1NEWSFEEDS(5) InterNetNews Documentation NEWSFEEDS(5)
2
6 newsfeeds - Determine where Usenet articles are sent
7
9 The newsfeeds file configures what innd does with incoming accepted
10 articles.
11 A minimalist newsfeeds file which parameters an outgoing NNTP feed to
13 "news.server.com", using path identity "server" (as it appears in Path
14 header fields), of all the articles you receive is:
15 # Mandatory line, with no exclusion patterns defined.
17 ME:::
18 # A real-time feed through innfeed of all your articles.
20 news.server.com/server\
21 :*\
22 :Tm:innfeed!
23 # The channel feed entry for innfeed.
25 # Change "/usr/lib/news/bin" to the directory path
26 # of the innfeed program.
27 innfeed!:!*:Tc,Wnm*:/usr/lib/news/bin/innfeed
28 In order to find an external feed, you can ask in the
30 news.admin.peering newsgroup. Several news administrators will
31 certainly respond and gracefully provide you with a news feed.
32 Other entries may be added in newsfeeds depending on your needs. See
34 the examples below and also in the archive(8), controlchan(8),
35 news2mail(8), ninpaths(8) and perl-nocem(8) man pages for the most
36 widely used.
37 If you exclude local hierarchies or newsgroups from the feed, make sure
39 to add "!control,!control.*" so that cancel articles or any kind of
40 control articles related to excluded hierarchies or newsgroups are not
41 propagated to the remote peer. Its newsfeeds entry then looks like:
42 news.server.com/server\
44 :*,!local.*,@*bina*,!control,!control.*\
45 :Tm:innfeed!
46 After any changes, run "inncheck" to perform basic syntax checks, and
48 reload this configuration file with the following command:
49 ctlinnd reload newsfeeds 'new peer'
51
53 The file pathetc/newsfeeds specifies how incoming articles should be
54 distributed to other programs and files on the server. It is parsed by
55 the InterNetNews server innd(8) when it starts up, or when directed to
56 by ctlinnd(8). innd doesn't send articles to remote sites itself, so
57 newsfeeds doesn't directly determine which remote news servers articles
58 are sent to. Instead, it specifies what batch files should be created
59 or which programs should be run (and what information should be sent to
60 them), and then this information is used by programs like innxmit(8)
61 and innfeed(8) to feed articles to remote sites.
62 The newsfeeds file isn't used solely to set up feeding accepted
64 articles to remote sites but also to pass them (or bits of information
65 about them) to any local programs or files that want that data. For
66 example, controlchan(8), a daemon that processes incoming control
67 messages, runs out of newsfeeds, as could a news to mail gateway.
68 The file is interpreted as a set of lines, parsed according to the
70 following rules: If a line ends with a backslash, the backslash, the
71 newline, and any whitespace at the start of the next line is deleted.
72 This is repeated until the entire "logical" line is collected. If the
73 logical line is blank or starts with a number sign ("#"), it is
74 ignored.
75 All other lines are interpreted as feed entries. An entry should
77 consist of four colon-separated fields; two of the fields may have
78 optional sub-fields, marked off by a slash. Fields or sub-fields that
79 take multiple parameters should be separated by a comma. Extra
80 whitespace can cause problems and should be avoided. Except for the
81 site names, case is significant. The format of an entry is:
82 sitename[/exclude,exclude,...]\
84 :pattern,pattern...[/distribution,distribution...]\
85 :flag,flag...\
86 :parameter
87 Each field is described below.
89 The sitename is the name of the site to which a news article can be
91 sent. It is used for writing log entries and for determining if an
92 article should be forwarded to a site. (A "site" is the generic term
93 for some destination of newsfeed data; it often corresponds to a remote
94 news peer, but doesn't have to. For example, a local archiving program
95 run from newsfeeds is also a "site".) If sitename already appears in
96 the article's Path: header, then the article will not be sent to the
97 site. The name is usually whatever the remote site uses to identify
98 itself in the Path: header, but can be almost any word.
99 Be careful, though, to avoid having the sitename accidentally match a
101 Path: header entry unintentionally. For this reason, special local
102 entries (such as archivers or gateways) should probably end with an
103 exclamation point to make sure that they do not have the same name as
104 any real site. For example, "gateway" is an obvious name for the local
105 entry that forwards articles out to a mailing list. If a site with the
106 name "gateway" posts an article, when the local site receives the
107 article it will see the name in the Path and not send the article to
108 its own "gateway" entry. Since "gateway!" can't appear as an
109 individual Path: entry since "!" is a delimiter in the Path: header,
110 that would be a better thing to use for sitename.
111 (Another way to avoid this problem is with the "Ap" flag; see the
113 description below.)
114 If an entry has an exclusion sub-field, the article will not be sent to
116 that site if any of exclude appear in the Path: header. (It's
117 sometimes convenient to have the sitename be an abbreviated form of the
118 name of the remote site, since all the sitenames to which an article is
119 sent are written to the log and using shorter sitenames can therefore
120 improve performance for large servers. In this case, the Path: header
121 entries of those sites should be given as exclude entries and the "Ap"
122 flag used so that the abbreviated sitename doesn't accidentally match
123 some other Path: header entry.)
124 The same sitename can be used more than once and the appropriate action
126 will be taken for each entry that should receive the article, but this
127 is recommended only for program feeds to avoid confusion. Case is not
128 significant in site names.
129 The comma-separated pattern specifies which groups to send to the site;
131 it is interpreted to build a "subscription list" for the site. The
132 default subscription is to get all groups carried by the server. It is
133 a uwildmat pattern supporting poison ("@") wildcards; see the
134 libinn_uwildmat man page for full details on the pattern matching
135 language. pattern will be matched against every newsgroup carried by
136 the server and all newsgroups that match will be added to the
137 subscription list for the site.
138 Normally, a given article (or information about it) is sent to a site
140 if any of the newsgroups to which the article was posted are in that
141 site's subscription list. If a newsgroup matches a "@" pattern in
142 pattern, then not only is it not added to the subscription list, but
143 any articles crossposted to that newsgroup also will not be sent to
144 that site even if other newsgroups to which it was crossposted are in
145 that site's subscription list. This is called a poison pattern
146 (because matching groups are "poisoned").
147 For example, to receive all comp.* groups, but only comp.sources.unix
149 within the sources newsgroups, the following pattern can be used:
150 comp.*,!comp.sources.*,comp.sources.unix
152 Note that the trailing ".*" is required; the pattern has to match the
154 whole newsgroup name. "comp.sources.*" could be written
155 "comp.sources*" and would exclude the newsgroup comp.sources (if it
156 exists) as well as the groups in the comp.sources.* hierarchy, but note
157 that this would also exclude a newsgroup named comp.sources-only
158 (whereas the above pattern would add that group to the site
159 subscription list since it matches "comp.*" and none of the other
160 patterns).
161 For another example, to feed alt.* and misc.* to a given site but not
163 any articles posted to alt.binaries.warez (even if they're also
164 crossposted to other alt.* or misc.* groups), the following pattern can
165 be used:
166 alt.*,@alt.binaries.warez,misc.*
168 Note, however, that if you reversed the "alt.*" and
170 "@alt.binaries.warez" entries, this pattern would be equivalent to
171 "alt.*,misc.*", since the last matching pattern determines whether a
172 given newsgroup matches and the poison logic only applies if the poison
173 entry is the last matching entry.
174 Control messages follow slightly different propagation rules than
176 normal articles; see innd(8) for the details. Note that most
177 subscriptions should have "!junk,!control,!control.*" in their pattern
178 list due to those propagation rules (and since "junk" is a special
179 internal newsgroup; see wanttrash in inn.conf(5) for more details on
180 what it's used for) and that the best way to keep control messages
181 local to a site is with a distribution.
182 A subscription can be further modified by specifying distributions that
184 the site should or should not receive. The default is to send all
185 articles to all sites that subscribe to any of the groups where it has
186 been posted, but if an article has a Distribution: header and any
187 distributions are specified, then they are checked according to the
188 following rules:
189 1. If the Distribution: header matches any of the values in the sub-
191 field, the article is sent.
192 2. If a distribution starts with an exclamation point, and it matches
194 the Distribution: header, the article is not sent.
195 3. If the Distribution: header does not match any distribution in the
197 site's entry and no negations were used, the article is not sent.
198 4. If the Distribution: header does not match any distribution in the
200 site's entry and any distribution started with an exclamation
201 point, the article is sent.
202 If an article has more than one distribution specified, then each one
204 is handled according according to the above rules. If any of the
205 specified distributions indicate that the article should be sent, it
206 is; if none do, it is not sent. In other words, the rules are used as
207 a logical or.
208 It is almost definitely a mistake to have a single feed that specifies
210 distributions that start with an exclamation point along with some that
211 don't.
212 Distributions are text words, not patterns; entries like "*" or "all"
214 have no special meaning.
215 The flag field is described in "FLAG VALUES". The interpretation of
217 the parameter field depends on the type of feed and is explained in
218 more detail in "FEED TYPES". It can be omitted for some types of
219 feeds.
220 The site named "ME" is special. There must be exactly one such entry,
222 and it should be the first entry in the file. If the "ME" entry has an
223 exclusion sub-field, incoming articles are rejected completely if any
224 of the names specified in that exclusion sub-field appear in their
225 Path: headers. If the "ME" entry has a subscription list, that list is
226 prepended to the subscription list of all other entries. For example,
227 "*,!control,!control.*,!junk,!foo.*" could be used to set the default
228 subscription list for all other feeds so that local postings are not
229 propagated unless "foo.*" explicitly appears in the site's subscription
230 list. This feature tends to be somewhat confusing since the default
231 subscription is prepended and can be overridden by other patterns.
232 If the "ME" entry has a distribution sub-field, only articles that
234 match that distribution list are accepted and all other articles with a
235 distribution are rejected. A common use for this is to put something
236 like "/!local" in the "ME" entry to reject local postings from other
237 misconfigured sites. The distribution sub-field of "ME" has no effect
238 on the acceptance or rejection of articles that do not have a
239 Distribution header field.
240 An empty "ME" entry is possible, in which case no exclusion patterns
242 will be defined.
243 Finally, it is also possible to set variables in newsfeeds and use them
245 later in the file. A line starting with "$" sets a variable. For
246 example:
247 $LOCALGROUPS=local.*,example.*
249 $CONTROLGROUPS=control,control.*
250 This sets the variable "LOCALGROUPS" to "local.*,example.*" and the
252 variable "CONTROLGROUPS" to "control,control.*". They can later be
253 used elsewhere in the file, such as in a site entry like:
254 news.example.com:$LOCALGROUPS:Tf,Wnm:
256 which is then completely equivalent to:
258 news.example.com:local.*,example.*:Tf,Wnm:
260 Variables aren't solely simple substitution. If either "!" or "@"
262 immediately preceds the variable and the value of the variable contains
263 commas, that character will be duplicated before each comma. This
264 somewhat odd-sounding behavior is designed to make it easier to use
265 variables to construct feed patterns. The utility becomes more obvious
266 when you observe that the line:
267 news.example.net:*,@$LOCALGROUPS,!$CONTROLGROUPS:Tf,Wnm:
269 is therefore equivalent to:
271 news.example.net:*,@local.*,@example.*,!control,!control.*:Tf,Wnm:
273 which (as explained below) excludes all of the groups in $LOCALGROUPS
275 and unwanted control articles from the feed to that site.
276
278 The flags parameter specifies miscellaneous parameters, including the
279 type of feed, what information should be sent to it, and various
280 limitations on what articles should be sent to a site. They may be
281 specified in any order and should be separated by commas. Flags that
282 take values should have the value immediately after the flag letter
283 with no whitespace. The valid flags are:
284 < size
286 An article will only be sent to this site if it is less than size
287 bytes long. The default is no limit.
288 > size
290 An article will only be sent to this site if it is greater than
291 size bytes long. The default is no limit.
292 A checks
294 An article will only be sent to this site if it meets the
295 requirements specified in checks, which should be chosen from the
296 following set. checks can be multiple letters if appropriate.
297 Note that this flag is not effective on funnel targets; it has to
298 be used on every funnel entry (for instance, Af is not effective on
299 the innfeed! funnel target and therefore has to be specified on
300 every funnelled news site).
301 c Exclude all kinds of control messages.
303 C Only send control messages, not regular articles.
305 d Only send articles with a Distribution: header. Combined with a
307 particular distribution value in the distribution part of the
308 site entry, this can be used to limit articles sent to a site to
309 just those with a particuliar distribution.
310 e Only send articles where every newsgroup listed in the
312 Newsgroups: header exists in the active file.
313 f Don't send articles rejected by filters. This is only useful
315 when dontrejectfiltered is set to true in inn.conf. With that
316 variable set, this lets one accept all articles but not
317 propagate filtered ones to some sites.
318 j Propagate articles according to their Newsgroups: header. This
320 is only useful when wanttrash is set to true in inn.conf. With
321 that variable set, articles accepted and filed in "junk" (due to
322 wanttrash) are fed to peers based on their subscription pattern
323 applied to the Newsgroups: header as though they were accepted
324 and all those groups were locally carried. Otherwise, they are
325 propagated to sites that receive the "junk" newsgroup.
326 This variable is useful if you want to run INN with a minimal
328 active file and propagate all posts.
329 o Only send articles for which overview data was stored.
331 O Send articles to this site that don't have an Injection-Info: or
333 X-Trace: header, even if the "O" flag is also given.
334 p Only check the exclusions against the Path: header of articles;
336 don't check the site name. This is useful if your site names
337 aren't the same as the Path: entries added by those remote
338 sites, or for program feeds where the site name is arbitrary and
339 unrelated to the Path: header.
340 If both "c" and "C" are given, the last specified one takes
342 precedence.
343 B high/low
345 If a site is being fed by a file, channel, or exploder (see below),
346 the server will normally start trying to write the information as
347 soon as possible. Providing a buffer may give better system
348 performance and help smooth out overall load if a large batch of
349 news comes in. The value of this flag should be two numbers
350 separated by a slash. high specifies the point at which the server
351 can start draining the feed's I/O buffer, and low specifies when to
352 stop writing and begin buffering again; the units are bytes. The
353 default is to do no buffering, sending output as soon as it is
354 possible to do so.
355 C count
357 If this flag is specified, an article will only be sent to this
358 site if the number of groups it is posted to, plus the square of
359 the number of groups followups would appear in, is no more than
360 count. 30 is a good value for this flag, allowing for instance:
361 • crossposts to only 5 groups when followups aren't set;
363 • crossposts to up to 26 groups when followups are set to two
365 groups;
366 • crossposts to up to 29 groups when followups are set to a single
368 group;
369 • crossposts to up to 30 groups when followups are set to poster.
371 Note that if an article does not contain a Followup-To: header
373 field, the number of groups followups would appear in is the number
374 of groups it is posted to.
375 F name
377 Specifies the name of the file that should be used if it's
378 necessary to begin spooling for the site (see below). If name is
379 not an absolute path, it is taken to be relative to pathoutgoing in
380 inn.conf. If name is a directory, the file togo in that directory
381 will be used as the file name.
382 G count
384 If this flag is specified, an article will only be sent to this
385 site if it is posted to no more than count newsgroups. This has
386 the problem of filtering out many FAQs, as well as newsgroup
387 creation postings and similar administrative announcements. Either
388 the C flag or the U flag is a better solution.
389 H count
391 If this flag is specified, an article will only be sent to this
392 site if it has count or fewer sites in its Path: line. This flag
393 should only be used as a rough guide because of the loose
394 interpretation of the Path: header; some sites put the poster's
395 name in the header, and some sites that might logically be
396 considered to be one hop become two because they put the posting
397 workstation's name in the header. The default value for count if
398 not specified is one. (Also see the O flag, which is sometimes
399 more appropriate for some uses of this flag.)
400 I size
402 The flag specifies the size of the internal buffer for a file feed.
403 If there are more file feeds than allowed by the system, they will
404 be buffered internally in least-recently-used order. If the
405 internal buffer grows bigger then size bytes, however, the data
406 will be written out to the appropriate file. The default value is
407 16 KB.
408 N status
410 Restricts the articles sent to this site to those in newsgroups
411 with the moderation status given by status. If status is "m", only
412 articles in moderated groups are sent; if status is "u", only
413 articles in unmoderated groups are sent.
414 O originator
416 If this flag is specified, an article will only be sent to this
417 site if it contains an Injection-Info header field (or an X-Trace
418 header field if no Injection-Info header field is found) and the
419 first field of this header field matches originator. originator is
420 a uwildmat expression without commas or a list of such expressions,
421 separated by "/". The article is never sent if the first character
422 of the pattern begins with "@" and the rest of the pattern matches.
423 One use of this flag is to restrict the feed to locally generated
424 posts by using an originator pattern that matches the Injection-
425 Info header field added by the local server.
426 P priority
428 The nice priority that this channel or program feed should receive.
429 This should be a positive number between 0 and 20 and is the
430 priority that the new process will run with. This flag can be used
431 to raise the priority to normal if you're using the nicekids
432 parameter in inn.conf.
433 Q hashfeed
435 Specifies the hashfeed match expression for this site. It must be
436 in the form "value/mod" or "start-end/mod". The Message-ID of the
437 article is hashed using MD5, which results in a 128-bit hash. The
438 lowest 32 bits are then taken by default as the hashfeed value
439 (which is an integer). If the hashfeed value modulus "mod" plus
440 one equals "value" or is between "start" and "end", the article
441 will be fed to this site. All these numbers must be integers.
442 It is a deterministic way to control the flow of articles and to
444 split a feed. For instance:
445 Q1/2 Feeds about 50% of all articles to this site.
447 Q2/2 Feeds the other 50% of all articles.
448 Another example with three sites:
450 Q1-3/10 Feeds about 30% of all articles.
452 Q4-5/10 Feeds about 20% of all articles.
453 Q6-10/10 Feeds about 50% of all articles.
454 If this flag is specified multiple times, the contents will be
456 logically "OR"ed together (just one match is needed).
457 You can use an extended syntax of the form "value/mod_offset" or
459 "start-end/mod_offset". As MD5 generates a 128-bit return value,
460 it is possible to specify from which byte-offset the 32-bit integer
461 used by hashfeed starts. The default value for "offset" is "_0"
462 and thirteen overlapping values from "_0" to "_12" can be used.
463 Only up to four totally independent values exist: "_0", "_4", "_8"
464 and "_12".
465 Therefore, it allows generating a second level of deterministic
467 distribution. Indeed, if a news server is fed "Q1/2", it can go on
468 splitting thanks to "Q1-3/9_4" for instance. Up to four levels of
469 deterministic distribution can be used.
470 The algorithm is compatible with the one used by Diablo 5.1 and up.
472 If you want to use the legacy quickhashing method used by Diablo
473 before 5.1, you can put an "@" sign just after the Q flag (for
474 instance "Q@1-3/10", but the distribution of the messages is not
475 perfect with this legacy method whose use is discouraged and for
476 which offsets cannot be used).
477 S size
479 If the amount of data queued for the site gets to be larger than
480 size bytes, the server will switch to spooling, appending to a file
481 specified by the F flag, or pathoutgoing/sitename if F is not
482 specified. Spooling usually happens only for channel or exploder
483 feeds, when the spawned program isn't keeping up with its input.
484 T type
486 This flag specifies the type of feed for this site. type should be
487 a letter chosen from the following set:
488 c Channel
490 f File
491 l Log entry only
492 m Funnel (multiple entries feed into one)
493 p Program
494 x Exploder
495 Each feed is described below in "FEED TYPES". The default is Tf,
497 for a file feed.
498 U count
500 If this flag is specified, an article will only be sent to this
501 site if followups to this article would be posted to no more than
502 count newsgroups. (Also see C for a more complex way of handling
503 this.)
504 W items
506 For a file, channel, or exploder feed, this flag controls what
507 information will be sent to this site. For a program feed, only
508 the asterisk ("*") has any effect. items should be chosen from the
509 following set:
510 b Size of the article (in wire format, meaning with CRLF at the
512 end of each line, periods doubled at the beginning of lines, and
513 ending in a line with a single period) in bytes.
514 e The time the article will expire as seconds since epoch if it
516 has an Expires: header, 0 otherwise.
517 f The storage API token of the article (the same as "n"). The
519 article can be retrieved given the storage API token by using
520 sm(8).
521 g The newsgroup the article is in; if cross-posted, then the first
523 of the groups to which the article was posted that this site
524 gets. (The difference from "G" is that this sends the newsgroup
525 to which the article was posted even if it is a control
526 message.)
527 h The history hash key of the article (derived from the message
529 ID).
530 m The message ID of the article.
532 n The storage API token of the article. The article can be
534 retrieved given the storage API token by using sm(8).
535 p The time the article was posted a seconds since epoch.
537 s The site that fed the article to the server. This is taken from
539 either the Path: header or the IP address of the sending site
540 depending on the value of logipaddr in inn.conf. If logipaddr
541 is true and the IP address is 0.0.0.0 (meaning that the article
542 was fed from localhost by a program like rnews(8)), the Path:
543 header value will be sent instead.
544 t The time the article was received as seconds since epoch.
546 * The names of the appropriate funnel entries, or all sites that
548 get the article (see below for more details).
549 D The value of the Distribution: header of the article, or "?" if
551 there is no such header in the article.
552 G Where the article is stored. If the newsgroup is crossposted,
554 this is generally the first of the groups to which it was posted
555 that this site receives; however, control messages are filed in
556 control or control.* (which is the difference between this item
557 and "g").
558 H All of the headers, followed by a blank line. The Xref header
560 will already be present, and a Bytes header containing the
561 article's size in bytes as in the "b" item will be added to the
562 headers. If used, this should be the only item in the list.
563 N The value of the Newsgroups: header.
565 P The value of the Path: header.
567 O Overview data for the article.
569 R Information needed for replication (the Xref header without the
571 site name).
572 More than one letter can be given. If multiple items are
574 specified, they will be written in the order specified separated by
575 spaces. ("H" should be the only item if given, but if it's not a
576 newline will be sent before the beginning of the headers.) The
577 default is Wn.
578 The "H" and "O" items are intended for use by programs that create
580 news overview databases or require similar information. WnteO is
581 the flag to generate input needed by the overchan(8) program.
582 The asterisk ("*") has special meaning. Normally it expands to a
584 space-separated list of all sites that received the current
585 article. If, however, this site is a target of a funnel feed (in
586 other words, if it is named by other sites which have the Tm flag),
587 then the asterisk expands to the names of the funnel feeds that
588 received the article. Similarly, if the site is a program feed, an
589 asterisk in the parameter field will be expanded into the list of
590 funnel feeds that received the article. A program feed cannot get
591 the site list unless it is the target of other Tm feeds.
592
594 innd provides four basic types of feeds: log, file, program, and
595 channel. An exploder is a special type of channel. In addition,
596 several entries can feed into the same feed; these are funnel feeds,
597 which refer to an entry that is one of the other types. Funnel feeds
598 are partially described above with the description of the W* flag. A
599 funnel feed gets every article that would be sent to any of the feeds
600 that funnel into it and normally include the W* flag in their flags so
601 that the program processing that feed knows which sites received which
602 articles. The most common funnel feed is innfeed(8).
603 Note that the term "feed" is technically a misnomer, since the server
605 doesn't transfer articles itself and only writes data to a file,
606 program, or log telling another program to transfer the articles.
607 The simplest feed is a log feed (Tl). Other than a mention in the news
609 log file, pathlog/news, no data is written out. This is equivalent to
610 a Tf entry writing to /dev/null, except that no file is ever opened.
611 Flushing a log feed does nothing.
612 A file feed (Tf) is the next simplest type of feed. When the site
614 should receive an article, the specified data is written out to the
615 file named by the parameter field. If parameter is not an absolute
616 path, it is taken to be relative to pathoutgoing in inn.conf. If
617 parameter is not given, it defaults to pathoutgoing/sitename. The file
618 name should be unique (two file feeds should not ever point to the same
619 file).
620 File feeds are designed for use by external programs that periodically
622 process the written data. To cooperate with innd properly, such
623 external programs should first rename the batch file and then send a
624 flush command for that site to innd using ctlinnd(8). innd will then
625 write out any buffered data, close the file, and reopen it (under the
626 original name), and the program can process the data in the renamed
627 file at its leisure. File feeds are most frequently used in
628 combination with nntpsend(8).
629 A program feed (Tp) spawns a given program for every article that the
631 site receives. The parameter field must be the command line to
632 execute, and should contain one instance of %s, which will be replaced
633 by the storage API token of the article (the actual article can be
634 retrieved by the program using sm(8)). The program will not receive
635 anything on standard input (unlike earlier versions of INN, where the
636 article is sent to the program on stdin), and standard output and error
637 from the program will be set to the error log (pathlog/errlog). innd
638 will try to avoid spawning a shell if the command has no shell meta-
639 characters; this feature can be defeated if necessary for some reason
640 by appending a semi-colon to the end of the command. The full path
641 name of the program to be run must be specified unless the command will
642 be run by the shell (and it is strongly recommended that the full path
643 name always be specified regardless).
644 If a program feed is the target of a funnel, and if W* appears in the
646 flags of the site, a single asterisk may be present in the parameter
647 and will be replaced by a space-separated list of names of the sites
648 feeding into the funnel which received the relevant article. If the
649 site is not the target of a funnel, or if the W* flag is not used, the
650 asterisk has no special meaning.
651 Flushing a program feed does nothing.
653 For a channel (Tc) or exploder (Tx) feed, the parameter field again
655 names the process to start. As with program feeds, the full path to
656 the program must be specified. However, rather than spawning the
657 program for every article, it is spawned once and then whenever the
658 site receives an article, the data specified by the site flags is
659 written to the standard input of the spawned program. Standard output
660 and error are set as with program feeds. If the process exits, it will
661 be restarted automatically. If the process cannot be started, the
662 server will spool input to a file named pathoutgoing/sitename and will
663 try to start the process again later.
664 When a channel or exploder feed is flushed, the server closes its end
666 of the pipe to the program's standard input. Any pending data that has
667 not been written will be spooled; see the description of the S flag
668 above. The server will then spawn a new instance of the program. No
669 signal is sent to the program; it is up to the program handling a
670 channel or exploder feed to notice end of file on its standard input
671 and exit appropriately.
672 Exploders are a special type of channel feed. In addition to the
674 channel feed behavior described above, exploders can also be sent
675 command lines. These lines start with an exclamation point and their
676 interpretation is up to the exploder. The following commands are
677 generated automatically by the server:
678 !newgroup group
680 !rmgroup group
681 !flush
682 !flush site
683 These commands are sent whenever the ctlinnd(8) command of the same
685 name is received by the server. In addition, the ctlinnd(8) "send"
686 command can be used to send an arbitrary command line to an exploder.
687 The primary exploder is buffchan(8).
688 Finally, Tm feeds are the input to a funnel. The parameter field of
690 the site should name the site handling articles for all of the funnel
691 inputs.
692
694 The syntax of the newsfeeds file is so complex because you can specify
695 a staggering variety of feeds. INN is capable of interacting with a
696 wide variety of programs that do various things with news articles.
697 Far and away the most common two entries in newsfeeds, however, are
698 file feeds for nntpsend(8) and funnel feeds for innfeed(8).
699 The former look like this:
701 feed.example.com:*,!control,!control.*,!junk:Tf,Wnm:
703 which generates a file named pathoutgoing/feed.example.com containing
705 one line per article consisting of the storage API token, a space, and
706 the message ID.
707 The latter look like this:
709 feed.example.com:*,!control,!control.*,!junk:Tm:innfeed!
711 Very similar, except that this is the input to a funnel feed named
713 "innfeed!". One could also write this as:
714 example/feed.example.com:*,!control,!control.*,!junk:Ap,Tm:innfeed!
716 (note the Ap so that articles that contain just "example" in the Path:
718 header will still be sent), which is completely equivalent except that
719 this will be logged in pathlog/news as going to the site "example"
720 rather than "feed.example.com".
721 The typical feed entry for innfeed(8) is a good example of a channel
723 feed that's the target of various funnel feeds (make sure the path to
724 innfeed is properly set):
725 innfeed!:!*:Tc,Wnm*:<pathbin in inn.conf>/innfeed -y
727 Note that the pattern for this feed is just "!*" so that it won't
729 receive any articles directly. The feed should only receive those
730 articles that would go to one of the funnel feeds that are feeding into
731 it. innfeed(8) will receive one line per article on its standard input
732 containing the storage API token, the message ID, and a space-separated
733 list of sites that should receive that article.
734 Here's a more esoteric example of a channel feed:
736 watcher!:*:Tc,Wbnm\
738 :exec awk '$1 > 1000000 { print "BIG", $2, $3 }' > /dev/console
739 This receives the byte size of each article along with the storage API
741 token and message ID, and prints to the console a line for every
742 article that's over a million bytes. This is actually rather a strange
743 way to write this since INN can do the size check itself; the following
744 is equivalent:
745 watcher!:*:Tc,>1000000,Wbnm\
747 :exec awk '{ print "BIG", $2, $3}' > /dev/console
748 Here's a cute, really simple news to mail gateway that also serves as
750 an example of a fairly fancy program feed:
751 mailer!:!*:W*,Tp\
753 :sm %s | innmail -s "News article" *
754 Remember that %s is replaced by the storage API token, so this
756 retrieves the article and pipes it into innmail (which is safer than
757 programs like Mail(1) because it doesn't parse the body for tilde
758 commands) with a given subject line. Note the use of "*" in the
759 command line and W* in the flags; this entry is designed to be used as
760 the target of funnel feeds such as:
761 peter@example.com:news.software.nntp:Tm:mailer!
763 sue@example.com:news.admin.misc:Tm:mailer!
764 Suppose that the server receives an article crossposted between
766 news.admin.misc and news.software.nntp. The server will notice that
767 the article should be sent to the site "peter@example.com" and the site
768 "bob@example.com", both of which funnel into "mailer!", so it will look
769 at the "mailer!" site and end up executing the command line:
770 sm @...@ | innmail -s "News article" peter@example.com sue@example.com
772 which will mail the article to both Peter and Sue.
774 Finally, another very useful example of a channel feed: the standard
776 entry for controlchan(8). Make sure its path is properly set.
777 controlchan!\
779 :!*,control,control.*,!control.cancel\
780 :AC,Tc,Wnsm:<pathbin in inn.conf>/controlchan
781 This program only wants information about articles posted to a control
783 newsgroup other than control.cancel, which due to the sorting of
784 control messages described in innd(8) will send it all control messages
785 except for cancel messages. controlchan gets the storage API token,
786 the name of the sending site (for processing old-style ihave and sendme
787 control messages, be sure to read about logipaddr in controlchan(8)),
788 and the message ID for each article.
789 For many other examples, including examples of the special "ME" site
791 entry, see the example newsfeeds file distributed with INN. Also see
792 the install documentation that comes with INN for information about
793 setting up the standard newsfeeds entries used by most sites.
794
796 Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews.
797 Reformatted and rewritten in POD by Russ Allbery <eagle@eyrie.org>.
798
800 active(5), buffchan(8), controlchan(8), ctlinnd(8), inn.conf(5),
801 innd(8), innfeed(8), innxmit(8), libinn_uwildmat(3), nntpsend(8).
802INN 2.6.5 2022-02-18 NEWSFEEDS(5)