1NEWSFEEDS(5) InterNetNews Documentation NEWSFEEDS(5)
2
6 newsfeeds - Determine where Usenet articles are sent
7
9 The newsfeeds file configures what innd does with incoming accepted
10 articles.
11 A minimalist newsfeeds file which parameters an outgoing NNTP feed to
13 "news.server.com", using path identity "server" (as it appears in Path
14 header fields), of all the articles you receive is:
15 # Mandatory line, with no exclusion patterns defined.
17 ME:::
18 # A real-time feed through innfeed of all your articles.
20 news.server.com/server\
21 :*\
22 :Tm:innfeed!
23 # The channel feed entry for innfeed.
25 # Change "/usr/lib/news/bin" to the directory path
26 # of the innfeed program.
27 innfeed!:!*:Tc,Wnm*:/usr/lib/news/bin/innfeed
28 In order to find an external feed, you can ask in the
30 news.admin.peering newsgroup. Several news administrators will
31 certainly respond and gracefully provide you with a news feed.
32 Other entries may be added in newsfeeds depending on your needs. See
34 the examples below and also in the archive(8), controlchan(8),
35 news2mail(8), ninpaths(8) and perl-nocem(8) man pages for the most
36 widely used.
37 If you exclude local hierarchies or newsgroups from the feed, make sure
39 to add "!control,!control.*" so that cancel articles or any kind of
40 control articles related to excluded hierarchies or newsgroups are not
41 propagated to the remote peer. Its newsfeeds entry then looks like:
42 news.server.com/server\
44 :*,!local.*,@*bina*,!control,!control.*\
45 :Tm:innfeed!
46 After any changes, run "inncheck" to perform basic syntax checks, and
48 reload this configuration file with the following command:
49 ctlinnd reload newsfeeds 'new peer'
51
53 The file pathetc/newsfeeds specifies how incoming articles should be
54 distributed to other programs and files on the server. It is parsed by
55 the InterNetNews server innd(8) when it starts up, or when directed to
56 by ctlinnd(8). innd doesn't send articles to remote sites itself, so
57 newsfeeds doesn't directly determine which remote news servers articles
58 are sent to. Instead, it specifies what batch files should be created
59 or which programs should be run (and what information should be sent to
60 them), and then this information is used by programs like innxmit(8)
61 and innfeed(8) to feed articles to remote sites.
62 The newsfeeds file isn't used solely to set up feeding accepted
64 articles to remote sites but also to pass them (or bits of information
65 about them) to any local programs or files that want that data. For
66 example, controlchan(8), a daemon that processes incoming control
67 messages, runs out of newsfeeds, as could a news to mail gateway.
68 The file is interpreted as a set of lines, parsed according to the
70 following rules: if a line ends with a backslash, the backslash, the
71 newline, and any whitespace at the start of the next line is deleted.
72 This is repeated until the entire "logical" line is collected. If the
73 logical line is blank or starts with a number sign ("#"), it is
74 ignored.
75 All other lines are interpreted as feed entries. An entry should
77 consist of four colon-separated fields; two of the fields may have
78 optional sub-fields, marked off by a slash. Fields or sub-fields that
79 take multiple parameters should be separated by a comma. Extra
80 whitespace can cause problems and should be avoided. Except for the
81 site names, case is significant. The format of an entry is:
82 sitename[/exclude,exclude,...]\
84 :pattern,pattern...[/distribution,distribution...]\
85 :flag,flag...\
86 :parameter
87 Each field is described below.
89 The sitename is the name of the site to which a news article can be
91 sent. It is used for writing log entries and for determining if an
92 article should be forwarded to a site. (A "site" is the generic term
93 for some destination of newsfeed data; it often corresponds to a remote
94 news peer, but doesn't have to. For example, a local archiving program
95 run from newsfeeds is also a "site".) If sitename already appears in
96 the article's Path header field body, then the article will not be sent
97 to the site. The name is usually whatever the remote site uses to
98 identify itself in the Path header field body, but can be almost any
99 word.
100 Be careful, though, to avoid having the sitename accidentally match a
102 path identity in the Path header field unintentionally. For this
103 reason, special local entries (such as archivers or gateways) should
104 probably end with an exclamation point to make sure that they do not
105 have the same name as any real site. For example, "gateway" is an
106 obvious name for the local entry that forwards articles out to a
107 mailing list. If a site with the name "gateway" posts an article, when
108 the local site receives the article it will see the name in the Path
109 header field and not send the article to its own "gateway" entry.
110 Since "gateway!" can't appear as an individual path identity in the
111 Path header field (as "!" is the delimiter), that would be a better
112 thing to use for sitename.
113 (Another way to avoid this problem is with the "Ap" flag; see the
115 description below.)
116 If an entry has an exclusion sub-field, the article will not be sent to
118 that site if any of exclude appear in the Path header field body.
119 (It's sometimes convenient to have the sitename be an abbreviated form
120 of the name of the remote site, since all the sitenames to which an
121 article is sent are written to the log and using shorter sitenames can
122 therefore improve performance for large servers. In this case, the
123 path identities of those sites should be given as exclude entries and
124 the "Ap" flag used so that the abbreviated sitename doesn't
125 accidentally match some other path identities in Path header fields.)
126 The same sitename can be used more than once and the appropriate action
128 will be taken for each entry that should receive the article, but this
129 is recommended only for program feeds to avoid confusion. Case is not
130 significant in site names.
131 The comma-separated pattern specifies which groups to send to the site;
133 it is interpreted to build a "subscription list" for the site. The
134 default subscription is to get all groups carried by the server. It is
135 a uwildmat pattern supporting poison ("@") wildcards; see the
136 libinn_uwildmat man page for full details on the pattern matching
137 language. pattern will be matched against every newsgroup carried by
138 the server and all newsgroups that match will be added to the
139 subscription list for the site.
140 Normally, a given article (or information about it) is sent to a site
142 if any of the newsgroups to which the article was posted are in that
143 site's subscription list. If a newsgroup matches an "@" pattern in
144 pattern, then not only is it not added to the subscription list, but
145 any articles crossposted to that newsgroup also will not be sent to
146 that site even if other newsgroups to which it was crossposted are in
147 that site's subscription list. This is called a poison pattern
148 (because matching groups are "poisoned").
149 For example, to receive all comp.* groups, but only comp.sources.unix
151 within the sources newsgroups, the following pattern can be used:
152 comp.*,!comp.sources.*,comp.sources.unix
154 Note that the trailing ".*" is required; the pattern has to match the
156 whole newsgroup name. "comp.sources.*" could be written
157 "comp.sources*" and would exclude the newsgroup comp.sources (if it
158 exists) as well as the groups in the comp.sources.* hierarchy, but note
159 that this would also exclude a newsgroup named comp.sources-only
160 (whereas the above pattern would add that group to the site
161 subscription list since it matches "comp.*" and none of the other
162 patterns).
163 For another example, to feed alt.* and misc.* to a given site but not
165 any articles posted to alt.binaries.warez (even if they're also
166 crossposted to other alt.* or misc.* groups), the following pattern can
167 be used:
168 alt.*,@alt.binaries.warez,misc.*
170 Note, however, that if you reversed the "alt.*" and
172 "@alt.binaries.warez" entries, this pattern would be equivalent to
173 "alt.*,misc.*", since the last matching pattern determines whether a
174 given newsgroup matches and the poison logic only applies if the poison
175 entry is the last matching entry.
176 Control messages follow slightly different propagation rules than
178 normal articles; see innd(8) for the details. Note that most
179 subscriptions should have "!junk,!control,!control.*" in their pattern
180 list due to those propagation rules (and since "junk" is a special
181 internal newsgroup; see wanttrash in inn.conf(5) for more details on
182 what it's used for) and that the best way to keep control messages
183 local to a site is with a distribution.
184 A subscription can be further modified by specifying distributions that
186 the site should or should not receive. The default is to send all
187 articles to all sites that subscribe to any of the groups where it has
188 been posted, but if an article has a Distribution header field and any
189 distributions are specified, then they are checked according to the
190 following rules:
191 1. If the Distribution header field matches any of the values in the
193 sub-field, the article is sent.
194 2. If a distribution starts with an exclamation point, and it matches
196 the Distribution header field, the article is not sent.
197 3. If the Distribution header field does not match any distribution in
199 the site's entry and no negations were used, the article is not
200 sent.
201 4. If the Distribution header field does not match any distribution in
203 the site's entry and any distribution started with an exclamation
204 point, the article is sent.
205 If an article has more than one distribution specified, then each one
207 is handled according according to the above rules. If any of the
208 specified distributions indicate that the article should be sent, it
209 is; if none do, it is not sent. In other words, the rules are used as
210 a logical or.
211 It is almost definitely a mistake to have a single feed that specifies
213 distributions that start with an exclamation point along with some that
214 don't.
215 Distributions are text words, not patterns; entries like "*" or "all"
217 have no special meaning.
218 The flag field is described in "FLAG VALUES". The interpretation of
220 the parameter field depends on the type of feed and is explained in
221 more detail in "FEED TYPES". It can be omitted for some types of
222 feeds.
223 The site named "ME" is special. There must be exactly one such entry,
225 and it should be the first entry in the file. If the "ME" entry has an
226 exclusion sub-field, incoming articles are rejected completely if any
227 of the names specified in that exclusion sub-field appear in their Path
228 header field bodies. If the "ME" entry has a subscription list, that
229 list is prepended to the subscription list of all other entries. For
230 example, "*,!control,!control.*,!junk,!foo.*" could be used to set the
231 default subscription list for all other feeds so that local postings
232 are not propagated unless "foo.*" explicitly appears in the site's
233 subscription list. This feature tends to be somewhat confusing since
234 the default subscription is prepended and can be overridden by other
235 patterns.
236 If the "ME" entry has a distribution sub-field, only articles that
238 match that distribution list are accepted and all other articles with a
239 distribution are rejected. A common use for this is to put something
240 like "/!local" in the "ME" entry to reject local postings from other
241 misconfigured sites. The distribution sub-field of "ME" has no effect
242 on the acceptance or rejection of articles that do not have a
243 Distribution header field.
244 An empty "ME" entry is possible, in which case no exclusion patterns
246 will be defined.
247 Finally, it is also possible to set variables in newsfeeds and use them
249 later in the file. A line starting with "$" sets a variable. For
250 example:
251 $LOCALGROUPS=local.*,example.*
253 $CONTROLGROUPS=control,control.*
254 This sets the variable "LOCALGROUPS" to "local.*,example.*" and the
256 variable "CONTROLGROUPS" to "control,control.*". They can later be
257 used elsewhere in the file, such as in a site entry like:
258 news.example.com:$LOCALGROUPS:Tf,Wnm:
260 which is then completely equivalent to:
262 news.example.com:local.*,example.*:Tf,Wnm:
264 Variables aren't solely simple substitution. If either "!" or "@"
266 immediately precedes the variable and the value of the variable
267 contains commas, that character will be duplicated before each comma.
268 This somewhat odd-sounding behavior is designed to make it easier to
269 use variables to construct feed patterns. The utility becomes more
270 obvious when you observe that the line:
271 news.example.net:*,@$LOCALGROUPS,!$CONTROLGROUPS:Tf,Wnm:
273 is therefore equivalent to:
275 news.example.net:*,@local.*,@example.*,!control,!control.*:Tf,Wnm:
277 which (as explained below) excludes all of the groups in $LOCALGROUPS
279 and unwanted control articles from the feed to that site.
280
282 The flags parameter specifies miscellaneous parameters, including the
283 type of feed, what information should be sent to it, and various
284 limitations on what articles should be sent to a site. They may be
285 specified in any order and should be separated by commas. Flags that
286 take values should have the value immediately after the flag letter
287 with no whitespace. The valid flags are:
288 < size
290 An article will only be sent to this site if it is less than size
291 bytes long. The default is no limit.
292 > size
294 An article will only be sent to this site if it is greater than
295 size bytes long. The default is no limit.
296 A checks
298 An article will only be sent to this site if it meets the
299 requirements specified in checks, which should be chosen from the
300 following set. checks can be multiple letters if appropriate.
301 Note that this flag is not effective on funnel targets; it has to
302 be used on every funnel entry (for instance, Af is not effective on
303 the innfeed! funnel target and therefore has to be specified on
304 every funnelled news site).
305 c Exclude all kinds of control messages.
307 C Only send control messages, not regular articles.
309 d Only send articles with a Distribution header field. Combined
311 with a particular distribution value in the distribution part of
312 the site entry, this can be used to limit articles sent to a
313 site to just those with a particular distribution.
314 e Only send articles where every newsgroup listed in the
316 Newsgroups header field body exists in the active file.
317 f Don't send articles rejected by filters. This is only useful
319 when dontrejectfiltered is set to true in inn.conf. With that
320 variable set, this lets one accept all articles but not
321 propagate filtered ones to some sites.
322 j Propagate articles according to their Newsgroups header field.
324 This is only useful when wanttrash is set to true in inn.conf.
325 With that variable set, articles accepted and filed in "junk"
326 (due to wanttrash) are fed to peers based on their subscription
327 pattern applied to the Newsgroups header field as though they
328 were accepted and all those groups were locally carried.
329 Otherwise, they are propagated to sites that receive the "junk"
330 newsgroup.
331 This variable is useful if you want to run INN with a minimal
333 active file and propagate all posts.
334 o Only send articles for which overview data was stored.
336 O Send articles to this site that don't have an Injection-Info or
338 X-Trace header field, even if the "O" flag is also given.
339 p Only check the exclusions against the Path header field of
341 articles; don't check the site name. This is useful if your
342 site names aren't the same as the path identities in Path header
343 fields added by those remote sites, or for program feeds where
344 the site name is arbitrary and unrelated to the Path header
345 field body.
346 If both "c" and "C" are given, the last specified one takes
348 precedence.
349 B high/low
351 If a site is being fed by a file, channel, or exploder (see below),
352 the server will normally start trying to write the information as
353 soon as possible. Providing a buffer may give better system
354 performance and help smooth out overall load if a large batch of
355 news comes in. The value of this flag should be two numbers
356 separated by a slash. high specifies the point at which the server
357 can start draining the feed's I/O buffer, and low specifies when to
358 stop writing and begin buffering again; the units are bytes. The
359 default is to do no buffering, sending output as soon as it is
360 possible to do so.
361 C count
363 If this flag is specified, an article will only be sent to this
364 site if the number of groups it is posted to, plus the square of
365 the number of groups followups would appear in, is no more than
366 count. 30 is a good value for this flag, allowing for instance:
367 • crossposts to only 5 groups when followups aren't set;
369 • crossposts to up to 26 groups when followups are set to two
371 groups;
372 • crossposts to up to 29 groups when followups are set to a single
374 group;
375 • crossposts to up to 30 groups when followups are set to poster.
377 Note that if an article does not contain a Followup-To header
379 field, the number of groups followups would appear in is the number
380 of groups it is posted to.
381 F name
383 Specifies the name of the file that should be used if it's
384 necessary to begin spooling for the site (see below). If name is
385 not an absolute path, it is taken to be relative to pathoutgoing in
386 inn.conf. If name is a directory, the file togo in that directory
387 will be used as the file name.
388 G count
390 If this flag is specified, an article will only be sent to this
391 site if it is posted to no more than count newsgroups. This has
392 the problem of filtering out many FAQs, as well as newsgroup
393 creation postings and similar administrative announcements. Either
394 the C flag or the U flag is a better solution.
395 H count
397 If this flag is specified, an article will only be sent to this
398 site if it has count or fewer sites in its Path header field body.
399 This flag should only be used as a rough guide because of the loose
400 interpretation of the Path header field; some sites put the
401 poster's name in the header field body, and some sites that might
402 logically be considered to be one hop become two because they put
403 the posting workstation's name in the header field body. The
404 default value for count if not specified is one. (Also see the O
405 flag, which is sometimes more appropriate for some uses of this
406 flag.)
407 I size
409 The flag specifies the size of the internal buffer for a file feed.
410 If there are more file feeds than allowed by the system, they will
411 be buffered internally in least-recently-used order. If the
412 internal buffer grows bigger than size bytes, however, the data
413 will be written out to the appropriate file. The default value is
414 16 KB.
415 N status
417 Restricts the articles sent to this site to those in newsgroups
418 with the moderation status given by status. If status is "m", only
419 articles in moderated groups are sent; if status is "u", only
420 articles in unmoderated groups are sent.
421 O originator
423 If this flag is specified, an article will only be sent to this
424 site if it contains an Injection-Info header field (or an X-Trace
425 header field if no Injection-Info header field is found) and the
426 first field of this header field matches originator. originator is
427 a uwildmat expression without commas or a list of such expressions,
428 separated by "/". The article is never sent if the first character
429 of the pattern begins with "@" and the rest of the pattern matches.
430 One use of this flag is to restrict the feed to locally generated
431 posts by using an originator pattern that matches the Injection-
432 Info header field added by the local server.
433 P priority
435 The nice priority that this channel or program feed should receive.
436 This should be a positive number between 0 and 20 and is the
437 priority that the new process will run with. This flag can be used
438 to raise the priority to normal if you're using the nicekids
439 parameter in inn.conf.
440 Q hashfeed
442 Specifies the hashfeed match expression for this site. It must be
443 in the form "value/mod" or "start-end/mod". The Message-ID of the
444 article is hashed using MD5, which results in a 128-bit hash. The
445 lowest 32 bits are then taken by default as the hashfeed value
446 (which is an integer). If the hashfeed value modulus "mod" plus
447 one equals "value" or is between "start" and "end", the article
448 will be fed to this site. All these numbers must be integers.
449 It is a deterministic way to control the flow of articles and to
451 split a feed. For instance:
452 Q1/2 Feeds about 50% of all articles to this site.
454 Q2/2 Feeds the other 50% of all articles.
455 Another example with three sites:
457 Q1-3/10 Feeds about 30% of all articles.
459 Q4-5/10 Feeds about 20% of all articles.
460 Q6-10/10 Feeds about 50% of all articles.
461 If this flag is specified multiple times, the contents will be
463 logically "OR"ed together (just one match is needed).
464 You can use an extended syntax of the form "value/mod_offset" or
466 "start-end/mod_offset". As MD5 generates a 128-bit return value,
467 it is possible to specify from which byte-offset the 32-bit integer
468 used by hashfeed starts. The default value for "offset" is "_0"
469 and thirteen overlapping values from "_0" to "_12" can be used.
470 Only up to four totally independent values exist: "_0", "_4", "_8"
471 and "_12".
472 Therefore, it allows generating a second level of deterministic
474 distribution. Indeed, if a news server is fed "Q1/2", it can go on
475 splitting thanks to "Q1-3/9_4" for instance. Up to four levels of
476 deterministic distribution can be used.
477 The algorithm is compatible with the one used by Diablo 5.1 and up.
479 If you want to use the legacy quickhashing method used by Diablo
480 before 5.1, you can put an "@" sign just after the Q flag (for
481 instance "Q@1-3/10", but the distribution of the messages is not
482 perfect with this legacy method whose use is discouraged and for
483 which offsets cannot be used).
484 S size
486 If the amount of data queued for the site gets to be larger than
487 size bytes, the server will switch to spooling, appending to a file
488 specified by the F flag, or pathoutgoing/sitename if F is not
489 specified. Spooling usually happens only for channel or exploder
490 feeds, when the spawned program isn't keeping up with its input.
491 T type
493 This flag specifies the type of feed for this site. type should be
494 a letter chosen from the following set:
495 c Channel
497 f File
498 l Log entry only
499 m Funnel (multiple entries feed into one)
500 p Program
501 x Exploder
502 Each feed is described below in "FEED TYPES". The default is Tf,
504 for a file feed.
505 U count
507 If this flag is specified, an article will only be sent to this
508 site if followups to this article would be posted to no more than
509 count newsgroups. (Also see C for a more complex way of handling
510 this.)
511 W items
513 For a file, channel, or exploder feed, this flag controls what
514 information will be sent to this site. For a program feed, only
515 the asterisk ("*") has any effect. items should be chosen from the
516 following set:
517 b Size of the article (in wire format, meaning with CRLF at the
519 end of each line, periods doubled at the beginning of lines, and
520 ending in a line with a single period) in bytes.
521 e The time the article will expire as seconds since epoch if it
523 has an Expires header field, 0 otherwise.
524 f The storage API token of the article (the same as "n"). The
526 article can be retrieved given the storage API token by using
527 sm(1).
528 g The newsgroup the article is in; if cross-posted, then the first
530 of the groups to which the article was posted that this site
531 gets. (The difference from "G" is that this sends the newsgroup
532 to which the article was posted even if it is a control
533 message.)
534 h The history hash key of the article (derived from the message
536 ID).
537 m The message ID of the article.
539 n The storage API token of the article. The article can be
541 retrieved given the storage API token by using sm(1).
542 p The time the article was posted a seconds since epoch.
544 s The site that fed the article to the server. This is taken from
546 either the Path header field body or the IP address of the
547 sending site depending on the value of logipaddr in inn.conf.
548 If logipaddr is true and the IP address is 0.0.0.0 (meaning that
549 the article was fed from localhost by a program like rnews(1)),
550 the Path header field body will be sent instead.
551 t The time the article was received as seconds since epoch.
553 * The names of the appropriate funnel entries, or all sites that
555 get the article (see below for more details).
556 D The body of the Distribution header field of the article, or "?"
558 if there is no such header field in the article.
559 G Where the article is stored. If the newsgroup is crossposted,
561 this is generally the first of the groups to which it was posted
562 that this site receives; however, control messages are filed in
563 control or control.* (which is the difference between this item
564 and "g").
565 H All of the headers, followed by a blank line. The Xref header
567 field will already be present, and a Bytes header field
568 containing the article's size in bytes as in the "b" item will
569 be added to the headers. If used, this should be the only item
570 in the list.
571 N The body of the Newsgroups header field.
573 P The body of the Path header field.
575 O Overview data for the article.
577 R Information needed for replication (the Xref header field
579 without the site name).
580 More than one letter can be given. If multiple items are
582 specified, they will be written in the order specified separated by
583 spaces. ("H" should be the only item if given, but if it's not a
584 newline will be sent before the beginning of the headers.) The
585 default is Wn.
586 The "H" and "O" items are intended for use by programs that create
588 news overview databases or require similar information. WnteO is
589 the flag to generate input needed by the overchan(8) program.
590 The asterisk ("*") has special meaning. Normally it expands to a
592 space-separated list of all sites that received the current
593 article. If, however, this site is a target of a funnel feed (in
594 other words, if it is named by other sites which have the Tm flag),
595 then the asterisk expands to the names of the funnel feeds that
596 received the article. Similarly, if the site is a program feed, an
597 asterisk in the parameter field will be expanded into the list of
598 funnel feeds that received the article. A program feed cannot get
599 the site list unless it is the target of other Tm feeds.
600
602 innd provides four basic types of feeds: log, file, program, and
603 channel. An exploder is a special type of channel. In addition,
604 several entries can feed into the same feed; these are funnel feeds,
605 which refer to an entry that is one of the other types. Funnel feeds
606 are partially described above with the description of the W* flag. A
607 funnel feed gets every article that would be sent to any of the feeds
608 that funnel into it and normally include the W* flag in their flags so
609 that the program processing that feed knows which sites received which
610 articles. The most common funnel feed is innfeed(8).
611 Note that the term "feed" is technically a misnomer, since the server
613 doesn't transfer articles itself and only writes data to a file,
614 program, or log telling another program to transfer the articles.
615 The simplest feed is a log feed (Tl). Other than a mention in the news
617 log file, pathlog/news, no data is written out. This is equivalent to
618 a Tf entry writing to /dev/null, except that no file is ever opened.
619 Flushing a log feed does nothing.
620 A file feed (Tf) is the next simplest type of feed. When the site
622 should receive an article, the specified data is written out to the
623 file named by the parameter field. If parameter is not an absolute
624 path, it is taken to be relative to pathoutgoing in inn.conf. If
625 parameter is not given, it defaults to pathoutgoing/sitename. The file
626 name should be unique (two file feeds should not ever point to the same
627 file).
628 File feeds are designed for use by external programs that periodically
630 process the written data. To cooperate with innd properly, such
631 external programs should first rename the batch file and then send a
632 flush command for that site to innd using ctlinnd(8). innd will then
633 write out any buffered data, close the file, and reopen it (under the
634 original name), and the program can process the data in the renamed
635 file at its leisure. File feeds are most frequently used in
636 combination with nntpsend(8).
637 A program feed (Tp) spawns a given program for every article that the
639 site receives. The parameter field must be the command line to
640 execute, and should contain one instance of %s, which will be replaced
641 by the storage API token of the article (the actual article can be
642 retrieved by the program using sm(1)). The program will not receive
643 anything on standard input (unlike earlier versions of INN, where the
644 article is sent to the program on stdin), and standard output and error
645 from the program will be set to the error log (pathlog/errlog). innd
646 will try to avoid spawning a shell if the command has no shell meta-
647 characters; this feature can be defeated if necessary for some reason
648 by appending a semi-colon to the end of the command. The full path
649 name of the program to be run must be specified unless the command will
650 be run by the shell (and it is strongly recommended that the full path
651 name always be specified regardless).
652 If a program feed is the target of a funnel, and if W* appears in the
654 flags of the site, a single asterisk may be present in the parameter
655 and will be replaced by a space-separated list of names of the sites
656 feeding into the funnel which received the relevant article. If the
657 site is not the target of a funnel, or if the W* flag is not used, the
658 asterisk has no special meaning.
659 Flushing a program feed does nothing.
661 For a channel (Tc) or exploder (Tx) feed, the parameter field again
663 names the process to start. As with program feeds, the full path to
664 the program must be specified. However, rather than spawning the
665 program for every article, it is spawned once and then whenever the
666 site receives an article, the data specified by the site flags is
667 written to the standard input of the spawned program. Standard output
668 and error are set as with program feeds. If the process exits, it will
669 be restarted automatically. If the process cannot be started, the
670 server will spool input to a file named pathoutgoing/sitename and will
671 try to start the process again later.
672 When a channel or exploder feed is flushed, the server closes its end
674 of the pipe to the program's standard input. Any pending data that has
675 not been written will be spooled; see the description of the S flag
676 above. The server will then spawn a new instance of the program. No
677 signal is sent to the program; it is up to the program handling a
678 channel or exploder feed to notice end of file on its standard input
679 and exit appropriately.
680 Exploders are a special type of channel feed. In addition to the
682 channel feed behavior described above, exploders can also be sent
683 command lines. These lines start with an exclamation point and their
684 interpretation is up to the exploder. The following commands are
685 generated automatically by the server:
686 !newgroup group
688 !rmgroup group
689 !flush
690 !flush site
691 These commands are sent whenever the ctlinnd(8) command of the same
693 name is received by the server. In addition, the ctlinnd(8) "send"
694 command can be used to send an arbitrary command line to an exploder.
695 The primary exploder is buffchan(8).
696 Finally, Tm feeds are the input to a funnel. The parameter field of
698 the site should name the site handling articles for all of the funnel
699 inputs.
700
702 The syntax of the newsfeeds file is so complex because you can specify
703 a staggering variety of feeds. INN is capable of interacting with a
704 wide variety of programs that do various things with news articles.
705 Far and away the most common two entries in newsfeeds, however, are
706 file feeds for nntpsend(8) and funnel feeds for innfeed(8).
707 The former look like this:
709 feed.example.com:*,!control,!control.*,!junk:Tf,Wnm:
711 which generates a file named pathoutgoing/feed.example.com containing
713 one line per article consisting of the storage API token, a space, and
714 the message ID.
715 The latter look like this:
717 feed.example.com:*,!control,!control.*,!junk:Tm:innfeed!
719 Very similar, except that this is the input to a funnel feed named
721 "innfeed!". One could also write this as:
722 example/feed.example.com:*,!control,!control.*,!junk:Ap,Tm:innfeed!
724 (note the Ap so that articles that contain just "example" in the Path
726 header field body will still be sent), which is completely equivalent
727 except that this will be logged in pathlog/news as going to the site
728 "example" rather than "feed.example.com".
729 The typical feed entry for innfeed(8) is a good example of a channel
731 feed that's the target of various funnel feeds (make sure the path to
732 innfeed is properly set):
733 innfeed!:!*:Tc,Wnm*:<pathbin in inn.conf>/innfeed -y
735 Note that the pattern for this feed is just "!*" so that it won't
737 receive any articles directly. The feed should only receive those
738 articles that would go to one of the funnel feeds that are feeding into
739 it. innfeed(8) will receive one line per article on its standard input
740 containing the storage API token, the message ID, and a space-separated
741 list of sites that should receive that article.
742 Here's a more esoteric example of a channel feed:
744 watcher!:*:Tc,Wbnm\
746 :exec awk '$1 > 1000000 { print "BIG", $2, $3 }' \
747 >> <pathlog>/bigart
748 This receives the byte size of each article along with the storage API
750 token and Message-ID, and appends to a bigart log file a line for every
751 article that's over a million bytes. This is actually rather a strange
752 way to write this since INN can do the size check itself; the following
753 is equivalent:
754 watcher!:*:Tc,>1000000,Wbnm\
756 :exec awk '{ print "BIG", $2, $3}' >> <pathlog>/bigart
757 Here's a cute, really simple news to mail gateway that also serves as
759 an example of a fairly fancy program feed:
760 mailer!:!*:W*,Tp\
762 :sm %s | innmail -s "News article" *
763 Remember that %s is replaced by the storage API token, so this
765 retrieves the article and pipes it into innmail (which is safer than
766 programs like Mail(1) because it doesn't parse the body for tilde
767 commands) with a given subject line. Note the use of "*" in the
768 command line and W* in the flags; this entry is designed to be used as
769 the target of funnel feeds such as:
770 peter@example.com:news.software.nntp:Tm:mailer!
772 sue@example.com:news.admin.misc:Tm:mailer!
773 Suppose that the server receives an article crossposted between
775 news.admin.misc and news.software.nntp. The server will notice that
776 the article should be sent to the site "peter@example.com" and the site
777 "bob@example.com", both of which funnel into "mailer!", so it will look
778 at the "mailer!" site and end up executing the command line:
779 sm @...@ | innmail -s "News article" \
781 peter@example.com sue@example.com
782 which will mail the article to both Peter and Sue.
784 Finally, another very useful example of a channel feed: the standard
786 entry for controlchan(8). Make sure its path is properly set.
787 controlchan!\
789 :!*,control,control.*,!control.cancel\
790 :AC,Tc,Wnsm:<pathbin in inn.conf>/controlchan
791 This program only wants information about articles posted to a control
793 newsgroup other than control.cancel, which due to the sorting of
794 control messages described in innd(8) will send it all control messages
795 except for cancel messages. controlchan gets the storage API token,
796 the name of the sending site (for processing old-style ihave and sendme
797 control messages, be sure to read about logipaddr in controlchan(8)),
798 and the message ID for each article.
799 For many other examples, including examples of the special "ME" site
801 entry, see the example newsfeeds file distributed with INN. Also see
802 the install documentation that comes with INN for information about
803 setting up the standard newsfeeds entries used by most sites.
804
806 Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews.
807 Reformatted and rewritten in POD by Russ Allbery <eagle@eyrie.org>.
808
810 active(5), buffchan(8), controlchan(8), ctlinnd(8), inn.conf(5),
811 innd(8), innfeed(8), innxmit(8), libinn_uwildmat(3), nntpsend(8).
812INN 2.7.1 2023-03-07 NEWSFEEDS(5)