1NEWSFEEDS(5)              InterNetNews Documentation              NEWSFEEDS(5)
2
3
4

NAME

6       newsfeeds - Determine where Usenet articles are sent
7

IN A NUTSHELL

9       The newsfeeds file configures what innd does with incoming accepted
10       articles.
11
12       A minimalist newsfeeds file which parameters an outgoing NNTP feed to
13       "news.server.com", using path identity "server" (as it appears in Path
14       header fields), of all the articles you receive is:
15
16           # Mandatory line, with no exclusion patterns defined.
17           ME:::
18
19           # A real-time feed through innfeed of all your articles.
20           news.server.com/server\
21               :*\
22               :Tm:innfeed!
23
24           # The channel feed entry for innfeed.
25           # Change "/usr/lib/news/bin" to the directory path
26           # of the innfeed program.
27           innfeed!:!*:Tc,Wnm*:/usr/lib/news/bin/innfeed
28
29       In order to find an external feed, you can ask in the
30       news.admin.peering newsgroup.  Several news administrators will
31       certainly respond and gracefully provide you with a news feed.
32
33       Other entries may be added in newsfeeds depending on your needs.  See
34       the examples below and also in the archive(8), controlchan(8),
35       news2mail(8), ninpaths(8) and perl-nocem(8) man pages for the most
36       widely used.
37
38       If you exclude local hierarchies or newsgroups from the feed, make sure
39       to add "!control,!control.*" so that cancel articles or any kind of
40       control articles related to excluded hierarchies or newsgroups are not
41       propagated to the remote peer.  Its newsfeeds entry then looks like:
42
43           news.server.com/server\
44               :*,!local.*,@*bina*,!control,!control.*\
45               :Tm:innfeed!
46
47       After any changes, run "inncheck" to perform basic syntax checks, and
48       reload this configuration file with the following command:
49
50           ctlinnd reload newsfeeds 'new peer'
51

DESCRIPTION

53       The file pathetc/newsfeeds specifies how incoming articles should be
54       distributed to other programs and files on the server.  It is parsed by
55       the InterNetNews server innd(8) when it starts up, or when directed to
56       by ctlinnd(8).  innd doesn't send articles to remote sites itself, so
57       newsfeeds doesn't directly determine which remote news servers articles
58       are sent to.  Instead, it specifies what batch files should be created
59       or which programs should be run (and what information should be sent to
60       them), and then this information is used by programs like innxmit(8)
61       and innfeed(8) to feed articles to remote sites.
62
63       The newsfeeds file isn't used solely to set up feeding accepted
64       articles to remote sites but also to pass them (or bits of information
65       about them) to any local programs or files that want that data.  For
66       example, controlchan(8), a daemon that processes incoming control
67       messages, runs out of newsfeeds, as could a news to mail gateway.
68
69       The file is interpreted as a set of lines, parsed according to the
70       following rules: if a line ends with a backslash, the backslash, the
71       newline, and any whitespace at the start of the next line is deleted.
72       This is repeated until the entire "logical" line is collected.  If the
73       logical line is blank or starts with a number sign ("#"), it is
74       ignored.
75
76       All other lines are interpreted as feed entries.  An entry should
77       consist of four colon-separated fields; two of the fields may have
78       optional sub-fields, marked off by a slash.  Fields or sub-fields that
79       take multiple parameters should be separated by a comma.  Extra
80       whitespace can cause problems and should be avoided.  Except for the
81       site names, case is significant.  The format of an entry is:
82
83           sitename[/exclude,exclude,...]\
84               :pattern,pattern...[/distribution,distribution...]\
85               :flag,flag...\
86               :parameter
87
88       Each field is described below.
89
90       The sitename is the name of the site to which a news article can be
91       sent.  It is used for writing log entries and for determining if an
92       article should be forwarded to a site.  (A "site" is the generic term
93       for some destination of newsfeed data; it often corresponds to a remote
94       news peer, but doesn't have to.  For example, a local archiving program
95       run from newsfeeds is also a "site".)  If sitename already appears in
96       the article's Path header field body, then the article will not be sent
97       to the site.  The name is usually whatever the remote site uses to
98       identify itself in the Path header field body, but can be almost any
99       word.
100
101       Be careful, though, to avoid having the sitename accidentally match a
102       path identity in the Path header field unintentionally.  For this
103       reason, special local entries (such as archivers or gateways) should
104       probably end with an exclamation point to make sure that they do not
105       have the same name as any real site.  For example, "gateway" is an
106       obvious name for the local entry that forwards articles out to a
107       mailing list.  If a site with the name "gateway" posts an article, when
108       the local site receives the article it will see the name in the Path
109       header field and not send the article to its own "gateway" entry.
110       Since "gateway!" can't appear as an individual path identity in the
111       Path header field (as "!" is the delimiter), that would be a better
112       thing to use for sitename.
113
114       (Another way to avoid this problem is with the "Ap" flag; see the
115       description below.)
116
117       If an entry has an exclusion sub-field, the article will not be sent to
118       that site if any of exclude appear in the Path header field body.
119       (It's sometimes convenient to have the sitename be an abbreviated form
120       of the name of the remote site, since all the sitenames to which an
121       article is sent are written to the log and using shorter sitenames can
122       therefore improve performance for large servers.  In this case, the
123       path identities of those sites should be given as exclude entries and
124       the "Ap" flag used so that the abbreviated sitename doesn't
125       accidentally match some other path identities in Path header fields.)
126
127       The same sitename can be used more than once and the appropriate action
128       will be taken for each entry that should receive the article, but this
129       is recommended only for program feeds to avoid confusion.  Case is not
130       significant in site names.
131
132       The comma-separated pattern specifies which groups to send to the site;
133       it is interpreted to build a "subscription list" for the site.  The
134       default subscription is to get all groups carried by the server.  It is
135       a uwildmat pattern supporting poison ("@") wildcards; see the
136       libinn_uwildmat man page for full details on the pattern matching
137       language.  pattern will be matched against every newsgroup carried by
138       the server and all newsgroups that match will be added to the
139       subscription list for the site.
140
141       Normally, a given article (or information about it) is sent to a site
142       if any of the newsgroups to which the article was posted are in that
143       site's subscription list.  If a newsgroup matches a "@" pattern in
144       pattern, then not only is it not added to the subscription list, but
145       any articles crossposted to that newsgroup also will not be sent to
146       that site even if other newsgroups to which it was crossposted are in
147       that site's subscription list.  This is called a poison pattern
148       (because matching groups are "poisoned").
149
150       For example, to receive all comp.* groups, but only comp.sources.unix
151       within the sources newsgroups, the following pattern can be used:
152
153           comp.*,!comp.sources.*,comp.sources.unix
154
155       Note that the trailing ".*" is required; the pattern has to match the
156       whole newsgroup name.  "comp.sources.*" could be written
157       "comp.sources*" and would exclude the newsgroup comp.sources (if it
158       exists) as well as the groups in the comp.sources.* hierarchy, but note
159       that this would also exclude a newsgroup named comp.sources-only
160       (whereas the above pattern would add that group to the site
161       subscription list since it matches "comp.*" and none of the other
162       patterns).
163
164       For another example, to feed alt.* and misc.* to a given site but not
165       any articles posted to alt.binaries.warez (even if they're also
166       crossposted to other alt.* or misc.* groups), the following pattern can
167       be used:
168
169           alt.*,@alt.binaries.warez,misc.*
170
171       Note, however, that if you reversed the "alt.*" and
172       "@alt.binaries.warez" entries, this pattern would be equivalent to
173       "alt.*,misc.*", since the last matching pattern determines whether a
174       given newsgroup matches and the poison logic only applies if the poison
175       entry is the last matching entry.
176
177       Control messages follow slightly different propagation rules than
178       normal articles; see innd(8) for the details.  Note that most
179       subscriptions should have "!junk,!control,!control.*" in their pattern
180       list due to those propagation rules (and since "junk" is a special
181       internal newsgroup; see wanttrash in inn.conf(5) for more details on
182       what it's used for) and that the best way to keep control messages
183       local to a site is with a distribution.
184
185       A subscription can be further modified by specifying distributions that
186       the site should or should not receive.  The default is to send all
187       articles to all sites that subscribe to any of the groups where it has
188       been posted, but if an article has a Distribution header field and any
189       distributions are specified, then they are checked according to the
190       following rules:
191
192       1.  If the Distribution header field matches any of the values in the
193           sub-field, the article is sent.
194
195       2.  If a distribution starts with an exclamation point, and it matches
196           the Distribution header field, the article is not sent.
197
198       3.  If the Distribution header field does not match any distribution in
199           the site's entry and no negations were used, the article is not
200           sent.
201
202       4.  If the Distribution header field does not match any distribution in
203           the site's entry and any distribution started with an exclamation
204           point, the article is sent.
205
206       If an article has more than one distribution specified, then each one
207       is handled according according to the above rules.  If any of the
208       specified distributions indicate that the article should be sent, it
209       is; if none do, it is not sent.  In other words, the rules are used as
210       a logical or.
211
212       It is almost definitely a mistake to have a single feed that specifies
213       distributions that start with an exclamation point along with some that
214       don't.
215
216       Distributions are text words, not patterns; entries like "*" or "all"
217       have no special meaning.
218
219       The flag field is described in "FLAG VALUES".  The interpretation of
220       the parameter field depends on the type of feed and is explained in
221       more detail in "FEED TYPES".  It can be omitted for some types of
222       feeds.
223
224       The site named "ME" is special.  There must be exactly one such entry,
225       and it should be the first entry in the file.  If the "ME" entry has an
226       exclusion sub-field, incoming articles are rejected completely if any
227       of the names specified in that exclusion sub-field appear in their Path
228       header field bodies.  If the "ME" entry has a subscription list, that
229       list is prepended to the subscription list of all other entries.  For
230       example, "*,!control,!control.*,!junk,!foo.*" could be used to set the
231       default subscription list for all other feeds so that local postings
232       are not propagated unless "foo.*" explicitly appears in the site's
233       subscription list.  This feature tends to be somewhat confusing since
234       the default subscription is prepended and can be overridden by other
235       patterns.
236
237       If the "ME" entry has a distribution sub-field, only articles that
238       match that distribution list are accepted and all other articles with a
239       distribution are rejected.  A common use for this is to put something
240       like "/!local" in the "ME" entry to reject local postings from other
241       misconfigured sites.  The distribution sub-field of "ME" has no effect
242       on the acceptance or rejection of articles that do not have a
243       Distribution header field.
244
245       An empty "ME" entry is possible, in which case no exclusion patterns
246       will be defined.
247
248       Finally, it is also possible to set variables in newsfeeds and use them
249       later in the file.  A line starting with "$" sets a variable.  For
250       example:
251
252           $LOCALGROUPS=local.*,example.*
253           $CONTROLGROUPS=control,control.*
254
255       This sets the variable "LOCALGROUPS" to "local.*,example.*" and the
256       variable "CONTROLGROUPS" to "control,control.*".  They can later be
257       used elsewhere in the file, such as in a site entry like:
258
259           news.example.com:$LOCALGROUPS:Tf,Wnm:
260
261       which is then completely equivalent to:
262
263           news.example.com:local.*,example.*:Tf,Wnm:
264
265       Variables aren't solely simple substitution.  If either "!" or "@"
266       immediately preceds the variable and the value of the variable contains
267       commas, that character will be duplicated before each comma.  This
268       somewhat odd-sounding behavior is designed to make it easier to use
269       variables to construct feed patterns.  The utility becomes more obvious
270       when you observe that the line:
271
272           news.example.net:*,@$LOCALGROUPS,!$CONTROLGROUPS:Tf,Wnm:
273
274       is therefore equivalent to:
275
276           news.example.net:*,@local.*,@example.*,!control,!control.*:Tf,Wnm:
277
278       which (as explained below) excludes all of the groups in $LOCALGROUPS
279       and unwanted control articles from the feed to that site.
280

FLAG VALUES

282       The flags parameter specifies miscellaneous parameters, including the
283       type of feed, what information should be sent to it, and various
284       limitations on what articles should be sent to a site.  They may be
285       specified in any order and should be separated by commas.  Flags that
286       take values should have the value immediately after the flag letter
287       with no whitespace.  The valid flags are:
288
289       < size
290           An article will only be sent to this site if it is less than size
291           bytes long.  The default is no limit.
292
293       > size
294           An article will only be sent to this site if it is greater than
295           size bytes long.  The default is no limit.
296
297       A checks
298           An article will only be sent to this site if it meets the
299           requirements specified in checks, which should be chosen from the
300           following set.  checks can be multiple letters if appropriate.
301           Note that this flag is not effective on funnel targets; it has to
302           be used on every funnel entry (for instance, Af is not effective on
303           the innfeed! funnel target and therefore has to be specified on
304           every funnelled news site).
305
306           c  Exclude all kinds of control messages.
307
308           C  Only send control messages, not regular articles.
309
310           d  Only send articles with a Distribution header field.  Combined
311              with a particular distribution value in the distribution part of
312              the site entry, this can be used to limit articles sent to a
313              site to just those with a particuliar distribution.
314
315           e  Only send articles where every newsgroup listed in the
316              Newsgroups header field body exists in the active file.
317
318           f  Don't send articles rejected by filters.  This is only useful
319              when dontrejectfiltered is set to true in inn.conf.  With that
320              variable set, this lets one accept all articles but not
321              propagate filtered ones to some sites.
322
323           j  Propagate articles according to their Newsgroups header field.
324              This is only useful when wanttrash is set to true in inn.conf.
325              With that variable set, articles accepted and filed in "junk"
326              (due to wanttrash) are fed to peers based on their subscription
327              pattern applied to the Newsgroups header field as though they
328              were accepted and all those groups were locally carried.
329              Otherwise, they are propagated to sites that receive the "junk"
330              newsgroup.
331
332              This variable is useful if you want to run INN with a minimal
333              active file and propagate all posts.
334
335           o  Only send articles for which overview data was stored.
336
337           O  Send articles to this site that don't have an Injection-Info or
338              X-Trace header field, even if the "O" flag is also given.
339
340           p  Only check the exclusions against the Path header field of
341              articles; don't check the site name.  This is useful if your
342              site names aren't the same as the path identities in Path header
343              fields added by those remote sites, or for program feeds where
344              the site name is arbitrary and unrelated to the Path header
345              field body.
346
347           If both "c" and "C" are given, the last specified one takes
348           precedence.
349
350       B high/low
351           If a site is being fed by a file, channel, or exploder (see below),
352           the server will normally start trying to write the information as
353           soon as possible.  Providing a buffer may give better system
354           performance and help smooth out overall load if a large batch of
355           news comes in.  The value of this flag should be two numbers
356           separated by a slash.  high specifies the point at which the server
357           can start draining the feed's I/O buffer, and low specifies when to
358           stop writing and begin buffering again; the units are bytes.  The
359           default is to do no buffering, sending output as soon as it is
360           possible to do so.
361
362       C count
363           If this flag is specified, an article will only be sent to this
364           site if the number of groups it is posted to, plus the square of
365           the number of groups followups would appear in, is no more than
366           count.  30 is a good value for this flag, allowing for instance:
367
368           •  crossposts to only 5 groups when followups aren't set;
369
370           •  crossposts to up to 26 groups when followups are set to two
371              groups;
372
373           •  crossposts to up to 29 groups when followups are set to a single
374              group;
375
376           •  crossposts to up to 30 groups when followups are set to poster.
377
378           Note that if an article does not contain a Followup-To header
379           field, the number of groups followups would appear in is the number
380           of groups it is posted to.
381
382       F name
383           Specifies the name of the file that should be used if it's
384           necessary to begin spooling for the site (see below).  If name is
385           not an absolute path, it is taken to be relative to pathoutgoing in
386           inn.conf.  If name is a directory, the file togo in that directory
387           will be used as the file name.
388
389       G count
390           If this flag is specified, an article will only be sent to this
391           site if it is posted to no more than count newsgroups.  This has
392           the problem of filtering out many FAQs, as well as newsgroup
393           creation postings and similar administrative announcements.  Either
394           the C flag or the U flag is a better solution.
395
396       H count
397           If this flag is specified, an article will only be sent to this
398           site if it has count or fewer sites in its Path header field body.
399           This flag should only be used as a rough guide because of the loose
400           interpretation of the Path header field; some sites put the
401           poster's name in the header field body, and some sites that might
402           logically be considered to be one hop become two because they put
403           the posting workstation's name in the header field body.  The
404           default value for count if not specified is one.  (Also see the O
405           flag, which is sometimes more appropriate for some uses of this
406           flag.)
407
408       I size
409           The flag specifies the size of the internal buffer for a file feed.
410           If there are more file feeds than allowed by the system, they will
411           be buffered internally in least-recently-used order.  If the
412           internal buffer grows bigger than size bytes, however, the data
413           will be written out to the appropriate file.  The default value is
414           16 KB.
415
416       N status
417           Restricts the articles sent to this site to those in newsgroups
418           with the moderation status given by status.  If status is "m", only
419           articles in moderated groups are sent; if status is "u", only
420           articles in unmoderated groups are sent.
421
422       O originator
423           If this flag is specified, an article will only be sent to this
424           site if it contains an Injection-Info header field (or an X-Trace
425           header field if no Injection-Info header field is found) and the
426           first field of this header field matches originator.  originator is
427           a uwildmat expression without commas or a list of such expressions,
428           separated by "/".  The article is never sent if the first character
429           of the pattern begins with "@" and the rest of the pattern matches.
430           One use of this flag is to restrict the feed to locally generated
431           posts by using an originator pattern that matches the Injection-
432           Info header field added by the local server.
433
434       P priority
435           The nice priority that this channel or program feed should receive.
436           This should be a positive number between 0 and 20 and is the
437           priority that the new process will run with.  This flag can be used
438           to raise the priority to normal if you're using the nicekids
439           parameter in inn.conf.
440
441       Q hashfeed
442           Specifies the hashfeed match expression for this site.  It must be
443           in the form "value/mod" or "start-end/mod".  The Message-ID of the
444           article is hashed using MD5, which results in a 128-bit hash.  The
445           lowest 32 bits are then taken by default as the hashfeed value
446           (which is an integer).  If the hashfeed value modulus "mod" plus
447           one equals "value" or is between "start" and "end", the article
448           will be fed to this site.  All these numbers must be integers.
449
450           It is a deterministic way to control the flow of articles and to
451           split a feed.  For instance:
452
453               Q1/2      Feeds about 50% of all articles to this site.
454               Q2/2      Feeds the other 50% of all articles.
455
456           Another example with three sites:
457
458               Q1-3/10   Feeds about 30% of all articles.
459               Q4-5/10   Feeds about 20% of all articles.
460               Q6-10/10  Feeds about 50% of all articles.
461
462           If this flag is specified multiple times, the contents will be
463           logically "OR"ed together (just one match is needed).
464
465           You can use an extended syntax of the form "value/mod_offset" or
466           "start-end/mod_offset".  As MD5 generates a 128-bit return value,
467           it is possible to specify from which byte-offset the 32-bit integer
468           used by hashfeed starts.  The default value for "offset" is "_0"
469           and thirteen overlapping values from "_0" to "_12" can be used.
470           Only up to four totally independent values exist: "_0", "_4", "_8"
471           and "_12".
472
473           Therefore, it allows generating a second level of deterministic
474           distribution.  Indeed, if a news server is fed "Q1/2", it can go on
475           splitting thanks to "Q1-3/9_4" for instance.  Up to four levels of
476           deterministic distribution can be used.
477
478           The algorithm is compatible with the one used by Diablo 5.1 and up.
479           If you want to use the legacy quickhashing method used by Diablo
480           before 5.1, you can put an "@" sign just after the Q flag (for
481           instance "Q@1-3/10", but the distribution of the messages is not
482           perfect with this legacy method whose use is discouraged and for
483           which offsets cannot be used).
484
485       S size
486           If the amount of data queued for the site gets to be larger than
487           size bytes, the server will switch to spooling, appending to a file
488           specified by the F flag, or pathoutgoing/sitename if F is not
489           specified.  Spooling usually happens only for channel or exploder
490           feeds, when the spawned program isn't keeping up with its input.
491
492       T type
493           This flag specifies the type of feed for this site.  type should be
494           a letter chosen from the following set:
495
496               c        Channel
497               f        File
498               l        Log entry only
499               m        Funnel (multiple entries feed into one)
500               p        Program
501               x        Exploder
502
503           Each feed is described below in "FEED TYPES".  The default is Tf,
504           for a file feed.
505
506       U count
507           If this flag is specified, an article will only be sent to this
508           site if followups to this article would be posted to no more than
509           count newsgroups.  (Also see C for a more complex way of handling
510           this.)
511
512       W items
513           For a file, channel, or exploder feed, this flag controls what
514           information will be sent to this site.  For a program feed, only
515           the asterisk ("*") has any effect.  items should be chosen from the
516           following set:
517
518           b  Size of the article (in wire format, meaning with CRLF at the
519              end of each line, periods doubled at the beginning of lines, and
520              ending in a line with a single period) in bytes.
521
522           e  The time the article will expire as seconds since epoch if it
523              has an Expires header field, 0 otherwise.
524
525           f  The storage API token of the article (the same as "n").  The
526              article can be retrieved given the storage API token by using
527              sm(1).
528
529           g  The newsgroup the article is in; if cross-posted, then the first
530              of the groups to which the article was posted that this site
531              gets.  (The difference from "G" is that this sends the newsgroup
532              to which the article was posted even if it is a control
533              message.)
534
535           h  The history hash key of the article (derived from the message
536              ID).
537
538           m  The message ID of the article.
539
540           n  The storage API token of the article.  The article can be
541              retrieved given the storage API token by using sm(1).
542
543           p  The time the article was posted a seconds since epoch.
544
545           s  The site that fed the article to the server.  This is taken from
546              either the Path header field body or the IP address of the
547              sending site depending on the value of logipaddr in inn.conf.
548              If logipaddr is true and the IP address is 0.0.0.0 (meaning that
549              the article was fed from localhost by a program like rnews(1)),
550              the Path header field body will be sent instead.
551
552           t  The time the article was received as seconds since epoch.
553
554           *  The names of the appropriate funnel entries, or all sites that
555              get the article (see below for more details).
556
557           D  The body of the Distribution header field of the article, or "?"
558              if there is no such header field in the article.
559
560           G  Where the article is stored.  If the newsgroup is crossposted,
561              this is generally the first of the groups to which it was posted
562              that this site receives; however, control messages are filed in
563              control or control.*  (which is the difference between this item
564              and "g").
565
566           H  All of the headers, followed by a blank line.  The Xref header
567              field will already be present, and a Bytes header field
568              containing the article's size in bytes as in the "b" item will
569              be added to the headers.  If used, this should be the only item
570              in the list.
571
572           N  The body of the Newsgroups header field.
573
574           P  The body of the Path header field.
575
576           O  Overview data for the article.
577
578           R  Information needed for replication (the Xref header field
579              without the site name).
580
581           More than one letter can be given.  If multiple items are
582           specified, they will be written in the order specified separated by
583           spaces.  ("H" should be the only item if given, but if it's not a
584           newline will be sent before the beginning of the headers.)  The
585           default is Wn.
586
587           The "H" and "O" items are intended for use by programs that create
588           news overview databases or require similar information.  WnteO is
589           the flag to generate input needed by the overchan(8) program.
590
591           The asterisk ("*") has special meaning.  Normally it expands to a
592           space-separated list of all sites that received the current
593           article.  If, however, this site is a target of a funnel feed (in
594           other words, if it is named by other sites which have the Tm flag),
595           then the asterisk expands to the names of the funnel feeds that
596           received the article.  Similarly, if the site is a program feed, an
597           asterisk in the parameter field will be expanded into the list of
598           funnel feeds that received the article.  A program feed cannot get
599           the site list unless it is the target of other Tm feeds.
600

FEED TYPES

602       innd provides four basic types of feeds: log, file, program, and
603       channel.  An exploder is a special type of channel.  In addition,
604       several entries can feed into the same feed; these are funnel feeds,
605       which refer to an entry that is one of the other types.  Funnel feeds
606       are partially described above with the description of the W* flag.  A
607       funnel feed gets every article that would be sent to any of the feeds
608       that funnel into it and normally include the W* flag in their flags so
609       that the program processing that feed knows which sites received which
610       articles.  The most common funnel feed is innfeed(8).
611
612       Note that the term "feed" is technically a misnomer, since the server
613       doesn't transfer articles itself and only writes data to a file,
614       program, or log telling another program to transfer the articles.
615
616       The simplest feed is a log feed (Tl).  Other than a mention in the news
617       log file, pathlog/news, no data is written out.  This is equivalent to
618       a Tf entry writing to /dev/null, except that no file is ever opened.
619       Flushing a log feed does nothing.
620
621       A file feed (Tf) is the next simplest type of feed.  When the site
622       should receive an article, the specified data is written out to the
623       file named by the parameter field.  If parameter is not an absolute
624       path, it is taken to be relative to pathoutgoing in inn.conf.  If
625       parameter is not given, it defaults to pathoutgoing/sitename.  The file
626       name should be unique (two file feeds should not ever point to the same
627       file).
628
629       File feeds are designed for use by external programs that periodically
630       process the written data.  To cooperate with innd properly, such
631       external programs should first rename the batch file and then send a
632       flush command for that site to innd using ctlinnd(8).  innd will then
633       write out any buffered data, close the file, and reopen it (under the
634       original name), and the program can process the data in the renamed
635       file at its leisure.  File feeds are most frequently used in
636       combination with nntpsend(8).
637
638       A program feed (Tp) spawns a given program for every article that the
639       site receives.  The parameter field must be the command line to
640       execute, and should contain one instance of %s, which will be replaced
641       by the storage API token of the article (the actual article can be
642       retrieved by the program using sm(1)).  The program will not receive
643       anything on standard input (unlike earlier versions of INN, where the
644       article is sent to the program on stdin), and standard output and error
645       from the program will be set to the error log (pathlog/errlog).  innd
646       will try to avoid spawning a shell if the command has no shell meta-
647       characters; this feature can be defeated if necessary for some reason
648       by appending a semi-colon to the end of the command.  The full path
649       name of the program to be run must be specified unless the command will
650       be run by the shell (and it is strongly recommended that the full path
651       name always be specified regardless).
652
653       If a program feed is the target of a funnel, and if W* appears in the
654       flags of the site, a single asterisk may be present in the parameter
655       and will be replaced by a space-separated list of names of the sites
656       feeding into the funnel which received the relevant article.  If the
657       site is not the target of a funnel, or if the W* flag is not used, the
658       asterisk has no special meaning.
659
660       Flushing a program feed does nothing.
661
662       For a channel (Tc) or exploder (Tx) feed, the parameter field again
663       names the process to start.  As with program feeds, the full path to
664       the program must be specified.  However, rather than spawning the
665       program for every article, it is spawned once and then whenever the
666       site receives an article, the data specified by the site flags is
667       written to the standard input of the spawned program.  Standard output
668       and error are set as with program feeds.  If the process exits, it will
669       be restarted automatically.  If the process cannot be started, the
670       server will spool input to a file named pathoutgoing/sitename and will
671       try to start the process again later.
672
673       When a channel or exploder feed is flushed, the server closes its end
674       of the pipe to the program's standard input.  Any pending data that has
675       not been written will be spooled; see the description of the S flag
676       above.  The server will then spawn a new instance of the program.  No
677       signal is sent to the program; it is up to the program handling a
678       channel or exploder feed to notice end of file on its standard input
679       and exit appropriately.
680
681       Exploders are a special type of channel feed.  In addition to the
682       channel feed behavior described above, exploders can also be sent
683       command lines.  These lines start with an exclamation point and their
684       interpretation is up to the exploder.  The following commands are
685       generated automatically by the server:
686
687           !newgroup group
688           !rmgroup group
689           !flush
690           !flush site
691
692       These commands are sent whenever the ctlinnd(8) command of the same
693       name is received by the server.  In addition, the ctlinnd(8) "send"
694       command can be used to send an arbitrary command line to an exploder.
695       The primary exploder is buffchan(8).
696
697       Finally, Tm feeds are the input to a funnel.  The parameter field of
698       the site should name the site handling articles for all of the funnel
699       inputs.
700

EXAMPLES

702       The syntax of the newsfeeds file is so complex because you can specify
703       a staggering variety of feeds.  INN is capable of interacting with a
704       wide variety of programs that do various things with news articles.
705       Far and away the most common two entries in newsfeeds, however, are
706       file feeds for nntpsend(8) and funnel feeds for innfeed(8).
707
708       The former look like this:
709
710           feed.example.com:*,!control,!control.*,!junk:Tf,Wnm:
711
712       which generates a file named pathoutgoing/feed.example.com containing
713       one line per article consisting of the storage API token, a space, and
714       the message ID.
715
716       The latter look like this:
717
718           feed.example.com:*,!control,!control.*,!junk:Tm:innfeed!
719
720       Very similar, except that this is the input to a funnel feed named
721       "innfeed!".  One could also write this as:
722
723           example/feed.example.com:*,!control,!control.*,!junk:Ap,Tm:innfeed!
724
725       (note the Ap so that articles that contain just "example" in the Path
726       header field body will still be sent), which is completely equivalent
727       except that this will be logged in pathlog/news as going to the site
728       "example" rather than "feed.example.com".
729
730       The typical feed entry for innfeed(8) is a good example of a channel
731       feed that's the target of various funnel feeds (make sure the path to
732       innfeed is properly set):
733
734           innfeed!:!*:Tc,Wnm*:<pathbin in inn.conf>/innfeed -y
735
736       Note that the pattern for this feed is just "!*" so that it won't
737       receive any articles directly.  The feed should only receive those
738       articles that would go to one of the funnel feeds that are feeding into
739       it.  innfeed(8) will receive one line per article on its standard input
740       containing the storage API token, the message ID, and a space-separated
741       list of sites that should receive that article.
742
743       Here's a more esoteric example of a channel feed:
744
745           watcher!:*:Tc,Wbnm\
746               :exec awk '$1 > 1000000 { print "BIG", $2, $3 }' >> <pathlog>/bigart
747
748       This receives the byte size of each article along with the storage API
749       token and Message-ID, and appends to a bigart log file a line for every
750       article that's over a million bytes.  This is actually rather a strange
751       way to write this since INN can do the size check itself; the following
752       is equivalent:
753
754           watcher!:*:Tc,>1000000,Wbnm\
755               :exec awk '{ print "BIG", $2, $3}' >> <pathlog>/bigart
756
757       Here's a cute, really simple news to mail gateway that also serves as
758       an example of a fairly fancy program feed:
759
760           mailer!:!*:W*,Tp\
761               :sm %s | innmail -s "News article" *
762
763       Remember that %s is replaced by the storage API token, so this
764       retrieves the article and pipes it into innmail (which is safer than
765       programs like Mail(1) because it doesn't parse the body for tilde
766       commands) with a given subject line.  Note the use of "*" in the
767       command line and W* in the flags; this entry is designed to be used as
768       the target of funnel feeds such as:
769
770           peter@example.com:news.software.nntp:Tm:mailer!
771           sue@example.com:news.admin.misc:Tm:mailer!
772
773       Suppose that the server receives an article crossposted between
774       news.admin.misc and news.software.nntp.  The server will notice that
775       the article should be sent to the site "peter@example.com" and the site
776       "bob@example.com", both of which funnel into "mailer!", so it will look
777       at the "mailer!" site and end up executing the command line:
778
779           sm @...@ | innmail -s "News article" peter@example.com sue@example.com
780
781       which will mail the article to both Peter and Sue.
782
783       Finally, another very useful example of a channel feed: the standard
784       entry for controlchan(8).  Make sure its path is properly set.
785
786           controlchan!\
787               :!*,control,control.*,!control.cancel\
788               :AC,Tc,Wnsm:<pathbin in inn.conf>/controlchan
789
790       This program only wants information about articles posted to a control
791       newsgroup other than control.cancel, which due to the sorting of
792       control messages described in innd(8) will send it all control messages
793       except for cancel messages.  controlchan gets the storage API token,
794       the name of the sending site (for processing old-style ihave and sendme
795       control messages, be sure to read about logipaddr in controlchan(8)),
796       and the message ID for each article.
797
798       For many other examples, including examples of the special "ME" site
799       entry, see the example newsfeeds file distributed with INN.  Also see
800       the install documentation that comes with INN for information about
801       setting up the standard newsfeeds entries used by most sites.
802

HISTORY

804       Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews.
805       Reformatted and rewritten in POD by Russ Allbery <eagle@eyrie.org>.
806

SEE ALSO

808       active(5), buffchan(8), controlchan(8), ctlinnd(8), inn.conf(5),
809       innd(8), innfeed(8), innxmit(8), libinn_uwildmat(3), nntpsend(8).
810
811
812
813INN 2.7.0                         2022-07-10                      NEWSFEEDS(5)
Impressum