1leafnode(8) System Manager's Manual leafnode(8)
2
6 leafnode - NNTP server for small (dialup) sites
7
10 leafnode
11
14 Leafnode is a USENET package intended for small sites, where there are
15 few users and little disk space, but where a large number of groups is
16 desired.
17 The design of leafnode is intended to self-repair after problems, and
19 to require no manual maintenance.
20 The leafnode program itself is the NNTP server. It is run from
22 inetd(8), xinetd(8) or tcpserver when someone wants to read news. The
23 other parts of the package, fetchnews and texpire, are responsible for
24 fetching new news from another server, and for deleting old news.
25
28 No authentication or access control is supported. This is a deliberate
29 omission: Implementing this is a job which should not be redone for
30 each and every service.
31 It is mandatory that you use external access control mechanisms like
33 tcpd, inetd/xinetd compiled with libwrap support, tcpserver with -x
34 option and the like and that these are in effect. tcpd and libwrap are
35 components of Wietse Venema's fine tcp_wrappers package.
36 As a very rough last line of defense against abuse, leafnode will drop
38 connections from outside your LANs by default. You can configure leafn‐
39 ode to let go of this restriction (look for the allowstrangers option),
40 but don't do that unless tight access control is in place. Someone
41 will abuse your computer sooner or later. Promised.
42 I recommend that either firewalling or tcpd be used for access control.
44
47 All these files and directories must be readable by the user "news". It
48 is recommended that, unless otherwise stated, that the user "news" be
49 the only user in the group "news" and these files belong to "root:news"
50 (user:group) so leafnode cannot modify your configuration or filter
51 files.
52 /etc/leafnode should not be writable by the user "news", but it must be
54 executable for at least any of the group that the user "news" is in.
55 /etc/leafnode/config contains the configuration parameters for leafn‐
56 ode. It must not be writable by the user "news". Set this to owner
57 root:news and mode 640. For details, see CONFIGURATION below.
58 /var/spool/news must also be readable and writable by the user "news".
60 It contains the news articles; e.g. /var/spool/news/alt/fan/agulbra
61 contains the articles in the alt.fan.agulbra group. Each directory
62 contains articles in numbered files (decimal numbers, monotonically
63 increasing), and a special file called .overview which contains the
64 "Subject", "From", "Date", "Message-ID", "References", "Bytes" and
65 "Lines" headers for each article in the group.
66 Several subdirectories are special:
68 /var/spool/news/leaf.node contains the files that leafnode creates dur‐
70 ing operation, for example the groupinfo file which contains informa‐
71 tion about each USENET newsgroup. This file is built by fetchnews [22m(8).
72 You can force a complete rebuild of the groupinfo file by calling
73 fetchnews with the parameter -f (see fetchnews (8)).
74 /var/spool/news/out.going contains local postings that fetchnews(8) is
76 to pass to the upstream NNTP server. After a posting has been success‐
77 fully written to disk, its u+r permission flag is set. This flag is
78 interpreted by fetchnews(8) as "you may post this article". This pre‐
79 vents fetchnews from posting articles that are still being received
80 from newsreaders. (Note: versions 1.9.23 to 1.9.32 inclusively used u+x
81 instead, which caused some "stuck post" problems with articles in the
82 spool when a prior leafnode version was updated to one of these 10 ver‐
83 sions. Updating to leafnode 1.9.33 or later fixes the problem.)
84 /var/spool/news/failed.postings contains local postings that the
86 upstream server rejected. fetchnews(8) will create files in this
87 directory, but none of the leafnode programs will delete anything in
88 it.
89 /var/spool/news/message.id contains hard links to each message; this is
91 used in place of the dbz database typically used by bigger servers. (A
92 directory such as this is probably more efficient for the small servers
93 leafnode is designed for but scales very badly.)
94 /var/spool/news/interesting.groups contains one file for each group an
96 NNTP client has asked to read. leafnode will update the ctime (ls -l
97 usually shows the mtime, try ls -lc) of the relevant file when a LIST‐
98 GROUP, XOVER, XHDR, STAT, HEAD, BODY or ARTICLE command is issued, when
99 a GROUP or LIST ACTIVE command (the latter only with a single group,
100 not with patterns) is issued for an interesting group (to avoid unsub‐
101 scribing low-traffic groups that are still read) and fetchnews(8) will
102 retrieve all new articles in all groups whose files have been either
103 - touched during the past two days, or
105 - touched more than once, and at least once within the past
107 week.
108 The timeout is configurable through the config file variables time‐
110 out_short and timeout_long. See also fetchnews(8) for the -n option.
111 /etc/inetd.conf or /etc/xinetd.conf contains the configuration which
113 starts leafnode. It is strongly recommended to start leafnode as user
114 news.
115
118 LN_REJECT_POST_PRE
119 If this variable exists, all POST commands are rejected with a
120 400 code. Use only for debugging clients.
121 LN_REJECT_POST_POST
123 If this variable exists, the POST command is rejected with a 400
124 code after the article and CRLF.CRLF has been received. Use only
125 for debugging clients.
126
129 All configuration is done using the file /etc/leafnode/config, which
130 may include a filter description file, filterfile for short, as
131 described below.
132 For the purposes of this section, whitespace shall be defined as an
134 arbitrary sequence consisting of one or more SPACE or HTAB characters,
135 ASCII positions 32 and 9, respectively.
136 The configuration file is strictly line-oriented with LF or CRLF as
138 line terminator.
139 Empty lines and lines consisting of only whitespace, possibly followed
141 by a comment (introduced by a hash mark (#) and extending through the
142 end of the line), are skipped.
143 All other lines have exactly three mandatory fields, a plain text
145 parameter, an assignment character (=) optionally surrounded by white‐
146 space and a value. The value is either plain text or - new since
147 leafnode v1.11 - a string in double quotes with trivial backslash
148 escape (see below).
149 Plain text starts at the first non-whitespace character and extends
151 through the last non-whitespace character on the line that is not a
152 comment. A trailing comment on a line is skipped.
153 Quoted strings are enclosed in double quote characters ("). The back‐
155 slash character (\) is skipped, but it copies the immediately following
156 character verbatim, so that you can specify the backslash itself by
157 doubling it (\\) or a double quote character as part of the string by
158 preceding it with a backslash (\"); the hash mark has no special mean‐
159 ing as command introducer inside quoted strings. Text after the end of
160 the string is silently ignored (this may change in future versions).
161 Comments after quoted strings are ignored.
162 MANDATORY PARAMETERS
164 These parameters must be specified for leafnode to work.
166 server = news02.example.com
168 "server" is used by fetchnews (8) to select what NNTP server(s)
169 to retrieve news from and to post your articles to. You can
170 specify more than one news server; in that case, the servers
171 will be queried from the top down. If you want to post articles,
172 at least one of your servers should allow you to post. In the
173 example above, news02.example.com is the news server.
174 This parameter can be given more than once. Each server starts
176 with a fresh set of default configuration options, no inheri‐
177 tance takes place from the previous server definition. Only
178 options explicitly marked "server-specific" can be set on a per
179 server basis, "general" options are set for all servers at the
180 same time.
181 expire = 5
183 "expire" is the number of days an article should be kept around.
184 In the example, five days after the article has last been read,
185 it is deleted by texpire (8). This value MUST be at least 1.
186 This parameter is global, see the introductory paragraph of the
187 following GENERAL OPTION PARAMETERS section to find out what
188 this means.
189 GENERAL OPTIONAL PARAMETERS
191 These options can only be configured once in the configuration file,
193 and take effect for leafnode as a whole. It does not matter where these
194 are specified relative to server= options, but for clarity, you are
195 encouraged to place these before the first server= line. Specifying
196 each of the global options more than once lets the last copy take
197 effect, but may cause errors in the future.
198 hostname = host.domain.country
200 By default, leafnode tries hard to figure the host name of your
201 computer, skipping inadequate (non-unique) names if possible. It
202 will look up your computer's host name with gethostname(3) and
203 then try to qualify the name with gethostbyname(3) if necessary.
204 Common sources for the full name therefore are /etc/hosts, NIS
205 and DNS, but consult your system documentation for details.
206 If leafnode fails to determine the host name, this is usually a
208 hint that your system is not configured properly, or it has a
209 hostname that is unsuitable for the domain part of a Message-ID,
210 for example, "localhost.localdomain", and you should fix the
211 name service configuration. Adding a unique fully-qualified host
212 name to /etc/hosts is usually sufficient. Please see README-FQDN
213 for more details.
214 You can configure the unique fully-qualified host name here as
216 well, but this is not recommended and discouraged.
217 create_all_links = 1
219 Normally, fetchnews will store articles only in the newsgroups
220 which it considers interesting. With this option set, fetchnews
221 will create hardlinks for all newsgroups in the Newsgroups:
222 header that it knows about. This may be of interest if you want
223 to apply a score- or killfile to the local Xref: line.
224 maxfetch = 1000
226 "maxfetch" specifies the maximum number of articles fetchnews
227 (8) should fetch from the upstream server in each group. Its use
228 is not advised, because if you use it you will not see all the
229 traffic in a group. By default there is no limit.
230 initialfetch = 1
232 "initialfetch" defines how many articles from a newly subscribed
233 group should be fetched. The default is to fetch all old arti‐
234 cles, which can get quite time-consuming when subscribing to a
235 very busy group. This is equivalent to setting initialfetch to
236 zero. If you want to get no old articles when subscribing to a
237 new group, you should set initialfetch to one, as in the example
238 above.
239 groupexpire very.crowded.group = 1
241 groupexpire very.crowded.hierarchy.* = 1
243 "groupexpire" makes it possible to adjust expiry times for indi‐
244 vidual groups. Expiry times are given in days. 0 means "use the
245 default", negative values prevent the expire process for this
246 group altogether (you can consider this an archive mode). This
247 value is used by texpire (8). You can specify as many groupex‐
248 pire lines as you like. It is possible to specify glob [22m(7)-like
249 wildcard expressions.
250 maxage = 10
252 If an article turns up on your upstream news server which is
253 older than "maxage" days it will not been fetched even if you
254 don't have it yet. This is useful if your upstream server gets
255 occasional "hiccups". The default is set to 10. If you want to
256 switch this feature off, set maxage to some very large value,
257 such as 20000 (this is equivalent to roughly 54 years).
258 maxold = 10
260 Is synonymous to maxage, see above.
261 maxlines = 2000
263 If you want to avoid receiving very large articles, you may set
264 the "maxlines" parameter to the maximal number of lines an arti‐
265 cle should have. By default, this feature is switched off.
266 minlines = 2
268 Sometimes newsgroups are spammed with empty postings. To reject
269 these postings, you can set the "minlines" parameter. Setting
270 minlines to a value larger 4 is probably not a good idea since
271 you will also start to kill "real" postings then. By default,
272 this feature is switched off.
273 maxbytes = 100000
275 If you want to avoid receiving very large articles, instead of
276 using the "maxlines" parameter you can also use the "maxbytes"
277 parameter. By default, this feature is switched off.
278 maxcrosspost = 5
280 If you want to combat spam, you can filter out all postings that
281 are posted to more than a certain number of newsgroups. The num‐
282 ber is defined by setting "maxcrosspost". Setting this parameter
283 to very low values is probably a bad idea. This feature is
284 switched off by default.
285 maxgroups = 5
287 Synonymous for maxcrosspost. See above.
288 filterfile = /etc/leafnode/filters
290 Leafnode can filter the input headers for arbitrary regular
291 expressions. These are stored in a file designated "filter‐
292 file". The format of "filterfile" is very simple: one perl-com‐
293 patible regular expression per line. If one of the regular
294 expressions fits to a header to be downloaded, the body of that
295 article will be rejected. This feature is switched off by
296 default. The format of the regular expressions is described in
297 pcre(3).
298 timeout_short = 2
300 By default, a group that has been accidentally touched is being
301 fetched for two days. You can change this time by changing time‐
302 out_short.
303 timeout_long = 7
305 By default, a group that has not been read at all is being
306 fetched for seven days before being unsubscribed. This interval
307 can be changed by setting timeout_long to a different value.
308 timeout_active = 90
310 By default, active files from the upstream servers are re-read
311 every 90 days. This interval can be changed by setting time‐
312 out_active to a different value. Be aware that reading an active
313 file transfers about one MB of information if the server that
314 you are using carries a reasonable number of groups (i. e.
315 around 20,000).
316 timeout_client = 900 (since v1.9.23)
318 By default, leafnode will drop the connection 900 seconds (15
319 minutes) after seeing the last command from the client. You can
320 change the timeout here. Setting it too low (like below 5 min‐
321 utes) will annoy your users and consume more system resources
322 for re-reading all the files.
323 timeout_fetchnews = 300 (since v1.9.52)
325 Fetchnews will, since v1.9.52, assume the upstream server has
326 become wedged after waiting for a reply for 300 seconds. You can
327 change the timeout here.
328 timeout_lock = 5 (since v1.9.54)
330 Configure how many seconds the leafnode programs (applyfilter,
331 checkgroups, fetchnews, texpire) will wait for the lock file
332 before aborting. Setting this to 0 means to wait indefinitely.
333 NOTE: you can override this by setting the environment variable
334 LN_LOCK_TIMEOUT (note it is not LN_TIMEOUT_LOCK). The default
335 is 5 seconds.
336 delaybody = 1
338 With this option set, fetchnews (8) fetches only the headers of
339 an article for visual inspection. Only when the headers have
340 been read, the bodies of the articles will be retrieved the next
341 time fetchnews (8) is called. This can save a huge amount of
342 download time and disk space.
343 delaybody_in_situ = 1 (since v1.9.41)
345 This is only applicable with delaybody=1.
346 By default, leafnode will give the full downloaded article a new
348 article number so they appear as new in your newsreader. This
349 does not work for all newsreaders. With this option set, leafn‐
350 ode will retain the original article number. You'll have to fig‐
351 ure out how to tell your newsreader to show old articles. This
352 option defaults to 0. It is highly recommended to leave it
353 unset.
354 debugmode = 1
357 With this option set, fetchnews (8), texpire (8) and leafnode
358 (8) will start to log lots of debugging output via syslog (8) at
359 facility news and priority debug. Use it for tracking down prob‐
360 lems with your feed. debugmode should be left at 0 for regular
361 use because it can log enormous amounts of data. The higher the
362 number, the more will be logged. Choosing a figure greater than
363 3 will not make a difference at the moment.
364 allow_8bit_headers = 1 (since v1.9.25)
366 By default, leafnode rejects local posts that have 8-bit charac‐
367 ters in their headers, because they violate relevant standards,
368 particularly RFC-2822 (which RFC-1036 is based on) that demands
369 that Usenet news headers (as mail headers) must be pure 7-bit
370 US-ASCII, with only whitespace allowed from the control charac‐
371 ters.
372 However, as UTF-8 is to come, and some national hierarchies,
374 particularly the Norwegian and Danish (no.*, dk.*) seem to have
375 agreed on preferring just-send-eight over RFC-2047, you can set
376 this option to allow 8-bit data in headers. Leafnode will how‐
377 ever add a warning header if 8-bit data is present, stating that
378 the site administrator allowed this.
379 There is no way to make leafnode accept non-whitespace control
381 characters in headers.
382 allowSTRANGERS = MAGIC (since v1.9.23)
384 By default, leafnode refuses connections from outside your LANs.
385 Check config.example for how to use this parameter to let
386 strangers connect to your leafnode. Instead of MAGIC, you have
387 to write a number as mentioned in config.example. Note that cap‐
388 italization matters.
389 linebuffer = 1
391 By default, stdout and sometimes stderr of applications are set
392 to "fully buffered" unless connected to terminals. Use this
393 option to explicitly request line buffered mode for stdout and
394 stderr.
395 clamp_maxage = 0
397 By default, leafnode will derive a "maxage" argument from the
398 expire time that is applicable to the group (groupexpire if set,
399 expire otherwise), to prevent fetching articles again that were
400 once there and then cleared by texpire(8). Set clamp_maxage=0 to
401 get rid of this behaviour.
402 article_despite_filter = 1 (since v1.9.33)
404 By default, fetchnews will request HEAD and BODY separately if a
405 filter file is defined and delaybody is off. For high latency,
406 high throughput links (such as interleaved DSL or satellite
407 links), it may be faster to request head and body together with
408 an ARTICLE command and ignore the body if the filters apply
409 (though it may not be cheaper if you pay per MByte), enabling
410 this option will force leafnode to use the ARTICLE command in
411 spite of filters being defined. (Note that in delaybody mode,
412 HEAD and BODY will ALWAYS be requested separately.)
413 newsadmin = news@leafnode.example.org (since v1.9.47)
415 This option sets the From: address for the placeholder article,
416 it should be the news administrator's mail address. It defaults
417 to news@HOSTNAME, where HOSTNAME is leafnode's hostname.
418 SERVER-SPECIFIC OPTIONAL PARAMETERS
420 These options can only be placed after the server= line of the server
422 to which you would like these to apply, and they always pertain to the
423 preceding server= line. Specifying them before the first server= line
424 is an error.
425 username = myname
427 If any of your news servers requires authentification, you can
428 enter your username on that server here. This field may occur
429 multiple times, once after each server definition. See the
430 introduction of this CONFIGURATION section for information on
431 how to quote myname.
432 password = mypassword
434 If any of your news servers requires authentification, you can
435 enter your password on that server here. This field may occur
436 multiple times, once after each server definition. Since the
437 password is available in clear text, it is recommended that you
438 set the rights on the config file as restrictive as possible,
439 otherwise other users of your computer will be able to get your
440 password(s) from that file. See the introduction of this CONFIG‐
441 URATION section for information on how to quote mypassword.
442 port = 8000
444 By default, fetchnews tries to connect to its upstream news
445 servers on the NNTP port (119). If your servers run on a differ‐
446 ent port, you can specify those here. This field may occur mul‐
447 tiple times, once after each server definition.
448 Note: to modify the port your own leafnode servers listens on,
450 change the inetd.conf, xinetd.conf configuration file or the
451 tcpsvd/tcpserver command line. leafnode does not set up its lis‐
452 ten port itself.
453 timeout = 30
455 By default, leafnode tries to connect for 10 seconds to a server
456 and then gives up. If you have a slow server, you can try for a
457 longer time by setting the timeout higher (in this example, 30
458 seconds). The timeout can be tuned individually for each server.
459 noactive = ANYTHING (v1.9.25 ... v1.11.4)
461 noactive = 1 (since v1.11.5)
463 If this parameter is set, the active file is never downloaded
464 from this server. Use this for very slow servers unless they
465 carry groups that other servers don't offer. Leafnode versions
466 1.9.25 to 1.11.4 would always assume that "ANYTHING" had been 1.
467 "noactive = 0" is supported since v1.11.5.
468 nodesc = ANYTHING (until v1.11.4)
470 nodesc = 1 (since v1.11.5)
472 Some servers do not deliver news groups descriptions correctly
473 because they cannot parse the XGTITLE and LIST NEWSGROUPS com‐
474 mands. In that case, put this line after the "server" line.
475 Leafnode versions up to v1.11.4 would always assume that "ANY‐
476 THING" had been 1. "nodesc = 0" is supported since v1.11.5.
477 nopost = 1 (since v1.9.23)
479 Prevent posting to this server. You can use this if the upstream
480 won't let you post but still greet leafnode with 200 or if the
481 upstream doesn't forward your postings reliably.
482 noread = 1 (since v1.9.33)
484 Prevent fetching news articles or active files from this server.
485 You can use this if the upstream is good to post, but too slow
486 to fetch news from.
487 noxover = 1 (since v1.9.47)
489 Prevent the use of XOVER on the current server. Fetchnews will
490 use XHDR instead.
491 only_groups_match_all = 1 (since v1.9.52)
493 Usually, when cross-posting an article, fetchnews will post the
494 article if ANY group listed in the Newsgroups: header is matched
495 by the PCRE. With this option on, ALL groups listed in the
496 Newsgroups: header must match. This can be used to avoid "poi‐
497 son" groups when you have multiple upstream servers.
498 only_groups_pcre = PCRE (since v1.9.28)
500 This parameter lists the Perl-compatible regular expression of
501 groups that are fetched or posted to this server. The PCRE is
502 automatically anchored at the left hand side, so you can omit
503 the leading ^. Remember to escape dots, as in:
504 de\.comp\.|de\.comm\.
505 If this parameter is omitted, all groups are fetched from and
507 posted to this server.
508 Note: you must run fetchnews with the -f option after changing,
510 adding or removing any only_groups_pcre option.
511 Hint: you can use something like this to check your
513 only_groups_pcre settings:
514 cut -f1 -d" " @spooldir@/leaf.node/groupinfo \
515 | pcregrep 'PATTERN'
516 post_anygroup = 1 (since v1.9.37)
518 This parameter makes leafnode post on this server without check‐
519 ing if it carries the group an article is posted to. The default
520 is post_anygroup = 0, which means that leafnode will check with
521 a "GROUP" command if the server carries the group the articles
522 is posted into. Use this on post-only servers that don't allow
523 the "GROUP" command. Note: inconsiderate use of this parameter
524 may cause articles to end up in the failed.postings directory.
525 OBSOLETE PARAMETERS
527 supplement
529 is synonymous to server. Don't use it on new installations.
530 fqdn is synonymous to hostname. Don't use it on new installations.
532
535 Here are the NNTP commands supported by this server:
536 ARTICLE, BODY, DATE, GROUP, HDR, HEAD, HELP, LAST, LIST, LIST ACTIVE,
538 LIST ACTIVE.TIMES, LIST EXTENSIONS, LIST NEWSGROUPS, LIST OVERVIEW.FMT,
539 LISTGROUP, MODE, NEWGROUPS, NEXT, POST, OVER, SLAVE, STAT, XHDR, XOVER.
540 These commands follow RFC-977 and RFC-2980, except HDR and OVER which
541 are recognized in anticipation of current NNTP drafts.
542 Note that the syntax of HDR and OVER may change.
544
546 Leafnode is totally unaware of UTF-8 and will reject a client that
547 posts UTF-8 characters in the header. Current Usefor drafts claim all
548 article headers UTF-8 encoded Unicode. Leafnode still expects RFC-2047
549 instead which may eventually be phased out in favour of UTF-8.
550 Leafnode stops reading a line at the first NUL character.
552 Leafnode may not cope well with crosspostings that cross hierarchies if
554 you have multiple upstream feeds and use the only_groups_pcre configu‐
555 ration option.
556 Leafnode will only bother to determine the time zone offset for gener‐
558 ated Date: headers for posts that lack them on systems that offer the
559 tm_gmtoff member in struct tm (commonly BSD and GNU systems).
560
563 Written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995 Troll
564 Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47 22646949.
565 Modified by Cornelius Krasel <krasel@wpxx02.toxi.uni-wuerzburg.de>,
567 Randolf Skerka <Randolf.Skerka@gmx.de> and Markus Enzenberger
568 <enz@cip.physik.uni-muenchen.de>. Copyright of the modifications
569 1997-1999.
570 Modified by Matthias Andree <matthias.andree@gmx.de>, Copyright 1999 -
572 2002. Modified by Ralf Wildenhues <ralf.wildenhues@gmx.de>, Copyright
573 2002.
574 Jonathan Larmour <jifl@jifvik.org> contributed the timeout_client fea‐
576 ture.
577 Andreas Meininger <a.meininger@gmx.net> contributed the code to let
579 texpire ignore groupexpire = -1 groups.
580 Mark Brown <broonie@debian.org> added the noactive option.
582 Numerous contributions by other people.
584 The initial development of leafnode has been paid for by Uninett AS
586 (http://www.uninett.no/).
587
590 applyfilter(8), checkgroups(8), fetchnews(8), newsq(1), texpire(8).
591 tcpd(8), hosts_access(5), glob(7), pcre(3), RFC 977, RFC 2980.
593leafnode 1.11.7 leafnode(8)