1CYCBUFF.CONF(5)           InterNetNews Documentation           CYCBUFF.CONF(5)
2
3
4

NAME

6       cycbuff.conf - Configuration file for INN CNFS storage method
7

DESCRIPTION

9       This file defines the cyclical buffers that make up the storage pools
10       for CNFS (Cyclic News File System).  Some options controlling the
11       behavior of the CNFS storage system can also be set here.  cycbuff.conf
12       is required if the CNFS (Cyclic News File System) storage method is
13       used.  INN will look for it in pathetc (as set in inn.conf).
14
15       CNFS stores articles in logical objects called metacycbuffs.  Each
16       metacycbuff is in turn composed of one or more physical buffers called
17       cycbuffs.  As articles are written to the metacycbuff, each article is
18       written to the next cycbuff in the list in a round-robin fashion
19       (unless "sequential" mode is specified, in which case each cycbuff is
20       filled before moving on to the next).  This is so that you can
21       distribute the individual cycbuffs across multiple physical disks and
22       balance the load between them.  Note that in order to use any cycbuff
23       larger than 2 GB on 32-bit platforms (and some very rare 64-bit
24       platforms that aren't Linux), you need to build INN with the
25       --enable-largefiles option.
26
27       For information about how to configure INN to use CNFS, see
28       storage.conf(5).
29
30       Blank lines and lines beginning with a hash sign ("#") are ignored.
31       All other lines must be of one of the following forms:
32
33           cycbuffupdate:<interval>
34           refreshinterval:<interval>
35           cycbuff:<name>:<file>:<size>
36           metacycbuff:<name>:<buffer>[,<buffer>,...][:<mode>]
37
38       (where items enclosed in [] are optional).  Order is mostly not
39       significant, but all cycbuff lines must occur before all metacycbuff
40       lines.  Long lines can be continued on the next line by ending the line
41       with a backslash ("\").
42
43       cycbuffupdate:<interval>
44           Sets the number of articles written before the cycbuff header is
45           written back to disk to <interval>.  Under most operating systems,
46           the header doesn't have to be written to disk for the updated data
47           to be available to other processes on the same system that are
48           reading articles out of CNFS, but any accesses to the CNFS cycbuffs
49           over NFS will only see the data present at the last write of the
50           header.  After a system crash, all updates since the last write of
51           the CNFS header may be lost.  The default value, if this line is
52           omitted, is 25, meaning that the header is written to disk after
53           every 25 articles stored in that cycbuff.
54
55       refreshinterval:<interval>
56           Sets the interval (in seconds) between re-reads of the cycbuff
57           header to <interval>.  This primarily affects nnrpd and controls
58           the frequency with which it updates its knowledge of the current
59           contents of the CNFS cycbuffs.  The default value, if this line is
60           omitted, is 30.
61
62       cycbuff:<name>:<file>:<size>
63           Configures a particular CNFS cycbuff.  <name> is a symbolic name
64           for the buffer, to be used later in a metacycbuff line.  It must be
65           no longer than seven characters.  <file> is the full path to the
66           buffer file or block device, and must be no longer than 63
67           characters.  <size> is the length of the buffer in kilobytes (1 KB
68           is 1024 bytes).  If <file> is not a block device, it should be
69           <size> * 1024 bytes long.
70
71           If you're trying to stay under 2 GB, keep your sizes below 2097152.
72
73       metacycbuff:<name>:<buffer>[,<buffer>,...][:<mode>]
74           Specifies a collection of CNFS buffers that make up a single
75           logical storage location from the perspective of INN.  Metacycbuffs
76           are referred to in storage.conf as storage locations for articles,
77           so in order to actually put articles in a cycbuff, it has to be
78           listed as part of some metacycbuff which is then referenced in
79           storage.conf.
80
81           <name> is the symbolic name of the metacycbuff, referred to in the
82           options: field of "cnfs" entries in storage.conf.  It must be no
83           longer than eight characters.  <buffer> is the name of a cycbuff
84           (the <name> part of a cycbuff line), and any number of cycbuffs may
85           be specified, separated by commas.
86
87           If there is more than one cycbuff in a metacycbuff, there are two
88           ways that INN can distribute articles between the cycbuffs.  The
89           default mode, "INTERLEAVE", stores the articles in each cycbuff in
90           a round-robin fashion, one article per cycbuff in the order listed.
91           If the cycbuffs are of wildly different sizes, this can cause some
92           of them to roll over much faster than others, and it may not give
93           the best performance depending on your disk layout.  The other
94           storage mode, "SEQUENTIAL", instead writes to each cycbuff in turn
95           until that cycbuff is full and then moves on to the next one,
96           returning to the first and starting a new cycle when the last one
97           is full.  To specify a mode rather than leaving it at the default,
98           add a colon and the mode ("INTERLEAVE" or "SEQUENTIAL") at the end
99           of the metacycbuff line.
100
101       innd only reads cycbuff.conf on startup, so if you change anything in
102       this file and want innd to pick up the changes, you have to use
103       "ctlinnd xexec innd"; "ctlinnd reload all ''" is not sufficient.
104
105       When articles are stored, the cycbuff into which they're stored is
106       saved as part of the article token.  In order for INN to retrieve
107       articles from a cycbuff, that cycbuff must be listed in cycbuff.conf.
108       However, if INN should not write to a cycbuff, it doesn't need to be
109       (and shouldn't be) listed in a metacycbuff.
110
111       This provides an easy way to retire a cycbuff.  Just remove it from its
112       metacycbuff, leaving in the cycbuff line, and restart innd (with, for
113       example, "ctlinnd xexec innd").  No new articles will be put into the
114       cycbuff, but neither will any articles expire from it.  After you no
115       longer need the articles in the cycbuff, just remove it entirely from
116       cycbuff.conf.  Then all of the articles will appear to have been
117       deleted to INN, and the next nightly expire run will clean up any
118       remaining references to them.
119
120       Adding a new cycbuff just requires creating it (see below), adding a
121       cycbuff line, adding it to a metacycbuff, and then restarting innd.
122       Similarly, changing the allocation of an existing cycbuff from a
123       metacycbuff to another one just requires modifying the two metacycbuffs
124       accordingly, and then restarting innd.  (Only the cycbuff is noted
125       noted in the storage API tokens.)
126

CREATING CYCBUFFS

128       When creating a new cycbuff, there are two different methods for
129       creating the buffers in which the articles will be stored.
130
131       1.  Create a large file on top of a regular file system.  The easiest
132           way to do this is probably with dd(1), using a command like:
133
134               dd if=/dev/zero of=/path/to/cycbuff bs=1024 count=<size>
135
136           where <size> is the size from the cycbuff line in cycbuff.conf.
137           INSTALL contains a script that will generate these commands for you
138           from your cycbuff.conf file.
139
140           This is the simplest method, but has the disadvantage that very
141           large files on regular file systems can be fairly slow to access,
142           particularly at the end of the file, and INN incurs unnecessary
143           file system overhead when accessing the cycbuff.
144
145       2.  Use block devices directly.  If your operating system allows you to
146           call mmap() on block devices (Solaris and recent versions of Linux
147           do, FreeBSD at last report does not), this is the recommended
148           method since you can avoid all of the native file system overhead.
149
150           Note that some OSes do not support files larger than 2 GB, which
151           will limit the size you can make a single cycbuff, but you can
152           still combine many cycbuffs into each metacycbuff.  Very old
153           versions of Linux (before 2.4 kernels, that raised the limit to
154           2 TB) are known to have this limitation; FreeBSD does not.  Some
155           OSes that support large files don't support direct access to block
156           devices for large partitions (Solaris prior to Solaris 7, or not
157           running in 64-bit mode, is in this category); on those OSes, if you
158           want cycbuffs over 2 GB, you'll have to use regular files.  If in
159           doubt, keep your cycbuffs smaller than 2 GB.
160
161           Partition the disk to make each partition equal to or smaller than
162           2 GB.  If you're using Solaris, set up your partitions to avoid the
163           first cylinder of the disk (or otherwise the cycbuff header will
164           overwrite the disk partition table and render the cycbuffs
165           inaccessible).  Then, create device files for each block device
166           you're going to use.
167
168           It's not recommended to use the block device files in /dev, since
169           the news system doesn't have permission to write to them and
170           changing the permissions of the system device files may affect
171           something else.  Instead, use mknod(1) to create a new set of block
172           devices (in somewhere like pathspool/cycbuffs that's only writable
173           by the news user).  To do this, run "ls -Ll" on the devices in /dev
174           that correspond to the block devices that you want to use.  The
175           major and minor device numbers are in the fifth and sixth columns
176           (right before the date), respectively.  Then run mknod like:
177
178               mknod <file> b <major> <minor>
179
180           where <file> is the path to the device to create (matching the
181           <file> part of the cycbuff line) and <major> and <minor> are the
182           major and minor device numbers as discovered above.
183
184           Here's a short script to do this when given the path to the system
185           device file as an argument:
186
187               #!/bin/sh
188               base=`echo "$1" | sed 's%.*/%%'`
189               major=`ls -Ll "$1" | awk '{print $5}' | tr -d ,`
190               minor=`ls -Ll "$1" | awk '{print $6}`
191               mkdir -p <pathspool in inn.conf>/cycbuffs
192               mknod <pathspool>/cycbuffs/"$base" b "$major" "$minor"
193               chown news:news <pathspool>/cycbuffs/"$base"
194               chmod 644 <pathspool>/cycbuffs/"$base"
195
196           Make sure that the created files are owned by the news user and
197           news group, as specified at configure time (the default being
198           "news" for both).  Also make sure that the permissions on the
199           devices allow the news user to read and write, and if you want
200           other users on the system to be able to use sm to retrieve
201           articles, make sure they're world-readable.
202
203       Once you have everything configured properly and you start innd, you
204       should see messages in news.notice that look like:
205
206           innd: CNFS: no magic cookie found for cycbuff ONE, initializing
207
208       where "ONE" will be whatever you called your cycbuff.
209

HISTORY

211       Written by Katsuhiro Kondou <kondou@nec.co.jp> for InterNetNews.
212       Rewritten into POD by Russ Allbery <eagle@eyrie.org>.
213
214       $Id: cycbuff.conf.pod 10230 2018-01-28 21:22:21Z iulius $
215

SEE ALSO

217       ctlinnd(8), innd(8), nnrpd(8), sm(1), storage.conf(5).
218
219
220
221INN 2.6.3                         2018-01-28                   CYCBUFF.CONF(5)
Impressum