1RRDTOOL(1) rrdtool RRDTOOL(1)
2
6 rrdtool - Round Robin Database Tool
7
9 rrdtool - [workdir]| function
10
12 OVERVIEW
13 It is pretty easy to gather status information from all sorts of
14 things, ranging from the temperature in your office to the number of
15 octets which have passed through the FDDI interface of your router. But
16 it is not so trivial to store this data in an efficient and systematic
17 manner. This is where RRDtool comes in handy. It lets you log and
18 analyze the data you gather from all kinds of data-sources (DS). The
19 data analysis part of RRDtool is based on the ability to quickly
20 generate graphical representations of the data values collected over a
21 definable time period.
22 In this man page you will find general information on the design and
24 functionality of the Round Robin Database Tool (RRDtool). For a more
25 detailed description of how to use the individual functions of RRDtool
26 check the corresponding man page.
27 For an introduction to the usage of RRDtool make sure you consult the
29 rrdtutorial.
30 FUNCTIONS
32 While the man pages talk of command line switches you have to set in
33 order to make RRDtool work it is important to note that RRDtool can be
34 remotely controlled through a set of pipes. This saves a considerable
35 amount of startup time when you plan to make RRDtool do a lot of things
36 quickly. Check the section on "REMOTE CONTROL" further down. There is
37 also a number of language bindings for RRDtool which allow you to use
38 it directly from Perl, python, Tcl, PHP, etc.
39 create Set up a new Round Robin Database (RRD). Check rrdcreate.
41 update Store new data values into an RRD. Check rrdupdate.
43 updatev Operationally equivalent to update except for output. Check
45 rrdupdate.
46 graph Create a graph from data stored in one or several RRDs. Apart
48 from generating graphs, data can also be extracted to stdout.
49 Check rrdgraph.
50 graphv Create a graph from data stored in one or several RRDs. Same as
52 graph, but metadata are printed before the graph. Check
53 rrdgraph.
54 dump Dump the contents of an RRD in plain ASCII. In connection with
56 restore you can use this to move an RRD from one computer
57 architecture to another. Check rrddump.
58 restore Restore an RRD in XML format to a binary RRD. Check rrdrestore
60 fetch Get data for a certain time period from a RRD. The graph
62 function uses fetch to retrieve its data from an RRD. Check
63 rrdfetch.
64 tune Alter setup and structure of an RRD. Check rrdtune.
66 first Find the first update time of an RRD. Check rrdfirst.
68 last Find the last update time of an RRD. Check rrdlast.
70 lastupdate
72 Find the last update time of an RRD. It also returns the value
73 stored for each datum in the most recent update. Check
74 rrdlastupdate.
75 info Get information about an RRD. Check rrdinfo.
77 resize Change the size of individual RRAs. This is dangerous! Check
79 rrdresize.
80 xport Export data retrieved from one or several RRDs. Check rrdxport.
82 flushcached
84 Flush the values for a specific RRD file from memory. Check
85 rrdflushcached.
86 list List the directories and rrd databases remotely. Check rrdlist.
88 HOW DOES RRDTOOL WORK?
90 Data Acquisition
91 When monitoring the state of a system, it is convenient to have
92 the data available at a constant time interval. Unfortunately,
93 you may not always be able to fetch data at exactly the time
94 you want to. Therefore RRDtool lets you update the log file at
95 any time you want. It will automatically interpolate the value
96 of the data-source (DS) at the latest official time-slot
97 (interval) and write this interpolated value to the log. The
98 original value you have supplied is stored as well and is also
99 taken into account when interpolating the next log entry.
100 Consolidation
102 You may log data at a 1 minute interval, but you might also be
103 interested to know the development of the data over the last
104 year. You could do this by simply storing the data in 1 minute
105 intervals for the whole year. While this would take
106 considerable disk space it would also take a lot of time to
107 analyze the data when you wanted to create a graph covering the
108 whole year. RRDtool offers a solution to this problem through
109 its data consolidation feature. When setting up a Round Robin
110 Database (RRD), you can define at which interval this
111 consolidation should occur, and what consolidation function
112 (CF) (average, minimum, maximum, last) should be used to build
113 the consolidated values (see rrdcreate). You can define any
114 number of different consolidation setups within one RRD. They
115 will all be maintained on the fly when new data is loaded into
116 the RRD.
117 Round Robin Archives
119 Data values of the same consolidation setup are stored into
120 Round Robin Archives (RRA). This is a very efficient manner to
121 store data for a certain amount of time, while using a known
122 and constant amount of storage space.
123 It works like this: If you want to store 1'000 values in 5
125 minute interval, RRDtool will allocate space for 1'000 data
126 values and a header area. In the header it will store a pointer
127 telling which slots (value) in the storage area was last
128 written to. New values are written to the Round Robin Archive
129 in, you guessed it, a round robin manner. This automatically
130 limits the history to the last 1'000 values (in our example).
131 Because you can define several RRAs within a single RRD, you
132 can setup another one, for storing 750 data values at a 2 hour
133 interval, for example, and thus keep a log for the last two
134 months at a lower resolution.
135 The use of RRAs guarantees that the RRD does not grow over time
137 and that old data is automatically eliminated. By using the
138 consolidation feature, you can still keep data for a very long
139 time, while gradually reducing the resolution of the data along
140 the time axis.
141 Using different consolidation functions (CF) allows you to
143 store exactly the type of information that actually interests
144 you: the maximum one minute traffic on the LAN, the minimum
145 temperature of your wine cellar, ... etc.
146 Unknown Data
148 As mentioned earlier, the RRD stores data at a constant
149 interval. Sometimes it may happen that no new data is available
150 when a value has to be written to the RRD. Data acquisition may
151 not be possible for one reason or other. With RRDtool you can
152 handle these situations by storing an *UNKNOWN* value into the
153 database. The value '*UNKNOWN*' is supported through all the
154 functions of the tool. When consolidating a data set, the
155 amount of *UNKNOWN* data values is accounted for and when a new
156 consolidated value is ready to be written to its Round Robin
157 Archive (RRA), a validity check is performed to make sure that
158 the percentage of unknown values in the data point is above a
159 configurable level. If not, an *UNKNOWN* value will be written
160 to the RRA.
161 Graphing
163 RRDtool allows you to generate reports in numerical and
164 graphical form based on the data stored in one or several RRDs.
165 The graphing feature is fully configurable. Size, color and
166 contents of the graph can be defined freely. Check rrdgraph for
167 more information on this.
168 Aberrant Behavior Detection
170 by Jake Brutlag
171 RRDtool provides the building blocks for near real-time
173 aberrant behavior detection. These components include:
174 • An algorithm for predicting the value of a time series one
176 time step into the future.
177 • A measure of deviation between predicted and observed
179 values.
180 • A mechanism to decide if and when an observed value or
182 sequence of observed values is too deviant from the
183 predicted value(s).
184 Here is a brief explanation of these components:
186 The Holt-Winters time series forecasting algorithm is an on-
188 line (or incremental) algorithm that adaptively predicts future
189 observations in a time series. Its forecast is the sum of three
190 components: a baseline (or intercept), a linear trend over time
191 (or slope), and a seasonal coefficient (a periodic effect, such
192 as a daily cycle). There is one seasonal coefficient for each
193 time point in the period (cycle). After a value is observed,
194 each of these components is updated via exponential smoothing.
195 This means that the algorithm "learns" from past values and
196 uses them to predict the future. The rate of adaptation is
197 governed by 3 parameters, alpha (intercept), beta (slope), and
198 gamma (seasonal). The prediction can also be viewed as a
199 smoothed value for the time series.
200 The measure of deviation is a seasonal weighted absolute
202 deviation. The term seasonal means deviation is measured
203 separately for each time point in the seasonal cycle. As with
204 Holt-Winters forecasting, deviation is predicted using the
205 measure computed from past values (but only at that point in
206 the seasonal cycle). After the value is observed, the algorithm
207 learns from the observed value via exponential smoothing.
208 Confidence bands for the observed time series are generated by
209 scaling the sequence of predicted deviation values (we usually
210 think of the sequence as a continuous line rather than a set of
211 discrete points).
212 Aberrant behavior (a potential failure) is reported whenever
214 the number of times the observed value violates the confidence
215 bands meets or exceeds a specified threshold within a specified
216 temporal window (e.g. 5 violations during the past 45 minutes
217 with a value observed every 5 minutes).
218 This functionality is embedded in a set of related RRAs. In
220 particular, a FAILURES RRA logs potential failures. With these
221 data you could, for example, use a front-end application to
222 RRDtool to initiate real-time alerts.
223 For a detailed description on how to set this up, see
225 rrdcreate.
226 REMOTE CONTROL
228 When you start RRDtool with the command line option '-' it waits for
229 input via standard input (STDIN). With this feature you can improve
230 performance by attaching RRDtool to another process (MRTG is one
231 example) through a set of pipes. Over these pipes RRDtool accepts the
232 same arguments as on the command line and some special commands like
233 cd, mkdir, pwd, ls and quit. For detailed help on the server commands
234 type:
235 rrdtool help cd
237 When a command is completed, RRDtool will print the string '"OK"',
239 followed by timing information of the form u:usertime s:systemtime.
240 Both values are the running totals of seconds since RRDtool was
241 started. If an error occurs, a line of the form '"ERROR:" Description
242 of error' will be printed instead. RRDtool will not abort, unless
243 something really serious happens. If a workdir is specified and the UID
244 is 0, RRDtool will do a chroot to that workdir. If the UID is not 0,
245 RRDtool only changes the current directory to workdir.
246 RRD Server
248 If you want to create a RRD-Server, you must choose a TCP/IP Service
249 number and add them to /etc/services like this:
250 rrdsrv 13900/tcp # RRD server
252 Attention: the TCP port 13900 isn't officially registered for rrdsrv.
254 You can use any unused port in your services file, but the server and
255 the client system must use the same port, of course.
256 With this configuration you can add RRDtool as meta-server to
258 /etc/inetd.conf. For example:
259 rrdsrv stream tcp nowait root /opt/rrd/bin/rrdtool rrdtool - /var/rrd
261 Don't forget to create the database directory /var/rrd and reinitialize
263 your inetd.
264 If all was setup correctly, you can access the server with Perl
266 sockets, tools like netcat, or in a quick interactive test by using
267 'telnet localhost rrdsrv'.
268 NOTE: that there is no authentication with this feature! Do not setup
270 such a port unless you are sure what you are doing.
271
273 For very big setups, updating thousands of RRD files often becomes a
274 serious IO problem. If you run into such problems, you might want to
275 take a look at rrdcached, a caching daemon for RRDtool which may help
276 you lessen the stress on your disks.
277
279 rrdcreate, rrdupdate, rrdgraph, rrddump, rrdfetch, rrdtune, rrdlast,
280 rrdxport, rrdflushcached, rrdcached
281
283 Bugs? Features!
284
286 Tobias Oetiker <tobi@oetiker.ch>
2871.8.0 2022-03-14 RRDTOOL(1)