1RRDGRAPH_RPN(1) rrdtool RRDGRAPH_RPN(1)
2
6 rrdgraph_rpn - About RPN Math in rrdtool graph
7
9 RPN expression:=vname|operator|value[,RPN expression]
10
12 If you have ever used a traditional HP calculator you already know RPN
13 (Reverse Polish Notation). The idea behind RPN is that you have a
14 stack and push your data onto this stack. Whenever you execute an
15 operation, it takes as many elements from the stack as needed. Pushing
16 is done implicitly, so whenever you specify a number or a variable, it
17 gets pushed onto the stack automatically.
18 At the end of the calculation there should be one and only one value
20 left on the stack. This is the outcome of the function and this is
21 what is put into the vname. For CDEF instructions, the stack is
22 processed for each data point on the graph. VDEF instructions work on
23 an entire data set in one run. Note, that currently VDEF instructions
24 only support a limited list of functions.
25 Example: "VDEF:maximum=mydata,MAXIMUM"
27 This will set variable "maximum" which you now can use in the rest of
29 your RRD script.
30 Example: "CDEF:mydatabits=mydata,8,*"
32 This means: push variable mydata, push the number 8, execute the
34 operator *. The operator needs two elements and uses those to return
35 one value. This value is then stored in mydatabits. As you may have
36 guessed, this instruction means nothing more than mydatabits = mydata *
37 8. The real power of RPN lies in the fact that it is always clear in
38 which order to process the input. For expressions like "a = b + 3 * 5"
39 you need to multiply 3 with 5 first before you add b to get a. However,
40 with parentheses you could change this order: "a = (b + 3) * 5". In
41 RPN, you would do "a = b, 3, +, 5, *" without the need for parentheses.
42
44 Boolean operators
45 LT, LE, GT, GE, EQ, NE
46 Less than, Less or equal, Greater than, Greater or equal, Equal,
48 Not equal all pop two elements from the stack, compare them for the
49 selected condition and return 1 for true or 0 for false. Comparing
50 an unknown or an infinite value will result in unknown returned ...
51 which will also be treated as false by the IF call.
52 UN, ISINF
54 Pop one element from the stack, compare this to unknown
56 respectively to positive or negative infinity. Returns 1 for true
57 or 0 for false.
58 then,else,condition,IF
60 Pops three elements from the stack. If the element popped last is
62 0 (false), the value popped first is pushed back onto the stack,
63 otherwise the value popped second is pushed back. This does,
64 indeed, mean that any value other than 0 is considered to be true.
65 Example: "A,B,C,IF" should be read as "if (A) then (B) else (C)"
67 Comparing values
71 MIN, MAX
72 Pops two elements from the stack and returns the smaller or larger,
74 respectively. Note that infinite is larger than anything else. If
75 one of the input numbers is unknown then the result of the
76 operation will be unknown too.
77 MINNAN, MAXNAN
79 NAN-safe version of MIN and MAX. If one of the input numbers is
81 unknown then the result of the operation will be the other one. If
82 both are unknown, then the result of the operation is unknown.
83 lower-limit,upper-limit,LIMIT
85 Pops two elements from the stack and uses them to define a range.
87 Then it pops another element and if it falls inside the range, it
88 is pushed back. If not, an unknown is pushed.
89 The range defined includes the two boundaries (so: a number equal
91 to one of the boundaries will be pushed back). If any of the three
92 numbers involved is either unknown or infinite this function will
93 always return an unknown
94 Example: "CDEF:a=alpha,0,100,LIMIT" will return unknown if alpha is
96 lower than 0 or if it is higher than 100.
97 Arithmetics
101 +, -, *, /, %
102 Add, subtract, multiply, divide, modulo
104 ADDNAN
106 NAN-safe addition. If one parameter is NAN/UNKNOWN it'll be treated
108 as zero. If both parameters are NAN/UNKNOWN, NAN/UNKNOWN will be
109 returned.
110 value,power,POW
112 Raise value to the power of power.
114 SIN, COS, LOG, EXP, SQRT
116 Sine and cosine (input in radians), log and exp (natural
118 logarithm), square root.
119 ATAN
121 Arctangent (output in radians).
123 ATAN2
125 Arctangent of y,x components (output in radians). This pops one
127 element from the stack, the x (cosine) component, and then a
128 second, which is the y (sine) component. It then pushes the
129 arctangent of their ratio, resolving the ambiguity between
130 quadrants.
131 Example: "CDEF:angle=Y,X,ATAN2,RAD2DEG" will convert "X,Y"
133 components into an angle in degrees.
134 FLOOR, CEIL
136 Round down or up to the nearest integer.
138 DEG2RAD, RAD2DEG
140 Convert angle in degrees to radians, or radians to degrees.
142 ABS
144 Take the absolute value.
146 Set Operations
148 count,SORT
149 Pop one element from the stack. This is the count of items to be
151 sorted. The top count of the remaining elements are then sorted
152 from the smallest to the largest, in place on the stack.
153 4,3,22.1,1,4,SORT -> 1,3,4,22.1
155 count,REV
157 Reverse the number
159 Example: "CDEF:x=v1,v2,v3,v4,v5,v6,6,SORT,POP,5,REV,POP,+,+,+,4,/"
161 will compute the average of the values v1 to v6 after removing the
162 smallest and largest.
163 count,AVG
165 Pop one element (count) from the stack. Now pop count elements and
167 build the average, ignoring all UNKNOWN values in the process.
168 Example: "CDEF:x=a,b,c,d,4,AVG"
170 count,SMIN and count,SMAX
172 Pop one element (count) from the stack. Now pop count elements and
174 push the minimum/maximum back onto the stack.
175 Example: "CDEF:x=a,b,c,d,4,AVG"
177 count,MEDIAN
179 pop one element (count) from the stack. Now pop count elements and
181 find the median, ignoring all UNKNOWN values in the process. If
182 there are an even number of non-UNKNOWN values, the average of the
183 middle two will be pushed on the stack.
184 Example: "CDEF:x=a,b,c,d,4,MEDIAN"
186 count,STDEV
188 pop one element (count) from the stack. Now pop count elements and
190 calculate the standard deviation over these values (ignoring any
191 NAN values). Push the result back on to the stack.
192 Example: "CDEF:x=a,b,c,d,4,STDEV"
194 percent,count,PERCENT
196 pop two elements (count,percent) from the stack. Now pop count
198 element, order them by size (while the smalles elements are -INF,
199 the largest are INF and NaN is larger than -INF but smaller than
200 anything else. No pick the element from the ordered list where
201 percent of the elements are equal then the one picked. Push the
202 result back on to the stack.
203 Example: "CDEF:x=a,b,c,d,95,4,PERCENT"
205 count,TREND, TRENDNAN
207 Create a "sliding window" average of another data series.
209 Usage: CDEF:smoothed=x,1800,TREND
211 This will create a half-hour (1800 second) sliding window average
213 of x. The average is essentially computed as shown here:
214 +---!---!---!---!---!---!---!---!--->
216 now
217 delay t0
218 <--------------->
219 delay t1
220 <--------------->
221 delay t2
222 <--------------->
223 Value at sample (t0) will be the average between (t0-delay) and (t0)
226 Value at sample (t1) will be the average between (t1-delay) and (t1)
227 Value at sample (t2) will be the average between (t2-delay) and (t2)
228 TRENDNAN is - in contrast to TREND - NAN-safe. If you use TREND and
230 one source value is NAN the complete sliding window is affected.
231 The TRENDNAN operation ignores all NAN-values in a sliding window
232 and computes the average of the remaining values.
233 PREDICT, PREDICTSIGMA, PREDICTPERC
235 Create a "sliding window" average/sigma/percentil of another data
237 series, that also shifts the data series by given amounts of time
238 as well
239 Usage - explicit stating shifts: "CDEF:predict=<shift n>,...,<shift
241 1>,n,<window>,x,PREDICT" "CDEF:sigma=<shift n>,...,<shift
242 1>,n,<window>,x,PREDICTSIGMA" "CDEF:perc=<shift n>,...,<shift
243 1>,n,<window>,<percentil>,x,PREDICTPERC"
244 Usage - shifts defined as a base shift and a number of time this is
246 applied "CDEF:predict=<shift multiplier>,-n,<window>,x,PREDICT"
247 "CDEF:sigma=<shift multiplier>,-n,<window>,x,PREDICTSIGMA"
248 "CDEF:sigma=<shift
249 multiplier>,-n,<window>,<percentil>,x,PREDICTPERC"
250 Example: CDEF:predict=172800,86400,2,1800,x,PREDICT
252 This will create a half-hour (1800 second) sliding window
254 average/sigma of x, that average is essentially computed as shown
255 here:
256 +---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!--->
258 now
259 shift 1 t0
260 <----------------------->
261 window
262 <--------------->
263 shift 2
264 <----------------------------------------------->
265 window
266 <--------------->
267 shift 1 t1
268 <----------------------->
269 window
270 <--------------->
271 shift 2
272 <----------------------------------------------->
273 window
274 <--------------->
275 Value at sample (t0) will be the average between (t0-shift1-window) and (t0-shift1)
277 and between (t0-shift2-window) and (t0-shift2)
278 Value at sample (t1) will be the average between (t1-shift1-window) and (t1-shift1)
279 and between (t1-shift2-window) and (t1-shift2)
280 The function is by design NAN-safe. This also allows for
282 extrapolation into the future (say a few days) - you may need to
283 define the data series with the optional start= parameter, so that
284 the source data series has enough data to provide prediction also
285 at the beginning of a graph...
286 The percentile can be between [-100:+100]. The positive
288 percentiles interpolates between values while the negative will
289 take the closest.
290 Example: you run 7 shifts with a window of 1800 seconds. Assuming
292 that the rrd-file has a step size of 300 seconds this means we have
293 to do the percentile calculation based on a max of 42 distinct
294 values (less if you got NAN). that means that in the best case you
295 get a step rate between values of 2.4 percent. so if you ask for
296 the 99th percentile, then you would need to look at the 41.59th
297 value. As we only have integers, either the 41st or the 42nd value.
298 With the positive percentile a linear interpolation between the 2
300 values is done to get the effective value.
301 The negative returns the closest value distance wise - so in the
303 above case 42nd value, which is effectively returning the
304 Percentile100 or the max of the previous 7 days in the window.
305 Here an example, that will create a 10 day graph that also shows
307 the prediction 3 days into the future with its uncertainty value
308 (as defined by avg+-4*sigma) This also shows if the prediction is
309 exceeded at a certain point.
310 rrdtool graph image.png --imgformat=PNG \
312 --start=-7days --end=+3days --width=1000 --height=200 --alt-autoscale-max \
313 DEF:value=value.rrd:value:AVERAGE:start=-14days \
314 LINE1:value#ff0000:value \
315 CDEF:predict=86400,-7,1800,value,PREDICT \
316 CDEF:sigma=86400,-7,1800,value,PREDICTSIGMA \
317 CDEF:upper=predict,sigma,3,*,+ \
318 CDEF:lower=predict,sigma,3,*,- \
319 LINE1:predict#00ff00:prediction \
320 LINE1:upper#0000ff:upper\ certainty\ limit \
321 LINE1:lower#0000ff:lower\ certainty\ limit \
322 CDEF:exceeds=value,UN,0,value,lower,upper,LIMIT,UN,IF \
323 TICK:exceeds#aa000080:1 \
324 CDEF:perc95=86400,-7,1800,95,value,PREDICTPERC \
325 LINE1:perc95#ffff00:95th_percentile
326 Note: Experience has shown that a factor between 3 and 5 to scale
328 sigma is a good discriminator to detect abnormal behavior. This
329 obviously depends also on the type of data and how "noisy" the data
330 series is.
331 Also Note the explicit use of start= in the CDEF - this is
333 necessary to load all the necessary data (even if it is not
334 displayed)
335 This prediction can only be used for short term extrapolations -
337 say a few days into the future.
338 Special values
340 UNKN
341 Pushes an unknown value on the stack
343 INF, NEGINF
345 Pushes a positive or negative infinite value on the stack. When
347 such a value is graphed, it appears at the top or bottom of the
348 graph, no matter what the actual value on the y-axis is.
349 PREV
351 Pushes an unknown value if this is the first value of a data set or
353 otherwise the result of this CDEF at the previous time step. This
354 allows you to do calculations across the data. This function
355 cannot be used in VDEF instructions.
356 PREV(vname)
358 Pushes an unknown value if this is the first value of a data set or
360 otherwise the result of the vname variable at the previous time
361 step. This allows you to do calculations across the data. This
362 function cannot be used in VDEF instructions.
363 COUNT
365 Pushes the number 1 if this is the first value of the data set, the
367 number 2 if it is the second, and so on. This special value allows
368 you to make calculations based on the position of the value within
369 the data set. This function cannot be used in VDEF instructions.
370 Time
372 Time inside RRDtool is measured in seconds since the epoch. The
373 epoch is defined to be "Thu Jan 1 00:00:00 UTC 1970".
374 NOW
376 Pushes the current time on the stack.
378 STEPWIDTH
380 The width of the current step in seconds. You can use this to go
382 back from rate based presentations to absolute numbers
383 CDEF:abs=rate,STEPWIDTH,*,PREV,ADDNAN
385 NEWDAY,NEWWEEK,NEWMONTH,NEWYEAR
387 These three operators will return 1.0 whenever a step is the first
389 of the given period. The periods are determined according to the
390 local timezone AND the "LC_TIME" settings.
391 CDEF:mtotal=rate,STEPWIDTH,*,NEWMONTH,0,PREV,IF,ADDNAN
393 TIME
395 Pushes the time the currently processed value was taken at onto the
397 stack.
398 LTIME
400 Takes the time as defined by TIME, applies the time zone offset
402 valid at that time including daylight saving time if your OS
403 supports it, and pushes the result on the stack. There is an
404 elaborate example in the examples section below on how to use this.
405 Processing the stack directly
407 DUP, POP, EXC
408 Duplicate the top element, remove the top element, exchange the two
410 top elements.
411 DEPTH
413 pushes the current depth of the stack onto the stack
415 a,b,DEPTH -> a,b,2
417 n,COPY
419 push a copy of the top n elements onto the stack
421 a,b,c,d,2,COPY => a,b,c,d,c,d
423 n,INDEX
425 push the nth element onto the stack.
427 a,b,c,d,3,INDEX -> a,b,c,d,b
429 n,m,ROLL
431 rotate the top n elements of the stack by m
433 a,b,c,d,3,1,ROLL => a,d,b,c
435 a,b,c,d,3,-1,ROLL => a,c,d,b
436
440 These operators work only on VDEF statements. Note that currently ONLY
441 these work for VDEF.
442 MAXIMUM, MINIMUM, AVERAGE
444 Return the corresponding value, MAXIMUM and MINIMUM also return the
445 first occurrence of that value in the time component.
446 Example: "VDEF:avg=mydata,AVERAGE"
448 STDEV
450 Returns the standard deviation of the values.
451 Example: "VDEF:stdev=mydata,STDEV"
453 LAST, FIRST
455 Return the last/first non-nan or infinite value for the selected
456 data stream, including its timestamp.
457 Example: "VDEF:first=mydata,FIRST"
459 TOTAL
461 Returns the rate from each defined time slot multiplied with the
462 step size. This can, for instance, return total bytes transferred
463 when you have logged bytes per second. The time component returns
464 the number of seconds.
465 Example: "VDEF:total=mydata,TOTAL"
467 PERCENT, PERCENTNAN
469 This should follow a DEF or CDEF vname. The vname is popped,
470 another number is popped which is a certain percentage (0..100).
471 The data set is then sorted and the value returned is chosen such
472 that percentage percent of the values is lower or equal than the
473 result. For PERCENTNAN Unknown values are ignored, but for PERCENT
474 Unknown values are considered lower than any finite number for this
475 purpose so if this operator returns an unknown you have quite a lot
476 of them in your data. Infinite numbers are lesser, or more, than
477 the finite numbers and are always more than the Unknown numbers.
478 (NaN < -INF < finite values < INF)
479 Example: "VDEF:perc95=mydata,95,PERCENT"
481 "VDEF:percnan95=mydata,95,PERCENTNAN"
482 LSLSLOPE, LSLINT, LSLCORREL
484 Return the parameters for a Least Squares Line (y = mx +b) which
485 approximate the provided dataset. LSLSLOPE is the slope (m) of the
486 line related to the COUNT position of the data. LSLINT is the
487 y-intercept (b), which happens also to be the first data point on
488 the graph. LSLCORREL is the Correlation Coefficient (also know as
489 Pearson's Product Moment Correlation Coefficient). It will range
490 from 0 to +/-1 and represents the quality of fit for the
491 approximation.
492 Example: "VDEF:slope=mydata,LSLSLOPE"
494
496 rrdgraph gives an overview of how rrdtool graph works. rrdgraph_data
497 describes DEF,CDEF and VDEF in detail. rrdgraph_rpn describes the RPN
498 language used in the ?DEF statements. rrdgraph_graph page describes
499 all of the graph and print functions.
500 Make sure to read rrdgraph_examples for tips&tricks.
502
504 Program by Tobias Oetiker <tobi@oetiker.ch>
505 This manual page by Alex van den Bogaerdt <alex@vandenbogaerdt.nl> with
507 corrections and/or additions by several people
5081.7.1 2019-02-04 RRDGRAPH_RPN(1)