1RRDGRAPH_RPN(1) rrdtool RRDGRAPH_RPN(1)
2
6 rrdgraph_rpn - About RPN Math in rrdtool graph
7
9 RPN expression:=vname|operator|value[,RPN expression]
10
12 If you have ever used a traditional HP calculator you already know RPN
13 (Reverse Polish Notation). The idea behind RPN is that you have a
14 stack and push your data onto this stack. Whenever you execute an
15 operation, it takes as many elements from the stack as needed. Pushing
16 is done implicitly, so whenever you specify a number or a variable, it
17 gets pushed onto the stack automatically.
18 At the end of the calculation there should be one and only one value
20 left on the stack. This is the outcome of the function and this is
21 what is put into the vname. For CDEF instructions, the stack is
22 processed for each data point on the graph. VDEF instructions work on
23 an entire data set in one run. Note, that currently VDEF instructions
24 only support a limited list of functions.
25 Example: "VDEF:maximum=mydata,MAXIMUM"
27 This will set variable "maximum" which you now can use in the rest of
29 your RRD script.
30 Example: "CDEF:mydatabits=mydata,8,*"
32 This means: push variable mydata, push the number 8, execute the
34 operator *. The operator needs two elements and uses those to return
35 one value. This value is then stored in mydatabits. As you may have
36 guessed, this instruction means nothing more than mydatabits = mydata *
37 8. The real power of RPN lies in the fact that it is always clear in
38 which order to process the input. For expressions like "a = b + 3 * 5"
39 you need to multiply 3 with 5 first before you add b to get a. However,
40 with parentheses you could change this order: "a = (b + 3) * 5". In
41 RPN, you would do "a = b, 3, +, 5, *" without the need for parentheses.
42
44 Boolean operators
45 LT, LE, GT, GE, EQ, NE
46 Less than, Less or equal, Greater than, Greater or equal, Equal,
48 Not equal all pop two elements from the stack, compare them for the
49 selected condition and return 1 for true or 0 for false. Comparing
50 an unknown or an infinite value will result in unknown returned ...
51 which will also be treated as false by the IF call.
52 UN, ISINF
54 Pop one element from the stack, compare this to unknown
56 respectively to positive or negative infinity. Returns 1 for true
57 or 0 for false.
58 then,else,condition,IF
60 Pops three elements from the stack. If the element popped last is
62 0 (false), the value popped first is pushed back onto the stack,
63 otherwise the value popped second is pushed back. This does,
64 indeed, mean that any value other than 0 is considered to be true.
65 Example: "A,B,C,IF" should be read as "if (A) then (B) else (C)"
67 Comparing values
71 MIN, MAX
72 Pops two elements from the stack and returns the smaller or larger,
74 respectively. Note that infinite is larger than anything else. If
75 one of the input numbers is unknown then the result of the
76 operation will be unknown too.
77 MINNAN, MAXNAN
79 NAN-safe version of MIN and MAX. If one of the input numbers is
81 unknown then the result of the operation will be the other one. If
82 both are unknown, then the result of the operation is unknown.
83 lower-limit,upper-limit,LIMIT
85 Pops two elements from the stack and uses them to define a range.
87 Then it pops another element and if it falls inside the range, it
88 is pushed back. If not, an unknown is pushed.
89 The range defined includes the two boundaries (so: a number equal
91 to one of the boundaries will be pushed back). If any of the three
92 numbers involved is either unknown or infinite this function will
93 always return an unknown
94 Example: "CDEF:a=alpha,0,100,LIMIT" will return unknown if alpha is
96 lower than 0 or if it is higher than 100.
97 Arithmetics
101 +, -, *, /, %
102 Add, subtract, multiply, divide, modulo
104 ADDNAN
106 NAN-safe addition. If one parameter is NAN/UNKNOWN it'll be treated
108 as zero. If both parameters are NAN/UNKNOWN, NAN/UNKNOWN will be
109 returned.
110 value,power,POW
112 Raise value to the power of power.
114 SIN, COS, LOG, EXP, SQRT
116 Sine and cosine (input in radians), log and exp (natural
118 logarithm), square root.
119 ATAN
121 Arctangent (output in radians).
123 ATAN2
125 Arctangent of y,x components (output in radians). This pops one
127 element from the stack, the x (cosine) component, and then a
128 second, which is the y (sine) component. It then pushes the
129 arctangent of their ratio, resolving the ambiguity between
130 quadrants.
131 Example: "CDEF:angle=Y,X,ATAN2,RAD2DEG" will convert "X,Y"
133 components into an angle in degrees.
134 FLOOR, CEIL
136 Round down or up to the nearest integer.
138 ROUND
140 Round to the nearest integer.
142 DEG2RAD, RAD2DEG
144 Convert angle in degrees to radians, or radians to degrees.
146 ABS
148 Take the absolute value.
150 Set Operations
152 count,SORT
153 Pop one element from the stack. This is the count of items to be
155 sorted. The top count of the remaining elements are then sorted
156 from the smallest to the largest, in place on the stack.
157 4,3,22.1,1,4,SORT -> 1,3,4,22.1
159 count,REV
161 Reverse the number
163 Example: "CDEF:x=v1,v2,v3,v4,v5,v6,6,SORT,POP,5,REV,POP,+,+,+,4,/"
165 will compute the average of the values v1 to v6 after removing the
166 smallest and largest.
167 count,AVG
169 Pop one element (count) from the stack. Now pop count elements and
171 build the average, ignoring all UNKNOWN values in the process.
172 Example: "CDEF:x=a,b,c,d,4,AVG"
174 count,SMIN and count,SMAX
176 Pop one element (count) from the stack. Now pop count elements and
178 push the minimum/maximum back onto the stack.
179 Example: "CDEF:x=a,b,c,d,4,SMIN"
181 count,MEDIAN
183 pop one element (count) from the stack. Now pop count elements and
185 find the median, ignoring all UNKNOWN values in the process. If
186 there are an even number of non-UNKNOWN values, the average of the
187 middle two will be pushed on the stack.
188 Example: "CDEF:x=a,b,c,d,4,MEDIAN"
190 count,STDEV
192 pop one element (count) from the stack. Now pop count elements and
194 calculate the standard deviation over these values (ignoring any
195 NAN values). Push the result back on to the stack.
196 Example: "CDEF:x=a,b,c,d,4,STDEV"
198 percent,count,PERCENT
200 pop two elements (count,percent) from the stack. Now pop count
202 element, order them by size (while the smallest elements are -INF,
203 the largest are INF and NaN is larger than -INF but smaller than
204 anything else. Pick the element from the ordered list where percent
205 of the elements are equal to the one picked. Push the result back
206 on to the stack.
207 Example: "CDEF:x=a,b,c,d,95,4,PERCENT"
209 count,TREND, TRENDNAN
211 Create a "sliding window" average of another data series.
213 Usage: CDEF:smoothed=x,1800,TREND
215 This will create a half-hour (1800 second) sliding window average
217 of x. The average is essentially computed as shown here:
218 +---!---!---!---!---!---!---!---!--->
220 now
221 delay t0
222 <--------------->
223 delay t1
224 <--------------->
225 delay t2
226 <--------------->
227 Value at sample (t0) will be the average between (t0-delay) and (t0)
230 Value at sample (t1) will be the average between (t1-delay) and (t1)
231 Value at sample (t2) will be the average between (t2-delay) and (t2)
232 TRENDNAN is - in contrast to TREND - NAN-safe. If you use TREND and
234 one source value is NAN the complete sliding window is affected.
235 The TRENDNAN operation ignores all NAN-values in a sliding window
236 and computes the average of the remaining values.
237 PREDICT, PREDICTSIGMA, PREDICTPERC
239 Create a "sliding window" average/sigma/percentil of another data
241 series, that also shifts the data series by given amounts of time
242 as well
243 Usage - explicit stating shifts: "CDEF:predict=<shift n>,...,<shift
245 1>,n,<window>,x,PREDICT" "CDEF:sigma=<shift n>,...,<shift
246 1>,n,<window>,x,PREDICTSIGMA" "CDEF:perc=<shift n>,...,<shift
247 1>,n,<window>,<percentil>,x,PREDICTPERC"
248 Usage - shifts defined as a base shift and a number of time this is
250 applied "CDEF:predict=<shift multiplier>,-n,<window>,x,PREDICT"
251 "CDEF:sigma=<shift multiplier>,-n,<window>,x,PREDICTSIGMA"
252 "CDEF:sigma=<shift
253 multiplier>,-n,<window>,<percentil>,x,PREDICTPERC"
254 Example: CDEF:predict=172800,86400,2,1800,x,PREDICT
256 This will create a half-hour (1800 second) sliding window
258 average/sigma of x, that average is essentially computed as shown
259 here:
260 +---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!--->
262 now
263 shift 1 t0
264 <----------------------->
265 window
266 <--------------->
267 shift 2
268 <----------------------------------------------->
269 window
270 <--------------->
271 shift 1 t1
272 <----------------------->
273 window
274 <--------------->
275 shift 2
276 <----------------------------------------------->
277 window
278 <--------------->
279 Value at sample (t0) will be the average between (t0-shift1-window) and (t0-shift1)
281 and between (t0-shift2-window) and (t0-shift2)
282 Value at sample (t1) will be the average between (t1-shift1-window) and (t1-shift1)
283 and between (t1-shift2-window) and (t1-shift2)
284 The function is by design NAN-safe. This also allows for
286 extrapolation into the future (say a few days) - you may need to
287 define the data series with the optional start= parameter, so that
288 the source data series has enough data to provide prediction also
289 at the beginning of a graph...
290 The percentile can be between [-100:+100]. The positive
292 percentiles interpolates between values while the negative will
293 take the closest.
294 Example: you run 7 shifts with a window of 1800 seconds. Assuming
296 that the rrd-file has a step size of 300 seconds this means we have
297 to do the percentile calculation based on a max of 42 distinct
298 values (less if you got NAN). that means that in the best case you
299 get a step rate between values of 2.4 percent. so if you ask for
300 the 99th percentile, then you would need to look at the 41.59th
301 value. As we only have integers, either the 41st or the 42nd value.
302 With the positive percentile a linear interpolation between the 2
304 values is done to get the effective value.
305 The negative returns the closest value distance wise - so in the
307 above case 42nd value, which is effectively returning the
308 Percentile100 or the max of the previous 7 days in the window.
309 Here an example, that will create a 10 day graph that also shows
311 the prediction 3 days into the future with its uncertainty value
312 (as defined by avg+-4*sigma) This also shows if the prediction is
313 exceeded at a certain point.
314 rrdtool graph image.png --imgformat=PNG \
316 --start=-7days --end=+3days --width=1000 --height=200 --alt-autoscale-max \
317 DEF:value=value.rrd:value:AVERAGE:start=-14days \
318 LINE1:value#ff0000:value \
319 CDEF:predict=86400,-7,1800,value,PREDICT \
320 CDEF:sigma=86400,-7,1800,value,PREDICTSIGMA \
321 CDEF:upper=predict,sigma,3,*,+ \
322 CDEF:lower=predict,sigma,3,*,- \
323 LINE1:predict#00ff00:prediction \
324 LINE1:upper#0000ff:upper\ certainty\ limit \
325 LINE1:lower#0000ff:lower\ certainty\ limit \
326 CDEF:exceeds=value,UN,0,value,lower,upper,LIMIT,UN,IF \
327 TICK:exceeds#aa000080:1 \
328 CDEF:perc95=86400,-7,1800,95,value,PREDICTPERC \
329 LINE1:perc95#ffff00:95th_percentile
330 Note: Experience has shown that a factor between 3 and 5 to scale
332 sigma is a good discriminator to detect abnormal behavior. This
333 obviously depends also on the type of data and how "noisy" the data
334 series is.
335 Also Note the explicit use of start= in the CDEF - this is
337 necessary to load all the necessary data (even if it is not
338 displayed)
339 This prediction can only be used for short term extrapolations -
341 say a few days into the future.
342 Special values
344 UNKN
345 Pushes an unknown value on the stack
347 INF, NEGINF
349 Pushes a positive or negative infinite value on the stack. When
351 such a value is graphed, it appears at the top or bottom of the
352 graph, no matter what the actual value on the y-axis is.
353 PREV
355 Pushes an unknown value if this is the first value of a data set or
357 otherwise the result of this CDEF at the previous time step. This
358 allows you to do calculations across the data. This function
359 cannot be used in VDEF instructions.
360 PREV(vname)
362 Pushes an unknown value if this is the first value of a data set or
364 otherwise the result of the vname variable at the previous time
365 step. This allows you to do calculations across the data. This
366 function cannot be used in VDEF instructions.
367 COUNT
369 Pushes the number 1 if this is the first value of the data set, the
371 number 2 if it is the second, and so on. This special value allows
372 you to make calculations based on the position of the value within
373 the data set. This function cannot be used in VDEF instructions.
374 Time
376 Time inside RRDtool is measured in seconds since the epoch. The
377 epoch is defined to be "Thu Jan 1 00:00:00 UTC 1970".
378 NOW
380 Pushes the current time on the stack.
382 STEPWIDTH
384 The width of the current step in seconds. You can use this to go
386 back from rate based presentations to absolute numbers
387 CDEF:abs=rate,STEPWIDTH,*,PREV,ADDNAN
389 NEWDAY,NEWWEEK,NEWMONTH,NEWYEAR
391 These three operators will return 1.0 whenever a step is the first
393 of the given period. The periods are determined according to the
394 local timezone AND the "LC_TIME" settings.
395 CDEF:mtotal=rate,STEPWIDTH,*,NEWMONTH,0,PREV,IF,ADDNAN
397 TIME
399 Pushes the time the currently processed value was taken at onto the
401 stack.
402 LTIME
404 Takes the time as defined by TIME, applies the time zone offset
406 valid at that time including daylight saving time if your OS
407 supports it, and pushes the result on the stack. There is an
408 elaborate example in the examples section below on how to use this.
409 Processing the stack directly
411 DUP, POP, EXC
412 Duplicate the top element, remove the top element, exchange the two
414 top elements.
415 DEPTH
417 pushes the current depth of the stack onto the stack
419 a,b,DEPTH -> a,b,2
421 n,COPY
423 push a copy of the top n elements onto the stack
425 a,b,c,d,2,COPY => a,b,c,d,c,d
427 n,INDEX
429 push the nth element onto the stack.
431 a,b,c,d,3,INDEX -> a,b,c,d,b
433 n,m,ROLL
435 rotate the top n elements of the stack by m
437 a,b,c,d,3,1,ROLL => a,d,b,c
439 a,b,c,d,3,-1,ROLL => a,c,d,b
440
444 These operators work only on VDEF statements. Note that currently ONLY
445 these work for VDEF.
446 MAXIMUM, MINIMUM, AVERAGE
448 Return the corresponding value, MAXIMUM and MINIMUM also return the
449 first occurrence of that value in the time component.
450 Example: "VDEF:avg=mydata,AVERAGE"
452 STDEV
454 Returns the standard deviation of the values.
455 Example: "VDEF:stdev=mydata,STDEV"
457 LAST, FIRST
459 Return the last/first non-nan or infinite value for the selected
460 data stream, including its timestamp.
461 Example: "VDEF:first=mydata,FIRST"
463 TOTAL
465 Returns the rate from each defined time slot multiplied with the
466 step size. This can, for instance, return total bytes transferred
467 when you have logged bytes per second. The time component returns
468 the number of seconds.
469 Example: "VDEF:total=mydata,TOTAL"
471 PERCENT, PERCENTNAN
473 This should follow a DEF or CDEF vname. The vname is popped,
474 another number is popped which is a certain percentage (0..100).
475 The data set is then sorted and the value returned is chosen such
476 that percentage percent of the values is lower or equal than the
477 result. For PERCENTNAN Unknown values are ignored, but for PERCENT
478 Unknown values are considered lower than any finite number for this
479 purpose so if this operator returns an unknown you have quite a lot
480 of them in your data. Infinite numbers are lesser, or more, than
481 the finite numbers and are always more than the Unknown numbers.
482 (NaN < -INF < finite values < INF)
483 Example: "VDEF:perc95=mydata,95,PERCENT"
485 "VDEF:percnan95=mydata,95,PERCENTNAN"
486 LSLSLOPE, LSLINT, LSLCORREL
488 Return the parameters for a Least Squares Line (y = mx +b) which
489 approximate the provided dataset. LSLSLOPE is the slope (m) of the
490 line related to the COUNT position of the data. LSLINT is the
491 y-intercept (b), which happens also to be the first data point on
492 the graph. LSLCORREL is the Correlation Coefficient (also know as
493 Pearson's Product Moment Correlation Coefficient). It will range
494 from 0 to +/-1 and represents the quality of fit for the
495 approximation.
496 Example: "VDEF:slope=mydata,LSLSLOPE"
498
500 rrdgraph gives an overview of how rrdtool graph works. rrdgraph_data
501 describes DEF,CDEF and VDEF in detail. rrdgraph_rpn describes the RPN
502 language used in the ?DEF statements. rrdgraph_graph page describes
503 all of the graph and print functions.
504 Make sure to read rrdgraph_examples for tips&tricks.
506
508 Program by Tobias Oetiker <tobi@oetiker.ch>
509 This manual page by Alex van den Bogaerdt <alex@vandenbogaerdt.nl> with
511 corrections and/or additions by several people
5121.8.0 2022-03-14 RRDGRAPH_RPN(1)