1QPERF(1)                         User Commands                        QPERF(1)
2
3
4

NAME

6       qperf - Measure RDMA and IP performance
7

SYNOPSIS

9       qperf
10       qperf SERVERNODE [OPTIONS] TESTS
11
12

DESCRIPTION

14       qperf  measures  bandwidth  and latency between two nodes.  It can work
15       over TCP/IP as well as the RDMA transports.  On one of the nodes, qperf
16       is typically run with no arguments designating it the server node.  One
17       may then run qperf on a client node  to  obtain  measurements  such  as
18       bandwidth, latency and cpu utilization.
19
20       In  its  most  basic  form,  qperf is run on one node in server mode by
21       invoking it with no arguments.  On the other node, it is run  with  two
22       arguments:  the  name  of  the  server node followed by the name of the
23       test.  A list of tests can be found in the section, TESTS.   A  variety
24       of options may also be specified.
25
26       One  can  get  more  detailed  information on qperf by using the --help
27       option.  Below are examples of using the --help option:
28
29           qperf --help examples       Some examples of using qperf
30           qperf --help opts           Summary of options
31           qperf --help options        Description of options
32           qperf --help tests          Short summary and description of tests
33           qperf --help TESTNAME       More information on test TESTNAME
34

EXAMPLES

36       In these examples, we first run qperf on  a  node  called  myserver  in
37       server  mode  by  invoking it with no arguments.  In all the subsequent
38       examples, we run qperf on another node and connect to the server  which
39       we assume has a hostname of myserver.
40
41       To run a TCP bandwidth and latency test:
42              qperf myserver tcp_bw tcp_lat
43
44       To run a SDP bandwidth test for 10 seconds:
45              qperf myserver -t 10 sdp_bw
46
47       To run a UDP latency test and then cause the server to terminate:
48              qperf myserver udp_lat quit
49
50       To measure the RDMA UD latency and bandwidth:
51              qperf myserver ud_lat ud_bw
52
53       To measure RDMA UC bi-directional bandwidth:
54              qperf myserver rc_bi_bw
55
56       To get a range of TCP latencies with a message size from 1 to 64K
57              qperf myserver -oo msg_size:1:64K:*2 -vu tcp_lat
58

OPTIONS

60       -ar, --access_recv OnOff
61              If  OnOff  is  non-zero, data is accessed once received.  Other‐
62              wise, data is ignored.  By default, OnOff is 0.  This  can  help
63              to mimic some applications.
64
65       -ar1   Cause received data to be accessed.
66
67       -ap, --alt_port Port
68              Set alternate path port. This enables automatic path failover.
69
70       -lap, --loc_alt_port Port
71              Set  local  alternate  path  port.  This  enables automatic path
72              failover.
73
74       -rap, --rem_alt_port Port
75              Set remote alternate path  port.  This  enables  automatic  path
76              failover.
77
78       -ca, --cpu_affinity PN
79              Set  cpu affinity to PN.  CPUs are numbered sequentially from 0.
80              If PN is "any", any cpu is allowed otherwise the cpu is  limited
81              to the one specified.
82
83       -lca, --loc_cpu_affinity PN
84              Set local processor affinity to PN.
85
86       -rca, --rem_cpu_affinity PN
87              Set remote processor affinity to PN.
88
89       -f, --flip OnOff
90              If non-zero, cause sender and receiver to play opposite roles.
91
92       -f1    Cause sender and receiver to play opposite roles.
93
94       -h, --help Topic
95              Print  out  information about Topic.  To see the list of topics,
96              type qperf --help
97
98       -H, --host Host
99              Run test between the current node and the qperf running on  node
100              Host.   This can also be specified as the first non-option argu‐
101              ment.
102
103       -i, --id Device:Port
104              Use RDMA Device and Port.
105
106       -li, --loc_id Device:Port
107              Use local RDMA Device and Port.
108
109       -ri, --rem_id Device:Port
110              Use remote RDMA Device and Port.
111
112       -lp, --listen_port Port
113              Set the port we listen on to ListenPort.  This must  be  set  to
114              the  same  port  on  both  the  server and client machines.  The
115              default value is 19765.
116
117       -oo, --loop Var:Init:Last:Incr
118              Run a test multiple times sequencing through a series of values.
119              Var is the loop variable; Init is the initial value; Last is the
120              value it must not exceed and Incr is the increment.  It is  use‐
121              ful  to  set the --verbose_used (-vu) option in conjunction with
122              this option.
123
124       -m, --msg_size Size
125              Set the message size to Size.  The default value varies by test.
126              It  is  assumed  that the value is specified in bytes however, a
127              trailing kib or K, mib or M, or gib or G indicates that the size
128              is  being specified in kibibytes, mebibytes or gibibytes respec‐
129              tively while a trailing kb or k, mb or m, or gb or  g  indicates
130              kilobytes, megabytes or gigabytes respectively.
131
132       -mt, --mtu_size Size
133              Set the MTU size.  Only relevant to the RDMA UC/RC tests.  Units
134              are specified in the same manner as the --msg_size option.
135
136       -n, --no_msgs N
137              Set test duration by number of messages sent instead of time.
138
139       -cp, --cq_poll OnOff
140              Turn polling mode on or off.  This is only relevant to the  RDMA
141              tests and determines whether they poll or wait on the completion
142              queues.  If OnOff is 0, they wait; otherwise they poll.
143
144       -lcp, --loc_cq_poll OnOff
145              Locally turn polling mode on or off.
146
147       -rcp, --rem_cq_poll OnOff
148              Remotely turn polling mode on or off.
149
150       -cp1   Turn polling mode on.
151
152       -lcp1  Turn local polling mode on.
153
154       -rcp1  Turn remote polling mode on.
155
156       -ip, --ip_port Port
157              Use Port to run the socket tests.  This is different from --lis‐
158              ten_port  which is used for synchronization.  This is only rele‐
159              vant for the socket tests and refers to the TCP/UDP/SDP/RDS/SCTP
160              port that the test is run on.
161
162       -e, --precision Digits
163              Set  the  number  of  significant digits that are used to report
164              results.
165
166       -nr, --rd_atomic Max
167              Set the number of in-flight operations that can be handled for a
168              RDMA  read or atomic operation to Max.  This is only relevant to
169              the RDMA Read and Atomic tests.
170
171       -lnr, --loc_rd_atomic Max
172              Set local read/atomic count.
173
174       -rnr, --rem_rd_atomic Max
175              Set remote read/atomic count.
176
177       -sl, --service_level SL
178              Set RDMA service level to SL.  This is only  used  by  the  RDMA
179              tests.  The service level must be between 0 and 15.  The default
180              service level is 0.
181
182       -lsl, --loc_service_level SL
183              Set local service level.
184
185       -rsl, --rem_service_level SL
186              Set remote service level.
187
188       -sb, --sock_buf_size Size
189              Set the socket buffer size.  This is only relevant to the socket
190              tests.
191
192       -lsb, --loc_sock_buf_size Size
193              Set local socket buffer size.
194
195       -rsb, --rem_sock_buf_size Size
196              Set remote socket buffer size.
197
198       -sp, --src_path_bits N
199              Set  source  path  bits. If the LMC is not zero, this will cause
200              the connection to use a LID with the low order LMC bits  set  to
201              N.
202
203       -lsp, --loc_src_path_bits N
204              Set local source path bits.
205
206       -rsp, --rem_src_path_bits N
207              Set remote source path bits.
208
209       -sr, --static_rate Rate
210              Force  InfiniBand  static rate.  Rate can be one of: 2.5, 5, 10,
211              20, 30, 40, 60, 80, 120, 1xSDR (2.5 Gbps), 1xDDR (5 Gbps), 1xQDR
212              (10  Gbps),  4xSDR  (2.5 Gbps), 4xDDR (5 Gbps), 4xQDR (10 Gbps),
213              8xSDR (2.5 Gbps), 8xDDR (5 Gbps), 8xQDR (10 Gbps).
214
215       -lsr, --loc_static_rate
216              Force local InfiniBand static rate
217
218       -rsr, --rem_static_rate
219              Force remote InfiniBand static rate
220
221       -t, --time Time
222              Set test duration to  Time.   Specified  in  seconds  however  a
223              trailing  m, h or d indicates that the time is specified in min‐
224              utes, hours or days respectively.
225
226       -to, --timeout Time
227              Set timeout to Time.  This  is  the  timeout  used  for  various
228              things such as exchanging messages.  The default is 5 seconds.
229
230       -lto, --loc_timeout Time
231              Set  local  timeout  to Time.  This may be used on the server to
232              set the timeout when initially exchanging data with each client.
233              However,  as  soon  as  we  receive the client's parameters, the
234              client's remote timeout will override this parameter.
235
236       -rto, --rem_timeout Time
237              Set remote timeout to Time.
238
239       -un, --unify_nodes
240              Unify the nodes.  Describe them in terms  of  local  and  remote
241              rather than send and receive.
242
243       -uu, --unify_units
244              Unify the units that results are shown in.  Uses the lowest com‐
245              mon denominator.  Helpful for scripts.
246
247       -ub, --use_bits_per_sec
248              Use bits/sec rather than bytes/sec  when  displaying  networking
249              speed.
250
251       -cm, --use_cm OnOff
252              Use  the  RDMA Connection Manager (CM) if OnOff is non-zero.  It
253              is necessary to use the CM for iWARP devices.  The default is to
254              establish  the connection without using the CM.  This only works
255              for the tests that use the RC transport.
256
257       -cm1   Use RDMA Connection Manager.
258
259       -v, --verbose
260              Provide more detailed output.  Turns on -vc, -vs, -vt and -vu.
261
262       -vc, --verbose_conf
263              Provide information on configuration.
264
265       -vs, --verbose_stat
266              Provide information on statistics.
267
268       -vt, --verbose_time
269              Provide information on timing.
270
271       -vu, --verbose_used
272              Provide information on parameters used.
273
274       -vv, --verbose_more
275              Provide even more detailed output.  Turns on  -vvc,  -vvs,  -vvt
276              and -vvu.
277
278       -vvc, --verbose_more_conf
279              Provide more information on configuration.
280
281       -vvs, --verbose_more_stat
282              Provide more information on statistics.
283
284       -vvt, --verbose_more_time
285              Provide more information on timing.
286
287       -vvu, --verbose_more_used
288              Provide more information on parameters used.
289
290       -V, --version
291              The current version of qperf is printed.
292
293       -ws, --wait_server Time
294              If  the server is not ready, continue to try connecting for Time
295              seconds before giving up.  The default is 5 seconds.
296

TESTS

298       conf   Show configuration
299
300       quit   Cause the server to quit
301
302       rds_bw RDS streaming one way bandwidth
303
304       rds_lat
305              RDS one way latency
306
307       sctp_bw
308              SCTP streaming one way bandwidth
309
310       sctp_lat
311              SCTP one way latency
312
313       sdp_bw SDP streaming one way bandwidth
314
315       sdp_lat
316              SDP one way latency
317
318       tcp_bw TCP streaming one way bandwidth
319
320       tcp_lat
321              TCP one way latency
322
323       udp_bw UDP streaming one way bandwidth
324
325       udp_lat
326              UDP one way latency
327
328       rc_bi_bw
329              RC streaming two way bandwidth
330
331       rc_bw  RC streaming one way bandwidth
332
333       rc_lat RC one way latency
334
335       uc_bi_bw
336              UC streaming two way bandwidth
337
338       uc_bw  UC streaming one way bandwidth
339
340       uc_lat UC one way latency
341
342       ud_bi_bw
343              UD streaming two way bandwidth
344
345       ud_bw  UD streaming one way bandwidth
346
347       ud_lat UD one way latency
348
349       xrc_bi_bw
350              XRC streaming two way bandwidth
351
352       xrc_bw XRC streaming one way bandwidth
353
354       xrc_lat
355              XRC one way latency
356
357       rc_rdma_read_bw
358              RC RDMA read streaming one way bandwidth
359
360       rc_rdma_read_lat
361              RC RDMA read one way latency
362
363       rc_rdma_write_bw
364              RC RDMA write streaming one way bandwidth
365
366       rc_rdma_write_lat
367              RC RDMA write one way latency
368
369       rc_rdma_write_poll_lat
370              RC RDMA write one way polling latency
371
372       uc_rdma_write_bw
373              UC RDMA write streaming one way bandwidth
374
375       uc_rdma_write_lat
376              UC RDMA write one way latency
377
378       uc_rdma_write_poll_lat
379              UC RDMA write one way polling latency
380
381       rc_compare_swap_mr
382              RC compare and swap messaging rate
383
384       rc_fetch_add_mr
385              RC fetch and add messaging rate
386
387       ver_rc_compare_swap
388              Verify RC compare and swap
389
390       ver_rc_fetch_add
391              Verify RC fetch and add
392

AUTHOR

394       Written by Johann George.
395

BUGS

397       None of the RDMA tests are available if qperf is compiled  without  the
398       RDMA  libraries.   None of the XRC tests are available if qperf is com‐
399       piled without the XRC extensions.  The -f option is not yet implemented
400       in many of the tests.
401
402
403
404qperf                             April 2018                          QPERF(1)
Impressum