1QPERF(1) User Commands QPERF(1)
2
6 qperf - Measure RDMA and IP performance
7
9 qperf
10 qperf SERVERNODE [OPTIONS] TESTS
11
14 qperf measures bandwidth and latency between two nodes. It can work
15 over TCP/IP as well as the RDMA transports. On one of the nodes, qperf
16 is typically run with no arguments designating it the server node. One
17 may then run qperf on a client node to obtain measurements such as
18 bandwidth, latency and cpu utilization.
19 In its most basic form, qperf is run on one node in server mode by
21 invoking it with no arguments. On the other node, it is run with two
22 arguments: the name of the server node followed by the name of the
23 test. A list of tests can be found in the section, TESTS. A variety
24 of options may also be specified.
25 One can get more detailed information on qperf by using the --help
27 option. Below are examples of using the --help option:
28 qperf --help examples Some examples of using qperf
30 qperf --help opts Summary of options
31 qperf --help options Description of options
32 qperf --help tests Short summary and description of tests
33 qperf --help TESTNAME More information on test TESTNAME
34
36 In these examples, we first run qperf on a node called myserver in
37 server mode by invoking it with no arguments. In all the subsequent
38 examples, we run qperf on another node and connect to the server which
39 we assume has a hostname of myserver.
40 To run a TCP bandwidth and latency test:
42 qperf myserver tcp_bw tcp_lat
43 To run a SDP bandwidth test for 10 seconds:
45 qperf myserver -t 10 sdp_bw
46 To run a UDP latency test and then cause the server to terminate:
48 qperf myserver udp_lat quit
49 To measure the RDMA UD latency and bandwidth:
51 qperf myserver ud_lat ud_bw
52 To measure RDMA UC bi-directional bandwidth:
54 qperf myserver rc_bi_bw
55 To get a range of TCP latencies with a message size from 1 to 64K
57 qperf myserver -oo msg_size:1:64K:*2 -vu tcp_lat
58
60 -ar, --access_recv OnOff
61 If OnOff is non-zero, data is accessed once received. Other‐
62 wise, data is ignored. By default, OnOff is 0. This can help
63 to mimic some applications.
64 -ar1 Cause received data to be accessed.
66 -ap, --alt_port Port
68 Set alternate path port. This enables automatic path failover.
69 -lap, --loc_alt_port Port
71 Set local alternate path port. This enables automatic path
72 failover.
73 -rap, --rem_alt_port Port
75 Set remote alternate path port. This enables automatic path
76 failover.
77 -ca, --cpu_affinity PN
79 Set cpu affinity to PN. CPUs are numbered sequentially from 0.
80 If PN is "any", any cpu is allowed otherwise the cpu is limited
81 to the one specified.
82 -lca, --loc_cpu_affinity PN
84 Set local processor affinity to PN.
85 -rca, --rem_cpu_affinity PN
87 Set remote processor affinity to PN.
88 -f, --flip OnOff
90 If non-zero, cause sender and receiver to play opposite roles.
91 -f1 Cause sender and receiver to play opposite roles.
93 -h, --help Topic
95 Print out information about Topic. To see the list of topics,
96 type qperf --help
97 -H, --host Host
99 Run test between the current node and the qperf running on node
100 Host. This can also be specified as the first non-option argu‐
101 ment.
102 -i, --id Device:Port
104 Use RDMA Device and Port.
105 -li, --loc_id Device:Port
107 Use local RDMA Device and Port.
108 -ri, --rem_id Device:Port
110 Use remote RDMA Device and Port.
111 -lp, --listen_port Port
113 Set the port we listen on to ListenPort. This must be set to
114 the same port on both the server and client machines. The
115 default value is 19765.
116 -oo, --loop Var:Init:Last:Incr
118 Run a test multiple times sequencing through a series of values.
119 Var is the loop variable; Init is the initial value; Last is the
120 value it must not exceed and Incr is the increment. It is use‐
121 ful to set the --verbose_used (-vu) option in conjunction with
122 this option.
123 -m, --msg_size Size
125 Set the message size to Size. The default value varies by test.
126 It is assumed that the value is specified in bytes however, a
127 trailing kib or K, mib or M, or gib or G indicates that the size
128 is being specified in kibibytes, mebibytes or gibibytes respec‐
129 tively while a trailing kb or k, mb or m, or gb or g indicates
130 kilobytes, megabytes or gigabytes respectively.
131 -mt, --mtu_size Size
133 Set the MTU size. Only relevant to the RDMA UC/RC tests. Units
134 are specified in the same manner as the --msg_size option.
135 -n, --no_msgs N
137 Set test duration by number of messages sent instead of time.
138 -cp, --cq_poll OnOff
140 Turn polling mode on or off. This is only relevant to the RDMA
141 tests and determines whether they poll or wait on the completion
142 queues. If OnOff is 0, they wait; otherwise they poll.
143 -lcp, --loc_cq_poll OnOff
145 Locally turn polling mode on or off.
146 -rcp, --rem_cq_poll OnOff
148 Remotely turn polling mode on or off.
149 -cp1 Turn polling mode on.
151 -lcp1 Turn local polling mode on.
153 -rcp1 Turn remote polling mode on.
155 -ip, --ip_port Port
157 Use Port to run the socket tests. This is different from --lis‐
158 ten_port which is used for synchronization. This is only rele‐
159 vant for the socket tests and refers to the TCP/UDP/SDP/RDS/SCTP
160 port that the test is run on.
161 -e, --precision Digits
163 Set the number of significant digits that are used to report
164 results.
165 -nr, --rd_atomic Max
167 Set the number of in-flight operations that can be handled for a
168 RDMA read or atomic operation to Max. This is only relevant to
169 the RDMA Read and Atomic tests.
170 -lnr, --loc_rd_atomic Max
172 Set local read/atomic count.
173 -rnr, --rem_rd_atomic Max
175 Set remote read/atomic count.
176 -sl, --service_level SL
178 Set RDMA service level to SL. This is only used by the RDMA
179 tests. The service level must be between 0 and 15. The default
180 service level is 0.
181 -lsl, --loc_service_level SL
183 Set local service level.
184 -rsl, --rem_service_level SL
186 Set remote service level.
187 -sb, --sock_buf_size Size
189 Set the socket buffer size. This is only relevant to the socket
190 tests.
191 -lsb, --loc_sock_buf_size Size
193 Set local socket buffer size.
194 -rsb, --rem_sock_buf_size Size
196 Set remote socket buffer size.
197 -sp, --src_path_bits N
199 Set source path bits. If the LMC is not zero, this will cause
200 the connection to use a LID with the low order LMC bits set to
201 N.
202 -lsp, --loc_src_path_bits N
204 Set local source path bits.
205 -rsp, --rem_src_path_bits N
207 Set remote source path bits.
208 -sr, --static_rate Rate
210 Force InfiniBand static rate. Rate can be one of: 2.5, 5, 10,
211 20, 30, 40, 60, 80, 120, 1xSDR (2.5 Gbps), 1xDDR (5 Gbps), 1xQDR
212 (10 Gbps), 4xSDR (2.5 Gbps), 4xDDR (5 Gbps), 4xQDR (10 Gbps),
213 8xSDR (2.5 Gbps), 8xDDR (5 Gbps), 8xQDR (10 Gbps).
214 -lsr, --loc_static_rate
216 Force local InfiniBand static rate
217 -rsr, --rem_static_rate
219 Force remote InfiniBand static rate
220 -t, --time Time
222 Set test duration to Time. Specified in seconds however a
223 trailing m, h or d indicates that the time is specified in min‐
224 utes, hours or days respectively.
225 -to, --timeout Time
227 Set timeout to Time. This is the timeout used for various
228 things such as exchanging messages. The default is 5 seconds.
229 -lto, --loc_timeout Time
231 Set local timeout to Time. This may be used on the server to
232 set the timeout when initially exchanging data with each client.
233 However, as soon as we receive the client's parameters, the
234 client's remote timeout will override this parameter.
235 -rto, --rem_timeout Time
237 Set remote timeout to Time.
238 -un, --unify_nodes
240 Unify the nodes. Describe them in terms of local and remote
241 rather than send and receive.
242 -uu, --unify_units
244 Unify the units that results are shown in. Uses the lowest com‐
245 mon denominator. Helpful for scripts.
246 -ub, --use_bits_per_sec
248 Use bits/sec rather than bytes/sec when displaying networking
249 speed.
250 -cm, --use_cm OnOff
252 Use the RDMA Connection Manager (CM) if OnOff is non-zero. It
253 is necessary to use the CM for iWARP devices. The default is to
254 establish the connection without using the CM. This only works
255 for the tests that use the RC transport.
256 -cm1 Use RDMA Connection Manager.
258 -v, --verbose
260 Provide more detailed output. Turns on -vc, -vs, -vt and -vu.
261 -vc, --verbose_conf
263 Provide information on configuration.
264 -vs, --verbose_stat
266 Provide information on statistics.
267 -vt, --verbose_time
269 Provide information on timing.
270 -vu, --verbose_used
272 Provide information on parameters used.
273 -vv, --verbose_more
275 Provide even more detailed output. Turns on -vvc, -vvs, -vvt
276 and -vvu.
277 -vvc, --verbose_more_conf
279 Provide more information on configuration.
280 -vvs, --verbose_more_stat
282 Provide more information on statistics.
283 -vvt, --verbose_more_time
285 Provide more information on timing.
286 -vvu, --verbose_more_used
288 Provide more information on parameters used.
289 -V, --version
291 The current version of qperf is printed.
292 -ws, --wait_server Time
294 If the server is not ready, continue to try connecting for Time
295 seconds before giving up. The default is 5 seconds.
296
298 conf Show configuration
299 quit Cause the server to quit
301 rds_bw RDS streaming one way bandwidth
303 rds_lat
305 RDS one way latency
306 sctp_bw
308 SCTP streaming one way bandwidth
309 sctp_lat
311 SCTP one way latency
312 sdp_bw SDP streaming one way bandwidth
314 sdp_lat
316 SDP one way latency
317 tcp_bw TCP streaming one way bandwidth
319 tcp_lat
321 TCP one way latency
322 udp_bw UDP streaming one way bandwidth
324 udp_lat
326 UDP one way latency
327 rc_bi_bw
329 RC streaming two way bandwidth
330 rc_bw RC streaming one way bandwidth
332 rc_lat RC one way latency
334 uc_bi_bw
336 UC streaming two way bandwidth
337 uc_bw UC streaming one way bandwidth
339 uc_lat UC one way latency
341 ud_bi_bw
343 UD streaming two way bandwidth
344 ud_bw UD streaming one way bandwidth
346 ud_lat UD one way latency
348 xrc_bi_bw
350 XRC streaming two way bandwidth
351 xrc_bw XRC streaming one way bandwidth
353 xrc_lat
355 XRC one way latency
356 rc_rdma_read_bw
358 RC RDMA read streaming one way bandwidth
359 rc_rdma_read_lat
361 RC RDMA read one way latency
362 rc_rdma_write_bw
364 RC RDMA write streaming one way bandwidth
365 rc_rdma_write_lat
367 RC RDMA write one way latency
368 rc_rdma_write_poll_lat
370 RC RDMA write one way polling latency
371 uc_rdma_write_bw
373 UC RDMA write streaming one way bandwidth
374 uc_rdma_write_lat
376 UC RDMA write one way latency
377 uc_rdma_write_poll_lat
379 UC RDMA write one way polling latency
380 rc_compare_swap_mr
382 RC compare and swap messaging rate
383 rc_fetch_add_mr
385 RC fetch and add messaging rate
386 ver_rc_compare_swap
388 Verify RC compare and swap
389 ver_rc_fetch_add
391 Verify RC fetch and add
392
394 Written by Johann George.
395
397 None of the RDMA tests are available if qperf is compiled without the
398 RDMA libraries. None of the XRC tests are available if qperf is com‐
399 piled without the XRC extensions. The -f option is not yet implemented
400 in many of the tests.
401qperf November 2012 QPERF(1)