1httperf(1) httperf(1)
2
6 httperf - HTTP performance measurement tool
7
9 httperf [--add-header R S ] [--burst-length R N ] [--client R I / N ]
10 [--close-with-reset] [-d|--debug R N ] [--failure-status R N ]
11 [-h|--help] [--hog] [--http-version R S ] [--max-connections R N ]
12 [--max-piped-calls R N ] [--method R S ] [--no-host-hdr] [--num-calls R
13 N ] [--num-conns R N ] [--period [d|u|e]T1[,T2]] [--port R N ]
14 [--print-reply [header|body]] [--print-request [header|body]] [--rate R
15 X ] [--recv-buffer R N ] [--retry-on-failure] [--send-buffer R N ]
16 [--server R S ] [--server-name R S ] [--session-cookie] [--ssl]
17 [--ssl-ciphers R L ] [--ssl-no-reuse] [--think-timeout R X ] [--timeout
18 R X ] [--uri R S ] [-v|--verbose] [-V|--version] [--wlog y|n,F]
19 [--wsess R N , N , X ] [--wsesslog R N , X , F ] [--wset R N , X ]
20
22 httperf is a tool to measure web server performance. It speaks the
23 HTTP protocol both in its HTTP/1.0 and HTTP/1.1 flavors and offers a
24 variety of workload generators. While running, it keeps track of a num‐
25 ber of performance metrics that are summarized in the form of statis‐
26 tics that are printed at the end of a test run. The most basic opera‐
27 tion of httperf is to generate a fixed number of HTTP GET requests and
28 to measure how many replies (responses) came back from the server and
29 at what rate the responses arrived.
30 IMPORTANT: To obtain correct results, it is necessary to run at most
32 one httperf process per client machine. Also, there should be as few
33 background processes as possible both on the client and server ma‐
34 chines.
35
38 httperf --hog --server www
39 This command causes httperf to create a connection to host www,
40 send a request for the root document (http://www/), receive the
41 reply, close the connection, and then print some performance
42 statistics.
43 httperf --hog --server www --num-conn 100 --ra 10 --timeout 5
45 Like above, except that a total of 100 connections are created
46 and that connections are created at a fixed rate of 10 per sec‐
47 ond. Note that option ``--rate'' has been abbreviated to
48 ``--ra''.
49 httperf --hog --server=www --wsess=10,5,2 --rate 1 --timeout 5
51 Causes httperf to generate a total of 10 sessions at a rate of 1
52 session per second. Each session consists of 5 calls that are
53 spaced out by 2 seconds.
54 httperf --hog --server=www --wsess=10,5,2 --rate=1 --timeout=5 --ssl
56 Like above, except that httperf contacts server www via SSL at
57 port 443 (the default port for SSL connections).
58 httperf --hog --server www --wsess=10,5,2 --rate=1 --timeout=5 --ssl
60 --ssl-ciphers=EXP-RC4-MD5:EXP-RC2-CBC-MD5 --ssl-no-reuse --http-ver‐
61 sion=1.0
62 Like above, except that httperf will inform the server that it
63 can only select from two cipher suites (EXP-RC4-MD5 or
64 EXP-RC2-CBC-MD5); furthermore, httperf will use HTTP version 1.0
65 which requires a new TCP connection for each request. Also, SSL
66 session ids are not reused, so the entire SSL connection estab‐
67 lishment process (known as the SSL handshake) occurs for each
68 connection.
69
71 The operation of httperf can be controlled through a number of options.
72 The tool supports both short (one-character) and long (arbi‐
73 trary-length) option names. Short options are prefixed with a single
74 leading dash (-), long options with a double-dash (--). Multiple short
75 options can be grouped together (e.g., ``-vV'' is equivalent to ``-v
76 -V'') and long options can be abbreviated so long as they remain
77 unique. Parameters to options can be specified either by following the
78 long option name with an equal sign and the parameter value (e.g.,
79 --burst=10) or by separating the option name and value with whitespace
80 (e.g., --burst 10).
81 --add-header=S
83 Specifies to include string S as an additional request header.
84 It is necessary to specify the terminating carriage-re‐
85 turn/line-feed sequence explicitly. This can be done by using
86 the escape sequence ``\n''. This makes it possible to include
87 multiple request headers. For example, ``--add-header "Referer:
88 foo\nAuth: secret\n"'' would add two request headers (``Ref‐
89 erer'' and ``Auth'') to each request. Other supported escape
90 sequences are ``\r'' (carriage-return), ``\a'' (line-feed),
91 ``\\'' (backslash), and ``\N'' where N is the code the character
92 to be inserted (in octal).
93 --burst-length=N
95 Specifies the length of bursts. Each burst consists of N calls
96 to the server. The exact meaning of this parameter depends on
97 the workload generator. For regular request-oriented workloads,
98 see the description of option --wsess.
99 --no-host-hdr
101 Specifies that the "Host:" header should not be included when
102 issuing an HTTP request.
103 --num-calls.
105 For session-oriented workloads, see the description of option
106 --wsess.
107 --client=I/N
109 Specifies that the machine httperf is running on is client I out
110 of a total of N clients. I should be in the range from 0 to R N
111 -1. Some of the workload generators (e.g., --wset) use the
112 client identity as a bias value to ensure that not all clients
113 generate perfectly identical workloads. When performing a test
114 that involves several client machines, it is generally a good
115 idea to specify this option.
116 --close-with-reset
118 Requests that httperf closes TCP connections by sending a RESET
119 instead of going through the normal TCP connection shutdown
120 handshake. Turning on this option can have ill effects such as
121 data corruption, stuck TCP control blocks, or wrong results.
122 For this reason, the option should not be used unless absolutely
123 necessary and even then it should not be used unless its impli‐
124 cations are fully understood.
125 -d=N
127 --debug=N
129 Set debug level to R N . Larger values of N will result in more
130 output.
131 --failure-status=N
133 Specifies that an HTTP response status code of N should be
134 treated as a failure (i.e., treated as if the request had timed
135 out, for example). For example, with ``--failure-status=504''
136 responses with an HTTP status of ``504 Gateway Time-out'' would
137 be considered failures. Caveat: this option is currently sup‐
138 ported for session workloads only (see the --wsess and --wsess‐
139 log options).
140 -h
142 --help Prints a summary of available options and their parameters.
144 --hog This option requests to use up as many TCP ports as necessary.
146 Without this option, httperf is typically limited to using
147 ephemeral ports (in the range from 1024 to 5000). This limited
148 port range can quickly become a bottleneck so it is generally a
149 good idea to specify this option for serious testing. Also,
150 this option must be specified when measuring NT servers since it
151 avoids a TCP incompatibility between NT and UNIX machines.
152 --http-version=S
154 Specifies the version string that should be included in the re‐
155 quests sent to the server. By default, version string ``1.1''
156 is used. This option can be set to ``1.0'' to force the genera‐
157 tion of HTTP/1.0 requests. Setting this option to any value
158 other than ``1.0'' or ``1.1'' may result in undefined behavior.
159 --max-connections=N
161 Specifies that at most N connections are opened for each ses‐
162 sion. This option is meaningful in conjunction with options
163 --wsess and --wsesslog only.
164 --max-piped-calls=N
166 Specifies that at most N pipelined calls are issued on each con‐
167 nection. This option is meaningful in conjunction with options
168 --wsess and --wsesslog only.
169 --method=S
171 Specifies the method that should be used when issuing an HTTP
172 request. If this option is not specified, the GET method is
173 used. The method S can be an arbitrary string but is usually
174 one of GET, HEAD, PUT, POST, etc.
175 --num-calls=N
177 This option is meaningful for request-oriented workloads only.
178 It specifies the total number of calls to issue on each connec‐
179 tion before closing it. If N is greater than 1, the server must
180 support persistent connections. The default value for this op‐
181 tion is 1. If --burst-length is set to R B , then the N calls
182 are issued in bursts of B pipelined calls each. Thus, the total
183 number of such bursts will be N/B (per connection).
184 --num-conns=N
186 This option is meaningful for request-oriented workloads only.
187 It specifies the total number of connections to create. On each
188 connection, calls are issued as specified by options --num-calls
189 and --burst-length. A test stops as soon as the N connections
190 have either completed or failed. A connection is considered to
191 have failed if any activity on the connection fails to make for‐
192 ward progress for more than the time specified by the timeout
193 options --timeout and --think-timeout. The default value for
194 this option is 1.
195 --period=[D]T1[,T2]
197 Specifies the time interval between the creation of connections
198 or sessions. Connections are created by default, sessions if
199 option --wsess or --wsesslog has been specified. This connec‐
200 tion/session ``interarrival time'' can alternatively be speci‐
201 fied by the --rate option, although more flexibility is avail‐
202 able with --period. The D parameter specifies the interarrival
203 time distribution. If omitted or set to ``d'', a deterministic
204 (i.e., fixed) period is used as specified by parameter R T1 in
205 units of seconds. If D is set to ``e'', an exponential (i.e.,
206 Poisson) distribution is used with a mean interarrival time of R
207 T1 . If D is set to ``u'', a uniform distribution over the in‐
208 terval [T1,T2) is used for the interarrival time. Finally, if D
209 is set to ``v'', a number of rates can be specified as follows:
210 --period=vT1,D1,T2,D2...Tn,Dn Where n <= NUM_RATES in httperf.h
211 and Ti,Di represent the period time (i.e., 1/rate) and duration
212 to maintain that rate (i.e., --period=v1,2,0.5,4 will generate 1
213 request/seconds for 2 seconds then 2 requests/seconds for 4 sec‐
214 onds). In all cases, a period of 0 results in connections or
215 sessions being generated sequentially (a new connection/session
216 is initiated as soon as the previous one completes). The de‐
217 fault value for this option is 0. Note that specifying, for ex‐
218 ample, --rate=5 is equivalent to specifying --period=d0.2 or
219 --period=0.2. By specifying --period=u1,3, the interarrival
220 times will be randomly chosen from the interval between 1 and 3
221 seconds. The specific sequence of (pseudo-)random interarrival
222 times are identical from one httperf run to another as long as
223 the values for the --period and --client options are identical.
224 --port=N
226 This option specifies the port number N on which the web server
227 is listening for HTTP requests. By default, httperf uses port
228 number 80.
229 --print-reply[=[header|body]]
231 Requests the printing of the reply headers, body, and summary.
232 The output is directed to standard output. Reply header lines
233 are prefixed by "RH", reply body lines are prefixed by "RB", and
234 the reply-size summary is prefixed by "RS". The prefix is fol‐
235 lowed by a serial number that uniquely identifies the call that
236 the reply line is for and a colon (":") character that marks the
237 beginning of the actual reply line. To print only reply head‐
238 ers, pass argument header to this option. To print only the re‐
239 ply body, pass argument body to this option.
240 --print-request[=[header|body]]
242 Requests the printing of the request headers, body (if one is
243 present), and summary. The output is directed to standard out‐
244 put. Request header lines are prefixed by "SH", request body
245 lines are prefixed by "SB", and the request summary is prefixed
246 by "SS". The prefix is followed by the call's serial number and
247 a colon (":") character that marks the beginning of the actual
248 reply line. To print only request headers, pass argument header
249 to this option. To print only the request body, pass argument
250 body to this option.
251 --rate=X
253 Specifies the fixed rate at which connections or sessions are
254 created. Connections are created by default, sessions if option
255 --wsess or --wsesslog has been specified. In both cases a rate
256 of 0 results in connections or sessions being generated sequen‐
257 tially (a new session/connection is initiated as soon as the
258 previous one completes). The default value for this option is
259 0.
260 --recv-buffer=N
262 Specifies the maximum size of the socket receive buffers used to
263 receive HTTP replies. By default, the limit is 16KB. A smaller
264 value may help memory-constrained clients whereas a larger value
265 may be necessary when communicating with a server over a
266 high-bandwidth, high-latency connection.
267 --retry-on-failure
269 This option is meaningful for session workloads only (see the
270 --wsess and --wsesslog options). If specified, a call that re‐
271 sults in a failure response (as defined by the --failure-status
272 option) is retried immediately instead of causing the session to
273 fail.
274 --send-buffer=N
276 Specifies the maximum size of the socket send buffers used to
277 send HTTP requests. By default, the limit is 4KB. A smaller
278 value may help memory-constrained clients whereas a larger value
279 may be necessary when generating large requests to a server con‐
280 nected via a high-bandwidth, high-latency connection.
281 --server=S
283 Specifies the IP hostname of the server. By default, the host‐
284 name ``localhost'' is used. This option should always be speci‐
285 fied as it is generally not a good idea to run the client and
286 the server on the same machine.
287 --server-name=S
289 Specifies the (default) server name that appears in the "Host:"
290 header of every request sent by httperf. Without this option,
291 the host name (or IP address) specified by option --server is
292 used instead.
293 --session-cookie
295 When this option is turned on, cookie managment is enabled on a
296 per-session basis. What this means is that if a reply to a re‐
297 quest that was generated by session R X contains a cookie, then
298 all future requests sent by session X will include this cookie
299 as well. At present, the cookie manager in httperf supports
300 only one cookie per session. If a second cookie is received,
301 the new cookie overwrites the existing one and a warning message
302 is printed if ``--debug 1'' is on.
303 --ssl Specifies that all communication between httperf and the server
305 should utilize the Secure Sockets Layer (SSL) protocol. This
306 option is available only if httperf was compiled with SSL sup‐
307 port enabled.
308 --ssl-ciphers=L
310 This option is only meaningful if SSL is in use (see --ssl op‐
311 tion). This option specifies the list L of cipher suites that
312 httperf may use in negotiating a secure connection with the
313 server. If the list contains more than one cipher suite, the
314 ciphers must be separated by a colon. If the server does not
315 accept any of the listed cipher suites, the connection estab‐
316 lishment will fail and httperf will exit immediately. If this
317 option is not specified when the --ssl option is present then
318 httperf will use all of the SSLv3 cipher suites provided by the
319 underlying SSL library.
320 --ssl-no-reuse
322 This option is only meaningful if SSL and sessions are in use
323 (see --ssl, --wsess, --wsesslog). When an SSL connection is es‐
324 tablished the client receives a session identifier (session id)
325 from the server. On subsequent SSL connections, the client nor‐
326 mally reuses this session id in order to avoid the expense of
327 repeating the (slow) SSL handshake to establish a new SSL ses‐
328 sion and obtain another session id (even if the client attempts
329 to re-use a session id, the server may force the client to rene‐
330 gotiate a session). By default httperf reuses the session id
331 across all connections in a session. If the --ssl-no-reuse op‐
332 tion is in effect, then httperf will not reuse the session id,
333 and the entire SSL handshake will be performed for each new con‐
334 nection in a session.
335 --think-timeout=X
337 Specifies the maximum time that the server may need to initiate
338 sending the reply for a given request. Note that this timeout
339 value is added to the normal timeout value (see option --time‐
340 out). When accessing static web content, it is usually not nec‐
341 essary to specify this option. However, when performing tests
342 with long-running CGI scripts, it may be necessary to use this
343 option to allow for larger response-times. The default value
344 for this option is zero seconds, meaning that the server has to
345 be able to respond within the normal timeout value.
346 --timeout=X
348 Specifies the amount of time X that httperf is willing to wait
349 for a server reaction. The timeout is specified in seconds and
350 can be a fractional number (e.g., --timeout 3.5). This timeout
351 value is used when establishing a TCP connection, when sending a
352 request, when waiting for a reply, and when receiving a reply.
353 If during any of those activities a request fails to make for‐
354 ward progress within the alloted time, httperf considers the re‐
355 quest to have died, closes the associated connection or session
356 and increases the client-timo error count. The actual timeout
357 value used when waiting for a reply is the sum of this timeout
358 and the think-timeout (see option --think-timeout). By default,
359 the timeout value is infinity.
360 --uri=S
362 Specifies that URI S should be accessed on the server. For some
363 of the workload generators (e.g., --wset), this option specifies
364 the prefix for the URIs being accessed.
365 --use-timer-cache
367 This feature allows the user to specify whether they want to
368 cache timestamps or not. Timestamps are not cached by default,
369 but the user can enable caching if higher performance is more
370 important than timing accuracy. For small response sizes, dis‐
371 abling timer caching reduced the performance of httperf by about
372 10%; for larger response sizes there was little or no effect.
373 -v
375 --verbose
377 Puts httperf into verbose mode. In this mode, additional output
378 such as the individual reply rate samples and connection life‐
379 time histogram are printed.
380 -V
382 --version
384 Prints the version of httperf.
385 --wlog=B,F
387 This option can be used to generate a specific sequence of URI
388 accesses. This is useful to replay the accesses recorded in a
389 server log file, for example. Parameter F is the name of a file
390 containing the ASCII NUL separated list of URIs that should be
391 accessed. If parameter B is set to ``y'', httperf will wrap
392 around to the beginning of the file when reaching the end of the
393 list (so the list of URIs is accessed repeatedly). With B set
394 to ``n'', the test will stop no later than when reaching the end
395 of the URI list.
396 --wsess=N1,N2,X
398 Requests the generation and measurement of sessions instead of
399 individual requests. A session consists of a sequence of bursts
400 which are spaced out by the user think-time. Each burst con‐
401 sists of a fixed number L of calls to the server (L is specified
402 by option --burst-length). The calls in a burst are issued as
403 follows: at first, a single call is issued. Once the reply to
404 this first call has been fully received, all remaining calls in
405 the burst are issued concurrently. The concurrent calls are is‐
406 sued either as pipelined calls on an existing persistent connec‐
407 tion or as individual calls on separate connections. Whether a
408 persistent connection is used depends on whether the server re‐
409 sponds to the first call with a reply that includes a ``Connec‐
410 tion: close'' header line. If such a line is present, separate
411 connections are used.
412 The option specifies the following parameters: N1 is the total
414 number of sessions to generate, N2 is the number of calls per
415 session, and X is the user think-time (in seconds) that sepa‐
416 rates consecutive call bursts. For example, the options
417 ``--wsess=100,50,10 --burst-len 5'' would result in 100 sessions
418 with a total of 50 calls each. Since each burst has a length of
419 5 calls, a total of 10 call bursts would be generated per ses‐
420 sion. The user think-time between call bursts would be 10 sec‐
421 onds. Note that user think-time X denotes the time between re‐
422 ceiving the last reply of the previous call burst and the send‐
423 ing of the first request of the next burst.
424 A test involving sessions finishes as soon as the requested num‐
426 ber N1 of sessions have either failed or completed. A session
427 is considered to have failed if any operation in a session takes
428 longer than the timeouts specified by options --timeout and
429 --think-timeout. In addition, a session also fails if the
430 server returns a reply with a status code matching the one spec‐
431 ified by option --failure-status.
432 --wsesslog=N,X,F
434 This specifies a session workload generator similar to --wsess
435 (please read that description first). With --wsesslog though,
436 many aspects of user sessions, including the number and sequence
437 of URI's, request method, think-time and burst-length parame‐
438 ters, can be specified in an input file F. Two other parameters
439 are retained from --wsess, namely N, the number of sessions to
440 initiate, and X, the burst-to-burst user think time (note that
441 this becomes a default time since the input file F can also
442 specify user think time on a per-burst basis. A small example
443 input file can most-easily show the settable parameters:
444 # Comment lines start with a ``#'' as the first
446 # character. Lines with only whitespace delimit
447 # sessions (multiple blank lines do not generate
448 # ``null'' sessions). All other lines specify a
449 # uri-sequence (1 uri per line). If the first
450 # character of the line is whitespace (e.g. space
451 # or tab), the uri is considered to be part of a
452 # burst that is sent out after the previous
453 # non-burst uri.
454 # session 1 definition (this is a comment)
456 /foo.html think=2.0
457 /pict1.gif
458 /pict2.gif
459 /foo2.html method=POST contents='Post data'
460 /pict3.gif
461 /pict4.gif
462 # session 2 definition
464 /foo3.html method=POST contents="Multiline\ndata"
465 /foo4.html method=HEAD
466 The above description specifies 2 sessions. The first session
468 will start with a request for /foo.html. When the /foo.html re‐
469 sponse comes back, a burst of 2 requests will follow (/pict1.gif
470 and /pict2.gif). When the last of those responses is received,
471 a two second user think time is inserted before the next request
472 of /foo2.html is issued. This request is sent as a POST. The
473 posted data can be contained between single- or double-quotes.
474 Newlines can appear within posted data as ``\n'' or as a
475 ``\<CR>''. The /foo2.html response is followed by a burst re‐
476 quest of /pict3.gif and /pict4.gif, which concludes this ses‐
477 sion. The second session is started some time after the first,
478 as specified by the --rate or --period options.
479 The second session consists of 2 requests separated by the de‐
481 fault user think time as specified by the X parameter of the
482 --wsesslog option. If the N parameter of --wsesslog is greater
483 than the number of sessions defined in input file R F , then the
484 defined sessions are used repeatedly until N sessions have been
485 created (i.e., the defined sessions are used in a round-robin
486 fashion).
487 One should avoid using --wsesslog in conjunction with other
489 httperf options that also control session behavior and workload
490 URI's, namely --burst-length, --wsess, --wlog, and --wset.
491 --wset=N,X
493 This option can be used to walk through a list of URIs at a
494 given rate. Parameter N specifies the number of distinct URIs
495 that should be generated and X specifies the rate at which new
496 URIs are accessed. A rate of 0.25 would mean that the same URI
497 would be accessed four times in a row before moving on to the
498 next URI. This type of access pattern is useful in generating a
499 workload that induces a relatively predictable amount of traffic
500 in the disk I/O subsystem of the server (assuming N and the ac‐
501 cessed files are big enough to exceed the server's buffer
502 cache). The URIs generated are of the form R prefix / path
503 .html, where prefix is the URI prefix specified by option --uri
504 and path is generated as follows: for the R i -th file in the
505 working set, write down i in decimal, prefixing the number with
506 as many zeroes as necessary to get a string that has as many
507 digits as R N -1. Then insert a slash character between each
508 digit. For example, the 103rd file in a working set consisting
509 of 1024 files would result in a path of ``0/1/0/3''. Thus, if
510 the URI-prefix is /wset1024, then the URI being accessed would
511 be /wset1024/0/1/0/3.html. In other words, the files on the
512 server need to be organized as a 10ary tree.
513
515 This section describes the statistics output at the end of each test
516 run. The basic information shown below is printed independent of the
517 selected workload generator.
518 Total: connections 30000 requests 29997 replies 29997 test-dura‐
520 tion 299.992 s
521 Connection rate: 100.0 conn/s (10.0 ms/conn, <=14 concurrent
523 connections)
524 Connection time [ms]: min 1.4 avg 3.0 max 163.4 median 1.5 std‐
525 dev 7.3
526 Connection time [ms]: connect 0.6
527 Connection length [replies/conn]: 1.000
528 Request rate: 100.0 req/s (10.0 ms/req)
530 Request size [B]: 75.0
531 Reply rate [replies/s]: min 98.8 avg 100.0 max 101.2 stddev 0.3
533 (60 samples)
534 Reply time [ms]: response 2.4 transfer 0.0
535 Reply size [B]: header 242.0 content 1010.0 footer 0.0 (total
536 1252.0)
537 Reply status: 1xx=0 2xx=29997 3xx=0 4xx=0 5xx=0
538 CPU time [s]: user 94.31 system 205.26 (user 31.4% system 68.4%
540 total 99.9%)
541 Net I/O: 129.6 KB/s (1.1*10^6 bps)
542 Errors: total 3 client-timo 0 socket-timo 0 connrefused 3 con‐
544 nreset 0
545 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
546 There are six groups of statistics: overall results (``Total''), con‐
548 nection related results (``Connection''), results relating to the issu‐
549 ing of HTTP requests (``Request''), results relating to the replies re‐
550 ceived from the server (``Reply''), miscellaneous results relating to
551 the CPU (``CPU'') and network (``Net I/O'') utilization and, last but
552 not least, a summary of errors encountered (``Errors'').
553 Total Section
555 This section summarizes how many TCP connections were initiated
556 by httperf, how many requests it sent out, how many replies it
557 received, and what the total test duration was. In the example
558 output shown above, 30,000 connections were created, 29,997 re‐
559 quests were sent out and 29,997 replies were received. The du‐
560 ration of the test was almost exactly 5 minutes (300 seconds).
561 Connection Section
563 This section conveys information related to TCP connections gen‐
564 erated by the tool. Specifically, the ``Connection rate'' line
565 shows that new connections were initiated at a rate of 100.0
566 connections per second. This rate corresponds to a period of
567 10.0 milliseconds per connection. The last number in this line
568 shows that at most 14 connections were open at any given time.
569 The first line labeled ``Connection time'' gives lifetime sta‐
571 tistics for successful connections. The lifetime of a connec‐
572 tion is the time between a TCP connection is initiated and the
573 time the connection is closed. A connection is considered suc‐
574 cessful if it had at least one call that completed successfully.
575 In the example output, the line indicates that the minimum
576 (``min'') connection lifetime was 1.4 milliseconds, the average
577 (``avg'') lifetime was 3.0 milliseconds, the maximum (``max'')
578 was 163.4 milliseconds, the median (``median'') lifetime was 1.5
579 milliseconds, and that the standard deviation of the lifetimes
580 was 7.3 milliseconds. The median lifetime is computed based on
581 a histogram with one millisecond resolution and a maximum life‐
582 time of 100 seconds. Thus, the median is accurate to within
583 half a millisecond if at least half of the successful connec‐
584 tions have a lifetime of no more than 100 seconds.
585 The next statistic in this section is the average time it took
587 to establish a TCP connection. Only successful TCP connection
588 establishments are counted. In the example, the second line la‐
589 beled ``Connection time'' shows that, on average, it took 0.6
590 milliseconds to establish a connection.
591 The final line in this section is labeled ``Connection length.''
593 It gives the average number of replies received on each connec‐
594 tion that received at least one reply (i.e., connections that
595 failed before yielding the first reply are not counted). This
596 number can be bigger than 1.0 due to persistent connections.
597 Request Section
599 The line labeled ``Request rate'' gives the rate at which HTTP
600 requests were issued and the period that this rate corresponds
601 to. In the example above, the request rate was 100.0 requests
602 per second, which corresponds to 10.0 milliseconds per request.
603 As long as no persistent connections are employed, the results
604 in this section are very similar or identical to results in the
605 connection section. However, when persistent connections are
606 used, several calls can be performed on a single connection in
607 which case the results would be different.
608 The line labeled ``Request size'' gives the average size of the
610 HTTP requests in bytes. In the example above, the average re‐
611 quest size was 75 bytes.
612 Reply Section
614 For simple measurements, this section is often the most inter‐
615 esting one as the line labeled ``Reply rate'' gives various sta‐
616 tistics for the reply rate. In the example above, the minimum
617 (``min'') reply rate was 98.8 replies per second, the average
618 (``avg'') was 100 replies per second, and the maximum (``max'')
619 rate was 101.2 replies per second. The standard deviation was
620 0.3 replies per second. The number enclosed in parentheses
621 shows that 60 reply rate samples were acquired. At present,
622 httperf collects a rate sample once every five seconds. To ob‐
623 tain a meaningful standard deviation, it is recommended to run
624 tests long enough so at least thirty samples are obtained. This
625 corresponds to a test duration of at least 150 seconds.
626 The line labeled ``Reply Time'' gives information on how long it
628 took for the server to respond and how long it took to receive
629 the reply. In the example, it took on average 2.4 milliseconds
630 between sending the first byte of the request and receiving the
631 first byte of the reply. The time to ``transfer'', or read, the
632 reply was too short to be measured, so it shows up as zero. The
633 is typical when the entire reply fits into a single TCP segment.
634 The next line, labeled ``Reply size'' contains statistics on the
636 average size of the replies---all numbers are in reported bytes.
637 Specifically, the line lists the average length of reply head‐
638 ers, the content, and footers (HTTP/1.1 uses footers to realize
639 the ``chunked'' transfer encoding). For convenience, the aver‐
640 age total number of bytes in the replies is also given in paren‐
641 theses. In the example, the average header length (``header'')
642 was 242 bytes, the average content length (``content'') was 1010
643 bytes, and there were no footers (``footer'' length is zero).
644 The total reply length of 1252 bytes on average.
645 The final line in this section is a histogram of the major sta‐
647 tus codes received in the replies from the server. The major
648 status code is the ``hundreds''-digit of the full HTTP status
649 code. In the example, all 29,997 replies had a major status
650 code of 2. It's a good guess that all status codes were ``200
651 OK'' but the information in the histogram is not detailed enough
652 to allow distinguishing status codes with the same major code.
653 Miscellaneous Section
655 This section starts with a summary of the CPU utilization on the
656 client machine. In the example, the line labeled ``CPU time''
657 shows that 94.31 seconds were spent executing in user mode
658 (``user''), 205.26 seconds were spent executing in system mode
659 (``system'') and that this corresponds to 31.4% user mode execu‐
660 tion and 68.4% system execution. The total utilization was
661 99.9%, which is expected given that httperf is a CPU hog. A to‐
662 tal CPU utilization of significantly less than 100% is a sign
663 that there were competing processes that interfered with the
664 test.
665 The line labeled ``Net I/O'' gives the average network through‐
667 put in kilobytes per second (where a kilobyte is 1024 bytes) and
668 in megabits per second (where a megabit is 10^6 bits). In the
669 example, an average network usage of about 129.6 kilobytes per
670 second was sustained. The number in parentheses shows that this
671 corresponds to about 1.1 megabits per second. This network
672 bandwidth is computed based on the number of bytes sent and re‐
673 ceived on the TCP connections. In other words, it does not ac‐
674 count for the network headers or TCP retransmissions that may
675 have occurred.
676 Errors Section
678 The last section contains statistics on the errors that were en‐
679 countered during a test. In the example, the two lines labeled
680 ``Errors'' show that there were a total of three errors and that
681 all three errors were due to the server refusing to accept a
682 connection (``connrefused''). A description of each error
683 counter follows:
684 client-timo: The number of times a session, connection, or call
686 failed due to a client timeout (as specified by the --timeout
687 and --think-timeout) options.
688 socket-timo: The number of times a TCP connection failed with a
690 socket-level timeout (ETIMEDOUT).
691 connrefused: The number of times a TCP connection attempt failed
693 with a ``connection refused by server'' error (ECONNREFUSED).
694 connreset: The number of times a TCP connection failed due to a
696 RESET from the server. Typically, a RESET is received when the
697 client attempts to send data to the server at a time the server
698 has already closed its end of the connection. NT servers also
699 send RESETs when attempting to establish a new connection when
700 the listen queue is full.
701 fd-unavail: The number of times the httperf process was out of
703 file descriptors. Whenever this count is non-zero, the test re‐
704 sults are meaningless because the client was overloaded (see
705 section "CHOOSING TIMEOUT VALUES").
706 addrunavail: The number of times the client was out of TCP port
708 numbers (EADDRNOTAVAIL). This error should never occur. If it
709 does, the results should be discarded.
710 ftab-full: The number of times the system's file descriptor ta‐
712 ble is full. Again, this error should never occur. If it does,
713 the results should be discarded.
714 other: The number of times some other type of error occurred.
716 Whenever this counter is non-zero, it is necessary to track down
717 the real cause of the error. To assist in doing this, httperf
718 prints the error code (errno) of the first unknown errors that
719 occurs during a test run.
720 When --wsess or --wsesslog is specified, httperf generates and measures
722 sessions instead of individual calls and additional statistics are
723 printed at the end of a test. An example output is shown below.
724 Session rate [sess/s]: min 0.00 avg 0.59 max 2.40 stddev 0.37
726 (240/450)
727 Session: avg 6.45 connections/session
728 Session lifetime [s]: 123.9
729 Session failtime [s]: 58.5
730 Session length histogram: 4 7 4 ... 3 3 240
731 The line labeled ``Session rate'' shows the minimum, average, and maxi‐
733 mum rate at which sessions completed (based on a 5 second sampling in‐
734 terval). It also shows the standard deviation of the session comple‐
735 tion rate. The numbers in parentheses show how many sessions succeeded
736 and how many sessions were initiated. In the example above, the mini‐
737 mum, average, and maximum session completion rates were 0.00, 0.59, and
738 2.40 sessions per second, respectively. The standard deviation was
739 0.37 sessions per second and 240 out of 450 sessions completed success‐
740 fully (210 failed due to errors such as timeouts).
741 The next line, labeled ``Session:'' shows the average length of a ses‐
743 sion measured in connections. In the example above, an average of 6.45
744 connections were required to complete a session.
745 The line labeled ``Session lifetime'' gives the average time it took to
747 complete a successful session. In the example above, it took an aver‐
748 age of 123.9 seconds.
749 The line labeled ``Session failtime'' gives the average time it took
751 before an unsuccessful session failed. In the example above, it took
752 on average 58.5 seconds for a session to fail.
753 Finally, the line labeled ``Session length histogram'' gives a his‐
755 togram of the number of replies received by each session. In the exam‐
756 ple above, 4 sessions ended after receiving no reply at all, 7 ended
757 after receiving one reply, and so on (the ellipsis indicates additional
758 histogram counts that were omitted from this manual for space reasons).
759 Note that this histogram does not distinguish between successful and
760 failed sessions.
761
764 Since the machine that httperf runs on has only a finite set of re‐
765 source available, it can not sustain arbitrarily high HTTP loads. For
766 example, one limiting factor is that there are only roughly 60,000 TCP
767 port numbers that can be in use at any given time. Since on most UNIX
768 systems it takes one minute for a TCP connection to be fully closed
769 (leave the TIME_WAIT state), the maximum rate a client can sustain is
770 at most 1,000 requests per second.
771 The actual sustainable rate is often much lower than that because be‐
773 fore running out of TCP ports, the machine is likely to run out of file
774 descriptors (one file descriptor is used up for each open TCP connec‐
775 tion). By default, HP-UX 10.20 allows 1,024 open file descriptors per
776 process. This means that without extra precautions, httperf could po‐
777 tentially very quickly use up all available file descriptors, at which
778 point it could not induce any additional load on the server. To avoid
779 this problem, httperf provides option --timeout to set a timeout for
780 all communication with the server. If the server does not respond be‐
781 fore the timeout expires, the client considers the corresponding ses‐
782 sion, connection, or call to be ``dead,'' closes the associated TCP
783 connection, and increases the ``client-timo'' error count. The only
784 exception to this rule is that after sending an entire request to the
785 server, httperf allows the server to take some additional time before
786 it starts sending the reply. This is to accommodate HTTP requests that
787 take a long time to complete on the server. This additional time is
788 called the ``server think time'' and can be specified by option
789 --think-timeout. By default, this additional think time is zero sec‐
790 onds, so the server would always have to respond within the time al‐
791 loted by option --timeout.
792 Timeouts allow httperf to sustain high offered loads even when the
794 server is overloaded. For example, with a timeout of 2 seconds and as‐
795 suming that 1,000 file-descriptors are available, the offered load
796 could be up to 500 requests per second (in practice, the sustainable
797 load is often somewhat smaller than the theoretical value). On the
798 downside, timeouts artificially truncate the connection lifetime dis‐
799 tribution. Thus, it is recommended to pick a timeout value that is as
800 large as possible yet small enough to allow sustaining the desired of‐
801 fered rate. A timeout as short as one second may be acceptable, but
802 larger timeouts (5-10 seconds) are preferable.
803 It is important to keep in mind that timeouts do not guarantee that a
805 client can sustain a particular offered load---there are many other po‐
806 tential resource bottlenecks. For example, in some cases the client
807 machine may simply run out of CPU time. To ensure that a given test
808 really measured the server's capabilities and not the client's, it is a
809 good idea to vary the number of machines participating in a test. If
810 observed performance remains the same as the number of client machines
811 is varied, the test results are likely to be valid.
812
814 httperf was developed by David Mosberger and was heavily influenced by
815 an earlier tool written by Tai Jin. Stephane Eranian contributed the
816 log-file based URI generator. Dick Carter contributed the --wsesslog
817 workload generator, the support behind the --period option, and bug
818 fixes. Ted Bullock has taken over maintenance and development activi‐
819 ties since September 2006.
820
822 Probably many. Always be sure to double-check results and don't fall
823 prey to measuring client-performance instead of server performance!
824 The user-interface definitely could be improved. A simple workload de‐
826 scription language might be more suitable than the dozens of little
827 command-line options the tool has right now.
828 01 Feb 2008 httperf(1)