1httperf(1) General Commands Manual httperf(1)
2
6 httperf - HTTP performance measurement tool
7
9 httperf [--add-header S] [--burst-length N] [--client I/N] [--close-
10 with-reset] [-d|--debug N] [--failure-status N] [-h|--help] [--hog]
11 [--http-version S] [--max-connections N] [--max-piped-calls N]
12 [--method S] [--no-host-hdr] [--num-calls N] [--num-conns N] [--period
13 [d|u|e]T1[,T2]] [--port N] [--print-reply [header|body]] [--print-
14 request [header|body]] [--rate X] [--recv-buffer N] [--retry-on-fail‐
15 ure] [--send-buffer N] [--server S] [--server-name S] [--session-
16 cookie] [--ssl] [--ssl-ciphers L] [--ssl-no-reuse] [--think-timeout X]
17 [--timeout X] [--uri S] [-v|--verbose] [-V|--version] [--wlog y|n,F]
18 [--wsess N,N,X] [--wsesslog N,X,F] [--wset N,X]
19
21 httperf is a tool to measure web server performance. It speaks the
22 HTTP protocol both in its HTTP/1.0 and HTTP/1.1 flavors and offers a
23 variety of workload generators. While running, it keeps track of a num‐
24 ber of performance metrics that are summarized in the form of statis‐
25 tics that are printed at the end of a test run. The most basic opera‐
26 tion of httperf is to generate a fixed number of HTTP GET requests and
27 to measure how many replies (responses) came back from the server and
28 at what rate the responses arrived.
29 IMPORTANT: To obtain correct results, it is necessary to run at most
31 one httperf process per client machine. Also, there should be as few
32 background processes as possible both on the client and server
33 machines.
34
37 httperf --hog --server www
38 This command causes httperf to create a connection to host www,
39 send a request for the root document (http://www/), receive the
40 reply, close the connection, and then print some performance
41 statistics.
42 httperf --hog --server www --num-conn 100 --ra 10 --timeout 5
44 Like above, except that a total of 100 connections are created
45 and that connections are created at a fixed rate of 10 per sec‐
46 ond. Note that option ``--rate'' has been abbreviated to
47 ``--ra''.
48 httperf --hog --server=www --wsess=10,5,2 --rate 1 --timeout 5
50 Causes httperf to generate a total of 10 sessions at a rate of 1
51 session per second. Each session consists of 5 calls that are
52 spaced out by 2 seconds.
53 httperf --hog --server=www --wsess=10,5,2 --rate=1 --timeout=5 --ssl
55 Like above, except that httperf contacts server www via SSL at
56 port 443 (the default port for SSL connections).
57 httperf --hog --server www --wsess=10,5,2 --rate=1 --timeout=5 --ssl
59 --ssl-ciphers=EXP-RC4-MD5:EXP-RC2-CBC-MD5 --ssl-no-reuse --http-ver‐
60 sion=1.0
61 Like above, except that httperf will inform the server that it
62 can only select from two cipher suites (EXP-RC4-MD5 or EXP-
63 RC2-CBC-MD5); furthermore, httperf will use HTTP version 1.0
64 which requires a new TCP connection for each request. Also, SSL
65 session ids are not reused, so the entire SSL connection estab‐
66 lishment process (known as the SSL handshake) occurs for each
67 connection.
68
70 The operation of httperf can be controlled through a number of options.
71 The tool supports both short (one-character) and long (arbitrary-
72 length) option names. Short options are prefixed with a single leading
73 dash (-), long options with a double-dash (--). Multiple short options
74 can be grouped together (e.g., ``-vV'' is equivalent to ``-v -V'') and
75 long options can be abbreviated so long as they remain unique. Parame‐
76 ters to options can be specified either by following the long option
77 name with an equal sign and the parameter value (e.g., --burst=10) or
78 by separating the option name and value with whitespace (e.g., --burst
79 10).
80 --add-header=S
82 Specifies to include string S as an additional request header.
83 It is necessary to specify the terminating carriage-return/line-
84 feed sequence explicitly. This can be done by using the escape
85 sequence ``\n''. This makes it possible to include multiple
86 request headers. For example, ``--add-header "Referer:
87 foo\nAuth: secret\n"'' would add two request headers (``Ref‐
88 erer'' and ``Auth'') to each request. Other supported escape
89 sequences are ``\r'' (carriage-return), ``\a'' (line-feed),
90 ``\\'' (backslash), and ``\N'' where N is the code the character
91 to be inserted (in octal).
92 --burst-length=N
94 Specifies the length of bursts. Each burst consists of N calls
95 to the server. The exact meaning of this parameter depends on
96 the workload generator. For regular request-oriented workloads,
97 see the description of option --wsess.
98 --no-host-hdr
100 Specifies that the "Host:" header should not be included when
101 issuing an HTTP request.
102 --num-calls.
104 For session-oriented workloads, see the description of option
105 --wsess.
106 --client=I/N
108 Specifies that the machine httperf is running on is client I out
109 of a total of N clients. I should be in the range from 0 to
110 N-1. Some of the workload generators (e.g., --wset) use the
111 client identity as a bias value to ensure that not all clients
112 generate perfectly identical workloads. When performing a test
113 that involves several client machines, it is generally a good
114 idea to specify this option.
115 --close-with-reset
117 Requests that httperf closes TCP connections by sending a RESET
118 instead of going through the normal TCP connection shutdown
119 handshake. Turning on this option can have ill effects such as
120 data corruption, stuck TCP control blocks, or wrong results.
121 For this reason, the option should not be used unless absolutely
122 necessary and even then it should not be used unless its impli‐
123 cations are fully understood.
124 -d=N
126 --debug=N
128 Set debug level to N. Larger values of N will result in more
129 output.
130 --failure-status=N
132 Specifies that an HTTP response status code of N should be
133 treated as a failure (i.e., treated as if the request had timed
134 out, for example). For example, with ``--failure-status=504''
135 responses with an HTTP status of ``504 Gateway Time-out'' would
136 be considered failures. Caveat: this option is currently sup‐
137 ported for session workloads only (see the --wsess and --wsess‐
138 log options).
139 -h
141 --help Prints a summary of available options and their parameters.
143 --hog This option requests to use up as many TCP ports as necessary.
145 Without this option, httperf is typically limited to using
146 ephemeral ports (in the range from 1024 to 5000). This limited
147 port range can quickly become a bottleneck so it is generally a
148 good idea to specify this option for serious testing. Also,
149 this option must be specified when measuring NT servers since it
150 avoids a TCP incompatibility between NT and UNIX machines.
151 --http-version=S
153 Specifies the version string that should be included in the
154 requests sent to the server. By default, version string ``1.1''
155 is used. This option can be set to ``1.0'' to force the genera‐
156 tion of HTTP/1.0 requests. Setting this option to any value
157 other than ``1.0'' or ``1.1'' may result in undefined behavior.
158 --max-connections=N
160 Specifies that at most N connections are opened for each ses‐
161 sion. This option is meaningful in conjunction with options
162 --wsess and --wsesslog only.
163 --max-piped-calls=N
165 Specifies that at most N pipelined calls are issued on each con‐
166 nection. This option is meaningful in conjunction with options
167 --wsess and --wsesslog only.
168 --method=S
170 Specifies the method that should be used when issuing an HTTP
171 request. If this option is not specified, the GET method is
172 used. The method S can be an arbitrary string but is usually
173 one of GET, HEAD, PUT, POST, etc.
174 --num-calls=N
176 This option is meaningful for request-oriented workloads only.
177 It specifies the total number of calls to issue on each connec‐
178 tion before closing it. If N is greater than 1, the server must
179 support persistent connections. The default value for this
180 option is 1. If --burst-length is set to B, then the N calls
181 are issued in bursts of B pipelined calls each. Thus, the total
182 number of such bursts will be N/B (per connection).
183 --num-conns=N
185 This option is meaningful for request-oriented workloads only.
186 It specifies the total number of connections to create. On each
187 connection, calls are issued as specified by options --num-calls
188 and --burst-length. A test stops as soon as the N connections
189 have either completed or failed. A connection is considered to
190 have failed if any activity on the connection fails to make for‐
191 ward progress for more than the time specified by the timeout
192 options --timeout and --think-timeout. The default value for
193 this option is 1.
194 --period=[D]T1[,T2]
196 Specifies the time interval between the creation of connections
197 or sessions. Connections are created by default, sessions if
198 option --wsess or --wsesslog has been specified. This connec‐
199 tion/session ``interarrival time'' can alternatively be speci‐
200 fied by the --rate option, although more flexibility is avail‐
201 able with --period. The D parameter specifies the interarrival
202 time distribution. If omitted or set to ``d'', a deterministic
203 (i.e., fixed) period is used as specified by parameter T1 in
204 units of seconds. If D is set to ``e'', an exponential (i.e.,
205 Poisson) distribution is used with a mean interarrival time of
206 T1. Finally, if D is set to ``u'', a uniform distribution over
207 the interval [T1,T2) is used for the interarrival time. In all
208 cases, a period of 0 results in connections or sessions being
209 generated sequentially (a new connection/session is initiated as
210 soon as the previous one completes). The default value for this
211 option is 0. Note that specifying, for example, --rate=5 is
212 equivalent to specifying --period=d0.2 or --period=0.2. By
213 specifying --period=u1,3, the interarrival times will be ran‐
214 domly chosen from the interval between 1 and 3 seconds. The
215 specific sequence of (pseudo-)random interarrival times are
216 identical from one httperf run to another as long as the values
217 for the --period and --client options are identical.
218 --port=N
220 This option specifies the port number N on which the web server
221 is listening for HTTP requests. By default, httperf uses port
222 number 80.
223 --print-reply[=[header|body]]
225 Requests the printing of the reply headers, body, and summary.
226 The output is directed to standard output. Reply header lines
227 are prefixed by "RH", reply body lines are prefixed by "RB", and
228 the reply-size summary is prefixed by "RS". The prefix is fol‐
229 lowed by a serial number that uniquely identifies the call that
230 the reply line is for and a colon (":") character that marks the
231 beginning of the actual reply line. To print only reply head‐
232 ers, pass argument header to this option. To print only the
233 reply body, pass argument body to this option.
234 --print-request[=[header|body]]
236 Requests the printing of the request headers, body (if one is
237 present), and summary. The output is directed to standard out‐
238 put. Request header lines are prefixed by "SH", request body
239 lines are prefixed by "SB", and the request summary is prefixed
240 by "SS". The prefix is followed by the call's serial number and
241 a colon (":") character that marks the beginning of the actual
242 reply line. To print only request headers, pass argument header
243 to this option. To print only the request body, pass argument
244 body to this option.
245 --rate=X
247 Specifies the fixed rate at which connections or sessions are
248 created. Connections are created by default, sessions if option
249 --wsess or --wsesslog has been specified. In both cases a rate
250 of 0 results in connections or sessions being generated sequen‐
251 tially (a new session/connection is initiated as soon as the
252 previous one completes). The default value for this option is
253 0.
254 --recv-buffer=N
256 Specifies the maximum size of the socket receive buffers used to
257 receive HTTP replies. By default, the limit is 16KB. A smaller
258 value may help memory-constrained clients whereas a larger value
259 may be necessary when communicating with a server over a high-
260 bandwidth, high-latency connection.
261 --retry-on-failure
263 This option is meaningful for session workloads only (see the
264 --wsess and --wsesslog options). If specified, a call that
265 results in a failure response (as defined by the --failure-sta‐
266 tus option) is retried immediately instead of causing the ses‐
267 sion to fail.
268 --send-buffer=N
270 Specifies the maximum size of the socket send buffers used to
271 send HTTP requests. By default, the limit is 4KB. A smaller
272 value may help memory-constrained clients whereas a larger value
273 may be necessary when generating large requests to a server con‐
274 nected via a high-bandwidth, high-latency connection.
275 --server=S
277 Specifies the IP hostname of the server. By default, the host‐
278 name ``localhost'' is used. This option should always be speci‐
279 fied as it is generally not a good idea to run the client and
280 the server on the same machine.
281 --server-name=S
283 Specifies the (default) server name that appears in the "Host:"
284 header of every request sent by httperf. Without this option,
285 the host name (or IP address) specified by option --server is
286 used instead.
287 --session-cookie
289 When this option is turned on, cookie managment is enabled on a
290 per-session basis. What this means is that if a reply to a
291 request that was generated by session X contains a cookie, then
292 all future requests sent by session X will include this cookie
293 as well. At present, the cookie manager in httperf supports
294 only one cookie per session. If a second cookie is received,
295 the new cookie overwrites the existing one and a warning message
296 is printed if ``--debug 1'' is on.
297 --ssl Specifies that all communication between httperf and the server
299 should utilize the Secure Sockets Layer (SSL) protocol. This
300 option is available only if httperf was compiled with SSL sup‐
301 port enabled.
302 --ssl-ciphers=L
304 This option is only meaningful if SSL is in use (see --ssl
305 option). This option specifies the list L of cipher suites that
306 httperf may use in negotiating a secure connection with the
307 server. If the list contains more than one cipher suite, the
308 ciphers must be separated by a colon. If the server does not
309 accept any of the listed cipher suites, the connection estab‐
310 lishment will fail and httperf will exit immediately. If this
311 option is not specified when the --ssl option is present then
312 httperf will use all of the SSLv3 cipher suites provided by the
313 underlying SSL library.
314 --ssl-no-reuse
316 This option is only meaningful if SSL and sessions are in use
317 (see --ssl, --wsess, --wsesslog). When an SSL connection is
318 established the client receives a session identifier (session
319 id) from the server. On subsequent SSL connections, the client
320 normally reuses this session id in order to avoid the expense of
321 repeating the (slow) SSL handshake to establish a new SSL ses‐
322 sion and obtain another session id (even if the client attempts
323 to re-use a session id, the server may force the client to rene‐
324 gotiate a session). By default httperf reuses the session id
325 across all connections in a session. If the --ssl-no-reuse
326 option is in effect, then httperf will not reuse the session id,
327 and the entire SSL handshake will be performed for each new con‐
328 nection in a session.
329 --think-timeout=X
331 Specifies the maximum time that the server may need to initiate
332 sending the reply for a given request. Note that this timeout
333 value is added to the normal timeout value (see option --time‐
334 out). When accessing static web content, it is usually not nec‐
335 essary to specify this option. However, when performing tests
336 with long-running CGI scripts, it may be necessary to use this
337 option to allow for larger response-times. The default value
338 for this option is zero seconds, meaning that the server has to
339 be able to respond within the normal timeout value.
340 --timeout=X
342 Specifies the amount of time X that httperf is willing to wait
343 for a server reaction. The timeout is specified in seconds and
344 can be a fractional number (e.g., --timeout 3.5). This timeout
345 value is used when establishing a TCP connection, when sending a
346 request, when waiting for a reply, and when receiving a reply.
347 If during any of those activities a request fails to make for‐
348 ward progress within the alloted time, httperf considers the
349 request to have died, closes the associated connection or ses‐
350 sion and increases the client-timo error count. The actual
351 timeout value used when waiting for a reply is the sum of this
352 timeout and the think-timeout (see option --think-timeout). By
353 default, the timeout value is infinity.
354 --uri=S
356 Specifies that URI S should be accessed on the server. For some
357 of the workload generators (e.g., --wset), this option specifies
358 the prefix for the URIs being accessed.
359 -v
361 --verbose
363 Puts httperf into verbose mode. In this mode, additional output
364 such as the individual reply rate samples and connection life‐
365 time histogram are printed.
366 -V
368 --version
370 Prints the version of httperf.
371 --wlog=B,F
373 This option can be used to generate a specific sequence of URI
374 accesses. This is useful to replay the accesses recorded in a
375 server log file, for example. Parameter F is the name of a file
376 containing the ASCII NUL separated list of URIs that should be
377 accessed. If parameter B is set to ``y'', httperf will wrap
378 around to the beginning of the file when reaching the end of the
379 list (so the list of URIs is accessed repeatedly). With B set
380 to ``n'', the test will stop no later than when reaching the end
381 of the URI list.
382 --wsess=N1,N2,X
384 Requests the generation and measurement of sessions instead of
385 individual requests. A session consists of a sequence of bursts
386 which are spaced out by the user think-time. Each burst con‐
387 sists of a fixed number L of calls to the server (L is specified
388 by option --burst-length). The calls in a burst are issued as
389 follows: at first, a single call is issued. Once the reply to
390 this first call has been fully received, all remaining calls in
391 the burst are issued concurrently. The concurrent calls are
392 issued either as pipelined calls on an existing persistent con‐
393 nection or as individual calls on separate connections. Whether
394 a persistent connection is used depends on whether the server
395 responds to the first call with a reply that includes a ``Con‐
396 nection: close'' header line. If such a line is present, sepa‐
397 rate connections are used.
398 The option specifies the following parameters: N1 is the total
400 number of sessions to generate, N2 is the number of calls per
401 session, and X is the user think-time (in seconds) that sepa‐
402 rates consecutive call bursts. For example, the options
403 ``--wsess=100,50,10 --burst-len 5'' would result in 100 sessions
404 with a total of 50 calls each. Since each burst has a length of
405 5 calls, a total of 10 call bursts would be generated per ses‐
406 sion. The user think-time between call bursts would be 10 sec‐
407 onds. Note that user think-time X denotes the time between
408 receiving the last reply of the previous call burst and the
409 sending of the first request of the next burst.
410 A test involving sessions finishes as soon as the requested num‐
412 ber N1 of sessions have either failed or completed. A session
413 is considered to have failed if any operation in a session takes
414 longer than the timeouts specified by options --timeout and
415 --think-timeout. In addition, a session also fails if the
416 server returns a reply with a status code matching the one spec‐
417 ified by option --failure-status.
418 --wsesslog=N,X,F
420 This specifies a session workload generator similar to --wsess
421 (please read that description first). With --wsesslog though,
422 many aspects of user sessions, including the number and sequence
423 of URI's, request method, think-time and burst-length parame‐
424 ters, can be specified in an input file F. Two other parameters
425 are retained from --wsess, namely N, the number of sessions to
426 initiate, and X, the burst-to-burst user think time (note that
427 this becomes a default time since the input file F can also
428 specify user think time on a per-burst basis. A small example
429 input file can most-easily show the settable parameters:
430 # Comment lines start with a ``#'' as the first
432 # character. Lines with only whitespace delimit
433 # sessions (multiple blank lines do not generate
434 # ``null'' sessions). All other lines specify a
435 # uri-sequence (1 uri per line). If the first
436 # character of the line is whitespace (e.g. space
437 # or tab), the uri is considered to be part of a
438 # burst that is sent out after the previous
439 # non-burst uri.
440 # session 1 definition (this is a comment)
442 /foo.html think=2.0
443 /pict1.gif
444 /pict2.gif
445 /foo2.html method=POST contents='Post data'
446 /pict3.gif
447 /pict4.gif
448 # session 2 definition
450 /foo3.html method=POST contents="Multiline\ndata"
451 /foo4.html method=HEAD
452 The above description specifies 2 sessions. The first session
454 will start with a request for /foo.html. When the /foo.html
455 response comes back, a burst of 2 requests will follow
456 (/pict1.gif and /pict2.gif). When the last of those responses
457 is received, a two second user think time is inserted before the
458 next request of /foo2.html is issued. This request is sent as a
459 POST. The posted data can be contained between single- or dou‐
460 ble-quotes. Newlines can appear within posted data as ``\n'' or
461 as a ``\<CR>''. The /foo2.html response is followed by a burst
462 request of /pict3.gif and /pict4.gif, which concludes this ses‐
463 sion. The second session is started some time after the first,
464 as specified by the --rate or --period options.
465 The second session consists of 2 requests separated by the
467 default user think time as specified by the X parameter of the
468 --wsesslog option. If the N parameter of --wsesslog is greater
469 than the number of sessions defined in input file F, then the
470 defined sessions are used repeatedly until N sessions have been
471 created (i.e., the defined sessions are used in a round-robin
472 fashion).
473 One should avoid using --wsesslog in conjunction with other
475 httperf options that also control session behavior and workload
476 URI's, namely --burst-length, --wsess, --wlog, and --wset.
477 --wset=N,X
479 This option can be used to walk through a list of URIs at a
480 given rate. Parameter N specifies the number of distinct URIs
481 that should be generated and X specifies the rate at which new
482 URIs are accessed. A rate of 0.25 would mean that the same URI
483 would be accessed four times in a row before moving on to the
484 next URI. This type of access pattern is useful in generating a
485 workload that induces a relatively predictable amount of traffic
486 in the disk I/O subsystem of the server (assuming N and the
487 accessed files are big enough to exceed the server's buffer
488 cache). The URIs generated are of the form prefix/path.html,
489 where prefix is the URI prefix specified by option --wset and
490 path is generated as follows: for the i-th file in the working
491 set, write down i in decimal, prefixing the number with as many
492 zeroes as necessary to get a string that has as many digits as
493 N-1. Then insert a slash character between each digit. For
494 example, the 103rd file in a working set consisting of 1024
495 files would result in a path of ``0/1/0/3''. Thus, if the URI-
496 prefix is /wset1024, then the URI being accessed would be
497 /wset1024/0/1/0/3.html. In other words, the files on the server
498 need to be organized as a 10ary tree.
499
501 This section describes the statistics output at the end of each test
502 run. The basic information shown below is printed independent of the
503 selected workload generator.
504 Total: connections 30000 requests 29997 replies 29997 test-dura‐
506 tion 299.992 s
507 Connection rate: 100.0 conn/s (10.0 ms/conn, <=14 concurrent
509 connections)
510 Connection time [ms]: min 1.4 avg 3.0 max 163.4 median 1.5 std‐
511 dev 7.3
512 Connection time [ms]: connect 0.6
513 Connection length [replies/conn]: 1.000
514 Request rate: 100.0 req/s (10.0 ms/req)
516 Request size [B]: 75.0
517 Reply rate [replies/s]: min 98.8 avg 100.0 max 101.2 stddev 0.3
519 (60 samples)
520 Reply time [ms]: response 2.4 transfer 0.0
521 Reply size [B]: header 242.0 content 1010.0 footer 0.0 (total
522 1252.0)
523 Reply status: 1xx=0 2xx=29997 3xx=0 4xx=0 5xx=0
524 CPU time [s]: user 94.31 system 205.26 (user 31.4% system 68.4%
526 total 99.9%)
527 Net I/O: 129.6 KB/s (1.1*10^6 bps)
528 Errors: total 3 client-timo 0 socket-timo 0 connrefused 3 con‐
530 nreset 0
531 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
532 There are six groups of statistics: overall results (``Total''), con‐
534 nection related results (``Connection''), results relating to the issu‐
535 ing of HTTP requests (``Request''), results relating to the replies
536 received from the server (``Reply''), miscellaneous results relating to
537 the CPU (``CPU'') and network (``Net I/O'') utilization and, last but
538 not least, a summary of errors encountered (``Errors'').
539 Total Section
541 This section summarizes how many TCP connections were initiated
542 by httperf, how many requests it sent out, how many replies it
543 received, and what the total test duration was. In the example
544 output shown above, 30,000 connections were created, 29,997
545 requests were sent out and 29,997 replies were received. The
546 duration of the test was almost exactly 5 minutes (300 seconds).
547 Connection Section
549 This section conveys information related to TCP connections gen‐
550 erated by the tool. Specifically, the ``Connection rate'' line
551 shows that new connections were initiated at a rate of 100.0
552 connections per second. This rate corresponds to a period of
553 10.0 milliseconds per connection. The last number in this line
554 shows that at most 14 connections were open at any given time.
555 The first line labeled ``Connection time'' gives lifetime sta‐
557 tistics for successful connections. The lifetime of a connec‐
558 tion is the time between a TCP connection is initiated and the
559 time the connection is closed. A connection is considered suc‐
560 cessful if it had at least one call that completed successfully.
561 In the example output, the line indicates that the minimum
562 (``min'') connection lifetime was 1.4 milliseconds, the average
563 (``avg'') lifetime was 3.0 milliseconds, the maximum (``max'')
564 was 163.4 milliseconds, the median (``median'') lifetime was 1.5
565 milliseconds, and that the standard deviation of the lifetimes
566 was 7.3 milliseconds. The median lifetime is computed based on
567 a histogram with one millisecond resolution and a maximum life‐
568 time of 100 seconds. Thus, the median is accurate to within
569 half a millisecond if at least half of the successful connec‐
570 tions have a lifetime of no more than 100 seconds.
571 The next statistic in this section is the average time it took
573 to establish a TCP connection. Only successful TCP connection
574 establishments are counted. In the example, the second line
575 labeled ``Connection time'' shows that, on average, it took 0.6
576 milliseconds to establish a connection.
577 The final line in this section is labeled ``Connection length.''
579 It gives the average number of replies received on each connec‐
580 tion that received at least one reply (i.e., connections that
581 failed before yielding the first reply are not counted). This
582 number can be bigger than 1.0 due to persistent connections.
583 Request Section
585 The line labeled ``Request rate'' gives the rate at which HTTP
586 requests were issued and the period that this rate corresponds
587 to. In the example above, the request rate was 100.0 requests
588 per second, which corresponds to 10.0 milliseconds per request.
589 As long as no persistent connections are employed, the results
590 in this section are very similar or identical to results in the
591 connection section. However, when persistent connections are
592 used, several calls can be performed on a single connection in
593 which case the results would be different.
594 The line labeled ``Request size'' gives the average size of the
596 HTTP requests in bytes. In the example above, the average
597 request size was 75 bytes.
598 Reply Section
600 For simple measurements, this section is often the most inter‐
601 esting one as the line labeled ``Reply rate'' gives various sta‐
602 tistics for the reply rate. In the example above, the minimum
603 (``min'') reply rate was 98.8 replies per second, the average
604 (``avg'') was 100 replies per second, and the maximum (``max'')
605 rate was 101.2 replies per second. The standard deviation was
606 0.3 replies per second. The number enclosed in parentheses
607 shows that 60 reply rate samples were acquired. At present,
608 httperf collects a rate sample once every five seconds. To
609 obtain a meaningful standard deviation, it is recommended to run
610 tests long enough so at least thirty samples are obtained. This
611 corresponds to a test duration of at least 150 seconds.
612 The line labeled ``Reply Time'' gives information on how long it
614 took for the server to respond and how long it took to receive
615 the reply. In the example, it took on average 2.4 milliseconds
616 between sending the first byte of the request and receiving the
617 first byte of the reply. The time to ``transfer'', or read, the
618 reply was too short to be measured, so it shows up as zero. The
619 is typical when the entire reply fits into a single TCP segment.
620 The next line, labeled ``Reply size'' contains statistics on the
622 average size of the replies---all numbers are in reported bytes.
623 Specifically, the line lists the average length of reply head‐
624 ers, the content, and footers (HTTP/1.1 uses footers to realize
625 the ``chunked'' transfer encoding). For convenience, the aver‐
626 age total number of bytes in the replies is also given in paren‐
627 theses. In the example, the average header length (``header'')
628 was 242 bytes, the average content length (``content'') was 1010
629 bytes, and there were no footers (``footer'' length is zero).
630 The total reply length of 1252 bytes on average.
631 The final line in this section is a histogram of the major sta‐
633 tus codes received in the replies from the server. The major
634 status code is the ``hundreds''-digit of the full HTTP status
635 code. In the example, all 29,997 replies had a major status
636 code of 2. It's a good guess that all status codes were ``200
637 OK'' but the information in the histogram is not detailed enough
638 to allow distinguishing status codes with the same major code.
639 Miscellaneous Section
641 This section starts with a summary of the CPU utilization on the
642 client machine. In the example, the line labeled ``CPU time''
643 shows that 94.31 seconds were spent executing in user mode
644 (``user''), 205.26 seconds were spent executing in system mode
645 (``system'') and that this corresponds to 31.4% user mode execu‐
646 tion and 68.4% system execution. The total utilization was
647 99.9%, which is expected given that httperf is a CPU hog. A
648 total CPU utilization of significantly less than 100% is a sign
649 that there were competing processes that interfered with the
650 test.
651 The line labeled ``Net I/O'' gives the average network through‐
653 put in kilobytes per second (where a kilobyte is 1024 bytes) and
654 in megabits per second (where a megabit is 10^6 bits). In the
655 example, an average network usage of about 129.6 kilobytes per
656 second was sustained. The number in parentheses shows that this
657 corresponds to about 1.1 megabits per second. This network
658 bandwidth is computed based on the number of bytes sent and
659 received on the TCP connections. In other words, it does not
660 account for the network headers or TCP retransmissions that may
661 have occurred.
662 Errors Section
664 The last section contains statistics on the errors that were
665 encountered during a test. In the example, the two lines
666 labeled ``Errors'' show that there were a total of three errors
667 and that all three errors were due to the server refusing to
668 accept a connection (``connrefused''). A description of each
669 error counter follows:
670 client-timo: The number of times a session, connection, or call
672 failed due to a client timeout (as specified by the --timeout
673 and --think-timeout) options.
674 socket-timo: The number of times a TCP connection failed with a
676 socket-level timeout (ETIMEDOUT).
677 connrefused: The number of times a TCP connection attempt failed
679 with a ``connection refused by server'' error (ECONNREFUSED).
680 connreset: The number of times a TCP connection failed due to a
682 RESET from the server. Typically, a RESET is received when the
683 client attempts to send data to the server at a time the server
684 has already closed its end of the connection. NT servers also
685 send RESETs when attempting to establish a new connection when
686 the listen queue is full.
687 fd-unavail: The number of times the httperf process was out of
689 file descriptors. Whenever this count is non-zero, the test
690 results are meaningless because the client was overloaded (see
691 section "CHOOSING TIMEOUT VALUES").
692 addrunavail: The number of times the client was out of TCP port
694 numbers (EADDRNOTAVAIL). This error should never occur. If it
695 does, the results should be discarded.
696 ftab-full: The number of times the system's file descriptor ta‐
698 ble is full. Again, this error should never occur. If it does,
699 the results should be discarded.
700 other: The number of times some other type of error occurred.
702 Whenever this counter is non-zero, it is necessary to track down
703 the real cause of the error. To assist in doing this, httperf
704 prints the error code (errno) of the first unknown errors that
705 occurs during a test run.
706 When --wsess or --wsesslog is specified, httperf generates and measures
708 sessions instead of individual calls and additional statistics are
709 printed at the end of a test. An example output is shown below.
710 Session rate [sess/s]: min 0.00 avg 0.59 max 2.40 stddev 0.37
712 (240/450)
713 Session: avg 6.45 connections/session
714 Session lifetime [s]: 123.9
715 Session failtime [s]: 58.5
716 Session length histogram: 4 7 4 ... 3 3 240
717 The line labeled ``Session rate'' shows the minium, average, and maxi‐
719 mum rate at which sessions completed (based on a 5 second sampling
720 interval). It also shows the standard deviation of the session comple‐
721 tion rate. The numbers in parentheses show how many sessions succeeded
722 and how many sessions were initiated. In the example above, the mini‐
723 mum, average, and maximum session completion rates were 0.00, 0.59, and
724 2.40 sessions per second, respectively. The standard deviation was
725 0.37 sessions per second and 240 out of 450 sessions completed success‐
726 fully (210 failed due to errors such as timeouts).
727 The next line, labeled ``Session:'' shows the average length of a ses‐
729 sion measured in connections. In the example above, an average of 6.45
730 connections were required to complete a session.
731 The line labeled ``Session lifetime'' gives the average time it took to
733 complete a successful session. In the example above, it took an aver‐
734 age of 123.9 seconds.
735 The line labeled ``Session failtime'' gives the average time it took
737 before an unsuccessful session failed. In the example above, it took
738 on average 58.5 seconds for a session to fail.
739 Finally, the line labeled ``Session length histogram'' gives a his‐
741 togram of the number of replies received by each session. In the exam‐
742 ple above, 4 sessions ended after receiving no reply at all, 7 ended
743 after receiving one reply, and so on (the ellipsis indicates additional
744 histogram counts that were omitted from this manual for space reasons).
745 Note that this histogram does not distinguish between successful and
746 failed sessions.
747
750 Since the machine that httperf runs on has only a finite set of
751 resource available, it can not sustain arbitrarily high HTTP loads.
752 For example, one limiting factor is that there are only roughly 60,000
753 TCP port numbers that can be in use at any given time. Since on most
754 UNIX systems it takes one minute for a TCP connection to be fully
755 closed (leave the TIME_WAIT state), the maximum rate a client can sus‐
756 tain is at most 1,000 requests per second.
757 The actual sustainable rate is often much lower than that because
759 before running out of TCP ports, the machine is likely to run out of
760 file descriptors (one file descriptor is used up for each open TCP con‐
761 nection). By default, HP-UX 10.20 allows 1,024 open file descriptors
762 per process. This means that without extra precautions, httperf could
763 potentially very quickly use up all available file descriptors, at
764 which point it could not induce any additional load on the server. To
765 avoid this problem, httperf provides option --timeout to set a timeout
766 for all communication with the server. If the server does not respond
767 before the timeout expires, the client considers the corresponding ses‐
768 sion, connection, or call to be ``dead,'' closes the associated TCP
769 connection, and increases the ``client-timo'' error count. The only
770 exception to this rule is that after sending an entire request to the
771 server, httperf allows the server to take some additional time before
772 it starts sending the reply. This is to accommodate HTTP requests that
773 take a long time to complete on the server. This additional time is
774 called the ``server think time'' and can be specified by option
775 --think-timeout. By default, this additional think time is zero sec‐
776 onds, so the server would always have to respond within the time
777 alloted by option --timeout.
778 Timeouts allow httperf to sustain high offered loads even when the
780 server is overloaded. For example, with a timeout of 2 seconds and
781 assuming that 1,000 file-descriptors are available, the offered load
782 could be up to 500 requests per second (in practice, the sustainable
783 load is often somewhat smaller than the theoretical value). On the
784 downside, timeouts artificially truncate the connection lifetime dis‐
785 tribution. Thus, it is recommended to pick a timeout value that is as
786 large as possible yet small enough to allow sustaining the desired
787 offered rate. A timeout as short as one second may be acceptable, but
788 larger timeouts (5-10 seconds) are preferable.
789 It is important to keep in mind that timeouts do not guarantee that a
791 client can sustain a particular offered load---there are many other
792 potential resource bottlenecks. For example, in some cases the client
793 machine may simply run out of CPU time. To ensure that a given test
794 really measured the server's capabilities and not the client's, it is a
795 good idea to vary the number of machines participating in a test. If
796 observed performance remains the same as the number of client machines
797 is varied, the test results are likely to be valid.
798
800 httperf was developed by David Mosberger and was heavily influenced by
801 an earlier tool written by Tai Jin. Stephane Eranian contributed the
802 log-file based URI generator. Dick Carter contributed the --wsesslog
803 workload generator, the support behind the --period option, and bug
804 fixes. All four authors are with Hewlett-Packard Research Laborato‐
805 ries.
806
808 Probably many. Always be sure to double-check results and don't fall
809 prey to measuring client-performance instead of server performance!
810 The user-interface definitely could be improved. A simple workload
812 description language might be more suitable than the dozens of little
813 command-line options the tool has right now.
814 30 Oct 2000 httperf(1)