1httperf(1)                  General Commands Manual                 httperf(1)
2
3
4

NAME

6       httperf - HTTP performance measurement tool
7

SYNOPSIS

9       httperf  [--add-header  S]  [--burst-length N] [--client I/N] [--close-
10       with-reset] [-d|--debug N]  [--failure-status  N]  [-h|--help]  [--hog]
11       [--http-version   S]   [--max-connections   N]   [--max-piped-calls  N]
12       [--method S] [--no-host-hdr] [--num-calls N] [--num-conns N]  [--period
13       [d|u|e]T1[,T2]]  [--port  N]  [--print-reply  [header|body]]  [--print-
14       request [header|body]] [--rate X] [--recv-buffer  N]  [--retry-on-fail‐
15       ure]  [--send-buffer  N]  [--server  S]  [--server-name  S] [--session-
16       cookie] [--ssl] [--ssl-ciphers L] [--ssl-no-reuse] [--think-timeout  X]
17       [--timeout  X]  [--uri  S] [-v|--verbose] [-V|--version] [--wlog y|n,F]
18       [--wsess N,N,X] [--wsesslog N,X,F] [--wset N,X]
19

DESCRIPTION

21       httperf is a tool to measure web server  performance.   It  speaks  the
22       HTTP  protocol  both  in its HTTP/1.0 and HTTP/1.1 flavors and offers a
23       variety of workload generators. While running, it keeps track of a num‐
24       ber  of  performance metrics that are summarized in the form of statis‐
25       tics that are printed at the end of a test run.  The most basic  opera‐
26       tion  of httperf is to generate a fixed number of HTTP GET requests and
27       to measure how many replies (responses) came back from the  server  and
28       at what rate the responses arrived.
29
30       IMPORTANT:  To  obtain  correct results, it is necessary to run at most
31       one httperf process per client machine.  Also, there should be  as  few
32       background  processes  as  possible  both  on  the  client  and  server
33       machines.
34
35

EXAMPLES

37       httperf --hog --server www
38              This command causes httperf to create a connection to host  www,
39              send  a request for the root document (http://www/), receive the
40              reply, close the connection, and  then  print  some  performance
41              statistics.
42
43       httperf --hog --server www --num-conn 100 --ra 10 --timeout 5
44              Like  above,  except that a total of 100 connections are created
45              and that connections are created at a fixed rate of 10 per  sec‐
46              ond.   Note  that  option  ``--rate''  has  been  abbreviated to
47              ``--ra''.
48
49       httperf --hog --server=www --wsess=10,5,2 --rate 1 --timeout 5
50              Causes httperf to generate a total of 10 sessions at a rate of 1
51              session  per  second.  Each session consists of 5 calls that are
52              spaced out by 2 seconds.
53
54       httperf --hog --server=www --wsess=10,5,2 --rate=1 --timeout=5 --ssl
55              Like above, except that httperf contacts server www via  SSL  at
56              port 443 (the default port for SSL connections).
57
58       httperf  --hog  --server  www --wsess=10,5,2 --rate=1 --timeout=5 --ssl
59       --ssl-ciphers=EXP-RC4-MD5:EXP-RC2-CBC-MD5  --ssl-no-reuse   --http-ver‐
60       sion=1.0
61              Like  above,  except that httperf will inform the server that it
62              can only select from two  cipher  suites  (EXP-RC4-MD5  or  EXP-
63              RC2-CBC-MD5);  furthermore,  httperf  will  use HTTP version 1.0
64              which requires a new TCP connection for each request.  Also, SSL
65              session  ids are not reused, so the entire SSL connection estab‐
66              lishment process (known as the SSL handshake)  occurs  for  each
67              connection.
68

OPTIONS

70       The operation of httperf can be controlled through a number of options.
71       The tool supports  both  short  (one-character)  and  long  (arbitrary-
72       length) option names.  Short options are prefixed with a single leading
73       dash (-), long options with a double-dash (--).  Multiple short options
74       can  be grouped together (e.g., ``-vV'' is equivalent to ``-v -V'') and
75       long options can be abbreviated so long as they remain unique.  Parame‐
76       ters  to  options  can be specified either by following the long option
77       name with an equal sign and the parameter value (e.g.,  --burst=10)  or
78       by  separating the option name and value with whitespace (e.g., --burst
79       10).
80
81       --add-header=S
82              Specifies to include string S as an additional  request  header.
83              It is necessary to specify the terminating carriage-return/line-
84              feed sequence explicitly.  This can be done by using the  escape
85              sequence  ``\n''.   This  makes  it possible to include multiple
86              request  headers.    For   example,   ``--add-header   "Referer:
87              foo\nAuth:  secret\n"''  would  add  two request headers (``Ref‐
88              erer'' and ``Auth'') to each request.   Other  supported  escape
89              sequences  are  ``\r''  (carriage-return),  ``\a''  (line-feed),
90              ``\\'' (backslash), and ``\N'' where N is the code the character
91              to be inserted (in octal).
92
93       --burst-length=N
94              Specifies  the length of bursts.  Each burst consists of N calls
95              to the server.  The exact meaning of this parameter  depends  on
96              the workload generator.  For regular request-oriented workloads,
97              see the description of option --wsess.
98
99       --no-host-hdr
100              Specifies that the "Host:" header should not  be  included  when
101              issuing an HTTP request.
102
103       --num-calls.
104              For  session-oriented  workloads,  see the description of option
105              --wsess.
106
107       --client=I/N
108              Specifies that the machine httperf is running on is client I out
109              of  a  total  of  N clients.  I should be in the range from 0 to
110              N-1.  Some of the workload generators  (e.g.,  --wset)  use  the
111              client  identity  as a bias value to ensure that not all clients
112              generate perfectly identical workloads.  When performing a  test
113              that  involves  several  client machines, it is generally a good
114              idea to specify this option.
115
116       --close-with-reset
117              Requests that httperf closes TCP connections by sending a  RESET
118              instead  of  going  through  the  normal TCP connection shutdown
119              handshake.  Turning on this option can have ill effects such  as
120              data  corruption,  stuck  TCP  control blocks, or wrong results.
121              For this reason, the option should not be used unless absolutely
122              necessary  and even then it should not be used unless its impli‐
123              cations are fully understood.
124
125       -d=N
126
127       --debug=N
128              Set debug level to N.  Larger values of N will  result  in  more
129              output.
130
131       --failure-status=N
132              Specifies  that  an  HTTP  response  status  code of N should be
133              treated as a failure (i.e., treated as if the request had  timed
134              out,  for  example).  For example, with ``--failure-status=504''
135              responses with an HTTP status of ``504 Gateway Time-out''  would
136              be  considered  failures.  Caveat: this option is currently sup‐
137              ported for session workloads only (see the --wsess and  --wsess‐
138              log options).
139
140       -h
141
142       --help Prints a summary of available options and their parameters.
143
144       --hog  This  option  requests to use up as many TCP ports as necessary.
145              Without this option,  httperf  is  typically  limited  to  using
146              ephemeral  ports (in the range from 1024 to 5000).  This limited
147              port range can quickly become a bottleneck so it is generally  a
148              good  idea  to  specify  this option for serious testing.  Also,
149              this option must be specified when measuring NT servers since it
150              avoids a TCP incompatibility between NT and UNIX machines.
151
152       --http-version=S
153              Specifies  the  version  string  that  should be included in the
154              requests sent to the server.  By default, version string ``1.1''
155              is used.  This option can be set to ``1.0'' to force the genera‐
156              tion of HTTP/1.0 requests.  Setting this  option  to  any  value
157              other than ``1.0'' or ``1.1'' may result in undefined behavior.
158
159       --max-connections=N
160              Specifies  that  at  most N connections are opened for each ses‐
161              sion.  This option is meaningful  in  conjunction  with  options
162              --wsess and --wsesslog only.
163
164       --max-piped-calls=N
165              Specifies that at most N pipelined calls are issued on each con‐
166              nection.  This option is meaningful in conjunction with  options
167              --wsess and --wsesslog only.
168
169       --method=S
170              Specifies  the  method  that should be used when issuing an HTTP
171              request.  If this option is not specified,  the  GET  method  is
172              used.   The  method  S can be an arbitrary string but is usually
173              one of GET, HEAD, PUT, POST, etc.
174
175       --num-calls=N
176              This option is meaningful for request-oriented  workloads  only.
177              It  specifies the total number of calls to issue on each connec‐
178              tion before closing it.  If N is greater than 1, the server must
179              support  persistent  connections.   The  default  value for this
180              option is 1.  If --burst-length is set to B, then  the  N  calls
181              are issued in bursts of B pipelined calls each.  Thus, the total
182              number of such bursts will be N/B (per connection).
183
184       --num-conns=N
185              This option is meaningful for request-oriented  workloads  only.
186              It specifies the total number of connections to create.  On each
187              connection, calls are issued as specified by options --num-calls
188              and  --burst-length.   A test stops as soon as the N connections
189              have either completed or failed.  A connection is considered  to
190              have failed if any activity on the connection fails to make for‐
191              ward progress for more than the time specified  by  the  timeout
192              options  --timeout  and  --think-timeout.  The default value for
193              this option is 1.
194
195       --period=[D]T1[,T2]
196              Specifies the time interval between the creation of  connections
197              or  sessions.   Connections  are created by default, sessions if
198              option --wsess or --wsesslog has been specified.   This  connec‐
199              tion/session  ``interarrival  time'' can alternatively be speci‐
200              fied by the --rate option, although more flexibility  is  avail‐
201              able  with --period.  The D parameter specifies the interarrival
202              time distribution.  If omitted or set to ``d'', a  deterministic
203              (i.e.,  fixed)  period  is  used as specified by parameter T1 in
204              units of seconds.  If D is set to ``e'', an  exponential  (i.e.,
205              Poisson)  distribution  is used with a mean interarrival time of
206              T1.  Finally, if D is set to ``u'', a uniform distribution  over
207              the  interval [T1,T2) is used for the interarrival time.  In all
208              cases, a period of 0 results in connections  or  sessions  being
209              generated sequentially (a new connection/session is initiated as
210              soon as the previous one completes).  The default value for this
211              option  is  0.   Note  that specifying, for example, --rate=5 is
212              equivalent to  specifying  --period=d0.2  or  --period=0.2.   By
213              specifying  --period=u1,3,  the  interarrival times will be ran‐
214              domly chosen from the interval between 1  and  3  seconds.   The
215              specific  sequence  of  (pseudo-)random  interarrival  times are
216              identical from one httperf run to another as long as the  values
217              for the --period and --client options are identical.
218
219       --port=N
220              This  option specifies the port number N on which the web server
221              is listening for HTTP requests.  By default, httperf  uses  port
222              number 80.
223
224       --print-reply[=[header|body]]
225              Requests  the  printing of the reply headers, body, and summary.
226              The output is directed to standard output.  Reply  header  lines
227              are prefixed by "RH", reply body lines are prefixed by "RB", and
228              the reply-size summary is prefixed by "RS".  The prefix is  fol‐
229              lowed  by a serial number that uniquely identifies the call that
230              the reply line is for and a colon (":") character that marks the
231              beginning  of  the actual reply line.  To print only reply head‐
232              ers, pass argument header to this option.   To  print  only  the
233              reply body, pass argument body to this option.
234
235       --print-request[=[header|body]]
236              Requests  the  printing  of the request headers, body (if one is
237              present), and summary.  The output is directed to standard  out‐
238              put.   Request  header  lines are prefixed by "SH", request body
239              lines are prefixed by "SB", and the request summary is  prefixed
240              by "SS".  The prefix is followed by the call's serial number and
241              a colon (":") character that marks the beginning of  the  actual
242              reply line.  To print only request headers, pass argument header
243              to this option.  To print only the request body,  pass  argument
244              body to this option.
245
246       --rate=X
247              Specifies  the  fixed  rate at which connections or sessions are
248              created.  Connections are created by default, sessions if option
249              --wsess  or --wsesslog has been specified.  In both cases a rate
250              of 0 results in connections or sessions being generated  sequen‐
251              tially  (a  new  session/connection  is initiated as soon as the
252              previous one completes).  The default value for this  option  is
253              0.
254
255       --recv-buffer=N
256              Specifies the maximum size of the socket receive buffers used to
257              receive HTTP replies.  By default, the limit is 16KB.  A smaller
258              value may help memory-constrained clients whereas a larger value
259              may be necessary when communicating with a server over  a  high-
260              bandwidth, high-latency connection.
261
262       --retry-on-failure
263              This  option  is  meaningful for session workloads only (see the
264              --wsess and --wsesslog options).   If  specified,  a  call  that
265              results  in a failure response (as defined by the --failure-sta‐
266              tus option) is retried immediately instead of causing  the  ses‐
267              sion to fail.
268
269       --send-buffer=N
270              Specifies  the  maximum  size of the socket send buffers used to
271              send HTTP requests.  By default, the limit is  4KB.   A  smaller
272              value may help memory-constrained clients whereas a larger value
273              may be necessary when generating large requests to a server con‐
274              nected via a high-bandwidth, high-latency connection.
275
276       --server=S
277              Specifies  the IP hostname of the server.  By default, the host‐
278              name ``localhost'' is used.  This option should always be speci‐
279              fied  as  it  is generally not a good idea to run the client and
280              the server on the same machine.
281
282       --server-name=S
283              Specifies the (default) server name that appears in the  "Host:"
284              header  of  every request sent by httperf.  Without this option,
285              the host name (or IP address) specified by  option  --server  is
286              used instead.
287
288       --session-cookie
289              When  this option is turned on, cookie managment is enabled on a
290              per-session basis.  What this means is that  if  a  reply  to  a
291              request  that was generated by session X contains a cookie, then
292              all future requests sent by session X will include  this  cookie
293              as  well.   At  present,  the cookie manager in httperf supports
294              only one cookie per session.  If a second  cookie  is  received,
295              the new cookie overwrites the existing one and a warning message
296              is printed if ``--debug 1'' is on.
297
298       --ssl  Specifies that all communication between httperf and the  server
299              should  utilize  the  Secure Sockets Layer (SSL) protocol.  This
300              option is available only if httperf was compiled with  SSL  sup‐
301              port enabled.
302
303       --ssl-ciphers=L
304              This  option  is  only  meaningful  if  SSL is in use (see --ssl
305              option).  This option specifies the list L of cipher suites that
306              httperf  may  use  in  negotiating  a secure connection with the
307              server.  If the list contains more than one  cipher  suite,  the
308              ciphers  must  be  separated by a colon.  If the server does not
309              accept any of the listed cipher suites,  the  connection  estab‐
310              lishment  will  fail and httperf will exit immediately.  If this
311              option is not specified when the --ssl option  is  present  then
312              httperf  will use all of the SSLv3 cipher suites provided by the
313              underlying SSL library.
314
315       --ssl-no-reuse
316              This option is only meaningful if SSL and sessions  are  in  use
317              (see  --ssl,  --wsess,  --wsesslog).   When an SSL connection is
318              established the client receives a  session  identifier  (session
319              id)  from the server.  On subsequent SSL connections, the client
320              normally reuses this session id in order to avoid the expense of
321              repeating  the  (slow) SSL handshake to establish a new SSL ses‐
322              sion and obtain another session id (even if the client  attempts
323              to re-use a session id, the server may force the client to rene‐
324              gotiate a session).  By default httperf reuses  the  session  id
325              across  all  connections  in  a  session.  If the --ssl-no-reuse
326              option is in effect, then httperf will not reuse the session id,
327              and the entire SSL handshake will be performed for each new con‐
328              nection in a session.
329
330       --think-timeout=X
331              Specifies the maximum time that the server may need to  initiate
332              sending  the  reply for a given request.  Note that this timeout
333              value is added to the normal timeout value (see  option  --time‐
334              out).  When accessing static web content, it is usually not nec‐
335              essary to specify this option.  However, when  performing  tests
336              with  long-running  CGI scripts, it may be necessary to use this
337              option to allow for larger response-times.   The  default  value
338              for  this option is zero seconds, meaning that the server has to
339              be able to respond within the normal timeout value.
340
341       --timeout=X
342              Specifies the amount of time X that httperf is willing  to  wait
343              for  a server reaction.  The timeout is specified in seconds and
344              can be a fractional number (e.g., --timeout 3.5).  This  timeout
345              value is used when establishing a TCP connection, when sending a
346              request, when waiting for a reply, and when receiving  a  reply.
347              If  during  any of those activities a request fails to make for‐
348              ward progress within the alloted  time,  httperf  considers  the
349              request  to  have died, closes the associated connection or ses‐
350              sion and increases the  client-timo  error  count.   The  actual
351              timeout  value  used when waiting for a reply is the sum of this
352              timeout and the think-timeout (see option --think-timeout).   By
353              default, the timeout value is infinity.
354
355       --uri=S
356              Specifies that URI S should be accessed on the server.  For some
357              of the workload generators (e.g., --wset), this option specifies
358              the prefix for the URIs being accessed.
359
360       -v
361
362       --verbose
363              Puts httperf into verbose mode.  In this mode, additional output
364              such as the individual reply rate samples and  connection  life‐
365              time histogram are printed.
366
367       -V
368
369       --version
370              Prints the version of httperf.
371
372       --wlog=B,F
373              This  option  can be used to generate a specific sequence of URI
374              accesses.  This is useful to replay the accesses recorded  in  a
375              server log file, for example.  Parameter F is the name of a file
376              containing the ASCII NUL separated list of URIs that  should  be
377              accessed.   If  parameter  B  is set to ``y'', httperf will wrap
378              around to the beginning of the file when reaching the end of the
379              list  (so  the list of URIs is accessed repeatedly).  With B set
380              to ``n'', the test will stop no later than when reaching the end
381              of the URI list.
382
383       --wsess=N1,N2,X
384              Requests  the  generation and measurement of sessions instead of
385              individual requests.  A session consists of a sequence of bursts
386              which  are  spaced  out by the user think-time.  Each burst con‐
387              sists of a fixed number L of calls to the server (L is specified
388              by  option  --burst-length).  The calls in a burst are issued as
389              follows: at first, a single call is issued.  Once the  reply  to
390              this  first call has been fully received, all remaining calls in
391              the burst are issued concurrently.   The  concurrent  calls  are
392              issued  either as pipelined calls on an existing persistent con‐
393              nection or as individual calls on separate connections.  Whether
394              a  persistent  connection  is used depends on whether the server
395              responds to the first call with a reply that includes  a  ``Con‐
396              nection:  close'' header line.  If such a line is present, sepa‐
397              rate connections are used.
398
399              The option specifies the following parameters: N1 is  the  total
400              number  of  sessions  to generate, N2 is the number of calls per
401              session, and X is the user think-time (in  seconds)  that  sepa‐
402              rates   consecutive  call  bursts.   For  example,  the  options
403              ``--wsess=100,50,10 --burst-len 5'' would result in 100 sessions
404              with a total of 50 calls each.  Since each burst has a length of
405              5 calls, a total of 10 call bursts would be generated  per  ses‐
406              sion.   The user think-time between call bursts would be 10 sec‐
407              onds.  Note that user think-time  X  denotes  the  time  between
408              receiving  the  last  reply  of  the previous call burst and the
409              sending of the first request of the next burst.
410
411              A test involving sessions finishes as soon as the requested num‐
412              ber  N1  of sessions have either failed or completed.  A session
413              is considered to have failed if any operation in a session takes
414              longer  than  the  timeouts  specified  by options --timeout and
415              --think-timeout.  In addition,  a  session  also  fails  if  the
416              server returns a reply with a status code matching the one spec‐
417              ified by option --failure-status.
418
419       --wsesslog=N,X,F
420              This specifies a session workload generator similar  to  --wsess
421              (please  read  that description first).  With --wsesslog though,
422              many aspects of user sessions, including the number and sequence
423              of  URI's,  request  method, think-time and burst-length parame‐
424              ters, can be specified in an input file F.  Two other parameters
425              are  retained  from --wsess, namely N, the number of sessions to
426              initiate, and X, the burst-to-burst user think time  (note  that
427              this  becomes  a  default  time  since the input file F can also
428              specify user think time on a per-burst basis.  A  small  example
429              input file can most-easily show the settable parameters:
430
431              # Comment lines start with a ``#'' as the first
432              # character.  Lines with only whitespace delimit
433              # sessions (multiple blank lines do not generate
434              # ``null'' sessions).  All other lines specify a
435              # uri-sequence (1 uri per line).  If the first
436              # character of the line is whitespace (e.g. space
437              # or tab), the uri is considered to be part of a
438              # burst that is sent out after the previous
439              # non-burst uri.
440
441              # session 1 definition (this is a comment)
442              /foo.html think=2.0
443                   /pict1.gif
444                   /pict2.gif
445              /foo2.html method=POST contents='Post data'
446                   /pict3.gif
447                   /pict4.gif
448
449              # session 2 definition
450              /foo3.html method=POST contents="Multiline\ndata"
451              /foo4.html method=HEAD
452
453              The  above  description specifies 2 sessions.  The first session
454              will start with a request for  /foo.html.   When  the  /foo.html
455              response   comes  back,  a  burst  of  2  requests  will  follow
456              (/pict1.gif and /pict2.gif).  When the last of  those  responses
457              is received, a two second user think time is inserted before the
458              next request of /foo2.html is issued.  This request is sent as a
459              POST.   The posted data can be contained between single- or dou‐
460              ble-quotes.  Newlines can appear within posted data as ``\n'' or
461              as  a ``\<CR>''.  The /foo2.html response is followed by a burst
462              request of /pict3.gif and /pict4.gif, which concludes this  ses‐
463              sion.   The second session is started some time after the first,
464              as specified by the --rate or --period options.
465
466              The second session consists  of  2  requests  separated  by  the
467              default  user  think time as specified by the X parameter of the
468              --wsesslog option.  If the N parameter of --wsesslog is  greater
469              than  the  number  of sessions defined in input file F, then the
470              defined sessions are used repeatedly until N sessions have  been
471              created  (i.e.,  the  defined sessions are used in a round-robin
472              fashion).
473
474              One should avoid using  --wsesslog  in  conjunction  with  other
475              httperf  options that also control session behavior and workload
476              URI's, namely --burst-length, --wsess, --wlog, and --wset.
477
478       --wset=N,X
479              This option can be used to walk through a  list  of  URIs  at  a
480              given  rate.   Parameter N specifies the number of distinct URIs
481              that should be generated and X specifies the rate at  which  new
482              URIs  are accessed.  A rate of 0.25 would mean that the same URI
483              would be accessed four times in a row before moving  on  to  the
484              next URI.  This type of access pattern is useful in generating a
485              workload that induces a relatively predictable amount of traffic
486              in  the  disk  I/O  subsystem  of the server (assuming N and the
487              accessed files are big enough  to  exceed  the  server's  buffer
488              cache).   The  URIs  generated are of the form prefix/path.html,
489              where prefix is the URI prefix specified by  option  --wset  and
490              path  is  generated as follows: for the i-th file in the working
491              set, write down i in decimal, prefixing the number with as  many
492              zeroes  as  necessary to get a string that has as many digits as
493              N-1.  Then insert a slash character  between  each  digit.   For
494              example,  the  103rd  file  in  a working set consisting of 1024
495              files would result in a path of ``0/1/0/3''.  Thus, if the  URI-
496              prefix  is  /wset1024,  then  the  URI  being  accessed would be
497              /wset1024/0/1/0/3.html.  In other words, the files on the server
498              need to be organized as a 10ary tree.
499

OUTPUT

501       This  section  describes  the statistics output at the end of each test
502       run.  The basic information shown below is printed independent  of  the
503       selected workload generator.
504
505              Total: connections 30000 requests 29997 replies 29997 test-dura‐
506              tion 299.992 s
507
508              Connection rate: 100.0 conn/s  (10.0  ms/conn,  <=14  concurrent
509              connections)
510              Connection  time [ms]: min 1.4 avg 3.0 max 163.4 median 1.5 std‐
511              dev 7.3
512              Connection time [ms]: connect 0.6
513              Connection length [replies/conn]: 1.000
514
515              Request rate: 100.0 req/s (10.0 ms/req)
516              Request size [B]: 75.0
517
518              Reply rate [replies/s]: min 98.8 avg 100.0 max 101.2 stddev  0.3
519              (60 samples)
520              Reply time [ms]: response 2.4 transfer 0.0
521              Reply  size  [B]:  header 242.0 content 1010.0 footer 0.0 (total
522              1252.0)
523              Reply status: 1xx=0 2xx=29997 3xx=0 4xx=0 5xx=0
524
525              CPU time [s]: user 94.31 system 205.26 (user 31.4% system  68.4%
526              total 99.9%)
527              Net I/O: 129.6 KB/s (1.1*10^6 bps)
528
529              Errors:  total  3 client-timo 0 socket-timo 0 connrefused 3 con‐
530              nreset 0
531              Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
532
533       There are six groups of statistics: overall results  (``Total''),  con‐
534       nection related results (``Connection''), results relating to the issu‐
535       ing of HTTP requests (``Request''), results  relating  to  the  replies
536       received from the server (``Reply''), miscellaneous results relating to
537       the CPU (``CPU'') and network (``Net I/O'') utilization and,  last  but
538       not least, a summary of errors encountered (``Errors'').
539
540       Total Section
541              This  section summarizes how many TCP connections were initiated
542              by httperf, how many requests it sent out, how many  replies  it
543              received,  and what the total test duration was.  In the example
544              output shown above,  30,000  connections  were  created,  29,997
545              requests  were  sent  out and 29,997 replies were received.  The
546              duration of the test was almost exactly 5 minutes (300 seconds).
547
548       Connection Section
549              This section conveys information related to TCP connections gen‐
550              erated  by the tool.  Specifically, the ``Connection rate'' line
551              shows that new connections were initiated at  a  rate  of  100.0
552              connections  per  second.   This rate corresponds to a period of
553              10.0 milliseconds per connection.  The last number in this  line
554              shows that at most 14 connections were open at any given time.
555
556              The  first  line labeled ``Connection time'' gives lifetime sta‐
557              tistics for successful connections.  The lifetime of  a  connec‐
558              tion  is  the time between a TCP connection is initiated and the
559              time the connection is closed.  A connection is considered  suc‐
560              cessful if it had at least one call that completed successfully.
561              In the example output,  the  line  indicates  that  the  minimum
562              (``min'')  connection lifetime was 1.4 milliseconds, the average
563              (``avg'') lifetime was 3.0 milliseconds, the  maximum  (``max'')
564              was 163.4 milliseconds, the median (``median'') lifetime was 1.5
565              milliseconds, and that the standard deviation of  the  lifetimes
566              was  7.3 milliseconds.  The median lifetime is computed based on
567              a histogram with one millisecond resolution and a maximum  life‐
568              time  of  100  seconds.   Thus, the median is accurate to within
569              half a millisecond if at least half of  the  successful  connec‐
570              tions have a lifetime of no more than 100 seconds.
571
572              The  next  statistic in this section is the average time it took
573              to establish a TCP connection.  Only successful  TCP  connection
574              establishments  are  counted.   In  the example, the second line
575              labeled ``Connection time'' shows that, on average, it took  0.6
576              milliseconds to establish a connection.
577
578              The final line in this section is labeled ``Connection length.''
579              It gives the average number of replies received on each  connec‐
580              tion  that  received  at least one reply (i.e., connections that
581              failed before yielding the first reply are not  counted).   This
582              number can be bigger than 1.0 due to persistent connections.
583
584       Request Section
585              The  line  labeled ``Request rate'' gives the rate at which HTTP
586              requests were issued and the period that this  rate  corresponds
587              to.   In  the example above, the request rate was 100.0 requests
588              per second, which corresponds to 10.0 milliseconds per  request.
589              As  long  as no persistent connections are employed, the results
590              in this section are very similar or identical to results in  the
591              connection  section.   However,  when persistent connections are
592              used, several calls can be performed on a single  connection  in
593              which case the results would be different.
594
595              The  line labeled ``Request size'' gives the average size of the
596              HTTP requests in bytes.   In  the  example  above,  the  average
597              request size was 75 bytes.
598
599       Reply Section
600              For  simple  measurements, this section is often the most inter‐
601              esting one as the line labeled ``Reply rate'' gives various sta‐
602              tistics  for  the reply rate.  In the example above, the minimum
603              (``min'') reply rate was 98.8 replies per  second,  the  average
604              (``avg'')  was 100 replies per second, and the maximum (``max'')
605              rate was 101.2 replies per second.  The standard  deviation  was
606              0.3  replies  per  second.   The  number enclosed in parentheses
607              shows that 60 reply rate samples  were  acquired.   At  present,
608              httperf  collects  a  rate  sample  once every five seconds.  To
609              obtain a meaningful standard deviation, it is recommended to run
610              tests long enough so at least thirty samples are obtained.  This
611              corresponds to a test duration of at least 150 seconds.
612
613              The line labeled ``Reply Time'' gives information on how long it
614              took  for  the server to respond and how long it took to receive
615              the reply.  In the example, it took on average 2.4  milliseconds
616              between  sending the first byte of the request and receiving the
617              first byte of the reply.  The time to ``transfer'', or read, the
618              reply was too short to be measured, so it shows up as zero.  The
619              is typical when the entire reply fits into a single TCP segment.
620
621              The next line, labeled ``Reply size'' contains statistics on the
622              average size of the replies---all numbers are in reported bytes.
623              Specifically, the line lists the average length of  reply  head‐
624              ers,  the content, and footers (HTTP/1.1 uses footers to realize
625              the ``chunked'' transfer encoding).  For convenience, the  aver‐
626              age total number of bytes in the replies is also given in paren‐
627              theses.  In the example, the average header length  (``header'')
628              was 242 bytes, the average content length (``content'') was 1010
629              bytes, and there were no footers (``footer''  length  is  zero).
630              The total reply length of 1252 bytes on average.
631
632              The  final line in this section is a histogram of the major sta‐
633              tus codes received in the replies from the  server.   The  major
634              status  code  is  the ``hundreds''-digit of the full HTTP status
635              code.  In the example, all 29,997 replies  had  a  major  status
636              code  of  2.  It's a good guess that all status codes were ``200
637              OK'' but the information in the histogram is not detailed enough
638              to allow distinguishing status codes with the same major code.
639
640       Miscellaneous Section
641              This section starts with a summary of the CPU utilization on the
642              client machine.  In the example, the line labeled  ``CPU  time''
643              shows  that  94.31  seconds  were  spent  executing in user mode
644              (``user''), 205.26 seconds were spent executing in  system  mode
645              (``system'') and that this corresponds to 31.4% user mode execu‐
646              tion and 68.4% system  execution.   The  total  utilization  was
647              99.9%,  which  is  expected  given that httperf is a CPU hog.  A
648              total CPU utilization of significantly less than 100% is a  sign
649              that  there  were  competing  processes that interfered with the
650              test.
651
652              The line labeled ``Net I/O'' gives the average network  through‐
653              put in kilobytes per second (where a kilobyte is 1024 bytes) and
654              in megabits per second (where a megabit is 10^6 bits).   In  the
655              example,  an  average network usage of about 129.6 kilobytes per
656              second was sustained.  The number in parentheses shows that this
657              corresponds  to  about  1.1  megabits  per second.  This network
658              bandwidth is computed based on the  number  of  bytes  sent  and
659              received  on  the  TCP connections.  In other words, it does not
660              account for the network headers or TCP retransmissions that  may
661              have occurred.
662
663       Errors Section
664              The  last  section  contains  statistics on the errors that were
665              encountered during a  test.   In  the  example,  the  two  lines
666              labeled  ``Errors'' show that there were a total of three errors
667              and that all three errors were due to  the  server  refusing  to
668              accept  a  connection  (``connrefused'').  A description of each
669              error counter follows:
670
671              client-timo: The number of times a session, connection, or  call
672              failed  due  to  a client timeout (as specified by the --timeout
673              and --think-timeout) options.
674
675              socket-timo: The number of times a TCP connection failed with  a
676              socket-level timeout (ETIMEDOUT).
677
678              connrefused: The number of times a TCP connection attempt failed
679              with a ``connection refused by server'' error (ECONNREFUSED).
680
681              connreset: The number of times a TCP connection failed due to  a
682              RESET  from the server.  Typically, a RESET is received when the
683              client attempts to send data to the server at a time the  server
684              has  already  closed its end of the connection.  NT servers also
685              send RESETs when attempting to establish a new  connection  when
686              the listen queue is full.
687
688              fd-unavail:  The  number of times the httperf process was out of
689              file descriptors.  Whenever this count  is  non-zero,  the  test
690              results  are  meaningless because the client was overloaded (see
691              section "CHOOSING TIMEOUT VALUES").
692
693              addrunavail: The number of times the client was out of TCP  port
694              numbers  (EADDRNOTAVAIL).  This error should never occur.  If it
695              does, the results should be discarded.
696
697              ftab-full: The number of times the system's file descriptor  ta‐
698              ble is full.  Again, this error should never occur.  If it does,
699              the results should be discarded.
700
701              other: The number of times some other type  of  error  occurred.
702              Whenever this counter is non-zero, it is necessary to track down
703              the real cause of the error.  To assist in doing  this,  httperf
704              prints  the  error code (errno) of the first unknown errors that
705              occurs during a test run.
706
707       When --wsess or --wsesslog is specified, httperf generates and measures
708       sessions  instead  of  individual  calls  and additional statistics are
709       printed at the end of a test.  An example output is shown below.
710
711              Session rate [sess/s]: min 0.00 avg 0.59 max  2.40  stddev  0.37
712              (240/450)
713              Session: avg 6.45 connections/session
714              Session lifetime [s]: 123.9
715              Session failtime [s]: 58.5
716              Session length histogram: 4 7 4 ... 3 3 240
717
718       The  line labeled ``Session rate'' shows the minium, average, and maxi‐
719       mum rate at which sessions completed (based  on  a  5  second  sampling
720       interval).  It also shows the standard deviation of the session comple‐
721       tion rate.  The numbers in parentheses show how many sessions succeeded
722       and  how many sessions were initiated.  In the example above, the mini‐
723       mum, average, and maximum session completion rates were 0.00, 0.59, and
724       2.40  sessions  per  second,  respectively.  The standard deviation was
725       0.37 sessions per second and 240 out of 450 sessions completed success‐
726       fully (210 failed due to errors such as timeouts).
727
728       The  next line, labeled ``Session:'' shows the average length of a ses‐
729       sion measured in connections.  In the example above, an average of 6.45
730       connections were required to complete a session.
731
732       The line labeled ``Session lifetime'' gives the average time it took to
733       complete a successful session.  In the example above, it took an  aver‐
734       age of 123.9 seconds.
735
736       The  line  labeled  ``Session failtime'' gives the average time it took
737       before an unsuccessful session failed.  In the example above,  it  took
738       on average 58.5 seconds for a session to fail.
739
740       Finally,  the  line  labeled  ``Session length histogram'' gives a his‐
741       togram of the number of replies received by each session.  In the exam‐
742       ple  above,  4  sessions ended after receiving no reply at all, 7 ended
743       after receiving one reply, and so on (the ellipsis indicates additional
744       histogram counts that were omitted from this manual for space reasons).
745       Note that this histogram does not distinguish  between  successful  and
746       failed sessions.
747
748

CHOOSING TIMEOUT VALUES

750       Since  the  machine  that  httperf  runs  on  has  only a finite set of
751       resource available, it can not sustain  arbitrarily  high  HTTP  loads.
752       For  example, one limiting factor is that there are only roughly 60,000
753       TCP port numbers that can be in use at any given time.  Since  on  most
754       UNIX  systems  it  takes  one  minute  for a TCP connection to be fully
755       closed (leave the TIME_WAIT state), the maximum rate a client can  sus‐
756       tain is at most 1,000 requests per second.
757
758       The  actual  sustainable  rate  is  often  much lower than that because
759       before running out of TCP ports, the machine is likely to  run  out  of
760       file descriptors (one file descriptor is used up for each open TCP con‐
761       nection).  By default, HP-UX 10.20 allows 1,024 open  file  descriptors
762       per  process.  This means that without extra precautions, httperf could
763       potentially very quickly use up  all  available  file  descriptors,  at
764       which  point it could not induce any additional load on the server.  To
765       avoid this problem, httperf provides option --timeout to set a  timeout
766       for  all communication with the server.  If the server does not respond
767       before the timeout expires, the client considers the corresponding ses‐
768       sion,  connection,  or  call  to be ``dead,'' closes the associated TCP
769       connection, and increases the ``client-timo'' error  count.   The  only
770       exception  to  this rule is that after sending an entire request to the
771       server, httperf allows the server to take some additional  time  before
772       it starts sending the reply.  This is to accommodate HTTP requests that
773       take a long time to complete on the server.  This  additional  time  is
774       called  the  ``server  think  time''  and  can  be  specified by option
775       --think-timeout.  By default, this additional think time is  zero  sec‐
776       onds,  so  the  server  would  always  have  to respond within the time
777       alloted by option --timeout.
778
779       Timeouts allow httperf to sustain high  offered  loads  even  when  the
780       server  is  overloaded.   For  example, with a timeout of 2 seconds and
781       assuming that 1,000 file-descriptors are available,  the  offered  load
782       could  be  up  to 500 requests per second (in practice, the sustainable
783       load is often somewhat smaller than the  theoretical  value).   On  the
784       downside,  timeouts  artificially truncate the connection lifetime dis‐
785       tribution.  Thus, it is recommended to pick a timeout value that is  as
786       large  as  possible  yet  small  enough to allow sustaining the desired
787       offered rate.  A timeout as short as one second may be acceptable,  but
788       larger timeouts (5-10 seconds) are preferable.
789
790       It  is  important to keep in mind that timeouts do not guarantee that a
791       client can sustain a particular offered  load---there  are  many  other
792       potential  resource bottlenecks.  For example, in some cases the client
793       machine may simply run out of CPU time.  To ensure that  a  given  test
794       really measured the server's capabilities and not the client's, it is a
795       good idea to vary the number of machines participating in a  test.   If
796       observed  performance remains the same as the number of client machines
797       is varied, the test results are likely to be valid.
798

AUTHOR

800       httperf was developed by David Mosberger and was heavily influenced  by
801       an  earlier  tool written by Tai Jin.  Stephane Eranian contributed the
802       log-file based URI generator.  Dick Carter contributed  the  --wsesslog
803       workload  generator,  the  support  behind the --period option, and bug
804       fixes.  All four authors are with  Hewlett-Packard  Research  Laborato‐
805       ries.
806

BUGS

808       Probably  many.   Always be sure to double-check results and don't fall
809       prey to measuring client-performance instead of server performance!
810
811       The user-interface definitely could be  improved.   A  simple  workload
812       description  language  might be more suitable than the dozens of little
813       command-line options the tool has right now.
814
815
816
817                                  30 Oct 2000                       httperf(1)
Impressum