1http(n)                      Tcl Bundled Packages                      http(n)
2
3
4
5______________________________________________________________________________
6

NAME

8       http - Client-side implementation of the HTTP/1.1 protocol
9

SYNOPSIS

11       package require http ?2.7?
12
13       ::http::config ?options?
14
15       ::http::geturl url ?options?
16
17       ::http::formatQuery key value ?key value ...?
18
19       ::http::reset token ?why?
20
21       ::http::wait token
22
23       ::http::status token
24
25       ::http::size token
26
27       ::http::code token
28
29       ::http::ncode token
30
31       ::http::meta token
32
33       ::http::data token
34
35       ::http::error token
36
37       ::http::cleanup token
38
39       ::http::register proto port command
40
41       ::http::unregister proto
42_________________________________________________________________
43

DESCRIPTION

45       The  http package provides the client side of the HTTP/1.1 protocol, as
46       defined in RFC 2616.  The package implements the GET,  POST,  and  HEAD
47       operations of HTTP/1.1.  It allows configuration of a proxy host to get
48       through firewalls.  The package is compatible with the  Safesock  secu‐
49       rity  policy, so it can be used by untrusted applets to do URL fetching
50       from a restricted set of hosts. This package can be extended to support
51       additional HTTP transport protocols, such as HTTPS, by providing a cus‐
52       tom socket command, via ::http::register.
53
54       The ::http::geturl procedure does  a  HTTP  transaction.   Its  options
55       determine  whether  a GET, POST, or HEAD transaction is performed.  The
56       return value of ::http::geturl is a token  for  the  transaction.   The
57       value  is  also  the name of an array in the ::http namespace that con‐
58       tains state information about the transaction.  The  elements  of  this
59       array are described in the STATE ARRAY section.
60
61       If the -command option is specified, then the HTTP operation is done in
62       the background.  ::http::geturl returns  immediately  after  generating
63       the  HTTP request and the callback is invoked when the transaction com‐
64       pletes.  For this to work, the Tcl event loop must be  active.   In  Tk
65       applications this is always true.  For pure-Tcl applications, the call‐
66       er can use ::http::wait after calling ::http::geturl to start the event
67       loop.
68

COMMANDS

70       ::http::config ?options?
71              The  ::http::config command is used to set and query the name of
72              the proxy server and port, and the User-Agent name used  in  the
73              HTTP  requests.   If  no options are specified, then the current
74              configuration is returned.  If a single argument  is  specified,
75              then  it  should  be  one of the flags described below.  In this
76              case the current value of that setting is returned.   Otherwise,
77              the  options should be a set of flags and values that define the
78              configuration:
79
80              -accept mimetypes
81                     The Accept header of the request.  The  default  is  */*,
82                     which  means  that  all  types of documents are accepted.
83                     Otherwise you can supply a comma-separated list  of  mime
84                     type patterns that you are willing to receive.  For exam‐
85                     ple, “image/gif, image/jpeg, text/*”.
86
87              -proxyhost hostname
88                     The name of the proxy host, if any.  If this value is the
89                     empty string, the URL host is contacted directly.
90
91              -proxyport number
92                     The proxy port number.
93
94              -proxyfilter command
95                     The   command   is   a   callback  that  is  made  during
96                     ::http::geturl to determine if a proxy is required for  a
97                     given  host.  One argument, a host name, is added to com‐
98                     mand when it is invoked.  If a  proxy  is  required,  the
99                     callback  should return a two-element list containing the
100                     proxy server and proxy port.  Otherwise the filter should
101                     return  an  empty  list.   The default filter returns the
102                     values of the -proxyhost and -proxyport settings if  they
103                     are non-empty.
104
105              -urlencoding encoding
106                     The  encoding  used  for  creating the x-url-encoded URLs
107                     with ::http::formatQuery.  The default is utf-8, as spec‐
108                     ified  by  RFC 2718.  Prior to http 2.5 this was unspeci‐
109                     fied, and that behavior can be returned by specifying the
110                     empty  string  ({}), although iso8859-1 is recommended to
111                     restore similar behavior but without the  ::http::format‐
112                     Query  throwing  an  error processing non-latin-1 charac‐
113                     ters.
114
115              -useragent string
116                     The value of the User-Agent header in the  HTTP  request.
117                     The default is “Tcl http client package 2.7”.
118
119       ::http::geturl url ?options?
120              The ::http::geturl command is the main procedure in the package.
121              The -query option causes a  POST  operation  and  the  -validate
122              option  causes  a  HEAD operation; otherwise, a GET operation is
123              performed.  The ::http::geturl command  returns  a  token  value
124              that  can be used to get information about the transaction.  See
125              the  STATE  ARRAY  and  ERRORS   section   for   details.    The
126              ::http::geturl  command  blocks  until  the operation completes,
127              unless the -command option specifies a callback that is  invoked
128              when  the HTTP transaction completes.  ::http::geturl takes sev‐
129              eral options:
130
131              -binary boolean
132                     Specifies whether to force interpreting the URL  data  as
133                     binary.   Normally  this  is  auto-detected (anything not
134                     beginning with a  text  content  type  or  whose  content
135                     encoding is gzip or compress is considered binary data).
136
137              -blocksize size
138                     The  block  size used when reading the URL.  At most size
139                     bytes are read at once.  After each block, a call to  the
140                     -progress callback is made (if that option is specified).
141
142              -channel name
143                     Copy  the  URL contents to channel name instead of saving
144                     it in state(body).
145
146              -command callback
147                     Invoke callback after  the  HTTP  transaction  completes.
148                     This  option causes ::http::geturl to return immediately.
149                     The callback gets an  additional  argument  that  is  the
150                     token  returned  from  ::http::geturl.  This token is the
151                     name of an array that is described  in  the  STATE  ARRAY
152                     section.  Here is a template for the callback:
153                            proc httpCallback {token} {
154                                upvar #0 $token state
155                                # Access state as a Tcl array
156                            }
157
158              -handler callback
159                     Invoke  callback  whenever  HTTP  data  is  available; if
160                     present, nothing else will be done with  the  HTTP  data.
161                     This  procedure gets two additional arguments: the socket
162                     for  the  HTTP  data  and   the   token   returned   from
163                     ::http::geturl.   The token is the name of a global array
164                     that is described in the STATE ARRAY section.  The proce‐
165                     dure  is expected to return the number of bytes read from
166                     the socket.  Here is a template for the callback:
167                            proc httpHandlerCallback {socket token} {
168                                upvar #0 $token state
169                                # Access socket, and state as a Tcl array
170                                # For example...
171                                ...
172                                set data [read $socket 1000]
173                                set nbytes [string length $data]
174                                ...
175                                return $nbytes
176                            }
177
178              -headers keyvaluelist
179                     This option is used to add  extra  headers  to  the  HTTP
180                     request.   The  keyvaluelist argument must be a list with
181                     an even number of elements that  alternate  between  keys
182                     and  values.   The  keys become header field names.  New‐
183                     lines are stripped from the values so the  header  cannot
184                     be corrupted.  For example, if keyvaluelist is Pragma no-
185                     cache then the following header is included in  the  HTTP
186                     request:
187                     Pragma: no-cache
188
189              -keepalive boolean
190                     If  true, attempt to keep the connection open for servic‐
191                     ing multiple requests.  Default is 0.
192
193              -method type
194                     Force the HTTP request  method  to  type.  ::http::geturl
195                     will  auto-select  GET,  POST  or  HEAD  based  on  other
196                     options, but this option enables  choices  like  PUT  and
197                     DELETE for webdav support.
198
199              -myaddr address
200                     Pass  an  specific local address to the underlying socket
201                     call in case multiple interfaces are available.
202
203              -progress callback
204                     The callback is made after each transfer of data from the
205                     URL.   The  callback gets three additional arguments: the
206                     token from ::http::geturl, the expected total size of the
207                     contents  from the Content-Length meta-data, and the cur‐
208                     rent number of bytes transferred so  far.   The  expected
209                     total  size  may be unknown, in which case zero is passed
210                     to the callback.  Here is a  template  for  the  progress
211                     callback:
212                            proc httpProgress {token total current} {
213                                upvar #0 $token state
214                            }
215
216              -protocol version
217                     Select  the  HTTP protocol version to use. This should be
218                     1.0 or 1.1 (the default). Should only  be  necessary  for
219                     servers  that  do  not  understand  or otherwise complain
220                     about HTTP/1.1.
221
222              -query query
223                     This flag causes ::http::geturl to do a POST request that
224                     passes  the  query to the server. The query must be an x-
225                     url-encoding formatted  query.   The  ::http::formatQuery
226                     procedure can be used to do the formatting.
227
228              -queryblocksize size
229                     The  block  size used when posting query data to the URL.
230                     At most size bytes  are  written  at  once.   After  each
231                     block,  a call to the -queryprogress callback is made (if
232                     that option is specified).
233
234              -querychannel channelID
235                     This flag causes ::http::geturl to do a POST request that
236                     passes the data contained in channelID to the server. The
237                     data contained in channelID  must  be  an  x-url-encoding
238                     formatted  query  unless  the -type option below is used.
239                     If a Content-Length  header  is  not  specified  via  the
240                     -headers  options,  ::http::geturl  attempts to determine
241                     the size of the post data in order to create that header.
242                     If  it  is  unable  to  determine the size, it returns an
243                     error.
244
245              -queryprogress callback
246                     The callback is made after each transfer of data  to  the
247                     URL  (i.e.  POST)  and  acts  exactly  like the -progress
248                     option (the callback format is the same).
249
250              -strict boolean
251                     Whether  to  enforce  RFC  3986  URL  validation  on  the
252                     request.  Default is 1.
253
254              -timeout milliseconds
255                     If  milliseconds is non-zero, then ::http::geturl sets up
256                     a timeout to occur after the  specified  number  of  mil‐
257                     liseconds.   A timeout results in a call to ::http::reset
258                     and to the -command callback, if specified.   The  return
259                     value  of  ::http::status  is timeout after a timeout has
260                     occurred.
261
262              -type mime-type
263                     Use mime-type as the Content-Type value, instead  of  the
264                     default  value (application/x-www-form-urlencoded) during
265                     a POST operation.
266
267              -validate boolean
268                     If boolean is non-zero, then ::http::geturl does an  HTTP
269                     HEAD  request.   This  request  returns  meta information
270                     about the URL, but the contents are  not  returned.   The
271                     meta  information  is available in the state(meta)  vari‐
272                     able after the transaction.  See the STATE ARRAY  section
273                     for details.
274
275       ::http::formatQuery key value ?key value ...?
276              This  procedure  does x-url-encoding of query data.  It takes an
277              even number of arguments that are the keys  and  values  of  the
278              query.  It encodes the keys and values, and generates one string
279              that has the proper & and = separators.  The result is  suitable
280              for the -query value passed to ::http::geturl.
281
282       ::http::reset token ?why?
283              This command resets the HTTP transaction identified by token, if
284              any.  This sets the state(status) value to why,  which  defaults
285              to reset, and then calls the registered -command callback.
286
287       ::http::wait token
288              This  is  a  convenience procedure that blocks and waits for the
289              transaction to  complete.   This  only  works  in  trusted  code
290              because  it  uses  vwait.   Also,  it is not useful for the case
291              where ::http::geturl  is  called  without  the  -command  option
292              because  in  this  case  the ::http::geturl call does not return
293              until the HTTP transaction is complete, and thus there is  noth‐
294              ing to wait for.
295
296       ::http::data token
297              This  is  a  convenience procedure that returns the body element
298              (i.e., the URL data) of the state array.
299
300       ::http::error token
301              This is a convenience procedure that returns the  error  element
302              of the state array.
303
304       ::http::status token
305              This  is a convenience procedure that returns the status element
306              of the state array.
307
308       ::http::code token
309              This is a convenience procedure that returns the http element of
310              the state array.
311
312       ::http::ncode token
313              This  is  a  convenience procedure that returns just the numeric
314              return code (200, 404, etc.) from the http element of the  state
315              array.
316
317       ::http::size token
318              This  is  a  convenience  procedure that returns the currentsize
319              element of the state array, which represents the number of bytes
320              received from the URL in the ::http::geturl call.
321
322       ::http::meta token
323              This is a convenience procedure that returns the meta element of
324              the state array which contains the HTTP  response  headers.  See
325              below for an explanation of this element.
326
327       ::http::cleanup token
328              This  procedure  cleans up the state associated with the connec‐
329              tion identified by token.  After this call, the procedures  like
330              ::http::data  cannot be used to get information about the opera‐
331              tion.  It is strongly recommended that you  call  this  function
332              after you are done with a given HTTP request.  Not doing so will
333              result in  memory  not  being  freed,  and  if  your  app  calls
334              ::http::geturl  enough times, the memory leak could cause a per‐
335              formance hit...or worse.
336
337       ::http::register proto port command
338              This procedure allows one to provide custom HTTP transport types
339              such  as  HTTPS,  by registering a prefix, the default port, and
340              the command to execute to create the Tcl channel. E.g.:
341                     package require http
342                     package require tls
343
344                     ::http::register https 443 ::tls::socket
345
346                     set token [::http::geturl https://my.secure.site/]
347
348       ::http::unregister proto
349              This procedure unregisters a protocol handler  that  was  previ‐
350              ously registered via ::http::register.
351

ERRORS

353       The  ::http::geturl procedure will raise errors in the following cases:
354       invalid command line options, an invalid URL, a URL on  a  non-existent
355       host,  or  a  URL at a bad port on an existing host.  These errors mean
356       that it cannot even start the network transaction.  It will also  raise
357       an  error  if  it  gets an I/O error while writing out the HTTP request
358       header.  For synchronous ::http::geturl calls (where  -command  is  not
359       specified),  it will raise an error if it gets an I/O error while read‐
360       ing the HTTP reply headers or data.  Because  ::http::geturl  does  not
361       return  a  token  in  these cases, it does all the required cleanup and
362       there is no issue of your app having to call ::http::cleanup.
363
364       For asynchronous ::http::geturl calls, all of the  above  error  situa‐
365       tions  apply,  except that if there is any error while reading the HTTP
366       reply headers or data, no exception is thrown.  This is  because  after
367       writing  the  HTTP headers, ::http::geturl returns, and the rest of the
368       HTTP transaction occurs in the background.  The  command  callback  can
369       check  if  any error occurred during the read by calling ::http::status
370       to check the status and if its error, calling ::http::error to get  the
371       error message.
372
373       Alternatively,  if the main program flow reaches a point where it needs
374       to know the result of  the  asynchronous  HTTP  request,  it  can  call
375       ::http::wait  and  then  check  status  and error, just as the callback
376       does.
377
378       In any case, you must still call ::http::cleanup to  delete  the  state
379       array when you are done.
380
381       There  are other possible results of the HTTP transaction determined by
382       examining the status from ::http::status.  These are described below.
383
384       ok     If the HTTP transaction completes entirely, then status will  be
385              ok.   However,  you should still check the ::http::code value to
386              get the HTTP status.  The ::http::ncode procedure provides  just
387              the numeric error (e.g., 200, 404 or 500) while the ::http::code
388              procedure returns a value like “HTTP 404 File not found”.
389
390       eof    If the server closes the socket without replying, then no  error
391              is raised, but the status of the transaction will be eof.
392
393       error  The  error message will also be stored in the error status array
394              element, accessible via ::http::error.
395
396       Another error possibility is that ::http::geturl is unable to write all
397       the post query data to the server before the server responds and closes
398       the socket.  The error message is saved in the posterror  status  array
399       element  and then  ::http::geturl attempts to complete the transaction.
400       If it can read the server's response it will end up with an ok  status,
401       otherwise it will have an eof status.
402

STATE ARRAY

404       The ::http::geturl procedure returns a token that can be used to get to
405       the state of the HTTP transaction in the form of a Tcl array.  Use this
406       construct to create an easy-to-use array variable:
407              upvar #0 $token state
408       Once  the  data  associated with the URL is no longer needed, the state
409       array should be unset to free up storage.  The  ::http::cleanup  proce‐
410       dure is provided for that purpose.  The following elements of the array
411       are supported:
412
413              body   The contents of the URL.   This  will  be  empty  if  the
414                     -channel  option  has  been  specified.   This  value  is
415                     returned by the ::http::data command.
416
417              charset
418                     The value of the charset attribute from the  Content-Type
419                     meta-data value.  If none was specified, this defaults to
420                     the   RFC   standard   iso8859-1,   or   the   value   of
421                     $::http::defaultCharset.   Incoming  text  data  will  be
422                     automatically converted from this charset to utf-8.
423
424              coding A copy of the Content-Encoding meta-data value.
425
426              currentsize
427                     The current number of bytes fetched from the  URL.   This
428                     value is returned by the ::http::size command.
429
430              error  If  defined,  this is the error string seen when the HTTP
431                     transaction was aborted.
432
433              http   The HTTP status reply from the  server.   This  value  is
434                     returned by the ::http::code command.  The format of this
435                     value is:
436                            HTTP/1.1 code string
437                     The code is a three-digit  number  defined  in  the  HTTP
438                     standard.   A  code of 200 is OK.  Codes beginning with 4
439                     or 5 indicate errors.  Codes beginning with 3  are  redi‐
440                     rection  errors.   In  this  case  the Location meta-data
441                     specifies a new URL that contains the requested  informa‐
442                     tion.
443
444              meta   The  HTTP  protocol  returns meta-data that describes the
445                     URL contents.  The meta element of the state array  is  a
446                     list of the keys and values of the meta-data.  This is in
447                     a format useful for initializing an array that just  con‐
448                     tains the meta-data:
449                            array set meta $state(meta)
450                     Some of the meta-data keys are listed below, but the HTTP
451                     standard defines more, and servers are free to add  their
452                     own.
453
454                     Content-Type
455                            The  type  of  the URL contents.  Examples include
456                            text/html, image/gif,  application/postscript  and
457                            application/x-tcl.
458
459                     Content-Length
460                            The  advertised  size of the contents.  The actual
461                            size obtained by ::http::geturl  is  available  as
462                            state(size).
463
464                     Location
465                            An alternate URL that contains the requested data.
466
467              posterror
468                     The  error,  if any, that occurred while writing the post
469                     query data to the server.
470
471              status Either ok, for successful  completion,  reset  for  user-
472                     reset,  timeout if a timeout occurred before the transac‐
473                     tion could complete, or error  for  an  error  condition.
474                     During the transaction this value is the empty string.
475
476              totalsize
477                     A copy of the Content-Length meta-data value.
478
479              type   A copy of the Content-Type meta-data value.
480
481              url    The requested URL.
482

EXAMPLE

484              # Copy a URL to a file and print meta-data
485              proc httpcopy { url file {chunk 4096} } {
486                 set out [open $file w]
487                 set token [::http::geturl $url -channel $out \
488                        -progress httpCopyProgress -blocksize $chunk]
489                 close $out
490
491                 # This ends the line started by httpCopyProgress
492                 puts stderr ""
493
494                 upvar #0 $token state
495                 set max 0
496                 foreach {name value} $state(meta) {
497                    if {[string length $name] > $max} {
498                       set max [string length $name]
499                    }
500                    if {[regexp -nocase ^location$ $name]} {
501                       # Handle URL redirects
502                       puts stderr "Location:$value"
503                       return [httpcopy [string trim $value] $file $chunk]
504                    }
505                 }
506                 incr max
507                 foreach {name value} $state(meta) {
508                    puts [format "%-*s %s" $max $name: $value]
509                 }
510
511                 return $token
512              }
513              proc httpCopyProgress {args} {
514                 puts -nonewline stderr .
515                 flush stderr
516              }
517

SEE ALSO

519       safe(n), socket(n), safesock(n)
520

KEYWORDS

522       security policy, socket
523
524
525
526http                                  2.7                              http(n)
Impressum