1tool(n)                         Tcl Web Server                         tool(n)
2
3
4
5______________________________________________________________________________
6

NAME

8       tool - A TclOO and coroutine based web server
9

SYNOPSIS

11       package require Tcl  8.6
12
13       package require httpd  ?4.1.1?
14
15       package require sha1
16
17       package require dicttool
18
19       package require oo::meta
20
21       package require oo::dialect
22
23       package require tool
24
25       package require coroutine
26
27       package require fileutil
28
29       package require fileutil::magic::filetype
30
31       package require websocket
32
33       package require mime
34
35       package require cron
36
37       package require uri
38
39       package require Markdown
40
41       constructor   ?port   ?port??   ?myaddr   ?ipaddr?|all?  ?server_string
42       ?string?? ?server_name ?string??
43
44       method add_uri pattern dict
45
46       method connect sock ip port
47
48       method Connect uuid sock ip
49
50       method counter which
51
52       method CheckTimeout
53
54       method dispatch header_dict
55
56       method log args
57
58       method port_listening
59
60       method PrefixNormalize prefix
61
62       method start
63
64       method stop
65
66       method template page
67
68       method TemplateSearch page
69
70       method Validate_Connection sock ip
71
72       method ENSEMBLE::add field element
73
74       method ENSEMBLE::dump
75
76       method ENSEMBLE::get field
77
78       method ENSEMBLE::reset
79
80       method ENSEMBLE::remove field element
81
82       method ENSEMBLE::replace keyvaluelist
83
84       method ENSEMBLE::reset
85
86       method ENSEMBLE::set field value
87
88       method http_info::netstring
89
90       method request::parse string
91
92       method reply::output
93
94       method close
95
96       method HttpHeaders sock ?debug?
97
98       method dispatch newsock datastate
99
100       method error code ?message? ?errorInfo?
101
102       method content
103
104       method EncodeStatus status
105
106       method FormData
107
108       method MimeParse mimetext
109
110       method output
111
112       method DoOutput
113
114       method PostData length
115
116       method puts string
117
118       method reset
119
120       method timeOutCheck
121
122       method timestamp
123
124       method TransferComplete args
125
126       method Url_Decode string
127
128       method cgi_info
129
130       option path
131
132       option prefix
133
134       method proxy_info
135
136       method scgi_info
137
138______________________________________________________________________________
139

DESCRIPTION

141       This module implements a web  server,  suitable  for  embedding  in  an
142       application.  The  server  is  object oriented, and contains all of the
143       fundamentals needed for a full service website.
144

MINIMAL EXAMPLE

146       Starting a web service requires starting a class of type httpd::server,
147       and  providing  that  server  with  one  or  more  URIs to service, and
148       httpd::reply derived classes to generate them.
149
150
151              tool::define ::reply.hello {
152                method content {} {
153                  my puts "<HTML><HEAD><TITLE>IRM Dispatch Server</TITLE></HEAD><BODY>"
154                  my puts "<h1>Hello World!</h1>"
155                  my puts </BODY></HTML>
156                }
157              }
158              ::docserver::server create HTTPD port 8015 myaddr 127.0.0.1
159              HTTPD add_uri /* [list mixin reply.hello]
160
161

CLASS ::HTTPD::SERVER

163       This class is the root object of the webserver. It is  responsible  for
164       opening the socket and providing the initial connection negotiation.
165
166       constructor   ?port   ?port??   ?myaddr   ?ipaddr?|all?  ?server_string
167       ?string?? ?server_name ?string??
168              Build a new server object. ?port? is the port to listen on
169
170       method add_uri pattern dict
171              Set the hander for a URI pattern. Information given in the  dict
172              is  stored  in the data structure the dispatch method uses. If a
173              field called mixin is given, that class will be mixed  into  the
174              reply object immediately after construction.
175
176       method connect sock ip port
177              Reply  to an open socket. This method builds a coroutine to man‐
178              age the remainder of the connection. The coroutine's  operations
179              are driven by the Connect method.
180
181       method Connect uuid sock ip
182              This  method  reads HTTP headers, and then consults the dispatch
183              method to determine if the request is valid, and/or what kind of
184              reply  to  generate.  Under  normal  cases,  an  object of class
185              ::http::reply is created.  Fields the server are looking for  in
186              particular  are:  class:  A class to use instead of the server's
187              own reply_class mixin: A class to be mixed into the  new  object
188              after  construction.   All  other fields are passed along to the
189              http_info structure of the reply object.   After  the  class  is
190              created  and the mixin is mixed in, the server invokes the reply
191              objects dispatch method.  This  action  passes  control  of  the
192              socket to the reply object. The reply object manages the rest of
193              the transaction, including closing the socket.
194
195       method counter which
196              Increment an internal counter.
197
198       method CheckTimeout
199              Check open connections for a time out event.
200
201       method dispatch header_dict
202              Given a key/value list of information, return a  data  structure
203              describing how the server should reply.
204
205       method log args
206              Log  an  event.  The input for args is free form. This method is
207              intended to be replaced by the user, and is a noop for  a  stock
208              http::server object.
209
210       method port_listening
211              Return the actual port that httpd is listening on.
212
213       method PrefixNormalize prefix
214              For  the stock version, trim trailing /'s and *'s from a prefix.
215              This method can be replaced by the end user to perform any other
216              transformations needed for the application.
217
218       method start
219              Open the socket listener.
220
221       method stop
222              Shut off the socket listener, and destroy any pending replies.
223
224       method template page
225              Return a template for the string page
226
227       method TemplateSearch page
228              Perform  a  search for the template that best matches page. This
229              can include local file searches, in-memory structures,  or  even
230              database  lookups.  The  stock  implementation  simply looks for
231              files with a .tml or .html extension in  the  ?doc_root?  direc‐
232              tory.
233
234       method Validate_Connection sock ip
235              Given a socket and an ip address, return true if this connection
236              should be terminated, or false if it should be allowed  to  con‐
237              tinue.  The  stock  implementation  always  returns  0.  This is
238              intended for applications to be able to  implement  black  lists
239              and/or provide security based on IP address.
240

CLASS ::HTTPD::REPLY

242       A  class  which shephards a request through the process of generating a
243       reply.  The socket associated with the reply is available at all  times
244       as the chan variable.  The process of generating a reply begins with an
245       httpd::server generating a http::class  object,  mixing  in  a  set  of
246       behaviors  and  then  invoking  the reply object's dispatch method.  In
247       normal operations the dispatch method:
248
249       [1]    Invokes the reset method for  the  object  to  populate  default
250              headers.
251
252       [2]    Invokes the HttpHeaders method to stream the MIME headers out of
253              the socket
254
255       [3]    Invokes the request parse method to convert the stream  of  MIME
256              headers into a dict that can be read via the request method.
257
258       [4]    Stores the raw stream of MIME headers in the rawrequest variable
259              of the object.
260
261       [5]    Invokes the content method for the object, generating an call to
262              the error method if an exception is raised.
263
264       [6]    Invokes the output method for the object
265

REPLY METHOD ENSEMBLES

267       The http::reply class and its derivatives maintain several variables as
268       dictionaries  internally.  Access  to  these  dictionaries  is  managed
269       through  a dedicated ensemble. The ensemble implements most of the same
270       behaviors as the dict command.  Each ensemble implements the  following
271       methods above, beyond, or modifying standard dicts:
272
273       method ENSEMBLE::add field element
274              Add  element  to  a  list stored in field, but only if it is not
275              already present om the list.
276
277       method ENSEMBLE::dump
278              Return the current contents of the data structure as a key/value
279              list.
280
281       method ENSEMBLE::get field
282              Return  the  value  of the field field, or an empty string if it
283              does not exist.
284
285       method ENSEMBLE::reset
286              Return a key/value list of the default contents  for  this  data
287              structure.
288
289       method ENSEMBLE::remove field element
290              Remove all instances of element from the list stored in field.
291
292       method ENSEMBLE::replace keyvaluelist
293              Replace the internal dict with the contents of keyvaluelist
294
295       method ENSEMBLE::reset
296              Replace the internal dict with the default state.
297
298       method ENSEMBLE::set field value
299              Set the value of field to value.
300

REPLY METHOD ENSEMBLE: HTTP_INFO

302       Manages HTTP headers passed in by the server.  Ensemble Methods:
303
304       method http_info::netstring
305              Return  the  contents  of  this  data  structure  as a netstring
306              encoded block.
307

REPLY METHOD ENSEMBLE: REQUEST

309       Managed data from MIME headers of the request.
310
311       method request::parse string
312              Replace the contents of  the  data  structure  with  information
313              encoded in a MIME formatted block of text (string).
314

REPLY METHOD ENSEMBLE: REPLY

316       Manage the headers sent in the reply.
317
318       method reply::output
319              Return  the  contents  of  this data structure as a MIME encoded
320              block appropriate for an HTTP response.
321

REPLY METHODS

323       method close
324              Terminate the transaction, and close the socket.
325
326       method HttpHeaders sock ?debug?
327              Stream MIME headers from the socket sock, stopping at  an  empty
328              line. Returns the stream as a block of text.
329
330       method dispatch newsock datastate
331              Take  over  control of the socket newsock, and store that as the
332              chan variable for the object. This method runs  through  all  of
333              the steps of reading HTTP headers, generating content, and clos‐
334              ing the connection. (See class writetup).
335
336       method error code ?message? ?errorInfo?
337              Generate an error message of the specified code, and display the
338              message  as the reason for the exception. errorInfo is passed in
339              from calls, but how or if it should be displayed is  a  preroga‐
340              tive of the developer.
341
342       method content
343              Generate  the  content for the reply. This method is intended to
344              be replaced by the mixin.  Developers have the option of stream‐
345              ing output to a buffer via the puts method of the reply, or sim‐
346              ply populating the  reply_body  variable  of  the  object.   The
347              information returned by the content method is not interpreted in
348              any way.  If an exception is thrown (via the  error  command  in
349              Tcl,  for example) the caller will auto-generate a 505 {Internal
350              Error} message.  A typical implementation of content look like:
351
352
353
354              tool::define ::test::content.file {
355                superclass ::httpd::content.file
356                # Return a file
357                # Note: this is using the content.file mixin which looks for the reply_file variable
358                # and will auto-compute the Content-Type
359                method content {} {
360                  my reset
361                  set doc_root [my http_info get doc_root]
362                  my variable reply_file
363                  set reply_file [file join $doc_root index.html]
364                }
365              }
366              tool::define ::test::content.time {
367                # return the current system time
368                method content {} {
369                   my variable reply_body
370                  my reply set Content-Type text/plain
371                   set reply_body [clock seconds]
372                }
373              }
374              tool::define ::test::content.echo {
375                method content {} {
376                   my variable reply_body
377                  my reply set Content-Type [my request get Content-Type]
378                   set reply_body [my PostData [my request get Content-Length]]
379                }
380              }
381              tool::define ::test::content.form_handler {
382                method content {} {
383                  set form [my FormData]
384                  my reply set Content-Type {text/html; charset=UTF-8}
385                  my puts "<HTML><HEADER><TITLE>My Dynamic Page</TITLE></HEADER>"
386                  my puts "<BODY>"
387                  my puts "You Sent<p>"
388                  my puts "<TABLE>"
389                  foreach {f v} $form {
390                    my puts "<TR><TH>$f</TH><TD><verbatim>$v</verbatim></TD>"
391                  }
392                  my puts "</TABLE><p>"
393                  my puts "Send some info:<p>"
394                  my puts "<FORM action=/[my http_info get REQUEST_PATH] method POST>"
395                  my puts "<TABLE>"
396                  foreach field {name rank serial_number} {
397                    set line "<TR><TH>$field</TH><TD><input name=\"$field\" "
398                    if {[dict exists $form $field]} {
399                      append line " value=\"[dict get $form $field]\"""
400                    }
401                    append line " /></TD></TR>"
402                    my puts $line
403                  }
404                  my puts "</TABLE>"
405                  my puts "</BODY></HTML>"
406                }
407              }
408
409
410
411       method EncodeStatus status
412              Formulate a standard HTTP status header from he string provided.
413
414       method FormData
415              For  GET  requests,  converts  the  QUERY_DATA  header  into   a
416              key/value list.  For POST requests, reads the Post data and con‐
417              verts that information to a key/value  list  for  application/x-
418              www-form-urlencoded  posts.  For  multipart posts, it composites
419              all of the MIME headers of the  post  to  a  singular  key/value
420              list,  and  provides  MIME_* information as computed by the mime
421              package, including the MIME_TOKEN, which can be  fed  back  into
422              the mime package to read out the contents.
423
424       method MimeParse mimetext
425              Converts a block of mime encoded text to a key/value list. If an
426              exception is encountered, the method will generate its own  call
427              to the error method, and immediately invoke the output method to
428              produce an error code and close the connection.
429
430       method output
431              Schedules a call to DoOutput when chan becomes writeable
432
433       method DoOutput
434              Generates the the HTTP reply, and streams that reply back across
435              chan.
436
437       method PostData length
438              Stream  length  bytes  from  the  chan  socket,  but only of the
439              request is a POST or PUSH. Returns an empty string otherwise.
440
441       method puts string
442              Appends the value of string to the end of reply_body, as well as
443              a trailing newline character.
444
445       method reset
446              Clear  the  contents  of  the reply_body variable, and reset all
447              headers in the reply structure back to  the  defaults  for  this
448              object.
449
450       method timeOutCheck
451              Called  from  the  http::server object which spawned this reply.
452              Checks to see if too much time has  elapsed  while  waiting  for
453              data  or  generating  a reply, and issues a timeout error to the
454              request if it has, as well as destroy the object and  close  the
455              chan socket.
456
457       method timestamp
458              Return the current system time in the format:
459
460              %a, %d %b %Y %T %Z
461
462       method TransferComplete args
463              Intended to be invoked from chan copy as a callback. This closes
464              every channel fed to it on the command line, and  then  destroys
465              the object.
466
467
468                  ###
469                  # Output the body
470                  ###
471                  chan configure $sock -translation binary -blocking 0 -buffering full -buffersize 4096
472                  chan configure $chan -translation binary -blocking 0 -buffering full -buffersize 4096
473                  if {$length} {
474                    ###
475                    # Send any POST/PUT/etc content
476                    ###
477                    chan copy $sock $chan -command [namespace code [list my TransferComplete $sock]]
478                  } else {
479                    catch {close $sock}
480                    chan flush $chan
481                    my destroy
482                  }
483
484
485       method Url_Decode string
486              De-httpizes a string.
487

CLASS ::HTTPD::CONTENT

489       The  httpd module includes several ready to use implementations of con‐
490       tent mixins for common use cases. Options are passed in to the  add_uri
491       method of the server.
492

CLASS ::HTTPD::CONTENT.CGI

494       An  implementation  to relay requests to process which will accept post
495       data streamed in vie stdin, and sent a reply streamed to stdout.
496
497       method cgi_info
498              Mandatory method to be replaced by  the  end  user.  If  needed,
499              activates the process to proxy, and then returns a list of three
500              values: exec - The arguments to send to exec  to  fire  off  the
501              responding process, minus the stdin/stdout redirection.
502

CLASS ::HTTPD::CONTENT.FILE

504       An implementation to deliver files from the local file system.
505
506       option path
507              The  root  directory  on the local file system to be exposed via
508              http.
509
510       option prefix
511              The prefix of the URI portion to ignore when  calculating  rela‐
512              tive file paths.
513

CLASS ::HTTPD::CONTENT.PROXY

515       An  implementation  to relay requests to another HTTP server, and relay
516       the results back across the request channel.
517
518       method proxy_info
519              Mandatory method to be replaced by  the  end  user.  If  needed,
520              activates the process to proxy, and then returns a list of three
521              values: proxyhost - The hostname  where  the  proxy  is  located
522              proxyport  -  The  port  to connect to proxyscript - A pre-amble
523              block of text to send prior to the mirrored request
524

CLASS ::HTTPD::CONTENT.SCGI

526       An implementation to relay requests to a server listening on  a  socket
527       expecting  SCGI encoded requests, and relay the results back across the
528       request channel.
529
530       method scgi_info
531              Mandatory method to be replaced by  the  end  user.  If  needed,
532              activates the process to proxy, and then returns a list of three
533              values: scgihost - The  hostname  where  the  scgi  listener  is
534              located  scgiport - The port to connect to scgiscript - The con‐
535              tents of the SCRIPT_NAME header to be sent
536

CLASS ::HTTPD::CONTENT.WEBSOCKET

538       A placeholder for a future implementation to manage requests  that  can
539       expect to be promoted to a Websocket. Currently it is an empty class.
540

SCGI SERVER FUNCTIONS

542       The HTTP module also provides an SCGI server implementation, as well as
543       an HTTP implementation. To use the SCGI functions, create an object  of
544       the http::server.scgi class instead of the http::server class.
545

CLASS ::HTTPD::REPLY.SCGI

547       An  modified  http::reply  implementation  that understands how to deal
548       with netstring encoded headers.
549

CLASS ::HTTPD::SERVER.SCGI

551       A modified http::server  which  is  tailored  to  replying  to  request
552       according to the SCGI standard instead of the HTTP standard.
553

AUTHORS

555       Sean Woods
556

BUGS, IDEAS, FEEDBACK

558       This  document,  and the package it describes, will undoubtedly contain
559       bugs and other problems.  Please report such in the category network of
560       the  Tcllib  Trackers  [http://core.tcl.tk/tcllib/reportlist].   Please
561       also report any ideas for enhancements you may have for either  package
562       and/or documentation.
563
564       When proposing code changes, please provide unified diffs, i.e the out‐
565       put of diff -u.
566
567       Note further that  attachments  are  strongly  preferred  over  inlined
568       patches.  Attachments  can  be  made  by  going to the Edit form of the
569       ticket immediately after its creation, and  then  using  the  left-most
570       button in the secondary navigation bar.
571

KEYWORDS

573       TclOO, WWW, http, httpd, httpserver, services
574

CATEGORY

576       Networking
577
579       Copyright (c) 2018 Sean Woods <yoda@etoyoc.com>
580
581
582
583
584tcllib                               4.1.1                             tool(n)
Impressum