1tool(n) Tcl Web Server tool(n)
2______________________________________________________________________________
6
8 tool - A TclOO and coroutine based web server
9
11 package require Tcl 8.6
12 package require httpd ?4.1.1?
14 package require sha1
16 package require dicttool
18 package require oo::meta
20 package require oo::dialect
22 package require tool
24 package require coroutine
26 package require fileutil
28 package require fileutil::magic::filetype
30 package require websocket
32 package require mime
34 package require cron
36 package require uri
38 package require Markdown
40 constructor ?port ?port?? ?myaddr ?ipaddr?|all? ?server_string
42 ?string?? ?server_name ?string??
43 method add_uri pattern dict
45 method connect sock ip port
47 method Connect uuid sock ip
49 method counter which
51 method CheckTimeout
53 method dispatch header_dict
55 method log args
57 method port_listening
59 method PrefixNormalize prefix
61 method start
63 method stop
65 method template page
67 method TemplateSearch page
69 method Validate_Connection sock ip
71 method ENSEMBLE::add field element
73 method ENSEMBLE::dump
75 method ENSEMBLE::get field
77 method ENSEMBLE::reset
79 method ENSEMBLE::remove field element
81 method ENSEMBLE::replace keyvaluelist
83 method ENSEMBLE::reset
85 method ENSEMBLE::set field value
87 method http_info::netstring
89 method request::parse string
91 method reply::output
93 method close
95 method HttpHeaders sock ?debug?
97 method dispatch newsock datastate
99 method error code ?message? ?errorInfo?
101 method content
103 method EncodeStatus status
105 method FormData
107 method MimeParse mimetext
109 method output
111 method DoOutput
113 method PostData length
115 method puts string
117 method reset
119 method timeOutCheck
121 method timestamp
123 method TransferComplete args
125 method Url_Decode string
127 method cgi_info
129 option path
131 option prefix
133 method proxy_info
135 method scgi_info
137______________________________________________________________________________
139
141 This module implements a web server, suitable for embedding in an
142 application. The server is object oriented, and contains all of the
143 fundamentals needed for a full service website.
144
146 Starting a web service requires starting a class of type httpd::server,
147 and providing that server with one or more URIs to service, and
148 httpd::reply derived classes to generate them.
149 tool::define ::reply.hello {
152 method content {} {
153 my puts "<HTML><HEAD><TITLE>IRM Dispatch Server</TITLE></HEAD><BODY>"
154 my puts "<h1>Hello World!</h1>"
155 my puts </BODY></HTML>
156 }
157 }
158 ::docserver::server create HTTPD port 8015 myaddr 127.0.0.1
159 HTTPD add_uri /* [list mixin reply.hello]
160
163 This class is the root object of the webserver. It is responsible for
164 opening the socket and providing the initial connection negotiation.
165 constructor ?port ?port?? ?myaddr ?ipaddr?|all? ?server_string
167 ?string?? ?server_name ?string??
168 Build a new server object. ?port? is the port to listen on
169 method add_uri pattern dict
171 Set the hander for a URI pattern. Information given in the dict
172 is stored in the data structure the dispatch method uses. If a
173 field called mixin is given, that class will be mixed into the
174 reply object immediately after construction.
175 method connect sock ip port
177 Reply to an open socket. This method builds a coroutine to man‐
178 age the remainder of the connection. The coroutine's operations
179 are driven by the Connect method.
180 method Connect uuid sock ip
182 This method reads HTTP headers, and then consults the dispatch
183 method to determine if the request is valid, and/or what kind of
184 reply to generate. Under normal cases, an object of class
185 ::http::reply is created. Fields the server are looking for in
186 particular are: class: A class to use instead of the server's
187 own reply_class mixin: A class to be mixed into the new object
188 after construction. All other fields are passed along to the
189 http_info structure of the reply object. After the class is
190 created and the mixin is mixed in, the server invokes the reply
191 objects dispatch method. This action passes control of the
192 socket to the reply object. The reply object manages the rest of
193 the transaction, including closing the socket.
194 method counter which
196 Increment an internal counter.
197 method CheckTimeout
199 Check open connections for a time out event.
200 method dispatch header_dict
202 Given a key/value list of information, return a data structure
203 describing how the server should reply.
204 method log args
206 Log an event. The input for args is free form. This method is
207 intended to be replaced by the user, and is a noop for a stock
208 http::server object.
209 method port_listening
211 Return the actual port that httpd is listening on.
212 method PrefixNormalize prefix
214 For the stock version, trim trailing /'s and *'s from a prefix.
215 This method can be replaced by the end user to perform any other
216 transformations needed for the application.
217 method start
219 Open the socket listener.
220 method stop
222 Shut off the socket listener, and destroy any pending replies.
223 method template page
225 Return a template for the string page
226 method TemplateSearch page
228 Perform a search for the template that best matches page. This
229 can include local file searches, in-memory structures, or even
230 database lookups. The stock implementation simply looks for
231 files with a .tml or .html extension in the ?doc_root? direc‐
232 tory.
233 method Validate_Connection sock ip
235 Given a socket and an ip address, return true if this connection
236 should be terminated, or false if it should be allowed to con‐
237 tinue. The stock implementation always returns 0. This is
238 intended for applications to be able to implement black lists
239 and/or provide security based on IP address.
240
242 A class which shephards a request through the process of generating a
243 reply. The socket associated with the reply is available at all times
244 as the chan variable. The process of generating a reply begins with an
245 httpd::server generating a http::class object, mixing in a set of
246 behaviors and then invoking the reply object's dispatch method. In
247 normal operations the dispatch method:
248 [1] Invokes the reset method for the object to populate default
250 headers.
251 [2] Invokes the HttpHeaders method to stream the MIME headers out of
253 the socket
254 [3] Invokes the request parse method to convert the stream of MIME
256 headers into a dict that can be read via the request method.
257 [4] Stores the raw stream of MIME headers in the rawrequest variable
259 of the object.
260 [5] Invokes the content method for the object, generating an call to
262 the error method if an exception is raised.
263 [6] Invokes the output method for the object
265
267 The http::reply class and its derivatives maintain several variables as
268 dictionaries internally. Access to these dictionaries is managed
269 through a dedicated ensemble. The ensemble implements most of the same
270 behaviors as the dict command. Each ensemble implements the following
271 methods above, beyond, or modifying standard dicts:
272 method ENSEMBLE::add field element
274 Add element to a list stored in field, but only if it is not
275 already present om the list.
276 method ENSEMBLE::dump
278 Return the current contents of the data structure as a key/value
279 list.
280 method ENSEMBLE::get field
282 Return the value of the field field, or an empty string if it
283 does not exist.
284 method ENSEMBLE::reset
286 Return a key/value list of the default contents for this data
287 structure.
288 method ENSEMBLE::remove field element
290 Remove all instances of element from the list stored in field.
291 method ENSEMBLE::replace keyvaluelist
293 Replace the internal dict with the contents of keyvaluelist
294 method ENSEMBLE::reset
296 Replace the internal dict with the default state.
297 method ENSEMBLE::set field value
299 Set the value of field to value.
300
302 Manages HTTP headers passed in by the server. Ensemble Methods:
303 method http_info::netstring
305 Return the contents of this data structure as a netstring
306 encoded block.
307
309 Managed data from MIME headers of the request.
310 method request::parse string
312 Replace the contents of the data structure with information
313 encoded in a MIME formatted block of text (string).
314
316 Manage the headers sent in the reply.
317 method reply::output
319 Return the contents of this data structure as a MIME encoded
320 block appropriate for an HTTP response.
321
323 method close
324 Terminate the transaction, and close the socket.
325 method HttpHeaders sock ?debug?
327 Stream MIME headers from the socket sock, stopping at an empty
328 line. Returns the stream as a block of text.
329 method dispatch newsock datastate
331 Take over control of the socket newsock, and store that as the
332 chan variable for the object. This method runs through all of
333 the steps of reading HTTP headers, generating content, and clos‐
334 ing the connection. (See class writetup).
335 method error code ?message? ?errorInfo?
337 Generate an error message of the specified code, and display the
338 message as the reason for the exception. errorInfo is passed in
339 from calls, but how or if it should be displayed is a preroga‐
340 tive of the developer.
341 method content
343 Generate the content for the reply. This method is intended to
344 be replaced by the mixin. Developers have the option of stream‐
345 ing output to a buffer via the puts method of the reply, or sim‐
346 ply populating the reply_body variable of the object. The
347 information returned by the content method is not interpreted in
348 any way. If an exception is thrown (via the error command in
349 Tcl, for example) the caller will auto-generate a 505 {Internal
350 Error} message. A typical implementation of content look like:
351 tool::define ::test::content.file {
355 superclass ::httpd::content.file
356 # Return a file
357 # Note: this is using the content.file mixin which looks for the reply_file variable
358 # and will auto-compute the Content-Type
359 method content {} {
360 my reset
361 set doc_root [my http_info get doc_root]
362 my variable reply_file
363 set reply_file [file join $doc_root index.html]
364 }
365 }
366 tool::define ::test::content.time {
367 # return the current system time
368 method content {} {
369 my variable reply_body
370 my reply set Content-Type text/plain
371 set reply_body [clock seconds]
372 }
373 }
374 tool::define ::test::content.echo {
375 method content {} {
376 my variable reply_body
377 my reply set Content-Type [my request get Content-Type]
378 set reply_body [my PostData [my request get Content-Length]]
379 }
380 }
381 tool::define ::test::content.form_handler {
382 method content {} {
383 set form [my FormData]
384 my reply set Content-Type {text/html; charset=UTF-8}
385 my puts "<HTML><HEADER><TITLE>My Dynamic Page</TITLE></HEADER>"
386 my puts "<BODY>"
387 my puts "You Sent<p>"
388 my puts "<TABLE>"
389 foreach {f v} $form {
390 my puts "<TR><TH>$f</TH><TD><verbatim>$v</verbatim></TD>"
391 }
392 my puts "</TABLE><p>"
393 my puts "Send some info:<p>"
394 my puts "<FORM action=/[my http_info get REQUEST_PATH] method POST>"
395 my puts "<TABLE>"
396 foreach field {name rank serial_number} {
397 set line "<TR><TH>$field</TH><TD><input name=\"$field\" "
398 if {[dict exists $form $field]} {
399 append line " value=\"[dict get $form $field]\"""
400 }
401 append line " /></TD></TR>"
402 my puts $line
403 }
404 my puts "</TABLE>"
405 my puts "</BODY></HTML>"
406 }
407 }
408 method EncodeStatus status
412 Formulate a standard HTTP status header from he string provided.
413 method FormData
415 For GET requests, converts the QUERY_DATA header into a
416 key/value list. For POST requests, reads the Post data and con‐
417 verts that information to a key/value list for application/x-
418 www-form-urlencoded posts. For multipart posts, it composites
419 all of the MIME headers of the post to a singular key/value
420 list, and provides MIME_* information as computed by the mime
421 package, including the MIME_TOKEN, which can be fed back into
422 the mime package to read out the contents.
423 method MimeParse mimetext
425 Converts a block of mime encoded text to a key/value list. If an
426 exception is encountered, the method will generate its own call
427 to the error method, and immediately invoke the output method to
428 produce an error code and close the connection.
429 method output
431 Schedules a call to DoOutput when chan becomes writeable
432 method DoOutput
434 Generates the the HTTP reply, and streams that reply back across
435 chan.
436 method PostData length
438 Stream length bytes from the chan socket, but only of the
439 request is a POST or PUSH. Returns an empty string otherwise.
440 method puts string
442 Appends the value of string to the end of reply_body, as well as
443 a trailing newline character.
444 method reset
446 Clear the contents of the reply_body variable, and reset all
447 headers in the reply structure back to the defaults for this
448 object.
449 method timeOutCheck
451 Called from the http::server object which spawned this reply.
452 Checks to see if too much time has elapsed while waiting for
453 data or generating a reply, and issues a timeout error to the
454 request if it has, as well as destroy the object and close the
455 chan socket.
456 method timestamp
458 Return the current system time in the format:
459 %a, %d %b %Y %T %Z
461 method TransferComplete args
463 Intended to be invoked from chan copy as a callback. This closes
464 every channel fed to it on the command line, and then destroys
465 the object.
466 ###
469 # Output the body
470 ###
471 chan configure $sock -translation binary -blocking 0 -buffering full -buffersize 4096
472 chan configure $chan -translation binary -blocking 0 -buffering full -buffersize 4096
473 if {$length} {
474 ###
475 # Send any POST/PUT/etc content
476 ###
477 chan copy $sock $chan -command [namespace code [list my TransferComplete $sock]]
478 } else {
479 catch {close $sock}
480 chan flush $chan
481 my destroy
482 }
483 method Url_Decode string
486 De-httpizes a string.
487
489 The httpd module includes several ready to use implementations of con‐
490 tent mixins for common use cases. Options are passed in to the add_uri
491 method of the server.
492
494 An implementation to relay requests to process which will accept post
495 data streamed in vie stdin, and sent a reply streamed to stdout.
496 method cgi_info
498 Mandatory method to be replaced by the end user. If needed,
499 activates the process to proxy, and then returns a list of three
500 values: exec - The arguments to send to exec to fire off the
501 responding process, minus the stdin/stdout redirection.
502
504 An implementation to deliver files from the local file system.
505 option path
507 The root directory on the local file system to be exposed via
508 http.
509 option prefix
511 The prefix of the URI portion to ignore when calculating rela‐
512 tive file paths.
513
515 An implementation to relay requests to another HTTP server, and relay
516 the results back across the request channel.
517 method proxy_info
519 Mandatory method to be replaced by the end user. If needed,
520 activates the process to proxy, and then returns a list of three
521 values: proxyhost - The hostname where the proxy is located
522 proxyport - The port to connect to proxyscript - A pre-amble
523 block of text to send prior to the mirrored request
524
526 An implementation to relay requests to a server listening on a socket
527 expecting SCGI encoded requests, and relay the results back across the
528 request channel.
529 method scgi_info
531 Mandatory method to be replaced by the end user. If needed,
532 activates the process to proxy, and then returns a list of three
533 values: scgihost - The hostname where the scgi listener is
534 located scgiport - The port to connect to scgiscript - The con‐
535 tents of the SCRIPT_NAME header to be sent
536
538 A placeholder for a future implementation to manage requests that can
539 expect to be promoted to a Websocket. Currently it is an empty class.
540
542 The HTTP module also provides an SCGI server implementation, as well as
543 an HTTP implementation. To use the SCGI functions, create an object of
544 the http::server.scgi class instead of the http::server class.
545
547 An modified http::reply implementation that understands how to deal
548 with netstring encoded headers.
549
551 A modified http::server which is tailored to replying to request
552 according to the SCGI standard instead of the HTTP standard.
553
555 Sean Woods
556
558 This document, and the package it describes, will undoubtedly contain
559 bugs and other problems. Please report such in the category network of
560 the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please
561 also report any ideas for enhancements you may have for either package
562 and/or documentation.
563 When proposing code changes, please provide unified diffs, i.e the out‐
565 put of diff -u.
566 Note further that attachments are strongly preferred over inlined
568 patches. Attachments can be made by going to the Edit form of the
569 ticket immediately after its creation, and then using the left-most
570 button in the secondary navigation bar.
571
573 TclOO, WWW, http, httpd, httpserver, services
574
576 Networking
577
579 Copyright (c) 2018 Sean Woods <yoda@etoyoc.com>
580tcllib 4.1.1 tool(n)