1rest(n)              A framework for RESTful web services              rest(n)
2
3
4
5______________________________________________________________________________
6

NAME

8       rest - define REST web APIs and call them inline or asychronously
9

SYNOPSIS

11       package require Tcl  8.5
12
13       package require rest  ?1.3.1?
14
15       ::rest::simple url query ?config? ?body?
16
17       ::rest::get url query ?config? ?body?
18
19       ::rest::post url query ?config? ?body?
20
21       ::rest::patch url query ?config? ?body?
22
23       ::rest::head url query ?config? ?body?
24
25       ::rest::put url query ?config? ?body?
26
27       ::rest::delete url query ?config? ?body?
28
29       ::rest::save name file
30
31       ::rest::describe name
32
33       ::rest::parameters url ?key?
34
35       ::rest::parse_opts static required optional words
36
37       ::rest::substitute string var
38
39       ::rest::create_interface name
40
41______________________________________________________________________________
42

DESCRIPTION

44       There  are  two types of usage this package supports: simple calls, and
45       complete interfaces.  In an interface you specify a set  of  rules  and
46       then the package builds the commands which correspond to the REST meth‐
47       ods. These commands can have many options  such  as  input  and  output
48       transformations  and  data  type specific formatting. This results in a
49       cleaner and simpler script.  On the other hand, while a simple call  is
50       easier  and  quicker  to implement it is also less featureful. It takes
51       the url and a few options about the  command  and  returns  the  result
52       directly. Any formatting or checking is up to rest of the script.
53

SIMPLE USAGE

55       In  simple  usage  you  make calls using the http method procedures and
56       then check or process the returned data yourself
57
58       ::rest::simple url query ?config? ?body?
59
60       ::rest::get url query ?config? ?body?
61
62       ::rest::post url query ?config? ?body?
63
64       ::rest::patch url query ?config? ?body?
65
66       ::rest::head url query ?config? ?body?
67
68       ::rest::put url query ?config? ?body?
69
70       ::rest::delete url query ?config? ?body?
71
72              These commands are all equivalent except  for  the  http  method
73              used.   If you use simple then the method should be specified as
74              an option in the config dictionary.  If  that  is  not  done  it
75              defaults  to get. If a body is needed then the config dictionary
76              must be present, however it is allowed to be empty.
77
78              The config dictionary supports the following keys
79
80              auth
81
82              content-type
83
84              cookie
85
86              error-body
87
88              format
89
90              headers
91
92              method
93
94       Two quick examples:
95
96       Example 1, Yahoo Boss:
97
98
99                  set appid APPID
100                  set search tcl
101                  set res [rest::get http://boss.yahooapis.com/ysearch/web/v1/$search [list appid $appid]]
102                  set res [rest::format_json $res]
103
104
105       Example 2, Twitter:
106
107
108                  set url   http://twitter.com/statuses/update.json
109                  set query [list status $text]
110                  set res [rest::simple $url $query {
111                      method post
112                      auth   {basic user password}
113                      format json
114                  }]
115
116

INTERFACE USAGE

118       An interface to a REST API consists of a series of definitions of  REST
119       calls  contained  in an array.  The name of that array becomes a names‐
120       pace containing the defined commands. Each key of the  array  specifies
121       the  name  of the call, with the associated configuration a dictionary,
122       i.e. key/value pairs.  The acceptable keys,  i.e.  legal  configuration
123       options  are  described  below.   After creating the definitions in the
124       array simply calling rest::create_interface with the array as  argument
125       will then create the desired commands.
126
127       Example, Yahoo Weather:
128
129
130                  package require rest
131
132                  set yweather(forecast) {
133                     url      http://weather.yahooapis.com/forecastrss
134                     req_args { p: }
135                     opt_args { u: }
136                  }
137                  rest::create_interface yweather
138                  puts [yweather::forecast -p 94089]
139
140
141       ::rest::save name file
142              This  command saves a copy of the dynamically created procedures
143              for all the API calls specified in the array  variable  name  to
144              the file, for later loading.
145
146              The result of the command is the empty string
147
148       ::rest::describe name
149              This  command prints a description of all API calls specified in
150              the array variable name to the channel stdout.
151
152              The result of the command is the empty string.
153
154       ::rest::parameters url ?key?
155              This command parses an url query string into  a  dictionary  and
156              returns said dictionary as its result.
157
158              If  key is specified the command will not return the entire dic‐
159              tionary, but only the value of that key.
160
161       ::rest::parse_opts static required optional words
162              This command implements a custom parserfor command options.
163
164              dict static
165                     A dictionary of options and their values that are  always
166                     present in the output.
167
168              list required
169                     A list of options that must be supplied by words
170
171              list optional
172                     A  list  of options that may appear in the words, but are
173                     not required.  The elements  must  be  in  one  of  three
174                     forms:
175
176                     name   The option may be present or not, no default.
177
178                     name:  When present the option requires an argument.
179
180                     name:value
181                            When not present use value as default.
182
183              list words
184                     The words to parse into options and values.
185
186       The result of the command is a list containing two elements.  The first
187       element is a dictionary containing the parsed options and their values.
188       The second element is a list of the remaining words.
189
190       ::rest::substitute string var
191              This  command  takes a string, substitutes values for any option
192              identifiers found inside and returns the modified string as  its
193              results.
194
195              The values to substitute are found in the variable var, which is
196              expected to contain a dictionary mapping from the option identi‐
197              fiers  to replace to their values.  Note that option identifiers
198              which have no key in var are replaced with the empty string.
199
200              The option identifiers in string have to follow the syntax %...%
201              where ... may contain any combination of lower-case alphanumeric
202              characters, plus underscore, colon and dash.
203
204       ::rest::create_interface name
205              This command creates procedures for all the API calls  specified
206              in the array variable name.
207
208              The  name  of  that  array  becomes  a  namespace containing the
209              defined commands. Each key of the array specifies  the  name  of
210              the  call,  with the associated configuration a dictionary, i.e.
211              key/value pairs.  The legal keys and their meanings are:
212
213              url    The value of this required option must be the  target  of
214                     the http request.
215
216              description
217                     The  value of this option must be a short string describ‐
218                     ing the call.  Default to the empty string, if not speci‐
219                     fied.  Used only by ::rest::describe.
220
221              body   The  value  of  this  option  indicates  if arguments are
222                     required for the call's request body or not. The  accept‐
223                     able  values  are  listed below.  Defaults to optional if
224                     not specified.
225
226                     none   The call has no request body, none  must  be  sup‐
227                            plied.
228
229                     optional
230                            A  request  body  can  be  supplied,  but  is  not
231                            required.
232
233                     required
234                            A request body must be supplied.
235
236                     argument
237                            This value must be followed  by  the  name  of  an
238                            option,  treating the entire string as a list. The
239                            request body will be used as  the  value  of  that
240                            option.
241
242                     mime_multipart
243                            A request body must be supplied and will be inter‐
244                            preted as each argument representing one part of a
245                            mime/multipart  document.  Arguments must be lists
246                            containing 2 elements, a list of header  keys  and
247                            values, and the mime part body, in this order.
248
249              method The  value  of  this  option must be the name of the HTTP
250                     method to call on the url.  Defaults to GET, if not spec‐
251                     ified.   The  acceptable  values  are GET, POST, and PUT,
252                     regardless of letter-case.
253
254              copy   When present the value of this option specifies the  name
255                     of a previously defined call. The definition of that call
256                     is copied to the current call,  except  for  the  options
257                     specified by the current call itself.
258
259              unset  When  present the value of this option contains a list of
260                     options in the current call. These  options  are  removed
261                     from  the  definition. Use this after copying an existing
262                     definition to remove options, instead of overriding their
263                     value.
264
265              headers
266                     Specification  of  additional header fields. The value of
267                     this option must be a dictionary, interpreted to  contain
268                     the new header fields and their values. The default is to
269                     not add any additional headers.
270
271              content-type
272                     The value of this option specifies the content  type  for
273                     the request data.
274
275              req_args
276                     The  value  of  this option is a list naming the required
277                     arguments of the call.  Names  ending  in  a  colon  will
278                     require a value.
279
280              opt_args
281                     The value of this option a list naming the arguments that
282                     may be present for a call but are not required.
283
284              static_args
285                     The value of this option a list naming the arguments that
286                     are always the same.  No sense in troubling the user with
287                     these.  A leading dash (-) is allowed but not required to
288                     maintain consistency with the command line.
289
290              auth   The  value  of  this option specifies how to authenticate
291                     the calls.  No authentication is done if  the  option  is
292                     not specified.
293
294                     basic  The user may configure the basic authentication by
295                            overriding the procedure basic_auth in the  names‐
296                            pace  of interface. This procedure takes two argu‐
297                            ments, the username and password, in this order.
298
299                     sign   The value must actually be a list with the  second
300                            element  the  name  of  a  procedure which will be
301                            called to perform request signing.
302
303              callback
304                     If this option is present then the method will be created
305                     as an async call. Such calls will return immediately with
306                     the value of the associated http  token  instead  of  the
307                     call's  result. The event loop must be active to use this
308                     option.
309
310                     The value of this option is treated as a  command  prefix
311                     which is invoked when the HTTP call is complete. The pre‐
312                     fix will receive at least two additional  arguments,  the
313                     name  of  the  calling  procedure  and  the status of the
314                     result (one of OK or ERROR), in this order.
315
316                     In case of OK a third argument is added, the data associ‐
317                     ated with the result.
318
319                     If  and  only if the ERROR is a redirection, the location
320                     redirected to will be added as argument.  Further, if the
321                     configuration  key  error-body  is  set  to true the data
322                     associated with the result will be added as  argument  as
323                     well.
324
325                     The  http request header will be available in that proce‐
326                     dure via upvar token token.
327
328              cookie The value of this option is  a  list  of  cookies  to  be
329                     passed  in  the  http  header.  This is a shortcut to the
330                     headers option.
331
332              input_transform
333                     The value of this option is a command prefix or script to
334                     perform a transformation on the query before invoking the
335                     call. A script transform is wrapped into an automatically
336                     generated internal procedure.
337
338                     If not specified no transformation is done.
339
340                     The  command  (prefix) must accept a single argument, the
341                     query (a dictionary) to transform, and  must  return  the
342                     modified  query (again as dictionary) as its result.  The
343                     request body is accessible in the transform  command  via
344                     upvar body body.
345
346              format
347
348              result The  value  of  this  option  specifies the format of the
349                     returned data.  Defaults to auto if not  specified.   The
350                     acceptable values are:
351
352                     auto   Auto detect between xml and json.
353
354                     discard
355
356                     json
357
358                     raw
359
360                     rss    This is formatted as a special case of xml.
361
362                     tdom
363
364                     xml
365
366              pre_transform
367                     The value of this option is a command prefix or script to
368                     perform a transformation on the result of a call  (before
369                     the application of the output transform as per format). A
370                     script transform is wrapped into an automatically  gener‐
371                     ated internal procedure.
372
373                     If not specified no transformation is done.
374
375                     The  command  (prefix) must accept a single argument, the
376                     result to transform, and must return the modified  result
377                     as its result.
378
379                     The  http  request  header is accessible in the transform
380                     command via upvar token token
381
382              post_transform
383                     The value of this option is a command prefix or script to
384                     perform  a  transformation on the result of a call (after
385                     the application of the output transform as per format). A
386                     script  transform is wrapped into an automatically gener‐
387                     ated internal procedure.
388
389                     If not specified no transformation is done.
390
391                     The command (prefix) must accept a single  argument,  the
392                     result  to transform, and must return the modified result
393                     as its result.
394
395                     The http request header is accessible  in  the  transform
396                     command via upvar token token
397
398              check_result
399                     The value of this option must be list of two expressions,
400                     either of which may be empty.
401
402                     The first expression is checks the OK condition, it  must
403                     return  true  when  the result is satisfactory, and false
404                     otherwise.
405
406                     The second expression is the  ERROR  condition,  it  must
407                     return  false  unless  there  is an error, then it has to
408                     return true.
409
410              error_body
411                     The value of this option determines whether to return the
412                     response  when  encountering  an  HTTP error, or not. The
413                     default is to not return the response body on error.
414
415                     See callback above for more information.
416

EXAMPLES

418       Yahoo Geo:
419
420
421              set ygeo(parse) {
422                  url http://wherein.yahooapis.com/v1/document
423                  method post
424                  body { arg documentContent }
425              }
426              ygeo::parse "san jose ca"
427              # "san jose ca" will be interpreted as if it were specified as the -documentContent option
428
429
430       Google Docs:
431
432
433              set gdocs(upload) {
434                  url http://docs.google.com/feeds/default/private/full
435                  body mime_multipart
436              }
437              gdocs::upload [list {Content-Type application/atom+xml} $xml] [list {Content-Type image/jpeg} $filedata]
438
439
440       Delicious:
441
442
443              set delicious(updated) {
444                  url https://api.del.icio.us/v1/posts/update
445                  auth basic
446              }
447
448              rest::create_interface flickr
449
450              flickr::basic_auth username password
451
452
453       Flickr:
454
455
456              set flickr(auth.getToken) {
457                 url http://api.flickr.com/services/rest/
458                 req_args { api_key: secret: }
459                 auth { sign do_signature }
460              }
461
462              rest::create_interface flickr
463
464              proc ::flickr::do_signature {query} {
465                  # perform some operations on the query here
466                  return $query
467              }
468
469

INCLUDED

471       The package provides functional but incomplete implementations for  the
472       following services:
473
474       del.icio.us
475
476       facebook
477
478       flickr
479
480       twitter
481
482       google calendar
483
484       yahoo boss
485
486       yahoo weather
487
488       Please  either read the package's implementation, or use rest::describe
489       after loading it for their details.
490
491       Do not forget developers' documentation on the respective sites either.
492

TLS

494       The rest package can be used with https-secured services, by  requiring
495       the  TLS  package  and  then registering it with the http package it is
496       sitting on top of.  Example
497
498
499                  package require tls
500                  http::register https 443 ::tls::socket
501
502

TLS SECURITY CONSIDERATIONS

504       This package uses the TLS package to handle the security for https urls
505       and other socket connections.
506
507       Policy  decisions like the set of protocols to support and what ciphers
508       to use are not the responsibility of TLS, nor of  this  package  itself
509       however.   Such  decisions are the responsibility of whichever applica‐
510       tion is using the package, and are likely  influenced  by  the  set  of
511       servers the application will talk to as well.
512
513       For  example,  in light of the recent POODLE attack [http://googleonli
514       nesecurity.blogspot.co.uk/2014/10/this-poodle-bites-exploiting-
515       ssl-30.html] discovered by Google many servers will disable support for
516       the SSLv3 protocol.  To handle this change the applications  using  TLS
517       must  be  patched,  and not this package, nor TLS itself.  Such a patch
518       may be as simple as generally activating tls1 support, as shown in  the
519       example below.
520
521
522                  package require tls
523                  tls::init -tls1 1 ;# forcibly activate support for the TLS1 protocol
524
525                  ... your own application code ...
526
527

BUGS, IDEAS, FEEDBACK

529       This  document,  and the package it describes, will undoubtedly contain
530       bugs and other problems.  Please report such in the  category  rest  of
531       the  Tcllib  Trackers  [http://core.tcl.tk/tcllib/reportlist].   Please
532       also report any ideas for enhancements you may have for either  package
533       and/or documentation.
534
535       When proposing code changes, please provide unified diffs, i.e the out‐
536       put of diff -u.
537
538       Note further that  attachments  are  strongly  preferred  over  inlined
539       patches.  Attachments  can  be  made  by  going to the Edit form of the
540       ticket immediately after its creation, and  then  using  the  left-most
541       button in the secondary navigation bar.
542
543
544
545tcllib                               1.3.1                             rest(n)
Impressum