1doctools::idx::import(n)      Documentation tools     doctools::idx::import(n)
2
3
4
5______________________________________________________________________________
6

NAME

8       doctools::idx::import - Importing keyword indices
9

SYNOPSIS

11       package require doctools::idx::import  ?0.2?
12
13       package require Tcl  8.4
14
15       package require doctools::config
16
17       package require doctools::idx::structure
18
19       package require snit
20
21       package require pluginmgr
22
23       ::doctools::idx::import objectName
24
25       objectName method ?arg arg ...?
26
27       objectName destroy
28
29       objectName import text text ?format?
30
31       objectName import file path ?format?
32
33       objectName import object text object text ?format?
34
35       objectName import object file object path ?format?
36
37       objectName config names
38
39       objectName config get
40
41       objectName config set name ?value?
42
43       objectName config unset pattern...
44
45       objectName includes
46
47       objectName include add path
48
49       objectName include remove path
50
51       objectName include clear
52
53       IncludeFile currentfile path
54
55       import text configuration
56
57______________________________________________________________________________
58

DESCRIPTION

60       This  package  provides a class to manage the plugins for the import of
61       keyword indices from other formats, i.e.  their  conversion  from,  for
62       example docidx, json, etc.
63
64       This  is  one  of  the  three  public pillars the management of keyword
65       indices resides on. The other two pillars are
66
67       [1]    Exporting keyword indices, and
68
69       [2]    Holding keyword indices
70
71       For information about the Concepts of keyword indices, and their parts,
72       see  the  same-named section.  For information about the data structure
73       which is the major output of the manager objects provided by this pack‐
74       age see the section Keyword index serialization format.
75
76       The  plugin  system of our class is based on the package pluginmgr, and
77       configured to look for plugins using
78
79       [1]    the environment variable DOCTOOLS_IDX_IMPORT_PLUGINS,
80
81       [2]    the environment variable DOCTOOLS_IDX_PLUGINS,
82
83       [3]    the environment variable DOCTOOLS_PLUGINS,
84
85       [4]    the path "~/.doctools/idx/import/plugin"
86
87       [5]    the path "~/.doctools/idx/plugin"
88
89       [6]    the path "~/.doctools/plugin"
90
91       [7]    the path "~/.doctools/idx/import/plugins"
92
93       [8]    the path "~/.doctools/idx/plugins"
94
95       [9]    the path "~/.doctools/plugins"
96
97       [10]   the     registry     entry      "HKEY_CURRENT_USER\SOFTWARE\DOC‐
98              TOOLS\IDX\IMPORT\PLUGINS"
99
100       [11]   the      registry     entry     "HKEY_CURRENT_USER\SOFTWARE\DOC‐
101              TOOLS\IDX\PLUGINS"
102
103       [12]   the registry entry "HKEY_CURRENT_USER\SOFTWARE\DOCTOOLS\PLUGINS"
104
105       The last three are used only when the package is run on a machine using
106       Windows(tm) operating system.
107
108       The  whole  system  is  delivered  with  two predefined import plugins,
109       namely
110
111       docidx See docidx import plugin for details.
112
113       json   See json import plugin for details.
114
115       Readers wishing to write their own import plugin for some format,  i.e.
116       plugin  writers  reading  and  understanding the section containing the
117       Import plugin API v2 reference is an absolute necessity, as  it  speci‐
118       fies the interaction between this package and its plugins in detail.
119

CONCEPTS

121       [1]    A keyword index consists of a (possibly empty) set of keywords.
122
123       [2]    Each keyword in the set is identified by its name.
124
125       [3]    Each keyword has a (possibly empty) set of references.
126
127       [4]    A reference can be associated with more than one keyword.
128
129       [5]    A reference not associated with at least one keyword is not pos‐
130              sible however.
131
132       [6]    Each reference is identified by its target, specified as  either
133              an  url or symbolic filename, depending on the type of reference
134              (url, or manpage).
135
136       [7]    The type of a reference (url, or manpage) depends  only  on  the
137              reference itself, and not the keywords it is associated with.
138
139       [8]    In  addition to a type each reference has a descriptive label as
140              well. This label depends only on the reference itself,  and  not
141              the keywords it is associated with.
142
143       A few notes
144
145       [1]    Manpage references are intended to be used for references to the
146              documents the index is made for. Their target is a symbolic file
147              name  identifying  the  document, and export plugins may replace
148              symbolic with actual file names, if specified.
149
150       [2]    Url references are intended on the othre hand are inteded to  be
151              used  for links to anything else, like websites. Their target is
152              an url.
153
154       [3]    While url and manpage references share  a  namespace  for  their
155              identifiers, this should be no problem, given that manpage iden‐
156              tifiers are symbolic filenames and as  such  they  should  never
157              look like urls, the identifiers for url references.
158

API

160   PACKAGE COMMANDS
161       ::doctools::idx::import objectName
162              This command creates a new import manager object with an associ‐
163              ated Tcl command whose name is objectName. This  object  command
164              is  explained  in full detail in the sections Object command and
165              Object methods. The object command will  be  created  under  the
166              current  namespace if the objectName is not fully qualified, and
167              in the specified namespace otherwise.
168
169   OBJECT COMMAND
170       All objects created by the  ::doctools::idx::import  command  have  the
171       following general form:
172
173       objectName method ?arg arg ...?
174              The  method method and its arg'uments determine the exact behav‐
175              ior of the command.  See section Object methods for the detailed
176              specifications.
177
178   OBJECT METHODS
179       objectName destroy
180              This method destroys the object it is invoked for.
181
182       objectName import text text ?format?
183              This  method  takes  the text and converts it from the specified
184              format to the canonical serialization of a keyword  index  using
185              the import plugin for the format. An error is thrown if no plug‐
186              in could be found for the format.  The  serialization  generated
187              by  the  conversion  process  is  returned as the result of this
188              method.
189
190              If no format is specified the method defaults to docidx.
191
192              The specification of what a canonical serialization  is  can  be
193              found in the section Keyword index serialization format.
194
195              The  plugin has to conform to the interface specified in section
196              Import plugin API v2 reference.
197
198       objectName import file path ?format?
199              This method is a  convenient  wrapper  around  the  import  text
200              method described by the previous item.  It reads the contents of
201              the specified file into memory, feeds  the  result  into  import
202              text and returns the resulting serialization as its own result.
203
204       objectName import object text object text ?format?
205              This  method  is  a  convenient  wrapper  around the import text
206              method described by the previous item.  It expects  that  object
207              is  an  object command supporting a deserialize method expecting
208              the canonical serialization of a keyword index.  It imports  the
209              text  using  import text and then feeds the resulting serializa‐
210              tion into the object via deserialize.  This method  returns  the
211              empty string as it result.
212
213       objectName import object file object path ?format?
214              This  method  behaves  like  import  object text, except that it
215              reads the text to convert from the  specified  file  instead  of
216              being given it as argument.
217
218       objectName config names
219              This  method returns a list containing the names of all configu‐
220              ration variables currently known to the object.
221
222       objectName config get
223              This method returns a dictionary containing the names and values
224              of all configuration variables currently known to the object.
225
226       objectName config set name ?value?
227              This  method  sets the configuration variable name to the speci‐
228              fied value and returns the new value of the variable.
229
230              If no value is specified it simply returns  the  current  value,
231              without changing it.
232
233              Note  that  while  the user can set the predefined configuration
234              variables user and format doing so will have  no  effect,  these
235              values  will  be  internally  overridden when invoking an import
236              plugin.
237
238       objectName config unset pattern...
239              This method unsets  all  configuration  variables  matching  the
240              specified  glob  patterns.  If  no  pattern is specified it will
241              unset all currently defined configuration variables.
242
243       objectName includes
244              This method returns a list containing  the  currently  specified
245              paths  to use to search for include files when processing input.
246              The order of paths in the list corresponds to the order in which
247              they  are  used, from first to last, and also corresponds to the
248              order in which they were added to the object.
249
250       objectName include add path
251              This methods adds the specified path to the list of paths to use
252              to  search  for include files when processing input. The path is
253              added to the end of the list, causing it to  be  searched  after
254              all  previously  added  paths.  The result of the command is the
255              empty string.
256
257              The method does nothing if the path is already known.
258
259       objectName include remove path
260              This methods removes the specified path from the list  of  paths
261              to  use  to  search for include files when processing input. The
262              result of the command is the empty string.
263
264              The method does nothing if the path is not known.
265
266       objectName include clear
267              This method clears the list  of  paths  to  use  to  search  for
268              include  files  when processing input. The result of the command
269              is the empty string.
270

IMPORT PLUGIN API V2 REFERENCE

272       Plugins are what this package uses to manage the support for any  input
273       format  beyond  the Keyword index serialization format. Here we specify
274       the API the objects created by this package use to interact with  their
275       plugins.
276
277       A plugin for this package has to follow the rules listed below:
278
279       [1]    A plugin is a package.
280
281       [2]    The   name   of   a   plugin   package   has   the   form   doc‐
282              tools::idx::import::FOO, where FOO is the name of the format the
283              plugin  will generate output for. This name is also the argument
284              to provide to the  various  import  methods  of  import  manager
285              objects to get a string encoding a keyword index in that format.
286
287       [3]    The    plugin    can    expect    that    the    package    doc‐
288              tools::idx::export::plugin is present, as indicator that it  was
289              invoked from a genuine plugin manager.
290
291       [4]    The  plugin  can  expect  that  a  command  named IncludeFile is
292              present, with the signature
293
294              IncludeFile currentfile path
295                     This command has to be invoked by the plugin when it  has
296                     to  process  an included file, if the format has the con‐
297                     cept of such. An  example  of  such  a  format  would  be
298                     docidx.
299
300                     The plugin has to supply the following arguments
301
302                     string currentfile
303                            The  path  of the file it is currently processing.
304                            This may be the empty string if no such is known.
305
306                     string path
307                            The path of the include file as specified  in  the
308                            include directive being processed.
309
310                     The  result  of the command will be a 5-element list con‐
311                     taining
312
313                     [1]    A boolean flag indicating the  success  (True)  or
314                            failure (False) of the operation.
315
316                     [2]    In  case  of  success the contents of the included
317                            file, and the empty string otherwise.
318
319                     [3]    The resolved, i.e. absolute path of  the  included
320                            file, if possible, or the unchanged path argument.
321                            This is for display in an error message, or as the
322                            currentfile  argument  of another call to Include‐
323                            File should this file contain more files.
324
325                     [4]    In case of success an empty string, and for  fail‐
326                            ure a code indicating the reason for it, one of
327
328                            notfound
329                                   The specified file could not be found.
330
331                            notread
332                                   The  specified  file  was found, but not be
333                                   read into memory.
334
335                     [5]    An empty string in case of success of  a  notfound
336                            failure,  and an additional error message describ‐
337                            ing the reason for a notread error in more detail.
338
339       [5]    A plugin has to provide one command, with  the  signature  shown
340              below.
341
342              import text configuration
343                     Whenever  an import manager of doctools::idx has to parse
344                     input for an index it will invoke this command.
345
346                     string text
347                            This argument will contain the text  encoding  the
348                            index per the format the plugin is for.
349
350                     dictionary configuration
351                            This  argument will contain the current configura‐
352                            tion to apply to the parsing, as a dictionary map‐
353                            ping from variable names to values.
354
355                            The  following configuration variables have a pre‐
356                            defined meaning all plugins have to obey, although
357                            they  can ignore this information at their discre‐
358                            tion. Any other other configuration variables rec‐
359                            ognized  by a plugin will be described in the man‐
360                            page for that plugin.
361
362                            user   This variable is expected  to  contain  the
363                                   name  of the user owning the process invok‐
364                                   ing the plugin.
365
366                            format This variable is expected  to  contain  the
367                                   name of the format whose plugin is invoked.
368
369       [6]    A  single usage cycle of a plugin consists of the invokations of
370              the command import. This call has to leave the plugin in a state
371              where another usage cycle can be run without problems.
372

KEYWORD INDEX SERIALIZATION FORMAT

374       Here  we specify the format used by the doctools v2 packages to serial‐
375       ize keyword indices as immutable values for transport, comparison, etc.
376
377       We distinguish between regular and canonical  serializations.  While  a
378       keyword index may have more than one regular serialization only exactly
379       one of them will be canonical.
380
381       regular serialization
382
383              [1]    An index serialization is a nested Tcl dictionary.
384
385              [2]    This dictionary holds a single  key,  doctools::idx,  and
386                     its value. This value holds the contents of the index.
387
388              [3]    The  contents  of  the index are a Tcl dictionary holding
389                     the title of the index, a label,  and  the  keywords  and
390                     references. The relevant keys and their values are
391
392                     title  The  value is a string containing the title of the
393                            index.
394
395                     label  The value is a string containing a label  for  the
396                            index.
397
398                     keywords
399                            The  value is a Tcl dictionary, using the keywords
400                            known to the index as keys. The associated  values
401                            are lists containing the identifiers of the refer‐
402                            ences associated with that particular keyword.
403
404                            Any reference identifier used in these  lists  has
405                            to  exist  as  a key in the references dictionary,
406                            see the next item for its definition.
407
408                     references
409                            The value is a Tcl dictionary, using  the  identi‐
410                            fiers  for  the  references  known to the index as
411                            keys. The associated values  are  2-element  lists
412                            containing the type and label of the reference, in
413                            this order.
414
415                            Any key here has to be associated  with  at  least
416                            one  keyword,  i.e.  occur  in at least one of the
417                            reference lists which are the values in  the  key‐
418                            words  dictionary, see previous item for its defi‐
419                            nition.
420
421              [4]    The type of a reference can be one of two values,
422
423                     manpage
424                            The identifier of the reference is interpreted  as
425                            symbolic  file name, referring to one of the docu‐
426                            ments the index was made for.
427
428                     url    The identifier of the reference is interpreted  as
429                            an  url, referring to some external location, like
430                            a website, etc.
431
432       canonical serialization
433              The canonical serialization of a keyword index has the format as
434              specified  in the previous item, and then additionally satisfies
435              the constraints below, which make it unique among all the possi‐
436              ble serializations of the keyword index.
437
438              [1]    The  keys  found  in  all the nested Tcl dictionaries are
439                     sorted in ascending dictionary  order,  as  generated  by
440                     Tcl's builtin command lsort -increasing -dict.
441
442              [2]    The  references  listed for each keyword of the index, if
443                     any, are listed in ascending dictionary  order  of  their
444                     labels,  as  generated  by  Tcl's  builtin  command lsort
445                     -increasing -dict.
446

BUGS, IDEAS, FEEDBACK

448       This document, and the package it describes, will  undoubtedly  contain
449       bugs  and  other problems.  Please report such in the category doctools
450       of the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist].   Please
451       also  report any ideas for enhancements you may have for either package
452       and/or documentation.
453
454       When proposing code changes, please provide unified diffs, i.e the out‐
455       put of diff -u.
456
457       Note  further  that  attachments  are  strongly  preferred over inlined
458       patches. Attachments can be made by going  to  the  Edit  form  of  the
459       ticket  immediately  after  its  creation, and then using the left-most
460       button in the secondary navigation bar.
461

KEYWORDS

463       conversion, docidx, documentation, import, index, json, keyword  index,
464       manpage, markup, parsing, plugin, reference, url
465

CATEGORY

467       Documentation tools
468
470       Copyright (c) 2009-2018 Andreas Kupries <andreas_kupries@users.sourceforge.net>
471
472
473
474
475tcllib                                0.2             doctools::idx::import(n)
Impressum