1tie(n) Tcl Data Structures tie(n)
2______________________________________________________________________________
6
8 tie - Array persistence
9
11 package require Tcl 8.5
12 package require tie ?1.2?
14 ::tie::tie arrayvarname options... dstype dsname...
16 ::tie::untie arrayvarname ?token?
18 ::tie::info ties arrayvarname
20 ::tie::info types
22 ::tie::info type dstype
24 ::tie::register dsclasscmd as dstype
26 dsclasscmd objname ?dsname...?
28 ds destroy
30 ds names
32 ds size
34 ds get
36 ds set dict
38 ds unset ?pattern?
40 ds setv index value
42 ds unsetv index
44 ds getv index
46______________________________________________________________________________
48
50 The tie package provides a framework for the creation of persistent Tcl
51 array variables. It should be noted that the provided mechanism is
52 generic enough to also allow its usage for the distribution of the con‐
53 tents of Tcl arrays over multiple threads and processes, i.e. communi‐
54 cation.
55 This, persistence and communication, is accomplished by tying) a Tcl
57 array variable to a data source. Examples of data sources are other Tcl
58 arrays and files.
59 It should be noted that a single Tcl array variable can be tied to more
61 than one data source. It is this feature which allows the framework to
62 be used for communication as well. Just tie several Tcl arrays in many
63 client processes to a Tcl array in a server and all changes to any of
64 them will be distributed to all. Less centralized variants of this are
65 of course possible as well.
66
68 TIE API
69 This section describes the basic API used to establish and remove ties
70 between Tcl array variables and data sources. This interface is the
71 only one a casual user has to be concerned about. The following sec‐
72 tions about the various internal interfaces can be safely skipped.
73 ::tie::tie arrayvarname options... dstype dsname...
75 This command establishes a tie between the Tcl array whose name
76 is provided by the argument arrayvarname and the data source
77 identified by the dstype and its series of dsname arguments. All
78 changes made to the Tcl array after this command returns will be
79 saved to the data source for safekeeping (or distribution).
80 The result of the command is always a token which identifies the
82 new tie. This token can be used later to destroy this specific
83 tie.
84 varname arrayvarname (in)
86 The name of the Tcl array variable to connect the new tie
87 to.
88 name|command dstype (in)
90 This argument specifies the type of the data source we
91 wish to access. The dstype can be one of log, array, re‐
92 motearray, file, growfile, or dsource; in addition, the
93 programmer can register additional data source types.
94 Each dstype is followed by one or more arguments that
95 identify the data source to which the array is to be
96 tied.
97 string dsname (in)
99 The series of dsname arguments coming after the dstype
100 identifies the data source we wish to connect to, and has
101 to be appropriate for the chosen type.
102 The command understands a number of additional options which guide the
104 process of setting up the connection between Tcl array and data source.
105 -open The Tcl array for the new tie is loaded from the data
108 source, and the previously existing contents of the Tcl
109 array are erased. Care is taken to not erase the previous
110 contents should the creation of the tie fail.
111 This option and the option -save exclude each other. If
113 neither this nor option -save are specified then this op‐
114 tion is assumed as default.
115 -save The Tcl array for the new tie is saved to the data
117 source, and the previously existing contents of the data
118 source are erased.
119 This option and the option -open exclude each other. If
121 neither this nor option -open are specified then option
122 -open is assumed as default.
123 -merge Using this option prevents the erasure of any previously
125 existing content and merges the data instead. It can be
126 specified in conjunction with either -open or -save. They
127 determine how data existing in both Tcl array and data
128 source, i.e duplicates, are dealt with.
129 When used with -open data in the data source has prece‐
131 dence. In other words, for duplicates the data in the
132 data source is loaded into the Tcl array.
133 When used with -save data in the Tcl array has prece‐
135 dence. In other words, for duplicates the data in the Tcl
136 array is saved into the data source.
137 ::tie::untie arrayvarname ?token?
140 This command dissolves one or more ties associated with the Tcl
141 array named by arrayvarname. If no token is specified then all
142 ties to that Tcl array are dissolved. Otherwise only the tie the
143 token stands for is removed, if it is actually connected to the
144 array. Trying to remove a specific tie not belonging to the pro‐
145 vided array will cause an error.
146 It should be noted that while severing a tie will destroy man‐
148 agement information internal to the package the data source
149 which was handled by the tie will not be touched, only closed.
150 After the command returns none of changes made to the array will
152 be saved to the data source anymore.
153 The result of the command is an empty string.
155 varname arrayname (in)
157 The name of a Tcl array variable which may have ties.
158 handle token (in)
160 A handle representing a specific tie. This argument is
161 optional.
162 ::tie::info ties arrayvarname
165 This command returns a list of ties associated with the Tcl ar‐
166 ray variable named by arrayvarname. The result list will be
167 empty if the variable has no ties associated with it.
168 ::tie::info types
170 This command returns a dictionary of registered types, and the
171 class commands they are associated with.
172 ::tie::info type dstype
174 This command returns the fully resolved class command for a type
175 name. This means that the command will follow a chain of type
176 definitions ot its end.
177 STANDARD DATA SOURCE TYPES
179 This package provides the six following types as examples and standard
180 data sources.
181 log This data source does not maintain any actual data, nor persis‐
183 tence. It does not accept any identifying arguments. All changes
184 are simply logged to stdout.
185 array This data source uses a regular Tcl array as the origin of the
187 persistent data. It accepts a single identifying argument, the
188 name of this Tcl array. All changes are mirrored to that array.
189 remotearray
191 This data source is similar to array. The difference is that the
192 Tcl array to which we are mirroring is not directly accessible,
193 but through a send-like command.
194 It accepts three identifying arguments, the name of the other
196 Tcl array, the command prefix for the send-like accessor com‐
197 mand, and an identifier for the remote entity hosting the array,
198 in this order. All changes are mirrored to that array, via the
199 command prefix. All commands will be executed in the context of
200 the global namespace.
201 send-like means that the command prefix has to have send syntax
203 and semantics. I.e. it is a channel over which we can send arbi‐
204 trary commands to some other entity. The remote array data
205 source however uses only the commands set, unset, array exists,
206 array names, array set, and array get to retrieve and set values
207 in the remote array.
208 The command prefix and the entity id are separate to allow the
210 data source to use options like -async when assembling the ac‐
211 tual commands.
212 Examples of command prefixes, listed with the id of the remote
214 entity, without options. In reality only the part before the id
215 is the command prefix:
216 send tkname
218 The Tcl array is in a remote interpreter and is accessed
219 via Tk's X communication.
220 comm::comm send hostportid
222 The Tcl array is in a remote interpreter and is accessed
223 through a socket.
224 thread::send threadid
226 The Tcl array is in a remote interpreter in a different
227 thread of this process.
228 file This data source uses a single file as origin of the persistent
231 data. It accepts a single identifying argument, the path to this
232 file. The file has to be both readable and writable. It may not
233 exist, the data source will create it in that case. This (and
234 only this) situation will require that the directory for the
235 file exists and is writable as well.
236 All changes are saved in the file, as proper Tcl commands, one
238 command per operation. In other words, the file will always con‐
239 tain a proper Tcl script.
240 If the file exists when the tie using it is set up, then it will
242 be compacted, i.e. superfluous operations are removed, if the
243 operations log stored in it contains either at least one opera‐
244 tion clearing the whole array, or at least 1.5 times more opera‐
245 tions than entries in the loaded array.
246 growfile
248 This data source is like file in terms of the storage medium for
249 the array data, and how it is configured. In constrast to the
250 former it however assumes and ensures that the tied array will
251 never shrink. I.e. the creation of new array entries, and the
252 modification of existing entries is allowed, but the deletion of
253 entries is not, and causes the data source to throw errors.
254 This restriction allows us to simplify both file format and ac‐
256 cess to the file radically. For one, the file is read only once
257 and the internal cache cannot be invalidated. Second, writing
258 data is reduced to a simple append, and no compaction step is
259 necessary. The format of the contents is the string representa‐
260 tion of a dictionary which can be incrementally extended forever
261 at the end.
262 dsource
264 This data source uses an explicitly specified data source object
265 as the source for the persistent data. It accepts a single iden‐
266 tifying argument, the command prefix, i.e. object command.
267 To use this type it is necessary to know how the framework man‐
269 ages ties and what data source objects are.
270 All changes are delegated to the specified object.
272
274 This section is of no interest to the casual user of ties. Only devel‐
275 opers wishing to create new data sources have to know the information
276 provided herein.
277 DATA SOURCE OBJECTS
279 All ties are represented internally by an in-memory object which medi‐
280 ates between the tie framework and the specific data source, like an
281 array, file, etc. This is the data source object.
282 Its class, the data source class is not generic, but specific to the
284 type of the data source. Writing a new data source requires us to write
285 such a class, and then registering it with the framework as a new type.
286 The following subsections describe the various APIs a data source class
288 and the objects it generates will have to follow to be compatible with
289 the tie framework.
290 Data source objects are normally automatically created and destroyed by
292 the framework when a tie is created, or removed. This management can be
293 explicitly bypassed through the usage of the "dsource" type. The data
294 source for this type is a data source object itself, and this object is
295 outside of the scope of the tie framework and not managed by it. In
296 other words, this type allows the creation of ties which talk to pre-
297 existing data source objects, and these objects will survive the re‐
298 moval of the ties using them as well.
299 REGISTERING A NEW DATA SOURCE CLASS
301 After a data source class has been written it is necessary to register
302 it as a new type with the framework.
303 ::tie::register dsclasscmd as dstype
305 Using this command causes the tie framework to remember the
306 class command dsclasscmd of a data source class under the type
307 name dstype.
308 After the call the argument dstype of the basic user command
310 ::tie::tie will accept dstype as a type name and translate it
311 internally to the appropriate class command for the creation of
312 data source objects for the new data source.
313 DATA SOURCE CLASS
315 Each data source class is represented by a single command, also called
316 the class command, or object creation command. Its syntax is
317 dsclasscmd objname ?dsname...?
319 The first argument of the class command is the name of the data
320 source object to create. The framework itself will always sup‐
321 ply the string %AUTO%, to signal that the class command has to
322 generate not only the object, but the object name as well.
323 This is followed by a series of arguments identifying the data
325 source the new object is for. These are the same dsname argu‐
326 ments which are given to the basic user command ::tie::tie.
327 Their actual meaning is dependent on the data source class.
328 The result of the class command has to be the fully-qualified
330 name of the new data source object, i.e. the name of the object
331 command. The interface this command has to follow is described
332 in the section DATA SOURCE OBJECT API
333 DATA SOURCE OBJECT API
335 Please read the section DATA SOURCE CLASS first, to know how to gener‐
336 ate new object commands.
337 Each object command for a data source object has to provide at least
339 the methods listed below for proper inter-operation with the tie frame‐
340 work. Note that the names of most of the methods match the subcommands
341 of the builtin array command.
342 ds destroy
344 This method is called when the object ds is destroyed. It now
345 has to release all its internal resources associated with the
346 external data source.
347 ds names
349 This command has to return a list containing the names of all
350 keys found in the data source the object talks to. This is
351 equivalent to array names.
352 ds size
354 This command has to return an integer number specifying the num‐
355 ber of keys found in the data source the object talks to. This
356 is equivalent to array size.
357 ds get This command has to return a dictionary containing the data
359 found in the data source the object talks to. This is equivalent
360 to array get.
361 ds set dict
363 This command takes a dictionary and adds its contents to the
364 data source the object talks to. This is equivalent to array
365 set.
366 ds unset ?pattern?
368 This command takes a pattern and removes all elements whose keys
369 matching it from the data source. If no pattern is specified it
370 defaults to *, causing the removal of all elements. This is
371 nearly equivalent to array unset.
372 ds setv index value
374 This command has to save the value in the data source the object
375 talks to, under the key index.
376 The result of the command is ignored. If an error is thrown then
378 this error will show up as error of the set operation which
379 caused the method call.
380 ds unsetv index
382 This command has to remove the value under the key index from
383 the data source the object talks to.
384 The result of the command is ignored. If an error is thrown then
386 this error will show up as error of the unset operation which
387 caused the method call.
388 ds getv index
390 This command has to return the value for the key index in the
391 data source the object talks to.
392 And here a small table comparing the data source methods to the regular
394 Tcl commands for accessing an array.
395 Regular Tcl Data source
397 ----------- -----------
398 array names a ds names
399 array size a ds size
400 array get a ds get
401 array set a dict ds set dict
402 array unset a pattern ds unset ?pattern?
403 ----------- -----------
404 set a($idx) $val ds setv idx val
405 unset a($idx) ds unsetv idx
406 $a($idx) ds getv idx
407 ----------- -----------
408
411 This document, and the package it describes, will undoubtedly contain
412 bugs and other problems. Please report such in the category tie of the
413 Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please also
414 report any ideas for enhancements you may have for either package
415 and/or documentation.
416 When proposing code changes, please provide unified diffs, i.e the out‐
418 put of diff -u.
419 Note further that attachments are strongly preferred over inlined
421 patches. Attachments can be made by going to the Edit form of the
422 ticket immediately after its creation, and then using the left-most
423 button in the secondary navigation bar.
424
426 array, database, file, metakit, persistence, tie, untie
427
429 Programming tools
430
432 Copyright (c) 2004-2021 Andreas Kupries <andreas_kupries@users.sourceforge.net>
433tcllib 1.2 tie(n)