1SharedCache(3)        User Contributed Perl Documentation       SharedCache(3)
2
3
4

NAME

6       IPC::SharedCache - a Perl module to manage a cache in SysV IPC shared
7       memory.
8

SYNOPSIS

10         use IPC::SharedCache;
11
12         # the cache is accessed using a tied hash.
13         tie %cache, 'IPC::SharedCache', ipc_key => 'AKEY',
14                                         load_callback => \&load,
15                                         validate_callback => \&validate;
16
17         # get an item from the cache
18         $config_file = $cache{'/some/path/to/some.config'};
19

DESCRIPTION

21       This module provides a shared memory cache accessed as a tied hash.
22
23       Shared memory is an area of memory that is available to all processes.
24       It is accessed by choosing a key, the ipc_key arguement to tie.  Every
25       process that accesses shared memory with the same key gets access to
26       the same region of memory.  In some ways it resembles a file system,
27       but it is not hierarchical and it is resident in memory.  This makes it
28       harder to use than a filesystem but much faster.  The data in shared
29       memory persists until the machine is rebooted or it is explicitely
30       deleted.
31
32       This module attempts to make shared memory easy to use for one specific
33       application - a shared memory cache.  For other uses of shared memory
34       see the documentation to the excelent module I use, IPC::ShareLite
35       (IPC::ShareLite).
36
37       A cache is a place where processes can store the results of their
38       computations for use at a later time, possibly by other instances of
39       the application.  A good example of the use of a cache is a web server.
40       When a web server receieves a request for an html page it goes to the
41       file system to read it.  This is pretty slow, so the web server will
42       probably save the file in memory and use the in memory copy the next
43       time a request for that file comes in, as long as the file hasn't
44       changed on disk.  This certainly speeds things up but web servers have
45       to serve multiple clients at once, and that means multiple copies of
46       the in-memory data.  If the web server uses a shared memory cache, like
47       the one this module provides, then all the servers can use the same
48       cache and much less memory is consumed.
49
50       This module handles all shared memory interaction using the
51       IPC::ShareLite module (version 0.06 and higher) and all data
52       serialization using Storable.  See IPC::ShareLite and Storable for
53       details.
54

MOTIVATION

56       This module began its life as an internal piece of HTML::Template (see
57       HTML::Template).  HTML::Template has the ability to maintain a cache of
58       parsed template structures when running in a persistent environment
59       like Apache/mod_perl.  Since parsing a template from disk takes a fair
60       ammount of time this can provide a big performance gain.  Unfortunately
61       it can also consume large ammounts of memory since each web server
62       maintains its own cache in its own memory space.
63
64       By using IPC::ShareLite and Storable (IPC::ShareLite and Storable),
65       HTML::Template was able to maintain a single shared cache of templates.
66       The downside was that HTML::Template's cache routines became
67       complicated by a lot of IPC code.  My solution is to break out the IPC
68       cache mechanisms into their own module, IPC::SharedCache.  Hopefully
69       over time it can become general enough to be usable by more than just
70       HTML::Template.
71

USAGE

73       This module allows you to store data in shared memory and have it load
74       automatically when needed.  You can also define a test to screen cached
75       data for vailidty - if the test fails the data will be reloaded.  This
76       is useful for defining a max-age for cached data or keeping cached data
77       in sync with other resources.  In the web server example above the
78       validation test would look to see wether the file had changed on disk.
79
80       To initialize this module you provide two callback subroutines.  The
81       first is the "load_callback".  This gets called when a user of the
82       cache requests an item from that is not yet present or is stale.  It
83       must return a reference to the data structure that will be stored in
84       the cache.  The second is the "validate_callback".  This gets called on
85       every cache access - its job is to check the cached object for
86       freshness (and/or some other validity, of course).  It must return true
87       or false.  When it returns true, the cached object is valid and is
88       retained in the cache.  When it returns false, the object is re-loaded
89       using the "load_callback" and the result is stored in the cache.
90
91       To use the module you just request entries for the objects you need.
92       If the object is present in the cache and the "validate_callback"
93       returns true, then you get the object from the cache.  If not, the
94       object is loaded into the cache with the "load_callback" and returned
95       to you.
96
97       The cache can be used to store any perl data structures that can be
98       serialized by the Storable module.  See Storable for details.
99

EXAMPLE

101       In this example a shared cache of files is maintained.  The
102       "load_callback" reads the file from disk into the cache and the
103       "validate_callback" checks its modification time using stat().  Note
104       that the "load_callback" stores information into the cached object that
105       "validate_callback" uses to check the freshness of the cache.
106
107         # the "load_callback", loads the file from disk, storing its stat()
108         # information along with the file into the cache.  The key in this
109         # case is the filename to load.
110         sub load_file {
111           my $key = shift;
112
113           open(FILE, $key) or die "Unable to open file named $key : $!");
114
115           # note the modification time of this file - the 9th element of a
116           # stat() is the modification time of the file.
117           my $mtime = (stat($key))[9];
118
119           # read the file into the variable $contents in 1k chunks
120           my ($buffer, $contents);
121           while(read(FILE, $buffer, 1024)) { $contents .= $buffer }
122           close(FILE);
123
124           # prepare the record to store in the cache
125           my %record = ( mtime => $mtime, contents => $contents );
126
127           # this record goes into the cache associated with $key, which is
128           # the filename.  Notice that we're returning a reference to the
129           # data structure here.
130           return \%record;
131         }
132
133         # the "validate" callback, checks the mtime of the file on disk and
134         # compares it to the cache value.  The $record is a reference to the
135         # cached values array returned from load_file above.
136         sub validate_file {
137           my ($key, $record) = @_;
138
139           # get the modification time out of the record
140           my $stored_mtime = $record->{mtime};
141
142           # get the current modification time from the filesystem - the 9th
143           # element of a stat() is the modification time of the file.
144           my $current_mtime = (stat($key))[9];
145
146           # compare and return the appropriate result.
147           if ($stored_mtime == $current_mtime) {
148             # the cached object is valid, return true
149             return 1;
150           } else {
151             # the cached object is stale, return false - load_callback will
152             # be called to load it afresh from disk.
153             return 0;
154           }
155         }
156
157         # now we can construct the IPC::SharedCache object, using as a root
158         # key 'SAMS'.
159
160         tie %cache 'IPC::SharedCache' ipc_key => 'SAMS',
161                                       load_callback => \&load_file,
162                                       validate_callback => \&validate_file;
163
164         # fetch an object from the cache - if it's already in the cache and
165         # validate_file() returns 1, then we'll get the cached file.  If it's
166         # not in the cache, or validate_file returns 0, then load_file is
167         # called to load the file into the cache.
168
169         $config_file = $cache{'/some/path/to/some.config'};
170

DETAILS

172       The module implements a full tied hash interface, meaning that you can
173       use exists(), delete(), keys() and each().  However, in normal usage
174       all you'll need to do is to fetch values from the cache and possible
175       delete keys.  Just in case you were wondering, exists() doesn't trigger
176       a cache load - it returns 1 if the given key is already in the cache
177       and 0 if it isn't.  Similarily, keys() and each() operate on key/value
178       pairs already loaded into the cache.
179
180       The most important thing to realize is that there is no need to
181       explicitely store into the cache since the load_callback is called
182       automatically when it is necessary to load new data.  If you find
183       yourself using more than just ""$data = $cache{'key'};"" you need to
184       make sure you really know what you're doing!
185
186   OPTIONS
187       There are a number parameters to tie that can be used to control the
188       behavior of IPC::SharedCache.  Some of them are required, and some art
189       optional. Here's a preview:
190
191          tie %cache, 'IPC::SharedCache',
192
193             # required parameters
194             ipc_key => 'MYKI',
195             load_callback => \&load,
196             validate_callback => \&validate,
197
198             # optional parameters
199             ipc_mode => 0666,
200             ipc_segment_size => 1_000_000,
201             max_size => 50_000_000,
202             debug => 1;
203
204   ipc_key (required)
205       This is the unique identifier for the particular cache.  It can be
206       specified as either a four-character string or an integer value.  Any
207       script that wishes to access the cache must use the same ipc_key value.
208       You can use the ftok() function from IPC::SysV to generate this value,
209       see IPC::SysV for details.  Using an ipc_key value that's already in
210       use by a non-IPC::SharedCache application will cause an error.  Many
211       systems provide a utility called 'ipcs' to examine shared memory; you
212       can use it to check for existing shared memory usage before choosing
213       your ipc_key.
214
215   load_callback and validate_callback (required)
216       These parameters both specify callbacks for IPC::SharedCache to use
217       when the cache gets a request for a key.  When you access the cache
218       ("$data = $cache{$key}"), the cache first looks to see if it already
219       has an object for the given key.  If it doesn't, it calls the
220       load_callback and returns the result which is also stored in the cache.
221       Alternately, if it does have the object in the cache it calls the
222       validate_callback to check if the object is still good.  If the
223       validate_callback returns true then object is good and is returned.  If
224       the validate_callback returns false then the object is discarded and
225       the load_callback is called.
226
227       The load_callback recieves a single parameter - the requested key.  It
228       must return a reference to the data object be stored in the cache.
229       Returning something that is not a reference results in an error.
230
231       The validate_callback recieves two parameters - the key and the
232       reference to the stored object.  It must return true or false.
233
234       There are two ways to specify the callbacks.  The first is simply to
235       specify a subroutine reference.  This can be an anonymous subroutine or
236       a named one.  Example:
237
238         tie %cache, 'IPC::SharedCache',
239             ipc_key => 'TEST',
240             load_callback => sub { ... },
241             validate_callback => \&validate;
242
243       The second method allows parameters to be passed to the subroutine when
244       it is called.  This is done by specifying a reference to an array of
245       values, the first being the subroutine reference and the rest are
246       parameters for the subroutine.  The extra parameters are passed in
247       before the IPC::SharedCache provided parameters.  Example:
248
249         tie %cache, 'IPC::SharedCache',
250             ipc_key => 'TEST',
251             load_callback => [\&load, $arg1, $arg2, $arg3]
252             validate_callback => [\&validate, $self];
253
254   ipc_mode (optional)
255       This option specifies the access mode of the IPC cache.  It defaults to
256       0666.  See IPC::ShareLite for more information on IPC access modes.
257       The default should be fine for most applications.
258
259   ipc_segment_size (optional)
260       This option allows you to specify the "chunk size" of the IPC shared
261       memory segments.  The default is 65,536, which is 64K.  This is a good
262       default and is very portable.  If you know that your system supports
263       larger IPC segment sizes and you know that your cache will be storing
264       large data items you might get better performance by increasing this
265       value.
266
267       This value places no limit on the size of an object stored in the cache
268       - IPC::SharedCache automatically spreads large objects across multiple
269       IPC segments.
270
271       WARNING: setting this value too low (below 1024 in my experience) can
272       cause errors.
273
274   max_size (optional)
275       By setting this parameter you are setting a logical maximum to the
276       ammount of data stored in the cache.  When an item is stored in the
277       cache and this limit is exceded the oldest item (or items, as
278       necessary) in the cache will be deleted to make room.  This value is
279       specified in bytes.  It defaults to 0, which specifies no limit on the
280       size of the cache.
281
282       Turning this feature on costs a fair ammount of performance - how much
283       depends largely on home much data is being stored into the cache versus
284       the size of max_cache.  In the worst case (where the max_size is set
285       much too low) this option can cause severe "thrashing" and negate the
286       benefit of maintaining a cache entirely.
287
288       NOTE: The size of the cache may in fact exceed this value - the book-
289       keeping data stored in the root segment is not counted towards the
290       total.  Also, extra padding imposed by the ipc_segment_size is not
291       counted.  This may change in the future if I learn that it would be
292       appropriate to count this padding as used memory.  It is not clear to
293       me that all IPC implementations will really waste this memory.
294
295   debug (optional)
296       Set this option to 1 to see a whole bunch of text on STDERR about what
297       IPC::SharedCache is doing.
298

UTILITIES

300       Two static functions are included in this package that are meant to be
301       used from the command-line.
302
303   walk
304       Walk prints out a detailed listing of the contents of a shared cache at
305       a given ipc_key.  It provides information the current keys stored and a
306       dump of the objects stored in each key.  Be warned, this can be quite a
307       lot of data!  Also, you'll need the Data::Dumper module installed to
308       use 'walk'.  You can get it on CPAN.
309
310       You can call walk like:
311
312          perl -MIPC::SharedCache -e 'IPC::SharedCache::walk AKEY'"
313
314       Example:
315
316          $ perl -MIPC::SharedCache -e 'IPC::SharedCache::walk MYKI'"
317          *===================*
318          IPC::SharedCache Root
319          *===================*
320          IPC_KEY: MYKI
321          ELEMENTS: 3
322          TOTAL SIZE: 99 bytes
323          KEYS: a, b, c
324
325          *=======*
326          Data List
327          *=======*
328
329          KEY: a
330          $CONTENTS = [
331                        950760892,
332                        950760892,
333                        950760892
334                      ];
335
336
337          KEY: b
338          $CONTENTS = [
339                        950760892,
340                        950760892,
341                        950760892
342                      ];
343
344
345          KEY: c
346          $CONTENTS = [
347                        950760892,
348                        950760892,
349                        950760892
350                      ];
351
352   remove
353       This function totally removes an entire cache given an ipc_key value.
354       This should not be done to a running system!  Still, it's an invaluable
355       tool during development when flawed data may become 'stuck' in the
356       cache.
357
358          $ perl -MIPC::SharedCache -e 'IPC::SharedCache::remove MYKI'
359
360       This function is silent and thus may be usefully called from within a
361       script if desired.
362

BUGS

364       I am aware of no bugs - if you find one please email me at
365       sam@tregar.com.  When submitting bug reports, be sure to include full
366       details, including the VERSION of the module and a test script
367       demonstrating the problem.
368

CREDITS

370       I would like to thank Maurice Aubrey for making this module possible by
371       producing the excelent IPC::ShareLite.
372
373       The following people have contributed patches, ideas or new features:
374
375          Tim Bunce
376          Roland Mas
377          Drew Taylor
378          Ed Loehr
379          Maverick
380
381       Thanks everyone!
382

AUTHOR

384       Sam Tregar, sam@tregar.com (you can also find me on the mailing list
385       for HTML::Template at htmltmpl@lists.vm.com - join it by sending a
386       blank message to htmltmpl-subscribe@lists.vm.com).
387

LICENSE

389       IPC::SharedCache - a Perl module to manage a SysV IPC shared cache.
390       Copyright (C) 2000 Sam Tregar (sam@tregar.com)
391
392       This program is free software; you can redistribute it and/or modify it
393       under the terms of the GNU General Public License as published by the
394       Free Software Foundation; either version 2 of the License, or (at your
395       option) any later version.
396
397       This program is distributed in the hope that it will be useful, but
398       WITHOUT ANY WARRANTY; without even the implied warranty of
399       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
400       General Public License for more details.
401
402       You should have received a copy of the GNU General Public License along
403       with this program; if not, write to the Free Software Foundation, Inc.,
404       59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
405

AUTHOR

407       Sam Tregar, sam@tregar.com
408

SEE ALSO

410       perl(1).
411
412
413
414perl v5.32.1                      2021-01-27                    SharedCache(3)
Impressum