1DBM_Filter(3pm) Perl Programmers Reference Guide DBM_Filter(3pm)
2
6 DBM_Filter -- Filter DBM keys/values
7
9 use DBM_Filter ;
10 use SDBM_File; # or DB_File, GDBM_File, NDBM_File, or ODBM_File
11 $db = tie %hash, ...
13 $db->Filter_Push(Fetch => sub {...},
15 Store => sub {...});
16 $db->Filter_Push('my_filter1');
18 $db->Filter_Push('my_filter2', params...);
19 $db->Filter_Key_Push(...) ;
21 $db->Filter_Value_Push(...) ;
22 $db->Filter_Pop();
24 $db->Filtered();
25 package DBM_Filter::my_filter1;
27 sub Store { ... }
29 sub Fetch { ... }
30 1;
32 package DBM_Filter::my_filter2;
34 sub Filter
36 {
37 my @opts = @_;
38 ...
39 return (
40 sub Store { ... },
41 sub Fetch { ... } );
42 }
43 1;
45
47 This module provides an interface that allows filters to be applied to
48 tied Hashes associated with DBM files. It builds on the DBM Filter
49 hooks that are present in all the *DB*_File modules included with the
50 standard Perl source distribution from version 5.6.1 onwards. In
51 addition to the *DB*_File modules distributed with Perl, the BerkeleyDB
52 module, available on CPAN, supports the DBM Filter hooks. See
53 perldbmfilter for more details on the DBM Filter hooks.
54
56 A DBM Filter allows the keys and/or values in a tied hash to be
57 modified by some user-defined code just before it is written to the DBM
58 file and just after it is read back from the DBM file. For example,
59 this snippet of code
60 $some_hash{"abc"} = 42;
62 could potentially trigger two filters, one for the writing of the key
64 "abc" and another for writing the value 42. Similarly, this snippet
65 my ($key, $value) = each %some_hash
67 will trigger two filters, one for the reading of the key and one for
69 the reading of the value.
70 Like the existing DBM Filter functionality, this module arranges for
72 the $_ variable to be populated with the key or value that a filter
73 will check. This usually means that most DBM filters tend to be very
74 short.
75 So what's new?
77 The main enhancements over the standard DBM Filter hooks are:
78 • A cleaner interface.
80 • The ability to easily apply multiple filters to a single DBM file.
82 • The ability to create "canned" filters. These allow commonly used
84 filters to be packaged into a stand-alone module.
85
87 This module will arrange for the following methods to be available via
88 the object returned from the "tie" call.
89 $db->Filter_Push() / $db->Filter_Key_Push() / $db->Filter_Value_Push()
91 Add a filter to filter stack for the database, $db. The three formats
92 vary only in whether they apply to the DBM key, the DBM value or both.
93 Filter_Push
95 The filter is applied to both keys and values.
96 Filter_Key_Push
98 The filter is applied to the key only.
99 Filter_Value_Push
101 The filter is applied to the value only.
102 $db->Filter_Pop()
104 Removes the last filter that was applied to the DBM file associated
105 with $db, if present.
106 $db->Filtered()
108 Returns TRUE if there are any filters applied to the DBM associated
109 with $db. Otherwise returns FALSE.
110
112 Filters can be created in two main ways
113 Immediate Filters
115 An immediate filter allows you to specify the filter code to be used at
116 the point where the filter is applied to a dbm. In this mode the
117 Filter_*_Push methods expects to receive exactly two parameters.
118 my $db = tie %hash, 'SDBM_File', ...
120 $db->Filter_Push( Store => sub { },
121 Fetch => sub { });
122 The code reference associated with "Store" will be called before any
124 key/value is written to the database and the code reference associated
125 with "Fetch" will be called after any key/value is read from the
126 database.
127 For example, here is a sample filter that adds a trailing NULL
129 character to all strings before they are written to the DBM file, and
130 removes the trailing NULL when they are read from the DBM file
131 my $db = tie %hash, 'SDBM_File', ...
133 $db->Filter_Push( Store => sub { $_ .= "\x00" ; },
134 Fetch => sub { s/\x00$// ; });
135 Points to note:
137 1. Both the Store and Fetch filters manipulate $_.
139 Canned Filters
141 Immediate filters are useful for one-off situations. For more generic
142 problems it can be useful to package the filter up in its own module.
143 The usage is for a canned filter is:
145 $db->Filter_Push("name", params)
147 where
149 "name"
151 is the name of the module to load. If the string specified does
152 not contain the package separator characters "::", it is assumed
153 to refer to the full module name "DBM_Filter::name". This means
154 that the full names for canned filters, "null" and "utf8",
155 included with this module are:
156 DBM_Filter::null
158 DBM_Filter::utf8
159 params
161 any optional parameters that need to be sent to the filter. See
162 the encode filter for an example of a module that uses parameters.
163 The module that implements the canned filter can take one of two forms.
165 Here is a template for the first
166 package DBM_Filter::null ;
168 use strict;
170 use warnings;
171 sub Store
173 {
174 # store code here
175 }
176 sub Fetch
178 {
179 # fetch code here
180 }
181 1;
183 Notes:
185 1. The package name uses the "DBM_Filter::" prefix.
187 2. The module must have both a Store and a Fetch method. If only one
189 is present, or neither are present, a fatal error will be thrown.
190 The second form allows the filter to hold state information using a
192 closure, thus:
193 package DBM_Filter::encoding ;
195 use strict;
197 use warnings;
198 sub Filter
200 {
201 my @params = @_ ;
202 ...
204 return {
205 Store => sub { $_ = $encoding->encode($_) },
206 Fetch => sub { $_ = $encoding->decode($_) }
207 } ;
208 }
209 1;
211 In this instance the "Store" and "Fetch" methods are encapsulated
213 inside a "Filter" method.
214
216 A number of canned filers are provided with this module. They cover a
217 number of the main areas that filters are needed when interfacing with
218 DBM files. They also act as templates for your own filters.
219 The filter included are:
221 • utf8
223 This module will ensure that all data written to the DBM will be
225 encoded in UTF-8.
226 This module needs the Encode module.
228 • encode
230 Allows you to choose the character encoding will be store in the
232 DBM file.
233 • compress
235 This filter will compress all data before it is written to the
237 database and uncompressed it on reading.
238 This module needs Compress::Zlib.
240 • int32
242 This module is used when interoperating with a C/C++ application
244 that uses a C int as either the key and/or value in the DBM file.
245 • null
247 This module ensures that all data written to the DBM file is null
249 terminated. This is useful when you have a perl script that needs
250 to interoperate with a DBM file that a C program also uses. A
251 fairly common issue is for the C application to include the
252 terminating null in a string when it writes to the DBM file. This
253 filter will ensure that all data written to the DBM file can be
254 read by the C application.
255
257 Maintain Round Trip Integrity
258 When writing a DBM filter it is very important to ensure that it is
259 possible to retrieve all data that you have written when the DBM filter
260 is in place. In practice, this means that whatever transformation is
261 applied to the data in the Store method, the exact inverse operation
262 should be applied in the Fetch method.
263 If you don't provide an exact inverse transformation, you will find
265 that code like this will not behave as you expect.
266 while (my ($k, $v) = each %hash)
268 {
269 ...
270 }
271 Depending on the transformation, you will find that one or more of the
273 following will happen
274 1. The loop will never terminate.
276 2. Too few records will be retrieved.
278 3. Too many will be retrieved.
280 4. The loop will do the right thing for a while, but it will
282 unexpectedly fail.
283 Don't mix filtered & non-filtered data in the same database file.
285 This is just a restatement of the previous section. Unless you are
286 completely certain you know what you are doing, avoid mixing filtered &
287 non-filtered data.
288
290 Say you need to interoperate with a legacy C application that stores
291 keys as C ints and the values and null terminated UTF-8 strings. Here
292 is how you would set that up
293 my $db = tie %hash, 'SDBM_File', ...
295 $db->Filter_Key_Push('int32') ;
297 $db->Filter_Value_Push('utf8');
299 $db->Filter_Value_Push('null');
300
302 <DB_File>, GDBM_File, NDBM_File, ODBM_File, SDBM_File, perldbmfilter
303
305 Paul Marquess <pmqs@cpan.org>
306perl v5.34.1 2022-03-15 DBM_Filter(3pm)