1DBM::Deep::Cookbook(3)User Contributed Perl DocumentationDBM::Deep::Cookbook(3)
2
6 DBM::Deep::Cookbook - Cookbook for DBM::Deep
7
9 This is the Cookbook for DBM::Deep. It contains useful tips and tricks,
10 plus some examples of how to do common tasks.
11
13 Unicode data
14 If possible, it is highly recommended that you upgrade your database to
15 version 2 (using the utils/upgrade_db.pl script in the CPAN
16 distribution), in order to use Unicode.
17 If your databases are still shared by perl installations with older
19 DBM::Deep versions, you can use filters to encode strings on the fly:
20 my $db = DBM::Deep->new( ... );
22 my $encode_sub = sub { my $s = shift; utf8::encode($s); $s };
23 my $decode_sub = sub { my $s = shift; utf8::decode($s); $s };
24 $db->set_filter( 'store_value' => $encode_sub );
25 $db->set_filter( 'fetch_value' => $decode_sub );
26 $db->set_filter( 'store_key' => $encode_sub );
27 $db->set_filter( 'fetch_key' => $decode_sub );
28 A previous version of this cookbook recommended using "binmode
30 $db->_fh, ":utf8"", but that is not a good idea, as it could easily
31 corrupt the database.
32 Real-time Encryption Example
34 NOTE: This is just an example of how to write a filter. This most
35 definitely should NOT be taken as a proper way to write a filter that
36 does encryption. (Furthermore, it fails to take Unicode into account.)
37 Here is a working example that uses the Crypt::Blowfish module to do
39 real-time encryption / decryption of keys & values with DBM::Deep
40 Filters. Please visit
41 <http://search.cpan.org/search?module=Crypt::Blowfish> for more on
42 Crypt::Blowfish. You'll also need the Crypt::CBC module.
43 use DBM::Deep;
45 use Crypt::Blowfish;
46 use Crypt::CBC;
47 my $cipher = Crypt::CBC->new({
49 'key' => 'my secret key',
50 'cipher' => 'Blowfish',
51 'iv' => '$KJh#(}q',
52 'regenerate_key' => 0,
53 'padding' => 'space',
54 'prepend_iv' => 0
55 });
56 my $db = DBM::Deep->new(
58 file => "foo-encrypt.db",
59 filter_store_key => \&my_encrypt,
60 filter_store_value => \&my_encrypt,
61 filter_fetch_key => \&my_decrypt,
62 filter_fetch_value => \&my_decrypt,
63 );
64 $db->{key1} = "value1";
66 $db->{key2} = "value2";
67 print "key1: " . $db->{key1} . "\n";
68 print "key2: " . $db->{key2} . "\n";
69 undef $db;
71 exit;
72 sub my_encrypt {
74 return $cipher->encrypt( $_[0] );
75 }
76 sub my_decrypt {
77 return $cipher->decrypt( $_[0] );
78 }
79 Real-time Compression Example
81 Here is a working example that uses the Compress::Zlib module to do
82 real-time compression / decompression of keys & values with DBM::Deep
83 Filters. Please visit
84 <http://search.cpan.org/search?module=Compress::Zlib> for more on
85 Compress::Zlib.
86 use DBM::Deep;
88 use Compress::Zlib;
89 my $db = DBM::Deep->new(
91 file => "foo-compress.db",
92 filter_store_key => \&my_compress,
93 filter_store_value => \&my_compress,
94 filter_fetch_key => \&my_decompress,
95 filter_fetch_value => \&my_decompress,
96 );
97 $db->{key1} = "value1";
99 $db->{key2} = "value2";
100 print "key1: " . $db->{key1} . "\n";
101 print "key2: " . $db->{key2} . "\n";
102 undef $db;
104 exit;
105 sub my_compress {
107 my $s = shift;
108 utf8::encode($s);
109 return Compress::Zlib::memGzip( $s ) ;
110 }
111 sub my_decompress {
112 my $s = Compress::Zlib::memGunzip( shift ) ;
113 utf8::decode($s);
114 return $s;
115 }
116 Note: Filtering of keys only applies to hashes. Array "keys" are
118 actually numerical index numbers, and are not filtered.
119
121 DBM::Deep by default uses the Message Digest 5 (MD5) algorithm for
122 hashing keys. However you can override this, and use another algorithm
123 (such as SHA-256) or even write your own. But please note that
124 DBM::Deep currently expects zero collisions, so your algorithm has to
125 be perfect, so to speak. Collision detection may be introduced in a
126 later version.
127 You can specify a custom digest algorithm by passing it into the
129 parameter list for new(), passing a reference to a subroutine as the
130 'digest' parameter, and the length of the algorithm's hashes (in bytes)
131 as the 'hash_size' parameter. Here is a working example that uses a
132 256-bit hash from the Digest::SHA256 module. Please see
133 <http://search.cpan.org/search?module=Digest::SHA256> for more
134 information.
135 The value passed to your digest function will be encoded as UTF-8 if
137 the database is in version 2 format or higher.
138 use DBM::Deep;
140 use Digest::SHA256;
141 my $context = Digest::SHA256::new(256);
143 my $db = DBM::Deep->new(
145 filename => "foo-sha.db",
146 digest => \&my_digest,
147 hash_size => 32,
148 );
149 $db->{key1} = "value1";
151 $db->{key2} = "value2";
152 print "key1: " . $db->{key1} . "\n";
153 print "key2: " . $db->{key2} . "\n";
154 undef $db;
156 exit;
157 sub my_digest {
159 return substr( $context->hash($_[0]), 0, 32 );
160 }
161 Note: Your returned digest strings must be EXACTLY the number of bytes
163 you specify in the hash_size parameter (in this case 32). Undefined
164 behavior will occur otherwise.
165 Note: If you do choose to use a custom digest algorithm, you must set
167 it every time you access this file. Otherwise, the default (MD5) will
168 be used.
169
171 Because DBM::Deep is a conncurrent datastore, every change is flushed
172 to disk immediately and every read goes to disk. This means that
173 DBM::Deep functions at the speed of disk (generally 10-20ms) vs. the
174 speed of RAM (generally 50-70ns), or at least 150-200x slower than the
175 comparable in-memory datastructure in Perl.
176 There are several techniques you can use to speed up how DBM::Deep
178 functions.
179 · Put it on a ramdisk
181 The easiest and quickest mechanism to making DBM::Deep run faster
183 is to create a ramdisk and locate the DBM::Deep file there. Doing
184 this as an option may become a feature of DBM::Deep, assuming there
185 is a good ramdisk wrapper on CPAN.
186 · Work at the tightest level possible
188 It is much faster to assign the level of your db that you are
190 working with to an intermediate variable than to re-look it up
191 every time. Thus
192 # BAD
194 while ( my ($k, $v) = each %{$db->{foo}{bar}{baz}} ) {
195 ...
196 }
197 # GOOD
199 my $x = $db->{foo}{bar}{baz};
200 while ( my ($k, $v) = each %$x ) {
201 ...
202 }
203 · Make your file as tight as possible
205 If you know that you are not going to use more than 65K in your
207 database, consider using the "pack_size => 'small'" option. This
208 will instruct DBM::Deep to use 16bit addresses, meaning that the
209 seek times will be less.
210
212 DBM::Deep(3), Digest::MD5(3), Digest::SHA256(3), Crypt::Blowfish(3),
213 Compress::Zlib(3)
214perl v5.28.0 2018-07-14 DBM::Deep::Cookbook(3)