1BerkeleyDB(3) User Contributed Perl Documentation BerkeleyDB(3)
2
3
4
6 BerkeleyDB - Perl extension for Berkeley DB version 2, 3, 4, 5 or 6
7
9 use BerkeleyDB;
10
11 $env = new BerkeleyDB::Env [OPTIONS] ;
12
13 $db = tie %hash, 'BerkeleyDB::Hash', [OPTIONS] ;
14 $db = new BerkeleyDB::Hash [OPTIONS] ;
15
16 $db = tie %hash, 'BerkeleyDB::Btree', [OPTIONS] ;
17 $db = new BerkeleyDB::Btree [OPTIONS] ;
18
19 $db = tie @array, 'BerkeleyDB::Recno', [OPTIONS] ;
20 $db = new BerkeleyDB::Recno [OPTIONS] ;
21
22 $db = tie @array, 'BerkeleyDB::Queue', [OPTIONS] ;
23 $db = new BerkeleyDB::Queue [OPTIONS] ;
24
25 $db = new BerkeleyDB::Heap [OPTIONS] ;
26
27 $db = new BerkeleyDB::Unknown [OPTIONS] ;
28
29 $status = BerkeleyDB::db_remove [OPTIONS]
30 $status = BerkeleyDB::db_rename [OPTIONS]
31 $status = BerkeleyDB::db_verify [OPTIONS]
32
33 $hash{$key} = $value ;
34 $value = $hash{$key} ;
35 each %hash ;
36 keys %hash ;
37 values %hash ;
38
39 $env = $db->Env()
40 $status = $db->db_get()
41 $status = $db->db_exists() ;
42 $status = $db->db_put() ;
43 $status = $db->db_del() ;
44 $status = $db->db_sync() ;
45 $status = $db->db_close() ;
46 $status = $db->db_pget()
47 $hash_ref = $db->db_stat() ;
48 $status = $db->db_key_range();
49 $type = $db->type() ;
50 $status = $db->status() ;
51 $boolean = $db->byteswapped() ;
52 $status = $db->truncate($count) ;
53 $status = $db->compact($start, $stop, $c_data, $flags, $end);
54 $status = $db->get_blob_threshold($t1) ;
55 $status = $db->get_blob_dir($dir) ;
56
57 $bool = $env->cds_enabled();
58 $bool = $db->cds_enabled();
59 $lock = $db->cds_lock();
60 $lock->cds_unlock();
61
62 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ;
63 ($flag, $old_offset, $old_length) = $db->partial_clear() ;
64
65 $cursor = $db->db_cursor([$flags]) ;
66 $newcursor = $cursor->c_dup([$flags]);
67 $status = $cursor->c_get() ;
68 $status = $cursor->c_put() ;
69 $status = $cursor->c_del() ;
70 $status = $cursor->c_count() ;
71 $status = $cursor->c_pget() ;
72 $status = $cursor->status() ;
73 $status = $cursor->c_close() ;
74 $stream = $cursor->db_stream() ;
75
76 $cursor = $db->db_join() ;
77 $status = $cursor->c_get() ;
78 $status = $cursor->c_close() ;
79
80 $status = $stream->size($S);
81 $status = $stream->read($data, $offset, $size);
82 $status = $stream->write($data, $offset);
83
84 $status = $env->txn_checkpoint()
85 $hash_ref = $env->txn_stat()
86 $status = $env->set_mutexlocks()
87 $status = $env->set_flags()
88 $status = $env->set_timeout()
89 $status = $env->lock_detect()
90 $status = $env->lsn_reset()
91 $status = $env->get_blob_threshold($t1) ;
92 $status = $env->get_blob_dir($dir) ;
93
94 $txn = $env->txn_begin() ;
95 $db->Txn($txn);
96 $txn->Txn($db1, $db2,...);
97 $status = $txn->txn_prepare()
98 $status = $txn->txn_commit()
99 $status = $txn->txn_abort()
100 $status = $txn->txn_id()
101 $status = $txn->txn_discard()
102 $status = $txn->set_timeout()
103
104 $status = $env->set_lg_dir();
105 $status = $env->set_lg_bsize();
106 $status = $env->set_lg_max();
107
108 $status = $env->set_data_dir() ;
109 $status = $env->set_tmp_dir() ;
110 $status = $env->set_verbose() ;
111 $db_env_ptr = $env->DB_ENV() ;
112
113 $BerkeleyDB::Error
114 $BerkeleyDB::db_version
115
116 # DBM Filters
117 $old_filter = $db->filter_store_key ( sub { ... } ) ;
118 $old_filter = $db->filter_store_value( sub { ... } ) ;
119 $old_filter = $db->filter_fetch_key ( sub { ... } ) ;
120 $old_filter = $db->filter_fetch_value( sub { ... } ) ;
121
122 # deprecated, but supported
123 $txn_mgr = $env->TxnMgr();
124 $status = $txn_mgr->txn_checkpoint()
125 $hash_ref = $txn_mgr->txn_stat()
126 $txn = $txn_mgr->txn_begin() ;
127
129 NOTE: This document is still under construction. Expect it to be
130 incomplete in places.
131
132 This Perl module provides an interface to most of the functionality
133 available in Berkeley DB versions 2, 3, 5 and 6. In general it is safe
134 to assume that the interface provided here to be identical to the
135 Berkeley DB interface. The main changes have been to make the Berkeley
136 DB API work in a Perl way. Note that if you are using Berkeley DB 2.x,
137 the new features available in Berkeley DB 3.x or later are not
138 available via this module.
139
140 The reader is expected to be familiar with the Berkeley DB
141 documentation. Where the interface provided here is identical to the
142 Berkeley DB library and the... TODO
143
144 The db_appinit, db_cursor, db_open and db_txn man pages are
145 particularly relevant.
146
147 The interface to Berkeley DB is implemented with a number of Perl
148 classes.
149
151 The BerkeleyDB::Env class provides an interface to the Berkeley DB
152 function db_appinit in Berkeley DB 2.x or db_env_create and DBENV->open
153 in Berkeley DB 3.x (or later). Its purpose is to initialise a number of
154 sub-systems that can then be used in a consistent way in all the
155 databases you make use of in the environment.
156
157 If you don't intend using transactions, locking or logging, then you
158 shouldn't need to make use of BerkeleyDB::Env.
159
160 Note that an environment consists of a number of files that Berkeley DB
161 manages behind the scenes for you. When you first use an environment,
162 it needs to be explicitly created. This is done by including
163 "DB_CREATE" with the "Flags" parameter, described below.
164
165 Synopsis
166 $env = new BerkeleyDB::Env
167 [ -Home => $path, ]
168 [ -Server => $name, ]
169 [ -CacheSize => $number, ]
170 [ -Config => { name => value, name => value }, ]
171 [ -ErrFile => filename, ]
172 [ -MsgFile => filename, ]
173 [ -ErrPrefix => "string", ]
174 [ -Flags => number, ]
175 [ -SetFlags => bitmask, ]
176 [ -LockDetect => number, ]
177 [ -TxMax => number, ]
178 [ -LogConfig => number, ]
179 [ -MaxLockers => number, ]
180 [ -MaxLocks => number, ]
181 [ -MaxObjects => number, ]
182 [ -SharedMemKey => number, ]
183 [ -Verbose => boolean, ]
184 [ -BlobThreshold=> $number, ]
185 [ -BlobDir => directory, ]
186 [ -Encrypt => { Password => "string",
187 Flags => number }, ]
188
189 All the parameters to the BerkeleyDB::Env constructor are optional.
190
191 -Home
192 If present, this parameter should point to an existing directory.
193 Any files that aren't specified with an absolute path in the sub-
194 systems that are initialised by the BerkeleyDB::Env class will be
195 assumed to live in the Home directory.
196
197 For example, in the code fragment below the database "fred.db"
198 will be opened in the directory "/home/databases" because it was
199 specified as a relative path, but "joe.db" will be opened in
200 "/other" because it was part of an absolute path.
201
202 $env = new BerkeleyDB::Env
203 -Home => "/home/databases"
204 ...
205
206 $db1 = new BerkeleyDB::Hash
207 -Filename => "fred.db",
208 -Env => $env
209 ...
210
211 $db2 = new BerkeleyDB::Hash
212 -Filename => "/other/joe.db",
213 -Env => $env
214 ...
215
216 -Server
217 If present, this parameter should be the hostname of a server that
218 is running the Berkeley DB RPC server. All databases will be
219 accessed via the RPC server.
220
221 -Encrypt
222 If present, this parameter will enable encryption of all data
223 before it is written to the database. This parameters must be
224 given a hash reference. The format is shown below.
225
226 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES }
227
228 Valid values for the Flags are 0 or "DB_ENCRYPT_AES".
229
230 This option requires Berkeley DB 4.1 or better.
231
232 -Cachesize
233 If present, this parameter sets the size of the environments
234 shared memory buffer pool.
235
236 -TxMax
237 If present, this parameter sets the number of simultaneous
238 transactions that are allowed. Default 100. This default is
239 definitely too low for programs using the MVCC capabilities.
240
241 -LogConfig
242 If present, this parameter is used to configure log options.
243
244 -MaxLockers
245 If present, this parameter is used to configure the maximum number
246 of processes doing locking on the database. Default 1000.
247
248 -MaxLocks
249 If present, this parameter is used to configure the maximum number
250 of locks on the database. Default 1000. This is often lower than
251 required.
252
253 -MaxObjects
254 If present, this parameter is used to configure the maximum number
255 of locked objects. Default 1000. This is often lower than
256 required.
257
258 -SharedMemKey
259 If present, this parameter sets the base segment ID for the shared
260 memory region used by Berkeley DB.
261
262 This option requires Berkeley DB 3.1 or better.
263
264 Use "$env->get_shm_key($id)" to find out the base segment ID used
265 once the environment is open.
266
267 -ThreadCount
268 If present, this parameter declares the approximate number of
269 threads that will be used in the database environment. This
270 parameter is only necessary when the $env->failchk method will be
271 used. It does not actually set the maximum number of threads but
272 rather is used to determine memory sizing.
273
274 This option requires Berkeley DB 4.4 or better. It is only
275 supported on Unix/Linux.
276
277 -BlobThreshold
278 Sets the size threshold that will be used to decide when data is
279 stored as a BLOB. This option must be set for a blobs to be used.
280
281 This option requires Berkeley DB 6.0 or better.
282
283 -BlobDir
284 The directory where the BLOB objects are stored.
285
286 If not specified blob files are stores in the environment
287 directoy.
288
289 This option requires Berkeley DB 6.0 or better.
290
291 -Config
292 This is a variation on the "-Home" parameter, but it allows finer
293 control of where specific types of files will be stored.
294
295 The parameter expects a reference to a hash. Valid keys are:
296 DB_DATA_DIR, DB_LOG_DIR and DB_TMP_DIR
297
298 The code below shows an example of how it can be used.
299
300 $env = new BerkeleyDB::Env
301 -Config => { DB_DATA_DIR => "/home/databases",
302 DB_LOG_DIR => "/home/logs",
303 DB_TMP_DIR => "/home/tmp"
304 }
305 ...
306
307 -ErrFile
308 Expects a filename or filehandle. Any errors generated internally
309 by Berkeley DB will be logged to this file. A useful debug setting
310 is to open environments with either
311
312 -ErrFile => *STDOUT
313
314 or
315
316 -ErrFile => *STDERR
317
318 -ErrPrefix
319 Allows a prefix to be added to the error messages before they are
320 sent to -ErrFile.
321
322 -Flags
323 The Flags parameter specifies both which sub-systems to
324 initialise, as well as a number of environment-wide options. See
325 the Berkeley DB documentation for more details of these options.
326
327 Any of the following can be specified by OR'ing them:
328
329 DB_CREATE
330
331 If any of the files specified do not already exist, create them.
332
333 DB_INIT_CDB
334
335 Initialise the Concurrent Access Methods
336
337 DB_INIT_LOCK
338
339 Initialise the Locking sub-system.
340
341 DB_INIT_LOG
342
343 Initialise the Logging sub-system.
344
345 DB_INIT_MPOOL
346
347 Initialize the shared memory buffer pool subsystem. This subsystem
348 should be used whenever an application is using any Berkeley DB
349 access method.
350
351 DB_INIT_TXN
352
353 Initialize the transaction subsystem. This subsystem should be
354 used when recovery and atomicity of multiple operations are
355 important. The DB_INIT_TXN flag implies the DB_INIT_LOG flag.
356
357 DB_MPOOL_PRIVATE
358
359 Create a private memory pool; see memp_open. Ignored unless
360 DB_INIT_MPOOL is also specified.
361
362 DB_INIT_MPOOL is also specified.
363
364 DB_NOMMAP
365
366 Do not map this database into process memory.
367
368 DB_RECOVER
369
370 Run normal recovery on this environment before opening it for
371 normal use. If this flag is set, the DB_CREATE flag must also be
372 set since the regions will be removed and recreated.
373
374 The db_appinit function returns successfully if DB_RECOVER is
375 specified and no log files exist, so it is necessary to ensure all
376 necessary log files are present before running recovery.
377
378 DB_PRIVATE
379
380 DB_RECOVER_FATAL
381
382 Run catastrophic recovery on this environment before opening it
383 for normal use. If this flag is set, the DB_CREATE flag must also
384 be set since the regions will be removed and recreated.
385
386 The db_appinit function returns successfully if DB_RECOVER_FATAL
387 is specified and no log files exist, so it is necessary to ensure
388 all necessary log files are present before running recovery.
389
390 DB_THREAD
391
392 Ensure that handles returned by the Berkeley DB subsystems are
393 useable by multiple threads within a single process, i.e., that
394 the system is free-threaded.
395
396 DB_TXN_NOSYNC
397
398 On transaction commit, do not synchronously flush the log; see
399 txn_open. Ignored unless DB_INIT_TXN is also specified.
400
401 DB_USE_ENVIRON
402
403 The Berkeley DB process' environment may be permitted to specify
404 information to be used when naming files; see Berkeley DB File
405 Naming. As permitting users to specify which files are used can
406 create security problems, environment information will be used in
407 file naming for all users only if the DB_USE_ENVIRON flag is set.
408
409 DB_USE_ENVIRON_ROOT
410
411 The Berkeley DB process' environment may be permitted to specify
412 information to be used when naming files; see Berkeley DB File
413 Naming. As permitting users to specify which files are used can
414 create security problems, if the DB_USE_ENVIRON_ROOT flag is set,
415 environment information will be used for file naming only for
416 users with a user-ID matching that of the superuser (specifically,
417 users for whom the getuid(2) system call returns the user-ID 0).
418
419 -SetFlags
420 Calls ENV->set_flags with the supplied bitmask. Use this when you
421 need to make use of DB_ENV->set_flags before DB_ENV->open is
422 called.
423
424 Only valid when Berkeley DB 3.x or better is used.
425
426 -LockDetect
427 Specifies what to do when a lock conflict occurs. The value should
428 be one of
429
430 DB_LOCK_DEFAULT
431
432 Use the default policy as specified by db_deadlock.
433
434 DB_LOCK_OLDEST
435
436 Abort the oldest transaction.
437
438 DB_LOCK_RANDOM
439
440 Abort a random transaction involved in the deadlock.
441
442 DB_LOCK_YOUNGEST
443
444 Abort the youngest transaction.
445
446 -Verbose
447 Add extra debugging information to the messages sent to -ErrFile.
448
449 Methods
450 The environment class has the following methods:
451
452 $env->errPrefix("string") ;
453 This method is identical to the -ErrPrefix flag. It allows the
454 error prefix string to be changed dynamically.
455
456 $env->set_flags(bitmask, 1|0);
457 $txn = $env->TxnMgr()
458 Constructor for creating a TxnMgr object. See "TRANSACTIONS" for
459 more details of using transactions.
460
461 This method is deprecated. Access the transaction methods using
462 the txn_ methods below from the environment object directly.
463
464 $env->txn_begin()
465 TODO
466
467 $env->txn_stat()
468 TODO
469
470 $env->txn_checkpoint()
471 TODO
472
473 $env->status()
474 Returns the status of the last BerkeleyDB::Env method.
475
476 $env->DB_ENV()
477 Returns a pointer to the underlying DB_ENV data structure that
478 Berkeley DB uses.
479
480 $env->get_shm_key($id)
481 Writes the base segment ID for the shared memory region used by
482 the Berkeley DB environment into $id. Returns 0 on success.
483
484 This option requires Berkeley DB 4.2 or better.
485
486 Use the "-SharedMemKey" option when opening the environmet to set
487 the base segment ID.
488
489 $env->set_isalive()
490 Set the callback that determines if the thread of control,
491 identified by the pid and tid arguments, is still running. This
492 method should only be used in combination with $env->failchk.
493
494 This option requires Berkeley DB 4.4 or better.
495
496 $env->failchk($flags)
497 The $env->failchk method checks for threads of control (either a
498 true thread or a process) that have exited while manipulating
499 Berkeley DB library data structures, while holding a logical
500 database lock, or with an unresolved transaction (that is, a
501 transaction that was never aborted or committed).
502
503 If $env->failchk determines a thread of control exited while
504 holding database read locks, it will release those locks. If
505 $env->failchk determines a thread of control exited with an
506 unresolved transaction, the transaction will be aborted.
507
508 Applications calling the $env->failchk method must have already
509 called the $env->set_isalive method, on the same DB environment,
510 and must have configured their database environment using the
511 -ThreadCount flag. The ThreadCount flag cannot be used on an
512 environment that wasn't previously initialized with it.
513
514 This option requires Berkeley DB 4.4 or better.
515
516 $env->stat_print
517 Prints statistical information.
518
519 If the "MsgFile" option is specified the output will be sent to
520 the file. Otherwise output is sent to standard output.
521
522 This option requires Berkeley DB 4.3 or better.
523
524 $env->lock_stat_print
525 Prints locking subsystem statistics.
526
527 If the "MsgFile" option is specified the output will be sent to
528 the file. Otherwise output is sent to standard output.
529
530 This option requires Berkeley DB 4.3 or better.
531
532 $env->mutex_stat_print
533 Prints mutex subsystem statistics.
534
535 If the "MsgFile" option is specified the output will be sent to
536 the file. Otherwise output is sent to standard output.
537
538 This option requires Berkeley DB 4.4 or better.
539
540 $status = $env->get_blob_threshold($t1) ;
541 Sets the parameter $t1 to the threshold value (in bytes) that is
542 used to determine when a data item is stored as a Blob.
543
544 $status = $env->get_blob_dir($dir) ;
545 Sets the $dir parameter to the directory where blob files are
546 stored.
547
548 $env->set_timeout($timeout, $flags)
549 $env->status()
550 Returns the status of the last BerkeleyDB::Env method.
551
552 Examples
553 TODO.
554
556 $status = BerkeleyDB::db_remove [OPTIONS]
557 $status = BerkeleyDB::db_rename [OPTIONS]
558 $status = BerkeleyDB::db_verify [OPTIONS]
559
561 BerkeleyDB supports the following database formats:
562
563 BerkeleyDB::Hash
564 This database type allows arbitrary key/value pairs to be stored
565 in data files. This is equivalent to the functionality provided by
566 other hashing packages like DBM, NDBM, ODBM, GDBM, and SDBM.
567 Remember though, the files created using BerkeleyDB::Hash are not
568 compatible with any of the other packages mentioned.
569
570 A default hashing algorithm, which will be adequate for most
571 applications, is built into BerkeleyDB. If you do need to use your
572 own hashing algorithm it is possible to write your own in Perl and
573 have BerkeleyDB use it instead.
574
575 BerkeleyDB::Btree
576 The Btree format allows arbitrary key/value pairs to be stored in
577 a B+tree.
578
579 As with the BerkeleyDB::Hash format, it is possible to provide a
580 user defined Perl routine to perform the comparison of keys. By
581 default, though, the keys are stored in lexical order.
582
583 BerkeleyDB::Recno
584 TODO.
585
586 BerkeleyDB::Queue
587 TODO.
588
589 BerkeleyDB::Heap
590 TODO.
591
592 BerkeleyDB::Unknown
593 This isn't a database format at all. It is used when you want to
594 open an existing Berkeley DB database without having to know what
595 type is it.
596
597 Each of the database formats described above is accessed via a
598 corresponding BerkeleyDB class. These will be described in turn in the
599 next sections.
600
602 Equivalent to calling db_open with type DB_HASH in Berkeley DB 2.x and
603 calling db_create followed by DB->open with type DB_HASH in Berkeley DB
604 3.x or greater.
605
606 Two forms of constructor are supported:
607
608 $db = new BerkeleyDB::Hash
609 [ -Filename => "filename", ]
610 [ -Subname => "sub-database name", ]
611 [ -Flags => flags,]
612 [ -Property => flags,]
613 [ -Mode => number,]
614 [ -Cachesize => number,]
615 [ -Lorder => number,]
616 [ -Pagesize => number,]
617 [ -Env => $env,]
618 [ -Txn => $txn,]
619 [ -Encrypt => { Password => "string",
620 Flags => number }, ],
621 [ -BlobThreshold=> $number, ]
622 [ -BlobDir => directory, ]
623 # BerkeleyDB::Hash specific
624 [ -Ffactor => number,]
625 [ -Nelem => number,]
626 [ -Hash => code reference,]
627 [ -DupCompare => code reference,]
628
629 and this
630
631 [$db =] tie %hash, 'BerkeleyDB::Hash',
632 [ -Filename => "filename", ]
633 [ -Subname => "sub-database name", ]
634 [ -Flags => flags,]
635 [ -Property => flags,]
636 [ -Mode => number,]
637 [ -Cachesize => number,]
638 [ -Lorder => number,]
639 [ -Pagesize => number,]
640 [ -Env => $env,]
641 [ -Txn => $txn,]
642 [ -Encrypt => { Password => "string",
643 Flags => number }, ],
644 [ -BlobThreshold=> $number, ]
645 [ -BlobDir => directory, ]
646 # BerkeleyDB::Hash specific
647 [ -Ffactor => number,]
648 [ -Nelem => number,]
649 [ -Hash => code reference,]
650 [ -DupCompare => code reference,]
651
652 When the "tie" interface is used, reading from and writing to the
653 database is achieved via the tied hash. In this case the database
654 operates like a Perl associative array that happens to be stored on
655 disk.
656
657 In addition to the high-level tied hash interface, it is possible to
658 make use of the underlying methods provided by Berkeley DB
659
660 Options
661 In addition to the standard set of options (see "COMMON OPTIONS")
662 BerkeleyDB::Hash supports these options:
663
664 -Property
665 Used to specify extra flags when opening a database. The following
666 flags may be specified by bitwise OR'ing together one or more of
667 the following values:
668
669 DB_DUP
670
671 When creating a new database, this flag enables the storing of
672 duplicate keys in the database. If DB_DUPSORT is not specified as
673 well, the duplicates are stored in the order they are created in
674 the database.
675
676 DB_DUPSORT
677
678 Enables the sorting of duplicate keys in the database. Ignored if
679 DB_DUP isn't also specified.
680
681 -Ffactor
682 -Nelem
683 See the Berkeley DB documentation for details of these options.
684
685 -Hash
686 Allows you to provide a user defined hash function. If not
687 specified, a default hash function is used. Here is a template for
688 a user-defined hash function
689
690 sub hash
691 {
692 my ($data) = shift ;
693 ...
694 # return the hash value for $data
695 return $hash ;
696 }
697
698 tie %h, "BerkeleyDB::Hash",
699 -Filename => $filename,
700 -Hash => \&hash,
701 ...
702
703 See "" for an example.
704
705 -DupCompare
706 Used in conjunction with the DB_DUPOSRT flag.
707
708 sub compare
709 {
710 my ($key, $key2) = @_ ;
711 ...
712 # return 0 if $key1 eq $key2
713 # -1 if $key1 lt $key2
714 # 1 if $key1 gt $key2
715 return (-1 , 0 or 1) ;
716 }
717
718 tie %h, "BerkeleyDB::Hash",
719 -Filename => $filename,
720 -Property => DB_DUP|DB_DUPSORT,
721 -DupCompare => \&compare,
722 ...
723
724 Methods
725 BerkeleyDB::Hash only supports the standard database methods. See
726 "COMMON DATABASE METHODS".
727
728 A Simple Tied Hash Example
729 use strict ;
730 use BerkeleyDB ;
731 use vars qw( %h $k $v ) ;
732
733 my $filename = "fruit" ;
734 unlink $filename ;
735 tie %h, "BerkeleyDB::Hash",
736 -Filename => $filename,
737 -Flags => DB_CREATE
738 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
739
740 # Add a few key/value pairs to the file
741 $h{"apple"} = "red" ;
742 $h{"orange"} = "orange" ;
743 $h{"banana"} = "yellow" ;
744 $h{"tomato"} = "red" ;
745
746 # Check for existence of a key
747 print "Banana Exists\n\n" if $h{"banana"} ;
748
749 # Delete a key/value pair.
750 delete $h{"apple"} ;
751
752 # print the contents of the file
753 while (($k, $v) = each %h)
754 { print "$k -> $v\n" }
755
756 untie %h ;
757
758 here is the output:
759
760 Banana Exists
761
762 orange -> orange
763 tomato -> red
764 banana -> yellow
765
766 Note that the like ordinary associative arrays, the order of the keys
767 retrieved from a Hash database are in an apparently random order.
768
769 Another Simple Hash Example
770 Do the same as the previous example but not using tie.
771
772 use strict ;
773 use BerkeleyDB ;
774
775 my $filename = "fruit" ;
776 unlink $filename ;
777 my $db = new BerkeleyDB::Hash
778 -Filename => $filename,
779 -Flags => DB_CREATE
780 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
781
782 # Add a few key/value pairs to the file
783 $db->db_put("apple", "red") ;
784 $db->db_put("orange", "orange") ;
785 $db->db_put("banana", "yellow") ;
786 $db->db_put("tomato", "red") ;
787
788 # Check for existence of a key
789 print "Banana Exists\n\n" if $db->db_get("banana", $v) == 0;
790
791 # Delete a key/value pair.
792 $db->db_del("apple") ;
793
794 # print the contents of the file
795 my ($k, $v) = ("", "") ;
796 my $cursor = $db->db_cursor() ;
797 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
798 { print "$k -> $v\n" }
799
800 undef $cursor ;
801 undef $db ;
802
803 Duplicate keys
804 The code below is a variation on the examples above. This time the hash
805 has been inverted. The key this time is colour and the value is the
806 fruit name. The DB_DUP flag has been specified to allow duplicates.
807
808 use strict ;
809 use BerkeleyDB ;
810
811 my $filename = "fruit" ;
812 unlink $filename ;
813 my $db = new BerkeleyDB::Hash
814 -Filename => $filename,
815 -Flags => DB_CREATE,
816 -Property => DB_DUP
817 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
818
819 # Add a few key/value pairs to the file
820 $db->db_put("red", "apple") ;
821 $db->db_put("orange", "orange") ;
822 $db->db_put("green", "banana") ;
823 $db->db_put("yellow", "banana") ;
824 $db->db_put("red", "tomato") ;
825 $db->db_put("green", "apple") ;
826
827 # print the contents of the file
828 my ($k, $v) = ("", "") ;
829 my $cursor = $db->db_cursor() ;
830 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
831 { print "$k -> $v\n" }
832
833 undef $cursor ;
834 undef $db ;
835
836 here is the output:
837
838 orange -> orange
839 yellow -> banana
840 red -> apple
841 red -> tomato
842 green -> banana
843 green -> apple
844
845 Sorting Duplicate Keys
846 In the previous example, when there were duplicate keys, the values are
847 sorted in the order they are stored in. The code below is identical to
848 the previous example except the DB_DUPSORT flag is specified.
849
850 use strict ;
851 use BerkeleyDB ;
852
853 my $filename = "fruit" ;
854 unlink $filename ;
855 my $db = new BerkeleyDB::Hash
856 -Filename => $filename,
857 -Flags => DB_CREATE,
858 -Property => DB_DUP | DB_DUPSORT
859 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
860
861 # Add a few key/value pairs to the file
862 $db->db_put("red", "apple") ;
863 $db->db_put("orange", "orange") ;
864 $db->db_put("green", "banana") ;
865 $db->db_put("yellow", "banana") ;
866 $db->db_put("red", "tomato") ;
867 $db->db_put("green", "apple") ;
868
869 # print the contents of the file
870 my ($k, $v) = ("", "") ;
871 my $cursor = $db->db_cursor() ;
872 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
873 { print "$k -> $v\n" }
874
875 undef $cursor ;
876 undef $db ;
877
878 Notice that in the output below the duplicate values are sorted.
879
880 orange -> orange
881 yellow -> banana
882 red -> apple
883 red -> tomato
884 green -> apple
885 green -> banana
886
887 Custom Sorting Duplicate Keys
888 Another variation
889
890 TODO
891
892 Changing the hash
893 TODO
894
895 Using db_stat
896 TODO
897
899 Equivalent to calling db_open with type DB_BTREE in Berkeley DB 2.x and
900 calling db_create followed by DB->open with type DB_BTREE in Berkeley
901 DB 3.x or greater.
902
903 Two forms of constructor are supported:
904
905 $db = new BerkeleyDB::Btree
906 [ -Filename => "filename", ]
907 [ -Subname => "sub-database name", ]
908 [ -Flags => flags,]
909 [ -Property => flags,]
910 [ -Mode => number,]
911 [ -Cachesize => number,]
912 [ -Lorder => number,]
913 [ -Pagesize => number,]
914 [ -Env => $env,]
915 [ -Txn => $txn,]
916 [ -Encrypt => { Password => "string",
917 Flags => number }, ],
918 [ -BlobThreshold=> $number, ]
919 [ -BlobDir => directory, ]
920 # BerkeleyDB::Btree specific
921 [ -Minkey => number,]
922 [ -Compare => code reference,]
923 [ -DupCompare => code reference,]
924 [ -Prefix => code reference,]
925
926 and this
927
928 [$db =] tie %hash, 'BerkeleyDB::Btree',
929 [ -Filename => "filename", ]
930 [ -Subname => "sub-database name", ]
931 [ -Flags => flags,]
932 [ -Property => flags,]
933 [ -Mode => number,]
934 [ -Cachesize => number,]
935 [ -Lorder => number,]
936 [ -Pagesize => number,]
937 [ -Env => $env,]
938 [ -Txn => $txn,]
939 [ -Encrypt => { Password => "string",
940 Flags => number }, ],
941 [ -BlobThreshold=> $number, ]
942 [ -BlobDir => directory, ]
943 # BerkeleyDB::Btree specific
944 [ -Minkey => number,]
945 [ -Compare => code reference,]
946 [ -DupCompare => code reference,]
947 [ -Prefix => code reference,]
948
949 Options
950 In addition to the standard set of options (see "COMMON OPTIONS")
951 BerkeleyDB::Btree supports these options:
952
953 -Property
954 Used to specify extra flags when opening a database. The following
955 flags may be specified by bitwise OR'ing together one or more of
956 the following values:
957
958 DB_DUP
959
960 When creating a new database, this flag enables the storing of
961 duplicate keys in the database. If DB_DUPSORT is not specified as
962 well, the duplicates are stored in the order they are created in
963 the database.
964
965 DB_DUPSORT
966
967 Enables the sorting of duplicate keys in the database. Ignored if
968 DB_DUP isn't also specified.
969
970 Minkey
971 TODO
972
973 Compare
974 Allow you to override the default sort order used in the database.
975 See "Changing the sort order" for an example.
976
977 sub compare
978 {
979 my ($key, $key2) = @_ ;
980 ...
981 # return 0 if $key1 eq $key2
982 # -1 if $key1 lt $key2
983 # 1 if $key1 gt $key2
984 return (-1 , 0 or 1) ;
985 }
986
987 tie %h, "BerkeleyDB::Hash",
988 -Filename => $filename,
989 -Compare => \&compare,
990 ...
991
992 Prefix
993 sub prefix
994 {
995 my ($key, $key2) = @_ ;
996 ...
997 # return number of bytes of $key2 which are
998 # necessary to determine that it is greater than $key1
999 return $bytes ;
1000 }
1001
1002 tie %h, "BerkeleyDB::Hash",
1003 -Filename => $filename,
1004 -Prefix => \&prefix,
1005 ...
1006 =item DupCompare
1007
1008 sub compare
1009 {
1010 my ($key, $key2) = @_ ;
1011 ...
1012 # return 0 if $key1 eq $key2
1013 # -1 if $key1 lt $key2
1014 # 1 if $key1 gt $key2
1015 return (-1 , 0 or 1) ;
1016 }
1017
1018 tie %h, "BerkeleyDB::Hash",
1019 -Filename => $filename,
1020 -DupCompare => \&compare,
1021 ...
1022
1023 set_bt_compress
1024 Enabled compression of the btree data. The callback interface is
1025 not supported at present. Need Berkeley DB 4.8 or better.
1026
1027 Methods
1028 BerkeleyDB::Btree supports the following database methods. See also
1029 "COMMON DATABASE METHODS".
1030
1031 All the methods below return 0 to indicate success.
1032
1033 $status = $db->db_key_range($key, $less, $equal, $greater [, $flags])
1034 Given a key, $key, this method returns the proportion of keys less
1035 than $key in $less, the proportion equal to $key in $equal and the
1036 proportion greater than $key in $greater.
1037
1038 The proportion is returned as a double in the range 0.0 to 1.0.
1039
1040 A Simple Btree Example
1041 The code below is a simple example of using a btree database.
1042
1043 use strict ;
1044 use BerkeleyDB ;
1045
1046 my $filename = "tree" ;
1047 unlink $filename ;
1048 my %h ;
1049 tie %h, 'BerkeleyDB::Btree',
1050 -Filename => $filename,
1051 -Flags => DB_CREATE
1052 or die "Cannot open $filename: $! $BerkeleyDB::Error\n" ;
1053
1054 # Add a key/value pair to the file
1055 $h{'Wall'} = 'Larry' ;
1056 $h{'Smith'} = 'John' ;
1057 $h{'mouse'} = 'mickey' ;
1058 $h{'duck'} = 'donald' ;
1059
1060 # Delete
1061 delete $h{"duck"} ;
1062
1063 # Cycle through the keys printing them in order.
1064 # Note it is not necessary to sort the keys as
1065 # the btree will have kept them in order automatically.
1066 foreach (keys %h)
1067 { print "$_\n" }
1068
1069 untie %h ;
1070
1071 Here is the output from the code above. The keys have been sorted using
1072 Berkeley DB's default sorting algorithm.
1073
1074 Smith
1075 Wall
1076 mouse
1077
1078 Changing the sort order
1079 It is possible to supply your own sorting algorithm if the one that
1080 Berkeley DB used isn't suitable. The code below is identical to the
1081 previous example except for the case insensitive compare function.
1082
1083 use strict ;
1084 use BerkeleyDB ;
1085
1086 my $filename = "tree" ;
1087 unlink $filename ;
1088 my %h ;
1089 tie %h, 'BerkeleyDB::Btree',
1090 -Filename => $filename,
1091 -Flags => DB_CREATE,
1092 -Compare => sub { lc $_[0] cmp lc $_[1] }
1093 or die "Cannot open $filename: $!\n" ;
1094
1095 # Add a key/value pair to the file
1096 $h{'Wall'} = 'Larry' ;
1097 $h{'Smith'} = 'John' ;
1098 $h{'mouse'} = 'mickey' ;
1099 $h{'duck'} = 'donald' ;
1100
1101 # Delete
1102 delete $h{"duck"} ;
1103
1104 # Cycle through the keys printing them in order.
1105 # Note it is not necessary to sort the keys as
1106 # the btree will have kept them in order automatically.
1107 foreach (keys %h)
1108 { print "$_\n" }
1109
1110 untie %h ;
1111
1112 Here is the output from the code above.
1113
1114 mouse
1115 Smith
1116 Wall
1117
1118 There are a few point to bear in mind if you want to change the
1119 ordering in a BTREE database:
1120
1121 1. The new compare function must be specified when you create the
1122 database.
1123
1124 2. You cannot change the ordering once the database has been created.
1125 Thus you must use the same compare function every time you access
1126 the database.
1127
1128 Using db_stat
1129 TODO
1130
1132 Equivalent to calling db_open with type DB_RECNO in Berkeley DB 2.x and
1133 calling db_create followed by DB->open with type DB_RECNO in Berkeley
1134 DB 3.x or greater.
1135
1136 Two forms of constructor are supported:
1137
1138 $db = new BerkeleyDB::Recno
1139 [ -Filename => "filename", ]
1140 [ -Subname => "sub-database name", ]
1141 [ -Flags => flags,]
1142 [ -Property => flags,]
1143 [ -Mode => number,]
1144 [ -Cachesize => number,]
1145 [ -Lorder => number,]
1146 [ -Pagesize => number,]
1147 [ -Env => $env,]
1148 [ -Txn => $txn,]
1149 [ -Encrypt => { Password => "string",
1150 Flags => number }, ],
1151 # BerkeleyDB::Recno specific
1152 [ -Delim => byte,]
1153 [ -Len => number,]
1154 [ -Pad => byte,]
1155 [ -Source => filename,]
1156
1157 and this
1158
1159 [$db =] tie @arry, 'BerkeleyDB::Recno',
1160 [ -Filename => "filename", ]
1161 [ -Subname => "sub-database name", ]
1162 [ -Flags => flags,]
1163 [ -Property => flags,]
1164 [ -Mode => number,]
1165 [ -Cachesize => number,]
1166 [ -Lorder => number,]
1167 [ -Pagesize => number,]
1168 [ -Env => $env,]
1169 [ -Txn => $txn,]
1170 [ -Encrypt => { Password => "string",
1171 Flags => number }, ],
1172 # BerkeleyDB::Recno specific
1173 [ -Delim => byte,]
1174 [ -Len => number,]
1175 [ -Pad => byte,]
1176 [ -Source => filename,]
1177
1178 A Recno Example
1179 Here is a simple example that uses RECNO (if you are using a version of
1180 Perl earlier than 5.004_57 this example won't work -- see "Extra RECNO
1181 Methods" for a workaround).
1182
1183 use strict ;
1184 use BerkeleyDB ;
1185
1186 my $filename = "text" ;
1187 unlink $filename ;
1188
1189 my @h ;
1190 tie @h, 'BerkeleyDB::Recno',
1191 -Filename => $filename,
1192 -Flags => DB_CREATE,
1193 -Property => DB_RENUMBER
1194 or die "Cannot open $filename: $!\n" ;
1195
1196 # Add a few key/value pairs to the file
1197 $h[0] = "orange" ;
1198 $h[1] = "blue" ;
1199 $h[2] = "yellow" ;
1200
1201 push @h, "green", "black" ;
1202
1203 my $elements = scalar @h ;
1204 print "The array contains $elements entries\n" ;
1205
1206 my $last = pop @h ;
1207 print "popped $last\n" ;
1208
1209 unshift @h, "white" ;
1210 my $first = shift @h ;
1211 print "shifted $first\n" ;
1212
1213 # Check for existence of a key
1214 print "Element 1 Exists with value $h[1]\n" if $h[1] ;
1215
1216 untie @h ;
1217
1218 Here is the output from the script:
1219
1220 The array contains 5 entries
1221 popped black
1222 shifted white
1223 Element 1 Exists with value blue
1224 The last element is green
1225 The 2nd last element is yellow
1226
1228 Equivalent to calling db_create followed by DB->open with type DB_QUEUE
1229 in Berkeley DB 3.x or greater. This database format isn't available if
1230 you use Berkeley DB 2.x.
1231
1232 Two forms of constructor are supported:
1233
1234 $db = new BerkeleyDB::Queue
1235 [ -Filename => "filename", ]
1236 [ -Subname => "sub-database name", ]
1237 [ -Flags => flags,]
1238 [ -Property => flags,]
1239 [ -Mode => number,]
1240 [ -Cachesize => number,]
1241 [ -Lorder => number,]
1242 [ -Pagesize => number,]
1243 [ -Env => $env,]
1244 [ -Txn => $txn,]
1245 [ -Encrypt => { Password => "string",
1246 Flags => number }, ],
1247 # BerkeleyDB::Queue specific
1248 [ -Len => number,]
1249 [ -Pad => byte,]
1250 [ -ExtentSize => number, ]
1251
1252 and this
1253
1254 [$db =] tie @arry, 'BerkeleyDB::Queue',
1255 [ -Filename => "filename", ]
1256 [ -Subname => "sub-database name", ]
1257 [ -Flags => flags,]
1258 [ -Property => flags,]
1259 [ -Mode => number,]
1260 [ -Cachesize => number,]
1261 [ -Lorder => number,]
1262 [ -Pagesize => number,]
1263 [ -Env => $env,]
1264 [ -Txn => $txn,]
1265 [ -Encrypt => { Password => "string",
1266 Flags => number }, ],
1267 # BerkeleyDB::Queue specific
1268 [ -Len => number,]
1269 [ -Pad => byte,]
1270
1272 Equivalent to calling db_create followed by DB->open with type DB_HEAP
1273 in Berkeley DB 5.2.x or greater. This database format isn't available
1274 if you use an older version of Berkeley DB.
1275
1276 One form of constructor is supported:
1277
1278 $db = new BerkeleyDB::Heap
1279 [ -Filename => "filename", ]
1280 [ -Subname => "sub-database name", ]
1281 [ -Flags => flags,]
1282 [ -Property => flags,]
1283 [ -Mode => number,]
1284 [ -Cachesize => number,]
1285 [ -Lorder => number,]
1286 [ -Pagesize => number,]
1287 [ -Env => $env,]
1288 [ -Txn => $txn,]
1289 [ -Encrypt => { Password => "string",
1290 Flags => number }, ],
1291 [ -BlobThreshold=> $number, ]
1292 [ -BlobDir => directory, ]
1293 # BerkeleyDB::Heap specific
1294 [ -HeapSize => number, ]
1295 [ -HeapSizeGb => number, ]
1296
1298 This class is used to open an existing database.
1299
1300 Equivalent to calling db_open with type DB_UNKNOWN in Berkeley DB 2.x
1301 and calling db_create followed by DB->open with type DB_UNKNOWN in
1302 Berkeley DB 3.x or greater.
1303
1304 The constructor looks like this:
1305
1306 $db = new BerkeleyDB::Unknown
1307 [ -Filename => "filename", ]
1308 [ -Subname => "sub-database name", ]
1309 [ -Flags => flags,]
1310 [ -Property => flags,]
1311 [ -Mode => number,]
1312 [ -Cachesize => number,]
1313 [ -Lorder => number,]
1314 [ -Pagesize => number,]
1315 [ -Env => $env,]
1316 [ -Txn => $txn,]
1317 [ -Encrypt => { Password => "string",
1318 Flags => number }, ],
1319
1320 An example
1322 All database access class constructors support the common set of
1323 options defined below. All are optional.
1324
1325 -Filename
1326 The database filename. If no filename is specified, a temporary
1327 file will be created and removed once the program terminates.
1328
1329 -Subname
1330 Specifies the name of the sub-database to open. This option is
1331 only valid if you are using Berkeley DB 3.x or greater.
1332
1333 -Flags
1334 Specify how the database will be opened/created. The valid flags
1335 are:
1336
1337 DB_CREATE
1338
1339 Create any underlying files, as necessary. If the files do not
1340 already exist and the DB_CREATE flag is not specified, the call
1341 will fail.
1342
1343 DB_NOMMAP
1344
1345 Not supported by BerkeleyDB.
1346
1347 DB_RDONLY
1348
1349 Opens the database in read-only mode.
1350
1351 DB_THREAD
1352
1353 Not supported by BerkeleyDB.
1354
1355 DB_TRUNCATE
1356
1357 If the database file already exists, remove all the data before
1358 opening it.
1359
1360 -Mode
1361 Determines the file protection when the database is created.
1362 Defaults to 0666.
1363
1364 -Cachesize
1365 -Lorder
1366 -Pagesize
1367 -Env When working under a Berkeley DB environment, this parameter
1368
1369 Defaults to no environment.
1370
1371 -Encrypt
1372 If present, this parameter will enable encryption of all data
1373 before it is written to the database. This parameters must be
1374 given a hash reference. The format is shown below.
1375
1376 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES }
1377
1378 Valid values for the Flags are 0 or "DB_ENCRYPT_AES".
1379
1380 This option requires Berkeley DB 4.1 or better.
1381
1382 -Txn TODO.
1383
1385 All the database interfaces support the common set of methods defined
1386 below.
1387
1388 All the methods below return 0 to indicate success.
1389
1390 $env = $db->Env();
1391 Returns the environment object the database is associated with or
1392 "undef" when no environment was used when opening the database.
1393
1394 $status = $db->db_get($key, $value [, $flags])
1395 Given a key ($key) this method reads the value associated with it from
1396 the database. If it exists, the value read from the database is
1397 returned in the $value parameter.
1398
1399 The $flags parameter is optional. If present, it must be set to one of
1400 the following values:
1401
1402 DB_GET_BOTH
1403 When the DB_GET_BOTH flag is specified, db_get checks for the
1404 existence of both the $key and $value in the database.
1405
1406 DB_SET_RECNO
1407 TODO.
1408
1409 In addition, the following value may be set by bitwise OR'ing it into
1410 the $flags parameter:
1411
1412 DB_RMW
1413 TODO
1414
1415 The variant "db_pget" allows you to query a secondary database:
1416
1417 $status = $sdb->db_pget($skey, $pkey, $value);
1418
1419 using the key $skey in the secondary db to lookup $pkey and $value from
1420 the primary db.
1421
1422 $status = $db->db_exists($key [, $flags])
1423 This method checks for the existence of the given key ($key), but does
1424 not read the value. If the key is not found, db_exists will return
1425 DB_NOTFOUND. Requires BDB 4.6 or better.
1426
1427 $status = $db->db_put($key, $value [, $flags])
1428 Stores a key/value pair in the database.
1429
1430 The $flags parameter is optional. If present it must be set to one of
1431 the following values:
1432
1433 DB_APPEND
1434 This flag is only applicable when accessing a BerkeleyDB::Recno
1435 database.
1436
1437 TODO.
1438
1439 DB_NOOVERWRITE
1440 If this flag is specified and $key already exists in the database,
1441 the call to db_put will return DB_KEYEXIST.
1442
1443 $status = $db->db_del($key [, $flags])
1444 Deletes a key/value pair in the database associated with $key. If
1445 duplicate keys are enabled in the database, db_del will delete all
1446 key/value pairs with key $key.
1447
1448 The $flags parameter is optional and is currently unused.
1449
1450 $status = $env->stat_print([$flags])
1451 Prints statistical information.
1452
1453 If the "MsgFile" option is specified the output will be sent to the
1454 file. Otherwise output is sent to standard output.
1455
1456 This option requires Berkeley DB 4.3 or better.
1457
1458 $status = $db->db_sync()
1459 If any parts of the database are in memory, write them to the database.
1460
1461 $cursor = $db->db_cursor([$flags])
1462 Creates a cursor object. This is used to access the contents of the
1463 database sequentially. See CURSORS for details of the methods available
1464 when working with cursors.
1465
1466 The $flags parameter is optional. If present it must be set to one of
1467 the following values:
1468
1469 DB_RMW
1470 TODO.
1471
1472 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ;
1473 TODO
1474
1475 ($flag, $old_offset, $old_length) = $db->partial_clear() ;
1476 TODO
1477
1478 $db->byteswapped()
1479 TODO
1480
1481 $status = $db->get_blob_threshold($t1) ;
1482 Sets the parameter $t1 to the threshold value (in bytes) that is used
1483 to determine when a data item is stored as a Blob.
1484
1485 $status = $db->get_blob_dir($dir) ;
1486 Sets the $dir parameter to the directory where blob files are stored.
1487
1488 $db->type()
1489 Returns the type of the database. The possible return code are DB_HASH
1490 for a BerkeleyDB::Hash database, DB_BTREE for a BerkeleyDB::Btree
1491 database and DB_RECNO for a BerkeleyDB::Recno database. This method is
1492 typically used when a database has been opened with
1493 BerkeleyDB::Unknown.
1494
1495 $bool = $env->cds_enabled();
1496 Returns true if the Berkeley DB environment $env has been opened on CDS
1497 mode.
1498
1499 $bool = $db->cds_enabled();
1500 Returns true if the database $db has been opened on CDS mode.
1501
1502 $lock = $db->cds_lock();
1503 Creates a CDS write lock object $lock.
1504
1505 It is a fatal error to attempt to create a cds_lock if the Berkeley DB
1506 environment has not been opened in CDS mode.
1507
1508 $lock->cds_unlock();
1509 Removes a CDS lock. The destruction of the CDS lock object
1510 automatically calls this method.
1511
1512 Note that if multiple CDS lock objects are created, the underlying
1513 write lock will not be released until all CDS lock objects are either
1514 explicitly unlocked with this method, or the CDS lock objects have been
1515 destroyed.
1516
1517 $ref = $db->db_stat()
1518 Returns a reference to an associative array containing information
1519 about the database. The keys of the associative array correspond
1520 directly to the names of the fields defined in the Berkeley DB
1521 documentation. For example, in the DB documentation, the field
1522 bt_version stores the version of the Btree database. Assuming you
1523 called db_stat on a Btree database the equivalent field would be
1524 accessed as follows:
1525
1526 $version = $ref->{'bt_version'} ;
1527
1528 If you are using Berkeley DB 3.x or better, this method will work will
1529 all database formats. When DB 2.x is used, it only works with
1530 BerkeleyDB::Btree.
1531
1532 $status = $db->status()
1533 Returns the status of the last $db method called.
1534
1535 $status = $db->truncate($count)
1536 Truncates the database and returns the number or records deleted in
1537 $count.
1538
1539 $status = $db->compact($start, $stop, $c_data, $flags, $end);
1540 Compacts the database $db.
1541
1542 All the parameters are optional - if only want to make use of some of
1543 them, use "undef" for those you don't want. Trailing unused parameters
1544 can be omitted. For example, if you only want to use the $c_data
1545 parameter to set the "compact_fillpercent", write you code like this
1546
1547 my %hash;
1548 $hash{compact_fillpercent} = 50;
1549 $db->compact(undef, undef, \%hash);
1550
1551 The parameters operate identically to the C equivalent of this method.
1552 The $c_data needs a bit of explanation - it must be a hash reference.
1553 The values of the following keys can be set before calling "compact"
1554 and will affect the operation of the compaction.
1555
1556 · compact_fillpercent
1557
1558 · compact_timeout
1559
1560 The following keys, along with associated values, will be created in
1561 the hash reference if the "compact" operation was successful.
1562
1563 · compact_deadlock
1564
1565 · compact_levels
1566
1567 · compact_pages_free
1568
1569 · compact_pages_examine
1570
1571 · compact_pages_truncated
1572
1573 You need to be running Berkeley DB 4.4 or better if you want to make
1574 use of "compact".
1575
1576 $status = $db->associate($secondary, \&key_callback)
1577 Associate $db with the secondary DB $secondary
1578
1579 New key/value pairs inserted to the database will be passed to the
1580 callback which must set its third argument to the secondary key to
1581 allow lookup. If an array reference is set multiple keys secondary keys
1582 will be associated with the primary database entry.
1583
1584 Data may be retrieved fro the secondary database using "db_pget" to
1585 also obtain the primary key.
1586
1587 Secondary databased are maintained automatically.
1588
1589 $status = $db->associate_foreign($secondary, callback, $flags)
1590 Associate a foreign key database $db with the secondary DB $secondary.
1591
1592 The second parameter must be a reference to a sub or "undef".
1593
1594 The $flags parameter must be either "DB_FOREIGN_CASCADE",
1595 "DB_FOREIGN_ABORT" or "DB_FOREIGN_NULLIFY".
1596
1597 When the flags parameter is "DB_FOREIGN_NULLIFY" the second parameter
1598 is a reference to a sub of the form
1599
1600 sub foreign_cb
1601 {
1602 my $key = \$_[0];
1603 my $value = \$_[1];
1604 my $foreignkey = \$_[2];
1605 my $changed = \$_[3] ;
1606
1607 # for ... set $$value and set $$changed to 1
1608
1609 return 0;
1610 }
1611
1612 $foreign_db->associate_foreign($secondary, \&foreign_cb, DB_FOREIGN_NULLIFY);
1613
1615 A cursor is used whenever you want to access the contents of a database
1616 in sequential order. A cursor object is created with the "db_cursor"
1617
1618 A cursor object has the following methods available:
1619
1620 $newcursor = $cursor->c_dup($flags)
1621 Creates a duplicate of $cursor. This method needs Berkeley DB 3.0.x or
1622 better.
1623
1624 The $flags parameter is optional and can take the following value:
1625
1626 DB_POSITION
1627 When present this flag will position the new cursor at the same
1628 place as the existing cursor.
1629
1630 $status = $cursor->c_get($key, $value, $flags)
1631 Reads a key/value pair from the database, returning the data in $key
1632 and $value. The key/value pair actually read is controlled by the
1633 $flags parameter, which can take one of the following values:
1634
1635 DB_FIRST
1636 Set the cursor to point to the first key/value pair in the
1637 database. Return the key/value pair in $key and $value.
1638
1639 DB_LAST
1640 Set the cursor to point to the last key/value pair in the
1641 database. Return the key/value pair in $key and $value.
1642
1643 DB_NEXT
1644 If the cursor is already pointing to a key/value pair, it will be
1645 incremented to point to the next key/value pair and return its
1646 contents.
1647
1648 If the cursor isn't initialised, DB_NEXT works just like DB_FIRST.
1649
1650 If the cursor is already positioned at the last key/value pair,
1651 c_get will return DB_NOTFOUND.
1652
1653 DB_NEXT_DUP
1654 This flag is only valid when duplicate keys have been enabled in a
1655 database. If the cursor is already pointing to a key/value pair
1656 and the key of the next key/value pair is identical, the cursor
1657 will be incremented to point to it and their contents returned.
1658
1659 DB_PREV
1660 If the cursor is already pointing to a key/value pair, it will be
1661 decremented to point to the previous key/value pair and return its
1662 contents.
1663
1664 If the cursor isn't initialised, DB_PREV works just like DB_LAST.
1665
1666 If the cursor is already positioned at the first key/value pair,
1667 c_get will return DB_NOTFOUND.
1668
1669 DB_CURRENT
1670 If the cursor has been set to point to a key/value pair, return
1671 their contents. If the key/value pair referenced by the cursor
1672 has been deleted, c_get will return DB_KEYEMPTY.
1673
1674 DB_SET
1675 Set the cursor to point to the key/value pair referenced by $key
1676 and return the value in $value.
1677
1678 DB_SET_RANGE
1679 This flag is a variation on the DB_SET flag. As well as returning
1680 the value, it also returns the key, via $key. When used with a
1681 BerkeleyDB::Btree database the key matched by c_get will be the
1682 shortest key (in length) which is greater than or equal to the key
1683 supplied, via $key. This allows partial key searches. See ??? for
1684 an example of how to use this flag.
1685
1686 DB_GET_BOTH
1687 Another variation on DB_SET. This one returns both the key and the
1688 value.
1689
1690 DB_SET_RECNO
1691 TODO.
1692
1693 DB_GET_RECNO
1694 TODO.
1695
1696 In addition, the following value may be set by bitwise OR'ing it into
1697 the $flags parameter:
1698
1699 DB_RMW
1700 TODO.
1701
1702 $status = $cursor->c_put($key, $value, $flags)
1703 Stores the key/value pair in the database. The position that the data
1704 is stored in the database is controlled by the $flags parameter, which
1705 must take one of the following values:
1706
1707 DB_AFTER
1708 When used with a Btree or Hash database, a duplicate of the key
1709 referenced by the current cursor position will be created and the
1710 contents of $value will be associated with it - $key is ignored.
1711 The new key/value pair will be stored immediately after the
1712 current cursor position. Obviously the database has to have been
1713 opened with DB_DUP.
1714
1715 When used with a Recno ... TODO
1716
1717 DB_BEFORE
1718 When used with a Btree or Hash database, a duplicate of the key
1719 referenced by the current cursor position will be created and the
1720 contents of $value will be associated with it - $key is ignored.
1721 The new key/value pair will be stored immediately before the
1722 current cursor position. Obviously the database has to have been
1723 opened with DB_DUP.
1724
1725 When used with a Recno ... TODO
1726
1727 DB_CURRENT
1728 If the cursor has been initialised, replace the value of the
1729 key/value pair stored in the database with the contents of $value.
1730
1731 DB_KEYFIRST
1732 Only valid with a Btree or Hash database. This flag is only really
1733 used when duplicates are enabled in the database and sorted
1734 duplicates haven't been specified. In this case the key/value
1735 pair will be inserted as the first entry in the duplicates for the
1736 particular key.
1737
1738 DB_KEYLAST
1739 Only valid with a Btree or Hash database. This flag is only really
1740 used when duplicates are enabled in the database and sorted
1741 duplicates haven't been specified. In this case the key/value
1742 pair will be inserted as the last entry in the duplicates for the
1743 particular key.
1744
1745 $status = $cursor->c_del([$flags])
1746 This method deletes the key/value pair associated with the current
1747 cursor position. The cursor position will not be changed by this
1748 operation, so any subsequent cursor operation must first initialise the
1749 cursor to point to a valid key/value pair.
1750
1751 If the key/value pair associated with the cursor have already been
1752 deleted, c_del will return DB_KEYEMPTY.
1753
1754 The $flags parameter is not used at present.
1755
1756 $status = $cursor->c_count($cnt [, $flags])
1757 Stores the number of duplicates at the current cursor position in $cnt.
1758
1759 The $flags parameter is not used at present. This method needs Berkeley
1760 DB 3.1 or better.
1761
1762 $status = $cursor->status()
1763 Returns the status of the last cursor method as a dual type.
1764
1765 $status = $cursor->c_pget() ;
1766 See "db_pget"
1767
1768 $status = $cursor->c_close()
1769 Closes the cursor $cursor.
1770
1771 $stream = $cursor->db_stream($flags);
1772 Create a BerkeleyDB::DbStream object to read the blob at the current
1773 cursor location. See Blob for details of the the BerkeleyDB::DbStream
1774 object.
1775
1776 $flags must be one or more of the following OR'ed together
1777
1778 DB_STREAM_READ DB_STREAM_WRITE DB_STREAM_SYNC_WRITE
1779
1780 For full information on the flags refer to the Berkeley DB Reference
1781 Guide.
1782
1783 Cursor Examples
1784 TODO
1785
1786 Iterating from first to last, then in reverse.
1787
1788 examples of each of the flags.
1789
1791 Join support for BerkeleyDB is in progress. Watch this space.
1792
1793 TODO
1794
1796 Transactions are created using the "txn_begin" method on
1797 BerkeleyDB::Env:
1798
1799 my $txn = $env->txn_begin;
1800
1801 If this is a nested transaction, supply the parent transaction as an
1802 argument:
1803
1804 my $child_txn = $env->txn_begin($parent_txn);
1805
1806 Then in order to work with the transaction, you must set it as the
1807 current transaction on the database handles you want to work with:
1808
1809 $db->Txn($txn);
1810
1811 Or for multiple handles:
1812
1813 $txn->Txn(@handles);
1814
1815 The current transaction is given by BerkeleyDB each time to the various
1816 BDB operations. In the C api it is required explicitly as an argument
1817 to every operation.
1818
1819 To commit a transaction call the "commit" method on it:
1820
1821 $txn->txn_commit;
1822
1823 and to roll back call abort:
1824
1825 $txn->txn_abort
1826
1827 After committing or aborting a child transaction you need to set the
1828 active transaction again using "Txn".
1829
1831 Blob support is available in Berkeley DB starting with version 6.0.
1832 Refer to the section "Blob Support" in the Berkeley DB Programmer
1833 Reference for details of how Blob supports works.
1834
1835 A Blob is access via a BerkeleyDBB::DbStream object. This is created
1836 via a cursor object.
1837
1838 # Note - error handling not shown below.
1839
1840 # Set the key we want
1841 my $k = "some key";
1842
1843 # Don't want the value retrieved by the cursor,
1844 # so use partial_set to make sure no data is retrieved.
1845 my $v = '';
1846 $cursor->partial_set(0,0) ;
1847 $cursor->c_get($k, $v, DB_SET) ;
1848 $cursor->partial_clear() ;
1849
1850 # Now create a stream to the blob
1851 my $stream = $cursor->db_stream(DB_STREAM_WRITE) ;
1852
1853 # get the size of the blob
1854 $stream->size(my $s) ;
1855
1856 # Read the first 1k of data from the blob
1857 my $data ;
1858 $stream->read($data, 0, 1024);
1859
1860 A BerkeleyDB::DbStream object has the following methods available:
1861
1862 $status = $stream->size($SIZE);
1863 Outputs the length of the Blob in the $SIZE parameter.
1864
1865 $status = $stream->read($data, $offset, $size);
1866 Read from the blob. $offset is the number of bytes from the start of
1867 the blob to read from. $size if the number of bytes to read.
1868
1869 $status = $stream->write($data, $offset, $flags);
1870 Write $data to the blob, starting at offset $offset.
1871
1872 Example
1873
1874 Below is an example of how to walk through a database when you don't
1875 know beforehand which entries are blobs and which are not.
1876
1877 while (1)
1878 {
1879 my $k = '';
1880 my $v = '';
1881 $cursor->partial_set(0,0) ;
1882 my $status = $cursor->c_get($k, $v, DB_NEXT) ;
1883 $cursor->partial_clear();
1884
1885 last if $status != 0 ;
1886
1887 my $stream = $cursor->db_stream(DB_STREAM_WRITE);
1888
1889 if (defined $stream)
1890 {
1891 # It's a Blob
1892 $stream->size(my $s) ;
1893 }
1894 else
1895 {
1896 # Not a Blob
1897 $cursor->c_get($k, $v, DB_CURRENT) ;
1898 }
1899 }
1900
1902 The Berkeley DB Concurrent Data Store (CDS) is a lightweight locking
1903 mechanism that is useful in scenarios where transactions are overkill.
1904
1905 What is CDS?
1906 The Berkeley DB CDS interface is a simple lightweight locking mechanism
1907 that allows safe concurrent access to Berkeley DB databases. Your
1908 application can have multiple reader and write processes, but Berkeley
1909 DB will arrange it so that only one process can have a write lock
1910 against the database at a time, i.e. multiple processes can read from a
1911 database concurrently, but all write processes will be serialised.
1912
1913 Should I use it?
1914 Whilst this simple locking model is perfectly adequate for some
1915 applications, it will be too restrictive for others. Before deciding on
1916 using CDS mode, you need to be sure that it is suitable for the
1917 expected behaviour of your application.
1918
1919 The key features of this model are
1920
1921 · All writes operations are serialised.
1922
1923 · A write operation will block until all reads have finished.
1924
1925 There are a few of the attributes of your application that you need to
1926 be aware of before choosing to use CDS.
1927
1928 Firstly, if you application needs either recoverability or transaction
1929 support, then CDS will not be suitable.
1930
1931 Next what is the ratio of read operation to write operations will your
1932 application have?
1933
1934 If it is carrying out mostly read operations, and very few writes, then
1935 CDS may be appropriate.
1936
1937 What is the expected throughput of reads/writes in your application?
1938
1939 If you application does 90% writes and 10% reads, but on average you
1940 only have a transaction every 5 seconds, then the fact that all writes
1941 are serialised will not matter, because there will hardly ever be
1942 multiple writes processes blocking.
1943
1944 In summary CDS mode may be appropriate for your application if it
1945 performs mostly reads and very few writes or there is a low throughput.
1946 Also, if you do not need to be able to roll back a series of database
1947 operations if an error occurs, then CDS is ok.
1948
1949 If any of these is not the case you will need to use Berkeley DB
1950 transactions. That is outside the scope of this document.
1951
1952 Locking Used
1953 Berkeley DB implements CDS mode using two kinds of lock behind the
1954 scenes - namely read locks and write locks. A read lock allows multiple
1955 processes to access the database for reading at the same time. A write
1956 lock will only get access to the database when there are no read or
1957 write locks active. The write lock will block until the process
1958 holding the lock releases it.
1959
1960 Multiple processes with read locks can all access the database at the
1961 same time as long as no process has a write lock. A process with a
1962 write lock can only access the database if there are no other active
1963 read or write locks.
1964
1965 The majority of the time the Berkeley DB CDS mode will handle all
1966 locking without your application having to do anything. There are a
1967 couple of exceptions you need to be aware of though - these will be
1968 discussed in "Safely Updating Records" and "Implicit Cursors" below.
1969
1970 A Berkeley DB Cursor (created with "$db->db_cursor") will by hold a
1971 lock on the database until it is either explicitly closed or destroyed.
1972 This means the lock has the potential to be long lived.
1973
1974 By default Berkeley DB cursors create a read lock, but it is possible
1975 to create a cursor that holds a write lock, thus
1976
1977 $cursor = $db->db_cursor(DB_WRITECURSOR);
1978
1979 Whilst either a read or write cursor is active, it will block any other
1980 processes that wants to write to the database.
1981
1982 To avoid blocking problems, only keep cursors open as long as they are
1983 needed. The same is true when you use the "cursor" method or the
1984 "cds_lock" method.
1985
1986 For full information on CDS see the "Berkeley DB Concurrent Data Store
1987 applications" section in the Berkeley DB Reference Guide.
1988
1989 Opening a database for CDS
1990 Here is the typical signature that is used when opening a database in
1991 CDS mode.
1992
1993 use BerkeleyDB ;
1994
1995 my $env = new BerkeleyDB::Env
1996 -Home => "./home" ,
1997 -Flags => DB_CREATE| DB_INIT_CDB | DB_INIT_MPOOL
1998 or die "cannot open environment: $BerkeleyDB::Error\n";
1999
2000 my $db = new BerkeleyDB::Hash
2001 -Filename => 'test1.db',
2002 -Flags => DB_CREATE,
2003 -Env => $env
2004 or die "cannot open database: $BerkeleyDB::Error\n";
2005
2006 or this, if you use the tied interface
2007
2008 tie %hash, "BerkeleyDB::Hash",
2009 -Filename => 'test2.db',
2010 -Flags => DB_CREATE,
2011 -Env => $env
2012 or die "cannot open database: $BerkeleyDB::Error\n";
2013
2014 The first thing to note is that you MUST always use a Berkeley DB
2015 environment if you want to use locking with Berkeley DB.
2016
2017 Remember, that apart from the actual database files you explicitly
2018 create yourself, Berkeley DB will create a few behind the scenes to
2019 handle locking - they usually have names like "__db.001". It is
2020 therefore a good idea to use the "-Home" option, unless you are happy
2021 for all these files to be written in the current directory.
2022
2023 Next, remember to include the "DB_CREATE" flag when opening the
2024 environment for the first time. A common mistake is to forget to add
2025 this option and then wonder why the application doesn't work.
2026
2027 Finally, it is vital that all processes that are going to access the
2028 database files use the same Berkeley DB environment.
2029
2030 Safely Updating a Record
2031 One of the main gotchas when using CDS is if you want to update a
2032 record in a database, i.e. you want to retrieve a record from a
2033 database, modify it in some way and put it back in the database.
2034
2035 For example, say you are writing a web application and you want to keep
2036 a record of the number of times your site is accessed in a Berkeley DB
2037 database. So your code will have a line of code like this (assume, of
2038 course, that %hash has been tied to a Berkeley DB database):
2039
2040 $hash{Counter} ++ ;
2041
2042 That may look innocent enough, but there is a race condition lurking in
2043 there. If I rewrite the line of code using the low-level Berkeley DB
2044 API, which is what will actually be executed, the race condition may be
2045 more apparent:
2046
2047 $db->db_get("Counter", $value);
2048 ++ $value ;
2049 $db->db_put("Counter", $value);
2050
2051 Consider what happens behind the scenes when you execute the commands
2052 above. Firstly, the existing value for the key "Counter" is fetched
2053 from the database using "db_get". A read lock will be used for this
2054 part of the update. The value is then incremented, and the new value
2055 is written back to the database using "db_put". This time a write lock
2056 will be used.
2057
2058 Here's the problem - there is nothing to stop two (or more) processes
2059 executing the read part at the same time. Remember multiple processes
2060 can hold a read lock on the database at the same time. So both will
2061 fetch the same value, let's say 7, from the database. Both increment
2062 the value to 8 and attempt to write it to the database. Berkeley DB
2063 will ensure that only one of the processes gets a write lock, while the
2064 other will be blocked. So the process that happened to get the write
2065 lock will store the value 8 to the database and release the write lock.
2066 Now the other process will be unblocked, and it too will write the
2067 value 8 to the database. The result, in this example, is we have missed
2068 a hit in the counter.
2069
2070 To deal with this kind of scenario, you need to make the update atomic.
2071 A convenience method, called "cds_lock", is supplied with the
2072 BerkeleyDB module for this purpose. Using "cds_lock", the counter
2073 update code can now be rewritten thus:
2074
2075 my $lk = $dbh->cds_lock() ;
2076 $hash{Counter} ++ ;
2077 $lk->cds_unlock;
2078
2079 or this, where scoping is used to limit the lifetime of the lock object
2080
2081 {
2082 my $lk = $dbh->cds_lock() ;
2083 $hash{Counter} ++ ;
2084 }
2085
2086 Similarly, "cds_lock" can be used with the native Berkeley DB API
2087
2088 my $lk = $dbh->cds_lock() ;
2089 $db->db_get("Counter", $value);
2090 ++ $value ;
2091 $db->db_put("Counter", $value);
2092 $lk->unlock;
2093
2094 The "cds_lock" method will ensure that the current process has
2095 exclusive access to the database until the lock is either explicitly
2096 released, via the "$lk->cds_unlock()" or by the lock object being
2097 destroyed.
2098
2099 If you are interested, all that "cds_lock" does is open a "write"
2100 cursor. This has the useful side-effect of holding a write-lock on the
2101 database until the cursor is deleted. This is how you create a write-
2102 cursor
2103
2104 $cursor = $db->db_cursor(DB_WRITECURSOR);
2105
2106 If you have instantiated multiple "cds_lock" objects for one database
2107 within a single process, that process will hold a write-lock on the
2108 database until ALL "cds_lock" objects have been destroyed.
2109
2110 As with all write-cursors, you should try to limit the scope of the
2111 "cds_lock" to as short a time as possible. Remember the complete
2112 database will be locked to other process whilst the write lock is in
2113 place.
2114
2115 Cannot write with a read cursor while a write cursor is active
2116 This issue is easier to demonstrate with an example, so consider the
2117 code below. The intention of the code is to increment the values of all
2118 the elements in a database by one.
2119
2120 # Assume $db is a database opened in a CDS environment.
2121
2122 # Create a write-lock
2123 my $lock = $db->db_cursor(DB_WRITECURSOR);
2124 # or
2125 # my $lock = $db->cds_lock();
2126
2127
2128 my $cursor = $db->db_cursor();
2129
2130 # Now loop through the database, and increment
2131 # each value using c_put.
2132 while ($cursor->c_get($key, $value, DB_NEXT) == 0)
2133 {
2134 $cursor->c_put($key, $value+1, DB_CURRENT) == 0
2135 or die "$BerkeleyDB::Error\n";
2136 }
2137
2138 When this code is run, it will fail on the "c_put" line with this error
2139
2140 Write attempted on read-only cursor
2141
2142 The read cursor has automatically disallowed a write operation to
2143 prevent a deadlock.
2144
2145 So the rule is -- you CANNOT carry out a write operation using a read-
2146 only cursor (i.e. you cannot use "c_put" or "c_del") whilst another
2147 write-cursor is already active.
2148
2149 The workaround for this issue is to just use "db_put" instead of
2150 "c_put", like this
2151
2152 # Assume $db is a database opened in a CDS environment.
2153
2154 # Create a write-lock
2155 my $lock = $db->db_cursor(DB_WRITECURSOR);
2156 # or
2157 # my $lock = $db->cds_lock();
2158
2159
2160 my $cursor = $db->db_cursor();
2161
2162 # Now loop through the database, and increment
2163 # each value using c_put.
2164 while ($cursor->c_get($key, $value, DB_NEXT) == 0)
2165 {
2166 $db->db_put($key, $value+1) == 0
2167 or die "$BerkeleyDB::Error\n";
2168 }
2169
2170 Implicit Cursors
2171 All Berkeley DB cursors will hold either a read lock or a write lock on
2172 the database for the existence of the cursor. In order to prevent
2173 blocking of other processes you need to make sure that they are not
2174 long lived.
2175
2176 There are a number of instances where the Perl interface to Berkeley DB
2177 will create a cursor behind the scenes without you being aware of it.
2178 Most of these are very short-lived and will not affect the running of
2179 your script, but there are a few notable exceptions.
2180
2181 Consider this snippet of code
2182
2183 while (my ($k, $v) = each %hash)
2184 {
2185 # do something
2186 }
2187
2188 To implement the "each" functionality, a read cursor will be created
2189 behind the scenes to allow you to iterate through the tied hash, %hash.
2190 While that cursor is still active, a read lock will obviously be held
2191 against the database. If your application has any other writing
2192 processes, these will be blocked until the read cursor is closed. That
2193 won't happen until the loop terminates.
2194
2195 To avoid blocking problems, only keep cursors open as long as they are
2196 needed. The same is true when you use the "cursor" method or the
2197 "cds_lock" method.
2198
2199 The locking behaviour of the "values" or "keys" functions, shown below,
2200 is subtly different.
2201
2202 foreach my $k (keys %hash)
2203 {
2204 # do something
2205 }
2206
2207 foreach my $v (values %hash)
2208 {
2209 # do something
2210 }
2211
2212 Just as in the "each" function, a read cursor will be created to
2213 iterate over the database in both of these cases. Where "keys" and
2214 "values" differ is the place where the cursor carries out the iteration
2215 through the database. Whilst "each" carried out a single iteration
2216 every time it was invoked, the "keys" and "values" functions will
2217 iterate through the entire database in one go -- the complete database
2218 will be read into memory before the first iteration of the loop.
2219
2220 Apart from the fact that a read lock will be held for the amount of
2221 time required to iterate through the database, the use of "keys" and
2222 "values" is not recommended because it will result in the complete
2223 database being read into memory.
2224
2225 Avoiding Deadlock with multiple databases
2226 If your CDS application uses multiple database files, and you need to
2227 write to more than one of them, you need to be careful you don't create
2228 a deadlock.
2229
2230 For example, say you have two databases, D1 and D2, and two processes,
2231 P1 and P2. Assume you want to write a record to each database. If P1
2232 writes the records to the databases in the order D1, D2 while process
2233 P2 writes the records in the order D2, D1, there is the potential for a
2234 deadlock to occur.
2235
2236 This scenario can be avoided by either always acquiring the write locks
2237 in exactly the same order in your application code, or by using the
2238 "DB_CDB_ALLDB" flag when opening the environment. This flag will make a
2239 write-lock apply to all the databases in the environment.
2240
2241 Add example here
2242
2244 A DBM Filter is a piece of code that is be used when you always want to
2245 make the same transformation to all keys and/or values in a DBM
2246 database. All of the database classes (BerkeleyDB::Hash,
2247 BerkeleyDB::Btree and BerkeleyDB::Recno) support DBM Filters.
2248
2249 An example is when you need to encode your data in UTF-8 before writing
2250 to the database and then decode the UTF-8 when reading from the
2251 database file.
2252
2253 There are two ways to use a DBM Filter.
2254
2255 1. Using the low-level API defined below.
2256
2257 2. Using the DBM_Filter module. This module hides the complexity of
2258 the API defined below and comes with a number of "canned" filters
2259 that cover some of the common use-cases.
2260
2261 Use of the DBM_Filter module is recommended.
2262
2263 DBM Filter Low-level API
2264 There are four methods associated with DBM Filters. All work
2265 identically, and each is used to install (or uninstall) a single DBM
2266 Filter. Each expects a single parameter, namely a reference to a sub.
2267 The only difference between them is the place that the filter is
2268 installed.
2269
2270 To summarise:
2271
2272 filter_store_key
2273 If a filter has been installed with this method, it will be
2274 invoked every time you write a key to a DBM database.
2275
2276 filter_store_value
2277 If a filter has been installed with this method, it will be
2278 invoked every time you write a value to a DBM database.
2279
2280 filter_fetch_key
2281 If a filter has been installed with this method, it will be
2282 invoked every time you read a key from a DBM database.
2283
2284 filter_fetch_value
2285 If a filter has been installed with this method, it will be
2286 invoked every time you read a value from a DBM database.
2287
2288 You can use any combination of the methods, from none, to all four.
2289
2290 All filter methods return the existing filter, if present, or "undef"
2291 in not.
2292
2293 To delete a filter pass "undef" to it.
2294
2295 The Filter
2296 When each filter is called by Perl, a local copy of $_ will contain the
2297 key or value to be filtered. Filtering is achieved by modifying the
2298 contents of $_. The return code from the filter is ignored.
2299
2300 An Example -- the NULL termination problem.
2301 Consider the following scenario. You have a DBM database that you need
2302 to share with a third-party C application. The C application assumes
2303 that all keys and values are NULL terminated. Unfortunately when Perl
2304 writes to DBM databases it doesn't use NULL termination, so your Perl
2305 application will have to manage NULL termination itself. When you write
2306 to the database you will have to use something like this:
2307
2308 $hash{"$key\0"} = "$value\0" ;
2309
2310 Similarly the NULL needs to be taken into account when you are
2311 considering the length of existing keys/values.
2312
2313 It would be much better if you could ignore the NULL terminations issue
2314 in the main application code and have a mechanism that automatically
2315 added the terminating NULL to all keys and values whenever you write to
2316 the database and have them removed when you read from the database. As
2317 I'm sure you have already guessed, this is a problem that DBM Filters
2318 can fix very easily.
2319
2320 use strict ;
2321 use BerkeleyDB ;
2322
2323 my %hash ;
2324 my $filename = "filt.db" ;
2325 unlink $filename ;
2326
2327 my $db = tie %hash, 'BerkeleyDB::Hash',
2328 -Filename => $filename,
2329 -Flags => DB_CREATE
2330 or die "Cannot open $filename: $!\n" ;
2331
2332 # Install DBM Filters
2333 $db->filter_fetch_key ( sub { s/\0$// } ) ;
2334 $db->filter_store_key ( sub { $_ .= "\0" } ) ;
2335 $db->filter_fetch_value( sub { s/\0$// } ) ;
2336 $db->filter_store_value( sub { $_ .= "\0" } ) ;
2337
2338 $hash{"abc"} = "def" ;
2339 my $a = $hash{"ABC"} ;
2340 # ...
2341 undef $db ;
2342 untie %hash ;
2343
2344 Hopefully the contents of each of the filters should be self-
2345 explanatory. Both "fetch" filters remove the terminating NULL, and both
2346 "store" filters add a terminating NULL.
2347
2348 Another Example -- Key is a C int.
2349 Here is another real-life example. By default, whenever Perl writes to
2350 a DBM database it always writes the key and value as strings. So when
2351 you use this:
2352
2353 $hash{12345} = "something" ;
2354
2355 the key 12345 will get stored in the DBM database as the 5 byte string
2356 "12345". If you actually want the key to be stored in the DBM database
2357 as a C int, you will have to use "pack" when writing, and "unpack" when
2358 reading.
2359
2360 Here is a DBM Filter that does it:
2361
2362 use strict ;
2363 use BerkeleyDB ;
2364 my %hash ;
2365 my $filename = "filt.db" ;
2366 unlink $filename ;
2367
2368
2369 my $db = tie %hash, 'BerkeleyDB::Btree',
2370 -Filename => $filename,
2371 -Flags => DB_CREATE
2372 or die "Cannot open $filename: $!\n" ;
2373
2374 $db->filter_fetch_key ( sub { $_ = unpack("i", $_) } ) ;
2375 $db->filter_store_key ( sub { $_ = pack ("i", $_) } ) ;
2376 $hash{123} = "def" ;
2377 # ...
2378 undef $db ;
2379 untie %hash ;
2380
2381 This time only two filters have been used -- we only need to manipulate
2382 the contents of the key, so it wasn't necessary to install any value
2383 filters.
2384
2386 Both BerkeleyDB::Hash and BerkeleyDB::Btree can be used with the MLDBM
2387 module. The code fragment below shows how to open associate MLDBM with
2388 BerkeleyDB::Btree. To use BerkeleyDB::Hash just replace
2389 BerkeleyDB::Btree with BerkeleyDB::Hash.
2390
2391 use strict ;
2392 use BerkeleyDB ;
2393 use MLDBM qw(BerkeleyDB::Btree) ;
2394 use Data::Dumper;
2395
2396 my $filename = 'testmldbm' ;
2397 my %o ;
2398
2399 unlink $filename ;
2400 tie %o, 'MLDBM', -Filename => $filename,
2401 -Flags => DB_CREATE
2402 or die "Cannot open database '$filename: $!\n";
2403
2404 See the MLDBM documentation for information on how to use the module
2405 and for details of its limitations.
2406
2408 TODO.
2409
2411 Sharing Databases With C Applications
2412 There is no technical reason why a Berkeley DB database cannot be
2413 shared by both a Perl and a C application.
2414
2415 The vast majority of problems that are reported in this area boil down
2416 to the fact that C strings are NULL terminated, whilst Perl strings are
2417 not. See "An Example -- the NULL termination problem." in the DBM
2418 FILTERS section for a generic way to work around this problem.
2419
2420 The untie Gotcha
2421 TODO
2422
2424 This section attempts to answer some of the more common questions that
2425 I get asked.
2426
2427 Relationship with DB_File
2428 Before Berkeley DB 2.x was written there was only one Perl module that
2429 interfaced to Berkeley DB. That module is called DB_File. Although
2430 DB_File can be build with Berkeley DB 1.x, 2.x, 3.x or 4.x, it only
2431 provides an interface to the functionality available in Berkeley DB
2432 1.x. That means that it doesn't support transactions, locking or any of
2433 the other new features available in DB 2.x or better.
2434
2435 How do I store Perl data structures with BerkeleyDB?
2436 See "Using BerkeleyDB with MLDBM".
2437
2439 See the Changes file.
2440
2442 The most recent version of BerkeleyDB can always be found on CPAN (see
2443 "CPAN" in perlmod for details), in the directory
2444 modules/by-module/BerkeleyDB.
2445
2446 The official web site for Berkeley DB is
2447 http://www.oracle.com/technology/products/berkeley-db/db/index.html.
2448
2450 Copyright (c) 1997-2019 Paul Marquess. All rights reserved. This
2451 program is free software; you can redistribute it and/or modify it
2452 under the same terms as Perl itself.
2453
2454 Although BerkeleyDB is covered by the Perl license, the library it
2455 makes use of, namely Berkeley DB, is not. Berkeley DB has its own
2456 copyright and its own license. Please take the time to read it.
2457
2458 Here are few words taken from the Berkeley DB FAQ (at
2459 http://www.oracle.com/technology/products/berkeley-db/db/index.html)
2460 regarding the license:
2461
2462 Do I have to license DB to use it in Perl scripts?
2463
2464 No. The Berkeley DB license requires that software that uses
2465 Berkeley DB be freely redistributable. In the case of Perl, that
2466 software is Perl, and not your scripts. Any Perl scripts that you
2467 write are your property, including scripts that make use of Berkeley
2468 DB. Neither the Perl license nor the Berkeley DB license
2469 place any restriction on what you may do with them.
2470
2471 If you are in any doubt about the license situation, contact either the
2472 Berkeley DB authors or the author of BerkeleyDB. See "AUTHOR" for
2473 details.
2474
2476 Paul Marquess <pmqs@cpan.org>.
2477
2479 perl(1), DB_File, Berkeley DB.
2480
2481
2482
2483perl v5.30.1 2020-01-29 BerkeleyDB(3)