1BerkeleyDB(3) User Contributed Perl Documentation BerkeleyDB(3)
2
6 BerkeleyDB - Perl extension for Berkeley DB version 2, 3 or 4
7
9 use BerkeleyDB;
10 $env = new BerkeleyDB::Env [OPTIONS] ;
12 $db = tie %hash, 'BerkeleyDB::Hash', [OPTIONS] ;
14 $db = new BerkeleyDB::Hash [OPTIONS] ;
15 $db = tie %hash, 'BerkeleyDB::Btree', [OPTIONS] ;
17 $db = new BerkeleyDB::Btree [OPTIONS] ;
18 $db = tie @array, 'BerkeleyDB::Recno', [OPTIONS] ;
20 $db = new BerkeleyDB::Recno [OPTIONS] ;
21 $db = tie @array, 'BerkeleyDB::Queue', [OPTIONS] ;
23 $db = new BerkeleyDB::Queue [OPTIONS] ;
24 $db = new BerkeleyDB::Unknown [OPTIONS] ;
26 $status = BerkeleyDB::db_remove [OPTIONS]
28 $status = BerkeleyDB::db_rename [OPTIONS]
29 $status = BerkeleyDB::db_verify [OPTIONS]
30 $hash{$key} = $value ;
32 $value = $hash{$key} ;
33 each %hash ;
34 keys %hash ;
35 values %hash ;
36 $status = $db->db_get()
38 $status = $db->db_put() ;
39 $status = $db->db_del() ;
40 $status = $db->db_sync() ;
41 $status = $db->db_close() ;
42 $status = $db->db_pget()
43 $hash_ref = $db->db_stat() ;
44 $status = $db->db_key_range();
45 $type = $db->type() ;
46 $status = $db->status() ;
47 $boolean = $db->byteswapped() ;
48 $status = $db->truncate($count) ;
49 $status = $db->compact($start, $stop, $c_data, $flags, $end);
50 $bool = $env->cds_enabled();
52 $bool = $db->cds_enabled();
53 $lock = $db->cds_lock();
54 $lock->cds_unlock();
55 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ;
57 ($flag, $old_offset, $old_length) = $db->partial_clear() ;
58 $cursor = $db->db_cursor([$flags]) ;
60 $newcursor = $cursor->c_dup([$flags]);
61 $status = $cursor->c_get() ;
62 $status = $cursor->c_put() ;
63 $status = $cursor->c_del() ;
64 $status = $cursor->c_count() ;
65 $status = $cursor->c_pget() ;
66 $status = $cursor->status() ;
67 $status = $cursor->c_close() ;
68 $cursor = $db->db_join() ;
70 $status = $cursor->c_get() ;
71 $status = $cursor->c_close() ;
72 $status = $env->txn_checkpoint()
74 $hash_ref = $env->txn_stat()
75 $status = $env->setmutexlocks()
76 $status = $env->set_flags()
77 $status = $env->set_timeout()
78 $status = $env->lsn_reset()
79 $txn = $env->txn_begin() ;
81 $db->Txn($txn);
82 $txn->Txn($db1, $db2,...);
83 $status = $txn->txn_prepare()
84 $status = $txn->txn_commit()
85 $status = $txn->txn_abort()
86 $status = $txn->txn_id()
87 $status = $txn->txn_discard()
88 $status = $txn->set_timeout()
89 $status = $env->set_lg_dir();
91 $status = $env->set_lg_bsize();
92 $status = $env->set_lg_max();
93 $status = $env->set_data_dir() ;
95 $status = $env->set_tmp_dir() ;
96 $status = $env->set_verbose() ;
97 $db_env_ptr = $env->DB_ENV() ;
98 $BerkeleyDB::Error
100 $BerkeleyDB::db_version
101 # DBM Filters
103 $old_filter = $db->filter_store_key ( sub { ... } ) ;
104 $old_filter = $db->filter_store_value( sub { ... } ) ;
105 $old_filter = $db->filter_fetch_key ( sub { ... } ) ;
106 $old_filter = $db->filter_fetch_value( sub { ... } ) ;
107 # deprecated, but supported
109 $txn_mgr = $env->TxnMgr();
110 $status = $txn_mgr->txn_checkpoint()
111 $hash_ref = $txn_mgr->txn_stat()
112 $txn = $txn_mgr->txn_begin() ;
113
115 NOTE: This document is still under construction. Expect it to be
116 incomplete in places.
117 This Perl module provides an interface to most of the functionality
119 available in Berkeley DB versions 2, 3 and 4. In general it is safe to
120 assume that the interface provided here to be identical to the Berkeley
121 DB interface. The main changes have been to make the Berkeley DB API
122 work in a Perl way. Note that if you are using Berkeley DB 2.x, the new
123 features available in Berkeley DB 3.x or DB 4.x are not available via
124 this module.
125 The reader is expected to be familiar with the Berkeley DB
127 documentation. Where the interface provided here is identical to the
128 Berkeley DB library and the... TODO
129 The db_appinit, db_cursor, db_open and db_txn man pages are
131 particularly relevant.
132 The interface to Berkeley DB is implemented with a number of Perl
134 classes.
135
137 The BerkeleyDB::Env class provides an interface to the Berkeley DB
138 function db_appinit in Berkeley DB 2.x or db_env_create and DBENV->open
139 in Berkeley DB 3.x/4.x. Its purpose is to initialise a number of sub-
140 systems that can then be used in a consistent way in all the databases
141 you make use of in the environment.
142 If you don't intend using transactions, locking or logging, then you
144 shouldn't need to make use of BerkeleyDB::Env.
145 Note that an environment consists of a number of files that Berkeley DB
147 manages behind the scenes for you. When you first use an environment,
148 it needs to be explicitly created. This is done by including
149 "DB_CREATE" with the "Flags" parameter, described below.
150 Synopsis
152 $env = new BerkeleyDB::Env
153 [ -Home => $path, ]
154 [ -Server => $name, ]
155 [ -CacheSize => $number, ]
156 [ -Config => { name => value, name => value }, ]
157 [ -ErrFile => filename, ]
158 [ -MsgFile => filename, ]
159 [ -ErrPrefix => "string", ]
160 [ -Flags => number, ]
161 [ -SetFlags => bitmask, ]
162 [ -LockDetect => number, ]
163 [ -TxMax => number, ]
164 [ -LogConfig => number, ]
165 [ -MaxLockers => number, ]
166 [ -MaxLocks => number, ]
167 [ -MaxObjects => number, ]
168 [ -SharedMemKey => number, ]
169 [ -Verbose => boolean, ]
170 [ -Encrypt => { Password => "string",
171 Flags => number }, ]
172 All the parameters to the BerkeleyDB::Env constructor are optional.
174 -Home
176 If present, this parameter should point to an existing directory.
177 Any files that aren't specified with an absolute path in the sub-
178 systems that are initialised by the BerkeleyDB::Env class will be
179 assumed to live in the Home directory.
180 For example, in the code fragment below the database "fred.db"
182 will be opened in the directory "/home/databases" because it was
183 specified as a relative path, but "joe.db" will be opened in
184 "/other" because it was part of an absolute path.
185 $env = new BerkeleyDB::Env
187 -Home => "/home/databases"
188 ...
189 $db1 = new BerkeleyDB::Hash
191 -Filename => "fred.db",
192 -Env => $env
193 ...
194 $db2 = new BerkeleyDB::Hash
196 -Filename => "/other/joe.db",
197 -Env => $env
198 ...
199 -Server
201 If present, this parameter should be the hostname of a server that
202 is running the Berkeley DB RPC server. All databases will be
203 accessed via the RPC server.
204 -Encrypt
206 If present, this parameter will enable encryption of all data
207 before it is written to the database. This parameters must be
208 given a hash reference. The format is shown below.
209 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES }
211 Valid values for the Flags are 0 or "DB_ENCRYPT_AES".
213 This option requires Berkeley DB 4.1 or better.
215 -Cachesize
217 If present, this parameter sets the size of the environments
218 shared memory buffer pool.
219 -TxMax
221 If present, this parameter sets the number of simultaneous
222 transactions that are allowed. Default 100. This default is
223 definitely too low for programs using the MVCC capabilities.
224 -LogConfig
226 If present, this parameter is used to configure log options.
227 -MaxLockers
229 If present, this parameter is used to configure the maximum number
230 of processes doing locking on the database. Default 1000.
231 -MaxLocks
233 If present, this parameter is used to configure the maximum number
234 of locks on the database. Default 1000. This is often lower than
235 required.
236 -MaxObjects
238 If present, this parameter is used to configure the maximum number
239 of locked objects. Default 1000. This is often lower than
240 required.
241 -SharedMemKey
243 If present, this parameter sets the base segment ID for the shared
244 memory region used by Berkeley DB.
245 This option requires Berkeley DB 3.1 or better.
247 Use "$env->get_shm_key($id)" to find out the base segment ID used
249 once the environment is open.
250 -ThreadCount
252 If present, this parameter declares the approximate number of
253 threads that will be used in the database environment. This
254 parameter is only necessary when the $env->failchk method will be
255 used. It does not actually set the maximum number of threads but
256 rather is used to determine memory sizing.
257 This option requires Berkeley DB 4.4 or better. It is only
259 supported on Unix/Linux.
260 -Config
262 This is a variation on the "-Home" parameter, but it allows finer
263 control of where specific types of files will be stored.
264 The parameter expects a reference to a hash. Valid keys are:
266 DB_DATA_DIR, DB_LOG_DIR and DB_TMP_DIR
267 The code below shows an example of how it can be used.
269 $env = new BerkeleyDB::Env
271 -Config => { DB_DATA_DIR => "/home/databases",
272 DB_LOG_DIR => "/home/logs",
273 DB_TMP_DIR => "/home/tmp"
274 }
275 ...
276 -ErrFile
278 Expects a filename or filenhandle. Any errors generated internally
279 by Berkeley DB will be logged to this file. A useful debug setting
280 is to open environments with either
281 -ErrFile => *STDOUT
283 or
285 -ErrFile => *STDERR
287 -ErrPrefix
289 Allows a prefix to be added to the error messages before they are
290 sent to -ErrFile.
291 -Flags
293 The Flags parameter specifies both which sub-systems to
294 initialise, as well as a number of environment-wide options. See
295 the Berkeley DB documentation for more details of these options.
296 Any of the following can be specified by OR'ing them:
298 DB_CREATE
300 If any of the files specified do not already exist, create them.
302 DB_INIT_CDB
304 Initialise the Concurrent Access Methods
306 DB_INIT_LOCK
308 Initialise the Locking sub-system.
310 DB_INIT_LOG
312 Initialise the Logging sub-system.
314 DB_INIT_MPOOL
316 Initialise the ...
318 DB_INIT_TXN
320 Initialise the ...
322 DB_MPOOL_PRIVATE
324 Initialise the ...
326 DB_INIT_MPOOL is also specified.
328 Initialise the ...
330 DB_NOMMAP
332 Initialise the ...
334 DB_RECOVER
336 DB_RECOVER_FATAL
338 DB_THREAD
340 DB_TXN_NOSYNC
342 DB_USE_ENVIRON
344 DB_USE_ENVIRON_ROOT
346 -SetFlags
348 Calls ENV->set_flags with the supplied bitmask. Use this when you
349 need to make use of DB_ENV->set_flags before DB_ENV->open is
350 called.
351 Only valid when Berkeley DB 3.x or better is used.
353 -LockDetect
355 Specifies what to do when a lock conflict occurs. The value should
356 be one of
357 DB_LOCK_DEFAULT
359 DB_LOCK_OLDEST
361 DB_LOCK_RANDOM
363 DB_LOCK_YOUNGEST
365 -Verbose
367 Add extra debugging information to the messages sent to -ErrFile.
368 Methods
370 The environment class has the following methods:
371 $env->errPrefix("string") ;
373 This method is identical to the -ErrPrefix flag. It allows the
374 error prefix string to be changed dynamically.
375 $env->set_flags(bitmask, 1|0);
377 $txn = $env->TxnMgr()
378 Constructor for creating a TxnMgr object. See "TRANSACTIONS" for
379 more details of using transactions.
380 This method is deprecated. Access the transaction methods using
382 the txn_ methods below from the environment object directly.
383 $env->txn_begin()
385 TODO
386 $env->txn_stat()
388 TODO
389 $env->txn_checkpoint()
391 TODO
392 $env->status()
394 Returns the status of the last BerkeleyDB::Env method.
395 $env->DB_ENV()
397 Returns a pointer to the underlying DB_ENV data structure that
398 Berkeley DB uses.
399 $env->get_shm_key($id)
401 Writes the base segment ID for the shared memory region used by
402 the Berkeley DB environment into $id. Returns 0 on success.
403 This option requires Berkeley DB 4.2 or better.
405 Use the "-SharedMemKey" option when opening the environemt to set
407 the base segment ID.
408 $env->set_isalive()
410 Set the callback that determines if the thread of control,
411 identified by the pid and tid arguments, is still running. This
412 method should only be used in combination with $env->failchk.
413 This option requires Berkeley DB 4.4 or better.
415 $env->failchk($flags)
417 The $env->failchk method checks for threads of control (either a
418 true thread or a process) that have exited while manipulating
419 Berkeley DB library data structures, while holding a logical
420 database lock, or with an unresolved transaction (that is, a
421 transaction that was never aborted or committed).
422 If $env->failchk determines a thread of control exited while
424 holding database read locks, it will release those locks. If
425 $env->failchk determines a thread of control exited with an
426 unresolved transaction, the transaction will be aborted.
427 Applications calling the $env->failchk method must have already
429 called the $env->set_isalive method, on the same DB environement,
430 and must have configured their database environment using the
431 -ThreadCount flag. The ThreadCount flag cannot be used on an
432 environment that wasn't previously initialized with it.
433 This option requires Berkeley DB 4.4 or better.
435 $env->stat_print
437 Prints statistical information.
438 If the "MsgFile" option is specified the output will be sent to
440 the file. Otherwise output is sent to standard output.
441 This option requires Berkeley DB 4.3 or better.
443 $env->lock_stat_print
445 Prints locking subsystem statistics.
446 If the "MsgFile" option is specified the output will be sent to
448 the file. Otherwise output is sent to standard output.
449 This option requires Berkeley DB 4.3 or better.
451 $env->mutex_stat_print
453 Prints mutex subsystem statistics.
454 If the "MsgFile" option is specified the output will be sent to
456 the file. Otherwise output is sent to standard output.
457 This option requires Berkeley DB 4.4 or better.
459 $env->set_timeout($timeout, $flags)
461 $env->status()
462 Returns the status of the last BerkeleyDB::Env method.
463 Examples
465 TODO.
466
468 $status = BerkeleyDB::db_remove [OPTIONS]
469 $status = BerkeleyDB::db_rename [OPTIONS]
470 $status = BerkeleyDB::db_verify [OPTIONS]
471
473 BerkeleyDB supports the following database formats:
474 BerkeleyDB::Hash
476 This database type allows arbitrary key/value pairs to be stored
477 in data files. This is equivalent to the functionality provided by
478 other hashing packages like DBM, NDBM, ODBM, GDBM, and SDBM.
479 Remember though, the files created using BerkeleyDB::Hash are not
480 compatible with any of the other packages mentioned.
481 A default hashing algorithm, which will be adequate for most
483 applications, is built into BerkeleyDB. If you do need to use your
484 own hashing algorithm it is possible to write your own in Perl and
485 have BerkeleyDB use it instead.
486 BerkeleyDB::Btree
488 The Btree format allows arbitrary key/value pairs to be stored in
489 a B+tree.
490 As with the BerkeleyDB::Hash format, it is possible to provide a
492 user defined Perl routine to perform the comparison of keys. By
493 default, though, the keys are stored in lexical order.
494 BerkeleyDB::Recno
496 TODO.
497 BerkeleyDB::Queue
499 TODO.
500 BerkeleyDB::Unknown
502 This isn't a database format at all. It is used when you want to
503 open an existing Berkeley DB database without having to know what
504 type is it.
505 Each of the database formats described above is accessed via a
507 corresponding BerkeleyDB class. These will be described in turn in the
508 next sections.
509
511 Equivalent to calling db_open with type DB_HASH in Berkeley DB 2.x and
512 calling db_create followed by DB->open with type DB_HASH in Berkeley DB
513 3.x or greater.
514 Two forms of constructor are supported:
516 $db = new BerkeleyDB::Hash
518 [ -Filename => "filename", ]
519 [ -Subname => "sub-database name", ]
520 [ -Flags => flags,]
521 [ -Property => flags,]
522 [ -Mode => number,]
523 [ -Cachesize => number,]
524 [ -Lorder => number,]
525 [ -Pagesize => number,]
526 [ -Env => $env,]
527 [ -Txn => $txn,]
528 [ -Encrypt => { Password => "string",
529 Flags => number }, ],
530 # BerkeleyDB::Hash specific
531 [ -Ffactor => number,]
532 [ -Nelem => number,]
533 [ -Hash => code reference,]
534 [ -DupCompare => code reference,]
535 and this
537 [$db =] tie %hash, 'BerkeleyDB::Hash',
539 [ -Filename => "filename", ]
540 [ -Subname => "sub-database name", ]
541 [ -Flags => flags,]
542 [ -Property => flags,]
543 [ -Mode => number,]
544 [ -Cachesize => number,]
545 [ -Lorder => number,]
546 [ -Pagesize => number,]
547 [ -Env => $env,]
548 [ -Txn => $txn,]
549 [ -Encrypt => { Password => "string",
550 Flags => number }, ],
551 # BerkeleyDB::Hash specific
552 [ -Ffactor => number,]
553 [ -Nelem => number,]
554 [ -Hash => code reference,]
555 [ -DupCompare => code reference,]
556 When the "tie" interface is used, reading from and writing to the
558 database is achieved via the tied hash. In this case the database
559 operates like a Perl associative array that happens to be stored on
560 disk.
561 In addition to the high-level tied hash interface, it is possible to
563 make use of the underlying methods provided by Berkeley DB
564 Options
566 In addition to the standard set of options (see "COMMON OPTIONS")
567 BerkeleyDB::Hash supports these options:
568 -Property
570 Used to specify extra flags when opening a database. The following
571 flags may be specified by bitwise OR'ing together one or more of
572 the following values:
573 DB_DUP
575 When creating a new database, this flag enables the storing of
577 duplicate keys in the database. If DB_DUPSORT is not specified as
578 well, the duplicates are stored in the order they are created in
579 the database.
580 DB_DUPSORT
582 Enables the sorting of duplicate keys in the database. Ignored if
584 DB_DUP isn't also specified.
585 -Ffactor
587 -Nelem
588 See the Berkeley DB documentation for details of these options.
589 -Hash
591 Allows you to provide a user defined hash function. If not
592 specified, a default hash function is used. Here is a template for
593 a user-defined hash function
594 sub hash
596 {
597 my ($data) = shift ;
598 ...
599 # return the hash value for $data
600 return $hash ;
601 }
602 tie %h, "BerkeleyDB::Hash",
604 -Filename => $filename,
605 -Hash => \&hash,
606 ...
607 See "" for an example.
609 -DupCompare
611 Used in conjunction with the DB_DUPOSRT flag.
612 sub compare
614 {
615 my ($key, $key2) = @_ ;
616 ...
617 # return 0 if $key1 eq $key2
618 # -1 if $key1 lt $key2
619 # 1 if $key1 gt $key2
620 return (-1 , 0 or 1) ;
621 }
622 tie %h, "BerkeleyDB::Hash",
624 -Filename => $filename,
625 -Property => DB_DUP|DB_DUPSORT,
626 -DupCompare => \&compare,
627 ...
628 Methods
630 BerkeleyDB::Hash only supports the standard database methods. See
631 "COMMON DATABASE METHODS".
632 A Simple Tied Hash Example
634 use strict ;
635 use BerkeleyDB ;
636 use vars qw( %h $k $v ) ;
637 my $filename = "fruit" ;
639 unlink $filename ;
640 tie %h, "BerkeleyDB::Hash",
641 -Filename => $filename,
642 -Flags => DB_CREATE
643 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
644 # Add a few key/value pairs to the file
646 $h{"apple"} = "red" ;
647 $h{"orange"} = "orange" ;
648 $h{"banana"} = "yellow" ;
649 $h{"tomato"} = "red" ;
650 # Check for existence of a key
652 print "Banana Exists\n\n" if $h{"banana"} ;
653 # Delete a key/value pair.
655 delete $h{"apple"} ;
656 # print the contents of the file
658 while (($k, $v) = each %h)
659 { print "$k -> $v\n" }
660 untie %h ;
662 here is the output:
664 Banana Exists
666 orange -> orange
668 tomato -> red
669 banana -> yellow
670 Note that the like ordinary associative arrays, the order of the keys
672 retrieved from a Hash database are in an apparently random order.
673 Another Simple Hash Example
675 Do the same as the previous example but not using tie.
676 use strict ;
678 use BerkeleyDB ;
679 my $filename = "fruit" ;
681 unlink $filename ;
682 my $db = new BerkeleyDB::Hash
683 -Filename => $filename,
684 -Flags => DB_CREATE
685 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
686 # Add a few key/value pairs to the file
688 $db->db_put("apple", "red") ;
689 $db->db_put("orange", "orange") ;
690 $db->db_put("banana", "yellow") ;
691 $db->db_put("tomato", "red") ;
692 # Check for existence of a key
694 print "Banana Exists\n\n" if $db->db_get("banana", $v) == 0;
695 # Delete a key/value pair.
697 $db->db_del("apple") ;
698 # print the contents of the file
700 my ($k, $v) = ("", "") ;
701 my $cursor = $db->db_cursor() ;
702 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
703 { print "$k -> $v\n" }
704 undef $cursor ;
706 undef $db ;
707 Duplicate keys
709 The code below is a variation on the examples above. This time the hash
710 has been inverted. The key this time is colour and the value is the
711 fruit name. The DB_DUP flag has been specified to allow duplicates.
712 use strict ;
714 use BerkeleyDB ;
715 my $filename = "fruit" ;
717 unlink $filename ;
718 my $db = new BerkeleyDB::Hash
719 -Filename => $filename,
720 -Flags => DB_CREATE,
721 -Property => DB_DUP
722 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
723 # Add a few key/value pairs to the file
725 $db->db_put("red", "apple") ;
726 $db->db_put("orange", "orange") ;
727 $db->db_put("green", "banana") ;
728 $db->db_put("yellow", "banana") ;
729 $db->db_put("red", "tomato") ;
730 $db->db_put("green", "apple") ;
731 # print the contents of the file
733 my ($k, $v) = ("", "") ;
734 my $cursor = $db->db_cursor() ;
735 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
736 { print "$k -> $v\n" }
737 undef $cursor ;
739 undef $db ;
740 here is the output:
742 orange -> orange
744 yellow -> banana
745 red -> apple
746 red -> tomato
747 green -> banana
748 green -> apple
749 Sorting Duplicate Keys
751 In the previous example, when there were duplicate keys, the values are
752 sorted in the order they are stored in. The code below is identical to
753 the previous example except the DB_DUPSORT flag is specified.
754 use strict ;
756 use BerkeleyDB ;
757 my $filename = "fruit" ;
759 unlink $filename ;
760 my $db = new BerkeleyDB::Hash
761 -Filename => $filename,
762 -Flags => DB_CREATE,
763 -Property => DB_DUP | DB_DUPSORT
764 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
765 # Add a few key/value pairs to the file
767 $db->db_put("red", "apple") ;
768 $db->db_put("orange", "orange") ;
769 $db->db_put("green", "banana") ;
770 $db->db_put("yellow", "banana") ;
771 $db->db_put("red", "tomato") ;
772 $db->db_put("green", "apple") ;
773 # print the contents of the file
775 my ($k, $v) = ("", "") ;
776 my $cursor = $db->db_cursor() ;
777 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
778 { print "$k -> $v\n" }
779 undef $cursor ;
781 undef $db ;
782 Notice that in the output below the duplicate values are sorted.
784 orange -> orange
786 yellow -> banana
787 red -> apple
788 red -> tomato
789 green -> apple
790 green -> banana
791 Custom Sorting Duplicate Keys
793 Another variation
794 TODO
796 Changing the hash
798 TODO
799 Using db_stat
801 TODO
802
804 Equivalent to calling db_open with type DB_BTREE in Berkeley DB 2.x and
805 calling db_create followed by DB->open with type DB_BTREE in Berkeley
806 DB 3.x or greater.
807 Two forms of constructor are supported:
809 $db = new BerkeleyDB::Btree
811 [ -Filename => "filename", ]
812 [ -Subname => "sub-database name", ]
813 [ -Flags => flags,]
814 [ -Property => flags,]
815 [ -Mode => number,]
816 [ -Cachesize => number,]
817 [ -Lorder => number,]
818 [ -Pagesize => number,]
819 [ -Env => $env,]
820 [ -Txn => $txn,]
821 [ -Encrypt => { Password => "string",
822 Flags => number }, ],
823 # BerkeleyDB::Btree specific
824 [ -Minkey => number,]
825 [ -Compare => code reference,]
826 [ -DupCompare => code reference,]
827 [ -Prefix => code reference,]
828 and this
830 [$db =] tie %hash, 'BerkeleyDB::Btree',
832 [ -Filename => "filename", ]
833 [ -Subname => "sub-database name", ]
834 [ -Flags => flags,]
835 [ -Property => flags,]
836 [ -Mode => number,]
837 [ -Cachesize => number,]
838 [ -Lorder => number,]
839 [ -Pagesize => number,]
840 [ -Env => $env,]
841 [ -Txn => $txn,]
842 [ -Encrypt => { Password => "string",
843 Flags => number }, ],
844 # BerkeleyDB::Btree specific
845 [ -Minkey => number,]
846 [ -Compare => code reference,]
847 [ -DupCompare => code reference,]
848 [ -Prefix => code reference,]
849 Options
851 In addition to the standard set of options (see "COMMON OPTIONS")
852 BerkeleyDB::Btree supports these options:
853 -Property
855 Used to specify extra flags when opening a database. The following
856 flags may be specified by bitwise OR'ing together one or more of
857 the following values:
858 DB_DUP
860 When creating a new database, this flag enables the storing of
862 duplicate keys in the database. If DB_DUPSORT is not specified as
863 well, the duplicates are stored in the order they are created in
864 the database.
865 DB_DUPSORT
867 Enables the sorting of duplicate keys in the database. Ignored if
869 DB_DUP isn't also specified.
870 Minkey
872 TODO
873 Compare
875 Allow you to override the default sort order used in the database.
876 See "Changing the sort order" for an example.
877 sub compare
879 {
880 my ($key, $key2) = @_ ;
881 ...
882 # return 0 if $key1 eq $key2
883 # -1 if $key1 lt $key2
884 # 1 if $key1 gt $key2
885 return (-1 , 0 or 1) ;
886 }
887 tie %h, "BerkeleyDB::Hash",
889 -Filename => $filename,
890 -Compare => \&compare,
891 ...
892 Prefix
894 sub prefix
895 {
896 my ($key, $key2) = @_ ;
897 ...
898 # return number of bytes of $key2 which are
899 # necessary to determine that it is greater than $key1
900 return $bytes ;
901 }
902 tie %h, "BerkeleyDB::Hash",
904 -Filename => $filename,
905 -Prefix => \&prefix,
906 ...
907 =item DupCompare
908 sub compare
910 {
911 my ($key, $key2) = @_ ;
912 ...
913 # return 0 if $key1 eq $key2
914 # -1 if $key1 lt $key2
915 # 1 if $key1 gt $key2
916 return (-1 , 0 or 1) ;
917 }
918 tie %h, "BerkeleyDB::Hash",
920 -Filename => $filename,
921 -DupCompare => \&compare,
922 ...
923 set_bt_compress
925 Enabled compression of the btree data. The callback interface is
926 not supported at present. Need Berkeley DB 4.8 or better.
927 Methods
929 BerkeleyDB::Btree supports the following database methods. See also
930 "COMMON DATABASE METHODS".
931 All the methods below return 0 to indicate success.
933 $status = $db->db_key_range($key, $less, $equal, $greater [, $flags])
935 Given a key, $key, this method returns the proportion of keys less
936 than $key in $less, the proportion equal to $key in $equal and the
937 proportion greater than $key in $greater.
938 The proportion is returned as a double in the range 0.0 to 1.0.
940 A Simple Btree Example
942 The code below is a simple example of using a btree database.
943 use strict ;
945 use BerkeleyDB ;
946 my $filename = "tree" ;
948 unlink $filename ;
949 my %h ;
950 tie %h, 'BerkeleyDB::Btree',
951 -Filename => $filename,
952 -Flags => DB_CREATE
953 or die "Cannot open $filename: $! $BerkeleyDB::Error\n" ;
954 # Add a key/value pair to the file
956 $h{'Wall'} = 'Larry' ;
957 $h{'Smith'} = 'John' ;
958 $h{'mouse'} = 'mickey' ;
959 $h{'duck'} = 'donald' ;
960 # Delete
962 delete $h{"duck"} ;
963 # Cycle through the keys printing them in order.
965 # Note it is not necessary to sort the keys as
966 # the btree will have kept them in order automatically.
967 foreach (keys %h)
968 { print "$_\n" }
969 untie %h ;
971 Here is the output from the code above. The keys have been sorted using
973 Berkeley DB's default sorting algorithm.
974 Smith
976 Wall
977 mouse
978 Changing the sort order
980 It is possible to supply your own sorting algorithm if the one that
981 Berkeley DB used isn't suitable. The code below is identical to the
982 previous example except for the case insensitive compare function.
983 use strict ;
985 use BerkeleyDB ;
986 my $filename = "tree" ;
988 unlink $filename ;
989 my %h ;
990 tie %h, 'BerkeleyDB::Btree',
991 -Filename => $filename,
992 -Flags => DB_CREATE,
993 -Compare => sub { lc $_[0] cmp lc $_[1] }
994 or die "Cannot open $filename: $!\n" ;
995 # Add a key/value pair to the file
997 $h{'Wall'} = 'Larry' ;
998 $h{'Smith'} = 'John' ;
999 $h{'mouse'} = 'mickey' ;
1000 $h{'duck'} = 'donald' ;
1001 # Delete
1003 delete $h{"duck"} ;
1004 # Cycle through the keys printing them in order.
1006 # Note it is not necessary to sort the keys as
1007 # the btree will have kept them in order automatically.
1008 foreach (keys %h)
1009 { print "$_\n" }
1010 untie %h ;
1012 Here is the output from the code above.
1014 mouse
1016 Smith
1017 Wall
1018 There are a few point to bear in mind if you want to change the
1020 ordering in a BTREE database:
1021 1. The new compare function must be specified when you create the
1023 database.
1024 2. You cannot change the ordering once the database has been created.
1026 Thus you must use the same compare function every time you access
1027 the database.
1028 Using db_stat
1030 TODO
1031
1033 Equivalent to calling db_open with type DB_RECNO in Berkeley DB 2.x and
1034 calling db_create followed by DB->open with type DB_RECNO in Berkeley
1035 DB 3.x or greater.
1036 Two forms of constructor are supported:
1038 $db = new BerkeleyDB::Recno
1040 [ -Filename => "filename", ]
1041 [ -Subname => "sub-database name", ]
1042 [ -Flags => flags,]
1043 [ -Property => flags,]
1044 [ -Mode => number,]
1045 [ -Cachesize => number,]
1046 [ -Lorder => number,]
1047 [ -Pagesize => number,]
1048 [ -Env => $env,]
1049 [ -Txn => $txn,]
1050 [ -Encrypt => { Password => "string",
1051 Flags => number }, ],
1052 # BerkeleyDB::Recno specific
1053 [ -Delim => byte,]
1054 [ -Len => number,]
1055 [ -Pad => byte,]
1056 [ -Source => filename,]
1057 and this
1059 [$db =] tie @arry, 'BerkeleyDB::Recno',
1061 [ -Filename => "filename", ]
1062 [ -Subname => "sub-database name", ]
1063 [ -Flags => flags,]
1064 [ -Property => flags,]
1065 [ -Mode => number,]
1066 [ -Cachesize => number,]
1067 [ -Lorder => number,]
1068 [ -Pagesize => number,]
1069 [ -Env => $env,]
1070 [ -Txn => $txn,]
1071 [ -Encrypt => { Password => "string",
1072 Flags => number }, ],
1073 # BerkeleyDB::Recno specific
1074 [ -Delim => byte,]
1075 [ -Len => number,]
1076 [ -Pad => byte,]
1077 [ -Source => filename,]
1078 A Recno Example
1080 Here is a simple example that uses RECNO (if you are using a version of
1081 Perl earlier than 5.004_57 this example won't work -- see "Extra RECNO
1082 Methods" for a workaround).
1083 use strict ;
1085 use BerkeleyDB ;
1086 my $filename = "text" ;
1088 unlink $filename ;
1089 my @h ;
1091 tie @h, 'BerkeleyDB::Recno',
1092 -Filename => $filename,
1093 -Flags => DB_CREATE,
1094 -Property => DB_RENUMBER
1095 or die "Cannot open $filename: $!\n" ;
1096 # Add a few key/value pairs to the file
1098 $h[0] = "orange" ;
1099 $h[1] = "blue" ;
1100 $h[2] = "yellow" ;
1101 push @h, "green", "black" ;
1103 my $elements = scalar @h ;
1105 print "The array contains $elements entries\n" ;
1106 my $last = pop @h ;
1108 print "popped $last\n" ;
1109 unshift @h, "white" ;
1111 my $first = shift @h ;
1112 print "shifted $first\n" ;
1113 # Check for existence of a key
1115 print "Element 1 Exists with value $h[1]\n" if $h[1] ;
1116 untie @h ;
1118 Here is the output from the script:
1120 The array contains 5 entries
1122 popped black
1123 shifted white
1124 Element 1 Exists with value blue
1125 The last element is green
1126 The 2nd last element is yellow
1127
1129 Equivalent to calling db_create followed by DB->open with type DB_QUEUE
1130 in Berkeley DB 3.x or greater. This database format isn't available if
1131 you use Berkeley DB 2.x.
1132 Two forms of constructor are supported:
1134 $db = new BerkeleyDB::Queue
1136 [ -Filename => "filename", ]
1137 [ -Subname => "sub-database name", ]
1138 [ -Flags => flags,]
1139 [ -Property => flags,]
1140 [ -Mode => number,]
1141 [ -Cachesize => number,]
1142 [ -Lorder => number,]
1143 [ -Pagesize => number,]
1144 [ -Env => $env,]
1145 [ -Txn => $txn,]
1146 [ -Encrypt => { Password => "string",
1147 Flags => number }, ],
1148 # BerkeleyDB::Queue specific
1149 [ -Len => number,]
1150 [ -Pad => byte,]
1151 [ -ExtentSize => number, ]
1152 and this
1154 [$db =] tie @arry, 'BerkeleyDB::Queue',
1156 [ -Filename => "filename", ]
1157 [ -Subname => "sub-database name", ]
1158 [ -Flags => flags,]
1159 [ -Property => flags,]
1160 [ -Mode => number,]
1161 [ -Cachesize => number,]
1162 [ -Lorder => number,]
1163 [ -Pagesize => number,]
1164 [ -Env => $env,]
1165 [ -Txn => $txn,]
1166 [ -Encrypt => { Password => "string",
1167 Flags => number }, ],
1168 # BerkeleyDB::Queue specific
1169 [ -Len => number,]
1170 [ -Pad => byte,]
1171
1173 This class is used to open an existing database.
1174 Equivalent to calling db_open with type DB_UNKNOWN in Berkeley DB 2.x
1176 and calling db_create followed by DB->open with type DB_UNKNOWN in
1177 Berkeley DB 3.x or greater.
1178 The constructor looks like this:
1180 $db = new BerkeleyDB::Unknown
1182 [ -Filename => "filename", ]
1183 [ -Subname => "sub-database name", ]
1184 [ -Flags => flags,]
1185 [ -Property => flags,]
1186 [ -Mode => number,]
1187 [ -Cachesize => number,]
1188 [ -Lorder => number,]
1189 [ -Pagesize => number,]
1190 [ -Env => $env,]
1191 [ -Txn => $txn,]
1192 [ -Encrypt => { Password => "string",
1193 Flags => number }, ],
1194 An example
1197 All database access class constructors support the common set of
1198 options defined below. All are optional.
1199 -Filename
1201 The database filename. If no filename is specified, a temporary
1202 file will be created and removed once the program terminates.
1203 -Subname
1205 Specifies the name of the sub-database to open. This option is
1206 only valid if you are using Berkeley DB 3.x or greater.
1207 -Flags
1209 Specify how the database will be opened/created. The valid flags
1210 are:
1211 DB_CREATE
1213 Create any underlying files, as necessary. If the files do not
1215 already exist and the DB_CREATE flag is not specified, the call
1216 will fail.
1217 DB_NOMMAP
1219 Not supported by BerkeleyDB.
1221 DB_RDONLY
1223 Opens the database in read-only mode.
1225 DB_THREAD
1227 Not supported by BerkeleyDB.
1229 DB_TRUNCATE
1231 If the database file already exists, remove all the data before
1233 opening it.
1234 -Mode
1236 Determines the file protection when the database is created.
1237 Defaults to 0666.
1238 -Cachesize
1240 -Lorder
1241 -Pagesize
1242 -Env When working under a Berkeley DB environment, this parameter
1243 Defaults to no environment.
1245 -Encrypt
1247 If present, this parameter will enable encryption of all data
1248 before it is written to the database. This parameters must be
1249 given a hash reference. The format is shown below.
1250 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES }
1252 Valid values for the Flags are 0 or "DB_ENCRYPT_AES".
1254 This option requires Berkeley DB 4.1 or better.
1256 -Txn TODO.
1258
1260 All the database interfaces support the common set of methods defined
1261 below.
1262 All the methods below return 0 to indicate success.
1264 $status = $db->db_get($key, $value [, $flags])
1266 Given a key ($key) this method reads the value associated with it from
1267 the database. If it exists, the value read from the database is
1268 returned in the $value parameter.
1269 The $flags parameter is optional. If present, it must be set to one of
1271 the following values:
1272 DB_GET_BOTH
1274 When the DB_GET_BOTH flag is specified, db_get checks for the
1275 existence of both the $key and $value in the database.
1276 DB_SET_RECNO
1278 TODO.
1279 In addition, the following value may be set by bitwise OR'ing it into
1281 the $flags parameter:
1282 DB_RMW
1284 TODO
1285 The variant "db_pget" allows you to query a secondary database:
1287 $status = $sdb->db_pget($skey, $pkey, $value);
1289 using the key $skey in the secondary db to lookup $pkey and $value from
1291 the primary db.
1292 $status = $db->db_put($key, $value [, $flags])
1294 Stores a key/value pair in the database.
1295 The $flags parameter is optional. If present it must be set to one of
1297 the following values:
1298 DB_APPEND
1300 This flag is only applicable when accessing a BerkeleyDB::Recno
1301 database.
1302 TODO.
1304 DB_NOOVERWRITE
1306 If this flag is specified and $key already exists in the database,
1307 the call to db_put will return DB_KEYEXIST.
1308 $status = $db->db_del($key [, $flags])
1310 Deletes a key/value pair in the database associated with $key. If
1311 duplicate keys are enabled in the database, db_del will delete all
1312 key/value pairs with key $key.
1313 The $flags parameter is optional and is currently unused.
1315 $status = $env->stat_print([$flags])
1317 Prints statistical information.
1318 If the "MsgFile" option is specified the output will be sent to the
1320 file. Otherwise output is sent to standard output.
1321 This option requires Berkeley DB 4.3 or better.
1323 $status = $db->db_sync()
1325 If any parts of the database are in memory, write them to the database.
1326 $cursor = $db->db_cursor([$flags])
1328 Creates a cursor object. This is used to access the contents of the
1329 database sequentially. See CURSORS for details of the methods available
1330 when working with cursors.
1331 The $flags parameter is optional. If present it must be set to one of
1333 the following values:
1334 DB_RMW
1336 TODO.
1337 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ;
1339 TODO
1340 ($flag, $old_offset, $old_length) = $db->partial_clear() ;
1342 TODO
1343 $db->byteswapped()
1345 TODO
1346 $db->type()
1348 Returns the type of the database. The possible return code are DB_HASH
1349 for a BerkeleyDB::Hash database, DB_BTREE for a BerkeleyDB::Btree
1350 database and DB_RECNO for a BerkeleyDB::Recno database. This method is
1351 typically used when a database has been opened with
1352 BerkeleyDB::Unknown.
1353 $bool = $env->cds_enabled();
1355 Returns true if the Berkeley DB environment $env has been opened on CDS
1356 mode.
1357 $bool = $db->cds_enabled();
1359 Returns true if the database $db has been opened on CDS mode.
1360 $lock = $db->cds_lock();
1362 Creates a CDS write lock object $lock.
1363 It is a fatal error to attempt to create a cds_lock if the Berkeley DB
1365 environment has not been opened in CDS mode.
1366 $lock->cds_unlock();
1368 Removes a CDS lock. The destruction of the CDS lock object
1369 automatically calls this method.
1370 Note that if multiple CDS lock objects are created, the underlying
1372 write lock will not be released until all CDS lock objects are either
1373 explictly unlocked with this method, or the CDS lock objects have been
1374 destroyed.
1375 $ref = $db->db_stat()
1377 Returns a reference to an associative array containing information
1378 about the database. The keys of the associative array correspond
1379 directly to the names of the fields defined in the Berkeley DB
1380 documentation. For example, in the DB documentation, the field
1381 bt_version stores the version of the Btree database. Assuming you
1382 called db_stat on a Btree database the equivalent field would be
1383 accessed as follows:
1384 $version = $ref->{'bt_version'} ;
1386 If you are using Berkeley DB 3.x or better, this method will work will
1388 all database formats. When DB 2.x is used, it only works with
1389 BerkeleyDB::Btree.
1390 $status = $db->status()
1392 Returns the status of the last $db method called.
1393 $status = $db->truncate($count)
1395 Truncates the datatabase and returns the number or records deleted in
1396 $count.
1397 $status = $db->compact($start, $stop, $c_data, $flags, $end);
1399 Compacts the database $db.
1400 All the parameters are optional - if only want to make use of some of
1402 them, use "undef" for those you don't want. Trailing unusused
1403 parameters can be omitted. For example, if you only want to use the
1404 $c_data parameter to set the "compact_fillpercent", write you code like
1405 this
1406 my %hash;
1408 $hash{compact_fillpercent} = 50;
1409 $db->compact(undef, undef, \%hash);
1410 The parameters operate identically to the C equivalent of this method.
1412 The $c_data needs a bit of explanation - it must be a hash reference.
1413 The values of the following keys can be set before calling "compact"
1414 and will affect the operation of the compaction.
1415 · compact_fillpercent
1417 · compact_timeout
1419 The following keys, along with associated values, will be created in
1421 the hash reference if the "compact" operation was successful.
1422 · compact_deadlock
1424 · compact_levels
1426 · compact_pages_free
1428 · compact_pages_examine
1430 · compact_pages_truncated
1432 You need to be running Berkeley DB 4.4 or better if you want to make
1434 use of "compact".
1435 $status = $db->associate($secondary, \&key_callback)
1437 Associate $db with the secondary DB $secondary
1438 New key/value pairs inserted to the database will be passed to the
1440 callback which must set its third argument to the secondary key to
1441 allow lookup. If an array reference is set multiple keys secondary keys
1442 will be associated with the primary database entry.
1443 Data may be retrieved fro the secondary database using "db_pget" to
1445 also obtain the primary key.
1446 Secondary databased are maintained automatically.
1448 $status = $db->associate_foreign($secondary, callback, $flags)
1450 Associate a foreign key database $db with the secondary DB $secondary.
1451 The second parameter must be a reference to a sub or "undef".
1453 The $flags parameter must be either "DB_FOREIGN_CASCADE",
1455 "DB_FOREIGN_ABORT" or "DB_FOREIGN_NULLIFY".
1456 When the flags parameter is "DB_FOREIGN_NULLIFY" the second parameter
1458 is a reference to a sub of the form
1459 sub foreign_cb
1461 {
1462 my $key = \$_[0];
1463 my $value = \$_[1];
1464 my $foreignkey = \$_[2];
1465 my $changed = \$_[3] ;
1466 # for ... set $$value and set $$changed to 1
1468 return 0;
1470 }
1471 $foreign_db->associate_foreign($secondary, \&foreign_cb, DB_FOREIGN_NULLIFY);
1473
1475 A cursor is used whenever you want to access the contents of a database
1476 in sequential order. A cursor object is created with the "db_cursor"
1477 A cursor object has the following methods available:
1479 $newcursor = $cursor->c_dup($flags)
1481 Creates a duplicate of $cursor. This method needs Berkeley DB 3.0.x or
1482 better.
1483 The $flags parameter is optional and can take the following value:
1485 DB_POSITION
1487 When present this flag will position the new cursor at the same
1488 place as the existing cursor.
1489 $status = $cursor->c_get($key, $value, $flags)
1491 Reads a key/value pair from the database, returning the data in $key
1492 and $value. The key/value pair actually read is controlled by the
1493 $flags parameter, which can take one of the following values:
1494 DB_FIRST
1496 Set the cursor to point to the first key/value pair in the
1497 database. Return the key/value pair in $key and $value.
1498 DB_LAST
1500 Set the cursor to point to the last key/value pair in the
1501 database. Return the key/value pair in $key and $value.
1502 DB_NEXT
1504 If the cursor is already pointing to a key/value pair, it will be
1505 incremented to point to the next key/value pair and return its
1506 contents.
1507 If the cursor isn't initialised, DB_NEXT works just like DB_FIRST.
1509 If the cursor is already positioned at the last key/value pair,
1511 c_get will return DB_NOTFOUND.
1512 DB_NEXT_DUP
1514 This flag is only valid when duplicate keys have been enabled in a
1515 database. If the cursor is already pointing to a key/value pair
1516 and the key of the next key/value pair is identical, the cursor
1517 will be incremented to point to it and their contents returned.
1518 DB_PREV
1520 If the cursor is already pointing to a key/value pair, it will be
1521 decremented to point to the previous key/value pair and return its
1522 contents.
1523 If the cursor isn't initialised, DB_PREV works just like DB_LAST.
1525 If the cursor is already positioned at the first key/value pair,
1527 c_get will return DB_NOTFOUND.
1528 DB_CURRENT
1530 If the cursor has been set to point to a key/value pair, return
1531 their contents. If the key/value pair referenced by the cursor
1532 has been deleted, c_get will return DB_KEYEMPTY.
1533 DB_SET
1535 Set the cursor to point to the key/value pair referenced by $key
1536 and return the value in $value.
1537 DB_SET_RANGE
1539 This flag is a variation on the DB_SET flag. As well as returning
1540 the value, it also returns the key, via $key. When used with a
1541 BerkeleyDB::Btree database the key matched by c_get will be the
1542 shortest key (in length) which is greater than or equal to the key
1543 supplied, via $key. This allows partial key searches. See ??? for
1544 an example of how to use this flag.
1545 DB_GET_BOTH
1547 Another variation on DB_SET. This one returns both the key and the
1548 value.
1549 DB_SET_RECNO
1551 TODO.
1552 DB_GET_RECNO
1554 TODO.
1555 In addition, the following value may be set by bitwise OR'ing it into
1557 the $flags parameter:
1558 DB_RMW
1560 TODO.
1561 $status = $cursor->c_put($key, $value, $flags)
1563 Stores the key/value pair in the database. The position that the data
1564 is stored in the database is controlled by the $flags parameter, which
1565 must take one of the following values:
1566 DB_AFTER
1568 When used with a Btree or Hash database, a duplicate of the key
1569 referenced by the current cursor position will be created and the
1570 contents of $value will be associated with it - $key is ignored.
1571 The new key/value pair will be stored immediately after the
1572 current cursor position. Obviously the database has to have been
1573 opened with DB_DUP.
1574 When used with a Recno ... TODO
1576 DB_BEFORE
1578 When used with a Btree or Hash database, a duplicate of the key
1579 referenced by the current cursor position will be created and the
1580 contents of $value will be associated with it - $key is ignored.
1581 The new key/value pair will be stored immediately before the
1582 current cursor position. Obviously the database has to have been
1583 opened with DB_DUP.
1584 When used with a Recno ... TODO
1586 DB_CURRENT
1588 If the cursor has been initialised, replace the value of the
1589 key/value pair stored in the database with the contents of $value.
1590 DB_KEYFIRST
1592 Only valid with a Btree or Hash database. This flag is only really
1593 used when duplicates are enabled in the database and sorted
1594 duplicates haven't been specified. In this case the key/value
1595 pair will be inserted as the first entry in the duplicates for the
1596 particular key.
1597 DB_KEYLAST
1599 Only valid with a Btree or Hash database. This flag is only really
1600 used when duplicates are enabled in the database and sorted
1601 duplicates haven't been specified. In this case the key/value
1602 pair will be inserted as the last entry in the duplicates for the
1603 particular key.
1604 $status = $cursor->c_del([$flags])
1606 This method deletes the key/value pair associated with the current
1607 cursor position. The cursor position will not be changed by this
1608 operation, so any subsequent cursor operation must first initialise the
1609 cursor to point to a valid key/value pair.
1610 If the key/value pair associated with the cursor have already been
1612 deleted, c_del will return DB_KEYEMPTY.
1613 The $flags parameter is not used at present.
1615 $status = $cursor->c_count($cnt [, $flags])
1617 Stores the number of duplicates at the current cursor position in $cnt.
1618 The $flags parameter is not used at present. This method needs Berkeley
1620 DB 3.1 or better.
1621 $status = $cursor->status()
1623 Returns the status of the last cursor method as a dual type.
1624 $status = $cursor->c_pget() ;
1626 See "db_pget"
1627 $status = $cursor->c_close()
1629 Closes the cursor $cursor.
1630 Cursor Examples
1632 TODO
1633 Iterating from first to last, then in reverse.
1635 examples of each of the flags.
1637
1639 Join support for BerkeleyDB is in progress. Watch this space.
1640 TODO
1642
1644 Transactions are created using the "txn_begin" method on
1645 BerkeleyDB::Env:
1646 my $txn = $env->txn_begin;
1648 If this is a nested transaction, supply the parent transaction as an
1650 argument:
1651 my $child_txn = $env->txn_begin($parent_txn);
1653 Then in order to work with the transaction, you must set it as the
1655 current transaction on the database handles you want to work with:
1656 $db->Txn($txn);
1658 Or for multiple handles:
1660 $txn->Txn(@handles);
1662 The current transaction is given by BerkeleyDB each time to the various
1664 BDB operations. In the C api it is required explicitly as an argument
1665 to every operation.
1666 To commit a transaction call the "commit" method on it:
1668 $txn->commit;
1670 and to roll back call abort:
1672 $txn->abort
1674 After committing or aborting a child transaction you need to set the
1676 active transaction again using "Txn".
1677
1679 The Berkeley DB Concurrent Data Store (CDS) is a lightweight locking
1680 mechanism that is useful in scenarios where transactions are overkill.
1681 What is CDS?
1683 The Berkeley DB CDS interface is a simple lightweight locking mechanism
1684 that allows safe concurrent access to Berkeley DB databases. Your
1685 application can have multiple reader and write processes, but Berkeley
1686 DB will arrange it so that only one process can have a write lock
1687 against the database at a time, i.e. multiple processes can read from a
1688 database concurrently, but all write processes will be serialised.
1689 Should I use it?
1691 Whilst this simple locking model is perfectly adequate for some
1692 applications, it will be too restrictive for others. Before deciding on
1693 using CDS mode, you need to be sure that it is suitable for the
1694 expected behaviour of your application.
1695 The key features of this model are
1697 · All writes operations are serialised.
1699 · A write operation will block until all reads have finished.
1701 There are a few of the attributes of your application that you need to
1703 be aware of before choosing to use CDS.
1704 Firstly, if you application needs either recoverability or transaction
1706 support, then CDS will not be suitable.
1707 Next what is the ratio of read operation to write operations will your
1709 application have?
1710 If it is carrying out mostly read operations, and very few writes, then
1712 CDS may be appropriate.
1713 What is the expected throughput of reads/writes in your application?
1715 If you application does 90% writes and 10% reads, but on average you
1717 only have a transaction every 5 seconds, then the fact that all writes
1718 are serialised will not matter, because there will hardly ever be
1719 multiple writes processes blocking.
1720 In summary CDS mode may be appropriate for your application if it
1722 performs mostly reads and very few writes or there is a low throughput.
1723 Also, if you do not need to be able to roll back a series of database
1724 operations if an error occurs, then CDS is ok.
1725 If any of these is not the case you will need to use Berkeley DB
1727 transactions. That is outside the scope of this document.
1728 Locking Used
1730 Berkeley DB implements CDS mode using two kinds of lock behind the
1731 scenes - namely read locks and write locks. A read lock allows multiple
1732 processes to access the database for reading at the same time. A write
1733 lock will only get access to the database when there are no read or
1734 write locks active. The write lock will block until the process
1735 holding the lock releases it.
1736 Multiple processes with read locks can all access the database at the
1738 same time as long as no process has a write lock. A process with a
1739 write lock can only access the database if there are no other active
1740 read or write locks.
1741 The majority of the time the Berkeley DB CDS mode will handle all
1743 locking without your application having to do anything. There are a
1744 couple of exceptions you need to be aware of though - these will be
1745 discussed in "Safely Updating Records" and "Implicit Cursors" below.
1746 A Berkeley DB Cursor (created with "$db->db_cursor") will by hold a
1748 lock on the database until it is either explicitly closed or destroyed.
1749 This means the lock has the potential to be long lived.
1750 By default Berkeley DB cursors create a read lock, but it is possible
1752 to create a cursor that holds a write lock, thus
1753 $cursor = $db->db_cursor(DB_WRITECURSOR);
1755 Whilst either a read or write cursor is active, it will block any other
1757 processes that wants to write to the database.
1758 To avoid blocking problems, only keep cursors open as long as they are
1760 needed. The same is true when you use the "cursor" method or the
1761 "cds_lock" method.
1762 For full information on CDS see the "Berkeley DB Concurrent Data Store
1764 applications" section in the Berkeley DB Reference Guide.
1765 Opening a database for CDS
1767 Here is the typical signature that is used when opening a database in
1768 CDS mode.
1769 use BerkeleyDB ;
1771 my $env = new BerkeleyDB::Env
1773 -Home => "./home" ,
1774 -Flags => DB_CREATE| DB_INIT_CDB | DB_INIT_MPOOL
1775 or die "cannot open environment: $BerkeleyDB::Error\n";
1776 my $db = new BerkeleyDB::Hash
1778 -Filename => 'test1.db',
1779 -Flags => DB_CREATE,
1780 -Env => $env
1781 or die "cannot open database: $BerkeleyDB::Error\n";
1782 or this, if you use the tied interface
1784 tie %hash, "BerkeleyDB::Hash",
1786 -Filename => 'test2.db',
1787 -Flags => DB_CREATE,
1788 -Env => $env
1789 or die "cannot open database: $BerkeleyDB::Error\n";
1790 The first thing to note is that you MUST always use a Berkeley DB
1792 environment if you want to use locking with Berkeley DB.
1793 Remember, that apart from the actual database files you explicitly
1795 create yourself, Berkeley DB will create a few behind the scenes to
1796 handle locking - they usually have names like "__db.001". It is
1797 therefore a good idea to use the "-Home" option, unless you are happy
1798 for all these files to be written in the current directory.
1799 Next, remember to include the "DB_CREATE" flag when opening the
1801 environment for the first time. A common mistake is to forget to add
1802 this option and then wonder why the application doesn't work.
1803 Finally, it is vital that all processes that are going to access the
1805 database files use the same Berkeley DB environment.
1806 Safely Updating a Record
1808 One of the main gotchas when using CDS is if you want to update a
1809 record in a database, i.e. you want to retrieve a record from a
1810 database, modify it in some way and put it back in the database.
1811 For example, say you are writing a web application and you want to keep
1813 a record of the number of times your site is accessed in a Berkeley DB
1814 database. So your code will have a line of code like this (assume, of
1815 course, that %hash has been tied to a Berkeley DB database):
1816 $hash{Counter} ++ ;
1818 That may look innocent enough, but there is a race condition lurking in
1820 there. If I rewrite the line of code using the low-level Berkeley DB
1821 API, which is what will actually be executed, the race condition may be
1822 more apparent:
1823 $db->db_get("Counter", $value);
1825 ++ $value ;
1826 $db->db_put("Counter", $value);
1827 Consider what happens behind the scenes when you execute the commands
1829 above. Firstly, the existing value for the key "Counter" is fetched
1830 from the database using "db_get". A read lock will be used for this
1831 part of the update. The value is then incremented, and the new value
1832 is written back to the database using "db_put". This time a write lock
1833 will be used.
1834 Here's the problem - there is nothing to stop two (or more) processes
1836 executing the read part at the same time. Remember multiple processes
1837 can hold a read lock on the database at the same time. So both will
1838 fetch the same value, let's say 7, from the database. Both increment
1839 the value to 8 and attempt to write it to the database. Berkeley DB
1840 will ensure that only one of the processes gets a write lock, while the
1841 other will be blocked. So the process that happened to get the write
1842 lock will store the value 8 to the database and release the write lock.
1843 Now the other process will be unblocked, and it too will write the
1844 value 8 to the database. The result, in this example, is we have missed
1845 a hit in the counter.
1846 To deal with this kind of scenario, you need to make the update atomic.
1848 A convenience method, called "cds_lock", is supplied with the
1849 BerkeleyDB module for this purpose. Using "cds_lock", the counter
1850 update code can now be rewritten thus:
1851 my $lk = $dbh->cds_lock() ;
1853 $hash{Counter} ++ ;
1854 $lk->cds_unlock;
1855 or this, where scoping is used to limit the lifetime of the lock object
1857 {
1859 my $lk = $dbh->cds_lock() ;
1860 $hash{Counter} ++ ;
1861 }
1862 Similarly, "cds_lock" can be used with the native Berkeley DB API
1864 my $lk = $dbh->cds_lock() ;
1866 $db->db_get("Counter", $value);
1867 ++ $value ;
1868 $db->db_put("Counter", $value);
1869 $lk->unlock;
1870 The "cds_lock" method will ensure that the current process has
1872 exclusive access to the database until the lock is either explicitly
1873 released, via the "$lk->cds_unlock()" or by the lock object being
1874 destroyed.
1875 If you are interested, all that "cds_lock" does is open a "write"
1877 cursor. This has the useful side-effect of holding a write-lock on the
1878 database until the cursor is deleted. This is how you create a write-
1879 cursor
1880 $cursor = $db->db_cursor(DB_WRITECURSOR);
1882 If you have instantiated multiple "cds_lock" objects for one database
1884 within a single process, that process will hold a write-lock on the
1885 database until ALL "cds_lock" objects have been destroyed.
1886 As with all write-cursors, you should try to limit the scope of the
1888 "cds_lock" to as short a time as possible. Remember the complete
1889 database will be locked to other process whilst the write lock is in
1890 place.
1891 Cannot write with a read cursor while a write cursor is active
1893 This issue is easier to demonstrate with an example, so consider the
1894 code below. The intention of the code is to increment the values of all
1895 the elements in a database by one.
1896 # Assume $db is a database opened in a CDS environment.
1898 # Create a write-lock
1900 my $lock = $db->db_cursor(DB_WRITECURSOR);
1901 # or
1902 # my $lock = $db->cds_lock();
1903 my $cursor = $db->db_cursor();
1906 # Now loop through the database, and increment
1908 # each value using c_put.
1909 while ($cursor->c_get($key, $value, DB_NEXT) == 0)
1910 {
1911 $cursor->c_put($key, $value+1, DB_CURRENT) == 0
1912 or die "$BerkeleyDB::Error\n";
1913 }
1914 When this code is run, it will fail on the "c_put" line with this error
1916 Write attempted on read-only cursor
1918 The read cursor has automatically disallowed a write operation to
1920 prevent a deadlock.
1921 So the rule is -- you CANNOT carry out a write operation using a read-
1923 only cursor (i.e. you cannot use "c_put" or "c_del") whilst another
1924 write-cursor is already active.
1925 The workaround for this issue is to just use "db_put" instead of
1927 "c_put", like this
1928 # Assume $db is a database opened in a CDS environment.
1930 # Create a write-lock
1932 my $lock = $db->db_cursor(DB_WRITECURSOR);
1933 # or
1934 # my $lock = $db->cds_lock();
1935 my $cursor = $db->db_cursor();
1938 # Now loop through the database, and increment
1940 # each value using c_put.
1941 while ($cursor->c_get($key, $value, DB_NEXT) == 0)
1942 {
1943 $db->db_put($key, $value+1) == 0
1944 or die "$BerkeleyDB::Error\n";
1945 }
1946 Implicit Cursors
1948 All Berkeley DB cursors will hold either a read lock or a write lock on
1949 the database for the existence of the cursor. In order to prevent
1950 blocking of other processes you need to make sure that they are not
1951 long lived.
1952 There are a number of instances where the Perl interface to Berkeley DB
1954 will create a cursor behind the scenes without you being aware of it.
1955 Most of these are very short-lived and will not affect the running of
1956 your script, but there are a few notable exceptions.
1957 Consider this snippet of code
1959 while (my ($k, $v) = each %hash)
1961 {
1962 # do something
1963 }
1964 To implement the "each" functionality, a read cursor will be created
1966 behind the scenes to allow you to iterate through the tied hash, %hash.
1967 While that cursor is still active, a read lock will obviously be held
1968 against the database. If your application has any other writing
1969 processes, these will be blocked until the read cursor is closed. That
1970 won't happen until the loop terminates.
1971 To avoid blocking problems, only keep cursors open as long as they are
1973 needed. The same is true when you use the "cursor" method or the
1974 "cds_lock" method.
1975 The locking behaviour of the "values" or "keys" functions, shown below,
1977 is subtly different.
1978 foreach my $k (keys %hash)
1980 {
1981 # do something
1982 }
1983 foreach my $v (values %hash)
1985 {
1986 # do something
1987 }
1988 Just as in the "each" function, a read cursor will be created to
1990 iterate over the database in both of these cases. Where "keys" and
1991 "values" differ is the place where the cursor carries out the iteration
1992 through the database. Whilst "each" carried out a single iteration
1993 every time it was invoked, the "keys" and "values" functions will
1994 iterate through the entire database in one go -- the complete database
1995 will be read into memory before the first iteration of the loop.
1996 Apart from the fact that a read lock will be held for the amount of
1998 time required to iterate through the database, the use of "keys" and
1999 "values" is not recommended because it will result in the complete
2000 database being read into memory.
2001 Avoiding Deadlock with multiple databases
2003 If your CDS application uses multiple database files, and you need to
2004 write to more than one of them, you need to be careful you don't create
2005 a deadlock.
2006 For example, say you have two databases, D1 and D2, and two processes,
2008 P1 and P2. Assume you want to write a record to each database. If P1
2009 writes the records to the databases in the order D1, D2 while process
2010 P2 writes the records in the order D2, D1, there is the potential for a
2011 deadlock to occur.
2012 This scenario can be avoided by either always acquiring the write locks
2014 in exactly the same order in your application code, or by using the
2015 "DB_CDB_ALLDB" flag when opening the environment. This flag will make a
2016 write-lock apply to all the databases in the environment.
2017 Add example here
2019
2021 A DBM Filter is a piece of code that is be used when you always want to
2022 make the same transformation to all keys and/or values in a DBM
2023 database. All of the database classes (BerkeleyDB::Hash,
2024 BerkeleyDB::Btree and BerkeleyDB::Recno) support DBM Filters.
2025 There are four methods associated with DBM Filters. All work
2027 identically, and each is used to install (or uninstall) a single DBM
2028 Filter. Each expects a single parameter, namely a reference to a sub.
2029 The only difference between them is the place that the filter is
2030 installed.
2031 To summarise:
2033 filter_store_key
2035 If a filter has been installed with this method, it will be
2036 invoked every time you write a key to a DBM database.
2037 filter_store_value
2039 If a filter has been installed with this method, it will be
2040 invoked every time you write a value to a DBM database.
2041 filter_fetch_key
2043 If a filter has been installed with this method, it will be
2044 invoked every time you read a key from a DBM database.
2045 filter_fetch_value
2047 If a filter has been installed with this method, it will be
2048 invoked every time you read a value from a DBM database.
2049 You can use any combination of the methods, from none, to all four.
2051 All filter methods return the existing filter, if present, or "undef"
2053 in not.
2054 To delete a filter pass "undef" to it.
2056 The Filter
2058 When each filter is called by Perl, a local copy of $_ will contain the
2059 key or value to be filtered. Filtering is achieved by modifying the
2060 contents of $_. The return code from the filter is ignored.
2061 An Example -- the NULL termination problem.
2063 Consider the following scenario. You have a DBM database that you need
2064 to share with a third-party C application. The C application assumes
2065 that all keys and values are NULL terminated. Unfortunately when Perl
2066 writes to DBM databases it doesn't use NULL termination, so your Perl
2067 application will have to manage NULL termination itself. When you write
2068 to the database you will have to use something like this:
2069 $hash{"$key\0"} = "$value\0" ;
2071 Similarly the NULL needs to be taken into account when you are
2073 considering the length of existing keys/values.
2074 It would be much better if you could ignore the NULL terminations issue
2076 in the main application code and have a mechanism that automatically
2077 added the terminating NULL to all keys and values whenever you write to
2078 the database and have them removed when you read from the database. As
2079 I'm sure you have already guessed, this is a problem that DBM Filters
2080 can fix very easily.
2081 use strict ;
2083 use BerkeleyDB ;
2084 my %hash ;
2086 my $filename = "filt.db" ;
2087 unlink $filename ;
2088 my $db = tie %hash, 'BerkeleyDB::Hash',
2090 -Filename => $filename,
2091 -Flags => DB_CREATE
2092 or die "Cannot open $filename: $!\n" ;
2093 # Install DBM Filters
2095 $db->filter_fetch_key ( sub { s/\0$// } ) ;
2096 $db->filter_store_key ( sub { $_ .= "\0" } ) ;
2097 $db->filter_fetch_value( sub { s/\0$// } ) ;
2098 $db->filter_store_value( sub { $_ .= "\0" } ) ;
2099 $hash{"abc"} = "def" ;
2101 my $a = $hash{"ABC"} ;
2102 # ...
2103 undef $db ;
2104 untie %hash ;
2105 Hopefully the contents of each of the filters should be self-
2107 explanatory. Both "fetch" filters remove the terminating NULL, and both
2108 "store" filters add a terminating NULL.
2109 Another Example -- Key is a C int.
2111 Here is another real-life example. By default, whenever Perl writes to
2112 a DBM database it always writes the key and value as strings. So when
2113 you use this:
2114 $hash{12345} = "something" ;
2116 the key 12345 will get stored in the DBM database as the 5 byte string
2118 "12345". If you actually want the key to be stored in the DBM database
2119 as a C int, you will have to use "pack" when writing, and "unpack" when
2120 reading.
2121 Here is a DBM Filter that does it:
2123 use strict ;
2125 use BerkeleyDB ;
2126 my %hash ;
2127 my $filename = "filt.db" ;
2128 unlink $filename ;
2129 my $db = tie %hash, 'BerkeleyDB::Btree',
2132 -Filename => $filename,
2133 -Flags => DB_CREATE
2134 or die "Cannot open $filename: $!\n" ;
2135 $db->filter_fetch_key ( sub { $_ = unpack("i", $_) } ) ;
2137 $db->filter_store_key ( sub { $_ = pack ("i", $_) } ) ;
2138 $hash{123} = "def" ;
2139 # ...
2140 undef $db ;
2141 untie %hash ;
2142 This time only two filters have been used -- we only need to manipulate
2144 the contents of the key, so it wasn't necessary to install any value
2145 filters.
2146
2148 Both BerkeleyDB::Hash and BerkeleyDB::Btree can be used with the MLDBM
2149 module. The code fragment below shows how to open associate MLDBM with
2150 BerkeleyDB::Btree. To use BerkeleyDB::Hash just replace
2151 BerkeleyDB::Btree with BerkeleyDB::Hash.
2152 use strict ;
2154 use BerkeleyDB ;
2155 use MLDBM qw(BerkeleyDB::Btree) ;
2156 use Data::Dumper;
2157 my $filename = 'testmldbm' ;
2159 my %o ;
2160 unlink $filename ;
2162 tie %o, 'MLDBM', -Filename => $filename,
2163 -Flags => DB_CREATE
2164 or die "Cannot open database '$filename: $!\n";
2165 See the MLDBM documentation for information on how to use the module
2167 and for details of its limitations.
2168
2170 TODO.
2171
2173 Sharing Databases With C Applications
2174 There is no technical reason why a Berkeley DB database cannot be
2175 shared by both a Perl and a C application.
2176 The vast majority of problems that are reported in this area boil down
2178 to the fact that C strings are NULL terminated, whilst Perl strings are
2179 not. See "An Example -- the NULL termination problem." in the DBM
2180 FILTERS section for a generic way to work around this problem.
2181 The untie Gotcha
2183 TODO
2184
2186 This section attempts to answer some of the more common questions that
2187 I get asked.
2188 Relationship with DB_File
2190 Before Berkeley DB 2.x was written there was only one Perl module that
2191 interfaced to Berkeley DB. That module is called DB_File. Although
2192 DB_File can be build with Berkeley DB 1.x, 2.x, 3.x or 4.x, it only
2193 provides an interface to the functionality available in Berkeley DB
2194 1.x. That means that it doesn't support transactions, locking or any of
2195 the other new features available in DB 2.x or better.
2196 How do I store Perl data structures with BerkeleyDB?
2198 See "Using BerkeleyDB with MLDBM".
2199
2201 See the Changes file.
2202
2204 The most recent version of BerkeleyDB can always be found on CPAN (see
2205 "CPAN" in perlmod for details), in the directory
2206 modules/by-module/BerkeleyDB.
2207 The official web site for Berkeley DB is
2209 http://www.oracle.com/technology/products/berkeley-db/db/index.html.
2210
2212 Copyright (c) 1997-2004 Paul Marquess. All rights reserved. This
2213 program is free software; you can redistribute it and/or modify it
2214 under the same terms as Perl itself.
2215 Although BerkeleyDB is covered by the Perl license, the library it
2217 makes use of, namely Berkeley DB, is not. Berkeley DB has its own
2218 copyright and its own license. Please take the time to read it.
2219 Here are few words taken from the Berkeley DB FAQ (at
2221 http://www.oracle.com/technology/products/berkeley-db/db/index.html)
2222 regarding the license:
2223 Do I have to license DB to use it in Perl scripts?
2225 No. The Berkeley DB license requires that software that uses
2227 Berkeley DB be freely redistributable. In the case of Perl, that
2228 software is Perl, and not your scripts. Any Perl scripts that you
2229 write are your property, including scripts that make use of Berkeley
2230 DB. Neither the Perl license nor the Berkeley DB license
2231 place any restriction on what you may do with them.
2232 If you are in any doubt about the license situation, contact either the
2234 Berkeley DB authors or the author of BerkeleyDB. See "AUTHOR" for
2235 details.
2236
2238 Paul Marquess <pmqs@cpan.org>.
2239
2241 perl(1), DB_File, Berkeley DB.
2242perl v5.12.0 2009-10-13 BerkeleyDB(3)