1BDB(3) User Contributed Perl Documentation BDB(3)
2
6 BDB - Asynchronous Berkeley DB access
7
9 use BDB;
10 my $env = db_env_create;
12 mkdir "bdtest", 0700;
14 db_env_open
15 $env,
16 "bdtest",
17 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL
18 | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
19 0600;
20 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
22 my $db = db_create $env;
24 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE
25 | BDB::READ_UNCOMMITTED, 0600;
26 db_put $db, undef, "key", "data", 0, sub {
27 db_del $db, undef, "key";
28 };
29 db_sync $db;
30 # when you also use Coro, management is easy:
32 use Coro::BDB;
33 # automatic event loop integration with AnyEvent:
35 use AnyEvent::BDB;
36 # automatic result processing with EV:
38 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
39 # with Glib:
41 add_watch Glib::IO BDB::poll_fileno,
42 in => sub { BDB::poll_cb; 1 };
43 # or simply flush manually
45 BDB::flush;
46
48 See the BerkeleyDB documentation
49 (http://www.oracle.com/technology/documentation/berkeley-db/db/index.html
50 <http://www.oracle.com/technology/documentation/berkeley-
51 db/db/index.html>). The BDB API is very similar to the C API (the
52 translation has been very faithful).
53 See also the example sections in the document below and possibly the
55 eg/ subdirectory of the BDB distribution. Last not least see the
56 IO::AIO documentation, as that module uses almost the same asynchronous
57 request model as this module.
58 I know this is woefully inadequate documentation. Send a patch!
60
62 Every request method creates a request. which is a C data structure not
63 directly visible to Perl.
64 During their existance, bdb requests travel through the following
66 states, in order:
67 ready
69 Immediately after a request is created it is put into the ready
70 state, waiting for a thread to execute it.
71 execute
73 A thread has accepted the request for processing and is currently
74 executing it (e.g. blocking in read).
75 pending
77 The request has been executed and is waiting for result processing.
78 While request submission and execution is fully asynchronous,
80 result processing is not and relies on the perl interpreter calling
81 "poll_cb" (or another function with the same effect).
82 result
84 The request results are processed synchronously by "poll_cb".
85 The "poll_cb" function will process all outstanding aio requests by
87 calling their callbacks, freeing memory associated with them and
88 managing any groups they are contained in.
89 done
91 Request has reached the end of its lifetime and holds no resources
92 anymore (except possibly for the Perl object, but its connection to
93 the actual aio request is severed and calling its methods will
94 either do nothing or result in a runtime error).
95
97 All of these are functions. The create functions simply return a new
98 object and never block. All the remaining functions take an optional
99 callback as last argument. If it is missing, then the function will be
100 executed synchronously. In both cases, $! will reflect the return value
101 of the function.
102 BDB functions that cannot block (mostly functions that manipulate
104 settings) are method calls on the relevant objects, so the rule of
105 thumb is: if it's a method, it's not blocking, if it's a function, it
106 takes a callback as last argument.
107 In the following, $int signifies an integer return value,
109 "bdb_filename" is a "filename" (octets on unix, madness on windows),
110 "U32" is an unsigned 32 bit integer, "int" is some integer, "NV" is a
111 floating point value.
112 Most "SV *" types are generic perl scalars (for input and output of
114 data values).
115 The various "DB_ENV" etc. arguments are handles return by
117 "db_env_create", "db_create", "txn_begin" and so on. If they have an
118 appended "_ornull" this means they are optional and you can pass
119 "undef" for them, resulting a NULL pointer on the C level.
120 The "SV *callback" is the optional callback function to call when the
122 request is completed. This last callback argument is special: the
123 callback is simply the last argument passed. If there are "optional"
124 arguments before the callback they can be left out. The callback itself
125 can be left out or specified as "undef", in which case the function
126 will be executed synchronously.
127 For example, "db_env_txn_checkpoint" usually is called with all integer
129 arguments zero. These can be left out, so all of these specify a call
130 to "DB_ENV->txn_checkpoint", to be executed asynchronously with a
131 callback to be called:
132 db_env_txn_checkpoint $db_env, 0, 0, 0, sub { };
134 db_env_txn_checkpoint $db_env, 0, 0, sub { };
135 db_env_txn_checkpoint $db_env, sub { };
136 While these all specify a call to "DB_ENV->txn_checkpoint" to be
138 executed synchronously:
139 db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
141 db_env_txn_checkpoint $db_env, 0, 0, 0;
142 db_env_txn_checkpoint $db_env, 0;
143 BDB functions
145 Functions in the BDB namespace, exported by default:
146 $env = db_env_create (U32 env_flags = 0)
148 flags: RPCCLIENT
149 db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = 0)
151 open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
152 db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = 0)
153 db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = 0)
154 flags: FORCE
155 db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = 0)
156 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
157 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = 0)
158 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = 0)
159 db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = 0)
160 db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = 0)
161 db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = 0)
162 $db = db_create (DB_ENV *env = 0, U32 flags = 0)
164 flags: XA_CREATE
165 db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = 0)
167 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
168 db_close (DB *db, U32 flags = 0, SV *callback = 0)
169 flags: DB_NOSYNC
170 db_verify (DB *db, bdb_filename file, bdb_filename database = 0, SV *dummy = 0, U32 flags = 0, SV *callback = 0)
171 db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = 0)
172 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = 0)
173 flags: FREELIST_ONLY FREE_SPACE
174 db_sync (DB *db, U32 flags = 0, SV *callback = 0)
175 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = 0)
176 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
177 flags: APPEND NODUPDATA NOOVERWRITE
178 db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6)
179 db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
180 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
181 db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0)
182 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
183 db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0)
184 db_txn_commit (DB_TXN *txn, U32 flags = 0, SV *callback = 0)
185 flags: TXN_NOSYNC TXN_SYNC
186 db_txn_abort (DB_TXN *txn, SV *callback = 0)
187 db_c_close (DBC *dbc, SV *callback = 0)
189 db_c_count (DBC *dbc, SV *count, U32 flags = 0, SV *callback = 0)
190 db_c_put (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
191 flags: AFTER BEFORE CURRENT KEYFIRST KEYLAST NODUPDATA
192 db_c_get (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
193 flags: CURRENT FIRST GET_BOTH GET_BOTH_RANGE GET_RECNO JOIN_ITEM LAST NEXT NEXT_DUP NEXT_NODUP PREV PREV_DUP PREV_NODUP SET SET_RANGE SET_RECNO READ_UNCOMMITTED MULTIPLE MULTIPLE_KEY RMW
194 db_c_pget (DBC *dbc, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0)
195 db_c_del (DBC *dbc, U32 flags = 0, SV *callback = 0)
196 db_sequence_open (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, SV *key, U32 flags = 0, SV *callback = 0)
198 flags: CREATE EXCL
199 db_sequence_close (DB_SEQUENCE *seq, U32 flags = 0, SV *callback = 0)
200 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = 0)
201 flags: TXN_NOSYNC
202 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = 0)
203 flags: TXN_NOSYNC
204 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = 0)
206 This is not actually a Berkeley DB function but a BDB module extension.
208 The background for this exytension is: It is very annoying to have to
209 check every single BDB function for error returns and provide a
210 codepath out of your transaction. While the BDB module still makes this
211 possible, it contains the following extensions:
212 When a transaction-protected function returns any operating system
214 error (errno > 0), BDB will set the "TXN_DEADLOCK" flag on the
215 transaction. This flag is also set by Berkeley DB functions themselves
216 when an operation fails with LOCK_DEADLOCK, and it causes all further
217 operations on that transaction (including "db_txn_commit") to fail.
218 The "db_txn_finish" request will look at this flag, and, if it is set,
220 will automatically call "db_txn_abort" (setting errno to
221 "LOCK_DEADLOCK" if it isn't set to something else yet). If it isn't
222 set, it will call "db_txn_commit" and return the error normally.
223 How to use this? Easy: just write your transaction normally:
225 my $txn = $db_env->txn_begin;
227 db_get $db, $txn, "key", my $data;
228 db_put $db, $txn, "key", $data + 1 unless $! == BDB::NOTFOUND;
229 db_txn_finish $txn;
230 die "transaction failed" if $!;
231 That is, handle only the expected errors. If something unexpected
233 happens (EIO, LOCK_NOTGRANTED or a deadlock in either db_get or
234 db_put), then the remaining requests (db_put in this case) will simply
235 be skipped (they will fail with LOCK_DEADLOCK) and the transaction will
236 be aborted.
237 You can use the "$txn->failed" method to check wether a transaction has
239 failed in this way and abort further processing (excluding
240 "db_txn_finish").
241 DB_ENV/database environment methods
243 Methods available on DB_ENV/$env handles:
244 DESTROY (DB_ENV_ornull *env)
246 CODE:
247 if (env)
248 env->close (env, 0);
249 $int = $env->set_data_dir (const char *dir)
251 $int = $env->set_tmp_dir (const char *dir)
252 $int = $env->set_lg_dir (const char *dir)
253 $int = $env->set_shm_key (long shm_key)
254 $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
255 $int = $env->set_flags (U32 flags, int onoff = 1)
256 $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7)
257 $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7)
258 $env->set_errfile (FILE *errfile = 0)
259 $env->set_msgfile (FILE *msgfile = 0)
260 $int = $env->set_verbose (U32 which, int onoff = 1)
261 $int = $env->set_encrypt (const char *password, U32 flags = 0)
262 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
263 $int = $env->set_mp_max_openfd (int maxopenfd);
264 $int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep);
265 $int = $env->set_mp_mmapsize (int mmapsize_mb)
266 $int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT)
267 $int = $env->set_lk_max_lockers (U32 max)
268 $int = $env->set_lk_max_locks (U32 max)
269 $int = $env->set_lk_max_objects (U32 max)
270 $int = $env->set_lg_bsize (U32 max)
271 $int = $env->set_lg_max (U32 max)
272 $int = $env->mutex_set_increment (U32 increment)
273 $int = $env->mutex_set_tas_spins (U32 tas_spins)
274 $int = $env->mutex_set_max (U32 max)
275 $int = $env->mutex_set_align (U32 align)
276 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
278 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
279 $txn = $env->cdsgroup_begin; (v4.5)
280 Example:
282 use AnyEvent;
284 use BDB;
285 our $FH; open $FH, "<&=" . BDB::poll_fileno;
287 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
288 BDB::min_parallel 8;
290 my $env = db_env_create;
292 mkdir "bdtest", 0700;
294 db_env_open
295 $env,
296 "bdtest",
297 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
298 0600;
299 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
301 DB/database methods
303 Methods available on DB/$db handles:
304 DESTROY (DB_ornull *db)
306 CODE:
307 if (db)
308 {
309 SV *env = (SV *)db->app_private;
310 db->close (db, 0);
311 SvREFCNT_dec (env);
312 }
313 $int = $db->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
315 $int = $db->set_flags (U32 flags)
316 flags: CHKSUM ENCRYPT TXN_NOT_DURABLE
317 Btree: DUP DUPSORT RECNUM REVSPLITOFF
318 Hash: DUP DUPSORT
319 Queue: INORDER
320 Recno: RENUMBER SNAPSHOT
321 $int = $db->set_encrypt (const char *password, U32 flags)
323 $int = $db->set_lorder (int lorder)
324 $int = $db->set_bt_minkey (U32 minkey)
325 $int = $db->set_re_delim (int delim)
326 $int = $db->set_re_pad (int re_pad)
327 $int = $db->set_re_source (char *source)
328 $int = $db->set_re_len (U32 re_len)
329 $int = $db->set_h_ffactor (U32 h_ffactor)
330 $int = $db->set_h_nelem (U32 h_nelem)
331 $int = $db->set_q_extentsize (U32 extentsize)
332 $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0)
334 flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT
335 $seq = $db->sequence (U32 flags = 0)
336 Example:
338 my $db = db_create $env;
340 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600;
341 for (1..1000) {
343 db_put $db, undef, "key $_", "data $_";
344 db_key_range $db, undef, "key $_", my $keyrange;
346 my ($lt, $eq, $gt) = @$keyrange;
347 }
348 db_del $db, undef, "key $_" for 1..1000;
350 db_sync $db;
352 DB_TXN/transaction methods
354 Methods available on DB_TXN/$txn handles:
355 DESTROY (DB_TXN_ornull *txn)
357 CODE:
358 if (txn)
359 txn->abort (txn);
360 $int = $txn->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
362 flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
363 $bool = $txn->failed
365 # see db_txn_finish documentation, above
366 DBC/cursor methods
368 Methods available on DBC/$dbc handles:
369 DESTROY (DBC_ornull *dbc)
371 CODE:
372 if (dbc)
373 dbc->c_close (dbc);
374 $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
376 Example:
378 my $c = $db->cursor;
380 for (;;) {
382 db_c_get $c, my $key, my $data, BDB::NEXT;
383 warn "<$!,$key,$data>";
384 last if $!;
385 }
386 db_c_close $c;
388 DB_SEQUENCE/sequence methods
390 Methods available on DB_SEQUENCE/$seq handles:
391 DESTROY (DB_SEQUENCE_ornull *seq)
393 CODE:
394 if (seq)
395 seq->close (seq, 0);
396 $int = $seq->initial_value (db_seq_t value)
398 $int = $seq->set_cachesize (U32 size)
399 $int = $seq->set_flags (U32 flags)
400 flags: SEQ_DEC SEQ_INC SEQ_WRAP
401 $int = $seq->set_range (db_seq_t min, db_seq_t max)
402 Example:
404 my $seq = $db->sequence;
406 db_sequence_open $seq, undef, "seq", BDB::CREATE;
408 db_sequence_get $seq, undef, 1, my $value;
409
411 EVENT PROCESSING AND EVENT LOOP INTEGRATION
412 $msg = BDB::strerror [$errno]
413 Returns the string corresponding to the given errno value. If no
414 argument is given, use $!.
415 Note that the BDB module also patches the $! variable directly, so
417 you should be able to get a bdb error string by simply stringifying
418 $!.
419 $fileno = BDB::poll_fileno
421 Return the request result pipe file descriptor. This filehandle
422 must be polled for reading by some mechanism outside this module
423 (e.g. Event or select, see below or the SYNOPSIS). If the pipe
424 becomes readable you have to call "poll_cb" to check the results.
425 See "poll_cb" for an example.
427 BDB::poll_cb
429 Process some outstanding events on the result pipe. You have to
430 call this regularly. Returns the number of events processed.
431 Returns immediately when no events are outstanding. The amount of
432 events processed depends on the settings of "BDB::max_poll_req" and
433 "BDB::max_poll_time".
434 If not all requests were processed for whatever reason, the
436 filehandle will still be ready when "poll_cb" returns.
437 Example: Install an Event watcher that automatically calls
439 BDB::poll_cb with high priority:
440 Event->io (fd => BDB::poll_fileno,
442 poll => 'r', async => 1,
443 cb => \&BDB::poll_cb);
444 BDB::max_poll_reqs $nreqs
446 BDB::max_poll_time $seconds
447 These set the maximum number of requests (default 0, meaning
448 infinity) that are being processed by "BDB::poll_cb" in one call,
449 respectively the maximum amount of time (default 0, meaning
450 infinity) spent in "BDB::poll_cb" to process requests (more
451 correctly the mininum amount of time "poll_cb" is allowed to use).
452 Setting "max_poll_time" to a non-zero value creates an overhead of
454 one syscall per request processed, which is not normally a problem
455 unless your callbacks are really really fast or your OS is really
456 really slow (I am not mentioning Solaris here). Using
457 "max_poll_reqs" incurs no overhead.
458 Setting these is useful if you want to ensure some level of
460 interactiveness when perl is not fast enough to process all
461 requests in time.
462 For interactive programs, values such as 0.01 to 0.1 should be
464 fine.
465 Example: Install an EV watcher that automatically calls
467 BDB::poll_cb with low priority, to ensure that other parts of the
468 program get the CPU sometimes even under high load.
469 # try not to spend much more than 0.1s in poll_cb
471 BDB::max_poll_time 0.1;
472 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb);
474 BDB::poll_wait
476 If there are any outstanding requests and none of them in the
477 result phase, wait till the result filehandle becomes ready for
478 reading (simply does a "select" on the filehandle. This is useful
479 if you want to synchronously wait for some requests to finish).
480 See "nreqs" for an example.
482 BDB::poll
484 Waits until some requests have been handled.
485 Returns the number of requests processed, but is otherwise strictly
487 equivalent to:
488 BDB::poll_wait, BDB::poll_cb
490 BDB::flush
492 Wait till all outstanding BDB requests have been handled.
493 Strictly equivalent to:
495 BDB::poll_wait, BDB::poll_cb
497 while BDB::nreqs;
498 VERSION CHECKING
500 BerkeleyDB comes in various versions, many of them have minor
501 incompatibilities. This means that traditional "at least version x.x"
502 checks are often not sufficient.
503 Example: set the log_autoremove option in a way compatible with <v4.7
505 and v4.7. Note the use of & on the constants to avoid triggering a
506 compiletime bug when the symbol isn't available.
507 $DB_ENV->set_flags (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7;
509 $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7;
510 BDB::VERSION
512 The "BDB::VERSION" function, when called without arguments, returns
513 the Berkeley DB version as a v-string (usually with 3 components).
514 You should use "lt" and "ge" operators exclusively to make
515 comparisons.
516 Example: check for at least version 4.7.
518 BDB::VERSION ge v4.7 or die;
520 BDB::VERSION min-version
522 Returns true if the BDB version is at least the given version
523 (specified as a v-string), false otherwise.
524 Example: check for at least version 4.5.
526 BDB::VERSION v4.7 or die;
528 BDB::VERSION min-version, max-version
530 Returns true of the BDB version is at least version "min-version"
531 (specify "undef" or "v0" for any minimum version) and less then
532 "max-version".
533 Example: check wether version is strictly less then v4.7.
535 BDB::VERSION v0, v4.7
537 or die "version 4.7 is not yet supported";
538 CONTROLLING THE NUMBER OF THREADS
540 BDB::min_parallel $nthreads
541 Set the minimum number of BDB threads to $nthreads. The current
542 default is 8, which means eight asynchronous operations can execute
543 concurrently at any one time (the number of outstanding requests,
544 however, is unlimited).
545 BDB starts threads only on demand, when an BDB request is queued
547 and no free thread exists. Please note that queueing up a hundred
548 requests can create demand for a hundred threads, even if it turns
549 out that everything is in the cache and could have been processed
550 faster by a single thread.
551 It is recommended to keep the number of threads relatively low, as
553 some Linux kernel versions will scale negatively with the number of
554 threads (higher parallelity => MUCH higher latency). With current
555 Linux 2.6 versions, 4-32 threads should be fine.
556 Under most circumstances you don't need to call this function, as
558 the module selects a default that is suitable for low to moderate
559 load.
560 BDB::max_parallel $nthreads
562 Sets the maximum number of BDB threads to $nthreads. If more than
563 the specified number of threads are currently running, this
564 function kills them. This function blocks until the limit is
565 reached.
566 While $nthreads are zero, aio requests get queued but not executed
568 until the number of threads has been increased again.
569 This module automatically runs "max_parallel 0" at program end, to
571 ensure that all threads are killed and that there are no
572 outstanding requests.
573 Under normal circumstances you don't need to call this function.
575 BDB::max_idle $nthreads
577 Limit the number of threads (default: 4) that are allowed to idle
578 (i.e., threads that did not get a request to process within 10
579 seconds). That means if a thread becomes idle while $nthreads other
580 threads are also idle, it will free its resources and exit.
581 This is useful when you allow a large number of threads (e.g. 100
583 or 1000) to allow for extremely high load situations, but want to
584 free resources under normal circumstances (1000 threads can easily
585 consume 30MB of RAM).
586 The default is probably ok in most situations, especially if thread
588 creation is fast. If thread creation is very slow on your system
589 you might want to use larger values.
590 $oldmaxreqs = BDB::max_outstanding $maxreqs
592 This is a very bad function to use in interactive programs because
593 it blocks, and a bad way to reduce concurrency because it is
594 inexact: Better use an "aio_group" together with a feed callback.
595 Sets the maximum number of outstanding requests to $nreqs. If you
597 to queue up more than this number of requests, the next call to the
598 "poll_cb" (and "poll_some" and other functions calling "poll_cb")
599 function will block until the limit is no longer exceeded.
600 The default value is very large, so there is no practical limit on
602 the number of outstanding requests.
603 You can still queue as many requests as you want. Therefore,
605 "max_oustsanding" is mainly useful in simple scripts (with low
606 values) or as a stop gap to shield against fatal memory overflow
607 (with large values).
608 $old_cb = BDB::set_sync_prepare $cb
610 Sets a callback that is called whenever a request is created
611 without an explicit callback. It has to return two code references.
612 The first is used as the request callback (it should save the
613 return status), and the second is called to wait until the first
614 callback has been called (it must set $! to the return status).
615 This mechanism can be used to include BDB into other event
617 mechanisms, such as Coro::BDB.
618 To allow other, callback-based, events to be executed while
620 callback-less ones are run, you could use this sync prepare
621 function:
622 sub {
624 my $status;
625 (
626 sub { $status = $! },
627 sub { BDB::poll while !defined $status; $! = $status },
628 )
629 }
630 It works by polling for results till the request has finished and
632 then sets $! to the return value. This means that if you don't use
633 a callback, BDB would simply fall back to synchronous operations.
634 By default, or if the sync prepare function is set to "undef", is
636 to execute callback-less BDB requests in the foreground thread,
637 setting $! to the return value, without polling for other events.
638 STATISTICAL INFORMATION
640 BDB::nreqs
641 Returns the number of requests currently in the ready, execute or
642 pending states (i.e. for which their callback has not been invoked
643 yet).
644 Example: wait till there are no outstanding requests anymore:
646 BDB::poll_wait, BDB::poll_cb
648 while BDB::nreqs;
649 BDB::nready
651 Returns the number of requests currently in the ready state (not
652 yet executed).
653 BDB::npending
655 Returns the number of requests currently in the pending state
656 (executed, but not yet processed by poll_cb).
657
659 Unexpected Crashes
660 Remember that, by default, BDB will execute requests in parallel, in
661 somewhat random order. That means that it is easy to run a "db_get"
662 request on thesa me database as a concurrent "db_close" request,
663 leading to a crash, silent data corruption, eventually the next world
664 war on terrorism.
665 If you only ever use foreground requests (without a callback), this
667 will not be an issue.
668 Unexpected Freezes or Deadlocks
670 Remember that, by default, BDB will execute requests in parallel, which
671 easily leads to deadlocks (even concurrent put's on the same database
672 can deadlock).
673 You either need to run deadlock detection (and handle the resulting
675 errors), or make sure only one process ever updates the database, ine
676 one thread, e.g. by using only foreground requests (without a
677 callback).
678
680 This module should do "the right thing" when the process using it
681 forks:
682 Before the fork, BDB enters a quiescent state where no requests can be
684 added in other threads and no results will be processed. After the fork
685 the parent simply leaves the quiescent state and continues
686 request/result processing, while the child frees the request/result
687 queue (so that the requests started before the fork will only be
688 handled in the parent). Threads will be started on demand until the
689 limit set in the parent process has been reached again.
690 In short: the parent will, after a short pause, continue as if fork had
692 not been called, while the child will act as if BDB has not been used
693 yet.
694 Win32 note: there is no fork on win32, and perls emulation of it is too
696 broken to be supported, so do not use BDB in a windows pseudo-fork,
697 better yet, switch to a more capable platform.
698
700 Per-request usage:
701 Each aio request uses - depending on your architecture - around 100-200
703 bytes of memory. In addition, stat requests need a stat buffer
704 (possibly a few hundred bytes), readdir requires a result buffer and so
705 on. Perl scalars and other data passed into aio requests will also be
706 locked and will consume memory till the request has entered the done
707 state.
708 This is not awfully much, so queuing lots of requests is not usually a
710 problem.
711 Per-thread usage:
713 In the execution phase, some aio requests require more memory for
715 temporary buffers, and each thread requires a stack and other data
716 structures (usually around 16k-128k, depending on the OS).
717
719 Perl on Win32 supports only ASCII filenames (the reason is that it
720 abuses an internal flag to store wether a filename is Unicode or ANSI,
721 but that flag is used for somethign else in the perl core, so there is
722 no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
723 module tries to work around this issue by assuming that the filename is
724 an ANSI filename and BDB was built for unicode support.
725
727 Known bugs will be fixed in the next release, except:
728 If you use a transaction in any request, and the request returns
730 with an operating system error or DB_LOCK_NOTGRANTED, the internal
731 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
732 above.
733
735 AnyEvent::BDB (event loop integration), Coro::BDB (more natural
736 syntax), IO::AIO (nice to have).
737
739 Marc Lehmann <schmorp@schmorp.de>
740 http://home.schmorp.de/
741perl v5.12.1 2010-03-31 BDB(3)