1BDB(3) User Contributed Perl Documentation BDB(3)
2
6 BDB - Asynchronous Berkeley DB access
7
9 use BDB;
10 my $env = db_env_create;
12 mkdir "bdtest", 0700;
14 db_env_open
15 $env,
16 "bdtest",
17 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL
18 | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
19 0600;
20 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
22 my $db = db_create $env;
24 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE
25 | BDB::READ_UNCOMMITTED, 0600;
26 db_put $db, undef, "key", "data", 0, sub {
27 db_del $db, undef, "key";
28 };
29 db_sync $db;
30 # when you also use Coro, management is easy:
32 use Coro::BDB;
33 # automatic event loop integration with AnyEvent:
35 use AnyEvent::BDB;
36 # automatic result processing with EV:
38 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
39 # with Glib:
41 add_watch Glib::IO BDB::poll_fileno,
42 in => sub { BDB::poll_cb; 1 };
43 # or simply flush manually
45 BDB::flush;
46
48 See the BerkeleyDB documentation
49 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.html>).
50 The BDB API is very similar to the C API (the translation has been very
51 faithful).
52 See also the example sections in the document below and possibly the
54 eg/ subdirectory of the BDB distribution. Last not least see the
55 IO::AIO documentation, as that module uses almost the same asynchronous
56 request model as this module.
57 I know this is woefully inadequate documentation. Send a patch!
59
61 Every request method creates a request. which is a C data structure not
62 directly visible to Perl.
63 During their existance, bdb requests travel through the following
65 states, in order:
66 ready
68 Immediately after a request is created it is put into the ready
69 state, waiting for a thread to execute it.
70 execute
72 A thread has accepted the request for processing and is currently
73 executing it (e.g. blocking in read).
74 pending
76 The request has been executed and is waiting for result processing.
77 While request submission and execution is fully asynchronous,
79 result processing is not and relies on the perl interpreter calling
80 "poll_cb" (or another function with the same effect).
81 result
83 The request results are processed synchronously by "poll_cb".
84 The "poll_cb" function will process all outstanding aio requests by
86 calling their callbacks, freeing memory associated with them and
87 managing any groups they are contained in.
88 done
90 Request has reached the end of its lifetime and holds no resources
91 anymore (except possibly for the Perl object, but its connection to
92 the actual aio request is severed and calling its methods will
93 either do nothing or result in a runtime error).
94
96 All of these are functions. The create functions simply return a new
97 object and never block. All the remaining functions take an optional
98 callback as last argument. If it is missing, then the function will be
99 executed synchronously. In both cases, $! will reflect the return value
100 of the function.
101 BDB functions that cannot block (mostly functions that manipulate
103 settings) are method calls on the relevant objects, so the rule of
104 thumb is: if it's a method, it's not blocking, if it's a function, it
105 takes a callback as last argument.
106 In the following, $int signifies an integer return value,
108 "bdb_filename" is a "filename" (octets on unix, madness on windows),
109 "U32" is an unsigned 32 bit integer, "int" is some integer, "NV" is a
110 floating point value.
111 Most "SV *" types are generic perl scalars (for input and output of
113 data values).
114 The various "DB_ENV" etc. arguments are handles return by
116 "db_env_create", "db_create", "txn_begin" and so on. If they have an
117 appended "_ornull" this means they are optional and you can pass
118 "undef" for them, resulting a NULL pointer on the C level.
119 The "SV *callback" is the optional callback function to call when the
121 request is completed. This last callback argument is special: the
122 callback is simply the last argument passed. If there are "optional"
123 arguments before the callback they can be left out. The callback itself
124 can be left out or specified as "undef", in which case the function
125 will be executed synchronously.
126 For example, "db_env_txn_checkpoint" usually is called with all integer
128 arguments zero. These can be left out, so all of these specify a call
129 to "DB_ENV->txn_checkpoint", to be executed asynchronously with a
130 callback to be called:
131 db_env_txn_checkpoint $db_env, 0, 0, 0, sub { };
133 db_env_txn_checkpoint $db_env, 0, 0, sub { };
134 db_env_txn_checkpoint $db_env, sub { };
135 While these all specify a call to "DB_ENV->txn_checkpoint" to be
137 executed synchronously:
138 db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
140 db_env_txn_checkpoint $db_env, 0, 0, 0;
141 db_env_txn_checkpoint $db_env, 0;
142 BDB functions
144 Functions in the BDB namespace, exported by default:
145 $env = db_env_create (U32 env_flags = 0)
147 flags: RPCCLIENT
148 db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = 0)
150 open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
151 db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = 0)
152 db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = 0)
153 flags: FORCE
154 db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = 0)
155 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
156 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = 0)
157 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = 0)
158 db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = 0)
159 db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = 0)
160 db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = 0)
161 db_env_lsn_reset (DB_ENV *env, bdb_filename db, U32 flags = 0, SV *callback = 0)
162 db_env_fileid_reset (DB_ENV *env, bdb_filename db, U32 flags = 0, SV *callback = 0)
163 $db = db_create (DB_ENV *env = 0, U32 flags = 0)
165 flags: XA_CREATE
166 db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = 0)
168 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
169 db_close (DB *db, U32 flags = 0, SV *callback = 0)
170 flags: DB_NOSYNC
171 db_verify (DB *db, bdb_filename file, bdb_filename database = 0, SV *dummy = 0, U32 flags = 0, SV *callback = 0)
172 db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = 0)
173 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = 0)
174 flags: FREELIST_ONLY FREE_SPACE
175 db_sync (DB *db, U32 flags = 0, SV *callback = 0)
176 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = 0)
177 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
178 flags: APPEND NODUPDATA NOOVERWRITE
179 db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6)
180 db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
181 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
182 db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0)
183 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
184 db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0)
185 db_txn_commit (DB_TXN *txn, U32 flags = 0, SV *callback = 0)
186 flags: TXN_NOSYNC TXN_SYNC
187 db_txn_abort (DB_TXN *txn, SV *callback = 0)
188 db_c_close (DBC *dbc, SV *callback = 0)
190 db_c_count (DBC *dbc, SV *count, U32 flags = 0, SV *callback = 0)
191 db_c_put (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
192 flags: AFTER BEFORE CURRENT KEYFIRST KEYLAST NODUPDATA
193 db_c_get (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
194 flags: CURRENT FIRST GET_BOTH GET_BOTH_RANGE GET_RECNO JOIN_ITEM LAST NEXT NEXT_DUP NEXT_NODUP PREV PREV_DUP PREV_NODUP SET SET_RANGE SET_RECNO READ_UNCOMMITTED MULTIPLE MULTIPLE_KEY RMW
195 db_c_pget (DBC *dbc, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0)
196 db_c_del (DBC *dbc, U32 flags = 0, SV *callback = 0)
197 db_sequence_open (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, SV *key, U32 flags = 0, SV *callback = 0)
199 flags: CREATE EXCL
200 db_sequence_close (DB_SEQUENCE *seq, U32 flags = 0, SV *callback = 0)
201 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = 0)
202 flags: TXN_NOSYNC
203 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = 0)
204 flags: TXN_NOSYNC
205 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = 0)
207 This is not actually a Berkeley DB function but a BDB module extension.
209 The background for this exytension is: It is very annoying to have to
210 check every single BDB function for error returns and provide a
211 codepath out of your transaction. While the BDB module still makes this
212 possible, it contains the following extensions:
213 When a transaction-protected function returns any operating system
215 error (errno > 0), BDB will set the "TXN_DEADLOCK" flag on the
216 transaction. This flag is also set by Berkeley DB functions themselves
217 when an operation fails with LOCK_DEADLOCK, and it causes all further
218 operations on that transaction (including "db_txn_commit") to fail.
219 The "db_txn_finish" request will look at this flag, and, if it is set,
221 will automatically call "db_txn_abort" (setting errno to
222 "LOCK_DEADLOCK" if it isn't set to something else yet). If it isn't
223 set, it will call "db_txn_commit" and return the error normally.
224 How to use this? Easy: just write your transaction normally:
226 my $txn = $db_env->txn_begin;
228 db_get $db, $txn, "key", my $data;
229 db_put $db, $txn, "key", $data + 1 unless $! == BDB::NOTFOUND;
230 db_txn_finish $txn;
231 die "transaction failed" if $!;
232 That is, handle only the expected errors. If something unexpected
234 happens (EIO, LOCK_NOTGRANTED or a deadlock in either db_get or
235 db_put), then the remaining requests (db_put in this case) will simply
236 be skipped (they will fail with LOCK_DEADLOCK) and the transaction will
237 be aborted.
238 You can use the "$txn->failed" method to check wether a transaction has
240 failed in this way and abort further processing (excluding
241 "db_txn_finish").
242 DB_ENV/database environment methods
244 Methods available on DB_ENV/$env handles:
245 DESTROY (DB_ENV_ornull *env)
247 CODE:
248 if (env)
249 env->close (env, 0);
250 $int = $env->set_data_dir (const char *dir)
252 $int = $env->set_tmp_dir (const char *dir)
253 $int = $env->set_lg_dir (const char *dir)
254 $int = $env->set_shm_key (long shm_key)
255 $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
256 $int = $env->set_flags (U32 flags, int onoff = 1)
257 $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7)
258 $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7)
259 $env->set_errfile (FILE *errfile = 0)
260 $env->set_msgfile (FILE *msgfile = 0)
261 $int = $env->set_verbose (U32 which, int onoff = 1)
262 $int = $env->set_encrypt (const char *password, U32 flags = 0)
263 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
264 $int = $env->set_mp_max_openfd (int maxopenfd);
265 $int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep);
266 $int = $env->set_mp_mmapsize (int mmapsize_mb)
267 $int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT)
268 $int = $env->set_lk_max_lockers (U32 max)
269 $int = $env->set_lk_max_locks (U32 max)
270 $int = $env->set_lk_max_objects (U32 max)
271 $int = $env->set_lg_bsize (U32 max)
272 $int = $env->set_lg_max (U32 max)
273 $int = $env->mutex_set_increment (U32 increment)
274 $int = $env->mutex_set_tas_spins (U32 tas_spins)
275 $int = $env->mutex_set_max (U32 max)
276 $int = $env->mutex_set_align (U32 align)
277 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
279 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
280 $txn = $env->cdsgroup_begin; (v4.5)
281 Example:
283 use AnyEvent;
285 use BDB;
286 our $FH; open $FH, "<&=" . BDB::poll_fileno;
288 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
289 BDB::min_parallel 8;
291 my $env = db_env_create;
293 mkdir "bdtest", 0700;
295 db_env_open
296 $env,
297 "bdtest",
298 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
299 0600;
300 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
302 DB/database methods
304 Methods available on DB/$db handles:
305 DESTROY (DB_ornull *db)
307 CODE:
308 if (db)
309 {
310 SV *env = (SV *)db->app_private;
311 db->close (db, 0);
312 SvREFCNT_dec (env);
313 }
314 $int = $db->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
316 $int = $db->set_flags (U32 flags)
317 flags: CHKSUM ENCRYPT TXN_NOT_DURABLE
318 Btree: DUP DUPSORT RECNUM REVSPLITOFF
319 Hash: DUP DUPSORT
320 Queue: INORDER
321 Recno: RENUMBER SNAPSHOT
322 $int = $db->set_encrypt (const char *password, U32 flags)
324 $int = $db->set_lorder (int lorder)
325 $int = $db->set_bt_minkey (U32 minkey)
326 $int = $db->set_re_delim (int delim)
327 $int = $db->set_re_pad (int re_pad)
328 $int = $db->set_re_source (char *source)
329 $int = $db->set_re_len (U32 re_len)
330 $int = $db->set_h_ffactor (U32 h_ffactor)
331 $int = $db->set_h_nelem (U32 h_nelem)
332 $int = $db->set_q_extentsize (U32 extentsize)
333 $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0)
335 flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT
336 $seq = $db->sequence (U32 flags = 0)
337 Example:
339 my $db = db_create $env;
341 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600;
342 for (1..1000) {
344 db_put $db, undef, "key $_", "data $_";
345 db_key_range $db, undef, "key $_", my $keyrange;
347 my ($lt, $eq, $gt) = @$keyrange;
348 }
349 db_del $db, undef, "key $_" for 1..1000;
351 db_sync $db;
353 DB_TXN/transaction methods
355 Methods available on DB_TXN/$txn handles:
356 DESTROY (DB_TXN_ornull *txn)
358 CODE:
359 if (txn)
360 txn->abort (txn);
361 $int = $txn->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
363 flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
364 $bool = $txn->failed
366 # see db_txn_finish documentation, above
367 DBC/cursor methods
369 Methods available on DBC/$dbc handles:
370 DESTROY (DBC_ornull *dbc)
372 CODE:
373 if (dbc)
374 dbc->c_close (dbc);
375 $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
377 Example:
379 my $c = $db->cursor;
381 for (;;) {
383 db_c_get $c, my $key, my $data, BDB::NEXT;
384 warn "<$!,$key,$data>";
385 last if $!;
386 }
387 db_c_close $c;
389 DB_SEQUENCE/sequence methods
391 Methods available on DB_SEQUENCE/$seq handles:
392 DESTROY (DB_SEQUENCE_ornull *seq)
394 CODE:
395 if (seq)
396 seq->close (seq, 0);
397 $int = $seq->initial_value (db_seq_t value)
399 $int = $seq->set_cachesize (U32 size)
400 $int = $seq->set_flags (U32 flags)
401 flags: SEQ_DEC SEQ_INC SEQ_WRAP
402 $int = $seq->set_range (db_seq_t min, db_seq_t max)
403 Example:
405 my $seq = $db->sequence;
407 db_sequence_open $seq, undef, "seq", BDB::CREATE;
409 db_sequence_get $seq, undef, 1, my $value;
410
412 EVENT PROCESSING AND EVENT LOOP INTEGRATION
413 $msg = BDB::strerror [$errno]
414 Returns the string corresponding to the given errno value. If no
415 argument is given, use $!.
416 Note that the BDB module also patches the $! variable directly, so
418 you should be able to get a bdb error string by simply stringifying
419 $!.
420 $fileno = BDB::poll_fileno
422 Return the request result pipe file descriptor. This filehandle
423 must be polled for reading by some mechanism outside this module
424 (e.g. Event or select, see below or the SYNOPSIS). If the pipe
425 becomes readable you have to call "poll_cb" to check the results.
426 See "poll_cb" for an example.
428 BDB::poll_cb
430 Process some outstanding events on the result pipe. You have to
431 call this regularly. Returns the number of events processed.
432 Returns immediately when no events are outstanding. The amount of
433 events processed depends on the settings of "BDB::max_poll_req" and
434 "BDB::max_poll_time".
435 If not all requests were processed for whatever reason, the
437 filehandle will still be ready when "poll_cb" returns.
438 Example: Install an Event watcher that automatically calls
440 BDB::poll_cb with high priority:
441 Event->io (fd => BDB::poll_fileno,
443 poll => 'r', async => 1,
444 cb => \&BDB::poll_cb);
445 BDB::max_poll_reqs $nreqs
447 BDB::max_poll_time $seconds
448 These set the maximum number of requests (default 0, meaning
449 infinity) that are being processed by "BDB::poll_cb" in one call,
450 respectively the maximum amount of time (default 0, meaning
451 infinity) spent in "BDB::poll_cb" to process requests (more
452 correctly the mininum amount of time "poll_cb" is allowed to use).
453 Setting "max_poll_time" to a non-zero value creates an overhead of
455 one syscall per request processed, which is not normally a problem
456 unless your callbacks are really really fast or your OS is really
457 really slow (I am not mentioning Solaris here). Using
458 "max_poll_reqs" incurs no overhead.
459 Setting these is useful if you want to ensure some level of
461 interactiveness when perl is not fast enough to process all
462 requests in time.
463 For interactive programs, values such as 0.01 to 0.1 should be
465 fine.
466 Example: Install an EV watcher that automatically calls
468 BDB::poll_cb with low priority, to ensure that other parts of the
469 program get the CPU sometimes even under high load.
470 # try not to spend much more than 0.1s in poll_cb
472 BDB::max_poll_time 0.1;
473 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb);
475 BDB::poll_wait
477 If there are any outstanding requests and none of them in the
478 result phase, wait till the result filehandle becomes ready for
479 reading (simply does a "select" on the filehandle. This is useful
480 if you want to synchronously wait for some requests to finish).
481 See "nreqs" for an example.
483 BDB::poll
485 Waits until some requests have been handled.
486 Returns the number of requests processed, but is otherwise strictly
488 equivalent to:
489 BDB::poll_wait, BDB::poll_cb
491 BDB::flush
493 Wait till all outstanding BDB requests have been handled.
494 Strictly equivalent to:
496 BDB::poll_wait, BDB::poll_cb
498 while BDB::nreqs;
499 VERSION CHECKING
501 BerkeleyDB comes in various versions, many of them have minor
502 incompatibilities. This means that traditional "at least version x.x"
503 checks are often not sufficient.
504 Example: set the log_autoremove option in a way compatible with <v4.7
506 and v4.7. Note the use of & on the constants to avoid triggering a
507 compiletime bug when the symbol isn't available.
508 $DB_ENV->set_flags (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7;
510 $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7;
511 BDB::VERSION
513 The "BDB::VERSION" function, when called without arguments, returns
514 the Berkeley DB version as a v-string (usually with 3 components).
515 You should use "lt" and "ge" operators exclusively to make
516 comparisons.
517 Example: check for at least version 4.7.
519 BDB::VERSION ge v4.7 or die;
521 BDB::VERSION min-version
523 Returns true if the BDB version is at least the given version
524 (specified as a v-string), false otherwise.
525 Example: check for at least version 4.5.
527 BDB::VERSION v4.7 or die;
529 BDB::VERSION min-version, max-version
531 Returns true of the BDB version is at least version "min-version"
532 (specify "undef" or "v0" for any minimum version) and less then
533 "max-version".
534 Example: check wether version is strictly less then v4.7.
536 BDB::VERSION v0, v4.7
538 or die "version 4.7 is not yet supported";
539 CONTROLLING THE NUMBER OF THREADS
541 BDB::min_parallel $nthreads
542 Set the minimum number of BDB threads to $nthreads. The current
543 default is 8, which means eight asynchronous operations can execute
544 concurrently at any one time (the number of outstanding requests,
545 however, is unlimited).
546 BDB starts threads only on demand, when an BDB request is queued
548 and no free thread exists. Please note that queueing up a hundred
549 requests can create demand for a hundred threads, even if it turns
550 out that everything is in the cache and could have been processed
551 faster by a single thread.
552 It is recommended to keep the number of threads relatively low, as
554 some Linux kernel versions will scale negatively with the number of
555 threads (higher parallelity => MUCH higher latency). With current
556 Linux 2.6 versions, 4-32 threads should be fine.
557 Under most circumstances you don't need to call this function, as
559 the module selects a default that is suitable for low to moderate
560 load.
561 BDB::max_parallel $nthreads
563 Sets the maximum number of BDB threads to $nthreads. If more than
564 the specified number of threads are currently running, this
565 function kills them. This function blocks until the limit is
566 reached.
567 While $nthreads are zero, aio requests get queued but not executed
569 until the number of threads has been increased again.
570 This module automatically runs "max_parallel 0" at program end, to
572 ensure that all threads are killed and that there are no
573 outstanding requests.
574 Under normal circumstances you don't need to call this function.
576 BDB::max_idle $nthreads
578 Limit the number of threads (default: 4) that are allowed to idle
579 (i.e., threads that did not get a request to process within 10
580 seconds). That means if a thread becomes idle while $nthreads other
581 threads are also idle, it will free its resources and exit.
582 This is useful when you allow a large number of threads (e.g. 100
584 or 1000) to allow for extremely high load situations, but want to
585 free resources under normal circumstances (1000 threads can easily
586 consume 30MB of RAM).
587 The default is probably ok in most situations, especially if thread
589 creation is fast. If thread creation is very slow on your system
590 you might want to use larger values.
591 $oldmaxreqs = BDB::max_outstanding $maxreqs
593 This is a very bad function to use in interactive programs because
594 it blocks, and a bad way to reduce concurrency because it is
595 inexact: Better use an "aio_group" together with a feed callback.
596 Sets the maximum number of outstanding requests to $nreqs. If you
598 to queue up more than this number of requests, the next call to the
599 "poll_cb" (and "poll_some" and other functions calling "poll_cb")
600 function will block until the limit is no longer exceeded.
601 The default value is very large, so there is no practical limit on
603 the number of outstanding requests.
604 You can still queue as many requests as you want. Therefore,
606 "max_oustsanding" is mainly useful in simple scripts (with low
607 values) or as a stop gap to shield against fatal memory overflow
608 (with large values).
609 $old_cb = BDB::set_sync_prepare $cb
611 Sets a callback that is called whenever a request is created
612 without an explicit callback. It has to return two code references.
613 The first is used as the request callback (it should save the
614 return status), and the second is called to wait until the first
615 callback has been called (it must set $! to the return status).
616 This mechanism can be used to include BDB into other event
618 mechanisms, such as Coro::BDB.
619 To allow other, callback-based, events to be executed while
621 callback-less ones are run, you could use this sync prepare
622 function:
623 sub {
625 my $status;
626 (
627 sub { $status = $! },
628 sub { BDB::poll while !defined $status; $! = $status },
629 )
630 }
631 It works by polling for results till the request has finished and
633 then sets $! to the return value. This means that if you don't use
634 a callback, BDB would simply fall back to synchronous operations.
635 By default, or if the sync prepare function is set to "undef", is
637 to execute callback-less BDB requests in the foreground thread,
638 setting $! to the return value, without polling for other events.
639 STATISTICAL INFORMATION
641 BDB::nreqs
642 Returns the number of requests currently in the ready, execute or
643 pending states (i.e. for which their callback has not been invoked
644 yet).
645 Example: wait till there are no outstanding requests anymore:
647 BDB::poll_wait, BDB::poll_cb
649 while BDB::nreqs;
650 BDB::nready
652 Returns the number of requests currently in the ready state (not
653 yet executed).
654 BDB::npending
656 Returns the number of requests currently in the pending state
657 (executed, but not yet processed by poll_cb).
658
660 Unexpected Crashes
661 Remember that, by default, BDB will execute requests in parallel, in
662 somewhat random order. That means that it is easy to run a "db_get"
663 request on the same database as a concurrent "db_close" request,
664 leading to a crash, silent data corruption, eventually the next world
665 war on terrorism.
666 If you only ever use foreground requests (without a callback), this
668 will not be an issue (unless you use threads).
669 Unexpected Freezes or Deadlocks
671 Remember that, by default, BDB will execute requests in parallel, which
672 easily leads to deadlocks (even concurrent put's on the same database
673 can deadlock).
674 You either need to run deadlock detection (and handle the resulting
676 errors), or make sure only one process ever updates the database, ine
677 one thread, e.g. by using only foreground requests (without a
678 callback).
679
681 This module should do "the right thing" when the process using it
682 forks:
683 Before the fork, BDB enters a quiescent state where no requests can be
685 added in other threads and no results will be processed. After the fork
686 the parent simply leaves the quiescent state and continues
687 request/result processing, while the child frees the request/result
688 queue (so that the requests started before the fork will only be
689 handled in the parent). Threads will be started on demand until the
690 limit set in the parent process has been reached again.
691 In short: the parent will, after a short pause, continue as if fork had
693 not been called, while the child will act as if BDB has not been used
694 yet.
695 Win32 note: there is no fork on win32, and perls emulation of it is too
697 broken to be supported, so do not use BDB in a windows pseudo-fork,
698 better yet, switch to a more capable platform.
699
701 Per-request usage:
702 Each aio request uses - depending on your architecture - around 100-200
704 bytes of memory. In addition, stat requests need a stat buffer
705 (possibly a few hundred bytes), readdir requires a result buffer and so
706 on. Perl scalars and other data passed into aio requests will also be
707 locked and will consume memory till the request has entered the done
708 state.
709 This is not awfully much, so queuing lots of requests is not usually a
711 problem.
712 Per-thread usage:
714 In the execution phase, some aio requests require more memory for
716 temporary buffers, and each thread requires a stack and other data
717 structures (usually around 16k-128k, depending on the OS).
718
720 Perl on Win32 supports only ASCII filenames (the reason is that it
721 abuses an internal flag to store wether a filename is Unicode or ANSI,
722 but that flag is used for somethign else in the perl core, so there is
723 no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
724 module tries to work around this issue by assuming that the filename is
725 an ANSI filename and BDB was built for unicode support.
726
728 Known bugs will be fixed in the next release, except:
729 If you use a transaction in any request, and the request returns
731 with an operating system error or DB_LOCK_NOTGRANTED, the internal
732 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
733 above.
734
736 AnyEvent::BDB (event loop integration), Coro::BDB (more natural
737 syntax), IO::AIO (nice to have).
738
740 Marc Lehmann <schmorp@schmorp.de>
741 http://home.schmorp.de/
742perl v5.32.0 2020-07-28 BDB(3)