1() PMDK Programmer's Manual ()
2
6 pmemobj_tx_stage(),
7 pmemobj_tx_begin(), pmemobj_tx_lock(), pmemobj_tx_xlock(), pmemo‐
9 bj_tx_abort(), pmemobj_tx_commit(), pmemobj_tx_end(), pmemobj_tx_er‐
10 rno(), pmemobj_tx_process(),
11 TX_BEGIN_PARAM(), TX_BEGIN_CB(), TX_BEGIN(), TX_ONABORT, TX_ONCOMMIT,
13 TX_FINALLY, TX_END,
14 pmemobj_tx_log_append_buffer(), pmemobj_tx_xlog_append_buffer(), pmemo‐
16 bj_tx_log_auto_alloc(), pmemobj_tx_log_snapshots_max_size(), pmemo‐
17 bj_tx_log_intents_max_size(),
18 pmemobj_tx_set_user_data(), pmemobj_tx_get_user_data(),
20 pmemobj_tx_set_failure_behavior(), pmemobj_tx_get_failure_behavior() -
22 transactional object manipulation
23
25 #include <libpmemobj.h>
26 enum pobj_tx_stage pmemobj_tx_stage(void);
28 int pmemobj_tx_begin(PMEMobjpool *pop, jmp_buf *env, enum pobj_tx_param, ...);
30 int pmemobj_tx_lock(enum tx_lock lock_type, void *lockp);
31 int pmemobj_tx_xlock(enum tx_lock lock_type, void *lockp, uint64_t flags);
32 void pmemobj_tx_abort(int errnum);
33 void pmemobj_tx_commit(void);
34 int pmemobj_tx_end(void);
35 int pmemobj_tx_errno(void);
36 void pmemobj_tx_process(void);
37 TX_BEGIN_PARAM(PMEMobjpool *pop, ...)
39 TX_BEGIN_CB(PMEMobjpool *pop, cb, arg, ...)
40 TX_BEGIN(PMEMobjpool *pop)
41 TX_ONABORT
42 TX_ONCOMMIT
43 TX_FINALLY
44 TX_END
45 int pmemobj_tx_log_append_buffer(enum pobj_log_type type, void *addr, size_t size);
47 int pmemobj_tx_xlog_append_buffer(enum pobj_log_type type, void *addr, size_t size, uint64_t flags);
48 int pmemobj_tx_log_auto_alloc(enum pobj_log_type type, int on_off);
49 size_t pmemobj_tx_log_snapshots_max_size(size_t *sizes, size_t nsizes);
50 size_t pmemobj_tx_log_intents_max_size(size_t nintents);
51 void pmemobj_tx_set_user_data(void *data);
53 void *pmemobj_tx_get_user_data(void);
54 void pmemobj_tx_set_failure_behavior(enum pobj_tx_failure_behavior behavior);
56 enum pobj_tx_failure_behavior pmemobj_tx_get_failure_behavior(void);
57
59 The non-transactional functions and macros described in pmemobj_al‐
60 loc(3), pmemobj_list_insert(3) and POBJ_LIST_HEAD(3) only guarantee the
61 atomicity of a single operation on an object. In case of more complex
62 changes involving multiple operations on an object, or allocation and
63 modification of multiple objects, data consistency and fail-safety may
64 be provided only by using atomic transactions.
65 A transaction is defined as series of operations on persistent memory
67 objects that either all occur, or nothing occurs. In particular, if
68 the execution of a transaction is interrupted by a power failure or a
69 system crash, it is guaranteed that after system restart, all the
70 changes made as a part of the uncompleted transaction will be rolled
71 back, restoring the consistent state of the memory pool from the moment
72 when the transaction was started.
73 Note that transactions do not provide atomicity with respect to other
75 threads. All the modifications performed within the transactions are
76 immediately visible to other threads. Therefore it is the responsibil‐
77 ity of the application to implement a proper thread synchronization
78 mechanism.
79 Each thread may have only one transaction open at a time, but that
81 transaction may be nested. Nested transactions are flattened. Commit‐
82 ting the nested transaction does not commit the outer transaction; how‐
83 ever, errors in the nested transaction are propagated up to the outer‐
84 most level, resulting in the interruption of the entire transaction.
85 Each transaction is visible only for the thread that started it. No
87 other threads can add operations, commit or abort the transaction ini‐
88 tiated by another thread. Multiple threads may have transactions open
89 on a given memory pool at the same time.
90 Please see the CAVEATS section below for known limitations of the
92 transactional API.
93 The pmemobj_tx_stage() function returns the current transaction stage
95 for a thread. Stages are changed only by the pmemobj_tx_*() functions.
96 Transaction stages are defined as follows:
97 • TX_STAGE_NONE - no open transaction in this thread
99 • TX_STAGE_WORK - transaction in progress
101 • TX_STAGE_ONCOMMIT - successfully committed
103 • TX_STAGE_ONABORT - starting the transaction failed or transaction
105 aborted
106 • TX_STAGE_FINALLY - ready for clean up
108 The pmemobj_tx_begin() function starts a new transaction in the current
110 thread. If called within an open transaction, it starts a nested
111 transaction. The caller may use the env argument to provide a pointer
112 to a calling environment to be restored in case of transaction abort.
113 This information must be provided by the caller using the setjmp(3)
114 macro.
115 A new transaction may be started only if the current stage is
117 TX_STAGE_NONE or TX_STAGE_WORK. If successful, the transaction stage
118 changes to TX_STAGE_WORK. Otherwise, the stage is changed to
119 TX_STAGE_ONABORT.
120 Optionally, a list of parameters for the transaction may be provided.
122 Each parameter consists of a type followed by a type-specific number of
123 values. Currently there are 4 types:
124 • TX_PARAM_NONE, used as a termination marker. No following value.
126 • TX_PARAM_MUTEX, followed by one value, a pmem-resident PMEMmutex
128 • TX_PARAM_RWLOCK, followed by one value, a pmem-resident PMEMrwlock
130 • TX_PARAM_CB, followed by two values: a callback function of type
132 pmemobj_tx_callback, and a void pointer
133 Using TX_PARAM_MUTEX or TX_PARAM_RWLOCK causes the specified lock to be
135 acquired at the beginning of the transaction. TX_PARAM_RWLOCK acquires
136 the lock for writing. It is guaranteed that pmemobj_tx_begin() will
137 acquire all locks prior to successful completion, and they will be held
138 by the current thread until the outermost transaction is finished.
139 Locks are taken in order from left to right. To avoid deadlocks, the
140 user is responsible for proper lock ordering.
141 TX_PARAM_CB registers the specified callback function to be executed at
143 each transaction stage. For TX_STAGE_WORK, the callback is executed
144 prior to commit. For all other stages, the callback is executed as the
145 first operation after a stage change. It will also be called after
146 each transaction; in this case the stage parameter will be set to
147 TX_STAGE_NONE. pmemobj_tx_callback must be compatible with:
148 void func(PMEMobjpool *pop, enum pobj_tx_stage stage, void *arg)
150 pop is a pool identifier used in pmemobj_tx_begin(), stage is a current
152 transaction stage and arg is the second parameter of TX_PARAM_CB.
153 Without considering transaction nesting, this mechanism can be consid‐
154 ered an alternative method for executing code between stages (instead
155 of TX_ONCOMMIT, TX_ONABORT, etc). However, there are 2 significant
156 differences when nested transactions are used:
157 • The registered function is executed only in the outermost transac‐
159 tion, even if registered in an inner transaction.
160 • There can be only one callback in the entire transaction, that is,
162 the callback cannot be changed in an inner transaction.
163 Note that TX_PARAM_CB does not replace the TX_ONCOMMIT, TX_ONABORT,
165 etc. macros. They can be used together: the callback will be executed
166 before a TX_ONCOMMIT, TX_ONABORT, etc. section.
167 TX_PARAM_CB can be used when the code dealing with transaction stage
169 changes is shared between multiple users or when it must be executed
170 only in the outer transaction. For example it can be very useful when
171 the application must synchronize persistent and transient state.
172 The pmemobj_tx_lock() function acquires the lock lockp of type
174 lock_type and adds it to the current transaction. lock_type may be
175 TX_LOCK_MUTEX or TX_LOCK_RWLOCK; lockp must be of type PMEMmutex or
176 PMEMrwlock, respectively. If lock_type is TX_LOCK_RWLOCK the lock is
177 acquired for writing. If the lock is not successfully acquired, the
178 function returns an error number. This function must be called during
179 TX_STAGE_WORK.
180 The pmemobj_tx_xlock() function behaves exactly the same as pmemo‐
182 bj_tx_lock() when flags equals POBJ_XLOCK_NO_ABORT. When flags equals
183 0 and if the lock is not successfully acquired,the transaction is
184 aborted. flags is a bitmask of the following values:
185 • POBJ_XLOCK_NO_ABORT - if the function does not end successfully, do
187 not abort the transaction.
188 pmemobj_tx_abort() aborts the current transaction and causes a transi‐
190 tion to TX_STAGE_ONABORT. If errnum is equal to 0, the transaction er‐
191 ror code is set to ECANCELED; otherwise, it is set to errnum. This
192 function must be called during TX_STAGE_WORK.
193 The pmemobj_tx_commit() function commits the current open transaction
195 and causes a transition to TX_STAGE_ONCOMMIT. If called in the context
196 of the outermost transaction, all the changes may be considered as
197 durably written upon successful completion. This function must be
198 called during TX_STAGE_WORK.
199 The pmemobj_tx_end() function performs a cleanup of the current trans‐
201 action. If called in the context of the outermost transaction, it re‐
202 leases all the locks acquired by pmemobj_tx_begin() for outer and nest‐
203 ed transactions. If called in the context of a nested transaction, it
204 returns to the context of the outer transaction in TX_STAGE_WORK, with‐
205 out releasing any locks. The pmemobj_tx_end() function can be called
206 during TX_STAGE_NONE if transitioned to this stage using pmemo‐
207 bj_tx_process(). If not already in TX_STAGE_NONE, it causes the tran‐
208 sition to TX_STAGE_NONE. pmemobj_tx_end must always be called for each
209 pmemobj_tx_begin(), even if starting the transaction failed. This
210 function must not be called during TX_STAGE_WORK.
211 The pmemobj_tx_errno() function returns the error code of the last
213 transaction.
214 The pmemobj_tx_process() function performs the actions associated with
216 the current stage of the transaction, and makes the transition to the
217 next stage. It must be called in a transaction. The current stage
218 must always be obtained by a call to pmemobj_tx_stage(). pmemo‐
219 bj_tx_process() performs the following transitions in the transaction
220 stage flow:
221 • TX_STAGE_WORK -> TX_STAGE_ONCOMMIT
223 • TX_STAGE_ONABORT -> TX_STAGE_FINALLY
225 • TX_STAGE_ONCOMMIT -> TX_STAGE_FINALLY
227 • TX_STAGE_FINALLY -> TX_STAGE_NONE
229 • TX_STAGE_NONE -> TX_STAGE_NONE
231 pmemobj_tx_process() must not be called after calling pmemobj_tx_end()
233 for the outermost transaction.
234 In addition to the above API, libpmemobj(7) offers a more intuitive
236 method of building transactions using the set of macros described be‐
237 low. When using these macros, the complete transaction flow looks like
238 this:
239 TX_BEGIN(Pop) {
241 /* the actual transaction code goes here... */
242 } TX_ONCOMMIT {
243 /*
244 * optional - executed only if the above block
245 * successfully completes
246 */
247 } TX_ONABORT {
248 /*
249 * optional - executed only if starting the transaction fails,
250 * or if transaction is aborted by an error or a call to
251 * pmemobj_tx_abort()
252 */
253 } TX_FINALLY {
254 /*
255 * optional - if exists, it is executed after
256 * TX_ONCOMMIT or TX_ONABORT block
257 */
258 } TX_END /* mandatory */
259 TX_BEGIN_PARAM(PMEMobjpool *pop, ...)
261 TX_BEGIN_CB(PMEMobjpool *pop, cb, arg, ...)
262 TX_BEGIN(PMEMobjpool *pop)
263 The TX_BEGIN_PARAM(), TX_BEGIN_CB() and TX_BEGIN() macros start a new
265 transaction in the same way as pmemobj_tx_begin(), except that instead
266 of the environment buffer provided by a caller, they set up the local
267 jmp_buf buffer and use it to catch the transaction abort. The TX_BE‐
268 GIN() macro starts a transaction without any options. TX_BEGIN_PARAM
269 may be used when there is a need to acquire locks prior to starting a
270 transaction (such as for a multi-threaded program) or set up a transac‐
271 tion stage callback. TX_BEGIN_CB is just a wrapper around TX_BE‐
272 GIN_PARAM that validates the callback signature. (For compatibility
273 there is also a TX_BEGIN_LOCK macro, which is an alias for TX_BE‐
274 GIN_PARAM). Each of these macros must be followed by a block of code
275 with all the operations that are to be performed atomically.
276 The TX_ONABORT macro starts a block of code that will be executed only
278 if starting the transaction fails due to an error in pmemobj_tx_be‐
279 gin(), or if the transaction is aborted. This block is optional, but
280 in practice it should not be omitted. If it is desirable to crash the
281 application when a transaction aborts and there is no TX_ONABORT sec‐
282 tion, the application can define the POBJ_TX_CRASH_ON_NO_ONABORT macro
283 before inclusion of <libpmemobj.h>. This provides a default TX_ONABORT
284 section which just calls abort(3).
285 The TX_ONCOMMIT macro starts a block of code that will be executed only
287 if the transaction is successfully committed, which means that the exe‐
288 cution of code in the TX_BEGIN() block has not been interrupted by an
289 error or by a call to pmemobj_tx_abort(). This block is optional.
290 The TX_FINALLY macro starts a block of code that will be executed re‐
292 gardless of whether the transaction is committed or aborted. This
293 block is optional.
294 The TX_END macro cleans up and closes the transaction started by the
296 TX_BEGIN() / TX_BEGIN_PARAM() / TX_BEGIN_CB() macros. It is mandatory
297 to terminate each transaction with this macro. If the transaction was
298 aborted, errno is set appropriately.
299 TRANSACTION LOG TUNING
301 From libpmemobj implementation perspective there are two types of oper‐
302 ations in a transaction:
303 • snapshots, where action must be persisted immediately,
305 • intents, where action can be persisted at the transaction commit
307 phase
308 pmemobj_tx_add_range(3) and all its variants belong to the snapshots
310 group.
311 pmemobj_tx_alloc(3) (with its variants), pmemobj_tx_free(3), pmemo‐
313 bj_tx_realloc(3) (with its variants) and pmemobj_tx_publish(3) belong
314 to the intents group. Even though pmemobj_tx_alloc() allocates memory
315 immediately, it modifies only the runtime state and postpones persis‐
316 tent memory modifications to the commit phase. pmemobj_tx_free(3) can‐
317 not free the object immediately, because of possible transaction roll‐
318 back, so it postpones both the action and persistent memory modifica‐
319 tions to the commit phase. pmemobj_tx_realloc(3) is just a combination
320 of those two. pmemobj_tx_publish(3) postpones reservations and de‐
321 ferred frees to the commit phase.
322 Those two types of operations (snapshots and intents) require that
324 libpmemobj builds a persistent log of operations. Intent log (also
325 known as a “redo log”) is applied on commit and snapshot log (also
326 known as an “undo log”) is applied on abort.
327 When libpmemobj transaction starts, it’s not possible to predict how
329 much persistent memory space will be needed for those logs. This means
330 that libpmemobj must internally allocate this space whenever it’s need‐
331 ed. This has two downsides:
332 • when transaction snapshots a lot of memory or does a lot of alloca‐
334 tions, libpmemobj may need to do many internal allocations, which
335 must be freed when transaction ends, adding time overhead when big
336 transactions are frequent,
337 • transactions can start to fail due to not enough space for logs -
339 this can be especially problematic for transactions that want to de‐
340 allocate objects, as those might also fail
341 To solve both of these problems libpmemobj exposes the following func‐
343 tions:
344 • pmemobj_tx_log_append_buffer(),
346 • pmemobj_tx_xlog_append_buffer(),
348 • pmemobj_tx_log_auto_alloc()
350 pmemobj_tx_log_append_buffer() appends a given range of memory [addr,
352 addr + size) to the log type of the current transaction. type can be
353 one of the two values (with meanings described above):
354 • TX_LOG_TYPE_SNAPSHOT,
356 • TX_LOG_TYPE_INTENT
358 The range of memory must belong to the same pool the transaction is on
360 and must not be used by more than one thread at the same time. The
361 latter condition can be verified with tx.debug.verify_user_buffers ctl
362 (see pmemobj_ctl_get(3)).
363 The pmemobj_tx_xlog_append_buffer() function behaves exactly the same
365 as pmemobj_tx_log_append_buffer() when flags equals zero. flags is a
366 bitmask of the following values:
367 • POBJ_XLOG_APPEND_BUFFER_NO_ABORT - if the function does not end suc‐
369 cessfully, do not abort the transaction.
370 pmemobj_tx_log_snapshots_max_size calculates the maximum size of a buf‐
372 fer which will be able to hold nsizes snapshots, each of size sizes[i].
373 Application should not expect this function to return the same value
374 between restarts. In future versions of libpmemobj this function can
375 return smaller (because of better accuracy or space optimizations) or
376 higher (because of higher alignment required for better performance)
377 value. This function is independent of transaction stage and can be
378 called both inside and outside of transaction. If the returned value S
379 is greater than PMEMOBJ_MAX_ALLOC_SIZE, the buffer should be split into
380 N chunks of size PMEMOBJ_MAX_ALLOC_SIZE, where N is equal to (S / PMEM‐
381 OBJ_MAX_ALLOC_SIZE) (rounded down) and the last chunk of size (S - (N *
382 PMEMOBJ_MAX_ALLOC_SIZE)).
383 pmemobj_tx_log_intents_max_size calculates the maximum size of a buffer
385 which will be able to hold nintents intents. Just like with pmemo‐
386 bj_tx_log_snapshots_max_size, application should not expect this func‐
387 tion to return the same value between restarts, for the same reasons.
388 This function is independent of transaction stage and can be called
389 both inside and outside of transaction.
390 pmemobj_tx_log_auto_alloc() disables (on_off set to 0) or enables
392 (on_off set to 1) automatic allocation of internal logs of given type.
393 It can be used to verify that the buffer set with pmemobj_tx_log_ap‐
394 pend_buffer() is big enough to hold the log, without reaching out-of-
395 space scenario.
396 The pmemobj_tx_set_user_data() function associates custom volatile
398 state, represented by pointer data, with the current transaction. This
399 state can later be retrieved using pmemobj_tx_get_user_data() function.
400 If pmemobj_tx_set_user_data() was not called for a current transaction,
401 pmemobj_tx_get_user_data() will return NULL. These functions must be
402 called during TX_STAGE_WORK or TX_STAGE_ONABORT or TX_STAGE_ONCOMMIT or
403 TX_STAGE_FINALLY.
404 pmemobj_tx_set_failure_behavior() specifies what should happen in case
406 of an error within the transaction. It only affects functions which
407 take a NO_ABORT flag. If pmemobj_tx_set_failure_behavior() is called
408 with POBJ_TX_FAILURE_RETURN a NO_ABORT flag is implicitly passed to all
409 functions which accept this flag. If called with POBJ_TX_FAILURE_ABORT
410 then all functions abort the transaction (unless NO_ABORT flag is
411 passed explicitly). This setting is inherited by inner transactions.
412 It does not affect any of the outer transactions. Aborting on failure
413 is the default behavior. pmemobj_tx_get_failure_behavior() returns
414 failure behavior for the current transaction. Both pmemo‐
415 bj_tx_set_failure_behavior() and pmemobj_tx_get_failure_behavior() must
416 be called during TX_STAGE_WORK.
417
419 The pmemobj_tx_stage() function returns the stage of the current trans‐
420 action stage for a thread.
421 On success, pmemobj_tx_begin() returns 0. Otherwise, an error number
423 is returned.
424 The pmemobj_tx_begin() and pmemobj_tx_lock() functions return zero if
426 lockp is successfully added to the transaction. Otherwise, an error
427 number is returned.
428 The pmemobj_tx_xlock() function return zero if lockp is successfully
430 added to the transaction. Otherwise, the error number is returned, er‐
431 rno is set and when flags do not contain POBJ_XLOCK_NO_ABORT, the
432 transaction is aborted.
433 The pmemobj_tx_abort() and pmemobj_tx_commit() functions return no val‐
435 ue.
436 The pmemobj_tx_end() function returns 0 if the transaction was success‐
438 ful. Otherwise it returns the error code set by pmemobj_tx_abort().
439 Note that pmemobj_tx_abort() can be called internally by the library.
440 The pmemobj_tx_errno() function returns the error code of the last
442 transaction.
443 The pmemobj_tx_process() function returns no value.
445 On success, pmemobj_tx_log_append_buffer() returns 0. Otherwise, the
447 stage is changed to TX_STAGE_ONABORT, errno is set appropriately and
448 transaction is aborted.
449 On success, pmemobj_tx_xlog_append_buffer() returns 0. Otherwise, the
451 error number is returned, errno is set and when flags do not contain
452 POBJ_XLOG_NO_ABORT, the transaction is aborted.
453 On success, pmemobj_tx_log_auto_alloc() returns 0. Otherwise, the
455 transaction is aborted and an error number is returned.
456 On success, pmemobj_tx_log_snapshots_max_size() returns size of the
458 buffer. On failure it returns SIZE_MAX and sets errno appropriately.
459 On success, pmemobj_tx_log_intents_max_size() returns size of the buf‐
461 fer. On failure it returns SIZE_MAX and sets errno appropriately.
462
464 Transaction flow control is governed by the setjmp(3) and longjmp(3)
465 macros, and they are used in both the macro and function flavors of the
466 API. The transaction will longjmp on transaction abort. This has one
467 major drawback, which is described in the ISO C standard subsection
468 7.13.2.1. It says that the values of objects of automatic storage du‐
469 ration that are local to the function containing the setjmp invocation
470 that do not have volatile-qualified type and have been changed between
471 the setjmp invocation and longjmp call are indeterminate.
472 The following example illustrates the issue described above.
474 int *bad_example_1 = (int *)0xBAADF00D;
476 int *bad_example_2 = (int *)0xBAADF00D;
477 int *bad_example_3 = (int *)0xBAADF00D;
478 int * volatile good_example = (int *)0xBAADF00D;
479 TX_BEGIN(pop) {
481 bad_example_1 = malloc(sizeof(int));
482 bad_example_2 = malloc(sizeof(int));
483 bad_example_3 = malloc(sizeof(int));
484 good_example = malloc(sizeof(int));
485 /* manual or library abort called here */
487 pmemobj_tx_abort(EINVAL);
488 } TX_ONCOMMIT {
489 /*
490 * This section is longjmp-safe
491 */
492 } TX_ONABORT {
493 /*
494 * This section is not longjmp-safe
495 */
496 free(good_example); /* OK */
497 free(bad_example_1); /* undefined behavior */
498 } TX_FINALLY {
499 /*
500 * This section is not longjmp-safe on transaction abort only
501 */
502 free(bad_example_2); /* undefined behavior */
503 } TX_END
504 free(bad_example_3); /* undefined behavior */
506 Objects which are not volatile-qualified, are of automatic storage du‐
508 ration and have been changed between the invocations of setjmp(3) and
509 longjmp(3) (that also means within the work section of the transaction
510 after TX_BEGIN()) should not be used after a transaction abort, or
511 should be used with utmost care. This also includes code after the
512 TX_END macro.
513 libpmemobj(7) is not cancellation-safe. The pool will never be cor‐
515 rupted because of a canceled thread, but other threads may stall wait‐
516 ing on locks taken by that thread. If the application wants to use
517 pthread_cancel(3), it must disable cancellation before calling any
518 libpmemobj(7) APIs (see pthread_setcancelstate(3) with PTHREAD_CAN‐
519 CEL_DISABLE), and re-enable it afterwards. Deferring cancellation
520 (pthread_setcanceltype(3) with PTHREAD_CANCEL_DEFERRED) is not safe
521 enough, because libpmemobj(7) internally may call functions that are
522 specified as cancellation points in POSIX.
523 libpmemobj(7) relies on the library destructor being called from the
525 main thread. For this reason, all functions that might trigger de‐
526 struction (e.g. dlclose(3)) should be called in the main thread. Oth‐
527 erwise some of the resources associated with that thread might not be
528 cleaned up properly.
529
531 dlclose(3), longjmp(3), pmemobj_tx_add_range(3), pmemobj_tx_alloc(3),
532 pthread_setcancelstate(3), pthread_setcanceltype(3), setjmp(3), libp‐
533 memobj(7) and <https://pmem.io>
534PMDK - 2023-06-05 ()