1PMEMKV(7) PMEMKV Programmer's Manual PMEMKV(7)
2
3
4
6 pmemkv - Key/Value Datastore for Persistent Memory
7
9 pmemkv is a key-value datastore framework optimized for persistent mem‐
10 ory. It provides native C API and C++ headers. Support for other lan‐
11 guages is described in the BINDINGS section below.
12
13 It has multiple storage engines, each optimized for a different use
14 case. They differ in implementation and capabilities:
15
16 · persistence - this is a trade-off between data preservation and per‐
17 formance; persistent engines retain their content and are power
18 fail/crash safe, but are slower; volatile engines are faster, but
19 keep their content only until the database is closed (or application
20 crashes; power fail occurs)
21
22 · concurrency - engines provide a varying degree of write scalability
23 in multi-threaded workloads. Concurrent engines support non-blocking
24 retrievals and, on average, highly scalable updates. For details see
25 the description of individual engines.
26
27 · keys’ ordering - “sorted” engines support querying above/below the
28 given key
29
30 Persistent engines usually use libpmemobj++ and PMDK to access NVDIMMs.
31 They can work with files on DAX filesystem (fsdax) or DAX device.
32
33 For description of pmemkv core API see libpmemkv(3). For description
34 of pmemkv configuration API see libpmemkv_config(3).
35
37 Engine Name Description Persistent? Concurrent? Sorted?
38 ──────────────────────────────────────────────────────────────────
39 cmap Concurrent Yes Yes No
40 hash map
41 vcmap Volatile con‐ No Yes No
42 current hash
43 map
44 vsmap Volatile No No Yes
45 sorted hash
46 map
47 blackhole Accepts ev‐ No Yes No
48 erything, re‐
49 turns nothing
50
51 The most mature and recommended engine to use for persistent use-cases
52 is cmap. It provides good performance results and stability.
53
54 Each engine can be manually turned on and off at build time, using
55 CMake options. All engines listed here are enabled and ready to use.
56
57 cmap
58 A persistent concurrent engine, backed by a hashmap that allows calling
59 get, put, and remove concurrently from multiple threads and ensures
60 good scalability. Rest of the methods (e.g. range query methods) are
61 not thread-safe and should not be called from more than one thread.
62 Data stored using this engine is persistent and guaranteed to be con‐
63 sistent in case of any kind of interruption (crash / power loss / etc).
64
65 Internally this engine uses persistent concurrent hashmap and persis‐
66 tent string from libpmemobj-cpp library (for details see
67 <https://github.com/pmem/libpmemobj-cpp>). Persistent string is used
68 as a type of a key and a value. Engine’s functions should not be
69 called within libpmemobj transactions (improper call by user will re‐
70 sult thrown exception). libpmemobj-cpp packages are required.
71
72 This engine requires the following config parameters (see libp‐
73 memkv_config(3) for details how to set them):
74
75 · path – Path to a database file or to a poolset file (see poolset(5)
76 for details). Not that when using poolset file, size should be 0
77
78 · type: string
79
80 · force_create – If 0, pmemkv opens file specified by `path', otherwise
81 it creates it.
82
83 · type: uint64_t
84
85 · default value: 0
86
87 · size – Only needed when force_create is not 0, specifies size of the
88 database [in bytes].
89
90 · type: uint64_t
91
92 · min value: 8388608 (8MB)
93
94 · oid – Pointer to oid (for details see libpmemobj(7)) which points to
95 engine data. If oid is null, engine will allocate new data, other‐
96 wise it will use existing one.
97
98 · type: object
99
100 The following table shows three possible combinations of parameters
101 (where `-' means `cannot be set'):
102
103 # path force_create size oid
104 ─────────────────────────────────────
105 1 set 0 - -
106 2 set 1 set -
107 3 - - - set
108
109 A database file or a poolset file can also be created using pmempool
110 utility (see pmempool-create(1)). When using pmempool create, “pmemkv”
111 should be passed as layout. Only PMEMOBJ pools are supported.
112
113 vcmap
114 A volatile concurrent engine, backed by memkind. Data written using
115 this engine is lost after database is closed.
116
117 This engine is built on top of tbb::concurrent_hash_map data structure
118 and uses PMEM C++ allocator to allocate memory. std::basic_string is
119 used as a type of a key and a value. Memkind, TBB and libpmemobj-cpp
120 packages are required.
121
122 This engine requires the following config parameters (see libp‐
123 memkv_config(3) for details how to set them):
124
125 · path – Path to an existing directory
126
127 · type: string
128
129 · size – Specifies size of the database [in bytes]
130
131 · type: uint64_t
132
133 · min value: 8388608 (8MB)
134
135 vsmap
136 A volatile single-threaded sorted engine, backed by memkind. Data
137 written using this engine is lost after database is closed.
138
139 This engine is built on top of std::map and uses PMEM C++ allocator to
140 allocate memory. std::basic_string is used as a type of a key and a
141 value. Memkind and libpmemobj-cpp packages are required.
142
143 This engine requires the following config parameters (see libp‐
144 memkv_config(3) for details how to set them):
145
146 · path – Path to an existing directory
147
148 · type: string
149
150 · size – Specifies size of the database [in bytes]
151
152 · type: uint64_t
153
154 · min value: 8388608 (8MB)
155
156 blackhole
157 A volatile engine that accepts an unlimited amount of data, but never
158 returns anything. Internally, blackhole does not use a persistent pool
159 or any durable structure. The intended use of this engine is to pro‐
160 file and tune high-level bindings, and similar cases when persistence
161 should be intentionally skipped. No additional packages are required.
162 No supported configuration parameters.
163
164 Experimental engines
165 There are also more engines in various states of development, for de‐
166 tails see <https://github.com/pmem/pmemkv>. Two of them (tree3 and
167 stree) requires the config parameters like cmap and similarly to cmap
168 should not be used within libpmemobj transaction(s).
169
171 Bindings for other languages are available on GitHub. Currently they
172 support only subset of native API.
173
174 Existing bindings:
175
176 · Java - for details see <https://github.com/pmem/pmemkv-java>
177
178 · JNI - for details see <https://github.com/pmem/pmemkv-jni>
179
180 · Node.js - for details see <https://github.com/pmem/pmemkv-nodejs>
181
182 · Python - for details see <https://github.com/pmem/pmemkv-python>
183
184 · Ruby - for details see <https://github.com/pmem/pmemkv-ruby>
185
187 libpmemkv(3), libpmemkv_config(3), pmempool(1), libpmemobj(7) and
188 <https://pmem.io>
189
190
191
192PMEMKV - pmemkv version 1.1 2020-02-12 PMEMKV(7)