1PMEMKV(7) PMEMKV Programmer's Manual PMEMKV(7)
2
6 pmemkv - Key/Value Datastore for Persistent Memory
7
9 pmemkv is a key-value datastore framework optimized for persistent mem‐
10 ory. It provides native C API and C++ headers. Support for other lan‐
11 guages is described in the BINDINGS section below.
12 It has multiple storage engines, each optimized for a different use
14 case. They differ in implementation and capabilities:
15 • persistence - this is a trade-off between data preservation and per‐
17 formance; persistent engines retain their content and are power
18 fail/crash safe, but are slower; volatile engines are faster, but
19 keep their content only until the database is closed (or application
20 crashes; power fail occurs)
21 • concurrency - engines provide a varying degree of write scalability
23 in multi-threaded workloads. Concurrent engines support non-blocking
24 retrievals and, on average, highly scalable updates. For details see
25 the description of individual engines.
26 • keys’ ordering - “sorted” engines support querying above/below the
28 given key. Most sorted engines also support passing a custom com‐
29 parator object (see libpmemkv_config(3)). By default, pmemkv stores
30 elements in binary order (comparison is done using a function equiva‐
31 lent to std::string::compare).
32 Persistent engines usually use libpmemobj++ and PMDK to access NVDIMMs.
34 They can work with files on DAX filesystem (fsdax) or DAX device.
35
37 Engine Name Description Persistent Concurrent Sorted
38 ───────────────────────────────────────────────────────────────
39 cmap Concurrent Yes Yes No
40 hash map
41 vcmap Volatile con‐ No Yes No
42 current hash
43 map
44 vsmap Volatile No No Yes
45 sorted hash
46 map
47 blackhole Accepts ev‐ No Yes No
48 erything, re‐
49 turns nothing
50 The most mature and recommended engine to use for persistent use-cases
52 is cmap. It provides good performance results and stability.
53 Each engine can be manually turned on and off at build time, using
55 CMake options. All engines listed here are enabled and ready to use.
56 To configure an engine, pmemkv_config is used (libpmemkv_config(3)).
58 Below is a list of engines along with config parameters they expect.
59 Each parameter has corresponding function (pmemkv_config_put_path,
60 pmemkv_config_put_comparator, etc.), which guarantees type safety. For
61 example to insert path parameter to the config, user should call
62 pmemkv_config_put_path(). For some use cases, like creating config
63 from parsed input, it may be more convenient to insert parameters by
64 their type instead of name. Each parameter has a certain type and may
65 be inserted to a config using appropriate function (pmemkv_con‐
66 fig_put_string, pmemkv_config_put_int64, etc.). For example, to insert
67 a parameter of type string, pmemkv_config_put_string function may be
68 used. Those two ways of inserting parameters into config may be used
69 interchangeably.
70 For description of pmemkv core API see libpmemkv(3).
72 cmap
74 A persistent concurrent engine, backed by a hashmap that allows calling
75 get, put, and remove concurrently from multiple threads and ensures
76 good scalability. Rest of the methods (e.g. range query methods) are
77 not thread-safe and should not be called from more than one thread.
78 Data stored using this engine is persistent and guaranteed to be con‐
79 sistent in case of any kind of interruption (crash / power loss / etc).
80 Internally this engine uses persistent concurrent hashmap and persis‐
82 tent string from libpmemobj-cpp library (for details see
83 <https://github.com/pmem/libpmemobj-cpp>). Persistent string is used
84 as a type of a key and a value. Engine’s functions should not be
85 called within libpmemobj transactions (improper call by user will re‐
86 sult thrown exception).
87 This engine requires the following config parameters (see libp‐
89 memkv_config(3) for details how to set them):
90 • path – Path to a database file or to a poolset file (see poolset(5)
92 for details). Note that when using poolset file, size should be 0.
93 It’s used to open or create pool (layout “pmemkv”).
94 • type: string
96 • create_if_missing – If 1, pmemkv tries to open the pool and if that
98 doesn’t succeed, it creates it. If 0, pmemkv will rely on cre‐
99 ate_or_error_if_exists flag setting. If both create_* flags will be
100 false - pmemkv will open the pool (unless the path does not exist -
101 then it’ll fail).
102 • type: uint64_t
104 • default value: 0
106 • create_or_error_if_exists – If 1, pmemkv creates the file (but it
108 will fail if path exists). If 0, pmemkv will rely on create_if_miss‐
109 ing flag setting. If both create_* flags will be false - pmemkv will
110 open the pool (unless the path does not exist - then it’ll fail).
111 • type: uint64_t
113 • default value: 0
115 • size – Only needed if any of the above flags is 1. It specifies size
117 of the database [in bytes] to create.
118 • type: uint64_t
120 • min value: 8388608 (8MB)
122 • oid – Pointer to oid (for details see libpmemobj(7)) which points to
124 engine data. If oid is null, engine will allocate new data, other‐
125 wise it will use existing one.
126 • type: object
128 The following table shows four possible combinations of parameters
130 (where `-' means `cannot be set'):
131 # path cre‐ cre‐ size oid
133 ate_if_miss‐ ate_or_er‐
134 ing ror_if_ex‐
135 ists
136 ──────────────────────────────────────────────────
137 1 set 0 0 N/A -
138 2 set 1 - set -
139 3 set - 1 set -
140 4 - N/A N/A N/A set
141 ad 1: If none of the flags are set (default flags’ value is
143 false), pmemkv will only try to open the file and it fails if
144 the path does not exist. Size is ignored.
145 ad 2: If create_if_missing flag is set, pmemkv will try to open
147 the file (based on path) and if it doesn’t succeed, pmemkv tries
148 to create the file. Flag create_or_error_if_exists can’t be set
149 (or it should be set to 0).
150 ad 3: If create_or_error_if_exists flag is set, pmemkv will try
152 to create the file (and fails if it exists). Flag cre‐
153 ate_if_missing can’t be set (or it should be set to 0).
154 ad 4: If oid is set, path should not be set. Both flags and
156 size are ignored.
157 A database file or a poolset file can also be created using pmempool
159 utility (see pmempool-create(1)). When using pmempool create, “pmemkv”
160 should be passed as layout for cmap engine and “pmemkv_<engine-name>”
161 for other engines (e.g. “pmemkv_stree” for stree engine). Only PMEMOBJ
162 pools are supported.
163 vcmap
165 A volatile concurrent engine, backed by memkind. Data written using
166 this engine is lost after database is closed.
167 This engine is built on top of tbb::concurrent_hash_map data structure
169 and uses PMEM C++ allocator to allocate memory. std::basic_string is
170 used as a type of a key and a value. Memkind and TBB packages are re‐
171 quired.
172 This engine requires the following config parameters (see libp‐
174 memkv_config(3) for details how to set them):
175 • path – Path to an existing directory
177 • type: string
179 • size – Specifies size of the database [in bytes]
181 • type: uint64_t
183 • min value: 16777216 (16MB) (value MEMKIND_PMEM_MIN_SIZE is speci‐
185 fied in memkind.h)
186 vsmap
188 A volatile single-threaded sorted engine, backed by memkind. Data
189 written using this engine is lost after database is closed.
190 This engine is built on top of std::map and uses PMEM C++ allocator to
192 allocate memory. std::basic_string is used as a type of a key and a
193 value. Memkind package is required.
194 This engine requires the following config parameters (see libp‐
196 memkv_config(3) for details how to set them):
197 • path – Path to an existing directory
199 • type: string
201 • size – Specifies size of the database [in bytes]
203 • type: uint64_t
205 • min value: 16777216 (16MB) (value MEMKIND_PMEM_MIN_SIZE is speci‐
207 fied in memkind.h)
208 • comparator – (optional) Specified comparator used by the engine
210 • type: object
212 blackhole
214 A volatile engine that accepts an unlimited amount of data, but never
215 returns anything. Internally, blackhole does not use a persistent pool
216 or any durable structure. The intended use of this engine is to pro‐
217 file and tune high-level bindings, and similar cases when persistence
218 should be intentionally skipped. No additional packages are required.
219 No supported configuration parameters.
220 Experimental engines
222 There are also more engines in various states of development, for de‐
223 tails see <https://github.com/pmem/pmemkv/blob/master/doc/ENGINES-ex‐
224 perimental.md>. Some of them (radix, tree3, stree and csmap) requires
225 the config parameters like cmap and similarly to cmap should not be
226 used within libpmemobj transaction(s).
227
229 Bindings for other languages are available on GitHub. Currently they
230 support only subset of native API.
231 Existing bindings:
233 • Java - for details see <https://github.com/pmem/pmemkv-java>
235 • Node.js - for details see <https://github.com/pmem/pmemkv-nodejs>
237 • Python - for details see <https://github.com/pmem/pmemkv-python>
239 • Ruby - for details see <https://github.com/pmem/pmemkv-ruby>
241
243 libpmemkv(3), libpmemkv_config(3), libpmemkv_iterator(3), pmempool(1),
244 libpmemobj(7) and <https://pmem.io>
245PMEMKV - pmemkv version 1.5.0 2022-01-28 PMEMKV(7)