1PMEMKV(7)                 PMEMKV Programmer's Manual                 PMEMKV(7)
2
3
4

NAME

6       pmemkv - Key/Value Datastore for Persistent Memory
7

DESCRIPTION

9       pmemkv is a key-value datastore framework optimized for persistent mem‐
10       ory.  It provides native C API and C++ headers.  Support for other lan‐
11       guages is described in the BINDINGS section below.
12
13       It  has  multiple  storage  engines, each optimized for a different use
14       case.  They differ in implementation and capabilities:
15
16       • persistence - this is a trade-off between data preservation and  per‐
17         formance;  persistent  engines  retain  their  content  and are power
18         fail/crash safe, but are slower; volatile  engines  are  faster,  but
19         keep  their content only until the database is closed (or application
20         crashes; power fail occurs)
21
22       • concurrency - engines provide a varying degree of  write  scalability
23         in multi-threaded workloads.  Concurrent engines support non-blocking
24         retrievals and, on average, highly scalable updates.  For details see
25         the description of individual engines.
26
27       • keys’  ordering  -  “sorted” engines support querying above/below the
28         given key.  Most sorted engines also support passing  a  custom  com‐
29         parator  object (see libpmemkv_config(3)).  By default, pmemkv stores
30         elements in binary order (comparison is done using a function equiva‐
31         lent to std::string::compare).
32
33       Persistent engines usually use libpmemobj++ and PMDK to access NVDIMMs.
34       They can work with files on DAX filesystem (fsdax) or DAX device.
35

ENGINES

37       Engine Name   Description     Persistent   Concurrent   Sorted
38       ───────────────────────────────────────────────────────────────
39       cmap          Concurrent         Yes          Yes         No
40                     hash map
41       vcmap         Volatile con‐       No          Yes         No
42                     current  hash
43                     map
44       vsmap         Volatile            No           No        Yes
45                     sorted   hash
46                     map
47       blackhole     Accepts   ev‐       No          Yes         No
48                     erything, re‐
49                     turns nothing
50
51       The  most mature and recommended engine to use for persistent use-cases
52       is cmap.  It provides good performance results and stability.
53
54       Each engine can be manually turned on and  off  at  build  time,  using
55       CMake options.  All engines listed here are enabled and ready to use.
56
57       To  configure  an  engine, pmemkv_config is used (libpmemkv_config(3)).
58       Below is a list of engines along with config  parameters  they  expect.
59       Each  parameter  has  corresponding  function  (pmemkv_config_put_path,
60       pmemkv_config_put_comparator, etc.), which guarantees type safety.  For
61       example  to  insert  path  parameter  to  the  config, user should call
62       pmemkv_config_put_path().  For some use  cases,  like  creating  config
63       from  parsed  input,  it may be more convenient to insert parameters by
64       their type instead of name.  Each parameter has a certain type and  may
65       be  inserted  to  a  config  using  appropriate  function  (pmemkv_con‐
66       fig_put_string, pmemkv_config_put_int64, etc.).  For example, to insert
67       a  parameter  of  type string, pmemkv_config_put_string function may be
68       used.  Those two ways of inserting parameters into config may  be  used
69       interchangeably.
70
71       For description of pmemkv core API see libpmemkv(3).
72
73   cmap
74       A persistent concurrent engine, backed by a hashmap that allows calling
75       get, put, and remove concurrently from  multiple  threads  and  ensures
76       good  scalability.   Rest of the methods (e.g. range query methods) are
77       not thread-safe and should not be called from  more  than  one  thread.
78       Data  stored  using this engine is persistent and guaranteed to be con‐
79       sistent in case of any kind of interruption (crash / power loss / etc).
80
81       Internally this engine uses persistent concurrent hashmap  and  persis‐
82       tent    string   from   libpmemobj-cpp   library   (for   details   see
83       <https://github.com/pmem/libpmemobj-cpp>).  Persistent string  is  used
84       as  a  type  of  a  key  and a value.  Engine’s functions should not be
85       called within libpmemobj transactions (improper call by user  will  re‐
86       sult thrown exception).
87
88       This  engine  requires  the  following  config  parameters  (see  libp‐
89       memkv_config(3) for details how to set them):
90
91path – Path to a database file or to a poolset file  (see  poolset(5)
92         for  details).   Note that when using poolset file, size should be 0.
93         It’s used to open or create pool (layout “pmemkv”).
94
95         • type: string
96
97create_if_missing – If 1, pmemkv tries to open the pool and  if  that
98         doesn’t  succeed,  it  creates  it.   If  0, pmemkv will rely on cre‐
99         ate_or_error_if_exists flag setting.  If both create_* flags will  be
100         false  -  pmemkv will open the pool (unless the path does not exist -
101         then it’ll fail).
102
103         • type: uint64_t
104
105         • default value: 0
106
107create_or_error_if_exists – If 1, pmemkv creates  the  file  (but  it
108         will fail if path exists).  If 0, pmemkv will rely on create_if_miss‐
109         ing flag setting.  If both create_* flags will be false - pmemkv will
110         open the pool (unless the path does not exist - then it’ll fail).
111
112         • type: uint64_t
113
114         • default value: 0
115
116size – Only needed if any of the above flags is 1.  It specifies size
117         of the database [in bytes] to create.
118
119         • type: uint64_t
120
121         • min value: 8388608 (8MB)
122
123oid – Pointer to oid (for details see libpmemobj(7)) which points  to
124         engine  data.   If oid is null, engine will allocate new data, other‐
125         wise it will use existing one.
126
127         • type: object
128
129       The following table shows  four  possible  combinations  of  parameters
130       (where `-' means `cannot be set'):
131
132       #   path   cre‐           cre‐         size   oid
133                  ate_if_miss‐   ate_or_er‐
134                  ing            ror_if_ex‐
135                                 ists
136       ──────────────────────────────────────────────────
137       1   set         0             0        N/A     -
138       2   set         1             -        set     -
139       3   set         -             1        set     -
140       4    -         N/A           N/A       N/A    set
141
142              ad 1: If none of the flags are  set  (default  flags’  value  is
143              false),  pmemkv  will  only try to open the file and it fails if
144              the path does not exist.  Size is ignored.
145
146              ad 2: If create_if_missing flag is set, pmemkv will try to  open
147              the file (based on path) and if it doesn’t succeed, pmemkv tries
148              to create the file.  Flag create_or_error_if_exists can’t be set
149              (or it should be set to 0).
150
151              ad  3: If create_or_error_if_exists flag is set, pmemkv will try
152              to create  the  file  (and  fails  if  it  exists).   Flag  cre‐
153              ate_if_missing can’t be set (or it should be set to 0).
154
155              ad  4:  If  oid  is set, path should not be set.  Both flags and
156              size are ignored.
157
158       A database file or a poolset file can also be  created  using  pmempool
159       utility (see pmempool-create(1)).  When using pmempool create, “pmemkv”
160       should be passed as layout for cmap engine  and  “pmemkv_<engine-name>”
161       for other engines (e.g. “pmemkv_stree” for stree engine).  Only PMEMOBJ
162       pools are supported.
163
164   vcmap
165       A volatile concurrent engine, backed by memkind.   Data  written  using
166       this engine is lost after database is closed.
167
168       This  engine is built on top of tbb::concurrent_hash_map data structure
169       and uses PMEM C++ allocator to allocate memory.   std::basic_string  is
170       used  as a type of a key and a value.  Memkind and TBB packages are re‐
171       quired.
172
173       This  engine  requires  the  following  config  parameters  (see  libp‐
174       memkv_config(3) for details how to set them):
175
176path – Path to an existing directory
177
178         • type: string
179
180size – Specifies size of the database [in bytes]
181
182         • type: uint64_t
183
184         • min  value:  16777216 (16MB) (value MEMKIND_PMEM_MIN_SIZE is speci‐
185           fied in memkind.h)
186
187   vsmap
188       A volatile single-threaded sorted  engine,  backed  by  memkind.   Data
189       written using this engine is lost after database is closed.
190
191       This  engine is built on top of std::map and uses PMEM C++ allocator to
192       allocate memory.  std::basic_string is used as a type of a  key  and  a
193       value.  Memkind package is required.
194
195       This  engine  requires  the  following  config  parameters  (see  libp‐
196       memkv_config(3) for details how to set them):
197
198path – Path to an existing directory
199
200         • type: string
201
202size – Specifies size of the database [in bytes]
203
204         • type: uint64_t
205
206         • min value: 16777216 (16MB) (value MEMKIND_PMEM_MIN_SIZE  is  speci‐
207           fied in memkind.h)
208
209comparator – (optional) Specified comparator used by the engine
210
211         • type: object
212
213   blackhole
214       A  volatile  engine that accepts an unlimited amount of data, but never
215       returns anything.  Internally, blackhole does not use a persistent pool
216       or  any  durable structure.  The intended use of this engine is to pro‐
217       file and tune high-level bindings, and similar cases  when  persistence
218       should  be intentionally skipped.  No additional packages are required.
219       No supported configuration parameters.
220
221   Experimental engines
222       There are also more engines in various states of development,  for  de‐
223       tails  see  <https://github.com/pmem/pmemkv/blob/master/doc/ENGINES-ex
224       perimental.md>.  Some of them (radix, tree3, stree and csmap)  requires
225       the  config  parameters  like  cmap and similarly to cmap should not be
226       used within libpmemobj transaction(s).
227

BINDINGS

229       Bindings for other languages are available on GitHub.   Currently  they
230       support only subset of native API.
231
232       Existing bindings:
233
234       • Java - for details see <https://github.com/pmem/pmemkv-java>
235
236       • Node.js - for details see <https://github.com/pmem/pmemkv-nodejs>
237
238       • Python - for details see <https://github.com/pmem/pmemkv-python>
239
240       • Ruby - for details see <https://github.com/pmem/pmemkv-ruby>
241

SEE ALSO

243       libpmemkv(3),  libpmemkv_config(3), libpmemkv_iterator(3), pmempool(1),
244       libpmemobj(7) and <https://pmem.io>
245
246
247
248PMEMKV - pmemkv version 1.5.0     2022-01-28                         PMEMKV(7)
Impressum