poolset(5)

1()                         PMDK Programmer's Manual                         ()
2
3
4

NAME

6       poolset - persistent memory pool configuration file format
7

SYNOPSIS

9              mypool.set
10

DESCRIPTION

12       Depending  on  the  configuration  of  the  system,  the available non-
13       volatile memory space may be divided into multiple memory devices.   In
14       such  case, the maximum size of the transactional object store could be
15       limited by the capacity of a single memory device.  Therefore, libpmem‐
16       obj(7),  libpmemblk(7)  and  libpmemlog(7) allow building object stores
17       spanning multiple memory devices by creation of persistent memory pools
18       consisting of multiple files, where each part of such a pool set may be
19       stored on a different pmem-aware filesystem.
20
21       To improve reliability and eliminate single point of failure, libpmemo‐
22       bj(7)  also  allows all the data written to a persistent memory pool to
23       be copied to local pool replicas, thereby providing backup for the per‐
24       sistent memory pool by producing a mirrored pool set.  In practice, the
25       pool replicas may be considered as binary copies of the  “master”  pool
26       set.   Data  replication is not supported in libpmemblk(7) and libpmem‐
27       log(7).
28
29       The set file for each type of pool is a plain text file.  Lines in  the
30       file are formatted as follows:
31
32       • The first line of the file must be the literal string “PMEMPOOLSET”
33
34       • The pool parts are specified, one per line, in the format:
35
36         size pathname
37
38       • Replica  sections,  if  any, start with the literal string “REPLICA”.
39         See REPLICAS, below, for further details.
40
41       • Pool set options, if any, start with literal string OPTION.  See POOL
42         SET OPTIONS below for details.
43
44       • Lines starting with “#” are considered comments and are ignored.
45
46       The  size  must be compliant with the format specified in IEC 80000-13,
47       IEEE 1541 or the Metric Interchange Format.  These standards accept  SI
48       units with obligatory B - kB, MB, GB, ...  (multiplier by 1000) suffix‐
49       es, and IEC units with optional “iB” - KiB, MiB, GiB, ..., K, M, G, ...
50       - (multiplier by 1024) suffixes.
51
52       pathname must be an absolute pathname.
53
54       The  pathname  of  a part can point to a Device DAX.  Device DAX is the
55       device-centric analogue of Filesystem DAX.  It allows memory ranges  to
56       be allocated and mapped without need of an intervening file system.
57
58       Pools created on Device DAX have additional options and restrictions:
59
60       • The  size  may be set to “AUTO”, in which case the size of the device
61         will be automatically resolved at pool creation time.
62
63       • To concatenate more than one Device DAX device  into  a  single  pool
64         set,  the  configured internal alignment of the devices must be 4KiB,
65         unless the SINGLEHDR or NOHDRS option is used in the pool  set  file.
66         See POOL SET OPTIONS below for details.
67
68       Please  see  ndctl-create-namespace(1)  for  more information on Device
69       DAX, including how to configure desired alignment.
70
71       The minimum file size of each part of the pool set is defined  as  fol‐
72       lows:
73
74       • For block pools, as PMEMBLK_MIN_PART in <libpmemblk.h>
75
76       • For object pools, as PMEMOBJ_MIN_PART in <libpmemobj.h>
77
78       • For log pools, as PMEMLOG_MIN_PART in <libpmemlog.h>
79
80       The net pool size of the pool set is equal to:
81
82              net_pool_size = sum_over_all_parts(page_aligned_part_size - 4KiB) + 4KiB
83
84       where
85
86              page_aligned_part_size = part_size & ~(page_size - 1)
87
88       Note  that  page  size is OS specific.  For more information please see
89       sysconf(3).
90
91       The minimum net pool size of a pool set is defined as follows:
92
93       • For block pools, as PMEMBLK_MIN_POOL in <libpmemblk.h>
94
95       • For object pools, as PMEMOBJ_MIN_POOL in <libpmemobj.h>
96
97       • For log pools, as PMEMLOG_MIN_POOL in <libpmemlog.h>
98
99       Here is an example “mypool.set” file:
100
101              PMEMPOOLSET
102              OPTION NOHDRS
103              100G /mountpoint0/myfile.part0
104              200G /mountpoint1/myfile.part1
105              400G /mountpoint2/myfile.part2
106
107       The files in the set may be created by running  one  of  the  following
108       commands.  To create a block pool:
109
110              $ pmempool create blk <bsize> mypool.set
111
112       To create a log pool:
113
114              $ pmempool create log mypool.set
115

REPLICAS

117       Sections  defining  replica  sets  are optional.  There may be multiple
118       replica sections.
119
120       Replica sections begin with a line containing only the  literal  string
121       “REPLICA”, followed by one or more pool part lines as described above.
122
123       Here is an example “myobjpool.set” file with replicas:
124
125              PMEMPOOLSET
126              100G /mountpoint0/myfile.part0
127              200G /mountpoint1/myfile.part1
128              400G /mountpoint2/myfile.part2
129
130              # local replica
131              REPLICA
132              500G /mountpoint3/mymirror.part0
133              200G /mountpoint4/mymirror.part1
134
135       The  files in the object pool set may be created by running the follow‐
136       ing command:
137
138              $ pmempool create --layout="mylayout" obj myobjpool.set
139

POOL SET OPTIONS

141       Pool set options can appear anywhere after the  line  with  PMEMPOOLSET
142       string.   Pool set file can contain several pool set options.  The fol‐
143       lowing options are supported:
144
145       • SINGLEHDR
146
147       If the SINGLEHDR option is used, only the first part  in  each  replica
148       contains  the  pool part internal metadata.  In that case the effective
149       size of a replica is the sum of sizes of all its part  files  decreased
150       once by 4096 bytes.
151
152       When  using the SINGLEHDR option, one can concatenate more than one De‐
153       vice DAX devices with any internal alignments in one replica.
154
155       Using the SINGLEHDR option has important implications for  data  integ‐
156       rity  checking  and  recoverability  in case of a pool set damage.  See
157       pmempool_sync() API for more information about pool set recovery.
158

DIRECTORIES

160       Providing a directory as a part’s pathname allows the pool  to  dynami‐
161       cally  create  files and consequently removes the user-imposed limit on
162       the size of the pool.
163
164       The size argument of a part in a directory poolset becomes the size  of
165       the  address  space reservation required for the pool.  In other words,
166       the size argument is the maximum theoretical size of the mapping.  This
167       value can be freely increased between instances of the application, but
168       decreasing it below the real required space will  result  in  an  error
169       when attempting to open the pool.
170
171       The directory must NOT contain user created files with extension .pmem,
172       otherwise the behavior is undefined.  If a file created by the  library
173       within  the  directory is in any way altered (resized, renamed) the be‐
174       havior is undefined.
175
176       A directory poolset must exclusively use directories to specify paths -
177       combining  files  and  directories  will  result in an error.  A single
178       replica can consist of one or more directories.  If there are  multiple
179       directories,  the  address space reservation is equal to the sum of the
180       sizes.
181
182       The order in which the files are created is unspecified,  but  the  li‐
183       brary will try to maintain equal usage of the directories.
184
185       By default pools grow in 128 megabyte increments.
186
187       Only poolsets with the SINGLEHDR option can safely use directories.
188

NOTES

190       Creation  of  all  the parts of the pool set and the associated replica
191       sets can be done with the pmemobj_create(3), pmemblk_create(3) or pmem‐
192       log_create(3) function, or by using the pmempool(1) utility.
193
194       Restoring data from a replica can be done by using the pmempool-sync(1)
195       command or the pmempool_sync() API from the libpmempool(7) library.
196
197       Modifications of a pool set file configuration can be done by using the
198       pmempool-transform(1)  command or the pmempool_transform() API from the
199       libpmempool(7) library.
200
201       When creating a pool set consisting of multiple files, or when creating
202       a  replicated  pool set, the path argument passed to pmemobj_create(3),
203       pmemblk_create(3) or pmemlog_create(3) must point to  the  special  set
204       file  that defines the pool layout and the location of all the parts of
205       the pool set.
206
207       When opening a pool set consisting of multiple files, or when opening a
208       replicated pool set, the path argument passed to pmemobj_open(3), pmem‐
209       blk_open(3) or pmemlog_open(3) must point to the same set file that was
210       used for pool set creation.
211