libvmmalloc(7)

1LIBVMMALLOC(7)             PMDK Programmer's Manual             LIBVMMALLOC(7)
2
3
4

NAME

6       libvmmalloc - general purpose volatile memory allocation library
7

SYNOPSIS

9              $ LD_PRELOAD=libvmmalloc.so.1 command [ args... ]
10
11       or
12
13              #include <stdlib.h>
14              #ifndef __FreeBSD__
15                  #include <malloc.h>
16              #else
17                  #include <malloc_np.h>
18              #endif
19              #include <libvmmalloc.h>
20
21              cc [ flag... ] file... -lvmmalloc [ library... ]
22
23              void *malloc(size_t size);
24              void free(void *ptr);
25              void *calloc(size_t number, size_t size);
26              void *realloc(void *ptr, size_t size);
27              int posix_memalign(void **memptr, size_t alignment, size_t size);
28              void *aligned_alloc(size_t alignment, size_t size);
29              void *memalign(size_t alignment, size_t size);
30              void *valloc(size_t size);
31              void *pvalloc(size_t size);
32              size_t malloc_usable_size(const void *ptr);
33              void cfree(void *ptr);
34

DESCRIPTION

36       libvmmalloc  transparently converts all dynamic memory allocations into
37       Persistent Memory allocations.
38
39       The typical usage of libvmmalloc does not require any  modification  of
40       the  target program.  It is enough to load libvmmalloc before all other
41       libraries by setting the environment variable LD_PRELOAD.  When used in
42       that  way, libvmmalloc interposes the standard system memory allocation
43       routines, as defined in  malloc(3),  posix_memalign(3)  and  malloc_us‐
44       able_size(3), and provides that all dynamic memory allocations are made
45       from a memory pool built on a memory-mapped file, instead of the system
46       heap.  The memory managed by libvmmalloc may have different attributes,
47       depending on the file system containing  the  memory-mapped  file.   In
48       particular,  libvmmalloc  is  part of the Persistent Memory Development
49       Kit because it is sometimes useful to  use  non-volatile  memory  as  a
50       volatile  memory  pool,  leveraging  its capacity, cost, or performance
51       characteristics.
52
53       This library is no longer actively developed,  and  is  in  maintenance
54       mode, same as its underlying code backend (libvmem).  It is mature, and
55       is expected to be supported for foreseable future.
56
57       libvmmalloc may be also linked to the program, by providing the **-lvm‐
58       malloc* argument to the linker.  Then it becomes the default memory al‐
59       locator for the program.
60
61              NOTE: Due to the fact the library operates  on  a  memory-mapped
62              file,  it  may  not  work  properly  with  programs that perform
63              fork(2) not followed by exec(3). There are two variants  of  ex‐
64              perimental  fork(2)  support  available in libvmmalloc.  The de‐
65              sired library behavior may be selected  by  setting  the  VMMAL‐
66              LOC_FORK  environment  variable.   By  default variant #1 is en‐
67              abled.  See ENVIRONMENT for more details.
68
69       libvmmalloc uses the mmap(2) system call to create a pool  of  volatile
70       memory.   The library is most useful when used with Direct Access stor‐
71       age (DAX), which is memory-addressable persistent storage that supports
72       load/store  access  without  being  paged via the system page cache.  A
73       Persistent Memory-aware file system is typically used to  provide  this
74       type  of  access.  Memory-mapping a file from a Persistent Memory-aware
75       file system provides the raw memory pools, and  this  library  supplies
76       the traditional malloc interfaces on top of those pools.
77
78       The memory pool acting as a system heap replacement is created automat‐
79       ically at library initialization time.  The user may control its  loca‐
80       tion  and  size by setting the environment variables described in ENVI‐
81       RONMENT, below.  The allocated file space is reclaimed when the process
82       terminates or in case of system crash.
83
84       Under normal usage, libvmmalloc will never print messages or intention‐
85       ally cause the process to exit.  The library  uses  pthreads(7)  to  be
86       fully  MT-safe,  but never creates or destroys threads itself.  The li‐
87       brary does not make use of any signals, networking, and never calls se‐
88       lect(2) or poll(2).
89

ENVIRONMENT

91       The VMMALLOC_POOL_DIR and VMMALLOC_POOL_SIZE environment variables must
92       be set for libvmmalloc to work properly.  If  either  of  them  is  not
93       specified,  or if their values are not valid, the library prints an ap‐
94       propriate error message and terminates the process.  Any other environ‐
95       ment variables are optional.
96
97       · VMMALLOC_POOL_DIR=path
98
99       Specifies  a path to the directory where the memory pool file should be
100       created.  The directory must exist and be writable.
101
102       · VMMALLOC_POOL_SIZE=len
103
104       Defines the desired size (in bytes) of the memory pool file.   It  must
105       be  not less than the minimum allowed size VMMALLOC_MIN_POOL as defined
106       in <libvmmalloc.h>.
107
108              NOTE: Due to the fact the library adds some metadata to the mem‐
109              ory  pool,  the  amount of actual usable space is typically less
110              than the size of the memory pool file.
111
112       · VMMALLOC_FORK=val (EXPERIMENTAL)
113
114       VMMALLOC_FORK controls the behavior of libvmmalloc in case of  fork(3),
115       and can be set to the following values:
116
117       · 0  -  fork(2)  support is disabled.  The behavior of fork(2) is unde‐
118         fined in this case, but most likely results in memory pool corruption
119         and a program crash due to segmentation fault.
120
121       · 1 - The memory pool file is remapped with the MAP_PRIVATE flag before
122         the fork completes.  From this moment, any access to memory that mod‐
123         ifies  the  heap  pages, both in the parent and in the child process,
124         will  trigger  creation  of  a   copy   of   those   pages   in   RAM
125         (copy-on-write).   The  benefit  of this approach is that it does not
126         significantly increase the time of the initial  fork  operation,  and
127         does  not  require additional space on the file system.  However, all
128         subsequent memory allocations, and modifications of any memory  allo‐
129         cated  before  fork,  will consume system memory resources instead of
130         the memory pool.
131
132       This is the default option if VMMALLOC_FORK is not set.
133
134       · 2 - A copy of the entire memory pool file is created for the  use  of
135         the  child  process.  This requires additional space on the file sys‐
136         tem, but both the parent and the child process may still  operate  on
137         their memory pools, not consuming system memory resources.
138
139         NOTE: In case of large memory pools, creating a copy of the pool file
140         may stall the fork operation for a quite long time.
141
142       · 3 - The library first attempts to create a copy of  the  memory  pool
143         (as  for  option  #2),  but if it fails (i.e. because of insufficient
144         free space on the file system), it will fall back to option #1.
145
146         NOTE: Options 2 and 3 are not currently supported on FreeBSD.
147
148       Environment variables used for debugging are  described  in  DEBUGGING,
149       below.
150

CAVEATS

152       libvmmalloc relies on the library destructor being called from the main
153       thread.  For this reason, all functions that might trigger  destruction
154       (e.g.  dlclose(3)) should be called in the main thread.  Otherwise some
155       of the resources associated with that thread might not  be  cleaned  up
156       properly.
157

DEBUGGING

159       Two  versions  of  libvmmalloc are typically available on a development
160       system.  The normal version is optimized for performance.  That version
161       skips  checks that impact performance and never logs any trace informa‐
162       tion or performs any run-time assertions.  A second  version,  accessed
163       when using libraries from /usr/lib/pmdk_debug, contains run-time asser‐
164       tions and trace points.  The typical way to access the debug version is
165       to  set the LD_LIBRARY_PATH environment variable to /usr/lib/pmdk_debug
166       or /usr/lib64/pmdk_debug, as appropriate.   Debugging  output  is  con‐
167       trolled  using  the  following  environment variables.  These variables
168       have no effect on the non-debug version of the library.
169
170       · VMMALLOC_LOG_LEVEL
171
172       The value of VMMALLOC_LOG_LEVEL enables trace points in the debug  ver‐
173       sion of the library, as follows:
174
175       · 0  -  Tracing  is  disabled.   This  is the default level when VMMAL‐
176         LOC_LOG_LEVEL is not set.
177
178       · 1 - Additional details on any errors detected are logged, in addition
179         to returning the errno-based errors as usual.
180
181       · 2 - A trace of basic operations is logged.
182
183       · 3 - Enables a very verbose amount of function call tracing in the li‐
184         brary.
185
186       · 4 - Enables voluminous tracing information about all  memory  alloca‐
187         tions and deallocations.
188
189       Unless VMMALLOC_LOG_FILE is set, debugging output is written to stderr.
190
191       · VMMALLOC_LOG_FILE
192
193       Specifies  the  name  of a file where all logging information should be
194       written.  If the last character in the name is “-”, the PID of the cur‐
195       rent  process  will  be  appended to the file name when the log file is
196       created.  If VMMALLOC_LOG_FILE is not set, output is written to stderr.
197
198       · VMMALLOC_LOG_STATS
199
200       Setting VMMALLOC_LOG_STATS to 1 enables logging human-readable  summary
201       statistics at program termination.
202

NOTES

204       Unlike the normal malloc(3), which asks the system for additional memo‐
205       ry when it runs out, libvmmalloc allocates the size it is told  to  and
206       never attempts to grow or shrink that memory pool.
207

BUGS

209       libvmmalloc  may  not  work properly with programs that perform fork(2)
210       and do not call exec(3) immediately afterwards.   See  ENVIRONMENT  for
211       more details about experimental fork(2) support.
212
213       If  logging  is  enabled  in  the  debug version of the library and the
214       process performs fork(2), no new log file  is  created  for  the  child
215       process,  even if the configured log file name ends with “-”.  All log‐
216       ging information from the child process will be written to the log file
217       owned  by  the  parent process, which may lead to corruption or partial
218       loss of log data.
219
220       Malloc hooks (see malloc_hook(3)), are not supported when using  libvm‐
221       malloc.
222

ACKNOWLEDGEMENTS

224       libvmmalloc  depends  on  jemalloc,  written  by Jason Evans, to do the
225       heavy  lifting   of   managing   dynamic   memory   allocation.    See:
226       <http://www.canonware.com/jemalloc>
227