1mm(3) Shared Memory Library mm(3)
2
6 OSSP mm - Shared Memory Allocation
7
9 OSSP mm 1.4.2 (15-Aug-2006)
10
12 #include "mm.h"
13 Global Malloc-Replacement API
15 int MM_create(size_t size, const char *file);
17 int MM_permission(mode_t mode, uid_t owner, gid_t group);
18 void MM_reset(void);
19 void MM_destroy(void);
20 int MM_lock(mm_lock_mode mode);
21 int MM_unlock(void);
22 void *MM_malloc(size_t size);
23 void *MM_realloc(void *ptr, size_t size);
24 void MM_free(void *ptr);
25 void *MM_calloc(size_t number, size_t size);
26 char *MM_strdup(const char *str);
27 size_t MM_sizeof(void *ptr);
28 size_t MM_maxsize(void);
29 size_t MM_available(void);
30 char *MM_error(void);
31 Standard Malloc-Style API
33 MM *mm_create(size_t size, char *file);
35 int mm_permission(MM *mm, mode_t mode, uid_t owner, gid_t group);
36 void mm_reset(MM *mm);
37 void mm_destroy(MM *mm);
38 int mm_lock(MM *mm, mm_lock_mode mode);
39 int mm_unlock(MM *mm);
40 void *mm_malloc(MM *mm, size_t size);
41 void *mm_realloc(MM *mm, void *ptr, size_t size);
42 void mm_free(MM *mm, void *ptr);
43 void *mm_calloc(MM *mm, size_t number, size_t size);
44 char *mm_strdup(MM *mm, const char *str);
45 size_t mm_sizeof(MM *mm, void *ptr);
46 size_t mm_maxsize(void);
47 size_t mm_available(MM *mm);
48 char *mm_error(void);
49 void mm_display_info(MM *mm);
50 Low-level Shared Memory API
52 void *mm_core_create(size_t size, char *file);
54 int mm_core_permission(void *core, mode_t mode, uid_t owner, gid_t group);
55 void mm_core_delete(void *core);
56 int mm_core_lock(void *core, mm_lock_mode mode);
57 int mm_core_unlock(void *core);
58 size_t mm_core_size(void *core);
59 size_t mm_core_maxsegsize(void);
60 size_t mm_core_align2page(size_t size);
61 size_t mm_core_align2click(size_t size);
62 Internal Library API
64 void mm_lib_error_set(unsigned int, const char *str);
66 char *mm_lib_error_get(void);
67 int mm_lib_version(void);
68
70 The OSSP mm library is a 2-layer abstraction library which simplifies
71 the usage of shared memory between forked (and this way strongly
72 related) processes under Unix platforms. On the first (lower) layer it
73 hides all platform dependent implementation details (allocation and
74 locking) when dealing with shared memory segments and on the second
75 (higher) layer it provides a high-level malloc(3)-style API for a con‐
76 venient and well known way to work with data-structures inside those
77 shared memory segments.
78 The abbreviation OSSP mm is historically and originally comes from the
80 phrase ``memory mapped'' as used by the POSIX.1 mmap(2) function.
81 Because this facility is internally used by this library on most plat‐
82 forms to establish the shared memory segments.
83 LIBRARY STRUCTURE
85 This library is structured into three main APIs which are internally
87 based on each other:
88 Global Malloc-Replacement API
90 This is the most high-level API which directly can be used as
91 replacement API for the POSIX.1 memory allocation API (malloc(2)
92 and friends). This is useful when converting heap based data struc‐
93 tures to shared memory based data structures without the need to
94 change the code dramatically. All which is needed is to prefix the
95 POSIX.1 memory allocation functions with `"MM_"', i.e. `"malloc"'
96 becomes `"MM_malloc"', `"strdup"' becomes `"MM_strdup"', etc. This
97 API internally uses just a global `"MM *"' pool for calling the
98 corresponding functions (those with prefix `"mm_"') of the Standard
99 Malloc-Style API.
100 Standard Malloc-Style API
102 This is the standard high-level memory allocation API. Its inter‐
103 face is similar to the Global Malloc-Replacement API but it uses an
104 explicit `"MM *"' pool to operate on. That is why every function of
105 this API has an argument of type `"MM *"' as its first argument.
106 This API provides a comfortable way to work with small dynamically
107 allocated shared memory chunks inside large statically allocated
108 shared memory segments. It is internally based on the Low-Level
109 Shared Memory API for creating the underlying shared memory seg‐
110 ment.
111 Low-Level Shared Memory API
113 This is the basis of the whole OSSP mm library. It provides low-
114 level functions for creating shared memory segments with mutual
115 exclusion (in short mutex) capabilities in a portable way. Inter‐
116 nally the shared memory and mutex facility is implemented in vari‐
117 ous platform-dependent ways. A list of implementation variants fol‐
118 lows under the next topic.
119 SHARED MEMORY IMPLEMENTATION
121 Internally the shared memory facility is implemented in various plat‐
123 form-dependent ways. Each way has its own advantages and disadvantages
124 (in addition to the fact that some variants aren't available at all on
125 some platforms). The OSSP mm library's configuration procedure tries
126 hard to make a good decision. The implemented variants are now given
127 for overview and background reasons with their advantages and disadvan‐
128 tages and in an ascending order, i.e. the OSSP mm configuration mecha‐
129 nism chooses the last available one in the list as the preferred vari‐
130 ant.
131 Classical mmap(2) on temporary file (MMFILE)
133 Advantage: maximum portable. Disadvantage: needs a temporary file
134 on the filesystem.
135 mmap(2) via POSIX.1 shm_open(3) on temporary file (MMPOSX)
137 Advantage: standardized by POSIX.1 and theoretically portable.
138 Disadvantage: needs a temporary file on the filesystem and is is
139 usually not available on existing Unix platform.
140 SVR4-style mmap(2) on "/dev/zero" device (MMZERO)
142 Advantage: widely available and mostly portable on SVR4 platforms.
143 Disadvantage: needs the "/dev/zero" device and a mmap(2) which sup‐
144 ports memory mapping through this device.
145 SysV IPC shmget(2) (IPCSHM)
147 Advantage: does not need a temporary file or external device. Dis‐
148 advantage: although available on mostly all modern Unix platforms,
149 it has strong restrictions like the maximum size of a single shared
150 memory segment (can be as small as 100KB, but depends on the plat‐
151 form).
152 4.4BSD-style mmap(2) via "MAP_ANON" facility (MMANON)
154 Advantage: does not need a temporary file or external device. Dis‐
155 advantage: usually only available on BSD platforms and derivatives.
156 LOCKING IMPLEMENTATION
158 As for the shared memory facility, internally the locking facility is
160 implemented in various platform-dependent ways. They are again listed
161 in ascending order, i.e. the OSSP mm configuration mechanism chooses
162 the last available one in the list as the preferred variant. The list
163 of implemented variants is:
164 4.2BSD-style flock(2) on temporary file (FLOCK)
166 Advantage: exists on a lot of platforms, especially on older Unix
167 derivatives. Disadvantage: needs a temporary file on the filesystem
168 and has to re-open file-descriptors to it in each(!) fork(2)'ed
169 child process.
170 SysV IPC semget(2) (IPCSEM)
172 Advantage: exists on a lot of platforms and does not need a tempo‐
173 rary file. Disadvantage: an unmeant termination of the application
174 leads to a semaphore leak because the facility does not allow a
175 ``remove in advance'' trick (as the IPC shared memory facility
176 does) for safe cleanups.
177 SVR4-style fcntl(2) on temporary file (FCNTL)
179 Advantage: exists on a lot of platforms and is also the most power‐
180 ful variant (although not always the fastest one). Disadvantage:
181 needs a temporary file.
182 MEMORY ALLOCATION STRATEGY
184 The memory allocation strategy the Standard Malloc-Style API functions
186 use internally is the following:
187 Allocation
189 If a chunk of memory has to be allocated, the internal list of free
190 chunks is searched for a minimal-size chunk which is larger or
191 equal than the size of the to be allocated chunk (a best fit strat‐
192 egy).
193 If a chunk is found which matches this best-fit criteria, but is
195 still a lot larger than the requested size, it is split into two
196 chunks: One with exactly the requested size (which is the resulting
197 chunk given back) and one with the remaining size (which is immedi‐
198 ately re-inserted into the list of free chunks).
199 If no fitting chunk is found at all in the list of free chunks, a
201 new one is created from the spare area of the shared memory segment
202 until the segment is full (in which case an out of memory error
203 occurs).
204 Deallocation
206 If a chunk of memory has to be deallocated, it is inserted in
207 sorted manner into the internal list of free chunks. The insertion
208 operation automatically merges the chunk with a previous and/or a
209 next free chunk if possible, i.e. if the free chunks stay physi‐
210 cally seamless (one after another) in memory, to automatically form
211 larger free chunks out of smaller ones.
212 This way the shared memory segment is automatically defragmented
214 when memory is deallocated.
215 This strategy reduces memory waste and fragmentation caused by small
217 and frequent allocations and deallocations to a minimum.
218 The internal implementation of the list of free chunks is not specially
220 optimized (for instance by using binary search trees or even splay
221 trees, etc), because it is assumed that the total amount of entries in
222 the list of free chunks is always small (caused both by the fact that
223 shared memory segments are usually a lot smaller than heaps and the
224 fact that we always defragment by merging the free chunks if possible).
225
227 In the following, all API functions are described in detail. The order
228 directly follows the one in the SYNOPSIS section above.
229 Global Malloc-Replacement API
231 int MM_create(size_t size, const char *file);
233 This initializes the global shared memory pool with size and file
234 and has to be called before any fork(2) operations are performed by
235 the application.
236 int MM_permission(mode_t mode, uid_t owner, gid_t group);
238 This sets the filesystem mode, owner and group for the global
239 shared memory pool (has effects only if the underlying shared mem‐
240 ory segment implementation is actually based on external auxiliary
241 files). The arguments are directly passed through to chmod(2) and
242 chown(2).
243 void MM_reset(void);
245 This resets the global shared memory pool: all chunks that have
246 been allocated in the pool are marked as free and are eligible for
247 reuse. The global memory pool itself is not destroyed.
248 void MM_destroy(void);
250 This destroys the global shared memory pool and should be called
251 after all child processes were killed.
252 int MM_lock(mm_lock_mode mode);
254 This locks the global shared memory pool for the current process in
255 order to perform either shared/read-only (mode is "MM_LOCK_RD") or
256 exclusive/read-write (mode is "MM_LOCK_RW") critical operations
257 inside the global shared memory pool.
258 int MM_unlock(void);
260 This unlocks the global shared memory pool for the current process
261 after the critical operations were performed inside the global
262 shared memory pool.
263 void *MM_malloc(size_t size);
265 Identical to the POSIX.1 malloc(3) function but instead of allocat‐
266 ing memory from the heap it allocates it from the global shared
267 memory pool.
268 void MM_free(void *ptr);
270 Identical to the POSIX.1 free(3) function but instead of deallocat‐
271 ing memory in the heap it deallocates it in the global shared mem‐
272 ory pool.
273 void *MM_realloc(void *ptr, size_t size);
275 Identical to the POSIX.1 realloc(3) function but instead of reallo‐
276 cating memory in the heap it reallocates it inside the global
277 shared memory pool.
278 void *MM_calloc(size_t number, size_t size);
280 Identical to the POSIX.1 calloc(3) function but instead of allocat‐
281 ing and initializing memory from the heap it allocates and initial‐
282 izes it from the global shared memory pool.
283 char *MM_strdup(const char *str);
285 Identical to the POSIX.1 strdup(3) function but instead of creating
286 the string copy in the heap it creates it in the global shared mem‐
287 ory pool.
288 size_t MM_sizeof(const void *ptr);
290 This function returns the size in bytes of the chunk starting at
291 ptr when ptr was previously allocated with MM_malloc(3). The result
292 is undefined if ptr was not previously allocated with MM_malloc(3).
293 size_t MM_maxsize(void);
295 This function returns the maximum size which is allowed as the
296 first argument to the MM_create(3) function.
297 size_t MM_available(void);
299 Returns the amount in bytes of still available (free) memory in the
300 global shared memory pool.
301 char *MM_error(void);
303 Returns the last error message which occurred inside the OSSP mm
304 library.
305 Standard Malloc-Style API
307 MM *mm_create(size_t size, const char *file);
309 This creates a shared memory pool which has space for approximately
310 a total of size bytes with the help of file. Here file is a
311 filesystem path to a file which need not to exist (and perhaps is
312 never created because this depends on the platform and chosen
313 shared memory and mutex implementation). The return value is a
314 pointer to a "MM" structure which should be treated as opaque by
315 the application. It describes the internals of the created shared
316 memory pool. In case of an error "NULL" is returned. A size of 0
317 means to allocate the maximum allowed size which is platform depen‐
318 dent and is between a few KB and the soft limit of 64MB.
319 int mm_permission(MM *mm, mode_t mode, uid_t owner, gid_t group);
321 This sets the filesystem mode, owner and group for the shared mem‐
322 ory pool mm (has effects only when the underlying shared memory
323 segment implementation is actually based on external auxiliary
324 files). The arguments are directly passed through to chmod(2) and
325 chown(2).
326 void mm_reset(MM *mm);
328 This resets the shared memory pool mm: all chunks that have been
329 allocated in the pool are marked as free and are eligible for re‐
330 use. The memory pool itself is not destroyed.
331 void mm_destroy(MM *mm);
333 This destroys the complete shared memory pool mm and with it all
334 chunks which were allocated in this pool. Additionally any created
335 files on the filesystem corresponding to the shared memory pool are
336 unlinked.
337 int mm_lock(MM *mm, mm_lock_mode mode);
339 This locks the shared memory pool mm for the current process in
340 order to perform either shared/read-only (mode is "MM_LOCK_RD") or
341 exclusive/read-write (mode is "MM_LOCK_RW") critical operations
342 inside the global shared memory pool.
343 int mm_unlock(MM *mm);
345 This unlocks the shared memory pool mm for the current process
346 after critical operations were performed inside the global shared
347 memory pool.
348 void *mm_malloc(MM *mm, size_t size);
350 This function allocates size bytes from the shared memory pool mm
351 and returns either a (virtual memory word aligned) pointer to it or
352 "NULL" in case of an error (out of memory). It behaves like the
353 POSIX.1 malloc(3) function but instead of allocating memory from
354 the heap it allocates it from the shared memory segment underlying
355 mm.
356 void mm_free(MM *mm, void *ptr);
358 This deallocates the chunk starting at ptr in the shared memory
359 pool mm. It behaves like the POSIX.1 free(3) function but instead
360 of deallocating memory from the heap it deallocates it from the
361 shared memory segment underlying mm.
362 void *mm_realloc(MM *mm, void *ptr, size_t size);
364 This function reallocates the chunk starting at ptr inside the
365 shared memory pool mm with the new size of size bytes. It behaves
366 like the POSIX.1 realloc(3) function but instead of reallocating
367 memory in the heap it reallocates it in the shared memory segment
368 underlying mm.
369 void *mm_calloc(MM *mm, size_t number, size_t size);
371 This is similar to mm_malloc(3), but additionally clears the chunk.
372 It behaves like the POSIX.1 calloc(3) function. It allocates space
373 for number objects, each size bytes in length from the shared mem‐
374 ory pool mm. The result is identical to calling mm_malloc(3) with
375 an argument of ``number * size'', with the exception that the allo‐
376 cated memory is initialized to nul bytes.
377 char *mm_strdup(MM *mm, const char *str);
379 This function behaves like the POSIX.1 strdup(3) function. It
380 allocates sufficient memory inside the shared memory pool mm for a
381 copy of the string str, does the copy, and returns a pointer to it.
382 The pointer may subsequently be used as an argument to the function
383 mm_free(3). If insufficient shared memory is available, "NULL" is
384 returned.
385 size_t mm_sizeof(MM *mm, const void *ptr);
387 This function returns the size in bytes of the chunk starting at
388 ptr when ptr was previously allocated with mm_malloc(3) inside the
389 shared memory pool mm. The result is undefined when ptr was not
390 previously allocated with mm_malloc(3).
391 size_t mm_maxsize(void);
393 This function returns the maximum size which is allowed as the
394 first argument to the mm_create(3) function.
395 size_t mm_available(MM *mm);
397 Returns the amount in bytes of still available (free) memory in the
398 shared memory pool mm.
399 char *mm_error(void);
401 Returns the last error message which occurred inside the OSSP mm
402 library.
403 void mm_display_info(MM *mm);
405 This is debugging function which displays a summary page for the
406 shared memory pool mm describing various internal sizes and coun‐
407 ters.
408 Low-Level Shared Memory API
410 void *mm_core_create(size_t size, const char *file);
412 This creates a shared memory area which is at least size bytes in
413 size with the help of file. The value size has to be greater than 0
414 and less or equal the value returned by mm_core_maxsegsize(3). Here
415 file is a filesystem path to a file which need not to exist (and
416 perhaps is never created because this depends on the platform and
417 chosen shared memory and mutex implementation). The return value
418 is either a (virtual memory word aligned) pointer to the shared
419 memory segment or "NULL" in case of an error. The application is
420 guaranteed to be able to access the shared memory segment from byte
421 0 to byte size-1 starting at the returned address.
422 int mm_core_permission(void *core, mode_t mode, uid_t owner, gid_t
424 group);
425 This sets the filesystem mode, owner and group for the shared mem‐
426 ory segment code (has effects only when the underlying shared mem‐
427 ory segment implementation is actually based on external auxiliary
428 files). The arguments are directly passed through to chmod(2) and
429 chown(2).
430 void mm_core_delete(void *core);
432 This deletes a shared memory segment core (as previously returned
433 by a mm_core_create(3) call). After this operation, accessing the
434 segment starting at core is no longer allowed and will usually lead
435 to a segmentation fault.
436 int mm_core_lock(const void *core, mm_lock_mode mode);
438 This function acquires an advisory lock for the current process on
439 the shared memory segment core for either shared/read-only (mode is
440 "MM_LOCK_RD") or exclusive/read-write (mode is "MM_LOCK_RW") criti‐
441 cal operations between fork(2)'ed child processes.
442 int mm_core_unlock(const void *core);
444 This function releases a previously acquired advisory lock for the
445 current process on the shared memory segment core.
446 size_t mm_core_size(const void *core);
448 This returns the size in bytes of core. This size is exactly the
449 size which was used for creating the shared memory area via
450 mm_core_create(3). The function is provided just for convenience
451 reasons to not require the application to remember the memory size
452 behind core itself.
453 size_t mm_core_maxsegsize(void);
455 This returns the number of bytes of a maximum-size shared memory
456 segment which is allowed to allocate via the MM library. It is
457 between a few KB and the soft limit of 64MB.
458 size_t mm_core_align2page(size_t size);
460 This is just a utility function which can be used to align the num‐
461 ber size to the next virtual memory page boundary used by the
462 underlying platform. The memory page boundary under Unix platforms
463 is usually somewhere between 2048 and 16384 bytes. You do not have
464 to align the size arguments of other OSSP mm library functions
465 yourself, because this is already done internally. This function
466 is exported by the OSSP mm library just for convenience reasons in
467 case an application wants to perform similar calculations for other
468 purposes.
469 size_t mm_core_align2word(size_t size);
471 This is another utility function which can be used to align the
472 number size to the next virtual memory word boundary used by the
473 underlying platform. The memory word boundary under Unix platforms
474 is usually somewhere between 4 and 16 bytes. You do not have to
475 align the size arguments of other OSSP mm library functions your‐
476 self, because this is already done internally. This function is
477 exported by the OSSP mm library just for convenience reasons in
478 case an application wants to perform similar calculations for other
479 purposes.
480 Low-Level Shared Memory API
482 void mm_lib_error_set(unsigned int, const char *str);
484 This is a function which is used internally by the various MM func‐
485 tion to set an error string. It's usually not called directly from
486 applications.
487 char *mm_lib_error_get(void);
489 This is a function which is used internally by MM_error(3) and
490 mm_error(3) functions to get the current error string. It is usu‐
491 ally not called directly from applications.
492 int mm_lib_version(void);
494 This function returns a hex-value ``0xVRRTLL'' which describes the
495 current OSSP mm library version. V is the version, RR the revi‐
496 sions, LL the level and T the type of the level (alphalevel=0,
497 betalevel=1, patchlevel=2, etc). For instance OSSP mm version 1.0.4
498 is encoded as 0x100204. The reason for this unusual mapping is
499 that this way the version number is steadily increasing.
500
502 The maximum size of a continuous shared memory segment one can allocate
503 depends on the underlying platform. This cannot be changed, of course.
504 But currently the high-level malloc(3)-style API just uses a single
505 shared memory segment as the underlying data structure for an "MM"
506 object which means that the maximum amount of memory an "MM" object
507 represents also depends on the platform.
508 This could be changed in later versions by allowing at least the high-
510 level malloc(3)-style API to internally use multiple shared memory seg‐
511 ments to form the "MM" object. This way "MM" objects could have arbi‐
512 trary sizes, although the maximum size of an allocatable continuous
513 chunk still is bounded by the maximum size of a shared memory segment.
514
516 mm-config(1).
517 malloc(3), calloc(3), realloc(3), strdup(3), free(3), mmap(2),
519 shmget(2), shmctl(2), flock(2), fcntl(2), semget(2), semctl(2),
520 semop(2).
521
523 http://www.ossp.org/pkg/lib/mm/
524
526 This library was originally written in January 1999 by Ralf S.
527 Engelschall <rse@engelschall.com> for use in the Extended API (EAPI) of
528 the Apache HTTP server project (see http://www.apache.org/), which was
529 originally invented for mod_ssl (see http://www.modssl.org/).
530 Its base idea (a malloc-style API for handling shared memory) was orig‐
532 inally derived from the non-publically available mm_malloc library
533 written in October 1997 by Charles Randall <crandall@matchlogic.com>
534 for MatchLogic, Inc.
535 In 2000 this library joined the OSSP project where all other software
537 development projects of Ralf S. Engelschall are located.
538
540 Ralf S. Engelschall
541 rse@engelschall.com
542 www.engelschall.com
54315-Aug-2006 MM 1.4.2 mm(3)