1erts_alloc(3) C Library Functions erts_alloc(3)
2
6 erts_alloc - An Erlang runtime system internal memory allocator
7 library.
8
11 erts_alloc is an Erlang runtime system internal memory allocator
12 library. erts_alloc provides the Erlang runtime system with a number of
13 memory allocators.
14
16 The following allocators are present:
17 temp_alloc:
19 Allocator used for temporary allocations.
20 eheap_alloc:
22 Allocator used for Erlang heap data, such as Erlang process heaps.
23 binary_alloc:
25 Allocator used for Erlang binary data.
26 ets_alloc:
28 Allocator used for ets data.
29 driver_alloc:
31 Allocator used for driver data.
32 literal_alloc:
34 Allocator used for constant terms in Erlang code.
35 sl_alloc:
37 Allocator used for memory blocks that are expected to be short-
38 lived.
39 ll_alloc:
41 Allocator used for memory blocks that are expected to be long-
42 lived, for example, Erlang code.
43 fix_alloc:
45 A fast allocator used for some frequently used fixed size data
46 types.
47 exec_alloc:
49 Allocator used by the HiPE application for native executable code
50 on specific architectures (x86_64).
51 std_alloc:
53 Allocator used for most memory blocks not allocated through any of
54 the other allocators described above.
55 sys_alloc:
57 This is normally the default malloc implementation used on the spe‐
58 cific OS.
59 mseg_alloc:
61 A memory segment allocator. It is used by other allocators for
62 allocating memory segments and is only available on systems that
63 have the mmap system call. Memory segments that are deallocated are
64 kept for a while in a segment cache before they are destroyed. When
65 segments are allocated, cached segments are used if possible
66 instead of creating new segments. This to reduce the number of sys‐
67 tem calls made.
68 sys_alloc and literal_alloc are always enabled and cannot be disabled.
70 exec_alloc is only available if it is needed and cannot be disabled.
71 mseg_alloc is always enabled if it is available and an allocator that
72 uses it is enabled. All other allocators can be enabled or disabled. By
73 default all allocators are enabled. When an allocator is disabled,
74 sys_alloc is used instead of the disabled allocator.
75 The main idea with the erts_alloc library is to separate memory blocks
77 that are used differently into different memory areas, to achieve less
78 memory fragmentation. By putting less effort in finding a good fit for
79 memory blocks that are frequently allocated than for those less fre‐
80 quently allocated, a performance gain can be achieved.
81
83 Internally a framework called alloc_util is used for implementing allo‐
84 cators. sys_alloc and mseg_alloc do not use this framework, so the fol‐
85 lowing does not apply to them.
86 An allocator manages multiple areas, called carriers, in which memory
88 blocks are placed. A carrier is either placed in a separate memory seg‐
89 ment (allocated through mseg_alloc), or in the heap segment (allocated
90 through sys_alloc).
91 * Multiblock carriers are used for storage of several blocks.
93 * Singleblock carriers are used for storage of one block.
95 * Blocks that are larger than the value of the singleblock carrier
97 threshold (sbct) parameter are placed in singleblock carriers.
98 * Blocks that are smaller than the value of parameter sbct are placed
100 in multiblock carriers.
101 Normally an allocator creates a "main multiblock carrier". Main multi‐
103 block carriers are never deallocated. The size of the main multiblock
104 carrier is determined by the value of parameter mmbcs.
105 Sizes of multiblock carriers allocated through mseg_alloc are decided
107 based on the following parameters:
108 * The values of the largest multiblock carrier size (lmbcs)
110 * The smallest multiblock carrier size (smbcs)
112 * The multiblock carrier growth stages (mbcgs)
114 If nc is the current number of multiblock carriers (the main multiblock
116 carrier excluded) managed by an allocator, the size of the next
117 mseg_alloc multiblock carrier allocated by this allocator is roughly
118 smbcs+nc*(lmbcs-smbcs)/mbcgs when nc <= mbcgs, and lmbcs when nc >
119 mbcgs. If the value of parameter sbct is larger than the value of
120 parameter lmbcs, the allocator may have to create multiblock carriers
121 that are larger than the value of parameter lmbcs, though. Singleblock
122 carriers allocated through mseg_alloc are sized to whole pages.
123 Sizes of carriers allocated through sys_alloc are decided based on the
125 value of the sys_alloc carrier size (ycs) parameter. The size of a car‐
126 rier is the least number of multiples of the value of parameter ycs
127 satisfying the request.
128 Coalescing of free blocks are always performed immediately. Boundary
130 tags (headers and footers) in free blocks are used, which makes the
131 time complexity for coalescing constant.
132 The memory allocation strategy used for multiblock carriers by an allo‐
134 cator can be configured using parameter as. The following strategies
135 are available:
136 Best fit:
138 Strategy: Find the smallest block satisfying the requested block
139 size.
140 Implementation: A balanced binary search tree is used. The time
142 complexity is proportional to log N, where N is the number of sizes
143 of free blocks.
144 Address order best fit:
146 Strategy: Find the smallest block satisfying the requested block
147 size. If multiple blocks are found, choose the one with the lowest
148 address.
149 Implementation: A balanced binary search tree is used. The time
151 complexity is proportional to log N, where N is the number of free
152 blocks.
153 Address order first fit:
155 Strategy: Find the block with the lowest address satisfying the
156 requested block size.
157 Implementation: A balanced binary search tree is used. The time
159 complexity is proportional to log N, where N is the number of free
160 blocks.
161 Address order first fit carrier best fit:
163 Strategy: Find the carrier with the lowest address that can satisfy
164 the requested block size, then find a block within that carrier
165 using the "best fit" strategy.
166 Implementation: Balanced binary search trees are used. The time
168 complexity is proportional to log N, where N is the number of free
169 blocks.
170 Address order first fit carrier address order best fit:
172 Strategy: Find the carrier with the lowest address that can satisfy
173 the requested block size, then find a block within that carrier
174 using the "address order best fit" strategy.
175 Implementation: Balanced binary search trees are used. The time
177 complexity is proportional to log N, where N is the number of free
178 blocks.
179 Age order first fit carrier address order first fit:
181 Strategy: Find the oldest carrier that can satisfy the requested
182 block size, then find a block within that carrier using the
183 "address order first fit" strategy.
184 Implementation: A balanced binary search tree is used. The time
186 complexity is proportional to log N, where N is the number of free
187 blocks.
188 Age order first fit carrier best fit:
190 Strategy: Find the oldest carrier that can satisfy the requested
191 block size, then find a block within that carrier using the "best
192 fit" strategy.
193 Implementation: Balanced binary search trees are used. The time
195 complexity is proportional to log N, where N is the number of free
196 blocks.
197 Age order first fit carrier address order best fit:
199 Strategy: Find the oldest carrier that can satisfy the requested
200 block size, then find a block within that carrier using the
201 "address order best fit" strategy.
202 Implementation: Balanced binary search trees are used. The time
204 complexity is proportional to log N, where N is the number of free
205 blocks.
206 Good fit:
208 Strategy: Try to find the best fit, but settle for the best fit
209 found during a limited search.
210 Implementation: The implementation uses segregated free lists with
212 a maximum block search depth (in each list) to find a good fit
213 fast. When the maximum block search depth is small (by default 3),
214 this implementation has a time complexity that is constant. The
215 maximum block search depth can be configured using parameter mbsd.
216 A fit:
218 Strategy: Do not search for a fit, inspect only one free block to
219 see if it satisfies the request. This strategy is only intended to
220 be used for temporary allocations.
221 Implementation: Inspect the first block in a free-list. If it sat‐
223 isfies the request, it is used, otherwise a new carrier is created.
224 The implementation has a time complexity that is constant.
225 As from ERTS 5.6.1 the emulator refuses to use this strategy on
227 other allocators than temp_alloc. This because it only causes prob‐
228 lems for other allocators.
229 Apart from the ordinary allocators described above, some pre-allocators
231 are used for some specific data types. These pre-allocators pre-allo‐
232 cate a fixed amount of memory for certain data types when the runtime
233 system starts. As long as pre-allocated memory is available, it is
234 used. When no pre-allocated memory is available, memory is allocated in
235 ordinary allocators. These pre-allocators are typically much faster
236 than the ordinary allocators, but can only satisfy a limited number of
237 requests.
238
240 Warning:
241 Only use these flags if you are sure what you are doing. Unsuitable
242 settings can cause serious performance degradation and even a system
243 crash at any time during operation.
244 Memory allocator system flags have the following syntax: +M<S><P> <V>,
247 where <S> is a letter identifying a subsystem, <P> is a parameter, and
248 <V> is the value to use. The flags can be passed to the Erlang emulator
249 (erl(1)) as command-line arguments.
250 System flags effecting specific allocators have an uppercase letter as
252 <S>. The following letters are used for the allocators:
253 * B: binary_alloc
255 * D: std_alloc
257 * E: ets_alloc
259 * F: fix_alloc
261 * H: eheap_alloc
263 * I: literal_alloc
265 * L: ll_alloc
267 * M: mseg_alloc
269 * R: driver_alloc
271 * S: sl_alloc
273 * T: temp_alloc
275 * X: exec_alloc
277 * Y: sys_alloc
279 Flags for Configuration of mseg_alloc
281 +MMamcbf <size>:
282 Absolute maximum cache bad fit (in kilobytes). A segment in the
283 memory segment cache is not reused if its size exceeds the
284 requested size with more than the value of this parameter. Defaults
285 to 4096.
286 +MMrmcbf <ratio>:
288 Relative maximum cache bad fit (in percent). A segment in the mem‐
289 ory segment cache is not reused if its size exceeds the requested
290 size with more than relative maximum cache bad fit percent of the
291 requested size. Defaults to 20.
292 +MMsco true|false:
294 Sets super carrier only flag. Defaults to true. When a super car‐
295 rier is used and this flag is true, mseg_alloc only creates carri‐
296 ers in the super carrier. Notice that the alloc_util framework can
297 create sys_alloc carriers, so if you want all carriers to be cre‐
298 ated in the super carrier, you therefore want to disable use of
299 sys_alloc carriers by also passing +Musac false. When the flag is
300 false, mseg_alloc tries to create carriers outside of the super
301 carrier when the super carrier is full.
302 Note:
304 Setting this flag to false is not supported on all systems. The flag
305 is then ignored.
306 +MMscrfsd <amount>:
309 Sets super carrier reserved free segment descriptors. Defaults to
310 65536. This parameter determines the amount of memory to reserve
311 for free segment descriptors used by the super carrier. If the sys‐
312 tem runs out of reserved memory for free segment descriptors, other
313 memory is used. This can however cause fragmentation issues, so you
314 want to ensure that this never happens. The maximum amount of free
315 segment descriptors used can be retrieved from the erts_mmap tuple
316 part of the result from calling erlang:system_info({allocator,
317 mseg_alloc}).
318 +MMscrpm true|false:
320 Sets super carrier reserve physical memory flag. Defaults to true.
321 When this flag is true, physical memory is reserved for the whole
322 super carrier at once when it is created. The reservation is after
323 that left unchanged. When this flag is set to false, only virtual
324 address space is reserved for the super carrier upon creation. The
325 system attempts to reserve physical memory upon carrier creations
326 in the super carrier, and attempt to unreserve physical memory upon
327 carrier destructions in the super carrier.
328 Note:
330 What reservation of physical memory means, highly depends on the
331 operating system, and how it is configured. For example, different
332 memory overcommit settings on Linux drastically change the behavior.
333 Setting this flag to false is possibly not supported on all systems.
335 The flag is then ignored.
336 +MMscs <size in MB>:
339 Sets super carrier size (in MB). Defaults to 0, that is, the super
340 carrier is by default disabled. The super carrier is a large con‐
341 tinuous area in the virtual address space. mseg_alloc always tries
342 to create new carriers in the super carrier if it exists. Notice
343 that the alloc_util framework can create sys_alloc carriers. For
344 more information, see +MMsco.
345 +MMmcs <amount>:
347 Maximum cached segments. The maximum number of memory segments
348 stored in the memory segment cache. Valid range is [0, 30].
349 Defaults to 10.
350 Flags for Configuration of sys_alloc
352 +MYe true:
353 Enables sys_alloc.
354 Note:
356 sys_alloc cannot be disabled.
357 +MYm libc:
360 malloc library to use. Only libc is available. libc enables the
361 standard libc malloc implementation. By default libc is used.
362 +MYtt <size>:
364 Trim threshold size (in kilobytes). This is the maximum amount of
365 free memory at the top of the heap (allocated by sbrk) that is kept
366 by malloc (not released to the operating system). When the amount
367 of free memory at the top of the heap exceeds the trim threshold,
368 malloc releases it (by calling sbrk). Trim threshold is specified
369 in kilobytes. Defaults to 128.
370 Note:
372 This flag has effect only when the emulator is linked with the GNU C
373 library, and uses its malloc implementation.
374 +MYtp <size>:
377 Top pad size (in kilobytes). This is the amount of extra memory
378 that is allocated by malloc when sbrk is called to get more memory
379 from the operating system. Defaults to 0.
380 Note:
382 This flag has effect only when the emulator is linked with the GNU C
383 library, and uses its malloc implementation.
384 Flags for Configuration of Allocators Based on alloc_util
387 If u is used as subsystem identifier (that is, <S> = u), all allocators
388 based on alloc_util are effected. If B, D, E, F, H, L, R, S, or T is
389 used as subsystem identifier, only the specific allocator identifier is
390 effected.
391 +M<S>acul <utilization>|de:
393 Abandon carrier utilization limit. A valid <utilization> is an
394 integer in the range [0, 100] representing utilization in percent.
395 When a utilization value > 0 is used, allocator instances are
396 allowed to abandon multiblock carriers. If de (default enabled) is
397 passed instead of a <utilization>, a recommended non-zero utiliza‐
398 tion value is used. The value chosen depends on the allocator type
399 and can be changed between ERTS versions. Defaults to de, but this
400 can be changed in the future.
401 Carriers are abandoned when memory utilization in the allocator
403 instance falls below the utilization value used. Once a carrier is
404 abandoned, no new allocations are made in it. When an allocator
405 instance gets an increased multiblock carrier need, it first tries
406 to fetch an abandoned carrier from an allocator instance of the
407 same allocator type. If no abandoned carrier can be fetched, it
408 creates a new empty carrier. When an abandoned carrier has been
409 fetched, it will function as an ordinary carrier. This feature has
410 special requirements on the allocation strategy used. Only the
411 strategies aoff, aoffcbf, aoffcaobf, ageffcaoffm, ageffcbf and
412 ageffcaobf support abandoned carriers.
413 This feature also requires multiple thread specific instances to be
415 enabled. When enabling this feature, multiple thread-specific
416 instances are enabled if not already enabled, and the aoffcbf
417 strategy is enabled if the current strategy does not support aban‐
418 doned carriers. This feature can be enabled on all allocators based
419 on the alloc_util framework, except temp_alloc (which would be
420 pointless).
421 +M<S>acfml <bytes>:
423 Abandon carrier free block min limit. A valid <bytes> is a positive
424 integer representing a block size limit. The largest free block in
425 a carrier must be at least bytes large, for the carrier to be aban‐
426 doned. The default is zero but can be changed in the future.
427 See also acul.
429 +M<S>acnl <amount>:
431 Abandon carrier number limit. A valid <amount> is a positive inte‐
432 ger representing max number of abandoned carriers per allocator
433 instance. Defaults to 1000 which will practically disable the
434 limit, but this can be changed in the future.
435 See also acul.
437 +M<S>as bf|aobf|aoff|aoffcbf|aoffcaobf|ageffcaoff|ageffcbf|ageff‐
439 caobf|gf|af:
440 Allocation strategy. The following strategies are valid:
441 * bf (best fit)
443 * aobf (address order best fit)
445 * aoff (address order first fit)
447 * aoffcbf (address order first fit carrier best fit)
449 * aoffcaobf (address order first fit carrier address order best
451 fit)
452 * ageffcaoff (age order first fit carrier address order first fit)
454 * ageffcbf (age order first fit carrier best fit)
456 * ageffcaobf (age order first fit carrier address order best fit)
458 * gf (good fit)
460 * af (a fit)
462 See the description of allocation strategies in section The
464 alloc_util Framework.
465 +M<S>asbcst <size>:
467 Absolute singleblock carrier shrink threshold (in kilobytes). When
468 a block located in an mseg_alloc singleblock carrier is shrunk, the
469 carrier is left unchanged if the amount of unused memory is less
470 than this threshold, otherwise the carrier is shrunk. See also
471 rsbcst.
472 +M<S>e true|false:
474 Enables allocator <S>.
475 +M<S>lmbcs <size>:
477 Largest (mseg_alloc) multiblock carrier size (in kilobytes). See
478 the description on how sizes for mseg_alloc multiblock carriers are
479 decided in section The alloc_util Framework. On 32-bit Unix style
480 OS this limit cannot be set > 128 MB.
481 +M<S>mbcgs <ratio>:
483 (mseg_alloc) multiblock carrier growth stages. See the description
484 on how sizes for mseg_alloc multiblock carriers are decided in sec‐
485 tion The alloc_util Framework.
486 +M<S>mbsd <depth>:
488 Maximum block search depth. This flag has effect only if the good
489 fit strategy is selected for allocator <S>. When the good fit
490 strategy is used, free blocks are placed in segregated free-lists.
491 Each free-list contains blocks of sizes in a specific range. The
492 maxiumum block search depth sets a limit on the maximum number of
493 blocks to inspect in a free-list during a search for suitable block
494 satisfying the request.
495 +M<S>mmbcs <size>:
497 Main multiblock carrier size. Sets the size of the main multiblock
498 carrier for allocator <S>. The main multiblock carrier is allocated
499 through sys_alloc and is never deallocated.
500 +M<S>mmmbc <amount>:
502 Maximum mseg_alloc multiblock carriers. Maximum number of multi‐
503 block carriers allocated through mseg_alloc by allocator <S>. When
504 this limit is reached, new multiblock carriers are allocated
505 through sys_alloc.
506 +M<S>mmsbc <amount>:
508 Maximum mseg_alloc singleblock carriers. Maximum number of single‐
509 block carriers allocated through mseg_alloc by allocator <S>. When
510 this limit is reached, new singleblock carriers are allocated
511 through sys_alloc.
512 +M<S>ramv <bool>:
514 Realloc always moves. When enabled, reallocate operations are more
515 or less translated into an allocate, copy, free sequence. This
516 often reduces memory fragmentation, but costs performance.
517 +M<S>rmbcmt <ratio>:
519 Relative multiblock carrier move threshold (in percent). When a
520 block located in a multiblock carrier is shrunk, the block is moved
521 if the ratio of the size of the returned memory compared to the
522 previous size is more than this threshold, otherwise the block is
523 shrunk at the current location.
524 +M<S>rsbcmt <ratio>:
526 Relative singleblock carrier move threshold (in percent). When a
527 block located in a singleblock carrier is shrunk to a size smaller
528 than the value of parameter sbct, the block is left unchanged in
529 the singleblock carrier if the ratio of unused memory is less than
530 this threshold, otherwise it is moved into a multiblock carrier.
531 +M<S>rsbcst <ratio>:
533 Relative singleblock carrier shrink threshold (in percent). When a
534 block located in an mseg_alloc singleblock carrier is shrunk, the
535 carrier is left unchanged if the ratio of unused memory is less
536 than this threshold, otherwise the carrier is shrunk. See also
537 asbcst.
538 +M<S>sbct <size>:
540 Singleblock carrier threshold (in kilobytes). Blocks larger than
541 this threshold are placed in singleblock carriers. Blocks smaller
542 than this threshold are placed in multiblock carriers. On 32-bit
543 Unix style OS this threshold cannot be set > 8 MB.
544 +M<S>smbcs <size>:
546 Smallest (mseg_alloc) multiblock carrier size (in kilobytes). See
547 the description on how sizes for mseg_alloc multiblock carriers are
548 decided in section The alloc_util Framework.
549 +M<S>t true|false:
551 Multiple, thread-specific instances of the allocator. This option
552 has only effect on the runtime system with SMP support. Default
553 behavior on the runtime system with SMP support is NoSchedulers+1
554 instances. Each scheduler uses a lock-free instance of its own and
555 other threads use a common instance.
556 Before ERTS 5.9 it was possible to configure a smaller number of
558 thread-specific instances than schedulers. This is, however, not
559 possible anymore.
560 Flags for Configuration of alloc_util
562 All allocators based on alloc_util are effected.
563 +Muycs <size>:
565 sys_alloc carrier size. Carriers allocated through sys_alloc are
566 allocated in sizes that are multiples of the sys_alloc carrier
567 size. This is not true for main multiblock carriers and carriers
568 allocated during a memory shortage, though.
569 +Mummc <amount>:
571 Maximum mseg_alloc carriers. Maximum number of carriers placed in
572 separate memory segments. When this limit is reached, new carriers
573 are placed in memory retrieved from sys_alloc.
574 +Musac <bool>:
576 Allow sys_alloc carriers. Defaults to true. If set to false,
577 sys_alloc carriers are never created by allocators using the
578 alloc_util framework.
579 Special Flag for literal_alloc
581 +MIscs <size in MB>:
582 literal_alloc super carrier size (in MB). The amount of virtual
583 address space reserved for literal terms in Erlang code on 64-bit
584 architectures. Defaults to 1024 (that is, 1 GB), which is usually
585 sufficient. The flag is ignored on 32-bit architectures.
586 Special Flag for exec_alloc
588 +MXscs <size in MB>:
589 exec_alloc super carrier size (in MB). The amount of virtual
590 address space reserved for native executable code used by the HiPE
591 application on specific architectures (x86_64). Defaults to 512.
592 Instrumentation Flags
594 +Mim true|false:
595 A map over current allocations is kept by the emulator. The alloca‐
596 tion map can be retrieved through module instrument(3). +Mim true
597 implies +Mis true. +Mim true is the same as flag -instr in erl(1).
598 +Mis true|false:
600 Status over allocated memory is kept by the emulator. The alloca‐
601 tion status can be retrieved through module instrument(3).
602 +Mit X:
604 Reserved for future use. Do not use this flag.
605 Note:
607 When instrumentation of the emulator is enabled, the emulator uses more
608 memory and runs slower.
609 Other Flags
612 +Mea min|max|r9c|r10b|r11b|config:
613 Options:
614 min:
616 Disables all allocators that can be disabled.
617 max:
619 Enables all allocators (default).
620 r9c|r10b|r11b:
622 Configures all allocators as they were configured in respective
623 Erlang/OTP release. These will eventually be removed.
624 config:
626 Disables features that cannot be enabled while creating an allo‐
627 cator configuration with erts_alloc_config(3).
628 Note:
630 This option is to be used only while running erts_alloc_config(3),
631 not when using the created configuration.
632 +Mlpm all|no:
635 Lock physical memory. Defaults to no, that is, no physical memory
636 is locked. If set to all, all memory mappings made by the runtime
637 system are locked into physical memory. If set to all, the runtime
638 system fails to start if this feature is not supported, the user
639 has not got enough privileges, or the user is not allowed to lock
640 enough physical memory. The runtime system also fails with an out
641 of memory condition if the user limit on the amount of locked mem‐
642 ory is reached.
643
645 Only some default values have been presented here. For information
646 about the currently used settings and the current status of the alloca‐
647 tors, see erlang:system_info(allocator) and erlang:system_info({alloca‐
648 tor, Alloc}).
649 Note:
651 Most of these flags are highly implementation-dependent and can be
652 changed or removed without prior notice.
653 erts_alloc is not obliged to strictly use the settings that have been
655 passed to it (it can even ignore them).
656 The erts_alloc_config(3) tool can be used to aid creation of an
659 erts_alloc configuration that is suitable for a limited number of run‐
660 time scenarios.
661
663 erl(1), erlang(3), erts_alloc_config(3), instrument(3)
664Ericsson AB erts 9.3.3.10 erts_alloc(3)