1File::Map(3) User Contributed Perl Documentation File::Map(3)
2
6 File::Map - Memory mapping made simple and safe.
7
9 version 0.67
10
12 use File::Map 'map_file';
13 map_file my $map, $filename, '+<';
15 $map =~ s/bar/quz/g;
16 substr $map, 1024, 11, "Hello world";
17
19 File::Map maps files or anonymous memory into perl variables.
20 Advantages of memory mapping
22 • Unlike normal perl variables, mapped memory is (usually) shared
23 between threads or forked processes.
24 • It is an efficient way to slurp an entire file. Unlike for example
26 File::Slurp, this module returns almost immediately, loading the
27 pages lazily on access. This means you only 'pay' for the parts of
28 the file you actually use.
29 • Perl usually doesn't return memory to the system while running,
31 mapped memory can be returned.
32 Advantages of this module over other similar modules
34 • Safety and Speed
35 This module is safe yet fast. Alternatives are either fast but can
37 cause segfaults or lose the mapping when not used correctly, or are
38 safe but rather slow. File::Map is as fast as a normal string yet
39 safe.
40 • Simplicity
42 It offers a simple interface targeted at common usage patterns
44 • Files are mapped into a variable that can be read just like any
46 other variable, and it can be written to using standard Perl
47 techniques such as regexps and "substr".
48 • Files can be mapped using a set of simple functions. There is
50 no need to know weird constants or the order of 6 arguments.
51 • It will automatically unmap the file when the scalar gets
53 destroyed. This works correctly even in multi-threaded
54 programs.
55 • Portability
57 File::Map supports Unix and Windows.
59 • Thread synchronization
61 It has built-in support for thread synchronization.
63
65 Mapping
66 The following functions for mapping a variable are available for
67 exportation. Note that all of these functions throw exceptions on
68 errors, unless noted otherwise.
69 map_handle $lvalue, $filehandle, $mode = '<', $offset = 0, $length =
71 -s(*handle) - $offset
72 Use a filehandle to map into an lvalue. $filehandle should be a scalar
74 filehandle. $mode uses the same format as "open" does (it currently
75 accepts "<", "+<", ">" and "+>"). $offset and $length are byte
76 positions in the file, and default to mapping the whole file.
77 * map_file $lvalue, $filename, $mode = '<', $offset = 0, $length =
79 -s($filename) - $offset
80 Open a file and map it into an lvalue. Other than $filename, all
82 arguments work as in map_handle.
83 * map_anonymous $lvalue, $length, $type
85 Map an anonymous piece of memory. $type can be either 'shared', in
87 which case it will be shared with child processes, or 'private', which
88 won't be shared.
89 * sys_map $lvalue, $length, $protection, $flags, $filehandle, $offset =
91 0
92 Low level map operation. It accepts the same constants as mmap does
94 (except its first argument obviously). If you don't know how mmap works
95 you probably shouldn't be using this.
96 * unmap $lvalue
98 Unmap a variable. Note that normally this is not necessary as variables
100 are unmapped automatically at destruction, but it is included for
101 completeness.
102 * remap $lvalue, $new_size
104 Try to remap $lvalue to a new size. This call is linux specific and not
106 supported on other systems. For a file backed mapping a file must be
107 long enough to hold the new size, otherwise you can expect bus faults.
108 For an anonymous map it must be private, shared maps can not be
109 remapped. Use with caution.
110 Auxiliary
112 * sync $lvalue, $synchronous = 1
113 Flush changes made to the memory map back to disk. Mappings are always
115 flushed when unmapped, so this is usually not necessary. If
116 $synchronous is true and your operating system supports it, the
117 flushing will be done synchronously.
118 * pin $lvalue
120 Disable paging for this map, thus locking it in physical memory.
122 Depending on your operating system there may be limits on pinning.
123 * unpin $lvalue
125 Unlock the map from physical memory.
127 * advise $lvalue, $advice
129 Advise a certain memory usage pattern. This is not implemented on all
131 operating systems, and may be a no-op. The following values for $advice
132 are always accepted:.
133 • normal
135 Specifies that the application has no advice to give on its behavior
137 with respect to the mapped variable. It is the default characteristic
138 if no advice is given.
139 • random
141 Specifies that the application expects to access the mapped variable
143 in a random order.
144 • sequential
146 Specifies that the application expects to access the mapped variable
148 sequentially from start to end.
149 • willneed
151 Specifies that the application expects to access the mapped variable
153 in the near future.
154 • dontneed
156 Specifies that the application expects that it will not access the
158 mapped variable in the near future.
159 On some systems there may be more values available, but this can not be
161 relied on. Unknown values for $advice will cause a warning but are
162 further ignored.
163 * protect $lvalue, $mode
165 Change the memory protection of the mapping. $mode takes the same
167 format as "open", but also accepts sys_map style constants.
168 Locking
170 These locking functions provide locking for threads for the mapped
171 region. The mapped region has an internal lock and condition variable.
172 The condition variable functions("wait_until", "notify", "broadcast")
173 can only be used inside a locked block. If your perl has been compiled
174 without thread support the condition functions will not be available.
175 * lock_map $lvalue
177 Lock $lvalue until the end of the scope. If your perl does not support
179 threads, this will be a no-op.
180 * wait_until { block } $lvalue
182 Wait for block to become true. After every failed attempt, wait for a
184 signal. It returns the value returned by the block.
185 * notify $lvalue
187 This will signal to one listener that the map is available.
189 * broadcast $lvalue
191 This will signal to all listeners that the map is available.
193 Constants
195 PROT_NONE, PROT_READ, PROT_WRITE, PROT_EXEC, MAP_ANONYMOUS, MAP_SHARED,
196 MAP_PRIVATE, MAP_ANON, MAP_FILE
197 These constants are used for sys_map. If you think you need them
198 your mmap manpage will explain them, but in most cases you can skip
199 sys_map altogether.
200
202 All previously mentioned functions are available for exportation, but
203 none are exported by default. Some functions may not be available on
204 your OS or your version of perl as specified above. A number of tags
205 are defined to make importation easier.
206 • :map
208 map_handle, map_file, map_anonymous, sys_map, unmap
210 • :extra
212 remap, sync, pin, unpin, advise, protect
214 • :lock
216 lock_map, wait_until, notify, broadcast
218 • :constants
220 PROT_NONE, PROT_READ, PROT_WRITE, PROT_EXEC, MAP_ANONYMOUS,
222 MAP_SHARED, MAP_PRIVATE, MAP_ANON, MAP_FILE
223 • :all
225 All functions defined in this module.
227
229 Exceptions
230 • Could not <function name>: this variable is not memory mapped
231 An attempt was made to "sync", "remap", "unmap", "pin", "unpin",
233 "advise" or "lock_map" an unmapped variable.
234 • Could not <function name>: <system error>
236 Your OS didn't allow File::Map to do what you asked it to do for
238 some reason.
239 • Trying to <function_name> on an unlocked map
241 You tried to "wait_until", "notify" or "broadcast" on an unlocked
243 variable.
244 • Zero length not allowed for anonymous map
246 A zero length anonymous map is not possible (or in any way useful).
248 • Can't remap a shared mapping
250 An attempt was made to remap a mapping that is shared among
252 different threads, this is not possible.
253 • Window (<start>, <end>) is outside the file
255 The offset and/or length you specified were invalid for this file.
257 • Can't map fake filehandle
259 The filehandle you provided is not real. This may mean it's a
261 scalar string handle or a tied handle.
262 • No such flag <flag_name>
264 The flag given for map_anonymous isn't valid, it should either be
266 "shared" or "private".
267 Warnings
269 • Writing directly to a memory mapped file is not recommended
270 Due to the way perl works internally, it's not possible to write a
272 mapping implementation that allows direct assignment yet performs
273 well. As a compromise, File::Map is capable of fixing up the mess
274 if you do it nonetheless, but it will warn you that you're doing
275 something you shouldn't. This warning is only given when "use
276 warnings 'substr'" is in effect.
277 • Truncating new value to size of the memory map
279 This warning is additional to the previous one, warning you that
281 you're losing data. This warning is only given when "use warnings
282 'substr'" is in effect.
283 • Shouldn't mmap non-binary filehandle
285 You tried to to map a filehandle that has some encoding layer.
287 Encoding layers are not supported by File::Map. This warning is
288 only given when "use warnings 'layer'" is in effect. Note that this
289 may become an exception in a future version.
290 • Unknown advice '<advice>'
292 You gave advise an advice it didn't know. This is either a typo or
294 a portability issue. This warning is only given when "use warnings
295 'portable'" is in effect.
296 • Syncing a readonly map makes no sense
298 "sync" flushes changes to the map to the filesystem. This obviously
300 is of little use when you can't change the map. This warning is
301 only given when "use warnings 'io'" is in effect.
302 • Can't overwrite an empty map
304 Overwriting an empty map is rather nonsensical, hence a warning is
306 given when this is tried. This warning is only given when "use
307 warnings 'substr'" is in effect.
308
310 This module depends on perl 5.8, Sub::Exporter::Progressive and
311 PerlIO::Layers. Perl 5.8.8 or higher is recommended because older
312 versions can give spurious warnings.
313 In perl versions before 5.11.5 many string functions including "substr"
315 are limited to 32bit logic
316 <http://rt.perl.org/rt3//Public/Bug/Display.html?id=72784>, even on
317 64bit architectures. Effectively this means you can't use them on
318 strings bigger than 2GB. If you are working with such large files, it
319 is strongly recommended to upgrade to 5.12.
320 In perl versions before 5.17.5, there is an off-by-one bug in Perl's
322 regexp engine, as explained here
323 <http://rt.perl.org/rt3//Public/Bug/Display.html?id=73542>. If the
324 length of the file is an exact multiple of the page size, some regexps
325 can trigger a segmentation fault.
326
328 • This module doesn't do any encoding or newline transformation for
329 you, and will reject any filehandle with such features enabled as
330 mapping it would return a different value than reading it normally.
331 Most importantly this means that on Windows you have to remember to
332 use the ":raw" open mode or binmode to make your filehandles binary
333 before mapping them, as by default it would do "crlf"
334 transformation. See PerlIO for more information on how that works.
335 • You can map a ":utf8" filehandle, but writing to it may be tricky.
337 Hic sunt dracones.
338 • You probably don't want to use ">" as a mode. This does not give
340 you reading permissions on many architectures, resulting in
341 segmentation faults when trying to read a variable (confusingly, it
342 will work on some others like x86).
343
345 As any piece of software, bugs are likely to exist here. Bug reports
346 are welcome.
347 Please report any bugs or feature requests to "bug-file-map at
349 rt.cpan.org", or through the web interface at
350 <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Map>. I will be
351 notified, and then you'll automatically be notified of progress on your
352 bug as I make changes.
353 Unicode file mappings are known to be buggy on perl 5.8.7 and lower.
355
357 • Sys::Mmap, the original Perl mmap module
358 • mmap(2), your mmap man page
360 • Win32::MMF
362 • CreateFileMapping at MSDN:
364 <http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx>
365
367 Leon Timmermans <fawaka@gmail.com>
368
370 This software is copyright (c) 2008 by Leon Timmermans.
371 This is free software; you can redistribute it and/or modify it under
373 the same terms as the Perl 5 programming language system itself.
374perl v5.32.1 2021-01-27 File::Map(3)