1File::Map(3) User Contributed Perl Documentation File::Map(3)
2
6 File::Map - Memory mapping made simple and safe.
7
9 Version 0.31
10
12 use File::Map 'map_file';
13 map_file my $map, $filename;
15 if ($map ne "foobar") {
16 $map =~ s/bar/quz/g;
17 substr $map, 1024, 11, "Hello world";
18 }
19
21 File::Map maps files or anonymous memory into perl variables.
22 Advantages of memory mapping
24 · Unlike normal perl variables, mapped memory is shared between
25 threads or forked processes.
26 · It is an efficient way to slurp an entire file. Unlike for example
28 File::Slurp, this module returns almost immediately, loading the
29 pages lazily on access. This means you only 'pay' for the parts of
30 the file you actually use.
31 · Perl usually doesn't return memory to the system while running,
33 mapped memory can be returned.
34 Advantages of this module over other similar modules
36 · Safety and Speed
37 This module is safe yet fast. Alternatives are either fast but can
39 cause segfaults or loose the mapping when not used correctly, or
40 are safe but rather slow. File::Map is as fast as a normal string
41 yet safe.
42 · Simplicity
44 It offers a simple interface targeted at common usage patterns
46 · Files are mapped into a variable that can be read just like any
48 other variable, and it can be written to using standard Perl
49 techniques such as regexps and "substr".
50 · Files can be mapped using a set of simple functions. There is
52 no need to know weird constants or the order of 6 arguments.
53 · It will automatically unmap the file when the scalar gets
55 destroyed. This works correctly even in multi-threaded
56 programs.
57 · Portability
59 File::Map supports Unix, Windows and VMS.
61 · Thread synchronization
63 It has built-in support for thread synchronization.
65
67 Mapping
68 The following functions for mapping a variable are available for
69 exportation.
70 · map_handle $lvalue, $filehandle, $mode = '<', $offset = 0, $length
72 = -s(*handle) - $offset
73 Use a filehandle to map into an lvalue. $filehandle should be a
75 scalar filehandle. $mode uses the same format as "open" does (it
76 currently accepts "<", "+<", ">" and "+>"). $offset and $length are
77 byte positions in the file, and default to mapping the whole file.
78 · map_file $lvalue, $filename, $mode = '<', $offset = 0, $length =
80 -s($filename) - $offset
81 Open a file and map it into an lvalue. Other than $filename, all
83 arguments work as in map_handle.
84 · map_anonymous $lvalue, $length
86 Map an anonymous piece of memory.
88 · sys_map $lvalue, $length, $protection, $flags, $filehandle, $offset
90 = 0
91 Low level map operation. It accepts the same constants as mmap does
93 (except its first argument obviously). If you don't know how mmap
94 works you probably shouldn't be using this.
95 · unmap $lvalue
97 Unmap a variable. Note that normally this is not necessary as
99 variables are unmapped automatically at destruction, but it is
100 included for completeness.
101 · remap $lvalue, $new_size
103 Try to remap $lvalue to a new size. It may fail if there is not
105 sufficient space to expand a mapping at its current location. This
106 call is linux specific and not supported on other systems.
107 Auxiliary
109 · sync $lvalue, $synchronous = 1
110 Flush changes made to the memory map back to disk. Mappings are
112 always flushed when unmapped, so this is usually not necessary. If
113 $synchronous is true and your operating system supports it, the
114 flushing will be done synchronously.
115 · pin $lvalue
117 Disable paging for this map, thus locking it in physical memory.
119 Depending on your operating system there may be limits on pinning.
120 · unpin $lvalue
122 Unlock the map from physical memory.
124 · advise $lvalue, $advice
126 Advise a certain memory usage pattern. This is not implemented on
128 all operating systems, and may be a no-op. The following values for
129 $advice are always accepted:.
130 · normal
132 Specifies that the application has no advice to give on its
134 behavior with respect to the mapped variable. It is the default
135 characteristic if no advice is given.
136 · random
138 Specifies that the application expects to access the mapped
140 variable in a random order.
141 · sequential
143 Specifies that the application expects to access the mapped
145 variable sequentially from start to end.
146 · willneed
148 Specifies that the application expects to access the mapped
150 variable in the near future.
151 · dontneed
153 Specifies that the application expects that it will not access
155 the mapped variable in the near future.
156 On some systems there may be more values available, but this can
158 not be relied on. Unknown values for $advice will cause a warning
159 but are further ignored.
160 · protect $lvalue, $mode
162 Change the memory protection of the mapping. $mode takes the same
164 format as, but also accepts sys_map style constants.
165 Locking
167 These locking functions provide locking for threads for the mapped
168 region. The mapped region has an internal lock and condition variable.
169 The condition variable functions("wait_until", "notify", "broadcast")
170 can only be used inside a locked block. If your perl has been compiled
171 without thread support the condition functions will not be available.
172 · lock_map $lvalue
174 Lock $lvalue until the end of the scope. If your perl does not
176 support threads, this will be a no-op.
177 · wait_until { block } $lvalue
179 Wait for block to become true. After every failed attempt, wait for
181 a signal. It returns the value returned by the block.
182 · notify $lvalue
184 This will signal to one listener that the map is available.
186 · broadcast $lvalue
188 This will signal to all listeners that the map is available.
190 CONSTANTS
192 PROT_NONE, PROT_READ, PROT_WRITE, PROT_EXEC, MAP_ANONYMOUS, MAP_SHARED,
193 MAP_PRIVATE, MAP_ANON, MAP_FILE
194 These constants are used for sys_map. If you think you need them
195 your mmap manpage will explain them, but in most cases you can skip
196 sys_map altogether.
197
199 All previously mentioned functions are available for exportation, but
200 none are exported by default. Some functions may not be available on
201 your OS or your version of perl as specified above. A number of tags
202 are defined to make importation easier.
203 · :map
205 map_handle, map_file, map_anonymous, sys_map, unmap
207 · :extra
209 remap, sync, pin, unpin, advise, protect
211 · :lock
213 lock_map, wait_until, notify, broadcast
215 · :constants
217 PROT_NONE, PROT_READ, PROT_WRITE, PROT_EXEC, MAP_ANONYMOUS,
219 MAP_SHARED, MAP_PRIVATE, MAP_ANON, MAP_FILE
220 · :all
222 All functions defined in this module.
224
226 In this overview %f is the name of the function that produced the
227 error, and %e is some error from your OS.
228 Exceptions
230 · Could not %f: this variable is not memory mapped
231 An attempt was made to "sync", "remap", "unmap", "pin", "unpin",
233 "advise" or "lock_map" an unmapped variable.
234 · Could not %f: %e
236 Your OS didn't allow File::Map to do what you asked it to do for
238 the reason specified in %e.
239 · Trying to %f on an unlocked map
241 You tried to "wait_until", "notify" or "broadcast" on an unlocked
243 variable.
244 · Zero length not allowed for anonymous map
246 A zero length anonymous map is not possible (or in any way useful).
248 · Can't remap a shared mapping
250 An attempts was made to remap a mapping that is shared among
252 different threads, this is not possible.
253 · Window ($start, $end) is outside the file
255 The offset and/or length you specified were invalid for this file.
257 Warnings
259 · Writing directly to a memory mapped file is not recommended
260 Due to the way perl works internally, it's not possible to write a
262 mapping implementation that allows direct assignment yet performs
263 well. As a compromise, File::Map is capable of fixing up the mess
264 if you do it nonetheless, but it will warn you that you're doing
265 something you shouldn't. This warning is only given when "use
266 warnings 'substr'" is in effect.
267 · Truncating new value to size of the memory map
269 This warning is additional to the previous one, warning you that
271 you're losing data. This warning is only given when "use warnings
272 'substr'" is in effect.
273 · Shouldn't mmap non-binary filehandle: layer '%s' is not binary
275 You tried to to map a filehandle that has some encoding layer.
277 Encoding layers are not supported by File::Map. This warning is
278 only given when "use warnings 'layer'" is in effect.
279 · Unknown advice '%s'
281 You gave advise an advice it didn't know. This is probably either a
283 typo or a portability issue. This warning is only given when "use
284 warnings 'portable'" is in effect.
285 · Syncing a readonly map makes no sense
287 "sync" flushes changes to the map to the filesystem. This obviously
289 is of little use when you can't change the map. This warning is
290 only given when "use warnings 'io'" is in effect.
291 · Can't overwrite an empty map
293 Overwriting an empty map is rather nonsensical, hence a warning is
295 given when this is tried. This warning is only given when "use
296 warnings 'substr'" is in effect.
297
299 This module depends on perl 5.8 and Const::Fast.
300
302 On perl versions before 5.11.5 many string functions including "substr"
303 are limited to 32bit logic
304 <http://rt.perl.org/rt3//Public/Bug/Display.html?id=72784>, even on
305 64bit architectures. Effectively this means you can't use them on
306 strings bigger than 2GB. If you are working with such large files, I
307 strongly recommend upgrading to 5.12.
308 This module assumes the file is binary data and doesn't do any encoding
310 or newline transformation for you. Most importantly this means that:
311 · On Windows, you have to remember to make your filehandles binary
313 before passing them to "map_handle" or "sys_map". It may warn on a
314 filehandle doing "crlf" transformation as it would return a different
315 value than reading it normally would. This can be remedied by using
316 open modes or binmode, see PerlIO for more information.
317 · You can make it a unicode string in-place by using utf8::decode if
319 it's valid utf-8, but writing to it requires you to really know what
320 you're doing. Currently "utf8" filehandles are not dealt with
321 gracefully, though in a future version that may change.
322 You probably don't want to use ">" as a mode. This does not give you
324 reading permissions on many architectures, resulting in segmentation
325 faults when trying to read a variable (confusingly, it will work on
326 some others like x86).
327
329 There is an off-by-one bug in Perl's regexp engine, as explained here
330 <http://rt.perl.org/rt3//Public/Bug/Display.html?id=73542>. If the
331 length of the file is an exact multiple of the page size, some regexps
332 can trigger a segmentation fault. This can not be fixed in this module
333 though.
334 As any piece of software, bugs are likely to exist here. Bug reports
336 are welcome.
337 Please report any bugs or feature requests to "bug-file-map at
339 rt.cpan.org", or through the web interface at
340 http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Map
341 <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Map>. I will be
342 notified, and then you'll automatically be notified of progress on your
343 bug as I make changes.
344
346 · Sys::Mmap, the original Perl mmap module
347 · mmap(2), your mmap man page
349 · Win32::MMF
351 · CreateFileMapping at MSDN:
353 http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx
354 <http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx>
355
357 Leon Timmermans, "<leont at cpan.org>"
358
360 You can find documentation for this module with the perldoc command.
361 perldoc File::Map
363 You can also look for information at:
365 · RT: CPAN's request tracker
367 http://rt.cpan.org/NoAuth/Bugs.html?Dist=File-Map
369 <http://rt.cpan.org/NoAuth/Bugs.html?Dist=File-Map>
370 · AnnoCPAN: Annotated CPAN documentation
372 http://annocpan.org/dist/File-Map <http://annocpan.org/dist/File-
374 Map>
375 · CPAN Ratings
377 http://cpanratings.perl.org/d/File-Map
379 <http://cpanratings.perl.org/d/File-Map>
380 · Search CPAN
382 http://search.cpan.org/dist/File-Map
384 <http://search.cpan.org/dist/File-Map>
385
387 Copyright 2008, 2009, 2010 Leon Timmermans, all rights reserved.
388 This program is free software; you can redistribute it and/or modify it
390 under the same terms as perl itself.
391perl v5.12.2 2011-01-17 File::Map(3)