1PERLFAQ5(1) Perl Programmers Reference Guide PERLFAQ5(1)
2
6 perlfaq5 - Files and Formats
7
9 This section deals with I/O and the "f" issues: filehandles, flushing,
10 formats, and footers.
11 How do I flush/unbuffer an output filehandle? Why must I do this?
13 (contributed by brian d foy)
14 You might like to read Mark Jason Dominus's "Suffering From Buffering"
16 at http://perl.plover.com/FAQs/Buffering.html .
17 Perl normally buffers output so it doesn't make a system call for every
19 bit of output. By saving up output, it makes fewer expensive system
20 calls. For instance, in this little bit of code, you want to print a
21 dot to the screen for every line you process to watch the progress of
22 your program. Instead of seeing a dot for every line, Perl buffers the
23 output and you have a long wait before you see a row of 50 dots all at
24 once:
25 # long wait, then row of dots all at once
27 while( <> ) {
28 print ".";
29 print "\n" unless ++$count % 50;
30 #... expensive line processing operations
32 }
33 To get around this, you have to unbuffer the output filehandle, in this
35 case, "STDOUT". You can set the special variable $| to a true value
36 (mnemonic: making your filehandles "piping hot"):
37 $|++;
39 # dot shown immediately
41 while( <> ) {
42 print ".";
43 print "\n" unless ++$count % 50;
44 #... expensive line processing operations
46 }
47 The $| is one of the per-filehandle special variables, so each
49 filehandle has its own copy of its value. If you want to merge standard
50 output and standard error for instance, you have to unbuffer each
51 (although STDERR might be unbuffered by default):
52 {
54 my $previous_default = select(STDOUT); # save previous default
55 $|++; # autoflush STDOUT
56 select(STDERR);
57 $|++; # autoflush STDERR, to be sure
58 select($previous_default); # restore previous default
59 }
60 # now should alternate . and +
62 while( 1 )
63 {
64 sleep 1;
65 print STDOUT ".";
66 print STDERR "+";
67 print STDOUT "\n" unless ++$count % 25;
68 }
69 Besides the $| special variable, you can use "binmode" to give your
71 filehandle a ":unix" layer, which is unbuffered:
72 binmode( STDOUT, ":unix" );
74 while( 1 ) {
76 sleep 1;
77 print ".";
78 print "\n" unless ++$count % 50;
79 }
80 For more information on output layers, see the entries for "binmode"
82 and "open" in perlfunc, and the "PerlIO" module documentation.
83 If you are using "IO::Handle" or one of its subclasses, you can call
85 the "autoflush" method to change the settings of the filehandle:
86 use IO::Handle;
88 open my( $io_fh ), ">", "output.txt";
89 $io_fh->autoflush(1);
90 The "IO::Handle" objects also have a "flush" method. You can flush the
92 buffer any time you want without auto-buffering
93 $io_fh->flush;
95 How do I change, delete, or insert a line in a file, or append to the
97 beginning of a file?
98 (contributed by brian d foy)
99 The basic idea of inserting, changing, or deleting a line from a text
101 file involves reading and printing the file to the point you want to
102 make the change, making the change, then reading and printing the rest
103 of the file. Perl doesn't provide random access to lines (especially
104 since the record input separator, $/, is mutable), although modules
105 such as "Tie::File" can fake it.
106 A Perl program to do these tasks takes the basic form of opening a
108 file, printing its lines, then closing the file:
109 open my $in, '<', $file or die "Can't read old file: $!";
111 open my $out, '>', "$file.new" or die "Can't write new file: $!";
112 while( <$in> )
114 {
115 print $out $_;
116 }
117 close $out;
119 Within that basic form, add the parts that you need to insert, change,
121 or delete lines.
122 To prepend lines to the beginning, print those lines before you enter
124 the loop that prints the existing lines.
125 open my $in, '<', $file or die "Can't read old file: $!";
127 open my $out, '>', "$file.new" or die "Can't write new file: $!";
128 print $out "# Add this line to the top\n"; # <--- HERE'S THE MAGIC
130 while( <$in> )
132 {
133 print $out $_;
134 }
135 close $out;
137 To change existing lines, insert the code to modify the lines inside
139 the "while" loop. In this case, the code finds all lowercased versions
140 of "perl" and uppercases them. The happens for every line, so be sure
141 that you're supposed to do that on every line!
142 open my $in, '<', $file or die "Can't read old file: $!";
144 open my $out, '>', "$file.new" or die "Can't write new file: $!";
145 print $out "# Add this line to the top\n";
147 while( <$in> )
149 {
150 s/\b(perl)\b/Perl/g;
151 print $out $_;
152 }
153 close $out;
155 To change only a particular line, the input line number, $., is useful.
157 First read and print the lines up to the one you want to change. Next,
158 read the single line you want to change, change it, and print it. After
159 that, read the rest of the lines and print those:
160 while( <$in> ) # print the lines before the change
162 {
163 print $out $_;
164 last if $. == 4; # line number before change
165 }
166 my $line = <$in>;
168 $line =~ s/\b(perl)\b/Perl/g;
169 print $out $line;
170 while( <$in> ) # print the rest of the lines
172 {
173 print $out $_;
174 }
175 To skip lines, use the looping controls. The "next" in this example
177 skips comment lines, and the "last" stops all processing once it
178 encounters either "__END__" or "__DATA__".
179 while( <$in> )
181 {
182 next if /^\s+#/; # skip comment lines
183 last if /^__(END|DATA)__$/; # stop at end of code marker
184 print $out $_;
185 }
186 Do the same sort of thing to delete a particular line by using "next"
188 to skip the lines you don't want to show up in the output. This example
189 skips every fifth line:
190 while( <$in> )
192 {
193 next unless $. % 5;
194 print $out $_;
195 }
196 If, for some odd reason, you really want to see the whole file at once
198 rather than processing line by line, you can slurp it in (as long as
199 you can fit the whole thing in memory!):
200 open my $in, '<', $file or die "Can't read old file: $!"
202 open my $out, '>', "$file.new" or die "Can't write new file: $!";
203 my @lines = do { local $/; <$in> }; # slurp!
205 # do your magic here
207 print $out @lines;
209 Modules such as "File::Slurp" and "Tie::File" can help with that too.
211 If you can, however, avoid reading the entire file at once. Perl won't
212 give that memory back to the operating system until the process
213 finishes.
214 You can also use Perl one-liners to modify a file in-place. The
216 following changes all 'Fred' to 'Barney' in inFile.txt, overwriting the
217 file with the new contents. With the "-p" switch, Perl wraps a "while"
218 loop around the code you specify with "-e", and "-i" turns on in-place
219 editing. The current line is in $_. With "-p", Perl automatically
220 prints the value of $_ at the end of the loop. See perlrun for more
221 details.
222 perl -pi -e 's/Fred/Barney/' inFile.txt
224 To make a backup of "inFile.txt", give "-i" a file extension to add:
226 perl -pi.bak -e 's/Fred/Barney/' inFile.txt
228 To change only the fifth line, you can add a test checking $., the
230 input line number, then only perform the operation when the test
231 passes:
232 perl -pi -e 's/Fred/Barney/ if $. == 5' inFile.txt
234 To add lines before a certain line, you can add a line (or lines!)
236 before Perl prints $_:
237 perl -pi -e 'print "Put before third line\n" if $. == 3' inFile.txt
239 You can even add a line to the beginning of a file, since the current
241 line prints at the end of the loop:
242 perl -pi -e 'print "Put before first line\n" if $. == 1' inFile.txt
244 To insert a line after one already in the file, use the "-n" switch.
246 It's just like "-p" except that it doesn't print $_ at the end of the
247 loop, so you have to do that yourself. In this case, print $_ first,
248 then print the line that you want to add.
249 perl -ni -e 'print; print "Put after fifth line\n" if $. == 5' inFile.txt
251 To delete lines, only print the ones that you want.
253 perl -ni -e 'print unless /d/' inFile.txt
255 ... or ...
257 perl -pi -e 'next unless /d/' inFile.txt
259 How do I count the number of lines in a file?
261 One fairly efficient way is to count newlines in the file. The
262 following program uses a feature of tr///, as documented in perlop. If
263 your text file doesn't end with a newline, then it's not really a
264 proper text file, so this may report one fewer line than you expect.
265 $lines = 0;
267 open(FILE, $filename) or die "Can't open `$filename': $!";
268 while (sysread FILE, $buffer, 4096) {
269 $lines += ($buffer =~ tr/\n//);
270 }
271 close FILE;
272 This assumes no funny games with newline translations.
274 How can I use Perl's "-i" option from within a program?
276 "-i" sets the value of Perl's $^I variable, which in turn affects the
277 behavior of "<>"; see perlrun for more details. By modifying the
278 appropriate variables directly, you can get the same behavior within a
279 larger program. For example:
280 # ...
282 {
283 local($^I, @ARGV) = ('.orig', glob("*.c"));
284 while (<>) {
285 if ($. == 1) {
286 print "This line should appear at the top of each file\n";
287 }
288 s/\b(p)earl\b/${1}erl/i; # Correct typos, preserving case
289 print;
290 close ARGV if eof; # Reset $.
291 }
292 }
293 # $^I and @ARGV return to their old values here
294 This block modifies all the ".c" files in the current directory,
296 leaving a backup of the original data from each file in a new ".c.orig"
297 file.
298 How can I copy a file?
300 (contributed by brian d foy)
301 Use the "File::Copy" module. It comes with Perl and can do a true copy
303 across file systems, and it does its magic in a portable fashion.
304 use File::Copy;
306 copy( $original, $new_copy ) or die "Copy failed: $!";
308 If you can't use "File::Copy", you'll have to do the work yourself:
310 open the original file, open the destination file, then print to the
311 destination file as you read the original. You also have to remember to
312 copy the permissions, owner, and group to the new file.
313 How do I make a temporary file name?
315 If you don't need to know the name of the file, you can use "open()"
316 with "undef" in place of the file name. In Perl 5.8 or later, the
317 "open()" function creates an anonymous temporary file:
318 open my $tmp, '+>', undef or die $!;
320 Otherwise, you can use the File::Temp module.
322 use File::Temp qw/ tempfile tempdir /;
324 $dir = tempdir( CLEANUP => 1 );
326 ($fh, $filename) = tempfile( DIR => $dir );
327 # or if you don't need to know the filename
329 $fh = tempfile( DIR => $dir );
331 The File::Temp has been a standard module since Perl 5.6.1. If you
333 don't have a modern enough Perl installed, use the "new_tmpfile" class
334 method from the IO::File module to get a filehandle opened for reading
335 and writing. Use it if you don't need to know the file's name:
336 use IO::File;
338 $fh = IO::File->new_tmpfile()
339 or die "Unable to make new temporary file: $!";
340 If you're committed to creating a temporary file by hand, use the
342 process ID and/or the current time-value. If you need to have many
343 temporary files in one process, use a counter:
344 BEGIN {
346 use Fcntl;
347 my $temp_dir = -d '/tmp' ? '/tmp' : $ENV{TMPDIR} || $ENV{TEMP};
348 my $base_name = sprintf "%s/%d-%d-0000", $temp_dir, $$, time;
349 sub temp_file {
351 local *FH;
352 my $count = 0;
353 until( defined(fileno(FH)) || $count++ > 100 ) {
354 $base_name =~ s/-(\d+)$/"-" . (1 + $1)/e;
355 # O_EXCL is required for security reasons.
356 sysopen FH, $base_name, O_WRONLY|O_EXCL|O_CREAT;
357 }
358 if( defined fileno(FH) ) {
360 return (*FH, $base_name);
361 }
362 else {
363 return ();
364 }
365 }
366 }
368 How can I manipulate fixed-record-length files?
370 The most efficient way is using pack() and unpack(). This is faster
371 than using substr() when taking many, many strings. It is slower for
372 just a few.
373 Here is a sample chunk of code to break up and put back together again
375 some fixed-format input lines, in this case from the output of a
376 normal, Berkeley-style ps:
377 # sample input line:
379 # 15158 p5 T 0:00 perl /home/tchrist/scripts/now-what
380 my $PS_T = 'A6 A4 A7 A5 A*';
381 open my $ps, '-|', 'ps';
382 print scalar <$ps>;
383 my @fields = qw( pid tt stat time command );
384 while (<$ps>) {
385 my %process;
386 @process{@fields} = unpack($PS_T, $_);
387 for my $field ( @fields ) {
388 print "$field: <$process{$field}>\n";
389 }
390 print 'line=', pack($PS_T, @process{@fields} ), "\n";
391 }
392 We've used a hash slice in order to easily handle the fields of each
394 row. Storing the keys in an array means it's easy to operate on them
395 as a group or loop over them with for. It also avoids polluting the
396 program with global variables and using symbolic references.
397 How can I make a filehandle local to a subroutine? How do I pass
399 filehandles between subroutines? How do I make an array of
400 filehandles?
401 As of perl5.6, open() autovivifies file and directory handles as
402 references if you pass it an uninitialized scalar variable. You can
403 then pass these references just like any other scalar, and use them in
404 the place of named handles.
405 open my $fh, $file_name;
407 open local $fh, $file_name;
409 print $fh "Hello World!\n";
411 process_file( $fh );
413 If you like, you can store these filehandles in an array or a hash. If
415 you access them directly, they aren't simple scalars and you need to
416 give "print" a little help by placing the filehandle reference in
417 braces. Perl can only figure it out on its own when the filehandle
418 reference is a simple scalar.
419 my @fhs = ( $fh1, $fh2, $fh3 );
421 for( $i = 0; $i <= $#fhs; $i++ ) {
423 print {$fhs[$i]} "just another Perl answer, \n";
424 }
425 Before perl5.6, you had to deal with various typeglob idioms which you
427 may see in older code.
428 open FILE, "> $filename";
430 process_typeglob( *FILE );
431 process_reference( \*FILE );
432 sub process_typeglob { local *FH = shift; print FH "Typeglob!" }
434 sub process_reference { local $fh = shift; print $fh "Reference!" }
435 If you want to create many anonymous handles, you should check out the
437 Symbol or IO::Handle modules.
438 How can I use a filehandle indirectly?
440 An indirect filehandle is using something other than a symbol in a
441 place that a filehandle is expected. Here are ways to get indirect
442 filehandles:
443 $fh = SOME_FH; # bareword is strict-subs hostile
445 $fh = "SOME_FH"; # strict-refs hostile; same package only
446 $fh = *SOME_FH; # typeglob
447 $fh = \*SOME_FH; # ref to typeglob (bless-able)
448 $fh = *SOME_FH{IO}; # blessed IO::Handle from *SOME_FH typeglob
449 Or, you can use the "new" method from one of the IO::* modules to
451 create an anonymous filehandle, store that in a scalar variable, and
452 use it as though it were a normal filehandle.
453 use IO::Handle; # 5.004 or higher
455 $fh = IO::Handle->new();
456 Then use any of those as you would a normal filehandle. Anywhere that
458 Perl is expecting a filehandle, an indirect filehandle may be used
459 instead. An indirect filehandle is just a scalar variable that contains
460 a filehandle. Functions like "print", "open", "seek", or the "<FH>"
461 diamond operator will accept either a named filehandle or a scalar
462 variable containing one:
463 ($ifh, $ofh, $efh) = (*STDIN, *STDOUT, *STDERR);
465 print $ofh "Type it: ";
466 $got = <$ifh>
467 print $efh "What was that: $got";
468 If you're passing a filehandle to a function, you can write the
470 function in two ways:
471 sub accept_fh {
473 my $fh = shift;
474 print $fh "Sending to indirect filehandle\n";
475 }
476 Or it can localize a typeglob and use the filehandle directly:
478 sub accept_fh {
480 local *FH = shift;
481 print FH "Sending to localized filehandle\n";
482 }
483 Both styles work with either objects or typeglobs of real filehandles.
485 (They might also work with strings under some circumstances, but this
486 is risky.)
487 accept_fh(*STDOUT);
489 accept_fh($handle);
490 In the examples above, we assigned the filehandle to a scalar variable
492 before using it. That is because only simple scalar variables, not
493 expressions or subscripts of hashes or arrays, can be used with built-
494 ins like "print", "printf", or the diamond operator. Using something
495 other than a simple scalar variable as a filehandle is illegal and
496 won't even compile:
497 @fd = (*STDIN, *STDOUT, *STDERR);
499 print $fd[1] "Type it: "; # WRONG
500 $got = <$fd[0]> # WRONG
501 print $fd[2] "What was that: $got"; # WRONG
502 With "print" and "printf", you get around this by using a block and an
504 expression where you would place the filehandle:
505 print { $fd[1] } "funny stuff\n";
507 printf { $fd[1] } "Pity the poor %x.\n", 3_735_928_559;
508 # Pity the poor deadbeef.
509 That block is a proper block like any other, so you can put more
511 complicated code there. This sends the message out to one of two
512 places:
513 $ok = -x "/bin/cat";
515 print { $ok ? $fd[1] : $fd[2] } "cat stat $ok\n";
516 print { $fd[ 1+ ($ok || 0) ] } "cat stat $ok\n";
517 This approach of treating "print" and "printf" like object methods
519 calls doesn't work for the diamond operator. That's because it's a
520 real operator, not just a function with a comma-less argument.
521 Assuming you've been storing typeglobs in your structure as we did
522 above, you can use the built-in function named "readline" to read a
523 record just as "<>" does. Given the initialization shown above for
524 @fd, this would work, but only because readline() requires a typeglob.
525 It doesn't work with objects or strings, which might be a bug we
526 haven't fixed yet.
527 $got = readline($fd[0]);
529 Let it be noted that the flakiness of indirect filehandles is not
531 related to whether they're strings, typeglobs, objects, or anything
532 else. It's the syntax of the fundamental operators. Playing the
533 object game doesn't help you at all here.
534 How can I set up a footer format to be used with write()?
536 There's no builtin way to do this, but perlform has a couple of
537 techniques to make it possible for the intrepid hacker.
538 How can I write() into a string?
540 See "Accessing Formatting Internals" in perlform for an "swrite()"
541 function.
542 How can I open a filehandle to a string?
544 (contributed by Peter J. Holzer, hjp-usenet2@hjp.at)
545 Since Perl 5.8.0 a file handle referring to a string can be created by
547 calling open with a reference to that string instead of the filename.
548 This file handle can then be used to read from or write to the string:
549 open(my $fh, '>', \$string) or die "Could not open string for writing";
551 print $fh "foo\n";
552 print $fh "bar\n"; # $string now contains "foo\nbar\n"
553 open(my $fh, '<', \$string) or die "Could not open string for reading";
555 my $x = <$fh>; # $x now contains "foo\n"
556 With older versions of Perl, the "IO::String" module provides similar
558 functionality.
559 How can I output my numbers with commas added?
561 (contributed by brian d foy and Benjamin Goldberg)
562 You can use Number::Format to separate places in a number. It handles
564 locale information for those of you who want to insert full stops
565 instead (or anything else that they want to use, really).
566 This subroutine will add commas to your number:
568 sub commify {
570 local $_ = shift;
571 1 while s/^([-+]?\d+)(\d{3})/$1,$2/;
572 return $_;
573 }
574 This regex from Benjamin Goldberg will add commas to numbers:
576 s/(^[-+]?\d+?(?=(?>(?:\d{3})+)(?!\d))|\G\d{3}(?=\d))/$1,/g;
578 It is easier to see with comments:
580 s/(
582 ^[-+]? # beginning of number.
583 \d+? # first digits before first comma
584 (?= # followed by, (but not included in the match) :
585 (?>(?:\d{3})+) # some positive multiple of three digits.
586 (?!\d) # an *exact* multiple, not x * 3 + 1 or whatever.
587 )
588 | # or:
589 \G\d{3} # after the last group, get three digits
590 (?=\d) # but they have to have more digits after them.
591 )/$1,/xg;
592 How can I translate tildes (~) in a filename?
594 Use the <> ("glob()") operator, documented in perlfunc. Versions of
595 Perl older than 5.6 require that you have a shell installed that groks
596 tildes. Later versions of Perl have this feature built in. The
597 "File::KGlob" module (available from CPAN) gives more portable glob
598 functionality.
599 Within Perl, you may use this directly:
601 $filename =~ s{
603 ^ ~ # find a leading tilde
604 ( # save this in $1
605 [^/] # a non-slash character
606 * # repeated 0 or more times (0 means me)
607 )
608 }{
609 $1
610 ? (getpwnam($1))[7]
611 : ( $ENV{HOME} || $ENV{LOGDIR} )
612 }ex;
613 How come when I open a file read-write it wipes it out?
615 Because you're using something like this, which truncates the file and
616 then gives you read-write access:
617 open(FH, "+> /path/name"); # WRONG (almost always)
619 Whoops. You should instead use this, which will fail if the file
621 doesn't exist.
622 open(FH, "+< /path/name"); # open for update
624 Using ">" always clobbers or creates. Using "<" never does either.
626 The "+" doesn't change this.
627 Here are examples of many kinds of file opens. Those using sysopen()
629 all assume
630 use Fcntl;
632 To open file for reading:
634 open(FH, "< $path") || die $!;
636 sysopen(FH, $path, O_RDONLY) || die $!;
637 To open file for writing, create new file if needed or else truncate
639 old file:
640 open(FH, "> $path") || die $!;
642 sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT) || die $!;
643 sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666) || die $!;
644 To open file for writing, create new file, file must not exist:
646 sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT) || die $!;
648 sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT, 0666) || die $!;
649 To open file for appending, create if necessary:
651 open(FH, ">> $path") || die $!;
653 sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT) || die $!;
654 sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT, 0666) || die $!;
655 To open file for appending, file must exist:
657 sysopen(FH, $path, O_WRONLY|O_APPEND) || die $!;
659 To open file for update, file must exist:
661 open(FH, "+< $path") || die $!;
663 sysopen(FH, $path, O_RDWR) || die $!;
664 To open file for update, create file if necessary:
666 sysopen(FH, $path, O_RDWR|O_CREAT) || die $!;
668 sysopen(FH, $path, O_RDWR|O_CREAT, 0666) || die $!;
669 To open file for update, file must not exist:
671 sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT) || die $!;
673 sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT, 0666) || die $!;
674 To open a file without blocking, creating if necessary:
676 sysopen(FH, "/foo/somefile", O_WRONLY|O_NDELAY|O_CREAT)
678 or die "can't open /foo/somefile: $!":
679 Be warned that neither creation nor deletion of files is guaranteed to
681 be an atomic operation over NFS. That is, two processes might both
682 successfully create or unlink the same file! Therefore O_EXCL isn't as
683 exclusive as you might wish.
684 See also the new perlopentut if you have it (new for 5.6).
686 Why do I sometimes get an "Argument list too long" when I use <*>?
688 The "<>" operator performs a globbing operation (see above). In Perl
689 versions earlier than v5.6.0, the internal glob() operator forks csh(1)
690 to do the actual glob expansion, but csh can't handle more than 127
691 items and so gives the error message "Argument list too long". People
692 who installed tcsh as csh won't have this problem, but their users may
693 be surprised by it.
694 To get around this, either upgrade to Perl v5.6.0 or later, do the glob
696 yourself with readdir() and patterns, or use a module like File::KGlob,
697 one that doesn't use the shell to do globbing.
698 Is there a leak/bug in glob()?
700 Due to the current implementation on some operating systems, when you
701 use the glob() function or its angle-bracket alias in a scalar context,
702 you may cause a memory leak and/or unpredictable behavior. It's best
703 therefore to use glob() only in list context.
704 How can I open a file with a leading ">" or trailing blanks?
706 (contributed by Brian McCauley)
707 The special two argument form of Perl's open() function ignores
709 trailing blanks in filenames and infers the mode from certain leading
710 characters (or a trailing "|"). In older versions of Perl this was the
711 only version of open() and so it is prevalent in old code and books.
712 Unless you have a particular reason to use the two argument form you
714 should use the three argument form of open() which does not treat any
715 characters in the filename as special.
716 open FILE, "<", " file "; # filename is " file "
718 open FILE, ">", ">file"; # filename is ">file"
719 How can I reliably rename a file?
721 If your operating system supports a proper mv(1) utility or its
722 functional equivalent, this works:
723 rename($old, $new) or system("mv", $old, $new);
725 It may be more portable to use the File::Copy module instead. You just
727 copy to the new file to the new name (checking return values), then
728 delete the old one. This isn't really the same semantically as a
729 rename(), which preserves meta-information like permissions,
730 timestamps, inode info, etc.
731 Newer versions of File::Copy export a move() function.
733 How can I lock a file?
735 Perl's builtin flock() function (see perlfunc for details) will call
736 flock(2) if that exists, fcntl(2) if it doesn't (on perl version 5.004
737 and later), and lockf(3) if neither of the two previous system calls
738 exists. On some systems, it may even use a different form of native
739 locking. Here are some gotchas with Perl's flock():
740 1. Produces a fatal error if none of the three system calls (or their
742 close equivalent) exists.
743 2. lockf(3) does not provide shared locking, and requires that the
745 filehandle be open for writing (or appending, or read/writing).
746 3. Some versions of flock() can't lock files over a network (e.g. on
748 NFS file systems), so you'd need to force the use of fcntl(2) when
749 you build Perl. But even this is dubious at best. See the flock
750 entry of perlfunc and the INSTALL file in the source distribution
751 for information on building Perl to do this.
752 Two potentially non-obvious but traditional flock semantics are
754 that it waits indefinitely until the lock is granted, and that its
755 locks are merely advisory. Such discretionary locks are more
756 flexible, but offer fewer guarantees. This means that files locked
757 with flock() may be modified by programs that do not also use
758 flock(). Cars that stop for red lights get on well with each
759 other, but not with cars that don't stop for red lights. See the
760 perlport manpage, your port's specific documentation, or your
761 system-specific local manpages for details. It's best to assume
762 traditional behavior if you're writing portable programs. (If
763 you're not, you should as always feel perfectly free to write for
764 your own system's idiosyncrasies (sometimes called "features").
765 Slavish adherence to portability concerns shouldn't get in the way
766 of your getting your job done.)
767 For more information on file locking, see also "File Locking" in
769 perlopentut if you have it (new for 5.6).
770 Why can't I just open(FH, ">file.lock")?
772 A common bit of code NOT TO USE is this:
773 sleep(3) while -e "file.lock"; # PLEASE DO NOT USE
775 open(LCK, "> file.lock"); # THIS BROKEN CODE
776 This is a classic race condition: you take two steps to do something
778 which must be done in one. That's why computer hardware provides an
779 atomic test-and-set instruction. In theory, this "ought" to work:
780 sysopen(FH, "file.lock", O_WRONLY|O_EXCL|O_CREAT)
782 or die "can't open file.lock: $!";
783 except that lamentably, file creation (and deletion) is not atomic over
785 NFS, so this won't work (at least, not every time) over the net.
786 Various schemes involving link() have been suggested, but these tend to
787 involve busy-wait, which is also less than desirable.
788 I still don't get locking. I just want to increment the number in the
790 file. How can I do this?
791 Didn't anyone ever tell you web-page hit counters were useless? They
792 don't count number of hits, they're a waste of time, and they serve
793 only to stroke the writer's vanity. It's better to pick a random
794 number; they're more realistic.
795 Anyway, this is what you can do if you can't help yourself.
797 use Fcntl qw(:DEFAULT :flock);
799 sysopen(FH, "numfile", O_RDWR|O_CREAT) or die "can't open numfile: $!";
800 flock(FH, LOCK_EX) or die "can't flock numfile: $!";
801 $num = <FH> || 0;
802 seek(FH, 0, 0) or die "can't rewind numfile: $!";
803 truncate(FH, 0) or die "can't truncate numfile: $!";
804 (print FH $num+1, "\n") or die "can't write numfile: $!";
805 close FH or die "can't close numfile: $!";
806 Here's a much better web-page hit counter:
808 $hits = int( (time() - 850_000_000) / rand(1_000) );
810 If the count doesn't impress your friends, then the code might. :-)
812 All I want to do is append a small amount of text to the end of a file. Do
814 I still have to use locking?
815 If you are on a system that correctly implements "flock" and you use
816 the example appending code from "perldoc -f flock" everything will be
817 OK even if the OS you are on doesn't implement append mode correctly
818 (if such a system exists.) So if you are happy to restrict yourself to
819 OSs that implement "flock" (and that's not really much of a
820 restriction) then that is what you should do.
821 If you know you are only going to use a system that does correctly
823 implement appending (i.e. not Win32) then you can omit the "seek" from
824 the code in the previous answer.
825 If you know you are only writing code to run on an OS and filesystem
827 that does implement append mode correctly (a local filesystem on a
828 modern Unix for example), and you keep the file in block-buffered mode
829 and you write less than one buffer-full of output between each manual
830 flushing of the buffer then each bufferload is almost guaranteed to be
831 written to the end of the file in one chunk without getting
832 intermingled with anyone else's output. You can also use the "syswrite"
833 function which is simply a wrapper around your system's write(2) system
834 call.
835 There is still a small theoretical chance that a signal will interrupt
837 the system level "write()" operation before completion. There is also a
838 possibility that some STDIO implementations may call multiple system
839 level "write()"s even if the buffer was empty to start. There may be
840 some systems where this probability is reduced to zero, and this is not
841 a concern when using ":perlio" instead of your system's STDIO.
842 How do I randomly update a binary file?
844 If you're just trying to patch a binary, in many cases something as
845 simple as this works:
846 perl -i -pe 's{window manager}{window mangler}g' /usr/bin/emacs
848 However, if you have fixed sized records, then you might do something
850 more like this:
851 $RECSIZE = 220; # size of record, in bytes
853 $recno = 37; # which record to update
854 open(FH, "+<somewhere") || die "can't update somewhere: $!";
855 seek(FH, $recno * $RECSIZE, 0);
856 read(FH, $record, $RECSIZE) == $RECSIZE || die "can't read record $recno: $!";
857 # munge the record
858 seek(FH, -$RECSIZE, 1);
859 print FH $record;
860 close FH;
861 Locking and error checking are left as an exercise for the reader.
863 Don't forget them or you'll be quite sorry.
864 How do I get a file's timestamp in perl?
866 If you want to retrieve the time at which the file was last read,
867 written, or had its meta-data (owner, etc) changed, you use the -A, -M,
868 or -C file test operations as documented in perlfunc. These retrieve
869 the age of the file (measured against the start-time of your program)
870 in days as a floating point number. Some platforms may not have all of
871 these times. See perlport for details. To retrieve the "raw" time in
872 seconds since the epoch, you would call the stat function, then use
873 localtime(), gmtime(), or POSIX::strftime() to convert this into human-
874 readable form.
875 Here's an example:
877 $write_secs = (stat($file))[9];
879 printf "file %s updated at %s\n", $file,
880 scalar localtime($write_secs);
881 If you prefer something more legible, use the File::stat module (part
883 of the standard distribution in version 5.004 and later):
884 # error checking left as an exercise for reader.
886 use File::stat;
887 use Time::localtime;
888 $date_string = ctime(stat($file)->mtime);
889 print "file $file updated at $date_string\n";
890 The POSIX::strftime() approach has the benefit of being, in theory,
892 independent of the current locale. See perllocale for details.
893 How do I set a file's timestamp in perl?
895 You use the utime() function documented in "utime" in perlfunc. By way
896 of example, here's a little program that copies the read and write
897 times from its first argument to all the rest of them.
898 if (@ARGV < 2) {
900 die "usage: cptimes timestamp_file other_files ...\n";
901 }
902 $timestamp = shift;
903 ($atime, $mtime) = (stat($timestamp))[8,9];
904 utime $atime, $mtime, @ARGV;
905 Error checking is, as usual, left as an exercise for the reader.
907 The perldoc for utime also has an example that has the same effect as
909 touch(1) on files that already exist.
910 Certain file systems have a limited ability to store the times on a
912 file at the expected level of precision. For example, the FAT and HPFS
913 filesystem are unable to create dates on files with a finer granularity
914 than two seconds. This is a limitation of the filesystems, not of
915 utime().
916 How do I print to more than one file at once?
918 To connect one filehandle to several output filehandles, you can use
919 the IO::Tee or Tie::FileHandle::Multiplex modules.
920 If you only have to do this once, you can print individually to each
922 filehandle.
923 for $fh (FH1, FH2, FH3) { print $fh "whatever\n" }
925 How can I read in an entire file all at once?
927 You can use the File::Slurp module to do it in one step.
928 use File::Slurp;
930 $all_of_it = read_file($filename); # entire file in scalar
932 @all_lines = read_file($filename); # one line per element
933 The customary Perl approach for processing all the lines in a file is
935 to do so one line at a time:
936 open (INPUT, $file) || die "can't open $file: $!";
938 while (<INPUT>) {
939 chomp;
940 # do something with $_
941 }
942 close(INPUT) || die "can't close $file: $!";
943 This is tremendously more efficient than reading the entire file into
945 memory as an array of lines and then processing it one element at a
946 time, which is often--if not almost always--the wrong approach.
947 Whenever you see someone do this:
948 @lines = <INPUT>;
950 you should think long and hard about why you need everything loaded at
952 once. It's just not a scalable solution. You might also find it more
953 fun to use the standard Tie::File module, or the DB_File module's
954 $DB_RECNO bindings, which allow you to tie an array to a file so that
955 accessing an element the array actually accesses the corresponding line
956 in the file.
957 You can read the entire filehandle contents into a scalar.
959 {
961 local(*INPUT, $/);
962 open (INPUT, $file) || die "can't open $file: $!";
963 $var = <INPUT>;
964 }
965 That temporarily undefs your record separator, and will automatically
967 close the file at block exit. If the file is already open, just use
968 this:
969 $var = do { local $/; <INPUT> };
971 For ordinary files you can also use the read function.
973 read( INPUT, $var, -s INPUT );
975 The third argument tests the byte size of the data on the INPUT
977 filehandle and reads that many bytes into the buffer $var.
978 How can I read in a file by paragraphs?
980 Use the $/ variable (see perlvar for details). You can either set it
981 to "" to eliminate empty paragraphs ("abc\n\n\n\ndef", for instance,
982 gets treated as two paragraphs and not three), or "\n\n" to accept
983 empty paragraphs.
984 Note that a blank line must have no blanks in it. Thus
986 "fred\n \nstuff\n\n" is one paragraph, but "fred\n\nstuff\n\n" is two.
987 How can I read a single character from a file? From the keyboard?
989 You can use the builtin "getc()" function for most filehandles, but it
990 won't (easily) work on a terminal device. For STDIN, either use the
991 Term::ReadKey module from CPAN or use the sample code in "getc" in
992 perlfunc.
993 If your system supports the portable operating system programming
995 interface (POSIX), you can use the following code, which you'll note
996 turns off echo processing as well.
997 #!/usr/bin/perl -w
999 use strict;
1000 $| = 1;
1001 for (1..4) {
1002 my $got;
1003 print "gimme: ";
1004 $got = getone();
1005 print "--> $got\n";
1006 }
1007 exit;
1008 BEGIN {
1010 use POSIX qw(:termios_h);
1011 my ($term, $oterm, $echo, $noecho, $fd_stdin);
1013 $fd_stdin = fileno(STDIN);
1015 $term = POSIX::Termios->new();
1017 $term->getattr($fd_stdin);
1018 $oterm = $term->getlflag();
1019 $echo = ECHO | ECHOK | ICANON;
1021 $noecho = $oterm & ~$echo;
1022 sub cbreak {
1024 $term->setlflag($noecho);
1025 $term->setcc(VTIME, 1);
1026 $term->setattr($fd_stdin, TCSANOW);
1027 }
1028 sub cooked {
1030 $term->setlflag($oterm);
1031 $term->setcc(VTIME, 0);
1032 $term->setattr($fd_stdin, TCSANOW);
1033 }
1034 sub getone {
1036 my $key = '';
1037 cbreak();
1038 sysread(STDIN, $key, 1);
1039 cooked();
1040 return $key;
1041 }
1042 }
1044 END { cooked() }
1046 The Term::ReadKey module from CPAN may be easier to use. Recent
1048 versions include also support for non-portable systems as well.
1049 use Term::ReadKey;
1051 open(TTY, "</dev/tty");
1052 print "Gimme a char: ";
1053 ReadMode "raw";
1054 $key = ReadKey 0, *TTY;
1055 ReadMode "normal";
1056 printf "\nYou said %s, char number %03d\n",
1057 $key, ord $key;
1058 How can I tell whether there's a character waiting on a filehandle?
1060 The very first thing you should do is look into getting the
1061 Term::ReadKey extension from CPAN. As we mentioned earlier, it now
1062 even has limited support for non-portable (read: not open systems,
1063 closed, proprietary, not POSIX, not Unix, etc) systems.
1064 You should also check out the Frequently Asked Questions list in
1066 comp.unix.* for things like this: the answer is essentially the same.
1067 It's very system dependent. Here's one solution that works on BSD
1068 systems:
1069 sub key_ready {
1071 my($rin, $nfd);
1072 vec($rin, fileno(STDIN), 1) = 1;
1073 return $nfd = select($rin,undef,undef,0);
1074 }
1075 If you want to find out how many characters are waiting, there's also
1077 the FIONREAD ioctl call to be looked at. The h2ph tool that comes with
1078 Perl tries to convert C include files to Perl code, which can be
1079 "require"d. FIONREAD ends up defined as a function in the sys/ioctl.ph
1080 file:
1081 require 'sys/ioctl.ph';
1083 $size = pack("L", 0);
1085 ioctl(FH, FIONREAD(), $size) or die "Couldn't call ioctl: $!\n";
1086 $size = unpack("L", $size);
1087 If h2ph wasn't installed or doesn't work for you, you can grep the
1089 include files by hand:
1090 % grep FIONREAD /usr/include/*/*
1092 /usr/include/asm/ioctls.h:#define FIONREAD 0x541B
1093 Or write a small C program using the editor of champions:
1095 % cat > fionread.c
1097 #include <sys/ioctl.h>
1098 main() {
1099 printf("%#08x\n", FIONREAD);
1100 }
1101 ^D
1102 % cc -o fionread fionread.c
1103 % ./fionread
1104 0x4004667f
1105 And then hard code it, leaving porting as an exercise to your
1107 successor.
1108 $FIONREAD = 0x4004667f; # XXX: opsys dependent
1110 $size = pack("L", 0);
1112 ioctl(FH, $FIONREAD, $size) or die "Couldn't call ioctl: $!\n";
1113 $size = unpack("L", $size);
1114 FIONREAD requires a filehandle connected to a stream, meaning that
1116 sockets, pipes, and tty devices work, but not files.
1117 How do I do a "tail -f" in perl?
1119 First try
1120 seek(GWFILE, 0, 1);
1122 The statement "seek(GWFILE, 0, 1)" doesn't change the current position,
1124 but it does clear the end-of-file condition on the handle, so that the
1125 next "<GWFILE>" makes Perl try again to read something.
1126 If that doesn't work (it relies on features of your stdio
1128 implementation), then you need something more like this:
1129 for (;;) {
1131 for ($curpos = tell(GWFILE); <GWFILE>; $curpos = tell(GWFILE)) {
1132 # search for some stuff and put it into files
1133 }
1134 # sleep for a while
1135 seek(GWFILE, $curpos, 0); # seek to where we had been
1136 }
1137 If this still doesn't work, look into the "clearerr" method from
1139 "IO::Handle", which resets the error and end-of-file states on the
1140 handle.
1141 There's also a "File::Tail" module from CPAN.
1143 How do I dup() a filehandle in Perl?
1145 If you check "open" in perlfunc, you'll see that several of the ways to
1146 call open() should do the trick. For example:
1147 open(LOG, ">>/foo/logfile");
1149 open(STDERR, ">&LOG");
1150 Or even with a literal numeric descriptor:
1152 $fd = $ENV{MHCONTEXTFD};
1154 open(MHCONTEXT, "<&=$fd"); # like fdopen(3S)
1155 Note that "<&STDIN" makes a copy, but "<&=STDIN" make an alias. That
1157 means if you close an aliased handle, all aliases become inaccessible.
1158 This is not true with a copied one.
1159 Error checking, as always, has been left as an exercise for the reader.
1161 How do I close a file descriptor by number?
1163 If, for some reason, you have a file descriptor instead of a filehandle
1164 (perhaps you used "POSIX::open"), you can use the "close()" function
1165 from the "POSIX" module:
1166 use POSIX ();
1168 POSIX::close( $fd );
1170 This should rarely be necessary, as the Perl "close()" function is to
1172 be used for things that Perl opened itself, even if it was a dup of a
1173 numeric descriptor as with "MHCONTEXT" above. But if you really have
1174 to, you may be able to do this:
1175 require 'sys/syscall.ph';
1177 $rc = syscall(&SYS_close, $fd + 0); # must force numeric
1178 die "can't sysclose $fd: $!" unless $rc == -1;
1179 Or, just use the fdopen(3S) feature of "open()":
1181 {
1183 open my( $fh ), "<&=$fd" or die "Cannot reopen fd=$fd: $!";
1184 close $fh;
1185 }
1186 Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe`
1188 work?
1189 Whoops! You just put a tab and a formfeed into that filename!
1190 Remember that within double quoted strings ("like\this"), the backslash
1191 is an escape character. The full list of these is in "Quote and Quote-
1192 like Operators" in perlop. Unsurprisingly, you don't have a file
1193 called "c:(tab)emp(formfeed)oo" or "c:(tab)emp(formfeed)oo.exe" on your
1194 legacy DOS filesystem.
1195 Either single-quote your strings, or (preferably) use forward slashes.
1197 Since all DOS and Windows versions since something like MS-DOS 2.0 or
1198 so have treated "/" and "\" the same in a path, you might as well use
1199 the one that doesn't clash with Perl--or the POSIX shell, ANSI C and
1200 C++, awk, Tcl, Java, or Python, just to mention a few. POSIX paths are
1201 more portable, too.
1202 Why doesn't glob("*.*") get all the files?
1204 Because even on non-Unix ports, Perl's glob function follows standard
1205 Unix globbing semantics. You'll need "glob("*")" to get all (non-
1206 hidden) files. This makes glob() portable even to legacy systems.
1207 Your port may include proprietary globbing functions as well. Check
1208 its documentation for details.
1209 Why does Perl let me delete read-only files? Why does "-i" clobber
1211 protected files? Isn't this a bug in Perl?
1212 This is elaborately and painstakingly described in the file-dir-perms
1213 article in the "Far More Than You Ever Wanted To Know" collection in
1214 http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz .
1215 The executive summary: learn how your filesystem works. The
1217 permissions on a file say what can happen to the data in that file.
1218 The permissions on a directory say what can happen to the list of files
1219 in that directory. If you delete a file, you're removing its name from
1220 the directory (so the operation depends on the permissions of the
1221 directory, not of the file). If you try to write to the file, the
1222 permissions of the file govern whether you're allowed to.
1223 How do I select a random line from a file?
1225 Short of loading the file into a database or pre-indexing the lines in
1226 the file, there are a couple of things that you can do.
1227 Here's a reservoir-sampling algorithm from the Camel Book:
1229 srand;
1231 rand($.) < 1 && ($line = $_) while <>;
1232 This has a significant advantage in space over reading the whole file
1234 in. You can find a proof of this method in The Art of Computer
1235 Programming, Volume 2, Section 3.4.2, by Donald E. Knuth.
1236 You can use the "File::Random" module which provides a function for
1238 that algorithm:
1239 use File::Random qw/random_line/;
1241 my $line = random_line($filename);
1242 Another way is to use the "Tie::File" module, which treats the entire
1244 file as an array. Simply access a random array element.
1245 Why do I get weird spaces when I print an array of lines?
1247 (contributed by brian d foy)
1248 If you are seeing spaces between the elements of your array when you
1250 print the array, you are probably interpolating the array in double
1251 quotes:
1252 my @animals = qw(camel llama alpaca vicuna);
1254 print "animals are: @animals\n";
1255 It's the double quotes, not the "print", doing this. Whenever you
1257 interpolate an array in a double quote context, Perl joins the elements
1258 with spaces (or whatever is in $", which is a space by default):
1259 animals are: camel llama alpaca vicuna
1261 This is different than printing the array without the interpolation:
1263 my @animals = qw(camel llama alpaca vicuna);
1265 print "animals are: ", @animals, "\n";
1266 Now the output doesn't have the spaces between the elements because the
1268 elements of @animals simply become part of the list to "print":
1269 animals are: camelllamaalpacavicuna
1271 You might notice this when each of the elements of @array end with a
1273 newline. You expect to print one element per line, but notice that
1274 every line after the first is indented:
1275 this is a line
1277 this is another line
1278 this is the third line
1279 That extra space comes from the interpolation of the array. If you
1281 don't want to put anything between your array elements, don't use the
1282 array in double quotes. You can send it to print without them:
1283 print @lines;
1285 How do I traverse a directory tree?
1287 (contributed by brian d foy)
1288 The "File::Find" module, which comes with Perl, does all of the hard
1290 work to traverse a directory structure. It comes with Perl. You simply
1291 call the "find" subroutine with a callback subroutine and the
1292 directories you want to traverse:
1293 use File::Find;
1295 find( \&wanted, @directories );
1297 sub wanted {
1299 # full path in $File::Find::name
1300 # just filename in $_
1301 ... do whatever you want to do ...
1302 }
1303 The "File::Find::Closures", which you can download from CPAN, provides
1305 many ready-to-use subroutines that you can use with "File::Find".
1306 The "File::Finder", which you can download from CPAN, can help you
1308 create the callback subroutine using something closer to the syntax of
1309 the "find" command-line utility:
1310 use File::Find;
1312 use File::Finder;
1313 my $deep_dirs = File::Finder->depth->type('d')->ls->exec('rmdir','{}');
1315 find( $deep_dirs->as_options, @places );
1317 The "File::Find::Rule" module, which you can download from CPAN, has a
1319 similar interface, but does the traversal for you too:
1320 use File::Find::Rule;
1322 my @files = File::Find::Rule->file()
1324 ->name( '*.pm' )
1325 ->in( @INC );
1326 How do I delete a directory tree?
1328 (contributed by brian d foy)
1329 If you have an empty directory, you can use Perl's built-in "rmdir". If
1331 the directory is not empty (so, no files or subdirectories), you either
1332 have to empty it yourself (a lot of work) or use a module to help you.
1333 The "File::Path" module, which comes with Perl, has a "rmtree" which
1335 can take care of all of the hard work for you:
1336 use File::Path qw(rmtree);
1338 rmtree( \@directories, 0, 0 );
1340 The first argument to "rmtree" is either a string representing a
1342 directory path or an array reference. The second argument controls
1343 progress messages, and the third argument controls the handling of
1344 files you don't have permissions to delete. See the "File::Path" module
1345 for the details.
1346 How do I copy an entire directory?
1348 (contributed by Shlomi Fish)
1349 To do the equivalent of "cp -R" (i.e. copy an entire directory tree
1351 recursively) in portable Perl, you'll either need to write something
1352 yourself or find a good CPAN module such as File::Copy::Recursive.
1353 =head1 REVISION
1354 Revision: $Revision$
1356 Date: $Date$
1358 See perlfaq for source control details and availability.
1360
1362 Copyright (c) 1997-2009 Tom Christiansen, Nathan Torkington, and other
1363 authors as noted. All rights reserved.
1364 This documentation is free; you can redistribute it and/or modify it
1366 under the same terms as Perl itself.
1367 Irrespective of its distribution, all code examples here are in the
1369 public domain. You are permitted and encouraged to use this code and
1370 any derivatives thereof in your own programs for fun or for profit as
1371 you see fit. A simple comment in the code giving credit to the FAQ
1372 would be courteous but is not required.
1373perl v5.10.1 2009-08-15 PERLFAQ5(1)