1Misc(3)               User Contributed Perl Documentation              Misc(3)
2
3
4

NAME

6       PDL::IO::Misc - misc IO routines for PDL
7

DESCRIPTION

9       Some basic I/O functionality: FITS, tables, byte-swapping
10

SYNOPSIS

12        use PDL::IO::Misc;
13

FUNCTIONS

15   bswap2
16         Signature: (x(); )
17
18       Swaps pairs of bytes in argument x()
19
20       bswap2 does not process bad values.  It will set the bad-value flag of
21       all output piddles if the flag is set for any of the input piddles.
22
23   bswap4
24         Signature: (x(); )
25
26       Swaps quads of bytes in argument x()
27
28       bswap4 does not process bad values.  It will set the bad-value flag of
29       all output piddles if the flag is set for any of the input piddles.
30
31   bswap8
32         Signature: (x(); )
33
34       Swaps octets of bytes in argument x()
35
36       bswap8 does not process bad values.  It will set the bad-value flag of
37       all output piddles if the flag is set for any of the input piddles.
38
39   rcols
40       Read specified ASCII cols from a file into piddles and perl arrays
41       (also see "rgrep").
42
43         Usage:
44           ($x,$y,...) = rcols( *HANDLE|"filename", { EXCLUDE => '/^!/' }, $col1, $col2, ... )
45                    $x = rcols( *HANDLE|"filename", { EXCLUDE => '/^!/' }, [] )
46           ($x,$y,...) = rcols( *HANDLE|"filename", $col1, $col2, ..., { EXCLUDE => '/^!/' } )
47           ($x,$y,...) = rcols( *HANDLE|"filename", "/foo/", $col1, $col2, ... )
48
49       For each column number specified, a 1D output PDL will be generated.
50       Anonymous arrays of column numbers generate 2D output piddles with dim0
51       for the column data and dim1 equal to the number of columns in the
52       anonymous array(s).
53
54       An empty anonymous array as column specification will produce a single
55       output data piddle with dim(1) equal to the number of columns
56       available.
57
58       There are two calling conventions - the old version, where a pattern
59       can be specified after the filename/handle, and the new version where
60       options are given as as hash reference.  This reference can be given as
61       either the second or last argument.
62
63       The default behaviour is to ignore lines beginning with a # character
64       and lines that only consist of whitespace.  Options exist to only read
65       from lines that match, or do not match, supplied patterns, and to set
66       the types of the created piddles.
67
68       Can take file name or *HANDLE, and if no explicit column numbers are
69       specified, all are assumed. For the allowed types, see
70       "Datatype_conversions" in PDL::Core.
71
72       Options (case insensitive):
73
74         EXCLUDE or IGNORE
75         - ignore lines matching this pattern (default B<'/^#/'>).
76
77         INCLUDE or KEEP
78         - only use lines which match this pattern (default B<''>).
79
80         LINES
81         - a string pattern specifying which line numbers to use.
82         Line numbers start at 0 and the syntax is 'a:b:c' to use
83         every c'th matching line between a and b (default B<''>).
84
85         DEFTYPE
86         - default data type for stored data (if not specified, use the type
87         stored in C<$PDL::IO::Misc::deftype>, which starts off as B<double>).
88
89         TYPES
90         - reference to an array of data types, one element for each column
91         to be read in.  Any missing columns use the DEFTYPE value (default B<[]>).
92
93         COLSEP
94         - splits on this string/pattern/qr{} between columns of data. Defaults to
95         $PDL::IO::Misc::defcolsep.
96
97         PERLCOLS
98         - an array of column numbers which are to be read into perl arrays
99         rather than piddles.  Any columns not specified in the explicit list
100         of columns to read will be returned after the explicit columns.
101         (default B<undef>).
102
103         COLIDS
104         - if defined to an array reference, it will be assigned the column
105         ID values obtained by splitting the first line of the file in the
106         identical fashion to the column data.
107
108         CHUNKSIZE
109         - the number of input data elements to batch together before appending
110         to each output data piddle (Default value is 100).  If CHUNKSIZE is
111         greater than the number of lines of data to read, the entire file is
112         slurped in, lines split, and perl lists of column data are generated.
113         At the end, effectively pdl(@column_data) produces any result piddles.
114
115         VERBOSE
116         - be verbose about IO processing (default C<$PDL::vebose>)
117
118       For example:
119
120         $x      = PDL->rcols 'file1';         # file1 has only one column of data
121         $x      = PDL->rcols 'file2', [];     # file2 can have multiple columns, still 1 piddle output
122                                               # (empty array ref spec means all possible data fields)
123
124         ($x,$y) = rcols 'table.csv', { COLSEP => ',' };  # read CSV data file
125         ($x,$y) = rcols *STDOUT;  # default separator for lines like '32 24'
126
127         # read in lines containing the string foo, where the first
128         # example also ignores lines that begin with a # character.
129         ($x,$y,$z) = rcols 'file2', 0,4,5, { INCLUDE => '/foo/' };
130         ($x,$y,$z) = rcols 'file2', 0,4,5, { INCLUDE => '/foo/', EXCLUDE => '' };
131
132         # ignore the first 27 lines of the file, reading in as ushort's
133         ($x,$y) = rcols 'file3', { LINES => '27:-1', DEFTYPE => ushort };
134         ($x,$y) = rcols 'file3', { LINES => '27:', TYPES => [ ushort, ushort ] };
135
136         # read in the first column as a perl array and the next two as piddles
137         # with the perl column returned after the piddle outputs
138         ($x,$y,$name) = rcols 'file4', 1, 2   , { PERLCOLS => [ 0 ] };
139         printf "Number of names read in = %d\n", 1 + $#$name;
140
141         # read in the first column as a perl array and the next two as piddles
142         # with PERLCOLS changing the type of the first returned value to perl list ref
143         ($name,$x,$y) = rcols 'file4', 0, 1, 2, { PERLCOLS => [ 0 ] };
144
145         # read in the first column as a perl array returned first followed by the
146         # the next two data columns in the file as a single Nx2 piddle
147         ($name,$xy) = rcols 'file4', 0, [1, 2], { PERLCOLS => [ 0 ] };
148
149
150         NOTES:
151
152         1. Quotes are required on patterns or use the qr{} quote regexp syntax.
153
154         2. Columns are separated by whitespace by default, use the COLSEP option
155            separator to specify an alternate split pattern or string or specify an
156            alternate default separator by setting C<$PDL::IO::Misc::defcolsep> .
157
158         3. Legacy support is present to use C<$PDL::IO::Misc::colsep> to set the
159            column separator but C<$PDL::IO::Misc::colsep> is not defined by default.
160            If you set the variable to a defined value it will get picked up.
161
162         4. LINES => '-1:0:3' may not work as you expect, since lines are skipped
163            when read in, then the whole array reversed.
164
165         5. For consistency with wcols and rcols 1D usage, column data is loaded
166            into the rows of the pdls (i.e., dim(0) is the elements read per column
167            in the file and dim(1) is the number of columns of data read.
168
169   wcols
170         Write ASCII columns into file from 1D or 2D piddles and/or 1D listrefs efficiently.
171
172       Can take file name or *HANDLE, and if no file/filehandle is given
173       defaults to STDOUT.
174
175         Options (case insensitive):
176
177           HEADER - prints this string before the data. If the string
178                    is not terminated by a newline, one is added. (default B<''>).
179
180           COLSEP - prints this string between columns of data. Defaults to
181                    $PDL::IO::Misc::defcolsep.
182
183           FORMAT - A printf-style format string that is cycled through
184                    column output for user controlled formatting.
185
186        Usage: wcols $data1, $data2, $data3,..., *HANDLE|"outfile", [\%options];  # or
187               wcols $format_string, $data1, $data2, $data3,..., *HANDLE|"outfile", [\%options];
188
189          where the $dataN args are either 1D piddles, 1D perl array refs,
190          or 2D piddles (as might be returned from rcols() with the [] column
191          syntax and/or using the PERLCOLS option).  dim(0) of all piddles
192          written must be the same size.  The printf-style $format_string,
193          if given, overrides any FORMAT key settings in the option hash.
194
195       e.g.,
196
197         $x = random(4); $y = ones(4);
198         wcols $x, $y+2, 'foo.dat';
199         wcols $x, $y+2, *STDERR;
200         wcols $x, $y+2, '|wc';
201
202         $a = sequence(3); $b = zeros(3); $c = random(3);
203         wcols $a,$b,$c; # Orthogonal version of 'print $a,$b,$c' :-)
204
205         wcols "%10.3f", $a,$b; # Formatted
206         wcols "%10.3f %10.5g", $a,$b; # Individual column formatting
207
208         $a = sequence(3); $b = zeros(3); $units = [ 'm/sec', 'kg', 'MPH' ];
209         wcols $a,$b, { HEADER => "#   a   b" };
210         wcols $a,$b, { Header => "#   a   b", Colsep => ', ' };  # case insensitive option names!
211         wcols " %4.1f  %4.1f  %s",$a,$b,$units, { header => "# Day  Time  Units" };
212
213         $a52 = sequence(5,2); $b = ones(5); $c = [ 1, 2, 4 ];
214         wcols $a52;         # now can write out 2D pdls (2 columns data in output)
215         wcols $b, $a52, $c  # ...and mix and match with 1D listrefs as well
216
217         NOTES:
218
219         1. Columns are separated by whitespace by default, use
220            C<$PDL::IO::Misc::defcolsep> to modify the default value or
221            the COLSEP option
222
223         2. Support for the C<$PDL::IO::Misc::colsep> global value
224            of PDL-2.4.6 and earlier is maintained but the initial value
225            of the global is undef until you set it.  The value will be
226            then be picked up and used as if defcolsep were specified.
227
228         3. Dim 0 corresponds to the column data dimension for both
229            rcols and wcols.  This makes wcols the reverse operation
230            of rcols.
231
232   swcols
233       generate string list from "sprintf" format specifier and a list of
234       piddles
235
236       "swcols" takes an (optional) format specifier of the printf sort and a
237       list of 1D piddles as input. It returns a perl array (or array
238       reference if called in scalar context) where each element of the array
239       is the string generated by printing the corresponding element of the
240       piddle(s) using the format specified. If no format is specified it uses
241       the default print format.
242
243        Usage: @str = swcols format, pdl1,pdl2,pdl3,...;
244           or  $str = swcols format, pdl1,pdl2,pdl3,...;
245
246   rgrep
247         Read columns into piddles using full regexp pattern matching.
248
249
250         Options:
251
252         UNDEFINED: This option determines what will be done for undefined
253         values. For instance when reading a comma-separated file of the type
254         C<1,2,,4> where the C<,,> indicates a missing value.
255
256         The default value is to assign C<$PDL::undefval> to undefined values,
257         but if C<UNDEFINED> is set this is used instead. This would normally
258         be set to a number, but if it is set to C<Bad> and PDL is compiled
259         with Badvalue support (see L<PDL::Bad/>) then undefined values are set to
260         the appropriate badvalue and the column is marked as bad.
261
262         DEFTYPE: Sets the default type of the columns - see the documentation for
263          L</rcols()>
264
265         TYPES:   A reference to a Perl array with types for each column - see
266         the documentation for L</rcols()>
267
268         BUFFERSIZE: The number of lines to extend the piddle by. It might speed
269         up the reading a little bit by setting this to the number of lines in the
270         file, but in general L</rasc()> is a better choice
271
272       Usage
273
274        ($x,$y,...) = rgrep(sub, *HANDLE|"filename")
275
276       e.g.
277
278        ($a,$b) = rgrep {/Foo (.*) Bar (.*) Mumble/} $file;
279
280       i.e. the vectors $a and $b get the progressive values of $1, $2 etc.
281
282   rdsa
283         Read a FIGARO/NDF format file.
284
285         Requires non-PDL DSA module. Contact Frossie (frossie@jach.hawaii.edu)
286
287       Usage:
288
289        ([$xaxis],$data) = rdsa($file)
290
291        $a = rdsa 'file.sdf'
292
293       Not yet tested with PDL-1.9X versions
294
295   isbigendian
296         Determine endianness of machine - returns 0 or 1 accordingly
297
298   rasc
299         Simple function to slurp in ASCII numbers quite quickly,
300         although error handling is marginal (to nonexistent).
301
302         $pdl->rasc("filename"|FILEHANDLE [,$noElements]);
303
304             Where:
305               filename is the name of the ASCII file to read or open file handle
306               $noElements is the optional number of elements in the file to read.
307                   (If not present, all of the file will be read to fill up $pdl).
308               $pdl can be of type float or double (for more precision).
309
310         #  (test.num is an ascii file with 20 numbers. One number per line.)
311         $in = PDL->null;
312         $num = 20;
313         $in->rasc('test.num',20);
314         $imm = zeroes(float,20,2);
315         $imm->rasc('test.num');
316
317   rcube
318        Read list of files directly into a large data cube (for efficiency)
319
320        $cube = rcube \&reader_function, @files;
321
322        $cube = rcube \&rfits, glob("*.fits");
323
324       This IO function allows direct reading of files into a large data cube,
325       Obviously one could use cat() but this is more memory efficient.
326
327       The reading function (e.g. rfits, readfraw) (passed as a reference) and
328       files are the arguments.
329
330       The cube is created as the same X,Y dims and datatype as the first
331       image specified. The Z dim is simply the number of images.
332

AUTHOR

334       Copyright (C) Karl Glazebrook 1997, Craig DeForest 2001, 2003, and
335       Chris Marshall 2010. All rights reserved. There is no warranty. You are
336       allowed to redistribute this software / documentation under certain
337       conditions. For details, see the file COPYING in the PDL distribution.
338       If this file is separated from the PDL distribution, the copyright
339       notice should be included in the file.
340
341
342
343perl v5.28.0                      2018-07-23                           Misc(3)
Impressum