1IO(3) User Contributed Perl Documentation IO(3)
2
6 PDL::IO - An overview of the modules in the PDL::IO namespace.
7
9 # At your system shell, type:
10 perldoc PDL::IO
11 # from perldl:
13 pdl> ?? PDL::IO
14
16 PDL contains many modules for displaying, loading, and saving data.
17 • Perlish or Text-Based
19 A few IO modules provide Perl-inspired capabilities. These are
21 PDL::IO::Dumper and PDL::IO::Storable. PDL::IO::Misc provides
22 simpler routines for dealing with delimited files, though its
23 capabilities are limited to tabular or at most 3-d data sets.
24 • Raw Format
26 PDL has two modules that store their data in a raw binary format;
28 they are PDL::IO::FastRaw and PDL::IO::FlexRaw. They are fast but
29 the files they produce will not be readable across different
30 architectures. These two modules are so similar that they could
31 probably be combined.
32 You can also directly access the data from Perl using "get_dataref"
34 in PDL::Core.
35 • Data Browsing
37 At the moment, only PDL::IO::Browser provides data browsing
39 functionality.
40 • Image Handling
42 PDL has a handful of modules that will load images into ndarrays
44 for you. They include PDL::IO::Dicom, PDL::IO::FITS, PDL::IO::GD,
45 PDL::IO::Pic, and PDL::IO::Pnm. However, PDL::IO::FITS should also
46 be considered something of a general data format.
47 • Disk Caching
49 Both PDL::IO::FastRaw and PDL::IO::FlexRaw provide for direct
51 ndarray-to-disk mapping, but they use PDL's underlying mmap
52 functionality to do it, and that doesn't work for Windows.
53 However, users of all operating systems can still use
54 PDL::DiskCache, which can use any desired IO read/write
55 functionality (though you may have to write a small wrapper
56 function).
57 • General Data Storage Formats
59 PDL has a number of modules that interface general data storage
61 libraries. They include PDL::IO::HDF and PDL::IO::NDF (the latter
62 is now a separate CPAN module). There is PDL::IO::IDL.
63 PDL::IO::FITS is something of a general data format, since ndarray
64 data can be stored to a FITS file without loss. PDL::IO::FlexRaw
65 and PDL::IO::FastRaw read and write data identical C's low-level
66 "write" function and PDL::IO::FlexRaw can work with FORTRAN 77
67 UNFORMATTED files. FlexRaw and Storable provide general data
68 storage capabilities. Finally, PDL can read Grib (weather-data)
69 files using the CPAN module PDL::IO::Grib.
70 • Making Movies
72 You can make an MPEG animation using PDL::IO::Pic's wmpeg function.
74 Here's a brief summary of all of the modules, in alphabetical order.
76 PDL::DiskCache
78 The DiskCache module allows you to tie a Perl array to a collection of
79 files on your disk, which will be loaded into and out of memory as
80 ndarrays. Although the module defaults to working with FITS files, it
81 allows you to specify your own reading and writing functions. This
82 allows you to vastly streamline your code by hiding the unnecessary
83 details of loading and saving files.
84 If you find yourself writing scripts to procss many data files,
86 especially if that data processing is not necessarily in sequential
87 order, you should consider using PDL::DiskCache. To read more, check
88 the PDL::DiskCache documentation.
89 PDL::IO::Browser
91 The Browser module provides a text-based data browser for 2D data sets.
92 It uses the CURSES library to do the scrolling, so if your operating
94 system does not have the curses library, you won't be able to install
95 this on your machine. (Note that the package containing the header
96 files for the CURSES library may be called "libcurses" or possibly
97 "libncurses".)
98 To see if the module is installed on your machine (and to get more
100 information about PDL::IO::Browser), follow this link or type at the
101 system prompt:
102 perldoc PDL::IO::Browser
104 If you want to get more information about PDL::IO::Browser and it's not
106 installed on your system, I'm afraid you'll have to pick out the pod
107 from the source file, which can be found online at
108 <https://github.com/PDLPorters/pdl/blob/master/IO/Browser/browser.pd>.
109 PDL::IO::Dicom
111 DICOM is an image format, and this module allows you to read image
112 files with the DICOM file format. To read more, check the
113 PDL::IO::Dicom documentation.
114 PDL::IO::Dumper
116 Provides functionality similar to Data::Dumper for ndarrays.
117 Data::Dumper stringifies a data structure, creating a string that can
118 be "eval"ed to reproduce the original data structure. It's also
119 usually suitable for printing, to visualize the structure.
120 To read more, check the PDL::IO::Dumper documentation. See also
122 PDL::IO::Storable for a more comprehensive structured data solution.
123 PDL::IO::FastRaw
125 Very simple module for quickly writing, reading, and memory-mapping
126 ndarrays to/from disk. It is fast to learn and fast to use, though you
127 may be frustrated by its lack of options. To quote from the original
128 POD:
129 "The binary files are in general NOT interchangeable between different
131 architectures since the binary file is simply dumped from the memory
132 region of the ndarray. This is what makes the approach efficient."
133 This creates two files for every ndarray saved - one that stores the
135 raw data and another that stores the header file, which indicates the
136 dimensions of the data stored in the raw file. Even if you save 1000
137 different ndarrays with the exact same dimensions, you will still need
138 to write out a header file for each one. You cannot store multiple
139 ndarrays in one file.
140 Note that at the time of writing, memory-mapping is not possible on
142 Windows.
143 For more details, see PDL::IO::FastRaw. For a more flexible raw IO
145 module, see PDL::IO::FlexRaw.
146 PDL::IO::FITS
148 Allows basic reading and writing of FITS files. You can read more
149 about FITS formatted files at
150 <http://fits.gsfc.nasa.gov/fits_intro.html> and
151 <http://en.wikipedia.org/wiki/FITS>. It is an image format commonly
152 used in Astronomy.
153 This module may or may not be installed on your machine. To get more
155 information, check online at
156 <http://pdl.perl.org/?docs=IO/FITS&title=PDL::IO::FITS>. To see if the
157 module is installed, look for PDL::IO::FITS on your machine by typing
158 at the system prompt:
159 perldoc PDL::IO::FITS
161 PDL::IO::FlexRaw
163 Somewhat smarter module (compared to FastRaw) for reading, writing, and
164 memory mapping ndarrays to disk. In addition to everything that
165 FastRaw can do, FlexRaw can also store multiple ndarrays in a single
166 file, take user-specified headers (so you can use one header file for
167 multiple files that have identical structure), and read compressed
168 data. However, FlexRaw cannot memory-map compressed data, and just as
169 with FastRaw, the format will not work across multiple architectures.
170 FlexRaw and FastRaw produce identical raw files and have essentially
172 identical performance. Use whichever module seems to be more
173 comfortable. I would generally recommend using FlexRaw over FastRaw,
174 but the differences are minor for most uses.
175 Note that at the time of writing, memory-mapping is not possible on
177 Windows.
178 For more details on FlexRaw, see PDL::IO::FlexRaw.
180 PDL::IO::GD
182 GD is a library for reading, creating, and writing bitmapped images,
183 written in C. You can read more about the C-library here:
184 <http://www.libgd.org/>.
185 In addition to reading and writing .png and .jpeg files, GD allows you
187 to modify the bitmap by drawing rectangles, adding text, and probably
188 much more. The documentation can be found here. As such, it should
189 probably be not only considered an IO module, but a Graphics module as
190 well.
191 This module provides PDL bindings for the GD library, which ought not
193 be confused with the Perl bindings. The perl bindings were developed
194 independently and can be found at GD, if you have Perl's GD bindings
195 installed.
196 PDL::IO::Grib
198 A CPAN module last updated in 2000 that allows you to read Grib files.
199 GRIB is a data format commonly used in meteorology. In the off-chance
200 that you have it installed, you should read PDL::IO::Grib's
201 documentation.
202 PDL::IO::HDF, PDL::IO::HDF5
204 Provides an interface to HDF4 and HDF5 file formats, which are kinda
205 like cross-platform binary XML files. HDF stands for Hierarchical Data
206 Format. HDF was originally developed at the NCSA. To read more about
207 HDF, see <http://www.hdfgroup.org/>. Note that HDF5 is not presently
208 distributed with PDL, and neither HDF4 nor HDF5 will be installed
209 unless you have the associated C libraries that these modules
210 interface. Also note that the HDF5 library on CPAN is rather old and
211 somebody from HDF contacted the mailing list in the Fall of 2009 to
212 develop new and better HDF5 bindings for Perl.
213 You should look into the PDL::IO::HDF (4) documentation or
215 PDL::IO::HDF5 documentation, depending upon which module you have
216 installed.
217 PDL::IO::IDL
219 PDL has a module for reading IDL data files: PDL::IO::IDL.
220 PDL::IO::Misc
222 Provides mostly text-based IO routines. Data input and output is
223 restricted mostly to tabular (i.e. two-dimensional) data sets, though
224 limited support is provided for 3d data sets.
225 Alternative text-based modules support higher dimensions, such as
227 PDL::IO::Dumper and PDL::IO::Storable. Check the PDL::IO::Misc
228 documentation for more details.
229 PDL::IO::NDF
231 Starlink developed a file format for N-Dimensional data Files, which it
232 cleverly dubbed NDF. If you work with these files, you're in luck!
233 Check the PDL::IO::NDF documentation for more details.
234 PDL::IO::Pic
236 Provides reading/writing of images to/from ndarrays, as well as
237 creating MPEG animations! The module uses the netpbm library, so you
238 will need that on your machine in order for this to work. To read
239 more, see the PDL::IO::Pic documentation. Also look into the next
240 module, as well as PDL::IO::GD.
241 PDL::IO::Pnm
243 Provides methods for reading and writing pnm files (of which pbm is but
244 one). Check the PDL::IO::Pnm documentation for more details. Also
245 check out the previous module and PDL::IO::GD.
246 PDL::IO::STL
248 Read and write STL (STereo Lithography) files, containing 3D objects.
249 There are many files available from the "Thingiverse". You can then
250 view the data in PDL::Graphics::TriD.
251 PDL::IO::Storable
253 Implements the relevant methods to be able to store and retrieve
254 ndarrays via Storable. True, you can use many methods to save a single
255 ndarray. In contrast, this module is particularly useful if you need
256 to save a complex Perl structure that contain ndarrays, such as an
257 array of hashes, each of which contains ndarrays.
258 Check the PDL::IO::Storable documentation for more details. See also
260 PDL::IO::Dumper for an alternative stringifier.
261
263 PDL::IO::CSV
264 Load/save PDL from/to CSV file (optimized for speed and large data).
265 See the PDL::IO::CSV documentation.
267 PDL::IO::DBI
269 Load PDL from a DBI handle. See PDL::IO::CSV.
270 PDL::IO::Dcm
272 Load PDL from a Dicom file. See PDL::IO::Dcm.
273 PDL::IO::Touchstone
275 A simple interface for reading and writing RF Touchstone files (also
276 known as ".sNp" files). Touchstone files contain complex-valued RF
277 sample data for a device or RF component with some number of ports. The
278 data is (typically) measured by a vector network analyzer under
279 stringent test conditions.
280 The resulting files are usually provided by manufacturers so RF design
282 engineers can estimate signal behavior at various frequencies in their
283 circuit designs. Examples of RF components include capacitors,
284 inductors, resistors, filters, power splitters, etc.
285 See the PDL::IO::Touchstone documentation.
287 PDL::IO::MDIF
289 A simple interface for reading and writing RF MDIF files (also known as
290 MDF or .mdf files). MDIF files contain multiple Touchstone files in a
291 text format for use in optimizing circuits. For example, a single MDIF
292 file could contain the Touchstone RF data for each available value in a
293 line of capacitors (ie, from 10pF to 1000pF) provided by a particular
294 manufacturer.
295 See the PDL::IO::MDIF documentation
297 PDL::IO::Image
299 Load/save PDL from/to image files. See PDL::IO::Image.
300 PDL::IO::XLSX
302 Load/save PDL from/to an Excel spreadsheet. See PDL::IO::XLSX.
303 PDL::NetCDF
305 Load/save PDL from/to NetCDF files. See PDL::NetCDF.
306 PDL::IO::Nifti
308 Load/save PDL from/to Nifti-1 files. See PDL::IO::Nifti.
309 PDL::IO::Matlab
311 Load/save PDL from/to MATLAB files. See PDL::IO::Matlab.
312 PDL::IO::Sereal
314 Load/save PDL from/to Sereal files. See PDL::IO::Sereal.
315 PDL::CCS
317 Has IO modules to load/save PDL from/to sparse data file formats,
318 including FITS, LDA-C, MatrixMarket, PETSc. See PDL::CCS.
319
321 Copyright 2010 David Mertens (dcmertens.perl@gmail.com). You can
322 distribute and/or modify this document under the same terms as the
323 current Perl license.
324 See: <http://dev.perl.org/licenses/>
326perl v5.38.0 2023-07-21 IO(3)