1dirfile(5) DATA FORMATS dirfile(5)
2
6 dirfile — a filesystem based database format for time-ordered binary
7 data
8
10 The dirfile database format is designed to provide a fast, simple for‐
11 mat for storing and reading binary time-ordered data. Dirfiles can be
12 read using the GetData library.
13 The dirfile database is centred around one or more time-ordered data
15 streams (a time stream). Each time stream is written to disk in a sep‐
16 arate file, in its native binary format. The name of these binary
17 files correspond to the time stream's field name. Two time streams may
18 have different constant sampling frequencies and mechanisms exist
19 within the dirfile format to ensure these time streams remain properly
20 sequenced in time.
21 To do this, the time streams in the dirfile are subdivided into frames.
23 Each frame contains an integer number of samples of each time stream.
24 When synchronous retrieval of data from more than one time stream is
25 required, position in the dirfile can be specified in frames, which
26 will ensure synchronicity.
27 The binary files are all located in a central directory, known as the
29 dirfile directory. The dirfile as a whole may be referred to by its
30 dirfile directory path.
31 Included in the dirfile along with the time streams is the dirfile for‐
33 mat specification, which is an ASCII text file called format located in
34 the dirfile directory. This file fully specifies the dirfile's meta‐
35 data. For the syntax of this file, see dirfile-format(5).
36 Version 3 of the Dirfile Standards introduced the large dirfile exten‐
38 sion. This extension added the ability to distribute the dirfile meta‐
39 data among multiple files (called fragments) in addition to the format
40 file, as well as the ability to house portions of the database in sub‐
41 dirfiles. These subdirfiles may be fully fledged dirfiles in their own
42 right, but may also be contained within a larger, parent dirfile. See
43 dirfile-format(5) for information on specifying these subdirfiles.
44 In addition to the raw fields on disk, the dirfile format specification
46 may also specify derived fields which are calculated from one or more
47 raw or derived time streams. Derived fields behave identically to raw
48 fields when read via GetData. See dirfile-format(5) for a complete
49 list of derived field types. Dirfiles may also contain both numerical
50 and character string constant scalar fields, also further outlined in
51 dirfile-format(5).
52 Dirfiles are designed to be written to and read simultaneously. The
54 dirfile specification dictates that one particular raw field (specified
55 either explicitly or implicitly by the format file) is to be used as
56 the reference field: all other vector fields are assumed to have at
57 least as many frames as the reference field has, and the size (in
58 frames) of the reference field is used as the size of the dirfile as a
59 whole.
60 Version 6 of the Dirfile Standards added the ability to encode the
62 binary files on disk. Each fragment may have its own encoding scheme.
63 Notably this can be used to compress these files. See dirfile-encod‐
64 ing(5) for information on encoding schemes.
65 Complex Number Storage Format
68 Version 7 of the Dirfile Standards added support for complex valued
69 data. Two types of complex valued data are supported by the Dirfile
70 Standards:
71 · A 64-bit complex number consisting of a IEEE-754 standard 32-bit
73 single precision floating point real part and a IEEE-754 standard
74 32-bit single precision floating point imaginary part, and
75 · A 128-bit complex number consisting of a IEEE-754 standard 64-bit
77 double precision floating point real part and a IEEE-754 standard
78 64-bit double precision floating point imaginary part.
79 No integer-type complex numbers are supported.
81 Unencoded complex numbers are stored on disk in "Fortran order", that
83 is with the IEEE-754 real part followed by the IEEE-754 imaginary part.
84 The specified endianness of the two components follows that of purely
85 real floating point numbers. Endianness does not affect the ordering
86 of the real and imaginary parts. This format also conforms to the C99
87 standard. The latest C++ standard (C++98) does not specify a standard
88 storage format for native complex numbers, but the upcoming standard,
89 (C++0x) is intended to specify the above format for compatibility with
90 C99 (see: ISO/IEC JTC1/SC22/WG21/N1388).
91 To aid in using complex valued data, dirfile field codes may contain a
93 representation suffix which specifies a norm to apply to the complex
94 valued data to convert it into purely real data. See dirfile-for‐
95 mat(5).
96
99 The dirfile specification was developed by C. B. Netterfield
100 <netterfield@astro.utoronto.ca>.
101 The dirfile specification is now maintained by D. V. Wiebe
103 <dvw@ketiltrout.net>.
104
107 dirfile-encoding(5), dirfile-format(5)
108Standards Version 7 9 February 2010 dirfile(5)