1X2SYS_INIT(1) Generic Mapping Tools X2SYS_INIT(1)
2
6 x2sys_init - Initialize data base for track data files
7
9 x2sys_init TAG -Ddeffile [ -Esuffix ] [ -F ] [ -Gd|g ] [ -Idx[/dy] ] [
10 -M[i|o][flag] ] [ -Rwest/east/south/north[r] ] [ -V ] [ -Wt|dgap ]
11
13 x2sys_init initializes a set of data bases that are particular to one
14 kind of track data. These data, their associated data bases, and key
15 parameters are given a short-hand notation called a system TAG. The
16 TAG remembers settings such as file format, whether the data are geo‐
17 graphic or not, and the binning resolution for track indexes. Running
18 x2sys_init is a prerequisite to running x2sys_binlist which will create
19 a crude representation of where data track go and which observations
20 are available; this information serves as input to x2sys_put which
21 updates the track data base. Then, x2sys_get can be used to find which
22 tracks and data are available inside a given region.
23 TAG The unique name of this data type system.
25 -D Definition file for this data set.
27
29 No space between the option flag and the associated arguments.
30 -E Specifies the file extension (suffix) for these data files. If
32 not given we use the definition file prefix as the suffix (see
33 -D).
34 -F Force creating new files if old ones are present [Default will
36 abort if old files are found].
37 -G Selects geographical coordinates. Append d for discontinuity at
39 the Dateline (makes longitude go from -180 to + 180) or g for
40 discontinuity at Greenwhich (makes longitude go from 0 to 360
41 [Default]).
42 -I x_inc [and optionally y_inc] is the grid spacing. Append m to
44 indicate minutes or c to indicate seconds.
45 -M Multiple segment file(s). Segments are separated by a special
47 record. For ASCII files the first character must be flag
48 [Default is '>']. For binary files all fields must be NaN.
49 -R west, east, south, and north specify the Region of interest, and
51 you may specify them in decimal degrees or in
52 [+-]dd:mm[:ss.xxx][W|E|S|N] format. Append r if lower left and
53 upper right map coordinates are given instead of w/e/s/n. The
54 two shorthands -Rg and -Rd stand for global domain (0/360 and
55 -180/+180 in longitude respectively, with -90/+90 in latitude).
56 -V Selects verbose mode, which will send progress reports to stderr
58 [Default runs "silently"].
59 -W Give t or d and append the corresponding maximum time gap (in
61 user units; this is typically seconds [Infinity]), or distance
62 gap (in user units; this is typically km [Infinity] ) allowed
63 between the two points immediately on either side of a cross‐
64 over.
65
67 The *.def files contain information about the data file format. The
68 *.def files have two sections: (1) header information and (2) column
69 information. All header information starts with the character # in the
70 first column, immediately followed by an upper-case directive. If the
71 directive takes an argument it is separated by white-space. You may
72 append a trailing # comments. Five directives are recognized:
73 ASCII states that the data files are in ASCII format.
75 BINARY states that the data files are binary files.
76 SKIP takes an integer argument which is either the number of lines to
77 skip (when reading ASCII files) or the number of bytes to skip (when
78 reading binary files).
79 GEO indicates that these are geographic data set, with periodicities in
80 the x-coordinate (longitudes).
81 MULTISEG means each track consists of multiple segments separated by a
82 GMT multisegment header (alternatively, use -M when defining the system
83 TAG).
84 The column information consists of one line per column in the order the
86 columns appear in the data file. For each column you must provide
87 seven attributes:
88 name type NaN NaN-proxy scale offset oformat
90 name is the name of the column variable. It is expected that you will
92 use the special names lon (or x if Cartesian) and lat (or y) for the
93 two required coordinate columns, and time when optional time data are
94 present.
95 type is a for ASCII representations of numbers, whereas for binary
96 files you may choose from c for signed 1-byte character (-127,+128), u
97 for unsigned byte (0-255), h for signed 2-byte integers
98 (-32768,+32767), i for signed 4-byte integers
99 (-2,147,483,648,+2,147,483,647), f for 4-byte floating points and d for
100 8-byte double precision floating points.
101 NaN is Y if certain values are to be replaced by NAN, and N otherwise.
102 NaN-proxy is that special value.
103 scale is used to multiply the data after reading.
104 offset is used to add to the scaled data.
105 oformat is a C-style format string used to print values from this col‐
106 umn.
107 For ASCII data the type format is always a. NaN should be Y if there is
109 a certain value (e.g., -9999) that is used to represent missing data
110 (NaNs). If so, the NaN-proxy value should be set accordingly (other‐
111 wise it is ignored). The scale and offset are applied to the data val‐
112 ues after they are read (first scaled, then offset), whereas the ofor‐
113 mat is used by some programs for formatted output. If you give - as
114 the oformat then GMT's formatting machinery will be used instead (i.e.,
115 PLOT_DEGREE_FORMAT, PLOT_DATE_FORMAT, PLOT_CLOCK_FORMAT). Some file
116 formats already have definition files premade. These include mgd77
117 (for plain ASCII MGD77 data files), mgd77+ (for enhanced MGD77+ netCDF
118 files), gmt (for old mgg supplement binary files), xy (for plain ASCII
119 x, y tables), xyz (same, with z-column), geo (for plain ASCII longi‐
120 tude, latitude files), and geoz (same, with z-column).
121
123 If you have a large set of track data files you can organize them using
124 the x2sys tools. Here we will outline the steps. Let us assume that
125 your track data file format consist of 2 header records with text
126 information followed by any number of identically formatted data
127 records with 6 columns (lat, lon, time, obs1, obs2, obs3) and that
128 files are called *.trk. We will call this the "line" format. First, we
129 create the line.def file:
130 # Define file for the line format
132 #ASCII # File is ASCII
133 #SKIP 2 # Skip 2 header records
134 #name type NaN NaN-proxy scale offset oformat
135 lon a N 0 1 0 %10.5f
136 lat a N 0 1 0 %9.5f
137 time a N 0 1 0 %7.1f
138 obs1 a N 0 1 0 %7.2f
139 obs2 a N 0 1 0 %7.2f
140 obs3 a N 0 1 0 %7.2f
141 Next we create the TAG and the databases for these line track files.
143 Assuming these contain geographic data and that we want to keep track
144 of the data distribution at a 1 x 1 degree resolution, we may run
145 x2sys_init LINE -V -G -Dline -R0/360/-90/90 -I1/1 -Etrk
147 where we have used LINE as our system tag. When x2sys tools try to
149 read your line data files they will first look in the current directory
150 and second look in the file TAG_paths.txt for a list of additional
151 directories to examine. Therefore, create such a file (here
152 LINE_paths.txt) and stick the full paths to your data directories
153 there. All TAG-related files (definition files, tag files, and track
154 data bases built) will be expected to be in the directory pointed to by
155 $X2SYS_HOME.
156 Create tbf file(s):
158 Once the (empty) TAG database have been initialized we go
159 through a two-step process to populate them. First we run
160 x2sys_binlist on all our track files to create one (or more)
161 multi-segment track bin-index files (tbf). These contain infor‐
162 mation on which 1 x 1 degree bins (or any other blocksize; see
163 -I) each track has visited and which observations (in your case
164 obs1, obs2, obs3) were actually observed (not all tracks may
165 have all three kinds of observations everywhere). For instance,
166 if your tracks are listed in the file tracks.lis we may run this
167 command:
168 x2sys_binlist -V -TLINE `cat tracks.lis` > tracks.tbf
170 Update index data base:
172 Next, the track bin-index files are fed to x2sys_put which will
173 insert the information into the TAG databases:
174 x2sys_put -V -TLINE tracks.tbf
176 Search for data:
178 You may now use x2sys_get to find all the tracks within a cer‐
179 tain sub-region, and optionally limit the search to those tracks
180 that have a particular combination of observables. E.g., to
181 find all the tracks which has both obs1 and obs3 inside the
182 specified region, run
183 x2sys_get -V -TLINE -R20/40/-40/-20 -Fobs1,obs3 > tracks.tbf
185 MGD77[+] or GMT:
187 Definition files already exist for MGD77 files (both standard
188 ASCII and enhanced netCDF-based MGD77+ files), the old *.gmt
189 files manipulated by the mgg supplements, and plain x,y[,z] and
190 lon,lat[,z] tracks. To initiate new track databases to be used
191 with MGD77 data from NGDC, try
192 x2sys_init MGD77 -V -Dmgd77 -Emgd77 -Rd -Gd -I1/1 -Wt900 -Wd5
194 where we have chosen a 15 minute (900 sec) or 5 km threshold to
196 indicate a data gap; the other steps are similar.
197 Binary files:
199 Let us pretend that your line files actually are binary files
200 with a 128-byte header structure (to be skipped) followed by the
201 data records and where lon, lat, time are double precision num‐
202 bers while the three observations are 2-byte integers which must
203 be multiplied by 0.1. Finally, the first two observations may
204 be -32768 which means there is no data available. All that is
205 needed is a different line.def file:
206 # Define file for the binary line format
208 #BINARY # File is now binary
209 #SKIP 128 # Skip 128 bytes
210 #name type NaN? NaN-proxy scale offset oformat
211 lon d N 0 1 0 %10.5f
212 lat d N 0 1 0 %9.5f
213 time d N 0 1 0 %7.1f
214 obs1 h Y -32768 0.1 0 %6.1f
215 obs2 h Y -32768 0.1 0 %6.1f
216 obs3 h N 0 0.1 0 %6.1f
217 The rest of the steps are identical.
219
221 x2sys_binlist(1) x2sys_get(1) x2sys_put(1)
222GMT 4.3.1 15 May 2008 X2SYS_INIT(1)