1X2SYS_INIT(1) Generic Mapping Tools X2SYS_INIT(1)
2
6 x2sys_init - Initialize x2sys data base for track data files
7
9 x2sys_init TAG -Ddeffile [ -Cc|f|g|e ] [ -Esuffix ] [ -F ] [ -Gd|g ] [
10 -Idx[/dy] ] [ -Nd|sunit ] [ -Rwest/east/south/north[r] ] [ -V ] [
11 -Wt|dgap ] [ -m[i|o][flag] ]
12
14 x2sys_init is the starting point for anyone wishing to use x2sys; it
15 initializes a set of data bases that are particular to one kind of
16 track data. These data, their associated data bases, and key parame‐
17 ters are given a short-hand notation called an x2sys TAG. The TAG
18 keeps track of settings such as file format, whether the data are geo‐
19 graphic or not, and the binning resolution for track indices. Running
20 x2sys_init is a prerequisite to running any of the other x2sys pro‐
21 grams, such as x2sys_binlist, which will create a crude representation
22 of where each data track go within the domain and which observations
23 are available; this information serves as input to x2sys_put which
24 updates the track data base. Then, x2sys_get can be used to find which
25 tracks and data are available inside a given region. With that list of
26 tracks you can use x2sys_cross to calculate track crossovers, use
27 x2sys_report to report crossover statistics or x2sys_list to pull out
28 selected crossover information that x2sys_solve can use to determine
29 track-specific systematic corrections. These corrections may be used
30 with x2sys_datalist to extract corrected data values for use in subse‐
31 quent work.
32 TAG The unique name of this data type x2sys TAG.
34 -C Select procedure for along-track distance calculation when
36 needed by other programs:
37 c Cartesian distances [Default, unless -G is set].
38 f Flat Earth distances.
39 g Great circle distances [Default if -G is set].
40 e Geodesic distances on current GMT ellipsoid.
41 -D Definition file prefix for this data set [See DEFINITION FILES
43 below for more information]. Specify full path if the file is
44 not in the current directory.
45
47 No space between the option flag and the associated arguments.
48 -E Specifies the file extension (suffix) for these data files. If
50 not given we use the definition file prefix as the suffix (see
51 -D).
52 -F Force creating new files if old ones are present [Default will
54 abort if old TAG files are found].
55 -G Selects geographical coordinates. Append d for discontinuity at
57 the Dateline (makes longitude go from -180 to + 180) or g for
58 discontinuity at Greenwich (makes longitude go from 0 to 360
59 [Default]). If not given we assume the data are Cartesian.
60 -I x_inc [and optionally y_inc] is the grid spacing. Append m to
62 indicate minutes or c to indicate seconds for geographic data.
63 These spacings refer to the binning used in the track bin-index
64 data base.
65 -m Multiple segment file(s). Segments are separated by a special
67 record. For ASCII files the first character must be flag
68 [Default is '>']. For binary files all fields must be NaN and
69 -b must set the number of output columns explicitly. By default
70 the -m setting applies to both input and output. Use -mi and
71 -mo to give separate settings to input and output.
72 -N Sets the units used for distance and speed when requested by
74 other programs. Append d for distance or s for speed, then give
75 the desired unit as c (Cartesian userdist or userdist/usertime),
76 e (meter or m/s), k (km or km/hr), m (miles or miles/hr), or n
77 (nautical miles or knots). [Default is -Ndk -Nse (km and m/s)
78 if -G is set and -Ndc and -Nsc otherwise (Cartesian units)].
79 -R west, east, south, and north specify the Region of interest, and
81 you may specify them in decimal degrees or in
82 [+-]dd:mm[:ss.xxx][W|E|S|N] format. Append r if lower left and
83 upper right map coordinates are given instead of w/e/s/n. The
84 two shorthands -Rg and -Rd stand for global domain (0/360 and
85 -180/+180 in longitude respectively, with -90/+90 in latitude).
86 Alternatively, specify the name of an existing grid file and the
87 -R settings (and grid spacing, if applicable) are copied from
88 the grid. For Cartesian data just give xin/xmax/ymin/ymax.
89 This sets the complete domain for the relevant track data set.
90 -V Selects verbose mode, which will send progress reports to stderr
92 [Default runs "silently"].
93 -W Give t or d and append the corresponding maximum time gap (in
95 user units; this is typically seconds [Infinity]), or distance
96 (for units, see -N) gap [Infinity]) allowed between the two data
97 points immediately on either side of a crossover. If these lim‐
98 its are exceeded then a data gap is assumed and no COE will be
99 determined.
100
102 These *.def files contain information about the data file format and
103 have two sections: (1) header information and (2) column information.
104 All header information starts with the character # in the first column,
105 immediately followed by an upper-case directive. If the directive
106 takes an argument it is separated by white-space. You may append a
107 trailing # comments. Five directives are recognized:
108 ASCII states that the data files are in ASCII format.
110 BINARY states that the data files are native binary files.
111 NETCDF states that the data files are COARDS-compliant 1-D netCDF
112 files.
113 SKIP takes an integer argument which is either the number of lines to
114 skip (when reading ASCII files) or the number of bytes to skip (when
115 reading native binary files). Not used with netCDF files.
116 GEO indicates that these files are geographic data sets, with periodic‐
117 ities in the x-coordinate (longitudes). Alternatively, use -G.
118 MULTISEG means each track consists of multiple segments separated by a
119 GMT multisegment header (alternatively, use -m when defining the system
120 TAG). Not used with netCDF files.
121 The column information consists of one line per column in the order the
123 columns appear in the data file. For each column you must provide
124 seven attributes:
125 name type NaN NaN-proxy scale offset oformat
127 name is the name of the column variable. It is expected that you will
129 use the special names lon (or x if Cartesian) and lat (or y) for the
130 two required coordinate columns, and time when optional time data are
131 present.
132 type is always a for ASCII representations of numbers, whereas for
133 binary files you may choose among c for signed 1-byte character
134 (-127,+128), u for unsigned byte (0-255), h for signed 2-byte integers
135 (-32768,+32767), i for signed 4-byte integers
136 (-2,147,483,648,+2,147,483,647), f for 4-byte floating points and d for
137 8-byte double precision floating points. For netCDF, simply use d as
138 netCDF will automatically handle type-conversions during reading.
139 NaN is Y if certain values (e.g, -9999) are to be replaced by NAN, and
140 N otherwise.
141 NaN-proxy is that special value (e.g., -9999).
142 scale is used to multiply the data after reading.
143 offset is used to add to the scaled data.
144 oformat is a C-style format string used to print values from this col‐
145 umn.
146 If you give - as the oformat then GMT's formatting machinery will be
148 used instead (i.e., D_FORMAT, PLOT_DEGREE_FORMAT, PLOT_DATE_FORMAT,
149 PLOT_CLOCK_FORMAT). Some file formats already have definition files
150 premade. These include mgd77 (for plain ASCII MGD77 data files),
151 mgd77+ (for enhanced MGD77+ netCDF files), gmt (for old mgg supplement
152 binary files), xy (for plain ASCII x, y tables), xyz (same, with one z-
153 column), geo (for plain ASCII longitude, latitude files), and geoz
154 (same, with one z-column).
155
157 If you have a large set of track data files you can organize them using
158 the x2sys tools. Here we will outline the steps. Let us assume that
159 your track data file format consist of 2 header records with text
160 information followed by any number of identically formatted data
161 records with 6 columns (lat, lon, time, obs1, obs2, obs3) and that
162 files are called *.trk. We will call this the "line" format. First, we
163 create the line.def file:
164 # Define file for the line format
166 #ASCII # File is ASCII
167 #SKIP 2 # Skip 2 header records
168 #GEO # Data are geographic
169 #name type NaN NaN-proxy scale offset oformat
170 lat a N 0 1 0 %9.5f
171 lon a N 0 1 0 %10.5f
172 time a N 0 1 0 %7.1f
173 obs1 a N 0 1 0 %7.2f
174 obs2 a N 0 1 0 %7.2f
175 obs3 a N 0 1 0 %7.2f
176 Next we create the TAG and the TAG directory with the databases for
178 these line track files. Assuming these contain geographic data and
179 that we want to keep track of the data distribution at a 1 x 1 degree
180 resolution, with distances in km calculated along geodesics and with
181 speeds given in knots, we may run
182 x2sys_init LINE -V -G -Dline -Rg -Ce -Ndk -NsN -I1/1 -Etrk
184 where we have selected LINE to be our x2sys tag. When x2sys tools try
186 to read your line data files they will first look in the current direc‐
187 tory and second look in the file TAG_paths.txt for a list of additional
188 directories to examine. Therefore, create such a file (here
189 LINE_paths.txt) and stick the full paths to your data directories
190 there. All TAG-related files (definition files, tag files, and track
191 data bases created) will be expected to be in the directory pointed to
192 by $X2SYS_HOME/TAG (in our case $X2SYS_HOME/LINE). Note that the argu‐
193 ment to -D must contain the full path if the *.def file is not in the
194 current directory. x2sys_init will copy this file to the
195 $X2SYS_HOME/TAG directory where all other x2sys tools will expect to
196 find it.
197 Create tbf file(s):
199 Once the (empty) TAG databases have been initialized we go
200 through a two-step process to populate them. First we run
201 x2sys_binlist on all our track files to create one (or more)
202 multi-segment track bin-index files (tbf). These contain infor‐
203 mation on which 1 x 1 degree bins (or any other blocksize; see
204 -I) each track has visited and which observations (in your case
205 obs1, obs2, obs3) were actually observed (not all tracks may
206 have all three kinds of observations everywhere). For instance,
207 if your tracks are listed in the file tracks.lis we may run this
208 command:
209 x2sys_binlist -V -TLINE :tracks.lis > tracks.tbf
211 Update index data base:
213 Next, the track bin-index files are fed to x2sys_put which will
214 insert the information into the TAG databases:
215 x2sys_put -V -TLINE tracks.tbf
217 Search for data:
219 You may now use x2sys_get to find all the tracks within a cer‐
220 tain sub-region, and optionally limit the search to those tracks
221 that have a particular combination of observables. E.g., to
222 find all the tracks which has both obs1 and obs3 inside the
223 specified region, run
224 x2sys_get -V -TLINE -R20/40/-40/-20 -Fobs1,obs3 > tracks.tbf
226 MGD77[+] or GMT:
228 Definition files already exist for MGD77 files (both standard
229 ASCII and enhanced netCDF-based MGD77+ files) and the old *.gmt
230 files manipulated by the mgg supplements; for these data sets
231 the -C and -N will default to great circle distance calculation
232 in km and speed in m/s. There are also definition files for
233 plain x,y[,z] and lon,lat[,z] tracks. To initiate new track
234 databases to be used with MGD77 data from NGDC, try
235 x2sys_init MGD77 -V -Dmgd77 -Emgd77 -Rd -Gd -Nsn -I1/1 -Wt900
237 -Wd5
238 where we have chosen a 15 minute (900 sec) or 5 km threshold to
240 indicate a data gap and selected knots as the speed; the other
241 steps are similar.
242 Binary files:
244 Let us pretend that your line files actually are binary files
245 with a 128-byte header structure (to be skipped) followed by the
246 data records and where lon, lat, time are double precision num‐
247 bers while the three observations are 2-byte integers which must
248 be multiplied by 0.1. Finally, the first two observations may
249 be -32768 which means there is no data available. All that is
250 needed is a different line.def file:
251 # Define file for the binary line format
253 #BINARY # File is now binary
254 #SKIP 128 # Skip 128 bytes
255 #GEO # Data are geographic
256 #name type NaN? NaN-proxy scale offset oformat
257 lon d N 0 1 0 %10.5f
258 lat d N 0 1 0 %9.5f
259 time d N 0 1 0 %7.1f
260 obs1 h Y -32768 0.1 0 %6.1f
261 obs2 h Y -32768 0.1 0 %6.1f
262 obs3 h N 0 0.1 0 %6.1f
263 The rest of the steps are identical.
265 COARDS 1-D netCDF files:
267 Finally, suppose that your line files actually are netCDF files
268 that conform to the COARDS convention, with data columns named
269 lon, lat, time, obs1, obs2, and obs3. All that is needed is a
270 different line.def file:
271 # Define file for the netCDF COARDS line format
273 #NETCDF # File is now netCDF
274 #GEO # Data are geographic
275 #name type NaN? NaN-proxy scale offset oformat
276 lon d N 0 1 0 %10.5f
277 lat d N 0 1 0 %9.5f
278 time d N 0 1 0 %7.1f
279 obs1 d N 0 1 0 %6.1f
280 obs2 d N 0 1 0 %6.1f
281 obs3 d N 0 1 0 %6.1f
282 Note we use no scaling or NAN proxies since those issues are
284 usually handled internally in the netCDF format description.
285
287 x2sys_binlist(1), x2sys_datalist(1), x2sys_get(1), x2sys_list(1),
288 x2sys_put(1), x2sys_report(1), x2sys_solve(1)
289GMT 4.5.6 10 Mar 2011 X2SYS_INIT(1)