1X2SYS_INIT(1) GMT X2SYS_INIT(1)
2
6 x2sys_init - Initialize a new x2sys track database
7
9 x2sys_init TAG -Ddeffile [ -Cc|f|g|e ] [ -Esuffix ] [ -F ] [ -Gd|g
10 ] [ -Idx[/dy] ] [ -Nd|sunit ] [ -Rregion ] [ -V[level] ] [
11 -Wt|dgap ]
12 Note: No space is allowed between the option flag and the associated
14 arguments.
15
17 x2sys_init is the starting point for anyone wishing to use x2sys; it
18 initializes a set of data bases that are particular to one kind of
19 track data. These data, their associated data bases, and key parameters
20 are given a short-hand notation called an x2sys TAG. The TAG keeps
21 track of settings such as file format, whether the data are geographic
22 or not, and the binning resolution for track indices. Running
23 x2sys_init is a prerequisite to running any of the other x2sys pro‐
24 grams, such as x2sys_binlist, which will create a crude representation
25 of where each data track go within the domain and which observations
26 are available; this information serves as input to x2sys_put which
27 updates the track data base. Then, x2sys_get can be used to find which
28 tracks and data are available inside a given region. With that list of
29 tracks you can use x2sys_cross to calculate track crossovers, use
30 x2sys_report to report crossover statistics or x2sys_list to pull out
31 selected crossover information that x2sys_solve can use to determine
32 track-specific systematic corrections. These corrections may be used
33 with x2sys_datalist to extract corrected data values for use in subse‐
34 quent work. Because you can run x2sys_init you must set the environ‐
35 mental parameter X2SYS_HOME to a directory where you have write permis‐
36 sion, which is where x2sys can keep track of your settings.
37
39 TAG The unique name of this data type x2sys TAG.
40 -Ddeffile
42 Definition file prefix for this data set [See DEFINITION FILES
43 below for more information]. Specify full path if the file is
44 not in the current directory.
45
47 -Cc|f|g|e
48 Select procedure for along-track distance calculation when
49 needed by other programs:
50 c Cartesian distances [Default, unless -G is set].
52 f Flat Earth distances.
54 g Great circle distances [Default if -G is set].
56 e Geodesic distances on current GMT ellipsoid.
58 -Esuffix
60 Specifies the file extension (suffix) for these data files. If
61 not given we use the definition file prefix as the suffix (see
62 -D).
63 -F Force creating new files if old ones are present [Default will
65 abort if old TAG files are found].
66 -Gd|g Selects geographical coordinates. Append d for discontinuity at
68 the Dateline (makes longitude go from -180 to + 180) or g for
69 discontinuity at Greenwich (makes longitude go from 0 to 360
70 [Default]). If not given we assume the data are Cartesian.
71 -Idx[/dy]
73 x_inc [and optionally y_inc] is the grid spacing. Append m to
74 indicate minutes or c to indicate seconds for geographic data.
75 These spacings refer to the binning used in the track bin-index
76 data base.
77 -Nd|sunit
79 Sets the units used for distance and speed when requested by
80 other programs. Append d for distance or s for speed, then give
81 the desired unit as c (Cartesian userdist or userdist/usertime),
82 e (meters or m/s), f (feet or feet/s), k (km or kms/hr), m
83 (miles or miles/hr), n (nautical miles or knots) or u (survey
84 feet or survey feet/s). [Default is -Ndk -Nse (km and m/s) if -G
85 is set and -Ndc and -Nsc otherwise (Cartesian units)].
86 -Rwest/east/south/north[/zmin/zmax][+r][+uunit]
88 west, east, south, and north specify the region of interest, and
89 you may specify them in decimal degrees or in
90 [±]dd:mm[:ss.xxx][W|E|S|N] format Append +r if lower left and
91 upper right map coordinates are given instead of w/e/s/n. The
92 two shorthands -Rg and -Rd stand for global domain (0/360 and
93 -180/+180 in longitude respectively, with -90/+90 in latitude).
94 Alternatively for grid creation, give Rcodelon/lat/nx/ny, where
95 code is a 2-character combination of L, C, R (for left, center,
96 or right) and T, M, B for top, middle, or bottom. e.g., BL for
97 lower left. This indicates which point on a rectangular region
98 the lon/lat coordinate refers to, and the grid dimensions nx and
99 ny with grid spacings via -I is used to create the corresponding
100 region. Alternatively, specify the name of an existing grid
101 file and the -R settings (and grid spacing, if applicable) are
102 copied from the grid. Appending +uunit expects projected (Carte‐
103 sian) coordinates compatible with chosen -J and we inversely
104 project to determine actual rectangular geographic region. For
105 perspective view (-p), optionally append /zmin/zmax. In case of
106 perspective view (-p), a z-range (zmin, zmax) can be appended to
107 indicate the third dimension. This needs to be done only when
108 using the -Jz option, not when using only the -p option. In the
109 latter case a perspective view of the plane is plotted, with no
110 third dimension. For Cartesian data just give
111 xmin/xmax/ymin/ymax. This option bases the statistics on those
112 COE that fall inside the specified domain.
113 -V[level] (more ...)
115 Select verbosity level [c].
116 -Wt|dgap
118 Give t or d and append the corresponding maximum time gap (in
119 user units; this is typically seconds [Infinity]), or distance
120 (for units, see -N) gap [Infinity]) allowed between the two
121 data points immediately on either side of a crossover. If these
122 limits are exceeded then a data gap is assumed and no COE will
123 be determined.
124 -^ or just -
126 Print a short message about the syntax of the command, then
127 exits (NOTE: on Windows just use -).
128 -+ or just +
130 Print an extensive usage (help) message, including the explana‐
131 tion of any module-specific option (but not the GMT common
132 options), then exits.
133 -? or no arguments
135 Print a complete usage (help) message, including the explanation
136 of all options, then exits.
137
139 These *.def files contain information about the data file format and
140 have two sections: (1) header information and (2) column information.
141 All header information starts with the character # in the first column,
142 immediately followed by an upper-case directive. If the directive takes
143 an argument it is separated by white-space. You may append a trailing #
144 comments. Five directives are recognized:
145 ASCII states that the data files are in ASCII format.
147 BINARY states that the data files are native binary files.
149 NETCDF states that the data files are COARDS-compliant 1-D netCDF
151 files.
152 SKIP takes an integer argument which is either the number of lines to
154 skip (when reading ASCII files) or the number of bytes to skip (when
155 reading native binary files). Not used with netCDF files.
156 GEO indicates that these files are geographic data sets, with periodic‐
158 ities in the x-coordinate (longitudes). Alternatively, use -G.
159 MULTISEG means each track consists of multiple segments separated by a
161 GMT segment header (alternatively, use -m when defining the system
162 TAG). Not used with netCDF files.
163 The column information consists of one line per column in the order the
165 columns appear in the data file. For each column you must provide seven
166 attributes:
167 name type NaN NaN-proxy scale offset oformat
169 name is the name of the column variable. It is expected that you will
171 use the special names lon (or x if Cartesian) and lat (or y) for the
172 two required coordinate columns, and time when optional time data are
173 present.
174 type is always a for ASCII representations of numbers, whereas for
176 binary files you may choose among c for signed 1-byte character
177 (-127,+128), u for unsigned byte (0-255), h for signed 2-byte integers
178 (-32768,+32767), i for signed 4-byte integers
179 (-2,147,483,648,+2,147,483,647), f for 4-byte floating points and d for
180 8-byte double precision floating points. For netCDF, simply use d as
181 netCDF will automatically handle type-conversions during reading.
182 NaN is Y if certain values (e.g, -9999) are to be replaced by NAN, and
184 N otherwise.
185 NaN-proxy is that special value (e.g., -9999).
187 scale is used to multiply the data after reading.
189 offset is used to add to the scaled data.
191 oformat is a C-style format string used to print values from this col‐
193 umn.
194 If you give - as the oformat then GMT's formatting machinery will be
196 used instead (i.e., FORMAT_FLOAT_OUT, FORMAT_GEO_MAP, FORMAT_DATE_MAP,
197 FORMAT_CLOCK_MAP). Some file formats already have definition files
198 premade. These include mgd77 (for plain ASCII MGD77 data files), mgd77+
199 (for enhanced MGD77+ netCDF files), gmt (for old mgg supplement binary
200 files), xy (for plain ASCII x, y tables), xyz (same, with one z-col‐
201 umn), geo (for plain ASCII longitude, latitude files), and geoz (same,
202 with one z-column).
203
205 If you have a large set of track data files you can organize them using
206 the x2sys tools. Here we will outline the steps. Let us assume that
207 your track data file format consist of 2 header records with text
208 information followed by any number of identically formatted data
209 records with 6 columns (lat, lon, time, obs1, obs2, obs3) and that
210 files are called *.trk. We will call this the "line" format. First, we
211 create the line.def file:
212 ┌───────────┬────────────┬─────┬───────────┬───────┬────────┬─────────┐
214 │# Define │ │ │ │ │ │ │
215 │file for │ │ │ │ │ │ │
216 │the line │ │ │ │ │ │ │
217 │format │ │ │ │ │ │ │
218 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
219 │# SKIP 2 │ # Skip 2 │ │ │ │ │ │
220 │ │ header │ │ │ │ │ │
221 │ │ records │ │ │ │ │ │
222 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
223 │# GEO │ # Data are │ │ │ │ │ │
224 │ │ geographic │ │ │ │ │ │
225 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
226 │#name │ type │ NaN │ NaN-proxy │ scale │ offset │ oformat │
227 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
228 │lat │ a │ N │ 0 │ 1 │ 0 │ %9.5f │
229 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
230 │lon │ a │ N │ 0 │ 1 │ 0 │ %10.5f │
231 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
232 │time │ a │ N │ 0 │ 1 │ 0 │ %7.1f │
233 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
234 │obs1 │ a │ N │ 0 │ 1 │ 0 │ %7.2f │
235 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
236 │obs2 │ a │ N │ 0 │ 1 │ 0 │ %7.2f │
237 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
238 │obs3 │ a │ N │ 0 │ 1 │ 0 │ %7.2f │
239 └───────────┴────────────┴─────┴───────────┴───────┴────────┴─────────┘
240 Next we create the TAG and the TAG directory with the databases for
242 these line track files. Assuming these contain geographic data and that
243 we want to keep track of the data distribution at a 1 x 1 degree reso‐
244 lution, with distances in km calculated along geodesics and with speeds
245 given in knots, we may run
246 gmt x2sys_init LINE -V -G -Dline -Rg -Ce -Ndk -NsN -I1/1 -Etrk
248 where we have selected LINE to be our x2sys tag. When x2sys tools try
250 to read your line data files they will first look in the current direc‐
251 tory and second look in the file TAG_paths.txt for a list of additional
252 directories to examine. Therefore, create such a file (here
253 LINE_paths.txt) and stick the full paths to your data directories
254 there. All TAG-related files (definition files, tag files, and track
255 data bases created) will be expected to be in the directory pointed to
256 by $X2SYS_HOME/TAG (in our case $X2SYS_HOME/LINE). Note that the argu‐
257 ment to -D must contain the full path if the *.def file is not in the
258 current directory. x2sys_init will copy this file to the
259 $X2SYS_HOME/TAG directory where all other x2sys tools will expect to
260 find it.
261 Create tbf file(s):
263 Once the (empty) TAG databases have been initialized we go
264 through a two-step process to populate them. First we run
265 x2sys_binlist on all our track files to create one (or more)
266 multisegment track bin-index files (tbf). These contain informa‐
267 tion on which 1 x 1 degree bins (or any other blocksize; see -I)
268 each track has visited and which observations (in your case
269 obs1, obs2, obs3) were actually observed (not all tracks may
270 have all three kinds of observations everywhere). For instance,
271 if your tracks are listed in the file tracks.lis we may run this
272 command:
273 gmt x2sys_binlist -V -TLINE :tracks.lis > tracks.tbf
275 Update index data base:
277 Next, the track bin-index files are fed to x2sys_put which will
278 insert the information into the TAG databases:
279 gmt x2sys_put -V -TLINE tracks.tbf
281 Search for data:
283 You may now use x2sys_get to find all the tracks within a cer‐
284 tain sub-region, and optionally limit the search to those tracks
285 that have a particular combination of observables. E.g., to find
286 all the tracks which has both obs1 and obs3 inside the specified
287 region, run
288 gmt x2sys_get -V -TLINE -R20/40/-40/-20 -Fobs1,obs3 > tracks.tbf
290 MGD77[+] or GMT:
292 Definition files already exist for MGD77 files (both standard
293 ASCII and enhanced netCDF-based MGD77+ files) and the old *.gmt
294 files manipulated by the mgg supplements; for these data sets
295 the -C and -N will default to great circle distance calculation
296 in km and speed in m/s. There are also definition files for
297 plain x,y[,z] and lon,lat[,z] tracks. To initiate new track
298 databases to be used with MGD77 data from NGDC, try
299 gmt x2sys_init MGD77 -V -Dmgd77 -Emgd77 -Rd -Gd -Nsn -I1/1 -Wt900 -Wd5
301 where we have chosen a 15 minute (900 sec) or 5 km threshold to
303 indicate a data gap and selected knots as the speed; the other
304 steps are similar.
305 Binary files:
307 Let us pretend that your line files actually are binary files
308 with a 128-byte header structure (to be skipped) followed by the
309 data records and where lon, lat, time are double precision num‐
310 bers while the three observations are 2-byte integers which must
311 be multiplied by 0.1. Finally, the first two observations may be
312 -32768 which means there is no data available. All that is
313 needed is a different line.def file:
314 ┌───────────┬────────────┬─────┬───────────┬───────┬────────┬─────────┐
316 │# Define │ │ │ │ │ │ │
317 │file for │ │ │ │ │ │ │
318 │the binary │ │ │ │ │ │ │
319 │line for‐ │ │ │ │ │ │ │
320 │mat │ │ │ │ │ │ │
321 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
322 │# BINARY │ # File is │ │ │ │ │ │
323 │ │ now binary │ │ │ │ │ │
324 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
325 │# SKIP 128 │ # Skip 128 │ │ │ │ │ │
326 │ │ bytes │ │ │ │ │ │
327 └───────────┴────────────┴─────┴───────────┴───────┴────────┴─────────┘
328 │# GEO │ # Data are │ │ │ │ │ │
332 │ │ geographic │ │ │ │ │ │
333 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
334 │#name │ type │ NaN │ NaN-proxy │ scale │ offset │ oformat │
335 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
336 │lon │ d │ N │ 0 │ 1 │ 0 │ %10.5f │
337 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
338 │lat │ d │ N │ 0 │ 1 │ 0 │ %9.5f │
339 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
340 │time │ d │ N │ 0 │ 1 │ 0 │ %7.1f │
341 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
342 │obs1 │ h │ Y │ -32768 │ 0.1 │ 0 │ %6.1f │
343 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
344 │obs2 │ h │ Y │ -32768 │ 0.1 │ 0 │ %6.1f │
345 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
346 │obs3 │ h │ N │ 0 │ 0.1 │ 0 │ %6.1f │
347 └───────────┴────────────┴─────┴───────────┴───────┴────────┴─────────┘
348 The rest of the steps are identical.
350 COARDS 1-D netCDF files:
352 Finally, suppose that your line files actually are netCDF files
353 that conform to the COARDS convention, with data columns named
354 lon, lat, time, obs1, obs2, and obs3. All that is needed is a
355 different line.def file:
356 ┌───────────┬────────────┬─────┬───────────┬───────┬────────┬─────────┐
358 │# Define │ │ │ │ │ │ │
359 │file for │ │ │ │ │ │ │
360 │the netCDF │ │ │ │ │ │ │
361 │COARDS │ │ │ │ │ │ │
362 │line for‐ │ │ │ │ │ │ │
363 │mat │ │ │ │ │ │ │
364 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
365 │# NETCDF │ # File is │ │ │ │ │ │
366 │ │ now netCDF │ │ │ │ │ │
367 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
368 │# GEO │ # Data are │ │ │ │ │ │
369 │ │ geographic │ │ │ │ │ │
370 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
371 │#name │ type │ NaN │ NaN-proxy │ scale │ offset │ oformat │
372 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
373 │lon │ d │ N │ 0 │ 1 │ 0 │ %10.5f │
374 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
375 │lat │ d │ N │ 0 │ 1 │ 0 │ %9.5f │
376 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
377 │time │ d │ N │ 0 │ 1 │ 0 │ %7.1f │
378 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
379 │obs1 │ d │ N │ 0 │ 1 │ 0 │ %6.1f │
380 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
381 │obs2 │ d │ N │ 0 │ 1 │ 0 │ %6.1f │
382 ├───────────┼────────────┼─────┼───────────┼───────┼────────┼─────────┤
383 │obs3 │ d │ N │ 0 │ 1 │ 0 │ %6.1f │
384 └───────────┴────────────┴─────┴───────────┴───────┴────────┴─────────┘
385 Note we use no scaling or NAN proxies since those issues are
387 usually handled internally in the netCDF format description.
388
390 x2sys_binlist, x2sys_datalist, x2sys_get, x2sys_list, x2sys_put,
391 x2sys_report, x2sys_solve, x2sys_cross
392
394 2019, P. Wessel, W. H. F. Smith, R. Scharroo, J. Luis, and F. Wobbe
3955.4.5 Feb 24, 2019 X2SYS_INIT(1)