1MGD77LIST(1) Generic Mapping Tools MGD77LIST(1)
2
6 mgd77list - A data-extractor for MGD77[+] files
7
9 mgd77list NGDC-ids -Fcolumns[,logic][:bittests] [ -A[+]c|d|f|m|tcode ]
10 [ -Cf|g|e ] [ -DA|astartdate ] [ -DB|bstopdate ] [ -E ] [ -Gastartrec ]
11 [ -Gbstoprec ] [ -H[i][nrec] ] [ -Iignore ] [ -L[corrtable] ] [
12 -Nd|sunit ] [ -Qa|vmin/max ] [ -Rwest/east/south/north[r] ] [ -Sastart‐
13 dist[unit] ] [ -Sbstopdist[unit] ] [ -T[m|e] ] [ -V ] [ -Wweight ] [
14 -Z+|- ] [ -bo[s|S|d|D[ncol]|c[var1/...]] ] [ -m[flag] ]
15
17 mgd77list reads <NGDC-id>.[mgd77|nc] files and produces an ASCII [or
18 binary] table. The <NGDC-id>.[mgd77|nc] files contain track informa‐
19 tion such as leg-id, time and position, geophysical observables such as
20 gravity, magnetics, and bathymetry, and control codes and corrections
21 such as Eotvos and diurnal corrections. The MGD77+ extended netCDF
22 files may also contain additional user columns (for a listing of avail‐
23 able columns, use mgd77info -C, and to learn how to add your own custom
24 columns, see mgd77manage). The user may extract any combination of
25 these parameters, any of six computed quantities (distance, heading,
26 velocity, Carter correction, and gravity and magnetic global reference
27 fields), calendar sub-units of time (year, month, day, hour, min, sec),
28 the NGDC id, and finally a preset weight (see -W). A sub-section can
29 be specified by passing time- or distance-intervals along track or by
30 selecting a geographical region. Finally, each output record may be
31 required to pass any number of logical tests involving data values or
32 bit flags.
33 NGDC-ids
36 Can be one or more of five kinds of specifiers:
37 1) 8-character NGDC IDs, e.g., 01010083, JA010010etc., etc.
38 2) 2-character <agency> codes which will return all cruises from
39 each agency.
40 3) 4-character <agency><vessel> codes, which will return all
41 cruises from those vessels.
42 4) =<list>, where <list> is a table with NGDC IDs, one per line.
43 5) If nothing is specified we return all cruises in the data
44 base.
45 (See mgd77info -L for agency and vessel codes). The ".mgd77" or
46 ".nc" extensions will automatically be appended, if needed (use
47 -I to ignore certain file types). Cruise files will be looked
48 for first in the current directory and second in all directories
49 listed in $MGD77_HOME/mgd77_paths.txt [If $MGD77_HOME is not set
50 it will default to $GMT_SHAREDIR/mgd77].
51 -F The required columns string must be a comma-separated list of
53 parameter abbreviations given in the desired output order. Any
54 parameters given in UPPER case must not be NaN in a record for
55 output to occur. Unless specified separately, the output format
56 (if ASCII) is controlled by the GMT parameter D_FORMAT. The
57 available abbreviations are:
58 drt The digital record type, usually 3 or 5 (for Y2K-compliant
60 cruises).
61 id The survey ID string (leg name).
63 ngdcid The 8-character NGDC cruise ID string (usually the file prefix).
65 time Choose between Absolute calendar time (atime, the default) in
67 the format dictated by the GMT parameters OUTPUT_DATE_FORMAT and
68 OUTPUT_CLOCK_FORMAT, Relative time (rtime) in the format dic‐
69 tated by the GMT parameters D_FORMAT and TIME_SYSTEM (or
70 TIME_EPOCH and TIME_UNIT)), or Fractional year (ytime) in the
71 format dictated by D_FORMAT.
72 lon Longitude in the format dictated by the GMT parameter OUT‐
74 PUT_DEGREE_FORMAT.
75 lat Longitude in the format dictated by the GMT parameter OUT‐
77 PUT_DEGREE_FORMAT.
78 twt Two-Way Travel time (in s).
80 depth Corrected bathymetry (in m, positive below sealevel).
82 mtf1 Magnetic Total Field intensity from sensor 1 (in nTesla).
84 mtf2 Magnetic Total Field intensity from sensor 2 (in nTesla).
86 mag Residual magnetic anomaly (in nTesla).
88 gobs Observed gravity (in mGal).
90 faa Free-air gravity anomaly (in mGal).
92 ptc Position Type Code (1 = fix, 3 = interpolated, 9 = unspecified).
94 bcc Bathymetric Correction Code, indicating the procedure used to
96 convert travel time to depth. (01-55 = Matthews' zone used to
97 correct the depth, 59 = Matthews' corrections used but the zones
98 is unspecified in the data record, 60 = S. Kuwahara formula for
99 T-S, 61 = Wilson formula for T-S, 62 = Del Grosso formula for T-
100 S, 63 = Carter's tables, 88 = Other, described in header sec‐
101 tions, 99 = unspecified).
102 btc Bathymetric Type Code, indicating how the bathymetry value was
104 obtained (1 = observed, 3 = interpolated, 9 = unspecified).
105 msens Magnetic sensor for used to evaluate the residual field (1 = 1st
107 or leading sensor, 2 = 2nd or trailing sensor, 9 = unspecified).
108 msd Depth (or altitude) of the magnetic sensor (in m, positive below
110 sealevel).
111 diur Magnetic diurnal correction (in nTesla).
113 eot Eotvos correction (in mGal).
115 sln Seismic Line Number string.
117 sspn Seismic Shot Point Number string.
119 nqc Navigation Quality Code (5 = suspected, by source institution, 6
121 = suspected, by NGDC, 9 = no problems identified).
122 In addition, the following derived quantities can be requested:
124 year The year of each record.
126 month The month of each record.
128 day The day of the month of each record.
130 hour The hour of each record.
132 min The minutes of each record.
134 sec The decimal seconds of each record.
136 dist Along-track distance from start of leg. For method of calcula‐
138 tion, see -C [spherical great circle distances], and for dis‐
139 tance units, see -N [km].
140 az Ship azimuth (heading) measured clockwise from north (in
142 degrees).
143 vel Ship speed; see -N for units [m/s].
145 weight Weight assigned to this data set (see -W).
147 carter Carter depth correction, if twt is present in file (in m).
149 Sign: Correction is to be subtracted from uncorrected depths to
150 yield a corrected depth.
151 igrf International geomagnetic reference field (total field) (in
153 nTesla).
154 ngrav International Gravity reference Field ("normal gravity") (in
156 mGal). Field is selected based on the parameter Gravity Theo‐
157 retical Formula Code in the cruise's MGD77 header. If this is
158 not set or is invalid we default to the IGF 1980. Alterna‐
159 tively, specify the field directly using -Af (see that option
160 for more details).
161 The following short-hand flags are also recognized:
163 all This returns all data columns in the file.
165 mgd77 This results in all 27 MGD77 fields being written out in the
167 offical MGD77 order.
168 geo This limits the output to 10 fields (time, lon, lat plus the
170 seven geophysical observations twt, depth, mtf1, mtf2, mag,
171 gobs, and faa). By appending + to either of these set we will
172 also append dist, azim, vel, and weight as listed above.
173 As an option, logical tests may be added for any of the observa‐
175 tions by appending ,logic, which is itself composed of one or
176 more comma-separated instructions of the form parOPvalue, where
177 par is one of the parameters listed above, OP is a logical oper‐
178 ator (<, <=, =, !=, >=, >, |), and value is a constant used in
179 the comparison. Floating point parameters are compared numeri‐
180 cally; character parameters are compared lexically (after lead‐
181 ing and trailing blanks have been removed). The bit comparison
182 (|) means that at least one of the bits in value must be turned
183 on in par. At least one of the tests must be true for the
184 record to be output, except for tests using UPPER case parame‐
185 ters which all must be true for output to occur. Note that
186 specifying a test does not imply that the corresponding column
187 will be included in the output stream; it must be present in
188 columns for that to occur. Note: some of the operators are spe‐
189 cial UNIX characters and you are advised to place quotes around
190 the entire argument to -F.
191 Finally, for MGD77+ files you may optionally append :bittests
193 which is : (a colon) followed by one or more comma-separated
194 +-col terms. This compares specific bitflags only for each
195 listed column. Here, + means the chosen bit must be 1 (ON)
196 whereas - means it must be 0 (OFF). All bit tests given must be
197 passed. By default, MGD77+ files that have the special
198 MGD77_flags column present will use those flags, and observa‐
199 tions associated with ON-bits (meaning they are flagged as bad)
200 will be set to NaN; append : with no trailing information to
201 turn this behavior off (i.e., no bit flags will be consulted).
202
204 No space between the option flag and the associated arguments.
205 -A By default, corrected depth (depth), magnetic residual anomaly
207 (mag), free-air gravity anomaly (faa), and the derived quantity
208 Carter depth correction (carter) are all output as is (if
209 selected in -F); this option adjusts that behavior. For each of
210 these columns there are 2-4 ways to adjust the data. Append
211 c(arter), d(epth), f(aa), or m(ag) and select the code for the
212 procedure you want applied. You may select more than one proce‐
213 dure for a data column by summing their numerical codes (1, 2,
214 4, and 8). E.g., -Ac3 will first try method -Ac1 to estimate a
215 Carter correction but if depth is NaN we will next try -Ac2
216 which only uses twt. In all cases, if any of the values
217 required by an adjustment procedure is NaN then the result will
218 be NaN. This is also true if the original anomaly is NaN.
219 Specify -A+ to recalculate anomalies even if the anomaly in the
220 file is NaN. Additionally, you can use -At to create fake times
221 for cruises that has no time; these are based on distances and
222 cruise duration.
223 -Ac Determines how the carter correction term is calculated. Below,
225 C(twt) stands for the Carter-corrected depth (it also depends on
226 lon, lat), U(twt, v) is the uncorrected depth (= twt * v / 2)
227 using as v the "Assumed Sound Velocity" parameter in the MGD77
228 header (if it is a valid velocity, otherwise we default to 1500
229 m/s); alternatively, append your preferred velocity v in m/s,
230 TU(depth, v) is the 2-way travel time estimated from the (pre‐
231 sumably) uncorrected depth, and TC(depth) is the 2-way travel
232 time obtained by inverting the (presumably) corrected depth
233 using the Carter correction formula. Select from
234 -Ac1[,v] returns difference between U(twt, v) and depth
235 [Default].
236 -Ac2[,v] returns difference between U(twt, v) and Carter (twt).
237 -Ac4[,v] returns difference between (assumed uncorrected) depth
238 and Carter (TU(depth)).
239 -Ac8[,v] returns difference between U(TC(depth), v) and depth.
240 -Ad Determines how the depth column output is obtained:
242 -Ad1 returns depth as stored in the data set [Default].
243 -Ad2[,v] returns calculated uncorrected depth U(twt, v).
244 -Ad4 returns calculated corrected depth C(twt).
245 -Af Determines how the faa column output is obtained. If ngrav
247 (i.e., the International Gravity reference Field (IGF), or "nor‐
248 mal gravity") is required it is selected based on the MGD77
249 header parameter "Theoretical Gravity Formula Code"; if this
250 code is not present or is invalid we default to 4. Alterna‐
251 tively, append the preferred field (1-4) to select 1 (Heiskanen
252 1924), 2 (IGF 1930), 3 (IGF 1967) or 4 (IGF 1980). Select from
253 -Af1[,field] returns faa as stored in the data set [Default].
254 Optionally, sets the IGF field to use if you also have requested
255 ngrav as an output column in -F.
256 -Af2[,field] returns the difference between gobs and ngrav (with
257 optional field directive).
258 -Af3[,field] returns the combination of gobs + eot - ngrav (with
259 optional field directive).
260 -Am Determines how the mag column output is obtained. There may be
262 one or two total field measurements in the file (mtf1 and mtf2),
263 and the column msens may state which one is the leading sensor
264 (1 or 2; it may also be undefined). Select from
265 -Am1 returns mag as stored in the data set [Default].
266 -Am2 returns the difference between mgfx and igrf, where x is
267 the leading sensor (1 or 2) indicated by the msens data field
268 (defaults to 1 if unspecified).
269 -Am4 returns the difference between mgfx and igrf, where x is
270 the sensor (2 or 1) not indicated by the msens data field
271 (defaults to 2 if unspecified).
272 -C Append a one-letter code to select the procedure for along-track
274 distance calculation (see -N for selecting units):
275 f Flat Earth distances.
276 g Great circle distances [Default].
277 e Geodesic distances on current GMT ellipsoid.
278 -Da Do not list data collected before startdate (yyyy-mm-
280 ddT[hh:mm:ss]) [Default is start of cruise]. Use -DA to
281 exclude records whose time is undefined (i.e., NaN). [Default
282 reports those records].
283 -Db Do not list data collected on or after stopdate (yyyy-mm-
285 ddT[hh:mm:ss]). [Default is end of cruise]. Use -DB to exclude
286 records whose time is undefined (i.e., NaN). [Default reports
287 those records].
288 -E Exact match: Only output records that match all the requested
290 geophysical columns [Default outputs records that matches at
291 least one of the observed columns].
292 -Ga Do not list records before startrec [Default is 0, the first
294 record].
295 -Gb Do not list data after stoprec. [Default is the last record].
297 -H Issue a header record with names for each data field.
299 -I Ignore certain data file formats from consideration. Append
301 a|c|t to ignore MGD77 ASCII, MGD77+ netCDF, or plain tab-sepa‐
302 rated ASCII table files, respectively. The option may be
303 repeated to ignore more than one format. [Default ignores
304 none].
305 -L Apply optimal corrections to columns where such corrections are
307 available. Append the correction table to use [Default uses the
308 correction table mgd77_corrections.txt in the $MGD77_HOME direc‐
309 tory]. For the format of this file, see CORRECTIONS below.
310 -n Issue a multi-segment header record with cruise ID for each
312 cruise.
313 -N Append d for distance or s for speed, then give the desired unit
315 as e (meter or m/s), k (km or km/hr), m (miles or miles/hr), or
316 n (nautical miles or knots). [Default is -Ndk -Nse (km and
317 m/s)].
318 -Qa Specify an accepted range (min/max) of azimuths. Records whose
320 track azimuth falls outside this range are ignored [0-360].
321 -Qv Specify an accepted range (min/max; or just min if there is no
323 upper limit) of velocities. Records whose track speed falls
324 outside this range are ignored [0-infinity].
325 -R west, east, south, and north specify the Region of interest, and
327 you may specify them in decimal degrees or in
328 [+-]dd:mm[:ss.xxx][W|E|S|N] format. Append r if lower left and
329 upper right map coordinates are given instead of w/e/s/n. The
330 two shorthands -Rg and -Rd stand for global domain (0/360 and
331 -180/+180 in longitude respectively, with -90/+90 in latitude).
332 Alternatively, specify the name of an existing grid file and the
333 -R settings (and grid spacing, if applicable) are copied from
334 the grid.
335 -Sa Do not list data that are less than startdist meter along track
337 from port of departure. Append k for km, m for miles, or n for
338 nautical miles [Default is 0 meters].
339 -Sb Do not list data that are stopdist or more meters along track
341 from port of departure. Append k for km, m for miles, or n for
342 nautical miles [Default is end of track].
343 -T Turns OFF the otherwise automatic adjustment of values based on
345 correction terms that are stored in the MGD77+ file and used to
346 counteract such things as wrong units used by the source insti‐
347 tution when creating the original MGD77 file from which the
348 MGD77+ file derives (the option has no effect on plain MGD77
349 ASCII files). Append m or e to limit the option to the MGD77 or
350 extended columns set only [Default applies to both].
351 -V Selects verbose mode, which will send progress reports to stderr
353 [Default runs "silently"].
354 -W Set the weight for these data. Weight output option must be set
356 in -F. This is useful if the data are to be processed with the
357 weighted averaging techniques offered by blockmean, blockmedian,
358 and blockmode [1].
359 -Z Append the sign you want for depth, carter, and msd values below
361 sea level (-Z- gives negative bathymetry) [Default is positive
362 down].
363 -bo Selects binary output mode (single precision; append d for dou‐
365 ble precision, or use S|D to swap bytes on output) [Default is
366 ASCII]. -H is ignored if -bo is selected. Likewise, string-
367 fields cannot be selected. Note that if time is one of the
368 binary output columns it will be stored as Unix-time (seconds
369 since 1970). To read this information in GMT to obtain absolute
370 calendar time will require you to use --TIME_SYSTEM=unix.
371
373 To get a (distance, heading, gravity, bathymetry) listing from
374 01010047.mgd77, starting at June 3 1971 20:45 and ending at distance =
375 5000 km, use the following command:
376 mgd77list 01010047 -Da1971-06-03T20:45 -Sb5000 -Fdist,azim,faa,depth >
378 myfile.d
379 To make input for blockmean and surface using free-air anomalies from
381 all the cruises listed in the file cruises.lis, but only the data that
382 are inside the specified area, and make the output binary:
383 mgd77list `cat cruises.lis` -Flon,lat,faa -R-40/-30/25/35 -bo > all‐
385 grav.b
386 To extract the locations of depths exceeding 9000 meter that were not
388 interpolated (btc != 1) from all the cruises listed in the file
389 cruises.lis:
390 mgd77list `cat cruises.lis` -F"depth,DEPTH>9000,BTC!=1" > really_deep.d
392 To extract dist, faa, and grav12_2 from records whose depths are shal‐
394 lower than 3 km and where none of the requested fields are NaN, from
395 all the MGD77+ netCDF files whose cruise ids are listed in the file
396 cruises.lis, we try
397 mgd77list `cat cruises.lis` -E -Ia -F"dist,faa,grav12_2,depth<3000" >
399 shallow_grav.d
400 To extract dist, faa, and grav12_2 from all the MGD77+ netCDF files
402 whose cruise ids are listed in the file cruises.lis, but only retrieve
403 records whose bitflag for faa indicates BAD values, we try
404 mgd77list `cat cruises.lis` -E -Ia -F"dist,faa,grav12_2:+faa" >
406 bad_grav.d
407 To output lon, lat, mag, and faa from all the cruises listed in the
409 file cruises.lis, but recalculate the two residuals based on the latest
410 reference fields, try:
411 mgd77list `cat cruises.lis` -Flon,lat,mag,faa -Af2,4 -Am2 > data.d
413
415 When recalculated anomalies are requested (either explicitly via the -A
416 option or implicitly via E77 metadata in the MGD77+ file) we only do so
417 for the records whose original anomaly was not a NaN. This restriction
418 is implemented since many anomaly columns contains corrections, usually
419 in the form of hand-edited changes, that cannot be duplicated from the
420 corresponding observation.
421
423 The IGRF calculations are based on a Fortran program written by Susan
424 Macmillan, British Geological Survey, translated to C via f2c by
425 Joaquim Luis, U Algarve, and adapted to GMT-style by Paul Wessel.
426
428 The equations used are reproduced here using coefficients extracted
429 directly from the source code (let us know if you find errors):
430 (1) g = 978052.0 * [1 + 0.005285 * sin^2(lat) - 7e-6 * sin^2(2*lat) +
431 27e-6 * cos^2(lat) * cos^2(lon-18)]
432 (2) g = 978049.0 * [1 + 0.0052884 * sin^2(lat) - 0.0000059 *
433 sin^2(2*lat)]
434 (3) g = 978031.846 * [1 + 0.0053024 * sin^2(lat) - 0.0000058 *
435 sin^2(2*lat)]
436 (4) g = 978032.67714 * [(1 + 0.00193185138639 * sin^2(lat)) / sqrt (1 -
437 0.00669437999013 * sin^2(lat))]
438
440 The correction table is an ASCII file with coefficients and parameters
441 needed to carry out corrections. Comment records beginning with # are
442 allowed. All correction records are of the form
443 cruiseID observation correction
445 where cruiseID is a NGDC prefix, observation is one of the abbrevia‐
447 tions for geophysical observations listed under -F above, and correc‐
448 tion consists of one or more terms that will be summed up and then sub‐
449 tracted from the observation before output. Each term must have this
450 exact syntax:
451 factor[*[function]([scale](abbrev[-origin]))[^power]]
453 where terms in brackets are optional (the brackets themselves are not
455 used but regular parentheses must be used as indicated). No spaces are
456 allowed except between terms. The factor is the amplitude of the basis
457 function, while the optional function can be one of sin, cos, or exp.
458 The optional scale and origin can be used to translate the argument
459 (before giving it to the optional function). The argument abbrev is
460 one of the abbreviations for observations listed above. If origin is
461 given as T it means that we should replace it with the value of abbrev
462 for the very first record in the file (this is usually only done for
463 time). If the first record entry is NaN we revert origin to zero.
464 Optionally, raise the entire expression to the given power, before mul‐
465 tiplying by the amplitude. The following is an example of fictitious
466 corrections to the cruise 99999999, implying the depth should have the
467 Carter correction removed, faa should have a linear trend removed, the
468 magnetic anomaly (mag) should be corrected by a strange dependency on
469 ship heading and latitude, and gobs needs to have 10 mGal added (hence
470 given as -10):
471 99999999 depth 1.0*((carter))
473 99999999 faa 14.1 1e-5*((time-T))
474 99999999 mag 0.5*cos(0.5*(azim-19))^2 1.0*exp(-1e-3(lat))^1.5
475 99999999 gobs -10
476
478 mgd77convert(1), mgd77info(1), mgd77manage(1), mgd77track(1)
479
481 Wessel, P., and W. H. F. Smith, 2009, The Generic Mapping Tools (GMT)
482 version 4.5.0 Technical Reference & Cookbook, SOEST/NOAA.
483 Wessel, P., and W. H. F. Smith, 1998, New, Improved Version of Generic
484 Mapping Tools Released, EOS Trans., AGU, 79 (47), p. 579.
485 Wessel, P., and W. H. F. Smith, 1995, New Version of the Generic Map‐
486 ping Tools Released, EOS Trans., AGU, 76 (33), p. 329.
487 Wessel, P., and W. H. F. Smith, 1995, New Version of the Generic Map‐
488 ping Tools Released, http://www.agu.org/eos_elec/95154e.html, Copyright
489 1995 by the American Geophysical Union.
490 Wessel, P., and W. H. F. Smith, 1991, Free Software Helps Map and Dis‐
491 play Data, EOS Trans., AGU, 72 (41), p. 441.
492 The Marine Geophysical Data Exchange Format - "MGD77", see
493 http://www.ngdc.noaa.gov/mgg/dat/geodas/docs/mgd77.txt
494 IGRF, see http://www.ngdc.noaa.gov/IAGA/vmod/igrf.html
495GMT 4.5.6 10 Mar 2011 MGD77LIST(1)