1MGD77SNIFFER(1) GMT MGD77SNIFFER(1)
2
6 mgd77sniffer - Along-track quality control of MGD77 cruises
7
9 mgd77sniffer NGDC-ids [ -Afieldabbrev,scale,offset ] [ -Cmaxspd ] [
10 -Dd|e|E|f|l|m|s|v[r] ] [ -E ] [ -Gfieldabbrev,imggrid,scale,mode or
11 -Gfieldabbrev,grid ] [ -H ] [ -Ifieldabbrev,rec1,recN ] [ -Lcus‐
12 tom-limits-file ] [ -M ] [ -N ] [ -Rregion ] [ -Sd|s|t ] [ -Tgap ]
13 [ -V[level] ] [ -Wc|g|o|s|t|v|x ] [ -Zlevel ] [ -bobinary ] [ -dono‐
14 data ] [ -nflags ]
15 Note: No space is allowed between the option flag and the associated
17 arguments.
18
20 mgd77sniffer scans old (pre-Y2K) and new format ASCII MGD77 files for
21 errors using point-by-point sanity checking, along-track detection of
22 excessive slopes, and optional comparison of cruise data with global
23 gravity and predicted bathymetry grids. Detected data problems are out‐
24 put by default as verbose descriptions of each detected error, often
25 resulting in multiple messages per scanned record. Data problems are
26 optionally output (-De option) using a computer-parseable format (see
27 E77 ERROR FORMAT description below). Default error thresholds are
28 derived from histograms of all MGD77 geophysical data collected between
29 1952 and January, 2006. Thresholds are adjustable with the -L option.
30
32 The mgd77sniffer links with Generic Mapping Tools 4.0 or later along
33 with the supplemental GMT packages x2sys and mgd77. See
34 http://gmt.soest.hawaii.edu for GMT details. Grids for comparison with
35 cruise data may be downloaded via the web.
36
38 NGDC-ids
39 Can be one or more of five kinds of specifiers:
40 1. 8-character NGDC IDs, e.g., 01010083, JA010010etc., etc.
42 2. 2-character agency codes which will return all cruises from
44 each agency.
45 3. 4-character <agency><vessel> codes, which will return all
47 cruises from those vessels.
48 4. =list, where list is a table with NGDC IDs, one per line.
50 5. If nothing is specified we return all cruises in the data
52 base.
53 (See mgd77info -L for agency and vessel codes). If no file
55 extension is given then we search for files with one of the four
56 known extensions. The search order (and the extensions) tried
57 is MGD77+ (".nc"), MGD77T (".m77t"), MGD77 (".mgd77" ) and plain
58 text file (".dat"). Use -I to ignore one or more of these file
59 types). Cruise files will be looked for first in the current
60 directory and second in all directories listed in
61 $MGD77_HOME/mgd77_paths.txt [If $MGD77_HOME is not set it will
62 default to $GMT_SHAREDIR/mgd77].
63
65 -Afieldabbrev,scale,offset
66 Apply scale factor and DC adjustment to specified data field.
67 Allows adjustment of cruise data prior to along-track analysis.
68 CAUTION: data must be thoroughly examined before applying these
69 global data adjustments. May not be used for multiple cruises.
70 -Cmaxspd
72 Set maximum ship speed in m/s, or knots with -N option. Ship
73 speeds exceeding 10 m/s (~20 knots) are flagged as excessive by
74 default.
75 -Dd|e|E|f|l|m|s|v[r]
77 Suppress default warning output and only dump cruise data
78 row-by-row such as values, gradients, grid-cruise differences,
79 E77 error summaries for each record, re-created MGD77 records or
80 sniffer limits. Append r to include all records (default omits
81 records where navigation errors were detected).
82 -Dd output differences between cruise and grid data. Requires -G
84 option. Output columns include:
85 lat lon dist cruiseZ gridZ diff [cruiseZ2 gridZ2 diff2 ...]
87 Note: grid values are subtracted from cruise data so a positive
89 difference implies cruise > grid. For multiple grid comparison,
90 cruiseZ gridZ diff are repeated for each grid comparison in com‐
91 mand line order.
92 -De output E77 error classification format. Error output is
94 divided into (1) a header containing information globally appli‐
95 cable to the cruise and (2) individual error records summarizing
96 all errors encountered in each cruise record. mgd77sniffer
97 writes E77 directly to <ngdc_id.e77> file handle. See E77 ERROR
98 FORMAT below for additional details.
99 -DE Same as -De but no regression tests will be carried out.
101 -Df output delta Z (change in geophysical field) column and
103 delta S (change in distance) for each geophysical field. Dis‐
104 tance between observations often differ for different fields
105 depending on instrument sampling rate, so ds is included for
106 each geophysical observation. Output columns include:
107 d[twt] ds d[depth] ds d[mtf1] ds d[mtf2] ds d[mag] ds d[diur] ds
109 d[msd] ds d[gobs] ds d[eot] ds d[faa] ds
110 -Dl display mgd77sniffer limits. Customize this output to create
112 a custom limits file for the -L option. No additional arguments
113 are required. Output columns include:
114 fieldabbrev min max maxSlope maxArea
116 -Dm output MGD77 format records in Y2K-compliant MGD77 format
118 -Dn output distance to coast for each record. Requires the -Gnav
120 option. Output columns include:
121 lat lon dist distToCoast
123 -Ds output calculated gradients for speed and geophysical
125 fields. Gradients correspond to the gradient type selected in
126 the -S option (spatial derivatives by default). Output columns
127 include:
128 speed d[twt] d[depth] d[mtf1] d[mtf2] d[mag] d[diur] d[msd]
130 d[gobs] d[eot] d[faa]
131 See MGD77 FIELD INFO below for field and abbreviations descrip‐
133 tions.
134 -Dv display values for the twelve position and geophysical
136 fields for each MGD77 data record (in this order):
137 lat lon twt depth mtf1 mtf2 mag diur msens gobs eot faa
139 See below for MGD77 FIELD INFO.
141 -E Reverse navigation quality flags (good to bad and vice versa).
143 May be necessary when a majority of navigation fixes are erro‐
144 neously flagged bad, which can happen when a cruise's first nav‐
145 igation fix is extremely erroneous. Caution! This will affect
146 sniffer output and should only be attempted after careful manual
147 navigation review.
148 -Ginformation
150 Compare cruise data to GMT or IMG grids. Use one of the formats
151 below. -Gfieldabbrev,imggrid,scale,mode Compare cruise data to
152 the specified grid in Sandwell/Smith Mercator format. Requires a
153 valid MGD77 field abbreviation (see MGD77 FIELD INFO below) fol‐
154 lowed by a comma, the path (if not in current directory) and
155 grid filename, a scale to multiply the data (1 or 0.1), and mode
156 which stand for the following: (0) Img files with no constraint
157 code, returns data at all points, (1) Img file with constraints
158 coded, return data at all points, (2) Img file with constraints
159 coded, return data only at constrained points and NaN elsewhere,
160 and (3) Img file with constraints coded, return 1 at constraints
161 and 0 elsewhere. -Gfieldabbrev,grid Compare cruise data to the
162 specified grid. Requires a valid MGD77 field abbreviation (see
163 MGD77 FIELD INFO below) followed by a comma, then the path (if
164 not in current directory) and grid filename. Multiple grid com‐
165 parison is supported by using separate -G calls for each grid.
166 See GRID FILE INFO below.
167 Grid comparison activates several additional error checks. (1)
169 Re-weighted Least Squares Regression of ship versus grid data
170 determines slope and DC shift, which when differing from
171 expected 1 and 0, respectively, may indicate incorrectly scaled
172 ship data, including incorrect units or instrument drift as well
173 as erroneous gravity tie-in. (2) Accumulated ship grid offsets
174 are computed along-track and excessive offsets are flagged
175 according to maxArea threshold (use -L option to adjust
176 maxArea). Warning: predicted bathymetry grids are constrained by
177 cruise data so grids and cruise data are not always independent.
178 Comparison of cruise bathymetry with predicted bathymetry grids
179 also activates a "navigation crossing over land" check.
180 -H (with -G|g only) disable (or force) decimation during RLS analy‐
182 sis of ship and gridded data. By default mgd77sniffer analyses
183 both the full and decimated data sets then reports RLS statis‐
184 tics for the higher correlation regression.
185 -Hb analyze both (default), report better of two.
187 -Hd to disable data decimation (equivalent to -H with no argu‐
189 ment).
190 -Hf to force data decimation.
192 -Ifieldabbrev,rec1,recN
194 Append a field abbreviation and the first and last record in a
195 range of records that should be flagged as bad (and set to NaN
196 prior to the analysis). Repeat as many times as needed. May not
197 be used for multiple cruises.
198 -Lcustom-limits-file
200 Override mgd77sniffer default error detection limits. Supply
201 path and filename to the custom limits file. Rows not beginning
202 with a valid MGD77 field abbreviation are ignored. Field abbre‐
203 viations are listed below in exact form under MGD77 FIELD INFO.
204 Multiple field limits may be modified using one default file,
205 one field per line. Field min, max, max slope and max area may
206 be changed for each field. Max slope pertains to the gradient
207 type selected using the -S option. Max area is used by the -G
208 option as the threshold for flagging excessive offsets from the
209 specified grid. Dump defaults -Dl to view syntax or to quickly
210 create an editable custom limits file.
211 Example custom default file contents (see below for units):
213 ┌─────────┬──────┬───────┬──────────┬─────────┐
215 │# abbrev │ min │ max │ maxSlope │ maxArea │
216 ├─────────┼──────┼───────┼──────────┼─────────┤
217 │twt │ 0 │ 15 │ 1 │ 0 │
218 ├─────────┼──────┼───────┼──────────┼─────────┤
219 │depth │ 0 │ 11000 │ 500 │ 5000 │
220 ├─────────┼──────┼───────┼──────────┼─────────┤
221 │mag │ -800 │ 800 │ -- │ -- │
222 ├─────────┼──────┼───────┼──────────┼─────────┤
223 │faa │ -300 │ 300 │ 100 │ 2500 │
224 └─────────┴──────┴───────┴──────────┴─────────┘
225 Use a dash '-' to retain a default limit. Hint: to test your
227 custom limits, try: mgd77sniffer -Dl -L<yourlimitsfile>
228 -M Adjust navigation on land threshold (meters inland) [100].
230 -N Use nautical units.
232 -Rwest/east/south/north[/zmin/zmax][+r][+uunit]
234 west, east, south, and north specify the region of interest, and
235 you may specify them in decimal degrees or in
236 [±]dd:mm[:ss.xxx][W|E|S|N] format Append +r if lower left and
237 upper right map coordinates are given instead of w/e/s/n. The
238 two shorthands -Rg and -Rd stand for global domain (0/360 and
239 -180/+180 in longitude respectively, with -90/+90 in latitude).
240 Alternatively for grid creation, give Rcodelon/lat/nx/ny, where
241 code is a 2-character combination of L, C, R (for left, center,
242 or right) and T, M, B for top, middle, or bottom. e.g., BL for
243 lower left. This indicates which point on a rectangular region
244 the lon/lat coordinate refers to, and the grid dimensions nx and
245 ny with grid spacings via -I is used to create the corresponding
246 region. Alternatively, specify the name of an existing grid
247 file and the -R settings (and grid spacing, if applicable) are
248 copied from the grid. Appending +uunit expects projected (Carte‐
249 sian) coordinates compatible with chosen -J and we inversely
250 project to determine actual rectangular geographic region. For
251 perspective view (-p), optionally append /zmin/zmax. In case of
252 perspective view (-p), a z-range (zmin, zmax) can be appended to
253 indicate the third dimension. This needs to be done only when
254 using the -Jz option, not when using only the -p option. In the
255 latter case a perspective view of the plane is plotted, with no
256 third dimension.
257 -Sd|s|t
259 Specify gradient type for along-track excessive slope checking.
260 -Sd Calculate change in z values along track (dz). Output is
261 given in geophysical units, e.g., mGal. -Ss Calculate spatial
262 gradients (dz/ds). Output is given in geophysical units per km
263 along the survey track, e.g., mGal/km. -St Calculate time gra‐
264 dients (dz/dt) [default]. Output is given in geophysical units
265 per second along the survey track, e.g., mGal/sec.
266 -Tgap Adjusts mgd77sniffer gap handling. By default, data gaps greater
268 than 5 km are skipped. Set to zero to de-activate gap skipping.
269 -Wc|g|o|s|t|v|x
271 Print out only certain warning types for verbose error messages.
272 Comma delimit any combination of c|g|o|s|t|v|x: where (c) type
273 code warnings, (g)radient out of range, (o)ffsets from grid
274 (requires -G|g), (s)peed out of range, (t)ime warnings, (v)alue
275 out of range, (x) warning summaries. By default ALL warning mes‐
276 sages are printed.Not compatible with any -D options.
277 -Z Flag regression statistics that are outside the specified confi‐
279 dence level. (i.e., -Z5 flags coefficients m, b, rms, and r that
280 fall outside 95%.)
281 -V[level] (more ...)
283 Select verbosity level [c].
284 -bo[ncols][type] (more ...)
286 Select native binary output. Output binary data for -Dd|f|s|v
287 option.
288 -donodata (more ...)
290 Replace output columns that equal NaN with nodata.
291 -n[b|c|l|n][+a][+bBC][+c][+tthreshold] (more ...)
293 Select interpolation mode for grids.
294 -^ or just -
296 Print a short message about the syntax of the command, then
297 exits (NOTE: on Windows just use -).
298 -+ or just +
300 Print an extensive usage (help) message, including the explana‐
301 tion of any module-specific option (but not the GMT common
302 options), then exits.
303 -? or no arguments
305 Print a complete usage (help) message, including the explanation
306 of all options, then exits.
307
309 Resample or sampling of grids will use various algorithms (see -n) that
310 may lead to possible distortions or unexpected results in the resampled
311 values. One expected effect of resampling with splines is the tendency
312 for the new resampled values to slightly exceed the global min/max lim‐
313 its of the original grid. If this is unacceptable, you can impose
314 clipping of the resampled values values so they do not exceed the input
315 min/max values by adding +c to your -n option.
316
318 ┌──────────┬──────────────┬───────────┐
319 │Field │ Abbreviation │ Units │
320 ├──────────┼──────────────┼───────────┤
321 │Two-way │ Travel │ Time │
322 ├──────────┼──────────────┼───────────┤
323 │Corrected │ Depth │ depth │
324 ├──────────┼──────────────┼───────────┤
325 │Mag │ Total │ Field1 │
326 ├──────────┼──────────────┼───────────┤
327 │Mag │ Total │ Field2 │
328 ├──────────┼──────────────┼───────────┤
329 │Residual │ Magnetic │ mag │
330 ├──────────┼──────────────┼───────────┤
331 │Diurnal │ Correction │ diur │
332 ├──────────┼──────────────┼───────────┤
333 │Mag │ Sensor │ Depth/Alt │
334 └──────────┴──────────────┴───────────┘
335 │Observed │ Gravity │ gobs │
337 ├──────────┼──────────────┼───────────┤
338 │Eotvos │ Correction │ eot │
339 ├──────────┼──────────────┼───────────┤
340 │Free │ Air │ Anomaly │
341 └──────────┴──────────────┴───────────┘
342
344 For -G the grids must eitehr be in the format used by Sandwell &
345 Smith, which is a spherical Mercator 2-byte grid with no header, or any
346 grid type supported by GMT and therefore must contain a GMT header. A
347 correctly formatted *.i2 grid file can be generated using grdraster as
348 shown below.
349 gmtset GRIDFILE_SHORTHAND TRUE
351 Create/edit .gmt_io file to include the following rows:
353 # GMT I/O shorthand file
355 # suffix format_id scale offset NaN
357 grd 0 - - - -
359 i2 2 - - 32767
361 gmt grdraster 1 -R0/359:55/-90/90 -Getopo5_hdr.i2
363 The new grid, etopo5_hdr.i2 in this example, contains a GMT header and
365 can be used in the -G option to compare cruise depth with grid values.
366
368 Header Information pertaining to an entire cruise, such as NGDC and
369 survey institution identification codes, cruise examination
370 time, two-way travel time corrector information, data precision
371 warnings, as well as systematic scales, DC shifts and correla‐
372 tion coefficients from global grid comparisons are reported as
373 E77 header information.
374 Sample
376 # Cruise 08010039 ID 74010908 MGD77 FILE VERSION: 19801230
378 N_RECS: 3066
379 # Examined: Wed Oct 3 16:30:13 2007 by mtchandl
381 # Arguments: -De -Gdepth,/data/GRIDS/etopo5_hdr.i2
383 N Errata table verification status
385 # mgd77manage applies corrections if the errata table is veri‐
387 fied (toggle 'N' above to 'Y' after review)
388 # For instructions on E77 format and usage, see
390 http://gmt.soest.hawaii.edu/mgd77/errata.php
391 # Verified by:
393 # Comments:
395 # Errata: Header
397 Y-E-08010039-H13-02: Invalid Magnetics Sampling Rate: (99) [ ]
399 Y-W-08010039-H13-10: Survey year (1975) outside magnetic refer‐
401 ence field IGRF 1965 time range (1965-1970)
402 Y-I-08010039-depth-00: RLS m: 1.00053 b: 0 rms: 127.851 r:
404 0.973422 significant: 1 decimation: 0
405 Y-W-08010039-twt-09: More recent bathymetry correction table
407 available
408 Y-W-08010039-mtf1-10: Integer precision
410 Y-W-08010039-mag-10: Integer precision
412 Error Record
414 Individual error records have strict format. Included is a time
415 or distance column followed by record number, a formatted error
416 code string, and finally a verbose description of errors
417 detected in the record. Three error classes are encoded into the
418 error code string with different alphabetic characters repre‐
419 senting unique error types. See below for error code format
420 description.
421 Format <time/distance> <record number> <error code string>
423 <description>
424 Sample
426 # Errata: Data
428 Y 08010039 1975-05-10T22:16:05.88 74 C-0-0 NAV: excessive
430 speed
431 Error Code Description
433 Each of the three error classes is separated by a dash - and
434 described by a combination of alphabetic characters or 0 signi‐
435 fying no detected problems.
436 Error classes: NAV-VAL-GRAD
438 Error Class Descriptions
440 NAV (navigation):
442 0 - fine
444 A - time out of range
446 B - time decreasing
448 C - excessive speed
450 D - above sea level
452 E - lat undefined
454 F - lon undefined
456 VAL (value):
458 0 - fine
460 K - twt invalid
462 L - depth invalid
464 O - mtf1 invalid
466 etc.
468 GRAD (gradient):
470 0 - fine
472 K - d[twt] excessive
474 L - d[depth] excessive
476 O - d[mtf1] excessive
478 etc.
479 The NAV error class has unique cases while VAL and GRAD classes
481 are described by alphabetic characters for each of the 24
482 numeric fields in MGD77 format order.
483 MGD77 bit-pattern w/ E77 alpha characters
485 |-------------------------------------------------|----------|
487 | X W V U T S R Q P O N M L K J I H G F E D C B A | E77 Code |
489 | - - - - - - - - - - - - - - - - - - - - - - - - | - - - - -|
491 | n f e g m d m m m m b b d t p l l m h d m y t d | F I |
493 | q a o o s i s a t t t c e w t o a i o a o e z r | i D |
495 | c a t b d u e g f f c c p t c n t n u y n a t | e |
497 | s r n 2 1 t r t r | l |
499 | s h h | d |
501 | - - - - - - - - - - - - - - - - - - - - - - - - | - - - - -|
503 | 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | Bit place|
505 | - G C G C C - G G G - - G G - - - T T T T T - - | Bit type |
507 |-------------------------------------------------|----------|
509 Bit types: (G)eophysical, (C)orrection, (T)ime
511
513 To scan for excessive values or gradients, try
514 gmt mgd77sniffer 08010001
516 To dump cruise gradients, try
518 gmt mgd77sniffer 08010001 -Ds
520 To compare cruise depth with ETOPO5 bathymetry and gravity with
522 Sandwell/Smith 2 min gravity version 11, try
523 mgd77sniffer 08010001 -Gdepth,/data/GRIDS/etopo5_hdr.i2 \
525 -Gfaa,/data/GRIDS/grav.11.2.img,0.1,1
526
528 grdraster, mgd77list, mgd77track x2sys_init
529
531 The Marine Geophysical Data Exchange Format - MGD77, see
532 http://www.ngdc.noaa.gov/mgg/dat/geodas/docs/mgd77.txt.
533
535 2019, P. Wessel, W. H. F. Smith, R. Scharroo, J. Luis, and F. Wobbe
5365.4.5 Feb 24, 2019 MGD77SNIFFER(1)