1JT(1) JT MANUAL JT(1)
2
6 jt - transform JSON data to tabular format
7
9 jt [-hV]
10 jt -u string
11 jt [-acj] [COMMAND ...]
12
14 Jt reads UTF-8 encoded JSON forms from stdin and writes tab separated
15 values (or CSV) to stdout. A simple stack-based programming language is
16 used to extract values from the JSON input for printing.
17
19 -h Print usage info and exit.
20 -V Print version and license info and exit.
22 -a Explicit iteration mode: require the . command to iterate over
24 arrays instead of iterating automatically.
25 -c CSV output mode: write RFC 4180 compliant CSV records.
27 -j Inner join mode: discard rows with missing columns.
29 -s A no-op, included for compatibility with earlier versions.
31 -u string
33 Unescape JSON string, print it, and exit. Double-quotes around
34 string are optional.
35
37 Non-option arguments are words (commands) in a stack-based programming
38 language. Commands operate on the stacks provided by the jt runtime:
39 · The DATA stack contains JSON values commands will operate on.
41 · The GOSUB stack contains pointers into the data stack.
43 · The INDEX stack contains pointers to iterators during iteration.
45 · The LOOP stack contains pointers to iterators between iterations.
47 · The OUTPUT stack contains values to print after each iteration.
49 When jt starts, the following happens:
53 1. One JSON form is read from stdin, parsed, and pushed onto the data
55 stack.
56 2. If the top of the data stack is a JSON array the next item in the
58 array is pushed onto the data stack based on state previously saved
59 on the loop stack. The loop stack is then popped and pushed onto
60 the index stack.
61 3. The next command is executed.
63 4. Steps 2 and 3 are repeated until no commands are left to execute.
65 5. Values in the output stack are printed, separated by tabs and fol‐
67 lowed by a newline (see OUTPUT FORMAT below). Data, output, and
68 gosub stacks are reset to their states as they were after step 1.
69 6. Loop variables are popped off of the index stack. They are then
71 incremented and pushed onto the loop stack as necessary.
72 7. Steps 2 through 6 are repeated until the loop stack is empty (i.e.
74 there are no more iterations left).
75 This process is repeated until there is no more JSON to read.
79
81 Jt provides the following commands:
82 [ Save the state of the data stack: the current data stack pointer
84 is pushed onto the gosub stack.
85 ] Restore the data stack to a saved state: pop the gosub stack and
87 restore the data stack pointer to that state.
88 % Push the value at the top of the data stack onto the output
90 stack.
91 %=NAME Like %, but also sets the heading for this column to NAME. Head‐
93 ers will be printed as the first line of output when this com‐
94 mand has been used.
95 ^ Push the value at the top of the index stack onto the output
97 stack. Note that the index stack will always have at least one
98 item: the index of the last JSON object read from stdin.
99 ^=NAME Like ^, but also sets the heading for this column to NAME. Head‐
101 ers will be printed as the first line of output when this com‐
102 mand has been used.
103 @ Print the keys of the object at the top of the data stack and
105 exit.
106 If the value at the top of the data stack is an array the first
108 item in the array is pushed onto the data stack and the command
109 is reevaluated.
110 If the value at the top of the data stack is not an object or an
112 array then its type is printed in brackets, eg. [array],
113 [string], [number], [boolean], or [null]. If there is no value
114 then [none] is printed.
115 + Parse embedded JSON: if the item at the top of the data stack is
117 a string, parse it and push the result onto the data stack.
118 If the item at the top of the data stack is not a string or if
120 there is an error parsing the JSON then nothing is done (the
121 operation is a no-op).
122 . Iterate over the values of the object at the top of the data
124 stack. The current value will be pushed onto the data stack and
125 the current key will be pushed onto the index stack.
126 [KEY] Drill down: get the value of the KEY property of the object at
128 the top of the data stack and push that value onto the data
129 stack.
130 If the item at the top of the data stack is not an object or if
132 the object has no KEY property a blank field is printed, unless
133 the -j option was specified in which case the entire row is
134 removed from the output.
135 If the KEY property of the object is an array subsequent com‐
137 mands will operate on one of the items in the array, chosen
138 automatically by jt. The array index will be available to subse‐
139 quent commands via the index stack.
140 KEY See [KEY] above — the [ and ] may be omitted if the property
142 name KEY does not conflict with any jt command.
143
145 The format of printed output from jt (see % in COMMANDS, above) depends
146 on the type of thing being printed:
147 Primitives
149 JSON primitives (i.e. null, true, and false) and numbers are printed
150 verbatim. Jt does not process them in any way. Numbers can be of arbi‐
151 trary precision, as long as they conform to the JSON parsing grammar.
152 If special formatting is required the printf program is your friend:
154 $ printf %.0f 2.99792458e9
158 2997924580
159 Strings
163 Strings are printed verbatim, minus the enclosing double quotes. No
164 unescaping is performed because tabs or newlines in JSON strings would
165 break the tabular output format.
166 If unescaped values are desired either use the -c flag to enable CSV
168 output (see CSV below) or use the -u option and pass the string as an
169 argument:
170 $ jt -u ´i love music \u266A´
174 i love music ♪
175 Collections
179 Objects and arrays are printed as JSON with whitespace removed. Note
180 that it is only possible to print arrays when the -a option is speci‐
181 fied (see the Iteration (Arrays) and Explicit Iteration sections
182 below).
183 CSV
185 The -c flag enables CSV output conforming to RFC 4180. This format sup‐
186 ports strings containing tabs, newlines, etc.:
187 $ jt -c [ foo % ] [ bar % ] baz % <<EOT
191 - {
192 - "foo": "one\ttwo",
193 - "bar": "three\nfour",
194 - "baz": "music \"\u266A\""
195 - }
196 - EOT
197 "one two","three
198 four","music ""♪"""
199
203 Jt will exit with a status of 1 if an error occurred, or 0 otherwise.
204
206 Below are a number of examples demonstrating how to use jt commands to
207 do some simple exploration and extraction of data from JSON and JSON
208 streams.
209 Explore
211 The @ command prints information about the item at the top of the data
212 stack. When the item is an object @ prints its keys:
213 $ jt @ <<EOT
217 - {
218 - "foo": 100,
219 - "bar": 200,
220 - "baz": 300
221 - }
222 - EOT
223 foo
224 bar
225 baz
226 When the top item is an array @ prints information about the first item
230 in the array:
231 $ jt @ <<EOT
235 - [
236 - {"foo": 100, "bar": 200},
237 - {"baz": 300, "baf": 400}
238 - ]
239 - EOT
240 foo
241 bar
242 Otherwise, @ prints the type of the item, or [none] if there is no
246 value:
247 $ echo ´["hello", "world"]´ | jt @
251 [string]
252 $ echo ´[1, 2]´ | jt @
258 [number]
259 $ echo ´[true, false]´ | jt @
265 [boolean]
266 $ echo ´[null, null]´ | jt @
272 [null]
273 $ echo ´[]´ | jt @
279 [none]
280 Drill Down
284 Property names are also commands. Use foo here as a command to drill
285 down into the foo property and then use @ to print its keys:
286 $ jt foo @ <<EOT
290 - {
291 - "foo": {
292 - "bar": 100,
293 - "baz": 200
294 - }
295 - }
296 - EOT
297 bar
298 baz
299 A property name that conflicts with a jt command may be wrapped with
303 square brackets to drill down:
304 $ jt [@] @ <<EOT
308 - {
309 - "@": {
310 - "bar": 100,
311 - "baz": 200
312 - }
313 - }
314 - EOT
315 bar
316 baz
317 Note: The property name must be a JSON escaped string, so if the prop‐
321 erty name in the JSON input contains eg. tab characters they must be
322 escaped in the command, like this:
323 $ jt ´foo\tbar´ @ <<EOT
327 - {
328 - "foo\tbar": {
329 - "bar": 100,
330 - "baz": 200
331 - }
332 - }
333 - EOT
334 bar
335 baz
336 Extract
340 The % command prints the item at the top of the data stack. Note that
341 when the top item is a collection it is printed as JSON (insiginificant
342 whitespace removed):
343 $ jt % <<EOT
347 - {
348 - "foo": 100,
349 - "bar": 200
350 - }
351 - EOT
352 {"foo":100,"bar":200}
353 Drill down and print:
357 $ jt foo bar % <<EOT
361 - {
362 - "foo": {
363 - "bar": 100
364 - }
365 - }
366 - EOT
367 100
368 The % command can be used multiple times. The printed values will be
372 separated by tabs:
373 $ jt % foo % bar % <<EOT
377 - {
378 - "foo": {
379 - "bar": 100
380 - }
381 - }
382 - EOT
383 {"foo":{"bar":100}} {"bar":100} 100
384 Save / Restore
388 The [ and ] commands provide a way to save the data stack´s state
389 before performing some operations and restore it afterwards. This makes
390 it possible to drill down into one part of an object and then "rewind"
391 to drill down into another part of the same object.
392 The [ command saves the state of the data stack and the ] command
394 restores the data stack to the corresponding saved state. It is an
395 error to use the ] command without a corresponding [ command.
396 $ jt [ foo % ] bar % <<EOT
400 - {
401 - "foo": 100,
402 - "bar": 200
403 - }
404 - EOT
405 100 200
406 The [ and ] commands can be nested:
410 $ jt [ foo [ bar % ] [ baz % ] ] baf % <<EOT
414 - {
415 - "foo": {
416 - "bar": 100,
417 - "baz": 200
418 - },
419 - "baf": "quux"
420 - }
421 - EOT
422 100 200 quux
423 Iteration (Arrays)
427 Jt automatically iterates over arrays (unless this behavior is disabled
428 — see Explicit Iteration below), producing one tab-delimited record per
429 iteration, records separated by newlines:
430 $ jt [ foo % ] bar baz % <<EOT
434 - {
435 - "foo": 100,
436 - "bar": [
437 - {"baz": 200},
438 - {"baz": 300},
439 - {"baz": 400}
440 - ]
441 - }
442 - EOT
443 100 200
444 100 300
445 100 400
446 The ^ command includes the array index as a column in the result:
450 $ jt [ foo % ] bar ^ baz % <<EOT
454 - {
455 - "foo": 100,
456 - "bar": [
457 - {"baz": 200},
458 - {"baz": 300},
459 - {"baz": 400}
460 - ]
461 - }
462 - EOT
463 100 0 200
464 100 1 300
465 100 2 400
466 The ^ command is scoped to the innermost enclosing loop:
470 $ jt foo ^ bar ^ % <<EOT
474 - {
475 - "foo": [
476 - {"bar": [100, 200]},
477 - {"bar": [300, 400]}
478 - ]
479 - }
480 - EOT
481 0 0 100
482 0 1 200
483 1 0 300
484 1 1 400
485 Iteration (Objects)
489 The . command iterates over the values of an object:
490 $ jt . % <<EOT
494 - {
495 - "foo": 100,
496 - "bar": 200,
497 - "baz": 300
498 - }
499 - EOT
500 100
501 200
502 300
503 When iterating over an object the ^ command prints the name of the cur‐
507 rent property:
508 $ jt . ^ % <<EOT
512 - {
513 - "foo": 100,
514 - "bar": 200,
515 - "baz": 300
516 - }
517 - EOT
518 foo 100
519 bar 200
520 baz 300
521 JSON Streams
525 Jt automatically iterates over entities in a JSON stream (optionally
526 delimited by whitespace):
527 $ jt [ foo % ] bar % <<EOT
531 - {"foo": 100, "bar": 200}
532 - {"foo": 200, "bar": 300}
533 - {"foo": 300, "bar": 400}
534 - EOT
535 100 200
536 200 300
537 300 400
538 The ^ command prints the current stream index:
542 $ jt ^ [ foo % ] bar % <<EOT
546 - {"foo": 100, "bar": 200}
547 - {"foo": 200, "bar": 300}
548 - {"foo": 300, "bar": 400}
549 - EOT
550 0 100 200
551 1 200 300
552 2 300 400
553 Nested JSON
557 The + command parses JSON embedded in strings:
558 $ jt [ foo + bar % ] baz % <<EOT
562 - {"foo":"{\"bar\":100}","baz":200}
563 - {"foo":"{\"bar\":200}","baz":300}
564 - {"foo":"{\"bar\":300}","baz":400}
565 - EOT
566 100 200
567 200 300
568 300 400
569 Note: The + command does not modify the original JSON:
573 $ jt [ foo + bar % ] % <<EOT
577 - {"foo":"{\"bar\":100}","baz":200}
578 - {"foo":"{\"bar\":200}","baz":300}
579 - {"foo":"{\"bar\":300}","baz":400}
580 - EOT
581 100 {"foo":"{\"bar\":100}","baz":200}
582 200 {"foo":"{\"bar\":200}","baz":300}
583 300 {"foo":"{\"bar\":300}","baz":400}
584 Column Headings
588 The %=NAME and ^=NAME commands are like % and ^ above, but they also
589 assign column headings, which are printed as the first line of output:
590 $ jt [ foo %=Foo ] bar ^=Bar baz %=Baz <<EOT
594 - {
595 - "foo": 100,
596 - "bar": [
597 - {"baz": 200},
598 - {"baz": 300},
599 - {"baz": 400}
600 - ]
601 - }
602 - EOT
603 Foo Bar Baz
604 100 0 200
605 100 1 300
606 100 2 400
607 Note: The heading NAME must be a JSON escaped string (eg. to have a
611 column heading with embedded tab characters use %=foo\tbar instead of
612 "%=foo bar").
613 Joins
615 Notice the empty column — some objects don´t have the bar key:
616 $ jt [ foo % ] bar % <<EOT
620 - {"foo":100,"bar":1000}
621 - {"foo":200}
622 - {"foo":300,"bar":3000}
623 - EOT
624 100 1000
625 200
626 300 3000
627 Enable inner join mode with the -j flag. This removes output rows when
631 a key in the traversal path doesn´t exist:
632 $ jt -j [ foo % ] bar % <<EOT
636 - {"foo":100,"bar":1000}
637 - {"foo":200}
638 - {"foo":300,"bar":3000}
639 - EOT
640 100 1000
641 300 3000
642 Note that this does not remove rows when the key exists and the value
646 is empty:
647 $ jt -j [ foo % ] bar % <<EOT
651 - {"foo":100,"bar":1000}
652 - {"foo":200,"bar":""}
653 - {"foo":300,"bar":3000}
654 - EOT
655 100 1000
656 200
657 300 3000
658 Explicit Iteration
662 Sometimes the implicit iteration over arrays is awkward:
663 $ jt . ^ . ^ % <<EOT
667 - {
668 - "foo": [
669 - {"bar":100},
670 - {"bar":200}
671 - ]
672 - }
673 - EOT
674 0 bar 100
675 1 bar 200
676 Should the first ^ be printing the array index of the implicit itera‐
680 tion (which it does, in this case) or the object key (i.e. foo) of the
681 explicit iteration of the . command?
682 Another awkward case is printing arrays:
684 $ jt foo % <<EOT
688 - {
689 - "foo": [
690 - {"bar":100},
691 - {"bar":200}
692 - ]
693 - }
694 - EOT
695 {"bar":100}
696 {"bar":200}
697 The array can not be printed with the % command because it is being
701 iterated over implicitly. Instead, the items in the array are printed,
702 which may not be the desired behavior.
703 The -a flag eliminates the ambiguity by enabling explicit iteration. In
705 this mode the . command must be used to iterate over both objects and
706 arrays — arrays are not automatically iterated over.
707 Now the array can be printed:
709 $ jt -a foo % <<EOT
713 - {
714 - "foo": [
715 - {"bar":100},
716 - {"bar":200}
717 - ]
718 - }
719 - EOT
720 [{"bar":100},{"bar":200}]
721 Or the first ^ can print the array index, as before:
725 $ jt -a . . ^ . ^ % <<EOT
729 - {
730 - "foo": [
731 - {"bar":100},
732 - {"bar":200}
733 - ]
734 - }
735 - EOT
736 0 bar 100
737 1 bar 200
738 Or it can print the object key:
742 $ jt -a . ^ . . ^ % <<EOT
746 - {
747 - "foo": [
748 - {"bar":100},
749 - {"bar":200}
750 - ]
751 - }
752 - EOT
753 foo bar 100
754 foo bar 200
755 Or, with the addition of one more ^ command, it can print both:
759 $ jt -a . ^ . ^ . ^ % <<EOT
763 - {
764 - "foo": [
765 - {"bar":100},
766 - {"bar":200}
767 - ]
768 - }
769 - EOT
770 foo 0 bar 100
771 foo 1 bar 200
772
776 Jt is based on ideas from the excellent jshon tool, which can be found
777 at http://kmkeen.com/jshon/. Source code for jt is available at
778 https://github.com/micha/json-table.
779
781 Please open an issue here: https://github.com/micha/json-table/issues.
782
784 Copyright © 2017 Micha Niskin <micha.niskin@gmail.com>, distributed
785 under the Eclipse Public License, version 1.0. This is free software:
786 you are free to change and redistribute it. There is NO WARRANTY, to
787 the extent permitted by law.
788 July 2017 JT(1)