1jc(1) JSON Convert jc(1)
2
3
4
6 jc - JSON Convert JSONifies the output of many CLI tools, file-types,
7 and strings
8
10 Standard syntax:
11
12 COMMAND | jc [OPTIONS] PARSER
13
14 cat FILE | jc [OPTIONS] PARSER
15
16 echo STRING | jc [OPTIONS] PARSER
17
18 Magic syntax:
19
20 jc [OPTIONS] COMMAND
21
22 jc [OPTIONS] /proc/<path-to-procfile>
23
24
26 jc JSONifies the output of many CLI tools, file-types, and common
27 strings for easier parsing in scripts. jc accepts piped input from
28 STDIN and outputs a JSON representation of the previous command's out‐
29 put to STDOUT. Alternatively, the "Magic" syntax can be used by
30 prepending jc to the command to be converted. Options can be passed to
31 jc immediately before the command is given. (Note: "Magic" syntax does
32 not support shell builtins or command aliases)
33
34
36 Parsers:
37
38
39
40 --acpi `acpi` command parser
41
42
43 --airport
44 `airport -I` command parser
45
46
47 --airport-s
48 `airport -s` command parser
49
50
51 --arp `arp` command parser
52
53
54 --asciitable
55 ASCII and Unicode table parser
56
57
58 --asciitable-m
59 multi-line ASCII and Unicode table parser
60
61
62 --blkid
63 `blkid` command parser
64
65
66 --cbt `cbt` (Google Bigtable) command parser
67
68
69 --cef CEF string parser
70
71
72 --cef-s
73 CEF string streaming parser
74
75
76 --chage
77 `chage --list` command parser
78
79
80 --cksum
81 `cksum` and `sum` command parser
82
83
84 --clf Common and Combined Log Format file parser
85
86
87 --clf-s
88 Common and Combined Log Format file streaming parser
89
90
91 --crontab
92 `crontab` command and file parser
93
94
95 --crontab-u
96 `crontab` file parser with user support
97
98
99 --csv CSV file parser
100
101
102 --csv-s
103 CSV file streaming parser
104
105
106 --date `date` command parser
107
108
109 --datetime-iso
110 ISO 8601 Datetime string parser
111
112
113 --df `df` command parser
114
115
116 --dig `dig` command parser
117
118
119 --dir `dir` command parser
120
121
122 --dmidecode
123 `dmidecode` command parser
124
125
126 --dpkg-l
127 `dpkg -l` command parser
128
129
130 --du `du` command parser
131
132
133 --email-address
134 Email Address string parser
135
136
137 --env `env` command parser
138
139
140 --file `file` command parser
141
142
143 --findmnt
144 `findmnt` command parser
145
146
147 --finger
148 `finger` command parser
149
150
151 --free `free` command parser
152
153
154 --fstab
155 `/etc/fstab` file parser
156
157
158 --git-log
159 `git log` command parser
160
161
162 --git-log-s
163 `git log` command streaming parser
164
165
166 --git-ls-remote
167 `git ls-remote` command parser
168
169
170 --gpg `gpg --with-colons` command parser
171
172
173 --group
174 `/etc/group` file parser
175
176
177 --gshadow
178 `/etc/gshadow` file parser
179
180
181 --hash `hash` command parser
182
183
184 --hashsum
185 hashsum command parser (`md5sum`, `shasum`, etc.)
186
187
188 --hciconfig
189 `hciconfig` command parser
190
191
192 --history
193 `history` command parser
194
195
196 --hosts
197 `/etc/hosts` file parser
198
199
200 --id `id` command parser
201
202
203 --ifconfig
204 `ifconfig` command parser
205
206
207 --ini INI file parser
208
209
210 --ini-dup
211 INI with duplicate key file parser
212
213
214 --iostat
215 `iostat` command parser
216
217
218 --iostat-s
219 `iostat` command streaming parser
220
221
222 --ip-address
223 IPv4 and IPv6 Address string parser
224
225
226 --iptables
227 `iptables` command parser
228
229
230 --iso-datetime
231 Deprecated - please use datetime-iso
232
233
234 --iw-scan
235 `iw dev [device] scan` command parser
236
237
238 --iwconfig
239 `iwconfig` command parser
240
241
242 --jar-manifest
243 Java MANIFEST.MF file parser
244
245
246 --jobs `jobs` command parser
247
248
249 --jwt JWT string parser
250
251
252 --kv Key/Value file and string parser
253
254
255 --last `last` and `lastb` command parser
256
257
258 --ls `ls` command parser
259
260
261 --ls-s `ls` command streaming parser
262
263
264 --lsblk
265 `lsblk` command parser
266
267
268 --lsmod
269 `lsmod` command parser
270
271
272 --lsof `lsof` command parser
273
274
275 --lspci
276 `lspci -mmv` command parser
277
278
279 --lsusb
280 `lsusb` command parser
281
282
283 --m3u M3U and M3U8 file parser
284
285
286 --mdadm
287 `mdadm` command parser
288
289
290 --mount
291 `mount` command parser
292
293
294 --mpstat
295 `mpstat` command parser
296
297
298 --mpstat-s
299 `mpstat` command streaming parser
300
301
302 --netstat
303 `netstat` command parser
304
305
306 --nmcli
307 `nmcli` command parser
308
309
310 --ntpq `ntpq -p` command parser
311
312
313 --openvpn
314 openvpn-status.log file parser
315
316
317 --os-prober
318 `os-prober` command parser
319
320
321 --passwd
322 `/etc/passwd` file parser
323
324
325 --pci-ids
326 `pci.ids` file parser
327
328
329 --pgpass
330 PostgreSQL password file parser
331
332
333 --pidstat
334 `pidstat -H` command parser
335
336
337 --pidstat-s
338 `pidstat -H` command streaming parser
339
340
341 --ping `ping` and `ping6` command parser
342
343
344 --ping-s
345 `ping` and `ping6` command streaming parser
346
347
348 --pip-list
349 `pip list` command parser
350
351
352 --pip-show
353 `pip show` command parser
354
355
356 --plist
357 PLIST file parser
358
359
360 --postconf
361 `postconf -M` command parser
362
363
364 --proc `/proc/` file parser
365
366
367 --proc-buddyinfo
368 `/proc/buddyinfo` file parser
369
370
371 --proc-consoles
372 `/proc/consoles` file parser
373
374
375 --proc-cpuinfo
376 `/proc/cpuinfo` file parser
377
378
379 --proc-crypto
380 `/proc/crypto` file parser
381
382
383 --proc-devices
384 `/proc/devices` file parser
385
386
387 --proc-diskstats
388 `/proc/diskstats` file parser
389
390
391 --proc-filesystems
392 `/proc/filesystems` file parser
393
394
395 --proc-interrupts
396 `/proc/interrupts` file parser
397
398
399 --proc-iomem
400 `/proc/iomem` file parser
401
402
403 --proc-ioports
404 `/proc/ioports` file parser
405
406
407 --proc-loadavg
408 `/proc/loadavg` file parser
409
410
411 --proc-locks
412 `/proc/locks` file parser
413
414
415 --proc-meminfo
416 `/proc/meminfo` file parser
417
418
419 --proc-modules
420 `/proc/modules` file parser
421
422
423 --proc-mtrr
424 `/proc/mtrr` file parser
425
426
427 --proc-pagetypeinfo
428 `/proc/pagetypeinfo` file parser
429
430
431 --proc-partitions
432 `/proc/partitions` file parser
433
434
435 --proc-slabinfo
436 `/proc/slabinfo` file parser
437
438
439 --proc-softirqs
440 `/proc/softirqs` file parser
441
442
443 --proc-stat
444 `/proc/stat` file parser
445
446
447 --proc-swaps
448 `/proc/swaps` file parser
449
450
451 --proc-uptime
452 `/proc/uptime` file parser
453
454
455 --proc-version
456 `/proc/version` file parser
457
458
459 --proc-vmallocinfo
460 `/proc/vmallocinfo` file parser
461
462
463 --proc-vmstat
464 `/proc/vmstat` file parser
465
466
467 --proc-zoneinfo
468 `/proc/zoneinfo` file parser
469
470
471 --proc-driver-rtc
472 `/proc/driver/rtc` file parser
473
474
475 --proc-net-arp
476 `/proc/net/arp` file parser
477
478
479 --proc-net-dev
480 `/proc/net/dev` file parser
481
482
483 --proc-net-dev-mcast
484 `/proc/net/dev_mcast` file parser
485
486
487 --proc-net-if-inet6
488 `/proc/net/if_inet6` file parser
489
490
491 --proc-net-igmp
492 `/proc/net/igmp` file parser
493
494
495 --proc-net-igmp6
496 `/proc/net/igmp6` file parser
497
498
499 --proc-net-ipv6-route
500 `/proc/net/ipv6_route` file parser
501
502
503 --proc-net-netlink
504 `/proc/net/netlink` file parser
505
506
507 --proc-net-netstat
508 `/proc/net/netstat` file parser
509
510
511 --proc-net-packet
512 `/proc/net/packet` file parser
513
514
515 --proc-net-protocols
516 `/proc/net/protocols` file parser
517
518
519 --proc-net-route
520 `/proc/net/route` file parser
521
522
523 --proc-net-unix
524 `/proc/net/unix` file parser
525
526
527 --proc-pid-fdinfo
528 `/proc/<pid>/fdinfo/<fd>` file parser
529
530
531 --proc-pid-io
532 `/proc/<pid>/io` file parser
533
534
535 --proc-pid-maps
536 `/proc/<pid>/maps` file parser
537
538
539 --proc-pid-mountinfo
540 `/proc/<pid>/mountinfo` file parser
541
542
543 --proc-pid-numa-maps
544 `/proc/<pid>/numa_maps` file parser
545
546
547 --proc-pid-smaps
548 `/proc/<pid>/smaps` file parser
549
550
551 --proc-pid-stat
552 `/proc/<pid>/stat` file parser
553
554
555 --proc-pid-statm
556 `/proc/<pid>/statm` file parser
557
558
559 --proc-pid-status
560 `/proc/<pid>/status` file parser
561
562
563 --ps `ps` command parser
564
565
566 --route
567 `route` command parser
568
569
570 --rpm-qi
571 `rpm -qi` command parser
572
573
574 --rsync
575 `rsync` command parser
576
577
578 --rsync-s
579 `rsync` command streaming parser
580
581
582 --semver
583 Semantic Version string parser
584
585
586 --sfdisk
587 `sfdisk` command parser
588
589
590 --shadow
591 `/etc/shadow` file parser
592
593
594 --ss `ss` command parser
595
596
597 --sshd-conf
598 sshd config file and `sshd -T` command parser
599
600
601 --stat `stat` command parser
602
603
604 --stat-s
605 `stat` command streaming parser
606
607
608 --sysctl
609 `sysctl` command parser
610
611
612 --syslog
613 Syslog RFC 5424 string parser
614
615
616 --syslog-s
617 Syslog RFC 5424 string streaming parser
618
619
620 --syslog-bsd
621 Syslog RFC 3164 string parser
622
623
624 --syslog-bsd-s
625 Syslog RFC 3164 string streaming parser
626
627
628 --systemctl
629 `systemctl` command parser
630
631
632 --systemctl-lj
633 `systemctl list-jobs` command parser
634
635
636 --systemctl-ls
637 `systemctl list-sockets` command parser
638
639
640 --systemctl-luf
641 `systemctl list-unit-files` command parser
642
643
644 --systeminfo
645 `systeminfo` command parser
646
647
648 --time `/usr/bin/time` command parser
649
650
651 --timedatectl
652 `timedatectl status` command parser
653
654
655 --timestamp
656 Unix Epoch Timestamp string parser
657
658
659 --toml TOML file parser
660
661
662 --top `top -b` command parser
663
664
665 --top-s
666 `top -b` command streaming parser
667
668
669 --tracepath
670 `tracepath` and `tracepath6` command parser
671
672
673 --traceroute
674 `traceroute` and `traceroute6` command parser
675
676
677 --udevadm
678 `udevadm info` command parser
679
680
681 --ufw `ufw status` command parser
682
683
684 --ufw-appinfo
685 `ufw app info [application]` command parser
686
687
688 --uname
689 `uname -a` command parser
690
691
692 --update-alt-gs
693 `update-alternatives --get-selections` command parser
694
695
696 --update-alt-q
697 `update-alternatives --query` command parser
698
699
700 --upower
701 `upower` command parser
702
703
704 --uptime
705 `uptime` command parser
706
707
708 --url URL string parser
709
710
711 --vmstat
712 `vmstat` command parser
713
714
715 --vmstat-s
716 `vmstat` command streaming parser
717
718
719 --w `w` command parser
720
721
722 --wc `wc` command parser
723
724
725 --who `who` command parser
726
727
728 --x509-cert
729 X.509 PEM and DER certificate file parser
730
731
732 --xml XML file parser
733
734
735 --xrandr
736 `xrandr` command parser
737
738
739 --yaml YAML file parser
740
741
742 --zipinfo
743 `zipinfo` command parser
744
745
746
747 Options:
748
749
750 -a, --about
751 About jc (JSON or YAML output)
752
753 -C, --force-color
754 Force color output even when using pipes (overrides -m
755 and the NO_COLOR env variable)
756
757 -d, --debug
758 Debug - show traceback (use -dd for verbose traceback)
759
760 -h, --help
761 Help (--help --parser_name for parser documentation). Use
762 twice to show hidden parsers (e.g. -hh)
763
764 -m, --monochrome
765 Monochrome output
766
767 -M, --meta-out
768 Add metadata to output including timestamp, parser name,
769 magic command, magic command exit code, etc.
770
771 -p, --pretty
772 Pretty print output
773
774 -q, --quiet
775 Quiet mode. Suppresses parser warning messages (use -qq
776 to ignore streaming parser errors)
777
778 -r, --raw
779 Raw output. Provides more literal output, typically with
780 string values and no additional semantic processing
781
782 -u, --unbuffer
783 Unbuffer output (useful for slow streaming data with
784 streaming parsers)
785
786 -v, --version
787 Version information
788
789 -y, --yaml-out
790 YAML output
791
792 -B, --bash-comp
793 Generate Bash shell completion script
794
795 -Z, --zsh-comp
796 Generate Zsh shell completion script
797
798
800 Any fatal errors within jc will generate an exit code of 100, otherwise
801 the exit code will be 0.
802
803 When using the "magic" syntax (e.g. jc ifconfig eth0), jc will store
804 the exit code of the program being parsed and add it to the jc exit
805 code. This way it is easier to determine if an error was from the
806 parsed program or jc.
807
808 Consider the following examples using ifconfig:
809
810 ifconfig exit code = 0, jc exit code = 0, combined exit code = 0
811 (no errors)
812
813 ifconfig exit code = 1, jc exit code = 0, combined exit code = 1
814 (error in ifconfig)
815
816 ifconfig exit code = 0, jc exit code = 100, combined exit code =
817 100 (error in jc)
818
819 ifconfig exit code = 1, jc exit code = 100, combined exit code =
820 101 (error in both ifconfig and jc)
821
822 When using the "magic" syntax you can also retrieve the exit code of
823 the called program by using the --meta-out or -M option. This will ap‐
824 pend a _jc_meta object to the output that will include the magic com‐
825 mand information, including the exit code.
826
827 Here is an example with ping:
828 $ jc --meta-out -p ping -c2 192.168.1.252
829 {
830 "destination_ip": "192.168.1.252",
831 "data_bytes": 56,
832 "pattern": null,
833 "destination": "192.168.1.252",
834 "packets_transmitted": 2,
835 "packets_received": 0,
836 "packet_loss_percent": 100.0,
837 "duplicates": 0,
838 "responses": [
839 {
840 "type": "timeout",
841 "icmp_seq": 0,
842 "duplicate": false
843 }
844 ],
845 "_jc_meta": {
846 "parser": "ping",
847 "timestamp": 1661357115.27949,
848 "magic_command": [
849 "ping",
850 "-c2",
851 "192.168.1.252"
852 ],
853 "magic_command_exit": 2
854 }
855 }
856 $ echo $?
857 2
858
859
861 Custom Colors
862
863 You can specify custom colors via the JC_COLORS environment variable.
864 The JC_COLORS environment variable takes four comma separated string
865 values in the following format:
866
867 JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>
868
869 Where colors are: black, red, green, yellow, blue, magenta, cyan, gray,
870 brightblack, brightred, brightgreen, brightyellow, brightblue, bright‐
871 magenta, brightcyan, white, or default
872
873 For example, to set to the default colors:
874
875 JC_COLORS=blue,brightblack,magenta,green
876
877 or
878
879 JC_COLORS=default,default,default,default
880
881 Disable Color Output
882
883 You can set the NO_COLOR environment variable to any value to disable
884 color output in jc. Note that using the -C option to force color output
885 will override both the NO_COLOR environment variable and the -m option.
886
887
889 Most parsers load all of the data from STDIN, parse it, then output the
890 entire JSON document serially. There are some streaming parsers (e.g.
891 ls-s, ping-s, etc.) that immediately start processing and outputting
892 the data line-by-line as JSON Lines (aka NDJSON) while it is being re‐
893 ceived from STDIN. This can significantly reduce the amount of memory
894 required to parse large amounts of command output (e.g. ls -lR /) and
895 can sometimes process the data more quickly. Streaming parsers have
896 slightly different behavior than standard parsers as outlined below.
897
898 Note: Streaming parsers cannot be used with the "magic" syntax
899
900 Ignoring Errors
901
902 You may want to ignore parsing errors when using streaming parsers
903 since these may be used in long-lived processing pipelines and errors
904 can break the pipe. To ignore parsing errors, use the -qq cli option.
905 This will add a _jc_meta object to the JSON output with a success at‐
906 tribute. If success is true, then there were no issues parsing the
907 line. If success is false, then a parsing issue was found and error and
908 line fields will be added to include a short error description and the
909 contents of the unparsable line, respectively:
910
911 Successfully parsed line with -qq option:
912 {
913 "command_data": "data",
914 "_jc_meta": {
915 "success": true
916 }
917 }
918
919 Unsuccessfully parsed line with -qq option:
920 {
921 "_jc_meta": {
922 "success": false,
923 "error": "error message",
924 "line": "original line data"
925 }
926 }
927
928 Unbuffering Output
929
930 Most operating systems will buffer output that is being piped from
931 process to process. The buffer is usually around 4KB. When viewing the
932 output in the terminal the OS buffer is not engaged so output is imme‐
933 diately displayed on the screen. When piping multiple processes to‐
934 gether, though, it may seem as if the output is hanging when the input
935 data is very slow (e.g. ping):
936
937 $ ping 1.1.1.1 | jc --ping-s | jq
938 <slow output>
939
940 This is because the OS engages the 4KB buffer between jc and jq in this
941 example. To display the data on the terminal in realtime, you can dis‐
942 able the buffer with the -u (unbuffer) cli option:
943
944 $ ping 1.1.1.1 | jc --ping-s -u | jq
945 {"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
946 {"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
947 etc...
948
949 Note: Unbuffered output can be slower for large data streams.
950
951
953 Custom local parser plugins may be placed in a jc/jcparsers folder in
954 your local "App data directory":
955
956 - Linux/unix: $HOME/.local/share/jc/jcparsers
957 - macOS: $HOME/Library/Application Support/jc/jcparsers
958 - Windows: $LOCALAPPDATA\jc\jc\jcparsers
959
960 Local parser plugins are standard python module files. Use the
961 jc/parsers/foo.py or jc/parsers/foo_s.py (streaming) parser as a tem‐
962 plate and simply place a .py file in the jcparsers subfolder.
963
964 Local plugin filenames must be valid python module names and therefore
965 must start with a letter and consist entirely of alphanumerics and un‐
966 derscores. Local plugins may override default parsers.
967
968 Note: The application data directory follows the XDG Base Directory
969 Specification
970
971
973 Locale
974
975 For best results set the locale environment variables to C or
976 en_US.UTF-8 by modifying the LC_ALL variable:
977
978 $ LC_ALL=C date | jc --date
979
980 You can also set the locale variables individually:
981
982 $ export LANG=C
983
984 $ export LC_NUMERIC=C
985
986 On some older systems UTF-8 output will be downgraded to ASCII with \u
987 escape sequences if the C locale does not support UTF-8 encoding.
988
989 Timezones
990
991 Some parsers have calculated epoch timestamp fields added to the out‐
992 put. Unless a timestamp field name has a _utc suffix it is considered
993 naive. (i.e. based on the local timezone of the system the jc parser
994 was run on).
995
996 If a UTC timezone can be detected in the text of the command output,
997 the timestamp will be timezone aware and have a _utc suffix on the key
998 name. (e.g. epoch_utc) No other timezones are supported for aware time‐
999 stamps.
1000
1001
1003 Standard Syntax:
1004 $ dig www.google.com | jc -p --dig
1005
1006 $ cat /proc/meminfo | jc --pretty --proc
1007
1008 Magic Syntax:
1009 $ jc --pretty dig www.google.com
1010
1011 $ jc --pretty /proc/meminfo
1012
1013 For parser documentation:
1014 $ jc --help --dig
1015
1017 Kelly Brazil (kellyjonbrazil@gmail.com)
1018
1019 https://github.com/kellyjonbrazil/jc
1020
1021
1023 Copyright (c) 2019-2023 Kelly Brazil
1024
1025 License: MIT License
1026
1027
1028
10291.22.5 2023-01-11 jc(1)