1jc(1) JSON Convert jc(1)
2
6 jc - JSON Convert JSONifies the output of many CLI tools, file-types,
7 and strings
8
10 Standard syntax:
11 COMMAND | jc [SLICE] [OPTIONS] PARSER
13 cat FILE | jc [SLICE] [OPTIONS] PARSER
15 echo STRING | jc [SLICE] [OPTIONS] PARSER
17 Magic syntax:
19 jc [SLICE] [OPTIONS] COMMAND
21 jc [SLICE] [OPTIONS] /proc/<path-to-procfile>
23
26 jc JSONifies the output of many CLI tools, file-types, and common
27 strings for easier parsing in scripts. jc accepts piped input from
28 STDIN and outputs a JSON representation of the previous command's out‐
29 put to STDOUT. Alternatively, the "Magic" syntax can be used by
30 prepending jc to the command to be converted. Options can be passed to
31 jc immediately before the command is given. (Note: "Magic" syntax does
32 not support shell builtins or command aliases)
33
36 Parsers:
37 --acpi `acpi` command parser
41 --airport
44 `airport -I` command parser
45 --airport-s
48 `airport -s` command parser
49 --arp `arp` command parser
52 --asciitable
55 ASCII and Unicode table parser
56 --asciitable-m
59 multi-line ASCII and Unicode table parser
60 --blkid
63 `blkid` command parser
64 --bluetoothctl
67 `bluetoothctl` command parser
68 --cbt `cbt` (Google Bigtable) command parser
71 --cef CEF string parser
74 --cef-s
77 CEF string streaming parser
78 --certbot
81 `certbot` command parser
82 --chage
85 `chage --list` command parser
86 --cksum
89 `cksum` and `sum` command parser
90 --clf Common and Combined Log Format file parser
93 --clf-s
96 Common and Combined Log Format file streaming parser
97 --crontab
100 `crontab` command and file parser
101 --crontab-u
104 `crontab` file parser with user support
105 --csv CSV file parser
108 --csv-s
111 CSV file streaming parser
112 --date `date` command parser
115 --datetime-iso
118 ISO 8601 Datetime string parser
119 --df `df` command parser
122 --dig `dig` command parser
125 --dir `dir` command parser
128 --dmidecode
131 `dmidecode` command parser
132 --dpkg-l
135 `dpkg -l` command parser
136 --du `du` command parser
139 --email-address
142 Email Address string parser
143 --env `env` command parser
146 --file `file` command parser
149 --find `find` command parser
152 --findmnt
155 `findmnt` command parser
156 --finger
159 `finger` command parser
160 --free `free` command parser
163 --fstab
166 `/etc/fstab` file parser
167 --git-log
170 `git log` command parser
171 --git-log-s
174 `git log` command streaming parser
175 --git-ls-remote
178 `git ls-remote` command parser
179 --gpg `gpg --with-colons` command parser
182 --group
185 `/etc/group` file parser
186 --gshadow
189 `/etc/gshadow` file parser
190 --hash `hash` command parser
193 --hashsum
196 hashsum command parser (`md5sum`, `shasum`, etc.)
197 --hciconfig
200 `hciconfig` command parser
201 --history
204 `history` command parser
205 --host `host` command parser
208 --hosts
211 `/etc/hosts` file parser
212 --id `id` command parser
215 --ifconfig
218 `ifconfig` command parser
219 --ini INI file parser
222 --ini-dup
225 INI with duplicate key file parser
226 --iostat
229 `iostat` command parser
230 --iostat-s
233 `iostat` command streaming parser
234 --ip-address
237 IPv4 and IPv6 Address string parser
238 --iptables
241 `iptables` command parser
242 --ip-route
245 `ip route` command parser
246 --iso-datetime
249 Deprecated - please use datetime-iso
250 --iw-scan
253 `iw dev [device] scan` command parser
254 --iwconfig
257 `iwconfig` command parser
258 --jar-manifest
261 Java MANIFEST.MF file parser
262 --jobs `jobs` command parser
265 --jwt JWT string parser
268 --kv Key/Value file and string parser
271 --last `last` and `lastb` command parser
274 --ls `ls` command parser
277 --ls-s `ls` command streaming parser
280 --lsattr
283 `lsattr` command parser
284 --lsb-release
287 `lsb_release` command parser
288 --lsblk
291 `lsblk` command parser
292 --lsmod
295 `lsmod` command parser
296 --lsof `lsof` command parser
299 --lspci
302 `lspci -mmv` command parser
303 --lsusb
306 `lsusb` command parser
307 --m3u M3U and M3U8 file parser
310 --mdadm
313 `mdadm` command parser
314 --mount
317 `mount` command parser
318 --mpstat
321 `mpstat` command parser
322 --mpstat-s
325 `mpstat` command streaming parser
326 --netstat
329 `netstat` command parser
330 --nmcli
333 `nmcli` command parser
334 --nsd-control
337 `nsd-control` command parser
338 --ntpq `ntpq -p` command parser
341 --openvpn
344 openvpn-status.log file parser
345 --os-prober
348 `os-prober` command parser
349 --os-release
352 `/etc/os-release` file parser
353 --passwd
356 `/etc/passwd` file parser
357 --pci-ids
360 `pci.ids` file parser
361 --pgpass
364 PostgreSQL password file parser
365 --pidstat
368 `pidstat -H` command parser
369 --pidstat-s
372 `pidstat -H` command streaming parser
373 --ping `ping` and `ping6` command parser
376 --ping-s
379 `ping` and `ping6` command streaming parser
380 --pip-list
383 `pip list` command parser
384 --pip-show
387 `pip show` command parser
388 --plist
391 PLIST file parser
392 --postconf
395 `postconf -M` command parser
396 --proc `/proc/` file parser
399 --proc-buddyinfo
402 `/proc/buddyinfo` file parser
403 --proc-consoles
406 `/proc/consoles` file parser
407 --proc-cpuinfo
410 `/proc/cpuinfo` file parser
411 --proc-crypto
414 `/proc/crypto` file parser
415 --proc-devices
418 `/proc/devices` file parser
419 --proc-diskstats
422 `/proc/diskstats` file parser
423 --proc-filesystems
426 `/proc/filesystems` file parser
427 --proc-interrupts
430 `/proc/interrupts` file parser
431 --proc-iomem
434 `/proc/iomem` file parser
435 --proc-ioports
438 `/proc/ioports` file parser
439 --proc-loadavg
442 `/proc/loadavg` file parser
443 --proc-locks
446 `/proc/locks` file parser
447 --proc-meminfo
450 `/proc/meminfo` file parser
451 --proc-modules
454 `/proc/modules` file parser
455 --proc-mtrr
458 `/proc/mtrr` file parser
459 --proc-pagetypeinfo
462 `/proc/pagetypeinfo` file parser
463 --proc-partitions
466 `/proc/partitions` file parser
467 --proc-slabinfo
470 `/proc/slabinfo` file parser
471 --proc-softirqs
474 `/proc/softirqs` file parser
475 --proc-stat
478 `/proc/stat` file parser
479 --proc-swaps
482 `/proc/swaps` file parser
483 --proc-uptime
486 `/proc/uptime` file parser
487 --proc-version
490 `/proc/version` file parser
491 --proc-vmallocinfo
494 `/proc/vmallocinfo` file parser
495 --proc-vmstat
498 `/proc/vmstat` file parser
499 --proc-zoneinfo
502 `/proc/zoneinfo` file parser
503 --proc-driver-rtc
506 `/proc/driver/rtc` file parser
507 --proc-net-arp
510 `/proc/net/arp` file parser
511 --proc-net-dev
514 `/proc/net/dev` file parser
515 --proc-net-dev-mcast
518 `/proc/net/dev_mcast` file parser
519 --proc-net-if-inet6
522 `/proc/net/if_inet6` file parser
523 --proc-net-igmp
526 `/proc/net/igmp` file parser
527 --proc-net-igmp6
530 `/proc/net/igmp6` file parser
531 --proc-net-ipv6-route
534 `/proc/net/ipv6_route` file parser
535 --proc-net-netlink
538 `/proc/net/netlink` file parser
539 --proc-net-netstat
542 `/proc/net/netstat` file parser
543 --proc-net-packet
546 `/proc/net/packet` file parser
547 --proc-net-protocols
550 `/proc/net/protocols` file parser
551 --proc-net-route
554 `/proc/net/route` file parser
555 --proc-net-tcp
558 `/proc/net/tcp` and `/proc/net/tcp6` file parser
559 --proc-net-unix
562 `/proc/net/unix` file parser
563 --proc-pid-fdinfo
566 `/proc/<pid>/fdinfo/<fd>` file parser
567 --proc-pid-io
570 `/proc/<pid>/io` file parser
571 --proc-pid-maps
574 `/proc/<pid>/maps` file parser
575 --proc-pid-mountinfo
578 `/proc/<pid>/mountinfo` file parser
579 --proc-pid-numa-maps
582 `/proc/<pid>/numa_maps` file parser
583 --proc-pid-smaps
586 `/proc/<pid>/smaps` file parser
587 --proc-pid-stat
590 `/proc/<pid>/stat` file parser
591 --proc-pid-statm
594 `/proc/<pid>/statm` file parser
595 --proc-pid-status
598 `/proc/<pid>/status` file parser
599 --ps `ps` command parser
602 --resolve-conf
605 `/etc/resolve.conf` file parser
606 --route
609 `route` command parser
610 --rpm-qi
613 `rpm -qi` command parser
614 --rsync
617 `rsync` command parser
618 --rsync-s
621 `rsync` command streaming parser
622 --semver
625 Semantic Version string parser
626 --sfdisk
629 `sfdisk` command parser
630 --shadow
633 `/etc/shadow` file parser
634 --srt SRT file parser
637 --ss `ss` command parser
640 --ssh-conf
643 `ssh` config file and `ssh -G` command parser
644 --sshd-conf
647 `sshd` config file and `sshd -T` command parser
648 --stat `stat` command parser
651 --stat-s
654 `stat` command streaming parser
655 --sysctl
658 `sysctl` command parser
659 --syslog
662 Syslog RFC 5424 string parser
663 --syslog-s
666 Syslog RFC 5424 string streaming parser
667 --syslog-bsd
670 Syslog RFC 3164 string parser
671 --syslog-bsd-s
674 Syslog RFC 3164 string streaming parser
675 --systemctl
678 `systemctl` command parser
679 --systemctl-lj
682 `systemctl list-jobs` command parser
683 --systemctl-ls
686 `systemctl list-sockets` command parser
687 --systemctl-luf
690 `systemctl list-unit-files` command parser
691 --systeminfo
694 `systeminfo` command parser
695 --time `/usr/bin/time` command parser
698 --timedatectl
701 `timedatectl status` command parser
702 --timestamp
705 Unix Epoch Timestamp string parser
706 --toml TOML file parser
709 --top `top -b` command parser
712 --top-s
715 `top -b` command streaming parser
716 --tracepath
719 `tracepath` and `tracepath6` command parser
720 --traceroute
723 `traceroute` and `traceroute6` command parser
724 --udevadm
727 `udevadm info` command parser
728 --ufw `ufw status` command parser
731 --ufw-appinfo
734 `ufw app info [application]` command parser
735 --uname
738 `uname -a` command parser
739 --update-alt-gs
742 `update-alternatives --get-selections` command parser
743 --update-alt-q
746 `update-alternatives --query` command parser
747 --upower
750 `upower` command parser
751 --uptime
754 `uptime` command parser
755 --url URL string parser
758 --ver Version string parser
761 --veracrypt
764 `veracrypt` command parser
765 --vmstat
768 `vmstat` command parser
769 --vmstat-s
772 `vmstat` command streaming parser
773 --w `w` command parser
776 --wc `wc` command parser
779 --who `who` command parser
782 --x509-cert
785 X.509 PEM and DER certificate file parser
786 --x509-csr
789 X.509 PEM and DER certificate request file parser
790 --xml XML file parser
793 --xrandr
796 `xrandr` command parser
797 --yaml YAML file parser
800 --zipinfo
803 `zipinfo` command parser
804 --zpool-iostat
807 `zpool iostat` command parser
808 --zpool-status
811 `zpool status` command parser
812 Options:
816 -a, --about
819 About jc (JSON or YAML output)
820 -C, --force-color
822 Force color output even when using pipes (overrides -m
823 and the NO_COLOR env variable)
824 -d, --debug
826 Debug - show traceback (use -dd for verbose traceback)
827 -h, --help
829 Help (--help --parser_name for parser documentation). Use
830 twice to show hidden parsers (e.g. -hh)
831 -m, --monochrome
833 Monochrome output
834 -M, --meta-out
836 Add metadata to output including timestamp, parser name,
837 magic command, magic command exit code, etc.
838 -p, --pretty
840 Pretty print output
841 -q, --quiet
843 Quiet mode. Suppresses parser warning messages (use -qq
844 to ignore streaming parser errors)
845 -r, --raw
847 Raw output. Provides more literal output, typically with
848 string values and no additional semantic processing
849 -u, --unbuffer
851 Unbuffer output (useful for slow streaming data with
852 streaming parsers)
853 -v, --version
855 Version information
856 -y, --yaml-out
858 YAML output
859 -B, --bash-comp
861 Generate Bash shell completion script
862 -Z, --zsh-comp
864 Generate Zsh shell completion script
865 Slice:
868 Line slicing is supported using the START:STOP syntax similar to
869 Python slicing. This allows you to skip lines at the beginning
870 and/or end of the STDIN input you would like jc to convert.
871 START and STOP can be positive or negative integers or blank and
873 allow you to specify how many lines to skip and how many lines
874 to process. Positive and blank slices are the most memory effi‐
875 cient. Any negative integers in the slice will use more memory.
876 For example, to skip the first and last line of the following
878 text, you could express the slice in a couple ways:
879 $ cat table.txt
881 ### We want to skip this header ###
882 col1 col2
883 foo 1
884 bar 2
885 ### We want to skip this footer ###
886 $ cat table.txt | jc 1:-1 --asciitable
887 [{"col1":"foo","col2":"1"},{"col1":"bar","col2":"2"}]
888 $ cat table.txt | jc 1:4 --asciitable
889 [{"col1":"foo","col2":"1"},{"col1":"bar","col2":"2"}]
890 In this example 1:-1 and 1:4 line slices provide the same out‐
892 put.
893 When using positive integers the index location of STOP is non-
895 inclusive. Positive slices count from the first line of the in‐
896 put toward the end starting at 0 as the first line. Negative
897 slices count from the last line toward the beginning starting at
898 -1 as the last line. This is also the way Python's slicing fea‐
899 ture works.
900 Here is a breakdown of line slice options:
902 START:STOP
905 lines START through STOP - 1
906 START: lines START through the rest of the output
908 :STOP lines from the beginning through STOP - 1
910 -START:STOP
912 START lines from the end through STOP - 1
913 START:-STOP
915 lines START through STOP lines from the end
916 -START:-STOP
918 START lines from the end through STOP lines from the end
919 -START:
921 START lines from the end through the rest of the output
922 :-STOP lines from the beginning through STOP lines from the end
924 : all lines
926
929 Any fatal errors within jc will generate an exit code of 100, otherwise
930 the exit code will be 0.
931 When using the "magic" syntax (e.g. jc ifconfig eth0), jc will store
933 the exit code of the program being parsed and add it to the jc exit
934 code. This way it is easier to determine if an error was from the
935 parsed program or jc.
936 Consider the following examples using ifconfig:
938 ifconfig exit code = 0, jc exit code = 0, combined exit code = 0
940 (no errors)
941 ifconfig exit code = 1, jc exit code = 0, combined exit code = 1
943 (error in ifconfig)
944 ifconfig exit code = 0, jc exit code = 100, combined exit code =
946 100 (error in jc)
947 ifconfig exit code = 1, jc exit code = 100, combined exit code =
949 101 (error in both ifconfig and jc)
950 When using the "magic" syntax you can also retrieve the exit code of
952 the called program by using the --meta-out or -M option. This will ap‐
953 pend a _jc_meta object to the output that will include the magic com‐
954 mand information, including the exit code.
955 Here is an example with ping:
957 $ jc --meta-out -p ping -c2 192.168.1.252
958 {
959 "destination_ip": "192.168.1.252",
960 "data_bytes": 56,
961 "pattern": null,
962 "destination": "192.168.1.252",
963 "packets_transmitted": 2,
964 "packets_received": 0,
965 "packet_loss_percent": 100.0,
966 "duplicates": 0,
967 "responses": [
968 {
969 "type": "timeout",
970 "icmp_seq": 0,
971 "duplicate": false
972 }
973 ],
974 "_jc_meta": {
975 "parser": "ping",
976 "timestamp": 1661357115.27949,
977 "magic_command": [
978 "ping",
979 "-c2",
980 "192.168.1.252"
981 ],
982 "magic_command_exit": 2
983 }
984 }
985 $ echo $?
986 2
987
990 Custom Colors
991 You can specify custom colors via the JC_COLORS environment variable.
993 The JC_COLORS environment variable takes four comma separated string
994 values in the following format:
995 JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>
997 Where colors are: black, red, green, yellow, blue, magenta, cyan, gray,
999 brightblack, brightred, brightgreen, brightyellow, brightblue, bright‐
1000 magenta, brightcyan, white, or default
1001 For example, to set to the default colors:
1003 JC_COLORS=blue,brightblack,magenta,green
1005 or
1007 JC_COLORS=default,default,default,default
1009 Disable Color Output
1011 You can set the NO_COLOR environment variable to any value to disable
1013 color output in jc. Note that using the -C option to force color output
1014 will override both the NO_COLOR environment variable and the -m option.
1015
1018 Most parsers load all of the data from STDIN, parse it, then output the
1019 entire JSON document serially. There are some streaming parsers (e.g.
1020 ls-s, ping-s, etc.) that immediately start processing and outputting
1021 the data line-by-line as JSON Lines (aka NDJSON) while it is being re‐
1022 ceived from STDIN. This can significantly reduce the amount of memory
1023 required to parse large amounts of command output (e.g. ls -lR /) and
1024 can sometimes process the data more quickly. Streaming parsers have
1025 slightly different behavior than standard parsers as outlined below.
1026 Note: Streaming parsers cannot be used with the "magic" syntax
1028 Ignoring Errors
1030 You may want to ignore parsing errors when using streaming parsers
1032 since these may be used in long-lived processing pipelines and errors
1033 can break the pipe. To ignore parsing errors, use the -qq cli option.
1034 This will add a _jc_meta object to the JSON output with a success at‐
1035 tribute. If success is true, then there were no issues parsing the
1036 line. If success is false, then a parsing issue was found and error and
1037 line fields will be added to include a short error description and the
1038 contents of the unparsable line, respectively:
1039 Successfully parsed line with -qq option:
1041 {
1042 "command_data": "data",
1043 "_jc_meta": {
1044 "success": true
1045 }
1046 }
1047 Unsuccessfully parsed line with -qq option:
1049 {
1050 "_jc_meta": {
1051 "success": false,
1052 "error": "error message",
1053 "line": "original line data"
1054 }
1055 }
1056 Unbuffering Output
1058 Most operating systems will buffer output that is being piped from
1060 process to process. The buffer is usually around 4KB. When viewing the
1061 output in the terminal the OS buffer is not engaged so output is imme‐
1062 diately displayed on the screen. When piping multiple processes to‐
1063 gether, though, it may seem as if the output is hanging when the input
1064 data is very slow (e.g. ping):
1065 $ ping 1.1.1.1 | jc --ping-s | jq
1067 <slow output>
1068 This is because the OS engages the 4KB buffer between jc and jq in this
1070 example. To display the data on the terminal in realtime, you can dis‐
1071 able the buffer with the -u (unbuffer) cli option:
1072 $ ping 1.1.1.1 | jc --ping-s -u | jq
1074 {"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
1075 {"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
1076 etc...
1077 Note: Unbuffered output can be slower for large data streams.
1079
1082 Parser plugins may be placed in a jc/jcparsers folder in your local
1083 "App data directory":
1084 - Linux/unix: $HOME/.local/share/jc/jcparsers
1086 - macOS: $HOME/Library/Application Support/jc/jcparsers
1087 - Windows: $LOCALAPPDATA\jc\jc\jcparsers
1088 Parser plugins are standard python module files. Use the
1090 jc/parsers/foo.py or jc/parsers/foo_s.py (streaming) parser as a tem‐
1091 plate and simply place a .py file in the jcparsers subfolder. Any de‐
1092 pendencies can be placed in the jc folder above jcparsers and can be
1093 imported in the parser code.
1094 Parser plugin filenames must be valid python module names and therefore
1096 must start with a letter and consist entirely of alphanumerics and un‐
1097 derscores. Local plugins may override default parsers.
1098 Note: The application data directory follows the XDG Base Directory
1100 Specification
1101
1104 Locale
1105 For best results set the locale environment variables to C or
1107 en_US.UTF-8 by modifying the LC_ALL variable:
1108 $ LC_ALL=C date | jc --date
1110 You can also set the locale variables individually:
1112 $ export LANG=C
1114 $ export LC_NUMERIC=C
1116 On some older systems UTF-8 output will be downgraded to ASCII with \u
1118 escape sequences if the C locale does not support UTF-8 encoding.
1119 Timezones
1121 Some parsers have calculated epoch timestamp fields added to the out‐
1123 put. Unless a timestamp field name has a _utc suffix it is considered
1124 naive. (i.e. based on the local timezone of the system the jc parser
1125 was run on).
1126 If a UTC timezone can be detected in the text of the command output,
1128 the timestamp will be timezone aware and have a _utc suffix on the key
1129 name. (e.g. epoch_utc) No other timezones are supported for aware
1130 timestamps.
1131
1134 Standard Syntax:
1135 $ dig www.google.com | jc -p --dig
1136 $ cat /proc/meminfo | jc --pretty --proc
1138 Magic Syntax:
1140 $ jc --pretty dig www.google.com
1141 $ jc --pretty /proc/meminfo
1143 Line Slicing:
1145 $ cat file.csv | jc :101 --csv # parse first 100 lines
1146 For parser documentation:
1148 $ jc --help --dig
1149 More Help:
1151 $ jc -hh # show hidden parsers
1152 $ jc -hhh # list parsers by category tags
1154
1157 Kelly Brazil (kellyjonbrazil@gmail.com)
1158 https://github.com/kellyjonbrazil/jc
1160
1163 Copyright (c) 2019-2023 Kelly Brazil
1164 License: MIT License
11661.23.6 2023-10-23 jc(1)