1jc(1) JSON Convert jc(1)
2
6 jc - JSON Convert JSONifies the output of many CLI tools, file-types,
7 and strings
8
10 Standard syntax:
11 COMMAND | jc [OPTIONS] PARSER
13 cat FILE | jc [OPTIONS] PARSER
15 echo STRING | jc [OPTIONS] PARSER
17 Magic syntax:
19 jc [OPTIONS] COMMAND
21 jc [OPTIONS] /proc/<path-to-procfile>
23
26 jc JSONifies the output of many CLI tools, file-types, and common
27 strings for easier parsing in scripts. jc accepts piped input from
28 STDIN and outputs a JSON representation of the previous command's out‐
29 put to STDOUT. Alternatively, the "Magic" syntax can be used by
30 prepending jc to the command to be converted. Options can be passed to
31 jc immediately before the command is given. (Note: "Magic" syntax does
32 not support shell builtins or command aliases)
33
36 Parsers:
37 --acpi `acpi` command parser
41 --airport
44 `airport -I` command parser
45 --airport-s
48 `airport -s` command parser
49 --arp `arp` command parser
52 --asciitable
55 ASCII and Unicode table parser
56 --asciitable-m
59 multi-line ASCII and Unicode table parser
60 --blkid
63 `blkid` command parser
64 --cbt `cbt` (Google Bigtable) command parser
67 --cef CEF string parser
70 --cef-s
73 CEF string streaming parser
74 --chage
77 `chage --list` command parser
78 --cksum
81 `cksum` and `sum` command parser
82 --clf Common and Combined Log Format file parser
85 --clf-s
88 Common and Combined Log Format file streaming parser
89 --crontab
92 `crontab` command and file parser
93 --crontab-u
96 `crontab` file parser with user support
97 --csv CSV file parser
100 --csv-s
103 CSV file streaming parser
104 --date `date` command parser
107 --datetime-iso
110 ISO 8601 Datetime string parser
111 --df `df` command parser
114 --dig `dig` command parser
117 --dir `dir` command parser
120 --dmidecode
123 `dmidecode` command parser
124 --dpkg-l
127 `dpkg -l` command parser
128 --du `du` command parser
131 --email-address
134 Email Address string parser
135 --env `env` command parser
138 --file `file` command parser
141 --findmnt
144 `findmnt` command parser
145 --finger
148 `finger` command parser
149 --free `free` command parser
152 --fstab
155 `/etc/fstab` file parser
156 --git-log
159 `git log` command parser
160 --git-log-s
163 `git log` command streaming parser
164 --git-ls-remote
167 `git ls-remote` command parser
168 --gpg `gpg --with-colons` command parser
171 --group
174 `/etc/group` file parser
175 --gshadow
178 `/etc/gshadow` file parser
179 --hash `hash` command parser
182 --hashsum
185 hashsum command parser (`md5sum`, `shasum`, etc.)
186 --hciconfig
189 `hciconfig` command parser
190 --history
193 `history` command parser
194 --hosts
197 `/etc/hosts` file parser
198 --id `id` command parser
201 --ifconfig
204 `ifconfig` command parser
205 --ini INI file parser
208 --ini-dup
211 INI with duplicate key file parser
212 --iostat
215 `iostat` command parser
216 --iostat-s
219 `iostat` command streaming parser
220 --ip-address
223 IPv4 and IPv6 Address string parser
224 --iptables
227 `iptables` command parser
228 --iso-datetime
231 Deprecated - please use datetime-iso
232 --iw-scan
235 `iw dev [device] scan` command parser
236 --iwconfig
239 `iwconfig` command parser
240 --jar-manifest
243 Java MANIFEST.MF file parser
244 --jobs `jobs` command parser
247 --jwt JWT string parser
250 --kv Key/Value file and string parser
253 --last `last` and `lastb` command parser
256 --ls `ls` command parser
259 --ls-s `ls` command streaming parser
262 --lsblk
265 `lsblk` command parser
266 --lsmod
269 `lsmod` command parser
270 --lsof `lsof` command parser
273 --lspci
276 `lspci -mmv` command parser
277 --lsusb
280 `lsusb` command parser
281 --m3u M3U and M3U8 file parser
284 --mdadm
287 `mdadm` command parser
288 --mount
291 `mount` command parser
292 --mpstat
295 `mpstat` command parser
296 --mpstat-s
299 `mpstat` command streaming parser
300 --netstat
303 `netstat` command parser
304 --nmcli
307 `nmcli` command parser
308 --ntpq `ntpq -p` command parser
311 --openvpn
314 openvpn-status.log file parser
315 --os-prober
318 `os-prober` command parser
319 --passwd
322 `/etc/passwd` file parser
323 --pci-ids
326 `pci.ids` file parser
327 --pgpass
330 PostgreSQL password file parser
331 --pidstat
334 `pidstat -H` command parser
335 --pidstat-s
338 `pidstat -H` command streaming parser
339 --ping `ping` and `ping6` command parser
342 --ping-s
345 `ping` and `ping6` command streaming parser
346 --pip-list
349 `pip list` command parser
350 --pip-show
353 `pip show` command parser
354 --plist
357 PLIST file parser
358 --postconf
361 `postconf -M` command parser
362 --proc `/proc/` file parser
365 --proc-buddyinfo
368 `/proc/buddyinfo` file parser
369 --proc-consoles
372 `/proc/consoles` file parser
373 --proc-cpuinfo
376 `/proc/cpuinfo` file parser
377 --proc-crypto
380 `/proc/crypto` file parser
381 --proc-devices
384 `/proc/devices` file parser
385 --proc-diskstats
388 `/proc/diskstats` file parser
389 --proc-filesystems
392 `/proc/filesystems` file parser
393 --proc-interrupts
396 `/proc/interrupts` file parser
397 --proc-iomem
400 `/proc/iomem` file parser
401 --proc-ioports
404 `/proc/ioports` file parser
405 --proc-loadavg
408 `/proc/loadavg` file parser
409 --proc-locks
412 `/proc/locks` file parser
413 --proc-meminfo
416 `/proc/meminfo` file parser
417 --proc-modules
420 `/proc/modules` file parser
421 --proc-mtrr
424 `/proc/mtrr` file parser
425 --proc-pagetypeinfo
428 `/proc/pagetypeinfo` file parser
429 --proc-partitions
432 `/proc/partitions` file parser
433 --proc-slabinfo
436 `/proc/slabinfo` file parser
437 --proc-softirqs
440 `/proc/softirqs` file parser
441 --proc-stat
444 `/proc/stat` file parser
445 --proc-swaps
448 `/proc/swaps` file parser
449 --proc-uptime
452 `/proc/uptime` file parser
453 --proc-version
456 `/proc/version` file parser
457 --proc-vmallocinfo
460 `/proc/vmallocinfo` file parser
461 --proc-vmstat
464 `/proc/vmstat` file parser
465 --proc-zoneinfo
468 `/proc/zoneinfo` file parser
469 --proc-driver-rtc
472 `/proc/driver/rtc` file parser
473 --proc-net-arp
476 `/proc/net/arp` file parser
477 --proc-net-dev
480 `/proc/net/dev` file parser
481 --proc-net-dev-mcast
484 `/proc/net/dev_mcast` file parser
485 --proc-net-if-inet6
488 `/proc/net/if_inet6` file parser
489 --proc-net-igmp
492 `/proc/net/igmp` file parser
493 --proc-net-igmp6
496 `/proc/net/igmp6` file parser
497 --proc-net-ipv6-route
500 `/proc/net/ipv6_route` file parser
501 --proc-net-netlink
504 `/proc/net/netlink` file parser
505 --proc-net-netstat
508 `/proc/net/netstat` file parser
509 --proc-net-packet
512 `/proc/net/packet` file parser
513 --proc-net-protocols
516 `/proc/net/protocols` file parser
517 --proc-net-route
520 `/proc/net/route` file parser
521 --proc-net-unix
524 `/proc/net/unix` file parser
525 --proc-pid-fdinfo
528 `/proc/<pid>/fdinfo/<fd>` file parser
529 --proc-pid-io
532 `/proc/<pid>/io` file parser
533 --proc-pid-maps
536 `/proc/<pid>/maps` file parser
537 --proc-pid-mountinfo
540 `/proc/<pid>/mountinfo` file parser
541 --proc-pid-numa-maps
544 `/proc/<pid>/numa_maps` file parser
545 --proc-pid-smaps
548 `/proc/<pid>/smaps` file parser
549 --proc-pid-stat
552 `/proc/<pid>/stat` file parser
553 --proc-pid-statm
556 `/proc/<pid>/statm` file parser
557 --proc-pid-status
560 `/proc/<pid>/status` file parser
561 --ps `ps` command parser
564 --route
567 `route` command parser
568 --rpm-qi
571 `rpm -qi` command parser
572 --rsync
575 `rsync` command parser
576 --rsync-s
579 `rsync` command streaming parser
580 --semver
583 Semantic Version string parser
584 --sfdisk
587 `sfdisk` command parser
588 --shadow
591 `/etc/shadow` file parser
592 --ss `ss` command parser
595 --sshd-conf
598 sshd config file and `sshd -T` command parser
599 --stat `stat` command parser
602 --stat-s
605 `stat` command streaming parser
606 --sysctl
609 `sysctl` command parser
610 --syslog
613 Syslog RFC 5424 string parser
614 --syslog-s
617 Syslog RFC 5424 string streaming parser
618 --syslog-bsd
621 Syslog RFC 3164 string parser
622 --syslog-bsd-s
625 Syslog RFC 3164 string streaming parser
626 --systemctl
629 `systemctl` command parser
630 --systemctl-lj
633 `systemctl list-jobs` command parser
634 --systemctl-ls
637 `systemctl list-sockets` command parser
638 --systemctl-luf
641 `systemctl list-unit-files` command parser
642 --systeminfo
645 `systeminfo` command parser
646 --time `/usr/bin/time` command parser
649 --timedatectl
652 `timedatectl status` command parser
653 --timestamp
656 Unix Epoch Timestamp string parser
657 --toml TOML file parser
660 --top `top -b` command parser
663 --top-s
666 `top -b` command streaming parser
667 --tracepath
670 `tracepath` and `tracepath6` command parser
671 --traceroute
674 `traceroute` and `traceroute6` command parser
675 --udevadm
678 `udevadm info` command parser
679 --ufw `ufw status` command parser
682 --ufw-appinfo
685 `ufw app info [application]` command parser
686 --uname
689 `uname -a` command parser
690 --update-alt-gs
693 `update-alternatives --get-selections` command parser
694 --update-alt-q
697 `update-alternatives --query` command parser
698 --upower
701 `upower` command parser
702 --uptime
705 `uptime` command parser
706 --url URL string parser
709 --vmstat
712 `vmstat` command parser
713 --vmstat-s
716 `vmstat` command streaming parser
717 --w `w` command parser
720 --wc `wc` command parser
723 --who `who` command parser
726 --x509-cert
729 X.509 PEM and DER certificate file parser
730 --xml XML file parser
733 --xrandr
736 `xrandr` command parser
737 --yaml YAML file parser
740 --zipinfo
743 `zipinfo` command parser
744 Options:
748 -a, --about
751 About jc (JSON or YAML output)
752 -C, --force-color
754 Force color output even when using pipes (overrides -m
755 and the NO_COLOR env variable)
756 -d, --debug
758 Debug - show traceback (use -dd for verbose traceback)
759 -h, --help
761 Help (--help --parser_name for parser documentation). Use
762 twice to show hidden parsers (e.g. -hh)
763 -m, --monochrome
765 Monochrome output
766 -M, --meta-out
768 Add metadata to output including timestamp, parser name,
769 magic command, magic command exit code, etc.
770 -p, --pretty
772 Pretty print output
773 -q, --quiet
775 Quiet mode. Suppresses parser warning messages (use -qq
776 to ignore streaming parser errors)
777 -r, --raw
779 Raw output. Provides more literal output, typically with
780 string values and no additional semantic processing
781 -u, --unbuffer
783 Unbuffer output (useful for slow streaming data with
784 streaming parsers)
785 -v, --version
787 Version information
788 -y, --yaml-out
790 YAML output
791 -B, --bash-comp
793 Generate Bash shell completion script
794 -Z, --zsh-comp
796 Generate Zsh shell completion script
797
800 Any fatal errors within jc will generate an exit code of 100, otherwise
801 the exit code will be 0.
802 When using the "magic" syntax (e.g. jc ifconfig eth0), jc will store
804 the exit code of the program being parsed and add it to the jc exit
805 code. This way it is easier to determine if an error was from the
806 parsed program or jc.
807 Consider the following examples using ifconfig:
809 ifconfig exit code = 0, jc exit code = 0, combined exit code = 0
811 (no errors)
812 ifconfig exit code = 1, jc exit code = 0, combined exit code = 1
814 (error in ifconfig)
815 ifconfig exit code = 0, jc exit code = 100, combined exit code =
817 100 (error in jc)
818 ifconfig exit code = 1, jc exit code = 100, combined exit code =
820 101 (error in both ifconfig and jc)
821 When using the "magic" syntax you can also retrieve the exit code of
823 the called program by using the --meta-out or -M option. This will ap‐
824 pend a _jc_meta object to the output that will include the magic com‐
825 mand information, including the exit code.
826 Here is an example with ping:
828 $ jc --meta-out -p ping -c2 192.168.1.252
829 {
830 "destination_ip": "192.168.1.252",
831 "data_bytes": 56,
832 "pattern": null,
833 "destination": "192.168.1.252",
834 "packets_transmitted": 2,
835 "packets_received": 0,
836 "packet_loss_percent": 100.0,
837 "duplicates": 0,
838 "responses": [
839 {
840 "type": "timeout",
841 "icmp_seq": 0,
842 "duplicate": false
843 }
844 ],
845 "_jc_meta": {
846 "parser": "ping",
847 "timestamp": 1661357115.27949,
848 "magic_command": [
849 "ping",
850 "-c2",
851 "192.168.1.252"
852 ],
853 "magic_command_exit": 2
854 }
855 }
856 $ echo $?
857 2
858
861 Custom Colors
862 You can specify custom colors via the JC_COLORS environment variable.
864 The JC_COLORS environment variable takes four comma separated string
865 values in the following format:
866 JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>
868 Where colors are: black, red, green, yellow, blue, magenta, cyan, gray,
870 brightblack, brightred, brightgreen, brightyellow, brightblue, bright‐
871 magenta, brightcyan, white, or default
872 For example, to set to the default colors:
874 JC_COLORS=blue,brightblack,magenta,green
876 or
878 JC_COLORS=default,default,default,default
880 Disable Color Output
882 You can set the NO_COLOR environment variable to any value to disable
884 color output in jc. Note that using the -C option to force color output
885 will override both the NO_COLOR environment variable and the -m option.
886
889 Most parsers load all of the data from STDIN, parse it, then output the
890 entire JSON document serially. There are some streaming parsers (e.g.
891 ls-s, ping-s, etc.) that immediately start processing and outputting
892 the data line-by-line as JSON Lines (aka NDJSON) while it is being re‐
893 ceived from STDIN. This can significantly reduce the amount of memory
894 required to parse large amounts of command output (e.g. ls -lR /) and
895 can sometimes process the data more quickly. Streaming parsers have
896 slightly different behavior than standard parsers as outlined below.
897 Note: Streaming parsers cannot be used with the "magic" syntax
899 Ignoring Errors
901 You may want to ignore parsing errors when using streaming parsers
903 since these may be used in long-lived processing pipelines and errors
904 can break the pipe. To ignore parsing errors, use the -qq cli option.
905 This will add a _jc_meta object to the JSON output with a success at‐
906 tribute. If success is true, then there were no issues parsing the
907 line. If success is false, then a parsing issue was found and error and
908 line fields will be added to include a short error description and the
909 contents of the unparsable line, respectively:
910 Successfully parsed line with -qq option:
912 {
913 "command_data": "data",
914 "_jc_meta": {
915 "success": true
916 }
917 }
918 Unsuccessfully parsed line with -qq option:
920 {
921 "_jc_meta": {
922 "success": false,
923 "error": "error message",
924 "line": "original line data"
925 }
926 }
927 Unbuffering Output
929 Most operating systems will buffer output that is being piped from
931 process to process. The buffer is usually around 4KB. When viewing the
932 output in the terminal the OS buffer is not engaged so output is imme‐
933 diately displayed on the screen. When piping multiple processes to‐
934 gether, though, it may seem as if the output is hanging when the input
935 data is very slow (e.g. ping):
936 $ ping 1.1.1.1 | jc --ping-s | jq
938 <slow output>
939 This is because the OS engages the 4KB buffer between jc and jq in this
941 example. To display the data on the terminal in realtime, you can dis‐
942 able the buffer with the -u (unbuffer) cli option:
943 $ ping 1.1.1.1 | jc --ping-s -u | jq
945 {"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
946 {"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
947 etc...
948 Note: Unbuffered output can be slower for large data streams.
950
953 Custom local parser plugins may be placed in a jc/jcparsers folder in
954 your local "App data directory":
955 - Linux/unix: $HOME/.local/share/jc/jcparsers
957 - macOS: $HOME/Library/Application Support/jc/jcparsers
958 - Windows: $LOCALAPPDATA\jc\jc\jcparsers
959 Local parser plugins are standard python module files. Use the
961 jc/parsers/foo.py or jc/parsers/foo_s.py (streaming) parser as a tem‐
962 plate and simply place a .py file in the jcparsers subfolder.
963 Local plugin filenames must be valid python module names and therefore
965 must start with a letter and consist entirely of alphanumerics and un‐
966 derscores. Local plugins may override default parsers.
967 Note: The application data directory follows the XDG Base Directory
969 Specification
970
973 Locale
974 For best results set the locale environment variables to C or
976 en_US.UTF-8 by modifying the LC_ALL variable:
977 $ LC_ALL=C date | jc --date
979 You can also set the locale variables individually:
981 $ export LANG=C
983 $ export LC_NUMERIC=C
985 On some older systems UTF-8 output will be downgraded to ASCII with \u
987 escape sequences if the C locale does not support UTF-8 encoding.
988 Timezones
990 Some parsers have calculated epoch timestamp fields added to the out‐
992 put. Unless a timestamp field name has a _utc suffix it is considered
993 naive. (i.e. based on the local timezone of the system the jc parser
994 was run on).
995 If a UTC timezone can be detected in the text of the command output,
997 the timestamp will be timezone aware and have a _utc suffix on the key
998 name. (e.g. epoch_utc) No other timezones are supported for aware time‐
999 stamps.
1000
1003 Standard Syntax:
1004 $ dig www.google.com | jc -p --dig
1005 $ cat /proc/meminfo | jc --pretty --proc
1007 Magic Syntax:
1009 $ jc --pretty dig www.google.com
1010 $ jc --pretty /proc/meminfo
1012 For parser documentation:
1014 $ jc --help --dig
1015
1017 Kelly Brazil (kellyjonbrazil@gmail.com)
1018 https://github.com/kellyjonbrazil/jc
1020
1023 Copyright (c) 2019-2023 Kelly Brazil
1024 License: MIT License
10261.22.5 2023-01-11 jc(1)