1JQ(1) JQ(1)
2
6 jq - Command-line JSON processor
7
9 jq [options...] filter [files...]
10 jq can transform JSON in various ways, by selecting, iterating, reduc‐
12 ing and otherwise mangling JSON documents. For instance, running the
13 command jq ´map(.price) | add´ will take an array of JSON objects as
14 input and return the sum of their "price" fields.
15 jq can accept text input as well, but by default, jq reads a stream of
17 JSON entities (including numbers and other literals) from stdin. White‐
18 space is only needed to separate entities such as 1 and 2, and true and
19 false. One or more files may be specified, in which case jq will read
20 input from those instead.
21 The options are described in the INVOKING JQ section; they mostly con‐
23 cern input and output formatting. The filter is written in the jq lan‐
24 guage and specifies how to transform the input file or document.
25
27 A jq program is a "filter": it takes an input, and produces an output.
28 There are a lot of builtin filters for extracting a particular field of
29 an object, or converting a number to a string, or various other stan‐
30 dard tasks.
31 Filters can be combined in various ways - you can pipe the output of
33 one filter into another filter, or collect the output of a filter into
34 an array.
35 Some filters produce multiple results, for instance there´s one that
37 produces all the elements of its input array. Piping that filter into a
38 second runs the second filter for each element of the array. Generally,
39 things that would be done with loops and iteration in other languages
40 are just done by gluing filters together in jq.
41 It´s important to remember that every filter has an input and an out‐
43 put. Even literals like "hello" or 42 are filters - they take an input
44 but always produce the same literal as output. Operations that combine
45 two filters, like addition, generally feed the same input to both and
46 combine the results. So, you can implement an averaging filter as add /
47 length - feeding the input array both to the add filter and the length
48 filter and then performing the division.
49 But that´s getting ahead of ourselves. :) Let´s start with something
51 simpler:
52
54 jq filters run on a stream of JSON data. The input to jq is parsed as a
55 sequence of whitespace-separated JSON values which are passed through
56 the provided filter one at a time. The output(s) of the filter are
57 written to standard out, again as a sequence of whitespace-separated
58 JSON data.
59 Note: it is important to mind the shell´s quoting rules. As a general
61 rule it´s best to always quote (with single-quote characters) the jq
62 program, as too many characters with special meaning to jq are also
63 shell meta-characters. For example, jq "foo" will fail on most Unix
64 shells because that will be the same as jq foo, which will generally
65 fail because foo is not defined. When using the Windows command shell
66 (cmd.exe) it´s best to use double quotes around your jq program when
67 given on the command-line (instead of the -f program-file option), but
68 then double-quotes in the jq program need backslash escaping.
69 You can affect how jq reads and writes its input and output using some
71 command-line options:
72 · --version:
74 Output the jq version and exit with zero.
76 · --seq:
78 Use the application/json-seq MIME type scheme for separating JSON
80 texts in jq´s input and output. This means that an ASCII RS (record
81 separator) character is printed before each value on output and an
82 ASCII LF (line feed) is printed after every output. Input JSON
83 texts that fail to parse are ignored (but warned about), discarding
84 all subsequent input until the next RS. This more also parses the
85 output of jq without the --seq option.
86 · --stream:
88 Parse the input in streaming fashion, outputing arrays of path and
90 leaf values (scalars and empty arrays or empty objects). For exam‐
91 ple, "a" becomes [[],"a"], and [[],"a",["b"]] becomes [[0],[]],
92 [[1],"a"], and [[1,0],"b"].
93 This is useful for processing very large inputs. Use this in con‐
95 junction with filtering and the reduce and foreach syntax to reduce
96 large inputs incrementally.
97 · --slurp/-s:
99 Instead of running the filter for each JSON object in the input,
101 read the entire input stream into a large array and run the filter
102 just once.
103 · --raw-input/-R:
105 Don´t parse the input as JSON. Instead, each line of text is passed
107 to the filter as a string. If combined with --slurp, then the
108 entire input is passed to the filter as a single long string.
109 · --null-input/-n:
111 Don´t read any input at all! Instead, the filter is run once using
113 null as the input. This is useful when using jq as a simple calcu‐
114 lator or to construct JSON data from scratch.
115 · --compact-output / -c:
117 By default, jq pretty-prints JSON output. Using this option will
119 result in more compact output by instead putting each JSON object
120 on a single line.
121 · --tab:
123 Use a tab for each indentation level instead of two spaces.
125 · --indent n:
127 Use the given number of spaces (no more than 8) for indentation.
129 · --color-output / -C and --monochrome-output / -M:
131 By default, jq outputs colored JSON if writing to a terminal. You
133 can force it to produce color even if writing to a pipe or a file
134 using -C, and disable color with -M.
135 · --ascii-output / -a:
137 jq usually outputs non-ASCII Unicode codepoints as UTF-8, even if
139 the input specified them as escape sequences (like "\u03bc"). Using
140 this option, you can force jq to produce pure ASCII output with
141 every non-ASCII character replaced with the equivalent escape
142 sequence.
143 · --unbuffered
145 Flush the output after each JSON object is printed (useful if
147 you´re piping a slow data source into jq and piping jq´s output
148 elsewhere).
149 · --sort-keys / -S:
151 Output the fields of each object with the keys in sorted order.
153 · --raw-output / -r:
155 With this option, if the filter´s result is a string then it will
157 be written directly to standard output rather than being formatted
158 as a JSON string with quotes. This can be useful for making jq fil‐
159 ters talk to non-JSON-based systems.
160 · --join-output / -j:
162 Like -r but jq won´t print a newline after each output.
164 · -f filename / --from-file filename:
166 Read filter from the file rather than from a command line, like
168 awk´s -f option. You can also use ´#´ to make comments.
169 · -Ldirectory / -L directory:
171 Prepend directory to the search list for modules. If this option is
173 used then no builtin search list is used. See the section on mod‐
174 ules below.
175 · -e / --exit-status:
177 Sets the exit status of jq to 0 if the last output values was nei‐
179 ther false nor null, 1 if the last output value was either false or
180 null, or 4 if no valid result was ever produced. Normally jq exits
181 with 2 if there was any usage problem or system error, 3 if there
182 was a jq program compile error, or 0 if the jq program ran.
183 · --arg name value:
185 This option passes a value to the jq program as a predefined vari‐
187 able. If you run jq with --arg foo bar, then $foo is available in
188 the program and has the value "bar". Note that value will be
189 treated as a string, so --arg foo 123 will bind $foo to "123".
190 · --argjson name JSON-text:
192 This option passes a JSON-encoded value to the jq program as a pre‐
194 defined variable. If you run jq with --argjson foo 123, then $foo
195 is available in the program and has the value 123.
196 · --slurpfile variable-name filename:
198 This option reads all the JSON texts in the named file and binds an
200 array of the parsed JSON values to the given global variable. If
201 you run jq with --argfile foo bar, then $foo is available in the
202 program and has an array whose elements correspond to the texts in
203 the file named bar.
204 · --argfile variable-name filename:
206 Do not use. Use --slurpfile instead.
208 (This option is like --slurpfile, but when the file has just one
210 text, then that is used, else an array of texts is used as in
211 --slurpfile.)
212 · --run-tests [filename]:
214 Runs the tests in the given file or standard input. This must be
216 the last option given and does not honor all preceding options. The
217 input consists of comment lines, empty lines, and program lines
218 followed by one input line, as many lines of output as are expected
219 (one per output), and a terminating empty line. Compilation failure
220 tests start with a line containing only "%%FAIL", then a line con‐
221 taining the program to compile, then a line containing an error
222 message to compare to the actual.
223 Be warned that this option can change backwards-incompatibly.
225
229 .
230 The absolute simplest (and least interesting) filter is .. This is a
231 filter that takes its input and produces it unchanged as output.
232 Since jq by default pretty-prints all output, this trivial program can
234 be a useful way of formatting JSON output from, say, curl.
235 jq ´.´
239 "Hello, world!"
240 => "Hello, world!"
241 .foo, .foo.bar
245 The simplest useful filter is .foo. When given a JSON object (aka dic‐
246 tionary or hash) as input, it produces the value at the key "foo", or
247 null if there´s none present.
248 If the key contains special characters, you need to surround it with
250 double quotes like this: ."foo$".
251 A filter of the form .foo.bar is equivalent to .foo|.bar.
253 jq ´.foo´
257 {"foo": 42, "bar": "less interesting data"}
258 => 42
259 jq ´.foo´
261 {"notfoo": true, "alsonotfoo": false}
262 => null
263 jq ´.["foo"]´
265 {"foo": 42}
266 => 42
267 .foo?
271 Just like .foo, but does not output even an error when . is not an
272 array or an object.
273 jq ´.foo?´
277 {"foo": 42, "bar": "less interesting data"}
278 => 42
279 jq ´.foo?´
281 {"notfoo": true, "alsonotfoo": false}
282 => null
283 jq ´.["foo"]?´
285 {"foo": 42}
286 => 42
287 jq ´[.foo?]´
289 [1,2]
290 => []
291 .[<string>], .[2], .[10:15]
295 You can also look up fields of an object using syntax like .["foo"]
296 (.foo above is a shorthand version of this). This one works for arrays
297 as well, if the key is an integer. Arrays are zero-based (like
298 javascript), so .[2] returns the third element of the array.
299 The .[10:15] syntax can be used to return a subarray of an array or
301 substring of a string. The array returned by .[10:15] will be of length
302 5, containing the elements from index 10 (inclusive) to index 15
303 (exclusive). Either index may be negative (in which case it counts
304 backwards from the end of the array), or omitted (in which case it
305 refers to the start or end of the array).
306 The .[2] syntax can be used to return the element at the given index.
308 Negative indices are allowed, with -1 referring to the last element, -2
309 referring to the next to last element, and so on.
310 The .foo syntax only works for simply keys i.e. keys that are all
312 alphanumeric characters. .[<string>] works with keys that contain spe‐
313 cial characters such as colons and dots. For example .["foo::bar"] and
314 .["foo.bar"] work while .foo::bar and .foo.bar would not.
315 The ? "operator" can also be used with the slice operator, as in
317 .[10:15]?, which outputs values where the inputs are slice-able.
318 jq ´.[0]´
322 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
323 => {"name":"JSON", "good":true}
324 jq ´.[2]´
326 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
327 => null
328 jq ´.[2:4]´
330 ["a","b","c","d","e"]
331 => ["c", "d"]
332 jq ´.[2:4]´
334 "abcdefghi"
335 => "cd"
336 jq ´.[:3]´
338 ["a","b","c","d","e"]
339 => ["a", "b", "c"]
340 jq ´.[-2:]´
342 ["a","b","c","d","e"]
343 => ["d", "e"]
344 jq ´.[-2]´
346 [1,2,3]
347 => 2
348 .[]
352 If you use the .[index] syntax, but omit the index entirely, it will
353 return all of the elements of an array. Running .[] with the input
354 [1,2,3] will produce the numbers as three separate results, rather than
355 as a single array.
356 You can also use this on an object, and it will return all the values
358 of the object.
359 jq ´.[]´
363 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
364 => {"name":"JSON", "good":true}, {"name":"XML", "good":false}
365 jq ´.[]´
367 []
368 =>
369 jq ´.[]´
371 {"a": 1, "b": 1}
372 => 1, 1
373 .[]?
377 Like .[], but no errors will be output if . is not an array or object.
378 ,
380 If two filters are separated by a comma, then the input will be fed
381 into both and there will be multiple outputs: first, all of the outputs
382 produced by the left expression, and then all of the outputs produced
383 by the right. For instance, filter .foo, .bar, produces both the "foo"
384 fields and "bar" fields as separate outputs.
385 jq ´.foo, .bar´
389 {"foo": 42, "bar": "something else", "baz": true}
390 => 42, "something else"
391 jq ´.user, .projects[]´
393 {"user":"stedolan", "projects": ["jq", "wikiflow"]}
394 => "stedolan", "jq", "wikiflow"
395 jq ´.[4,2]´
397 ["a","b","c","d","e"]
398 => "e", "c"
399 |
403 The | operator combines two filters by feeding the output(s) of the one
404 on the left into the input of the one on the right. It´s pretty much
405 the same as the Unix shell´s pipe, if you´re used to that.
406 If the one on the left produces multiple results, the one on the right
408 will be run for each of those results. So, the expression .[] | .foo
409 retrieves the "foo" field of each element of the input array.
410 jq ´.[] | .name´
414 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
415 => "JSON", "XML"
416
420 jq supports the same set of datatypes as JSON - numbers, strings, bool‐
421 eans, arrays, objects (which in JSON-speak are hashes with only string
422 keys), and "null".
423 Booleans, null, strings and numbers are written the same way as in
425 javascript. Just like everything else in jq, these simple values take
426 an input and produce an output - 42 is a valid jq expression that takes
427 an input, ignores it, and returns 42 instead.
428 Array construction - []
430 As in JSON, [] is used to construct arrays, as in [1,2,3]. The elements
431 of the arrays can be any jq expression. All of the results produced by
432 all of the expressions are collected into one big array. You can use it
433 to construct an array out of a known quantity of values (as in [.foo,
434 .bar, .baz]) or to "collect" all the results of a filter into an array
435 (as in [.items[].name])
436 Once you understand the "," operator, you can look at jq´s array syntax
438 in a different light: the expression [1,2,3] is not using a built-in
439 syntax for comma-separated arrays, but is instead applying the [] oper‐
440 ator (collect results) to the expression 1,2,3 (which produces three
441 different results).
442 If you have a filter X that produces four results, then the expression
444 [X] will produce a single result, an array of four elements.
445 jq ´[.user, .projects[]]´
449 {"user":"stedolan", "projects": ["jq", "wikiflow"]}
450 => ["stedolan", "jq", "wikiflow"]
451 Objects - {}
455 Like JSON, {} is for constructing objects (aka dictionaries or hashes),
456 as in: {"a": 42, "b": 17}.
457 If the keys are "sensible" (all alphabetic characters), then the quotes
459 can be left off. The value can be any expression (although you may need
460 to wrap it in parentheses if it´s a complicated one), which gets
461 applied to the {} expression´s input (remember, all filters have an
462 input and an output).
463 {foo: .bar}
467 will produce the JSON object {"foo": 42} if given the JSON object
471 {"bar":42, "baz":43}. You can use this to select particular fields of
472 an object: if the input is an object with "user", "title", "id", and
473 "content" fields and you just want "user" and "title", you can write
474 {user: .user, title: .title}
478 Because that´s so common, there´s a shortcut syntax: {user, title}.
482 If one of the expressions produces multiple results, multiple dictio‐
484 naries will be produced. If the input´s
485 {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
489 then the expression
493 {user, title: .titles[]}
497 will produce two outputs:
501 {"user":"stedolan", "title": "JQ Primer"}
505 {"user":"stedolan", "title": "More JQ"}
506 Putting parentheses around the key means it will be evaluated as an
510 expression. With the same input as above,
511 {(.user): .titles}
515 produces
519 {"stedolan": ["JQ Primer", "More JQ"]}
523 jq ´{user, title: .titles[]}´
525 {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
526 => {"user":"stedolan", "title": "JQ Primer"}, {"user":"stedolan", "title": "More JQ"}
527 jq ´{(.user): .titles}´
529 {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
530 => {"stedolan": ["JQ Primer", "More JQ"]}
531
535 Some jq operator (for instance, +) do different things depending on the
536 type of their arguments (arrays, numbers, etc.). However, jq never does
537 implicit type conversions. If you try to add a string to an object
538 you´ll get an error message and no result.
539 Addition - +
541 The operator + takes two filters, applies them both to the same input,
542 and adds the results together. What "adding" means depends on the types
543 involved:
544 · Numbers are added by normal arithmetic.
546 · Arrays are added by being concatenated into a larger array.
548 · Strings are added by being joined into a larger string.
550 · Objects are added by merging, that is, inserting all the key-value
552 pairs from both objects into a single combined object. If both
553 objects contain a value for the same key, the object on the right
554 of the + wins. (For recursive merge use the * operator.)
555 null can be added to any value, and returns the other value unchanged.
559 jq ´.a + 1´
563 {"a": 7}
564 => 8
565 jq ´.a + .b´
567 {"a": [1,2], "b": [3,4]}
568 => [1,2,3,4]
569 jq ´.a + null´
571 {"a": 1}
572 => 1
573 jq ´.a + 1´
575 {}
576 => 1
577 jq ´{a: 1} + {b: 2} + {c: 3} + {a: 42}´
579 null
580 => {"a": 42, "b": 2, "c": 3}
581 Subtraction - -
585 As well as normal arithmetic subtraction on numbers, the - operator can
586 be used on arrays to remove all occurrences of the second array´s ele‐
587 ments from the first array.
588 jq ´4 - .a´
592 {"a":3}
593 => 1
594 jq ´. - ["xml", "yaml"]´
596 ["xml", "yaml", "json"]
597 => ["json"]
598 Multiplication, division, modulo - *, /, and %
602 These infix operators behave as expected when given two numbers. Divi‐
603 sion by zero raises an error. x % y computes x modulo y.
604 Multiplying a string by a number produces the concatenation of that
606 string that many times. "x" * 0 produces null.
607 Dividing a string by another splits the first using the second as sepa‐
609 rators.
610 Multiplying two objects will merge them recursively: this works like
612 addition but if both objects contain a value for the same key, and the
613 values are objects, the two are merged with the same strategy.
614 jq ´10 / . * 3´
618 5
619 => 6
620 jq ´. / ", "´
622 "a, b,c,d, e"
623 => ["a","b,c,d","e"]
624 jq ´{"k": {"a": 1, "b": 2}} * {"k": {"a": 0,"c": 3}}´
626 null
627 => {"k": {"a": 0, "b": 2, "c": 3}}
628 jq ´.[] | (1 / .)?´
630 [1,0,-1]
631 => 1, -1
632 length
636 The builtin function length gets the length of various different types
637 of value:
638 · The length of a string is the number of Unicode codepoints it con‐
640 tains (which will be the same as its JSON-encoded length in bytes
641 if it´s pure ASCII).
642 · The length of an array is the number of elements.
644 · The length of an object is the number of key-value pairs.
646 · The length of null is zero.
648 jq ´.[] | length´ [[1,2], "string", {"a":2}, null] => 2, 6, 1, 0
650 keys, keys_unsorted
654 The builtin function keys, when given an object, returns its keys in an
655 array.
656 The keys are sorted "alphabetically", by unicode codepoint order. This
658 is not an order that makes particular sense in any particular language,
659 but you can count on it being the same for any two objects with the
660 same set of keys, regardless of locale settings.
661 When keys is given an array, it returns the valid indices for that
663 array: the integers from 0 to length-1.
664 The keys_unsorted function is just like keys, but if the input is an
666 object then the keys will not be sorted, instead the keys will roughly
667 be in insertion order.
668 jq ´keys´
672 {"abc": 1, "abcd": 2, "Foo": 3}
673 => ["Foo", "abc", "abcd"]
674 jq ´keys´
676 [42,3,35]
677 => [0,1,2]
678 has(key)
682 The builtin function has returns whether the input object has the given
683 key, or the input array has an element at the given index.
684 has($key) has the same effect as checking whether $key is a member of
686 the array returned by keys, although has will be faster.
687 jq ´map(has("foo"))´
691 [{"foo": 42}, {}]
692 => [true, false]
693 jq ´map(has(2))´
695 [[0,1], ["a","b","c"]]
696 => [false, true]
697 in
701 The builtin function in returns the input key is in the given object,
702 or the input index corresponds to an element in the given array. It is,
703 essentially, an inversed version of has.
704 jq ´.[] | in({"foo": 42})´
708 ["foo", "bar"]
709 => true, false
710 jq ´map(in([0,1]))´
712 [2, 0]
713 => [false, true]
714 path(path_expression)
718 Outputs array representations of the given path expression in .. The
719 outputs are arrays of strings (keys in objects0 and/or numbers (array
720 indices.
721 Path expressions are jq expressions like .a, but also .[]. There are
723 two types of path expressions: ones that can match exactly, and ones
724 that cannot. For example, .a.b.c is an exact match path expression,
725 while .a[].b is not.
726 path(exact_path_expression) will produce the array representation of
728 the path expression even if it does not exist in ., if . is null or an
729 array or an object.
730 path(pattern) will produce array representations of the paths matching
732 pattern if the paths exist in ..
733 Note that the path expressions are not different from normal expres‐
735 sions. The expression path(..|select(type=="boolean")) outputs all the
736 paths to boolean values in ., and only those paths.
737 jq ´path(.a[0].b)´
741 null
742 => ["a",0,"b"]
743 jq ´[path(..)]´
745 {"a":[{"b":1}]}
746 => [[],["a"],["a",0],["a",0,"b"]]
747 del(path_expression)
751 The builtin function del removes a key and its corresponding value from
752 an object.
753 jq ´del(.foo)´
757 {"foo": 42, "bar": 9001, "baz": 42}
758 => {"bar": 9001, "baz": 42}
759 jq ´del(.[1, 2])´
761 ["foo", "bar", "baz"]
762 => ["foo"]
763 to_entries, from_entries, with_entries
767 These functions convert between an object and an array of key-value
768 pairs. If to_entries is passed an object, then for each k: v entry in
769 the input, the output array includes {"key": k, "value": v}.
770 from_entries does the opposite conversion, and with_entries(foo) is a
772 shorthand for to_entries | map(foo) | from_entries, useful for doing
773 some operation to all keys and values of an object. from_entries
774 accepts key, Key, Name, value and Value as keys.
775 jq ´to_entries´
779 {"a": 1, "b": 2}
780 => [{"key":"a", "value":1}, {"key":"b", "value":2}]
781 jq ´from_entries´
783 [{"key":"a", "value":1}, {"key":"b", "value":2}]
784 => {"a": 1, "b": 2}
785 jq ´with_entries(.key |= "KEY_" + .)´
787 {"a": 1, "b": 2}
788 => {"KEY_a": 1, "KEY_b": 2}
789 select(boolean_expression)
793 The function select(foo) produces its input unchanged if foo returns
794 true for that input, and produces no output otherwise.
795 It´s useful for filtering lists: [1,2,3] | map(select(. >= 2)) will
797 give you [2,3].
798 jq ´map(select(. >= 2))´
802 [1,5,3,0,7]
803 => [5,3,7]
804 jq ´.[] | select(.id == "second")´
806 [{"id": "first", "val": 1}, {"id": "second", "val": 2}]
807 => {"id": "second", "val": 2}
808 arrays, objects, iterables, booleans, numbers, normals, finites, strings,
812 nulls, values, scalars
813 These built-ins select only inputs that are arrays, objects, iterables
814 (arrays or objects), booleans, numbers, normal numbers, finite numbers,
815 strings, null, non-null values, and non-iterables, respectively.
816 jq ´.[]|numbers´
820 [[],{},1,"foo",null,true,false]
821 => 1
822 empty
826 empty returns no results. None at all. Not even null.
827 It´s useful on occasion. You´ll know if you need it :)
829 jq ´1, empty, 2´
833 null
834 => 1, 2
835 jq ´[1,2,empty,3]´
837 null
838 => [1,2,3]
839 error(message)
843 Produces an error, just like .a applied to values other than null and
844 objects would, but with the given message as the error´s value.
845 $__loc__
847 Produces an object with a "file" key and a "line" key, with the file‐
848 name and line number where $__loc__ occurs, as values.
849 jq ´try error("\($__loc__)") catch .´
853 null
854 => "{\"file\":\"<top-level>\",\"line\":1}"
855 map(x), map_values(x)
859 For any filter x, map(x) will run that filter for each element of the
860 input array, and produce the outputs a new array. map(.+1) will incre‐
861 ment each element of an array of numbers.
862 Similarly, map_values(x) will run that filter for each element, but it
864 will return an object when an object is passed.
865 map(x) is equivalent to [.[] | x]. In fact, this is how it´s defined.
867 Similarly, map_values(x) is defined as .[] |= x.
868 jq ´map(.+1)´
872 [1,2,3]
873 => [2,3,4]
874 jq ´map_values(.+1)´
876 {"a": 1, "b": 2, "c": 3}
877 => {"a": 2, "b": 3, "c": 4}
878 paths, paths(node_filter), leaf_paths
882 paths outputs the paths to all the elements in its input (except it
883 does not output the empty list, representing . itself).
884 paths(f) outputs the paths to any values for which f is true. That is,
886 paths(numbers) outputs the paths to all numeric values.
887 leaf_paths is an alias of paths(scalars); leaf_paths is deprecated and
889 will be removed in the next major release.
890 jq ´[paths]´
894 [1,[[],{"a":2}]]
895 => [[0],[1],[1,0],[1,1],[1,1,"a"]]
896 jq ´[paths(scalars)]´
898 [1,[[],{"a":2}]]
899 => [[0],[1,1,"a"]]
900 add
904 The filter add takes as input an array, and produces as output the ele‐
905 ments of the array added together. This might mean summed, concatenated
906 or merged depending on the types of the elements of the input array -
907 the rules are the same as those for the + operator (described above).
908 If the input is an empty array, add returns null.
910 jq ´add´
914 ["a","b","c"]
915 => "abc"
916 jq ´add´
918 [1, 2, 3]
919 => 6
920 jq ´add´
922 []
923 => null
924 any, any(condition), any(generator; condition)
928 The filter any takes as input an array of boolean values, and produces
929 true as output if any of the the elements of the array is true.
930 If the input is an empty array, any returns false.
932 The any(condition) form applies the given condition to the elements of
934 the input array.
935 The any(generator; condition) form applies the given condition to all
937 the outputs of the given generator.
938 jq ´any´
942 [true, false]
943 => true
944 jq ´any´
946 [false, false]
947 => false
948 jq ´any´
950 []
951 => false
952 all, all(condition), all(generator; condition)
956 The filter all takes as input an array of boolean values, and produces
957 true as output if all of the the elements of the array are true.
958 The all(condition) form applies the given condition to the elements of
960 the input array.
961 The all(generator; condition) form applies the given condition to all
963 the outputs of the given generator.
964 If the input is an empty array, all returns true.
966 jq ´all´
970 [true, false]
971 => false
972 jq ´all´
974 [true, true]
975 => true
976 jq ´all´
978 []
979 => true
980 [Requires 1.5] flatten, flatten(depth)
984 The filter flatten takes as input an array of nested arrays, and pro‐
985 duces a flat array in which all arrays inside the original array have
986 been recursively replaced by their values. You can pass an argument to
987 it to specify how many levels of nesting to flatten.
988 flatten(2) is like flatten, but going only up to two levels deep.
990 jq ´flatten´
994 [1, [2], [[3]]]
995 => [1, 2, 3]
996 jq ´flatten(1)´
998 [1, [2], [[3]]]
999 => [1, 2, [3]]
1000 jq ´flatten´
1002 [[]]
1003 => []
1004 jq ´flatten´
1006 [{"foo": "bar"}, [{"foo": "baz"}]]
1007 => [{"foo": "bar"}, {"foo": "baz"}]
1008 range(upto), range(from;upto) range(from;upto;by)
1012 The range function produces a range of numbers. range(4;10) produces 6
1013 numbers, from 4 (inclusive) to 10 (exclusive). The numbers are produced
1014 as separate outputs. Use [range(4;10)] to get a range as an array.
1015 The one argument form generates numbers from 0 to the given number,
1017 with an increment of 1.
1018 The two argument form generates numbers from from to upto with an
1020 increment of 1.
1021 The three argument form generates numbers from to upto with an incre‐
1023 ment of by.
1024 jq ´range(2;4)´
1028 null
1029 => 2, 3
1030 jq ´[range(2;4)]´
1032 null
1033 => [2,3]
1034 jq ´[range(4)]´
1036 null
1037 => [0,1,2,3]
1038 jq ´[range(0;10;3)]´
1040 null
1041 => [0,3,6,9]
1042 jq ´[range(0;10;-1)]´
1044 null
1045 => []
1046 jq ´[range(0;-5;-1)]´
1048 null
1049 => [0,-1,-2,-3,-4]
1050 floor
1054 The floor function returns the floor of its numeric input.
1055 jq ´floor´
1059 3.14159
1060 => 3
1061 sqrt
1065 The sqrt function returns the square root of its numeric input.
1066 jq ´sqrt´
1070 9
1071 => 3
1072 tonumber
1076 The tonumber function parses its input as a number. It will convert
1077 correctly-formatted strings to their numeric equivalent, leave numbers
1078 alone, and give an error on all other input.
1079 jq ´.[] | tonumber´
1083 [1, "1"]
1084 => 1, 1
1085 tostring
1089 The tostring function prints its input as a string. Strings are left
1090 unchanged, and all other values are JSON-encoded.
1091 jq ´.[] | tostring´
1095 [1, "1", [1]]
1096 => "1", "1", "[1]"
1097 type
1101 The type function returns the type of its argument as a string, which
1102 is one of null, boolean, number, string, array or object.
1103 jq ´map(type)´
1107 [0, false, [], {}, null, "hello"]
1108 => ["number", "boolean", "array", "object", "null", "string"]
1109 infinite, nan, isinfinite, isnan, isfinite, isnormal
1113 Some arithmetic operations can yield infinities and "not a number"
1114 (NaN) values. The isinfinite builtin returns true if its input is infi‐
1115 nite. The isnan builtin returns true if its input is a NaN. The infi‐
1116 nite builtin returns a positive infinite value. The nan builtin returns
1117 a NaN. The isnormal builtin returns true if its input is a normal num‐
1118 ber.
1119 Note that division by zero raises an error.
1121 Currently most arithmetic operations operating on infinities, NaNs, and
1123 sub-normals do not raise errors.
1124 jq ´.[] | (infinite * .) < 0´
1128 [-1, 1]
1129 => true, false
1130 jq ´infinite, nan | type´
1132 null
1133 => "number", "number"
1134 sort, sort_by(path_expression)
1138 The sort functions sorts its input, which must be an array. Values are
1139 sorted in the following order:
1140 · null
1142 · false
1144 · true
1146 · numbers
1148 · strings, in alphabetical order (by unicode codepoint value)
1150 · arrays, in lexical order
1152 · objects
1154 The ordering for objects is a little complex: first they´re compared by
1158 comparing their sets of keys (as arrays in sorted order), and if their
1159 keys are equal then the values are compared key by key.
1160 sort may be used to sort by a particular field of an object, or by
1162 applying any jq filter.
1163 sort_by(foo) compares two elements by comparing the result of foo on
1165 each element.
1166 jq ´sort´
1170 [8,3,null,6]
1171 => [null,3,6,8]
1172 jq ´sort_by(.foo)´
1174 [{"foo":4, "bar":10}, {"foo":3, "bar":100}, {"foo":2, "bar":1}]
1175 => [{"foo":2, "bar":1}, {"foo":3, "bar":100}, {"foo":4, "bar":10}]
1176 group_by(path_expression)
1180 group_by(.foo) takes as input an array, groups the elements having the
1181 same .foo field into separate arrays, and produces all of these arrays
1182 as elements of a larger array, sorted by the value of the .foo field.
1183 Any jq expression, not just a field access, may be used in place of
1185 .foo. The sorting order is the same as described in the sort function
1186 above.
1187 jq ´group_by(.foo)´
1191 [{"foo":1, "bar":10}, {"foo":3, "bar":100}, {"foo":1, "bar":1}]
1192 => [[{"foo":1, "bar":10}, {"foo":1, "bar":1}], [{"foo":3, "bar":100}]]
1193 min, max, min_by(path_exp), max_by(path_exp)
1197 Find the minimum or maximum element of the input array.
1198 The min_by(path_exp) and max_by(path_exp) functions allow you to spec‐
1200 ify a particular field or property to examine, e.g. min_by(.foo) finds
1201 the object with the smallest foo field.
1202 jq ´min´
1206 [5,4,2,7]
1207 => 2
1208 jq ´max_by(.foo)´
1210 [{"foo":1, "bar":14}, {"foo":2, "bar":3}]
1211 => {"foo":2, "bar":3}
1212 unique, unique_by(path_exp)
1216 The unique function takes as input an array and produces an array of
1217 the same elements, in sorted order, with duplicates removed.
1218 The unique_by(path_exp) function will keep only one element for each
1220 value obtained by applying the argument. Think of it as making an array
1221 by taking one element out of every group produced by group.
1222 jq ´unique´
1226 [1,2,5,3,5,3,1,3]
1227 => [1,2,3,5]
1228 jq ´unique_by(.foo)´
1230 [{"foo": 1, "bar": 2}, {"foo": 1, "bar": 3}, {"foo": 4, "bar": 5}]
1231 => [{"foo": 1, "bar": 2}, {"foo": 4, "bar": 5}]
1232 jq ´unique_by(length)´
1234 ["chunky", "bacon", "kitten", "cicada", "asparagus"]
1235 => ["bacon", "chunky", "asparagus"]
1236 reverse
1240 This function reverses an array.
1241 jq ´reverse´
1245 [1,2,3,4]
1246 => [4,3,2,1]
1247 contains(element)
1251 The filter contains(b) will produce true if b is completely contained
1252 within the input. A string B is contained in a string A if B is a sub‐
1253 string of A. An array B is contained in an array A if all elements in B
1254 are contained in any element in A. An object B is contained in object A
1255 if all of the values in B are contained in the value in A with the same
1256 key. All other types are assumed to be contained in each other if they
1257 are equal.
1258 jq ´contains("bar")´
1262 "foobar"
1263 => true
1264 jq ´contains(["baz", "bar"])´
1266 ["foobar", "foobaz", "blarp"]
1267 => true
1268 jq ´contains(["bazzzzz", "bar"])´
1270 ["foobar", "foobaz", "blarp"]
1271 => false
1272 jq ´contains({foo: 12, bar: [{barp: 12}]})´
1274 {"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]}
1275 => true
1276 jq ´contains({foo: 12, bar: [{barp: 15}]})´
1278 {"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]}
1279 => false
1280 indices(s)
1284 Outputs an array containing the indices in . where s occurs. The input
1285 may be an array, in which case if s is an array then the indices output
1286 will be those where all elements in . match those of s.
1287 jq ´indices(", ")´
1291 "a,b, cd, efg, hijk"
1292 => [3,7,12]
1293 jq ´indices(1)´
1295 [0,1,2,1,3,1,4]
1296 => [1,3,5]
1297 jq ´indices([1,2])´
1299 [0,1,2,3,1,4,2,5,1,2,6,7]
1300 => [1,8]
1301 index(s), rindex(s)
1305 Outputs the index of the first (index) or last (rindex) occurrence of s
1306 in the input.
1307 jq ´index(", ")´
1311 "a,b, cd, efg, hijk"
1312 => 3
1313 jq ´rindex(", ")´
1315 "a,b, cd, efg, hijk"
1316 => 12
1317 inside
1321 The filter inside(b) will produce true if the input is completely con‐
1322 tained within b. It is, essentially, an inversed version of contains.
1323 jq ´inside("foobar")´
1327 "bar"
1328 => true
1329 jq ´inside(["foobar", "foobaz", "blarp"])´
1331 ["baz", "bar"]
1332 => true
1333 jq ´inside(["foobar", "foobaz", "blarp"])´
1335 ["bazzzzz", "bar"]
1336 => false
1337 jq ´inside({"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]})´
1339 {"foo": 12, "bar": [{"barp": 12}]}
1340 => true
1341 jq ´inside({"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]})´
1343 {"foo": 12, "bar": [{"barp": 15}]}
1344 => false
1345 startswith(str)
1349 Outputs true if . starts with the given string argument.
1350 jq ´[.[]|startswith("foo")]´
1354 ["fo", "foo", "barfoo", "foobar", "barfoob"]
1355 => [false, true, false, true, false]
1356 endswith(str)
1360 Outputs true if . ends with the given string argument.
1361 jq ´[.[]|endswith("foo")]´
1365 ["foobar", "barfoo"]
1366 => [false, true]
1367 combinations, combinations(n)
1371 Outputs all combinations of the elements of the arrays in the input
1372 array. If given an argument n, it outputs all combinations of n repeti‐
1373 tions of the input array.
1374 jq ´combinations´
1378 [[1,2], [3, 4]]
1379 => [1, 3], [1, 4], [2, 3], [2, 4]
1380 jq ´combinations(2)´
1382 [0, 1]
1383 => [0, 0], [0, 1], [1, 0], [1, 1]
1384 ltrimstr(str)
1388 Outputs its input with the given prefix string removed, if it starts
1389 with it.
1390 jq ´[.[]|ltrimstr("foo")]´
1394 ["fo", "foo", "barfoo", "foobar", "afoo"]
1395 => ["fo","","barfoo","bar","afoo"]
1396 rtrimstr(str)
1400 Outputs its input with the given suffix string removed, if it ends with
1401 it.
1402 jq ´[.[]|rtrimstr("foo")]´
1406 ["fo", "foo", "barfoo", "foobar", "foob"]
1407 => ["fo","","bar","foobar","foob"]
1408 explode
1412 Converts an input string into an array of the string´s codepoint num‐
1413 bers.
1414 jq ´explode´
1418 "foobar"
1419 => [102,111,111,98,97,114]
1420 implode
1424 The inverse of explode.
1425 jq ´implode´
1429 [65, 66, 67]
1430 => "ABC"
1431 split
1435 Splits an input string on the separator argument.
1436 jq ´split(", ")´
1440 "a, b,c,d, e, "
1441 => ["a","b,c,d","e",""]
1442 join(str)
1446 Joins the array of elements given as input, using the argument as sepa‐
1447 rator. It is the inverse of split: that is, running split("foo") |
1448 join("foo") over any input string returns said input string.
1449 jq ´join(", ")´
1453 ["a","b,c,d","e"]
1454 => "a, b,c,d, e"
1455 ascii_downcase, ascii_upcase
1459 Emit a copy of the input string with its alphabetic characters (a-z and
1460 A-Z) converted to the specified case.
1461 while(cond; update)
1463 The while(cond; update) function allows you to repeatedly apply an
1464 update to . until cond is false.
1465 Note that while(cond; update) is internally defined as a recursive jq
1467 function. Recursive calls within while will not consume additional mem‐
1468 ory if update produces at most one output for each input. See advanced
1469 topics below.
1470 jq ´[while(.<100; .*2)]´
1474 1
1475 => [1,2,4,8,16,32,64]
1476 until(cond; next)
1480 The until(cond; next) function allows you to repeatedly apply the
1481 expression next, initially to . then to its own output, until cond is
1482 true. For example, this can be used to implement a factorial function
1483 (see below).
1484 Note that until(cond; next) is internally defined as a recursive jq
1486 function. Recursive calls within until() will not consume additional
1487 memory if next produces at most one output for each input. See advanced
1488 topics below.
1489 jq ´[.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1]´
1493 4
1494 => 24
1495 recurse(f), recurse, recurse(f; condition), recurse_down
1499 The recurse(f) function allows you to search through a recursive struc‐
1500 ture, and extract interesting data from all levels. Suppose your input
1501 represents a filesystem:
1502 {"name": "/", "children": [
1506 {"name": "/bin", "children": [
1507 {"name": "/bin/ls", "children": []},
1508 {"name": "/bin/sh", "children": []}]},
1509 {"name": "/home", "children": [
1510 {"name": "/home/stephen", "children": [
1511 {"name": "/home/stephen/jq", "children": []}]}]}]}
1512 Now suppose you want to extract all of the filenames present. You need
1516 to retrieve .name, .children[].name, .children[].children[].name, and
1517 so on. You can do this with:
1518 recurse(.children[]) | .name
1522 When called without an argument, recurse is equivalent to
1526 recurse(.[]?).
1527 recurse(f) is identical to recurse(f; . != null) and can be used with‐
1529 out concerns about recursion depth.
1530 recurse(f; condition) is a generator which begins by emitting . and
1532 then emits in turn .|f, .|f|f, .|f|f|f, ... so long as the computed
1533 value satisfies the condition. For example, to generate all the inte‐
1534 gers, at least in principle, one could write recurse(.+1; true).
1535 For legacy reasons, recurse_down exists as an alias to calling recurse
1537 without arguments. This alias is considered deprecated and will be
1538 removed in the next major release.
1539 The recursive calls in recurse will not consume additional memory when‐
1541 ever f produces at most a single output for each input.
1542 jq ´recurse(.foo[])´
1546 {"foo":[{"foo": []}, {"foo":[{"foo":[]}]}]}
1547 => {"foo":[{"foo":[]},{"foo":[{"foo":[]}]}]}, {"foo":[]}, {"foo":[{"foo":[]}]}, {"foo":[]}
1548 jq ´recurse´
1550 {"a":0,"b":[1]}
1551 => {"a":0,"b":[1]}, 0, [1], 1
1552 jq ´recurse(. * .; . < 20)´
1554 2
1555 => 2, 4, 16
1556 ..
1560 Short-hand for recurse without arguments. This is intended to resemble
1561 the XPath // operator. Note that ..a does not work; use ..|a instead.
1562 In the example below we use ..|.a? to find all the values of object
1563 keys "a" in any object found "below" ..
1564 jq ´..|.a?´
1568 [[{"a":1}]]
1569 => 1
1570 env
1574 Outputs an object representing jq´s environment.
1575 jq ´env.PAGER´
1579 null
1580 => "less"
1581 transpose
1585 Transpose a possibly jagged matrix (an array of arrays). Rows are
1586 padded with nulls so the result is always rectangular.
1587 jq ´transpose´
1591 [[1], [2,3]]
1592 => [[1,2],[null,3]]
1593 bsearch(x)
1597 bsearch(x) conducts a binary search for x in the input array. If the
1598 input is sorted and contains x, then bsearch(x) will return its index
1599 in the array; otherwise, if the array is sorted, it will return (-1 -
1600 ix) where ix is an insertion point such that the array would still be
1601 sorted after the insertion of x at ix. If the array is not sorted,
1602 bsearch(x) will return an integer that is probably of no interest.
1603 jq ´bsearch(0)´
1607 [0,1]
1608 => 0
1609 jq ´bsearch(0)´
1611 [1,2,3]
1612 => -1
1613 jq ´bsearch(4) as $ix | if $ix < 0 then .[-(1+$ix)] = 4 else . end´
1615 [1,2,3]
1616 => [1,2,3,4]
1617 String interpolation - \(foo)
1621 Inside a string, you can put an expression inside parens after a back‐
1622 slash. Whatever the expression returns will be interpolated into the
1623 string.
1624 jq ´"The input was \(.), which is one less than \(.+1)"´
1628 42
1629 => "The input was 42, which is one less than 43"
1630 Convert to/from JSON
1634 The tojson and fromjson builtins dump values as JSON texts or parse
1635 JSON texts into values, respectively. The tojson builtin differs from
1636 tostring in that tostring returns strings unmodified, while tojson
1637 encodes strings as JSON strings.
1638 jq ´[.[]|tostring]´
1642 [1, "foo", ["foo"]]
1643 => ["1","foo","[\"foo\"]"]
1644 jq ´[.[]|tojson]´
1646 [1, "foo", ["foo"]]
1647 => ["1","\"foo\"","[\"foo\"]"]
1648 jq ´[.[]|tojson|fromjson]´
1650 [1, "foo", ["foo"]]
1651 => [1,"foo",["foo"]]
1652 Format strings and escaping
1656 The @foo syntax is used to format and escape strings, which is useful
1657 for building URLs, documents in a language like HTML or XML, and so
1658 forth. @foo can be used as a filter on its own, the possible escapings
1659 are:
1660 @text:
1662 Calls tostring, see that function for details.
1664 @json:
1666 Serializes the input as JSON.
1668 @html:
1670 Applies HTML/XML escaping, by mapping the characters <>&´" to
1672 their entity equivalents <, >, &, ', ".
1673 @uri:
1675 Applies percent-encoding, by mapping all reserved URI characters
1677 to a %XX sequence.
1678 @csv:
1680 The input must be an array, and it is rendered as CSV with dou‐
1682 ble quotes for strings, and quotes escaped by repetition.
1683 @tsv:
1685 The input must be an array, and it is rendered as TSV (tab-sepa‐
1687 rated values). Each input array will be printed as a single
1688 line. Fields are separated by a single tab (ascii 0x09). Input
1689 characters line-feed (ascii 0x0a), carriage-return (ascii 0x0d),
1690 tab (ascii 0x09) and backslash (ascii 0x5c) will be output as
1691 escape sequences \n, \r, \t, \\ respectively.
1692 @sh:
1694 The input is escaped suitable for use in a command-line for a
1696 POSIX shell. If the input is an array, the output will be a
1697 series of space-separated strings.
1698 @base64:
1700 The input is converted to base64 as specified by RFC 4648.
1702 This syntax can be combined with string interpolation in a useful way.
1704 You can follow a @foo token with a string literal. The contents of the
1705 string literal will not be escaped. However, all interpolations made
1706 inside that string literal will be escaped. For instance,
1707 @uri "https://www.google.com/search?q=\(.search)"
1711 will produce the following output for the input {"search":"what is
1715 jq?"}:
1716 "https://www.google.com/search?q=what%20is%20jq%3F"
1720 Note that the slashes, question mark, etc. in the URL are not escaped,
1724 as they were part of the string literal.
1725 jq ´@html´
1729 "This works if x < y"
1730 => "This works if x < y"
1731 jq ´@sh "echo \(.)"´
1733 "O´Hara´s Ale"
1734 => "echo ´O´\\´´Hara´\\´´s Ale´"
1735 Dates
1739 jq provides some basic date handling functionality, with some
1740 high-level and low-level builtins. In all cases these builtins deal
1741 exclusively with time in UTC.
1742 The fromdateiso8601 builtin parses datetimes in the ISO 8601 format to
1744 a number of seconds since the Unix epoch (1970-01-01T00:00:00Z). The
1745 todateiso8601 builtin does the inverse.
1746 The fromdate builtin parses datetime strings. Currently fromdate only
1748 supports ISO 8601 datetime strings, but in the future it will attempt
1749 to parse datetime strings in more formats.
1750 The todate builtin is an alias for todateiso8601.
1752 The now builtin outputs the current time, in seconds since the Unix
1754 epoch.
1755 Low-level jq interfaces to the C-library time functions are also pro‐
1757 vided: strptime, strftime, mktime, and gmtime. Refer to your host oper‐
1758 ating system´s documentation for the format strings used by strptime
1759 and strftime. Note: these are not necessarily stable interfaces in jq,
1760 particularly as to their localization functionality.
1761 The gmtime builtin consumes a number of seconds since the Unix epoch
1763 and outputs a "broken down time" representation of time as an array of
1764 numbers representing (in this order): the year, the month (zero-based),
1765 the day of the month, the hour of the day, the minute of the hour, the
1766 second of the minute, the day of the week, and the day of the year --
1767 all one-based unless otherwise stated.
1768 The mktime builtin consumes "broken down time" representations of time
1770 output by gmtime and strptime.
1771 The strptime(fmt) builtin parses input strings matching the fmt argu‐
1773 ment. The output is in the "broken down time" representation consumed
1774 by gmtime and output by mktime.
1775 The strftime(fmt) builtin formats a time with the given format.
1777 The format strings for strptime and strftime are described in typical C
1779 library documentation. The format string for ISO 8601 datetime is
1780 "%Y-%m-%dT%H:%M:%SZ".
1781 jq may not support some or all of this date functionality on some sys‐
1783 tems.
1784 jq ´fromdate´
1788 "2015-03-05T23:51:47Z"
1789 => 1425599507
1790 jq ´strptime("%Y-%m-%dT%H:%M:%SZ")´
1792 "2015-03-05T23:51:47Z"
1793 => [2015,2,5,23,51,47,4,63]
1794 jq ´strptime("%Y-%m-%dT%H:%M:%SZ")|mktime´
1796 "2015-03-05T23:51:47Z"
1797 => 1425599507
1798
1802 ==, !=
1803 The expression ´a == b´ will produce ´true´ if the result of a and b
1804 are equal (that is, if they represent equivalent JSON documents) and
1805 ´false´ otherwise. In particular, strings are never considered equal to
1806 numbers. If you´re coming from Javascript, jq´s == is like Javascript´s
1807 === - considering values equal only when they have the same type as
1808 well as the same value.
1809 != is "not equal", and ´a != b´ returns the opposite value of ´a == b´
1811 jq ´.[] == 1´
1815 [1, 1.0, "1", "banana"]
1816 => true, true, false, false
1817 if-then-else
1821 if A then B else C end will act the same as B if A produces a value
1822 other than false or null, but act the same as C otherwise.
1823 Checking for false or null is a simpler notion of "truthiness" than is
1825 found in Javascript or Python, but it means that you´ll sometimes have
1826 to be more explicit about the condition you want: you can´t test
1827 whether, e.g. a string is empty using if .name then A else B end,
1828 you´ll need something more like if (.name | length) > 0 then A else B
1829 end instead.
1830 If the condition A produces multiple results, it is considered "true"
1832 if any of those results is not false or null. If it produces zero
1833 results, it´s considered false.
1834 More cases can be added to an if using elif A then B syntax.
1836 jq ´if . == 0 then
1840 "zero" elif . == 1 then "one" else "many" end´ 2 => "many"
1844 >, >=, <=, <
1846 The comparison operators >, >=, <=, < return whether their left argu‐
1847 ment is greater than, greater than or equal to, less than or equal to
1848 or less than their right argument (respectively).
1849 The ordering is the same as that described for sort, above.
1851 jq ´. < 5´
1855 2
1856 => true
1857 and/or/not
1861 jq supports the normal Boolean operators and/or/not. They have the same
1862 standard of truth as if expressions - false and null are considered
1863 "false values", and anything else is a "true value".
1864 If an operand of one of these operators produces multiple results, the
1866 operator itself will produce a result for each input.
1867 not is in fact a builtin function rather than an operator, so it is
1869 called as a filter to which things can be piped rather than with spe‐
1870 cial syntax, as in .foo and .bar | not.
1871 These three only produce the values "true" and "false", and so are only
1873 useful for genuine Boolean operations, rather than the common
1874 Perl/Python/Ruby idiom of "value_that_may_be_null or default". If you
1875 want to use this form of "or", picking between two values rather than
1876 evaluating a condition, see the "//" operator below.
1877 jq ´42 and "a string"´
1881 null
1882 => true
1883 jq ´(true, false) or false´
1885 null
1886 => true, false
1887 jq ´(true, true) and (true, false)´
1889 null
1890 => true, false, true, false
1891 jq ´[true, false | not]´
1893 null
1894 => [false, true]
1895 Alternative operator - //
1899 A filter of the form a // b produces the same results as a, if a pro‐
1900 duces results other than false and null. Otherwise, a // b produces the
1901 same results as b.
1902 This is useful for providing defaults: .foo // 1 will evaluate to 1 if
1904 there´s no .foo element in the input. It´s similar to how or is some‐
1905 times used in Python (jq´s or operator is reserved for strictly Boolean
1906 operations).
1907 jq ´.foo // 42´
1911 {"foo": 19}
1912 => 19
1913 jq ´.foo // 42´
1915 {}
1916 => 42
1917 try-catch
1921 Errors can be caught by using try EXP catch EXP. The first expression
1922 is executed, and if it fails then the second is executed with the error
1923 message. The output of the handler, if any, is output as if it had been
1924 the output of the expression to try.
1925 The try EXP form uses empty as the exception handler.
1927 jq ´try .a catch ". is not an object"´
1931 true
1932 => ". is not an object"
1933 jq ´[.[]|try .a]´
1935 [{}, true, {"a":1}]
1936 => [null, 1]
1937 jq ´try error("some exception") catch .´
1939 true
1940 => "some exception"
1941 Breaking out of control structures
1945 A convenient use of try/catch is to break out of control structures
1946 like reduce, foreach, while, and so on.
1947 For example:
1949 # Repeat an expression until it raises "break" as an
1953 # error, then stop repeating without re-raising the error.
1954 # But if the error caught is not "break" then re-raise it.
1955 try repeat(exp) catch .=="break" then empty else error;
1956 jq has a syntax for named lexical labels to "break" or "go (back) to":
1960 label $out | ... break $out ...
1964 The break $label_name expression will cause the program to to act as
1968 though the nearest (to the left) label $label_name produced empty.
1969 The relationship between the break and corresponding label is lexical:
1971 the label has to be "visible" from the break.
1972 To break out of a reduce, for example:
1974 label $out | reduce .[] as $item (null; if .==false then break $out else ... end)
1978 The following jq program produces a syntax error:
1982 break $out
1986 because no label $out is visible.
1990 ? operator
1992 The ? operator, used as EXP?, is shorthand for try EXP.
1993 jq ´[.[]|(.a)?]´
1997 [{}, true, {"a":1}]
1998 => [null, 1]
1999
2003 jq uses the Oniguruma regular expression library, as do php, ruby,
2004 TextMate, Sublime Text, etc, so the description here will focus on jq
2005 specifics.
2006 The jq regex filters are defined so that they can be used using one of
2008 these patterns:
2009 STRING | FILTER( REGEX )
2013 STRING | FILTER( REGEX; FLAGS )
2014 STRING | FILTER( [REGEX] )
2015 STRING | FILTER( [REGEX, FLAGS] )
2016 where: * STRING, REGEX and FLAGS are jq strings and subject to jq
2020 string interpolation; * REGEX, after string interpolation, should be a
2021 valid PCRE regex; * FILTER is one of test, match, or capture, as
2022 described below.
2023 FLAGS is a string consisting of one of more of the supported flags:
2025 · g - Global search (find all matches, not just the first)
2027 · i - Case insensitive search
2029 · m - Multi line mode (´.´ will match newlines)
2031 · n - Ignore empty matches
2033 · p - Both s and m modes are enabled
2035 · s - Single line mode (´^´ -> ´\A´, ´$´ -> ´\Z´)
2037 · l - Find longest possible matches
2039 · x - Extended regex format (ignore whitespace and comments)
2041 To match whitespace in an x pattern use an escape such as \s, e.g.
2045 · test( "a\sb", "x" ).
2047 Note that certain flags may also be specified within REGEX, e.g.
2051 · jq -n ´("test", "TEst", "teST", "TEST") | test( "(?i)te(?-i)st" )´
2053 evaluates to: true, true, false, false.
2057 [Requires 1.5] test(val), test(regex; flags)
2059 Like match, but does not return match objects, only true or false for
2060 whether or not the regex matches the input.
2061 jq ´test("foo")´
2065 "foo"
2066 => true
2067 jq ´.[] | test("a b c # spaces are ignored"; "ix")´
2069 ["xabcd", "ABC"]
2070 => true, true
2071 [Requires 1.5] match(val), match(regex; flags)
2075 match outputs an object for each match it finds. Matches have the fol‐
2076 lowing fields:
2077 · offset - offset in UTF-8 codepoints from the beginning of the input
2079 · length - length in UTF-8 codepoints of the match
2081 · string - the string that it matched
2083 · captures - an array of objects representing capturing groups.
2085 Capturing group objects have the following fields:
2089 · offset - offset in UTF-8 codepoints from the beginning of the input
2091 · length - length in UTF-8 codepoints of this capturing group
2093 · string - the string that was captured
2095 · name - the name of the capturing group (or null if it was unnamed)
2097 Capturing groups that did not match anything return an offset of -1
2101 jq ´match("(abc)+"; "g")´
2105 "abc abc"
2106 => {"offset": 0, "length": 3, "string": "abc", "captures": [{"offset": 0, "length": 3, "string": "abc", "name": null}]}, {"offset": 4, "length": 3, "string": "abc", "captures": [{"offset": 4, "length": 3, "string": "abc", "name": null}]}
2107 jq ´match("foo")´
2109 "foo bar foo"
2110 => {"offset": 0, "length": 3, "string": "foo", "captures": []}
2111 jq ´match(["foo", "ig"])´
2113 "foo bar FOO"
2114 => {"offset": 0, "length": 3, "string": "foo", "captures": []}, {"offset": 8, "length": 3, "string": "FOO", "captures": []}
2115 jq ´match("foo (?<bar123>bar)? foo"; "ig")´
2117 "foo bar foo foo foo"
2118 => {"offset": 0, "length": 11, "string": "foo bar foo", "captures": [{"offset": 4, "length": 3, "string": "bar", "name": "bar123"}]}, {"offset": 12, "length": 8, "string": "foo foo", "captures": [{"offset": -1, "length": 0, "string": null, "name": "bar123"}]}
2119 jq ´[ match("."; "g")] | length´
2121 "abc"
2122 => 3
2123 [Requires 1.5] capture(val), capture(regex; flags)
2127 Collects the named captures in a JSON object, with the name of each
2128 capture as the key, and the matched string as the corresponding value.
2129 jq ´capture("(?<a>[a-z]+)-(?<n>[0-9]+)")´
2133 "xyzzy-14"
2134 => { "a": "xyzzy", "n": "14" }
2135 [Requires 1.5] scan(regex), scan(regex; flags)
2139 Emit a stream of the non-overlapping substrings of the input that match
2140 the regex in accordance with the flags, if any have been specified. If
2141 there is no match, the stream is empty. To capture all the matches for
2142 each input string, use the idiom [ expr ], e.g. [ scan(regex) ].
2143 split(regex; flags)
2145 For backwards compatibility, split splits on a string, not a regex.
2146 [Requires 1.5] splits(regex), splits(regex; flags)
2148 These provide the same results as their split counterparts, but as a
2149 stream instead of an array.
2150 [Requires 1.5] sub(regex; tostring) sub(regex; string; flags)
2152 Emit the string obtained by replacing the first match of regex in the
2153 input string with tostring, after interpolation. tostring should be a
2154 jq string, and may contain references to named captures. The named cap‐
2155 tures are, in effect, presented as a JSON object (as constructed by
2156 capture) to tostring, so a reference to a captured variable named "x"
2157 would take the form: "(.x)".
2158 [Requires 1.5] gsub(regex; string), gsub(regex; string; flags)
2160 gsub is like sub but all the non-overlapping occurrences of the regex
2161 are replaced by the string, after interpolation.
2162
2164 Variables are an absolute necessity in most programming languages, but
2165 they´re relegated to an "advanced feature" in jq.
2166 In most languages, variables are the only means of passing around data.
2168 If you calculate a value, and you want to use it more than once, you´ll
2169 need to store it in a variable. To pass a value to another part of the
2170 program, you´ll need that part of the program to define a variable (as
2171 a function parameter, object member, or whatever) in which to place the
2172 data.
2173 It is also possible to define functions in jq, although this is is a
2175 feature whose biggest use is defining jq´s standard library (many jq
2176 functions such as map and find are in fact written in jq).
2177 jq has reduction operators, which are very powerful but a bit tricky.
2179 Again, these are mostly used internally, to define some useful bits of
2180 jq´s standard library.
2181 It may not be obvious at first, but jq is all about generators (yes, as
2183 often found in other languages). Some utilities are provided to help
2184 deal with generators.
2185 Some minimal I/O support (besides reading JSON from standard input, and
2187 writing JSON to standard output) is available.
2188 Finally, there is a module/library system.
2190 Variables
2192 In jq, all filters have an input and an output, so manual plumbing is
2193 not necessary to pass a value from one part of a program to the next.
2194 Many expressions, for instance a + b, pass their input to two distinct
2195 subexpressions (here a and b are both passed the same input), so vari‐
2196 ables aren´t usually necessary in order to use a value twice.
2197 For instance, calculating the average value of an array of numbers
2199 requires a few variables in most languages - at least one to hold the
2200 array, perhaps one for each element or for a loop counter. In jq, it´s
2201 simply add / length - the add expression is given the array and pro‐
2202 duces its sum, and the length expression is given the array and pro‐
2203 duces its length.
2204 So, there´s generally a cleaner way to solve most problems in jq than
2206 defining variables. Still, sometimes they do make things easier, so jq
2207 lets you define variables using expression as $variable. All variable
2208 names start with $. Here´s a slightly uglier version of the array-aver‐
2209 aging example:
2210 length as $array_length | add / $array_length
2214 We´ll need a more complicated problem to find a situation where using
2218 variables actually makes our lives easier.
2219 Suppose we have an array of blog posts, with "author" and "title"
2221 fields, and another object which is used to map author usernames to
2222 real names. Our input looks like:
2223 {"posts": [{"title": "Frist psot", "author": "anon"},
2227 {"title": "A well-written article", "author": "person1"}],
2228 "realnames": {"anon": "Anonymous Coward",
2229 "person1": "Person McPherson"}}
2230 We want to produce the posts with the author field containing a real
2234 name, as in:
2235 {"title": "Frist psot", "author": "Anonymous Coward"}
2239 {"title": "A well-written article", "author": "Person McPherson"}
2240 We use a variable, $names, to store the realnames object, so that we
2244 can refer to it later when looking up author usernames:
2245 .realnames as $names | .posts[] | {title, author: $names[.author]}
2249 The expression exp as $x | ... means: for each value of expression exp,
2253 run the rest of the pipeline with the entire original input, and with
2254 $x set to that value. Thus as functions as something of a foreach loop.
2255 Just as {foo} is a handy way of writing {foo: .foo}, so {$foo} is a
2257 handy way of writing {foo:$foo}.
2258 Multiple variables may be declared using a single as expression by pro‐
2260 viding a pattern that matches the structure of the input (this is known
2261 as "destructuring"):
2262 . as {realnames: $names, posts: [$first, $second]} | ...
2266 The variable declarations in array patterns (e.g., . as [$first, $sec‐
2270 ond]) bind to the elements of the array in from the element at index
2271 zero on up, in order. When there is no value at the index for an array
2272 pattern element, null is bound to that variable.
2273 Variables are scoped over the rest of the expression that defines them,
2275 so
2276 .realnames as $names | (.posts[] | {title, author: $names[.author]})
2280 will work, but
2284 (.realnames as $names | .posts[]) | {title, author: $names[.author]}
2288 won´t.
2292 For programming language theorists, it´s more accurate to say that jq
2294 variables are lexically-scoped bindings. In particular there´s no way
2295 to change the value of a binding; one can only setup a new binding with
2296 the same name, but which will not be visible where the old one was.
2297 jq ´.bar as $x | .foo | . + $x´
2301 {"foo":10, "bar":200}
2302 => 210
2303 jq ´. as $i|[(.*2|. as $i| $i), $i]´
2305 5
2306 => [10,5]
2307 jq ´. as [$a, $b, {c: $c}] | $a + $b + $c´
2309 [2, 3, {"c": 4, "d": 5}]
2310 => 9
2311 jq ´.[] as [$a, $b] | {a: $a, b: $b}´
2313 [[0], [0, 1], [2, 1, 0]]
2314 => {"a":0,"b":null}, {"a":0,"b":1}, {"a":2,"b":1}
2315 Defining Functions
2319 You can give a filter a name using "def" syntax:
2320 def increment: . + 1;
2324 From then on, increment is usable as a filter just like a builtin func‐
2328 tion (in fact, this is how some of the builtins are defined). A func‐
2329 tion may take arguments:
2330 def map(f): [.[] | f];
2334 Arguments are passed as filters, not as values. The same argument may
2338 be referenced multiple times with different inputs (here f is run for
2339 each element of the input array). Arguments to a function work more
2340 like callbacks than like value arguments. This is important to under‐
2341 stand. Consider:
2342 def foo(f): f|f;
2346 5|foo(.*2)
2347 The result will be 20 because f is .*2, and during the first invocation
2351 of f . will be 5, and the second time it will be 10 (5 * 2), so the
2352 result will be 20. Function arguments are filters, and filters expect
2353 an input when invoked.
2354 If you want the value-argument behaviour for defining simple functions,
2356 you can just use a variable:
2357 def addvalue(f): f as $f | map(. + $f);
2361 Or use the short-hand:
2365 def addvalue($f): ...;
2369 With either definition, addvalue(.foo) will add the current input´s
2373 .foo field to each element of the array.
2374 Multiple definitions using the same function name are allowed. Each
2376 re-definition replaces the previous one for the same number of function
2377 arguments, but only for references from functions (or main program)
2378 subsequent to the re-definition.
2379 jq ´def addvalue(f): . + [f]; map(addvalue(.[0]))´
2383 [[1,2],[10,20]]
2384 => [[1,2,1], [10,20,10]]
2385 jq ´def addvalue(f): f as $x | map(. + $x); addvalue(.[0])´
2387 [[1,2],[10,20]]
2388 => [[1,2,1,2], [10,20,1,2]]
2389 Reduce
2393 The reduce syntax in jq allows you to combine all of the results of an
2394 expression by accumulating them into a single answer. As an example,
2395 we´ll pass [3,2,1] to this expression:
2396 reduce .[] as $item (0; . + $item)
2400 For each result that .[] produces, . + $item is run to accumulate a
2404 running total, starting from 0. In this example, .[] produces the
2405 results 3, 2, and 1, so the effect is similar to running something like
2406 this:
2407 0 | (3 as $item | . + $item) |
2411 (2 as $item | . + $item) |
2412 (1 as $item | . + $item)
2413 jq ´reduce .[] as $item (0; . + $item)´
2415 [10,2,5,3]
2416 => 20
2417 limit(n; exp)
2421 The limit function extracts up to n outputs from exp.
2422 jq ´[limit(3;.[])]´
2426 [0,1,2,3,4,5,6,7,8,9]
2427 => [0,1,2]
2428 first(expr), last(expr), nth(n; expr)
2432 The first(expr) and last(expr) functions extract the first and last
2433 values from expr, respectively.
2434 The nth(n; expr) function extracts the nth value output by expr. This
2436 can be defined as def nth(n; expr): last(limit(n + 1; expr));. Note
2437 that nth(n; expr) doesn´t support negative values of n.
2438 jq ´[first(range(.)), last(range(.)), nth(./2; range(.))]´
2442 10
2443 => [0,9,5]
2444 first, last, nth(n)
2448 The first and last functions extract the first and last values from any
2449 array at ..
2450 The nth(n) function extracts the nth value of any array at ..
2452 jq ´[range(.)]|[first, last, nth(5)]´
2456 10
2457 => [0,9,5]
2458 foreach
2462 The foreach syntax is similar to reduce, but intended to allow the con‐
2463 struction of limit and reducers that produce intermediate results (see
2464 example).
2465 The form is foreach EXP as $var (INIT; UPDATE; EXTRACT). Like reduce,
2467 INIT is evaluated once to produce a state value, then each output of
2468 EXP is bound to $var, UPDATE is evaluated for each output of EXP with
2469 the current state and with $var visible. Each value output by UPDATE
2470 replaces the previous state. Finally, EXTRACT is evaluated for each new
2471 state to extract an output of foreach.
2472 This is mostly useful only for constructing reduce- and limit-like
2474 functions. But it is much more general, as it allows for partial reduc‐
2475 tions (see the example below).
2476 jq ´[foreach .[] as $item ([[],[]]; if $item == null then [[],.[0]] else [(.[0] + [$item]),[]] end; if $item == null then .[1] else empty end)]´
2480 [1,2,3,4,null,"a","b",null]
2481 => [[1,2,3,4],["a","b"]]
2482 Recursion
2486 As described above, recurse uses recursion, and any jq function can be
2487 recursive. The while builtin is also implemented in terms of recursion.
2488 Tail calls are optimized whenever the expression to the left of the
2490 recursive call outputs its last value. In practice this means that the
2491 expression to the left of the recursive call should not produce more
2492 than one output for each input.
2493 For example:
2495 def recurse(f): def r: ., (f | select(. != null) | r); r;
2499 def while(cond; update):
2501 def _while:
2502 if cond then ., (update | _while) else empty end;
2503 _while;
2504 def repeat(exp):
2506 def _repeat:
2507 exp, _repeat;
2508 _repeat;
2509 Generators and iterators
2513 Some jq operators and functions are actually generators in that they
2514 can produce zero, one, or more values for each input, just as one might
2515 expect in other programming languages that have generators. For exam‐
2516 ple, .[] generates all the values in its input (which must be an array
2517 or an object), range(0; 10) generates the integers between 0 and 10,
2518 and so on.
2519 Even the comma operator is a generator, generating first the values
2521 generated by the expression to the left of the comma, then for each of
2522 those, the values generate by the expression on the right of the comma.
2523 The empty builtin is the generator that produces zero outputs. The
2525 empty builtin backtracks to the preceding generator expression.
2526 All jq functions can be generators just by using builtin generators. It
2528 is also possible to define new generators using only recursion and the
2529 comma operator. If the recursive call(s) is(are) "in tail position"
2530 then the generator will be efficient. In the example below the recur‐
2531 sive call by _range to itself is in tail position. The example shows
2532 off three advanced topics: tail recursion, generator construction, and
2533 sub-functions.
2534 jq ´def range(init; upto; by): def _range: if (by > 0 and . < upto) or (by < 0 and . > upto) then ., ((.+by)|_range) else . end; if by == 0 then init else init|_range end | select((by > 0 and . < upto) or (by < 0 and . > upto)); range(0; 10; 3)´
2538 null
2539 => 0, 3, 6, 9
2540 jq ´def while(cond; update): def _while: if cond then ., (update | _while) else empty end; _while; [while(.<100; .*2)]´
2542 1
2543 => [1,2,4,8,16,32,64]
2544
2548 jq currently only has IEEE754 double-precision (64-bit) floating point
2549 number support.
2550 Besides simple arithmetic operators such as +, jq also has most stan‐
2552 dard math functions from the C math library. C math functions that take
2553 a single input argument (e.g., sin()) are available as zero-argument jq
2554 functions. C math functions that take two input arguments (e.g., pow())
2555 are available as two-argument jq functions that ignore ..
2556 Availability of standard math functions depends on the availability of
2558 the corresponding math functions in your operating system and C math
2559 library. Unavailable math functions will be defined but will raise an
2560 error.
2561
2563 At this time jq has minimal support for I/O, mostly in the form of con‐
2564 trol over when inputs are read. Two builtins functions are provided for
2565 this, input and inputs, that read from the same sources (e.g., stdin,
2566 files named on the command-line) as jq itself. These two builtins, and
2567 jq´s own reading actions, can be interleaved with each other.
2568 One builtin provides minimal output capabilities, debug. (Recall that a
2570 jq program´s output values are always output as JSON texts on stdout.)
2571 The debug builtin can have application-specific behavior, such as for
2572 executables that use the libjq C API but aren´t the jq executable
2573 itself.
2574 input
2576 Outputs one new input.
2577 inputs
2579 Outputs all remaining inputs, one by one.
2580 This is primarily useful for reductions over a program´s inputs.
2582 debug
2584 Causes a debug message based on the input value to be produced. The jq
2585 executable wraps the input value with ["DEBUG:", <input-value>] and
2586 prints that and a newline on stderr, compactly. This may change in the
2587 future.
2588 input_filename
2590 Returns the name of the file whose input is currently being filtered.
2591 Note that this will not work well unless jq is running in a UTF-8
2592 locale.
2593 input_line_number
2595 Returns the line number of the input currently being filtered.
2596
2598 With the --stream option jq can parse input texts in a streaming fash‐
2599 ion, allowing jq programs to start processing large JSON texts immedi‐
2600 ately rather than after the parse completes. If you have a single JSON
2601 text that is 1GB in size, streaming it will allow you to process it
2602 much more quickly.
2603 However, streaming isn´t easy to deal with as the jq program will have
2605 [<path>, <leaf-value>] (and a few other forms) as inputs.
2606 Several builtins are provided to make handling streams easier.
2608 The examples below use the the streamed form of [0,[1]], which is
2610 [[0],0],[[1,0],1],[[1,0]],[[1]].
2611 Streaming forms include [<path>, <leaf-value>] (to indicate any scalar
2613 value, empty array, or empty object), and [<path>] (to indicate the end
2614 of an array or object). Future versions of jq run with --stream and
2615 -seq may output additional forms such as ["error message"] when an
2616 input text fails to parse.
2617 truncate_stream(stream_expression)
2619 Consumes a number as input and truncates the corresponding number of
2620 path elements from the left of the outputs of the given streaming
2621 expression.
2622 jq ´[1|truncate_stream([[0],1],[[1,0],2],[[1,0]],[[1]])]´
2626 1
2627 => [[[0],2],[[0]]]
2628 fromstream(stream_expression)
2632 Outputs values corresponding to the stream expression´s outputs.
2633 jq ´fromstream(1|truncate_stream([[0],1],[[1,0],2],[[1,0]],[[1]]))´
2637 null
2638 => [2]
2639 tostream
2643 The tostream builtin outputs the streamed form of its input.
2644 jq ´. as $dot|fromstream($dot|tostream)|.==$dot´
2648 [0,[1,{"a":1},{"b":2}]]
2649 => true
2650
2654 Assignment works a little differently in jq than in most programming
2655 languages. jq doesn´t distinguish between references to and copies of
2656 something - two objects or arrays are either equal or not equal, with‐
2657 out any further notion of being "the same object" or "not the same
2658 object".
2659 If an object has two fields which are arrays, .foo and .bar, and you
2661 append something to .foo, then .bar will not get bigger. Even if you´ve
2662 just set .bar = .foo. If you´re used to programming in languages like
2663 Python, Java, Ruby, Javascript, etc. then you can think of it as though
2664 jq does a full deep copy of every object before it does the assignment
2665 (for performance, it doesn´t actually do that, but that´s the general
2666 idea).
2667 All the assignment operators in jq have path expressions on the
2669 left-hand side.
2670 =
2672 The filter .foo = 1 will take as input an object and produce as output
2673 an object with the "foo" field set to 1. There is no notion of "modify‐
2674 ing" or "changing" something in jq - all jq values are immutable. For
2675 instance,
2676 .foo = .bar | .foo.baz = 1
2678 will not have the side-effect of setting .bar.baz to be set to 1, as
2680 the similar-looking program in Javascript, Python, Ruby or other lan‐
2681 guages would. Unlike these languages (but like Haskell and some other
2682 functional languages), there is no notion of two arrays or objects
2683 being "the same array" or "the same object". They can be equal, or not
2684 equal, but if we change one of them in no circumstances will the other
2685 change behind our backs.
2686 This means that it´s impossible to build circular values in jq (such as
2688 an array whose first element is itself). This is quite intentional, and
2689 ensures that anything a jq program can produce can be represented in
2690 JSON.
2691 Note that the left-hand side of ´=´ refers to a value in .. Thus
2693 $var.foo = 1 won´t work as expected ($var.foo is not a valid or useful
2694 path expression in .); use $var | .foo = 1 instead.
2695 If the right-hand side of ´=´ produces multiple values, then for each
2697 such value jq will set the paths on the left-hand side to the value and
2698 then it will output the modified .. For example, (.a,.b)=range(2) out‐
2699 puts {"a":0,"b":0}, then {"a":1,"b":1}. The "update" assignment forms
2700 (see below) do not do this.
2701 Note too that .a,.b=0 does not set .a and .b, but (.a,.b)=0 sets both.
2703 |=
2705 As well as the assignment operator ´=´, jq provides the "update" opera‐
2706 tor ´|=´, which takes a filter on the right-hand side and works out the
2707 new value for the property of . being assigned to by running the old
2708 value through this expression. For instance, .foo |= .+1 will build an
2709 object with the "foo" field set to the input´s "foo" plus 1.
2710 This example should show the difference between ´=´ and ´|=´:
2712 Provide input ´{"a": {"b": 10}, "b": 20}´ to the programs:
2714 .a = .b .a |= .b
2716 The former will set the "a" field of the input to the "b" field of the
2718 input, and produce the output {"a": 20}. The latter will set the "a"
2719 field of the input to the "a" field´s "b" field, producing {"a": 10}.
2720 The left-hand side can be any general path expression; see path().
2722 Note that the left-hand side of ´|=´ refers to a value in .. Thus
2724 $var.foo |= . + 1 won´t work as expected ($var.foo is not a valid or
2725 useful path expression in .); use $var | .foo |= . + 1 instead.
2726 If the right-hand side outputs multiple values, only the last one will
2728 be used.
2729 jq ´(..|select(type=="boolean")) |= if . then 1 else 0 end´
2733 [true,false,[5,true,[true,[false]],false]]
2734 => [1,0,[5,1,[1,[0]],0]]
2735 +=, -=, *=, /=, %=, //=
2739 jq has a few operators of the form a op= b, which are all equivalent to
2740 a |= . op b. So, += 1 can be used to increment values.
2741 jq ´.foo += 1´
2745 {"foo": 42}
2746 => {"foo": 43}
2747 Complex assignments
2751 Lots more things are allowed on the left-hand side of a jq assignment
2752 than in most languages. We´ve already seen simple field accesses on the
2753 left hand side, and it´s no surprise that array accesses work just as
2754 well:
2755 .posts[0].title = "JQ Manual"
2759 What may come as a surprise is that the expression on the left may pro‐
2763 duce multiple results, referring to different points in the input docu‐
2764 ment:
2765 .posts[].comments |= . + ["this is great"]
2769 That example appends the string "this is great" to the "comments" array
2773 of each post in the input (where the input is an object with a field
2774 "posts" which is an array of posts).
2775 When jq encounters an assignment like ´a = b´, it records the "path"
2777 taken to select a part of the input document while executing a. This
2778 path is then used to find which part of the input to change while exe‐
2779 cuting the assignment. Any filter may be used on the left-hand side of
2780 an equals - whichever paths it selects from the input will be where the
2781 assignment is performed.
2782 This is a very powerful operation. Suppose we wanted to add a comment
2784 to blog posts, using the same "blog" input above. This time, we only
2785 want to comment on the posts written by "stedolan". We can find those
2786 posts using the "select" function described earlier:
2787 .posts[] | select(.author == "stedolan")
2791 The paths provided by this operation point to each of the posts that
2795 "stedolan" wrote, and we can comment on each of them in the same way
2796 that we did before:
2797 (.posts[] | select(.author == "stedolan") | .comments) |=
2801 . + ["terrible."]
2802
2806 jq has a library/module system. Modules are files whose names end in
2807 .jq.
2808 Modules imported by a program are searched for in a default search path
2810 (see below). The import and include directives allow the importer to
2811 alter this path.
2812 Paths in the a search path are subject to various substitutions.
2814 For paths starting with "~/", the user´s home directory is substituted
2816 for "~".
2817 For paths starting with "$ORIGIN/", the path of the jq executable is
2819 substituted for "$ORIGIN".
2820 For paths starting with "./" or paths that are ".", the path of the
2822 including file is substituted for ".". For top-level programs given on
2823 the command-line, the current directory is used.
2824 Import directives can optionally specify a search path to which the
2826 default is appended.
2827 The default search path is the search path given to the -L command-line
2829 option, else ["~/.jq", "$ORIGIN/../lib/jq", "$ORIGIN/../lib"].
2830 Null and empty string path elements terminate search path processing.
2832 A dependency with relative path "foo/bar" would be searched for in
2834 "foo/bar.jq" and "foo/bar/bar.jq" in the given search path. This is
2835 intended to allow modules to be placed in a directory along with, for
2836 example, version control files, README files, and so on, but also to
2837 allow for single-file modules.
2838 Consecutive components with the same name are not allowed to avoid
2840 ambiguities (e.g., "foo/foo").
2841 For example, with -L$HOME/.jq a module foo can be found in
2843 $HOME/.jq/foo.jq and $HOME/.jq/foo/foo.jq.
2844 If "$HOME/.jq" is a file, it is sourced into the main program.
2846 import RelativePathString as NAME [<metadata>];
2848 Imports a module found at the given path relative to a directory in a
2849 search path. A ".jq" suffix will be added to the relative path string.
2850 The module´s symbols are prefixed with "NAME::".
2851 The optional metadata must be a constant jq expression. It should be an
2853 object with keys like "homepage" and so on. At this time jq only uses
2854 the "search" key/value of the metadata. The metadata is also made
2855 available to users via the modulemeta builtin.
2856 The "search" key in the metadata, if present, should have a string or
2858 array value (array of strings); this is the search path to be prefixed
2859 to the top-level search path.
2860 include RelativePathString [<metadata>];
2862 Imports a module found at the given path relative to a directory in a
2863 search path as if it were included in place. A ".jq" suffix will be
2864 added to the relative path string. The module´s symbols are imported
2865 into the caller´s namespace as if the module´s content had been
2866 included directly.
2867 The optional metadata must be a constant jq expression. It should be an
2869 object with keys like "homepage" and so on. At this time jq only uses
2870 the "search" key/value of the metadata. The metadata is also made
2871 available to users via the modulemeta builtin.
2872 import RelativePathString as $NAME [<metadata>];
2874 Imports a JSON file found at the given path relative to a directory in
2875 a search path. A ".json" suffix will be added to the relative path
2876 string. The file´s data will be available as $NAME::NAME.
2877 The optional metadata must be a constant jq expression. It should be an
2879 object with keys like "homepage" and so on. At this time jq only uses
2880 the "search" key/value of the metadata. The metadata is also made
2881 available to users via the modulemeta builtin.
2882 The "search" key in the metadata, if present, should have a string or
2884 array value (array of strings); this is the search path to be prefixed
2885 to the top-level search path.
2886 module <metadata>;
2888 This directive is entirely optional. It´s not required for proper oper‐
2889 ation. It serves only the purpose of providing metadata that can be
2890 read with the modulemeta builtin.
2891 The metadata must be a constant jq expression. It should be an object
2893 with keys like "homepage". At this time jq doesn´t use this metadata,
2894 but it is made available to users via the modulemeta builtin.
2895 modulemeta
2897 Takes a module name as input and outputs the module´s metadata as an
2898 object, with the module´s imports (including metadata) as an array
2899 value for the "deps" key.
2900 Programs can use this to query a module´s metadata, which they could
2902 then use to, for example, search for, download, and install missing
2903 dependencies.
2904
2906 Presumably. Report them or discuss them at:
2907 https://github.com/stedolan/jq/issues
2911
2915 Stephen Dolan <mu@netsoc.tcd.ie>
2916 August 2015 JQ(1)