The pipe is the next-most used feature of jq
after filters. If you are already familiar with pipes in the shell, it’s a very similar concept.
Pipes are fundamental to using jq
, so this section is essential to understand. We will cover:
|
).[]
‘ filterCreate a folder to work in, and move into it:
$ mkdir ljqthw_pipes
$ cd ljqthw_pipes
.[]
FilterBefore we get to the pipe, we’re going to talk about another filter. This filter
doesn’t have a snappy name, so we’re going to refer to it as the .[]
filter. It’s
technically known as the ‘array/object value iterator’.
Create a simple JSON document in a file:
$ echo '[{"username": "alice", "id": 1 }, {"username": "bob", "id": 2 }]' > doc1
This document is an array representing users on a system, with two objects in it.
Each object has a username
and id
name-value pair.
Since the file is a simple array, we can filter the individual objects by referring
to the array index in a fashion similar to other languages, by putting the index
number in square brackets:
$ jq '.[0]' doc1
$ jq '.[1]' doc1
If the referenced index doesn’t exist, then a null
is returned:
$ jq '.[2]' doc1
You can also refer to negative indexes to reference the array from the end rather than the start:
$ jq '.[-1]' doc1
$ jq '.[-2]' doc1
$ jq '.[-3]' doc1
If you don’t provide an index, then all the items in the array are returned:
$ jq '.[]' doc1
Look at the output. How are the items returned?
Running these commands may help you decide. Note that in the below commands we are using a shell pipe, not a jq
pipe. The two uses of the pipe (|
) character are analogous, but not the same:
$ jq '.[]' doc1 | jq '.'
$ jq '.[]' doc1 | jq '.[0]'
It’s really important to understand that the .[]
filter does not just work on
arrays, even though it uses an array’s square brackets. If a series of JSON
objects are supplied, then it will iterate through the name value pairs, returning
the values one by one. That’s why it’s called the ‘array/object value iterator’.
Type this out to see that in practice:
$ echo '{"username": "alice"} {"id": 1}' > doc2
$ jq '.' doc2
$ jq '.[]' doc2
This time you had no array, just a series of two JSON objects. The .[]
filter
iterates over a series of objects, returning the values in it. Remember, it’s
formally known as the ‘array/object value iterator’.
Now try this. What happens? Why?
$ echo '"username"' > doc3
$ jq '.[]' doc3
This filter confused me for a long time when I was learning jq
, as I thought the.[]
filter just removed the square brackets from the json, returning an array’s
contents, like this example does:
$ echo '[{"a": "b", "c": "d" }]' > doc4
$ jq '.' doc4
$ jq '.[]' doc4
While it obviously does do that in this case, you should now be aware that this
filter does more than just ‘remove the square brackets’!
OK, back to doc1
. This time you’re going to run the .[]
filter over it, and then
use a jq
pipe to process it further. Run these commands and think about what
the pipe character (|
) is doing.
$ jq '.' doc1
$ jq '.[]' doc1
$ jq '.[] | .["username"]' doc1
The first command shows you the JSON document pretty-printed, to remind you of the
raw content.
The second command (the ‘array/object value iterator’) iterates over the objects in
the array, and produces two JSON documents. This is an example of what was pointed
out above: the .[]
filter does not simply ‘remove the square brackets’. If it
did, the resulting two objects would have commas between them. It turns the two
objects within the array into a series of JSON documents. Make sure you understand
this before continuing.
The third command introduces the jq
pipe. Note that this time, the pipe character
(|
) is inside the jq
string’s single quotes, making it a jq
pipe rather than
the ‘shell pipe’ we saw earlier. The jq
pipe passes the output of the .[]
operator
on to the next filter (.["username"]
). This filter then effectively retrieves
the value of the username
name/value pair from each JSON document from the output
of the first .[]
filter.
You can run the last command in other ways too, with identical results:
$ jq '.[] | ."username"' doc1
$ jq '.[] | .username' doc1
jq
will figure out what you mean if you use either of those shorthands.
Not all JSON documents have objects in them with consistent names. Here you create
a document with inconsistent names in its objects (username
and uname
):
$ echo '[{"username": "alice"}, {"uname": "bob" }]' > doc4
Now see what happens when you reference one of the names in your query:
$ jq '.[] | .username' doc4
The object with the matching name outputs the its value, but the other
object outputs a null
value, indicating that there was no match for
the query for that object.
Referencing the other object’s name swaps which value it outputs and which
results in a null
:
$ jq '.[] | .uname' doc4
.[]
(array/object value iterator) filter does, and how it works on arrays and objects1) Create a file with three JSON documents in them: an array with a single name-value pair object in it, an array with two name-value pair objects in it, and a single bare object. Run it through the .[]
filter and predict what output it will give.