Data
JSON & jq — query, filter, transform, and build JSON
Query, filter, reshape, and build JSON on the command line with jq: path selectors, pipes, select/map/has, object and array construction, group_by/unique/sort_by, reduce, string ops, raw output, --arg injection, from_entries/to_entries, and slurp/stream for large files.
Quick reference — reach for these first
jq '.' file.jsonParse, reformat, and color JSON; exits non-zero on invalid input, so it doubles as a validator.jq '.items[]' file.jsonStream each element of the .items array as a separate value for the rest of the pipe.jq 'map(.x)' file.jsonApply a filter to every element of an input array and collect the results back into an array.jq -r '.name' file.jsonEmit string values without surrounding quotes or escaping — needed when piping to other shell tools.jq 'select(.ok)' file.jsonKeep only inputs where the boolean expression is truthy; drop everything else from the stream.A jq filter is a pipeline: data flows left to right through stages joined by |
Text equivalent: Left-to-right pipeline of five boxes. Stage 1 input JSON object with an items array. Arrow to stage 2 .items[] which iterates the array. Arrow to stage 3 select(.status != "ok") which filters the stream. Arrow to stage 4 [.id, .name, .status] which builds a flat array per item. Arrow to stage 5 @csv which formats each array into a CSV row. Output below is a three-line CSV report.
Common tasks
Select & filter · Build & reshape · I/O & arguments
| Select & filter | Build & reshape | I/O & arguments |
|---|---|---|
| Pretty-print and validate a file | jq '.' data.json | Reformatted JSON prints and exit code is 0; a parse error prints to stderr and exits non-zero. |
| Reach into nested keys | jq '.a.b.c' data.json | Prints the value at that path, or null if any segment is missing. |
| Iterate an array of objects | jq '.items[]' data.json | Each object prints on its own as a separate top-level value. |
| Iterate safely when the field may be absent | jq '.items[]?' data.json | No 'Cannot iterate' error when .items is null or missing — it yields nothing instead. |
| Filter a stream by a predicate | jq '.items[] | select(.status == "fail")','data.json | Only elements whose status equals fail survive the pipe. |
| Map and pick fields into new objects | jq '[.items[] | {id, label: .name}]' data.json | Returns an array of trimmed objects; {id} is shorthand for id: .id. |
| Keep objects that have a key | jq '[.items[] | select(has("error"))]' data.json | Only objects containing an error key are returned. |
| Group, dedupe, and sort | jq 'group_by(.status) | map({status: .[0].status, n: length})' data.json | One row per distinct status with its count; use unique and sort_by(.x) for related shaping. |
| Sum or fold with reduce | jq 'reduce .items[].latency_ms as $x (0; . + $x)' data.json | Prints the running total; the accumulator starts at 0 and updates per element. |
| Inject shell values safely | jq --arg s fail '.items[] | select(.status == $s)' data.json | $s is a quoted string; use --argjson n 5 to pass a number/JSON literal instead. |
| Convert object <-> entries | jq 'to_entries | map(select(.value > 0)) | from_entries' data.json | Round-trips an object through {key,value} pairs so you can filter by value. |
| Slurp multiple inputs into one array | jq -s '.' a.json b.json | All inputs are read into a single array before the filter runs (-s / --slurp). |
| Extract text from a Claude API response | jq -r '.content[0].text' response.json | Prints just the assistant text from a Messages API reply (Opus 4.x / Sonnet) without JSON quoting. |
| Stream a file too big for memory | jq --stream -c '. ' huge.json | Emits [path, leaf-value] events incrementally instead of loading the whole document. |
Turn an API response into a failing-checks CSV report
csv-report
curl -s https://API_HOST/v1/healthchecks
# {"items":[{"id":1,"name":"api","status":"ok","latency_ms":42},{"id":2,"name":"db","status":"fail","latency_ms":1200},{"id":3,"name":"cache","status":"degraded","latency_ms":300}]}curl -s https://API_HOST/v1/healthchecks | jq '.items[]'
# {"id":1,"name":"api","status":"ok","latency_ms":42}
# {"id":2,"name":"db","status":"fail","latency_ms":1200}
# {"id":3,"name":"cache","status":"degraded","latency_ms":300}... | jq '.items[] | select(.status != "ok")'
# {"id":2,"name":"db","status":"fail","latency_ms":1200}
# {"id":3,"name":"cache","status":"degraded","latency_ms":300}... | jq -r '["id","name","status","latency_ms"], (.items[] | select(.status != "ok") | [.id, .name, .status, .latency_ms])'
# ["id","name","status","latency_ms"] (header, then one array per failing row)... | jq -r '["id","name","status","latency_ms"], (.items[] | select(.status != "ok") | [.id, .name, .status, .latency_ms]) | @csv'
# id,name,status,latency_ms
# 2,"db","fail",1200
# 3,"cache","degraded",300Troubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| jq: error: Cannot index array with "name" | Check whether the value at that point is an array; print the parent with 'jq '.items''. | You applied an object accessor like .name directly to an array value. | Insert .[] to iterate first: '.items[] | .name' instead of '.items.name'. |
| Output is just 'null' | Print the parent level and compare key names exactly, including case: 'jq 'keys''. | The path points at a key that does not exist, so jq returns null rather than erroring. | Fix the path to the real key name, or use '?' / '// "default"' to handle absence. |
| Shell complains about ! $ ( ) or splits the program | Confirm the whole jq program is wrapped in single quotes. | The shell is expanding the unquoted filter before jq sees it. | Single-quote the program; pass dynamic values with --arg / --argjson, not string interpolation. |
| Strings come out wrapped in double quotes | Look at whether you need a raw value for a downstream shell command. | jq emits JSON-encoded strings by default, with quotes and escapes. | Add -r (--raw-output) to print the bare string contents. |
| @csv / @tsv errors with 'Cannot be csv-formatted' | Check the value immediately before @csv is an array. | @csv formats an array of scalars into one row; it rejects objects or bare scalars. | Wrap the fields in brackets first: '[.id, .name, .status] | @csv'. |
| jq hangs or eats all memory on a huge file | Check the file size and whether you really need the whole document in memory. | Default mode parses the entire input before filtering. | Use --stream for incremental parsing, or -c with per-line NDJSON input. |
| Program with newlines fails from the command line | Check whether the filter was quoted across lines or lost its pipes. | Long multi-line programs are awkward to pass inline. | Move the program into a file and run 'jq -f program.jq data.json'. |
References
- jq Manual (official)jqlang.github.io
- jq Tutorialjqlang.github.io
- jq Download & Installjqlang.github.io
- jqlang/jq on GitHubgithub.com
jq 1.7+ on a POSIX shell; examples single-quote the program so the shell does not expand it. API_HOST and other CAPS tokens are placeholders. @csv quotes strings and leaves numbers unquoted. The Claude example assumes a Messages API response whose top-level content is an array of blocks.