yamafaktory/jql

---

jql is a JSON Query Language tool built with Rust 🦀.

Pronounce it as jackal 🐺.

📖 Documentation and examples — the whole grammar, with every example generated by running jql itself.

📜 Philosophy

• ⚡Be fast
• 🪶 Stay lightweight
• 🎮 Keep its features as simple as possible
• 🧠 Avoid redundancy
• 💡 Provide meaningful error messages
• 🍰 Eat JSON as input, process, output JSON back

🚀 Installation

Alpine Linux

The package is maintained by @jirutka.

apk add jql

Archlinux

The AUR package is maintained by @barklan.

yay -S jql

Cargo

cargo install jql

Cargo Binstall

cargo binstall jql

Fedora

dnf install jql

FreeBSD

pkg install jql

Homebrew

brew install jql

Nix

nix-env -i jql

openSUSE

zypper install jql

Manual installation from GitHub

Compiled binary versions are automatically uploaded to GitHub when a new release is made. You can install jql manually by downloading a release.

🛠️ Usage

To make a selection from a JSON input, jql expects a query as a sequence of tokens.

To be fully compliant with the JSON format, jql always expect key selectors to be double-quoted, see The JavaScript Object Notation (JSON) Data Interchange Format.

{
  ".valid": 1337,
  "": "yeah!",
  "\"": "yup, valid too!"
}

Consequently, to be shell compliant, a query must be either enclosed by single quotation marks or every inner double quotation mark must be escaped.

The input may hold more than one JSON document — concatenated, separated by whitespace or newlines, each pretty-printed or not. The query is applied to every one of them and a result is written per document. Anything trailing that is not itself a document is an error rather than being ignored.

printf '{ "a": 1 }{ "a": 2 }' | jql '"a"'

1
2

Separators

Group separator

Group separators build up an array from sub-queries.

JSON input

{ "a": 1, "b": 2, "c": 3 }

Query

'"a","b","c"'

JSON output

[1, 2, 3]

Selectors

Arrays

Array index selector

Indexes can be used in arbitrary order.

JSON input

[1, 2, 3]

Query

'[2,1]'

JSON output

[3, 2]

Array range selector

Range can be in natural order [0:2], reversed [2:0], without lower [:2] or upper bound [0:].

JSON input

[1, 2, 3]

Query

'[2:1]'

JSON output

[3, 2]

Lens selector

Lens can be a combination of one or more selectors with or an optional value, a value being any of boolean | null | number | string.

JSON input

[
  { "a": 1, "b": { "d": 2 } },
  { "a": 2, "b": "some" },
  { "a": 2, "b": { "d": null } },
  { "a": 2, "b": true },
  { "c": 3, "b": 4 }
]

Query

'|={"b""d"=2, "c"}'

JSON output

[
  { "a": 1, "b": { "d": 2 } },
  { "c": 3, "b": 4 }
]

Objects

Key selector

Any valid JSON key can be used. A key carrying an escape sequence is written the way JSON writes it — \", \\, \/, \b, \f, \n, \r, \t and \uXXXX — so a key that contains a double quote or a backslash is reachable.

JSON input

{ "a": 1, "b": 2, "c": 3 }

Query

'"c"'

JSON output

3

JSON input

{ "a\"b": 1 }

Query

'"a\"b"'

JSON output

1

Multi key selector

Keys can be used in arbitrary order.

JSON input

{ "a": 1, "b": 2, "c": 3 }

Query

'{"c","a"}'

JSON output

{ "c": 3, "a": 1 }

Object index selector

Indexes can be used in arbitrary order.

JSON input

{ "a": 1, "b": 2, "c": 3 }

Query

'{2,0}'

JSON output

{ "c": 3, "a": 1 }

Object range selector

Range can be in natural order {0:2}, reversed {2:0}, without lower {:2} or upper bound {0:}.

JSON input

{ "a": 1, "b": 2, "c": 3 }

Query

'{2:1}'

JSON output

{ "c": 3, "b": 2 }

Operators

Flatten operator

Flattens arrays and objects.

JSON input

[[[[[[[[[[[[[[{ "a": 1 }]]]]]]]]]]]]], [[[[[{ "b": 2 }]]]], { "c": 3 }], null]

Query

'..'

JSON output

[{ "a": 1 }, { "b": 2 }, { "c": 3 }, null]

JSON input

{ "a": { "c": false }, "b": { "d": { "e": { "f": 1, "g": { "h": 2 } } } } }

Query

'..'

JSON output

{
  "a.c": false,
  "b.d.e.f": 1,
  "b.d.e.g.h": 2
}

Keys operator

Returns the keys of an object or the indices of an array. Other primitives are returned as is.

JSON input

{ "a": 1, "b": 2, "c": 3 }

Query

'@'

JSON output

["a", "b", "c"]

Pipe in operator

Applies the next tokens on each element of an array.

JSON input

{ "a": [{ "b": { "c": 1 } }, { "b": { "c": 2 } }] }

Query

'"a"|>"b""c"'

JSON output

[1, 2]

Pipe out operator

Stops the iteration initiated by the pipe in operator.

JSON input

{ "a": [{ "b": { "c": 1 } }, { "b": { "c": 2 } }] }

Query

'"a"|>"b""c"<|[1]'

JSON output

2

Truncate operator

Maps the output into simple JSON primitives boolean | null | number | string | [] | {}.

JSON input

{ "a": [1, 2, 3] }

Query

'"a"!'

JSON output

[]

💻 Shell integration

How to save the output

jql '"a"' input.json > output.json

How to read from stdin

cat test.json | jql '"a"'

Available flags

Inline the JSON output

By default, the output is pretty printed in a more human-readable way, this can be disabled.

-i, --inline

Read the query from file

The command will read the provided query from a file instead of the stdin.

-q, --query <FILE>

Write to stdout without JSON double-quotes

This can be useful to drop the double-quotes surrounding a string primitive.

-r, --raw-string

Sort the keys of every object in the JSON output

The keys of every object in the output are recursively sorted in alphanumerical order, matching the behavior of jq's -S flag.

-S, --sort-keys

Read a stream of JSON data line by line

Without this flag the whole input is read before anything is written, which is fine for a file or a finished pipe. This flag processes each line as it arrives instead, for output that is still being produced (e.g. Docker logs with the --follow flag). It expects one document per line; a document spread over several lines needs the default. This is not an option to read an incomplete streamed content (e.g. a very large input).

-s, --stream

Validate the JSON data

The command will return a matching exit code based on the validity of the JSON content or file provided.

-v, --validate

Print help

-h, --help

Print version

-V, --version

Help

jql -h
jql --help

🦀 Workspace

This project is composed of following crates:

• jql (binary)
• jql-parser (library)
• jql-runner (library)

Development

Some commands are available as a justfile at the root of the workspace (testing / fuzzing / changelog).

Prerequisites

• cargo-fuzz (fuzzing, needs a nightly toolchain)
• cargo-nextest
• git-cliff (changelog)
• just

Commands

just --list

⚠️ Non-goal

There's no plan to align jql with jq or any other similar tool.

⚡ Performance

Some benchmarks comparing a set of similar functionalities provided by this tool and jq are available here.

Selection queries — keys, indexes, ranges, multi keys, @, !, |= and a single |> — are evaluated against a simd-json tape, so only the part of the document a query actually selects is built. Input holding more than one document is evaluated a document at a time, on the tape as well. Queries using the flatten operator or nested pipes, and input that is deeply nested, are parsed in full instead.

Two JSON parsers are therefore in play, and they round the last bit of some numbers written in scientific notation differently, by up to two ULP. The tape returns the correctly rounded value; it is the fallback that is off, so a query answered on the tape is the more accurate of the two. A short mantissa is no protection — 9e75 reads as 9e75 on the tape and as 8.999999999999999e75 through the fallback. Plain decimals, integers, strings and every other value are unaffected.

📔 Licenses

• Apache License, Version 2.0
• MIT license