Expression reference

An expression computes one value from a record's fields. Use expressions in Eval, Code, Drop and If steps, step filters, and route conditions.

How it works

A name reads the field of that name on the record being processed, and operators and functions compute from those values. A method call and a function call are one call. Every value has a type, and the pipeline compiles only when every expression's types fit.

let host: str = "web-01" host.length() 6
let host: str = "web-01" length(host) 6

Types

A name's first write sets its type. A name's type is fixed for the whole pipeline. A declaration, name: T = e, states it, and e as T converts a value to another type.

Code

y = 1;
ratio: f64 = 3;
let label: str = "checkout";
service = label;

Input

{  "id": 7}

Output

{  "id": 7,  "y": 1,  "ratio": 3.0,  "service": "checkout"}

After that, three kinds of write treat a value of another type differently:

Write	What it converts	Example
`x = e`	Nothing. It accepts null, and an `i64` into an `f64` name.	`r: f64 = 1.5; r = 2;` stores `2.0`
`x: T = e`	A field a parser did not declare, record by record. It also changes the type of `x`.	`n: i64 = extra;` turns `"5"` into `5`
`e as T`	Any value.	`"5" as i64` is `5`; `2.7 as i64` is `3`

Code

ratio: f64 = 1.5;
ratio = 2;
count: i64 = extra;
rounded = 2.7 as i64;

Input

{  "extra": "5"}

Output

{  "extra": "5",  "ratio": 2.0,  "count": 5,  "rounded": 3}

After a JSON or Key/value parser, you can copy a field the parser does not declare as it is: copy = extra;. Its type is only known per record, so to calculate with it, give it a type first, as in extra: str = extra;, or convert the value: (extra as str) + "!".

Type	Holds	Example
`bool`	`true` or `false`.	`true` `true`
`i64`	A 64-bit signed integer. A result that does not fit is null, with an `Overflow` exception.	`443` `443`
`f64`	A 64-bit floating-point number. `/` on two numbers always gives one.	`1.5` `1.5`
`str`	Text, always valid UTF-8. Positions and lengths count characters.	`"GET"` `"GET"`
`bytes`	Bytes, not read as text. Text functions refuse them; `decode` reads them as text in a named encoding. They write as base64, and `toString` names another form.
`datetime`	An instant in UTC, to the microsecond, written as ISO 8601.	`"2024-05-01T12:00:00Z".parseTime()` `"2024-05-01T12:00:00.000000Z"`
`duration`	A signed span, to the microsecond, written as its count of microseconds.	`duration(90, "s")` `90000000`
`variant`	A value whose type is known only per row, such as a field of an open object that no declaration names.	`p = json("{\"port\": 443}")` `p.port` `443`
`array<T>`	Elements of one type, as many as each row has.	`[200, 404]` `[200,404]`
`{name: T, ...}`	Named fields. `{ip: str}` holds only the fields it names; `{ip: str, ...}` also keeps the others. See Objects.	`{ip: "10.0.0.1", port: 443}` `{"ip":"10.0.0.1","port":443}`

Any other mismatch is an error when you save, and the error names the fix:

x: i64 = "5"; does not compile. Write "5" as i64.
n: i64 = 1; n = 2.5; does not compile. Write 2.5 as i64, orn: f64 = 2.5.

An expression whose types do not fit on any row does not compile.

Does not compile
`"10" * 2`	Text and a number have no common type. Convert first: `"10".parseInt() * 2`.

Null

A declared field a record lacks reads as null, and a name no step declares or writes does not compile. Arithmetic on null gives null. A condition reads null as false: a ternary takes the value after :, an If step its else branch, and a step whose filter is null leaves the record unchanged. ?? gives a fallback for null. Writing null to a field removes it.

let host: str = null host.length() null
let host: str = null host ?? "unknown" "unknown"
let message: str = null !message.startsWith("debug") true // a negated test is true when the field is null
let message: str = null message != null && !message.startsWith("debug") false

Fields

Every record has these fields. A source's own fields are on its source page, and the fields a destination reads are on its destination page. Smeltr never delivers a field at the record's root whose name starts with __.

Field	Type	Set by	Holds
`_raw`	`str`	source	The text the source received, decoded with the source's `encoding`. A parser reads it. The destination delivers it with the record unless a step sets it to `null`. A source whose `encoding` is `none` gives no `_raw`.
`__source.name`	`str`	source	The name of the source that read the record.
`__source.type`	`str`	source	The kind of source that read it, its `type`, such as `kafka`, `s3List`, `httpApi` or `hecReceiver`.
`smeltr.exception`	`struct`	Smeltr	The exceptions of a record answered tag or deliver: `route`, `pipeline`, and `exceptions`, a list of each one's `step`, `kind` and `message`.

Naming a field

A bare name is letters, digits, _ and $, dotted. source.ip is one column. A name is the same name wherever it stands: read, written, or passed. $ is the record itself, an object: $.source.ip and source.ip are one read. A field whose name holds any other character is written $["..."], so $["source.ip"] is a different field: one key with a dot in it.

$["..."] takes a string literal and names a field, not a value: $["x-y"].z is the one field x-y.z, as a.b is. $[k] with k a value reads whichever field k names on that row, as any object's o[k] does.

On a named field, a fixed string key follows the same path: geo["city"] reads geo.city. A declared field a record lacks reads as null.

`.`	the field of that dotted name, or a child of an object	`geo = {"city": "Oslo", "lat": 59}` `geo.city` `"Oslo"` `source.ip = "10.0.0.1"` `source.ip.startsWith("10.")` `true` // one flat column
`$[]`	a field of the record by its name, for one a bare spelling cannot hold	`$["user-agent"] = "curl"` `$["user-agent"].toUpperCase()` `"CURL"`

Does not compile
`@timestamp`	`@` cannot start a bare name. Write `$["@timestamp"]`.

Objects

The record is an object: $ is the whole record, and each field is one of its keys. An object is written {"a": 1, "b": 2}. o[k] reads the key that k holds on each row.

{ip: str, port: i64} is closed: converting a value to it with as keeps the fields it names and drops the rest. {ip: str, ...} is open and keeps them.

Code

closed = obj as {ip: str};
open = obj as {ip: str, ...};
text = obj as {ip: str, port: str};

Input

{  "obj": {    "ip": "a",    "port": 1  }}

Output

{  "obj": {    "ip": "a",    "port": 1  },  "closed": {    "ip": "a"  },  "open": {    "ip": "a",    "port": 1  },  "text": {    "ip": "a",    "port": "1"  }}

Writing a field by name adds it to either kind. A computed key, such as o[key], names a field only the record knows, so it writes only to an open object: one declared with ..., or one you build without declaring its type. A computed key written to a closed object is an error when you save.

o: {ip: str} = {ip: "a"}; o[key] = 1; does not compile. Write o: {...} = o as {...}.

`object.keys()`

The object's keys, as an array of strings, in the order the record writes them. A field holding null is left out.

{"a": 1, "b": 2}.keys() ["a","b"]

`object.entries()`

Each key with its value, as {key, value}, in the order of keys. Every value must be of one type.

{"a": 1, "b": 2}.entries() [{"key":"a","value":1},{"key":"b","value":2}]

`fromEntries(pairs)`

An object of an array of {key, value}. A later key replaces an earlier one, and a null key is left out.

fromEntries([{"key": "a", "value": 1}, {"key": "a", "value": 2}]) {"a":2}

`object.omit(key, ...)`

The object without those keys.

{"a": 1, "b": 2, "c": 3}.omit("a", "c") {"b":2}

Statements

A statement ends with a semicolon, and a block in braces holds statements. The language has these statements:

`name = e;`	Writes the value of `e` to `name`, a field of the record or a local. `e` must have `name`'s type, an `i64` widens to an `f64`, and `null` fits any type. Writing `null` to a field removes it.
`name: T = e;`	Gives `name` the type `T` and writes `e`. This also converts the type declaration to `T`.
`name: T;`	Declares the type of `name` in advance. Use before parse functions to coerce types and improve performance.
`let name = e;`	Binds a local variable scoped to this block. It never reaches the record, and it takes the type of `e`. `let name: T = e;` gives it a type.
`$ = e;`	Replaces the whole record with the object `e`.
`Object.assign(o, e);`	Writes each field of the object `e` into the object `o`, converting each value to the type of its field in `o`. `Object.assign($, e)` writes into the record.
`if (c) { … } else { … }`	Runs the first block where `c` is true, and the `else` block where it is false or null. `else if` chains.
`switch (e) { case v: … default: … }`	Compares `e` with each `case` value, in order, with `==`. Runs from the first that matches through every case after it, until a `break`. `default` matches when no case does, and comes last.
`for (let x of a) { … }`	Runs the block once per element of the array `a`, with `x` that element.
`while (c) { … }`	Runs the block while `c` is true. The code step has a time budget, 100 ms by default and configured by `SMELTR_LOOP_BUDGET_MS`, to prevent infinite loop stalls. When it runs out, a loop still running stops and marks each record whose condition is still true with a `LoopBudgetExceeded` exception.
`break;`	Leaves the innermost loop or `switch`.
`continue;`	Ends this trip of the innermost loop and starts the next.
`function f(p: T): R { … }`	A local function. A parameter or result with no type takes its value's type. The body's last expression, written without a semicolon, is its value.
`return e;`	Ends the function with the value `e`.
`try { … } catch (e) { … } finally { … }`	Runs the block until a record's first exception that loses a value. The write that failed keeps the name's old value. A `catch` then runs on that record, with the exception cleared; `e.kind` is its kind and `e.message` its text. `finally` runs on every record that entered the `try`. A `try` needs a `catch`, a `finally` or both.
`catch (e: K1 \| K2) { … }`	Runs on the records holding one of the kinds it names, and clears only those. Several catches are tried in order; a catch with no type takes every kind left, and comes last.
`throw e;`	In a `catch`, puts the caught exception back on the record, for an outer `try` or its exception answer.
`delete(a, b.c);`	Removes each named field from the record. A field bound outside an `if`, loop or `try` cannot be deleted inside one; write `a = null` there.
`move(from, to);`	Renames the field `from` to `to`, and removes `from`.
`drop();`	Discards the record.
`unroll(a);`	Replaces the record with one record per element of the array field `a`, each holding its element at `a`. `unroll(a, true)` keeps the record's other fields on each. It stands outside every `if`, loop and `try`.

Parse and lookup

The parse functions read text into an object, as a Parser step does, and lookup reads a lookup table, as a Lookup step does. A call stands where a statement writes its object: Object.assign($, f(…)), $ = f(…), x = f(…) and x: T = f(…), or a let. Smeltr reads everything but the text and the key when the pipeline compiles, so you write it as literals. A parse function converts a field that a declared type names to it; every other field keeps the type the format gives, which is str for all but JSON.

`json(text, {splitDots, formats})`

The object the JSON text holds. It reads lenient JSON too, such as a trailing comma, raw control characters and \xHH escapes.

Option	Holds	Default
`splitDots`	Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`.	`false`
`formats`	The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s.	any format the parser recognizes

`kv(text, {pairSep, kvSep, quote, escapes, nextKey, splitDots, formats})`

An object of the key=value pairs in text. Every value is a str.

Option	Holds	Default
`pairSep`	The character between one pair and the next.	a space
`kvSep`	The character between a key and its value.	`=`
`quote`	The character around a value that holds `pairSep`; `null` for none.	`"`
`escapes`	Whether a backslash escapes any character in any value, quoted or not: `\n`, `\r` and `\t` are control characters and any other `\x` is `x`. Without it, only `\"` and `\\` inside quotes are escapes.	`false`
`nextKey`	Whether an unquoted value runs to the `pairSep` before the next key: in `level=info debug x=1`, `level` is `info debug`. Off, a value ends at the first `pairSep` and `debug` is a bare key, `debug=true`.	`true`
`splitDots`	Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`.	`true`
`formats`	The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s.	any format the parser recognizes

`cef(text, {splitDots, formats})`

An object of the ArcSight CEF record in text: the seven header fields as version, deviceVendor, deviceProduct, deviceVersion, deviceEventClassId, name and severity, and each extension key as written. Text before CEF: is skipped. Every value is a str.

Option	Holds	Default
`splitDots`	Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`.	`false`
`formats`	The text format of each `datetime` field, by name: `{rt: "%b %d %Y %H:%M:%S"}`. The specifiers are `parseTime`'s.	any format the parser recognizes

`csv(text, [headers], {delimiter, quote, escape, squeeze, trim, splitDots, formats})`

An object of one CSV line's columns, named by headers in order. A null header skips its column. Every value is a str.

Option	Holds	Default
`delimiter`	The character between columns.	`,`
`quote`	The character that quotes a column, which may then hold the delimiter.	`"`
`escape`	The character that escapes a quote, or itself, inside a quoted column. `null` reads a doubled quote as one.	`null`
`squeeze`	Whether a run of delimiters is one, as aligned columns write.	`false`
`trim`	Whether spaces around each unquoted column are dropped.	`false`
`splitDots`	Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`.	`false`
`formats`	The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s.	any format the parser recognizes

`grok(text, [patterns], {patterns, splitDots, formats})`

An object of the fields the first matching pattern captures. Each %{NAME:field} captures field. Every value is a str.

Option	Holds	Default
`patterns`	Your own patterns by name, `{NAME: "regex"}`, which the patterns may use as `%{NAME}`.	none
`splitDots`	Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`.	`true`
`formats`	The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s.	any format the parser recognizes

`lookup(table, key, {match, column, first, all, fields, formats})`

An object of the fields read from the row of the lookup table table that key matches. A field with no match and no default is left out.

Option	Holds	Default
`match`	How `key` matches the key column: `"exact"`, `"cidr"` or `"regex"`.	`"exact"`
`column`	The key column. A MaxMind database has none.	required, except for a MaxMind database
`first`	`true`: the first matching row, in table order.	`false`: the most exact match
`all`	`true`: every matching row; each field is an array.	`false`
`fields`	Each output named for its column, `"column"`, or for its column and its default, `["column", "default"]`.	required
`formats`	The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s.	any format the parser recognizes

Comparison and logic

The operators that answer a question or pick a value rather than compute one. None belongs to a single type: comparison works across numbers, text, instants and spans, and ?? takes a value of any type.

Both operands of a comparison must have the same type, except that an i64 beside an f64 widens to f64. Ordering works on numbers, text, instants and spans. Text orders by Unicode code point, so it is case-sensitive: "Z" < "a" is true. bool and bytes take == and != only.

Comparisons always return a boolean. Equality is true when both values are null, and false when only one is null. != gives the opposite result. Ordering (<, <=, > and >=) is false if either value is null; null is not converted to zero.

!, && and || read null as false and always return a boolean. For example, !null is true, true && null is false, and false || null is false. Non-null operands must be booleans; numbers and strings are not converted to true or false.

?? tests null and nothing else, so 0 and "" pass through: "" ?? "fallback" is "". It binds looser than && and ||, so a ?? b && c is a ?? (b && c).

Every branch must compile. Branches whose types do not meet, such as a str and an i64, give a variant.

A variant, a value read from an open object {...} or made from one, has a type only per row. == null, != null, isNull and ?? compile on it, and so does a read by a str or i64 key, as v.k, v[k] or v[0]. A read gives a variant, null where the row's value holds nothing; a negative index counts from the end. Any other operator, condition or function argument needs a type first, as in n: i64 = v. A ternary whose branches are a variant and another type is a variant: each row keeps the value of the branch it took.

Null and an empty array are different. A conditional array value that selects null stays null: == null is true and ?? uses its fallback. The length of a null array is null; length([]) is 0.

`==`	equal	`404 == 404` `true` `null == null` `true` `null == 0` `false`
`!=`	not equal	`404 != 200` `true` `null != 0` `true` `null != null` `false`
`<`	less	`1.5 < 2` `true` `null < 1` `false`
`<=`	less or equal	`2 <= 2` `true` `null <= 1` `false`
`>`	greater	`duration(1, "s") > duration(999, "ms")` `true` `1 > null` `false`
`>=`	greater or equal	`2 >= 3` `false` `null >= null` `false`
`&&`	both / and	`"GET" == "GET" && 200 < 300` `true` `true && false` `false` `true && null` `false`
`\|\|`	either / or	`true \|\| false` `true` `false \|\| null` `false` `true \|\| null` `true`
`!`	not	`!true` `false` `!null` `true`
`??`	the left value unless it is null	`"nope".parseInt() ?? -1` `-1` throws `CoercionRefused` // the left is null, so the fallback stands in `(true ? null : [1, 2]) ?? [9]` `[9]` // a null array value uses the fallback `[] ?? [9]` `[]` // an empty array is already a value
`?:`	ternary; if true, then take the first, otherwise take the second	`200 < 300 ? "ok" : "bad"` `"ok"` `true ? 1 : 2.5` `1.0` // the branches widen to a common type `null ? "yes" : "no"` `"no"`

Does not compile
`"10" < 9`	Text and a number have no common type. Convert first: `"10".parseInt() < 9` is `false`.
`true > false`	`bool` has no ordering.
`0 \|\| 200 < 300`	There is no truthiness: `\|\|` takes `bool` operands.

`value.isNull()`

(any) → bool

True when the value is null or the field is absent.

A declared field a record lacks reads as null, so isNull(user.id) is how you test for absence.

"".isNull() false
"nope".parseInt().isNull() true throws CoercionRefused
"smeltr".at(99).isNull() true throws IndexOutOfRange
isNull("") false // the same call, written as a function

Conversion

Reading one type as another. A parse whose text does not convert is null on that row and throws a CoercionRefused exception on the record; it never returns an approximation. Blank text is null and throws nothing. Text that reads as an instant is read with parseTime.

`value.toString(form = "base64")`

(i64/f64/bool/str/bytes/datetime/duration) → str(bytes, str) → str

Renders a number, bool, datetime or duration as text, exactly as the JSON output would. Bytes render in the form named: "base64", "hex" (two lowercase digits a byte) or "binary" (eight 0s and 1s a byte, high bit first). A form it does not know is null and throws a CoercionRefused exception.

404.toString() "404"
"1998-08-30T12:00:00Z".parseTime().toString() "1998-08-30T12:00:00.000000Z"
duration(1, "s").toString() "1000000" // a span writes as its microsecond count
"café".encode("utf-8").toString() "Y2Fmw6k=" // bytes write as base64 where no form is named, as the JSON output writes them
"café".encode("utf-8").toString("hex") "636166c3a9"
toString(404) "404" // the same call, written as a function

`value.parseInt(radix = 10)`

(str) → i64(str, i64) → i64

Parses a whole string as an integer. Surrounding whitespace is allowed; trailing junk is not.

"42".parseInt() 42
" 42 ".parseInt() 42 // surrounding space is ignored
"12abc".parseInt() null throws CoercionRefused // not 12: the whole text must be a number
"0x1f".parseInt(16) 31 // an explicit radix, 2 to 36
parseInt("ff", 16) 255 // the same call, written as a function

`value.parseHex()`

(str) → i64

Parses a hexadecimal string, with or without a 0x prefix. An alias for parseInt(value, 16).

"0x1f".parseHex() 31
"ff".parseHex() 255 // the 0x is optional
"zz".parseHex() null throws CoercionRefused
parseHex("ff") 255 // the same call, written as a function

`value.parseFloat()`

(str) → f64

Parses a whole string as a floating-point number.

"3.5".parseFloat() 3.5
"1e3".parseFloat() 1000.0 // an f64 always writes a decimal point, so it cannot be read back as an integer
"3.5kb".parseFloat() null throws CoercionRefused
parseFloat("3.5") 3.5 // the same call, written as a function

`value.parseBytes(form = "base64")`

(str) → bytes(str, str) → bytes

Reads text back into the bytes toString wrote, in the form named: "base64" (padded or not), "hex" (either case) or "binary".

"Y2Fmw6k=".parseBytes().decode("utf-8") "café"
"636166c3a9".parseBytes("hex").toString() "Y2Fmw6k="
"not base64!".parseBytes() null throws CoercionRefused
parseBytes("Y2Fmw6k=").toString("hex") "636166c3a9" // the same call, written as a function

`bytes.decode(encoding)`

(bytes, str) → str

Reads bytes as text in the encoding named: a WHATWG label such as "utf-8", "utf-16le" or "latin1", or "auto", which detects as a source's encoding does. Bytes that do not decode become U+FFFD. An encoding it does not know is null and throws a CoercionRefused exception.

"6361 66e9".replaceAll(/ /, "").parseBytes("hex").decode("latin1") "café"
"Y2Fmw6k=".parseBytes().decode("auto") "café"
"Y2Fmw6k=".parseBytes().decode("klingon") null throws CoercionRefused
decode(parseBytes("Y2Fmw6k="), "utf-8") "café" // the same call, written as a function

`text.encode(encoding)`

(str, str) → bytes

The text's bytes in the encoding named, by the labels decode takes. "auto" names no encoding to write. A character the encoding cannot spell makes the result null and throws a CoercionRefused exception.

"café".encode("latin1").toString("hex") "636166e9"
"hi".encode("utf-16le").toString("hex") "68006900"
"日本".encode("latin1") null throws CoercionRefused
encode("café", "utf-8").toString("hex") "636166c3a9" // the same call, written as a function

Numbers

i64 and f64. An i64 beside an f64 widens to f64. An arithmetic i64 result never wraps. The bitwise operators take i64 only and work over all 64 bits.

An i64 result that does not fit, from +, -, *, negation, abs, idiv, round, floor, ceil or trunc, is null on that row and throws an Overflow exception on the record: 9223372036854775807 * 2 is null, not a wrapped negative.

A value divided by zero with /, % or idiv is null on that row and throws a DivideByZero exception on the record. A null divided by zero is null and throws nothing.

An f64 result that is not finite, such as 1e308 * 10, throws no exception. JSON has no spelling for it, so the output omits the field.

Bitwise operands are signed and already 64 bits wide, so there is no narrowing step and no unsigned shift. The shift count is taken modulo 64, and bits pushed off the end are gone: 1 << 65 is 2, and (1 << 62) << 2 is 0.

`+`	add numbers	`200 + 1` `201`
`-`	subtract numbers	`1000 - 1` `999`
`-`	negate	`-(41)` `-41`
`*`	multiply	`1.5 * 4` `6.0`
`/`	divide, always `f64`	`1500 / 1000` `1.5` `1 / 0` `null` throws `DivideByZero` // by zero is null, not infinity
`%`	remainder, signed like the left operand	`-5 % 3` `-2` `5 % -3` `2` // the sign comes from the left operand, not the right `5 % 0` `null` throws `DivideByZero` // a zero divisor is null, as dividing by zero is
`**`	power	`3 8` `6561.0` `2 3 2` `512.0` // right-associative, so `2 (3 ** 2)`
`&`	bits set in both	`0xff & 0x0f` `15` // `11111111 & 00001111` = `00001111`
`\|`	bits set in either	`6 \| 3` `7` // `110 \| 011` = `111`
`^`	bits set in exactly one	`6 ^ 3` `5` // `110 ^ 011` = `101`
`~`	every bit flipped, so `~n` is `-n - 1`	`~0` `-1` // `000…0` flipped is `111…1`, all 64 bits set `~6` `-7` // `000…0110` flipped is `111…1001`
`<<`	shift toward the high end	`1 << 8` `256`
`>>`	shift toward the low end, sign-preserving	`-8 >> 1` `-4` `256 >> 4` `16`

Does not compile
`1.5 & 1`	Bitwise operators take `i64` only.

`value.abs()`

(i64/f64) → the value's type

Absolute value, keeping the input's type.

(-3).abs() 3
abs(-2.5) 2.5 // the same call, written as a function

`min(a, b, ...)`

(i64/f64, i64/f64, ...) → i64 when every argument is, else f64

The smallest of its arguments: an i64 when every argument is one, else an f64.

min(200, 64) 64
status = 404 min(status, 500) 404
min(200, 64, 128) 64 // any number of arguments
min(3, 5.5) 3.0 // one f64 argument widens the result, so the answer is 3.0 and not 3
status = 404 status.min(500) 404 // the same call, written as a method

`max(a, b, ...)`

(i64/f64, i64/f64, ...) → i64 when every argument is, else f64

The largest of its arguments: an i64 when every argument is one, else an f64.

max(200, 64) 200
max(64, 200, 128) 200 // any number of arguments
max(3, 2.5) 3.0 // the integer wins, and the answer is still 3.0 and not 3
status = 404 status.max(500) 500 // the same call as max(status, 500), written as a method

`value.clamp(lo, hi)`

(i64/f64, i64/f64, i64/f64) → i64 when every argument is, else f64

The value pulled into lo..hi: lo when it is below, hi when it is above, itself in between. An f64 if any argument is one.

width = 150 width.clamp(0, 100) 100
width = -20 width.clamp(0, 100) 0
width = 64 width.clamp(0, 100) 64
(1).clamp(0.5, 2) 1.0 // one f64 bound widens the result even when the value is kept, so the answer is 1.0 and not 1
(1).clamp(10, 0) null // an inverted range satisfies nothing, so it is null rather than one of the ends
width = 150 clamp(width, 0, 100) 100 // the same call, written as a function

`value.round()`

(f64) → i64

The nearest integer. A tie rounds away from zero, as as i64 does.

2.4.round() 2
2.5.round() 3 // a tie rounds away from zero
-2.5.round() -3 // a literal binds the method before the minus: -(2.5.round())
(-2.5).round() -3 // parenthesized, the value itself rounds
round(2.5) 3 // the same call, written as a function

`value.floor()`

(f64) → i64

The largest integer no greater than the value.

2.9.floor() 2
floor(-2.1) -3 // down, away from zero

`value.ceil()`

(f64) → i64

The smallest integer no less than the value.

2.1.ceil() 3
ceil(-2.9) -2 // up, toward zero

`value.trunc()`

(f64) → i64

The fraction dropped, toward zero.

2.9.trunc() 2
trunc(-2.9) -2 // toward zero, which is where it differs from floor

`idiv(a, b)`

(i64/f64, i64/f64) → i64 when every argument is, else f64

Integer division, truncating toward zero: the pair of %.

idiv(7, 2) 3
idiv(-7, 2) -3 // toward zero, so idiv(a, b) * b + a % b is a

`value.sign()`

(i64/f64) → i64

-1, 0 or 1.

(-42).sign() -1
sign(0) 0
sign(0.5) 1

Sequences

A string and an array both have a length and positions, and the same functions reach into either. Positions count from zero and a negative one counts from the end; positions and lengths count Unicode characters, so a slice can never land mid-character.

For arrays and strings, x[i] is at(x, i), including negative positions and out-of-range results.

A position past either end reads null and throws an IndexOutOfRange exception on the record.

On a named field, a fixed string key is part of the field name: geo["city"] reads geo.city.

`[]`	the element of an array at that position; negative counts from the end	`fields = "a,b,c".split(",")` `fields[2]` `"c"` `fields = "a,b,c".split(",")` `fields[-1]` `"c"` // the last element `fields = "a,b,c".split(",")` `fields[5]` `null` throws `IndexOutOfRange` // past the end of an array is null
`[]`	the character of a string at that position	`method = "GET"` `method[0]` `"G"` `method = "GET"` `method[99]` `null` throws `IndexOutOfRange` // past the end of a string is null
`[]`	the child of an object under that name	`geo = {"city": "Oslo", "lat": 59}` `geo["city"]` `"Oslo"` // a name, not a position

`value.length()`

(str/bytes/array) → i64

Characters in a string, bytes in a bytes value, or elements in an array.

line = "disk full" line.length() 9
"🔥 alert".length() 7 // characters, not bytes and not UTF-16 units: an emoji counts once
line = "a,b,c" line.split(",").length() 3 // elements, for an array
line = "disk full" length(line) 9 // the same call, written as a function

`value.slice(start[, end])`

(str, i64) → str(str, i64, i64) → str(array of scalars, i64) → the array's type(array of scalars, i64, i64) → the array's type

The part of a string — or of an array — from start up to but not including end. A negative index counts from the end; end defaults to the length.

line = "error: disk full" line.slice(7) "disk full"
"1998-08-30".slice(0, 4) "1998" // end is exclusive
path = "/api/v2/things" path.slice(-6) "things" // a negative start counts from the end
"a,b,c,d".split(",").slice(1, 3) ["b","c"] // arrays slice the same way
line = "error: disk full" slice(line, 7) "disk full" // the same call, written as a function

`value.at(index)`

(str, i64) → str(array of scalars, i64) → the element type

One element of an array, one character of a string, or one field of a declared object.

For arrays and strings, value[index] is the same as value.at(index).

On a named field, payload["status"] resolves as payload.status. A declared field a record lacks reads as null. This named-path form differs from calling at on an object value.

An object value passed to at needs a declared field set and a literal key. An unknown key does not compile. For a declared array of objects, tags[0]["name"] reads a field of one element.

"smeltr".at(0) "s"
"smeltr".at(-1) "r" // a negative index counts from the end
"smeltr".at(99) null throws IndexOutOfRange // past either end of a string is null
line = "a,b,c" line.split(",")[-1] "c" // [...] is at, negative index included
at("smeltr", 0) "s" // the same call, written as a function
payload.status = 200 payload["status"] 200 // a fixed string key on a named field

`value.indexOf(search)`

(str, str) → i64(array of scalars, value) → i64

Where search first occurs in a string, or which element of an array equals it. -1 when it does not.

line = "a,b,c" line.indexOf(",") 1
"GET /health".indexOf("POST") -1 // -1, not null
line = "a,b,c" line.split(",").indexOf("c") 2
line = "a,b,c" indexOf(line, ",") 1 // the same call, written as a function

`value.includes(search)`

(str, str) → bool(array of scalars, value) → bool

Whether a string contains search, or an array contains an element equal to it.

msg = "connection reset by peer" msg.includes("reset") true
"GET,HEAD".split(",").includes("HEAD") true // arrays too
msg = "connection reset by peer" includes(msg, "reset") true // the same call, written as a function

Strings

A str is text, always valid UTF-8. + joins two. length, slice, at, indexOf and includes take a string or an array.

There is no implicit stringification: a number has to go through toString() before it can be joined to text.

+

join text

"GET" + " /health" "GET /health"

Does not compile
`"a" + 1`	No implicit stringification. Write `"a" + (1).toString()`.

`value.lastIndexOf(search)`

(str, str) → i64

Where search last occurs, or -1. Useful for splitting on the final separator.

"/api/v2/things".lastIndexOf("/") 7
"a.b.c".lastIndexOf(".") 3
lastIndexOf("/api/v2/things", "/") 7 // the same call, written as a function

`value.startsWith(prefix)`

(str, str) → bool

Whether the string begins with prefix.

path = "/api/v2/things" path.startsWith("/api/") true
"GET /health".startsWith("POST") false
path = "/api/v2/things" startsWith(path, "/api/") true // the same call, written as a function

`value.endsWith(suffix)`

(str, str) → bool

Whether the string ends with suffix.

"app.log.gz".endsWith(".gz") true
endsWith("app.log.gz", ".gz") true // the same call, written as a function

`value.toUpperCase()`

(str) → str

Uppercased. Unicode-aware, so a character can expand.

level = "warn" level.toUpperCase() "WARN"
"straße".toUpperCase() "STRASSE" // Unicode-aware, so one character can become two
level = "warn" toUpperCase(level) "WARN" // the same call, written as a function

`value.toLowerCase()`

(str) → str

Lowercased, Unicode-aware. Lowercase both sides to compare them without regard to case.

"WARN".toLowerCase() "warn"
"Get".toLowerCase() == "get" true
toLowerCase("WARN") "warn" // the same call, written as a function

`value.trim()`

(str) → str

Whitespace removed from both ends.

raw = " disk full " raw.trim() "disk full"
"\ttabbed\n".trim() "tabbed" // every kind of whitespace, not just spaces
raw = " disk full " trim(raw) "disk full" // the same call, written as a function

`value.trimStart()`

(str) → str

Whitespace removed from the start only.

" indented".trimStart() "indented"
trimStart(" indented") "indented" // the same call, written as a function

`value.trimEnd()`

(str) → str

Whitespace removed from the end only.

"trailing ".trimEnd() "trailing"
trimEnd("trailing ") "trailing" // the same call, written as a function

`value.padStart(length, pad = " ")`

(str, i64) → str(str, i64, str) → str

Pads the front with pad — a space by default — until the string is length characters. Already long enough is returned unchanged.

"7".padStart(3, "0") "007"
"5".padStart(3) " 5" // the default pad is a space
id = 42 id.toString().padStart(5, "0") "00042"
"toolong".padStart(3, "0") "toolong" // never truncates
"x".padStart(2000000000) null // a result over 16 MiB is null and throws nothing
padStart("7", 3, "0") "007" // the same call, written as a function

`value.padEnd(length, pad = " ")`

(str, i64) → str(str, i64, str) → str

Pads the end with pad, a space by default, until the string is length characters. Already long enough is returned unchanged.

"id".padEnd(5, ".") "id..."
"hi".padEnd(4) "hi " // the default pad is a space
padEnd("id", 5, ".") "id..." // the same call, written as a function

`value.repeat(count)`

(str, i64) → str

The string repeated count times.

"ab".repeat(3) "ababab"
"x".repeat(0) ""
"x".repeat(-1) null // a negative count, or a result over 16 MiB, is null
repeat("ab", 3) "ababab" // the same call, written as a function

`value.md5()`

(str/bytes) → str

The MD5 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex. A fingerprint for spotting repeats, never a security check.

"abc".md5() "900150983cd24fb0d6963f7d28e17f72"

`value.sha1()`

(str/bytes) → str

The SHA-1 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex.

"abc".sha1() "a9993e364706816aba3e25717850c26c9cd0d89d"

`value.sha256()`

(str/bytes) → str

The SHA-256 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex.

"abc".sha256() "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"

`value.sha512()`

(str/bytes) → str

The SHA-512 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex.

"abc".sha512().slice(0, 16) "ddaf35a193617aba" // the first 16 of 128 digits

Regular expressions

A pattern is written as a literal, /[0-9]+/, and compiled once, when the pipeline is compiled, not per row. The flags are i to ignore case, m to let ^ and $ match at each line, and s to let . match a newline.

`value.test(/pattern/)`

(str, /pattern/) → bool

Whether the pattern matches anywhere in the value, as a bool. Use it in a Drop condition or a route.

"GET /health".test(/^GET/) true
"id=abc".test(/^id=[0-9]+$/) false
"Error: nope".test(/error/i) true
test("GET /health", /^GET/) true // the same call, written as a function

`value.match(/pattern/)`

(str, /pattern/) → array(str)

The whole match followed by each capture group, as an array — or null when nothing matches.

Element 0 is the whole match, so a capture group is at 1.

"2026-08-30".match(/([0-9]+)-([0-9]+)-([0-9]+)/) ["2026-08-30","2026","08","30"]
"v=17".match(/v=([0-9]+)/).at(1) "17"
"nothing here".match(/[0-9]+/) null
match("2026-08-30", /([0-9]+)-([0-9]+)-([0-9]+)/) ["2026-08-30","2026","08","30"] // the same call, written as a function

`value.replaceFirst(/pattern/, replacement)`

(str, /pattern/, str) → str

The value with the first match replaced. $1, $<name> and $& in the replacement refer back to the match.

"a1b2".replaceFirst(/[0-9]/, "#") "a#b2" // the first match only
"2026-08-30".replaceFirst(/([0-9]+)-([0-9]+)-([0-9]+)/, "$3/$2/$1") "30/08/2026" // capture groups, reordered
replaceFirst("user=alice", /user=(.+)/, "user=***") "user=***" // the same call, written as a function

`value.replaceAll(/pattern/, replacement)`

(str, /pattern/, str) → str

The value with every match replaced.

"a1b2".replaceAll(/[0-9]/, "#") "a#b#" // every match, where replaceFirst takes one
"user=alice".replaceAll(/user=(.+)/, "user=***") "user=***"
replaceAll("a1b2", /[0-9]/, "#") "a#b#" // the same call, written as a function

Arrays

An array<T> holds elements of the one type T. filter keeps the elements whose condition is true and map computes a value for each element. length, slice, at, indexOf and includes take an array or a string.

+ joins two arrays of the same element type. It never appends a single element, since xs + y would be ambiguous when y is itself an array; push appends.

+

join two arrays

line = "a,b"more = "c,d" line.split(",") + more.split(",") ["a","b","c","d"]

Does not compile
`["a", "b"] + "c"`	`+` joins two arrays; use `push` to append an element.

`array.map(item => expression)`

A new array with one computed value for each element. The source array is unchanged.

It compiles in an Eval or Code step. In a route condition, a delivery condition or a capture filter it does not compile.

The arrow takes one parameter. Its body is one expression, or a block whose last expression is the value. An arrow may call local functions and hold another arrow.

[1, 2, 3].map(x => x * 2) [2,4,6]
map([1, 2, 3], x => x * 2) [2,4,6] // the same call in function form

`array.filter(item => condition)`

A new array of the elements whose condition is true. It filters elements inside a record; a Drop step discards the whole record.

It compiles in an Eval or Code step. In a route condition, a delivery condition or a capture filter it does not compile.

The arrow takes one parameter. Its body is one expression, or a block whose last expression is the value. An arrow may call local functions and hold another arrow.

A null condition counts as false. When no element matches, the result is an empty array.

[1, 2, 3, 4].filter(x => x > 2) [3,4]
[1, 2].filter(x => x > 9) []
filter([1, 2, 3], x => x > 1).map(x => x * 2) [4,6] // filter and then transform the kept elements

`value.split(delimiter)`

(str, str) → array(str)(str, /pattern/) → array(str)

Splits a string into an array on a literal delimiter or a regex. An empty delimiter splits into characters.

"GET /health HTTP/1.1".split(" ")[2] "HTTP/1.1" // index the array with [...] -- one field out of a delimited line
line = "a,b,c" line.split(",") ["a","b","c"]
"a1b22c".split(/[0-9]+/) ["a","b","c"] // a regex separator splits on runs
"abc".split("") ["a","b","c"]
line = "GET /health HTTP/1.1" split(line, " ")[2] "HTTP/1.1" // the same call, written as a function

`array.join(separator)`

(array of scalars, str) → str

The elements as one string, separated by separator.

line = "a,b,c" line.split(",").join(" | ") "a | b | c"
"a,b".split(",").join("") "ab"
line = "a,b,c" join(line.split(","), " | ") "a | b | c" // the same call, written as a function

`array.push(value, ...)`

(array, value, ...) → the array's type

The array with every value appended. A null value is a position, as it is in ["a", null].

line = "a,b" line.split(",").push("c") ["a","b","c"]
x = [] as array<str> x.push("a") ["a"] // [] has no element type of its own, so the name that holds it declares one
line = "a,b" line.split(",").push("c", "d", "e") ["a","b","c","d","e"] // every value in one call -- a loop that pushes rebuilds the whole array each time round
line = "a,b" line.split(",").push(parseInt("x").toString()) ["a","b",null] throws CoercionRefused // a null value is a position, so every value keeps its place
line = "a,b" line.split(",").push("c").join(" | ") "a | b | c" // the new array comes back, so it goes straight into the next call
line = "a,b" push(line.split(","), "c") ["a","b","c"] // the same call, written as a function

`array.insert(index, value, ...)`

(array, i64, value, ...) → the array's type

The array with every value inserted before index. push is this with the position at the end.

An index past either end clamps. A null index makes the result null. A null value takes a position.

line = "a,b,d" line.split(",").insert(2, "c") ["a","b","c","d"]
line = "a,b" line.split(",").insert(-1, "z") ["a","z","b"] // -1 is the last element, so the value lands before it -- to append, use push
line = "a,b" line.split(",").insert(1, "x", "y") ["a","x","y","b"]
line = "a,b" line.split(",").insert(1, parseInt("x").toString()) ["a",null,"b"] throws CoercionRefused // a null value is a position here too
line = "a,b,d" insert(line.split(","), 2, "c") ["a","b","c","d"] // the same call, written as a function

`array.remove(index)`

(array of scalars, i64) → the array's type

The array without the element at index. The elements after it move up one.

A null array or index makes the result null.

line = "a,b,c" line.split(",").remove(1) ["a","c"]
line = "a,b,c" line.split(",").remove(-1) ["a","b"] // -1 is the last element, as it is for at
line = "a,b,c" line.split(",").remove(5) ["a","b","c"] // an index outside the array removes nothing

`array.resize(size)`

(array of scalars, i64) → the array's type

The array with exactly size positions: its first size, or its own followed by positions that hold nothing.

A negative or null size, or a null array, makes the result null.

line = "a,b,c" line.split(",").resize(2) ["a","b"]
line = "a,b,c" line.split(",").resize(4) ["a","b","c",null] // a new position holds nothing until a write fills it
resize([] as array<str>, 2) [null,null] // sized before any data arrives; each position holds nothing until a write fills it

`array.concat(other, ...)`

(array of scalars, array of scalars, ...) → the array's type

One array with every element of all of them, in order. a + b is the same call.

line = "a,b" line.split(",").concat(["c", "d"]) ["a","b","c","d"]
line = "a,b" line.split(",").concat(["c"], ["d", "e"]) ["a","b","c","d","e"] // any number of arrays, in one pass
line = "a,b" concat(line.split(","), ["c", "d"]) ["a","b","c","d"] // the same call, written as a function

`array.sort(descending)`

(array of scalars) → the array's type(array of scalars, bool) → the array's type

The elements in order: text by its bytes, numbers by value, false before true. descending is optional. Equal elements keep their order and nulls go last.

["b", "c", "a"].sort() ["a","b","c"]
[3, 1, 2].sort(true) [3,2,1] // descending
[2, null, 1].sort() [1,2,null] // nulls last

Dates and times

A datetime is an instant in UTC, microsecond-resolution, and serializes as ISO 8601. now() is read once per record, so every route a record takes sees the same instant. There is no local time zone: a log line's own offset is read when it is parsed and kept nowhere else.

Comparison orders instants: start < end is true when start is earlier.

`+`	advance an instant	`"1998-08-30".parseTime() + duration(90, "s")` `"1998-08-30T00:01:30.000000Z"` `now() + duration(1, "h") > now()` `true` // `now()` is read once per record, so both of these are the same instant
`-`	wind an instant back	`"1998-08-30".parseTime() - duration(1, "d")` `"1998-08-29T00:00:00.000000Z"`
`-`	the span between two instants	`"1998-08-31".parseTime() - "1998-08-30".parseTime() == duration(1, "d")` `true`

Does not compile
`"1998-08-30T12:00:00Z".parseTime() + "1998-08-30T12:00:00Z".parseTime()`	Two instants have no sum. Subtract them for a span.

The clock

`now()`

() → datetime

The current instant. Captured once per record, so every route a record takes sees the same value.

now().year() >= 2026 true

Reading text

`value.parseTime([format][, reference = now()])`

(str) → datetime(str, str) → datetime(str, datetime) → datetime(str, str, datetime) → datetime

Parses text into a datetime. With no format, a run of digits is a compact date when it reads as one (YYYYMMDD or YYYYMMDDHHMMSS), else a Unix epoch whose unit comes from its magnitude — seconds below 10^11, then milliseconds, microseconds and nanoseconds at each further thousand — and other text is matched against the common shapes — ISO 8601, YYYY-MM-DD HH:MM:SS, YYYY/MM/DD HH:MM:SS, syslog's Mon D HH:MM:SS, and Apache's [D/Mon/YYYY:HH:MM:SS +ZZZZ]. Give a strftime format to pin it, and a reference instant to fix the year a syslog stamp does not carry. A parser's declared datetime field is this call: the bare type detects, and {"type":"datetime","format":...} pins.

The reference is a datetime, not a year — usually the field carrying when the record was collected. Pass none and it is now(), which is what makes the no-argument form right for a live stream.

A yearless stamp takes the reference's year, and rolls back a year rather than landing in the future: parsed against 1 January 1999, Dec 31 is 1998. That also means an anchor *earlier* than the stamp reads as the year before it.

Text that does not parse is null on that row and throws a CoercionRefused exception on the record. An impossible date is such text, never a silently shifted one: 1998-02-30 does not become 2 March.

A zone name is not one offset: CST is six hours behind UTC in Chicago and eight ahead in Shanghai, and one pipeline reads both. %z takes the fixed set the table names. Replace any other name with the offset that source means — replaceAll(/CEST/, "+0200") — and the choice sits beside the source it belongs to.

Reading what it finds

"1998-08-30T12:00:00Z".parseTime() "1998-08-30T12:00:00.000000Z"
"1998-08-30 12:34:56".parseTime().hour() 12
"904478400".parseTime() "1998-08-30T12:00:00.000000Z" // ten digits of seconds; the same instant in milliseconds, 904478400000, reads the same
"904478400000".parseTime() == "904478400".parseTime() true
"19980830".parseTime() "1998-08-30T00:00:00.000000Z" // eight digits that read as a date are that date, not a count of seconds

Pinning the shape

"30/08/1998".parseTime("%d/%m/%Y") "1998-08-30T00:00:00.000000Z"
"2026-08-30 12:00:00 CEST".parseTime("%Y-%m-%d %H:%M:%S %z") null throws CoercionRefused // a zone name outside the set %z takes is null, never a guess
"2026-08-30 12:00:00 CEST".replaceAll(/CEST/, "+0200").parseTime("%Y-%m-%d %H:%M:%S %z") "2026-08-30T10:00:00.000000Z" // give the name the offset it means for this source, then parse
"1000".parseTime("%Q") "1970-01-01T00:00:01.000000Z" // auto-detect reads four digits as seconds; %Q says milliseconds and %s says seconds

Yearless stamps

"Feb 3 04:05:06".parseTime() == "Feb 3 04:05:06".parseTime(now()) true // no reference means now(), which is where a yearless stamp gets its year
"02-03 04:05:06".parseTime("%m-%d %H:%M:%S") "2026-02-03T04:05:06.000000Z" // a shape auto-detect does not know; the year comes from now()
"Dec 31 23:59:00".parseTime("1999-01-01T00:05:00Z".parseTime()) "1998-12-31T23:59:00.000000Z" // a December line read five minutes into January is last year's, not next year's

parseTime("1998-08-30T12:00:00Z") "1998-08-30T12:00:00.000000Z" // the same call, written as a function

parseTime(value, format) takes these, and only these. Every other byte of the format must match the text literally, and text left over when the format runs out makes the result null rather than a partial read. Auto-detect tries an ordered list of formats built from the same specifiers, so every rule below holds for parseTime(value) too.

Format specifiers for `parseTime`
The date
`%Y`	The year, exactly four digits. There is no two-digit year: every century pivot is wrong for someone.	`"2026-08-30".parseTime("%Y-%m-%d")` `"2026-08-30T00:00:00.000000Z"`
`%m`	Month number, `1`-`12`, with or without a leading zero.	`"8/2026".parseTime("%m/%Y")` `"2026-08-01T00:00:00.000000Z"`
`%d`	Day of month, `1`-`31`, with or without a leading zero. A day past the month's end is null.	`"2026-02-30".parseTime("%Y-%m-%d")` `null` throws `CoercionRefused` // February has no 30th
`%e`	Day of month with an optional leading space, as in `Aug 3`.	`"Aug 3 2026".parseTime("%b %e %Y")` `"2026-08-03T00:00:00.000000Z"`
The time of day
`%H`	Hour on a 24-hour clock, `0`-`23`. `24` is ISO 8601's end-of-day and is allowed only at a zero minute, second and fraction.	`"2026-08-30 24:00:00".parseTime("%Y-%m-%d %H:%M:%S")` `"2026-08-31T00:00:00.000000Z"` // the midnight that closes the date
`%I`	Hour on a 12-hour clock, `1`-`12`. Pair it with `%p`.	`"2026-08-30 02:00 PM".parseTime("%Y-%m-%d %I:%M %p")` `"2026-08-30T14:00:00.000000Z"`
`%M`	Minute, `0`-`59`, two digits.	`"2026-08-30 00:45".parseTime("%Y-%m-%d %H:%M")` `"2026-08-30T00:45:00.000000Z"`
`%S`	Second, `0`-`60`, two digits. `60` is a leap second and lands on the following minute.	`"2016-12-31 23:59:60".parseTime("%Y-%m-%d %H:%M:%S")` `"2017-01-01T00:00:00.000000Z"`
`%f`	Fractional seconds. Every digit is read and the first six kept, so a nanosecond stamp is truncated rather than refused; a shorter run is a position, not a count.	`"2026-08-30 00:00:00.123456789".parseTime("%Y-%m-%d %H:%M:%S.%f")` `"2026-08-30T00:00:00.123456Z"`
The offset
`%z`	A UTC offset — `Z`, `+HH`, `+HHMM` or `+HH:MM` — or one of the named zones `UT`, `UTC`, `GMT`, `EST`, `EDT`, `CST`, `CDT`, `MST`, `MDT`, `PST`, `PDT`. Any other name is null. Without `%z` the text reads as UTC.	`"2026-08-30T00:00:00+05".parseTime("%Y-%m-%dT%H:%M:%S%z")` `"2026-08-29T19:00:00.000000Z"` // A whole-hour offset without its minutes
Names and AM/PM
`%b`	An English month name, abbreviated or full. The same specifier as `%B`.	`"30 August 2026".parseTime("%d %b %Y")` `"2026-08-30T00:00:00.000000Z"`
`%B`	An English month name, abbreviated or full. The same specifier as `%b`.	`"30 Aug 2026".parseTime("%d %B %Y")` `"2026-08-30T00:00:00.000000Z"`
`%a`	An English weekday name, abbreviated or full. Consumed and discarded — it is never checked against the date.	`"Sun, 30 Aug 2026".parseTime("%a, %d %b %Y")` `"2026-08-30T00:00:00.000000Z"`
`%A`	An English weekday name, abbreviated or full. The same specifier as `%a`.	`"Sunday 30 Aug 2026".parseTime("%A %d %b %Y")` `"2026-08-30T00:00:00.000000Z"`
`%p`	`AM` or `PM`, either case. Reads `%I`; `12 AM` is midnight and `12 PM` is noon.	`"2026-08-30 12:00 AM".parseTime("%Y-%m-%d %I:%M %p")` `"2026-08-30T00:00:00.000000Z"`
`%s`	Seconds since the Unix epoch, with an optional sign and fraction. It names the whole instant, so the format holds no other specifier.	`"1000.5".parseTime("%s")` `"1970-01-01T00:16:40.500000Z"` // auto-detect would read a count this small as seconds too; the specifier says so outright
`%Q`	Milliseconds since the Unix epoch, with an optional sign and fraction. It names the whole instant, so the format holds no other specifier.	`"1000".parseTime("%Q")` `"1970-01-01T00:00:01.000000Z"` // auto-detect reads a count this small as seconds
`%%`	A literal percent sign.	`"2026 100%".parseTime("%Y 100%%")` `"2026-01-01T00:00:00.000000Z"`

`value.parseDuration()`

(str) → duration

Parses text into a duration: the count of microseconds toString() writes, whole or as a decimal rounded to the microsecond. Text that is no count is null.

"90000000".parseDuration().countIn("s") 90.0
duration(250, "ms").toString().parseDuration() == duration(250, "ms") true // what toString() writes reads back as the same span
"soon".parseDuration() null throws CoercionRefused

A part of the instant

`value.year()`

(datetime) → i64

The calendar year.

"1998-08-30".parseTime().year() 1998

`value.month()`

(datetime) → i64

The month, 1 to 12.

"1998-08-30".parseTime().month() 8

`value.day()`

(datetime) → i64

The day of the month.

"1998-08-30".parseTime().day() 30

`value.hour()`

(datetime) → i64

The hour, 0 to 23, in UTC.

"1998-08-30 12:34:56".parseTime().hour() 12

`value.minute()`

(datetime) → i64

The minute, 0 to 59.

"1998-08-30 12:34:56".parseTime().minute() 34

`value.second()`

(datetime) → i64

The second, 0 to 59.

"1998-08-30 12:34:56".parseTime().second() 56

`value.millisecond()`

(datetime) → i64

The millisecond within the second, 0 to 999.

"1998-08-30T12:00:00.250Z".parseTime().millisecond() 250

`value.dayOfWeek()`

(datetime) → i64

The day of the week: 1 is Monday, 7 is Sunday.

"1998-08-31".parseTime().dayOfWeek() 1

Epoch counts

`value.epochMillis()`

(datetime) → i64

Milliseconds since the Unix epoch.

"1998-08-30 12:00:00".parseTime().epochMillis() 904478400000

`epochSeconds(value)`

(datetime) → i64

Whole seconds since the Unix epoch, flooring so a time before it counts backward rather than toward zero.

epochSeconds(parseTime("1998-08-30T12:00:00Z")) 904478400

`fromEpochMillis(millis)`

(i64) → datetime

A datetime from milliseconds since the Unix epoch, for a producer that already sends one.

fromEpochMillis(904478400000) "1998-08-30T12:00:00.000000Z"

`fromEpochSeconds(seconds)`

(i64) → datetime

A datetime from seconds since the Unix epoch. A count past the datetime range is null.

fromEpochSeconds(904478400) "1998-08-30T12:00:00.000000Z"

Durations

A duration is a signed span, microsecond-resolution. Subtracting two datetimes produces one. It serializes as a bare count of microseconds, toString() produces the same text, and parseDuration() reads it back.

Comparison orders spans: age > duration(5, "m") is true when age is longer than five minutes.

`+`	add two spans	`duration(30, "m") + duration(45, "s") == duration(1845, "s")` `true`
`-`	the difference of two spans	`duration(1, "h") - duration(30, "m") == duration(30, "m")` `true`
`*`	scale a span	`duration(90, "s") * 2 == duration(3, "m")` `true`
`/`	the ratio of two spans	`duration(1, "h") / duration(1, "m")` `60.0`
`/`	divide a span	`duration(1, "h") / 4 == duration(15, "m")` `true`

Does not compile
`duration(1, "y")`	A year varies in length, so a span cannot be counted in one. Days are fixed: `duration(365, "d")`.

`duration(count, unit)`

(i64/f64, str) → duration

A span of count units. The unit is "ns", "us", "ms", "s", "m", "h", "d", "w" or a long name, and the count may be fractional.

A span is held to the microsecond, so anything finer truncates toward zero: duration(1500, "ns") is one microsecond and duration(1, "ns") is a zero span. A fractional count of any unit rounds the same way.

There is no month or year: how long either one is depends on which one, and a span is a count with no calendar attached to answer that. A week is seven days, so it is here.

A literal unit is checked when the pipeline compiles: duration(1, "y") does not compile. A unit computed per row that names no unit makes the result null. A span too large to hold is null.

duration(90, "s").toString() "90000000"
duration(250, "ms").toString() "250000"
duration(1.5, "d").toString() "129600000000"
duration(1000, "ns").toString() "1" // a thousand nanoseconds is one microsecond, exactly
duration(2, "w") == duration(14, "d") true // a week is exactly seven days
duration(90, "s") > duration(1, "minute") true

`duration.countIn(unit)`

(duration, str) → f64

The whole span counted in one unit, as an f64, not a remainder.

duration(90, "s").countIn("m") 1.5
duration(36, "h").countIn("d") 1.5
duration(1, "d").countIn("h") 24.0
duration(1, "s").countIn("us") 1000000.0 // the unit the wire form counts in
duration(14, "d").countIn("w") 2.0
started = "1998-08-30 12:00:00".parseTime()ended = "1998-08-30 12:30:00".parseTime() (ended - started).countIn("m") 30.0
countIn(duration(90, "s"), "m") 1.5 // the same call, written as a function

Addresses and networks

An address is text. A function parses it and writes it back as text. Text that is not an address makes the result null and throws no exception. Space around an address is ignored. To match an address against a table of blocks, use a lookup.

`text.isIp()`

(str) → bool

The text is an address, v4 or v6. Text that is not one is false. Null stays null.

"10.1.2.3".isIp() true
"fe80::1".isIp() true
"10.1.2".isIp() false
isIp("10.1.2.3") true // the same call, written as a function

`text.ipVersion()`

(str) → i64

4 or 6.

"10.1.2.3".ipVersion() 4
"::1".ipVersion() 6
"example.com".ipVersion() null

`text.ipNormalize()`

(str) → str

The address in one spelling. Leading zeros go. IPv6 is lowercase and its longest run of zero groups is compressed. Two spellings of one address come out the same.

"2001:0DB8:0000:0000:0000:0000:0000:0001".ipNormalize() "2001:db8::1"
" 10.1.2.3 ".ipNormalize() "10.1.2.3"
"010.1.2.3".ipNormalize() null // a dotted quad takes no leading zero

`text.ipScope()`

(str) → str

Where the address reaches: unspecified, loopback, link-local, multicast, private or public. Private is RFC 1918 and fc00::/7.

"10.1.2.3".ipScope() "private"
"8.8.8.8".ipScope() "public"
"127.0.0.1".ipScope() "loopback"
"fe80::1".ipScope() "link-local"
"224.0.0.1".ipScope() "multicast"
"::".ipScope() "unspecified"
"fd12::1".ipScope() "private"

`address.inCidr(block, ...)`

(str, str, ...) → bool

The address is in one of the blocks. A block of the other family holds nothing. A bare address is a block of one.

"10.1.2.3".inCidr("10.0.0.0/8") true
"10.1.2.3".inCidr("192.168.0.0/16", "10.0.0.0/8") true // any number of blocks
"10.1.2.3".inCidr("::/0") false // the other family
"10.1.2.3".inCidr("10.1.2.3") true // a bare address is a block of one
"10.1.2.3".inCidr("10.0.0.0/33") null // a block that does not parse

`address.cidrNetwork(prefix)`

(str, i64) → str

The address masked to prefix bits, written as a block. Group by it to count per subnet.

"10.1.2.3".cidrNetwork(24) "10.1.2.0/24"
"10.1.2.3".cidrNetwork(12) "10.0.0.0/12"
"2001:db8:abcd::1".cidrNetwork(32) "2001:db8::/32"
"10.1.2.3".cidrNetwork(33) null // past the width of the address

`communityId(srcIp, srcPort, dstIp, dstPort, transport, seed)`

(str, i64, str, i64, str) → str(str, i64, str, i64, i64) → str(str, i64, str, i64, str, i64) → str(str, i64, str, i64, i64, i64) → str

The flow's Community ID, version 1. transport is an IANA protocol number, 0 to 255, as a number or as text, or one of these names in any case: icmp, igmp, tcp, udp, gre, icmpv6 (or ipv6-icmp), eigrp, ospf, pim, sctp. Any other name is null. TCP, UDP and SCTP need both ports, 1 to 65535; for ICMP and ICMPv6 the ports are the type and the code; any other protocol has no ports. Both addresses must be IPv4 or both IPv6. seed is optional, 0 to 65535, 0 by default. Both directions of a flow give one ID.

communityId("128.232.110.120", 34855, "66.35.250.204", 80, "tcp") "1:LQU9qZlK+B5F3KDmev6m5PMibrg="
communityId("66.35.250.204", 80, "128.232.110.120", 34855, 6) "1:LQU9qZlK+B5F3KDmev6m5PMibrg=" // the reply, with the protocol's number
communityId("128.232.110.120", 34855, "66.35.250.204", 80, "tcp", 123) "1:hTSGlFQnR58UCk+NfKRZzA32dPg=" // a seed
communityId("128.232.110.120", 0, "66.35.250.204", 80, "tcp") null // TCP needs both ports
communityId("10.0.0.1", 5, "10.0.0.2", 6, "quic") null // not a protocol name it knows

URLs, domains and user agents

Text that does not read is null and throws a CoercionRefused exception on the record; blank text is null and throws nothing.

`text.urlDecode()`

(str) → str

%XX as the byte it names and + as a space, as a form field is encoded, and the bytes read as UTF-8; a byte that is not UTF-8 becomes U+FFFD. A % that two hex digits do not follow stays a %, so every text reads.

"a%20b+c%C3%A9".urlDecode() "a b cé"
"100%".urlDecode() "100%" // a bare % stays

`text.htmlStrip()`

(str) → str

The text without its markup. A tag goes, in any case. A block-level tag, start or end, becomes a line break: address, article, aside, blockquote, br, dd, div, dl, dt, fieldset, figcaption, figure, footer, form, h1 to h6, header, hgroup, hr, li, nav, noscript, ol, output, p, pre, section, table, tbody, td, tfoot, th, thead, tr and ul. A comment, a <!...> or <?...> declaration, and a <script> or <style> element go with what they hold; a <![CDATA[...]]> section leaves what it holds. A character reference becomes its character: a number, é or é, or any of HTML 4's names, such as &,   or ö. A reference that does not read, and a < that starts no tag, stay as they are. Text without both < and > is returned as it is.

"<b>bold</b> & plain".htmlStrip() "bold & plain"
"<p>café — ok</p>".htmlStrip() "\ncafé — ok\n" // named and numeric references
"one<br>two".htmlStrip() "one\ntwo" // a block-level tag breaks the line

`text.parseUrl()`

(str) → {domain: str, fragment: str, path: str, extension: str, port: i64, query: str, scheme: str, user_info: str, username: str, password: str}

A URL's parts as an object: domain, fragment, path, extension, port, query, scheme, user_info, username and password. Text that reads as an RFC 3986 URI or relative reference, such as a bare path, has its path, query, fragment and user info decoded. An http, https, ftp, file or jar URL that does not, holding a space or a | say, is split as written, undecoded. extension is what follows the last . of the path's last segment. username is the user info up to its first :, all of it where there is none, and password what follows that :. A part the URL lacks is null.

"https://www.example.com:8443/files/report.pdf?x=1#top".parseUrl() {"domain":"www.example.com","fragment":"top","path":"/files/report.pdf","extension":"pdf","port":8443,"query":"x=1","scheme":"https"}
"http://user:pw@host/a%20b".parseUrl().path "/a b" // a part read on its own
"/search/index.html?q=x".parseUrl() {"path": "/search/index.html", "extension": "html", "query": "q=x"} // a path alone
"not a url".parseUrl() null throws CoercionRefused

`host.registeredDomain()`

(str) → {domain: str, registered_domain: str, top_level_domain: str, subdomain: str}

The host, the name registered under its public suffix, that suffix and what is left, as an object of domain, registered_domain, top_level_domain and subdomain. The suffixes are the ICANN section of the Public Suffix List, compiled in. A host with no known suffix does not read.

"www.foo.example.co.uk".registeredDomain() {"domain":"www.foo.example.co.uk","registered_domain":"example.co.uk","top_level_domain":"co.uk","subdomain":"www.foo"}
"co.uk".registeredDomain() {"domain":"co.uk","top_level_domain":"co.uk"} // a suffix alone
"intranet.local".registeredDomain() null throws CoercionRefused // no known suffix

`text.parseUserAgent()`

(str) → {name: str, version: str, os: {name: str, version: str, full: str}, device: {name: str}}

The browser, operating system and device a User-Agent header names, as an object of name, version, os.name, os.version, os.full and device.name. The rules are uap-core's, compiled in. name and device.name are Other where no rule matches, and os is null where none names an operating system. os.full is os.name, then os.version where there is one.

"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36".parseUserAgent() {"name":"Chrome","version":"120.0.6099.109","os":{"name":"Windows","version":"10","full":"Windows 10"},"device":{"name":"Other"}}
"curl/8.4.0".parseUserAgent().name "curl" // one part

Exceptions and try

A calculation that cannot give a value leaves its field null and marks the record with an exception: text that is not a number, a division by zero, an overflow. Each exception has a kind, such as CoercionRefused or DivideByZero. A conversion that fails, such as "five" as i64, gives null and marks the record with a CoercionRefused exception. A parser's declared field that the parser cannot read as its type also gives null, and marks the record with an UnparsableField exception. An exception no catch takes stays on the record to the end of the pipeline, where its exception answers apply.

A function's entry says when it throws one. Each kind belongs to a category. The group's exception answers answer a kind or its category, and a route may override them. By default an encoding exception is ignored, since its value is kept, a delivery exception blocks, and every other kind is tagged.

10 / 0 null throws DivideByZero
"many".parseInt() null throws CoercionRefused
9223372036854775807 + 1 null throws Overflow

Kind	Category	When
`Unparsable`	parse	A parser got nothing from the record: malformed JSON, an unterminated CSV quote, a Grok pattern that matched nothing.
`UnparsableField`	parse	One declared field could not be read as its declared type while the rest of the record parsed. That field is null.
`KeyContradiction`	parse	Two keys of a record contradict after flattening, such as the same key twice. One is kept.
`CoercionRefused`	parse	A value could not convert to the type asked for, as in `parseInt("many")`. The result is null.
`Overflow`	math	`i64` arithmetic overflowed. The result is null.
`DivideByZero`	math	A division by zero. The result is null, not Infinity.
`NaN`	math	A parse met `NaN`, `Infinity` or a number past `f64`'s range. That key has no value.
`LoopBudgetExceeded`	limits	A `while` loop hit its time budget with the record's condition still true.
`IndexOutOfRange`	limits	An array read or write, or a string read, named a position the value does not have. The read is null and the write does nothing.
`InvalidUtf8`	encoding	Bytes that did not decode under the source's encoding were replaced with U+FFFD.
`GuessedEncoding`	encoding	A source set to `auto` recognized neither UTF-8 nor UTF-16 and guessed the encoding. The text may well be right.
`Unmatched`	delivery	No route on the record's source took it.
`Unprocessable`	delivery	No pipeline could finish the record: one crashed on its batch, or the record does not fit the memory budget alone.
`Refused`	delivery	A destination refused the record.

A try block stops at a record's first exception that loses a value. The write that failed does not happen, so the field keeps what it held. Its catch then runs on that record, with the exception cleared. catch (e) names the exception: e.kind is its kind and e.message its text. When one statement leaves several, e.kind is the first and e.message holds all their texts, and the catch clears them all. A finally block runs on every record that entered the try. A try needs a catch, a finally or both.

Code

try {
  count = parseInt(text);
} catch (e) {
  count = 0;
  problem = e.kind;
  detail = e.message;
} finally {
  checked = true;
}

Input

{  "text": "many"}

Output

{  "checked": true,  "count": 0,  "detail": "text is \"many\", which could not convert to i64 in count",  "problem": "CoercionRefused",  "text": "many"}

A typed catch, such as catch (e: DivideByZero | NaN), runs on the records holding a kind it names and clears only those kinds. A try can have several catches, and it tries them in order. A catch with no type takes every kind left, and comes last. A kind no catch names stays on the record, for an outer try or its exception answer. throw e in a catch puts the exception back on the record, which goes on to the next try out, or to its exception answer when there is none. A catch reads e and cannot change it. A try catches every kind but the encoding and delivery ones. An encoding exception repairs its value instead, so the block goes on and no catch takes it.

Expression reference

How it works

Types

Null

Fields

Statements

Parse and lookup

json(text, {splitDots, formats})

kv(text, {pairSep, kvSep, quote, escapes, nextKey, splitDots, formats})

cef(text, {splitDots, formats})

csv(text, [headers], {delimiter, quote, escape, squeeze, trim, splitDots, formats})

grok(text, [patterns], {patterns, splitDots, formats})

lookup(table, key, {match, column, first, all, fields, formats})

Exceptions and try

See also

`json(text, {splitDots, formats})`

`kv(text, {pairSep, kvSep, quote, escapes, nextKey, splitDots, formats})`

`cef(text, {splitDots, formats})`

`csv(text, [headers], {delimiter, quote, escape, squeeze, trim, splitDots, formats})`

`grok(text, [patterns], {patterns, splitDots, formats})`

`lookup(table, key, {match, column, first, all, fields, formats})`