# Expression reference

An expression computes one value from a record's fields. Use expressions in Eval, Code, Drop and If steps, step filters, and route conditions.

## How it works

A name reads the field of that name on the record being processed, and operators and functions compute from those values. A method call and a function call are one call. Every value has a type, and the pipeline compiles only when every expression's types fit.

-   `let host: str = "web-01"`: `host.length()` → `6`
-   `let host: str = "web-01"`: `length(host)` → `6`

### Types

A name's first write sets its type. A name's type is fixed for the whole pipeline. A declaration, `name: T = e`, states it, and `e as T` converts a value to another type.

Code

```
y = 1;
ratio: f64 = 3;
let label: str = "checkout";
service = label;
```

Input

```
{
  "id": 7
}
```

→

Output

```
{
  "id": 7,
  "y": 1,
  "ratio": 3.0,
  "service": "checkout"
}
```

After that, three kinds of write treat a value of another type differently:

| Write | What it converts | Example |
| --- | --- | --- |
| `x = e` | Nothing. It accepts null, and an `i64` into an `f64` name. | `r: f64 = 1.5; r = 2;` stores `2.0` |
| `x: T = e` | A field a parser did not declare, record by record. It also changes the type of `x`. | `n: i64 = extra;` turns `"5"` into `5` |
| `e as T` | Any value. | `"5" as i64` is `5`; `2.7 as i64` is `3` |

Code

```
ratio: f64 = 1.5;
ratio = 2;
count: i64 = extra;
rounded = 2.7 as i64;
```

Input

```
{
  "extra": "5"
}
```

→

Output

```
{
  "extra": "5",
  "ratio": 2.0,
  "count": 5,
  "rounded": 3
}
```

After a JSON or Key/value parser, you can copy a field the parser does not declare as it is: `copy = extra;`. Its type is only known per record, so to calculate with it, give it a type first, as in `extra: str = extra;`, or convert the value: `(extra as str) + "!"`.

| Type | Holds | Example |
| --- | --- | --- |
| `bool` | `true` or `false`. | `true` → `true` |
| `i64` | A 64-bit signed integer. A result that does not fit is null, with an `Overflow` exception. | `443` → `443` |
| `f64` | A 64-bit floating-point number. `/` on two numbers always gives one. | `1.5` → `1.5` |
| `str` | Text, always valid UTF-8. Positions and lengths count characters. | `"GET"` → `"GET"` |
| `bytes` | Bytes, not read as text. Text functions refuse them; `decode` reads them as text in a named encoding. They write as base64, and `toString` names another form. |  |
| `datetime` | An instant in UTC, to the microsecond, written as ISO 8601. | `"2024-05-01T12:00:00Z".parseTime()` → `"2024-05-01T12:00:00.000000Z"` |
| `duration` | A signed span, to the microsecond, written as its count of microseconds. | `duration(90, "s")` → `90000000` |
| `variant` | A value whose type is known only per row, such as a field of an open object that no declaration names. | `p = json("{\"port\": 443}")`: `p.port` → `443` |
| `array<T>` | Elements of one type, as many as each row has. | `[200, 404]` → `[200,404]` |
| `{name: T, ...}` | Named fields. `{ip: str}` holds only the fields it names; `{ip: str, ...}` also keeps the others. See [Objects](#objects). | `{ip: "10.0.0.1", port: 443}` → `{"ip":"10.0.0.1","port":443}` |

Any other mismatch is an error when you save, and the error names the fix:

-   `x: i64 = "5";` does not compile. Write `"5" as i64`.
-   `n: i64 = 1; n = 2.5;` does not compile. Write `2.5 as i64`, or`n: f64 = 2.5`.

An expression whose types do not fit on any row does not compile.

Does not compile:

|  |  |
| --- | --- |
| `"10" * 2` | Text and a number have no common type. Convert first: `"10".parseInt() * 2`. |

### Null

A declared field a record lacks reads as null, and a name no step declares or writes does not compile. Arithmetic on null gives null. A condition reads null as false: a ternary takes the value after `:`, an If step its else branch, and a step whose filter is null leaves the record unchanged. `??` gives a fallback for null. Writing `null` to a field removes it.

-   `let host: str = null`: `host.length()` → `null`
-   `let host: str = null`: `host ?? "unknown"` → `"unknown"`
-   `let message: str = null`: `!message.startsWith("debug")` → `true` // a negated test is true when the field is null
-   `let message: str = null`: `message != null && !message.startsWith("debug")` → `false`

## Fields

Every record has these fields. A source's own fields are on its [source page](/docs/sources), and the fields a destination reads are on its [destination page](/docs/destinations). Smeltr never delivers a field at the record's root whose name starts with `__`.

| Field | Type | Set by | Holds |
| --- | --- | --- | --- |
| `_raw` | `str` | source | The text the source received, decoded with the source's `encoding`. A parser reads it. The destination delivers it with the record unless a step sets it to `null`. A source whose `encoding` is `none` gives no `_raw`. |
| `__source.name` | `str` | source | The name of the source that read the record. |
| `__source.type` | `str` | source | The kind of source that read it, its `type`, such as `kafka`, `s3List`, `httpApi` or `hecReceiver`. |
| `smeltr.exception` | `struct` | Smeltr | The exceptions of a record answered tag or deliver: `route`, `pipeline`, and `exceptions`, a list of each one's `step`, `kind` and `message`. |

## Naming a field

A bare name is letters, digits, `_` and `$`, dotted. `source.ip` is one column. A name is the same name wherever it stands: read, written, or passed. `$` is the record itself, an [object](#objects): `$.source.ip` and `source.ip` are one read. A field whose name holds any other character is written `$["..."]`, so `$["source.ip"]` is a different field: one key with a dot in it.

`$["..."]` takes a string literal and names a field, not a value: `$["x-y"].z` is the one field `x-y.z`, as `a.b` is. `$[k]` with `k` a value reads whichever field `k` names on that row, as any object's `o[k]` does.

On a named field, a fixed string key follows the same path: `geo["city"]` reads `geo.city`. A declared field a record lacks reads as null.

|  |  |  |
| --- | --- | --- |
| `.` | the field of that dotted name, or a child of an object | `geo = {"city": "Oslo", "lat": 59}`: `geo.city` → `"Oslo"`<br>`source.ip = "10.0.0.1"`: `source.ip.startsWith("10.")` → `true` // one flat column |
| `$[]` | a field of the record by its name, for one a bare spelling cannot hold | `$["user-agent"] = "curl"`: `$["user-agent"].toUpperCase()` → `"CURL"` |

Does not compile:

|  |  |
| --- | --- |
| `@timestamp` | `@` cannot start a bare name. Write `$["@timestamp"]`. |

## Objects

The record is an object: `$` is the whole record, and each field is one of its keys. An object is written `{"a": 1, "b": 2}`. `o[k]` reads the key that `k` holds on each row.

`{ip: str, port: i64}` is closed: converting a value to it with `as` keeps the fields it names and drops the rest. `{ip: str, ...}` is open and keeps them.

Code

```
closed = obj as {ip: str};
open = obj as {ip: str, ...};
text = obj as {ip: str, port: str};
```

Input

```
{
  "obj": {
    "ip": "a",
    "port": 1
  }
}
```

→

Output

```
{
  "obj": {
    "ip": "a",
    "port": 1
  },
  "closed": {
    "ip": "a"
  },
  "open": {
    "ip": "a",
    "port": 1
  },
  "text": {
    "ip": "a",
    "port": "1"
  }
}
```

Writing a field by name adds it to either kind. A computed key, such as `o[key]`, names a field only the record knows, so it writes only to an open object: one declared with `...`, or one you build without declaring its type. A computed key written to a closed object is an error when you save.

-   `o: {ip: str} = {ip: "a"}; o[key] = 1;` does not compile. Write `o: {...} = o as {...}`.

### `object.keys()`

The object's keys, as an array of strings, in the order the record writes them. A field holding null is left out.

-   `{"a": 1, "b": 2}.keys()` → `["a","b"]`

### `object.entries()`

Each key with its value, as `{key, value}`, in the order of `keys`. Every value must be of one type.

-   `{"a": 1, "b": 2}.entries()` → `[{"key":"a","value":1},{"key":"b","value":2}]`

### `fromEntries(pairs)`

An object of an array of `{key, value}`. A later key replaces an earlier one, and a null key is left out.

-   `fromEntries([{"key": "a", "value": 1}, {"key": "a", "value": 2}])` → `{"a":2}`

### `object.omit(key, ...)`

The object without those keys.

-   `{"a": 1, "b": 2, "c": 3}.omit("a", "c")` → `{"b":2}`

## Statements

A statement ends with a semicolon, and a block in braces holds statements. The language has these statements:

|  |  |
| --- | --- |
| `name = e;` | Writes the value of `e` to `name`, a field of the record or a local. `e` must have `name`'s type, an `i64` widens to an `f64`, and `null` fits any type. Writing `null` to a field removes it. |
| `name: T = e;` | Gives `name` the type `T` and writes `e`. This also converts the type declaration to `T`. |
| `name: T;` | Declares the type of `name` in advance. Use before parse functions to coerce types and improve performance. |
| `let name = e;` | Binds a local variable scoped to this block. It never reaches the record, and it takes the type of `e`. `let name: T = e;` gives it a type. |
| `$ = e;` | Replaces the whole record with the [object](#objects) `e`. |
| `Object.assign(o, e);` | Writes each field of the object `e` into the object `o`, converting each value to the type of its field in `o`. `Object.assign($, e)` writes into the record. |
| `if (c) { … } else { … }` | Runs the first block where `c` is true, and the `else` block where it is false or null. `else if` chains. |
| `switch (e) { case v: … default: … }` | Compares `e` with each `case` value, in order, with `==`. Runs from the first that matches through every case after it, until a `break`. `default` matches when no case does, and comes last. |
| `for (let x of a) { … }` | Runs the block once per element of the array `a`, with `x` that element. |
| `while (c) { … }` | Runs the block while `c` is true. The code step has a time budget, 100 ms by default and configured by `SMELTR_LOOP_BUDGET_MS`, to prevent infinite loop stalls. When it runs out, a loop still running stops and marks each record whose condition is still true with a `LoopBudgetExceeded` exception. |
| `break;` | Leaves the innermost loop or `switch`. |
| `continue;` | Ends this trip of the innermost loop and starts the next. |
| `function f(p: T): R { … }` | A local function. A parameter or result with no type takes its value's type. The body's last expression, written without a semicolon, is its value. |
| `return e;` | Ends the function with the value `e`. |
| `try { … } catch (e) { … } finally { … }` | Runs the block until a record's first exception that loses a value. The write that failed keeps the name's old value. A `catch` then runs on that record, with the exception cleared; `e.kind` is its kind and `e.message` its text. `finally` runs on every record that entered the `try`. A `try` needs a `catch`, a `finally` or both. |
| `catch (e: K1 \| K2) { … }` | Runs on the records holding one of the kinds it names, and clears only those. Several catches are tried in order; a catch with no type takes every kind left, and comes last. |
| `throw e;` | In a `catch`, puts the caught exception back on the record, for an outer `try` or its exception answer. |
| `delete(a, b.c);` | Removes each named field from the record. A field bound outside an `if`, loop or `try` cannot be deleted inside one; write `a = null` there. |
| `move(from, to);` | Renames the field `from` to `to`, and removes `from`. |
| `drop();` | Discards the record. |
| `unroll(a);` | Replaces the record with one record per element of the array field `a`, each holding its element at `a`. `unroll(a, true)` keeps the record's other fields on each. It stands outside every `if`, loop and `try`. |

## Parse and lookup

The parse functions read text into an object, as a Parser step does, and `lookup` reads a lookup table, as a Lookup step does. A call stands where a statement writes its object: `Object.assign($, f(…))`, `$ = f(…)`, `x = f(…)` and `x: T = f(…)`, or a `let`. Smeltr reads everything but the text and the key when the pipeline compiles, so you write it as literals. A parse function converts a field that a declared type names to it; every other field keeps the type the format gives, which is `str` for all but JSON.

### `json(text, {splitDots, formats})`

The object the JSON `text` holds. It reads lenient JSON too, such as a trailing comma, raw control characters and `\xHH` escapes.

| Option | Holds | Default |
| --- | --- | --- |
| `splitDots` | Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`. | `false` |
| `formats` | The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s. | any format the parser recognizes |

### `kv(text, {pairSep, kvSep, quote, escapes, nextKey, splitDots, formats})`

An object of the `key=value` pairs in `text`. Every value is a `str`.

| Option | Holds | Default |
| --- | --- | --- |
| `pairSep` | The character between one pair and the next. | a space |
| `kvSep` | The character between a key and its value. | `=` |
| `quote` | The character around a value that holds `pairSep`; `null` for none. | `"` |
| `escapes` | Whether a backslash escapes any character in any value, quoted or not: `\n`, `\r` and `\t` are control characters and any other `\x` is `x`. Without it, only `\"` and `\\` inside quotes are escapes. | `false` |
| `nextKey` | Whether an unquoted value runs to the `pairSep` before the next key: in `level=info debug x=1`, `level` is `info debug`. Off, a value ends at the first `pairSep` and `debug` is a bare key, `debug=true`. | `true` |
| `splitDots` | Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`. | `true` |
| `formats` | The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s. | any format the parser recognizes |

### `cef(text, {splitDots, formats})`

An object of the ArcSight CEF record in `text`: the seven header fields as `version`, `deviceVendor`, `deviceProduct`, `deviceVersion`, `deviceEventClassId`, `name` and `severity`, and each extension key as written. Text before `CEF:` is skipped. Every value is a `str`.

| Option | Holds | Default |
| --- | --- | --- |
| `splitDots` | Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`. | `false` |
| `formats` | The text format of each `datetime` field, by name: `{rt: "%b %d %Y %H:%M:%S"}`. The specifiers are `parseTime`'s. | any format the parser recognizes |

### `csv(text, [headers], {delimiter, quote, escape, squeeze, trim, splitDots, formats})`

An object of one CSV line's columns, named by `headers` in order. A `null` header skips its column. Every value is a `str`.

| Option | Holds | Default |
| --- | --- | --- |
| `delimiter` | The character between columns. | `,` |
| `quote` | The character that quotes a column, which may then hold the delimiter. | `"` |
| `escape` | The character that escapes a quote, or itself, inside a quoted column. `null` reads a doubled quote as one. | `null` |
| `squeeze` | Whether a run of delimiters is one, as aligned columns write. | `false` |
| `trim` | Whether spaces around each unquoted column are dropped. | `false` |
| `splitDots` | Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`. | `false` |
| `formats` | The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s. | any format the parser recognizes |

### `grok(text, [patterns], {patterns, splitDots, formats})`

An object of the fields the first matching pattern captures. Each `%{NAME:field}` captures `field`. Every value is a `str`.

| Option | Holds | Default |
| --- | --- | --- |
| `patterns` | Your own patterns by name, `{NAME: "regex"}`, which the patterns may use as `%{NAME}`. | none |
| `splitDots` | Whether a key holding dots is split into a nested path: `a.b` becomes `{a: {b}}`. | `true` |
| `formats` | The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s. | any format the parser recognizes |

### `lookup(table, key, {match, column, first, all, fields, formats})`

An object of the `fields` read from the row of the lookup table `table` that `key` matches. A field with no match and no default is left out.

| Option | Holds | Default |
| --- | --- | --- |
| `match` | How `key` matches the key column: `"exact"`, `"cidr"` or `"regex"`. | `"exact"` |
| `column` | The key column. A MaxMind database has none. | required, except for a MaxMind database |
| `first` | `true`: the first matching row, in table order. | `false`: the most exact match |
| `all` | `true`: every matching row; each field is an array. | `false` |
| `fields` | Each output named for its column, `"column"`, or for its column and its default, `["column", "default"]`. | required |
| `formats` | The text format of each `datetime` field, by name: `{ts: "%d/%m/%Y"}`. The specifiers are `parseTime`'s. | any format the parser recognizes |

## Comparison and logic

The operators that answer a question or pick a value rather than compute one. None belongs to a single type: comparison works across numbers, text, instants and spans, and `??` takes a value of any type.

Both operands of a comparison must have the same type, except that an `i64` beside an `f64` widens to `f64`. Ordering works on numbers, text, instants and spans. Text orders by Unicode code point, so it is case-sensitive: `"Z" < "a"` is `true`. `bool` and `bytes` take `==` and `!=` only.

Comparisons always return a boolean. Equality is true when both values are null, and false when only one is null. `!=` gives the opposite result. Ordering (`<`, `<=`, `>` and `>=`) is false if either value is null; null is not converted to zero.

`!`, `&&` and `||` read null as false and always return a boolean. For example, `!null` is `true`, `true && null` is `false`, and `false || null` is `false`. Non-null operands must be booleans; numbers and strings are not converted to true or false.

`??` tests null and nothing else, so `0` and `""` pass through: `"" ?? "fallback"` is `""`. It binds looser than `&&` and `||`, so `a ?? b && c` is `a ?? (b && c)`.

Every branch must compile. Branches whose types do not meet, such as a `str` and an `i64`, give a `variant`.

A `variant`, a value read from an open object `{...}` or made from one, has a type only per row. `== null`, `!= null`, `isNull` and `??` compile on it, and so does a read by a `str` or `i64` key, as `v.k`, `v[k]` or `v[0]`. A read gives a `variant`, null where the row's value holds nothing; a negative index counts from the end. Any other operator, condition or function argument needs a type first, as in `n: i64 = v`. A ternary whose branches are a `variant` and another type is a `variant`: each row keeps the value of the branch it took.

Null and an empty array are different. A conditional array value that selects null stays null: `== null` is true and `??` uses its fallback. The length of a null array is null; `length([])` is `0`.

|  |  |  |
| --- | --- | --- |
| `==` | equal | `404 == 404` → `true`<br>`null == null` → `true`<br>`null == 0` → `false` |
| `!=` | not equal | `404 != 200` → `true`<br>`null != 0` → `true`<br>`null != null` → `false` |
| `<` | less | `1.5 < 2` → `true`<br>`null < 1` → `false` |
| `<=` | less or equal | `2 <= 2` → `true`<br>`null <= 1` → `false` |
| `>` | greater | `duration(1, "s") > duration(999, "ms")` → `true`<br>`1 > null` → `false` |
| `>=` | greater or equal | `2 >= 3` → `false`<br>`null >= null` → `false` |
| `&&` | both / and | `"GET" == "GET" && 200 < 300` → `true`<br>`true && false` → `false`<br>`true && null` → `false` |
| `\|\|` | either / or | `true \|\| false` → `true`<br>`false \|\| null` → `false`<br>`true \|\| null` → `true` |
| `!` | not | `!true` → `false`<br>`!null` → `true` |
| `??` | the left value unless it is null | `"nope".parseInt() ?? -1` → `-1` throws `CoercionRefused` // the left is null, so the fallback stands in<br>`(true ? null : [1, 2]) ?? [9]` → `[9]` // a null array value uses the fallback<br>`[] ?? [9]` → `[]` // an empty array is already a value |
| `?:` | ternary; if true, then take the first, otherwise take the second | `200 < 300 ? "ok" : "bad"` → `"ok"`<br>`true ? 1 : 2.5` → `1.0` // the branches widen to a common type<br>`null ? "yes" : "no"` → `"no"` |

Does not compile:

|  |  |
| --- | --- |
| `"10" < 9` | Text and a number have no common type. Convert first: `"10".parseInt() < 9` is `false`. |
| `true > false` | `bool` has no ordering. |
| `0 \|\| 200 < 300` | There is no truthiness: `\|\|` takes `bool` operands. |

### `value.isNull()`

`(any)` → `bool`

True when the value is null or the field is absent.

A declared field a record lacks reads as null, so `isNull(user.id)` is how you test for absence.

-   `"".isNull()` → `false`
-   `"nope".parseInt().isNull()` → `true` throws `CoercionRefused`
-   `"smeltr".at(99).isNull()` → `true` throws `IndexOutOfRange`
-   `isNull("")` → `false` // the same call, written as a function

## Conversion

Reading one type as another. A parse whose text does not convert is null on that row and throws a `CoercionRefused` exception on the record; it never returns an approximation. Blank text is null and throws nothing. Text that reads as an instant is read with [parseTime](#parseTime).

### `value.toString(form = "base64")`

`(i64/f64/bool/str/bytes/datetime/duration)` → `str`; `(bytes, str)` → `str`

Renders a number, bool, datetime or duration as text, exactly as the JSON output would. Bytes render in the form named: `"base64"`, `"hex"` (two lowercase digits a byte) or `"binary"` (eight `0`s and `1`s a byte, high bit first). A form it does not know is null and throws a `CoercionRefused` exception.

-   `404.toString()` → `"404"`
-   `"1998-08-30T12:00:00Z".parseTime().toString()` → `"1998-08-30T12:00:00.000000Z"`
-   `duration(1, "s").toString()` → `"1000000"` // a span writes as its microsecond count
-   `"café".encode("utf-8").toString()` → `"Y2Fmw6k="` // bytes write as base64 where no form is named, as the JSON output writes them
-   `"café".encode("utf-8").toString("hex")` → `"636166c3a9"`
-   `toString(404)` → `"404"` // the same call, written as a function

### `value.parseInt(radix = 10)`

`(str)` → `i64`; `(str, i64)` → `i64`

Parses a whole string as an integer. Surrounding whitespace is allowed; trailing junk is not.

-   `"42".parseInt()` → `42`
-   `" 42 ".parseInt()` → `42` // surrounding space is ignored
-   `"12abc".parseInt()` → `null` throws `CoercionRefused` // not `12`: the whole text must be a number
-   `"0x1f".parseInt(16)` → `31` // an explicit radix, 2 to 36
-   `parseInt("ff", 16)` → `255` // the same call, written as a function

### `value.parseHex()`

`(str)` → `i64`

Parses a hexadecimal string, with or without a `0x` prefix. An alias for `parseInt(value, 16)`.

-   `"0x1f".parseHex()` → `31`
-   `"ff".parseHex()` → `255` // the `0x` is optional
-   `"zz".parseHex()` → `null` throws `CoercionRefused`
-   `parseHex("ff")` → `255` // the same call, written as a function

### `value.parseFloat()`

`(str)` → `f64`

Parses a whole string as a floating-point number.

-   `"3.5".parseFloat()` → `3.5`
-   `"1e3".parseFloat()` → `1000.0` // an `f64` always writes a decimal point, so it cannot be read back as an integer
-   `"3.5kb".parseFloat()` → `null` throws `CoercionRefused`
-   `parseFloat("3.5")` → `3.5` // the same call, written as a function

### `value.parseBytes(form = "base64")`

`(str)` → `bytes`; `(str, str)` → `bytes`

Reads text back into the bytes `toString` wrote, in the form named: `"base64"` (padded or not), `"hex"` (either case) or `"binary"`.

-   `"Y2Fmw6k=".parseBytes().decode("utf-8")` → `"café"`
-   `"636166c3a9".parseBytes("hex").toString()` → `"Y2Fmw6k="`
-   `"not base64!".parseBytes()` → `null` throws `CoercionRefused`
-   `parseBytes("Y2Fmw6k=").toString("hex")` → `"636166c3a9"` // the same call, written as a function

### `bytes.decode(encoding)`

`(bytes, str)` → `str`

Reads bytes as text in the encoding named: a WHATWG label such as `"utf-8"`, `"utf-16le"` or `"latin1"`, or `"auto"`, which detects as a source's `encoding` does. Bytes that do not decode become U+FFFD. An encoding it does not know is null and throws a `CoercionRefused` exception.

-   `"6361 66e9".replaceAll(/ /, "").parseBytes("hex").decode("latin1")` → `"café"`
-   `"Y2Fmw6k=".parseBytes().decode("auto")` → `"café"`
-   `"Y2Fmw6k=".parseBytes().decode("klingon")` → `null` throws `CoercionRefused`
-   `decode(parseBytes("Y2Fmw6k="), "utf-8")` → `"café"` // the same call, written as a function

### `text.encode(encoding)`

`(str, str)` → `bytes`

The text's bytes in the encoding named, by the labels `decode` takes. `"auto"` names no encoding to write. A character the encoding cannot spell makes the result null and throws a `CoercionRefused` exception.

-   `"café".encode("latin1").toString("hex")` → `"636166e9"`
-   `"hi".encode("utf-16le").toString("hex")` → `"68006900"`
-   `"日本".encode("latin1")` → `null` throws `CoercionRefused`
-   `encode("café", "utf-8").toString("hex")` → `"636166c3a9"` // the same call, written as a function

## Numbers

`i64` and `f64`. An `i64` beside an `f64` widens to `f64`. An arithmetic `i64` result never wraps. The bitwise operators take `i64` only and work over all 64 bits.

An `i64` result that does not fit, from `+`, `-`, `*`, negation, `abs`, `idiv`, `round`, `floor`, `ceil` or `trunc`, is null on that row and throws an `Overflow` exception on the record: `9223372036854775807 * 2` is null, not a wrapped negative.

A value divided by zero with `/`, `%` or `idiv` is null on that row and throws a `DivideByZero` exception on the record. A null divided by zero is null and throws nothing.

An `f64` result that is not finite, such as `1e308 * 10`, throws no exception. JSON has no spelling for it, so the output omits the field.

Bitwise operands are signed and already 64 bits wide, so there is no narrowing step and no unsigned shift. The shift count is taken modulo 64, and bits pushed off the end are gone: `1 << 65` is `2`, and `(1 << 62) << 2` is `0`.

|  |  |  |
| --- | --- | --- |
| `+` | add numbers | `200 + 1` → `201` |
| `-` | subtract numbers | `1000 - 1` → `999` |
| `-` | negate | `-(41)` → `-41` |
| `*` | multiply | `1.5 * 4` → `6.0` |
| `/` | divide, always `f64` | `1500 / 1000` → `1.5`<br>`1 / 0` → `null` throws `DivideByZero` // by zero is null, not infinity |
| `%` | remainder, signed like the left operand | `-5 % 3` → `-2`<br>`5 % -3` → `2` // the sign comes from the left operand, not the right<br>`5 % 0` → `null` throws `DivideByZero` // a zero divisor is null, as dividing by zero is |
| `**` | power | `3 ** 8` → `6561.0`<br>`2 ** 3 ** 2` → `512.0` // right-associative, so `2 ** (3 ** 2)` |
| `&` | bits set in both | `0xff & 0x0f` → `15` // `11111111 & 00001111` = `00001111` |
| `\|` | bits set in either | `6 \| 3` → `7` // `110 \| 011` = `111` |
| `^` | bits set in exactly one | `6 ^ 3` → `5` // `110 ^ 011` = `101` |
| `~` | every bit flipped, so `~n` is `-n - 1` | `~0` → `-1` // `000…0` flipped is `111…1`, all 64 bits set<br>`~6` → `-7` // `000…0110` flipped is `111…1001` |
| `<<` | shift toward the high end | `1 << 8` → `256` |
| `>>` | shift toward the low end, sign-preserving | `-8 >> 1` → `-4`<br>`256 >> 4` → `16` |

Does not compile:

|  |  |
| --- | --- |
| `1.5 & 1` | Bitwise operators take `i64` only. |

### `value.abs()`

`(i64/f64)` → `the value's type`

Absolute value, keeping the input's type.

-   `(-3).abs()` → `3`
-   `abs(-2.5)` → `2.5` // the same call, written as a function

### `min(a, b, ...)`

`(i64/f64, i64/f64, ...)` → `i64 when every argument is, else f64`

The smallest of its arguments: an `i64` when every argument is one, else an `f64`.

-   `min(200, 64)` → `64`
-   `status = 404`: `min(status, 500)` → `404`
-   `min(200, 64, 128)` → `64` // any number of arguments
-   `min(3, 5.5)` → `3.0` // one `f64` argument widens the result, so the answer is `3.0` and not `3`
-   `status = 404`: `status.min(500)` → `404` // the same call, written as a method

### `max(a, b, ...)`

`(i64/f64, i64/f64, ...)` → `i64 when every argument is, else f64`

The largest of its arguments: an `i64` when every argument is one, else an `f64`.

-   `max(200, 64)` → `200`
-   `max(64, 200, 128)` → `200` // any number of arguments
-   `max(3, 2.5)` → `3.0` // the integer wins, and the answer is still `3.0` and not `3`
-   `status = 404`: `status.max(500)` → `500` // the same call as `max(status, 500)`, written as a method

### `value.clamp(lo, hi)`

`(i64/f64, i64/f64, i64/f64)` → `i64 when every argument is, else f64`

The value pulled into `lo..hi`: `lo` when it is below, `hi` when it is above, itself in between. An `f64` if any argument is one.

-   `width = 150`: `width.clamp(0, 100)` → `100`
-   `width = -20`: `width.clamp(0, 100)` → `0`
-   `width = 64`: `width.clamp(0, 100)` → `64`
-   `(1).clamp(0.5, 2)` → `1.0` // one `f64` bound widens the result even when the value is kept, so the answer is `1.0` and not `1`
-   `(1).clamp(10, 0)` → `null` // an inverted range satisfies nothing, so it is null rather than one of the ends
-   `width = 150`: `clamp(width, 0, 100)` → `100` // the same call, written as a function

### `value.round()`

`(f64)` → `i64`

The nearest integer. A tie rounds away from zero, as `as i64` does.

-   `2.4.round()` → `2`
-   `2.5.round()` → `3` // a tie rounds away from zero
-   `-2.5.round()` → `-3` // a literal binds the method before the minus: `-(2.5.round())`
-   `(-2.5).round()` → `-3` // parenthesized, the value itself rounds
-   `round(2.5)` → `3` // the same call, written as a function

### `value.floor()`

`(f64)` → `i64`

The largest integer no greater than the value.

-   `2.9.floor()` → `2`
-   `floor(-2.1)` → `-3` // down, away from zero

### `value.ceil()`

`(f64)` → `i64`

The smallest integer no less than the value.

-   `2.1.ceil()` → `3`
-   `ceil(-2.9)` → `-2` // up, toward zero

### `value.trunc()`

`(f64)` → `i64`

The fraction dropped, toward zero.

-   `2.9.trunc()` → `2`
-   `trunc(-2.9)` → `-2` // toward zero, which is where it differs from `floor`

### `idiv(a, b)`

`(i64/f64, i64/f64)` → `i64 when every argument is, else f64`

Integer division, truncating toward zero: the pair of `%`.

-   `idiv(7, 2)` → `3`
-   `idiv(-7, 2)` → `-3` // toward zero, so `idiv(a, b) * b + a % b` is `a`

### `value.sign()`

`(i64/f64)` → `i64`

`-1`, `0` or `1`.

-   `(-42).sign()` → `-1`
-   `sign(0)` → `0`
-   `sign(0.5)` → `1`

## Sequences

A string and an array both have a length and positions, and the same functions reach into either. Positions count from zero and a negative one counts from the end; positions and lengths count Unicode characters, so a slice can never land mid-character.

For arrays and strings, `x[i]` is `at(x, i)`, including negative positions and out-of-range results.

A position past either end reads null and throws an `IndexOutOfRange` exception on the record.

On a named field, a fixed string key is part of the [field name](#names): `geo["city"]` reads `geo.city`.

|  |  |  |
| --- | --- | --- |
| `[]` | the element of an array at that position; negative counts from the end | `fields = "a,b,c".split(",")`: `fields[2]` → `"c"`<br>`fields = "a,b,c".split(",")`: `fields[-1]` → `"c"` // the last element<br>`fields = "a,b,c".split(",")`: `fields[5]` → `null` throws `IndexOutOfRange` // past the end of an array is null |
| `[]` | the character of a string at that position | `method = "GET"`: `method[0]` → `"G"`<br>`method = "GET"`: `method[99]` → `null` throws `IndexOutOfRange` // past the end of a string is null |
| `[]` | the child of an object under that name | `geo = {"city": "Oslo", "lat": 59}`: `geo["city"]` → `"Oslo"` // a name, not a position |

### `value.length()`

`(str/bytes/array)` → `i64`

Characters in a string, bytes in a `bytes` value, or elements in an array.

-   `line = "disk full"`: `line.length()` → `9`
-   `"🔥 alert".length()` → `7` // characters, not bytes and not UTF-16 units: an emoji counts once
-   `line = "a,b,c"`: `line.split(",").length()` → `3` // elements, for an array
-   `line = "disk full"`: `length(line)` → `9` // the same call, written as a function

### `value.slice(start[, end])`

`(str, i64)` → `str`; `(str, i64, i64)` → `str`; `(array of scalars, i64)` → `the array's type`; `(array of scalars, i64, i64)` → `the array's type`

The part of a string — or of an array — from `start` up to but not including `end`. A negative index counts from the end; `end` defaults to the length.

-   `line = "error: disk full"`: `line.slice(7)` → `"disk full"`
-   `"1998-08-30".slice(0, 4)` → `"1998"` // `end` is exclusive
-   `path = "/api/v2/things"`: `path.slice(-6)` → `"things"` // a negative start counts from the end
-   `"a,b,c,d".split(",").slice(1, 3)` → `["b","c"]` // arrays slice the same way
-   `line = "error: disk full"`: `slice(line, 7)` → `"disk full"` // the same call, written as a function

### `value.at(index)`

`(str, i64)` → `str`; `(array of scalars, i64)` → `the element type`

One element of an array, one character of a string, or one field of a declared object.

For arrays and strings, `value[index]` is the same as `value.at(index)`.

On a named field, `payload["status"]` resolves as `payload.status`. A declared field a record lacks reads as null. This named-path form differs from calling `at` on an object value.

An object value passed to `at` needs a declared field set and a literal key. An unknown key does not compile. For a declared array of objects, `tags[0]["name"]` reads a field of one element.

-   `"smeltr".at(0)` → `"s"`
-   `"smeltr".at(-1)` → `"r"` // a negative index counts from the end
-   `"smeltr".at(99)` → `null` throws `IndexOutOfRange` // past either end of a string is null
-   `line = "a,b,c"`: `line.split(",")[-1]` → `"c"` // `[...]` is `at`, negative index included
-   `at("smeltr", 0)` → `"s"` // the same call, written as a function
-   `payload.status = 200`: `payload["status"]` → `200` // a fixed string key on a named field

### `value.indexOf(search)`

`(str, str)` → `i64`; `(array of scalars, value)` → `i64`

Where `search` first occurs in a string, or which element of an array equals it. `-1` when it does not.

-   `line = "a,b,c"`: `line.indexOf(",")` → `1`
-   `"GET /health".indexOf("POST")` → `-1` // `-1`, not null
-   `line = "a,b,c"`: `line.split(",").indexOf("c")` → `2`
-   `line = "a,b,c"`: `indexOf(line, ",")` → `1` // the same call, written as a function

### `value.includes(search)`

`(str, str)` → `bool`; `(array of scalars, value)` → `bool`

Whether a string contains `search`, or an array contains an element equal to it.

-   `msg = "connection reset by peer"`: `msg.includes("reset")` → `true`
-   `"GET,HEAD".split(",").includes("HEAD")` → `true` // arrays too
-   `msg = "connection reset by peer"`: `includes(msg, "reset")` → `true` // the same call, written as a function

## Strings

A `str` is text, always valid UTF-8. `+` joins two. [length](#length), [slice](#slice), [at](#at), [indexOf](#indexOf) and [includes](#includes) take a string or an array.

There is no implicit stringification: a number has to go through `toString()` before it can be joined to text.

|  |  |  |
| --- | --- | --- |
| `+` | join text | `"GET" + " /health"` → `"GET /health"` |

Does not compile:

|  |  |
| --- | --- |
| `"a" + 1` | No implicit stringification. Write `"a" + (1).toString()`. |

### `value.lastIndexOf(search)`

`(str, str)` → `i64`

Where `search` last occurs, or `-1`. Useful for splitting on the final separator.

-   `"/api/v2/things".lastIndexOf("/")` → `7`
-   `"a.b.c".lastIndexOf(".")` → `3`
-   `lastIndexOf("/api/v2/things", "/")` → `7` // the same call, written as a function

### `value.startsWith(prefix)`

`(str, str)` → `bool`

Whether the string begins with `prefix`.

-   `path = "/api/v2/things"`: `path.startsWith("/api/")` → `true`
-   `"GET /health".startsWith("POST")` → `false`
-   `path = "/api/v2/things"`: `startsWith(path, "/api/")` → `true` // the same call, written as a function

### `value.endsWith(suffix)`

`(str, str)` → `bool`

Whether the string ends with `suffix`.

-   `"app.log.gz".endsWith(".gz")` → `true`
-   `endsWith("app.log.gz", ".gz")` → `true` // the same call, written as a function

### `value.toUpperCase()`

`(str)` → `str`

Uppercased. Unicode-aware, so a character can expand.

-   `level = "warn"`: `level.toUpperCase()` → `"WARN"`
-   `"straße".toUpperCase()` → `"STRASSE"` // Unicode-aware, so one character can become two
-   `level = "warn"`: `toUpperCase(level)` → `"WARN"` // the same call, written as a function

### `value.toLowerCase()`

`(str)` → `str`

Lowercased, Unicode-aware. Lowercase both sides to compare them without regard to case.

-   `"WARN".toLowerCase()` → `"warn"`
-   `"Get".toLowerCase() == "get"` → `true`
-   `toLowerCase("WARN")` → `"warn"` // the same call, written as a function

### `value.trim()`

`(str)` → `str`

Whitespace removed from both ends.

-   `raw = " disk full "`: `raw.trim()` → `"disk full"`
-   `"\ttabbed\n".trim()` → `"tabbed"` // every kind of whitespace, not just spaces
-   `raw = " disk full "`: `trim(raw)` → `"disk full"` // the same call, written as a function

### `value.trimStart()`

`(str)` → `str`

Whitespace removed from the start only.

-   `" indented".trimStart()` → `"indented"`
-   `trimStart(" indented")` → `"indented"` // the same call, written as a function

### `value.trimEnd()`

`(str)` → `str`

Whitespace removed from the end only.

-   `"trailing ".trimEnd()` → `"trailing"`
-   `trimEnd("trailing ")` → `"trailing"` // the same call, written as a function

### `value.padStart(length, pad = " ")`

`(str, i64)` → `str`; `(str, i64, str)` → `str`

Pads the front with `pad` — a space by default — until the string is `length` characters. Already long enough is returned unchanged.

-   `"7".padStart(3, "0")` → `"007"`
-   `"5".padStart(3)` → `" 5"` // the default pad is a space
-   `id = 42`: `id.toString().padStart(5, "0")` → `"00042"`
-   `"toolong".padStart(3, "0")` → `"toolong"` // never truncates
-   `"x".padStart(2000000000)` → `null` // a result over 16 MiB is null and throws nothing
-   `padStart("7", 3, "0")` → `"007"` // the same call, written as a function

### `value.padEnd(length, pad = " ")`

`(str, i64)` → `str`; `(str, i64, str)` → `str`

Pads the end with `pad`, a space by default, until the string is `length` characters. Already long enough is returned unchanged.

-   `"id".padEnd(5, ".")` → `"id..."`
-   `"hi".padEnd(4)` → `"hi "` // the default pad is a space
-   `padEnd("id", 5, ".")` → `"id..."` // the same call, written as a function

### `value.repeat(count)`

`(str, i64)` → `str`

The string repeated `count` times.

-   `"ab".repeat(3)` → `"ababab"`
-   `"x".repeat(0)` → `""`
-   `"x".repeat(-1)` → `null` // a negative count, or a result over 16 MiB, is null
-   `repeat("ab", 3)` → `"ababab"` // the same call, written as a function

### `value.md5()`

`(str/bytes)` → `str`

The MD5 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex. A fingerprint for spotting repeats, never a security check.

-   `"abc".md5()` → `"900150983cd24fb0d6963f7d28e17f72"`

### `value.sha1()`

`(str/bytes)` → `str`

The SHA-1 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex.

-   `"abc".sha1()` → `"a9993e364706816aba3e25717850c26c9cd0d89d"`

### `value.sha256()`

`(str/bytes)` → `str`

The SHA-256 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex.

-   `"abc".sha256()` → `"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"`

### `value.sha512()`

`(str/bytes)` → `str`

The SHA-512 digest of the bytes, or of a text's UTF-8 bytes, in lowercase hex.

-   `"abc".sha512().slice(0, 16)` → `"ddaf35a193617aba"` // the first 16 of 128 digits

## Regular expressions

A pattern is written as a literal, `/[0-9]+/`, and compiled once, when the pipeline is compiled, not per row. The flags are `i` to ignore case, `m` to let `^` and `$` match at each line, and `s` to let `.` match a newline.

### `value.test(/pattern/)`

`(str, /pattern/)` → `bool`

Whether the pattern matches anywhere in the value, as a `bool`. Use it in a `Drop` condition or a route.

-   `"GET /health".test(/^GET/)` → `true`
-   `"id=abc".test(/^id=[0-9]+$/)` → `false`
-   `"Error: nope".test(/error/i)` → `true`
-   `test("GET /health", /^GET/)` → `true` // the same call, written as a function

### `value.match(/pattern/)`

`(str, /pattern/)` → `array(str)`

The whole match followed by each capture group, as an array — or null when nothing matches.

Element 0 is the whole match, so a capture group is at 1.

-   `"2026-08-30".match(/([0-9]+)-([0-9]+)-([0-9]+)/)` → `["2026-08-30","2026","08","30"]`
-   `"v=17".match(/v=([0-9]+)/).at(1)` → `"17"`
-   `"nothing here".match(/[0-9]+/)` → `null`
-   `match("2026-08-30", /([0-9]+)-([0-9]+)-([0-9]+)/)` → `["2026-08-30","2026","08","30"]` // the same call, written as a function

### `value.replaceFirst(/pattern/, replacement)`

`(str, /pattern/, str)` → `str`

The value with the first match replaced. `$1`, `$<name>` and `$&` in the replacement refer back to the match.

-   `"a1b2".replaceFirst(/[0-9]/, "#")` → `"a#b2"` // the first match only
-   `"2026-08-30".replaceFirst(/([0-9]+)-([0-9]+)-([0-9]+)/, "$3/$2/$1")` → `"30/08/2026"` // capture groups, reordered
-   `replaceFirst("user=alice", /user=(.+)/, "user=***")` → `"user=***"` // the same call, written as a function

### `value.replaceAll(/pattern/, replacement)`

`(str, /pattern/, str)` → `str`

The value with every match replaced.

-   `"a1b2".replaceAll(/[0-9]/, "#")` → `"a#b#"` // every match, where `replaceFirst` takes one
-   `"user=alice".replaceAll(/user=(.+)/, "user=***")` → `"user=***"`
-   `replaceAll("a1b2", /[0-9]/, "#")` → `"a#b#"` // the same call, written as a function

## Arrays

An `array<T>` holds elements of the one type `T`. `filter` keeps the elements whose condition is true and `map` computes a value for each element. [length](#length), [slice](#slice), [at](#at), [indexOf](#indexOf) and [includes](#includes) take an array or a string.

`+` joins two arrays of the same element type. It never appends a single element, since `xs + y` would be ambiguous when `y` is itself an array; [push](#push) appends.

|  |  |  |
| --- | --- | --- |
| `+` | join two arrays | `line = "a,b"`;`more = "c,d"`: `line.split(",") + more.split(",")` → `["a","b","c","d"]` |

Does not compile:

|  |  |
| --- | --- |
| `["a", "b"] + "c"` | `+` joins two arrays; use `push` to append an element. |

### `array.map(item => expression)`

A new array with one computed value for each element. The source array is unchanged.

It compiles in an Eval or Code step. In a route condition, a delivery condition or a capture filter it does not compile.

The arrow takes one parameter. Its body is one expression, or a block whose last expression is the value. An arrow may call local functions and hold another arrow.

-   `[1, 2, 3].map(x => x * 2)` → `[2,4,6]`
-   `map([1, 2, 3], x => x * 2)` → `[2,4,6]` // the same call in function form

### `array.filter(item => condition)`

A new array of the elements whose condition is true. It filters elements inside a record; a Drop step discards the whole record.

It compiles in an Eval or Code step. In a route condition, a delivery condition or a capture filter it does not compile.

The arrow takes one parameter. Its body is one expression, or a block whose last expression is the value. An arrow may call local functions and hold another arrow.

A null condition counts as false. When no element matches, the result is an empty array.

-   `[1, 2, 3, 4].filter(x => x > 2)` → `[3,4]`
-   `[1, 2].filter(x => x > 9)` → `[]`
-   `filter([1, 2, 3], x => x > 1).map(x => x * 2)` → `[4,6]` // filter and then transform the kept elements

### `value.split(delimiter)`

`(str, str)` → `array(str)`; `(str, /pattern/)` → `array(str)`

Splits a string into an array on a literal delimiter or a regex. An empty delimiter splits into characters.

-   `"GET /health HTTP/1.1".split(" ")[2]` → `"HTTP/1.1"` // index the array with `[...]` -- one field out of a delimited line
-   `line = "a,b,c"`: `line.split(",")` → `["a","b","c"]`
-   `"a1b22c".split(/[0-9]+/)` → `["a","b","c"]` // a regex separator splits on runs
-   `"abc".split("")` → `["a","b","c"]`
-   `line = "GET /health HTTP/1.1"`: `split(line, " ")[2]` → `"HTTP/1.1"` // the same call, written as a function

### `array.join(separator)`

`(array of scalars, str)` → `str`

The elements as one string, separated by `separator`.

-   `line = "a,b,c"`: `line.split(",").join(" | ")` → `"a | b | c"`
-   `"a,b".split(",").join("")` → `"ab"`
-   `line = "a,b,c"`: `join(line.split(","), " | ")` → `"a | b | c"` // the same call, written as a function

### `array.push(value, ...)`

`(array, value, ...)` → `the array's type`

The array with every value appended. A null value is a position, as it is in `["a", null]`.

-   `line = "a,b"`: `line.split(",").push("c")` → `["a","b","c"]`
-   `x = [] as array<str>`: `x.push("a")` → `["a"]` // `[]` has no element type of its own, so the name that holds it declares one
-   `line = "a,b"`: `line.split(",").push("c", "d", "e")` → `["a","b","c","d","e"]` // every value in one call -- a loop that pushes rebuilds the whole array each time round
-   `line = "a,b"`: `line.split(",").push(parseInt("x").toString())` → `["a","b",null]` throws `CoercionRefused` // a null value is a position, so every value keeps its place
-   `line = "a,b"`: `line.split(",").push("c").join(" | ")` → `"a | b | c"` // the new array comes back, so it goes straight into the next call
-   `line = "a,b"`: `push(line.split(","), "c")` → `["a","b","c"]` // the same call, written as a function

### `array.insert(index, value, ...)`

`(array, i64, value, ...)` → `the array's type`

The array with every value inserted before `index`. `push` is this with the position at the end.

An index past either end clamps. A null index makes the result null. A null value takes a position.

-   `line = "a,b,d"`: `line.split(",").insert(2, "c")` → `["a","b","c","d"]`
-   `line = "a,b"`: `line.split(",").insert(-1, "z")` → `["a","z","b"]` // `-1` is the last element, so the value lands before it -- to append, use `push`
-   `line = "a,b"`: `line.split(",").insert(1, "x", "y")` → `["a","x","y","b"]`
-   `line = "a,b"`: `line.split(",").insert(1, parseInt("x").toString())` → `["a",null,"b"]` throws `CoercionRefused` // a null value is a position here too
-   `line = "a,b,d"`: `insert(line.split(","), 2, "c")` → `["a","b","c","d"]` // the same call, written as a function

### `array.remove(index)`

`(array of scalars, i64)` → `the array's type`

The array without the element at `index`. The elements after it move up one.

A null array or index makes the result null.

-   `line = "a,b,c"`: `line.split(",").remove(1)` → `["a","c"]`
-   `line = "a,b,c"`: `line.split(",").remove(-1)` → `["a","b"]` // `-1` is the last element, as it is for `at`
-   `line = "a,b,c"`: `line.split(",").remove(5)` → `["a","b","c"]` // an index outside the array removes nothing

### `array.resize(size)`

`(array of scalars, i64)` → `the array's type`

The array with exactly `size` positions: its first `size`, or its own followed by positions that hold nothing.

A negative or null size, or a null array, makes the result null.

-   `line = "a,b,c"`: `line.split(",").resize(2)` → `["a","b"]`
-   `line = "a,b,c"`: `line.split(",").resize(4)` → `["a","b","c",null]` // a new position holds nothing until a write fills it
-   `resize([] as array<str>, 2)` → `[null,null]` // sized before any data arrives; each position holds nothing until a write fills it

### `array.concat(other, ...)`

`(array of scalars, array of scalars, ...)` → `the array's type`

One array with every element of all of them, in order. `a + b` is the same call.

-   `line = "a,b"`: `line.split(",").concat(["c", "d"])` → `["a","b","c","d"]`
-   `line = "a,b"`: `line.split(",").concat(["c"], ["d", "e"])` → `["a","b","c","d","e"]` // any number of arrays, in one pass
-   `line = "a,b"`: `concat(line.split(","), ["c", "d"])` → `["a","b","c","d"]` // the same call, written as a function

### `array.sort(descending)`

`(array of scalars)` → `the array's type`; `(array of scalars, bool)` → `the array's type`

The elements in order: text by its bytes, numbers by value, `false` before `true`. `descending` is optional. Equal elements keep their order and nulls go last.

-   `["b", "c", "a"].sort()` → `["a","b","c"]`
-   `[3, 1, 2].sort(true)` → `[3,2,1]` // descending
-   `[2, null, 1].sort()` → `[1,2,null]` // nulls last

## Dates and times

A datetime is an instant in UTC, microsecond-resolution, and serializes as ISO 8601. `now()` is read once per record, so every route a record takes sees the same instant. There is no local time zone: a log line's own offset is read when it is parsed and kept nowhere else.

[Comparison](#tests) orders instants: `start < end` is true when `start` is earlier.

|  |  |  |
| --- | --- | --- |
| `+` | advance an instant | `"1998-08-30".parseTime() + duration(90, "s")` → `"1998-08-30T00:01:30.000000Z"`<br>`now() + duration(1, "h") > now()` → `true` // `now()` is read once per record, so both of these are the same instant |
| `-` | wind an instant back | `"1998-08-30".parseTime() - duration(1, "d")` → `"1998-08-29T00:00:00.000000Z"` |
| `-` | the span between two instants | `"1998-08-31".parseTime() - "1998-08-30".parseTime() == duration(1, "d")` → `true` |

Does not compile:

|  |  |
| --- | --- |
| `"1998-08-30T12:00:00Z".parseTime() + "1998-08-30T12:00:00Z".parseTime()` | Two instants have no sum. Subtract them for a span. |

The clock

### `now()`

`()` → `datetime`

The current instant. Captured once per record, so every route a record takes sees the same value.

-   `now().year() >= 2026` → `true`

Reading text

### `value.parseTime([format][, reference = now()])`

`(str)` → `datetime`; `(str, str)` → `datetime`; `(str, datetime)` → `datetime`; `(str, str, datetime)` → `datetime`

Parses text into a datetime. With no format, a run of digits is a compact date when it reads as one (`YYYYMMDD` or `YYYYMMDDHHMMSS`), else a Unix epoch whose unit comes from its magnitude — seconds below 10^11, then milliseconds, microseconds and nanoseconds at each further thousand — and other text is matched against the common shapes — ISO 8601, `YYYY-MM-DD HH:MM:SS`, `YYYY/MM/DD HH:MM:SS`, syslog's `Mon D HH:MM:SS`, and Apache's `[D/Mon/YYYY:HH:MM:SS +ZZZZ]`. Give a `strftime` format to pin it, and a reference instant to fix the year a syslog stamp does not carry. A parser's declared `datetime` field is this call: the bare type detects, and `{"type":"datetime","format":...}` pins.

The reference is a datetime, not a year — usually the field carrying when the record was collected. Pass none and it is `now()`, which is what makes the no-argument form right for a live stream.

A yearless stamp takes the reference's year, and rolls back a year rather than landing in the future: parsed against 1 January 1999, `Dec 31` is 1998. That also means an anchor \*earlier\* than the stamp reads as the year before it.

Text that does not parse is null on that row and throws a `CoercionRefused` exception on the record. An impossible date is such text, never a silently shifted one: `1998-02-30` does not become 2 March.

A zone name is not one offset: `CST` is six hours behind UTC in Chicago and eight ahead in Shanghai, and one pipeline reads both. `%z` takes the fixed set the table names. Replace any other name with the offset that source means — `replaceAll(/CEST/, "+0200")` — and the choice sits beside the source it belongs to.

Reading what it finds

-   `"1998-08-30T12:00:00Z".parseTime()` → `"1998-08-30T12:00:00.000000Z"`
-   `"1998-08-30 12:34:56".parseTime().hour()` → `12`
-   `"904478400".parseTime()` → `"1998-08-30T12:00:00.000000Z"` // ten digits of seconds; the same instant in milliseconds, `904478400000`, reads the same
-   `"904478400000".parseTime() == "904478400".parseTime()` → `true`
-   `"19980830".parseTime()` → `"1998-08-30T00:00:00.000000Z"` // eight digits that read as a date are that date, not a count of seconds

Pinning the shape

-   `"30/08/1998".parseTime("%d/%m/%Y")` → `"1998-08-30T00:00:00.000000Z"`
-   `"2026-08-30 12:00:00 CEST".parseTime("%Y-%m-%d %H:%M:%S %z")` → `null` throws `CoercionRefused` // a zone name outside the set `%z` takes is null, never a guess
-   `"2026-08-30 12:00:00 CEST".replaceAll(/CEST/, "+0200").parseTime("%Y-%m-%d %H:%M:%S %z")` → `"2026-08-30T10:00:00.000000Z"` // give the name the offset it means for this source, then parse
-   `"1000".parseTime("%Q")` → `"1970-01-01T00:00:01.000000Z"` // auto-detect reads four digits as seconds; `%Q` says milliseconds and `%s` says seconds

Yearless stamps

-   `"Feb 3 04:05:06".parseTime() == "Feb 3 04:05:06".parseTime(now())` → `true` // no reference means `now()`, which is where a yearless stamp gets its year
-   `"02-03 04:05:06".parseTime("%m-%d %H:%M:%S")` → `"2026-02-03T04:05:06.000000Z"` // a shape auto-detect does not know; the year comes from `now()`
-   `"Dec 31 23:59:00".parseTime("1999-01-01T00:05:00Z".parseTime())` → `"1998-12-31T23:59:00.000000Z"` // a December line read five minutes into January is last year's, not next year's

-   `parseTime("1998-08-30T12:00:00Z")` → `"1998-08-30T12:00:00.000000Z"` // the same call, written as a function

`parseTime(value, format)` takes these, and only these. Every other byte of the format must match the text literally, and text left over when the format runs out makes the result null rather than a partial read. Auto-detect tries an ordered list of formats built from the same specifiers, so every rule below holds for `parseTime(value)` too.

Format specifiers for parseTime:

| The date |
| --- | --- | --- |
| `%Y` | The year, exactly four digits. There is no two-digit year: every century pivot is wrong for someone. | `"2026-08-30".parseTime("%Y-%m-%d")` → `"2026-08-30T00:00:00.000000Z"` |
| `%m` | Month number, `1`\-`12`, with or without a leading zero. | `"8/2026".parseTime("%m/%Y")` → `"2026-08-01T00:00:00.000000Z"` |
| `%d` | Day of month, `1`\-`31`, with or without a leading zero. A day past the month's end is null. | `"2026-02-30".parseTime("%Y-%m-%d")` → `null` throws `CoercionRefused` // February has no 30th |
| `%e` | Day of month with an optional leading space, as in `Aug 3`. | `"Aug 3 2026".parseTime("%b %e %Y")` → `"2026-08-03T00:00:00.000000Z"` |
| The time of day |
| `%H` | Hour on a 24-hour clock, `0`\-`23`. `24` is ISO 8601's end-of-day and is allowed only at a zero minute, second and fraction. | `"2026-08-30 24:00:00".parseTime("%Y-%m-%d %H:%M:%S")` → `"2026-08-31T00:00:00.000000Z"` // the midnight that closes the date |
| `%I` | Hour on a 12-hour clock, `1`\-`12`. Pair it with `%p`. | `"2026-08-30 02:00 PM".parseTime("%Y-%m-%d %I:%M %p")` → `"2026-08-30T14:00:00.000000Z"` |
| `%M` | Minute, `0`\-`59`, two digits. | `"2026-08-30 00:45".parseTime("%Y-%m-%d %H:%M")` → `"2026-08-30T00:45:00.000000Z"` |
| `%S` | Second, `0`\-`60`, two digits. `60` is a leap second and lands on the following minute. | `"2016-12-31 23:59:60".parseTime("%Y-%m-%d %H:%M:%S")` → `"2017-01-01T00:00:00.000000Z"` |
| `%f` | Fractional seconds. Every digit is read and the first six kept, so a nanosecond stamp is truncated rather than refused; a shorter run is a position, not a count. | `"2026-08-30 00:00:00.123456789".parseTime("%Y-%m-%d %H:%M:%S.%f")` → `"2026-08-30T00:00:00.123456Z"` |
| The offset |
| `%z` | A UTC offset — `Z`, `+HH`, `+HHMM` or `+HH:MM` — or one of the named zones `UT`, `UTC`, `GMT`, `EST`, `EDT`, `CST`, `CDT`, `MST`, `MDT`, `PST`, `PDT`. Any other name is null. Without `%z` the text reads as UTC. | `"2026-08-30T00:00:00+05".parseTime("%Y-%m-%dT%H:%M:%S%z")` → `"2026-08-29T19:00:00.000000Z"` // A whole-hour offset without its minutes |
| Names and AM/PM |
| `%b` | An English month name, abbreviated or full. The same specifier as `%B`. | `"30 August 2026".parseTime("%d %b %Y")` → `"2026-08-30T00:00:00.000000Z"` |
| `%B` | An English month name, abbreviated or full. The same specifier as `%b`. | `"30 Aug 2026".parseTime("%d %B %Y")` → `"2026-08-30T00:00:00.000000Z"` |
| `%a` | An English weekday name, abbreviated or full. Consumed and discarded — it is never checked against the date. | `"Sun, 30 Aug 2026".parseTime("%a, %d %b %Y")` → `"2026-08-30T00:00:00.000000Z"` |
| `%A` | An English weekday name, abbreviated or full. The same specifier as `%a`. | `"Sunday 30 Aug 2026".parseTime("%A %d %b %Y")` → `"2026-08-30T00:00:00.000000Z"` |
| `%p` | `AM` or `PM`, either case. Reads `%I`; `12 AM` is midnight and `12 PM` is noon. | `"2026-08-30 12:00 AM".parseTime("%Y-%m-%d %I:%M %p")` → `"2026-08-30T00:00:00.000000Z"` |
| `%s` | Seconds since the Unix epoch, with an optional sign and fraction. It names the whole instant, so the format holds no other specifier. | `"1000.5".parseTime("%s")` → `"1970-01-01T00:16:40.500000Z"` // auto-detect would read a count this small as seconds too; the specifier says so outright |
| `%Q` | Milliseconds since the Unix epoch, with an optional sign and fraction. It names the whole instant, so the format holds no other specifier. | `"1000".parseTime("%Q")` → `"1970-01-01T00:00:01.000000Z"` // auto-detect reads a count this small as seconds |
| `%%` | A literal percent sign. | `"2026 100%".parseTime("%Y 100%%")` → `"2026-01-01T00:00:00.000000Z"` |

### `value.parseDuration()`

`(str)` → `duration`

Parses text into a duration: the count of microseconds `toString()` writes, whole or as a decimal rounded to the microsecond. Text that is no count is null.

-   `"90000000".parseDuration().countIn("s")` → `90.0`
-   `duration(250, "ms").toString().parseDuration() == duration(250, "ms")` → `true` // what `toString()` writes reads back as the same span
-   `"soon".parseDuration()` → `null` throws `CoercionRefused`

A part of the instant

### `value.year()`

`(datetime)` → `i64`

The calendar year.

-   `"1998-08-30".parseTime().year()` → `1998`

### `value.month()`

`(datetime)` → `i64`

The month, 1 to 12.

-   `"1998-08-30".parseTime().month()` → `8`

### `value.day()`

`(datetime)` → `i64`

The day of the month.

-   `"1998-08-30".parseTime().day()` → `30`

### `value.hour()`

`(datetime)` → `i64`

The hour, 0 to 23, in UTC.

-   `"1998-08-30 12:34:56".parseTime().hour()` → `12`

### `value.minute()`

`(datetime)` → `i64`

The minute, 0 to 59.

-   `"1998-08-30 12:34:56".parseTime().minute()` → `34`

### `value.second()`

`(datetime)` → `i64`

The second, 0 to 59.

-   `"1998-08-30 12:34:56".parseTime().second()` → `56`

### `value.millisecond()`

`(datetime)` → `i64`

The millisecond within the second, 0 to 999.

-   `"1998-08-30T12:00:00.250Z".parseTime().millisecond()` → `250`

### `value.dayOfWeek()`

`(datetime)` → `i64`

The day of the week: 1 is Monday, 7 is Sunday.

-   `"1998-08-31".parseTime().dayOfWeek()` → `1`

Epoch counts

### `value.epochMillis()`

`(datetime)` → `i64`

Milliseconds since the Unix epoch.

-   `"1998-08-30 12:00:00".parseTime().epochMillis()` → `904478400000`

### `epochSeconds(value)`

`(datetime)` → `i64`

Whole seconds since the Unix epoch, flooring so a time before it counts backward rather than toward zero.

-   `epochSeconds(parseTime("1998-08-30T12:00:00Z"))` → `904478400`

### `fromEpochMillis(millis)`

`(i64)` → `datetime`

A datetime from milliseconds since the Unix epoch, for a producer that already sends one.

-   `fromEpochMillis(904478400000)` → `"1998-08-30T12:00:00.000000Z"`

### `fromEpochSeconds(seconds)`

`(i64)` → `datetime`

A datetime from seconds since the Unix epoch. A count past the datetime range is null.

-   `fromEpochSeconds(904478400)` → `"1998-08-30T12:00:00.000000Z"`

## Durations

A duration is a signed span, microsecond-resolution. Subtracting two datetimes produces one. It serializes as a bare count of microseconds, `toString()` produces the same text, and `parseDuration()` reads it back.

[Comparison](#tests) orders spans: `age > duration(5, "m")` is true when `age` is longer than five minutes.

|  |  |  |
| --- | --- | --- |
| `+` | add two spans | `duration(30, "m") + duration(45, "s") == duration(1845, "s")` → `true` |
| `-` | the difference of two spans | `duration(1, "h") - duration(30, "m") == duration(30, "m")` → `true` |
| `*` | scale a span | `duration(90, "s") * 2 == duration(3, "m")` → `true` |
| `/` | the ratio of two spans | `duration(1, "h") / duration(1, "m")` → `60.0` |
| `/` | divide a span | `duration(1, "h") / 4 == duration(15, "m")` → `true` |

Does not compile:

|  |  |
| --- | --- |
| `duration(1, "y")` | A year varies in length, so a span cannot be counted in one. Days are fixed: `duration(365, "d")`. |

### `duration(count, unit)`

`(i64/f64, str)` → `duration`

A span of `count` units. The unit is `"ns"`, `"us"`, `"ms"`, `"s"`, `"m"`, `"h"`, `"d"`, `"w"` or a long name, and the count may be fractional.

A span is held to the microsecond, so anything finer truncates toward zero: `duration(1500, "ns")` is one microsecond and `duration(1, "ns")` is a zero span. A fractional count of any unit rounds the same way.

There is no month or year: how long either one is depends on which one, and a span is a count with no calendar attached to answer that. A week is seven days, so it is here.

A literal unit is checked when the pipeline compiles: `duration(1, "y")` does not compile. A unit computed per row that names no unit makes the result null. A span too large to hold is null.

-   `duration(90, "s").toString()` → `"90000000"`
-   `duration(250, "ms").toString()` → `"250000"`
-   `duration(1.5, "d").toString()` → `"129600000000"`
-   `duration(1000, "ns").toString()` → `"1"` // a thousand nanoseconds is one microsecond, exactly
-   `duration(2, "w") == duration(14, "d")` → `true` // a week is exactly seven days
-   `duration(90, "s") > duration(1, "minute")` → `true`

### `duration.countIn(unit)`

`(duration, str)` → `f64`

The whole span counted in one unit, as an `f64`, not a remainder.

-   `duration(90, "s").countIn("m")` → `1.5`
-   `duration(36, "h").countIn("d")` → `1.5`
-   `duration(1, "d").countIn("h")` → `24.0`
-   `duration(1, "s").countIn("us")` → `1000000.0` // the unit the wire form counts in
-   `duration(14, "d").countIn("w")` → `2.0`
-   `started = "1998-08-30 12:00:00".parseTime()`;`ended = "1998-08-30 12:30:00".parseTime()`: `(ended - started).countIn("m")` → `30.0`
-   `countIn(duration(90, "s"), "m")` → `1.5` // the same call, written as a function

## Addresses and networks

An address is text. A function parses it and writes it back as text. Text that is not an address makes the result null and throws no exception. Space around an address is ignored. To match an address against a table of blocks, use a lookup.

### `text.isIp()`

`(str)` → `bool`

The text is an address, v4 or v6. Text that is not one is `false`. Null stays null.

-   `"10.1.2.3".isIp()` → `true`
-   `"fe80::1".isIp()` → `true`
-   `"10.1.2".isIp()` → `false`
-   `isIp("10.1.2.3")` → `true` // the same call, written as a function

### `text.ipVersion()`

`(str)` → `i64`

`4` or `6`.

-   `"10.1.2.3".ipVersion()` → `4`
-   `"::1".ipVersion()` → `6`
-   `"example.com".ipVersion()` → `null`

### `text.ipNormalize()`

`(str)` → `str`

The address in one spelling. Leading zeros go. IPv6 is lowercase and its longest run of zero groups is compressed. Two spellings of one address come out the same.

-   `"2001:0DB8:0000:0000:0000:0000:0000:0001".ipNormalize()` → `"2001:db8::1"`
-   `" 10.1.2.3 ".ipNormalize()` → `"10.1.2.3"`
-   `"010.1.2.3".ipNormalize()` → `null` // a dotted quad takes no leading zero

### `text.ipScope()`

`(str)` → `str`

Where the address reaches: `unspecified`, `loopback`, `link-local`, `multicast`, `private` or `public`. Private is RFC 1918 and `fc00::/7`.

-   `"10.1.2.3".ipScope()` → `"private"`
-   `"8.8.8.8".ipScope()` → `"public"`
-   `"127.0.0.1".ipScope()` → `"loopback"`
-   `"fe80::1".ipScope()` → `"link-local"`
-   `"224.0.0.1".ipScope()` → `"multicast"`
-   `"::".ipScope()` → `"unspecified"`
-   `"fd12::1".ipScope()` → `"private"`

### `address.inCidr(block, ...)`

`(str, str, ...)` → `bool`

The address is in one of the blocks. A block of the other family holds nothing. A bare address is a block of one.

-   `"10.1.2.3".inCidr("10.0.0.0/8")` → `true`
-   `"10.1.2.3".inCidr("192.168.0.0/16", "10.0.0.0/8")` → `true` // any number of blocks
-   `"10.1.2.3".inCidr("::/0")` → `false` // the other family
-   `"10.1.2.3".inCidr("10.1.2.3")` → `true` // a bare address is a block of one
-   `"10.1.2.3".inCidr("10.0.0.0/33")` → `null` // a block that does not parse

### `address.cidrNetwork(prefix)`

`(str, i64)` → `str`

The address masked to `prefix` bits, written as a block. Group by it to count per subnet.

-   `"10.1.2.3".cidrNetwork(24)` → `"10.1.2.0/24"`
-   `"10.1.2.3".cidrNetwork(12)` → `"10.0.0.0/12"`
-   `"2001:db8:abcd::1".cidrNetwork(32)` → `"2001:db8::/32"`
-   `"10.1.2.3".cidrNetwork(33)` → `null` // past the width of the address

### `communityId(srcIp, srcPort, dstIp, dstPort, transport, seed)`

`(str, i64, str, i64, str)` → `str`; `(str, i64, str, i64, i64)` → `str`; `(str, i64, str, i64, str, i64)` → `str`; `(str, i64, str, i64, i64, i64)` → `str`

The flow's [Community ID](https://github.com/corelight/community-id-spec), version 1. `transport` is an IANA protocol number, 0 to 255, as a number or as text, or one of these names in any case: `icmp`, `igmp`, `tcp`, `udp`, `gre`, `icmpv6` (or `ipv6-icmp`), `eigrp`, `ospf`, `pim`, `sctp`. Any other name is null. TCP, UDP and SCTP need both ports, 1 to 65535; for ICMP and ICMPv6 the ports are the type and the code; any other protocol has no ports. Both addresses must be IPv4 or both IPv6. `seed` is optional, 0 to 65535, 0 by default. Both directions of a flow give one ID.

-   `communityId("128.232.110.120", 34855, "66.35.250.204", 80, "tcp")` → `"1:LQU9qZlK+B5F3KDmev6m5PMibrg="`
-   `communityId("66.35.250.204", 80, "128.232.110.120", 34855, 6)` → `"1:LQU9qZlK+B5F3KDmev6m5PMibrg="` // the reply, with the protocol's number
-   `communityId("128.232.110.120", 34855, "66.35.250.204", 80, "tcp", 123)` → `"1:hTSGlFQnR58UCk+NfKRZzA32dPg="` // a seed
-   `communityId("128.232.110.120", 0, "66.35.250.204", 80, "tcp")` → `null` // TCP needs both ports
-   `communityId("10.0.0.1", 5, "10.0.0.2", 6, "quic")` → `null` // not a protocol name it knows

## URLs, domains and user agents

Text that does not read is null and throws a `CoercionRefused` exception on the record; blank text is null and throws nothing.

### `text.urlDecode()`

`(str)` → `str`

`%XX` as the byte it names and `+` as a space, as a form field is encoded, and the bytes read as UTF-8; a byte that is not UTF-8 becomes U+FFFD. A `%` that two hex digits do not follow stays a `%`, so every text reads.

-   `"a%20b+c%C3%A9".urlDecode()` → `"a b cé"`
-   `"100%".urlDecode()` → `"100%"` // a bare % stays

### `text.htmlStrip()`

`(str)` → `str`

The text without its markup. A tag goes, in any case. A block-level tag, start or end, becomes a line break: `address`, `article`, `aside`, `blockquote`, `br`, `dd`, `div`, `dl`, `dt`, `fieldset`, `figcaption`, `figure`, `footer`, `form`, `h1` to `h6`, `header`, `hgroup`, `hr`, `li`, `nav`, `noscript`, `ol`, `output`, `p`, `pre`, `section`, `table`, `tbody`, `td`, `tfoot`, `th`, `thead`, `tr` and `ul`. A comment, a `<!...>` or `<?...>` declaration, and a `<script>` or `<style>` element go with what they hold; a `<![CDATA[...]]>` section leaves what it holds. A character reference becomes its character: a number, `&#233;` or `&#xE9;`, or any of HTML 4's names, such as `&amp;`, `&nbsp;` or `&ouml;`. A reference that does not read, and a `<` that starts no tag, stay as they are. Text without both `<` and `>` is returned as it is.

-   `"<b>bold</b> &amp; plain".htmlStrip()` → `"bold & plain"`
-   `"<p>caf&eacute; &#8212; ok</p>".htmlStrip()` → `"\ncafé — ok\n"` // named and numeric references
-   `"one<br>two".htmlStrip()` → `"one\ntwo"` // a block-level tag breaks the line

### `text.parseUrl()`

`(str)` → `{domain: str, fragment: str, path: str, extension: str, port: i64, query: str, scheme: str, user_info: str, username: str, password: str}`

A URL's parts as an object: `domain`, `fragment`, `path`, `extension`, `port`, `query`, `scheme`, `user_info`, `username` and `password`. Text that reads as an RFC 3986 URI or relative reference, such as a bare path, has its path, query, fragment and user info decoded. An `http`, `https`, `ftp`, `file` or `jar` URL that does not, holding a space or a `|` say, is split as written, undecoded. `extension` is what follows the last `.` of the path's last segment. `username` is the user info up to its first `:`, all of it where there is none, and `password` what follows that `:`. A part the URL lacks is null.

-   `"https://www.example.com:8443/files/report.pdf?x=1#top".parseUrl()` → `{"domain":"www.example.com","fragment":"top","path":"/files/report.pdf","extension":"pdf","port":8443,"query":"x=1","scheme":"https"}`
-   `"http://user:pw@host/a%20b".parseUrl().path` → `"/a b"` // a part read on its own
-   `"/search/index.html?q=x".parseUrl()` → `{"path": "/search/index.html", "extension": "html", "query": "q=x"}` // a path alone
-   `"not a url".parseUrl()` → `null` throws `CoercionRefused`

### `host.registeredDomain()`

`(str)` → `{domain: str, registered_domain: str, top_level_domain: str, subdomain: str}`

The host, the name registered under its public suffix, that suffix and what is left, as an object of `domain`, `registered_domain`, `top_level_domain` and `subdomain`. The suffixes are the ICANN section of the Public Suffix List, compiled in. A host with no known suffix does not read.

-   `"www.foo.example.co.uk".registeredDomain()` → `{"domain":"www.foo.example.co.uk","registered_domain":"example.co.uk","top_level_domain":"co.uk","subdomain":"www.foo"}`
-   `"co.uk".registeredDomain()` → `{"domain":"co.uk","top_level_domain":"co.uk"}` // a suffix alone
-   `"intranet.local".registeredDomain()` → `null` throws `CoercionRefused` // no known suffix

### `text.parseUserAgent()`

`(str)` → `{name: str, version: str, os: {name: str, version: str, full: str}, device: {name: str}}`

The browser, operating system and device a User-Agent header names, as an object of `name`, `version`, `os.name`, `os.version`, `os.full` and `device.name`. The rules are uap-core's, compiled in. `name` and `device.name` are `Other` where no rule matches, and `os` is null where none names an operating system. `os.full` is `os.name`, then `os.version` where there is one.

-   `"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36".parseUserAgent()` → `{"name":"Chrome","version":"120.0.6099.109","os":{"name":"Windows","version":"10","full":"Windows 10"},"device":{"name":"Other"}}`
-   `"curl/8.4.0".parseUserAgent().name` → `"curl"` // one part

## Exceptions and try

A calculation that cannot give a value leaves its field null and marks the record with an exception: text that is not a number, a division by zero, an overflow. Each exception has a kind, such as `CoercionRefused` or `DivideByZero`. A conversion that fails, such as `"five" as i64`, gives null and marks the record with a `CoercionRefused` exception. A parser's declared field that the parser cannot read as its type also gives null, and marks the record with an `UnparsableField` exception. An exception no `catch` takes stays on the record to the end of the pipeline, where its [exception answers](/docs/routing/exceptions#answers) apply.

A function's entry says when it throws one. Each kind belongs to a category. The group's [exception answers](/docs/routing/exceptions#answers) answer a kind or its category, and a route may override them. By default an encoding exception is ignored, since its value is kept, a delivery exception blocks, and every other kind is tagged.

-   `10 / 0` → `null` throws `DivideByZero`
-   `"many".parseInt()` → `null` throws `CoercionRefused`
-   `9223372036854775807 + 1` → `null` throws `Overflow`

| Kind | Category | When |
| --- | --- | --- |
| `Unparsable` | parse | A parser got nothing from the record: malformed JSON, an unterminated CSV quote, a Grok pattern that matched nothing. |
| `UnparsableField` | parse | One declared field could not be read as its declared type while the rest of the record parsed. That field is null. |
| `KeyContradiction` | parse | Two keys of a record contradict after flattening, such as the same key twice. One is kept. |
| `CoercionRefused` | parse | A value could not convert to the type asked for, as in `parseInt("many")`. The result is null. |
| `Overflow` | math | `i64` arithmetic overflowed. The result is null. |
| `DivideByZero` | math | A division by zero. The result is null, not Infinity. |
| `NaN` | math | A parse met `NaN`, `Infinity` or a number past `f64`'s range. That key has no value. |
| `LoopBudgetExceeded` | limits | A `while` loop hit its time budget with the record's condition still true. |
| `IndexOutOfRange` | limits | An array read or write, or a string read, named a position the value does not have. The read is null and the write does nothing. |
| `InvalidUtf8` | encoding | Bytes that did not decode under the source's encoding were replaced with U+FFFD. |
| `GuessedEncoding` | encoding | A source set to `auto` recognized neither UTF-8 nor UTF-16 and guessed the encoding. The text may well be right. |
| `Unmatched` | delivery | No route on the record's source took it. |
| `Unprocessable` | delivery | No pipeline could finish the record: one crashed on its batch, or the record does not fit the memory budget alone. |
| `Refused` | delivery | A destination refused the record. |

A `try` block stops at a record's first exception that loses a value. The write that failed does not happen, so the field keeps what it held. Its `catch` then runs on that record, with the exception cleared. `catch (e)` names the exception: `e.kind` is its kind and `e.message` its text. When one statement leaves several, `e.kind` is the first and `e.message` holds all their texts, and the `catch` clears them all. A `finally` block runs on every record that entered the `try`. A `try` needs a `catch`, a `finally` or both.

Code

```
try {
  count = parseInt(text);
} catch (e) {
  count = 0;
  problem = e.kind;
  detail = e.message;
} finally {
  checked = true;
}
```

Input

```
{
  "text": "many"
}
```

→

Output

```
{
  "checked": true,
  "count": 0,
  "detail": "text is \"many\", which could not convert to i64 in count",
  "problem": "CoercionRefused",
  "text": "many"
}
```

A typed catch, such as `catch (e: DivideByZero | NaN)`, runs on the records holding a kind it names and clears only those kinds. A `try` can have several catches, and it tries them in order. A catch with no type takes every kind left, and comes last. A kind no catch names stays on the record, for an outer `try` or its exception answer. `throw e` in a catch puts the exception back on the record, which goes on to the next `try` out, or to its exception answer when there is none. A catch reads `e` and cannot change it. A `try` catches every kind but the encoding and delivery ones. An encoding exception repairs its value instead, so the block goes on and no `catch` takes it.

## See also

-   [Code step](/docs/steps/code)
-   [Eval step](/docs/steps/eval)
-   [Exception answers](/docs/routing/exceptions#answers)