JSON TransformArray
unique
Remove duplicate elements from an array
unique
Returns a new array with duplicates removed. The first occurrence of each value wins; subsequent duplicates are dropped.
Options
| Option | Type | Required | Default | Description |
|---|---|---|---|---|
key | string | No | — | For object arrays, dedupe by this field instead of by full structural equality. |
Example
Input
[
{ "id": 1, "label": "a" },
{ "id": 2, "label": "b" },
{ "id": 1, "label": "a-dup" }
]Operation
{ "type": "unique", "options": { "key": "id" } }Output
[
{ "id": 1, "label": "a" },
{ "id": 2, "label": "b" }
]In a Circuit Step
{
"name": "dedupe_by_id",
"description": "Drop duplicate records, keyed on id",
"function": "circuit.core.transform.json",
"input": {
"source": "${{ records }}",
"operations": [
{ "type": "unique", "options": { "key": "id" } }
]
},
"outputs": [
{
"name": "unique_records",
"description": "Records deduplicated on id",
"value": "${{ dedupe_by_id.output.transformed_result }}"
}
]
}Notes & gotchas
- Errors with
unique: expected arrayif the input is not an array. - Without
key, equality is determined by the JSON string representation of each element, so{ "a": 1, "b": 2 }and{ "b": 2, "a": 1 }may compare as different (key order is preserved byserde_json::Mapinsertion order). - With
key, missing keys fall back to the element's full string representation — i.e., a missing key still de-dupes per-element. - The output preserves original order (first occurrence wins).