Large language model applications frequently move structured data through prompts: search results, tool outputs, product catalogs, event logs, table rows, customer records, benchmark fixtures, and intermediate agent state. JSON is a reliable programmatic interchange format, but its repeated keys, braces, quotes, and commas consume context budget that could otherwise carry more evidence or instructions.
Token-Oriented Object Notation, or TOON, is a compact text encoding of the JSON data model for LLM input. It keeps JSON’s object, array, primitive, and lossless round-trip semantics, but renders nested objects with indentation and renders uniform arrays of objects as CSV-like tables with an explicit row count and field header. The reference project describes it as a translation layer: keep JSON inside software systems, then encode selected payloads as TOON before placing them in model context.
The teachable mechanism is not just “shorter JSON.” TOON combines schema cues, length markers, deterministic field order, minimal quoting, and tabular row streaming. Those choices work best when a payload contains repeated records with the same primitive fields. They work poorly when the payload is deeply nested, highly irregular, or latency-sensitive enough that fewer tokens do not offset parser or model unfamiliarity costs.
TOON is designed to represent the same logical data that JSON represents: objects, arrays, strings, numbers, booleans, and null values. That compatibility matters because upstream services can continue producing ordinary JSON and downstream code can decode the model-facing form back into typed values without inventing a second application schema.
The largest savings come from arrays whose elements share the same primitive fields. Instead of repeating every key in every row, TOON writes one header such as users[2]{id,name,role}: and then writes row values. The length marker gives the model and decoder a simple structural check, while the field list carries a compact schema.
TOON removes most braces, brackets, repeated quotes, and repeated keys. It keeps enough delimiters to remain readable: indentation for nesting, array lengths for cardinality, field headers for row shape, and quotes only when a string would otherwise be ambiguous.
TOON is a prompt transport format. It should be generated from structured data, measured with the actual model tokenizer and task, and decoded or discarded after the model interaction. It is not a universal replacement for JSON, Parquet, CSV, or protocol buffers.
An encoder first normalizes input into JSON-like values. Objects become sorted key-value mappings, primitive arrays can be written inline, nested objects become indented blocks, and uniform arrays of objects become tabular arrays. The tabular form declares the row count and fields once, then streams values in that order.
{
"users": [
{"id": 1, "name": "Alice", "role": "admin"},
{"id": 2, "name": "Bob", "role": "user"}
]
}
users[2]{id,name,role}:
1,Alice,admin
2,Bob,userA practical encoder needs a policy for when tabular output is valid. A common rule is that every element must be an object with the same field set, and those fields must contain primitive values. If the elements differ, contain nested arrays, or contain nested objects, the encoder falls back to list-style output so the representation remains lossless.
function encodeArray(name, rows):
if rows are uniform objects with primitive fields:
fields = sorted(keys(rows[0]))
write name + "[" + len(rows) + "]{" + join(fields, ",") + "}:"
for row in rows:
write join([encodeScalar(row[field]) for field in fields], ",")
else:
write name + "[" + len(rows) + "]:"
for item in rows:
write "- " + encodeValue(item)Strings can often be left unquoted, which saves tokens and improves readability. The encoder must quote empty strings, values containing delimiters or control characters, leading or trailing whitespace, and strings that look like booleans, numbers, or null. Otherwise a decoder or model may confuse a literal string with a typed scalar.
Agents often call search, database, CRM, observability, or file-index tools that return many similarly shaped rows. TOON can shrink those results before they enter the transcript while preserving field names and row counts. That is especially useful when a model needs to inspect many candidates rather than one verbose object.
Retrieval and structured-answer evaluations need controlled inputs. A TOON fixture can hold repeated records compactly while still exposing enough structure for a model to answer field retrieval, filtering, aggregation, and structure-awareness questions. The reference benchmarks compare those outcomes across JSON, YAML, XML, CSV, and TOON.
When an application already has rows from SQL, analytics events, vector-search hits, or catalog records, TOON provides a readable alternative to dumping raw JSON arrays. The model sees one schema header plus rows, while application code keeps the canonical records in ordinary structured objects.
Use TOON at the model boundary, not as the authoritative in-memory representation. Convert typed application data to JSON-compatible values, encode TOON for the prompt, and keep the original object graph available for validation, retries, and audit trails.
def build_prompt(records):
payload = {
"task": "Find delayed shipments by region",
"shipments": records,
}
compact_context = toon_encode(payload)
return (
"Use the structured context below. Return JSON.\\n\\n"
"```toon\\n" + compact_context + "\\n```"
)Token savings do not automatically mean better throughput or accuracy. The TOON reference benchmarks report strong efficiency for mixed structured datasets, but also identify cases where compact JSON, CSV, or another format is better. Run task-specific measurements for token count, model accuracy, time to first token, total latency, and parse failures.
TOON can be used as input context, output format, or both. For production workflows, prefer structured outputs such as JSON Schema for final answers unless TOON output is a deliberate contract. If the model generates TOON, decode it with a strict parser and validate row counts, field counts, scalar types, and required fields before applying side effects.