Token-Oriented Object Notation (TOON) is a compact, human-readable serialization format designed specifically for Large Language Model (LLM) prompts. It was created to reduce the verbosity of JSON when sending structured data to AI models. As the official spec notes, TOON “is a compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage”. In practice, TOON can convey the same information as JSON but typically at roughly half the token count. This was motivated by the high token costs of JSON in LLM contexts (GPT-like models charge per token).

The format’s purpose is not to replace JSON in APIs or storage, but rather to serve as a more token-efficient representation for LLM inputs. It is intended as a lossless, drop-in representation of JSON when used in prompts, not as a general data interchange format. TOON was created by Johann Schopplich (et al.) in 2025 and has since gained traction in the AI community. In summary, TOON was developed to cut prompt size and cost by removing redundant JSON punctuation, leveraging indentation and tabular structures, while preserving the data’s semantics for the model.

Technical Format and Syntax

TOON’s syntax is a blend of YAML-like indentation and CSV-like tabular rows. The key features are: minimal punctuation, explicit length markers, and reuse of field names for uniform arrays. Formally, TOON omits JSON braces/brackets and most quotes, using whitespace and simple markers instead. Nested objects are represented by indenting under a key (like YAML). Arrays of uniform objects are written in tabular form: the header declares the length and field names once, then each row lists values comma-separated. Mixed or non-uniform arrays fall back to a list format with dash (-) markers.

For example, consider a simple JSON object and its TOON equivalent:

JSON Example	TOON Format (token-efficient)
json<br>{"id":1,"name":"Ada"}	toon<br>id: 1<br>name: Ada

JSON { "id": 1, "name": "Ada" } becomes two lines id: 1 and name: Ada in TOON, removing braces and quotes. For an array of objects:

JSON Example	TOON Format
json<br>{"users":[ {"id":1,"name":"Alice"}, {"id":2,"name":"Bob"} ]}	toon<br>users[2]{id,name}:<br> 1,Alice<br> 2,Bob

Here, the JSON list of two user objects is encoded as users[2]{id,name}: (the header indicates 2 rows with fields id,name), followed by two data rows 1,Alice and 2,Bob. Because each object has the same fields, those names need only appear once. If values contain commas or special chars, TOON will quote them (e.g. "hello, world" → "hello, world").

Do Check our Free JSON to TOON converter

Key syntax points:

• Objects: Each key-value pair appears as key: value on its own line. Nested objects increase indent.
• Primitive arrays (inline): If an array contains primitives (strings/numbers), it’s shown inline: e.g. tags: [“a”,”b”] becomes tags[2]: a,b (length [2] with comma-separated values).
• Tabular arrays: If an array contains uniform objects (same keys, all values primitive), TOON writes a header field1,field2,… in braces and then each object’s values as a CSV row. Example:{"items":[{"sku":"A1","qty":2}, {"sku":"B2","qty":1}]} becomes:items[2]{sku,qty}: A1,2 B2,1 (see).
• Lists/mixed arrays: If an array is mixed or its objects differ, TOON uses a dash list: e.g., [1, {"a":1}, "x"] →items[3]: - 1 - a: 1 - x (the first field of each object may follow the - on the same indent).
• Length markers: Every array header includes a length [N], which helps the model validate the number of items (e.g. users[3]{id,name}: if 3 users).
• Indentation: TOON uses 2-space indent by convention, but any consistent indent works.
• Minimal quotes: Only values containing delimiters, spaces, or brackets need quotes. E.g., note: "hello, world" or items[2]: "true",true (one string, one boolean).

These syntax rules ensure the output is still human-readable while dramatically shortening token count compared to JSON. (See the Syntax Cheatsheet for more examples.)

Token Efficiency

TOON’s headline claim is large token savings over JSON. Benchmarks provided by the creators show roughly 30–60% fewer tokens when encoding typical JSON data. Token counts are measured with a GPT-style tokenizer; savings vary by model but are consistently significant for uniform, tabular data. For example, the official benchmarks report:

Dataset	JSON tokens	TOON tokens	Reduction
GitHub Repositories (100 rows)	15,145	8,745	–42.3%
Daily Analytics (180 days)	10,977	4,507	–58.9%
E-Commerce Order (small)	257	166	–35.4%
Total (combined)	26,379	13,418	–49.1%

In words, a realistic mix of data saw TOON using only 13,418 tokens versus JSON’s 26,379 (about 49% fewer). On just the “Daily Analytics” dataset, TOON used 4,507 tokens vs JSON’s 10,977 (59% reduction). This roughly halves the prompt size. In an independent test with OpenAI’s GPT-4o tokenizer, a developer found a 56% drop in prompt tokens: JSON input was 1,344 tokens, while the TOON equivalent was 589 tokens.

Another study compared model accuracy vs tokens: passing the same tabular data to GPT-4.1 (nano) resulted in JSON using 66,396 tokens vs TOON’s 21,518 (about 68% savings). (TOON’s accuracy was slightly lower than JSON’s, but this confirmed dramatic token cuts.) In short, TOON cuts cost and latency: fewer tokens means cheaper API calls and faster LLM processing. Industry blogs emphasize potential savings (e.g. up to $90K annually for heavy users).

Taken together, both official and third-party benchmarks confirm that TOON can reduce prompt token usage by roughly 30–60% on large, structured payloads.

Use Cases

TOON is most beneficial anytime you feed large structured data to an LLM. Common scenarios include:

• Chatbots and agents with data: When a chatbot or AI agent must process a list of records, commands, or files, TOON shrinks the input. For example, an AI “agent” that outputs a series of file-write commands can use TOON to list those commands compactly.
• Multi-step workflows: Complex prompts often carry tables of information between steps. TOON reduces the accumulated context size, speeding up each step.
• Retrieval-Augmented Generation (RAG): In systems retrieving JSON documents for LLM context, converting them to TOON saves tokens on each API call.
• Data analysis prompts: Supplying spreadsheets or log data in prompts (e.g. “Here is 6 months of metrics data…”) becomes cheaper with TOON’s tabular style.
• Prompt templates and datasets: Developers preparing prompt templates or fine-tuning data with JSON structures can use TOON to pack more data per token.

As one industry author notes, “If you’re building anything that regularly feeds structured data into LLMs – for example, chatbots, AI-assisted code generation, or multi-step workflows – TOON can significantly reduce prompt size.”. In essence, prompt compression is the key use case: any LLM workload where token counts and cost are a concern. (Conversely, for small or rare payloads, TOON’s benefit is minor.)

Tooling and Ecosystem

TOON already has a growing ecosystem of tools and libraries:

• Official TypeScript library: The core implementation is in TypeScript (npm package @toon-format/toon). Install via npm install @toon-format/toon or pnpm add @toon-format/toon. This library provides encode(data) and decode(toonString) APIs.
• CLI tool: A command-line interface is included (@toon-format/cli). For example: npx @toon-format/cli input.json -o output.toon will auto-encode JSON to TOON. It auto-detects by file extension and supports flags (delimiter, length-marker, stats, etc). The Python implementation also offers a toon CLI (see below).
• Python library: A Python implementation (toon) is available via PyPI (pip install python-toon). It claims 100% compatibility with the spec and similarly encodes/decodes data. Usage is from toon import encode; encode(your_json_data).
• Online converters: Several free web tools let you paste JSON and get TOON instantly. For example, JSON2TOON (json2toon.com) and JSONtoON (jsontoon.com) advertise “30–60% token reduction” and instant conversion. They run in your browser (no server upload) and show token-saving stats.
• Interactive playgrounds: CuriouslyChase’s Format Tokenization Playground lets you compare token usage of JSON, YAML, CSV, TOON, etc, on your data. This helps developers visualize the savings.
• Language support: Beyond JS/TS and Python, the community has ports to many languages. Official or community libraries exist for C# (.NET ToonSharp), C++ (ctoon), Java (JToon), Go (gotoon), PHP (toon-php), Ruby, Swift (TOONEncoder), Elixir, Dart (toon_format on pub.dev), and more. For Dart/Flutter, the toon_format package on pub.dev (version 0.1.0) provides encoding/decoding.
• Specification and tests: The full TOON specification (v1.3) is public on GitHub, along with language-agnostic conformance tests. This aids library authors in ensuring compatibility.
• Ecosystem tools: Additional utilities like VSCode extensions, GitHub Actions, or integration into LLM pipelines may appear as the format gains awareness.

In short, TOON support spans multiple languages and platforms. Developers can embed TOON conversion anywhere: in code (via libraries), in build pipelines, or interactively with web tools. The ecosystem is actively growing to make TOON accessible in diverse workflows.

Installation and Usage

Installing TOON libraries:

• Node.js/TypeScript:
  Install via npm/PNPM/Yarn (e.g. npm install @toon-format/toon). Then in code:import { encode, decode } from "@toon-format/toon"; const data = { users: [ {id:1,name:"Alice"}, {id:2,name:"Bob"} ] }; const toonText = encode(data); // toonText is: // users[2]{id,name}: // 1,Alice // 2,Bob

To convert back: decode(toonText) yields the original object.

• Python:
  Install with pip install python-toon. Then:from toon import encode, decode data = {"name":"Ada", "age":30} print(encode(data)) # Output: # name: Ada # age: 30 For a list of objects:users = [ {"id":1,"name":"Alice"},{"id":2,"name":"Bob"} ] print(encode(users)) # Output: # [2,]{id,name}: # 1,Alice # 2,Bob

The Python package also provides a CLI: e.g. toon input.json -o output.toon (auto-detects mode by file extension).

CLI usage:

• TypeScript CLI: Use the npm CLI via npx:
  • Encode JSON file to TOON:npx @toon-format/cli input.json -o output.toon
  • Decode TOON back to JSON:
    npx @toon-format/cli data.toon -o output.json
• You can pipe via stdin, change delimiters, view token stats, etc.
• Python CLI: Similarly, after pip install python-toon, use the toon command (as in the Quick Start above).

Incorporating into LLM pipelines: Once TOON text is generated, include it in the prompt as a fenced code block. For example:

```toon
users[2]{id,name}:
  1,Alice
  2,Bob
```

Labeling it as toon (toon) helps the model recognize the format. As the docs advise, “Wrap your encoded data in a fenced code block (label it toon for clarity). The indentation and headers are usually enough – models treat it like familiar YAML or CSV”. When asking the model to produce TOON, it’s best to show an example header and specify the rules (indent, [N] counts, etc).

Integration tips: TOON conversion can be inserted into existing code with minimal change. For example, in Node.js you might require('@toon-format/toon') where you would otherwise JSON.stringify. The TOON output can be sent directly to any LLM API (OpenAI, Anthropic, etc) as part of the prompt. The JSON ↔ TOON encoding is lossless, so you can always decode model output back into standard JSON if needed.

Limitations and Considerations

While TOON excels on repetitive, tabular data, it has several constraints:

• Uniform structure requirement: TOON’s greatest savings come when arrays contain uniform objects with the same keys and primitive values. In this case it uses the compact tabular form. If your data is highly mixed, nested, or objects have varying fields, TOON falls back to a more verbose list format. As the spec warns, “TOON’s tabular format requires arrays of objects with identical keys and primitive values only. When this doesn’t hold… TOON switches to list format where JSON can be more [efficient].”. In other words, non-uniform data may yield less token saving, and could even be longer than JSON in worst cases.
• Deep nesting overhead: Deeply nested JSON becomes deeply indented TOON. The extra indentation lines can sometimes offset the removal of braces. In practice, if your JSON is very hierarchical, JSON or another format (like YAML) might use fewer tokens. For example, one guide notes that complex nested hierarchies can make TOON longer, so it’s “not ideal for deeply nested structures.”. Benchmark examples on nested data even showed a slight increase in tokens for TOON vs JSON.
• Limited to LLM contexts: TOON is not designed for general-purpose data interchange. It is not a drop-in replacement for JSON in APIs, databases, or storage. It lacks some JSON features (e.g. custom types, comments) and its syntax (e.g. length markers) has no meaning outside LLM prompts. Only use TOON where both sender and receiver (i.e. the LLM) understand the format.
• Model familiarity: Most LLMs have not been explicitly trained on TOON, so you must teach them the format by example or instruction. The “guardrails” like [N] lengths and field lists help, but model accuracy with TOON can differ. One evaluation found GPT-4 accuracy with TOON was slightly lower than with JSON (e.g. 47.5% vs 52.3% on one table task). This is expected until models see more TOON data. When generating TOON output, be explicit in prompts (show schema and instruct on indentation/format) to minimize errors.
• Edge cases and quoting: If a value contains the delimiter (comma) or a colon, TOON will quote it, adding tokens. Long string values or texts may bloat a TOON line. Also, some JSON types (like null or empty arrays) have special handling (nulls are omitted, empty arrays still show key[0]:) which may confuse novice users. See the spec for details.
• CSV comparison: It’s worth noting that for flat tabular data, pure CSV is even more compact than TOON (since CSV has no array length or field-list overhead). The spec remarks: “For flat tabular data, CSV is more compact. TOON adds minimal overhead to provide explicit structure and validation…”. So if you only need a simple table without nested context, CSV might be better – TOON’s value is in mixing tables with nested context.
• Readability vs tokens: TOON sacrifices some human readability (e.g. lost braces) for tokens, though it remains fairly readable. Just be aware that it may look odd to those used to JSON, and misuse (extra/missing indents, wrong length marker) can cause confusion. Always validate TOON output with the official decoder or tests.

In summary, TOON is a niche format optimized for specific scenarios. It works best on flat or mildly nested, uniform data where repeated keys drive cost. It is not ideal for highly irregular JSON, deeply nested documents, or as a generic file format. As with any tool, weigh the token savings against these limitations before adoption.

---

Example Syntax Comparison (JSON vs TOON):

JSON (input)	TOON (token-friendly output)
json <br>{"users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]}	toon <br>users[2]{id,name}:<br> 1,Alice<br> 2,Bob
json <br>{"settings":{"theme":"dark","notifications":true}}	toon <br>settings:<br> theme: dark<br> notifications: true

Table: Two examples showing how TOON removes JSON’s braces/brackets and quotes while explicitly listing array lengths and fields (based on the TOON spec).

Example Token Savings: (from official benchmarks)

Dataset	JSON tokens	TOON tokens	Savings
GitHub Repos (100 items)	15,145	8,745	–42.3%
Daily Analytics (180 days)	10,977	4,507	–58.9%
E-Commerce Order (small)	257	166	–35.4%

Table: Token counts for sample datasets (JSON vs TOON). TOON consistently uses far fewer tokens, cutting prompt size roughly in half.

Summary: TOON is a specialized format: when your LLM prompt carries significant structured data, and token costs or context size are critical, it can offer major savings. It comes with robust tooling (libraries, CLI, converters) and an active community, but users must apply it judiciously given its constraints. For LLM-centric pipelines, however, TOON provides a powerful way to compress structured input without losing fidelity.