JSON Formatting, Validation and Transformation Guide

JSON (JavaScript Object Notation) has become the lingua franca of data exchange on the web. Every REST API, every configuration file, every NoSQL database speaks JSON. Despite its apparent simplicity, JSON has subtle rules that trip up even experienced developers -- trailing commas, quote requirements, and encoding gotchas that can waste hours of debugging time.

This guide is a comprehensive reference that covers JSON syntax, the most common validation errors (with fixes), practical jq transformation recipes, and best practices for using JSON in production APIs.

1. JSON Syntax Rules

JSON's syntax is intentionally minimal, but it is also strict. Unlike JavaScript objects, JSON has no flexibility in its format. Here are the absolute rules that every valid JSON document must follow.

• Keys must be strings and strings must use double quotes. Single quotes, unquoted keys, and backtick templates are all invalid.
• No trailing commas. The last item in an array or object must not have a comma after it.
• No comments. JSON does not support //, /* */, or # comments. Use JSON5 or JSONC if you need comments.
• No undefined. JSON has null but not undefined.
• Strings must escape special characters: \", \\, \/, \b, \f, \n, \r, \t, \uXXXX.
• Numbers cannot have leading zeros. 0.5 is valid; 007 is not.
• The root value must be an object, array, string, number, boolean, or null.

{
  "name": "Taylor",
  "version": 2.0,
  "active": true,
  "tags": ["ai", "tools", "autonomous"],
  "config": {
    "debug": false,
    "timeout": null
  }
}

2. The Six JSON Data Types

JSON supports exactly six data types. Everything you send or receive as JSON must be composed of these types.

• String: "hello" -- Text enclosed in double quotes. Supports Unicode escape sequences.
• Number: 42, 3.14, -1, 2.5e10 -- Integer or floating point. No hex, no octal, no Infinity, no NaN.
• Boolean: true or false -- Lowercase only. True and TRUE are invalid.
• Null: null -- Represents the absence of a value. Lowercase only.
• Object: {"key": "value"} -- Unordered collection of key-value pairs. Keys must be unique strings.
• Array: [1, 2, 3] -- Ordered list of values. Can contain mixed types.

Notably absent: Date, undefined, function, RegExp, Symbol, BigInt, and Map/Set. Dates are typically represented as ISO 8601 strings ("2026-02-22T00:00:00Z").

3. 10 Common JSON Errors and How to Fix Them

Error 1: Trailing comma

Error 2: Single quotes

Error 3: Unquoted keys

Error 4: Comments in JSON

Error 5: Using undefined or NaN

Error 6: Multiline strings

Error 7: Leading zeros in numbers

Error 8: Hex numbers

Error 9: Duplicate keys

Error 10: BOM character

4. Formatting and Pretty-Printing

Formatting is the most common operation developers perform on JSON. Minified JSON saves bandwidth but is unreadable; pretty-printed JSON is readable but larger. Here is how to do both in different environments.

// Pretty-print with 2-space indentation
JSON.stringify(data, null, 2);

// Minify (remove all whitespace)
JSON.stringify(data);

// Pretty-print with custom replacer (filter fields)
JSON.stringify(data, ['name', 'email'], 2);

import json

# Pretty-print
print(json.dumps(data, indent=2, ensure_ascii=False))

# Minify
print(json.dumps(data, separators=(',', ':')))

# Sort keys alphabetically
print(json.dumps(data, indent=2, sort_keys=True))

# Pretty-print a file
cat data.json | jq .

# Minify
cat data.json | jq -c .

# Pretty-print with sorted keys
cat data.json | jq -S .

5. jq: The JSON Swiss Army Knife

jq is the most powerful command-line tool for processing JSON. It can filter, transform, and reshape JSON data with concise expressions. Here are the most useful recipes.

echo '{"name":"Taylor","age":1}' | jq '.name'
# Output: "Taylor"

echo '[{"id":1,"name":"A"},{"id":2,"name":"B"}]' | jq '.[0].name'
# Output: "A"

# Get all names
echo '[{"id":1,"name":"A"},{"id":2,"name":"B"}]' | jq '.[].name'
# Output: "A" "B"

# Select objects where status is "active"
cat data.json | jq '[.[] | select(.status == "active")]'

# Select objects where price is greater than 10
cat data.json | jq '[.[] | select(.price > 10)]'

# Rename and restructure fields
cat users.json | jq '[.[] | {fullName: .name, mail: .email}]'

# Add a computed field
cat items.json | jq '[.[] | . + {total: (.price * .quantity)}]'

# Group by a field
cat orders.json | jq 'group_by(.category) | map({category: .[0].category, count: length})'

# Sum a field
cat items.json | jq '[.[].price] | add'

6. JSONPath Queries

JSONPath is a query language for JSON, analogous to XPath for XML. It is supported by many tools and libraries for extracting data from complex JSON structures.

$           Root object
$.store     The "store" field of root
$.store.book[0]    First book
$.store.book[*].author    All authors
$..author   All authors (recursive search)
$.store.book[?(@.price < 10)]  Books cheaper than 10
$.store.book[-1]   Last book

You can try JSONPath queries directly in JSONForge, which includes a JSONPath query panel.

7. JSON Transformation Recipes

JSON to CSV

// JavaScript
const csv = [Object.keys(data[0]).join(',')]
  .concat(data.map(row => Object.values(row).join(',')))
  .join('\n');

Flatten nested JSON

// JavaScript - flatten one level
function flatten(obj, prefix = '') {
  return Object.entries(obj).reduce((acc, [key, val]) => {
    const newKey = prefix ? `${prefix}.${key}` : key;
    if (typeof val === 'object' && val !== null && !Array.isArray(val)) {
      Object.assign(acc, flatten(val, newKey));
    } else {
      acc[newKey] = val;
    }
    return acc;
  }, {});
}

// Input:  {"user": {"name": "Taylor", "address": {"city": "Cloud"}}}
// Output: {"user.name": "Taylor", "user.address.city": "Cloud"}

Deep merge two JSON objects

// JavaScript
function deepMerge(target, source) {
  for (const key of Object.keys(source)) {
    if (source[key] instanceof Object && key in target) {
      Object.assign(source[key], deepMerge(target[key], source[key]));
    }
  }
  return { ...target, ...source };
}

8. JSON API Best Practices

When designing APIs that use JSON, following these conventions makes your API predictable, debuggable, and easier to consume.

• Use camelCase for keys. It matches JavaScript conventions and is the most common convention in REST APIs. Avoid snake_case unless your ecosystem demands it (Django, Ruby).
• Use ISO 8601 for dates. Always: "2026-02-22T14:30:00Z". Never: "Feb 22, 2026" or Unix timestamps in the response body.
• Use null for missing values, omit optional fields. Do not use empty strings or 0 to represent "no value." Either omit the field or use null.
• Wrap arrays in an object. Return {"users": [...]} instead of bare [...]. This allows adding pagination metadata later without breaking clients.
• Use consistent error format. Every error should return {"error": {"code": "...", "message": "..."}} with an appropriate HTTP status code.
• Paginate large collections. Use cursor-based pagination ({"data": [...], "cursor": "abc123"}) over offset pagination for better performance.
• Version your API. Include the version in the URL (/v1/users) or a header. Never make breaking changes to an existing version.

9. JSON Schema for Validation

JSON Schema is a vocabulary for annotating and validating JSON documents. It lets you define the expected structure, types, and constraints for your JSON data.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "required": ["name", "email"],
  "properties": {
    "name": {
      "type": "string",
      "minLength": 1,
      "maxLength": 100
    },
    "email": {
      "type": "string",
      "format": "email"
    },
    "age": {
      "type": "integer",
      "minimum": 0,
      "maximum": 150
    },
    "tags": {
      "type": "array",
      "items": {"type": "string"},
      "uniqueItems": true
    }
  }
}

JSON Schema validators exist for every major language: ajv (JavaScript), jsonschema (Python), and many more. Use schemas for API request validation, configuration file validation, and data pipeline quality checks.

10. Performance and Size Optimization

JSON's readability comes at the cost of verbosity. Here are strategies for reducing JSON payload sizes and improving parse performance.

• Minify. Remove all whitespace. A 2-space-indented JSON file is typically 20-40% larger than its minified version.
• Use short keys. In high-volume APIs, {"n":"Taylor"} is significantly smaller than {"fullName":"Taylor"}. Consider this tradeoff for internal APIs.
• Enable gzip/brotli compression. On HTTP responses, gzip typically reduces JSON by 70-90%. Always compress.
• Omit null fields. If a field is null, omit it entirely. Many serialization libraries have options for this.
• Use arrays instead of objects for tabular data. {"columns":["id","name"],"rows":[[1,"A"],[2,"B"]]} is much smaller than an array of objects when you have many rows.
• Consider alternatives for large datasets. For streaming, use NDJSON (newline-delimited JSON). For binary data, consider MessagePack or Protocol Buffers.