Turning Nested JSON Streams Into Compact Parquet Tables
API event logs, webhook payloads, and LLM evaluation runs are usually recorded as newline-delimited JSON (.jsonl / .ndjson), repeating every key name on every single line. Repacking those logs into Apache Parquet removes redundant keys, applies Snappy columnar compression, and lets DuckDB or Athena query fields dramatically faster.
This browser tool accepts both JSON arrays and JSONL streams, discovers the union of top-level keys across records (automatically marking optional keys as nullable), infers INT32, DOUBLE, BOOLEAN, or STRING column types, serializes nested sub-objects cleanly as JSON text, and outputs a standard PAR1 Parquet file locally without uploading log tokens.
Workflow & Local CLI Equivalents
- Drop a
.jsonl,.ndjson, or.jsonfile above, or click Try sample .jsonl (40 events). - Review skipped-line warnings (if any) and verify the inferred column types in the schema table.
- Click Download .parquet to save the compressed binary directly from browser memory.
When processing multi-gigabyte log archives offline on a server, run DuckDB or Python Polars locally:
# Convert JSONL to Parquet via DuckDB CLI
duckdb -c "COPY (SELECT * FROM read_json_auto('events.jsonl')) TO 'events.parquet' (FORMAT PARQUET);"
# Or with Python + Polars
python3 -c "import polars as pl; pl.read_ndjson('events.jsonl').write_parquet('events.parquet')"Practical Boundaries
- 10,000-record browser limit: Files stop at 10,000 records to prevent Chrome or Safari tabs from exhausting heap memory while compiling binary column chunks.
- Schema evolution across rows: If a field is an integer on row 1 and a string on row 50, type inference widens that column to
STRINGsafely rather than failing halfway through encoding.
Frequently Asked Questions
Does this accept both standard JSON arrays and JSONL / NDJSON logs?
Yes. Top-level [...] JSON arrays, single JSON objects, and newline-delimited JSONL/NDJSON files are all auto-detected and converted into the union of keys discovered across rows.
How are nested objects handled inside Parquet columns?
Nested child objects or arrays within top-level JSON records serialize cleanly as compact JSON strings inside STRING columns so DuckDB, Spark, and Postgres can read them directly via JSON functions.
What happens if one line in a JSONL export is corrupted or truncated?
Invalid lines are skipped without crashing the entire conversion, and the exact failing line number plus parser error are reported immediately above the schema table.