GZip support in ILP ingestion

pdeng · September 25, 2025, 1:05pm

Hi there,

InfluxDB supports gzip message to optimise ILP ingestion.

Can QuestDB support it? I cannot find anywhere to configure it, or do you have ETA for this feature?

Cheers,

Paul

nwoolmer · September 25, 2025, 1:05pm

Should come in next release:

github.com/questdb/questdb

feat(ilp): add gzip support to ILP/HTTP server

master ← nw_ilp_gzip

opened 11:16PM - 19 Sep 25 UTC

nwoolmer

+177 -15

ILP can be fairly inefficient with bandwidth. A 2x-3x increase in bandwidth woul…d not be unusual for ILP streaming, due to its text protocol nature. Upgrading the clients to a binary protocol is a WIP. In the meantime, this change allows the server to accept `gzip` encoded ILP. In a local test, this will reduce overall throughput, due to the overhead of encoding and decoding the compressed buffer: **TSBS, cpu-only** - Uncompressed, 1 thread: `568582 rows/sec` - Compressed, 1 thread: `295322 rows/sec` - Uncompressed, 8 threads: `1333824 rows/sec` - Compressed, 8 threads: `785233 rows/sec` However, in aggregate, this can be an advantage; for example, when receiving text-heavy data from multiple sources (logs, documents). The encoding cpu-overhead is masked on the client-side, and the reduction in bandwidth usage raises the ceiling for maximum throughput.

Topic		Replies	Views
What's the best way to upload an ILP file into QuestDB? Community question	15	473	September 19, 2024
Delayed Updates/Commits via Influx Line Protocol (ILP) in QuestDB Community config , question	9	342	February 3, 2025
Data Compression Info for storage evaluation Community storage , question	5	207	October 29, 2025
Is there anything I can check if my ILP CPU is at 100%? Community question	11	267	February 11, 2025
QuestDB stopped writing data to the table, ILP gives no errors, table is not suspended Community question	9	536	October 10, 2024

GZip support in ILP ingestion

Related topics