# QWP ready for production use?

**URL:** <https://community.questdb.com/t/qwp-ready-for-production-use/999>\
**Category:** Community\
**Created:** [June 10, 2026, 3:57pm UTC](https://community.questdb.com/t/qwp-ready-for-production-use/999 "2026-06-10T15:57:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vangreen](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.questdb.com/vangreen/32/384_2.png) [@Vangreen](https://community.questdb.com/u/Vangreen)\
**Post date:** [June 10, 2026, 3:57pm UTC](https://community.questdb.com/t/qwp-ready-for-production-use/999/1 "2026-06-10T15:57:18Z")

</div>

Hi,

I noticed mentions of QWP in the release notes for both the Java QuestDB client and the main repository. Is QWP already ready for production use?

I also saw a LinkedIn post some time ago mentioning a pub/sub feature.

> **[QuestDB Pub/Sub Handles 500 Subscribers at 115M Rows/s | Vlad Ilyushchenko 🇺🇦...](https://www.linkedin.com/posts/ilyushchenko_500-subscribers-each-one-consuming-every-activity-7459639562726731776-48Ah?utm_source=share&utm_medium=member_desktop&rcm=ACoAACqFUE0BXzFLFlqHzQ3FkJeMc1D2Z6F3L5w)**
>
> 500 subscribers. Each one consuming every row the producer publishes, with no slowdown for anyone. Aggregate egress: 115 M rows/s. Stale subscribers: 0.
> Fan-out should be free. In QuestDB's new pub/sub it is - the per-subscriber rate tracks the...

According to the public roadmap, it’s planned for Q3 2026, and it would be a perfect fit for our use case.

Will it be available in nightly builds before the official release so that we can evaluate and test it in advance?

Thanks!

---

<div class="post-metadata">

**Author:** ![nwoolmer](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.questdb.com/nwoolmer/32/24_2.png) [@nwoolmer](https://community.questdb.com/u/nwoolmer)\
**Post date:** [June 10, 2026, 3:58pm UTC](https://community.questdb.com/t/qwp-ready-for-production-use/999/2 "2026-06-10T15:58:36Z")

</div>

Hi @Vangreen ,

Official release will be in QuestDB 10.0, we are just refining some bits and rolling out the compatible clients. A questdb:nightly container is published every night based on the master branch, which does have QWP support.

Pub/sub is a fast follow, but not in OSS, only Enterprise.

---

<div class="post-metadata">

**Author:** ![Vangreen](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.questdb.com/vangreen/32/384_2.png) [@Vangreen](https://community.questdb.com/u/Vangreen)\
**Post date:** [June 12, 2026, 8:35am UTC](https://community.questdb.com/t/qwp-ready-for-production-use/999/3 "2026-06-12T08:35:54Z")

</div>

Thanks for answer.  
How about Arrow IPC

> <https://github.com/questdb/roadmap/issues/108>
>
> \## Summary
> 
> Native SQL drivers with a high-performance binary protocol for query… results, replacing the PostgreSQL wire protocol for applications that need maximum throughput. Uses the same binary protocol as the new ingestion layer (#105). The wire format is Arrow IPC compatible — received bytes are directly usable as Apache Arrow record batches with zero conversion overhead on the client.
> 
> \## Current State
> 
> \- Queries use PostgreSQL wire protocol
> \- Text-based encoding adds serialization overhead
> \- Limited by PG protocol constraints
> \- Clients must parse text, convert types, and build columnar arrays — redundant work
> 
> \## Design Goal: Zero Conversion Anywhere
> 
> The QuestDB Wire Protocol (QWP) encodes query results as Arrow IPC record batches on the wire. This is a deliberate design choice: the fastest possible data delivery means \*\*no conversion at any point in the path\*\*.
> 
> \`\`\`
> QuestDB server Client
> ┌────────────┐ ┌────────────────────┐
> │ Query │ │ │
> │ engine │ │ Received bytes │
> │ │ │ Arrow IPC on wire │ ═══════════ │
> │ ▼ │ ─────────────────────► │ ARE the Arrow │
> │ Serialize │ │ RecordBatch │
> │ to Arrow │ │ │ │
> │ IPC │ │ ├─► .to\_numpy() │ zero-copy
> └────────────┘ │ ├─► .to\_pandas() │ zero-copy
> │ ├─► polars.from\_ │ zero-copy
> │ │ arrow() │
> │ └─► torch.from\_ │ zero-copy
> │ numpy() │
> └────────────────────┘
> \`\`\`
> 
> The server does one serialization (internal format → Arrow IPC). The client does none. The received network buffer is memory-mapped directly as Arrow arrays. Column data pointers reference the receive buffer — no parsing loop, no field-by-field deserialization, no intermediate representation.
> 
> This matters because:
> \- Arrow IPC is the lingua franca of the data ecosystem — Pandas, Polars, DuckDB, Spark, PyTorch, TensorFlow all consume Arrow natively
> \- Any Arrow-aware tool can consume QWP results without a QuestDB-specific driver implementation
> \- The conversion cost that normally sits in every client driver is eliminated entirely
> 
> \## What This Enables
> 
> | Feature | Description |
> |---------|-------------|
> | \*\*Arrow IPC wire format\*\* | Query results are Arrow record batches on the wire — no client-side conversion |
> | \*\*Native drivers\*\* | Purpose-built drivers for Python, Java, Go, etc. — thin wrappers, not parsers |
> | \*\*True zero-copy\*\* | Client memory-maps received bytes directly as Arrow arrays |
> | \*\*Unified protocol\*\* | Same binary format for both ingestion (#105) and egress |
> | \*\*Cursor-based streaming\*\* | Stream large result sets as a sequence of Arrow record batches without loading everything into memory |
> | \*\*Ecosystem compatibility\*\* | Any Arrow-aware tool (Polars, DuckDB, Spark, Pandas) can consume results without custom integration |
> 
> \## Benefits
> 
> \- \*\*Fastest possible egress\*\* — One serialization on the server, zero on the client. No conversion overhead anywhere.
> \- \*\*Lower bandwidth\*\* — Binary columnar format vs text-based PG protocol
> \- \*\*Consistent API\*\* — Single protocol for read and write operations
> \- \*\*Better type fidelity\*\* — Arrow types preserve QuestDB semantics without text conversion edge cases
> \- \*\*Thin drivers\*\* — Client drivers become Arrow IPC readers with QWP framing, not full parsers. Less code, fewer bugs, easier to maintain across languages.
> 
> \## Use Cases
> 
> \### High-frequency data retrieval
> 
> Trading systems, real-time dashboards, and monitoring applications that poll QuestDB frequently benefit from the reduced serialization overhead and compact binary format.
> 
> \### Bulk data export and ETL pipelines
> 
> Large-scale data extraction for warehousing, compliance reporting, or analytics. Cursor-based streaming of Arrow record batches keeps memory bounded even for multi-billion row exports. Downstream tools (Spark, Polars, DuckDB) consume the batches directly.
> 
> \### ML training data pipeline
> 
> Native drivers are the egress foundation for ML training workflows in physical AI and other domains. Since the wire format is Arrow IPC, the path from database to tensor is zero-copy end to end:
> 
> \`\`\`python
> import questdb
> import torch
> from torch.utils.data import DataLoader, IterableDataset
> 
> class QuestDBDataset(IterableDataset):
> def \_\_init\_\_(self, client, episode\_query, data\_query):
> self.client = client
> self.episode\_query = episode\_query
> self.data\_query = data\_query
> 
> def \_\_iter\_\_(self):
> # Find matching episodes
> episodes = self.client.query(self.episode\_query)
> random.shuffle(episodes)
> 
> for ep in episodes:
> # Stream episode data as Arrow record batches
> # Wire bytes ARE Arrow — no conversion
> # Compound sort keys (#119) make per-episode reads sequential
> batches = self.client.query\_streaming(
> self.data\_query,
> params={"episode\_id": ep.episode\_id},
> batch\_size=1024
> )
> for batch in batches:
> # Arrow → NumPy → PyTorch (zero-copy all the way)
> positions = torch.from\_numpy(batch.column("positions").to\_numpy())
> velocities = torch.from\_numpy(batch.column("velocities").to\_numpy())
> torques = torch.from\_numpy(batch.column("torques").to\_numpy())
> yield {"positions": positions, "velocities": velocities, "torques": torques}
> 
> dataset = QuestDBDataset(
> client=questdb.connect("questdb://central:9000"),
> episode\_query="""
> SELECT episode\_id FROM episodes
> WHERE outcome = 'success' AND model\_version = 'v2.3'
> """,
> data\_query="""
> SELECT ts, positions, velocities, torques
> FROM joint\_telemetry
> WHERE episode\_id = :episode\_id
> ORDER BY ts
> """
> )
> 
> dataloader = DataLoader(dataset, batch\_size=32, num\_workers=4)
> 
> for batch in dataloader:
> loss = model(batch)
> loss.backward()
> \`\`\`
> 
> Why this matters:
> \- \*\*No export step\*\* — train directly against QuestDB, no intermediate Parquet files
> \- \*\*Always fresh\*\* — new episodes from today's robot runs are available in the next training epoch
> \- \*\*No storage duplication\*\* — data lives in QuestDB, not copied to a separate training store
> \- \*\*Zero-copy end to end\*\* — Arrow IPC on wire → Arrow RecordBatch → NumPy view → PyTorch tensor, no serialization at any step
> \- \*\*Streaming\*\* — cursor-based delivery keeps memory bounded regardless of dataset size
> \- \*\*Parallelizable\*\* — multiple DataLoader workers query different episodes concurrently
> 
> For multi-modal training data (scalar telemetry + LiDAR + camera), this driver provides the scalar egress layer. File resolution is handled by the multi-modal data management client (#122), which uses the native driver for metadata queries and fetches files from object storage.
> 
> \### Foundation for distributed queries
> 
> Database links (#110) use QWP for remote-to-remote communication. Arrow IPC on the wire means the receiving QuestDB instance can process remote result batches without deserialization — it reads the Arrow buffers directly into its query engine.
> 
> \## Related
> 
> \- #105 — Binary ingestion protocol (same binary format, unified for ingestion and egress)
> \- #119 — Compound sort keys (efficient per-episode, per-series reads that the driver streams)
> \- #122 — Multi-modal data management (uses native driver for metadata queries, extends with file resolution)
> \- #110 — Database links (uses QWP/Arrow IPC for remote queries — zero-copy between instances)
> \- Supersedes #51 (Apache Arrow ADBC) — QWP delivers the same zero-copy goals with tighter QuestDB integration, and the Arrow IPC wire format provides equivalent ecosystem compatibility

This will avaliable in OSS version of QuestDB?