Hi everyone !
I’m having an issue when querying our QuestDB (8.1.2)
A simple query that worked perfectly a few days ago now randomly started crashing and I don’t seem to understand why.
I’m not quite certain but I haven’t found any similar issues on the forum.
Thanks for your help in advance !
Hi Mat, do you mind trying to upgrade to the latest version? 8.1.4?
Hi !
I’ll see if we can upgrade, it’s not on my side but it shouldn’t be to hard, I’ll keep you updated.
In the meantime, do you have an idea as to why it’s happening ? Just out of curiosity !
Mat & Blue,
We are running into a very similar issue. We tried upgrading to 8.2.0 and it is still present.
In our case, the same query works fine in the Web console, but is repeatedly failing inside automated jobs.
Any advice for a fix?
H8 @chandler150,
We will need more info to debug this!
Hey @nwoolmer ,
Here’s some output from our code:
Building DataOccupancy table for ASC09L:
Looking for data available in QuestDB… Found no existing data. Done
Finding available ‘[’.Parquet’, ‘.parquet’]’ logs… Found 56628. 309 days 16:24:00 with 0 gaps to fill. Done
Finding available ‘[’.cms’]’ logs… C:\code/PCMC_LBCT/Python/questdb\QuestdbLink.py:320: UserWarning: Query exception occurred, so returning an empty Dataframe: (psycopg2.DatabaseError) unexpected filter error
LINE 1:
^
[SQL:
SELECT * FROM LogAttributes
WHERE Extension in (‘.cms’)
AND Deleted=False
AND Crane IN ('09L')
AND ((to_timezone(StartDateTime,'America/Los_Angeles') BETWEEN '2021-08-01T07:00:00.000000Z' AND '2024-12-02T17:35:59.949901Z')
OR (to_timezone(EndDateTime,'America/Los_Angeles') BETWEEN '2021-08-01T07:00:00.000000Z' AND '2024-12-02T17:35:59.949901Z'))
ORDER BY StartDateTime;
]
(Background on this error at: Error Messages — SQLAlchemy 2.0 Documentation)
warnings.warn(f"Query exception occurred, so returning an empty Dataframe: {error}")
Attempted Query: <
SELECT * FROM LogAttributes
WHERE Extension in (‘.cms’)
AND Deleted=False
AND Crane IN ('09L')
AND ((to_timezone(StartDateTime,'America/Los_Angeles') BETWEEN '2021-08-01T07:00:00.000000Z' AND '2024-12-02T17:35:59.949901Z')
OR (to_timezone(EndDateTime,'America/Los_Angeles') BETWEEN '2021-08-01T07:00:00.000000Z' AND '2024-12-02T17:35:59.949901Z'))
ORDER BY StartDateTime;
>
Found 0. Traceback (most recent call last):
Here’s the query that works fine in the Web console:
SELECT * FROM LogAttributes
WHERE Extension in (‘.cms’)
AND Deleted=False
AND Crane IN (‘09L’)
AND ((to_timezone(StartDateTime,‘America/Los_Angeles’) BETWEEN ‘2021-08-01T07:00:00.000000Z’ AND ‘2024-12-02T17:35:59.949901Z’)
OR (to_timezone(EndDateTime,‘America/Los_Angeles’) BETWEEN ‘2021-08-01T07:00:00.000000Z’ AND ‘2024-12-02T17:35:59.949901Z’))
ORDER BY StartDateTime;
and here it is from the docker logs:
2024-12-02T17:38:08.705967Z E i.q.g.e.QueryProgress err [id=-1, sql=WITH one as ( SELECT TransactionTime, Compute, row_number() OVER (PARTITION BY Compute ORDER BY Compute, TransactionTime) as rn FROM DataLineage WHERE OutputTarget = 'Parquet' AND Compute NOT IN ('CMSTest', 'DEVCMS19', 'DEVCMS20', 'DEVCMS21', 'DEVCMS22', 'DEVCMS23', 'DEVCMS24', 'DEVCMS25', 'DEVCMS26') AND TransactionTime BETWEEN '2024-12-02T16:38:08.441Z' AND '2024-12-02T17:38:08.441Z' ), two as (SELECT a.TransactionTime, a.Compute, datediff('s', a.TransactionTime, b.TransactionTime) as CycleTime FROM one a LEFT JOIN one b ON a.Compute = b.Compute AND a.rn = b.rn - 1) SELECT Date_trunc('hour',dateadd('h',1,TransactionTime)) as TransactionTime, (avg(CycleTime))/60 as AvgCycleTime FROM two ORDER BY TransactionTime SAMPLE BY 1h ALIGN TO CALENDAR TIME ZONE 'America/Los_Angeles';, principal=admin, cache=false, jit=false, time=5713487, msg=unexpected token [SAMPLE], errno=0, pos=794]
2024-12-02T17:38:19.055408Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=73, frameCount=235]
2024-12-02T17:38:19.055218Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=74, frameCount=235]
2024-12-02T17:38:19.055547Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=75, frameCount=235]
2024-12-02T17:38:19.055585Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=76, frameCount=235]
2024-12-02T17:38:19.055928Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=77, frameCount=235]
2024-12-02T17:38:19.056272Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=78, frameCount=235]
2024-12-02T17:38:19.056312Z E i.q.c.s.a.PageFrameReduceJob reduce error [error=
java.lang.NullPointerException
, id=876487, taskType=0, frameIndex=79, frameCount=235]
2024-12-02T17:38:19.063517Z E i.q.g.e.t.AsyncFilteredRecordCursor filter error [ex=
io.questdb.cairo.CairoException: [-1] unexpected filter error
at io.questdb.cairo.CairoException.instance(CairoException.java:316)
at io.questdb.cairo.CairoException.nonCritical(CairoException.java:121)
at io.questdb.griffin.engine.table.AsyncFilteredRecordCursor.fetchNextFrame(AsyncFilteredRecordCursor.java:333)
at io.questdb.griffin.engine.table.AsyncFilteredRecordCursor.hasNext(AsyncFilteredRecordCursor.java:203)
at io.questdb.griffin.engine.QueryProgress$RegisteredRecordCursor.hasNext(QueryProgress.java:319)
at io.questdb.cutlass.pgwire.modern.PGPipelineEntry.outCursor(PGPipelineEntry.java:1883)
at io.questdb.cutlass.pgwire.modern.PGPipelineEntry.outCursor(PGPipelineEntry.java:1854)
at io.questdb.cutlass.pgwire.modern.PGPipelineEntry.msgSync(PGPipelineEntry.java:680)
at io.questdb.cutlass.pgwire.modern.PGConnectionContextModern.syncPipeline(PGConnectionContextModern.java:1312)
at io.questdb.cutlass.pgwire.modern.PGConnectionContextModern.msgSync0(PGConnectionContextModern.java:1142)
at io.questdb.cutlass.pgwire.modern.PGConnectionContextModern.msgSync(PGConnectionContextModern.java:1138)
at io.questdb.cutlass.pgwire.modern.PGConnectionContextModern.msgQuery(PGConnectionContextModern.java:1111)
at io.questdb.cutlass.pgwire.modern.PGConnectionContextModern.parseMessage(PGConnectionContextModern.java:1239)
at io.questdb.cutlass.pgwire.modern.PGConnectionContextModern.handleClientOperation(PGConnectionContextModern.java:450)
at io.questdb.cutlass.pgwire.modern.PGWireServerModern$1.lambda$$0(PGWireServerModern.java:111)
at io.questdb.network.AbstractIODispatcher.processIOQueue(AbstractIODispatcher.java:199)
at io.questdb.cutlass.pgwire.modern.PGWireServerModern$1.run(PGWireServerModern.java:141)
at io.questdb.mp.Worker.run(Worker.java:152)
]
2024-12-02T17:38:19.075939Z E i.q.g.e.QueryProgress err [id=6556152, sql=`SELECT * FROM LogAttributes
WHERE Extension in (‘.cms’)
AND Deleted=False
AND Crane IN ('09L')
AND ((to_timezone(StartDateTime,'America/Los_Angeles') BETWEEN '2021-08-01T07:00:00.000000Z' AND '2024-12-02T17:35:59.949901Z')
OR (to_timezone(EndDateTime,‘America/Los_Angeles’) BETWEEN ‘2021-08-01T07:00:00.000000Z’ AND ‘2024-12-02T17:35:59.949901Z’))
ORDER BY StartDateTime;`, principal=admin, cache=false, jit=false, time=1007026188, msg=unexpected filter error, errno=-1, pos=0]