How can I ensure JIT is enabled on the SQL queries? I have set the server.conf property but still find that its not working as intended. Also, I see that jit flag is set to false for all queries that I am sending from my python code and the Web QuestDB tool.
We are running the DB on x86 machine only. Also, I do not find this being set for any of the 1000’s of queries that we run which have where clauses. I have even enabled the debug option.
Sample output from logs:
2025-01-28T14:43:15.651167Z I i.q.g.e.QueryProgress exe [id=1991, sql=`select count() from quote where trade_date='2025-01-27'`, principal=admin, cache=false, jit=false]
2025-01-28T14:43:15.651203Z I i.q.c.TableReader open partition [path=/quote~57/2025-01-27.3944, rowCount=1162663, partitionIndex=83, partitionCount=86]
2025-01-28T14:43:15.652176Z I i.q.g.e.QueryProgress fin [id=1991, sql=`select count() from quote where trade_date='2025-01-27'`, principal=admin, cache=false, jit=false, time=1014764]
2025-01-28T14:43:15.652181Z I i.q.c.h.p.JsonQueryProcessorState [7082401209047] timings [compiler: 5790695, count: 0, execute: 6832354, q=`select count() from quote where trade_date='2025-01-27'`]
BTW, is there any documentation on how I can read the logs?
explain select trade_date, ticker, close from daily
where
currency = 'USD' and close is not null
and ticker = 'AAPL'
and close > 100
SelectedRecord
Filter filter: (currency=‘USD’ and close is not null and ticker=‘AAPL’ and 100<close)
PageFrame
Row forward scan
Frame forward scan on: daily
I ran this from the questDB webUI and see the following in the logs:
2025-01-29T02:39:17.350349Z I i.q.g.e.QueryProgress fin [id=3278, sql=`explain select trade_date, ticker, close from daily
where
currency = 'USD' and close is not null
and ticker = 'AAPL'
and close > 100`, principal=admin, cache=false, jit=false, time=17385286]
Can you can see the http message is setting jit to false - does this override the config setting? How do we set the WebUI to pass jit to true?
Regarding logs: I meant how do we interpret the log output as shown above. What does “fin” or “exe” mean? What does time value after the jit flag indicate, etc.
It is false because the query does not compile to a JIT filter - the plan should have a node with JIT in it. How many cores is this running on, as it doesn’t compile to an async (parallel) filter either?
Also, what are the types involved? Most likely, a simple number based filter will be JIT’ed, but not necessarily more complex types that rely on Java-code to execute. As a rough rule, if you could pass the memory to C++ and fully realise the filter, then there is a good chance of compiling to a JIT filter.
Re: logs, roughly exe is the start of query execution and fin is the end. The time parameter is the execution time.
You can also search the github repo at questdb/questdb for logs to see lower-level context and where they originate.
We don’t have detailed docs about interpreting the logs - and many users prefer to ask us ( or our slack chatbot) their questions anyway!
Its running on a 8 core machine. The only types in the table are symbol, timestamp, long and boolean.
BTW, I just updated to 8.2.2 and I see that the jit parameter is getting set to true on the exe log message. But I still not see the JIT being mentioned in the ‘explain’ node. What is the best way to debug this? I think getting JIT working on top of 8.2.2 performance improvements will be even faster.
BTW, I also notice that after upgrading to 8.2.2 the DB freezes for random amounts of time between 25sec to 70 sec when executing the same queries that I used to run with 8.2.1.
… see that the jit parameter is getting set to true on the exe log message. But I still not see the JIT being mentioned in the ‘explain’ node.
Please could you share the log message and EXPLAIN plan again?
BTW, I also notice that after upgrading to 8.2.2 the DB freezes for random amounts of time between 25sec to 70 sec when executing the same queries that I used to run with 8.2.1.
If may be that you have a very heavy query - or something that is holding lots of resources.
We made a lot of changes in the last release, including quite a few performance related ones. It may be that your queries are running faster, and using more resources than before - but that is having a knock-on effect.