SELECT from multiple timeseries tables with the same query

Yup dropping the index boosts the query a lot! Selecting the count in the interval can returns also more then 200mln of rows so I guess removing it is fine… I don’t even need other complex queries, I just need to stream ticks from quest to my application which is a simulator running on VPS with 12 vCore (I know not too much, so trying to keep things as much efficient as possibile)…

Union is not a solution btw, it’s incredibly slow, amost 30sec to merge 2000 minutes of series, most of the time it goes in timeout.

So I think the final solution will be to make a big table for a single broker and put all symbols there, drop index and use the proper WHERE clause… Thanks a lot for the help!

May I ask you if you can review my snippet to execute queries? I think it’s fine but maybe I might have forgot some important performance consideration :slight_smile:

long startTime = currentTime - (timeframe.millis * (count + 1));

        String query = "SELECT timestamp AS tick_time, first(bid) AS open, last(bid) AS close, min(bid) AS low, max(bid) AS high FROM " + symbol.dataSource + " WHERE timestamp >= " + startTime + " AND timestamp <= " + currentTime + "000 AND symbol = '" + symbol.symbol + "' SAMPLE BY " + tf + " ALIGN TO CALENDAR ORDER BY tick_time DESC LIMIT " + count;
        System.out.println(query);
        int cores = Runtime.getRuntime().availableProcessors();
        System.out.println("Number of cores: " + cores);
        try (SqlExecutionContext ctx = new SqlExecutionContextImpl(cairoEngine, cores)
                .with(AllowAllSecurityContext.INSTANCE, null)) {
            try (RecordCursorFactory factory = cairoEngine.select(query, ctx)) {
                try (RecordCursor cursor = factory.getCursor(ctx)) {
                    final Record record = cursor.getRecord();
                    while (cursor.hasNext()) {
                        Period period = new Period();
                        period.period = timeframe;
                        period.symbol = symbolId;

                        period.startDate = record.getLong(0) / 1000;
                        period.endDate = period.startDate + period.period.millis;

                        period.open = record.getDouble(1);
                        period.close = record.getDouble(2);
                        period.low = record.getDouble(3);
                        period.high = record.getDouble(4);

                        periods.insert(0, period);
                    }
                }
            }
        } catch (SqlException e) {
            throw new RuntimeException(e);
        }