I’m running QuestDB and seeing high memory usage. I’d like to check what contributes to the process’ RSS. Is it possible to do that?
Yes, that’s possible. To do that, you should configure Prometheus metrics and scrape them.
First, QuestDB runs in Java Virtual Machine (JVM), so JVM itself may require some memory for its heap. To see the heap size, you should check the questdb_memory_jvm_max metric (it’s in bytes). If the heap size seems too high, you should consider collecting a heap dump, and then analyze it or report an issue.
Next, QuestDB uses native (a.k.a. off-heap) memory, anonymous and mmapped. Native memory usage is grouped into different metrics with questdb_memory_tag_ prefix in the name. E.g. questdb_memory_tag_MMAP_TABLE_READER stands for the amount of memory in bytes allocated for table reader mmapped files. An example of anonymous memory is the questdb_memory_tag_NATIVE_FAST_MAP metric. This one stands for memory used by hash tables allocated by the query engine, for JOIN and GROUP BY queries.
Mmapped memory is usually not a problem when it comes to memory usage. That’s because this memory is elastic and automatically handled by the OS: the OS evicts mmapped pages if a process requests memory and loads them later when they’re accessed. So, you should be looking at native memory metrics - if any of these has a high, potentially growing value, it may be an indication of a problem.