# Query Engine Architectures A reference for the main ways query engines are structured internally. --- ## Short answer There is no single query engine architecture. Most engines are built by making choices along a few major axes: - row-oriented vs columnar - interpreted vs compiled - pipelined vs materializing - centralized vs distributed - tightly coupled to storage vs separated through a source interface Different systems pick different combinations depending on whether they optimize for transactional access, analytics, low latency, concurrency, distribution, or implementation simplicity. --- ## A useful mental model When people say "query engine architecture," they usually mean the answer to questions like: - what is the unit of execution: row, batch, or whole relation? - how do operators communicate: pull, push, or staged pipelines? - when are intermediates materialized? - where does optimization happen? - how tightly is execution tied to the underlying storage? - does one machine run the whole plan, or is the plan split across many workers? So architecture is less about one named pattern and more about a set of design choices. --- ## The classic row-at-a-time iterator model One of the most common traditional designs is the Volcano-style iterator model. In that architecture: - each operator exposes a method like `next()` - parent operators pull one tuple at a time from child operators - data flows upward through a tree of operators Typical shape: 1. `Projection` 2. pulls from `Filter` 3. which pulls from `Scan` ### Strengths - simple and modular - easy to understand and implement - operators compose cleanly - works well for many relational operators ### Weaknesses - pays per-row virtual-call or dispatch overhead - poor cache locality for analytical scans - not ideal for SIMD or vectorized processing This architecture was historically very influential, but many modern analytical engines move away from pure row-at-a-time execution. --- ## Vectorized and batch-oriented engines Modern analytical engines often process batches of rows, frequently in columnar form. Instead of asking for one row at a time, an operator processes a chunk such as a `RecordBatch` containing many values per column. ### Strengths - lower per-row overhead - better cache behavior - fits columnar storage and in-memory formats well - makes vectorized computation practical ### Weaknesses - more complex operator implementations - more bookkeeping around batch boundaries - some control-flow-heavy operations fit less naturally This style is common in engines optimized for scans, filters, aggregations, and joins over large datasets. --- ## Interpreted vs compiled execution Another major architecture decision is whether plans are interpreted directly or turned into specialized executable code. ### Interpreted execution The engine walks the plan at runtime and runs generic operator implementations. Strengths: - easier to build - easier to debug - more flexible for dynamic plans Weaknesses: - more dispatch overhead - fewer opportunities for low-level specialization ### Compiled or code-generated execution The engine turns expressions or even whole pipelines into specialized machine code, bytecode, or generated source. Strengths: - less runtime interpretation overhead - more aggressive specialization - can fuse multiple operations together Weaknesses: - much more implementation complexity - longer compile/startup path - harder debugging and observability Many real systems are hybrid: interpreted at some layers, compiled at others. --- ## Pipelined vs materializing architectures Query engines also differ in when they store intermediate results. ### Pipelined execution Operators stream data incrementally from one stage to the next. Strengths: - lower latency - less memory for some workloads - natural for streaming or interactive execution Weaknesses: - harder to reuse intermediates - blocking operators still force barriers ### Materializing execution The engine fully computes an intermediate result before the next stage uses it. Strengths: - simpler control flow - easier fault recovery in some systems - easier sharing of intermediates Weaknesses: - more memory and I/O overhead - higher latency Real engines mix both. For example, filter and projection may pipeline, while sort and some aggregates necessarily materialize or partially buffer. --- ## Row stores, column stores, and storage coupling Some engines are tightly integrated with one storage layout. Others sit above many storage systems. ### Storage-coupled engines In these systems, the executor is deeply aware of the storage engine's pages, indexes, and on-disk layout. Strengths: - can exploit detailed storage-level knowledge - often strong performance for the target workload Weaknesses: - harder to adapt to new sources - architecture is less modular ### Storage-decoupled engines In these systems, execution interacts with storage through a cleaner source boundary such as scan interfaces and pushdowns. Strengths: - easier support for many formats and backends - cleaner separation of concerns Weaknesses: - some storage-specific optimizations become harder - abstraction can hide useful low-level details This is a major difference between database kernels and more modular data-processing engines. --- ## Centralized vs distributed architectures Some engines run entirely inside one process or one machine. Others split work across nodes. ### Centralized engines All planning and execution happens locally. Strengths: - much simpler design - easier debugging - lower coordination overhead Weaknesses: - limited by one machine's CPU, memory, and I/O ### Distributed engines The plan is divided into stages or fragments that run on multiple workers. Common added components include: - exchange or shuffle operators - task schedulers - remote data transport - fault handling and retries Strengths: - can scale to much larger datasets - can exploit cluster parallelism Weaknesses: - much more complexity - network and serialization overhead - harder optimization problem Distributed query engines are not just "single-node engines on more machines." They need an additional architecture for task placement and data movement. --- ## Blocking vs non-blocking operators Operator behavior also shapes architecture. ### Non-blocking operators These can start producing output before consuming all input. Examples: - scan - filter - projection ### Blocking operators These require all or much of the input before producing useful output. Examples: - sort - some aggregates - some join strategies Blocking operators create execution barriers and often determine where memory pressure and latency appear in the engine. --- ## Common architecture patterns ### 1. Classic relational database kernel Typical traits: - row-oriented execution - iterator model - tight coupling to storage and indexes - strong support for transactional workloads ### 2. Modern analytical columnar engine Typical traits: - columnar in-memory representation - batch or vectorized execution - pushdown into file formats or source connectors - optimized for scans, joins, and aggregates on large datasets ### 3. Distributed SQL or lakehouse engine Typical traits: - logical and physical planning on a coordinator - distributed execution stages on workers - shuffle or exchange boundaries - object storage or remote tables as sources ### 4. Streaming query engine Typical traits: - push-style or event-driven data flow - unbounded inputs - windows, watermarks, and incremental state - low-latency continuous execution ### 5. Logic or rule engine Typical traits: - facts and rules instead of only relational algebra - recursion and fixpoint evaluation - unification, derivation, or chase-like execution - architecture organized around inference as much as scanning This last category matters when the "query engine" is not mainly about SQL tables at all. --- ## Comparison table | Architecture style | Typical unit of work | Best for | Main weakness | |:-------------------|:---------------------|:---------|:--------------| | Row iterator | one row | simplicity, OLTP-style access | high per-row overhead | | Vectorized columnar| batch / column chunk | analytics | more implementation complexity | | Materializing | full intermediate | simpler barriers and reuse | memory and latency cost | | Pipelined | incremental stream | low latency | harder coordination around blocking operators | | Distributed | stage / partition | scale-out analytics | network and scheduling complexity | | Rule/fixpoint | fact/rule step | recursion and inference | different optimization model from relational SQL | --- ## How to think about architectural fit A good architecture depends on what the engine is for. If the workload is mostly: - point lookups and updates, row-oriented designs can be reasonable - large scans and aggregates, vectorized columnar designs usually win - recursive inference or logical closure, rule/fixpoint architectures become central - cluster-scale datasets, distributed planning and exchange operators become unavoidable So the architecture should follow the workload, not the other way around. --- ## A practical summary If you need the shortest useful summary, this is a good one: > Query engine architecture is mostly about execution granularity, operator communication, intermediate storage, and the boundary with storage and > distribution. Or more concretely: - row vs batch - interpreted vs compiled - pipelined vs materialized - local vs distributed - relational vs rule-oriented execution Those choices explain most of the important differences between engines. --- ## Questions to keep in mind - What is the unit of execution? - Which operators are blocking? - How much is pushed down to the data source? - Is storage tightly coupled or abstracted behind connectors? - Does the workload look more transactional, analytical, streaming, or logical? - Is distribution a first-order requirement or premature complexity? --- ## Changelog * **Mar 31, 2026** -- First version created.