query-engine/ROADMAP.md

Project Roadmap

This document tracks the current state and next steps for the repository.

Important

The project is still early-stage. Treat this roadmap as directional rather than fixed, and do not read unchecked items as implied existing behavior.

Current Foundation

• Rust crate and binary named query_engine / query-engine
• Core term representation (Constant, Null, Variable)
• Atom structure (predicate with terms)
• Fact storage via Instance
• Tuple-generating dependencies (TGDs)
• Equality-generating dependencies (EGDs)
• Rule and EGD builders
• Variable substitutions and atom unification
• Restricted chase implementation
• Standard chase implementation
• Existential variable support (labeled null generation)
• Multi-atom rule bodies and heads
• Equality merging with union-find support
• REPL, script runner, and local web UI
• Provenance-oriented explanation support
• Relational schema, value, row, and result-set scaffolding
• Predicate-backed catalog inference
• Minimal SQL AST and parser
• Logical plan scaffolding
• Logical-plan execution for the first SQL slice
• SELECT-FROM-WHERE-ORDER BY-LIMIT support with positional or named columns
• Basic multi-table SQL joins via qualified-column filtering
• Table aliases for self-joins and qualified references
• Basic ORDER BY support over output columns
• !=/<> inequality and OR disjunction in WHERE clauses
• LIMIT clause for restricting output row count
• Integer literal and DataType::Integer support
• COUNT, SUM, MIN, MAX, AVG aggregates with GROUP BY
• Filter push-down rewrite across NestedLoopJoin in the physical layer

Near-Term Cleanup

• Keep all public docs aligned with actual implemented behavior
• Remove remaining stale terminology in comments and help text
• Expand examples for the current rule-engine and SQL workflows
• Add rustdoc coverage for the main public types
• Document the current SQL subset and its limits

Query-Engine Structure

• Introduce a dedicated logical representation module
• Define clear front-end, planning, and execution boundaries
• Add engine-level abstractions that are not chase-specific
• Establish common schema and typed-value representations
• Design a source boundary for future scans and pushdown

Front End and Planning

• Add a parser-oriented module beyond the current rule REPL language
• Add AST types for a structured query front end
• Add logical plan node types
• Add name resolution and schema validation hooks
• Add expression typing and nullability tracking
• Add aliases and richer projection expressions
• Add joins across multiple predicate-backed tables
• Add a catalog path for stable column naming beyond c0, c1, ...

Execution and Optimization

• Introduce physical operator abstractions
• Add a planning step from logical operators to executable operators
• Add basic rule-based logical rewrites
• Add statistics and cost-model scaffolding
• Add indexing and access-path abstractions

Rule-Engine Evolution

• Restricted chase
• Standard chase
• Oblivious chase
• Skolem chase
• Core chase
• Negative constraints
• Stratified negation in rule bodies
• Disjunctive heads
• Aggregation support in rule evaluation (available in SQL; not yet exposed to chase rules)
• Semi-naive evaluation
• Termination analysis helpers

Data and Interoperability

• Fact import/export
• File-backed data source experiments
• Table-like row or batch abstractions
• Stable script/query file format
• Integration with external storage or file formats

Performance and Reliability

• Predicate indexing for fact lookup
• Incremental evaluation
• Benchmarks
• Fuzzing
• Profiling-guided optimization passes

Testing and Tooling

• Unit tests
• Integration tests
• Property-based tests
• Regression tests
• Initial SQL pipeline tests
• Benchmark coverage (chase and SQL pipeline via cargo bench)
• Snapshot-style frontend tests
• More planner/executor tests as those layers are added