Add note files about fast search and rule engines

This commit is contained in:
Hassan Abedi 2026-04-01 13:52:34 +02:00
parent 9c6f689447
commit 40ccf7ae69
2 changed files with 202 additions and 0 deletions

View File

@ -0,0 +1,104 @@
# Search and Vector Query Engines
A reference for how search-oriented engines differ from classical relational ones.
---
## Short answer
Search and vector engines are still query engines, but their core operators, indexes, and ranking models differ from standard relational SQL systems.
They are often built around retrieval rather than exact relational transformation.
---
## Search engines
Traditional search engines center on:
- inverted indexes
- document retrieval
- ranking and scoring
- boolean and text queries
The core question is often not just "which rows satisfy a predicate?" but "which documents are most relevant?"
That makes ranking a first-class part of execution.
---
## Vector engines
Vector engines center on:
- embeddings
- nearest-neighbor search
- approximate similarity retrieval
- metadata filtering
The core operation is often "find the closest vectors to this query vector," which is different from equality joins or exact predicate evaluation.
Approximation is often acceptable because speed matters and exact nearest-neighbor search can be too expensive at scale.
---
## Hybrid systems
Many modern systems combine:
- lexical search
- vector search
- metadata filters
- reranking
This means the engine may need to merge several candidate-generation and scoring paths into one final result.
---
## How these engines differ from relational ones
Search and vector engines often emphasize:
- ranking
- retrieval quality
- approximate indexing
- candidate generation
- top-k execution
Relational engines more often emphasize:
- exact semantics
- joins
- aggregations
- transactional or analytical correctness
The difference is not that one has a query engine and the other does not. The difference is what the engine is optimizing for.
---
## Example systems
Examples of search and vector query engines include:
- Lucene
- Vespa
- Weaviate
- Qdrant
They differ in packaging and scope, but all have real planning and execution concerns around retrieval.
---
## Practical mental model
Relational engines are often about exact transformation of structured data.
Search and vector engines are often about efficient retrieval and ranking over high-dimensional or text-heavy data.
That is the cleanest conceptual distinction.
---
## Changelog
* **April 1, 2026** -- First version created.

View File

@ -0,0 +1,98 @@
# Rule Engines and Fixpoint Evaluation
A reference for query engines whose core work is inference, recursion, or repeated rule application.
---
## Short answer
Not all query engines are centered on SQL-style scans, joins, and aggregates.
Some are organized around:
- facts
- rules
- recursion
- derivation until no new information appears
That is the world of rule engines, Datalog engines, and chase-style systems.
---
## The core idea
Instead of asking only for direct data retrieval, a rule engine often asks:
- what facts follow from these rules?
- what closure or least fixpoint do these inputs imply?
- what new values must exist to satisfy constraints?
This makes the engine iterative in a deeper way than many classical relational plans.
---
## Fixpoint evaluation
Fixpoint evaluation means:
1. start with known facts
2. apply rules
3. add newly derived facts
4. repeat until nothing new is produced
That final stable state is the fixpoint.
This is central to:
- recursive Datalog
- transitive closure
- many inference systems
---
## How this differs from standard relational execution
Relational engines often execute one finite plan over a fixed input.
Rule engines often need:
- iterative scheduling
- duplicate elimination
- dependency tracking
- recursion support
- possibly witness generation or equality merging
So the runtime model is often different even when some individual operators overlap.
---
## Why this matters
Rule/fixpoint execution is especially relevant when the system needs:
- recursive reasoning
- closure computation
- derivation of implicit facts
- chase-like enforcement of dependencies
This makes it a natural topic whenever query engines overlap with logic or theorem-like data processing.
---
## Practical mental model
Standard query engines answer:
- what result follows from this query over this data?
Rule engines also ask:
- what additional facts become true once the rules have run to completion?
That is the essential distinction.
---
## Changelog
* **April 1, 2026** -- First version created.