The Creation of Loka — Emma Leonhart

The Problem

I was building knowledge graphs. Not toy ones — real ones, with millions of triples extracted from structured and unstructured sources. The kind where you need to traverse relationships and search by semantic similarity in the same breath.

The standard answer in 2024 was: run two databases. A triplestore like Apache Jena Fuseki for your RDF graph, and a vector database like Qdrant or Pinecone for your embeddings. Glue them together in application code. Pray they stay in sync.

The frustration: Every GraphRAG pipeline I saw had this same two-database architecture. Two services to deploy. Two query languages. Two consistency boundaries. Two things to monitor, backup, and keep alive. And if you wanted an AI agent to set it up? Good luck. That's two install scripts, two config files, two health checks.

For production teams with DevOps engineers, this is manageable. For a solo developer, a researcher, or an AI agent trying to spin up a knowledge base autonomously? It's a wall.

I wanted something that felt like SQLite. Open a file, start querying. No daemon. No Docker compose. No "Step 1: install Java."

The Insight

The breakthrough wasn't a clever algorithm. It was noticing something obvious that everyone seemed to overlook.

HNSW — Hierarchical Navigable Small World — is the algorithm behind almost every production vector database. Qdrant uses it. Pinecone uses it. Weaviate uses it. It's the reason approximate nearest neighbor search works at scale.

And HNSW is, itself, a graph.

The realization: An HNSW index is a navigable graph where nodes are vectors and edges connect nearby points across hierarchical layers. A triplestore is a graph where nodes are entities and edges are predicates. These aren't two different things. They're two views of the same data structure. Why was everyone running them in separate processes?

If your knowledge graph already has a graph engine for traversal, and your vector index is already a graph internally, then unifying them isn't a hack — it's the natural architecture. You just need one system where vectors are stored as triples and HNSW is treated as a fourth index type alongside SPO, POS, and OSP.

Instead of:

Application
  |--- SPARQL query ---> [Fuseki]     (graph)
  |--- ANN search  ---> [Qdrant]      (vectors)
  |--- sync logic  ---> [application] (glue)

You get:

Application
  |--- one SPARQL query ---> [Loka] (graph + vectors)

Why Rust, Why SQLite's Philosophy

The choice of Rust wasn't about performance cult. It was about two things: correctness guarantees for concurrent HNSW reads (the borrow checker is the best tool for this), and single-binary distribution. No JVM, no Python runtime, no shared libraries to manage.

The SQLite inspiration went deeper than "just a file." SQLite proved that a database can be both embedded and serious. It's the most deployed database in the world, not because it's simple, but because it made the right tradeoffs: zero-config for the common case, opt-in complexity when you need it.

I wanted those same tradeoffs for graph+vector:

Embedded mode — Open a .sdb file, insert triples, search vectors. No config.
Server mode — Run loka serve when you need HTTP access. Opt-in, not mandatory.
Agent-installable — An AI agent can install, configure, and operate everything without ever touching a GUI. Because in 2025, your DBA might be a language model.

The Key Decision: Vectors Are Triples

The most important design decision in Loka is that a vector embedding is just a triple with a typed literal.

# An embedding on a node — it's just a triple
:paper_42 :hasEmbedding "0.23 -0.11 0.87 ..."^^loka:f32vec .

# An embedding on an edge — RDF-star makes this natural
<< :paper_42 :discusses :TransformerArchitecture >>
    :hasEmbedding "0.23 -0.11 ..."^^loka:f32vec .

This means there's no separate "vector collection" or "embedding table." Vectors live in the same triple store, participate in the same SPARQL queries, and get backed up in the same file. The HNSW index is built from these triples automatically, just like SPO/POS/OSP indexes are built from regular triples.

The result: you can write a single query that traverses a citation graph, jumps into vector space to find similar papers, filters by author, and ranks by similarity score — all without leaving SPARQL.

The Architecture

loka-core

The triple storage engine. SPO/POS/OSP indexes on interned u64 IDs. IRI interning, RDF-star quoted triple support, LSM-tree persistence. This is the foundation everything else sits on.

loka-hnsw

The HNSW vector index. Completely independent of SPARQL — it's a pure data structure crate. One index per vector predicate. Cosine, euclidean, dot product. Tombstone deletion with background compaction.

loka-sparql

SPARQL 1.1 parser, query planner, and executor with the hybrid extension: VECTOR_SIMILAR and VECTOR_SCORE. The planner decides whether to start from the graph or the vector index based on what's bound.

loka-proto

The HTTP layer. SPARQL Protocol, Graph Store Protocol, REST API. Only loaded in server mode.

loka-cli

The command-line interface. Import, export, query, serve. Also includes the agent-first installer that outputs config-as-markdown.

What I Learned

Building Loka taught me that the gap in the ecosystem wasn't a missing algorithm or a missing optimization. It was a missing opinion. Everyone agreed that knowledge graphs need vectors. Nobody was willing to say: "these should be the same database."

The technical challenges — content-addressed triple IDs for RDF-star, HNSW tombstone management, the query planner heuristic for choosing vector-first vs. graph-first execution — were all solvable once the architecture was clear. The hard part was committing to the idea that a vector is just a triple with a funny datatype.

The lesson: Sometimes the right architecture isn't the one that's technically hardest. It's the one that's conceptually simplest. If HNSW is a graph and your data is a graph, build one engine that does both. Don't run two.