SeKondBrainPerspective · Illustrated
Perspective · Illustrated edition

Created by Benjamin J Brey, CFA

Chief Strategist, SeKondBrain.ai

b@sekondbrain.ai

Memory without a map. Why AI memory needs a concept graph.

A vector index can tell you what sounds similar. It cannot tell you what is true, what changed, or why. That takes a concept graph.

· 8 min read · six figures

Figure 1 · The essay as a concept graphHover or tap a concept
PROPOSES TORECORDSKEYED BYVALID FROMWEIGHED BYCITESJOINS MANYENTRY POINT TOLAST RESORT FORFUSESRANKS BY PROXIMITYDECAYS TO ARCHIVEKEEPS CITATIONSRUNSMERGES, SPLITSDATEDConceptCORE IDEAIdentityPROPERTYProvenancePROPERTYTimePROPERTYCertaintyPROPERTYEvent nodePATTERNForgettingPRACTICECognitionOSSYSTEMReasoning engineSYSTEMKemorySYSTEMVector indexENTRY POINTHousekeepingAGENTSPROPOSES TORECORDSKEYED BYVALID FROMWEIGHED BYCITESJOINS MANYENTRY POINT TOLAST RESORT FORFUSESRANKS BY PROXIMITYDECAYS TO ARCHIVEKEEPS CITATIONSRUNSMERGES, SPLITSDATEDConceptCORE IDEAIdentityPROPERTYProvenancePROPERTYTimePROPERTYCertaintyPROPERTYEvent nodePATTERNForgettingPRACTICECognitionOSSYSTEMReasoning engineSYSTEMKemorySYSTEMVector indexENTRY POINTHousekeepingAGENTS

Hover, tap or tab to any concept to see what it connects to and why.

Figure 1 The essay as a concept graph. Twelve ideas from this essay and sixteen typed relationships between them, drawn the way the essay argues memory should be kept: concepts as nodes, meaning on the arrows. Dashed lines enter from outside the graph. Every claim is sourced in the text below.

This is the illustrated edition. Where a picture makes the point, the prose steps aside.

In investment analysis, a number without a source is a rumour. The CFA curriculum does not teach you to love spreadsheets so much as to distrust any figure you cannot trace back to a filing, a ledger line or a signed confirmation. That habit is the most useful thing I bring to AI, because most of what the industry calls "AI memory" would fail an audit.

The default recipe goes like this: split everything into chunks, embed the chunks, put them in a vector index, and at question time pull back whatever sits nearest to the question. It demos beautifully. Then it meets a real organisation, where the same customer appears under three names, last quarter's decision was reversed on Tuesday, and the question that matters is not "what sounds like this?" but "what depends on this, and who said so?"

Similarity is not knowledge. An AI that is meant to remember needs a map of the things it knows and how they connect. It needs a concept graph. The vector index still earns its place, but as one way into the map, not as the map itself.

Vector indexConcept graph

Tells you what sounds similar.

Tells you what is true, what changed, and why.

Three spellings, three neighbours.

Three spellings, one client.

Accepts anything you embed.

Refuses anything that does not fit.

Pulls back whatever sits nearest.

Walks the map: what depends on this, and who said so?

Knowledge lives at the level of the concept

The cleanest framing I know is on SeKondBrain's research page: models work at the level of tokens, retrieval works at the level of documents, but knowledge lives at the level of the concept. Our home page says the same thing in plainer words: extract the meaning underneath your information, the people, decisions and ideas, and connect it into a living map.

Illustration: scattered fragments on the left resolve into a violet network of connected concept nodes on the right.

That is what I mean by a concept graph: SeKondBrain's take on the knowledge graph, with facts organised around persistent, versioned concepts that carry their own time, provenance and certainty. Our Concept-Grounded Attention paper calls this a Concept Lifecycle Model.

To people outside the field I describe it as a spreadsheet with arrows. Each row is a thing you care about; each column is a fact about it. The arrow is the part a table cannot say cleanly: this feature has that requirement; this conclusion was derived from that call. Once you have arrows, questions become walks rather than lookups, and a walk is how most real questions are shaped.

A graph that records, a reasoner that proposes

The design decision I would defend hardest in CognitionOS is a separation of powers. CognitionOS is a linked property graph served over REST, with one isolated graph per organisation and retrieval that combines graph traversal with vector similarity. It hosts no language model of its own. Every model call goes through the separate Reasoning engine and has to come back as schema-validated JSON rather than a paragraph to parse.

The model proposes; the graph records. In finance we would call it segregation of duties: the person who books the trade does not reconcile it. When a language model can write straight into your system of record, every confident mistake becomes a fact. When it has to submit a draft that is checked, the mistakes stay drafts.

Three things a vector index cannot do

Illustration: three broken shapes flow into a single glowing node.

It cannot know that two things are one thing. Three spellings of a client are three neighbours in embedding space, close but separate. A graph has to decide. CognitionOS starts with identity: write with your own id and the write merges on (id, org_id) instead of duplicating, so a retry converges rather than doubling the graph. For free text, its resolution cascade looks first for exact anchors, then their neighbours in the hierarchy, and only then for vector discoveries. Similarity is the last resort, not the first.

VECTOR SPACENEAREST NEIGHBOURSclose, but three separate pointsClient A LtdClient-A LimitedCLIENT Amerge on(id, org_id)Client AONE CONCEPT · ORGANISATIONa retry converges, it does not doubleRESOLUTION CASCADE FOR FREE TEXT1 Exact anchoralready in your graph2 Hierarchy neighboursof those anchors3 Vector discoverylast resort, not first
VECTOR SPACENEAREST NEIGHBOURSclose, but three separate pointsClient A LtdClient-A LimitedCLIENT Amerge on(id, org_id)Client AONE CONCEPT · ORGANISATIONa retry converges, it does not doubleRESOLUTION CASCADE FOR FREE TEXT1 Exact anchoralready in your graph2 Hierarchy neighboursof those anchors3 Vector discoverylast resort, not first
Figure 2 One client, three spellings. Close in vector space is not the same as one thing. CognitionOS merges writes on (id, org_id); free text resolves through exact anchors, then hierarchy neighbours, and only then vector discovery. Names are illustrative. Source: CognitionOS documentation, what it is and concepts.

It cannot refuse bad data. A vector index will accept anything you embed, which is exactly the problem. A graph that accepts anything soon contains everything. In CognitionOS an unknown node or relationship type is rejected with a 400 rather than quietly becoming a new type, circular hierarchy writes are rejected before insertion, and extractions below a confidence threshold are staged for human review rather than written blind. Every node roots at one of five primitives, Person, Organisation, Concept, Artefact and a tenant anchor, so the vocabulary can grow without sprawling.

It cannot tell you why. An embedding has no memory of where it came from. In a graph, provenance can live on the arrow itself: relationships in CognitionOS carry properties such as a match score and its source. A Graph of Thought records observations, inferences and decisions as a directed acyclic graph, so you can ask any entity what produced it, and unified search returns a match_explanation with each result. That is the difference between an answer and an answer you can defend.

IN A VECTOR INDEXtext chunk[0.12, -0.48, 0.33, 0.07, ...]no source · no date · no reasonIN A CONCEPT GRAPHConclusionCONCEPTClient call notesARTEFACTDERIVED_FROMsourcecall notesscore0.91date2026-04-12statusconfirmedthe arrow carries its own whyGRAPH OF THOUGHT · ASK ANY ENTITY WHAT PRODUCED ITObservationInferenceDecisionmatch_explanationreturned with each search result
IN A VECTOR INDEXtext chunk[0.12, -0.48, ...]no source · no date · no reasonIN A CONCEPT GRAPHConclusionCONCEPTCall notesARTEFACTDERIVED_FROMsourcecall notesscore0.91date2026-04-12statusconfirmedthe arrow carries its own whyGRAPH OF THOUGHTObservationInferenceDecisionask any entity what produced itmatch_explanationreturned with each search result
Figure 3 The arrow that remembers why. An embedding keeps no record of where it came from. A relationship can carry its own properties, and a Graph of Thought links each decision to what produced it. Property values are illustrative. Source: CognitionOS documentation on the data model, Graph of Thought and search.

The why is usually a route, not a document. As Context Graphs Are the Bridge puts it, organisations don't forget facts; they forget paths. The cheapest way to keep the path is to capture it where decisions already happen: approvals, overrides, escalations and exceptions, in thin slices rather than everything.

Illustration: a violet arrow with a blank tag hanging from it.

What we measured, and what we threw away

Here is the part I am proudest of, and it is not a feature. Kemory by SeKondBrain, our memory layer, publishes the retrieval fixes it built, measured against a labelled set, and threw away. Chunking before embedding was actively harmful. An asymmetric query prefix made things slightly worse. A fixed similarity floor did nothing at scale, because nearest-neighbour scores rise as the collection grows, so genuine matches and background noise end up in the same band. The list also records changes of mind. A cross-encoder re-ranker looked useless at first, then turned out to be the largest single improvement once it was measured properly. Now it ships on by default.

Most of those are standard advice, and most of them failed our own test. A humbler habit still helps, even with the re-ranker in place: phrasing the query in the vocabulary the memory was written in. What Kemory does is fuse semantic, lexical and blind-index legs by rank, then re-rank on five signals, one of which is proximity in the memory graph. The graph is not decoration on top of retrieval. It is one of the signals that decides what comes back.

Our research follows the same discipline. In Concept-Grounded Attention, the team put the concept graph inside a transformer and tested each mechanism against itself switched off. Steering attention with graph structure changed nothing. What helped was the memory model underneath: explicit temporal validity and an epistemic status on every claim. Atomic Units of X tested consolidation on three open-source software projects, hit its target on two, missed on the third, and reported the miss as plainly as the wins.

An investment memo that only lists the winners is a sales document. A research programme that publishes its off-switch results is one you can underwrite.

Time and certainty belong in the data

If I had to compress the lesson into one sentence for anyone building AI memory, it would be this: put time and certainty in the data model before you reach for cleverer retrieval.

To be clear about where we stand: these results come from our research. In CognitionOS, time validity and multi-party facts are being rolled out carefully, not switched on everywhere at once.

A fact without a valid-from date is a fact that will eventually be wrong without telling you. A claim without a status (observed, inferred or confirmed) cannot be weighed. The numbers are not small. In the Concept-Grounded Attention experiments, representing time explicitly added 13 to 25 points of accuracy on the LongMemEval memory benchmark across every model tested, up to 122 billion parameters, and giving each claim an epistemic status cut unsupported assertions in 72–122B models from 19–68% to 0–5%. Accountants solved this centuries ago with dated, double-entry ledgers in which nothing is overwritten and every adjustment has a reason. A concept graph with temporal validity and provenance is the same idea applied to what an organisation knows.

ONE CONCEPT · CLIENT A · THREE DATESt1t2t3nowReplacedpredecessor keptLead: Person XLead: Person YCONFIRMEDRetiredno successorStatus: pilotOBSERVEDretiredArchivedinto a summarySummary · 12 citations keptnotes+13 to +25points of LongMemEval accuracyfrom explicit time, every model19–68% → 0–5%unsupported assertions, 72–122B models, with epistemic status
ONE CONCEPT · CLIENT A · THREE DATESt1t2t3nowReplacedpredecessor keptLead: Person XLead: Person YRetiredno successorStatus: pilotOBSERVEDretiredArchivedinto a summarySummary · 9 citations kept+13 to +25LongMemEval points fromexplicit time, every model
Figure 4 Memory over time. The same concept at three dates: a fact replaced, with its predecessor kept and closed; a fact retired, with its history kept; and a run of notes archived to a summary that keeps its citations. In Concept-Grounded Attention (arXiv 2609.38684), representing time explicitly added 13 to 25 points of accuracy on the LongMemEval memory benchmark across every model tested, and an epistemic status on each claim cut unsupported assertions in 72–122B models from 19–68% to 0–5%. Lanes are illustrative. Further sources: Kemory documentation; The Librarian in Your Head.

Some facts are not pairs. "Bank X led a $5B round in Company Y in March, according to this filing" has five parts. Squeeze it into arrows between two nodes and the amount ends up hanging off the wrong thing. Mathematicians would call the fix a hypergraph, an edge that joins many nodes at once. The practical version is older and simpler: give the event its own node, and hang the parties, the date, the amount and the source off it. If the graph cannot hold the shape of what happened, it will eventually misremember it (Neurosymbolic Is a Direction of Travel, July 2026).

SQUEEZED INTO PAIRSLEDInvestor AORGANISATIONCompany BORGANISATIONAmountVALUEWhose amount? Which date?Which filing says so?hangs offthe wrong thingAS AN EVENT NODELEDRAISEDAMOUNTONCITED INFunding roundEVENT NODEInvestor AORGANISATIONCompany BORGANISATIONAmountVALUEDateTIMEFilingSOURCE
SQUEEZED INTO PAIRSLEDInvestor AORGANISATIONCompany BORGANISATIONAmount hangs off the wrong thingWhose amount? Which date? Which filing?AS AN EVENT NODELEDRAISEDAMOUNTONCITED INFunding roundEVENT NODEInvestor AORGANISATIONCompany BORGANISATIONAmountVALUEDateTIMEFilingSOURCE
Figure 5 A fact with many parts. Squeezed into pairs, a five-part fact leaves the amount on the wrong node. A hypergraph would join all five with one edge; the event node is the practical version. Generic labels, no real parties or figures. Source: Neurosymbolic Is a Direction of Travel.

The same honesty extends to how results are read. Kemory keeps capture separate from consolidation: capture keeps discrete items the user confirmed, while consolidation only refreshes a session summary and adds nothing new. And the documentation tells you to read a result honestly: an empty result in encrypted mode is not evidence that a memory does not exist. Absence of evidence is not evidence of absence. Any auditor will tell you the same.

A graph is a living thing

Illustration: an archive bookcase with glowing objects filed into its shelves and drawers.

The last mistake is to treat a graph as a project with an end date. Graphs drift. Two nodes turn out to be one; one node turns out to conflate two; a hierarchy that made sense in spring is wrong by autumn. CognitionOS runs housekeeping agents for merge and split, link prediction, hierarchy coherence, concept drift and quality, and because housekeeping changes the graph, every change is recorded and attributable. Maintenance with an audit trail is the only kind I trust.

Forgetting belongs in the design too. In Kemory, memories age through normal decay to an archive. The pattern set out in The Librarian in Your Head goes further: compress long histories into summaries that keep their citations, and down-weight a memory that keeps misleading. A garden compounds because it is pruned; a warehouse just gets bigger.

generalspecificRESEARCH DIRECTION · NOT A PRODUCT FEATUREA hierarchy in a Poincaré discRoom grows exponentially towards the edge,the way a hierarchy fans out, so deep concepttrees embed with little distortion in fewdimensions.Distance from the centre reads as howgeneral a concept is.A root and four levels, three branches each:1 + 3 + 9 + 27 + 81 nodes
generalRESEARCH DIRECTION · NOT A PRODUCT FEATURERoom grows exponentially towards the edge,as a hierarchy fans out, so deep concepttrees embed with little distortion.Distance from the centre reads as how general.A root and four levels: 1 + 3 + 9 + 27 + 81 nodes
Figure 6 Going further: a hierarchy in hyperbolic space. A research direction, not a product feature. In a Poincaré disc, room grows exponentially with distance from the centre, so a branching hierarchy embeds with little distortion in few dimensions. Edges are drawn as true geodesics of the disc. Source: Nickel and Kiela, Poincaré Embeddings for Learning Hierarchical Representations (2017).

Where to start

You do not need a licence to try the idea. CognitionOS is licensed and provisioned rather than self-serve, but Kemory Community Edition is an Apache-2.0 memory server for agents, Postgres with pgvector behind REST and MCP, that starts with a single npx command. The source sits in the public SeKondBrainAILabs organisation on GitHub.

Before you write a single node, though, write down the five to ten questions your graph must answer. Most first graphs fail because nobody did. Then pick your nouns, your verbs and a stable key, and make the graph refuse anything that does not fit.

Illustration: an engraved world map with violet routes linking points.

Vector search tells you what is nearby. A concept graph tells you what is known, how it connects, when it was true and who said so. If you want an AI you can rely on, build it the second way. Understanding, unshackled, starts with a map.

Illustrations generated with NotebookLM from this essay. Figures 1 to 6 drawn for this edition.

Go deeper

Read the primers

Two companion primers turn this argument into a build: questions, model, extract, resolve, validate, store, query.

SeKondBrain edition

Concept Graphs, the SeKondBrain Way

The step-by-step primer this piece draws on, told through CognitionOS and Kemory, with every source linked.

sekondbrain-kg-way.vercel.app
General edition

The Concept Graph Primer

The same pipeline for any stack, with a mapped library of how-tos, tools and papers.

sekondbrain-kg-primer.vercel.app
The company

SeKondBrain.ai

CognitionOS, Kemory, and the research behind this essay.

sekondbrain.ai