Beyond Flat Embeddings for Production AI Agents

Introduction

Since the launch of ChatGPT, large language models have become an important part of how people work. They are excellent at solving problems when all the required information fits inside their context window. But, most real-world enterprise tasks are far more complex. An AI agent may need to understand hundreds of documents, analyze an organization’s financial structure, follow decisions across months of Slack conversations, or combine information from emails, GitHub, Notion, and internal systems. Standard LLMs struggle with these tasks because they are trained on historical data, do not continuously learn from new information, and can only reason over a limited amount of context at a time.

Retrieval-Augmented Generation, or RAG, attempts to solve this problem by retrieving relevant text from a vector database. This works well for simple semantic search, but it becomes unreliable when information is fragmented across thousands of messages, documents, emails, and conversations.

Vector similarity may find related text, but it does not reliably understand relationships, changing ownership, dependencies, timelines, or the latest valid state of information. Scaling the model’s context window does not fully solve the problem either; it increases computational cost and can introduce irrelevant information, and hallucinations. The model remains stateless and does not build persistent memory.

HydraDB approaches the problem differently. It treats organizational memory as a data-modeling problem rather than only a context-length problem. HydraDB continuously converts evolving information from Slack, Gmail, Notion, GitHub, documents, and other artifacts into a fast, temporally aware knowledge graph. Using Git-style versioning, it preserves historical changes, tracks the latest state, and gives AI agents the structured context they need to reason reliably over an organization’s continuously changing knowledge.

Why Vector Search Fails

Fragmented context. Chunking splits related facts across different segments. If an entity is introduced in one chunk and updated in another, the system may fail to connect them and miss important information.

Outdated information. A flat vector index does not understand time. An old policy and its latest version can appear equally relevant because similarity search cannot tell which fact is current.

Similarity is not Relevance. Two similar vectors don’t mean they are related to the context. For instance: 24 June log: "Deploy version 2.4 to production on Monday.”, after two days, 26 June log: “Do not deploy version 2.4 on Monday, deploy on Friday”. Querying about “Deployment Status” will retrieve both results because similarity search doesn’t have temporal understanding. Similarity-based vector search can retrieve both because they are similar, but it can’t comprehend the difference. Microsoft Research has shown this baseline "struggles to connect the dots" when answers require traversing disparate information.

Methodology

Our methodology replaces flat chunk retrieval with a context-enriched, temporally versioned knowledge graph designed for enterprise AI agents.

First, each document, conversation, ticket, meeting note, and operational record is processed using a sliding-window context window. For a segment $s_i$, the system uses its neighboring segments $W_i$ to resolve pronouns, implicit references, missing subjects, and local dependencies:

This produces self-contained segments that preserve the original text while adding the context required to interpret each record independently. The system then extracts typed entities and relationships. Mentions referring to the same employee, customer, project, policy, issue, product, or service are resolved to a shared entity identifier. Facts distributed across Slack, email, CRM records, support tickets, and internal documents can therefore be connected into one organizational memory.

Each relationship is stored as a versioned edge:

where $u$ and $v$ are entities, $r$ is the relationship type, $t_{\text{commit}}$ records when the fact entered the system, $t_{\text{valid}}$ records when it became true, and $m$ stores provenance, confidence, ownership, and access-control metadata.

Updates are append-only. A new fact does not destroy the previous state, it creates a new version. The current state is reconstructed by ordering related facts by validity time and selecting the latest valid version.

For retrieval, the system combines semantic, contextual, and lexical signals:

Content embeddings retrieve direct semantic matches. Enriched embeddings retrieve records whose meaning depends on surrounding context. Lexical search captures exact identifiers such as customer IDs, project names, ticket numbers, service names, and error codes. The strongest text matches are mapped to their connected entities. The system then performs bounded graph traversal across permitted relationship types, valid time ranges, and access-control boundaries. This expands retrieval from isolated passages to the wider operational context around them. The system can retrieve the project, traverse to the blocking issue, identify its assignee, inspect the affected service, and recover the latest status update. The agent receives a connected explanation rather than several unrelated chunks containing similar words.

Finally, vector results, lexical matches, and graph-expanded evidence are deduplicated, temporally filtered, reranked, and compressed into the final model context. Every retrieved fact remains linked to its original source, making the answer traceable and auditable.

Why This Matters for Enterprise AI Agents

Enterprise agents do more than answer questions. They monitor operations, coordinate workflows, recommend actions, and make decisions across changing organizational data. To perform reliably, they need memory that is connected, current, permission-aware, and explainable.

This methodology provides that Knowledge Database.

Persistent organizational context. The agent can retain relationships across sessions instead of treating every interaction as isolated. It can remember who owns a project, which issue blocks it, what decision was made, and how that state changed.

Current-state reasoning. Temporal versioning prevents the agent from acting on obsolete policies, ownership records, customer states, or operational instructions.

Cross-system understanding. Entity resolution connects information distributed across communication tools, project trackers, CRMs, documents, and databases into a unified operational model.

Multi-hop reasoning. Graph traversal allows the agent to follow dependencies that cannot be recovered through similarity search alone, such as:

Permission-aware memory. Access metadata can be enforced during retrieval and traversal, preventing the agent from exposing information that the requesting user is not authorized to access.

Auditable responses. Provenance links every fact back to its source, timestamp, and extraction confidence. This makes agent outputs easier to verify, debug, and govern.

Reliable action planning. Because the agent can reconstruct the latest valid organizational state, it can make better recommendations, identify responsible owners, detect unresolved dependencies, and trigger workflows using current evidence.

The result is a structured memory architecture that allows enterprise AI agents to understand how organizational facts are connected, how they change over time, and which information is valid for a specific user and task.

Controlled inference

Context enrichment should clarify ambiguous text, not invent new facts. The model may use nearby messages to resolve references, but every extracted fact must remain linked to its evidence:

This allows low-confidence resolutions to be excluded, reviewed, or stored as uncertain claims rather than accepted facts. The original text must also remain available. Enrichment improves searchability, it does not replace source evidence.

Enterprise communication is highly contextual. Employees frequently use phrases such as:

• “Assign it to Arnav.”
• “The customer approved the second option.”
• “That service is still failing.”
• “Use the previous pricing policy.”
• “She owns the migration now.”

A flat chunk index stores these statements without reliably identifying the referenced project, customer, service, policy, or employee. Sliding-window enrichment converts them into explicit operational records:

1. “The engineering manager assigned the Ontology-Based Migration to Arnav.”
2. “Customer Acme approved Pricing Proposal Version 2.”
3. “The Authentication Service continues to return Error 503.”

This gives enterprise agents memory that remains retrievable across sessions, systems, and changes in vocabulary

Bio-Mimetic Context Consolidation

Persistent memory creates a second problem: GROWTH!

An enterprise agent may ingest millions of messages, tickets, documents, decisions, status updates, and transient events. Keeping every record equally active increases retrieval latency and allows weak or obsolete memories to compete with high-value information. The system therefore needs to decide not only what to store, but also what should remain immediately retrievable.

Retention score

For a memory $m$ at time $t$, we define a retention score:

• $I_{\text{salience}}$ is the memory’s initial importance.
• $\lambda$ controls the rate of temporal decay.
• $\Delta t$ is the time since creation or last consolidation.
• $t_{\text{access}_i}$ is a successful retrieval time.
• $w_i$ measures the usefulness of that retrieval.
• $\sigma$ controls reinforcement strength.
• $\epsilon$ prevents division by zero.

The first term gradually reduces the priority of unused memories. The second term reinforces memories that repeatedly contribute to successful answers or actions. A retrieved record should not be reinforced merely because it appeared in a candidate set. Reinforcement should depend on evidence that it was useful.

Multi-Representation Vector Substrate

The knowledge graph stores entities and relationships, but the system must first locate the relevant records and entities from which graph traversal can begin.

For each record $c$, the system indexes three representations:

content vector embeds the original text and retrieves records with similar meaning.

”Who owns the authentication migration?”

This matches to:

”Harsh owns the authentication migration.”

Sparse Representation

Sparse search preserves exact terms that dense embeddings may miss, including ticket IDs, error codes, API routes, customer IDs, acronyms, and version numbers. For example, it can precisely retrieve AUTH-503 .

Latent-context vector

Here, $c'$ is the context-enriched version of the record produced by the sliding-window pipeline.

Original record:

“Error 503: Service Unavailable”

Enriched record:

“The application is unavailable because the authentication service is returning HTTP 503.”

This allows an abstract query such as:

“Why is the application not working?”

to match the operational meaning of the record, even when the words are different.

Hybrid retrieval

The system searches all three indexes:

Candidates are then ranked using multiple signals:

The weights depend on the query. Exact identifiers favour sparse search, conceptual questions favour dense vectors, and current-state questions favour temporal validity and recency.

The retrieved records identify graph entry points. The system can then follow relationships such as:

Vector retrieval finds the relevant starting point; graph traversal reconstructs the complete explanation. This combination lets enterprise agents retrieve exact identifiers, semantic matches, implicit meaning, connected dependencies, and the latest valid state—even when the same event is described using different vocabulary.

Multi-Stage Recall Pipeline

At query time, HydraDB combines hybrid search with bounded graph traversal. The pipeline retrieves not only semantically similar text, but also exact identifiers, connected entities, and the latest valid state.

Adaptive query expansion

The original query (q) is preserved and expanded into multiple interpretations:

The expansions may include paraphrases, domain-specific terms, and explicit time ranges. For instance,

All queries run in parallel to improve recall when stored records use different wording.

Weighted hybrid retrieval

Each query searches the three representations introduced in the above section:

The content vector finds direct semantic matches. The latent vector finds implicit contextual matches. BM25 preserves exact identifiers such as AUTH-503, usernames, API routes, and error codes.

The weights adapt to the query: lexical signals dominate identifier-heavy searches, while dense signals dominate conceptual questions.

Entity-based graph retrieval

Entities mentioned in the query are resolved to graph nodes. HydraDB then follows bounded paths across allowed relationship types:

Each path is converted into structured text containing its nodes, relationships, provenance, and temporal state:

A cross-encoder scores each path against the query:

Chunk-level graph expansion

Every chunk is linked to its entities during ingestion. When a strong vector match is found, HydraDB expands directly through those pre-linked entities:

This retrieves nearby dependencies that were not explicitly mentioned in the query. For instance, a chunk about AUTH-503 may expand to the project it blocks, the service it affects, and the engineer assigned to it. Because these links are created during ingestion, the system avoids repeating entity extraction at query time.

Reranking and fusion

The pipeline produces three candidate streams:

Vector candidates are reranked using semantic and lexical relevance:

The final context merges the highest-ranked vector chunks with their graph neighborhoods, then adds the strongest independent graph paths:

Temporal validity, permissions, provenance, duplicate evidence, and token budget are applied before the context is sent to the language model.

In simple terms, query expansion improves recall, hybrid search finds the entry points, graph traversal reconstructs dependencies, and reranking selects the smallest set of evidence needed to answer correctly.

3 · Results: Implications for Enterprise AI Agents

We evaluate HydraDB on LongMemEval-s, a long-term memory benchmark containing 500 answerable questions across conversations averaging more than 115,000 tokens and roughly 50 sessions.

Unlike shorter benchmarks, LongMemEval-s tests whether an agent can recover facts, preserve preferences, combine evidence across sessions, reason over time, and select current information after updates. Data is ingested session by session to approximate an agent receiving information continuously rather than processing the complete history at once.

Gemini 3.0 Pro is used as the primary answer model and evaluator. GPT-5.2 and GPT-5 Mini test whether the memory architecture generalizes across different model capacities.

3.1 · Overall performance

HydraDB achieves 90.79% overall accuracy, outperforming the strongest reported competing system by 5.0 percentage points and the full-context baseline by approximately 30 points.

Reliable factual recall

Perfect performance on user and assistant fact extraction shows that the system can recover explicit information even when it is surrounded by large amounts of unrelated history.

This is important for agents that must remember:

• Decisions made during meetings.
• Customer requirements from support conversations.
• Tasks assigned by an assistant.
• Project owners and deadlines.
• Constraints introduced during previous workflow steps.

The result indicates that relevant evidence remains retrievable without repeatedly placing the complete conversation history inside the model context.

Persistent Personalization

HydraDB reaches 96.67% on preference questions, compared with 70.00% for the strongest baseline in this category. Preference memory is essential for customer-service agents, sales copilots, employee assistants and personalized applications. These systems must distinguish persistent preferences from temporary requests and recover them even when they were expressed indirectly or several sessions earlier.

For instance, an agent may need to remember that a customer prefers annual billing, that an engineer avoids deploying on Fridays, or that a user wants technical explanations without excessive abstraction.

Sliding-window enrichment makes these preferences explicit, while entity linking attaches them to the correct user, account or project.

Current-state reasoning

HydraDB achieves 97.43% on knowledge updates and 90.97% on temporal reasoning.

These categories directly measure whether an agent can distinguish current state from historical state. This is necessary for enterprise questions such as:

Who owns the migration now?

Which policy is currently active?

Was the incident resolved before the deployment?

What changed after the customer escalation?

The versioned graph preserves historical facts but allows retrieval to prioritize the latest valid state. This prevents an agent from treating an obsolete owner, policy, status or instruction as current merely because it is semantically similar to the query.

This capability is especially useful for:

• Compliance and policy assistants.
• Incident-response agents.
• Project-management copilots.
• CRM and account-management agents.
• Approval and workflow automation.
• Employee knowledge assistants.

Cross-session reasoning

Multi-session reasoning remains the hardest category. HydraDB scores 76.69%, matching the strongest competing result but leaving a meaningful gap relative to simpler memory tasks.

These questions require the system to combine evidence distributed across different sessions rather than retrieve one explicit statement.

For instance,

Answering correctly may require retrieving multiple records, resolving entities, selecting temporally valid edges and composing the resulting path. The result suggests that graph-based retrieval substantially improves long-horizon reasoning, but multi-session evidence composition remains the primary area for further improvement.

3.2 · Cross-Model Generalization

A useful memory system should improve weaker models rather than depend entirely on a large reasoning model.

HydraDB remains within approximately six percentage points across all three models. GPT-5 Mini reaches 85.80%, close to the strongest competing system’s flagship-model score of 85.20%.

This indicates that retrieval quality, temporal indexing and context construction can partially compensate for lower model capacity. A smaller model receives a compact set of connected, current and relevant evidence instead of reasoning over a large, noisy history.

3.3 · Practical Interpretation

The benchmark results show that HydraDB is strongest where enterprise agents commonly fail: preserving preferences, tracking updates, reasoning over chronology and retrieving facts across long histories.

The architecture is particularly suitable for agents that operate continuously over changing data:

• Support agents that remember customer history and unresolved issues.
• Project agents that track owners, blockers, decisions and deadlines.
• Sales agents that preserve account preferences and relationship history.
• Compliance agents that retrieve the currently valid policy with provenance.
• Incident agents that reconstruct failures, dependencies and remediation timelines.
• Research agents that accumulate evidence across long-running investigations.
• Employee assistants that connect conversations, documents, tickets and organizational entities.

Conclusion

Enterprise AI agents in research, finance, law, and other knowledge-intensive fields must reason across years of conversations, emails, documents, and operational artifacts. Giving these agents memory through vector search alone is not reliable: similarity search retrieves what looks related, not necessarily correct. HydraDB introduces a context engine that models knowledge as versioned, relational, and time-aware state. Its architecture combines sliding-window context enrichment, a Git-style temporal knowledge graph, hybrid retrieval, and multi-stage reranking to give AI agents a stronger and more dependable memory layer. On LongMemEval-s, HydraDB achieves 90.79% overall accuracy, including 97.43% on knowledge updates and 90.97% on temporal reasoning. These results show that production AI agents need more than flat embeddings; they need structured context that preserves relationships, chronology, and provenance.

References

1. LLMs: Bigger Is Not Always Better. Rigoni, T. — Ampere Computing Blog (2024). amperecomputing.com
2. Lost in the Middle: How Language Models Use Long Contexts. Liu, N.F. et al. (2023). arXiv:2307.03172
3. Context Rot: How Increasing Input Tokens Impacts LLM Performance. Hong, K., Troynikov, A., Huber, J. (2025). research.trychroma.com
4. Introducing Contextual Retrieval. Ford, D. — Anthropic Engineering (2024). anthropic.com
5. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Lewis, P. et al. (2021). arXiv:2005.11401
6. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. Wu, D. et al. (2025). arXiv:2410.10813
7. Evaluating Very Long-Term Conversational Memory of LLM Agents. Maharana, A. et al. (2024). arXiv:2402.17753
8. Lies, Damn Lies, & Statistics: Is Mem0 Really SOTA in Agent Memory? Chalef, D., Rasmussen, P. (2025). blog.getzep.com
9. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Edge, D. et al. (2024). arXiv:2404.16130
10. Supermemory: State-of-the-Art Agent Memory on LongMemEval. Daga, S., Sreedhar, S., Shah, D. (2026). supermemory.ai/research
11. Zep: A Temporal Knowledge Graph Architecture for Agent Memory. Rasmussen, P. et al. (2025). arXiv:2501.13956
12. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. Chhikara, P. et al. (2025). arXiv:2504.19413
13. BEAM: Benchmark for Evaluation of AI Memory. Tavakoli, M. et al. (2025). arXiv:2510.27246
14. FinanceBench: A New Benchmark for Financial Question Answering. Islam, P. et al. (2023). arXiv:2311.11944
15. Hydra DB: Beyond Flat Embeddings for Production AI Agents. Hydra DB Research Team (2026). Read the HydraDB research paper