· Engineering · 8 min read · By Graylark Labs

Building Kai: an AI knowledge agent with CAG and RAG

“What can I do here?” is a hard question for a search box. The answer depends on who is asking, what they can access, and which screen is open. Kai is the knowledge agent we built for exactly that moment, and we use it inside Graylark LRM and in our own support work.

Two colleagues reviewing documents at a laptop with linked knowledge cards
Maintained product guides provide the knowledge Kai uses to answer platform questions.

A good knowledge agent should do more than produce fluent text. It should find the right passage, keep the passage attached to its context, and give a useful answer without inventing a button, path, or procedure. Kai brings those requirements into the retrieval pipeline itself.

Start with the person and the page

When someone opens Kai in Graylark LRM, the platform can send their role, the modules they can open, the current module, and the current page with record identifiers removed. Kai uses that context to leave out guide chapters for unavailable modules and, for a single-sided role, the other side's quick start. It gives the current module's chapter a ranking boost. If someone asks “what can I do here?”, Kai reserves part of the retrieval set for the chapter behind the screen, beginning with its overview.

This is more precise than putting a role label into every search query. The user's question remains the general search text; the role controls which chapters are eligible, and the screen supplies local context when the question needs it. A representative asking about agreements and an employee relations colleague asking the same words should not be steered to the same chapter merely because both contain “agreements”.

Page context is the route and module, not the data displayed on the page. Kai does not inspect proposal fields, rows or tenant business records; those require separate, governed workflows. The platform also discards a page hint for a module the user cannot open, and Kai filters action links that point into unavailable modules.

CAG, RAG, and hybrid mode in one architecture

Kai supports cache-augmented generation (CAG), retrieval-augmented generation (RAG), and a hybrid mode. CAG puts a bounded set of stable guides into the model's working context. RAG searches a larger indexed corpus for the passages relevant to this question. Hybrid mode can combine both. The production configuration checked into our Kai repository currently uses RAG, while the same service can be configured for CAG or hybrid use when the corpus calls for it.

That flexibility matters because a short, stable orientation guide and a growing collection of detailed product procedures have different retrieval needs. The architecture lets us choose the knowledge path for the material, rather than treat every document as the same kind of context.

Markdown-aware chunking and GrayEmbed research

Kai's ingestion is aware of Markdown headings. It splits a guide into sections, carries the section path into each chunk, packs short adjacent sections, and overlaps longer ones. A passage about searching agreements therefore retains its route through a chapter such as Landscape → Country Drill-Down → Agreements. Metadata records the guide, module, and audience. Those small pieces of structure make a large difference when two parts of a platform use the same vocabulary for different work.

Our GrayEmbed work extends this domain-tuning idea to embeddings: the aim is to make representation and chunking choices responsive to the material being indexed. We are evaluating it as a way to retrieve useful context with less wasted chunk space across different knowledge domains. We will publish comparative efficiency results when the evaluation is complete; we are not treating an unmeasured gain as a production benchmark.

Search for the question people would ask

In its question-indexing mode, Kai generates likely user questions for each chunk, indexes those questions, and returns the original source passage when one matches. It then blends the question match with the similarity of the passage itself. This helps a detailed, directly relevant section outrank a broad summary that happens to mention the same term.

The retrieval suite tests both what should appear and what must stay out. In the repository's documented tuning run, a range of chunk-text weights passed all 42 retrieval cases, while relying only on generated-question similarity missed four. That is a retrieval result, not a claim that every generated answer is correct.

Answer with useful boundaries

Kai is instructed to answer support questions from the supplied documentation and to say when it cannot confirm a procedure. It can suggest direct links to relevant Graylark pages, but returned action links are checked against documented targets and the user's available modules. Short follow-up questions can carry forward the previous topic; “what can I do here?” instead returns to the current screen, even if the conversation began elsewhere.

The service also has an optional second-pass grounding verifier that can challenge unsupported details and trigger a rewrite or evidence-only fallback. It is a configurable safeguard, not an always-on guarantee: the checked-in base configuration disables it. Kai's value comes from the combined engineering around the model—structured source material, retrieval tests, scoped context, valid destinations, and a clear way to acknowledge a gap.

Built where the work happens

Kai runs as a separate knowledge service behind Graylark's AI service and appears in the LRM support flow when enabled for a deployment and tenant. Its source corpus is built from platform guides, with build information that tracks the guide version. Changed files are re-indexed and unchanged files are skipped. That gives us a repeatable path from product documentation to the help people see in the platform.

For us, this is the compelling part of a knowledge agent: a question can become a grounded explanation and, when the documentation supports it, a link to the right place to act. See how Kai helps HR and labour relations teams use Graylark LRM.

Back to all articles