Grounding and Hallucinations

The Assistant builds its answers from your organization’s content. This is called grounding. This page explains how grounding works, the cases where a response can include content that didn’t come from your sources, and how to check a response.

Common questions this page answers:

“Why did the Assistant return information that isn’t in our knowledge base?”
“Why do some responses have no citations?”
“Where did this response come from?”

Check whether a response was grounded

Two questions usually tell you whether a response was grounded:

Did the engine call any tools? Check the AI Reasoning for the response. If no tools were called, the response was likely ungrounded.
Are there citations? No citations is the clearest sign of an ungrounded response. If there are citations, you can trace which content the response was built from.

The Assistant’s chat responses don’t come from the open internet. The exception is World Knowledge App, which can run a live web search on the Moveworks web app when it’s enabled, and cites the web sources it used. Outside of that, an answer that didn’t come from your sources may be ungrounded.

How Moveworks keeps responses grounded

Guardrails on input. Requests that are harmful or not appropriate for work are declined before the engine looks for an answer. These topics rarely have relevant content in your sources anyway. See Guardrails.
Search across your sources. Moveworks search returns semantically relevant results from your knowledge sources: knowledge articles, forms, generally available user roster information, and custom queries built in Tool Studio. This gives the engine the most relevant material to work with.
Retrieval-augmented generation. The Assistant is constrained to answer only from your enterprise data. If it doesn’t find relevant information, it says so and offers to help file a ticket.
References and inline citations. Each response lists the exact references it used and cites them inline, so users can check the answer themselves.
Clarifying questions. When a request is unclear, the Assistant may ask a follow-up question before it answers or takes an action.
Disclaimers. You can add a disclaimer about generative AI for your users. See AI Disclaimers.

You can also close gaps in your own content with Knowledge Studio. For how Moveworks evaluates and improves grounding over time, see Models.

When responses include ungrounded content

The LLMs behind the Assistant were trained on a large body of public text. That training lets them follow instructions and write natural, contextual responses. What the model learned in training is called world knowledge. For the models Moveworks runs, see Models.

In these cases, a response can include content that didn’t come from your data.

Definitions and acronyms

The LLM can add context that isn’t in your content. For example:

It may know the full form of an acronym that your documents never spell out.
It may know how two entities relate, such as two versions of a mobile device.

Helpfulness as an enterprise assistant

The Assistant is instructed to be responsible, helpful, knowledgeable, and empathetic. To fit that persona it may add wording nobody programmed, such as acknowledging the user’s issue and concerns or offering general, high-level suggestions.

In most cases the Assistant tells the user when it can’t find specific information in your content, but it may still offer high-level suggestions. It’s instructed not to engage on certain topics, which limits how often and how widely it does this.

Drafting content

The Assistant has a tool for drafting emails, chat messages, summaries, and other text, often from grounded answers earlier in the conversation. Users can ask for a particular format or tone. Because drafting needs more creative generation, this tool is less constrained to grounded content. It still follows the other policies, including the work-related scope. See Email Composer.

LLMs are useful for many work tasks, and Moveworks adds formal support for more of them over time. Formal support means Moveworks has tested the use case with high-priority requests and added guidelines, policies, and prompt tuning so it works predictably.

Some tasks work in the Assistant today without formal support. For example, the Assistant can respond when a user asks it to analyze or write code in chat, but Moveworks doesn’t rigorously test this. Code Interpreter and World Knowledge App are documented separately.

Tone and formatting

There are endless ways to describe how a response should read, and only a small number of explicit instructions at any time. Summary style, tone, length, and formatting can change over time.

Knowledge questions that get ungrounded answers

Ungrounded answers to knowledge questions are rare but possible. The Assistant is instructed and tested to tell the user when it can’t find relevant knowledge, instead of answering from what the model learned in training. Most Moveworks customers have said they want only grounded answers.

Moveworks reviewers annotate usage data for quality, including cases where the Assistant produced an ungrounded response. Machine learning engineers use that data to find patterns in these cases, which make up less than 0.1% of all interactions, and Moveworks adjusts the LLM’s instructions to keep grounded answers high.

Knowledge questions that get grounded but inaccurate answers

A response can misread its sources when the context is ambiguous or spread across a long document. Moveworks splits large documents into smaller pieces called snippets so it can match a question to the right part of an article or file. Sometimes context that spans several parts of a document gets lost, and the Assistant misreads one snippet because it can’t see the related ones.

Two examples:

You ask about an abbreviation that’s defined in one part of a document and used in another.
The engine combines several tool responses, and it isn’t clear how they fit together. This can happen when an API response is missing the data that shows how it relates to the rest of the content. See Multi-Tool Responses.

Guidance for stakeholders and users

Treat ungrounded behavior as subject to change. Don’t build strong dependencies on it.
Look for citations to see which references a response used, so you can check whether it’s based on your organization’s content.
Ask follow-up questions to get more detail on the sources the Assistant used.
Use permitted but formally unsupported tasks at your own discretion, and check the accuracy of those responses the same way.