Review and Improve Your Bot

This page provides information about reviewing conversation data in CXone Bot Builder to improve your bot's performance. This is the fifth step in the bot implementation process.

After you configure and begin to test your bot's initial use cases, Bot Builder will have data you can use to examine how effective the current configurations are. At first, the data come from test conversations with the bot. Later on, after releasing the bot to production, the data will include live conversations with contacts.

By reviewing this data, you can spot places where you can improve how your bot performs. A bot's performance is indicated by how well it correctly predicts intents. If the bot predicts the wrong intent, it's harder for contacts to achieve their goals.

Glossary

Concept	Definition	Example	What the Bot Does
Utterance	Anything a contact says in an interaction. Sometimes called a message.	"I lost my password." "What is my balance?" "Are you a bot?"	The bot uses Natural Language Understanding (NLU) to analyze each contact utterance to determine its meaning, or intent.
Intent	What the contact wants to communicate or accomplish. Every message the contact sends has an intent.	"I lost my password" has the intent of "reset password". "Hello" has the intent of "greeting".	The bot analyzes a contact's message using NLU This process expands on Natural Language Processing (NLP) to make decisions or take action based on what it understands. to determine the intent. Once it knows that, it can respond with a message of its own. You configure the response you want the bot to use for each intent.
Entity	A defined piece of information in a contact's message.	Person or product name, phone number, account number, location, and so on.	The bot uses NLU to identify entities in a contact's message. Entities help the bot understand what the contact's message means.
Slot	An entity extracted from a contact's message and saved for use in bot responses. Similar to a variable.	Creating a slot for contact name lets the bot use that name in responses during an interaction, making it more personal.	When configured to do so, the bot extracts an entity from a contact message and saves it in a slot. You can have the bot use this information later in the conversation.
Rule	Defines a bot's response to messages that don't change meaning with context.	Single-turn interactions with fixed responses: What are your hours? What is your address? Conversation building blocks: Greetings, good-byes, thank yous and transitions; simple yes/no questions; and acknowledgments. CXone Bot Builder comes with default intents and rules for several of these, including greetings, handover Any contact message that should trigger transfer to a live agent requests, and more. FAQs Insults and classic bot challenges	Rules are one of two ways you can configure how the bot responds to an intent. Rules are useful for certain kinds of intents, but not all intents.
Story	Trains a bot to handle an interaction based on message intent and conversational context.	In an interaction about a forgotten password, the bot would respond to, "How do I do that?" in one way. If the interaction were about creating a new account, the response would be quite different even though in both cases the contact is using the same words with the same intent – to get more information.	Stories are the second of two ways you can configure how the bot responds to an intent. Stories teach the bot how to use the context of the conversation to respond appropriately.
Bot Action	Anything a bot says or does while handling an interaction.	In an interaction about a forgotten password, the bot responds by sending the link to the password reset FAQ on the website. When a contact expresses frustration, such as "I don't understand! It's not working!!!" the bot responds with "I'm sorry. Would you like me to transfer you to a human agent?" When the contact says yes, the bot initiates the transfer.	Actions are the options you have when defining how you want the bot to respond to each intent. They give you the flexibility to configure each response to achieve the outcome that meets the contact's needs.

Review Conversation Data

You can review every conversation bot handles. This allows you to see first-hand how your bot responds, where the bot has difficulty, as well as the way contacts interact with the bot and any issues they have. This information is valuable, as you can use it to improve your bot's intents The meaning or purpose behind what a contact says/types; what the contact wants to communicate or accomplish, rules Used to define bot's response to messages that don't change with context., and stories Used to train bot for interaction handling based on intent and context.

There following options in Bot Builder allow you to review conversation data:

Insights: Provides reporting and real-time, interactive analytics for your bots:
NLU Inbox: Helps you manage your NLU This process expands on Natural Language Processing (NLP) to make decisions or take action based on what it understands. data to improve the quality of your bot. It shows all new messages from contacts.
Query search: Use the search bar to narrow results in the NLU inbox or the Insights section.

Evaluate Intents

Evaluating and adjusting intents can help you translate their challenges into solutions when you modify your bot's responses. While you can't cater to every user behavior, you can address common points of friction or frustration.

As you review conversations, evaluate intent data to see whether the intents are effective and efficient. Does the contact achieve their desired outcome easily? If not, determine if the intents are not specific enough, are too specific, or if the training data is inadequate:

Does the bot reliably understand what the contact wants? If not, add more training examples to the intents where the bot is unsure.
Are any of the intents actually similar enough that they're essentially the same? If so, consider combining them under a more general intent and using the examples to train the bot to recognize the different scenarios.
Are contacts saying things that aren't covered by the existing intents? If so, consider adding more intents, or adding training data to your current intents.

Refine Your Bot Responses

Bot Builder has features that can help you to improve your bot's performance. Your first set of intents, stories, and rules might not have taken advantage of these features. As you work to improve how your bot responds, the following features might be helpful:

Examine your responses to see if you could use different actions in bot responses to make the response more slick, human-like, or user-friendly.
Consider whether any of the intents could be multi-intents. Multi-intents are useful when the contact combines two intents in one utterance. For example, when a contact says Thank you. Bye as a single message to the bot, they're combining a thank you intent and a goodbye intent.
Identify the unhappy paths Story that produces a wrong outcome for the intent that correspond to the happy paths you already mapped out in your intents, if you haven't done so already. Plan how to handle them, then add stories or rules as needed, along with the necessary training examples.
Consider what entities or slots are required. Entities are pieces of information collected from a conversation. Slots are like variables and can hold the collected entities.
Determine if a form is needed to simplify the process of collecting information from the contact to fill slots. The bot can follow a form to ask questions and gather information from the contact. You can also have the bot display a form to the contact.
Configure additional options in your bot responses, such as rich messaging, variations of bot messages, Smart Typing, fallback, and safety nets.

Situations Where You Can Improve Your Bot

While reviewing, you can take actions to teach your bot how to perform even better in future conversations. The following list describes situations that require improvement and what you can do:

Low intent classification confidence: If the intent classification is correct but the bot's confidence is low, add more training data for the intent to make the bot more confident. Training data includes intent examples and stories. If the intent classification is not correct, change it.
Low bot action prediction confidence: Add more training data to make the bot more confident in what action it should take in this situation.
User frustration: This may include requests for transfer to a human (handover Any contact message that should trigger transfer to a live agent), repeating what they already said previously, or insults. Add more training data about their issue, or adjust the NLU confidence threshold in fallback. This can tell your bot to use handover sooner if it isn't confident how to help the user.
“Out_of_scope” intent or fallback behavior: This may include a user asking for something your bot is not capable of, or your bot using fallback too early. Check for and fix any misclassified intents to verify that your bot didn't simply misunderstand the situation. Add a new story or rule to show your bot what to do next time.

Fallback and Safety Nets

As you review and improve your bot, you will start to identify places where contacts are likely to get frustrated with your bot. This is the time to add fallback and safety nets to your bot.

Fallback: This teaches your bot what to do when it's not sure how to proceed. There are two kinds of fallback:
- NLU fallback: when the bot isn't confident its understanding the contact.
- Action fallback: when the bot isn't confident in its ability to predict the next action.
Safety Net: A safety net allows you to configure what happens when there's another problem with the bot or the systems it connects to. This could include things such as the bot taking longer than normal to respond to the contact.

Additionally, this is a good time to make sure you have out-of-scope paths so the bot can respond to contact messages that are outside its domain.