Configure AI Prompt for QM Auto Score

You can use NiCE CXone GenAI Prompts to configure the prompt that NiCE CXone Quality Management (CXone) uses for Auto Score. This lets you control the scoring logic and evaluation criteria applied during automated quality assessments. After configuring your prompt, you can import or enter up to five transcripts to test your settings. If the test results are unsatisfactory, refine the configuration until you achieve the desired outcome. Once finalized, publish the prompt to make it available for selection within Quality Management (CXone) Auto Score.

If AI Studio and the Prompt Editor are enabled, you can configure the scoring instructions and evaluation criteria used by Auto Score. The underlying AI model is managed by NiCE CXone and is not configurable per prompt profile.

Launch GenAI Prompts

  1. Click the app selector icon of app selector .

  2. Under General, select AI Studio.

  3. In the AI Studio left navigation, select GenAI Prompts.

Create Prompt Profile

Required permissions: GenAI Prompts, Prompt Access Edit

You will also need a license for Quality Management (CXone) Advanced AI enabled.

Each flow includes one default profile. Profiles save automatically as you work, but remain private until you click Publish Prompt. Unpublished profiles cannot be selected as live prompts. If you do not publish a new or edited profile within 30 days of the last edit, the settings revert to the latest published version or defaults.

To create an Auto Score profile:

  1. In the Profiles tree expand Quality Management > Auto Score.

  2. To create a new profile, click the + icon next to Auto Score.

  3. In GenAI Prompts, configure the prompt and add transcripts.

  4. Go to the Prompt Configuration tab to configure the prompt profile and define how interactions are evaluated. In this tab:

    • Enter a unique name for the prompt profile in the Profile Name field.

    • Add a short description of the profile in the Profile Description field.

    • Specify the criteria for scoring interactions in the Evaluation Criteria field. You can include time-bound conditions in your criteria, for example, "the agent must state the compliance script within the first 30 seconds of the call." The AI evaluates these conditions against the transcript's timing.

      The values entered in the Profile Name, Profile Description, and Evaluation Criteria fields are all sent to the Auto Score LLM and are considered when evaluating the profile against an interaction.

    • Out of the Box Profiles

      Auto Score offers predefined Profiles that are available as soon as the Scorecard is activated. These Profiles provide an immediate evaluation framework, enabling customers to start scoring interactions without additional setup. Each Profile represents a core behavior common to most contact center interactions.

      Included Out of the Box Profiles

      Proper Greeting

      Evaluates whether the agent began the interaction professionally, introduced themselves, and acknowledged the customer.

      Problem Solving

      Measures the agent's ability to identify, analyze, and resolve the customer issues effectively.

      Closing Summary

      Verifies that the agent concluded the interaction with a clear summary, confirmed resolution, and outlined next steps if necessary.

      Key Benefits

      • Available for immediate use without any setup required.

      • Universal behaviors applicable across industries and use cases.

      • Ensures a consistent evaluation baseline before adding custom Profiles.

      • Enables customers to activate Auto Score quickly and begin gaining insights on day one.

      Flexibility

      Out of the Box Profiles are fully compatible with:

      • Multiple Scorecards

      • Custom Profiles

      • Weight adjustments

      The Proper Greeting Example

      Many agents assume that a greeting is simply a variation of one of these lines:

      • “ACME, hello, John speaking.”

      • “ACME, good morning, John speaking.”

      • “ACME, good morning, John speaking, how can I be of service?”

      While these greetings are polite and familiar, we must clearly define what qualifies as a measurable Proper Greeting for scoring purposes. We also need to address cases where the customer speaks first, immediately raises the issue, or skips the greeting.

      For this reason, our Proper Greeting Profile is structured to accommodate all possible scenarios.

      However, for scoring, the LLM requires clear criteria and ground rules to ensure consistent and transparent evaluation.

      For Example:

      To explain the scoring process, we divide the behavior into three key questions. Who spoke first: If the customer initiates the conversation, the representative does not receive full credit. This approach ensures the agent leads the call.

      Does the greeting include the required elements? In this example, let us assume a complete greeting includes four measurable components:

      • Warm opening

      • Self introduction

      • Company or department name

      • Offer of assistance

      Each element helps establish a clear and professional start to the conversation. The number of missing elements determines the score range.

      • All four elements → high score

      • One or two missing → mid range score

      • Three or more missing → low mid range score

      • Customer spoke first → capped at 40 percent

      • No greeting at all → very low score

      Proper Greeting Out Of The Box Profile Setup

      The Proper Greeting Profile evaluates whether a representative opens an interaction according to the expected greeting standard. It includes three components: the Hard Rule, Scoring Logic, and Quick Decision Flow. These elements ensure the LLM applies scoring criteria consistently across all interactions.

      Hard Rule

      The initial setup defines the primary rule, which takes precedence over all other scoring conditions.

      If the customer speaks first, the score is capped at 40 percent. This ensures the CSR leads the call and initiates the greeting. When this occurs, the LLM automatically applies the cap before assessing the remainder of the greeting.

      Scoring Logic

      If the CSR speaks first, the system evaluates the greeting quality using four measurable elements:

      • Warm opening

      • Self introduction

      • Company or department name

      • Offer of assistance

      The number of elements determines the score range:


      The table below outlines the range and score criteria:
      Range Score Criteria
      81–100 percent (Exceeding) All four required greeting elements are included. The tone remains confident, friendly, and professional. Example: Good morning, this is Sarah with ABC Support, how can I help you today.
      61–80 percent (Meeting) The customer service representative greets the customer but omits one or two key elements. The tone is polite, though the greeting is incomplete. For Example: Hi, this is Mike, how can I assist you.
      41–60 percent (Needs Improvement) The greeting is rushed, overly procedural, or lacks key components. For example: Hello, what is your account number?.
      21–40 percent (Poor) The customer initiated the conversation, or the greeting was strictly procedural. Tone does not affect this category.
      0–20 percent (Unsatisfactory) The message lacks a proper greeting and begins abruptly, which may appear unprofessional.

  5. Click the Transcripts tab next to Prompt Configuration. This section lets you add sample transcripts for testing your prompt.

    • Click New Transcript radio button and click Create New Transcript.

    • Choose a Type of Transcript File.

      • Segment for partial conversation snippets.

      • New Transcript for full conversation samples.

    • Enter a Name and Description for the transcript.

    • Click the Edit icon next to the transcript name.

    • Add conversation lines under Client and Agent sections. For Example:

      • Client: “Hi, I have a question about my latest bill.”

      • Agent: “Sure, can you give me your account number?”

    • Use Agent Phrase and Client Phrase buttons to insert structured dialogue.

    For more information, see Manage Profiles, to learn more about the other actions you can perform on a profile.

Understanding Auto Score Results

About Auto Score Scoring

Auto Score uses AI (LLMs) to evaluate interactions against your scorecard criteria. Since AI is non-deterministic, scores may occasionally vary even for the same interaction evaluated multiple times.

Scores in AI Studio may differ from those in the Auto Score dashboard sometimes. AI Studio evaluates interactions one profile at a time, while Auto Score processes all profiles together as a batch, which can lead to minor scoring differences.

Best Practices for Writing Evaluation Criteria and Prompts

Well-written evaluation criteria and prompts help improve the accuracy and consistency of Auto Score results.

Writing Effective Prompts

Follow these guidelines when creating evaluation criteria and prompts:

  • Keep prompts concise and focused.

  • Use clear and specific language.

  • Define the expected behavior or outcome directly.

  • Include only the information needed to evaluate the criterion.

  • Avoid lengthy instructions, excessive context, or multiple unrelated requirements in a single prompt.

Example

Less effective- Evaluate the interaction by considering all aspects of the conversation, including tone, empathy, professionalism, compliance, conversation flow, issue resolution, and any additional behaviors that may indicate a positive or negative customer experience.

More effective-Determine whether the agent demonstrated empathy when responding to the customer's concern.

Test the Prompt

Before publishing a prompt profile, you can validate its logic by testing it with sample interactions. Testing ensures that the scoring criteria and instructions behave as expected when applied to real or simulated conversations.

You can test the profile in the following ways:

  • Fetch an existing transcript by entering a Segment ID.

  • Manually add transcript text directly in the editor.

Once the transcript is provided, the LLM generates a response that includes an executive summary, score, reasoning, and highlights. This allows you to review and refine the prompt configuration before making it live.

You can test a prompt with up to five interaction segments at a time. This allows you to evaluate prompt performance across multiple conversation segments before saving or publishing the prompt.

To test the prompt:

  1. If you want to test using an existing transcript from your system, enter the Segment ID in the designated field. This fetches the transcript automatically for testing.

  2. To generate test response, click Test Prompt at the bottom of the screen. The Generated Response panel displays the following:

    • Executive Summary

    • Score

    • Reasoning

    • Highlights

    • If your evaluation criteria include a time-bound condition, the response also includes the relevant transcript timestamp, so you can verify when in the call the scored event occurred.

    You can review the output to ensure the logic aligns with your evaluation criteria. If the test results are unsatisfactory, return to Prompt Configuration and refine the profile details or evaluation criteria. You can repeat testing until the output meets your expectations.

  3. Click Publish Prompt to make the profile live.