The Quality Management Advanced AI Usage report provides visibility into the consumption of advanced AI features within Quality Management. It tracks usage for Evaluation Summary and Auto Score, enabling Quality Managers to monitor adoption and performance trends.
The paradigm date for this report is always the Usage Date. The Usage Date is derived from the exact timestamp of each event. For example, when you select Last 30 days, the report displays daily totals for each of those 30 calendar days, based on the date each event was processed (Usage Date). Data in the report refreshes every 4 hours.
This report is available under the Reports page as a prebuilt report.
Below are descriptions of each widget and the insights it provides:
Use Case
Quality Managers can use this report to:
-
Track AI Adoption: Monitor how frequently advanced AI features are utilized in evaluations.
-
Optimize Resource Allocation: Identify trends in usage to plan for capacity and licensing needs.
-
Measure ROI: Assess whether advanced AI tools are delivering expected value by comparing usage against performance outcomes.
-
Support Compliance: Ensure that AI-driven evaluations align with organizational quality standards.
Permissions
You must have these permissions:
-
Dashboard > Dashboard Templates > Quality Management Advanced AI Usage: On
-
Dashboard > Dashboards: View
-
Dashboard > Dashboards: Edit (optional)
If you cannot access the reporting templates or Dashboard, or if widget data is not loading, check with your administrator. The administrator can verify your role's reporting, dashboard, and widget permissions by visiting Admin > Security Settings > Roles and Permissions.
Data Overview
This report includes:
Evaluation Summary Usage Overview
Displays the number of evaluations processed through Evaluation Summary.
View the data in this report
|
Column |
description |
|---|---|
|
Date |
The processing date of the Evaluation Summary. |
|
Number of Evaluations |
The quantity of evaluations charged. |
Consumption-Based Billing for Evaluation Summary (Evaluation Insights)
Evaluation Summary (Evaluation Insights) usage is charged per evaluation summarized.
-
After Evaluation Summary is enabled for your tenant, all evaluations completed by human evaluators are automatically summarized and incur one consumption-based charge per evaluation.
-
Historical evaluations completed before the feature was enabled are not automatically summarized. To generate summaries for those evaluations, you must create them manually, and each generated summary incurs a consumption-based charge.
-
If an evaluation is summarized more than once, including through manual or bulk summary operations, each summarization incurs an additional charge.
Example
Evaluation Summary is enabled on your tenant. A newly completed evaluation is automatically summarized and incurs one charge. Later, you manually generate a summary for a historical evaluation, resulting in another charge. If that same historical evaluation is summarized again as part of a bulk summary operation, it incurs an additional charge.
Consumption charges are based on each evaluation summary generated, including repeated summaries of the same evaluation, not on unique evaluations.
Auto Score Usage Overview
Displays the number of interactions processed through Auto Score.
View the data in this report
|
Column |
description |
|---|---|
|
Date |
Processing date of the Evaluation Summary. |
|
Number of Evaluations |
The quantity of evaluations charged. |
Consumption-Based Billing for Auto Score
Auto Score usage is charged per interaction that is auto scored.
-
Billing is based on the scorecard that performs the auto-scoring and is not affected by the number of GenAI profiles configured within the scorecard.
-
If multiple scorecards auto-score the same interaction, a separate consumption-based charge is incurred for each scorecard.
Example 1
One interaction is auto-scored by a single scorecard that contains multiple GenAI profiles.
Result: One consumption-based charge is incurred.
Example 2
One interaction is auto-scored by two different scorecards.
Result: Two consumption-based charges are incurred, one for each scorecard that auto-scored the interaction.
Consumption charges are based on each auto-scoring event, not on unique interactions.
How Consumption Charges Are Calculated
Advanced AI features use consumption-based billing. Charges are incurred each time an evaluation is summarized or an interaction is auto scored. If the same evaluation or interaction is processed multiple times, each processing event is counted separately and may result in additional charges. This applies even when the same evaluation or interaction is included in bulk operations or processed by multiple scorecards.