Evaluation Question and Answer Analysis Report

Related Products or Features: Business Intelligence (BI) Reports, Quality Management

Required permissions: Report Viewer, Evaluation Question and Answer Analysis

Required Employee Settings: Can View BI Reports

The Evaluation Question and Answer Analysis report helps you to analyze the quality of an evaluation form. The report provides detailed information on the responses and answers received during evaluation.

Using this report, you can:

  • Determine whether important questions are impacting the quality score as intended.

  • See if the form could be revised to align behaviors with overall performance.

This report is an advanced business intelligence (BI) report. BI reports allow you to perform actions such as drilling down to more details or grouping and exporting data.

The data for this report refreshes every hour.

  1. To access and run this report, go to Reporting > Prebuilt Reports.

  2. Select the report you would like to run from the list.

  3. Complete the report prompt. The available steps and options will vary for each report.

  4. Click Run.

For most reports you can filter the report data using a variety of parameters. You cannot use filters to change certain parameters you selected on the report prompt. However, you can use Re-Prompt to change the parameters of the report.

  1. To change report parameters click Re-Prompt.

  2. Edit the desired parameters you want to change. For example, if you want to change the time period you selected before, set Time Period to the new desired range.

  3. When you are finished changing the report parameters, click Run.

This report allows you to filter by Evaluation Submitted Time Period or Recorded Interaction Time Period.

  1. Click the app selector and select Reporting > Prebuilt Reports.

  2. Click the report name to open it.

  3. In the Time Period field, select one of the following options:

    • Evaluation Submitted Time Period—When the evaluations were finished and submitted.

    • Recorded Interaction Time Period—When the interactions related to the evaluations occurred.

  4. Select the Teams you want to see the data for.

  5. Select the Form you want to see the data for.

  6. Click Run.

Example of Use

Joshua wants to make sure questions and answers are driving the quality program in the correct direction. They should conform to the business strategy for scoreable and non-scoreable questions.

Joshua opens the Evaluation Question and Answer Analysis report. He defines the time period as the last 30 days, and selects his team. He chooses the form to be analyzed and clicks Run.

The resulting Overview page provides detailed statistics on the answers to each question on the form.

The Average Eval Score % column shows the average score for each question, and can help Joshua see which answers were scored poorly. A low value in the Answer Representation % column lets Joshua know that evaluators may have had problems understanding the relevance of the answer. Together these can help him pinpoint where the form needs improvement.

Joshua can check that he has a reasonable number of evaluations to make a decision by looking at the Answer Count column.

Overview Page

The Overview page shows the following details about the questions and answers on the selected evaluation form:

  • The page shows the answers given for each question.

    For answers that have a fixed list of values, there is a row for each possible answer. If the evaluator could select more than one of the values as the answer, the selected values are shown separated by a comma.

    For questions that have open answers where the evaluator can type their own answer, there is just one entry showing the type of answer. The possible answer types are:

    • Text answer is given / is not given—A short single line answer.

    • Textarea answer is given / is not given—A long, possibly multiple line answer.

    • Hyperlink answer is given / is not given.

    • Datetime answer is given / is not given.

    An asterisk * after a question indicates that it is required or critical.

  • For each answer, the page shows:

    • A count of the number of times the answer was scored in the evaluation.

      For questions with open answers, this is the total count of all the answer values.

    • A representation value giving the percentage of users whochose the answer out of all the users who scored the question. For example, if the question was scored 10 times and the answer Yes was selected 4 times, then the representation for that answer is 40%.

      For an answer that is one of the open answer types, the representation value is based on the number of times the text answer is given or is not given. For example, suppose the number of times a text answer is given is 2 and the number of times it is not given is 1. The total number of responses is 3. The representation value is 67% (2/3) for Text answer is given and 33% (1/3) for Text answer is not given.

    • An average evaluation score for the answer across all the evaluations when the answer was selected.

      For an answer that is one of the open answer types, this is always shown as N/A.

    You can filter the data on the page by team, form version, and form section.