In the manual scoring interface, you will find a place to provide scores of 1 - 5 stars for candidate responses to the Customer Service (Immersive Role-Play) test. This guide will help you score their responses accurately. Before you start to score candidate responses, please review the guidelines below.
Tips for Accurate Scores
First, review these tips to increase the accuracy of your scores.
- Listen Actively and Impartially: Focus solely on the content and delivery of the response. Avoid distractions and personal opinions unrelated to the evaluation criteria.
- Consistent Application: Apply the same criteria and standards to every candidate. Do not adjust the scale for different candidates. This ensures fairness and consistency.
- Avoid Halo/Horn Effect: Don't let your impression of one aspect of a candidate's response (e.g., communicating effectively) unduly influence your scoring of other aspects (e.g., navigating complexity). Each criterion should be evaluated independently.
- Be Mindful of Personal Biases: Recognize that everyone has biases. Be aware of your own tendencies and try to set them aside. For example, if a candidate’s accent is similar to yours, ensure it doesn't unfairly influence your scoring of other styles.
- Focus on Evidence: Base your scores on specific evidence from candidates’ audio responses. Note specific examples of what the candidate did well or where they fell short.
- Take Notes: Taking brief notes while listening can help you remember key points and avoid biases. This will help to justify your scores and make the process more objective.
- Listen Multiple Times (If Needed): For complex or nuanced responses, listen to the candidate’s audio response more than once. This can help you catch details you might have missed the first time and lead to a more accurate evaluation.
Scoring Criteria:
Each question in the role-play is designed to assess specific skills. Please rate the candidate’s responses to those questions based on the criteria below.
Score Levels:
You will use a star system to assign scores in the manual scoring interface. First, review the scoring levels below to understand each star rating. Then, refer to the scoring criteria for each skill area to assess candidate responses accurately at each level.
5 stars = Exceptional
4 stars = Above Average
3 stars = Average
2 stars = Below Average
1 star = Unsatisfactory
Scoring Criteria for Each Skill Area:
Before assigning scores to candidate responses, review the skill definitions provided below to understand what is being measured. Examine the criteria for each score level carefully to determine what a candidate must demonstrate to achieve that rating.
After reviewing this Scoring Guide, you can start scoring candidate responses using the following criteria. To do so, listen carefully to each candidate’s responses, score them using the criteria below, and consider the tips provided in this guide.
Skill Area 1 - Communicating Effectively:
- Definition: Using the right methods to exchange information, thoughts, and ideas clearly, concisely, and accurately.
- Exceptional:
- Provides an exceptionally clear, logical explanation.
- Uses engaging, simplified language.
- Maintains a professional and empathetic tone.
- Above Average:
- Provides a clear, logical, and concise explanation.
- Uses simple, understandable language.
- Maintains a professional and empathetic tone.
- Satisfactory:
- Provides a clear explanation.
- Uses understandable language.
- Maintains a professional tone.
- Below Average: Provides a somewhat clear explanation.
- Uses simple terms, but the explanation is unconvincing or hard to follow.
- Maintains a neutral, detached tone.
- Unsatisfactory:
- Provides an inaccurate or unclear explanation.
- Uses confusing or unconvincing language.
- Demonstrates an unprofessional tone.
- Exceptional:
Skill Area 2 - Navigating Complexity:
- Definition: Identifying, analyzing, and solving complex problems in a resourceful and effective manner.
- Exceptional:
- Clearly recognizes the complexity of the situation.
- Discusses multiple viable solutions from different angles.
- Selects an effective resolution while maintaining policy integrity.
- Above Average:
- Recognizes relevant factors but may miss some complexities.
- Discusses multiple viable options but may not consider different angles.
- Selects an effective resolution but may lack detail.
- Satisfactory:
- Recognizes relevant factors but misses key complexities.
- Discusses one viable option.
- Selects an effective resolution but leaves some aspects unresolved.
- Below Average:
- Acknowledges only one relevant factor.
- Suggests only one viable option, which may lack effectiveness.
- Selects a resolution that only partially addresses the problem.
- Unsatisfactory:
- Fails to recognize relevant factors.
- Fails to discuss any viable options or discusses inappropriate options.
- Chooses an ineffective resolution or fails to offer a resolution.
- Exceptional:
Skill Area 3 - Customer Focus:
- Definition: Understanding, managing, meeting, and exceeding the needs and expectations of customers.
- Exceptional:
- Recognizes and validates customer feelings and loyalty.
- Clearly explains how the company values long-term customers.
- Tailors the response to the customer's situation.
- Suggests future support aligned with customer needs.
- Above Average:
- Acknowledges customer feelings but may not fully validate loyalty.
- Explains company values but lacks depth.
- Adapts the response but could be more personalized.
- Suggests future support but lacks detail.
- Satisfactory:
- Recognizes customer feelings but does not explicitly validate loyalty.
- Confirms that the company values long-term customers, but the response is vague.
- Adapts the response minimally and lacks personalization.
- Suggests one future way to support the customer but lacks detail.
- Below Average:
- Shows limited acknowledgment of customer feelings and does not address loyalty.
- Gives a basic response about company policies without showing customer appreciation.
- Offers a generic answer without tailoring the response.
- Suggests one future way to support the customer but lacks personalization and detail.
- Unsatisfactory:
- Does not acknowledge customer feelings or loyalty.
- Fails to explain how the company values long-term customers.
- Gives a dismissive response.
- Provides no suggestions for future support.
- Exceptional: