Writing situational judgement questions

Article 3 of 5 in a series on creating your own tests.

This is the third article in a series of five, to help you create your own tests using the TestGorilla platform. The full series includes:

1. Developing an effective screening test
2. Choosing a question type
3. Writing situational judgment questions
4. How to create your own test
5. Creating a coding question or test (optional)

The option for adding your own tests is available to account owners, admins, and recruiters who are on our Scale and Business plans.

Approx. reading time 8 minutes

 

In this article

1. Intro to situational judgment tests — or SJTs
   1. Elements and scoring
   2. Challenges
2. Template for creating good SJTs
   1. Template for scenarios
   2. Creating a scenario using the template
3. Template for response options
   1. Checking response options using the template
4. Common questions
5. Next steps

Introduction to situational judgment tests — or SJTs

Situational Judgment Tests — or SJTs — assess a candidate’s ability to respond appropriately to specific situations in the workplace. How well a candidate responds in each situation depends on previous training, experience with similar situations, and behavioral tendencies.

When the questions clearly define the situations to be considered by the candidate, SJTs can be used across a wide range of professions.

For example, a leadership test doesn’t need to be specifically tailored to a marketing or design leader, as long as the leadership situations are clearly defined. This makes SJTs some of the most widely used tests in pre-employment assessments.

The elements and scoring of SJTs

Situational judgment test items consist of two elements:

1. The scenario which provides the situation to be solved 
2. The possible response options that a candidate can choose from.

In the same way that real-world situations are never entirely black or white, SJT scenarios don’t always have just one right answer. The response options of an SJT item can contain:

• One action that’s the most appropriate for the question asked in that situation, which earns full points. 
• One or two actions that are somewhat appropriate, and earn partial points 
• One or two actions that would be inappropriate for the question asked in that situation, earning no points.

The challenge with writing good SJTs

Creating scenarios for a timed test presents a unique challenge. You need to provide candidates with all the necessary context of a specific situation — with no extraneous information — so they can choose a response.

Extraneous information not only adds reading time and stress for the candidate, it can also introduce doubts about which information is most relevant for solving the scenario.

In a 10-minute test of 15 questions, candidates have 40 seconds to dedicate to each question. This means scenarios and answer options need to be succinct and to the point.

We recommend that:

• Scenarios are 80 words or less and
• Each response option is 25 words or less.

Most scenarios and response options should be significantly shorter than the max.

To help you achieve that, we’ve developed a template for writing good SJT items that help you zero in on the specific skill being assessed in each scenario, and the appropriate response actions — without going off-topic or going overboard with extra details.

 

Template for creating good SJTs

Situational judgment test items can vary widely in context and details depending on the ability they aim to test. What remains the same in good scenarios is the structure and logistics of introducing complexity and making decisions.

Our template takes care of the structure and logistics for you so you can focus on the most important part of each item that requires your expertise — the situation and its context.

Template for situational judgment scenarios

The most important thing in creating a situational judgment scenario is identifying the goal of your question. Each question should assess only one specific skill, and the scenario you develop should be directly linked to that. You can see our article on developing an effective screening test to help with this.

Once you identify the skill to test, you’ll come up with a general scenario related to that skill and refine it using the following four steps:

 

Creating a scenario using the template

Let’s see the template in action by going through the process step by step for one of the sample items from our negotiation test. 

The final question reads:

You work for a company that makes plastic bottle tops. You are negotiating with supplier X to sell you dye so you can color your plastics. You know that company X has the best dyes. You know company Y also sells dyes. Y’s dyes are of inferior quality to X’s dyes, but are nevertheless acceptable for you as a Plan B. 

Which of the following is the best negotiation tactic to take with supplier X?

We’ll break down how we can get to a brief scenario for what is a complex situation by applying the template.

---

 

You’ll start by selecting the skill area and specific category of skill you want to work on from the ones you have defined for your test. Our question above corresponds to the following skill area and skill.

What skill area does this scenario belong to?

• • The ability to control and drive discussion.

What specific skill does this scenario test?

• • Negotiating for the best option when alternatives are available.

From this, we’ll come up with a rough scenario that’s relevant to the skill area and category.

You are in negotiations with a vendor for a product. This vendor has a high-quality product and you’d like to buy from them, but their asking price is higher than what you can pay. Another vendor also sells a similar product of lower quality for less. The second vendor is a good Plan B, but ideally, you want to get a better price from Vendor A. How do you accomplish that?

We’ll put our rough scenario through the four steps of the template to define its structure and logistics and turn it into a good situational judgment question:

We’ll then put together the sentences from each of the steps, and edit any overlap between sentences so they flow together in a natural and logical way. 

Here’s our resulting scenario from the steps above, with each part of the question highlighted with the color of the part of the template it belongs to.

 

Template for response options

The response options of situational judgment scenarios should follow the same rules as all response options for multiple-choice questions. Specifically, response options should:

• Follow the same grammatical structure.
• Be related in terms of content. They should be different, but different in the same way
• Be internally consistent. They should logically flow from the scenario and question asked.
• Not contain additional context that’s not given in your scenario.

In addition to the above rules, response options for situational judgment questions must be actions a test taker can take to resolve the situation, not the results of actions taken. Even wrong answers should be logically possible for the specific scenario. 

Checking response options using the template

The response options to the sample question we reviewed above are as follows: 

 

Notice how all the response options adhere to the rules discussed above by:

• Following the same grammatical structure. They all start with action verbs that the candidate will take or not take.
• Being related in content. All options include the same actions to take — talk with X, talk with Y — but differ in the order in which the actions are taken. 

• Being internally consistent by answering the question directly.
• Not containing additional context that’s not given in the original scenario. Additional context here could be an option like “Get your manager to negotiate with X.” The additional context of the manager generates confusion. Are you even authorized to have these negotiations or is the manager the best option?

Finally, notice that all answer options are actions to take with suppliers X and Y. Our scenario already defines the result — negotiate with supplier X for the best price — so our response options can focus on actions that could lead to that result.

 

Common questions

Why should I use situational judgment tests?
It’s one thing to know the skills that a candidate has, but to see how they would actually use them is another thing entirely. SJTs are designed to assess how a candidate would react in hypothetical situations based on real-life work-based scenarios. The questions require your candidates to call upon the skills and experience they have gained in previous roles and put them into practice to identify the best possible solution to the scenario. A candidate’s answers to these questions can help shed some light on their potential job performance.

Do I have to use the template to create my questions?
Of course not! You are welcome to create your own questions in your own way. We eat our own dog food here at TestGorilla — so we follow this template when creating content for our tests as we believe it is a great way to structure and create your questions. It has worked very well for us, so we thought we would share it with you.

 

Next steps

As mentioned above, this is the third article in a series of five, to help you create your own tests using the TestGorilla platform.

The full series includes:

1. Developing an effective screening test
2. Choosing a question type
3. Writing situational judgment questions
4. How to create your own test
5. Creating a coding question or test

We recommend you next read article four: How to create your own test so you can begin the process of putting your test into TestGorilla.