1. TestGorilla Help Center
  2. Creating your own tests

Creating a custom coding test

Updated

Article 5 of 5 in a series on creating your own tests.

This is the fifth article in a series of five, to help you create your own tests using the TestGorilla platform. The full series can be found on the right. Account owners, admins, and recruiters on any plan can create their own tests. To add one to an assessment and invite candidates, you need a Plus plan.

When creating your own test, we recommend outlining it before putting it into the TestGorilla platform. This will help ensure that everything is organized and laid out exactly as you want.

Approx. reading time 8 minutes

In this article

  1. What is a custom coding test?
  2. Creating a custom coding test
  3. Language-agnostic coding tests
  4. Creating a SQL coding test
  5. Common questions
  6. Next step

What is a custom coding test?

Custom coding tests are used to assess a candidate’s ability to create or debug code in a specified language within a certain time limit. 

Custom coding tests are:

  • Language agnostic, allowing candidates to choose from a variety of programming languages.
  • Designed to provide a consistent candidate experience by including one question per test.
  • Built for collaboration, enabling multiple users to contribute to the creation process.

 

Who can create a coding test?

Any user with the Owner, Admin, or Recruiter role is able to create a custom coding test. 

Note: TestGorilla cannot assist you with the content of custom tests. Any tests you create yourself must be an original creation.

Available coding languages

We currently have 20 coding languages available:

∙ C: 10.2.0
∙ C# : 6.12.0
∙ C++ : 10.2.0
∙ Go:  1.16.2
∙ Java: 15.0.2
∙ JavaScript: 18.15.0
∙ Kotlin: 1.8.20
∙ PHP:  8.2.3
∙ Python:  3.10.0
∙ R: 4.1.1
∙ Ruby:  3.0.1
∙ Scala: 3.2.2
∙ SQL: SQLite 3.31.1
∙ Swift: 5.3.3
∙ Typescript:  5.0.3

The following languages are available for language-agnostic tests:

∙ Dart: 2.19.6
∙ Elixir: 1.11.3
∙ Erlang: 23.0.0
∙ Julia: 1.8.5
∙ Perl: 5.36.0
∙ Rust: 1.68.2

Note: SQL tests use SQL only. For other coding tests, you can allow as many languages as you like, and each candidate picks one of them.

Creating a custom coding test

Coding tests use the same three steps as every custom test: Test details, Questions and Settings.

Step 1: Test details

1

Go to Library, open the Skills tests tab and click Create new.
2

Enter a Test name and choose the Language the test is written in.
3

In Test type, select Coding: Programming languages. You can't change the test type later.
4

Under Programming languages, click Add and select the languages candidates can take the test in, or Select all languages. Select at least one.
5

Optionally, add Instructions for candidates and the internal Additional details (difficulty level, who the test is relevant for, test summary).
6

Click Continue.

Step 2: Questions

You can add several coding questions, but each candidate completes only one question. You can also add 1 practice question so candidates can try the format first.

Click Create new. The question editor opens with four steps:

1

Question: the task candidates see. Explain the problem, the inputs and the expected output.
2

Initial code: the starting code candidates see, one section per language you selected. For the task to work, the code needs to:

  • Read the inputs, one per line
  • Contain the function that solves the task
  • Run the function and print the expected output

Example for Python, for a function with three inputs and one output:

n = int(input())
x = int(input())
y = int(input())
def FunctionName(n, x, y):
    # Write your code here
result = FunctionName(n, x, y)
print(result)
3

Test cases: pairs of inputs and the output a correct solution should produce. Each test case has a Name, Points, an Input and an Expected output. If the function has several inputs, put each on its own line (for n=1, x=2, y=5, write 1, 2 and 5 on three lines). There are two sections, and each needs at least 4 test cases:

  • Visible to candidates for self-evaluation: candidates can run their code against these during the test.
  • Hidden to candidates, used for calculating the test score: the candidate's score is based on which of these pass. The points from all hidden test cases must add up to 100.
4

Verification code: your model answer. Run it to confirm it passes your test cases. You must run it successfully in at least one language before you save the question.

Click Create, or Create and add another to write the next question.

Step 3: Settings

Set the Test duration and click Finalize to publish the test. You can only publish once every question's verification code has run without errors. Questions that still need a run show Setup incomplete.

 

Demo video

While the video showcases our old UI, it still reflects the key concepts and aspects of the test creation flow:

 

Creating a SQL coding test

SQL tests follow the same three steps, with a few differences.

Test details: in Test type, select Coding: SQL. SQL tests always run against an SQLite3 database, so there are no languages to choose.

Questions: click Create new. The question editor has five steps:

1

Question: the task candidates see.
2

Database structure: choose the Type of query (SELECT by default) and write the Script initial database that creates the database.
3

Initial code: the starting query candidates see.
4

Test cases: add at least 4 hidden test cases. Each has a Name, a Test case setup (an SQL script that runs right after the database is created and before the candidate's code) and the expected Result. If the query type isn't SELECT, add a Checking query: it runs after the candidate's code, and its result is compared with the expected result.
5

Verification code: your model answer. It must run successfully before you save the question. If you change the code, the test cases, the database script or the query type, run it again.

Settings: set the duration and click Finalize.

 

Demo video

While the video showcases our old UI, it still reflects the key concepts and aspects of the test creation flow:

 

Common questions

Is there a limit to the number of coding tests in an assessment?
Depending on your plan, you can include up to 5 tests per assessment. Consider assessment length carefully to avoid overwhelming candidates.

Can I edit an existing custom coding test?
Yes. While the test is a draft, you can change anything. Once it's published, you can only fix wording. To change languages, test cases or other details, open the test's menu in the Library and choose Enable edits. This creates a new draft version; assessments that already use the test keep the version they have.

Why does a question show Setup incomplete?
Its verification code hasn't run without errors yet. Open the question, go to Verification code and run it. You can publish the test once every question is complete.

Next steps

While not included in this series, we do have other articles that are helpful when making your own test that are worth taking a look at. They are:

Was this article helpful?

Articles in this section