35 key questions to evaluate an online assessment platform

You’ll need to ensure the platform works with (or improves) your current processes, syncs with existing systems, and most importantly, allows you to run efficient, smooth large-scale exams for your students.

To reach the point where you can make a confident decision, you’ll need to complete plenty of research, including questions to the platform’s sales team – lots of them.

In this article, we’ve compiled what we believe are the most crucial questions you need to ask. They’ll help you thoroughly evaluate whether an online assessment platform will work for your institution or organisation.

Scale and resilience

Q: How many concurrent students can your platform handle for large-scale exams?
You’ll need to ensure the online assessment platform can comfortably handle your test volumes, and what systems are in place to avoid crashes. The number of tests you run will also increase your subscription costs, so this is important information to clarify.

Q: Can your platform scale easily as our needs grow?
If you spend months (perhaps years) selecting and switching to new exam software, it needs to not only handle your current test volumes, but your future assessment volumes after your company or institution has grown.

This is a crucial point. You’ll want to understand these potential test limits, and how the platform comfortably scales to your needs.

Q: How do you prevent freezing, crashing or other test interruptions?
What systems, features, and processes are in place to ensure smooth exams? As well as meticulous event planning, they might have features like server auto-scaling, load balancing, rigorous security protocols, and more. You’ll need to know that the online assessment platform has features that make it resilient.

Q: Do students need a stable internet connection to complete their tests?
Some areas are cursed with poor internet. You need assurance that your students can complete their tests without freezes, crashes, or data issues. Essentially, you need an exam platform that works offline, regardless of internet reliability.

Authoring and marking tests

Q: Will my paper tests translate to your platform?
If you’re switching from paper to online tests, you’ll need to understand whether the paper tests can easily be imported/converted to the online assessment platform, what this process looks like, and how you’ll be supported.

As you’re evolving from paper to digital tests, it’s also important to fully embrace the switch by reducing or cutting out your old paper-based processes. This means committing to a more efficient digital-first approach, not trying to shoehorn your paper processes into the new digital system. The platform’s salesperson can help you achieve this.

Q: Does the platform support branching tests or adaptive tests?
If you run branching or adaptive tests where students take easier or harder “routes” depending on their answers, you’ll need to understand whether the exam software supports them.

Q: What question/item types does your platform support?
Your tests may include question types like multiple-choice, essays, drawings, hotspot interactions, and more. You’ll need to confirm they all work properly in the new exam platform and also look out for fresh question types that might enhance your assessments.

Q: How does the authoring system work? Can it be customised?
Companies and institutions tend to author assessments in their own unique ways, and it’s important to understand whether your process will work with the system, or what may need to be adapted (ideally improved).

Q: How does the item banking system work?
Item banks are a crucial aspect of authoring assessments, allowing your test creators to quickly build assessments from the library of questions. Does the exam software’s item bank work with your own processes? How flexible is it? How easy is it for test creators to collaborate when creating new items?

These are just a few things you’ll need to understand.

Q: How does your marking system work? Can it be customised?
As with authoring tests, companies and institutions tend to have their own unique processes for marking, and you’ll need to ensure that the exam software can accommodate them. This includes who marks the tests, how frequently, how a test is passed from marker to marker, how markers communicate, and more. The software must work for you and ideally provide the features to improve your marking processes.

Q: What marking modes does your platform allow?
Your marking teams will likely use a mixture of automatic and manual marking depending on question types. They might also use methods like holistic marking, peer marking, competency-based marking, and more. You’ll need to know whether the online assessment platform supports these.

Q: What marking quality controls can be applied?
For assessment results to be accurate and credible, markers use quality controls like standardisation, double marking, rubrics, benchmarking, and more. You’ll need to find out if the exam software supports the quality controls that you use at your company or institution.

Q: Can tests include multimedia like images, video, and audio?
Modern digital tests are made more engaging with multimedia, and can help assess students’ skills and knowledge more accurately. Depending on the types of multimedia you use in your tests, you’ll want to find out whether the online assessment platform can support them.

Q: Can your platform accommodate different languages?
If you deliver tests in other languages, you’ll need to confirm that the new online assessment platform can handle them. You may also aspire to expand your assessments to new countries with different languages – will the platform support these plans?

Test events

Q: How are live exam events managed and invigilated?
Being able to oversee and manage exam events is essential. When exams are in progress, you may want a live view of the following and more:

• How many unique assessment events there are in total
• How many test sessions are active
• How many students are underway with their tests
• Who is invigilating each test sessions

The online assessment platform should provide you with a usable dashboard with every key piece of information you need, so that you can efficiently monitor and manage your exam events.

Test experience

Q: How do students access tests?
It’s vital for students to access their tests easily. They should be provided with clear instructions on their device requirements and exactly what they need to do start. You’ll want to learn how this process works, including how students are instructed to complete the tests, and how they’re supported if things go wrong.

Q: How are students with disabilities accommodated?
A quality exam platform caters to many different disabilities: blindness, colour blindness, hearing loss or deafness, problems with motor skills that make keyboards and mice hard to use, and more. People with these disabilities have a right to better themselves through educational assessment, so exam software must be able to handle their needs. This includes screen reader support, text and background colour changes, and more.

Q: What test tools are available to students?
What are the essential test tools your students need to work through their assessments? These might include standard exam timers and question/progress trackers as they work through the test, as well as more technical tools like rulers, protractors and calculators.

You’ll need to ensure the online assessment software has every tool that your students need.

Q: How and when do students receive their results?
Students are anxious to receive their results, especially for high-stakes tests. You’ll want to ensure that the process is as fast and easy as possible, with students receiving results electronically and in a format they can clearly access and understand.

Q: How is cheating prevented for tests?
To retain your academic integrity, you’ll want to learn how the online assessment software deters and prevents students from cheating – especially the most common forms of cheating.

Cheating may be prevented by blocking other websites and apps using lockdown software, verifying their identities and monitoring them through remote proctoring software, and more.

Security

Q: What security protocols and processes do you have in place?
Your IT security team will want to understand how your assessment and student data is protected, which may include the exact standards or accreditations that are gained – like ISO 27001.

Q: How do you protect my items and assessments against theft?
Assessments are expensive to produce, and staff members have been known to download and sell them. To combat this, an online assessment platform should allow you to define what each user can access in the system, so that only the appropriate staff members can access your item banks and tests.

Q: What user roles / permissions does your platform allow?
Similar to the question above, you’ll want to understand how the online assessment platform restricts users’ access so that they only have the information and features they need to do their jobs. This protects sensitive data and prevents staff members from accidentally breaking anything.

Typical roles might be administrators who have access to everything, authors who can access the test and item creation features, and markers who can access the marking system.

Remote proctoring

Q: How does your remote proctoring software detect cheating?
Cheating is the number one concern for companies and institutions considering remote proctoring. And it’s valid. You’ll want to understand how the remote proctoring software prevents students from using the internet, AI, their phones, and other cheating methods to get better scores.

How does the algorithm work? And how accurate does it tend to be? Your academic integrity depends on it.

Q: Can you customise what students can do during their tests?
You may want to run remotely-proctored tests where notes are allowed, or where students can access the Internet. In this instance, the online assessment platform should allow you to enable access to these things and ensure that the AI doesn’t flag them as violations.

Q: How are human proctors involved in your remotely-proctored tests?
AI has become excellent at detecting cheating for remotely-proctored tests, but it still requires the discerning eye of a human to review flagged violations. You may want to go one step further and employ live proctors to monitor students as they complete their tests (in addition to the AI monitoring them).

You’ll want to understand exactly how your proctors/invigilators can work with the remote proctoring software to achieve academic integrity.

Setup and services

Q: How does the setup process work for your platform?
Switching to a new online assessment platform is a lot of work. You’ll want to understand exactly what is involved during the setup, how long it takes, and most importantly, how the platform supports you along the way. For companies or institutions that run thousands of tests each year, it all comes down to one question: is it worth the risk?

This is why we strongly recommend forming a partnership of sorts with your new online assessment provider. It ensures that you’re both invested in the process and are willing to support each other along the way.

Q: What support or training do you provide for setting up the platform?
Similar to the above – switching to a new exam platform requires a tonne of support and training, so you’ll want to understand exactly what is offered, when it happens, and for whom. The last thing you want is to sign a contract and be left to struggle alone.

Q: What support is provided to students?
Some students run into technical issues with online assessments, and when they do, you’ll need to know the support systems that are in place to help them. It might be a mix of your own company and the exam platform providing the support, broken down into complexity levels. Clear rules need to be in place to define who is providing support and for what types of common problems.

Whatever the support system in place, it should be efficient and reliable so that students can solve their problems quickly.

Q: Which parts of the exam can your company manage?
You may want someone else to manage certain parts of your exams – their setup, invigilation, marking, or the entire process. The online assessment platform might offer these accompanying services, so it’s important to understand exactly how they can help, and how much they charge.

Reporting

Q: What reports does your platform provide?
To understand how your company or institution is performing, you need detailed reports on exam events, assessments, and students. What out-of-the-box reports does the online assessment software provide? And do they meet your needs?

Q: Can I create custom reports?
Sometimes, an exam platform may not provide you with the exact insights you need from their standard reports, and a custom report is required. You’ll want to learn whether these reports are possible, how customisable they are, and how easy they are to set up.

Technical

Q: What are the system requirements for test devices?
Whether you provide test devices to students or ask them to supply their own (or a mix of the two), you’ll want to ensure the devices are up-to-spec and able to comfortably run the assessment or remote proctoring software.

If the software requires mid or high-spec devices, you may have to upgrade many of your own devices, which can become expensive. Or you might need to ask students to upgrade their own, which isn’t equitable. Either way, it’s important to understand the exact system requirements for test devices, to ensure students can complete their assessments.

Q: Can your engineers customise the platform to meet our needs?
Your company or institution may have very specific requirements that the platform wouldn’t ordinarily provide, like integrating with a rare data analytics application. You’ll want assurance that the platform can be customised to meet your needs, including the cost and time involved in doing so.

Q: Can your exam software integrate with other apps?
You may want to continue using your student management system, learning management system, and other applications alongside the new exam platform, with data passing between them. If this is the case, you’ll need to confirm whether the various systems can be properly connected (via APIs).

Q: How does your system use AI?
AI is unstoppable, and has the potential to save you a lot of time and money. Whether it helps with marking, creating assessments, invigilating test-takers, or some other crucial function for your business, you’ll want to understand how AI is integrated into the online assessment software and what future plans they might have to do so.