The 5 biggest problems with online exams

Many companies, universities and schools use paper tests to assess their students. It’s a method they trust. But they’re also uncomfortably aware of its cost – especially those that set thousands of tests a year.

Moving to online exams is tempting but has problems of its own. In this article, we offer our honest opinion on the biggest problems that businesses, universities, and schools face when running online exams, and how we believe they’re mitigated.

1. Computer-based tests have technical issues

For a paper exam, the worst technical issue a student has is their pen running out of ink. But for online exams, which use complex software and computers, technical issues can stretch to the horizon.

For hardware, computers might have overloaded RAM or CPUs. They can overheat. Their batteries might be shot. Mice and keyboards may be caked in dust or chip crumbs and suddenly break without warning.

For software, operating systems may be incompatible with the exam app. The app might have lurking bugs, or the computer could be riddled with viruses. The student’s internet connection might be sloth-like or prone to tumble, or they may have forgotten their login password.

This is a small sample of potential technical issues. But intimidating as the list may seem, it can be shrunk with the right measures in place, namely:

• Clear instructions and support: if using their own devices, students need clear prior instructions on what to install and how to access the test on exam day. They also need a knowledgeable support team in the wings. If a company or institution is using its own devices, this support and instruction is given to their exam coordinators instead.
• System requirements: to ensure their devices can run the assessment software smoothly, students should be given a list of system requirements that includes compatible operating systems and web browsers, minimum CPU and RAM needs, minimum internet speeds (especially if the test uses large videos and images), and minimum battery life.
• A usable test platform: a confusing test platform is the last thing a nervous student needs before starting a high-stakes exam. The system must be easy to use – in a sense invisible – so that they can get their highest score without it getting in the way. Aspects like text sizing and colour contrasts must also be editable for students with disabilities, so they’re assessed on a level playing field.

With considered processes, good exam software, and skilled support, many technical issues can be eliminated before they pop up.

2. Online exams need the internet

For companies or institutions that run large-scale online exams, some of their students are in remote areas with patchy internet. The students need a stable connection to first download the exam app, then the test itself before they can start.

This is doable for some, but despite us being well into the digital age, some people don’t have the luxury of the internet. To get online they may need to visit a library or pay for a data plan with money they can’t afford to spend. For these students, online assessments aren’t as equitable as paper tests, and companies and institutions need to provide accommodations so students can access the tests fairly – perhaps using some of the budget saved from not running paper tests.

The internet’s stability also used to be an issue with online testing, with the tests freezing, crashing, and losing progress when the connection dropped. But quality digital assessment providers now allow students to complete their tests offline, with progress safely stored in their web browser and uploaded to the cloud when back online.

This was a game changer. While some students might still need to download the exam app and test in a public place like a library, they go back to their (ideally) quieter home to focus on the test. And for those with unstable internet connections, they can be confident that their online exam should work properly from start to finish.

3. There are more cheating options for online assessments

For as long as there’s been high-stakes tests, there’s been students cheating them. It’s human nature. When online testing became more common after the millennium,  students had a new method to exploit and no beady-eyed invigilators to foil them.

Students could read their notes, call a willing friend, scan a textbook, search Google…all manner of unethical, undetectable methods that would give them a leg-up.

Today, things are better. To uphold academic integrity, high-stakes online exams always have some form of invigilation. The exams typically falling into two categories: those held in exam venues, or those taken at home.

Online tests held in exam venues

For computer-based tests held in exam venues, cheating tends to be curbed in the same way as paper tests: by hiring invigilators.

But they only get you so far. There are other cheating opportunities for students completing their tests online:

• The internet. Provided there are no invigilators watching, what’s stopping a student from using Google or ChatGPT to cheat? Thankfully, this is easily controlled by lockdown software that blocks access to websites or apps.
• “Computer issues”. A student may sabotage their computer or internet connection to give themselves more time to think. This is solved by restricting access to the computer’s settings, again through lockdown software.
• Earphones can be tiny today. If discrete enough, students can slyly listen to notes or receive instruction from friends. Invigilators must (reluctantly) peer into students’ ear canals to check for nothing more than wax.
• Copy-pasting. When using their own computer, a student might cunningly copy notes before starting the test and paste them later. Lockdown software should clear their computer’s clipboard.
• Students can use their phones to access the internet or message their friends for help. Invigilators must monitor for phone usage during the online assessment, just as they’d monitor for written notes.

Online tests taken at home

For online tests taken at home, human invigilators are replaced by remote proctoring software. This uses AI-driven technology to record and monitor students during their tests, flagging potential infractions for human review.

But despite the software evolving into an expert invigilator, it isn’t perfect. These are some ways students try to evade it:

• Proxy test-taking. If a student has the cash, it’s easy to find someone else to take their high-stakes test. But remote proctoring software blocks this by scanning students’ faces and comparing it to their identification. If a positive match is found, they can start their tests, and the software keeps a look out for sneaky substitutions throughout.
• This tried-and-tested cheating method remains popular. Students can write notes on paper, mobiles, watches, on their palms, and other creative surfaces. Remote proctoring software catches this by scanning the room before the test starts and then watching for suspicious head and eye movements. A lockdown feature also blocks students from opening digital notes.
• Another device. Devices like laptops, mobiles, and smartwatches make cheating easy, and remote proctoring software catches them in the same way as notes – by using its image recognition technology to spot them in the room and flag them.
• “Internet issues”. As with tests held at exam venues, students might sabotage their internet connection to buy extra time. That’s why remote proctoring software must work through dropouts – genuine or not.
• Helper in the room. It’s easy for someone to enter a room mid-way through a test and hold up notes for the student. But it’s hard for the student to keep looking at the notes and avoid being detected by the proctoring software.
• As with tests held in exam venues, students at home can wear discrete earphones to listen to notes or an accomplice. It’s more challenging to detect for at-home tests, but the proctoring software can scan students’ ear canals for the devices.

Those are the cheating methods students favour for online exams. As well as the solutions mentioned for each, companies and institutions can slash cheating by setting time limits on exams, using adaptive tests, using plagiarism detection software, and enforcing other strong deterrents. With these in place, they can be confident that their high-stakes digital assessments are fair.

However, there is always risk with online testing – especially for those taken at students’ homes. If they have excellent computer skills and want to cheat, they might find a loophole to exploit, so those setting the tests must decide how much risk they’re willing to accept. Given that students can also cheat on paper tests, are online exams worth the cost savings? Often, they are.

To conclude the topic of cheating, we must shift our focus from students to assessment authors. Given the sheer amount of skilled work that goes into creating quality, fit-for-purpose high-stakes assessments, staff with access to the system have been known to steal them and sell them to the highest bidder.

When these kinds of leaks happen to companies or institutions, they can badly damage their reputations – not to mention the cost of recreating the tests. To avoid this, an online exam system must include an ironclad permissions feature that so that the correct people can access your precious assessments and item banks, and no-one else.

4. Managing devices for online exams is expensive

For the companies and institutions that provide their own test-taking devices to students – at least to students who don’t have their own – managing them can be a big job that requires an I.T person.

This person – or team of people depending on how many devices there are – need to maintain an inventory, manage software updates, stay on top of operating system viruses and patches, ensure the devices meet compliance requirements, test and confirm they work properly, and carry out other tasks that take time and expertise. When there are hundreds or thousands of devices, this is expensive.

There isn’t a way around this. For the devices to run smooth, compliant online tests, they need to be managed properly. That’s why companies and institutions usually ask students to bring their own devices (BYOD), to reduce costs.

5. Making the switch to online exams takes work

This isn’t a problem with online exams themselves, but rather a hurdle to adopting them. And it’s a giant one.

When a company or institution has been running paper tests for decades, and its staff have become masters, switching to online assessments seems like a lot of trouble. The efficiencies and cost savings of online tests are clearly there, but adopting them feels longwinded, risky, and overwhelming. So they don’t end up switching, and remain stuck with expensive, old-fashioned processes.

But where there’s a will, there’s a way. The switch can be made easier by chunking it into manageable (though challenging) tasks. If you’d like to learn about these, we explore them a recent article: Switching from paper-based to online exams – key considerations.

Those are the five biggest problems with online exams. Some are weighty as rhinos, but each have solutions that either mitigate or eliminate them.

If you’re considering making the switch to online exams, it’s about weighing up the time, effort, and money involved in adopting them, and how well that investment eventually pays off.

When the shift is made and staff are comfortably settled into the new system, many companies and institutions believe it was worth the effort.

Image credits
Printing press image by vlasta2 – Flickr: PrintMus 038, CC BY 2.0