CALL US TODAY 1300 857 687  

Category: Assessment

Solving logistics and identity for Online Exams at University

Assessments are used in the Tertiary sector for formative and summative purposes. The economics and student demands are increasing the pressure for the Tertiary sector to provide online exams. The cost of Assessment can be ~$50 per high-stakes test per student, equating in a cost of $287 million dollars to the sector in Australia alone*.

The potential cost saving for a typical 50,000 person University are $6 million dollars per annum.**

Schools are responding to these economic pressures and exams are shifting online with NAPLAN Online in 2018, and the State-accreditation bodies stating to move for budget approval.

What will school leavers expect from Tertiary institutions after having already used computers for exams for a number of years? It won’t be a pencil and piece of paper.

Before Tertiary can embrace Online Assessments, Australia’s leading Universities identified two key challenges: logistics and identity. Here’s what Janison is doing to tackle these head on.

How do you manage logistics?

When planning for a digital exam, you want a solution that can meet your needs for resiliency, security and bandwidth. For the NAPLAN Online project, we’re building three options: Online, School Server and USB technology. In this way, schools and University campuses can opt for the solution that best meets their level of readiness.

cloud-picture

Online

The online solution requires a reliable internet connection and the use of a lock-down browser. This can be done on Windows, Mac, Chromebook, iPad and Android tablet.

School Server

School Server

The School Server solution runs a Deployed Application on a server within the local network of the school. This can be via installation on a physical hardware or on a virtual machine. The school server is capable of delivering tests with no Internet connectivity. Afterwards, test attempts can be uploaded to the central server.

USB

USB

The USB solution is self-contained, designed to run on a Windows computer with no internet nor local network connectivity. Some installation is required, all school, test, student information, and student test attempts will remain stored on the USB drive.

We are extending on the above to develop hybrid solution of software and hardware. Janison physical mobile Exam Appliances can be setup in any location, which will provide a local high-density WiFi access point. Student devices connect will to the appliance via a mesh network. With battery backups and optional 3G/4G connectivity, this can convert even the most unlikely venue into a digitally-ready Exam Centre. These four options provide a broad range of delivery options to meet the unique challenges of Universities.

 

How do you manage identity?

identity

The critical challenge in this decade for all institutions is ensuring that the identity of the person who is being assessed is the same as the person receiving the accreditation. This important issue is prevalent across essays, submissions, course work and exams.

Janison is conducting research and development in this area that allows institutions to improve their confidence in identity Forms of identity factors that can be developed include:

  • Facial recognition – Did you know algorithms have been shown to exceed human ability in terms of recognition? We’re keen to help Universities reduce any margin of error during the exam registration process.
  • Voice recognition.
  • Fingerprint biometrics – We are working with hardware partners such as Google to ensure we can integrate with the next generation of devices fingerprint capabilities as part of the authentication process of gaining access to an exam.
  • Typing signatures.
  • Editing and creative writing signatures.

 

What are your University’s next steps?

students-on-devices

Open the discussion

Talk to us about your university’s unique challenges and opportunities. We’ll help you identify the approaches that help you realise business value sooner.

Request a demo

After identifying the features and needs that matter most to you, we can tailor a presentation that matches your context so you can make better decisions on what next. One size doesn’t fit all here.

Run a pilot

The fastest way to learn, is to do. We’re helping organisations run pilot programs so they can see first-hand how online exams will work in their environment. We then work with you to scale up when you’re ready.

Talk to us about your goals.

 

*Based on $50 per test, 8 exams per year per student, 718,679 students taking Bachelor Degree in 2015, as per Education.gov.au.
**Based on $50 per test, 8 exams per year per student, 30% savings in end-to-end costs.

What are the five key questions towards improving exams by 2020?

Universities must aim to improve themselves over the next 3 years. There are many opportunities, challenges and threats they need to fend off to remain competitive and relevant.

Wait — Universities will always be relevant… won’t they?

Not without a committed plan to remain so. Historically, Universities have had a monopoly on education. There were no alternatives to learn advanced concepts but through formal education. With the rise of the Internet, Learning Management Systems and MOOCs, anyone with a mobile device has access to ever-increasing banks of knowledge. Universities are even starting to award credits from completed MOOCs towards degrees — how long until they compete directly?

The rise of self-taught professionals, entrepreneur & startup culture, and just-in-time learning resources mean there are more viable alternatives than going to University than ever before. More than simply viable; these alternatives reflect modern life more accurately, and evolve faster, than Universities do. People live, work and learn in collaboration with others on digital platforms. However, in a case of massive disconnect, students at Universities aren’t assessed on this ability – instead are graded on their ability to transfer knowledge through a pencil onto paper. After which, they go back to the ‘real world’ again.

Even the geographic advantage of Universities is disappearing. Students in Australia can now enroll online at American Institutions. Universities need to provide a compelling case for students to enroll locally. Improving exams by 2020 is one way Universities can fight towards remaining competitive and relevant for future students.

What are the five key questions towards improving exams by 2020?

 

1. How will we address the disconnect between pen on paper exams, and students’ everyday experiences of study, work and life?

One option is to switch the focus from ‘traditional assessment’ to being performance-based. This involves assessors supervising and marking group activities, conducting interviews, and setting up other real-world scenarios in which to determine a candidates ability. This approach is used heavily in medical and sporting fields, and could be extended to many others with some creative thinking.

Another option is to migrate pen-and-paper exams to a digital delivery platform. There are many benefits to this — for both the candidates and staff at all levels of the institution. Moving online is a journey — see this 3-minute video for an quick overview.

2. How will we leverage the proliferation of personal mobile devices?

 

A survey conducted by University of Queensland found that Mobile device ownership (excluding desktop computers) was an average of 2.3 devices per student ( Hillier, M (2015) “e-Exams: The story so far”). Students already have devices they are familiar and comfortable with, and use for study and work purposes. We are now at a point where these same devices can be leveraged for examinations, via bring-your-on-device policies There are at least two approaches here:

  1. Exams are run in ‘lock-down’ modes — preventing access to the internet. This itself can be achieved in a variety of ways, two of which are:
    -Device uses a bootable environment (USB) which has exam software and prevents internet access.
    -A lightweight client is installed (via download/app store), which either acts as a ‘safe-exam browser’ (no internet), or which goes on to download an encrypted exam, only unlockable to those in the exam center later on.
  2. Exam papers are designed to be ‘open book’ — allowing internet access (and tracking it). Personally I hope to see this as a future movement, where the ability to filter and find information online is part of a given assessment — better reflecting skills required in today’s workforce.

Leveraging the technology that students already have means there are less hurdles than ever for Universities to consider migrating their exams online.

3. How will we improve the quality and efficiency of our exam process?

Two key drivers for change that are sector agnostic are quality and efficiency. These often seem like opposing forces, however digital exams is a great example where both can be optimised.

Quality
Across an exam paper lifecycle, there are key points where quality can be improved:

  1. Authoring — item banks, review workflows, item analysis, item seeding (testing non-scoring items during exams to see how well they perform), and smart branching and adaptive tests all contribute to a better quality exam paper.
  2. Delivery — digital student responses are more legible and easier to format and edit than doing so on paper.
  3. Marking — digital marking systems can facilitate far greater transparency, process, quality controls and real time metrics and monitoring than paper based marking. Tolerances for double-marking and other quality controls can be fine-tuned mid-process, to ensure the marking activity is a well-oiled machine. It’s a key factor in why Singapore, #1 in PISA rankings, is moving their national O-level and A-levels online.

When marking quality is paramount, the best Education organisations in the world have gone digital.

Efficiency
As above, efficiency gains can be seen in a number of areas of an exam paper lifecycle. Authoring can leverage item banks of pre-seeded and pre-tagged questions, meaning writing new papers is less about starting from scratch, and more about curating the exam. It frees up exam-setters to spend more time thinking about the exam as a whole, instead of doing grunt work.

Digital marking again comes to the fore in terms of efficiency. Marking can begin as soon as the candidates have submitted their electronic responses, various levels of artificial intelligence can automark, semi-mark, and pre-score different question types, while the system handles all the response-management, metrics, quality controls and response distribution. Australia’s NAPLAN program is moving online in a large part due to the efficiency gains it will bring, enabling data to get flow back to the schools sooner.

4. How will we mitigate the risks inherent in high stakes exams?

There are many risks to think about when delivering exams. Let’s focus on four of them: exam paper leaks, poor exam quality, plagiarism, and impersonation.

Exam Paper Leaks: Last month, the UK’s national spelling test wasmistakenly published on a government website. While it was a low-stakes test, the public were urged not to ‘pass on’ the information (really?). Again in the UK (this month), a rogue marker published a SATS test online. What if these were high-stakes tests? The disruption to school’s schedules and reputational damage would be immense.

Technology can play a part in mitigating these risks in a number of ways:

  1. Increased security for login, such as two-factor authentication. This reduces the impact of a password-on-a-post-it-note, falling into the wrong hands.
  2. Limited access to the whole exam — item banks and randomisation mean only the most trusted sources can see the ‘full picture’ of a possible exam paper.
  3. Workflow statuses and tags reduce the chance of mistaken identity when it comes to differentiating practice and real exams.

Poor exam quality: Without digital systems that enforce process, reviews, quality and are backed by item analysis, mistakes are inevitable. This month, students took to Twitter on masse accusing Exexcel of setting the wrong paperafter the questions were seemed impossible without a calculator.

Plagiarism: Some students will go to extraordinar lengths to copy the work of others during an exam. Last month, three Thai students brought in smart-glasses linked to smart-watches to communicate the exam paper to outside tutors. While cell-phone blocking technology has been used in China to prevent exam cheating, most of that tech remains illegal to use in the West (so we don’t recommend it). However digital exams can play a smaller part here; enhanced randomisation and exam-branching reduce the chance of copying the work of any student around you, while linear navigation options mean you don’t have access to the whole exam upfront (only one question at a time), which could reduce efficiency of any back-and-forth comms.

Impersonation: A study from the University of Sydney found students are “increasingly paying impersonators to sit their exams”.Exam invigilators are often external contractors, and do not know the students personally.

Technology solutions such as facial recognition, audio recognition and keyboard signature profiling can alert invigilators to suspicious behavior . They can then investigate further to determine if the real candidate is present.

5. How will we maximise our capability to collect and analyse exam results?

Without an end-to-end online exam system, the process of measuring, calculating and analysing results is either lacking or inefficient. With online exams, you can achieve a closed-loop system which facilitates consistent and automated processes:

  • Data mining, available as soon as delivery and marking activities are complete.
  • Calculation and psychometric analysis that feeds back into your Item Bank.
  • Additional insights and trends that can be extracted from BI tools.

“We’re going to start answering these questions!” — Head Change Agents @ your University.

Here’s some more resources to help you get started on the journey:

  • Beg/borrow/steal/leverage this Business Case for Online Exams.
  • Get in contact around our maturity model, and how we can help you start realising business value now, without the need for a big bang approach.

Universities are great — let’s ensure the student experience remains great too (in addition to the toga parties).

Why Online Assessments should be ‘Open Book’

Did you ever sit an ‘open book’ exam growing up? You probably went through the three emotions I did.

  1. Happiness — that we don’t need to study, since we’d have our text books.
  2. Panic — as we sat down and read the exam, and realised the book was going to be useless.
  3. Understanding — that the Teacher didn’t want us to recall facts and figures, but to display a deeper understanding.

These exams were actually more engaging as a student. You really had to use a combination of memory, logical thinking and creativity to get through them. Not to mention time management, as the brief flicker of hope that the text book might somehow help you never truly died out.

Continue reading Why Online Assessments should be ‘Open Book’

Why Collaborative Assessment will be King

The last piece in the 21st century skills puzzle

School curricula of the past have focused on classic disciplines; algebra, history, chemistry, geography. While these have merit, there is a change in the education zeitgeist; a focus on 21st century skills.

In 2012, a study called ATC21S identified 10 skills that people will need to master to be successful in today’s world.

Continue reading Why Collaborative Assessment will be King

ABOUT US

We are at the forefront of innovation in the development and delivery of award-winning learning and assessment technologies that improve the performance of organisations and the learning experiences of employees and students, around the globe.

HELP & SUPPORT

Help: help.janison.com.au

Australia: 1300 857 687

International: +61 2 6652 9850

Fax: +61 2 6652 9855