Report on Exploring the Efficacy and Challenges of Proctorio
in Open Book Examinations

Abstract: As the landscape of education evolves, online proctoring tools like
Proctorio have emerged as potential solutions for facilitating open-book
examinations. This case study examines the efficacy and challenges of using
Proctorio in open-book exams through a mixed-methods approach, analyzing
student surveys and lecturer feedback from a sample of 150 law students and
their corresponding lecturers. While the findings reveal a mixed response, they
offer valuable insights into the potential and pitfalls of this technology,
informing future decisions about its implementation in open-book assessment
formats.

Introduction: Open-book examinations offer a unique opportunity for students
to demonstrate their critical thinking and application skills while accessing
relevant resources.1 However, the shift away from traditional, invigilated
settings raises concerns about academic integrity and the need for effective
proctoring solutions.2 Proctorio, an online proctoring tool utilizing webcam and
microphone monitoring, has gained traction in recent years, but its efficacy,
particularly in open-book settings, remains a subject of debate.3 It’s use,
prompts the question as to whether Proctorio enables authentic assessments.4
This case study delves into the experiences of both students and lecturers to
shed light on the benefits and drawbacks of using Proctorio in open-book exams

Methodology: To capture a comprehensive picture, the study employed a
mixed-methods approach.5 A survey was administered to 150 law students who
had taken open-book exams using Proctorio, exploring their perceptions of the
technology’s ease of use, effectiveness in maintaining academic integrity, and
overall impact on their exam experience. Additionally, feedback from lecturers

__________________________________________________________________

¹ Green, S. G., Ferrante, C. J., & Heppard, K. A. ‘Using Open-Book Exams to Enhance Student Learning,
Performance, and Motivation’ (2016) 16 (1) The Journal of Effective Teaching 19-35. ² Reedy, A., Pfitzner, D., Rook, L. et al. ‘Responding to the COVID-19 emergency: Student and Academic
Staff Perceptions of Academic Integrity in the Transition to Online Exams at three Australian Universities’
(2021) 17 (9) Int J Educ Integr (2021). https://doi.org/10.1007/s40979-021-00075-9 ³ Bergmans, L., Bouali, N., Luttikhuis, M. and Rensink, A. ‘On the Efficacy of Online Proctoring using
Proctorio’ in proceedings of the 13th International Conference on Computer Supported Education (CSEDU
2021) – Volume 1, pages 279-290, DOI: 10.5220/0010399602790290. See also, Mohammed Juned Hussein,
Javed Yusuf, Arpana Sandhya Deb, Letila Fong & Som Naidu, ‘An evaluation of Online Proctoring Tools’
(2020) 12 (4) Open Praxis 509-525 ⁴ Jacalyn Lund (1997) Authentic Assessment: Its Development & Applications, Journal of Physical
Education, Recreation & Dance, 68:7, 25-28, DOI: 10.1080/07303084.1997.10604979 ⁵ Roberta Heale & Alison Twycross, Validity and reliability in quantitative studies, Evid Based Nurs, Vol 18(3),
2015, 66-67

who had to address student queries and concerns during Proctorio-proctored
exams was collected and analyzed. This qualitative data provided valuable
insights into the logistical and emotional challenges faced by both students and
lecturers.
Findings: The student survey revealed a divided response to Proctorio. While
60% found the platform smooth and user-friendly, highlighting its convenience
and time-saving benefits of taking exams from home, 40% encountered
technical difficulties, reporting frustration and anxiety during the exam. These
technical issues included software glitches, internet connectivity problems, and
difficulties navigating the interface. Notably, students who experienced
technical hiccups expressed concerns about the fairness and validity of their
exam performance, raising questions about the reliability of Proctorio as a
proctoring tool._⁶
The lecturer feedback corroborated these concerns. Some lecturers reported a
significant increase in student queries and anxieties related to Proctorio
technical issues. Addressing these concerns often required additional time and
resources, exceeding the workload typically associated with traditional
invigilated exams. Additionally, some lecturers expressed discomfort with the
intrusive nature of webcam monitoring, highlighting concerns about student
privacy and the potential for psychological pressure.⁷
However, the lecturer feedback also included positive aspects. Some lecturers
appreciated the flexibility and convenience offered by Proctorio, particularly for
students facing geographical or logistical challenges. Additionally, the potential
for increased academic integrity through real-time monitoring was seen as a
benefit by some, particularly in cases where traditional invigilation is
impractical.

Discussion: The mixed findings of this case study highlight the complex interplay
of factors influencing the efficacy of Proctorio in open-book exams. While the
technology offers undeniable advantages in terms of convenience and flexibility,
the potential for technical difficulties, student anxiety, and increased workload
for lecturers raises significant concerns.

________________________________________________________

⁶ Bergmans (n3) 287 ⁷ Mohammad Khalil, Paul Prinsloo & Sharon Slade ‘In the nexus of integrity and surveillance: Proctoring (re)
considered’ (2022) 38 Journal of Computer Assisted Learning 1589–1602.