Introduction

The onset of the COVID-19 pandemic brought expanded use of digital platforms designed to support remote testing and uphold academic integrity standards using automated features powered by artificial intelligence (AI). Eproctoring services existed before the pandemic (Dimeo, 2017), but COVID-19 has been a catalyst for the rapid increase in the use of these services and has inspired institutions to seek novel ways to take advantage of the increasingly sophisticated methods available to detect and deter cheaters in the online environment (Flaherty, 2020; Lawson, 2020). It is difficult to find aggregated data on eproctoring as an industry, but considering that a single company oversaw over a million exams in one month and is adding 100s of new clients in a single year (Harwell, 2020), it is safe to say that this segment of education technology is a burgeoning one. During the pandemic, over half of institutions who responded to an EDUCAUSE poll reported using eproctoring and another 23% indicated that they are planning to adopt this practice (Grajek, 2020). Companies already in operation have expanded their range of services and increased their level of technological sophistication, and new companies have emerged to take advantage of the sudden increase in demand (Flaherty, 2020).

Multiple choice question-based exams have increased over the past several decades, due to a trend towards increased class sizes, limited resources, and the availability of new technologies (Nichols, 2017). The rapid transition in 2019-2020 towards remote learning (dubbed by some as “pandemic pedagogy”) (Barbour et al., 2020) saw many colleges and universities translate traditional activities into a digital mode (including standard assessment methods), without adequate time and resources to fully understand and respond to this new teaching environment and the pedagogical challenges it presented. The phenomenon of transplanting pedagogical conventions into an online dimension has been dubbed “emergency remote teaching”, and Hodges et al. (2020) argued that it should be considered as a separate category to true online learning. Anticipating that students may be more likely to cheat in an unproctored environment (Dyer, 2020), institutions and faculty increasingly chose to rely on eproctoring as a logical solution to a clearly defined problem. While defensible in a time of crisis, many ethical issues and serious concerns related to the adoption of eproctoring tools should now prompt post-secondary institutions to rethink their use of tools that rely on invasive surveillance. As a second-order consequence, it also prompts reconsideration of the use of timed, high-stakes, memory-based testing in the post-secondary education (PSE) environment.

These ethical concerns with eproctoring tools are related to privacy and security; mental health, diversity and discrimination; and autonomy and independence. As a contrast to many of the articles published in the popular media on this topic, this paper will not argue that institutions should immediately divest themselves of all eproctoring tools. Instead, this paper argues that institutions should limit the risk presented by eproctoring software solutions while simultaneously engaging in a process that would see a shift towards authentic assessments that simulate what students might expect to encounter in their career, require critical thinking, and allow for multiple attempts and access to resource material (Center for Teaching and Learning, 2019).

Overview of Ethical Concerns

Educational institutions have a long history of using private educational technology companies that rely on large data sets as partners in academic content and service delivery (Regan & Jesse, 2019). As such, there was a ready supply of companies able to meet the perceived needs of institutions to operate in a digital mode in a way that was analogous to face-to-face delivery. However, there is a price to be paid for the convenience and efficiencies these platforms purport to offer, and students may have been burdened with much of this cost. As will be described below, the costs for students disqualifies these tools as a long-term solution to the question of how to ensure knowledge transmission and meet the institution’s need to provide instructional continuity.

Privacy and Security

For eproctoring service providers to deliver on their value proposition, it is necessary for them to adopt a variety of computer-mediated, automated features that impinge on the privacy of its users. This includes facial detection/recognition, recording of the user’s screen and immediate physical environment, browser lock-down, monitoring and analysis of system usage, keystroke/mouse movement capturing, audio monitoring, and other features, depending on the selected vendor and level of service. If a student behaves in a way that either the remote proctor or the algorithm deems to be suspicious, the system triggers a notification either for review by the faculty member upon completion of the exam or in some cases the student may be notified in real-time of the potential infraction (Dimeo, 2018; Flaherty, 2020).

The result is a large amount of identifiable data located on the servers of a private company, including unique facial, voice, and behavioral data (Stewart, 2020). The data gleaned from these practices is a valuable commodity to the company and, given the propensity of the vendors to guard their source code, some institutions feel that they are in the position of having to trust that the companies are behaving in a responsible way (Hippensteel, 2021). There are safeguards in place, in the form of privacy and security terms offered by vendors; contracts that specify acceptable use of this data; and privacy legislation, but the misuse of data can be difficult to detect and police (Swauger, 2020a). The risk of a data breach may be relatively low but is still a genuine concern, given that an eproctoring-related breach already occurred in Australia, and privacy breaches in general are quite common for digital service providers (Johnston, 2020). Given the amount of private information stored in these recordings, the consequences may be higher than with other educational technology applications that store data on private servers. It has been suggested that a feeling of being watched may be anxiety-provoking (Reed, 2020), and this feeling would be accentuated when the viewer is provided with heightened access to intimate details of the students’ home environment, including their living spaces, physical attributes, and even bathroom habits.

Mental Health, Diversity, and Discrimination

While concern for privacy has been identified as a major issue for students and faculty alike, many recent articles focus primarily on the psychological impacts of being watched while undertaking an already stressful activity (Swauger, 2020b). Users object to the “creepiness” factor related to eproctoring: the disturbing feeling of being watched, the additional stress related to a sense of someone tracking your movements, fear of being penalized for bathroom breaks, and the surreal sensation of exposing environments such as one’s bedroom to a stranger (Chin, 2020b). Journalists have shared stories of extreme anxiety caused by the fear that the AI imbedded in an eproctoring service might flag a long gaze, unusual typing patterns, scrolling too quickly, completing the exam too early, or the potential that another person in their house might accidentally enter the frame (Harwell, 2020).

Students and their advocates have argued that the adoption of eproctoring tools unnecessarily increases students’ stress associated with test taking to excessive degrees, especially impacting students with mental health conditions (Chin, 2020a). Students with cognitive disabilities report being flagged for reading questions aloud and students with involuntary vocal or physical tics or a need for self-stimulation have claimed that they were flagged by the algorithm for suspicious sounds and motion (Harris, 2020; Flaherty, 2020). Students of colour have reported that the facial recognition technology is not effective at identifying them and creates the conditions for potential consequences relating to their academic standing and progression (Harris, 2020). Their concerns are well founded, as it has been demonstrated that facial recognition and detection is much less effective for women and students of colour due to a variety of factors. These factors may include implicit bias built into the programming of the hardware and software as well as the lack of black and brown faces in the databanks that these programs use to execute this function (Buolamwini & Gebru, 2018).

It could be argued that test-taking is an inherently stressful experience and while it has not been conclusively determined that eproctored tests are more stressful than exam-taking in general, and there is some evidence that some students feel less anxious in remote testing environments (Japp et al., 2021). It is also true that there is no guarantee that human proctors in a face-to-face setting do not harbour implicit or explicit biases. However, there is enough anecdotal evidence to suggest that the trauma caused by surveillance-based software—coupled with evidence that students are still able to use services like CHEGG to cheat, despite every effort being made to ensure that students complete the assessment without unsanctioned assistance (Adams, 2021)—is enough to motivate educational institutions to investigate and consider implementing alternative approaches to eproctoring services.

Autonomy and Independence

A common theme that pervades this topic is one of agency, power, and control. This concerns both students who feel subject to the whims of their institution and their chosen corporate partners, and faculty who may feel similarly constrained by choices made by their administration. Ceding too much control to third parties is not only detrimental to the student but could weaken the foundation of the educational institution and may hamper the academic freedom of the faculty member. However, a strict prohibition on eproctoring solutions also places limits on academic freedom by denying the faculty member the right to choose their preferred mode of assessment for their circumstances. Institutions and the administrators who are responsible for business decisions that impact the overall success of the college or university also have the right to keep at their disposal tools that may, in a limited and prescribed manner, enable them to remain innovative in a highly competitive environment.

All the above considerations represent a complex interplay of competing rights. The students’ rights relating to mental health, disability accommodations, privacy, or objections of conscience should be respected. Honouring student rights must also be viewed in tandem with the faculty member’s academic freedom and right to assess students in a manner they deem to be the most appropriate for their field, industry, or to meet certification requirements. This speaks to the need to find a balanced approach that emphasizes the careful application of policy as well as reason, compassion, and respect for the autonomy of both staff and students. In addition, the rights of the institution and its mission to provide quality and reputable programs must also be considered.

Institutions can mitigate concerns about rights by yielding power back to those affected by these choices. Giving students and faculty clear explanations of their options for remote assessment and the rationale for the decision behind the use of the software better facilitates informed decision-making and builds trust between the affected parties. Institutions should balance their interest in maintaining high academic integrity standards with an equally robust concern for empathy and agency. Also, institutions should recognize their power and influence by limiting their business relationships to only companies that comport to their institutional values. There are examples of companies posting consumer-support chat logs into a Reddit thread to shame a student who complained about the service (Zhou, 2020) and excessively litigious companies who have resorted to taking legal action against a professor who posted training videos on Twitter (Chin, 2020a). Given the preponderance of eproctoring options in the private sector, institutions should only consider companies that demonstrate and commit to the highest ethical standards.

Recommendations

The COVID-19 pandemic is a tragedy, but also presented an unprecedented opportunity to affect new ways of teaching and learning in online environments. The pandemic and shift to online teaching and learning has exposed weaknesses in the operations of post-secondary institutions. In particular, a digital-default or hybrid content delivery mode, with large class sizes and limited faculty development time, poses an inherent challenge to PSE’s core mission. However, an impulsive decision to immediately sever eproctoring contracts may be as problematic as the reactive decision to implement eproctoring modules at the onset of the pandemic.

Post-secondary institutions are heterogenous, complex, and often conservative entities that are made up of faculty who enjoy considerable autonomy and academic freedom. Instead of a prohibitive approach, educational institutions should embark on a long-term strategy that encourages faculty to adopt more creative, authentic assessments and foster a spirit of collaboration and community (Anderson, 2017). Faculty with an interest in simulating exam conditions (e.g., in regulated professions that require a standardized test administered by a governing body) could consider frequent low-stakes testing (or “retrieval practice”) as an alternative to “make-or-break” exams that create the type of stressful conditions that may motivate some students to cheat (Paul, 2015). Online assessments may offer great value as a pedagogical tool and have many practical applications that go beyond a mere determiner of grades. Means et al. (2014) cite several other roles for online assessments, from determining if a student is ready for new content, providing faculty with information about learning states, and identifying struggling students.

The nature of authentic assessments can discourage or make cheating an impossibility, as these types of assessments are based on providing unique responses to specific circumstances instead of choosing the “right” answers from a predetermined list (Sotiriadou et al., 2020). By adopting authentic assessment strategies, we can further shift the conversation towards questions about whether students are learning instead of how to deter cheating (Bertram Gallant, 2008). Institutions should initiate one-time or ongoing investment in faculty development and increase support for curriculum designers and online learning specialists to facilitate this culture shift. These costs could be recouped or defrayed in the long term by avoiding the licensing fees related to the software solutions currently in place. In the short-term, a process could be developed that requires faculty to justify the use of eproctoring and demonstrate that alternative assessments have been considered and ruled out—not related to personal preference or convenience, but guided by theoretical frameworks or exigencies such as student-instructor ratio (Hodges et al, 2020).

If one accepts the argument that eproctoring may be necessary to offer as an option while longer-term strategies for authentic assessments are pursued, there are several practices institutions can employ to reduce the impact on student privacy and mental health and regain their ethical footing:

• Use eproctoring solutions with the least invasive features. In line with recommendations of the Dutch Data Protection Agency (2020), avoid platforms that use synchronous monitoring of student workspaces while the exam is in progress. Limit access to the recordings to the smallest number of trained adjudicators with the faculty member as the determiner if the flag is a false-positive or genuine concern (Dawson, 2020).
• Require that potential and current vendors complete a process like the HECVAT questionnaire (EDUCAUSE, 2020), a report that provides the institution with an exhaustive overview of their policies and practices related to data security and privacy including information on staff training protocols and procedures.
• Conduct a Privacy Impact Assessment using a process similar to what is recommended by the Privacy Commissioner of Ontario (2015). New and existing contracts should be assessed to determine compliance with privacy legislation. Information used via this process can be used to affect changes in practice with vendors currently not meeting the standards or expectations of the institution (e.g., reducing retention schedule to statutory minimums).
• Enter a separately negotiated contract and privacy terms with vendors that exist outside of their standard service level agreement and override any posted terms on the vendor’s website. Publicly post institution-specific contracts on dedicated website. Queen’s University is an exemplar institution vis-a-vis clear communication with respect to transparency and clarity (Queen’s University, n.d.)
• Set clear expectations for vendors regarding professional conduct and demonstrate a willingness to terminate contracts with companies that do not align with institutional values, policies, and legislation.
• Conspicuously post information about all aspects of the eproctoring experience, including details about technical requirements, expectations for the in-exam experience, how to prepare your workspace, what the faculty role is with respect to following up on potential AI flags, and a coherent rationale for why eproctoring may be chosen by faculty as the best or only option available to meet the learning objectives of the course (Dawson, 2020)
• Broadly communicate the options available to students with disability accommodations and clearly outline pathways for these students to access alternative assessments that ensure the learning outcomes are met. Examples include testing in a physical space, or non-algorithmic online, remote testing in which a human invigilator monitors the students’ screens and uses their discretion to manually flag suspicious behavior.

Conclusion

Eproctoring raises concerns related to privacy, mental health and safety, and student autonomy and independence. The widespread adoption of eproctoring software represents a textbook case of products being acquired as a response to external conditions, where administrations make long-term sacrifices (of student privacy and potentially their online safety) for the sake of the short-term maintenance of the status quo (with respect to assessment practices). This should not preclude institutions from reconsidering their choices and honestly evaluating their decisions now that their impact is more easily measured. In the short term, institutions can recognize their power and agency and compel vendors to improve their terms and practices to lower the risk of harm for all students, with the option to patronize other vendors if their chosen corporate partner does not share their values.

While these short-term solutions may limit harm, the long-term solution should be to improve student outcomes by transitioning away from multiple-choice assessments and ending harmful practices that rely on surveillance and breed a sense of distrust among students and faculty. In the long term, institutions should support faculty and students in moving towards authentic assessments of meaningful student work. Achieving this type of attitudinal, cultural, and operational shift requires time, resources, and buy-in, and hence a short-term continuation of eproctoring in a conditional and limited manner will allow institutions to reprioritize budget allocations and revise academic planning processes to support this effort and give faculty the space they require to make the necessary adjustments, all the while respecting their academic freedom.

References

Adams, S. (2021, March 31). This $12 billion company is getting rich off students cheating their way through COVID. Forbes. https://www.forbes.com/sites/susanadams/2021/01/28/this-12-billion-company-is-getting-rich-off-students-cheating-their-way-through-covid/

Anderson, T. (2017, February 1). How communities of inquiry drive teaching and learning in the digital age. Contact North Online Learning. Contact North. https://teachonline.ca/tools-trends/how-communities-inquiry-drive-teaching-and-learning-digital-age

Barbour, M., LaBonte, R., Kelly, K., Hodges, C., Moore, S., Lockee, B., Trust, T., Bond, A., & Hill, P. (2020). Understanding pandemic pedagogy. Differences between emergency remote, remote, and online teaching. CANeLearn: K-12 Remote Learning in Canada. https://www.researchgate.net/publication/347535715_Understanding_Pandemic_Pedagogy_Differences_Between_Emergency_Remote_Remote_and_Online_Teaching

Bertram Gallant, T. (2008). Academic Integrity in the 21st Century: A teaching and learning imperative. ASHE Higher Education Report.

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1-15. http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf

Dimeo, J. (2017, May 10). Online exam proctoring catches cheaters, raises concerns. Inside Higher Ed.  https://www.insidehighered.com/digital-learning/article/2017/05/10/online-exam-proctoring-catches-cheaters-raises-concerns

Center for Teaching and Learning. (2019, March 26). Authentic Assessment in the Online Classroom. Wiley Education Services. https://ctl.wiley.com/authentic-assessment-in-the-online-classroom/

Chin, M. (2020a, October 22). An ed-tech specialist spoke out about remote testing software—And now he’s being sued. The Verge. https://www.theverge.com/2020/10/22/21526792/proctorio-online-test-proctoring-lawsuit-universities-students-coronavirus

Chin, M. (2020b, April 29). Exam anxiety: How remote test-proctoring is creeping students out. The Verge. https://www.theverge.com/2020/4/29/21232777/examity-remote-test-proctoring-online-class-education

EDUCAUSE. (2020, April 6). Higher Education Community Vendor Assessment Toolkit. https://library.educause.edu/resources/2020/4/higher-education-community-vendor-assessment-toolkit

Dawson, P. (2020). Defending assessment security in a digital world: Preventing e-cheating and supporting academic integrity in higher education. Taylor & Francis Group. http://ebookcentral.proquest.com/lib/fleming-ebooks/detail.action?docID=6340937

Dutch Data Protection Authority (2020, December 17). Investigation of online voice and video calls and online proctoring in education. https://autoriteitpersoonsgegevens.nl/en/news/recommendations-privacy-during-remote-online-education

Dyer, J. M. (2020). Academic dishonesty and testing: How student beliefs and test settings impact decisions to cheat. Journal of the National College Testing Association, 4 (1), 1-30. https://www.ncta-testing.org/assets/docs/JNCTA/2020%20-%20JNCTA%20-%20Academic%20Dishonesty%20and%20Testing.pdf

Flaherty, C. (2020, May 11). Online proctoring is surging during COVID-19. Inside Higher Ed. https://www.insidehighered.com/news/2020/05/11/online-proctoring-surging-during-covid-19

Grajek, S. (2020, April 10). QuickPoll results: Grading and proctoring. EDUCAUSE. https://er.educause.edu/blogs/2020/4/educause-covid-19-quickpoll-results-grading-and-proctoring

Harris, M. (2020, October 4). Viral TikTok: Student fails exam after AI software flags for cheating. Insider.  https://www.insider.com/viral-tiktok-student-fails-exam-after-ai-software-flags-cheating-2020-10

Harwell, D. (2020, April 1). Mass school closures in the wake of the coronavirus are driving a new wave of student surveillance. Washington Post. https://www.washingtonpost.com/technology/2020/04/01/online-proctoring-college-exams-coronavirus/

Hippensteel, C. (2021, February 9). Online learning threatens students’ privacy, experts say. The Daily Orange. http://dailyorange.com/2021/02/online-learning-threatens-students-privacy-experts-say/

Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond A. (2020, March 17). The difference between emergency remote teaching and online learning. EDCAUSE Review.  https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning

Information and Privacy Commissioner for Ontario (2015, May). Planning for success: Privacy impact assessment guide. https://www.ipc.on.ca/wp-content/uploads/2015/05/planning-for-success-pia-guide.pdf

Jaap, A., Dewar, A., Duncan, C., Fairhurst, K., Hope, D., & Kluth, D. (2021). Effect of remote online exam delivery on student experience and performance in applied knowledge tests. BMC Medical Education, 21(1), 86–86. https://doi.org/10.1186/s12909-021-02521-1

Johnson, M. (2020. August 6). Australian universities investigate online exam tool data breach. ITnews.  https://www.itnews.com.au/news/australian-universities-investigate-online-exam-tool-data-breach-551373

Lawson, S. (2020, April 24). Are schools forcing students to install spyware that invades their privacy as a result of the coronavirus lockdown? Forbes.  https://www.forbes.com/sites/seanlawson/2020/04/24/are-schools-forcing-students-to-install-spyware-that-invades-their-privacy-as-a-result-of-the-coronavirus-lockdown/

Means, B., Bakia, M., & Murphy R. (2014). Learning online: What research tells us about whether, when and how. Routledge.

Paul, A.M. (2015, August 1). Researchers find that frequent tests can boost learning. Scientific American. https://doi.org/10.1038/scientificamerican0815-54

Queen’s University: Office of the University Registrar (n.d.). Remote Proctoring.  http://www.queensu.ca/registrar/students/examinations/exams-office-services/remote-proctoring

Reed, M. (2020, May 13). Should showing faces by mandatory? A new question posed by technology. Inside Higher Ed. https://www.insidehighered.com/blogs/confessions-community-college-dean/should-showing-faces-be-mandatory

Regan, P.M., & Jesse, J. (2019). Ethical challenges of edtech, big data and personalized learning: Twenty-first century student sorting and tracking. Ethics and Information Technology, 21(3), 167–179. https://doi.org/10.1007/s10676-018-9492-2

Sotiriadou, P., Logan, D., Daly, A., & Guest, R. (2020). The role of authentic assessment to preserve academic integrity and promote skill development and employability. Studies in Higher Education, 45(11), 2132–2148. https://doi.org/10.1080/03075079.2019.1582015

Stewart, B. (2020, December 3). Online exam monitoring can invade privacy and erode trust at universities. The Conversation. http://theconversation.com/online-exam-monitoring-can-invade-privacy-and-erode-trust-at-universities-149335

Swauger, S. (2020a, April 2). Our bodies encoded: Algorithmic test proctoring in higher education. Hybrid Pedagogy. https://hybridpedagogy.org/our-bodies-encoded-algorithmic-test-proctoring-in-higher-education/

Swauger, S. (2020b, Nov 7). What’s worse than remote school? Remote test-taking with AI proctors. NBC News.  https://www.nbcnews.com/think/opinion/remote-testing-monitored-ai-failing-students-forced-undergo-it-ncna1246769

Zhou, N. (2020, July 1). CEO of exam monitoring software Proctorio apologises for posting student’s chat logs on Reddit. The Guardian. http://www.theguardian.com/australia-news/2020/jul/01/ceo-of-exam-monitoring-software-proctorio-apologises-for-posting-students-chat-logs-on-reddit