At SmarterServices, we regularly have conversations with colleagues about their assessment strategies and how they ensure academic integrity. One comment that we sometimes hear is, "We don't use proctored exams because we use authentic assessment." But with the rapid proliferation of ChatGPT and other artificial intelligence assisted writing tools, educators need to ask, "How authentic is authentic assessment?"
In this blog post, we will explore strategies that schools can use to foster academic integrity, even with written forms of authentic assessment.
There are many ways to assess learning. The type of assessment selected is not a matter of right or wrong, but a decision based on several factors such as the nature of the subject matter and the rigor of the exam. Here are a few types of assessment questions that instructors may use to determine the level of content mastery:
Objective Assessment – This is the most common form of assessment, especially for online courses. Objective assessment items typically take the form of multiple choice, true or false, or matching questions. An advanced form of objective assessment is a situational judgement test in which students are presented with a scenario and then they must indicate how they would respond. Based on their response, further scenario details may be provided, resulting in additional response choices. These tests can be administered entirely online and because they can be automatically scored, students instantly view their grades. Students may also receive qualitative feedback that explains the questions they missed. Item banks can be used to provide a unique set of questions for each student.
Skills Test – While objective assessments are great at allowing a student to demonstrate what they know, skills tests are appropriate for following a student to demonstrate what they can do. Sometimes the instructor needs to see that the learner can utilize certain tools that they will likely use in the workplace. This could be an actual tool, such as a welder. While skills tests such as this are commonly done face-to-face, they could be achieved in an online course using student-produced video evidence. Other skills tests can be done online could be using software such as a spreadsheet that a student could use to demonstrate application of accounting processes.
Oral Assessment – Perhaps the assessment type that many students fear the most is the dreaded oral assessment. Students are asked a question by the instructor, and they must then immediately provide an oral response. Historically, when this type of assessment is used in a classroom, other learners may comprise an audience, making the experience even more anxiety provoking. In online courses, students may be connected via a video conferencing platform for the oral test.
Written Assignment – The form of authentic assessment that schools most often use is a written assignment. Students are tasked to compose an essay on a given topic. Being able to articulate one's knowledge in writing is a classic form of assessment. But with technology now being able to instantly do the composition, this form of assessment is coming under scrutiny.
According to the History of Calculators provided by EdTech magazine, hand-held calculators first became affordable enough for use in schools in the mid to last 1970s. By the mid-80s, handheld graphing calculators were commonplace. The advent of this disruptive technology prompted great debate among math faculty about whether students should be able to use calculators, especially during tests.
A parallel disruptive technology is now emerging that is to writing what the calculator is to math. Artificial Intelligence Writing, also referred to as AI Writing, is content composed by a computing device. Just like a calculator performs the math computations and provides the answer, AI assisted writing gathers the data, performs the composition, and provides the written assignment. ChatGPT is currently one of the most common platforms, and others such as GrammarlyGO are rapidly growing in usage. According to a survey conducted by Forbes.com, 43% of college students reported that they have used ChatGPT for their assignments.
So, how are schools to respond to such technology? Typically, students are taught how to do mathematical operations, such as long division, so that they comprehend and appreciate the process. But after that, they are allowed to use the calculator since in "real life" they will likely have access to a calculator as well.
What will "real life" look like over the next decade for writing? Is there a way that schools can teach students the art and science of writing correctly, but then allow them to use AI assisted tools to make the process more efficient?
In an article titled "How to Use ChatGPT as a Learning Tool," the American Psychological Association recommends and provides suggestions for how educators can view it as an opportunity rather than a threat. While the AI may assemble a quick representation of the facts, educators should encourage the students to be creative and use critical thinking to apply the facts to a specific situation.
Does this mean that students should be allowed to use ChatGPT for written assignments during the course but not during an exam? Not necessarily. SmarterProctoring can actually be configured to allow students to open other tabs during a proctored session, while still maintaining control of other computing functions.
Here is one use case of this strategy. Suppose you are teaching an Introduction to Psychology course. For the two-hour final exam, students will be instructed to compose four essays on the general topics of (1) the human brain, (2) sensation and perception, (3) memory and learning, and (4) psychological theories. Students are informed before the exam that during the exam, the instructor will allow the student to view other websites related to each topic and the student can use ChatGPT as well. Then during the exam, SmarterProctoring records the screen, video, and audio of the student as they use these resources to compose their own unique essays. This would allow the student to demonstrate that they can use these information gathering and essay composing tools in a way that allows them to highlight their own creativity and critical thinking.
So, just like SmarterProctoring provides an integrated online calculator, ChatGPT or other AI writing tools can also still be used even during a proctored exam.
As your school is grappling with how to respond to ChatGPT or other AI writing tools, check out this article by Jack Caulfield, which is a compilation of existing instructional policies. He found that 51% of universities allow individual instructors to decide on their own policy, 27% of schools have no clear guidance or policy, 18% of institutions ban the tools by default, and 4% allow their use with proper citation.
Here is one closing thought about "authentic assessment." The title "authentic assessment" seems to imply that other forms of assessment are not authentic. As stated earlier in this post, the choice of assessment question type is not a matter of right or wrong. It is not a matter of authentic vs. non-authentic. But rather the nature of the subject matter should determine the optimal way for students to demonstrate their mastery. If that demonstration can be done online, then SmarterProctoring can be used to control and monitor the online environment during the testing session.