AI.1 Case Study: Large Language Models
Is ChatGPT a writing scaffold?
Chapter 5 described various writing scaffolds but recommended avoiding one modern scaffold: large language models, such as ChatGPT. Large language models learn about a huge corpus of text (billions of words). Given a sentence, a large language model predicts what words should occur next. Given a short prompt, a large language model uses its predictive power to generate long, meaningful responses. Some scientists report using tools like ChatGPT to help edit scientific manuscripts (Mitchell & Krakauer, 2023).
However, I worry about calling a large language model a ‘scaffold’ because the model does most of the writing for you. Professors fret their students turn in exams or essays written not by themselves, but by a large language model. Scientific journals fear researchers use large language models to create research papers, or to write reviews of papers. Danger exists in such practice. (Stokel-Walker & Van Noorden, 2023) The journal Science refuses to publish articles created with the help of a large language model.
Table 5-1 provided one example of ChatGPT’s writing to a prompt. Consider another example of what large language models can do. My ‘History of Modern Psychology’ students recently wrote a take-home, open-book exam in which they answered a general question with an essay. What if some students yielded to the temptation to use ChatGPT? I wondered what ChatGPT would do when prompted by my exam questions. The following paragraphs provide ChatGPT’s response to one question:
Exam Question Prompt: “To some, the cognitive revolution against behaviorism returned psychology to its proper domain: the scientific study of consciousness. Others disagree, believing consciousness has little role in cognitive psychology. Adopt one of these two views, and argue in favor of it, using appropriate examples from the text and the lectures.”
ChatGPT Response: “I will argue in favor of the view that the cognitive revolution against behaviorism returned psychology to its proper domain: the scientific study of consciousness.
Behaviorism, which dominated psychology in the early 20th century, rejected the study of mental processes and focused exclusively on observable behavior. This approach largely ignored the role of consciousness in shaping behavior and mental processes, leading to a gap in our understanding of human cognition.
The cognitive revolution, which emerged in the 1950s and 1960s, challenged the behaviorist approach and emphasized the importance of studying mental processes such as attention, perception, memory, and reasoning. This shift marked a return to the scientific study of consciousness, as cognitive psychologists sought to understand the underlying processes that give rise to our subjective experiences.
One key example of the role of consciousness in cognitive psychology is the study of attention. Attention is a fundamental cognitive process that allows us to selectively attend to certain stimuli in our environment while ignoring others. However, attention is also a conscious experience that we are aware of and can reflect on. Cognitive psychologists have studied attention extensively, examining its neural basis, its role in perception and memory, and its relation to other cognitive processes such as emotion and decision making. These studies have shed light on both the objective and subjective aspects of attention, providing a more complete understanding of this complex cognitive process.
Another example is the study of consciousness in cognitive neuroscience. Advances in brain imaging technologies have allowed researchers to investigate the neural basis of consciousness and the mechanisms that give rise to our subjective experiences. These studies have revealed the complex interplay between brain regions involved in perception, attention, and memory, and have provided insights into the neural correlates of conscious awareness.
In conclusion, the cognitive revolution against behaviorism marked a return to the scientific study of consciousness in psychology. By emphasizing the importance of studying mental processes and their relation to conscious experience, cognitive psychology has provided a more complete understanding of human cognition. The study of attention and consciousness in cognitive psychology and cognitive neuroscience exemplify the role of consciousness in cognitive psychology, supporting the view that consciousness has a central role in the scientific study of human cognition.”
Clearly, ChatGPT generated an interpretable answer to the exam question. It produced an impressive first paragraph which suggested ChatGPT understood the question raised two opposite positions and asked a student to defend one. ChatGPT chose one and then wrote an understandable essay to defend the choice. Given such ability, why did I recommend not using large language models in Chapter 5?
Appendix I addresses my understanding of ChatGPT and its cousins. To prepare, I read several articles about large language models. As I read, I began to think about how large language models relate to cognitive science. Appendix I provides an essay which communicates my thoughts to you.
While Appendix I deals with a topic you might find dry or uninteresting, I try to wrap the topic in a more enticing package. I created my essay using the scaffolds described in earlier chapters. In addition to my essay’s final version, Appendix I provides my intermediate work – the index cards I created – while outlining and drafting my essay.
Appendix I proceeds as follows. Section AI.2 presents my essay’s final version. Section AI.3 presents my initial topic cards. Section AI.4 presents new topic cards which emerged when I refined and organized the Section AI.3 cards. Section AI.5 provides the topic sentences I created for each paragraph; Section AI.6 provides the concluding sentences. Section AI.7 provides the essay’s first draft, which I created by adding supporting sentences between each topic sentence/concluding sentence pair. Section AI.8 describes how I worked to revise and polish the essay and compares the Section AI.7 first draft to the final version from Section AI.2.
