AI.3 Initial Topic Cards For The Essay

A paper begins as a collection of ideas you could write about.

I created the Section AI.2 essay by using the scaffold detailed in Chapter 2. I began by preparing – I needed material to give me ideas before starting the essay. I knew the main point I wanted to explore: the relationship between large language models (like ChatGPT) and cognitive science. However, I needed to learn a bit more about large language models; I also needed to determine what others might have written about my intended topic.

I prepared for a week, using Clarivate’s Web of Science database to collect relevant articles (which I then read). I also searched through articles published in the New York Times. I remembered the Times had published a series which introduced large language models to the general public, and I knew the Times frequently published articles with alarming headlines concerning new developments in AI.

As I read, I related the new material to my own ideas about cognitive science. Potential topics which I could write about were popping into my head. With my reading finished, I felt prepared to start writing.

However, for me, starting to write is actually starting to outline. My outlining process follows the method detailed in Chapter 2. My first step was generating broad topics (Section 2.6).

I performed the first step by taking blank index cards to use to jot down ideas as they came to mind. I began by writing down the basic thread which I planned to communicate (the first four entries in the left column of Table AI-1). For my essay about large language models and cognitive science I found my topics came to mind in related groups. For instance, I first generated some general topics related to how modern AI is being received, and then found myself generating ideas concerning particular properties of an example of modern AI, ChatGPT. As topics arose in related groups, I wrote a title to classify a group of topics on a separate index card.

Table AI-1 provides the topics which I jotted down on my index cards. Creating the topics required about a half hour on May 26, 2023. I laid out related topic cards in a column on a coffee table. I needed to see the topics I had already generated. The first card in each column was my title for related topics. I sat on my sofa generating topics for a while, until topic ideas dried up. I then spent some time looking at the cards I had created. On occasion looking at the cards reminded me of topics which I could add. In particular, I realized – after looking at my topics – I could talk about the evidence cognitive science used, evidence motivated by theory in cognitive science. So, the last topics I generated were the ‘Theory’ topics at Table AI-1’s end.

Table AI-1lists the topics I generated in the order I generated them. The table can be read by reading down its columns, first the left column then the right one.

After creating my topic cards, two observations became apparent to me. First, I felt I had generated topics by ‘thinking in paragraphs’, because I felt capable of using a paragraph to express the idea jotted down on many different topic cards. Second, I realized I needed to prune topics. My plan was to write a fairly short essay. However, Table AI-1 shows I produced nearly 60 topics. If I was indeed thinking in paragraphs, and needed a paragraph to express each topic, then my planned essay would be far too long. As I moved to my scaffold’s next stages, I realized my primary task was to prune topics which I didn’t have space to include.

Table AI-1 The initial set of topics generated to outline an essay which explored how large language models might inform cognitive science.
Overall Thread Cards (May 26 2023)
Title: From ChatGPT to Cognitive Science
Point 1: ChatGPT leads to gee whiz connectionism
Point 2: For ChatGPT to inform cognitive science, we need detailed interpretation of its internal structure
Introductory Topics (May 26 2023)
Modern AIs perform amazing stuff
Modern AIs are generating lots of excitement and press
Lots of fear about modern AIs
Fear of AI expressed as scary questions
Fear of AI due to its performance
AI performance leads to less scary (more boring) question: what does it say about cognitive science?
Radical answer to this question: Piantadosi – modern AI refutes all of Chomsky!
But radical cognitive science answers are just new gee whiz connectionism
Westbury question at candidacy exam – don’t need to look inside ChatGPT to inform cognitive science
My point: modern AI can inform cognitive science, but only if we look past performance and interpret internal structure
Overall Thread Cards (May 26 2023)
Title: From ChatGPT to Cognitive Science
Point 1: ChatGPT leads to gee whiz connectionism
ChatGPT Topics (May 26 2023)
What is NLP?
What is ChatGPT
What does ChatGPT do?
What new ideas give ChatGPT power?
What is ChatGPT trained on?
What is ChatGPT’s architecture?
How big is ChatGPT?
Classical Linguistics Topics (May 26 2023)
What is classical cognitive science?
Chomsky as classical prototypes
Syntax vs semantics; grammar; universal ; learning
Gee Whiz Connectionism Topics (May 26 2023)
Connectionism is old
1980s connectionist revolution due to network power (multilayer perceptron)
Connectionist revolution attacked classical rules
Problem: revolution assumed nets don’t have rules
Musical networks capture informal (i.e., no rules)
But if you look inside nets you find rules (logic network, mushroom network)
But if you look inside musical networks you find formal music theory
Moral: look inside networks to confirm the revolution
Kicker Topics (May 26 2023)
P. claims ChatGPT refutes Chomsky
Refutation: ChatGPT is better than any linguistics theory, but ChatGPT does not use rules
Issue: P. is doing gee whiz connectionism
Look inside ChatGPT: find hierarchies (Manning) – but P. misinterprets this result
What else will be found inside? ChatGPT is big!
ChatGPT may or may not refute Chomsky – to know, have to look inside
Theory Topics (May 26 2023)
Levels of analysis
Computation vs algorithm
Bonini’s paradox
Strong vs weak equivalence
Theory vs technology
Humans: first and second order effects
Networks: look inside
Theory Topics (May 26 2023)
Levels of analysis
Bonini’s paradox
Strong vs weak equivalence
Theory vs technology
Computation vs algorithm
Humans: first and second order effects
Networks: look inside
Computation vs algorithm