AI.3 Initial Topic Cards For The Essay
A paper begins as a collection of ideas you could write about.
I created the Section AI.2 essay by using the scaffold detailed in Chapter 2. I began by preparing – I needed material to give me ideas before starting the essay. I knew the main point I wanted to explore: the relationship between large language models (like ChatGPT) and cognitive science. However, I needed to learn a bit more about large language models; I also needed to determine what others might have written about my intended topic.
I prepared for a week, using Clarivate’s Web of Science database to collect relevant articles (which I then read). I also searched through articles published in the New York Times. I remembered the Times had published a series which introduced large language models to the general public, and I knew the Times frequently published articles with alarming headlines concerning new developments in AI.
As I read, I related the new material to my own ideas about cognitive science. Potential topics which I could write about were popping into my head. With my reading finished, I felt prepared to start writing.
However, for me, starting to write is actually starting to outline. My outlining process follows the method detailed in Chapter 2. My first step was generating broad topics (Section 2.6).
I performed the first step by taking blank index cards to use to jot down ideas as they came to mind. I began by writing down the basic thread which I planned to communicate (the first four entries in the left column of Table AI-1). For my essay about large language models and cognitive science I found my topics came to mind in related groups. For instance, I first generated some general topics related to how modern AI is being received, and then found myself generating ideas concerning particular properties of an example of modern AI, ChatGPT. As topics arose in related groups, I wrote a title to classify a group of topics on a separate index card.
Table AI-1 provides the topics which I jotted down on my index cards. Creating the topics required about a half hour on May 26, 2023. I laid out related topic cards in a column on a coffee table. I needed to see the topics I had already generated. The first card in each column was my title for related topics. I sat on my sofa generating topics for a while, until topic ideas dried up. I then spent some time looking at the cards I had created. On occasion looking at the cards reminded me of topics which I could add. In particular, I realized – after looking at my topics – I could talk about the evidence cognitive science used, evidence motivated by theory in cognitive science. So, the last topics I generated were the ‘Theory’ topics at Table AI-1’s end.
Table AI-1lists the topics I generated in the order I generated them. The table can be read by reading down its columns, first the left column then the right one.
After creating my topic cards, two observations became apparent to me. First, I felt I had generated topics by ‘thinking in paragraphs’, because I felt capable of using a paragraph to express the idea jotted down on many different topic cards. Second, I realized I needed to prune topics. My plan was to write a fairly short essay. However, Table AI-1 shows I produced nearly 60 topics. If I was indeed thinking in paragraphs, and needed a paragraph to express each topic, then my planned essay would be far too long. As I moved to my scaffold’s next stages, I realized my primary task was to prune topics which I didn’t have space to include.
| Table AI-1 The initial set of topics generated to outline an essay which explored how large language models might inform cognitive science. |
|---|
| Overall Thread Cards (May 26 2023) |
| Title: From ChatGPT to Cognitive Science |
| Point 1: ChatGPT leads to gee whiz connectionism |
| Point 2: For ChatGPT to inform cognitive science, we need detailed interpretation of its internal structure |
| Introductory Topics (May 26 2023) |
| Modern AIs perform amazing stuff |
| Modern AIs are generating lots of excitement and press |
| Lots of fear about modern AIs |
| Fear of AI expressed as scary questions |
| Fear of AI due to its performance |
| AI performance leads to less scary (more boring) question: what does it say about cognitive science? |
| Radical answer to this question: Piantadosi – modern AI refutes all of Chomsky! |
| But radical cognitive science answers are just new gee whiz connectionism |
| Westbury question at candidacy exam – don’t need to look inside ChatGPT to inform cognitive science |
| My point: modern AI can inform cognitive science, but only if we look past performance and interpret internal structure |
| Overall Thread Cards (May 26 2023) |
| Title: From ChatGPT to Cognitive Science |
| Point 1: ChatGPT leads to gee whiz connectionism |
| ChatGPT Topics (May 26 2023) |
| What is NLP? |
| What is ChatGPT |
| What does ChatGPT do? |
| What new ideas give ChatGPT power? |
| What is ChatGPT trained on? |
| What is ChatGPT’s architecture? |
| How big is ChatGPT? |
| Classical Linguistics Topics (May 26 2023) |
| What is classical cognitive science? |
| Chomsky as classical prototypes |
| Syntax vs semantics; grammar; universal ; learning |
| Gee Whiz Connectionism Topics (May 26 2023) |
| Connectionism is old |
| 1980s connectionist revolution due to network power (multilayer perceptron) |
| Connectionist revolution attacked classical rules |
| Problem: revolution assumed nets don’t have rules |
| Musical networks capture informal (i.e., no rules) |
| But if you look inside nets you find rules (logic network, mushroom network) |
| But if you look inside musical networks you find formal music theory |
| Moral: look inside networks to confirm the revolution |
| Kicker Topics (May 26 2023) |
| P. claims ChatGPT refutes Chomsky |
| Refutation: ChatGPT is better than any linguistics theory, but ChatGPT does not use rules |
| Issue: P. is doing gee whiz connectionism |
| Look inside ChatGPT: find hierarchies (Manning) – but P. misinterprets this result |
| What else will be found inside? ChatGPT is big! |
| ChatGPT may or may not refute Chomsky – to know, have to look inside |
| Theory Topics (May 26 2023) |
| Levels of analysis |
| Computation vs algorithm |
| Bonini’s paradox |
| Strong vs weak equivalence |
| Theory vs technology |
| Humans: first and second order effects |
| Networks: look inside |
| Theory Topics (May 26 2023) |
| Levels of analysis |
| Bonini’s paradox |
| Strong vs weak equivalence |
| Theory vs technology |
| Computation vs algorithm |
| Humans: first and second order effects |
| Networks: look inside |
| Computation vs algorithm |