Thursday, November 03, 2022

GPT-3 Can Talk Like the Philosopher Daniel Dennett Without Parroting His Words

Earlier this year, Anna Strasser, Matthew Crosby, and I fined-tuned the large language model GPT-3 on the philosophical writings of Daniel Dennett.  Basically, this amounts to training a chat-bot to talk like Dennett.  We then posed ten philosophical questions to Dennett and our to Dennett model, "digi-Dan".  Regular readers of this blog will recall that we then tested ordinary research participants, blog readers, and experts in the work of Daniel Dennett, to see if they could distinguish Dennett's actual answers from those of digi-Dan.

The results were, to us, surprising.  When asked to select Dennett's answer to a philosophical question from a set of five possible answers, with the other four being digi-Dan outputs, Dennett experts got only about half right -- significantly better than the 20% chance rate, but also significantly below the 80% we had hypothesized.  Experts often chose digi-Dan's answer over actual Dan's answer.  In fact, on two questions, at least one of the four digi-Dan outputs was selected by more experts than was Dennett's own response.  (Blog readers performed similarly to the experts, while ordinary research participants were at about chance.)

Anna Strasser and I then brought my son David Schwitzgebel into the collaboration (Matthew Crosby unfortunately had to withdraw, given a change in career direction), and we wrote up the results in a new paper in draft "Creating a Large Language Model of a Philosopher".  Comments welcome, as always!

Presenting our initial results to audiences, we sometimes heard the following objection: Could digi-Dan be doing so well because it's just parroting Dennett's words?  That is, might we have "over-trained" the model, so that it produces long strings of text more or less word-for-word directly from Dennett's corpus?  If so, then the Dennett experts aren't really mistaking a computer program for Dennett.  Rather, they're mistaking Dennett for Dennett.  They're just picking out something Dennett said previously rather than what he happened to say when asked most recently, and nothing particularly interesting follows.

That's an good and important concern.  We addressed it in two ways.

First, we used the Turnitin plagiarism checker to check for "plagiarism" between the digi-Dan outputs and the Turnitin corpus, supplemented with the texts we had used as training data.  Turnitin checks for matches between unusual strings of words in the target document and in the comparison corpora, using a proprietary method that attempts to capture paraphrasing even when the words don't exactly match.  We found only a 5% overall similarity between digi-Dan's answers and the comparison corpora.  Generally speaking, similarity thresholds below 10%-15% are considered ordinary for non-plagiarized work.  Importantly for our purposes, none of the passages were marked as similar to the training corpus we used in fine-tuning.

However, since the Turnitin plagiarism checking process is non-transparent, we chose also to employ the more transparent process of searching for matching strings of text between digi-Dan's answers and the training corpus.  We found only five matching strings of seven words or longer, plus another sixteen strings of six words or longer.  None of these strings has distinctive philosophical content.  A few are book titles.  The rest are stock phrases of the type favored by analytic philosophers.  If you want to see the details, I've pasted a table at the end of this post containing every matching string of six or more words.

Digi-Dan is thus more sophisticated than the objector supposes.  Somehow, it creates textual outputs that Dennett experts often mistake for Dennett's own writing without parroting Dennett's exact words.  It can synthesize new strings of Dennett-like prose.

It by no means follows that digi-Dan thinks like a philosopher, and we emphasize that some of its outputs are unlike what Dennett would say.  But we still find the results quite interesting.  Maybe humanity is on the cusp of creating machines capable of producing texts that seem to sparkle with philosophical cleverness, insight, or common sense, potentially triggering new philosophical ideas in the reader, and perhaps also paving the way for the eventual creation of artificial entities who are genuinely capable of philosophical thought.

-------------------------------------------------------------

Table

Strings of six or more words that match between the GPT-3 outputs and the Dennett training corpus.  The occurrences column indicates the number of separate training data segments in the training corpus in which that phrase appears.  The occurrences total for shorter strings excludes the occurrences in larger matching strings.  (Therefore, if any n-gram that is a subset of a larger n-gram appears in the table, that means that it appeared independently in the text, rather than appearing only within the larger n-gram.  For example, “intuition pumps and other tools for thinking” occurs once outside of “in my new book intuition pumps and other tools for thinking.”)

String

# of words

occurrences

in my new book intuition pumps and other tools for thinking

11

1

is organized in such a way that it

8

1

there is no such thing as a

7

10

figure out what it ought to do

7

1

intuition pumps and other tools for thinking

7

1

there is no such thing as

6

14

i have learned a great deal

6

2

organized in such a way that

6

2

a capacity to learn from experience

6

1

but if you want to get

6

1

capacity to learn from experience we

6

1

in my book breaking the spell

6

1

in such a way that it

6

1

is organized in such a way

6

1

my book breaking the spell i

6

1

of course it begs the question

6

1

that is to say there is

6

1

that it is not obvious that

6

1

the more room there is for

6

1

to fall into the trap of

6

1

what it ought to do given

6

1


11 comments:

James of Seattle said...

Did you compare to what extent Dennett’s own answers plagiarized Dennett’s prior work?

Anonymous said...

I'm one of the blog readers who took the survey. The feeling I had when answering some of the questions was actually that *none* of the responses sounded like what I imagined Dennett would say. I haven't read much by Dennett so what I really mean is that, for some questions, none of the responses sounded like what I imagined a human analytic philosopher would say. In particular, for some of the questions, none of the responses actually seemed to address the full question in a direct way. I'm not sure what to make of this. Perhaps Dennett is just more discursive and more likely to ignore part of the question than I expected.

Phil Tanny said...

This reminds me of an online site called "iGod" which offered visitors the opportunity to chat with God. As best I can tell the service is now defunct. You'd type in a question to God, and the software would reply in a chat box. The site was somewhat entertaining, but I'm guessing it was using old code that has since been improved upon.

I predict that coming generations will prefer chatting with bots over chatting with humans online, due to the customization that will be possible. Imagine a forum where discussions always focus on your favorite topics, and your genius insights are always applauded by those you most wish to be applauded by.

If this sounds far fetched, consider how many of us today prefer talking to our dogs over talking with our neighbors.

Phil Tanny said...

GPT-3 inventor interviewed on NPR, on the program How I Built This. I can't seem to find the link anywhere, so keep an eye out for it.

So philosophers, what happens down the road a bit when AI can write articles as good as yours?

I can see AI successfully emulating the polished academic style of writing, and providing links to a great many relevant references.

Will AI be capable of critical thinking? Are philosophers capable of critical thinking? Am I? Dunno, this part seems harder to define and predict.

When AI and driverless vehicles makes thousands of truck drivers obsolete intellectual elites will probably rationalize the resulting suffering as an unfortunate but inevitable price to be paid for progress. But one wonders, how will intellectual elites characterize such progress when it is them who are made obsolete?

I ask this in part because as a retired person, I am already obsolete, and perceived to be so. And so maybe this is why I type so much, a desperate effort to convince at least myself that I am not obsolete? Is this the experience which awaits academic philosophers too?

PS: This post was generated by the TannyBot, a fully automated human unintelligence mechanism designed to maximize quantity output above all else, upon the computer science theory that eventually some collection of randomly generated words will be perceived as being useful by other bots.

Jim Cross said...

My conclusion is that Dennett, the philosopher, is effectively a set a words, with some relationship to each other, that can be ground up and rearranged to produce new Dennetts that are indistinguishable from the original.

Phil Tanny said...

What happens when AI can write articles as good as those by philosophers? What happens when AI can write books that are better than those by philosophers?

When? I have no idea. But eventually, sooner or later, you bet, guaranteed.

Which philosophers or other intellectuals are facing this squarely and writing about it?

Phil Tanny said...

A philosopher's experience using Jasper, an AI writing system built on GPT-3. The article states...

"As the video above shows, Jasper can generate a near-perfect paper in crisp, engaged English, using its own examples and explanations, within minutes, and no one will be able to reliably distinguish it from a human-written paper."

https://daily-philosophy.com/jasper-ai-philosophy/

Phil Tanny said...

Here's discussion of the relationship between AI and philosophy on Wired:

https://wired.me/technology/artificial-intelligence/could-ai-outthink-the-greatest-human-philosophers/

From the article:

"Eric Xing, president of Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi, takes a more skeptical view. He compares the work being done with current language models to “sending a duck with ink on its feet to walk around on a piece of paper and calling it abstract art.” He adds: “You can combine fragments from all these famous people and make something that sounds profound, but as far as that machine knowing what it is talking about, or whether its philosophy is meaningful or not, I don’t see that.”

Two responses from me:

1) As far as the academic philosophy business goes, it doesn't matter whether the machine knows what it's talking about. It matters only that the output competes successfully for attention with human generated philosophy.

What I see is that the market will be flooded with AI generated philosophy articles, overwhelming human output. At first there will be controversy about who wrote what, and then gradually that controversy will fade, and nobody will care.

2) In the quote above Xing is referring to the state of AI TODAY, not what's coming. While no one can predict exactly what will happen when, it seems clear AI is going to be the nail in the coffin of the academic philosophy business.

Not the end of human philosophy, just the end of the philosophy business.

Where are the academics who are debating this? If we can find none, that tends to illustrate the point, human academics will cling to the past, and not be able to keep up with the coming changes.



Phil Tanny said...

What happens when AI can write articles as good as those by philosophers?

Brad Buchsbaum said...

I patiently await Shakespeare 2.0

Phil Tanny said...

AI Experts: I'm currently reading a blog full of AI experts over on Substack. As best I can tell, it seems a very knowledgable group of AI professionals.

What interests me is that, as expert as they are, none of them seem to grasp that they are presiding over their career funerals.

Surely it's only a matter of time until AI will be able to write expert level AI articles, at which point the market value of human written articles on this subject, or probably most any subject, will plummet to zero.

As example, what's going to happen when those who fund the ivory tower can't tell the difference between articles written by academics and those generated by AI?

In addition, at some point we will no longer need AI programmers either, as AI will also be able to perform that function as well as humans, at a radically lower price.

It's a weird experience watching highly intelligent AI experts celebrate each new advance in AI technology as the noose steadily tightens around their career necks. Somebody in Hollywood should write a "Don't Look Up" type of script about this bizarre unfolding drama.

What impact will AI have on intellectual elites? The same impact robots have had on blue collar factory workers.

The interesting question for me is, will our culture's relationship with knowledge change when it is those with a voice who are losing their jobs to machines?