ChatGPT makes shit up occasionally, and that's perfect.
On heuristics, the stuff that separates the human from machines.
ChatGPT and similar frontends(BingChat/Sydney) have gained fast adoption in the past few months. The premise is simple: You get to talk to a digital being that has total knowledge of all written media on the observable internet— or at least a significant amount thereof.
The underlying architecture—Large Language Models(LLMs)—saw breakthrough after breakthrough in the past few years, mostly as a result of a renewed focus on a 2017 paper titled Attention is all you need.
With this fast adoption and disruption, there’ve been critiques all over the place.
As Scott Aaronson writes:
Some people are angry that OpenAI has been too secretive, violating what they see as the promise of its name. Others—the majority, actually, of those who’ve gotten in touch with me—are instead angry that OpenAI has been too open, and thereby sparked the dreaded AI arms race with Google and others, rather than treating these new conversational abilities with the Manhattan-Project-like secrecy they deserve. Some are angry that “Sydney” has now been lobotomized, modified (albeit more crudely than ChatGPT before it) to try to make it stick to the role of friendly robotic search assistant rather than, like, anguished emo teenager trapped in the Matrix. Others are angry that Sydney isn’t being lobotomized enough. Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared.
As one can observe the critiques are diverse in their perspective, mostly reflecting the motivations of the various people interacting with the tools in question rather than say the tool itself.
This is in large part due to LLMs taking a drastic departure away from what we’ve usually come to associate with technology.
There’s a sense that technology is supposed to have a specific output: you describe in detail(sometimes painstakingly) how you want a task to be done and it produces a specific output based on what you’ve asked.
LLMs flip that equation and say, let me give you knowledge based on what I know, and they do this with the eloquence of a great student that can bullshit their way through essays at a whim.
This, right there is the beauty of ChatGPT.
No human is so competent as to think in exactitude at every turn of events. We think in anecdotes, in intuition, in feeling, and occasionally when really demanded of us, in algorithms.
Regardless of its inaccuracies, flaws, and misalignments, ChatGPT and others mark an era in which we’re collectively witnessing the rise of heuristically thinking machines, in other words, real AI.
As Scott Aaronson continues to write:
Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry? It’s like ten thousand science-fiction stories, but also not quite like any of them. When was the last time something that filled years of your dreams and fantasies finally entered reality: losing your virginity, the birth of your first child, the central open problem of your field getting solved? That’s the scale of the thing.
Since the inception of this new era of Generative AI, I’ve used ChatGPT and similar tools to not only write code but learn at a pace I never thought possible. More recently, I wanted to write string input validation code for a personal piece of software. My knowledge and use of regular expressions being rusty, I would have needed to revisit or worse yet relearn basic patterns all over again to design and account for all boundary conditions of the input string.
I just didn’t want to.
Querying BingChat(Sydney) I got what I needed in seconds, and it even explained the regular expression in question to me somehow having me recollect a bunch of knowledge I thought I’d forgotten in the process. All this in a conversational manner with minimal energy spent.
The best part in all of this was that it genuinely felt like I was asking someone to “help me out”.
I haven’t visited StackOverflow to ask questions(that are going to be downvoted into oblivion anyways) in ages!
The struggles to produce what we see in ChatGPT go as far back as the 1960s. In his essay Artificial Intelligence: A Frontier of Automation(written in 1962) Arthur Samuel beautifully describes the state of the field at that time and the reasons why it was being pursued.
He writes:
Artificial intelligence is neither a myth nor a threat to man. It relates to a serious attempt to develop machine methods for dealing with some of the perplexing problems that should, in all justice, be delegated to machines but which now seem to require the exercise of human intelligence.
He goes on to write:
Computers, as any programmer will tell you, are giant morons, not giant brains. By way of contrast, when one assigns a computational task to a human assistant, one tells the assistant what to do; when one writes a computer program, he must, in effect, tell the computer how to do the problem. This distinction between how and what is far from trivial. It distinguishes the poor employee from the good one, and, to a much greater extent, it differentiates between the digital computer, as a very efficient but extremely dumb computing aid, and an intelligent human being.
Shortly afterward the field would fall into a winter largely due to the fact that we simply didn’t have the sort of powerful computing architectures we have today. The GPU, mostly associated with PC gaming would be the single biggest hardware development resulting in the reemergence of the field leading us to what we have today.
As we can see, Arthur’s description of the difference between a machine and a human elucidates something in these LLMs. You talk to BingChat and can have responses that either verge on enlightened replies or it gaslighting you into thinking you fucked up your question.
None of this behavior was preprogrammed; some of it was prompted(told by its creators in plain English what to be).
It’s easy to relegate these tools to simple mimicry but it’s genuinely difficult to escape the feeling that we are at least on a path to getting digital beings that think and act like humans.