Fears surrounding the development of Artificial Intelligence are nothing new and have long been the basis of science fiction plots. But recently, a conversation between a researcher and a chatbot at Google has reinvigorated the discussion of what AI is and when and if something can be called “sentient.” That’s not a question I can answer (frankly, I’ve worked with honest-to-god biological human beings in the past that I’d have trouble labeling “sentient”), but I can tell you more about how Google’s LaMDA conversational AI works.

A “breakthrough conversation technology”

At Google I/O 2021, Google CEO Sundar Pichai revealed the company’s “latest breakthrough in natural language understanding,” called LaMDA.

The name literally stands for “Language Model for Dialogue Applications.” At its core, it’s basically a machine learning-powered chatbot designed to speak generally about any topic, sort of like IBM’s Watson loaded up with a wide-ranging set of facts, but enhanced with better language understanding and language-generation skills. Google said at the time that it had been trained only on text, meaning LaMDA can’t create or respond to things like images, audio, or video (though Google was actively planning to expand those capabilities).

At the time, LaMDA was just a novel curiosity with an impressive ability to parse questions and generate surprisingly spontaneous answers that escaped the realm of pure fact. One demonstration that Google showed included LaMDA speaking from the perspective of the demoted solar satellite Pluto, advising someone on how to dress if they visit and when the last time they had guests was, while sprinkling in a little salt about the lack of respect that dwarf planets get. As Pichai stressed in the presentation, this isn’t the sort of chatbot that responds to a question about the weather with a set of numbers; it’s got a little more style or perspective and is better able to model how we actually use language.

Another big benefit to LaMDA is that it’s a single model that doesn’t have to be re-trained for different conversations or subjects. It’s what’s called an “open domain” model. That means LaMDA-as-Pluto is the same LaMDA that can answer questions about history or the weather. But even this early version of LaMDA had drawbacks. Some of its answers bordered on nonsensical, with it losing its frame of reference in lines of questioning where it’s meant to be an object. As an example, a question about the gravity on Pluto had it responding about jumping high and practicing its flips (Pluto jumping on Pluto) or playing fetch with the Moon.

In 2021, Pichai said it was still “early research” and under active development, but that the company had been using it internally to “explore novel interactions” and improve the language processing skills of other voice-powered tools like Search, Google Assistant, and Workspace. Ongoing work was meant to ensure it meets Google’s high standards for fairness, accuracy, safety, and privacy, “consistent with [Google’s] AI principles.” That’s important to test for and control because LaMDA’s answers to questions could become a source of facts for some people, depending on where and how it ends up being used.

At Google I/O in 2022, Google revealed "LaMDA 2," a more advanced version of the conversational AI. This time, Google allowed "thousands of Googlers" to test it — partly to reduce instances of problematic or offensive answers. LaMDA 2, by all appearances, has much of the same features and functionalities as the original, operating as a sophisticated general purpose chatbot. However, Google's demonstration at the time was more focused, pointing out technical demos like keeping a conversation on the topic, generating lists tied to a subject, or imagining being in a specific place. LaMDA's abilities certainly aren't limited to these workflows, they're just one avenue that Google wants to take to test and refine how LaMDA works. Google reportedly plans to expand testing to larger groups over time through an AI Test Kitchen.

LaMDA isn't even Google's most sophisticated language processing model. PaLM (also revealed at I/O) is an even bigger and more sophisticated system that can handle problems LaMDA can't, like math and code generation, with a more advanced processing system that proves its work for greater accuracy.

Google’s AI team has been the subject of more drama than you’d expect over the years, but the company has been concerned about the possible ethics of developing AI for some time. In 2018 it established a series of ethical AI guidelines for how AI could be made and what it can be used to do, since distilled down into a set of defined principles and practices. The short version is that Google wants to ensure its work in AI is for the good of society, safe, and respects privacy while following best practices for things like data and model testing. Lastly, Google claims it won’t pursue AI applications that are likely to be used to harm others (as in a weapon), for surveillance, or to violate laws.

Building up to LaMDA

LaMDA is built on a sort of neural architecture called Transformer that Google helped devise back in 2017. This stuff gets complex fast, but Transformer basically delivers an advantage in both training performance (the time to build the model) and resulting accuracy (how good it is at doing its thing) compared to recurrent and convolutional neural models (the more typical machine learning systems of the late teens) when it comes to language processing.

An animation abstracting how a Transformer model translates text.

Rather than relying on a step-by-step analysis of text input, Transformer analyzes whole sentences simultaneously and can model relationships between them to better understand a contextually nuanced meaning. And since it does a big-picture analysis like this all at once, it needs fewer steps to do its work — the fewer steps in a machine learning model, the more likely you can train it to do its job well.

In 2020, Google showed off its first genuinely good AI-powered chatbot built on this technology, called Meena.

A real human (right) chatting with Meena (left).

The announcement cruised under the radar at most venues (no one rushed to the press claiming that Meena was “sentient”), but by some very specific metrics, it was nearly able to pass as a thinking, breathing human.

The “Sensibleness and Specificity Average” isn’t a widespread standard, but a metric Google’s engineers used to measure their Meena's success based on two simple questions: Is an answer specific, and does it make sense?

Meena was trained on 341 GB of text “filtered from public domain social media conversations,” meaning it learned the nuance of conversation from some of the most difficult but true-to-life examples possible. None of us commoners ever had a chance to play with it, but an analysis of transcripts provided in the academic paper (direct PDF warning) shows how human-like Meena could seem at times, expressing things as mundane as a curiosity in seeing specific films. But it still stumbles in ways anyone that’s played with the 2011-era darling Cleverbot will notice, steering the conversation away from subjects at odd times or demonstrating a lack of memory by ignoring its own prior answers.

The Two Minute Papers YouTube Channel did an excellent analysis of Meena’s academic paper.

Meena may not be as sophisticated as LaMDA, but it was a necessary step in demonstrating that an open-domain chatbot might be able to better understand the nuance of how we use language and return answers that make sense, feeling human — or at least “sensible and specific” like a human’s are.

“Parameters” are a ballpark measure of a machine learning model’s sophistication, encompassing different values the system can use for lots of its internal mechanisms, including various weights and triggers for its decision-making pathways. It’s a very imprecise analogy, but you can think of them sort of like the connections between neurons in your own brain. More complexity generally means more sophistication and power. Meena ultimately had 2.6 billion parameters. That’s an impressive number, but nothing compared to LaMDA, which can have up to 137 billion parameters. And remember PaLM, Google's even more advanced system? It has a staggering 540 billion parameters. Compared to more modern solutions, Meena is a machine-learnng tinker toy.

How LaMDA works

In January of 2022, months before showing it off at I/O, Google published a more detailed explanation for how LaMDA and its language model worked, describing the progress and development made so far on its AI Blog (some of my favorite weekend reading material).

LaMDA was specifically created to meet a series of metrics by human raters that prior chatbots had trouble with. Things like the internal consistency (i.e., memory) of its responses and how unexpected its answers are correspond to an SSI (Sensibility, Specificity, and Interestingness) score. Rather than just meeting expectations, can it make a joke or provide real insight? And how factual and informative are its answers?

Using these as measures of success — in essence, targeting the creation of a chatbot that can seem human — Google trained LaMDA’s models on a massive dataset, essentially just trying to get it to predict the next components of a sentence. At its core, LaMDA is an ad-lib generator meant to fill in the blanks with details humans will like. The model is then fine-tuned for other applications to extend its generative capabilities to create response candidates for longer responses, further trained using a dialog dataset of back-and-forth conversation between two authors — in short, it goes from fill in the blank to fill in the sentence, trained to mimic real conversations.

Lamda generates multiple candidates in response to input, picking the best one according to a range of scores.

Like many of ML-based systems, rather than generate a single response, LaMDA creates multiple candidates and picks what different internal ranking systems choose as the best one, so when it’s asked a question, it doesn’t “think” along a single path to one answer, it creates several of them, with another model choosing which scores highest on that SSI score we mentioned, actively trying to pick out the most interesting, insightful, and curious answers.

SSI itself is a human-judged metric, but in its prior Meena experiments, Google demonstrated how other human-judged metrics can be simulated with reasonable accuracy by yet another model, and according to the researcher’s paper on LaMDA (direct PDF warning), that’s what’s happening here, too. This SSI-judging model is created with human-generated responses to random sample of evaluation data sets. Basically, a bunch of people looked at question-answer pairs and determined whether they were good quality or not, and a model was trained off of that, augmented with similar training in other categories like safety and helpfulness. If you remember the Pluto conversation demonstration, LaMDA is also trained to consider role consistency — how closely its answers hew to what the target role would have said.

Training models on crowdsourced datasets without some fine-tuning is a bad idea.

Without this additional training, LaMDA had a tendency to offer… let's say “problematic” answers to certain lines of questioning, with room still left for improvement.

On top of that, Google’s working on ways to make LaMDA even more factually grounded by essentially including citations to external sources for facts in the conversation data sets it trains from. The company is also working to give it the ability to retrieve external information (i.e., fact check) during response generation. Google says it’s seeing promising results, but this is “very early work.”

Ultimately, LaMDA’s responses can beat average human responses on its interestingness metric in Google’s testing and come very close in sensibleness, specificity, and safety metrics (though it still falls short in other areas).

And LaMDA isn't even the end here. As touched on before, Google's shiny new PaLM system has capabilities LaMDA can't approach, like the ability to prove its work, write code, solve text-based math problems, and even explain jokes, with a parameter "brain" that's almost four times as big. On top of that, PaLM acquired the ability to translate and answer questions without being trained specifically for the task — the model is so big and sophisticated, the presence of related information in the training dataset was enough.

The big question everyone is asking: Is LaMDA sentient?

Everyone with a platform to talk about it has been weighing in on the question of LaMDA’s sentience — whether it can “feel” things like emotions and whether it has consciousness. Some of the answers it has provided to questions seem to imply to some people that it can. As with most pop science philosophical matters, opinions (and their quality) vary, and I’m not a philosopher. But now that the curtain has been raised on how LaMDA works, there are some mental gymnastics one has to make to assign a sense of agency to an ad-libbing decision tree in the absence of other evidence.

At a fundamental level, LaMDA isn’t just a software-based machine; it’s a machine that was explicitly made and trained to provide the most human-like answers possible through a selection process that’s meant to literally please humans into believing its responses came from one. That was the expressed goal here, so should we be surprised if it succeeds in doing that? We built it with this purpose in mind.

That people-pleasing nature is highly adaptable, too. In 2021 when LaMDA was in its relative infancy, Pichai stressed that “LaMDA is able to carry a conversation no matter what we talk about” (emphasis mine). If you have a model that is trained with all the data it can be fed, and that includes even science fiction, is it surprising that it would respond to questions about a sentience and awareness in the affirmative if the direction of the conversation and the questions are clearly leading in that direction? LaMDA was explicitly made to be role-consistent, and if you give it the role of a sentient machine, it will try to oblige.

As a necessary devil’s advocate: Being a bit of software in a computer may not intrinsically preclude sentience. What are our brains and bodies if not biochemical machines running their own learned and DNA-based software? That's one of the reasons why many futurologists have been concerned about the subject of AI for decades. After all, the only way we really have to express our sentience to one another is through how we interact — there’s no spark of thought or “soul” you can squeeze out and measure inside the human mind. However, how it interacts with the world isn’t the only way we have to judge LaMDA.

So far as I can tell, based on an understanding of its technologies, my own limited expertise in the subject of machine learning, and the team’s published research paper, LaMDA doesn’t actually have a working memory like you or I do. Its model is trained to generate responses that “make sense in context and do not contradict anything that was said earlier,” but apart from being re-trained, LaMDA can’t acquire new knowledge or store things in a way that would persist between conversations. Although LaMDA might offer claims to the contrary when asked in certain lines of leading questions, the model isn’t constantly running in self-reference. By all appearances to its structure, it can’t have the sort of internal monologue that you or I do.

Much like the debate in biology regarding extraterrestrial life and how hard it might be to recognize in other places if it follows alien chemistries and lineages, sentience may ultimately be difficult for us to recognize if it doesn’t work in the same way that we do.

Philosophies vary, but most working definitions of sentience assign requirements like intelligence, self-awareness, and intentionality — the ability to have thoughts about something. LaMDA can’t learn by itself in a way that persists, limiting its intelligence, and it can’t “run” in the absence of input. What limited intentionality it can offer is constrained to picking the best response among a handful of candidates it generates in parallel, judging them purely based on the weights that researchers assigned it. As for being self-aware, LaMDA may have itself told one researcher that it is, but it’s a people-pleaser by design. A different line of questioning would almost certainly have seen it offer a different response.

By all appearances, LaMDA lacks many of the required characteristics we consider necessary for sentience as we know it.

I can’t offer you a concrete answer as to whether LaMDA is sentient, just as I can’t factually verify that you, dear reader, are sentient, nor can I prove that I am. Ryne Hager could simply be an Android Police chatbot. But I can tell you that LaMDA doesn’t appear to be sentient or self-aware in the way that you or I perceive with one another — though I question whether we will ever find a way to quantify and recognize sentience in a concrete way.