Amazon races to transplant Alexa’s “brain” with generative AI

Amazon is preparing to relaunch its Alexa voice-activated digital assistant as an artificial intelligence “agent” that can complete real-world tasks, and the technology team is racing to solve the challenges that have plagued the system's artificial intelligence reform.

The $2.4 trillion company has spent the past two years looking to redesign Alexa, its conversational system embedded in 500 million consumer devices worldwide, so that the software's “brains” are transplanted into generative artificial intelligence .

Rohit Prasad, head of Amazon's artificial general intelligence (AGI) team, told the Financial Times that the voice assistant still needs to overcome several technical hurdles before it can be launched.

This includes addressing issues with “illusions” or fabricated answers, their response speed or “latency” and reliability. “The hallucination has to be close to zero,” Prasad said. “This is still an open issue in the industry, but we are working hard to resolve it.”

Amazon leaders' vision is to transform Alexa, which is still used for limited, simple tasks like playing music and setting alarms, into an “agent” product that acts as a personalized concierge. This could include anything from suggesting restaurants to configuring bedroom lighting based on a person's sleep cycle.

Alexa's redesign has been underway since the launch of OpenAI's ChatGPT in late 2022 with support from Microsoft. While Microsoft, Google, Meta and others have been quick to embed generative AI into their computing platforms and enhance their software services, critics question whether Amazon can resolve its technical and organizational struggles in time to compete with rivals compete.

After years of artificial intelligence research and development, the team's work has hit a snag, according to multiple employees who have worked on Amazon's voice assistant team in recent years.

Several former employees said the long wait for launch was largely due to unexpected difficulties in switching and combining the simpler, predefined algorithms Alexa is based on with the more powerful but unpredictable large language models.

In response, Amazon said it is “working to provide more proactive and capable assistance for its voice assistant.” It added that it is unprecedented to implement technology of this scale into a suite of live services and devices used by customers around the world, rather than being as simple as overlaying an LLM onto an Alexa service.

Prasad, Alexa's former chief architect, said last month's launch of the company's in-house Amazon Nova model, led by his AGI team, was driven in part by a specific need for optimal speed, cost and reliability to help applications like artificial intelligence Alexa The program “gets to that last mile, which is really hard.”

In order to function as an agent, Alexa's “brain” must be able to call hundreds of third-party software and services, Prasad said.

“Sometimes we underestimate the number of services integrated into Alexa, and it's a huge number. These applications receive billions of requests every week, so when you're trying to do something quickly and reliably . . . you have to be able to Do it in a very cost-effective way,” he added.

The complexity comes from the fact that Alexa users expect fast responses and extremely high accuracy. These qualities contradict the probabilistic nature inherent in today’s generative AI, a type of statistical software that predicts words based on speech and language patterns.

Some former employees also noted the need to preserve the Assistant's original attributes, including its consistency and functionality, while infusing it with new generative capabilities (such as creativity and free conversation).

Because the LLM is more personal and talkative, the company also plans to hire experts to shape the AI's personality, voice and phrasing so that it remains familiar to Alexa users, according to a person familiar with the matter.

A former senior member of the Alexa team said that while LLMs are complex, they also come with risks, such as producing “sometimes completely invented” answers.

“At the scale of Amazon's operations, this could happen many times a day,” they said, damaging its brand and reputation.

In June of this year, Mihail Eric, a former machine learning scientist at Alexa and a founding member of its “Conversational Modeling Team,” publicly stated that Amazon has become “the clear market leader in conversational artificial intelligence with Alexa.” “failed”.

Eric said that despite the company's strong scientific talent and “tremendous” financial resources, the company was “riddled with technical and bureaucratic problems,” showing that “data annotation was poor” and “documentation was either non-existent or outdated.” .

The historic technology underpinning the voice assistant lacks flexibility and is difficult to change quickly because the code base is clunky and disorganized and the engineering team is “too fragmented,” according to two former employees who worked on Alexa-related artificial intelligence.

The original Alexa software, built on technology acquired from British startup Evi in ​​2012, was a question-and-answer machine that found the right response by searching within a defined range of facts, such as the day's weather or specific information. Songs from your music library.

The new Alexa uses a range of different artificial intelligence models to recognize and translate voice queries and generate responses, as well as identify policy violations such as picking up inappropriate responses and hallucinations. Building software that translates between legacy systems and new AI models has been a major obstacle to Alexa-LLM integration.

The models include Amazon's own in-house software, including the latest Nova model, as well as Claude, an AI model from startup Anthropic, in which Amazon has invested $8 billion over the past 18 months.

“The most challenging thing about AI agents is making sure they are safe, reliable and predictable,” Anthropic CEO Dario Amodei told the Financial Times last year.

Agent-like artificial intelligence software needs to reach the point of “…” . . People can actually trust the system,” he added. “Once we get to that point, we will release these systems. “

One current employee said more steps are still needed, such as covering child-safety filters and testing custom integrations with Alexa, such as smart lights and doorbells.

“Reliability is the issue – getting it to work close to 100 percent,” the employee added. “That's why you see us . . . or Apple or Google shipping slowly and incrementally.”

Many third parties developing “skills” or features for Alexa say they are unsure when the new AI-generating device will launch and how to create new features for it.

“We are waiting for details and understanding,” said Thomas Lindgren, co-founder of Swedish content developer Wanderword. “When we started working with them, they were more open . . . and then over time, they changed.”

Another partner said that after Amazon put initial “pressure” on developers to start preparing for the next generation of Alexa, things have calmed down.

Amazon's Alexa team has suffered massive layoffs in 2023, and a persistent challenge is how to make money. Jared Roesch, co-founder of generative artificial intelligence group OctoAI, said figuring out how to make assistants “cheap enough to work at scale” will be a major undertaking.

A former Alexa employee said options being discussed include creating a new Alexa subscription service or taking a commission from the sale of goods and services.

Prasad said Amazon's goal is to create a variety of artificial intelligence models that can serve as “building blocks” for a variety of applications beyond Alexa.

“We've always been based on customers and practical AI, we're not doing science for science's sake,” Prasad said. “We’re doing this . . . delivering customer value and impact is more important than ever in this era of generative AI because customers want to see a return on their investment.”