Sony demos an AI-powered PlayStation character

Sony has used AI to imbue a PlayStation character with the ability to converse with a player, The Verge reports. A source shared a since-deleted video with the publication featuring an AI-powered version of Aloy from Horizon Forbidden West. Engadget viewed the video before it was pulled.

In the demo, Aloy can hold a conversation with voice prompts during gameplay, and respond to questions with a synthesized voice and facial movements. It starts with a delay from Aloy, after being asked how they are, followed by the character stating they are “managing alright, just dealing with a sore throat,” a weird aside. The technology appears to work both within a controlled demo as well as the entire Horizon Forbidden West game. During gameplay, Aloy answers queries about where they are and what the time is, though Aloy states it’s afternoon and hot, rather than a specific time. After the game, Aloy returns to interview mode and gives an overview of what just happened.

The creepy bit, at least to me, came when Aloy is prompted to say goodbye to the audience and thank everyone. The character does, but the robotic voice creates an eerie vibe.

Sony used a few different models to create this prototype, including GPT-4 and Llama 3 for powering decision making and conversation, along with OpenAI’s Whisper allows for speech-to-text. In the demo, Sony reportedly said it is using its own Emotional Voice Synthesis (EVS) system and Mockingbird technology for speech generation and audio to face animation, two technologies that the company hasn’t talked about much yet publicly.

NVIDIA and Microsoft have been working on similar technology. The former has used ACE, its suite of technology for enlivening game characters with AI, to create AI-powered conversations for NPCs. Engadget’s senior editor, Devindra Hardawar, was far from impressed, stating in January, that “seeing several NVIDIA ACE demos back-to-back made me genuinely sick to my stomach.”