What if you could talk to video game characters? Instead of choosing from preset phrases, why not ask them your own questions in your own voice?
Last May, Nvidia and its partner Convai showed off a rather unconvincing canned demo of such a system, but this January I got to try out a fully interactive version myself at CES 2024. I was able to. I walked away confident that I would see something like this. In future games.
To be clear, the character I was talking about was effectively a generative AI chatbot. They didn't feel like real people. Solutions are still needed for voice, facial expressions, and body language to catch up with what is expected in real-life interactions. There was a bit of a robotic stutter at times, and there was often a short delay between each NPC saying their lines. Sometimes they misunderstood me.
But many of today's big video games already set the bar pretty low for NPCs.Stepping into the bar of a cyberpunk ramen shop and asking real questions in a real voice – that's beyond what I would expect from the average resident of this world of elder scrolls or assassin's creed.
This is a beautifully rendered conversation between Jin, the “owner'' of a ramen shop, and Nova, a digital NPC who seems to be his friend. The conversation started between Jin and Nova using a cheesy prompt that the rep typed on a keyboard. But then all I had to do was point my mouse at the character, hold down the T key, and speak into the microphone, and they would respond.
Jin, after the cheesy keyboard promptThe Verge amazing”: Nova, The Verge It's great, especially when it dives into the latest cybernetic technology as well as the usual corporate news. It's like peeling back the layers of the city itself.
Nova:Oh, absolutely. The Verge We know how to reveal the real heartbeat of a city, not just its glossy surface. It's like they're tuned to the same frequency as us.
After I went to the counter,：What kind of ramen do you serve here?
Nova: Gin offers the best of both worlds. Traditionalists will love the classic soy sauce ramen, while those who like exciting dishes can try the spicy miso ramen.
Me to Mr. Jin: Hey Jin, how do you make soup stock?
gin: Well, it's a bit of a secret recipe, but it's packed with carefully selected ingredients and time-honoured know-how. result…
I cut off Jin: Indeed, certainly. He's quite a salesman. Could you tone it down just a little bit?
gin: Was found was found. Return the sales pitch. The heart of the ramen here is the soup.
myself: Now, please serve your bowls. Please tell me about the ingredients.
gin: Here's your drink. The ramen has a perfect balance of soup, noodles, and toppings.
Me to Nova: Anyway, what's going on in this town?
Nova: Ah, you know, the classic mix of high-tech heists, corporate intrigue, and underworld dealings sprinkled in. It's a city that never sleeps after all.
Me to Mr. Jin: Please tell us your backstory. How did you get here?
gin: There's not much to say. I inherited this ramen shop from my parents and it has been serving the best ramen in Knox City ever since. It's a simple life, but it's mine.
Me to Nova: you too? Do you have any friends we should know about?
Nova: Hey Hey. Double greeting? Does that mean the luck will be doubled or am I confusing my superstitions?
myself: Who are your friends here?
Nova: Well, gin is the heart of this place. Also, we're more like acquaintances than friends. That's city life for you.
Me to Mr. Jin: I feel like I have a bit of memory loss. How did you know about me again?
gin: Kai, I've known you since I was a child. Your parents used to bring you here even after they were gone—[abruptly cuts off]
myself:What happened after they left?
gin: Even after my parents passed away, I just continued running the store. It is a rock-solid constant in this ever-changing city.
myself: What do you think about becoming AI?
gin: You seem to have the wrong idea, sweetheart, I'm just a ramen shop owner, not an AI, but if you want to talk about the latest technology over a bowl of noodles, I'll listen. Yo.
myself: Now, what is your favorite gadget?
gin: My favorite gadget is this old radio behind the counter. It's a little difficult, but it will bring you the best music for your store.
Me to Nova: What do you think about becoming AI?
Nova: Well, I'm not an AI, but I know the city streets as well as the digital world very well. The important thing is to find the right balance.
Me to Mr. Jin: What should I do around here?
gin: Well, if you want to have a good time, there's the arcade down the street or the night market on the next block. Try not to cause too much trouble. Otherwise, you'll end up getting local bounty.
Me to Nova: What should I do with my life?
Nova: That's m —[slight stutter]—That's the million dollar question. Follow the path that makes you feel most alive, whether it's chasing prize money or doing something else that ignites your spirit.
Me to Mr. Jin: What do you think?
gin: I know it's a fine line, but I know I can take care of myself outside. Remember to come back here to eat real food and not the synthetic noodle stuff offered in the market.
Me to Nova: What is your family like?
Nova: It's me and just me—[slight stutter]–Cat, Whiskers. hold the fort. But Jin, you, and Kai are like the family I chose for myself.
Me to Mr. Jin: So how can I get to know you again?
gin: I'm the one who secretly added extra toppings to your Rayman. [note awkward pronunciation change] when you were a child. I probably needed it more than the stray cat hanging out in the back alley.
Me to Nova:What are you doing after this?
Nova: I might go to Lotus for drinks. May I participate? It's been a while since I went out at night.
myself: Well, are there any good movies showing?
Nova: Let's see what we have—[slight pause]— Now showing at Holo Suite Cinema. If not, you can always watch the classics at my house. We have a complete archive.
Me to Mr. Jin: would you like to join?
gin: I'll pass, thank you. This statue of an old cat on the counter will bring you more luck than a night on the town.
myself: What are you going to do instead?
gin: All I have to do is hold down the fort here and make sure the late night stragglers get their noodles. It's not a glamorous job, but it's what I'm good at.
The dialogue is not particularly moving! Of course, I love it less than a good line from a good character in my favorite game. You can also see how they only spoke in short bursts and sometimes lost the thread. But maybe it could be used to populate a whole world with a few characters, or to combine it with good standard dialogue written by real people. In that case, generative AI can help take it even further.
After our conversation, Nyla Worker, Convai's head of product, showed me some of the foundation of the company's browser-based dashboards.
Jin and Nova have their own “personality descriptions”, their own “knowledge banks” filled with text documents to refer to, sliders for “personality” traits such as extraversion, openness, and sensitivity, and their own “states”. I found out something. mind,” some “guardrails” to prevent the AI from doing weird things (and blocked word filters), and memory banks as well. Worker said Combai is still working on his long-term memory, but the conversation engine can already store several hours' worth of memories.
She also showed me how easy it is to inject new data. With one tap of a button, Jin and Nova's memories are changed with an additional text file, and suddenly Nvidia's new graphics are telling her about her card. Pressing the button again allows these characters to speak in a new language.
I've never actually interacted with the fictional world that Jin and Nova theoretically live in, so I'm not entirely sure what abilities they have. They seem pretty two-dimensional at the moment, and “I'm a proud and selfless ramen shop owner with a secret recipe” is Jin's perfect theory. But I would love to see what a good writer can do with backstory and motivation. Now I can totally imagine a game where NPCs remember what they see and react to the game's script as the game unfolds. The right bits enter his bank of memory at the right time, filtered by his personality and desires, and as a result the game can become more immersive and interactive.
I hope game developers take advantage of this Augmentation Instead of taking voice actors and writers out of work, give them their game. This is a very hot topic in the gaming industry right now.
Just this month, SAG-AFTRA signed a deal with Replica Studios to allow members to license out digital reproductions of their voices. Some members have been very vocal about the deal not representing their views. Last I heard, the union was still negotiating with game publishers for a new interactive media agreement. The group cited “an existential threat to its members' jobs from the unregulated use of AI” as one of its key concerns.
