An international team of UC Davis students is designing the future of conversational AI—for Amazon and the World.

On November 27, 2018, a UC Davis team of 11 international students won the Amazon Alexa Prize—and $500,000—thanks to the performance of their artificial intelligence (AI) chatbot, affectionately named Gunrock. During the competition, the social bot was able to chit-chat, just like a human, for nearly 10 minutes on topics ranging from technology to sports, with Gunrock’s speech even conveying the nuances of everyday conversation.

But the real challenge came not in the form of dialog systems and natural language processing algorithms, but in programming Gunrock to read into what people are really saying when they engage in conversation.

For team members Mingyang Zhou and Dian Yu, computer science and engineering students from the College of Engineering, capturing what users actually want to talk about with the chatbot was a challenge they both determinedly took on.

Alexa, Let's Chat

By Andy Fell | UC Davis Magazine | Spring/Summer 2019

UC Davis students built a chat bot using data from millions of Amazon Alexa user conversations. Their goal? Ask it anything. Last fall, UC Davis’ Team Gunrock won first prize and $500,000 in the Amazon Alexa Challenge to create a social chat bot that can converse engagingly with humans on a variety of subjects.

“The diversity of how users come to the same intention is pretty hard to capture when you’re developing a chatbot,” says Mingyang, who was responsible for designing the chatbot’s sports and food modules. “Sometimes you know what the user is talking about, sometimes you don’t. So we have to think about the possible questions and answers of the user and then design specific rules to generate appropriate responses.”

For instance, if the chatbot asks what sports a user would like to talk about, the user response may be more indirect: “I really like watching Stephen Curry’s game” is by no means a direct answer, but the chatbot is nonetheless designed to understand that the user wants to talk about basketball.

Mingyang, whose doctoral work focuses on AI and multimodal learning systems, says that reading through the dialogs of the more than 1.7 million Amazon Echo speaker users who participated in conversations with the finalist chatbots was one of the best ways to program how Gunrock should think.

Dian Yu agrees, going as far as to wade through the stream of Twitter and Reddit user responses to questions they didn’t understand in order to come up with enough varied utterances for Gunrock to pull from.

“Accounting for human differences when building this chatbot model has been the most challenging part,” he says, “but it has also bee the most fun.”

For both students, it’s precisely these aspects of conversational AI that brought them to the team. “Reading the user dialogs with our chatbot helped us understand how different people carry conversations in diverse ways,” says Mingyang. “Making this more scalable with automatic machine learning methods is something we definitely want to do in the next challenge.”

Collaborative Hacking

A project as big as the Alexa Prize needs an even bigger brain overseeing the parts and the whole. Enter Zhou Yu, an assistant professor in the Department of Computer Science, a 2018 Forbes’ 30 Under 30 science winner and the director of her Language, Multimodal and Interaction Lab at UC Davis.

“The students definitely worked together to get this done,” she says. “It’s a very collaborative project because it’s a big system, and everybody’s responsible for different parts. So we do hackathons every week, where everybody works together for a half day, then they split up to do individual work during the rest of the week.”

Given that Professor Yu’s work focuses on designing algorithms for human-machine communication, her expertise was key in helping the team design the Gunrock chatbot—pioneering research that was also one of the reasons Ph.D. student Mingyang decided to attend UC Davis in the first place, once he realized he’d be learning from a world expert in his chosen subfield: natural language processing.

For Professor Yu, building better technology to automate things she finds tedious is one of the drivers of her work. Accordingly, the competition afforded her the opportunity to expand on this research while gaining experience in leading a big project with many minds on board.

“Understanding how to motivate students, how to help them collaborate together, how to coordinate the team leads and give them ideas and direction on what we might do to improve the system was an important lesson for me,” she says.

For example, when one team member decided to build specific data science tools to help his teammates understand the data, he soon encountered trouble selling the tools’ value to various members—until mentorship from Professor Yu helped him adjust his work so that it became uniquely accessible to everyone.

“I think the biggest lesson for this project has been how to communicate effectively—to understand the needs of other people—so you can make yourself more useful to the entire team,” she says.

Why AI?

A small department, computer science has nonetheless become one of the most competitive programs at UC Davis. During the Ph.D. application process, students can indicate which faculty they would like to work with, and when it comes to Professor Yu, around 800 students put her name down every year.

In addition to Yu’s work within the computer science department, she also has an affinity for interdisciplinary research and is currently collaborating with faculty in UC Davis’ communication department.

The topic: persuasion. 

“Persuasion is everywhere in everyday life,” she says. “For example, advertisement is one form of persuasion, but it’s not adapted to individual users. With the internet and social media showing personalized ads based on things like browsing history, there is now more we can do to adapt to a user’s particular aspects—for example decision-making style, personality—to tailor the persuasion message automatically and in an interactive way.”

This spirit of interdisciplinary collaboration is a hallmark of the computer science department. 

“I spent most of my life in big cities before coming to UC Davis, so the small town was a change for me, but I’m really starting to enjoy the lab here,” says Dian.  “As a Ph.D. student, I spend most of my time in the lab, so it’s provided a really great environment for me to do research and talk about different collaborations and projects with other faculty in the computer science department.”

For Mingyang, the first project he worked on as a Ph.D. student was co-advised by a computer vision professor, who was happy to provide insightful feedback and suggestions coming from his own purview.

“A lot of other schools have pretty strict requirements on the number of courses to take every quarter or year, but this university has provided a pretty flexible plan for Ph.D. students and natural language processing students to balance between research and courses,” he says.

And when it comes to research, Professor Yu’s students arrive from not all over—often through student exchanges with China’s top three universities.

“I have visitors from all over the place!” she says. “These students have a lot of potential, they just don't have the experience of getting into research through their own institutes in a more international, collaborative setting. So, giving them the opportunity to see whether they like research, whether they like doing research in the U.S., is very beneficial for their own personal growth.”

Originally from China, Mingyang, Dian and Professor Yu became fascinated by how computers work at an early age. And because the department they call home has a much higher ratio of international faculty and students than other campus departments, this has proven beneficial to their field.

“Building specific technologies—and dealing with processing language in English and Chinese—is very different,” says Yu. “I started learning program language in my elementary school. It’s a natural call for me to think about, ‘Oh, I’m doing all these chores and it’s tedious work, can the computer do it for me?’”

“This kind of automation is definitely something I really wanted to do when I was little, and I'm still doing it now,” she says.

While some of the 2018 Alexa Prize team members have graduated from the university, additional international students have since come on board, bringing with them innovative and interdisciplinary experience that may just help the team in this year’s challenge win the still-unclaimed Grand Challenge—a $1 million unrestricted university gift for maintaining 20 minutes of conversation.

The sky’s the AI limit for what Gunrock can do.