Conversational AI is the term that refers to AI applications that use spoken-word or chat-based conversations to automate communication and provide personalized experiences to people. Chatbots and voice-assistants have allowed us to interact with computers at a complex level, enabling the rise of innovative and time-saving technologies. With the help of superior natural language processing (NLP) techniques, neural networks, and deep learning, we have reached a stage wherein we can employ chatbots and voice assistants in a variety of use cases. Popular examples include:
- Alexa: A voice assistant created by Amazon, Alexa can answer browsable questions, play music, and stay contextual, while conversing. It also comes with in-depth integration into home automation.
- U-Report: A chatbot developed by UNICEF, it is used for helping citizens in developing countries speak about their community’s urgent needs.
- DoNotPay: A chatbot that helps users dispute parking tickets, by guiding you through the legal process.
In this case study, we’re detailing how Bridged worked with a conversational AI company, Galaxy (placeholder name, the company would like its name to remain confidential), to create a voice-assistant designed to act as a friend, a smart digital assistant, and a life-coach to its users. Bridged created bespoke datasets to train their neural networks model over multiple iterations.
Defining the character and scope of the AI
Human nature is complex, and every individual is unique. Conversations bring out personalities, emotions, memories, dreams, aspirations, and more. Understanding and empathizing with people is a multi-dimensional problem-statement that requires a very nuanced and complex solution.
Luna, the voice-assistant, is designed to increase the user’s self-awareness, self-belief, and promote possibilities to realize full potential. It will be a judgment-free companion encouraging its users to share thoughts and feelings, feel listened to, and understood. To realize such an outcome, we defined individual verticals that address different aspects of its training – Language & Vocabulary, Sensitivity & Tone, Knowledge & Intellect, Personality & Tact, and Memory.
- The datasets would be created in both UK and US English, with over 16,000 words included – words known by more than 98% of the people.
- The AI should be empathetic, motivating, inspiring, curious, playful, non-confrontational, and cannot induce guilt or shame.
- It should be able to understand the context and have a conversation in 5 major subjects, without necessarily offering its opinion:
- Career and goals
- Health – Physical and mental
- Personal relationships
- Money and financial stability
- Luna’s personality will be dynamically modified to suit the needs of the user but measured using the Big 5 OCEAN model.
- The AI would also be able to identify people and events from the user’s life and use its learning to talk about memories.
Training writers and generating couplets
We selected a 1500 strong team of writers, and we trained them in all aspects of the requirement. The brief was to create conversation ‘couplets’ in the form of user input and AI agent output, with the content in-line with the required guidelines.
To add diversity and aid creativity, the five major subjects were further classified into over 50 categories and more than 500 sub-categories. Each category was assigned a priority level, modified in each phase according to the training results of the ML model till then.
Examples of single-turn couplets:
|User:||I’m not sure about my new manager. I think I can do a better job myself.|
|Luna:||Try to stay optimistic. Perhaps you can share your ideas with them!|
|User:||I don’t think I should be in a relationship anymore.|
|Luna:||Sounds like something about it is troubling you. Would you mind sharing it?|
The couplets were also written in part as multi-turn (extended) conversation threads, to train the AI on memory, call-backs, and complex contextual narrative.
We split the entire project into 3 phases, with the client training their model at the end of each generation phase to refine the content requirements further.
Complex data require constant quality checks
As part of general training, writers were regularly briefed about the requirements along with performance enhancement suggestions. Also, individual training was provided to people at both ends of the performance spectrum to ensure better returns and higher quality.
Our quality control process involved each writer being subjected to three levels of evaluation:
- Threshold percentage for on-boarding
- Weekly evaluation of content
- Automated checks on our task execution platform during live data generation
The datasets also went through multi-level quality checks:
- By trained checkers evaluating at a couplet level
- Automated checks on the platform
- Final in-house analysis carried out as random sample evaluation
The first two phases of the project saw Bridged create 250,000 conversational datasets each. In the third phase, we generated more than 500,000 couplets. In total, Bridged delivered 1 million datasets to Galaxy as training data, which would allow Luna to have life-like conversations with humans.
One of Galaxy’s data scientists, speaking about the collaboration with Bridged, said,
“Bridged has been an indispensable partner for us, helping to build datasets has allowed us to develop ground-breaking new techniques where correctly formatted data sets simply didn’t exist. We hope to work with Bridged for a long time to come.”
How conversational datasets help
On receiving the conversational datasets, Galaxy had the fuel required to train their conversational AI model. Having understood the traits to be instilled while developing Luna, Bridged created datasets that helped the chatbot engage in real-life conversations. Multi-turn conversational sets help Luna indulge in long-form conversations and maintain context.
The potential of Galaxy’s chatbot holds scope beyond responding to simple commands, and thus, paving the way for AI to be more than a voice assistant and play a larger role in the users’ lives.