UX and chatbots –
the fundamental things apply.
We recently completed an employee-facing chatbot project for an international company with offices in every country in the world (except one).
Working on this project, I found that while new deliverables were needed, the fundamentals of User-Centered Design still apply.
The UX of conversational interfaces
Chatbots are gaining momentum in the digital space, and with them comes a clear shift from the right to the left brain, from layout and design to content and logic. Luckily, as UX professionals, we usually have a rather split personality and are comfortable on both sides.
Also lucky for us - the fundamental things apply as time goes by. (Play it again here, for old time's sake.)
Not to get overly distracted (or teary, for that matter), I wanted to share some insights on the UX of conversational interfaces.
1. Do your homework first
UX practitioners will recognize this as user research.
Before jumping into interviewing a certain number of users, we looked at existing support information, such as helpdesk scripts and logs, as well as FAQs and other help information on the intranet.
Next step was to interview the first and second level support agents. As these people were directly concerned by the initiative, it was important to position the project as a useful extension to their work, and not as a threat aiming to replace them.
For the end user interviews we had a nice mix of geographies, genders, job functions and seniority. It gave us insight not only into their daily tasks and potential frustrations, but also into the corporate culture. It was particularly important for us to reach out to the small remote offices, because the chatbot would likely be even more needed in these remote offices than in the larger locations where onsite technical and HR support is more readily available.
Aside from discovering current user habits and key pain points, we needed to define the tone of voice for the chatbot. Not an easy task for a company as global as this one. Too formal and the chatbot might come across as boring. Too informal and you risk offending someone. Our interviews, therefore, explored the degree to which the company culture permeated the dispersed offices versus the need for sensitivity to local cultural norms.
And finally, we were curious to find out how people would interact with an internal chatbot, what their mindset was towards automated support. When asked "what would be the first question you would as the bot?" most people started either by saying hello, asking questions about the weather, or finding out about the chatbot's likes and dislikes. Proof that small talk is an important part of chatbot design and should be taken seriously.
To complete our initial research, we ran a company-wide survey to get more quantitative information about the tools and services used, the habits and experience of using external chatbots and voicebots (such as Alexa), and the general satisfaction level with internal support functions.
2. Define the bot personality
Humans have a tendency to personify just about everything: animals, objects, gadgets. At home, we have a vacuum cleaning robot who sends me little text messages, "help, I'm stuck!". We call him Jeeves – see the Jeeves & Wooster series based on the books by P.G. Wodehouse – and generally agree that while he's helpful, he's also a bit dumb at times and not a quick learner either. In fact, our robot Jeeves is quite unlike the literary Jeeves, who is always a couple of steps ahead of his master, Wooster!
The point is that it’s only natural for companies to give their chatbots names. Hence the need for UX professionals to help define a bot’s personality. Creating a bot persona is a logical extension to traditional user personas. It can be a bit more "creative" than the former since we are inventing a new “individual”. However, a bot persona should also be solidly anchored in user research.
During our project, we presented the findings of the user research phase during our first bot workshop and, together with key internal stakeholders, co-created the personality of the bot by identifying some typical small talk, its likes and dislikes, a background story, the overall goal of the bot, and key services the bot would offer.
3. Prototype structured dialogues
At this point, one could jump to the tool of choice to implement the bot. It makes a lot of sense though to look at all the functions and define to what extent the dialog should be structured or open. For the structured parts, modeling these dialogues in form of dialog flows (similar to tasks flows in traditional apps) is essential.
Initially, hand-drawn sketches on the wall work best to get everybody to agree on the general flow and logic. Subsequently these can be presented in a clean way using software specifically designed for bot dialog modeling (we tested out Botmock for this occasion) or your standard flow charting tool.
As important as the flow and logic are to successful chatbot design, the copy should get just as much (if not more) attention. Interestingly, copywriting has always been a key factor in creating good feedback, confirmation and error messages, but unfortunately for websites and apps these dialogs have often been often neglected. With bots, copy quickly takes center stage and ensures the "content first" doctrine is truly respected.
Along with structured dialogs that guide a user down a task-oriented path, we also had to consider simple Q&A dialogs related to company knowledge and processes. Replacing FAQs (which have always been problematic) may be one of the most useful things an internal bot can do. Instead of wading through FAQs to find questions that may correspond to my needs (cognitive overload and frustration), I can simply ask the bot and get my answer.
During the user interviews and again during the bot workshop, we challenged participants to think about what kind of questions they would ask their intranet if they could only talk to "it". This exercise revealed that simply putting the intranet’s existing FAQ answers into the bot framework wouldn’t cut it. From the UX side, we needed to make sure that the answers were clear and could be used for a multitude of similarly worded questions.
4. Test, learn, train – more than ever!
Bots start out quite dumb – and we knew that a beta version was definitely in order before launching the chatbot company-wide!
Testing is not an option, it's a necessary and integral part of bot development. No one can predict all the use cases and edge cases, since a lot of the control formerly granted by forms and structured data is gone. From my perspective, the more guidance we can give, the better. Our chatbot was intended for deployment within a mobile app channel in the first release, so canned responses in the form of buttons (rather than typing) also made sense.
Learning is a big part of the process – looking at analytics and the feedback received from the beta users gave us insights into what subjects mattered and where the bot failed to give good answers. How and when to integrate feedback was a flow we worked on, the goal being to have an integrated feedback mechanism without annoying the users!
Training is where the new world differs from the old one. We used to train users and aimed at making applications as intuitive as possible; now we train the bot, making it "more intelligent" to learn frequent patterns and recognize user intents. Bots actually learn from conversations using artificial intelligence, and "bot training and monitoring" is carried out jointly by bot developers and UX specialists. Analytics and user feedback have to be monitored constantly, while conversation flows, tone of voice, and content need to be adjusted accordingly.
A famous example of AI gone very wrong happened to Microsoft's twitter chatbot Tay in 2016, who turned into a racist, misogynistic bot within 24 hours by simply mimicking insults and nasty comments it received. Maybe teaching Tay the right values in the first place would have been a good idea?
5. Collaborate closely with internal stakeholders, developers, and users
I saved this one for the end, as it is truly valid for any project.
The ISO standard 9241-210 states: "The design team includes multidisciplinary skills and perspectives" and "Users are involved throughout design and development."
Internal chatbot projects tend to involve a lot of stakeholders – basically all the support functions – and therefore represent an excellent opportunity to break down internal communication barriers. While a classic intranet still allows each department to communicate separately (definitely not encouraged from a UX perspective, but of course widely practiced) a chatbot requires alignment and common sources of information.
On the design and development end, consistent and frequent collaboration is needed and required throughout the entire project, and typical waterfall behavior is simply not possible since both sides need to be involved in the design, testing, and training process.
Users are still – and should always be – at the center of any project. From initial interviews, analysis of support tickets and surveys to usability testing, bot analytics and feedback, the users' needs and satisfaction should be the main drivers of any project. As mentioned earlier, the fundamental things apply, as time goes by.