Yell AI Conversational Commerce Chatbot Case Study

1

Background

Yell is the UK’s No.1 provider of managed digital marketing services for all types of local businesses. For over 50 years they have supported the marketing needs of local businesses across every industry and every region of the UK. Trusted by over 100,000 businesses they tailor make product solutions using their relationships with the leading players in the industry – including Apple, Facebook, Google, Microsoft to ensure their solutions are cutting-edge, using automation and machine learning to deliver the best returns.

2

The Problem

Yell was looking for ways to quickly and easily create a marketplace between its consumers (who had a need) with their businesses (who could service that need). They also saw an opportunity to embrace conversational commerce using Apple Maps and Spotlight search on iPhone and iPad.

The Chatbot using ‘Apple Business Chat’ (a powerful new way for businesses to connect directly with customers using iPhones, iPads, Macs and Apple Watches) would need to be able to handle thousands of conversations, guiding customers to the information they needed, and then finally handover to the merchant/business owner, who could then respond via the Yell Business app through their working day.

3

My Role

As interim Lead, I managed 2 UX Designers, 1 senior and 1 junior that I mentored for the period. I communicated consistently with all key stakeholders while constantly discussing the project with the product lead and head of consumer to ensure everything was consistently moving forward at the pace required. Also, on regular intervals, I would meet with the key 3rd party technology platform, the software company hired to develop the conversational commerce and AI software, to gain an understanding of how it can be best served to meet the users needs.

Outside of this I would also be advising on UX issues for other Yell products, working with SEO, interviewing potential UX candidates for my role and put together design ideas of what a new Yell website could look like with guides for a pattern library.

4

Business and User Goals

As with every UX project it’s essential to define and establish the Business and User goals as these are the foundational elements. From a meeting with the stakeholders I was able to ascertain both.

Business goal: Enable business owners to talk directly with customers via the Yell Messaging network once contact has been made, with a full understanding of their requirements.

User Goal: Connect with businesses after talking with ‘Hartley’ the Chatbot, ensuring a swift experience with all expectations met to keep customer retention and deliver a phenomenal customer experience.

5

User Research and Personas

With access to millions of businesses that were signed up with Yell we were able to gain valuable insights through web analytics for constructing personas and empathy maps to better understand our audience.

We were able to personify our audience/users/clients with their name, age, business type, geographical location and income. Using this information we were able to create and run 12 seperate workshops around the UK with segmented demographics using stimulus material to understand where the pain points occured, understand better the goals, motivations, frustrations and create stronger persona types to get a clear picture of our audience.

Due to sensitivity of data I am unable to show our findings.

6

Chatbot Analysis

To gain an understanding of how the Apple Business Chat software worked I had a series of questions I wanted to find answers for:

  1. How the conversations performed?
  2. How informative and instructive they were?
  3. How they set up expectations with the user?
  4. How quick each question was answered?
  5. How and when the bot handed over to the merchant?
  6. What tone of voice did they use?
  7. How easy or hard the conversation was? And why was it easy or hard work?
  8. How each question lead to the next?
  9. What information the bot handed over and in what format?
  10. What did the bot look like?

I researched who was currently using Apple’s Business Chat and then set myself a task on each determined by the nature of their business and proceeded through a set of questions to evaluate how well it worked for the user.

The main businesses I selected to learn about conversational commerce were:

  1. Apple
  2. Buddy Bank
  3. Burberry
  4. Four Seasons
  5. Hilton
  6. T-Mobile

I also looked at other conversational commerce examples that didn’t use Apple Business Chat including:

  1. Facebook
  2. Dominoes
  3. DPD
  4. Santander
  5. Starbucks
  6. XBOX
  7. Volkswagen

Overall across the board it worked very well. One business stated it only worked in the United States, some were very robotic, others had a touch of personality, using emojis and ending the conversation with ‘Have a nice day’. A few ended with surveys asking the user questions on their experience.

Chatbot Examples for competitor analysis

7

Chatbot Research

With an understanding of how others were using conversational commerce, I also researched articles to gain extra knowledge on how to approach designing and structuring a Chatbot’s conversation. Jakob Nielson, the Nielson Norman Group www.nngroup.com, is one of my first ports of call for researched-based UX guidance that I more often than not find incredibly useful.

I came away with these findings which we added to our growing knowledge base on Chatbot conversations.

  1. The most important advantage was speed
  2. If you have a bot, you should have a really good one
  3. Disclose upfront to their customers that they are interacting with a bot
  4. When users realized they were talking to a bot, they tended to be more direct, use keyword-based language, and avoid politeness markers
  5. Users complained when a bot did not allow them to pick an option and instead required them to type.
  6. Make it easy for user, add date pickers, reduce typing for example
  7. Carousels, the UI element that bots use for showing sets of results, are simply not the best choice for displaying long lists.
  8. Users were generally annoyed when the bot repeated the same answers over and over again.
  9. Owning the failure and offering an escape hatch (phone number or a live agent) were generally perceived favourably.
  10. If the bot is too rudimentary, people will lose trust in the company and will feel ignored and unappreciated.
  11. Do chatbots have any advantages? In their current embodiment, they just have one: less information overload.
  12. Be upfront about using a bot and not a human.
  13. Clearly tell people what tasks the bot can do. Make sure you don’t create false expectations.
  14. Don’t be overly ambitious: create bots for simple tasks. Complexity is not well handled in the limited bot interface.
  15. Tolerate typos and ambiguity.
  16. Allow people to interact with the bot both through free-text input and selection of links.
  17. Allow sorting and filtering to let people narrow down through results.
  18. Save information from one task to the next.
  19. Program some flexibility into the bot: infer context and allow people to jump forward and backward in the linear flow.
  20. Be honest about not understanding. Offer an escape hatch in the form of a real human, a phone number, or a link to a different interaction channel.

Amongst other articles and websites I found www.chatbotguide.org also to be a source of excellent information.

8

Information Architecture and Wireframing

Given that we were new to the idea of designing a Chatbot’s conversation, we had to start somewhere. We had gathered plenty of information and knew essentially what we wanted to achieve. We had put together a hypothesis for how we imagined and perceived it should work, and how we wanted the bot to steer the conversation to meet our merchants and customers needs.

For us the main objectives were:

  1. Set the expectation up from the first message:
    a. Be honest. Tell the customer it’s a Chatbot and can help with information
    b. Explain to the customer that they can reach a human by typing ‘human’, we wanted to give the ability to connect to merchant should the customer require it immediately
    c. Give sets of information options so customer can easily select by typing in a letter to select and therefore speed up the process eliminating lengthy texts that can’t be understood by the Chatbot
  2. Provide a summary at the end before handover to ensure customer information is correct for the merchant to read
  3. From the merchants end it was required that they select a response time, which could be provided to the customer at point of handover to set an expectation of when they would be contacted
  4. We wanted the Chatbot to send prompts to the merchant as the time window for their response started to close, an alert so they didn’t miss a potential job and the app didn’t annoy the customer
  5. The chat box conversation would end with a quick survey so we could learn about where we could improve

Ideally a conversation would follow the format in the wireframe below. As you can see the goal was to make the process as simple and swift as possible eliminating any chance of the bot not understanding a question, removing all superfluous information that could act as impediments and slow the process down while guiding the customer to make choices with the data the Yell bot has to offer before seamlessly handing over to the merchant with the information he needs about the customer’s request.

We ran 4 more workshops to test our hypothesis using the clickable prototypes below.

Information Architecture and Wireframing examples

9

High Fidelity Designs

Based on the information we had collected about our customers and turned into Personas for our own understanding of their search habits, excellent advice and tutoring from the key 3rd party technology platform about how a Chatbot behaves and learns, testing results from the wireframes, the technical issues and the research with other businesses using Apple Business Chat we decided to design two different journeys.

One journey that would take the customer down a successful path and the other down paths which weren’t as straight forward. This would enable us to understand what we were trying to achieve and also show all the stakeholders our vision.

The ‘happy’ journey would show the customer entering the Yell app via Apple maps and then enter in a conversation with the Chatbot. From here we put together a series of conversations with all different ways to answer questions and convey those answers. The happy journey would also illustrate the handover to the merchant and how this is displayed on the customers side of the Yell Business app and also the merchants. It would also show when the bot has enough information to hand over and how different the conversation tone would be from Chatbot to real person.

The ‘unhappy journey’ would illustrate where we believed our biggest blockers and impediments would be and how we overcame them such as:

1. How to handle a message that was too hard for the bot to understand
2. What to ask when the customer refused to give a name as this was a requirement
3. Prompt to merchant and message to customer when their enquiry has had a response

Of course, some of this for the moment was conjecture, but as a UX team we wanted to collect people’s thoughts, ideas and opinions on what we were trying to achieve. The designs would also serve as a great start for lab user testing our hypothesis with suitable candidates.

10

Designing the Chatbot

Designing the structure and taxonomy of the Chatbots language was highly important, but equally, as there are mainly facets to the user experience of a project, there were many questions to be asked with different stakeholders to ascertain it’s identity.

  1. Would it be male, female, non gender or another type of gender?
  2. What should the tone of voice be like? Happy? Cheeky? Staid and informative? Corporate? Funny?
  3. What should it look like? A robot? A person? A cartoon? A photo?
  4. Should it have a name? Should the name be related to Yell?
  5. What colour should it be?
11

User Testing and Outcome

For our user testing sessions we decided to have 4 x 2 hour face to face discussion groups of two customer sessions with under 35’s in one and over 35’s in the other so we could see if age played a role in what they thought about the app and the offering. We also had 2 x SME owners for the merchant section of the app, across the ages.

Overall the app was received very well, highlighting in some areas we need to change/amend parts and in others that our hypothesis indeed was correct.

With enough information to put together an MVP, the app was first trialled in Manchester to a reduced number of people for testing purposes and shown to be highly successful in this period of social distancing and lockdown where businesses are looking to connect with their customers in new ways, proving it supports this brilliantly. It has been so popular that the whole Yell Messaging is now being rolled out nationwide.

12

Testimonial

"I have worked with Kai a number of times now and I know if I need someone who will work to understand the customer and business needs swiftly, who will conduct thorough research, who will confidently walk around the office, introduce himself and find who he needs to speak to, who will work with boundless energy and have an excellent understanding of UX procedures, Kai will provide this.

In this role I needed Kai to manage the UX team as interim lead, to be more hands off than focused on production, to mentor members of the team, to evangelise UX/UI, work with the Product Leads, other department heads and stakeholders, interview candidates and ensure we were always up to date and moving forward.

I would recommend Kai anytime and I look forward to working with him again soon."

James Loar - Head of Consumer at Yell