Designing Voice User Interfaces: Principles of Conversational Experiences

by Cathy Pearl

meant to test the flow and ease of use.

your users have problems because they’re not being recognized, that is clearly an issue that needs addressing.

Ann Thyme-Gobbel, currently VUI design lead at D+M Holdings,

“Yes, Chef,” Conversant Labs

if you have less control, you’ll need more subjects.

rejection error, for example, is different than a mismatch

car company NEXTEV,

self-driving car company Zoox,

Just because you can say “book me a taxi and then make a reservation and then send my mom flowers” doesn’t mean any users actually will say it.

Dialog Traversal Testing (DTT).

exercising a test script

How can we get these from the call centers?

Recognition Testing

Before starting your pilot, you need to define your goals.

How to Build a Speech Recognition Application, Bruce Balentine and David Morgan

Here are some examples of success criteria:

task completion rate

people quit after the initial part of the task:

the “pay bill” task did not start until the user went past the balance prompt and began the bill pay workflow.

task:

Some of these are squishier

Dropout Rate

call duration.

the user feels in control, duration is not necessarily a bad thing.

your VUI could have shortened prompts for users who have interacted more than a minimum number of times.

high number of NSP timeouts can be an indication of a confusing prompt,

out-of-domain

of-domain.

academic.

Academic is the wrong word here

these.

You could easily handle this -- treat it as "of my house"

N-best list

automated speech recognition (ASR), the natural-language understanding (NLU),

Previous or Back button, tracking usage

Back

We should track the difference between cold and warm launch numbers

“repeat” function,

repeated queries,

Latency

whole call recording

Conversation Dashboard (originally created by Volio’s CTO, Bernt Habermeier).

Without logging, when you launch your app into the wild, you will be blind.

includes the following:

analysis.

I think it's also important to disclose the analysis to the users.

money,

Manually verifying the STT. Our QA is doing this.

Users are more likely to do a survey when it’s within the app, rather than having to click a link at a later time.

those who choose to take a survey are the ones who are the most or the least satisfied.

irrelevant.

This isn't 100% true. They're using software and need to know how to use it

you’re using confidence scores to determine your confirmation strategy, it’s important to verify that it’s set correctly at each state.

Nuance, for example, has a “My Vocabulary”

At this time, not many tools exist to easily allow designers to improve their VUIs based on user data.

without having to access a database, or roll out a new version of the app.

Voxcom, which develops speech recognition apps for call centers,

Transcription, in which humans