VoiceOver: App Testing Beyond The Visuals

Session 226 WWDC 2018

VoiceOver provides spoken descriptions and braille output of onscreen elements. Learn how to go beyond the visuals to identify VoiceOver and functional issues to help improve the experience for all users.

[ Music ]

[ Applause ]

Hello and welcome.

I'm James Vest, and I'm here to teach you today how learning VoiceOver can help make your app more usable for everyone.

And here's how I'm going to do that.

I'll tell you what VoiceOver is and why you should support VoiceOver and also how to use VoiceOver if you have never done it before and also how VoiceOver benefits your app.

So what is VoiceOver?

I get some common answers like well VoiceOver is a screen reader.

That's true, but VoiceOver doesn't just read the screen, or VoiceOver is for the visually impaired.

It's true, but it's not just for the visually impaired.

So if VoiceOver is not just for the visually impaired, and it's not just a screen reader, what is it?

VoiceOver is an alternative way of using you app, and it uses gestures, a keyboard, or a Braille display to give a variety of users equal access to your experience, and it's built in to every Apple device.

Now think about that last point.

It's kind of awesome.

I mean that means that every person who uses your app has access to VoiceOver.

But not every VoiceOver user has access to every app.

And that's why you're here.

That's why you are such an important role in this story because you must update your app for us to take advantage of VoiceOver.

Now when I say updating your app or supporting VoiceOver, I'm not talking about creating or maintaining a alternative or simplified layout or tailored to a subsection of users.

It's about allowing all users to access the experience that you created.

So why support VoiceOver?

Well, supporting VoiceOver makes Well, supporting VoiceOver makes your app more inclusive.

VoiceOver users are more likely to choose you app over other apps that don't support VoiceOver.

And this is a great community, so they are more likely to in turn recommend your app to other VoiceOver users.

But sadly, for many developers, those reasons are not enough, and I get it.

There's only so much time.

There's only so many resources, and you want to make the best decisions to create the best experience for the most of your users.

So that's why it's so important today that you realize that learning VoiceOver can help make your app more usable for everyone, not just VoiceOver users, for all users.

So I bet you're asking yourself, right?

Can a screen reader do that?

Yeah, absolutely.

I mean look at some of the latest apps coming out, those larger texts.

Those cleaner layouts.

They're not just benefiting low vision users or users with cognitive needs.

They benefit us all, and the same is true with VoiceOver.

So here's how VoiceOver could help you with your app.

Well, testing with VoiceOver can give you a new perspective.

Now it's easy to take for granted your app because it's your app.

You know exactly what it does, what it's supposed to do, how it's supposed to do it.

But we take a lot of those things for granted, and when I say testing with VoiceOver gives you a new perspective, it's getting a chance to experience your app in a new way.

That's because VoiceOver goes beyond the way your app location looks.

I mean an app might look good, but that doesn't make it usable.

So side-scrolling states, hover So side-scrolling states, hover states, they can all give the allusion of straightforward, clean, easy-to-use interface.

But testing with VoiceOver goes beyond that, and so it allows you to discover new challenges and insights, and finally, I believe VoiceOver gets really at the heart of what we do.

As iOS developers, we pride ourselves on creating easy-to-use, easy-to-understand apps.

Well those values are especially apparent when using VoiceOver.

So my question to you today is your app so easy to use you could use it with your eyes closed?

And that's not a question just for testers.

That's a question for designers, for developers.

Anyone responsible for creating the best experience possible for our users.

And the only way to answer that question is to start testing question is to start testing with VoiceOver, and that's what I'm here for today.

I'm going to give you some ways of being able to turn on VoiceOver and start using it.

Figure out where you're at, where you could improve, and then I'll give you some things else to watch out for that might not only improve your VoiceOver experience but possibly the experience that all users get.

And here's where we're going to start.

We need to know how to use VoiceOver.

Now in the next slide, I'm going to show you how to enable VoiceOver in settings.

What I don't cover in the next slide is how to navigate with VoiceOver, activate with VoiceOver, heck even turn VoiceOver off using VoiceOver.

So if you're following along out there on your iOS device, you may want to wait.

But I promise you, everyone in this room is going to have a chance to turn on VoiceOver before my presentation is done.

before my presentation is done.

So how do we turn on VoiceOver?

Well, it starts in settings, and we go to general.

In general, I can open up the accessibility menu.

It's a great list of all the features that we include on our iOS devices so that we can have equal access to our users.

And VoiceOver is right at the top of that list, but I discourage you from opening it there.

I discourage you from going in and enabling VoiceOver because, again, we might not know enough about VoiceOver to turn it back off.

So when I teach people how to use VoiceOver for the first time, I encourage you to go to the bottom of this list to the accessibility shortcut.

It's here inside of a list of options.

If I just check VoiceOver, that means I can easily turn VoiceOver on when I want to test it and also turn VoiceOver back off when I'm not ready to test it or my phone goes to sleep or I get a phone.

So just checking VoiceOver in this box I can turn it on by triple clicking the home button.

Click, click, click.

And if you have a brand new device that doesn't have a home button, well then you'd use the side button, click, click, click.

When VoiceOver is enabled, we see the VoiceOver cursor.

It's a visual reinforcement of what VoiceOver is trying to describe, and that's the first objective when testing VoiceOver, is to make sure that these descriptions are accurate and complete.

And the only way you're going to test every single element inside of your app is to navigate with VoiceOver, and here's how to do it.

Navigating can be as easy as just reaching out and touching the screen.

Dragging your finger across the screen, VoiceOver will just read whatever is under your finger.

But that's actually not really how a lot of people review their how a lot of people review their apps.

I start at the top.

I work through everything in read order to the very end.

VoiceOver users accomplish using flick navigation, and that's taking my finger and putting it on the screen and then flicking to the right, flicking to the right.

Now if I went slow enough, it would just read under my finger, so you have to really give it a quick flick until your finger flies off the screen.

And conversely, I could go back to the left, and I could go to the previous item or the next item.

And it's important that you also want to keep in mind the starting position of your VoiceOver cursor, because if will start in the middle of your page.

Well, that would assume as a user that there isn't anything before it, and so you might be leaving your users without all the story.

So, we have two ways to navigate, and we have no way to control how a user navigates.

So you want to test both these So you want to test both these ways to make sure they work as expected in your app.

Now while you're navigating your app, you may need to pause that output.

So, if you ever need to have so stop speaking, just take two fingers and tap the screen.

And if you make that peace sign again and tap the screen, VoiceOver will just pick up right where you left off.

Now, if you accidentally enabled VoiceOver earlier in settings when I said not to, don't worry.

You're halfway to turning it back off.

But there's one piece that's still missing, and that's activating.

You know, we take for granted that if I see a button on the screen I just tap it and it opens, but with VoiceOver when I tap an element, it describes it to me, and it gives me the choice to decide.

Do I want to open this element or do I want to move to another?

So, when I reach out and I hear a description of what I want to open, then I follow that up with a single finger, double tap, a single finger, double tap, tap, tap.

And the selected item will open.

So to illustrate how navigating and activating with VoiceOver, well, I'd like to welcome to my stage my colleague, Ryan Dour, to the stage.

[ Applause ]

Thanks James.

So you guys have all heard about how VoiceOver is used, but in order to really see this thing in action, we got to turn it on.

So I'm going to go ahead here, and on this phone, I have an iPhone X, I'm going to triple click the side button.

There we go.

Flash Thursday, June 7.

Excellent.

Heading not found.

[ Background Noise ]

Facetime.

There we go, okay.

Fantastic.

So we've started off here on the home screen, and the first thing I'd like to show you is that we've got a VoiceOver cursor on the Facetime icon.

It's a black cursor.

That's going to follow me everywhere I go while I'm using everywhere I go while I'm using VoiceOver, and there's really several different ways that we can move the VoiceOver cursor around.

The first is that I can reach out and touch just about anything on the screen, and the VoiceOver cursor is going to follow along, and it's also going to speak that item.

So if I just reach out and touch Reminders.

One task due today.

Excellent.

I can also just drag my finger around the screen to move that VoiceOver cursor, and it's going to follow along, so I'll try that.

No mail [inaudible] maps, weather.

We also have flick navigation.

If I flick left.

Maps.

I move back an item.

If I flick right.

Weather.

I move right an item.

And this reads left to right, top to bottom.

Notes.

See how it wrapped to the next line there?

So from here I can actually go ahead and if I want flick my way to an app and launch it.

Remind, news, stocks, TV, stocks, weather.

Or I can find it this way, and I can double tap.

Cupertino.

No weather info San Jose, [inaudible].

There we go, that's the right There we go, that's the right location.

Seventy-three degrees.

Except I don't want to hear the entire thing, so I paused it with a two-finger tap, again, using that peace sign, just performed that two-finger tap, and it paused the speech.

If I do it again.

Thursday, clear.

High 76.

This feature zooms.

Low 52.

And, again, I can pause it.

That way I can send it back to James.

Thanks James.

[ Applause ]

Thanks Ryan.

Now I just want to stop here and point out that using these steps alone you can actually start testing your app with VoiceOver.

You can turn it on.

You can navigate your app, and you can see how well it works but just with VoiceOver on.

So now the other side of testing VoiceOver is about getting familiar with the expected result, and that means getting outside your app and starting to experience areas that already have a good VoiceOver experience.

So here's how we do that.

Now if I needed to go home, well, I could press the home button.

But on a new device, like iPhone X, we need to go home by taking our finger from the very bottom edge of your device and starting to slide it up.

The first vibration I feel will go home, and if I keep dragging up, the second vibration I feel will open the app switcher.

I can also repeat this gesture from the very top edge of my display and open things like the Navigation Center and the Control Center.

And when you're not testing your app, you can three-finger flick.

It's basically the same as a one-finger flick to allow me to scroll or move between my home screens.

But in this case, I'm using three fingers.

So using three fingers back and forth, up and down, will allow me to get to those areas.

So whether or not you're exploring other apps or going exploring other apps or going into Control Center or Notification Center, these areas are full of complex UI that's waiting to teach you the expected results when we focus the VoiceOver cursor on them.

Now another gesture you may need is called the two-finger scrub, and that's where I take that same two-finger tap from before, but instead of letting go, I scrub back and forth very quickly, almost like in a fast Z pattern, and that will allow me to get out of a window without having to navigate back to cancel buttons or back buttons.

Or in the case of UI, where they don't even have close buttons, to quickly scrub back and forth and dismiss the window.

So while we're busy navigating and exploring and learning about this great expected behavior, we also kind of need to know what we're listening for.

So when you open a newspaper, you receive a lot more visual you receive a lot more visual information than just words on a page.

Here's that same newspaper.

If we fail to infer all of that additional information, like headings or images, so we need to explore and listen to headings and images, tabs and buttons, and make sure we're honoring that experience.

Now to demonstrate these initial gestures, here's Ryan Dour.

Cool. Thanks James.

So we're still here in our weather app, and I would actually like to go back to the home screen.

This is an iPhone X, so in order to do that, I need to use our new home gesture, and that is going to be again, touching my finger at the bottom in the home [inaudible], and then sliding my finger up until I feel that first vibration to go home.

VoiceOver will also tell me, if I haven't let go of the screen yet, that this is exactly what I'm going to do if I let go.

Let's go ahead and check that Let's go ahead and check that out.

Lift for home.

And I'm going to do that.

Weather.

Okay, so we're back on the home screen.

Many of us have many pages of apps, so in order to switch between those with VoiceOver, we use a three-finger swipe.

Actually it's a three-finger flick, but, yeah, it's funny, we actually interchange that a lot here at Apple.

We say three-finger swipe or three-finger flick, but they're interchangeable.

I'm going to go ahead and three-finger swipe left.

Page three of three.

There we go.

Utilities folder.

Four apps.

I'm going to go ahead and double tap on this utilities folder so that we can explore the inside.

Utilities, heading.

Let's explore.

I'm going to go ahead and flick right.

Voice memo, compass, measure, calculate, calculator.

Hear that bonk?

I've reached the end.

If I flick left Measure, compass, voice, utilities, heading.

Well, there's a lot of apps and a great heading here.

However, there's no apparent way However, there's no apparent way to close this folder.

Without voiceover on, you would just tap outside of the folder's area in order to be able to close it, but with VoiceOver, we use the two-finger scrub.

Now, you can use it just as James described it, like a Z gesture, but I'm a DJ, and I kind of think of it more like a scrub, like you're scratching a record.

So let's go ahead and take those two fingers and Utilities folder.

Boom that folder's closed.

Four apps.

Cool. Thanks, James.

[ Applause ]

Thanks, Ryan.

So when faced with a lot of information, a sided user may choose not to go through every single object in read order to the very end.

No, instead we choose to pay attention to maybe a select group of elements, like these headings on this newspaper.

VoiceOver's users can also navigate this way using the Rotor.

And the Rotor is a VoiceOver And the Rotor is a VoiceOver feature that dynamically presents options that the user can adjust or navigate by.

A VoiceOver user can set the Rotor by making two fingers on the screen and turning it around a center point, almost like you're turning an invisible knob on screen.

And as we adjust the rotor, it'll speak all the options that are supported within the view that we're focused in.

So in this case, we could turn that rotor and switch it to headings, and once it's set to headings, I'd be able to go to the next heading by taking a single finger and flicking down.

Or, conversely, I could flick up with one finger, and that would go to the previous heading.

And here's why the Rotor matters to you as a developer.

Because a Rotor only works if we as developers remember to include headings in our apps.

include headings in our apps.

So when your app doesn't support one of these rotor options, well then we're kind of taking a tool out of that user's toolbox to have a great experience in their app.

So I invite you to go back into settings, general, accessibility.

Inside of the VoiceOver menu is a lot of great settings that VoiceOver users customize, and one of them is the Rotor settings.

So review what we can support and make sure that we are honoring those in our apps.

So here's Ryan Dour to demonstrate how the Rotor works.

Cook, thanks James.

So again we're back here on the home screen.

We're on our second page of apps, and I'm going to open the App Store.

So I'm going to flick once, right.

App Store.

There it is.

Image going to double tap again to activate.

Apps, heading.

So I'm on the apps tab of the App Store.

There are an awful lot of fantastic apps here.

However, I don't want to have to explore each and every one of explore each and every one of them to get to the various categories.

So I can actually touch one of them.

WWDC18, celebrating the Apple design award winners.

And I'm pausing this speech again with that two-finger tap.

Now, to do the Rotor gesture, all I have to do is take two fingers and turn either clockwise or counterclockwise.

Characters, words, headings.

There's the one I want.

As soon as I let go, I am on that rotor.

Now, if I flick down New apps we love, best new updates, stream yourself selfie with light skin tone, heading.

Now, if I flick up Best new updates, new apps we love, heading.

I move back up that list, and of course if I want, I can go ahead and dive into a category, but before I do, I want to tell you guys about one more gesture, one that is, at least for me, extremely important.

So James started this whole thing off with asking you a very thing off with asking you a very important question, which is, is your app easy enough to use with your eyes closed?

And I'm going to guess that for a lot of you, you might say, geez, I have no idea.

I'm going to find out, but for me that is my experience.

I don't get to see your app.

I cants see your app.

So for me I really need that question to actually be the most serious one you ask yourself.

Now, keeping your eyes closed is a challenge.

It's very difficult to keep it up, and also you might want to be looking at your computer, taking notes, and doing other things.

So in order to do this, the most effective way I know possible, I encourage you to use the screen curtain.

The screen curtain blacks out your entire screen and puts you in my shoes.

You have a chance to use your app the way I use it entirely, and there's no cheating here.

That is the only interface you're going to get.

Let's turn it on.

So three-finger triple tap.

Again, three fingers, three times.

times.

Screen curtain on.

There we go.

At this point, we're on the same plane, and if I, again, flick down Best new updates, heading.

And I flick right, let's start exploring.

See all, button.

Of course now I know that's a button, and I can activate that button.

So I'm going to go ahead and do a double tap.

See apps, back button.

I'm in the apps areas, I'm going to start flicking through to the right, let's explore.

Best new update, [inaudible] get, but, world of dinosaurs, $1.99, in-app purchases, button.

That's a lot of information that you're getting that's usually very visual.

Procreate pocket, entertainment, button.

And of course, if I want to go back, that two-finger scrub Apps, heading.

And we're back.

Screen curtain off.

So that is screen curtain, and I can't encourage you enough to use it because when you make your app experience something that you can take VoiceOver through, both enjoy and make through, both enjoy and make sure that it's completely amazing, then so can I.

I can enjoy your app in full if you take this through as the final step before you call it done.

Thank you.

[ Applause ]

Thanks Ryan.

So now it's time to talk about how VoiceOver can benefit your app.

Because you're ready.

Today you can leave this area and go and test your app, and I would encourage you to test it like you normally would just with VoiceOver on.

But I want you to pay attention to these additional insights that could make your app better for all users.

So one, if something doesn't sound right, it may not read right either.

It's like when a writer reads something aloud that they wrote.

It's just another way to check that your message is coming through, and it doesn't just through, and it doesn't just look right, it actually sounds right as well.

Two, if navigating with VoiceOver takes effort, it probably takes a lot of effort visually as well.

So maybe it's time to add section headings or break up your content to allow users of your app to benefit from a cleaner, more straightforward visual and spoken experience.

Three, if your app sounds out of order, your content may be too.

VoiceOver navigates your app in read order.

So if you're navigating your app wondering where's the login?

Well, likely is a lot of your audience asking the same question.

There it is.

So these issues can't hide from VoiceOver and neither should you.

So let's turn it on.

Are you guys ready?

Come on, do you remember how?

All right, let's review then, all right.

So inside of settings, we're going to go to general.

If you have an iOS device, feel free to take it out.

Inside of general is accessibility.

Accessibility, we're going to ignore the very top of the screen, and we're going to go all the way to the bottom.

If we can enable today the accessibility shortcut, not only are we going to have a chance to turn on VoiceOver, potentially for the very first time, but it's also a quick way to go back and turn it on again, back to show your colleagues.

Turn it on again when you go back to your office or maybe when you're trying to make a case for trying to give more resource time to focus on VoiceOver.

So, if we've gone into settings, general, accessibility, and gone into the accessibility shortcut and checked VoiceOver, now we'll and checked VoiceOver, now we'll be able to turn on VoiceOver quickly with the home button.

Click, click, click.

Three clicks of the home button or on iPhone X, three clicks of the side button, click, click, click.

Ready to activate VoiceOver.

Let's do that now.

Yeah. And how does it feel, right?

For a lot of us, it's brand new, and so we could have mixed feelings or uncertain feelings.

This could be challenging, but I've never done anything worthwhile in my life that wasn't challenging.

All right.

So our next challenge will be turn VoiceOver back off.

[laughter] Let's see if we can do that now.

So with the home button, give it a triple click, click, click, click.

Or with the side button on an iPhone X, click, click, click.

So, yes, I'm going to call you out individually now, and, but no, in review.

VoiceOver is an alternative way of using your app.

And you can navigate and activate elements based on just how they're labeled.

And you can explore other apps to learn expected behavior.

And then rediscover your app by hearing it to help identify other issues.

So here's some steps I want you to do as soon as you leave today.

Just like Ryan suggested, I want you to close your eyes and test your app.

So really experience this.

Start identify issues that prevent users from getting the most out of your app.

And then open your eyes and confirm that experience, because confirm that experience, because if you've ever wondered what a VoiceOver experience should be, just remember, the goal is just to match the experience everyone else gets.

So compare the two and improve both based on what you learn.

And then make VoiceOver a part of your regular testing.

Talk to your team.

Make VoiceOver a part of your practice, not just so that you can continue to improve your experience with every release, but also to avoid losing the lessons that we gained here today.

And lastly, go out there and teach others what you know because that's why I'm here today, because teaching is the best way I know how to master any subject.

Now helping you become a master, I also want to share these resources.

Apple.com/accessibility is a Apple.com/accessibility is a great resource to get to know VoiceOver and other accessibility features and also have access to resources such as the guides, not only to reinforce what we learned today but build on it.

And the same is true for developer.apple.com/ accessibility/ios.

And for more information, feel free to rewatch this video.

Anytime we do something for the first time, it's hard to master, so watch it again.

Practice with me until it feels comfortable and you're ready to share.

And also, take advantage of these other opportunities.

For instance, tomorrow starting at 7:00 in the morning until 7:30, you can sign up for accessibility design by appointment lab, and that will allow you to talk to an engineer like me, who can also help you go through your app, identify issues, and then you can take those issues to the accessibility lab and technology lab nine tomorrow at 11 a.m. so lab nine tomorrow at 11 a.m. so you can meet with other engineers and say, how do I start fixing some of these issues?

How do I start improving that experience?

And then, also go to deliver an exceptional accessibility experience.

That's the book and presentation to the presentation that you watch today.

We showed you how to start using VoiceOvers to identify issues, and tomorrow we'll take the stage again, and we'll start showing you how to resolve some of those issues.

Thank you everyone.

[ Applause ]

Apple, Inc. AAPL
1 Infinite Loop Cupertino CA 95014 US