Quick Interaction Techniques for watchOS

Session 211 WWDC 2016

Quick interactions are essential for creating a compelling Apple Watch app. Gain insights into new gesture, Digital Crown, navigation, and notification APIs, and see how to use these API alongside SceneKit and SpriteKit to create rich, glanceable interfaces. Explore how this expanded toolbox can help make your watch apps quick and responsive.

[ Music ]

[ Applause ]

So, good morning, everyone.

So, my name is Tom Witkin, and I'm an Engineer here at Apple on watchOS.

And this morning, we're going to be talking about building quick interactions into your Apple Watch apps.

And first off, we'd like to start with the question - what do we even mean by a quick interaction?

So I'm sure that many of you have developed apps for iOS or for Mac OS.

And on those platforms, we're typically measuring the amount of time a user is spending in your applications in terms of minutes or even hours.

But in watchOS, it's just not the case.

A user wants to be able to raise their wrist and quickly get to specific information or perform a certain action, and then drop their wrist and go back to whatever they were doing.

And so the interaction model is entirely different.

So here are some examples of what a quick interaction on Apple Watch are.

So, for example, starting a timer, or checking the weather, or marking an item off your grocery list.

And so we think that these are some of the prototypical types of interactions on Apple Watch, and there are some key philosophies behind these that you've probably heard this week.

So, first off, apps on Apple Watch they need to be glanceable.

Which means that, with just a glance, your users should be able to get the information that they need.

Apps need to be actionable, which means that your users should be able to make quick and well-informed decisions.

And lastly, they should be responsive.

You should never be making your users wait.

And if you're interested in learning more about these, I highly recommend that you check out the Designing Great Apple Watch Experiences talk this afternoon, or watch the video of the What's New in watchOS 3 session.

And so now we know a bit about what we mean by a quick interaction.

But how long is a quick interaction on Apple Watch?

And so we think that the ideal length of interaction is only 2 seconds long.

So, 2 seconds.

That's not very long at all.

And so you have to be very deliberate in how you design and build your applications.

But we think that it's more than possible, especially with all the new features and API that we've added in watchOS 3.

And so that's how we're going to start off is by talking about these new features and API.

And then afterward, we're going to look at some practical techniques that you can use in your apps to build these types of quick interactions.

And with that, I'd like to hand the stage over to my colleague to talk through the new features and API.

[ Applause ]

Thanks, Tom.

Hi, everybody.

My name is Miguel.

I'll be giving you a quick overview of the new APIs that we're introducing in WatchKit to allow you to implement quick interactions.

I'll be talking about five sets of API that allow you to make your apps glanceable, actionable, and responsive.

The first has to do with giving you more access to new kinds of input.

You can now do gestures with gesture recognizers in your apps.

We are giving you access to the rotation events in the Digital Crown.

And we're also having three other areas.

One is for improved table navigation, the support for the new user notification framework, as well as integration with SpriteKit and SceneKit.

So let's jump right in.

I'll be giving you an overview of the API, and then Tom will come back on stage and show you examples of how he combines multiple of these APIs.

So if you have used gesture recognizers in iOS, you pretty much already know all there is to know for gesture recognition in watchOS.

If you haven't, gesture recognizers are objects that help you parse the lower-level touch events into recognizable gestures.

We're writing support for four recognizers.

Two discretes the SwipeGesture, the TapGesture.

And two continuous gesture recognizers the PanGesture, and the LongPress.

So the way you use this, like most of the development you do in WatchKit, is an interface builder in your storyboard.

You are dragging the objects into your interface.

So if you look at the library, you can now see four instances of the recognizers.

These are dragged into the elements that you want to have gestures attached to them.

And you do the configuration interface builder in the inspector.

The last thing you do is drag the action of the recognizer into a function within your code.

This is a function that's invoked when the gesture is recognized by the system.

As you can see, one of the parameters is the gesture recognizer itself.

From it, you can now get the location where the gesture was recognized, the bounds of the object, as well as the state that the gesture is in.

So, a few words about the state of gestures.

Gestures begin in the possible state before they're recognized, and they can transition either to a recognized or a failed state.

When you're dealing with discrete gestures, you really don't think too much in terms of states.

All you need to know is that when the gesture is recognized, your action method is called.

This is slightly different from continuous gestures.

They also begin in the possible state.

But as their name implies, they progress over multiple updates into your code.

So just assume that your action method will be called multiple times as your gesture is changing, giving you a chance to update your user interface.

Some tips for using gestures in watchOS.

We recommend that you attach them to the group elements rather than the controls.

You can do that, but you might conflict with some of the gestures that are already present in the controls.

And remember that you have a smaller screen real estate in the Watch app, so the groups tend to be the bigger interface elements in your application.

You can still set dependency relationships between gestures.

But for watchOS, this is only possible in the inspector and interface builder.

And be mindful that some of the system gestures will take precedence over the gestures that you will define.

For example, if you have an element that already has scrolling enabled and you're trying to attach a SwipeDown gesture, that will not be recognized.

The scrolling will take precedence.

The same thing for the ForceTouch and the LongPress.

Now let's move onto our second input element the Digital Crown.

This is a unique element for the Apple Watch that allows your users quick and precise control of their input.

You've had access to the Crown since watchOS 2 through the WKInterfacePicker object.

To use that, you are giving us a list of picker items and setting a style.

And WatchKit takes care of drawing the element for you.

Here is the list style.

You have the stacks style and the image sequence style.

But for this element, WatchKit is taking care of a lot of stuff for you.

We're drawing the element, we're taking care of interpreting the Crown events, and we're simply calling your action when a selection is made.

So, in watchOS 3, we are now giving you direct access to the rotation events so that you can write your own UI elements and drive them with the rotation values.

This is achieved through two new objects through new classes we're introducing WKCrownSequencer, which gives you access to the rotation events I'm sorry, the rotations per second.

And the new delegate protocol, WKCrownDelegate, with callbacks on rotational deltas.

So you have two kinds of values you're receiving rotations per second and rotational deltas.

What do you do with each one of them, or how do you access them?

If you're interested in rotations per second, you want to access the crownSequencer property which is now present in all of the interface controller classes.

And you query its rotations per second property.

This is probably something you're going to be using when you're interested in the speed of the Crown rotation in maybe a physics-based animation, a game, something like that.

Rotational deltas are communicated to you through the delegate protocol.

This is how you're able to count how many times the Crown has rotated.

So you will get a callback in the crownDidRotate method.

You will have the rotationalDelta as one of its arguments.

And what you want to do in this method is you want to maintain an accumulator variable where you add the deltas that we're passing on to you.

The math you want to use is a full rotation is equal to accumulated deltas of 1.

Another example is if you're only interested in detecting half rotations, you accumulate your deltas up to .5.

So you're adding the deltas as we give them to you.

And when they cross this threshold, you know that a rotation has occurred.

It's really up to you to figure out how these rotation counts correspond to the sensitivity of the content that you want to display.

When we're communicating values to you, keep in mind the sign either positive or negative.

When your users are scrolling with the intent of moving their content up, we will be giving you positive values.

And when they're scrolling with the intent to moving the content down, we're giving you negative values.

And these values are orientation agnostic, so you don't have to worry about what counter-clockwise or clockwise means.

If your users change their Crown orientation, we will continue to give you a positive value or a negative value for up and down, so you don't have to worry about that.

Finally, there's a concept of focus on the Crown.

The focus can be taken away by other interface elements for example, a picker, a slider, any scrolling element.

So it's up to you to determine when your interface element needs to be the focus of the Crown.

This is a great place to be using the new gestures.

And once you determine that your element needs to be in focus again, please call the function "focus".

And don't forget to call this initially when your interface controller first comes up.

Now let's put aside the input elements and let's talk a lot about the vertical paging that we're introducing.

The typical way that your users navigate in your tables is by selecting a row and seeing the detail for that particular row in a separate controller.

They need to navigate back into the master table to select another row before they're able to see that detail.

In watchOS 3, we have the vertical paging concept.

So when you enable this in your tables, once your users go into their first detail, they're able to swipe with their fingers or with the Crown up and down and see the subsequent row detail.

So this eliminates their need to go back into the master table, making your app seem a lot more responsive.

So how do you opt into this mode?

It's actually pretty straightforward.

All you need to do is enable the Item Pagination checkbox in your table inspector.

But you also need to make sure that your table is using the segues API.

That's where you're giving us what context you want to instantiate a controller with and we instantiate it with the given context.

You're probably already doing that.

So the API itself hasn't changed much.

There's a couple of changes.

There's a single method that allows you to jump into the segue mode from code.

This is useful when you're launching your application from tapping on a complication or tapping from a notification and you want to directly go into a specific row in the segue mode.

The second thing you need to be aware of is WatchKit is trying to be smart and help you out with the speed of your application.

So we're doing preheating of controllers, anticipating what the user could do in terms of scrolling up and down.

So some of the lifecycle methods for your controllers will be called before your elements are on screen.

I won't be going too much into detail about this.

But we have a whole new session, Architecting for Performance, where they give a fairly detailed example on all the badging of the lifecycle methods.

Now let's talk about notifications.

These are the ultimate quick interactions that we've introduced since the beginning of the Apple Watch product.

So from a user's point of view, your user's point of view, the way they interact with notifications is simply by feeling the haptic in their arm.

They raise their wrist and they automatically see this short-look interface for your notification.

If they keep their hand raised, we transition into the long-look of the notification.

The way they respond to your notifications is I'm going to walk through the long-look example, which is the one that has the most detail here.

So if you've done a good job in terms of communicating the content of your notification through the text, they'll do nothing other than dismiss the notification.

That is the shortest quick interaction.

They could also tap on the notification to launch your app.

Or, remember that notifications can have custom actions associated with them.

In this case, I'm showing three the Accept, Maybe, and Decline button.

You can have up to a total of four.

When you're associating actions with your notifications, they can either be foreground actions meaning that if the user taps on the button, your application will become active or background actions.

Background actions in watchOS 2 were always forwarded to your phone.

The big change in watchOS 3 for a notification is that there's a whole new notification framework.

Please go to the Introduction to Notifications session to learn all of the details about the new framework itself.

But the high-level points for you as WatchKit developers are three.

You now have a set of similar notification APIs across platforms iOS, tvOS, and watchOS.

You can now schedule local notifications on your watch.

And notifications with background actions will now be delivered to your extension as long as they were scheduled on the Watch.

Let's look at the API.

The new object, UNUserNotificationCenter, you want to become the delegate of that object probably when your application is launching in the applicationDidFinishLaunching method.

And you want to indicate that you are adopting the notification center delegate protocol.

There are two methods of interest to you in this protocol.

The first is the willPresent method.

This is called when your application is already active on the watch face and the system is giving you an opportunity to tell it how you want it to handle that notification.

So, depending on the option that you use in the completion handler there, the system will do one of two things.

If you say that you want to use the alert option, we will revert back to the regular long-look interface for the notification and it will be overlaid on top of your app.

If you say that you want to do nothing, that means that system should do nothing.

Since your app is already active, you are the ones taking care of displaying the notification content somehow in your application.

The second method is the didReceive response.

This is when your users act on your notification.

So this is the callback when the app becomes foreground because the user tapped on a button.

And this is also the new entry point in watchOS 3 where you're getting a callback for background actions.

But, remember, this is only for background actions that are scheduled on the Watch itself.

This is a good moment to make a quick review of the scheduling and delivery of notifications on Apple Watch.

Anytime a notification fires on your phone, it is forwarded to the Watch if the screen on the phone is off and you have your Watch on your wrist and it's unlocked.

That's always been the behavior.

But now in watchOS 3, you have the ability to schedule your own notifications on the Apple Watch.

So keep in mind that these are only delivered on the Watch.

And if your application needs to also have a corresponding notification on the phone side, it's up to you to communicate to the phone and schedule your notification there.

Now you might be wondering, "If I schedule multiple notifications, what happens when they both fire?"

Well, you have the ability to dedupe the notifications as long as you use the same identifier both on the Watch side and on the iPhone side of the notification.

So if we're in a situation where we're forwarding the notification to the Watch, we look at the ID first.

And if there's a similar ID notification firing at that moment on the Watch, that is giving precedence and that's the one that's displayed so your users don't see two things fire at the same time.

Finally, let's talk about the SpriteKit and SceneKit objects.

So SpriteKit and SceneKit are very rich APIs that allow you to do the 2D and 3D graphics, and they allow you to expand significantly the visual language of your applications.

We have two new classes for WatchKit that support these frameworks WKInterfaceSKScene for SpriteKit and WKInterfaceSCNScene for the SceneKit.

Again, you do all of the configuration in the storyboard and interface builder.

You will now see the two classes appear in the library.

All you do is drag them into your controller, and they really pretty much give you the canvas where you're going to be drawing your scene.

And the configuration is done on the inspector and interface builder.

Once you're ready to display your scenes, all of your SpriteKit and SceneKit code continues to be that.

You continue to use the APIs.

I won't be going into the APIs of those frameworks themselves.

But once you have your scene ready to present, the integration with WatchKit is simply in the presenting of the scene.

So if you're dealing with a SpriteKit scene, you call the percent function or functions depending on whether you have a transition.

And if you have a SceneKit scene, all you do is set the scene property on the object.

And that's all there is in terms of the integration.

So I've given you a quick overview of the new ToolKit items that you have for creating glanceable, actionable, and responsive applications.

Now Tom will come back and give you specific examples of how he combines multiple of these interfaces to make your app more responsive.

Thank you.

[ Applause ]

All right.

And so now we have that ToolKit, so we can look at some practical techniques to actually build these types of quick interactions.

So we're going to talk through adding complications and notifications in your app.

And now these are not new, but we're going to talk through them in some unique and new ways.

Then afterward, we're going to talk about having a simple navigation model within your apps.

And then lastly, we're going to talk about giving your users the confidence to drop their wrist as soon as they're done interacting inside your applications.

And so we'll start off with actionable complications.

And so we introduced complications for third-party apps in watchOS 2.

And now in watchOS 3, they're playing an even larger role.

And now there are more faces that have complications, and there's some of the existing faces that have even more complication slots.

So, also, now you have the ability to quickly swipe across your faces to see all your complications.

So because of this, we think that every app on Apple Watch should have a complication.

And you'll now see that all the native apps have complications as well, as you can see with the Workout app or the Heartrate app.

So why complications?

They're extremely glanceable.

They're on their wrist at all times.

And with just a glance, you can see a lot of really important information.

They are frequently updated.

If your app has a complication on the current face, it's updated at least twice an hour.

And so you can always ensure that you have really fresh and current data always available to your users.

Your app is kept in memory.

And this is really important because it makes the launch of your app extremely fast and makes all the interactions within your applications a lot more responsive.

And complications provide a great way to accelerate your users into your app to perform specific types of functionality.

So, for example, if we look at the Music complication, I tap on that and I'm sent directly to the Now Playing screen because that's what's most important to me if I'm playing music.

I can play and pause the song or skip the track.

And it makes the whole interaction a lot faster because I'm sent directly to where I need.

Also new in watchOS 3 and iOS 10 is the new Gallery in the Apple Watch app on iPhone.

This is great because it allows you to add new faces to your Apple Watch.

But it also allows you to customize the faces and choose among all the complications.

And third-party complications appear here as well, so it's a great way for your users to discover your complications and add them directly to the watch face.

And if you want to be part of the list of complications, you create a complication bundle.

And there's documentation for that on the developer website.

So I highly recommend that you check that out.

So complications on Apple Watch are just a really fantastic mechanism to get your users into your app and really provide a lot of these fast interactions.

So if you don't have a complication in your current app or you're building a new app, make sure to include them.

Next up, we're going to move onto glanceable notifications.

And so we all know that notifications on Apple Watch are really fantastic, and one of the main reasons is that they're entirely customizable.

And so you can make them look however you want.

But also, it allows you to display really valuable information right inline in the notification.

So, for example, here in the Calendar notification, I can see the name of the event and when it is and where it is.

And ultimately, this can often times remove the necessity to even go and launch the Watch app because all the important information is right inline in the notification.

And now in watchOS 3, all notifications support SpriteKit, and SceneKit, and inline video.

And so you can provide even more customization inside of your notifications.

So, SpriteKit and SceneKit I'm sure a lot of you think of these in terms of gaming.

And they're great for gaming.

And there's a session on Friday about game technologies for Apple Watch.

But here, we'd like to talk about them in terms of nongaming types of interfaces.

And so why are they important for nongaming?

They allow you entire customization over the content, and the layout, and the animation of your interfaces.

And so it allows you to create really custom types of looks.

And so, here, we're going to look at an example of a Photos app.

So, here, you can see I got a notification from my friend, Ivy, and she has shared some photos with me.

But this does not include a lot of information.

I have no idea what the photos are.

I have to read the text in order to see who has shared them.

And so we can definitely take a lot more advantage of the customization possibilities in our notification.

So here's an alternative which is now using SpriteKit along the top.

It actually has the photos that were shared with me.

It has the photo of my friend, Ivy.

And with just a glance, I can see a lot more information.

And ultimately, I can make a more well-informed decision on whether I want to go launch the Watch app or even go and grab my iPhone and check these photos out more.

And if we look at how this was done, across the top we have the new WKInterfaceSKScene object.

And then below that, we just have a regular label.

And the key thing I want to point out here is that you can mix and match your SpriteKit and your WatchKit user interface elements.

And so maybe you already are using an animated image or the view animation API.

You can swap those out with SpriteKit.

Or if you just have an existing interface and want to add animation, you don't have to rewrite everything in order to adopt SpriteKit.

So it provides a really powerful way to add a lot of richness to your interfaces without having to throw everything out and start over, which is really cool.

And so if you have notifications within your app, make sure to take full advantage of what they have to offer.

If you just kind of have the default interface, think about what information is really important inside the notification and add a custom interface to reflect that.

OK. So, next up, we're moving on to a simple navigation model within your app.

Earlier, we talked about the new vertical paging API for tables.

And that's great for any hierarchical app.

But here, we'd like to talk about navigation in terms of two other of the technologies that we talked about and those are Gestures and the Digital Crown.

And so the Digital Crown is unique to Apple Watch, and it's a really fantastic input mechanism.

It allows you to quickly and precisely interact with the interface.

And it can be used in many, many different ways.

So, for example, in the Timer application, it can be used to scrub through durations and choose the time you want.

Or in the Photos application, it can be used to zoom in and out of the photo.

So you can get really creative with how you use the Digital Crown in your applications, and we're really excited to see how you'll be doing that.

And here, though, we'd like to talk through an example of a running app that puts both Gestures and the Digital Crown to use to simplify the navigation.

So, here, we have one version of a running app.

The goal is to show my previous run, and to show kind of the average time, and distance, and the altitude.

So one version is just to kind of have a list of every mile.

And maybe I can tap in on the mile and see more information, or I can scroll the list and see the entire run.

And while this version has all the information that I would need, you kind of have to hunt around in order to find what you're looking for.

You have to tap in on rows, you have to scroll the list, and the information just isn't that dense or glanceable.

And so it can definitely be done in a lot better way.

So here's an alternative which now is using SpriteKit to have a graph at the top, and it shows the entire run.

And so with just a glance, I can actually see a lot more of the information.

And now with the use of gestures, I can tap on the graph and it will animate between the pace and the altitude.

And then I can also use the Digital Crown to scrub through the graph to specific data points.

So even though there's a lot more information on screen, it's actually a lot easier to navigate to the specific information that I'm looking for, and it makes the overall interaction a lot faster.

And so, as you can see now, we're starting to mix and match all the technologies that we introduced earlier so Gestures, and the Digital Crown, and SpriteKit and they create some really powerful combinations.

So as you're building your own applications, you should make sure to take into account all the technologies and see how they can help to simplify the navigation within your app.

A lot of focus has gone into navigation in watchOS 3, and we really want to make sure that carries through to all of your apps as well.

And so, last up, we're going to talk about a technique that allows your users to drop their wrist as soon as they're done interacting.

And so a key aspect in making a quick interaction on Apple Watch is that your app feels very, very responsive.

And this is especially true if you're doing any type of networking or communicating with a companion iPhone.

Because often times, that can lead to loading indicators, which are not good.

And one aspect of this is if you're updating the data within your app.

And that's a large enough topic that there's actually a whole talk on keeping your Watch app current and fresh.

So make sure and check that out.

But here, we'd like to talk about networking in terms of responding to user action.

So, for example, you have a food-delivery app, and you want to buy some food and you tap the Buy button.

How does your app respond in that type of situation?

And we want to recommend that you provide instant feedback to your users rather than showing a loading indicator.

So we'll walk through an example of what that means.

So, here, we have that food delivery app I was talking about.

I can tap on one of my favorite items and there's a big Buy button, and so we think this is going to be a really fast and quick interaction.

And I tap on Buy and I get a loading indicator.

And that's because the app has to go talk with the server, it has to process the order, it has to get that confirmation.

And then once it's done that, it has to send back to the Watch.

And only then is it going to remove the loading indicator and say, "Hey, your order is on its way."

And if we look at a timeline of this, it really shows the problem.

Right? We start with choosing the item.

We tap Buy.

And then we show the loading indicator.

And during this time, the user just raises their wrist and stares at your app until you can show that confirmation.

And so the user is only actively interacting with this at the beginning and the end, and so most of the time is just spent waiting.

So we can definitely improve on this type of interaction.

So, alternatively, if we go back to our food app and now we tap the Buy button, we have instant feedback.

The key thing here is this isn't confirmation because we haven't done any networking.

We can't actually say the order is on its way because we have no idea.

But we want to provide this feedback and ultimately give the user confidence that the app is taking care of their order.

And this allows the user then to go and drop their wrist and not have to wait on the application.

And now, in the background, the app will use a background NSURLSession to go talk with the server, and then eventually gets a response.

And then, in the background, the app is woken up and it can use the new local notification support to say, "OK, now your order is actually confirmed.

It's on its way."

And then you have all the same information that you had before.

And looking at a timeline of this, it really shows how much it helps and improves things.

So we choose an item, we tap Buy, and so we start out the same way.

And then instead of showing the loading indicator, we show the feedback.

And then after we do the networking, we show the notification.

And so as far as the app is concerned, the overall interaction is actually about the same as it was in the first example.

But from the user's point of view, it feels far faster and far more responsive.

And so it's just a better interaction overall.

I do want to point out here how you can actually do that processing after the user has dropped their wrist.

If you're doing networking, you can use a background NSURLSession.

And make sure to use a download task because that will enable your app to be woken up in the background after it's been done.

And then at that point, you can do a local notification that makes sense in your application.

Or if you have to do processing on the Watch itself, you can take out a background task assertion.

We had a great walkthrough of this in a session last year called WatchKit Tips and Tricks, so make sure to check that out if you need to do any type of processing on the Watch itself.

So 2 seconds is a very audacious goal in terms of how fast interactions on Apple Watch should be.

But we really think that it should be the goal as you're building your applications because it can really inform how you design and build them.

So as you're going about creating new applications or thinking about your existing ones, make sure to focus on quick interactions right from the very beginning.

And think about what are the features that your users are going to want to be using in your applications, and make sure that those are surfaced to the very front and are really fast and responsive.

Use complications and notifications because they just create really fast interactions overall, and they just provide a lot more utility to the user for your applications.

Use SceneKit and SpriteKit to create really rich but also interactive types of interfaces.

And they can add a lot of power to your existing apps or new apps.

And so we're really excited about what you're going to be doing with those.

Simplify the navigation.

Again, a lot of work has been done in watchOS to simplify the navigation at a system level.

But we also want that to carry through within all of our applications as well.

And, again, never make the user wait.

If you're showing a loading indicator on screen, try and reorganize your app to allow the user to drop their wrist.

Provide the instant type of feedback, and just give the user confidence that your app is taking care of things.

And so for more information on anything that we talked about, go and check out the developer website.

And there are tons of sessions all week.

I highly recommend the Designing Great Apple Watch Experiences talk this afternoon or the watchOS talks tomorrow as well.

With that, I want to thank you all for coming.

And enjoy the rest of the week in San Francisco.

[ Applause ]

Apple, Inc. AAPL
1 Infinite Loop Cupertino CA 95014 US