The Keys to a Better Text Input Experience

Session 242 WWDC 2017

Go beyond the basics and provide a first class text input experience in your app. Learn best practices and how to expand your app by supporting text input traits and QuickType. Get tips on how to optimize your UI when incorporating the system keyboard into your layout. Discover how to create a custom keyboard and upgrade it to a system-wide keyboard extension.

[ Cheering ]

[ Applause ]

Okay, thank you.

Thank you.

Good morning, everybody.

My name is Kasia.

I'm here with my colleagues, Shuchen and James.

And we are here to hand you the keys to a better text input experience in your app.

So, let's see what we're going to talk about today.

We're going to start with integrating the keyboard into your layout.

And we're also going to talk about creating dynamic and more interesting input accessory views.

We'll talk about making your app reach multilingual audiences, using traits to make predictive quick type smarter in your app.

Supporting hardware keyboards.

Creating custom input views.

And finally, tips and best practices for keyboard extensions.

So, I'm going to start with integrating the keyboard into your app.

And for that, I'm going to show you a screenshot of the app we've created for this talk.

It's called CatChat.

It's for talking with your pets.

Not just cats, but in my case, just cats.

And we do have this available as sample code so you can grab it from the site.

Cat pictures not included.

So, let's get started with that.

You're going to see a lot of this in the talk.

And we're going to start with accounting for changing keyboard heights.

So, keyboards come in different sizes.

Depending on the language that the user is using and the settings that they've enabled, you might see different heights of the keyboards that are used in your app.

The English keyboard, with the predictive bar, is one height, whereas the Polish keyboard is shorter, and the Chinese 10-key pinyin keyboard, is taller than both of those.

And there are third party keyboards, which some of you may have even written, and those do not have a specific height requirement.

So, you're not ever guaranteed to get a specific height for a keyboard that a user is using in your app.

So, how do you account for this?

Well, we have a set of notifications.

We tell you when the keyboard shows, when it hides, and when the frame changes.

And for each of these notifications, we'll give you the beginning and ending frame that you get from the keyboard.

Now before we go into how to work with that frame we're giving you, let's talk about a bit of a special case, and that's the undocked keyboard.

In iPads, the user can actually lift the keyboard off the bottom of the screen, and move it around, as well as split it apart.

In that instance, you generally don't want your app to be trying to stay on top of the keyboard, because they might move it all the way to the top of the screen, and then all your content is gone and that's no fun.

So, when the keyboard gets undocked, we will send you a "Hide" notification, just like we would if the keyboard was dismissed.

In most cases, that's how you're going to handle it, as though the keyboard was dismissed.

If you do want to do something clever, we will continue sending you frame change notifications, but for most people, what you're going to want to do is track the most recent hide or show event, and then deal with the keyboard accordingly before you deal with any frame changes.

So, the undocked keyboard is something like this.

We have this nice scroll view.

And when the keyboard comes up, the scroll view compensates.

When it's undocked, it takes up the full screen again.

And when the user docks it again, it comes back up so that you can see the entire transcript of your delightful chat with your cat, which is important.

So, that's the kind of special case.

So, let's move on to actually working with that frame notification that we give you.

First things first.

The keyboard is always full screen.

It's always in the full screen coordinate space.

So, the first thing you do when you get a keyboard frame from one of our notifications, is convert it from the frame the screen coordinates, to your local coordinates before you start working with anything else.

Because the keyboard can be full screen when you are not, and because you can you know there are all sorts of fun situations where keyboards come up and down.

You always want to make sure to do this first.

You can't count on the frame that you get being local to the frame of your app.

Think about multitasking, for instance.

Then, if you do use the intersection of the frame that we give you once you've converted it, and the view you want to use, you can then get the difference in height between the two of those or the height of the keyboard as it will overlap your view, and use that to adjust for the keyboard.

So, let's talk about a couple different kinds of layouts.

Let's start with layouts that do not scroll.

So, static layouts are generally a couple of text views, maybe a button, where the user doesn't have to move around a lot in order to see everything in your view.

But you still want it to look nice with all keyboards.

In this instance, as an example, we have five text fields and they are tied together with UI Layout Guides acting as spacers.

AT the top of the view, they're using The Safe Area Edge Insets, and at the bottom well, we'll talk about that in a second, but the spacers change size depending on what keyboard appears, so that it always shows all five of the text views to the user.

So, how did we do that?

Well, we created a custom UI Layout Guide with a height constraint that we hold onto so that we can adjust it later.

The Layout Guide gets tied to the bottom Safe Area Layout Guide of your view.

And the top of the Layout Guide should be tied to the bottom of the view or spacer that you want to appear directly on top of the keyboard, when the keyboard has appeared.

Once you do that, you can do your height conversion after checking that the keyboard is visible, and simply set that height constraint you held onto, to the height of the keyboard.

If you pop that into a view animation block with N call layout if needed, you'll get the cool little animation between the keyboards that you saw, just a moment ago.

The more common case however, are scrolling layouts.

Layouts embedded in the scroll view so that the user can move between your views, and see everything that they need to see.

In our case, we had this conversation View Controller in our Cat Chat app.

And we want to make sure that you can scroll through the entire conversation at any time.

So, we need to make sure that the bottom of the table view is adjusted to how the entirety of your chat transcript.

So, how do you do this?

Very similar to before but a little less complicated.

You simply make sure the keyboard's visible gain, do your frame conversion, get your height, and set the content inset of the scroll view.

This is where that intersection bit is helpful because if you have a scroll view that does not take up the full screen, you only want to offset it by the bit that's covered by the keyboard.

Finally, you do need to handle scrolling the content, once you've adjusted your scroll insets.

In the previous example, we were actually using a UI Table View Controller, which is even easier because it sets the insets for you, and all you have to do is scroll the content, which is nice.

In this case, we have a table view and we want the most recent item to show right above the keyboard.

So, we scroll the bottom cell, to the bottom of our inset table view.

So, that is adjusting for the keyboard in your layout.

Now, let's talk about extending the keyboard with an input accessory view.

The input accessory view, is a view right above the keyboard.

Here, it's a text view, and a button.

IT can also be a tool bar with some navigation buttons.

It can be a row of buttons.

If can a couple of cool little, you know, photos or something.

Anything you want to put on there, you can because it's just a view.

How do you add this to the keyboard?

Well, here we have the an input accessory view in our chat conversation view controller.

And so, it's the host view controller that wants to display the keyboard with the input accessory view.

So, we make it part of the responder chain by returning True for, "Can become first responder," on our conversation view controller.

Then, using either a custom view or a custom view controller, depending on what exactly you want to do with that view, you override the applicable method.

Input accessory view, if you're using a view.

Input accessory view controller, if you're using a view controller.

So, the input accessory view that you add, is accounted for in frame change notifications, and if the height of that input accessory view changes, it will send a frame change notification.

So, you can take care of that in the app that the keyboard is in.

Let's talk about making a dynamic height changing input accessory view, because this is something that a lot of people want to be able to do.

So, when you're doing this, this acts a lot like a self-sizing table view in that you want to take whatever view is going to be changing size, and use it to define the height of the view.

And you can do this with more than one, but in this example, we'll use a UI text view to show how this might be done.

If you pin you can pin it to the top and bottom of the content view with spacing if applicable.

And if there are views in between it, that's fine as long as the height is fully defined.

Again, it's very similar to self-sizing table view cells.

At that point, you need to use or define the intrinsic content size of your growing view, so that the input accessory view knows how talk it needs to be.

With a text view specifically, in order for it to have its own intrinsic content size so that you don't have to define it, you need to tell its text container to have its height track the text view.

And you also need to disable scrolling.

At that point, the text view defines its internal intrinsic content size, based on the amount of text in the view.

And you can get something like this.

So, this is one of the few cases where you may need to actually override intrinsic content size in the input accessory view.

And to do that, you don't want to spend, you know don't want to make it too complicated.

It's actually not too bad.

Grab the bounds as they come in.

That's going to be your original bounds, and we'll make sure that you have the correct width, which is already given to you by the system.

In my case, I always like to make sure that the view has a minimum height, so I set it to minimum height first, just in case everything else all my other ifs are elses.

And then, if my expanding view is the correct height, I use that plus some vertical padding I've previously defined to define the new height of the view.

I also have a maximum height because there's only so much I actually should type to my cats at once.

They don't have great attention spans.

So, at a point, it stops.

At that point, you simply return your new intrinsic content size, and your input accessory view will grow.

Okay, so we have now talked about integrating the keyboard into your layout, and adding a dynamic input accessory view.

Now, we have an app where I can talk to my cats, and I can say more than a couple of words at once, which is nice.

He doesn't need very many words.

He's very specific in what he needs from me.

But there's more cool things that we can do with this app to make it a lot more interesting.

So, I'm going to hand this over to Shuchen, and she's going to tell you all about how to make your app feel more magical.

[ Applause ]

Hello, everyone.

So, in iOS, we provide a lot of sweet features so that you can improve our user text input experience.

My name is Shuchen Li.

I'm going to talk a few of them.

So, I hope after my talk you can utilize these tricks and make your app feel magical.

So, let's get started.

So, first of all, let me ask a question.

Who are multilingual people here?

Oh, I see quite a few.

So, am I. Do you face the situation that you have to keep switch keyboard?

Oh, I see one there.

Oh, I see a few of you there.

Good. I suffer that problem too.

And I have to talk to my boss with one keyboard.

I talk to my mom with another keyboard.

I have to keep switch between these.

What if your app just know what keyboard you use?

What if when keyboard brings up, it's just the right keyboard for you?

Actually, you may know this.

In our Message app, it does that for you.

I talk to John with English keyboard.

I talk to my [inaudible] with Chinese keyboard.

And Magic knows what keyboard I'm using.

How does that work?

In iOS, we can associate a unique identifier called Text Input Context Identifier, with your text view.

Actually, we can associate that with all your UI Responder.

That is, you can also do that with your view controllers.

So, when we see you have a unique identifier set up, whether you're a responder, we will associate that automatically with the user selected keyboard, and then save that into user default.

Next time, when you are a responder, becomes a first responder, the keyboard will be retrieved automatically.

Sounds complicated.

The only thing you need to worry about is to find a good unique identifier.

What are good unique identifiers?

For example, your user ID, your conversation ID, the group IDs, these are all good examples.

So, you may ask, "Where should I put this unique identifier?

You mentioned, yes, I can define it in all the UI responder.

Do I define it everywhere?"

Of course not.

So, let's see how we put it.

Let's recall our CatChat app.

When we bring out the keyboard, the text view becomes the first responder.

And then what's next, in the responder chain, is our conversation view controller.

And then, it's navigation controller.

And then, in iOS, when we look for the unique identifier for your responder, we will search and look up the responder chain to look for the first one.

So, in our app, we handle all our conversation in our conversation view controller.

So, we put it there.

In such way, you don't have to find a way you pass down your unique identifier to your [inaudible] or other places that it want to pass in.

So, you just put in where you handle all your conversations.

Let's see how we transfer all these into code.

In our conversation view controller, we override the [inaudible] for Text Input Context Identifier.

You can see this method.

It's very simple.

It's just return a unique identifier.

In our case, we return our recipient's ID.

Now, Kasia is going to show you a demo to see how we give our app a memory.

Kasia?

[ Applause ]

You may not remember me.

It's been a while since I've seen you all.

But Kasia's still on keyboards.

Okay, so first here we have our CatChat app that we've been looking at today.

And I've got a couple of pets that speak different languages.

I have my Misiu here.

He speaks Polish.

And actually, I learned as a child that all cats speak Polish.

So, it's important to speak Polish to all of them.

But, my default keyboard as an English speaker is English.

So, if I go ahead and change it to Polish, and say "Hello," let's see.

And see, you know, how he's doing today.

This is me saying, "Hey, how are you doing?"

And he wants food.

This is so surprising.

Well, let's check in on Shuchen's dog, Jiwang.

And Jiwang would like to go out for a walk, but I don't currently have time.

However, he speaks Chinese, so I'm going to switch keyboards again to the Chinese pinyin keyboard, and say, "I don't have time right now.

Please be patient."

And dogs are a little bit friendlier sometimes with their responses than cats.

So, now, this is the first time I've used this to talk to both of them.

So, if I go back into Misiu's conversation, the Polish keyboard is still there.

And if I go back here, I still have the Chinese keyboard.

So, I don't have to keep switching keyboards, depending on the conversation I'm in.

And all that needed to happen for that to happen, was to set that text input context identifier, as Shuchen showed you, with the unique ID of each individual chat.

If I change languages now, my app doesn't need to know what language I'm in.

It just needs to know that it's in a specific conversation and we will take care of setting the proper keyboard for you.

So.

[ Applause ]

Okay, back to Shuchen.

Thanks, Kasia.

A quite neat feature, right?

It'll save quite a bit of wear and tear on your [inaudible] I believe.

So, in this whole process, you may notice, you don't really need to care what keyboard the user selected.

You don't need to know.

The only thing you need to worry about is the unique identifier that was set up.

Now you may ask, what if you want to know what's the language the user used?

What if you want to develop a specific feature for a certain language?

Now you may want to look up UI text input mode.

In UI text input mode, you can easily access the primary language for the user.

In such way, you can for example, develop your own spell checker for your own language or you can develop other specific feature for the specific language that you want that probably [inaudible] support where we will support maybe in future.

Now, next, we've been working really hard on how to accurately and quickly input text.

We made it further through a context scroll predictions.

Also, we take user personalized information into account.

Back in iOS 4, we introduced keyboard types.

There's different keyboard types like email keyboards, URL keyboards.

If you're typing an email address, the @ and the .com keys are there for easy typing.

Last year, we introduced UI text content type.

UI text content type, it's a tag that you can set to your text view.

It will provide all the context predictions through the tags.

Let's see an example.

So, now, John wants to add his cat, Penny, into our CatChat app.

This is John Appleseed's cat.

From the video, you can see actually John didn't even type on any of the letter keys.

He was just typing on the Quick Type bar.

All the informations are input.

Now, John wants to type the address for the cat.

So, that's John Appleseed's cat.

So, it lives in must be living in an Apple store.

Which one?

The one is Los Gatos.

For someone if we don't know, Los Gatos means "cat" in Spanish.

So, we can see, John went to the map and browsed the Apple store and see what's the address.

And then come back to our CatChat app.

And then you will see, ta-da, the address are there, magically.

And by typing on the Quick Type bar, you can thank you.

If I type on the Quick Type bar, you can directly input the whole address.

If you look closely, you can see its source.

It's from Map, in our case.

So, in iOS, we've been pulling all user information and user activities as a whole, to come with the data source, to improve user's experience.

For example, our keyboard personalization also can extract names to form keyboard personalization lexicon.

Also, one more thing I want to mention.

In this whole process, your app is not only a passive receiver.

Your app can be a contributor.

Your app can contribute your app's activity through NS User Activity APIs.

So, for example, if your browse your favorite restaurant in Yelp, the address can be displayed on the Quick Type bar, if you set the tag properly.

So, sounds interested?

If you want to learn more, don't forget to look at the session from last year called "Increase Usage of Your App Through Proactive Suggestion."

So, this year, we made it further.

We add some new tags for user name and password.

When you label your log-in field properly, we will automatically suggest user name and password for your app.

And also, fill that for you.

The only thing you need to do, if you set your tags properly, this is a one line API super simple, and it will guarantee that some of the Quick Type suggestion will display.

However, this is not the full story.

If you want to make sure all your credentials displayed properly with your app, because you may have a website to associate with your app, you'd want to check out "Introducing Password Auto Fill for App" session.

So, next up, we talk about some new smartness.

This year, we introduced some smartness into our text view.

First one is Smart Quote.

Here are typewriter quotes.

They are straight and some people don't like it.

I don't like it.

These are typographic quotes.

I like it.

They are curly.

They're more legible.

And actually, they're more grammatically correct in the language perspective.

And this year, iOS will automatically convert your typewriter quote into typographic quote.

[ Applause ]

What's next?

When you type on the middle bar line, sometimes you just think about, "Okay, a hyphen, an end dash, and add dash," how to do that.

From this year, we can input a hyphen with one dash.

We will automatically convert two dashes into an end dash.

And then, add dash with three dashes of course.

[ Applause ]

Also, we introduce Smart Insertion and Smart Deletion.

If you work with tags before, you have to insert and delete a paragraph or a phrase, you know what I'm talking about.

And not even mention the trouble caused by punctuation.

So, for example, in our example, Vivian wants to bring Jiwang also to the party, but then Jiwang was sick.

So, we have to remove him.

Now, in this whole process, you have to deal with a space.

That's the trouble.

Troublemaker.

So, from today, we will take care of all of that for you.

With Smart Insertion and Deletion, we'll take care of all the leading and trailing spaces for you.

[ Applause ]

Also, I want to mention, all the smartness that we introduced that I have talked about today, they are all by default.

But I want you note, for these smart text input traits, they have three values.

They have yes, no, and the default value.

We set the default value for default for all of you.

We will automatically do the right thing for you.

For example, if you have a text field, you set the Secured Text Entry to Yes.

Probably it's the password field.

In that case, if you set your smart traits to default, we will automatically disable the smart quote conversion for you, because we think probably that's the user password.

We don't want to switch user typing for a password field.

If you recall, actually we can config auto correction types.

We can config auto capitalization types.

There are a lot of sweet traits that we have in Text Input traits.

All of them have this default value.

What I want to emphasize here is, before you play with any of these traits, always understand what your tags is going to be.

Always understand what the tags you're expecting the user to type.

So, let me ask another question.

Who flew to San Jose for WWDC?

Well, welcome.

So, am I. Kidding this time.

If you have a text entry, only take flight code as an input, in such case, probably you want to disable auto correction type and auto capitalization type setting to No.

Right? Because that makes sense.

So, you always want to fully understand your text before set all these traits.

Not just, okay, type on the tab and just say, "Okay, yes," or "No," just by thing.

Always fully understand it.

This is my point.

Next, while work with text, you can now ignore marked texts.

Two big names, Chinese and Japanese.

They all use marked texts.

So, marked text language call word, the marked text into final text by select candidates.

In iOS, actually you can easily access marked text from UI Text Input.

You can directly use the marked text.

We have an example in our system is the Spotlight.

In Spotlight, when user type in Chinese for example, before users select the candidate, Spotlight engine already start to show you all the results.

You can also do that in your app through the marked text by retrieving it from UI Text Input.

In such way, user won't be interrupted by selecting candidate.

The user can spend more time to enjoy your app.

I won't dive into too much about internationalization stuff, but if you're interested, please watch Internationalization Best Practices from last year.

Last but not least, hardware keyboard.

I think everyone use hardware keyboard and I believe you all love the keyboard shortcuts, like Command C, Command V, probably your favorite is Command Z.

We brought that to iOS with Smart Keyboard.

So, in iOS, you can define your own shortcut for your user if I have iPad app.

So, let's see, how we do that.

Let's dive right into the code.

It's very straightforward and easy.

Again, in UI Responder, you can override the key commands.

Key commands return an array of key commands.

That means you can define a couple shortcuts for your app at the same time.

To define a key command is very straightforward.

You give it inputs.

You give a modifier.

You put your selector there.

And then also give it a name.

Also, don't forget to localize your name.

This is our second shortcut.

Now, press on the Command key, it will display the keyboard shortcut for you and then you can see in our CatChat app, we have Command Up and Command Down to switch between the conversations.

So, let's recap.

I've talked about the multilingual problem to how to give your app a memory.

I've talked about how to take most advantages out of the context for predictions by giving your text field a tag through text content type.

I talked about the new smartness for text input fields.

I also talked about the mark text.

Last but not least, there's keyboard shortcuts you'll want to try out.

Try them all.

Next, James is going to talk about how to create a custom input views.

James?

[ Applause ]

Thanks, Shuchen.

I'm very nervous I mean, excited to tell you about creating custom input views.

So, Shuchen briefly mentioned a UI Responder property called Keyboard Type, which allows you to access a lot of the built-in iOS system keyboards for specialized input, such as the decimal pad for entering decimal numbers, and we also have a phone pad for entering phone numbers, and a special keyboard just for URLs and email addresses.

Sometimes though, this may not be enough for your application.

You may want to go beyond what the system keyboard provides you.

Let's take a look at a couple of examples.

So, mathematics is a pretty good example.

In math, there's a lot of custom symbols like Greek letters, superscript operators, and subscript operators.

It could be pretty difficult to type something like this with the built-in system keyboard.

Swift Playgrounds is another good example.

This is the Swift Playgrounds app.

They have the standard QWERTY layout, but they provide a lot of additional functionality for frequently used symbols in the Swift programming language.

And they also have a really nice accessory bar in the top here, for quickly inputting relevant functions and symbols.

And then lastly, Apple numbers, spreadsheets, is another good example.

So, there's a formula entry bar here, which also requires a lot of custom symbols, and they also have a nice, dark gray background here, so it blends in really nicely with the system keyboard.

So, going back to our CatChat app, it's another pretty good example for something that needs a custom input view.

Why? Well, right now, we have a way to talk to our pets, but our pets don't have a way to talk back to us.

And that's because, they have a lot of trouble using the QWERTY keyboard.

Their paws are quite large.

The keys are really small.

And so, they have a lot of difficulty hitting those keys.

Plus, they have a really limited vocabulary too.

So, they don't really need to say a whole lot.

So, let's say we want to make a keyboard just for them, accessible to cats and dogs.

So, maybe we want just large graphical buttons to represent their entire small vocabulary.

So, here's what that might look like.

This is our PetChat app, again.

It's the it's got the cool, expanding text view that Kasia built for us earlier.

And now, on the pet side, we have three, large graphical buttons with representing their entire vocabulary of things they might want to say like, "I need food," "I want to go outside," and "Stop petting me."

Something like that.

The other cool thing too is that this behaves just like the system keyboard does.

It sits at the bottom of the screen.

There's an accessory bar at the top that's laid out, right at the top of the where the keyboard would go.

And then also the scroll view in the background, automatically adapts its content insets to the size of your custom input view.

So, let's go into a little bit more detail, about how to build something like this.

So, for our PetChat app, to create something like this, all we need to do is subclass UI Input View Controller.

And by doing that, we get a lot of stuff for free, such as the blurry gray background that mimics the system keyboard's appearance.

But most importantly, we provide you a way to access the currently interactive text field that the user has selected.

And we do this via a special property, called Text Document Proxy.

Now the Text Document Proxy allows you to access things like the text input traits for the currently selected text field.

So, that would be stuff like whether or not it's a password field, what kind of content type it prefers, and even what kind of keyboard type it prefers.

So, you can adapt as necessary.

But most importantly, the Text Document Proxy also allows you to insert text into the text field, via a simple method called Insert Text, which just takes a string.

Oh, and one more thing that you can do with the Text Document Proxy too.

You can access where the insertion point position is, and the text before and after the insertion point.

Pretty cool.

So, let's talk about how we can actually get this inside of our PetChat app.

Well, let's go back to our old friend here, the conversation view controller.

So, to get it to show up in our conversation view controller, we simply need to instantiate our custom input view as a member variable.

And then, override a UI Responder method called Can Become First Responder, to return True, so that it becomes a part of the responder chain.

And then lastly, all we need to do, is override this other UI Responder method called Input View.

And then just return our custom input view instance.

Now, if you're using things like UI Text View or UI Text Field, you actually don't need to do most of this.

UI Text View and UI Text Field, the input view property is a writable property.

So, all you really need to do is actually just assign your custom input view instance to the input view property.

So, now, our pets have a way of talking back to us in the PetChat app, but let's say they actually do kind of want to break out a little bit and start Tweeting or maybe writing a blog.

I'm sure they have a lot of interesting things to say with that small vocabulary of theirs.

So, in iOS 8, we actually added the ability to create custom keyboard extensions, so that it runs anywhere in the system.

This is actually our first time talking about it at WWDC.

So, here's what our custom keyboard extension might look like, running inside of messages.

So, you can your pets can chat anywhere, or type into Twitter or keep a blog or something.

So, it's actually really easy to go from an input view into a custom keyboard extension.

And so, I'm going to give you a quick demo and show you how to do that.

So, let's switch over to the computer here.

Okay, so this is our custom input view controller here.

I'm not going to go into too much detail about how it was built, but our sample code is uploaded online.

So, you can download all this yourself if you want to take a look at it.

But to take let's go ahead and build and run this first to see what it looks like inside of our app.

Build and run.

Alright, so here's our custom input view running inside of our app.

And so, I'm going to pretend like I'm a pet and just start saying what I'm feeling.

I didn't have breakfast this morning, so I'll do something like that.

So, now, if we actually want to take this custom input view and convert it into a third-party keyboard extension, it's quite easy.

So, let's switch back over to Xcode here and I'll show you how to do that.

So, first, I'm going to go over here to the top left, to the Project Settings.

And I'm going to create a new target by clicking this plus button at the bottom here.

And so, I'm going to go ahead and choose the custom keyboard extension template, and click Next.

And give it a name.

I'll just call it Animal Keyboard.

So, now, by doing this, Xcode actually already gives us a bunch of stuff for free.

For instance, it sets it up so that this extension is automatically embedded inside of your app bundle.

So, you don't need to mess with build settings or build phases or anything like that.

And it also gives us a template, UI Input View Controller, here.

But we actually don't need to use this because we already wrote one.

So, I'm just going to tweak around with a couple of settings here, so that our extension loads are already built, Custom Input View Controller, instead of the one that the template gives us.

So, I'm going to go over here to our Custom Input View Controller, and I'm going to check the box over here for our keyboard extension so that it's a member of that target.

And then, our keyboard uses a couple of other resources.

It has a couple of localized strings.

So, I'm also going to go ahead and switch to localized strings and check this checkbox to make sure it's part of that target.

And then lastly, it also has a couple of assets that it uses for images.

So, I'm going to check this box.

And then, one more thing I have to do.

I'm going to go over to the info P-list here, that it generated for our custom keyboard extension, and change its principle class to the one that we already made, from the one that the template gave us.

And that's pretty much it.

So, I'm going to switch back to the app target here and build and run.

So, now that the app is installed on the system, the extension came along with it.

And there used to be now we have to enable the custom keyboard extension in the System Settings, and there used to be a really convoluted way to do that.

But in new, in iOS 11, we actually made it much simpler for your users to enable your custom keyboard extension.

So, I'm going to show you how that works right now.

So, I'm going to go back to Settings.

So, all you need to do, to enable your to allow your users to enable your custom keyboard, is provide your app with a Settings Bundle.

It can be empty or it can contain some of your settings.

And we have this new option, right underneath there, under Siri and search that just says, "keyboards."

And by clicking this, the user all the user has to do is check this one switch.

And now the keyboard's enabled.

[ Applause ]

I'm sure you third-party keyboard developers are going to love that.

So, now, I'm just going to go over to messages here.

And then, I'm going to switch to our custom keyboard extension.

And there we go.

Now we have it running, all over the system.

And so, I'm just going to express how I'm feeling about the demo right now.

And that's all it takes.

So, let's go back over to the slides.

[ Applause ]

Thanks. So, this is actually my personally recommended way for you to develop third-party keyboards.

It's really easy to test it inside of your app first, just as a custom input view.

And then, as you saw, going from a custom input view to a custom keyboard extension is really easy.

So, that's really the way to go.

So, just a quick recap of how to convert from a custom input view to a keyboard extension.

We just made a new target and we used the Custom Keyboard Extension Template.

You can either fill in the provided UI Input View Controller Subclass, or change it so that it works with your own.

And then lastly, I showed you a cool, new user acceleration or a cool new acceleration for your users to enable your keyboard from within your Settings Bundle.

But that's not all.

We have a couple of new APIs in iOS 11 that I want to share with you tonight or today.

So, you can now access the currently selected text that the user has selected, which is pretty useful if you want to do retro correction with your Auto Complete Engine, for instance.

Or provide some kind of cool context around the text that the user had selected.

We also provide you this new thing called the Document Identifier Handle, which is just a UUID, that uniquely identifies one text field from the next.

Now, this is real useful for a lot of things, but one of the things that it's useful for is for resetting your keyboard state when the user switches text fields, so you can clear your predictions if you have predictions displayed, or switch back to the alphabetic plane if the user had switched to your numeric plane, or something like that.

And lastly, we also added the ability for you to query whether or not the user has granted your extension full access.

And I'll go over what full access means in just a moment, in case you're not familiar.

But first, I want to talk about some other lesser known, third-party keyboard APIs that maybe you didn't know about.

So, in iOS 10, we actually added the ability for you to incorporate the System Globe Menu inside of your keyboard, which is really nice so that your user can quickly switch between all of the keyboards that they have selected.

It's a lot better than that next keyboard thing that we had before.

Oh, thanks.

[ Applause ]

You can also personalize your auto complete results with something that we call the Supplementary Lexicon.

The Supplementary Lexicon is basically just a list of words containing the portion of the user's address book, so you can auto complete various names and stuff, that the user might be referencing in the text that they type.

And finally, tying back to some of the multilingual support that we added in iOS 11, that Shuchen was talking about earlier, your keyboard can use those document identifier handle things that I was mentioning before, as a multilingual hint.

So, if your keyboard supports multiple languages, you too can behave like the system keyboard does, and change when appropriate.

So, lastly, we can't talk about third-party keyboard extensions without talking about privacy and user trust.

So, privacy is very much a standard iOS feature that users expect.

And users are made aware that all of their keystrokes are handled by your extension.

So, if you're doing something with those keystrokes, such as sending them offsite or analyzing them in some way, you should make sure to inform the user about this, and anonymize the data that's coming in, if you are sending it offsite.

And lastly, most applications shouldn't actually need it, but you can ask the user to provide your extension with full access, which means access to some lower level parts of the system.

Let's talk a little bit about what full access really means.

So, first of all, there's actually quite a bit of value in not asking for full access.

This is why we built the full access switch to begin with.

It's not a speed bump for you guys.

Instead, our Apple customers are very privacy conscious.

Plus, your keyboards already provide a lot of awesome functionality right out of the box, without full access.

So, granting this should really only supplement your existing feature set.

So, some examples of things that you might need full access for, is communicating with your main app.

So, if you need to build a bridge from your main app bundle to your custom keyboard extension, if you want to access like a database or something that you have stored there, you'll need to request for full access for that.

If you want to access a if you want to provide your user with a repository of gif images for instance, then you'll need to ask for full access to grant access to the networking stack on iOS.

So, for current location, in case you want to enrich your auto complete results with references to locations that the user might be referencing to in the text that they type, you'll need to request for full access for the current location.

And you'll also have to have the user grant your main app access to your current location too.

So, keep that in mind.

And for the address book, most applications don't actually need this because of what I said about the supplementary lexicon.

So, you actually don't need full access to get to the address book.

But, if you want to access some more of the address book, like the notes fields, or nickname, or something like that.

Then you will need full access for that.

And lastly, make sure that your keyboard works without requesting for full access.

Like I said, there's a lot of value in not asking for full access, and plus, your app will actually get rejected if it doesn't.

Take a look at the App Store Guidelines for more information about that.

So, just a quick summary before I leave you tonight, or today, sorry.

I did that again.

Kasia showed you how to design your app with the keyboard in mind and about keyboard avoidance.

Actually, we don't say "avoidance."

We say, "embrace the keyboard."

Don't avoid it.

And Shuchen mentioned some advanced traits to enhance the user's experience with some of the built-in system keyboards that you can pick from.

And if that wasn't enough, we showed you how to build custom keyboard extensions and custom input views, and how going from a custom input view to a keyboard extension, is actually quite a lot easier than you think.

So, make sure to like, comment, and subscribe, or check out our sample page or sample app, here at WWDC 17, Session 242.

All the code that you saw today, is right there.

So, all the sample code that you saw today is right there.

So, go check it out.

Here's some related sessions.

The Password Autofill for Apps is definitely a good one to check out.

Mysteries of Auto Layout is great too for keyboard avoidance and or keyboard embracing.

Thank you.

[ Applause ]

Apple, Inc. AAPL
1 Infinite Loop Cupertino CA 95014 US