Integrating Apps and Content with AR Quick Look 

Session 603 WWDC 2018

With iOS 12, ARKit includes a built-in viewer for displaying and sharing high-quality 3D content using Pixar’s usdz file format. Get introduced to the capabilities of AR Quick Look and see how to easily integrate AR into your app. Learn best practices for preparing 3D models for AR, and see how to deliver usdz content for viewing across Safari, Mail, Messages, and other built-in apps.

[ Music ]

[ Applause ]

Good afternoon, everyone, and welcome to Integrating Apps and Content with AR Quick Look.

My name is David Lui, and I’m so excited to talk to you about AR Quick Look.

As you all saw yesterday at the Keynote, we brought this new capability of previewing 3D models in AR space throughout iOS.

AR is becoming the tool to see what objects look like in the world, from a new sofa you’re purchasing for your living room to that cute animated character you’re reading about in your book.

AR Quick Look was designed to enrich any application on the OS with AR content with a simpler way to adopt AR previewing for consistent viewing experience.

So, in this session, we’ll take a deeper look at what AR Quick Look is, how to design your app and web page to take advantage of AR Quick Look, and creating content specifically for it.

So, let’s get started.

So, let me show you what AR Quick Look is with the files app.

Let’s preview the 3D silver pitcher model and push it into AR.

We find the table, and we place the pitcher on the surface.

You can see the pitcher on a 1-to-1 scale and get a better understanding of how it fits with your surrounding.

On my table, I have an Apple mug, some Post-its, and a kettle, and it looks so realistic with these items.

And what’s really amazing is that you can see the colorful Post-its reflecting on the lower body of the pitcher, giving this illusion that it’s actually really there.

So, it’s really magical how the virtual pitcher blends in with the physical world.

To help get my lovely pitcher in the world, we’re using AR Quick Look.

The way to preview 3D content in AR.

AR Quick Look is built in and deeply integrated into the OS to allow previewing from any application and websites.

The deep integration comes from us using, of course, Quick Look technologies and expanding its capabilities to enhance the system experience with AR.

The AR portion of Quick Look is available in iOS 12 on ARKit compatible devices and only object mode on non-ARKit supported devices.

You’ll see this new AR file icon throughout the OS to represent the new 3D file format usdz.

Let’s invite you to Quick Look, the 3D content in our viewer and see how it looks like in your world.

So, let’s tap on the icon to launch AR Quick Look.

AR Quick Look handles setting up the AR experience like plane detection, object placement, gesture manipulations, and creating contact shadows.

But these are only a subset of all the viewer capabilities that together with rendering a 3D model, provides an immersive experience.

So, use AR Quick Look if your goal is to preview 3D objects and push them into the world.

By adopting our new system viewer, you got a consistent 3D content previewing experience, so you don’t need to create your own.

It’s really easy to adopt and integrate the viewer into a website and application and it could be as simple as embedding a usdz file if your application already supports and uses Quick Look.

Just as you saw from the previous slide, AR Quick Look does all the heavy work for you in providing and creating a great AR experience.

And it’s truly magical how you get an amazing AR experience by just providing the 3D content and we make this super easy for you so you don’t have to deal with any AR specifics.

So, how about we see AR Quick Look in action with the web demo.

[ Applause ]

So, here, I have the files up, and I have a catalogue of 3D objects.

You’ll notice that there’s these rich and beautiful thumbnails with the AR badge stamp at the top right-hand corner, and that’s there to tell you that there’s an AR experience behind this.

So, I have a lot of great models to choose from, but my personal favorite is this little purple guy at the top right-hand corner.

So, Radar is our bug tracking software, and if you don’t know, the purple aardvark is our mascot.

So, I’ll go ahead and Quick Look the purple aardvark into the viewer where we can see her in our studio-like environment, which we call object mode.

So, you can really pan around and see the model from various different angles.

We can pinch to size an object and make it look a lot larger and really see the fine details that went into this model.

So, you can see how there’s animation baked into this model, and AR Quick Look supports transform animation and skeletal animation, so the purple aardvark is really lively and animated with personality.

And just like a photo, you can double tap on the model to recess the position and the size.

But of course, the beauty really comes from pushing my purple aardvark into the world.

So, I’ll tap on the AR tab, and there, the purple aardvark is in the world.

And so we were able to place the model so quickly into the world because we started the AR session the moment AR Quick Look was launched, and we gathered all the feature points of the environment and pushed the objects quickly out into the world.

So, right here, you can notice that the controls are hidden for a full immersive experience.

Controls like the status bar, on-screen controls and iPhone X, we hide the home indicator.

So, you can go ahead and rotate the objects using your standard iOS rotation gestures.

You can scale the object down using your two-finger pinch and scale it larger if you want it to be a lot cuter.

And so, you can see that animation there where the purple aardvark is just fixing all my bugs and just slurping them up.

So, I can go ahead and double-tap to reset the size back to 100%, and you can translate the model by just tapping on it and dragging it to a different place on the table.

Or you can tap on it and move the device around for device movement, really seamless interactions.

Of course, since this is an AR experience, we really encourage you to walk around and be really engaged and immerse and see that animation, see the model that’s pushed out into the world.

So, we’ve also provided a contact shadow.

So, you’ll notice right there just sitting on the edge, this tells you that the purple aardvark is grounded and placed on that table in the world.

And what makes the aardvark look so amazing right now is that we’re matching the color intensity and the temperature that’s reported by ARKit from the world, and we’re using that with our lighting setup.

So, I think this is really amazing.

I want to remember this moment.

So, I’m going to go ahead and tap to show the controls.

I’m going to take a snapshot of this.

So, right here, I’m going to go ahead and mark up the moment, and this is really amazing, so I can save this later and take a look at this.

But with AR Quick Look, we really wanted to make AR accessible to everyone, and we do mean everyone.

Accessibility is one of our core values, and so we made AR Quick Look work with VoiceOver and switch control.

So, let me show you the VoiceOver flow right now.

So, I’ll triple click the home button to activate VoiceOver.

VoiceOver on.

Files. Close.

Button. Landscape.

So, I navigate over to the AR tab.

Home button to the right.

AR button.

Selected AR.

Radar aardvark is now positioned in the world.

And what’s really cool is now you get audible feedback for when my Radar aardvark is off screen.

Radar aardvark is now off screen.

AR when is back on screen.

Radar aardvark is now on screen.

And this is really amazing, and we really encourage developers and users to really think about the full accessibility workflow in your applications and websites as you adopt and integrate AR Quick Look.

[ Applause ]

So, I just showed you AR Quick Look integrated into the files app, but that’s one of six first party apps to have adopted and integrate the viewer.

The others are mail, messages, notes, news, and Safari.

And the great news is with AR Quick Look built into Safari itself, developers can integrate it into their websites by using our notational [phonetic] markup that I’ll talk more about later.

Lastly, you can adopt a viewer into your iOS application to provide a more immersive experience with AR.

So, sharing a 3D model between all these apps requires the models to be bundled up in a single sharable file.

To enable this, AR Quick Look supports usdz, which is a new file format for mobile distribution of 3D models.

usdz packages all these models and textures into a single file for efficient delivery of 3D content without having to work with reference files.

It’s based on Pixar’s open-source universal scene description format, or short, USD.

And usdz is supported on iOS and macOS, in SceneKit and Model I/O, and later in the session, we’ll show you how to convert 3D models into the usdz format using the new usdz Converter tool that ships as part of Xcode 10.

Now that you know more about AR Quick Look, let’s talk about how to integrate the viewer itself.

You can integrate AR Quick Look in two different mediums.

The first, in an application.

Apps that showcase a lot of photos and text can provide better imagery and a really immersive experience with AR Quick Look integrated.

And here, we’re using Quick Look API to facilitate this integration, to preview usdz files, and load them into the viewer.

Second, is in websites in Safari.

So, integrating AR Quick Look in news articles and product pages of online storefronts really complements the reading and shopping experience by providing a new way to visualize the content and develop a better understanding of what you’re actually seeing, and we’ll be writing HTML to allow previewing AR content on the web.

So, let’s get started with applications and how the Quick Look API is used to provide an AR experience.

So, if you’re not already familiar, let me quickly explain Quick Look.

Quick Look is all about previewing documents like Keynotes, PDFs, images, and now 3D model files like usdz.

Quick Look itself is an iOS framework that not only provides support to preview documents, but also will be really to create and provide custom previews for your custom file types.

It comes with everything you need to preview documents.

You can control the transition and the different presentation modes for Quick Look, like being embedded inline or incorporating a push navigation flow and the standard modal presentation.

And what’s really amazing is that security is handled for you where the contents of the usdz file is safely read to present a preview for you.

Now, there’s a dedicated endup session about Quick Look and previewing all file types, including the new usdz file type that I encourage you to check out.

So, let’s take a quick look at the preview flow.

Here’s the apps ViewController on the left.

It has a grid of thumbnails for various 3D models.

When someone taps on one of these thumbnails, I want to show a Quick Look preview of the model for that thumbnail.

The first thing I want to do is create a QLPreviewController.

I have instance, I have ViewController that allows you to preview documents.

And I set myself as a dataSource and a delegate.

So it can ask me questions about what should be previewed.

With that configured, I present that QLPreviewController.

Now, it doesn’t appear immediately, however.

It first asks me some questions.

It starts by asking how many items do you have for me to preview?

And for usdz files, the answer is one.

Now, it asks me for a URL for the first item to be previewed, and I respond with the file URL for my app bundle for the model that was tapped.

Finally, it asks me for the view to use as a source view for the transition animation for when the preview is presented.

And here, I want a seamless animation from the view that was tapped to the view that was going to be presented.

And so, I return View #3.

With all that configured, we’re ready to present our preview controller, and so it’s going to animate and scale up from the view that was tapped.

And this is what it looks like in code.

In order to preview and present documents, we need to instantiate a QLPreviewController.

It’s part of the Quick Look framework.

It has a dataSource and a delegate, which I’ll get into in a bit.

And then we then present the previewController modally.

Pretty simple, right.

So, let’s take a look at the protocol for a dataSource.

So, as I just mentioned, we need to conform to the QLPreviewControllerDataSource protocol.

This protocol has two required functions; one to provide data to the Quick Look Preview Controller.

The first being, the number of preview items in controller.

And here, you return the number of items that your Preview Controller should preview.

For AR Quick Look, it only allows previewing a single object in one dedicated session, and so we returned the one item.

The second function we’re implementing is PreviewController previewItemAtindex.

And here, you returned the one item that we we’re requesting to be previewed.

This is the URL where to find a 3D object on disk.

Now, this should look very familiar to a lot of you, and this is because we’re using a very similar data source and delicate pattern from UIKit like Table Views.

So, moving to the protocol for that delegate, there are a few ways to present the preview.

AR Quick Look is intended to be presented full screen.

To share the same consistent experience with AR Quick Look, I recommend you configure your Quick Look PreviewController’s appearance to have the same zoom transition animation.

So when the file’s up, you can see the nice transition from the thumbnail view to the viewer and back down as it goes down for dismissal.

In code, the easiest and recommended way to achieve this effect is to implement PreviewController, controller transitionViewfor item and return a UIView.

So, Quick Look does all of the heavy work here for you and uses the UIView to infer the rectangular frame that the animation should start from when presenting and ending when dismissing.

If possible, I suggest the final or UIView to have a square frame, to have the best seamless transition.

AR Quick Look nicely handles the transition from this view to the full screen viewer on presentation and dismissal, creating this effect where the model just magically appears.

And for an overall better experience, I recommend using a UI button as the view and setting the thumbnail as the button’s image.

That way, you get visual feedback whenever the view is tapped with the button highlight, and this is extremely important so that the user knows that some action is about to happen.

And that is how to integrate AR Quick Look into your iOS application.

It’s never been easier to introduce AR preview and capabilities in your apps today.

Now, let’s talk about integrating AR content on the web.

We really want AR content to be accessible online.

So, starting in iOS 12, Safari has built-in support for previewing usdz files in AR.

By hosting usdz files on your website, you got a way of delivering previews of 3D objects.

So, not only can you read about the object and see it through the image, you can now experience what the model looks like in your world.

So, here, I have a modern version of my online garden store with various thumbnails of gardening tools I have listed for sale.

For better integration of AR on my web content, I’ll use the new HTML markup that I’m about to show you shortly.

This opens up and allows customers to preview my gardening tools before they actually purchase it.

So, by following this markup, you had a badge that I mentioned earlier, rendered on the image in the top right-hand corner, and this is useful and there to tell you that our system viewer is behind this so you can preview my watering can in AR.

So, you can provide your own cover artwork associated to the 3D object, and this can be a portrait shot or nice hero image like I’ve chosen for my lovely watering can.

And with this flow, we support drag and drop and, of course, long-press Safari actions from Safari like Add to Reading List or Copy and Share.

And the best part is you get a more immediate and optimized workflow into AR Quick Look.

So, you’re not following any links, and you get a response without updating your web URL.

The HTML Markup is very straightforward and simple, and this is it.

So, by adding this attribute, this will turn your webpage into an AR experience.

The requirement is an a elements with rel=ar, and this tells WebKit that this is AR content.

You then provide the link to the location of the usdz file.

Lastly, we have a single child that is the image element, and this is the cover artwork that I mentioned earlier about that you provide as your thumbnail for the 3D content.

The badge will appear above your thumbnail image inviting you to preview and it screens the object in AR.

And this can also be done with the picture element.

Similar to the image element, the rules here are single picture child of the a rel=ar tag.

The picture itself can have all the children it needs, from being very simple to having very complicated roles about which child source image it picks to render based off of device, size, and other platform logic.

usdz content must be served with the correct media type, and so make sure that the MIME type is set for these files.

The registration of an official type is in progress.

So, in the meantime, WebKit is affecting both of these media types if you’re using Apache.

And that’s how to integrate AR Quick Look.

The goal was to make it really easy to put AR everywhere, on not just websites in Safari but also in apps in iOS.

Now, you might be wondering where can I get usdz content.

I’ll invite my colleague Dave who will talk more about how to create content for AR Quick Look.

[ Applause ]

Thanks David.

Well, let’s start by taking a look at what it is that we want to achieve with our 3D models.

Here, I have a model of a teapot that I’ve loaded in AR Quick Look, and we can see that it sits perfectly on the ground, as indicated by its shadow.

It spins around its natural pivot point, the center of the bowl, and if I place it in the world, we can really believe that it’s there on the surface in front of me.

We even see the environment reflected in its shiny metal handle.

So, how do we achieve this effect with our 3D models?

Well, when we’re creating our models, there are six things we need to get right, and we’ll see how to do each of these in the remainder of the session.

We need to set the model’s placement, we need to set its physical size to be correct in AR, we need to create any animation we want for our model, add its contact shadow, modify its appearance, and then add any transparency that our model needs.

Once these six things are done, we can optimize and export our models for use in AR Quick Look.

We’ll start with placement, and there are three things we need to get right in our 3D software to make our model appear correctly in the world.

The first is that the object should face toward the camera, toward positive z.

Secondly, the base of the object should sit on the ground plane, the plane where y = 0.

And finally, we should put its natural pivot point at the origin.

So, let’s see how that looks for a sample model.

Here, I’m in my 3D modeling software.

I have the origin at the center.

I have positive x heading off to the side, positive y heading up from the origin, and positive z toward the camera.

Now, if I import a model, if I import my teapot without having set it up, we can immediately see there’s a problem here.

The model is below the ground plane.

If we look at it from the side, we can see this model has been created around the origin.

So, let’s move it up such that its base is set on our plane where y = 0.

And now, it’s ready to position in the world on the ground.

If we look at it from the front, however, there are still some esthetic improvements we can make here.

This kind of looks like a teapot, but it could look more like a teapot.

This is what we think of when we think of a teapot, with a spout to the side.

When you’re thinking how to position your objects for this optimal recognition, think how you would place that object if you put it on a shelf or in a display cabinet.

If in doubt, ask a young child to draw the object for you.

Chances are the profile they choose is the one you should use.

Now, this profile really matters.

Not only is it what people will see when your object first loads in our Quick Look, but it’s also the profile that’s used for the object’s thumbnail, and these two deliberately match when the thumbnail is in files or in messages.

So, we get that seamless transition between the two.

So, choosing a good profile is really important.

We’ll head back into our 3D software and rotate our model so the spout is to the side.

As one final check before we export it, let’s take a top-down view and make sure that that bowl is centered on the origin, so it will spin around the point we want it to.

The second thing we want to set up for our model is its physical size.

Here, I have a gramophone record player, and size matters here.

This record player is designed to play 12-inch records, so we need to make sure it’s the right size in the world.

If I export this model and load it in object mode in AR Quick Look, it looks great because we always scale models to fit in this mode.

But if I place it in the world, we can see there’s a problem when I put it next to a 12-inch record.

It’s far too small.

Right now, it’s set up to play 6-inch records, and they don’t exist.

So, let’s head back into my 3D software and create a reference, 12-inch cylinder.

Now, I know this is exactly 12 inches in diameter because I know the units that this project is working in.

I can use this reference cylinder to resize my turntable and make sure that the platter is exactly the right size to play a 12-inch record.

Having done so, if I re-export my model, it looks exactly the same in object mode because it’s still scaled to fit.

But when I place it in the world, we can now see it’s the right size to play this 12-inch record.

We could imagine picking it up and placing it on that turntable.

Now, not all objects have a natural size.

This little guy is Chatterbox, and he doesn’t exist in the real world.

He only exists in our imaginations.

But he’s pretty cute, and he has a fun animation, and he’s good to take a photo with.

So, we’ve created him at desktop size, which makes him easy to place in the world in common environments, then people can resize him, shrink him down.

They can move him around or scale him up and make him the size they want him to be for a photo.

But by making him desktop size by default, he’s always easy to place.

We saw here that Chatterbox has animation, and this is something we recommend you use where it helps to bring that object to life with an idle animation that makes it feel more like it’s in the world.

Note that animations always loop, and the animations you could create can use skeletal and/or transform animation to bring them to life.

Here’s another example.

This is a cartoon style chicken character without animation, and if I place him in the world, he looks pretty cool, but he’s kind of static.

He doesn’t really feel like he’s alive there in the room with me.

So, there’s more we can do here.

So, if we add a little bit of skeletal animation to our character to make him bob his head, now when we place him in the world, he feels much more realistic, much more like he’s there on the table in front of me.

Some tips as you add animation to your models.

Firstly, choose animations that enhance the AR immersiveness, that make you feel more like that object is there in the room.

Now, because people can manipulate your objects, they can retape them, scale them, and move them around, it’s important not to move them away from the origin.

If I create a car that zooms around in a meter-wide circle, when someone tries to put their finger on that car to pick it up, it’s already moved away and zooms somewhere else.

So, that gesture is very confusing to somebody trying to manipulate that object.

So, try and keep those objects centered at the origin.

For the same reasons, try and keep a consistent bounding box throughout the animation as well.

Even if the object is at one location, it’s much easier for me to manipulate it if it stays in the same place under my fingers throughout the entire gesture.

As a result, prefer animations that make sense at a static location.

If you have a T-rex character, rather than giving it a walk cycle that would make it look like it was running on the spot, make it roar or stamp its feet or something else that makes sense at a static location.

Now, there is an alternative.

You can also consider creating animations as self-contained scenes to help with these gesture manipulations.

And let’s see an example of how that can work.

Here, I have a swimming koi fish, and it makes sense for this fish to have some translation.

So, we created a scene with an aquarium base, and this means when we place this in the world, if we rotate it, or if we move it, the scene we’re manipulating is the scene, not the fish, and so the gesture makes a lot more sense.

It also means by having a base to our scene that we have a consistent bounding box throughout the entire animation.

Another thing that really brings your models to life in the world is that contact shadow.

Now, note here that AR Quick Look provides the contact shadow for you.

This means that it can turn the shadow on and off as you transition between modes.

It also means it can apply ambient lighting conditions to the shadow as the lighting changes around you.

Because of this, don’t bake a contact shadow into the models you provide.

If you do, you’ll end up with two shadows.

Note also that the first frame of any animation is used to create the shadow for a model, and sometimes this means it’s important to choose the right first frame.

We’ll see an example.

Here, I have a model of a Newton’s cradle, a desktop type, and it has a swinging animation, a transform animation to bring it to life.

Now, if we freeze this with its first frame, here the first frame has one of the ball bearings out to the side, we can see there’s a problem.

The shadow we’ve generated from this first frame also has that ball bearing out to the side.

And so, if we animate to the other extreme of this animation, at this point, the shadow doesn’t really make sense.

Now, because animations loop, we could choose a different starting point, a more neutral one where the ball’s in the center and still have the same animation, the same overall effect.

But now, by choosing a better first frame, our shadow makes sense throughout the entire animation.

One of the biggest things we can modify about our model to make it feel real in the world is the model’s appearance.

Now, here, AR Quick Look uses a Physically Based Rendering, or PBR shader, and this gives us six things that we can modify about our model’s appearance to make it feel more real in the world.

We do this by providing textures, images, for each of these six things if we want to.

The first is the model’s albedo.

This is the base color of the model, its underlying color.

The second is indicating which parts of the model are metallic.

This changes whether they are a conductor or an insulator, which changes how they interact with the physics of light in the real world.

We could also specify which parts of our model are rough or shiny by providing a roughness texture.

We can add in a normal map.

This creates the illusion of depth and variances within the surfaces, the model’s surface, without changing the underlying mesh.

We can add an ambient occlusion texture to specify where the model casts shadows on itself in the crevices and nooks and crannies of the model.

And if our model emits lights, we can provide an emissive texture for things like a TV screen or a computer monitor.

Now, if you’re just getting started with PBR, we have a session on Thursday about creating PBR-based models for any AR experience, and I highly recommend you check it out.

For now, we’ll take a look at how these six textures affect AR Quick Look particularly.

So, I’ll start with a model of a TV.

Right now, it has no textures.

It’s pretty bland.

We’ll bring in an albedo texture to give it some base color, and it’s much more recognizable as a TV now, but it’s still very flat.

It doesn’t feel realistic.

The next thing we’ll do is to introduce a metallic texture to make the trim around the TV screen and the antenna on top to indicate that they’re made from metal.

Note here that our metallic texture is pretty much black and white.

Things either are a metal, or they aren’t.

But the metal still doesn’t look as realistic as we want it to do, and that’s because the metal, like all of the model, is entirely rough.

There’s no shine, there’s no smooth surfaces on this TV right now.

To fix that, we’ll bring in a roughness texture, and at this point the model really comes to life.

The metal around the TV screen and the aerial now looks like metal.

It looks like chrome, and the wood on the top of our TV actually looks like it has a varnish or lacquer effect.

We can believe it much more realistically as a TV.

And roughness is one of the biggest points of creative control you have over the look of your model and how it feels in the real world.

And you’ll see, we have quite a lot of variance in our roughness texture, even within the same parts of the model, to give more realism and to bring it more to life.

Next, we’ll add a normal map to add the illusion of variance, particularly in the woodgrain on top of the TV here.

The underlying mesh hasn’t changed, but it feels more 3D.

Also, on the speaker grill.

Because we have some crevices and some hidden areas on the front of the TV that would cast shadows on themselves, we’ll add in an ambient occlusion texture.

The effect is pretty subtle, but it does add to the depth of the model and make it feel more real when it’s in the world.

And because this is a TV with a screen, we’ll add an emissive texture as well to indicate that the screen emits light into the world.

The effect of this is best seen when pushing the object into a room.

Here, we can see that the main body of the TV reflects the ambient lighting conditions of the room it’s in, but the screen is brighter.

It’s actually emitting light.

Emissive texture is a fairly special case but not something you’d use every day.

But if the object you’re modeling would glow if you turned the lights off, it might be a good candidate for an emissive texture.

We can also make parts of our models transparent.

This is useful for making objects made out of partly glass, for example.

If you do this, be sure to use a separate material for the transparent and non-transparent parts of your model to ensure they render correctly.

You do this by providing an albedo texture that has the transparency in its alpha channel.

So, a transparent PNG file, for example.

Note that transparency is really intended for see-through parts of the model, not for creating cutouts like a leaf edge or a butterfly wing.

We’ll see this in action for our TV model from before.

In addition to the mesh we already have for the base of the model, I’m going to add a second mesh for a curved screen, a screen cover that’s made of glass, with a slight amount of dirt on the screen.

Here’s how it looks in AR Quick Look.

We can see that the light reflects on the screen and shines off the curved surface, and if we turn it to the side, we can see the color of the screen there as well where the glass as some grime to it.

We achieve this by using this kind of texture for our screen.

It’s pretty simple.

It’s just adding that small amount of dirt.

But crucially, it has the transparency in its alpha channel to indicate that the screen is see-through, and we can see the main screen behind it.

When you are setting up the textures for these individual properties.

These are the formats to use.

Albedo should RGB, or RBGA, if you’re also providing transparency.

Normal and emissive should also be RGB.

And metallic roughness and ambient occlusion should be Greyscale.

You can use any image format supported on iOS, and note that these textures should be square.

They should be powers of 2 squared.

So, 2k, 1k, 512 pixels, so on.

David mentioned earlier that AR Quick Look is available on all devices that support iOS 12, and this is actually quite a range of devices with different capabilities.

Now, because AR Quick Look is a system-wide extension, this means it has to share the available system memory with the applications that launch it.

And so, there’s only so much memory available to display and load these models.

To help with this, we recommend that you create and optimize and test your models for high memory devices like the iPhone 7 Plus, 8 Plus, and X, and the iPad Pro 12.9 inch.

If you do this, AR Quick Look will dynamically downsample the textures in your model for other devices of different capabilities, if needed.

This means you can make the devices, your models look great on those devices but still know they’ll work everywhere.

Know that the meshes and the animations won’t be modified.

It’s only the textures that will be downsampled.

You’re probably thinking at this point, okay, so how big can my model be?

How big and complex can my textures be?

And the answer is there is no one answer.

Many factors affect a model’s memory usage, its mesh, and its animation complexity, and its textures size and count.

But as a guide, for a model that has a single PBR texture, using those six textures we saw earlier on, if you aim for about 100k polygons, one set of 2k by 2k PBR textures and about ten seconds of animation, this is typically good for those high-memory devices we mentioned earlier on.

If in doubt, always test your model on a device to make sure that it renders and loads as you would expect.

As you optimize your models for use in AR Quick Look, a few things to keep in mind.

Do be sure to freeze any transforms in your models and merge any adjacent vertices to tidy them up ready for export.

And if you can, try and use a single material and texture set for the entire model.

This enables you to optimize how you pack the different parts of that model into those square textures that you’re using.

And if you don’t need a texture, don’t include it.

If none of the parts of your model are made from metal, there’s no need to provide a metallic texture.

The default is non-metallic for any model that’s loaded.

When you’re choosing where to spend your texture budget, prioritize the areas that give most realism to that model.

For some models, that might be a high-resolution albedo texture.

For others, it might be better spent on roughness or the normal map.

And remember the pixels actually have a physical size in AR.

If you’re printing a model of a thimble, and it’s one inch across, there’s probably no need to use a 2k by 2k texture for it.

Those pixels will be tiny in the real world.

And although we want models to look great, we also need to balance texture size and quality against file size.

If a model is being downloaded from a website, we want it to be a good size for downloading and also for sharing via messages and mail and other means.

So, with all of that in mind, how do we make these things?

Well for this, we have a usdz Converter.

This is a command line tool that will convert existing 3D models to usdz format.

It ships inside Xcode 10, and in addition to creating usdz files, you could also use it to map PBR textures to the meshes and submeshes inside those existing models.

It’s X3 input format, OBJ files, Alembic files, and existing USD files, either USDA or USDC, the ASCII and binary versions of USD.

Before we create one, let’s take a look at what it is we’re going to make.

Let’s look inside a usdz file.

In essence, these are uncompressed zip archives.

The first file is always a usdc file.

This contains the model’s mesh, its animation, if it has some, and any material definitions it needs.

And then, the remainder of the files in the archive are any textures, any images, like the ones we saw earlier on.

If you’re thinking of creating your own tooling to build usdz files, the great news is that this is that this is an open format and Pixar have published the direct specification for usdz on the site.

To create one with a usdz converter, first note that we call the tool with xcrun because it shifts inside Xcode.

We pass in the name of the model we want to convert, an OBJ file in this case, and the file name of the usdz we want to have the other end.

To map PBR textures to the meshes they’re in, we use the -g option, followed by the name of one of the groups, a mesh or a submesh that we want to map them to, and then we can provide any number of these textures for that particular group.

To map multiple textures to multiple groups, we can use the -g option multiple times, as many as we need, to map all the textures in our model.

And if you’re not sure what the groups in your model are called, you can pass in the -v option for verbose output.

This will print the names of the groups that we find in the model plus useful other information about the conversion process.

In addition to making your own models, we’ve also put together a gallery of example models that show good practices for PBR.

If you visit this gallery in an iOS 12 device, you can tap any of these models, open them in AR Quick Look, and place them in the world.

This gallery also showcases the Safari integration that David showed you earlier on.

So, if you have iOS 12 on a device, visit  and give them a try.

So, in summary, AR Quick Look is a system-wide way to view AR content in the real world.

It can be integrated into your own apps and websites.

It uses the usdz file format for 3D model distribution and sharing.

It supports PBR, animation, and transparency, and you can use the usdz converter tool that ships in Xcode 10 to convert your existing models to usdz.

For more information on the subjects in this talk, check out the session’s website.

And we also have ARKit labs throughout the week.

So, please do bring any questions you have about AR Quick Look, and we’d be really happy to help you answer them.

Thank you.

[ Applause ]

Apple, Inc. AAPL
1 Infinite Loop Cupertino CA 95014 US