Painting the Future

Session 109 WWDC 2013

Be among the first to see the unveiling of MARI on OS X, the supercharged 3D painting package coming to the Mac later this year, running on the next generation Mac Pro. Jack Greasley, The Foundry’s MARI Product Manager, will chart MARI's rapid rise from in-house studio tool to the film industry’s painting tool of choice. He will detail MARI's evolution, from its original conception during Weta Digital's visual effects work on Avatar to becoming a critical element in making countless blockbusters a reality. Then watch as Pixar character shading artist, Jonathan Hoffman, makes full use of the power of the next generation Mac Pro to demonstrate how MARI was used to help bring Monsters University to life.

[ Music ]

[ Applause ]

Hi, I'm Jack Greasley.

I work for The Foundry.

And we make software to do that sort of stuff.

It's a pretty cool job.

The Foundry was started about 15 years ago by two guys, Bruno and Simon, working out off a basement in Soho, London.

And they were making image processing tools for the local film and visual effects industry.

Since then, we've grown a bit.

And we've really made a name for ourselves by taking tools and technologies out of visual effects companies, out of animation companies, and commercializing them and making them available to everybody, so with tools like NUKE from Digital Domain and KATANA from Sony Pictures Imageworks.

We've taken tools and technologies which were really only available to the elite few and made them available to everybody.

One of the tools that we've done that way is my particular baby which is MARI.

And MARI is a GPU-accelerated 3D digital painting system.

And it was originally developed at Weta Digital in New Zealand which is Peter Jackson's visual effects company.

I'm the product manager for MARI but I'm also the original author.

And what I'd like to do is just take you through just exactly what a 3D digital painting system actually is.

When you're making a movie and you're making a character like Gollum here, he starts off as a model, as a mesh.

And that model gives you the overall shape.

It gives you the muscle and the size of the object.

But it lacks the fine-grain detail on the surface that really make an object compelling.

And it's the job of a texture artist to go in there literally with a brush and paint in individual freckles, individual veins, and put the detail in there that really makes it, you know, go from a gray blank model like this to something looking pretty good.

And if any of you guys have ever painted any figurines or any models, you know just how long it can take to get that looking good and how much detail you have to put in there.

So imagine that your model is 30-feet wide and it's going to be projected on a screen this size.

And this is the problem that Weta Digital were facing.

How do you put enough detail into your models where the director can choose to zoom into any part of it and have it full screen?

In a game, you might put, you know, one or two textures on a character, and they might be 1,000 pixel square or 2,000 pixel square.

To get the level of detail that they needed, Weta Digital would often put a patchwork of 500 textures over the surface of a single character.

And each of those textures would be big.

It would be at least 2,000 pixel square and sometimes much, much bigger.

And it doesn't stop there.

Those 500 textures only tell you one thing about the surface.

They only tell you what color it is, but you also need to tell the system how bumpy it is, how shiny it is, how scratchy, how dirty, how dusty.

And each one of those pieces of information gives you another 500 textures.

So very quickly you can be up at 10,000 huge textures for a single character, and that can be 20, 30, 40 gigabytes of information.

And this was slowing people down.

This was slowing the artists down.

The one thing that artists love to do is paint.

Things that they hate doing are copying data around, renaming files and managing complexity.

And so, this is really where MARI comes in.

I was hired by Weta Digital in 2006 to work with their texture artists to really come up with tools and technologies to get them back to doing what they love which is painting.

So I got on a plane from London down to Wellington, New Zealand and worked with the texture artists while they were working on this little movie for James Cameron called Avatar.

So my team and I were sitting next to a room of fifty kind of busy, kind of caffeinated texture artists for about four years.

And it got to the point where we were releasing MARI three times a day.

We'd do a version in the morning and then fix it at lunchtime and then another one in the afternoon.

And if releasing three times a day wasn't, you know, enough work, my brand new girlfriend at that time was a texture artist using MARI.

It took me a little while to work out how to prioritize her bugs properly.

[ Laughter ]

But I'm glad to say it worked out and we're still together.

[ Applause ]

Thank you.

[ Applause ]

So MARI was incredibly successful at Weta.

Every tool so every model, every character, every plant, everything that Weta Digital painted for Avatar was done in MARI, and that was several thousand assets.

Avatar did OK as well.

You know, I think three billion dollars at the Box Office.

And MARI continues to be used at Weta Digital until today.

Every movie that they've released since 2008 has used MARI as its as their main texture tool.

So this includes things like The Avengers, The Adventures of Tintin, Rise of the Planet of the Apes, and most recently, this guy, again, for The Hobbit.

In 2010, Weta Digital and The Foundry came to an agreement to commercialize MARI.

So The Foundry licensed MARI from Weta.

But what that really meant is that they sold it and they sold me along with it.

So I got on a plane from Wellington, New Zealand back to London which is 32 hours door-to-door.

And this time, I was coming back with a USB stick for the source code and an invoice for The Foundry to sign.

My role at The Foundry was simple.

We wanted to make MARI the industry standard for visual effects, 3D digital painting.

Over the last three years, we've done that.

We've been incredibly successful to the point that nine out of ten of the movies nominated for visual effects Oscars this year used MARI as their main texture painting system.

The only reason that the tenth didn't is that they didn't buy it in time enough to use on the movie.

MARI is at use in use at a whole bunch of companies you will have heard of, so there's Industrial Light & Magic over in the Presidio, DreamWorks Animations, Sony Pictures Imageworks.

So really if you've seen a movie over the last couple of years in which there has been an alien, a spaceship, a monster, something blowing up, you've seen MARI at work.

One of the really great things though about taking an in-house tool something thats design for one specific task and making it available to everybody is that you start seeing uses that you never expected for it.

We've seen it cropping up in games, in architecture, in digital design, and even in amazing artwork like this which is from an album cover.

One of our most technically innovative artists is a good friend of mine called Scott Metzger who works out of LA.

And he's coming up with new novel techniques for capturing digital environments and making photorealistic rendering incredibly easy to do.

So what this really means is Scott went out one weekend and rented an industrial laser scanner and started scanning things, and he scanned his apartment.

And he scanned it down to millimeter accuracy.

And ran around his apartment with a camera taking hundreds and hundreds of reference photographs making this enormous dataset which he pulled into MARI and used MARI to paint those photos down onto to this enormous scan that he made.

And this created I think it was a 20 gigabyte dataset that he was able to fly around inside of MARI, edit in real time.

But it also meant that he could make amazing photorealistic renderings like this.

You can actually zoom down into the work surfaces and you can see fingerprints, you can see fine-grained detail in the paint work.

And this is an incredibly useful thing for the visual effects industry and the film industry because when you're making a film, what you normally do is you build a set, you put some cameras on the set, you get the actors on the set, you film it, and then the set gets torn down and thrown away.

But what often happens is two months later, the director has this flash of inspiration that the thing that's really going to give him the Oscar this time is a zoom through this set which is now sitting somewhere in a dumpster in Hollywood.

And these are called pickup shots, and they're incredibly expensive to do.

You need to go and find the bits of set that still exists, you need to rebuild the set, you need to reconstruct the lighting, you need to match everything perfectly.

But with this technique, you can actually capture the set and its entirety, all of the information, all the lighting, when you're shooting.

And when the director has this amazing flash of inspiration, it's then incredibly cheap to do pickup shots rather than incredibly expensive.

All of the examples that I've shown up until now have been running on Windows and Linux.

But MARI was originally developed for the first couple of years of its life on both Macintosh and Linux.

Unfortunately, as I mentioned, we were releasing three times a day and maintaining six builds a day in full production was really more than we could deal with.

So we concentrated on Linux at the time, but I'm incredibly pleased that MARI is actually coming back to the Mac later this summer.

Thank you.

[ Applause ]

We've been developing on and targeting common generation Mac hardware.

And partially this is really selfish.

I want to demo on my brand new shiny retina Macbook Pro and not on a big ugly laptop I've got at the moment.

The porting process for us has been pretty straightforward.

It's really just some housekeeping.

We need to take MARI's OpenGL usage because it really is fully OpenGL based and update it to 3.2 core profile.

This is something we've been meaning to do for a while anyway.

It really gives us a lot of benefits in terms of performance, speed, simplicity of code and stability.

And it's taken us about six weeks to go from absolutely nothing, MARI didn't compile on the Mac, didn't run on the Mac, to being able to demo here today which you guys will be seeing in a few minutes.

The screenshots you can see on the screen were taken about a week apart by my developers as they were bringing MARI up.

The first image is the first time MARI ran on the Mac in five years.

The second one is a week later.

We have rendering up and working.

And the third one is a week after that where we're dealing with big datasets we can paint and we can animate.

So it really has been a pretty seamless process.

However, we have been incredibly lucky during this whole process have been given access to the new Mac Pro.

We originally designed MARI with multiple GPUs in mind.

For me it makes so much sense that you're going to want to be processing huge amounts of data on one device while you're displaying it fluidly at 60 frames per second on a second.

However, we've not really ever seen a viable platform for us to do this before.

We don't see multiple GPUs installed in visual effects artist machines.

We see it in the gaming world, you know, people who put two GPUs in their gaming rig so they can play Gears of War at 1080p 60 hertz.

But that power has never been available to artists.

The multiple GPUs in the new Mac Pro especially combined with the updated OpenGL and OpenCL support makes it just a dream platform for us to develop on it.

It's been really a fantastic experience.

It's also going to make my life a lot easier.

A large part of my job is actually working with our most demanding customers to help them design and build systems that will meet their, you know, incredible requirements.

I often get emails in the morning along the lines of, "So Jack, we tried to load 250 gigs of data into MARI and it's running a little bit slowly.

What can we do?"

So this machine is going to make my life really easy.

I can say, "Go buy the new Mac Pro."

[ Laughter ]

I can genuinely say it is the best out-of-the-box experience I've ever seen with MARI.

In terms of our most demanding customers, I think Pixar Animation Studios are right up there with them.

I love working with them as a company because they represent this amazing fusion of being deeply artistic but at the same time being deeply technical.

Pixar were a very early adopter of MARI.

And we've been working with them for the last, about three years, to integrate MARI into their pipeline to put together new techniques for creating incredibly realistic characters for their upcoming movies.

We're very lucky to have Jonathan Hoffman here today who's a shading technical artist from Pixar.

Using MARI on the new Mac Pro, he's going to be giving us a sneak peak behind the scenes into the process of how Pixar design and build the characters that are going to their movies.

But before we do that, I think we should have a quick look at some of the amazing work that they've been up to for the last couple of years.

[ Applause ]

[ Music ]

Tonight we party like scarers!

I want to piece of that action.

My friends call me Sully.

Mike Wazowski.

I'm officially a college student.

Yeah!

We don't need to study scaring, you know, do it.

That is a good point.

What's so scary about a little librarian?

A bunch of guys went to the hospital last year.

You can totally die.

And it's worth it.

Let the animal out!

Is that legal?

I've never seen him this way in my life!

Trust me when I say you are not going to want to touch this bad boy.

I want to touch it.

Yeah, I want to touch him.

Hey, what are you doing up there?

I can't go back to jail.

I'm going to rule!

[ Applause ]

Obviously a lot of work goes into every one of our films.

It's truly a collaborative effort between hundreds of really talented people.

What I'm going to talk to you about today is really only one small part of the whole process.

You saw on the trailer, there are a lot of different characters in this upcoming film and each character presents its own unique set of challenges.

With Monsters University, obviously since this is a prequel, we're going back to the world of Monsters Inc.

which we haven't been to since 2001.

And recreating that world, you know, we wanted to stay true to the essence of the design while still pushing the limits of what we can do now that it's 2013.

Obviously the technology has come a long way since then.

Our two primary characters in this film are still Mike Wazowski and James Sullivan from Monsters Incorporated.

Now, since this is their college days, we needed to redesign the characters to look like teenagers, and that might be more difficult than you would think because how exactly does one make a walking eyeball look younger, right?

You don't normally think about this sort of thing.

But the people in the art team did an amazing job in figuring out and envisioning, you know, how you take these monsters and make them look younger.

With Mike Wazowski, this image by our production designer, Ricky, is this great progression of Mike all the way from toddler to adulthood.

And this is the final design for both of the main characters.

And there's a lot of real subtle things here.

I'm going to kind of show you some of the things that they did to make them look younger.

With Mike Wazowski, you know, we took lots of pictures of all of the people of the studio and looked at them when they were teenagers and then looked at their adult pictures.

And we learned a few things, you know, besides, you know, we all are, you know, little bit less hair, you know, some things like that.

But with teenagers, teenagers reach their adult height way before they really fill in.

And so with Mike, you'll notice he's the same height as he was before.

His limbs are the same length.

His hands and his feet are the same size.

But he's a lot skinnier, right?

He's lanky.

And there's a lot of other things too.

The colors are much more vibrant and saturated to give them that sort of youthful glow.

The horns are shorter and more blunted because, obviously as you get older, your horns get longer.

And with Mike, you know, you've got this great detail of that retainer just coming out of high school, you know.

I'm sure a lot of you had to deal with that.

And then with Sullivan, my favorite detail is he's got this great bedhead look.

But since it's all over his body, it's his bed-body look.

It looks like he just rolled out of bed and all of his hair is disheveled.

So a lot of thought and time and effort goes into really making these character designs perfect for these characters, to really stay true to that original design but really take them in a new direction for this film.

But for every main character, for every, you know, for these our core characters, we need a good foil, we need a good antagonist for the story.

And so, I'm going to introduce you to the character that I have the chance to work on for this film and I'm going to show you a never before seen clip from the movie to introduce that character.

Please, let me try the simulator.

I'll surprise you.

Surprise me?

I doubt that very much.

All right, so that is Dean Hardscrabble of the Monsters University School of Scaring.

Now, she is the scariest monster on campus.

And she is a scaring legend.

And with a character like this, where does it start?

How does a character go from start to finish?

Where does it begin?

Well it starts with a bunch of drawings.

The art team does hundreds of drawings for weeks to try and really figure out what a character is, to experiment with different ideas.

And with Hardscrabble, they had a very challenging goal.

They needed her to be very poised and elegant and have this sense of authority.

But they also needed her to be terrifying.

And they were trying a bunch of different ideas but it wasn't until they saw this video of a Amazonian centipede which these things are huge.

It's a giant Amazonian centipede.

They saw this video and it had this amazing grace to it despite the fact that the first things that you see is "Wow, that is scary.

I'm going to" you know, if you see that you want to run away.

But you see it and it had this movement, this deadly grace to it, and that is exactly what they wanted for Dean Hardscrabble.

So the director, Dan Scanlon, drew the centipede tail onto the back of Hardscrabble and they realized that is the design.

That was the central part of her visual look.

So this is the final design by Jason Deamer and it really the central figure of the design is that centipede, but then he's incorporated all these other really scary animals into that design.

She's got these bat wings, the sort of dragon/lizard head.

And this, you know, this was the character that they've been searching for, this elegance and grace, authority along with this idea of being very, very scary and terrifying.

The first film, we had a you know, we had this cast of very candy colored characters, these very bright, you know, and saturated characters.

And with the next film, we wanted to stay true to that.

But with Hardscrabble, we needed to make sure that she still fit in the world of all these other characters and yet was still scarier.

In Monsters Incorporated, if you remember the character of Waternoose, he was a little bit different.

And so we were trying to go for something similar to him where he still fit in the world but was different than all the other characters.

She again needed to be scarier and yet still feel like she came from the same planet.

So once the design is settled, it really gets passed in to kind of two different areas.

The first is the modeling department where, in this case, Michael Honsel took this design and began to sculpt it on the computer, very much how he would sculpt an object in real life in real clay.

But he's sculpting it digitally on the computer.

But it also gets passed to the art department again to start to develop the textural look of the character.

And so this is a painting by Shelly Wan and it was given to me to show me, OK, this is the way that the surface should look when it's reacting to light.

This is how the character needs to look in color and with specularity.

And I'll explain some of the details of that later.

And so we've got this beautiful image.

And along with this image comes a bunch of reference photography of the different creatures that make up what Hardscrabble is.

And so, we we start with her makeup, you know, and all these little great details about what her face makeup is going to look like.

But then we moved into her different aspects like the horns, how those are going to be shaped, what the colors are going to be, how shiny they are.

And we go through all of these details of these different aspects.

But one of the things about her shells and her legs, with a centipede that's only this big, I mean only this big, it has this sort of translucency to it but that would give her a sense of being very small.

So instead of going with the insect variety of shells, we went with more of a crustacean look because that would indicate a larger scale creature.

So we have these great images of her legs, of the shells, and also these great batwings and the translucency that we could get with the batwings.

All these things were thought about.

But with this photography with these images, I wasn't supposed to take them and make a photorealistic character.

The goal at Pixar is never to go with just loads and loads of detail and just make it look like a creature because we've hit this area called "Uncanny Valley".

It's something we try to avoid very much where you've added so much detail that it looks too realistic.

And at Pixar, we want to find the essence of these designs.

We want to boil it down to its simplest idea and then really pull that idea out.

And so with all these photography, I was supposed to take inspiration but not copy exactly.

I was supposed to find that simple idea.

So, the next thing that we're going to do is actually demonstrate the process of how I began to paint this character on the computer.

Now, bear with me because this is, you know, the new Mac Pro, it's unreleased.

We're running on an unreleased OS and the software, MARI, has only been on the Mac for about six weeks.

So, what could possibly go wrong, right?

So it'll be exciting for me, hopefully, exciting for you as well to do this.

So, all right.

So this is the new Mac Pro and this is MARI running on the new Mac Pro.

And this is how I get the model from the modeling department.

It's gray.

It doesn't have any detail yet because it's my job to add the detail to the character.

So I'm going to navigate around and you can see, you know, this is a 3D model brought in to the computer.

And the first thing that I do when I'm creating a character, when I'm starting to paint in the detail, is I start with the big details and then I move into the smaller details.

So for her, I'm going to start with the color and I'm going to start with her base color, basically her flat base color.

So she's more of this color than anything else.

So this is where I'm going to start.

And the first big read that I got from that shader packet that I showed you earlier is that her horns are sort of different material than her face.

It's a darker shell.

And so, I'm going to paint in a little bit of that variation first.

And I'm going to be a little bit rough and sloppy here because I'm compressing a couple I would at least want to work on this character for a few weeks and I'm going to compress that into about 15 minutes.

So I apologize if I'm a little bit sloppy with this.

So I'm grabbing a brush, and this is very similar to a program like Photoshop where you have a bunch of different brushes that can do different things.

And I'm just going to start painting right onto this model if I can.

Let's see.

Again, live demo, I apologize.

All right, here we go.

So I'm painting straight onto this model and you can see that it's, you know, accepting this paint and I'm getting a nice gradient towards her horns.

I'm just trying to lay in, you know, a nice amount of detail here.

I'm going to move the camera and that paint is going to be dropped onto the model.

So you can see it's actually been projected through the model here.

I'm just going to add a little bit more from this angle.

Then I'm going to go in and do a little bit more detail on one of these horns.

Now, again, I'm trying to lay in some of the bigger base details first and then I'll start moving into the smaller details.

You never want to start with small details.

And any artist will tell you this, "If you start painting the eyelashes before you've really, you know, figured out the shape of the head, you're doing something wrong."

So I'm starting with the big broad strokes first.

And this horn, if you remember that image from the lizard, it had some really nice striations that kind of ran along the horn.

And so, I'm going to start painting some of those in.

But I don't want to go I don't want to go too detailed.

I'm trying to keep this these shapes graphic and really strong here, get that simple idea without going into, you know, all those tiny little nitty-gritties.

So I'm just rotating this around.

Now, you'll notice that this feels very liquid.

It feels very smooth, right?

As I'm tumbling, it's so responsive.

And this really feels just like I've got an object, a model in my hand and I'm just turning it and painting it.

And the way the reason that that's possible is two things.

First, MARI is very, very slick at allowing me to see all these detail.

But secondly, it's really so responsive because the hardware is supporting it is so well.

It's just you know, I forget that I'm painting on a computer and it feels like I'm painting in real life which is exactly what I want as an artist.

So I painted this first horn.

I'm relatively satisfied with it.

As with everything, as an artist, I'd want to sit here and noodle it for an hour.

But I'm going to skip ahead, not bore you with painting the rest of these horns.

So I'll just this is another layer here that shows you, you know, the same treatment to all these other horns.

And now as I zoom back, you get that first pass at her color.

Like if you were to see her from this far away, she looks pretty good.

This is the biggest read of the character.

But her skin is way too simple, right?

Every all skin has variety to it.

It has mottling and freckles and you know, even the most perfect person has a large amount of variation in their skin.

If they look like this, they look like a mannequin.

So I'm going to start adding in some variety to her skin.

And I'm not you know, I don't want to go too far again because I don't want to make her look grotesque.

But I want her to look I want her to look old.

She needs to be sort of this figure of wisdom and authority.

And so, I need her to kind of feel like she's had some, you know, had some years, had some experience.

Not exactly sure how quickly lizard, centipede, bats age, but, you know, she's got to look a little bit lived-in.

So I'm just going to add in some basic variety to her skin here.

You know, MARI comes with these, you know, a lot of really great brushes that I can kind of start with and lay on some detail.

Add some freckles, a little bit of darkening.

And I can put some stuff on and then pull it off if I've gone too far.

That's nice.

Paint some freckles in the back of her neck here.

It's always interesting, the back of people's arms, the back of people's necks always get a little bit dark and I'm not sure if that's because of you know, that you just get more sun there, if the skin is a bit more aged in those areas, but it always seems that way.

It's one of those things you start staring at people differently when you've been painting, you know?

And on creatures and humans and characters for a long time, you just you're staring at someone's you know, the back of their leg for whatever reason and they're thinking, you know, "What's he doing?

Why is he looking at me?"

And it's just because there's this really cool vein doing this really cool thing.

And it's not, you know, I'm not trying to be insulting, it's just that I'm fascinated by it, right?

Because I'm going to use it to on the next day when I'm painting this character.

So, anyway.

I hope I haven't offended too many people.

Anyway, so I'm OK with where she is right now.

And I want to talk a little bit about how MARI is storing this data.

You know, Jack gave you a really great overview of what MARI does, and we're going to get in a little bit into the nuts and bolts as I'm demonstrating here without being too technical because I don't really understand all the nuts and bolts.

So you can see what I've been painting on the left side is mirrored over here on the right side.

And what we're looking at here is a series of slightly disturbing images where the character has been flattened out or kind of pelted into these tiles, right?

And what these tiles represent, each one represents a single color image, just like an image you would take on your phone.

And these images keep all of that color data that I've been painting.

And they can be scaled to whatever amount of detail you need.

So for a video game, they'd be down at like 256 pixel squared which is pretty small.

But for a feature film, you never know how much detail you're going to need.

So these are actually 8,000 pixels by 8,000 pixels.

That's 33 times bigger than an HD frame from a movie, all right?

But the reason you need that is because you never know if the character is going to walk right past the camera and we're going to see all of this detail on one of these legs, right?

And that actually happened in the film.

She walks right past the camera.

So we really need this level of fidelity to make sure that she holds up in any situation that she shows up in.

So, the next thing I'm going to do and the reason why I came into this view is to show you how I would paint her makeup.

And I painted on the flat version because it's a little bit easier than trying to work around the 3D shape.

But as I paint on the right, you'll see that it updates on the left.

So I'm going to grab this great eye shadow color.

Now, a couple of weeks ago, we had the wrap party where the movie is the movie is finished and we all go to see it for the first time and we dress up.

And my wife usually gets her makeup professionally done.

But I told her, "Honey, I painted makeup before, you know.

I do it for work.

It's not a big deal.

I can handle it."

So, I painted her makeup, I painted her eye shadow.

I thought it looked great, you know?

I thought I did this great job.

She looks in the mirror, was not pleased with my efforts.

So, we ended up being about half an hour late to the movie.

Well we made it before the movie.

We're half an hour late to the party.

So we made it for the important part, but I doubt she'll trust me to do her makeup again after that.

I guess, you know, painting on monsters doesn't necessarily transfer directly over to painting on humans.

Would never have guessed.

Anyway, so I'm painting this sweet purple eye shadow onto this character.

And you can see, again, as I'm painting on the right it's updating on the left.

And So what we've got here is basically two views of the same thing.

So now I've done that, I'm going to just go ahead and visit on the other side of I've got another layer that has her makeup on the other side.

I always forget character right or my right.

Guess I guessed right, great.

I'm not even going to bother with her lipstick.

Painting lipstick for me is like when your four-year-old daughter tries to paint her lipstick for the first time, I've never gotten past that because I just don't do it often enough.

Anyway, so we've got her makeup done, and her skin color looks good.

But it's still lacking some that detail that we saw from the packet.

Now, this is the color channel and I've got some nice variety to her color.

But with every character in a 3D film, there are multiple channels that define different parts of what the character looks like.

So this is the slide from the shader packet that shows me what the characters look like, what the character's face looks like in more detail.

And there's some really graphic shapes to her wrinkles and to the, you know, the shape of her face, more than just the color.

And so we would actually paint that on a separate channel.

It would be on what we call the bump or the displacement map.

And that's a grayscale channel that defines where the wrinkles go in and where things bump out.

And so, I'm going to show you how I would do that over here.

So, here we're starting with a color and I'm going to switch over to the bump channel.

It starts out with this you know, just this gray solid image.

And the first thing I'm going to do is flood in some texture, flood in just this basic leather variety.

Now, it looks good but it doesn't have the specificity that we'd really want for this character to match that artwork, that beautiful artwork.

So the first thing that I'm going to do is actually bring in a picture of an elephant.

So this is the side of an elephant as you can see.

And I'm actually going to use that to paint through and get some wrinkles on the surface of this character.

So I'm going to start with her eye.

And I'm trying to find the right angle to paint through because with wrinkles, if you look at some with really wrinkly skin, none of the wrinkles are arbitrary.

All of them flow exactly where the skin would fold and compress over time.

And so, I can't just paint these willy-nilly.

I have to really make sure that they are feeling what we would call motivated, right?

That they would be they would be naturally where the character would be wrinkling.

Now, this is obviously a made up character.

But the character still has to feel believable, right?

If this character creature were to exist, how would she wrinkle in her face?

So, I'm good with these.

I'm going to zoom out.

And this is one of my it's really one of my favorite features in MARI because I can get so much detail so rapidly onto the character.

And so I'm just going to paint some of these wrinkles in.

And again, since this is the black and white map, it's really hard to tell it how this is going to affect the final look of the character.

And I'll get more into that later.

But this is, you know, this is where we start from in getting a lot of this detail in.

There's this great spot on the texture that looks just like forehead, if I can find it.

Right about I'm upside down.

There we go.

So the next image that I'm actually going to pull in came from a picture that I took on my iPhone from the Pixar parking lot.

There are these trees there.

You can the tire of my car over here.

These trees there that spray sap in all the cars for some reason, and I'm not sure, you know, why.

But I was walking into work and I saw the texture on the side and I just stared at it, like, you know, weirdo.

I'm like, wow.

That texture looks really cool.

I'm not sure what I'm going to use it for.

So I took the picture and then I realized it was actually this really perfect way to transition from the organic leather on the front of her face to the more hardened shells on the side of her head.

So, I really loved it for that particular purpose.

Though when I took the picture, I had no idea when I was going to use it.

Again, I'm trying to get sort of this material transition because we're talking about two different textures here.

This, again, is one of my favorite parts because I can really paint in as much detail as I would like.

Now, so we've got these shapes and it's starting to feel good but it's still in the realm of being a bit too photorealistic, right?

We could be moving into Uncanny Valley here if I'm not careful with all of this photo detail that I'm using.

And so, the next thing that I would do is actually start to paint on top of this to really nail the details down the way that the way that it was in the art packet.

So, let's see.

I remember from the packet there were these really graphic shapes around her eyes.

I'm going to try and pull those out.

Again, like I said, I'm going pretty darn fast here.

So, I apologize I'm being a bit sloppy in how I do this.

Once I've painted kind of on top of this, it starts to make it feel more graphic and make it feel a bit more painterly and less creature-shop.

I'm just trying to get in some of these folds.

I'm going to get some great crow's feet on the side of her face.

It's funny because, shaders, we always love to work on old characters because their faces really tell a story.

You can see, you know, the most common expression a person has had for their whole life.

You know, if you frown a lot, those lines are kind of etched permanently into your face.

And with older characters, I actually get to take part in telling the story by what the character looks like.

And that was you know, my goal here is I with Hardscrabble, I really wanted her to look as good as this character was, I mean, because she's such an awesome character and I really wanted to take part in telling that story and make her look visually as cool as I possibly could.

So this is good enough for now, a good starting point.

So I've painted into this bump channel.

And I can actually take that channel and start to use it in the color.

So when I visit that channel here, I can multiply the color just like in Photoshop.

And we can start to see how it will affect the final look of the character.

But there's a whole lot of other channels that are needed for the final look.

The first is the specular channel.

What this is, and I'm not going to bother painting it today, what this is, is it's showing where the shininess is, if that makes any sense.

It's showing what parts are going to be shinier than other parts.

And then the roughness channel, which makes it even less sense, is showing how glossy or how rough those specular highlights are going to be.

Now, as an artist, these things don't make a whole lot of sense to me visually.

I look at this gray and I'm like, "OK.

What does that mean, right?

Her eyes are black.

Her lips are a slightly darker gray than her skin.

Her color is this other gray.

I don't know how that affects the final look."

And before a program like MARI, I would have to load up these channels as individual flattened images in a program like Photoshop.

I'd paint the gray her of her lips and hope that it was going to be the right shade of gray.

And then I'd kick off a render, and that render would take anywhere from two minutes for a really kind of inexpensive render that shows me a vague idea of what it's going to look like to a full 12-hour render if I'm going into the actual shot from the film with all of the lighting and all the bells and whistles turned on.

So if I really wanted to see how her lips looked in the shot, I would need to wait up to 12 hours to see that.

And so, I'd paint my gray.

I'd save the image.

I'd kick off the renders.

And I would wait.

And then I get to render back and I go like, "OK, her lips are too glossy," so I'd go back.

I'd paint a slightly darker gray.

And then I'd send it and then I'd realize I'd gone the wrong direction, and now her lips are even shinier.

And I'd wait another 12 hours, right?

So, you can see how that iteration could completely kill the creative process.

What MARI allows us to do is it allows us to take all of these separate kind of nonsensical channels and create a shader that we can actually preview in real time that lets us look at what the final look of the character is going to be in that finished render.

And instead of waiting for anywhere from 2 minutes to 12 hours, we can see that at 60 frames per second.

So I'm going to switch over to her shader and turn on the lighting.

And you can see here how now we've got all of those different channels that I've painted working at the same time.

We've got those bumps that I just painted, those wrinkles.

We've got the color.

And then the shininess, you can see how her eyelid is much shinier than the rest of her skin.

And this is, you know, this is possible, you know, again, through the hardware and the software working together to give us this image.

And if you think about it, remember how I said that each of those frames was 8,000 pixel squared.

Well there's nine of those for every single channel.

So for one channel, the color channel, that's 600 million pixels, all right, 600 million points of image data.

Now we're looking at four channels simultaneously which is about 2.4 billion pixels, all right?

That is a vast amount of data.

That's 10 gigabytes of image data.

And MARI is letting us preview that all in real time and it's running so smoothly on this hardware.

I've been working on MARI for about two or three years now and I've never seen it run smoother than it does on the new Mac Pro.

By the way, so I've been working on this for a couple of weeks now and it's been in this giant box.

I had no idea what it looked like until yesterday, all right?

No idea.

[ Applause ]

The funny thing was it could've been the size of a mini fridge.

I had no idea.

Or an iPhone.

We were thinking it was this floating sphere, you know.

I had no idea.

But I just knew that it was running really well.

I just had no idea it was inside this mystery box.

So, it was just as exciting for me yesterday as for you guys.

Anyway, I lost my train of thought.

So here we are.

We're looking at all these shaders.

Now, the great thing is even though it's really pretty to see this shader in real time, here's where the actual real advantage comes in.

Working at an animation studio, you paint all these details and then the animators have to come in and start moving stuff around, right?

I mean the audacity of that.

So, I paint the wrinkles on her and then once the character actually is talking and moving, sometimes what I painted doesn't look right anymore and I have to fix it.

Now, in the past, it's really hard to identify those problems or at least it takes a lot of time because to render the animation, I definitely need to wait at least a few hours to see that.

And if I want to see it in the context of the shot, I would need to wait up to 12 hours.

So, MARI allows us to bring the final animation from the shot into this environment so I can actually preview it.

So, I'm going to scrub to her animation here.

And this is the same shot that I showed you.

And you can see her speaking and moving around.

And I noticed at this point where she raises eyebrow like, wow, OK, my textures are stretching right there, all right?

So let me fix that.

So I'm going to go to my bump channel.

I'm going to pull in my elephant texture again.

And I'm going to grab these wrinkles.

And I can paint straight onto this animated frame, right, in real time.

[ Applause ]

It's pretty cool, huh?

[ Applause ]

And now when I play the animation, I can say, "OK, you know, how does it look, you know, at this moment, right?

How does it look, you know, at any given frame?"

And I've just identified and fixed a problem in about 30 seconds that would've typically taken me about 2 days before if I were to wait for the full all the bells and whistles renders, right?

And that is fantastic.

It is you know, I'm able to work ridiculously faster in an environment like this.

And it is, you know, again so, like I said, MARI has never run faster than I've seen it on this machine.

And we can push the limits of that.

So, this character represents about 10 gigabytes worth of image data but we can do more than that.

I can bring in Mike Wazowski which essentially doubles it to about 20 gigabytes because he's got his own set of textures.

And so he's over here, right, and we can preview his animation as well.

So he's saying, "Please, let me try the simulator.

I'll surprise you."

[ Laughter ]

So there he is.

And we've got both of these characters and I can see them in context together, right?

I can see them right here and that is so helpful.

It's so amazing to be able to find and identify any problems that we have by having these characters in the same room together.

And I can even bring in the set and see them with their shadows in this environment.

And it's been so amazing.

And, you know, I think that's it for the demo.

I've it's been such a privilege to speak to you today to able to work with this amazing hardware.

I can't wait to get my hands on one.

I can't wait 'til the fall.

And I've been so it's been such a privilege to work on Monsters University.

We're really proud of this film.

And I hope that all of you guys go see it because it's amazing.

We're going to end with another trailer from the movie.

And, yeah, let's go to that now.

[ Applause ]

[Background Music] Welcome to the Scare Floor.

Wow.

How do I become a scarer?

I'm officially a college student!

Mike Wazowski.

James P.

Sullivan.

Like Bill Sullivan?

He's my dad.

I expect big things from you.

I just need to ace my classes, graduate with honors, and become the greatest scarer ever.

Help!

[ Noise ]

Were you kissing my hand?

And what about you with all your shedding.

I don't shed.

Really?

The star player has just arrived.

You're my hero.

Yeah!

Slow down, squirt.

This is a party for scare students.

I am a scare student.

I mean for scare students who actually, you know, never change.

Just wait hotshot.

I'm going to scare circles around you this year.

[ Noise ]

My brothers appose my [inaudible].

[Inaudible] everybody let's keep a dream journal.

I'm a dance major.

And I'm not.

Not exactly the scariest group in the world.

Smile!

Hi mom!

[ Laughter ]

I thought I could show everybody that Mike Wazowski is something special.

Just reach deep down and let the scary out.

Boys, oh, scary.

Up top.

The whole school is finally going to see what Mike Wazowski can do.

Time to go to work.

Scary feet, scary feet, scary feet.

The kid's in the bathroom.

[Shouting]

Angry poodle.

Dance clown.

My Aunt Phyllis.

In the morning.

Shhh! Quiet.

What's so scary about a little old librarian?

I said quiet!

Move it!

Move it! [Shouting] [ Music ]

I want to piece of that action.

Tentacles and serpent's wings, they ugh!

Can't wait to start scaring with you brothers.

We're going to hit you to pieces.

[ Music ]

Stop doing that.

Have fun kids.

I'll just be here listening to my tunes.

[ Music ]

[ Applause ]

OK. So thank you very much to Jonathan and thank you very much to Apple for the opportunity for being here today.

And thank you for being here to watch.

Thank you.

Thanks for coming.

[ Applause ]

Apple, Inc. AAPL
1 Infinite Loop Cupertino CA 95014 US