Tuesday, May 28, 2013

Pixar's Lightspeed Brings New Light to Monsters University



No, literally. Pixar's lighting/rendering systems were completely redone for their new film Monsters University. In my correspondence with Chris Horne, who studied Visualization at A&M and works Pixar's Lightspeed, he mentioned his work on MU:

I was on MU from May 2012 until April 2013 -- so yeah I worked on it for a *very very* long time as far as the lighting department is concerned. When I joined it was only leads + 1 shot lighter and me. It was a blast to work on because we completely rewrote our lighting system to be a raytraced/GI system -- and since I was on early I got to test the boundaries of it and figure out how we should light the show from a technical aspect. We really explored the lighting system - and I feel like a significant amount of work we did back then is going to live on in the way we light shows with this new technology. It was really unnerving and weird though to run into something new, ask some really smart people what the hell is going on, and hear back "We have no idea. It's all new. Good luck!" P.S. - the film is fully done and in the can as of this week (including credits, stereo, and all their international permutations, and the audio related to all of those). Creative production here finished maybe the 2nd week of April?

I was surprised that Pixar was just now using all ray tracing in their system. A few emails later I asked him if audiences will notice the difference in lighting:

There's a huge difference in MU compared to past films. Even people that don't know anything about our tech change going in walk out going "HOLY CRAP!"....but they have a hard time putting their finger on why it looks so awesome. Personally I see a huge difference between MU and Brave - there's more shaping, more little splashes of color, and everything feels a little bit more dynamic and pulled together. This is particularly evident in the toxic urchins sequence - where every single urchin is a light source. We couldn't have done that sequence in the past with our old technology. 

Historically we don't use raytracing. It wasn't until Cars that we actually supported raytracing (and even then it was a haphazard and mostly broken support). We really only used it for highly reflective smooth curved surfaces that absolutely needed to be truly reflected and not faked. We fake almost everything - mirrors, wet surfaces, eyes, shiny props like belt buckles/spoons/swords/etc. We obviously can't get away with that on Lightning McQueen - so we would cache out the scene into a brickmap (essentially a kd-tree with shading attached to the voxels) and fire rays against that (so even then....we aren't doing traditional raytracing). For shadows - we would sometimes use raytracing when we needed particularly awesome looking contact shadows. The same shadow would ramp off to using a shadowmap to help lower the expense. 

So our Director of Photography went to a studio that is so clearly raytracing averse and essentially said "We're raytracing everything. True reflection and refraction in the eyes reflecting actual SCENE GEOMETRY and not a brickmap. Yep - we're refracting through the cornea onto the sclera and iris. Oh and all your shadows are raytraced now - no more shadowmaps. Nope. None. Yes I know you like them but no. And global illumination! We're doing that now. By default. Everywhere. Oh and I almost forgot - all reflective surfaces will do real true reflection....and deciding what's reflective will be a shading decision instead of a lighting one. Yes you heard me right. Now get to work" It was extremely controversial, but it made a huge impact and really was one of the true success stories of the film. And now I'm working on that.

If you're advanced enough to understand the tech jargon above, good for you. For the uninitiated, ray tracing is a relatively advanced CG lighting technique which virtually simulates actual rays of light and all their interactions with the objects in your environment. In technical terms, Global Illumination lighting is like super hard core ray tracing. Both techniques use up a lot of memory. A more comprehensive explanation can be found here.

I was surprised that ray tracing in Pixar was historically a clunky, haphazard process. I always thought of it as this smooth, polished machine like something you would see at an Apple store. It's cool to see Pixar making history and advancing themselves over the past couple of years, I mean you can already see the difference in the images above.

Follow Chris on Twitter@distastee
And follow me on Twitter: @masonsmtx

15 comments:

  1. I'm not surprised. Pixar was way ahead in developing alternatives to ray tracing back when using it was utterly impractical (had they tried to raytrace Toy Story on the computers of that day, they would still be rendering RIGHT NOW) and that expertise kept them ahead of the game for a long time, including vis-a-vis other studios that, lacking Pixar's sunk costs, moved to raytracing earlier. We are only now reaching the stage where computers are fast enough to make it worth Pixar's time to make the jump.

    ReplyDelete
  2. FYI, the CSS for your anchors could be a bit better.
    They don't highlight anchors unless you mouse over them (making them hard to see within the text) such as the link to codermine

    ReplyDelete
    Replies
    1. +1, thought the exact same thing.

      Delete
    2. That, plus it's smarter to put the actual link on the words that explain the link.

      So don't write:
      A more comprehensive explanation can be found ___here___.

      But do write:
      Here's a ___more comprehensive explanation of ray-tracing___.

      Delete
    3. Thanks for the advice folks. I'm in the process of re-doing the look of the blog; honestly I didn't think I'd get so many pageviews in one day. A little embarrassed here.

      Delete
    4. M.S., it's probably because this page is in The Code Project's Daily News email.

      Delete
  3. IIRC, they first used ray tracing in A Bugs Life for the glass bottle at the "Mexican" resort the grasshoppers hung out at. But that was (again) a hack - a shader that sent requests to BMRT (Larry Gritz worked at Pixar at the time).

    ReplyDelete
    Replies
    1. Yes, that's correct. I was there at the time and Larry explained the "hack" to me at some point.

      Delete
  4. As I understand it, the vast majority of scenes were rendered with subdivision surfaces refined into triangles that were on the order of the desired pixel size or smaller.

    ReplyDelete
  5. What's wrong with the top pic? It's that the lightbulb is not on, it's lighted by some other source above and nearer to the "camera" and it doesn't cast a shadow. No realism there.

    ReplyDelete
    Replies
    1. Good observation Doc. Going to the sources for the image, there was a disclaimer made that the images are not complete. They are 3 images released from Pixar's Graphics Research page graphics.pixar.com. For a more powerful demonstration of Lightspeed's latest accomplishments, see the trailer and video clips.

      Delete
  6. So, does that mean no more point based approximate color bleeding? And when you say raytraced global illumination, do you mean monte carlo raytracing?

    ReplyDelete
    Replies
    1. http://graphics.pixar.com/library/RadiosityCaching/paper.pdf
      Brent, I've been slowly digesting the above memo from Pixar's research dept, and from what I understand their new GI system is meant to compliment, not replace point-based color bleeding. Correct me if I'm wrong.
      Not sure what specific type of raytracing their utilizing.

      Delete
  7. Ligh bulb is not on..It does not looks real..
    http://www.gameyan.com/character-shading-lighting.html

    ReplyDelete