People of ACM - Paul Debevec
March 8, 2016
How did you become interested in computer-generated imagery? Do you believe that having a PhD in computer science gives you a different perspective than other colleagues in the special effects field?
My favorite classes in elementary school were math and art, and when I realized you could use math to create art through the use of a computer, I became very interested! I had also gotten involved as a photographer for my high school newspaper and yearbook and, when I saw there were ways of digitizing photographs and processing them to compute new photoreal imagery, I decided to work in this area for my graduate degree at UC Berkeley. I’m very happy to have a Ph.D.: it’s a license to choose a problem of strategic interest to your organization and to lead the charge to develop a way to solve it. In both academia and industry, a Ph.D. will give you much more of a chance to work on what you find interesting. While most of visual effects involves the artistic application of existing technology, sometimes new technology needs to be invented, and it’s very likely there’s a job for a Ph.D. at the core of that process.
Your upcoming ACM talk at SXSW is titled "Lighting Hollywood’s Real and Virtual Actors." Can you explain why lighting is so central to creating believable virtual humans?
A frame of a film is a grid of pixels: a few million measurements of the color and intensity of light. Watching a movie, we perceive people and environments, but all we really see is the light reflected off of them. To create realistic imagery with human characters, we have to synthesize convincing patterns of light, and to do that, we can simulate where light comes from, how it reflects off of the skin’s surface and scatters through tissues, how some of the light is absorbed by the pigments and blood of the skin, and how the light reflects and diffuses out toward an observer. Many of the ACM SIGGRAPH papers I’ve worked on have been about how to digitize light, how to digitize the reflectance of human faces, and how to render photorealistic digital models of people under any possible lighting condition. It’s been very exciting to work with the visual effects community to see some of these ideas added into the mix of how movies are made.
In a recent interview you said that, just 10 years ago, the processes now being used at ICT were only vaguely imagined. What new advances in computing are transforming the field?
Computer graphics, like most fields, takes advantage of the vast increases in computing power, bandwidth, and storage capability we’ve had in the past few decades. Many of the things we can do today, such as simulating all the rays of light in a scene, or animating the mechanics of complex biological materials, or photographing a densely-sampled light field at high resolution, have become practical as a result of these advances and are now producing barely-dreamed-of results. What’s even more exciting is the multiplicative power of the additional computation and the increasingly clever and efficient algorithms our community invents, making it possible to see compelling solutions to longstanding challenges such as photoreal facial animation and realistic scene rendering.
If younger colleagues asked your advice about getting involved in your area of specialization, what foundational computer science coursework would you suggest they take, and what are some exciting growth areas in the field you would encourage them to explore?
The most important math classes I took were the first two semesters of linear algebra and calculus, which help me approach problems sensibly from the very beginning. Often I end up writing down the math I used to solve a problem only after we have a working algorithm. And something very important happened when I took the Digital Design class at the University of Michigan and learned how to make a complete working computer—including memory, branching, and math operations—out of NAND gates and a clock. Since then, nothing about computers has seemed fundamentally mysterious.
The best way to widen your skill set is to learn by doing, even on just a toy version of a problem. When I needed to take my computer graphics qualifying exam at UC Berkeley, I needed to be able to answer questions about ray tracing and radiosity. So I wrote an extremely simple ray tracer which rendered a few spheres, and I wrote a very simple radiosity program which computed the bounce light between three planes. That’s when these lighting simulation algorithms lost their mystery, and I simply started thinking that way. I’d be scrambling a lot more to understand today’s exciting advances in Artificial Intelligence if I hadn’t programmed a neural network to detect facial features for my undergraduate AI class project back in the late 1980’s.
Today, many people in computer science look to Deep Learning algorithms and Convolutional Neural Networks as ways of developing very powerful and useful tools based on the wealth of information now available on the internet, and to Virtual and Augmented Reality as exciting ways to deliver new visual experiences, empowered by the recent advances in hand-held displays from the smartphone industry. Learning about these areas will be exciting and could very well lead to interesting job opportunities! But remember, these fields have been around for decades, and have gotten renewed interest due to unexpected disruptive advances in other areas. Taking the time to learn what’s going on in anything new you find mysterious has a great chance to open possibilities for invention and accomplishment far into the future.
One of the three exciting ACM sessions featured at this year's South by Southwest (SXSW) Interactive Conference in March is "Lighting Hollywood's Real and Virtual Actors." Presenter Paul Debevec is a former Vice President of ACM SIGGRAPH and an Academy Award recipient. His unique expertise stems from leading the Graphics Laboratory at the University of Southern California's Institute for Creative Technologies (ICT) and serving as a research professor in the computer science department. Debevec has worked on some of Hollywood's top-grossing films including Avatar, The Curious Case of Benjamin Button and Maleficent.
His most high-profile project at ICT has been using advances in computing technology to produce believable virtual humans. The magic at ICT begins with a real actor sitting in a device Debevec invented called a “Light Stage,” a 360-degree sphere outfitted with more than 6,000 LED lights and 50 cameras. The cameras capture images of the actor from every angle, which are then processed by computers to create a composite. In addition to the film industry, ICT technology is used by the US military for training, and it is expected to be part of the anticipated holographic technology boom.