Deep learning: the answer to display noise?
27 July 2017
To achieve the striking imagery in modern animation, animators must apply painstaking details and remove the dreaded display ‘noise’ – but research now shows that neural networking can do so for them.
Modern films and TV shows are filled with spectacular computer-generated sequences computed by rendering systems that simulate the flow of light in a three-dimensional scene and convert the information into a two-dimensional image.
However, computing the thousands of light rays (per frame) to achieve accurate colour, shadows, reflectivity and other light-based characteristics is a labour-intensive, time-consuming and expensive undertaking. While a more time and labour-efficient alternative is to render the images using only a few light rays, this results in inaccuracies that show up as objectionable ‘noise’ in the final image.
Over the past couple of years, analysts Pradeep Sen, electrical and computer engineering professor; and PhD student Steve Bako, have worked with researchers at Disney Research and Pixar Animation Studios to develop a new technology based on artificial intelligence and deep learning.
This is to eliminate the aforementioned noise and enable production-quality rendering at much higher speeds.
Steve Bako spent a year working at Pixar. The team tested the software by using millions of examples from the film Finding Dory to train a deep-learning model known as a convolutional neural network. Through this process, the system learned to transform noisy images into noise-free versions that resemble those computed with significantly more light rays. Once trained, the system successfully removed the noise on test images from entirely different Pixar films – despite their completely disparate styles and colour palettes.
“Noise is a really big problem for production rendering,” said Tony DeRose, head of research at Pixar. “This new technology allows us to automatically remove the noise while preserving the detail in our scenes.”
The work presents a significant step forward over previous state-of-the-art denoising methods, which often left artifacts or residual noise that required artists to either render more light rays or to tweak the denoising filter to improve the quality of a specific image. Disney and Pixar plan to incorporate the technology in their production pipelines to accelerate the movie-making process.
“Other approaches for removing image noise have grown increasingly complex, with diminishing returns,” said Markus Gross, vice president for research at Disney Research. “By leveraging deep learning, this work presents an important step forward for removing undesirable artifacts from animated films.”
Credit: Ben-Gurion University of the Negev, aabgu.org
Contact Details and Archive...