[Expert Talks] Advanced techniques for Realistic renderings in Virtual Try-On

About Christophe Dehais, Computer Graphics Manager at Fittingbox

Christophe has been part of Fittingbox team for 15 years now. Hand in hand with his team of 5 engineers, he works on techniques to keep improving our Virtual Try-On engine.

Computer Graphics team's overall goal is to keep improving the rendering of our real-time Virtual Try-On solutions to make them look as realistic as possible. 

How is Virtual Try-On so close to reality?

Virtual Try-On has revolutionized the eyewear industry. It provides customers with the confidence that what they see is very close to the real appearance of the glasses they’re considering buying.

Most techniques we develop are based on the principles of Physically-Based Rendering (PBR), which aims at modelling the materials’ appearance of the 3D digital frame in a way that closely matches how real light, and objects interact. 

Indeed, when light hits an object, it interacts with the irregularities of the surface before reaching our eyes. PBR helps our experts to model this phenomenon with remarkable accuracy. 

Learn more about the challenge of lighting in Virtual Try-On 

What are the challenges involved when it comes to transparent digital frames?

One of the significant challenges we've encountered is rendering transparent surfaces in a realistic manner. It is the case for crystal frames and tinted lenses.

Usually, transparent surfaces are drawn using a technique called Alpha Blending, which is easy to compute but unfortunately not very realistic. In contrast, our transmission-based model correctly takes reflections and background into account and makes justice to these tinted lenses colour. 

After considerable research and development, we overcame this challenge by devising a material model that renders transparent surfaces authentically. This was no small feat, but our dedication to realism drove us to find a solution that met our standards and exceeded our end users' expectations. 

Another unique rendering challenge we met head on was about complex designs that mix multiple layers. The question for us is how to correctly render those parts and in which order do we need to draw them to replicate the intended look.

For this we use a variant of a technique called Depth Peeling, which “slices” all the objects in as many layers as necessary, and render each one back to front, so that they all compose in a correct and realistic manner. Contrary to other methods that require human intervention when authoring the 3D asset, this is done automatically, in real-time, based on the current viewpoint, and importantly for every pixel.

One of the main challenges we faced was to make this fast for all the devices we target, especially on the mobile web platform. 

What’s next for even more realistic renderings?

All these techniques and more are available today in our virtual try-on renderer. But we’re continuing to add new features to represent a wider variety of materials more realistically.

One of those developments is about how to faithfully render diffuse materials, which let the light pass through but with a blurred, “frosted” appearance. Diffuse materials are transparent so they let the light pass through but their texture scatters light, which give the background a soft look.

We're currently working on improving this feature to continue elevating the virtual try-on experience for our users.