Reflectance Transformation Imaging (RTI) in VFX

In VFX, real-life objects are commonly used within virtual scenes, either to compose these into plates or to render full shots built out of digital assets. There are three basic methods of capturing real objects for virtual scenes: 3D scanning, photogrammetry, and simple image or video plates. All of these methods have difficulty reflecting changing light situations and interactions with virtual lights. The objects must either be captured under the correct lighting conditions, or changes must be made manually using compositing techniques. Alternatively, a PBR pipeline can be used, but extracting the correct physical properties, such as diffuse color, roughness and subsurface scattering values, can be challenging. Another option that has not yet been used in this context is Reflectance Transformation Imaging (RTI). This technique has the potential to provide an alternative capable of approximating realistic light interactions with minimal performance impact.

What is RTI?

Reflectance Transformation Imaging is an image-based technique used to capture reflectance properties of objects. It is widely used in the conservation and documentation of cultural heritage, as well as in fields of archaeology and art history. It enables images to be relit with virtual light sources, revealing and examining surface characteristics of objects without touching them. Extending existing methods and algorithms opens up new possibilities for incorporating RTI into VFX pipelines for relighting.

[//] # (Continue here! How does it work?)

With a fixed camera perspective and one light source each with a known position. The lamps are usually arranged in a dome shape, are located in the hemisphere facing the camera and are aligned at a constant distance from the object.

Physical lights and custom renderer

Integrating RTI into DCCs and render engines

RTI BSDF

Challenges and further improvements

ML based approach for estimating reflectance values

Going 3D