Is It Possible to Assign Custom Environments to Materials in Three.js Path Tracing?

[RafaeldelM]

Hello,
I am using three-gpu-pathtracer and I truly love how easy it is to use. However, I have noticed that it does not support assigning a different environment to a material than the one used for the main scene.
Could you please let me know if this is possible with THREE.js-PathTracing-Renderer?
Thank you very much in advance.

[erichlof]

Hello @RafaeldelM

Apologies for the late reply - I was on my family vacation and just got back. I want to make sure I am understanding your question correctly: are you wanting to change the background environment HDR texture that covers the top hemisphere surrounding the entire scene, and then have it automatically light the scene? If so, I would imagine that the three-gpu-pathtracer that you are currently using lets you change out the surrounding HDR image texture for another one of your choosing. However, I am not familiar enough with Garrett's project to speak on this definitively. There should be a way for you to manually edit the string that contains the path and name of the HDR that you want to use instead of the default one that Garrett has assigned to his demo.

The case would be similar if you wanted to use my path tracer instead. If you look at the source code for my HDRI_Environment demo, you'll see here that I provide the path and image name in the JavaScript setup code:

THREE.js-PathTracing-Renderer/js/HDRI_Environment.js

Lines 502 to 506 in 457fb53

	hdrPath = 'textures/symmetrical_garden_2k.hdr';
	//hdrPath = 'textures/cloud_layers_2k.hdr';
	//hdrPath = 'textures/delta_2_2k.hdr';
	//hdrPath = 'textures/kiara_5_noon_2k.hdr';
	//hdrPath = 'textures/noon_grass_2k.hdr';

If you simply comment out the current path line and uncomment one of the following lines, that particular HDR background will be loaded instead (although, currently I haven't uploaded the other HDR options to my GitHub repo here, so you would need to grab those files from HDR Haven, or make sure the ones that you personally want to use are in the 'textures' folder, otherwise no background will be loaded). The more user friendly way to do it would be to stick all of this into a GUI menu dropdown list. That way, the end user can simply pick from a list which background environment image they would like to have illuminate all of the path traced objects in the scene. To see an example of this in action, check out my Babylon JS PathTracing Renderer project and its HDRI_Environment_Path_Tracing demo: https://erichlof.github.io/Babylon.js-PathTracing-Renderer/HDRI_Environment_Path_Tracing.html
and its JavaScript setup code to get the dropdown HDR environment list working with the GUI:
https://github.com/erichlof/Babylon.js-PathTracing-Renderer/blob/main/js/HDRI_Environment_Path_Tracing.js

That hopefully covers the first part of your question. The remaining part is a little more complicated though. If you change the environment while your demo is running, you need a way for the path tracer to learn where the important bright points are in the new HDR image texture. Once it locates the points of interest that have the highest luminosity (like the Sun on an outdoor scene), it then must convert those brightest 2D texels into a narrow cone or range of directions in 3D space. This will allow the path tracer to evaluate the lighting at every step along the path of each individual ray that was sent out from the camera through each pixel in the camera's viewing area. When a camera ray hits an object out in the scene, most path tracers will send out what's known as a 'shadow ray' towards an important bright area of the scene. This will often be artificial lights on an indoor scene (like rectangular or spherical area light sources), and the relatively small Sun disk on outdoor scenes (which usually dominates the lighting when you're outdoors). If the test shadow ray escapes the scene and is able to reach the Sun (or other important light source), then the material at that point is lit according to the material properties and BDRF for that surface. However, if something is obstructing the shadow ray's path towards the Sun (or other important light source), we leave that point on the surface unlit, or in shadow - hence the name shadow ray.

You'll notice that I only offer outdoor scene choices for HDR environments on my demos, because although tricky, programmatically stepping through each pixel in an HDR texture and locating the Sun's texels (usually the brightest by far), and then converting that into a 3D direction that the path tracer can sample at every step of the way, is doable. Now doing this for an indoor HDR environment with many arbitrary artificial light sources is another story. I'm not sure about Garrett's three-gpu-pathtracer, but at the moment, I have not figured out how to systematically go through each pixel of an indoor HDR image and make a list of all the important points, and then somehow sample from all of those simultaneously back in the path tracer's shadow ray code. It is doable, but it is way more involved because you have to understand and implement multiple importance sampling on the image, and all the possible light sources have to be weighted correctly (versus each other), otherwise the image will be incorrectly lit and will look slightly 'off' to the end user.

I hope that I have given you enough info to change out your desired outdoor HDR images, and maybe food for thought when it comes to using indoor images with arbitrary artificial lights. Apologies if I misunderstood your question or use case and if you already knew some of the topics I discussed. Let me know if you have any further issues or questions, and I'll do my best to either answer them, or at least point you in the right direction. 😊

-Erich