3D Scanning and Gaussian Splats for 3D Mesh Reconstruction.
3D Scanning and Gaussian Splats for 3D Mesh Reconstruction.
3D Scanning and Gaussian Splats for 3D Mesh Reconstruction.
3D Scanning and Gaussian Splats for 3D Mesh Reconstruction.
With rapid development of Artificial Intelligence and extended Research in development of 3D reconstruction algorithms, new approach to 3D scanning emerged - Gaussian Splatting.
With rapid development of Artificial Intelligence and extended Research in development of 3D reconstruction algorithms, new approach to 3D scanning emerged - Gaussian Splatting.
With rapid development of Artificial Intelligence and extended Research in development of 3D reconstruction algorithms, new approach to 3D scanning emerged - Gaussian Splatting.
With rapid development of Artificial Intelligence and extended Research in development of 3D reconstruction algorithms, new approach to 3D scanning emerged - Gaussian Splatting.
Sep 15, 2023
Sep 15, 2023
Sep 15, 2023
Sep 15, 2023
3D Reconstructed Digital Twin of the technocrane 15
Gaussian Splatting to Cinema Hardware and Gear for cataloguing purposes.
The paper on Gaussian Splatting appeared during SIGGRAPH 2023 and inspired me to apply the technology of 3D reconstruction to a cinema gear leasing company based in Vancouver, BC - William F. Whites. I prototyped a workflow enabling technicians at the company to capture a video of a hardware gear (cameras, technocranes, poles, lights, etc) resulting in a 3D digital visualization of the hardware.
What is the process behind 3D Reconstruction through Gaussian Splatting Technology?
The process starts with capturing visual data with camera. A technician takes an iPhone or smartphone camera and captures the object walking around it to get as many point of views of the object recorded on camera. Once the visual data is collected, it is being uploaded to a remote server. From there, the video is parsed to become a large data set of images. The Gaussian Splatting algorithm takes this data and trains an Artificial Intelligence pattern recognition to approximate the travelling of light rays relevant to the camera angles captured with the device. The algorithm relies on two sets of data: camera visual input and camera angles relevant to the object. After this complicated step, the algorithm starts to plot out a set of image data that represents different camera angles. The camera angles are further meshed together for consistent preview on the object. Lastly, the algorithm exports a 3D model out of the capture sharing the model with other developers and designers for clean up and further deployment.
Processing Time and Improvements.
Utilizing the latest hardware NVIDIA 4090 and Intel i9-13500K, it takes about an hour to receive the result from a 120 seconds video capture. Rendering on consumer hardware is a bottleneck and could be further improved by deployment on production hardware. Additionally, an important factor is the quality of the image. 4k 60fps footage is not yet supported by Gaussian Splatting algorithms and needs to be downscaled to 2K. Preprocessed 3D Data is a point cloud which did not possess much use to the client, so an algorithm to convert from .PLY to .GLB was used to enhance the workflow.
Why is not Gaussian Splatting widely used in 3D reconstruction of Hardware?
Striving for high quality and full control over the meshes being scanned, it is impossible to 'disassemble' the scanned mesh into sub-meshes. There will a significant loss in quality of assets and textures. 3D CAD modelling proves to be more time and quality efficient approach to hardware virtualization. Even though Gaussian Splats output a visualization faster than 3D CAD modeller could, it's purpose and application is still limited for full production use.
Gaussian Splatting to Cinema Hardware and Gear for cataloguing purposes.
The paper on Gaussian Splatting appeared during SIGGRAPH 2023 and inspired me to apply the technology of 3D reconstruction to a cinema gear leasing company based in Vancouver, BC - William F. Whites. I prototyped a workflow enabling technicians at the company to capture a video of a hardware gear (cameras, technocranes, poles, lights, etc) resulting in a 3D digital visualization of the hardware.
What is the process behind 3D Reconstruction through Gaussian Splatting Technology?
The process starts with capturing visual data with camera. A technician takes an iPhone or smartphone camera and captures the object walking around it to get as many point of views of the object recorded on camera. Once the visual data is collected, it is being uploaded to a remote server. From there, the video is parsed to become a large data set of images. The Gaussian Splatting algorithm takes this data and trains an Artificial Intelligence pattern recognition to approximate the travelling of light rays relevant to the camera angles captured with the device. The algorithm relies on two sets of data: camera visual input and camera angles relevant to the object. After this complicated step, the algorithm starts to plot out a set of image data that represents different camera angles. The camera angles are further meshed together for consistent preview on the object. Lastly, the algorithm exports a 3D model out of the capture sharing the model with other developers and designers for clean up and further deployment.
Processing Time and Improvements.
Utilizing the latest hardware NVIDIA 4090 and Intel i9-13500K, it takes about an hour to receive the result from a 120 seconds video capture. Rendering on consumer hardware is a bottleneck and could be further improved by deployment on production hardware. Additionally, an important factor is the quality of the image. 4k 60fps footage is not yet supported by Gaussian Splatting algorithms and needs to be downscaled to 2K. Preprocessed 3D Data is a point cloud which did not possess much use to the client, so an algorithm to convert from .PLY to .GLB was used to enhance the workflow.
Why is not Gaussian Splatting widely used in 3D reconstruction of Hardware?
Striving for high quality and full control over the meshes being scanned, it is impossible to 'disassemble' the scanned mesh into sub-meshes. There will a significant loss in quality of assets and textures. 3D CAD modelling proves to be more time and quality efficient approach to hardware virtualization. Even though Gaussian Splats output a visualization faster than 3D CAD modeller could, it's purpose and application is still limited for full production use.
Gaussian Splatting to Cinema Hardware and Gear for cataloguing purposes.
The paper on Gaussian Splatting appeared during SIGGRAPH 2023 and inspired me to apply the technology of 3D reconstruction to a cinema gear leasing company based in Vancouver, BC - William F. Whites. I prototyped a workflow enabling technicians at the company to capture a video of a hardware gear (cameras, technocranes, poles, lights, etc) resulting in a 3D digital visualization of the hardware.
What is the process behind 3D Reconstruction through Gaussian Splatting Technology?
The process starts with capturing visual data with camera. A technician takes an iPhone or smartphone camera and captures the object walking around it to get as many point of views of the object recorded on camera. Once the visual data is collected, it is being uploaded to a remote server. From there, the video is parsed to become a large data set of images. The Gaussian Splatting algorithm takes this data and trains an Artificial Intelligence pattern recognition to approximate the travelling of light rays relevant to the camera angles captured with the device. The algorithm relies on two sets of data: camera visual input and camera angles relevant to the object. After this complicated step, the algorithm starts to plot out a set of image data that represents different camera angles. The camera angles are further meshed together for consistent preview on the object. Lastly, the algorithm exports a 3D model out of the capture sharing the model with other developers and designers for clean up and further deployment.
Processing Time and Improvements.
Utilizing the latest hardware NVIDIA 4090 and Intel i9-13500K, it takes about an hour to receive the result from a 120 seconds video capture. Rendering on consumer hardware is a bottleneck and could be further improved by deployment on production hardware. Additionally, an important factor is the quality of the image. 4k 60fps footage is not yet supported by Gaussian Splatting algorithms and needs to be downscaled to 2K. Preprocessed 3D Data is a point cloud which did not possess much use to the client, so an algorithm to convert from .PLY to .GLB was used to enhance the workflow.
Why is not Gaussian Splatting widely used in 3D reconstruction of Hardware?
Striving for high quality and full control over the meshes being scanned, it is impossible to 'disassemble' the scanned mesh into sub-meshes. There will a significant loss in quality of assets and textures. 3D CAD modelling proves to be more time and quality efficient approach to hardware virtualization. Even though Gaussian Splats output a visualization faster than 3D CAD modeller could, it's purpose and application is still limited for full production use.
Gaussian Splatting to Cinema Hardware and Gear for cataloguing purposes.
The paper on Gaussian Splatting appeared during SIGGRAPH 2023 and inspired me to apply the technology of 3D reconstruction to a cinema gear leasing company based in Vancouver, BC - William F. Whites. I prototyped a workflow enabling technicians at the company to capture a video of a hardware gear (cameras, technocranes, poles, lights, etc) resulting in a 3D digital visualization of the hardware.
What is the process behind 3D Reconstruction through Gaussian Splatting Technology?
The process starts with capturing visual data with camera. A technician takes an iPhone or smartphone camera and captures the object walking around it to get as many point of views of the object recorded on camera. Once the visual data is collected, it is being uploaded to a remote server. From there, the video is parsed to become a large data set of images. The Gaussian Splatting algorithm takes this data and trains an Artificial Intelligence pattern recognition to approximate the travelling of light rays relevant to the camera angles captured with the device. The algorithm relies on two sets of data: camera visual input and camera angles relevant to the object. After this complicated step, the algorithm starts to plot out a set of image data that represents different camera angles. The camera angles are further meshed together for consistent preview on the object. Lastly, the algorithm exports a 3D model out of the capture sharing the model with other developers and designers for clean up and further deployment.
Processing Time and Improvements.
Utilizing the latest hardware NVIDIA 4090 and Intel i9-13500K, it takes about an hour to receive the result from a 120 seconds video capture. Rendering on consumer hardware is a bottleneck and could be further improved by deployment on production hardware. Additionally, an important factor is the quality of the image. 4k 60fps footage is not yet supported by Gaussian Splatting algorithms and needs to be downscaled to 2K. Preprocessed 3D Data is a point cloud which did not possess much use to the client, so an algorithm to convert from .PLY to .GLB was used to enhance the workflow.
Why is not Gaussian Splatting widely used in 3D reconstruction of Hardware?
Striving for high quality and full control over the meshes being scanned, it is impossible to 'disassemble' the scanned mesh into sub-meshes. There will a significant loss in quality of assets and textures. 3D CAD modelling proves to be more time and quality efficient approach to hardware virtualization. Even though Gaussian Splats output a visualization faster than 3D CAD modeller could, it's purpose and application is still limited for full production use.
Further reading
I wrote an article further explaining deployment and application of Gaussian Splatting to 3D reconstruction which you can read here.
