Visualizing Point Cloud Data in Omniverse Using Kaolin

3D Visualization Pipeline for Omniverse Using Point Cloud Data

3D data processing is becoming increasingly vital across industries such as robotics, simulation, and digital twin development. In this post, we walk through a full pipeline that converts raw point cloud data into triangular mesh format using NVIDIA Kaolin, and visualizes it in NVIDIA Omniverse.

This guide covers the step-by-step transformation process from .ply format to .usda format, suitable for use in Omniverse.

Overview and Pipeline

The goal of this pipeline is to convert a .ply point cloud file into a triangular mesh in .usda format, which can be rendered and simulated in Omniverse. The steps involved are:

Point Cloud → NumPy array
NumPy → Voxel grid
Voxel → Triangle Mesh
Mesh → Omniverse visualization

This structured transformation makes the data easy to pre-process and use in simulation or machine learning environments.

Point Cloud Dataset: FoundationStereo

We used NVIDIA’s FoundationStereo, a project that provides high-resolution stereo matching models and datasets for 3D reconstruction and depth estimation.

Input: cloud_denoise.ply – noise-reduced point cloud

Intermediate files:
- converted.npy (NumPy array)
- converted_voxel.pt (Voxel data)
Final output:
- converted_mesh.usda (Mesh for Omniverse)

Libraries and Environment

Library	Purpose	Installation
NumPy	Convert to array format	pip install numpy
Open3D	Load and process .ply files	pip install open3d
PyTorch	Tensor computation and GPU acceleration	pip install torch torchvision torchaudio
Kaolin	Voxelization and mesh generation
pip install kaolin==0.17.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-{TORCH_VERSION}_cu{CUDA_VERSION}.html
USD Core	Save mesh in .usda format for Omniverse	pip install usd-core

Step 1: Convert .ply to NumPy

Output: converted.npy

Step 2: Convert NumPy to Voxel Grid

Output: converted_voxel_02.pt

Step 3: Convert Voxel to Triangle Mesh

Output: converted_mesh_02.usda

Step 4: Visualize in Omniverse

You can now load the .usda file into Omniverse Isaac Sim or USD Composer.

Voxel resolution: 128

The 3D mesh will appear based on the voxel resolution and mesh detail created in previous steps. A higher resolution like 512 provides finer geometry but also increases memory usage, so adjust accordingly based on your system and use case.

Voxel resolution: 512

Conclusion

This workflow shows how to transform stereo-based point cloud data into a renderable mesh for Omniverse using Kaolin. From raw .ply files to .usda output, every step is reproducible with open tools from NVIDIA and the open-source community.

We found this pipeline particularly useful for simulations, digital twin creation, and robotic AI environments. Feel free to adapt this process for your own 3D recognition or visualization projects.