Depth Point Cloud – Lidar Point Cloud from Depth Image¶

We can save 3D pointclouds effciently using Jpeg compression, by representing it as RGBA or grayscale images. In this example, we use a grayscale PNG image for depth. Note that PNG encoding and decoding can be 20 \(\times\) slower than the FFT operations in a jpeg, so for visualization purposes, you can use jpeg for both color and depth.

Generate point clouds directly from depth images using the DepthPointCloud component.

Basic Example¶

import asyncio

from vuer import Vuer, VuerSession
from vuer.schemas import DepthPointCloud

vuer = Vuer(workspace="./rgb_depth", killport=True)


@vuer.spawn(start=True)
async def main(sess: VuerSession):
  sess.upsert @ DepthPointCloud(
    key="depth-point-cloud",
    depth=vuer.localhost_prefix / "depth.png",
    cmap="viridis",
    colorMode=4,
  )
  await asyncio.sleep(0.001)

  await vuer.loop_forever()

Performance Tip

When rendering multiple DepthPointCloud instances, wrap them in a DepthPointCloudProvider for better performance. The provider enables frustum culling, level-of-detail rendering, and shared caching.

With Provider (Recommended for Multiple Point Clouds)¶

import asyncio

from vuer import Vuer, VuerSession
from vuer.schemas import DepthPointCloud, DepthPointCloudProvider

vuer = Vuer(workspace="./rgb_depth", killport=True)


@vuer.spawn(start=True)
async def main(sess: VuerSession):
  sess.upsert @ DepthPointCloudProvider(
    DepthPointCloud( key="pc-0", depth=vuer.localhost_prefix / "depth.png", position=[0, 0, 0] ),
    DepthPointCloud( key="pc-1", depth=vuer.localhost_prefix / "depth.png", position=[2, 0, 0] ),
    key="provider",
    frustumCulling=True,
  )

  await vuer.loop_forever()

DepthPointCloud Parameters¶

Parameter	Type	Default	Description
`depth`	str	(required)	URL to 16-bit depth PNG image
`rgb`	str	None	URL to RGB image (uses depth grayscale if not provided)
`position`	tuple	[0, 0, 0]	Position in 3D space [x, y, z]
`rotation`	tuple	[0, 0, 0]	Rotation in Euler angles [x, y, z]
`scale`	tuple	[1, 1, 1]	Scale factors [x, y, z]
`fov`	float	58	Vertical field of view in degrees (58 = RealSense D435)
`pointSize`	float	2.0	Point size in pixels or world units
`screenSpaceSizing`	bool	True	If true, points have constant pixel size
`cmap`	str	None	Colormap: `"turbo"`, `"viridis"`, `"inferno"`, `"jet"`, or None for RGB
`colorMode`	int	0	Color mode (see table below)
`depthMin`	float	0.1	Minimum depth for visualization mapping
`depthMax`	float	50	Maximum depth for visualization mapping
`heightMin`	float	-2	Minimum height for visualization mapping
`heightMax`	float	2	Maximum height for visualization mapping

Color Modes¶

Value	Mode	Description
0	depth	Color by raw depth value
1	camZ	Color by camera Z distance
2	camDist	Color by Euclidean distance from camera
3	localY	Color by local Y coordinate
4	worldY	Color by world Y coordinate (height)

DepthPointCloudProvider Parameters¶

Parameter	Type	Default	Description
`frustumCulling`	bool	True	Skip rendering point clouds outside camera view
`cullingMargin`	float	2.0	Margin multiplier for culling bounds
`lod`	dict	None	Level-of-detail configuration with strides and distances
`bake`	dict	None	Bake configuration for depth processing
`children`	list	[]	Child `DepthPointCloud` elements