Component Library
vuer.schemas

vuer.schema contains the schemas for the various components of the Vuer system. These schemas are used to validate the data that is passed to the various components of the system.

For detailed view of how these components are implemented, please refer to the typescript source code at https://github.com/vuer-ai/vuer-ts/tree/master/src/schemas.

Example Usage:

from vuer import Vuer
from vuer.schemas import DefaultScene, Sphere

vuer = Vuer()
vuer.set @ DefaultScene(up=[0, 1, 0])
vuer.upsert @ Sphere(args=[0.1, 20, 20], position=[0, 0.1, 0], key="sphere")

HTML Components

class vuer.schemas.html_components.Element

Bases: object

Base class for all elements

tag: str = 'div'

serialize()

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.html_components.BlockElement

Bases: Element

serialize()

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.html_components.AutoScroll

Bases: BlockElement

tag: str = 'AutoScroll'

class vuer.schemas.html_components.Markdown

Bases: BlockElement

tag: str = 'Markdown'

class vuer.schemas.html_components.Page

Bases: BlockElement

A Page is an element that contains other elements. It is represented by a div element in the DOM.

tag: str = 'article'

class vuer.schemas.html_components.div

Bases: BlockElement

tag: str = 'Div'

class vuer.schemas.html_components.InputBox

Bases: Element

An InputBox is an element that allows the user to input text. It is represented by an input element in the DOM.

tag: str = 'Input'

class vuer.schemas.html_components.Header1

Bases: BlockElement

A Text element is an element that displays text. It is represented by a text, or p element in the DOM.

tag: str = 'h1'

vuer.schemas.html_components.Header

alias of Header1

class vuer.schemas.html_components.Header2

Bases: Header1

Header 2

tag: str = 'h2'

class vuer.schemas.html_components.Header3

Bases: Header1

Header 2

tag: str = 'h3'

class vuer.schemas.html_components.Paragraph

Bases: BlockElement

A Text element is an element that displays text. It is represented by a text, or p element in the DOM.

tag: str = 'p'

class vuer.schemas.html_components.span

Bases: Element

A Text element is an element that displays text. It is represented by a text, or p element in the DOM.

tag: str = 'Span'

class vuer.schemas.html_components.Bold

Bases: span

class vuer.schemas.html_components.Italic

Bases: span

class vuer.schemas.html_components.Link

Bases: span

tag: str = 'a'

class vuer.schemas.html_components.Button

Bases: Element

A Button element is an element that allows the user to click on it. It is represented by a button element in the DOM.

tag: str = 'Button'

class vuer.schemas.html_components.Slider

Bases: Element

A Slider element is an element that allows the user to slide a value. It is represented by a slider element in the DOM.

tag: str = 'Slider'

class vuer.schemas.html_components.Image

Bases: Element

An Image element is an element that displays an image. It is represented by an img element in the DOM.

tag: str = 'Img'

class vuer.schemas.html_components.ImageUpload

Bases: Element

A ImageUpload element is an element that allows the user to upload a file. It is represented by a file upload element in the DOM.

tag: str = 'ImageUpload'

3D Scene Components

class vuer.schemas.scene_components.BlockElement

Bases: Element

serialize()

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.scene_components.Image

Bases: Element

An Image element is an element that displays an image. It is represented by an img element in the DOM.

tag: str = 'Img'

class vuer.schemas.scene_components.Element

Bases: object

Base class for all elements

tag: str = 'div'

serialize()

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.scene_components.SceneElement

Bases: BlockElement

class vuer.schemas.scene_components.Frustum

Bases: SceneElement

Camera Frustum

Parameters:

• position (tuple[float, float, float]) – An optional tuple of three numbers representing the position.

• rotation (tuple[float, float, float]) – An optional tuple of three numbers representing the rotation.

• matrix (tuple[float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float]) – An optional tuple of sixteen numbers representing the matrix.

• aspect (float) – An optional number representing the aspect.

• focus (float) – An optional number representing the focus.

• fov (float) – An optional number representing the field of view.

• near (float) – An optional number representing the near field.

• far (float) – An optional number representing the far field.

• scale (float) – An optional number representing the scale.

• upScale (float) – An optional number representing the up scale.

• focalLength (float) – An optional number representing the focal length.

• showUp (bool) – An optional boolean indicating whether to show up.

• showFrustum (bool) – An optional boolean indicating whether to show the frustum.

• showFocalPlane (bool) – An optional boolean indicating whether to show the focal plane.

• showImagePlane (bool) – An optional boolean indicating whether to show the image plane.

• src (str) – An optional string representing the source.

• colorOrigin (ColorRepresentation) – An optional ColorRepresentation for the origin color.

• colorFrustum (ColorRepresentation) – An optional ColorRepresentation for the frustum color.

• colorCone (ColorRepresentation) – An optional ColorRepresentation for the cone color.

• colorFocalPlane (ColorRepresentation) – An optional ColorRepresentation for the focal plane color.

• colorUp (ColorRepresentation) – An optional ColorRepresentation for the up color.

• colorTarget (ColorRepresentation) – An optional ColorRepresentation for the target color.

• colorCross (ColorRepresentation) – An optional ColorRepresentation for the cross color.

tag: str = 'Frustum'

class vuer.schemas.scene_components.CameraHelper

Bases: SceneElement

tag: str = 'CameraHelper'

class vuer.schemas.scene_components.group

Bases: SceneElement

tag: str = 'group'

children = []

class vuer.schemas.scene_components.mesh

Bases: SceneElement

tag: str = 'mesh'

children = []

class vuer.schemas.scene_components.TriMesh

Bases: SceneElement

tag: str = 'TriMesh'

children = []

vertices: ndarray[Any, dtype[float16]] = None

faces: ndarray[Any, dtype[uint32]] = None

colors: ndarray[Any, dtype[uint8]] = None

uv: ndarray[Any, dtype[float16]] = None

class vuer.schemas.scene_components.PointCloud

Bases: SceneElement

PointCould element, highly optimized for payload size and speed.

Parameters:

• vertices (NDArray[np.float16]) – An optional numpy array of shape (N, 3) containing the vertices of the pointcloud.

• colors – An optional numpy array of shape (N, 3) containing the colors of the point cloud.

• size (float) – An optional float that sets the size of the points.

• key (str) – str An optional string that sets the key of the element.

Usage:

sess.upsert @ PointCloud(
    vertices=np.random.rand(1000, 3),
    colors=np.random.rand(1000, 3),
    size=0.01,
    key="pointcloud",
)

tag: str = 'PointCloud'

vertices: ndarray[Any, dtype[float16]] = None

An optional numpy array of shape (N, 3) containing the vertices of the point cloud.

colors: ndarray[Any, dtype[uint8]] = None

An optional numpy array of shape (N, 3) containing the colors of the point cloud.

children = []

vuer.schemas.scene_components.p

alias of PointCloud

class vuer.schemas.scene_components.Box

Bases: SceneElement

tag: str = 'Box'

class vuer.schemas.scene_components.Capsule

Bases: SceneElement

tag: str = 'Capsule'

class vuer.schemas.scene_components.Cone

Bases: SceneElement

tag: str = 'Cone'

class vuer.schemas.scene_components.Circle

Bases: SceneElement

tag: str = 'Circle'

class vuer.schemas.scene_components.Cylinder

Bases: SceneElement

tag: str = 'Cylinder'

class vuer.schemas.scene_components.Dodecahedron

Bases: SceneElement

tag: str = 'Dodecahedron'

class vuer.schemas.scene_components.Edges

Bases: SceneElement

tag: str = 'Edges'

class vuer.schemas.scene_components.Extrude

Bases: SceneElement

tag: str = 'Extrude'

class vuer.schemas.scene_components.Icosahedron

Bases: SceneElement

tag: str = 'Icosahedron'

class vuer.schemas.scene_components.Lathe

Bases: SceneElement

tag: str = 'Lathe'

class vuer.schemas.scene_components.Octahedron

Bases: SceneElement

tag: str = 'Octahedron'

class vuer.schemas.scene_components.Plane

Bases: SceneElement

tag: str = 'Plane'

class vuer.schemas.scene_components.Polyhedron

Bases: SceneElement

tag: str = 'Polyhedron'

class vuer.schemas.scene_components.Ring

Bases: SceneElement

tag: str = 'Ring'

class vuer.schemas.scene_components.Shape

Bases: SceneElement

tag: str = 'Shape'

class vuer.schemas.scene_components.Sphere

Bases: SceneElement

tag: str = 'Sphere'

class vuer.schemas.scene_components.Tetrahedron

Bases: SceneElement

tag: str = 'Tetrahedron'

class vuer.schemas.scene_components.Torus

Bases: SceneElement

tag: str = 'Torus'

class vuer.schemas.scene_components.TorusKnot

Bases: SceneElement

tag: str = 'TorusKnot'

class vuer.schemas.scene_components.Tube

Bases: SceneElement

tag: str = 'Tube'

class vuer.schemas.scene_components.Fog

Bases: SceneElement

Fog is a scene element that adds fog to the scene. This can be used to approximate depth.

Args:

color: The color of the fog. near: The distance to the near plane. far: The distance to the far plane.

Example Usage:

Fog(color=”green”, near=3, far=7)

tag: str = 'fog'

class vuer.schemas.scene_components.Wireframe

Bases: SceneElement

tag: str = 'Wireframe'

class vuer.schemas.scene_components.Splat

Bases: SceneElement

tag: str = 'Splat'

class vuer.schemas.scene_components.LumaSplats

Bases: SceneElement

tag: str = 'Splats'

class vuer.schemas.scene_components.Pcd

Bases: SceneElement

tag: str = 'Pcd'

class vuer.schemas.scene_components.CameraView

Bases: SceneElement

CameraView for rendering from arbitrary camera poses.

Parameters:

• fov (float, optional) – The vertical field of view of the camera. Defaults to 50.

• width (int, optional) – The width of the camera image. Defaults to 320.

• height (int, optional) – The height of the camera image. Defaults to 240.

• key (str, optional) – The key of the camera view. Defaults to “ego”.

• position (List[float], optional) – The position of the camera. Defaults to [0, 0, 0].

• rotation (List[float], optional) – The rotation of the camera. Defaults to [0, 0, 0]

• stream (str, optional) – The stream of the camera. Defaults to “ondemand”.

• fps (int, optional) – The frames per second of the camera. Defaults to 30.

• near (float, optional) – The near field of the camera. Defaults to 0.1.

• far (float, optional) – The far field of the camera. Defaults to 20.

• renderDepth (bool, optional) – Whether to render depth. If set, returns value[“depthFrame”]. Defaults to True.

• showFrustum (bool, optional) – Whether to show the frustum. Defaults to True.

• downsample (int, optional) – The downsample rate. Defaults to 1.

• distanceToCamera (float, optional) – The distance to the camera. Defaults to 2.

tag: str = 'CameraView'

class vuer.schemas.scene_components.SceneBackground

Bases: Image, SceneElement

Sets the background of the scene to a static image. Does not work well with high frame rates. For displaying movies, use the ImageBackground element.

tag: str = 'SceneBackground'

class vuer.schemas.scene_components.ImageBackground

Bases: Image, SceneElement

Sets the background of the scene to an image, Supports high frame rates.

We use a plane that is always facing the camera to display the image.

tag: str = 'ImageBackground'

class vuer.schemas.scene_components.HUDPlane

Bases: Image, SceneElement

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

tag: str = 'HUDPlane'

class vuer.schemas.scene_components.VideoMaterial

Bases: Image, SceneElement

A Video Material for loading from a file hosted at a url.

tag: str = 'VideoMaterial'

class vuer.schemas.scene_components.WebRTCVideoMaterial

Bases: Image, SceneElement

A Video Material for loading from a media stream.

tag: str = 'WebRTCVideoMaterial'

class vuer.schemas.scene_components.VideoPlane

Bases: Image, SceneElement

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

tag: str = 'VideoPlane'

class vuer.schemas.scene_components.WebRTCVideoPlane

Bases: Image, SceneElement

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

tag: str = 'WebRTCVideoPlane'

class vuer.schemas.scene_components.StereoVideoPlane

Bases: Image, SceneElement

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

tag: str = 'StereoVideoPlane'

class vuer.schemas.scene_components.WebRTCStereoVideoPlane

Bases: Image, SceneElement

A Head-up display (HUD) plane that is always facing the camera. Requires mounting a material.

tag: str = 'WebRTCStereoVideoPlane'

class vuer.schemas.scene_components.Group

Bases: SceneElement

tag: str = 'VuerGroup'

class vuer.schemas.scene_components.HemisphereLight

Bases: SceneElement

tag: str = 'HemisphereLight'

class vuer.schemas.scene_components.RectAreaLight

Bases: SceneElement

tag: str = 'RectAreaLight'

class vuer.schemas.scene_components.Stage

Bases: SceneElement

tag: str = 'Stage'

class vuer.schemas.scene_components.Gamepads

Bases: SceneElement

tag: str = 'Gamepads'

class vuer.schemas.scene_components.DirectionalLight

Bases: SceneElement

tag: str = 'DirectionalLight'

class vuer.schemas.scene_components.PointLight

Bases: SceneElement

tag: str = 'PointLight'

class vuer.schemas.scene_components.SpotLight

Bases: SceneElement

tag: str = 'SpotLight'

class vuer.schemas.scene_components.AmbientLight

Bases: SceneElement

tag: str = 'AmbientLight'

class vuer.schemas.scene_components.Html

Bases: SceneElement

Usage:

as='div' // Wrapping element (default: 'div')
wrapperClass // The className of the wrapping element (default: undefined)
prepend // Project content behind the canvas (default: false)
center // Adds a -50%/-50% css transform (default: false) [ignored in transform mode]
fullscreen // Aligns to the upper-left corner, fills the screen (default:false) [ignored in transform mode]
distanceFactor={10} // If set (default: undefined), children will be scaled by this factor, and also by distance to a PerspectiveCamera / zoom by a OrthographicCamera.
zIndexRange={[100, 0]} // Z-order range (default=[16777271, 0])
portal={domnodeRef} // Reference to target container (default=undefined)
transform // If true, applies matrix3d transformations (default=false)
sprite // Renders as sprite, but only in transform mode (default=false)
calculatePosition={(el: Object3D, camera: Camera, size: { width: number; height: number }) => number[]} // Override default positioning function. (default=undefined) [ignored in transform mode]
occlude={[ref]} // Can be true or a Ref<Object3D>[], true occludes the entire scene (default: undefined)
onOcclude={(visible) => null} // Callback when the visibility changes (default: undefined)
{...groupProps} // All THREE.Group props are valid
{...divProps} // All HTMLDivElement props are valid

tag: str = 'Html'

class vuer.schemas.scene_components.Pivot

Bases: SceneElement

tag: str = 'Pivot'

class vuer.schemas.scene_components.Movable

Bases: SceneElement

tag: str = 'Movable'

class vuer.schemas.scene_components.Hands

Bases: SceneElement

tip Setting stream to True

Important: You need to set the stream option to True to start streaming the hand movement.

The Hand component offers a way to stream the current pose of the hand to the server. You can get the full pose of the hands by listening to the HAND_MOVE event. You can add flags left and right to specify which hand you want to track. The returned data looks like the following:

/**
 * Left and right pose are relative to the wrist transformations.
 */
export type HandsData = {
  left?: Float32Array;       // 25 * 16 values.
  right?: Float32Array;      // 25 * 16 values.
  leftState: HandState;
  rightState: HandState;
};

export type HandState = {
  pinch: boolean;
  squeeze: boolean;
  tap: boolean;

  pinchValue: number;
  squeezeValue: number;
  tapValue: number;
}

Matrix format

All 4x4 transform matrices used in WebGL are stored in 16-element Float32Arrays. The values are stored in the array in column-major order; that is, each column is written into the array top-down before moving to the next column to the right and writing it into the array. Therefore, for the array [a0, a1, a2, …, a13, a14, a15], the matrix looks like this:

For details, refer to the MDN documentation on [XR Rigid Body Transformation](https://developer.mozilla.org/en-US/docs/Web/API/XRRigidTransform/matrix)

Hand Landmarks

We follow the [XR Hand](https://developer.mozilla.org/en-US/docs/Web/API/XRHand) conventions, and return the landmarks in a single array of 25 * 16 values in the following order:

Hand joint	Index	Hand joint (continue)	Index
wrist	0	middle-finger-phalanx-distal	13
thumb-metacarpal	1	middle-finger-tip	14
thumb-phalanx-proximal	2	ring-finger-metacarpal	15
thumb-phalanx-distal	3	ring-finger-phalanx-proximal	16
thumb-tip	4	ring-finger-phalanx-intermediate	17
index-finger-metacarpal	5	ring-finger-phalanx-distal	18
index-finger-phalanx-proximal	6	ring-finger-tip	19
index-finger-phalanx-intermediate	7	pinky-finger-metacarpal	20
index-finger-phalanx-distal	8	pinky-finger-phalanx-proximal	21
index-finger-tip	9	pinky-finger-phalanx-intermediate	22
middle-finger-metacarpal	10	pinky-finger-phalanx-distal	23
middle-finger-phalanx-proximal	11	pinky-finger-tip	24
middle-finger-phalanx-intermediate	12

Usage Example:

from vuer import Vuer, VuerSession
from vuer.schemas import Hands
from asyncio import sleep

app = Vuer()


@app.add_handler("HAND_MOVE")
async def handler(event, session):
    print(f"Movement Event: key-{event.key}", event.value)


@app.spawn(start=True)
async def main(session: VuerSession):
    # Important: You need to set the `stream` option to `True` to start
    # streaming the hand movement.
    session.upsert @ Hands(stream=True, key="hands")

    while True:
        await sleep(1)

tag: str = 'Hands'

class vuer.schemas.scene_components.MotionControllers

Bases: SceneElement

MotionController for tracking XR controller poses and button states.

Parameters:

• key (str, optional) – Unique identifier for the controller.

• eventTypes (tuple or list, optional) – A tuple or list of events to track (e.g., “trigger”, “squeeze”).

• stream (bool, optional) – Whether to enable streaming of controller data.

• left (bool, optional) – Boolean indicating if the left controller should be tracked.

• right (bool, optional) – Boolean indicating if the right controller should be tracked.

• showLeft (bool, optional) – Boolean indicating if the left controller visualization should be shown.

• showRight (bool, optional) – Boolean indicating if the right controller visualization should be shown.

tag: str = 'MotionControllers'

class vuer.schemas.scene_components.Obj

Bases: SceneElement

tag: str = 'Obj'

class vuer.schemas.scene_components.CoordsMarker

Bases: SceneElement

Coordinates Marker Component.

Args:

position: A list of 3 numbers representing the position. rotation: A list of 3 numbers representing the rotation. matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. scale: 1.0 headScale: 1.0 lod: Level of detail. The number of segments for the cone and the stem.

tag: str = 'CoordsMarker'

class vuer.schemas.scene_components.Arrow

Bases: SceneElement

Coordinates Marker Component.

Args:

matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. position: A list of 3 numbers representing the position. rotation: A list of 3 numbers representing the rotation. scale: 1.0 headScale: 1.0 lod: Level of detail. The number of segments for the cone and the stem.

tag: str = 'Arrow'

class vuer.schemas.scene_components.Ply

Bases: SceneElement

tag: str = 'Ply'

class vuer.schemas.scene_components.Glb

Bases: SceneElement

Glb Component

# this follows the material type

Parameters:

• materialType – Literal[“basic”, …]

• material –

  { side=0: inside, side=1: outsie, side=2: both.

}

tag: str = 'Glb'

class vuer.schemas.scene_components.Urdf

Bases: SceneElement

tag: str = 'Urdf'

class vuer.schemas.scene_components.Gripper

Bases: SceneElement

tag: str = 'Gripper'

class vuer.schemas.scene_components.SkeletalGripper

Bases: SceneElement

tag: str = 'SkeletalGripper'

class vuer.schemas.scene_components.Grid

Bases: SceneElement

tag: str = 'Grid'

class vuer.schemas.scene_components.GrabRender

Bases: SceneElement

tag: str = 'GrabRender'

key = 'DEFAULT'

We do not want the client to set keys automatically since GrabRender is usually used a singleton component as default.

class vuer.schemas.scene_components.TimelineControls

Bases: SceneElement

tag: str = 'TimelineControls'

class vuer.schemas.scene_components.PointerControls

Bases: SceneElement

tag: str = 'PointerControls'

class vuer.schemas.scene_components.Scene

Bases: BlockElement

tag: str = 'Scene'

serialize()

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.scene_components.DefaultScene

Bases: Scene

Default Scene that includes a basic setup of ambient lights.

Parameters:

• children (SceneElement, ...) – list of children elements to be rendered in the scene.

• rawChildren – list of children elements to be rendered in the scene.

• htmlChildren – list of children elements to be rendered in the scene.

• bgChildren – list of children elements to be rendered in the scene.

• show_helper – list of children elements to be rendered in the scene.

• startStep – list of children elements to be rendered in the scene.

• endStep – list of children elements to be rendered in the scene.

• up – list of children elements to be rendered in the scene.

• kwargs – list of children elements to be rendered in the scene.

Example Usage:

DefaultScene(
    # Ambient Light does not have helper because it is ambient.
    AmbientLight(intensity=1.0, key="default_ambient_light"),
    DirectionalLight(
        intensity=1, key="default_directional_light", helper=show_helper
    ),
    *children,
    rawChildren=rawChildren,
    htmlChildren=htmlChildren,
    bgChildren=[
        GrabRender(),
        *[
            # we use a key here so that we can replace the timeline controls via update
            TimelineControls(start=startStep, end=endStep, key="timeline")
            if endStep
            else None,
        ],
        PointerControls(),
        Grid(),
        *bgChildren,
    ],
    up=up,
    **kwargs,
)

Drei Components

class vuer.schemas.drei_components.SceneElement

Bases: BlockElement

class vuer.schemas.drei_components.Html

Bases: SceneElement

tag: str = 'Html'

class vuer.schemas.drei_components.Splat

Bases: SceneElement

tag: str = 'Splat'

class vuer.schemas.drei_components.Line

Bases: SceneElement

Renders a THREE.Line2 or THREE.LineSegments2 (depending on the value of segments).

Usage:

<Line
  points={[[0, 0, 0], ...]}       // Array of points, Array<Vector3 | Vector2 | [number, number, number] | [number, number] | number>
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  segments                        // If true, renders a THREE.LineSegments2. Otherwise, renders a THREE.Line2
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>

tag: str = 'Line'

class vuer.schemas.drei_components.QuadraticBezierLine

Bases: SceneElement

Renders a THREE.Line2 using THREE.QuadraticBezierCurve3 for interpolation.

Usage:

<QuadraticBezierLine
  start={[0, 0, 0]}               // Starting point, can be an array or a vec3
  end={[10, 0, 10]}               // Ending point, can be an array or a vec3
  mid={[5, 0, 5]}                 // Optional control point, can be an array or a vec3
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>

You can also update the line runtime.

code:

const ref = useRef()
useFrame((state) => {
  ref.current.setPoints(
    [0, 0, 0],
    [10, 0, 0],
    // [5, 0, 0] // Optional: mid-point
  )
}, [])
return <QuadraticBezierLine ref={ref} />
}

tag: str = 'QuadraticBezierLine'

class vuer.schemas.drei_components.CubicBezierLine

Bases: SceneElement

Renders a THREE.Line2 using THREE.CubicBezierCurve3 for interpolation.

Usage:

<CubicBezierLine
  start={[0, 0, 0]}               // Starting point
  end={[10, 0, 10]}               // Ending point
  midA={[5, 0, 0]}                // First control point
  midB={[0, 0, 5]}                // Second control point
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>

tag: str = 'CubicBezierLine'

class vuer.schemas.drei_components.CatmullRomLine

Bases: SceneElement

Renders a THREE.Line2 using THREE.CatmullRomCurve3 for interpolation.

Usage:

<CatmullRomLine
  points={[[0, 0, 0], ...]}       // Array of Points
  closed={false}                  // Default
  curveType="centripetal"         // One of "centripetal" (default), "chordal", or "catmullrom"
  tension={0.5}                   // Default (only applies to "catmullrom" curveType)
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>

tag: str = 'CatmullRomLine'

class vuer.schemas.drei_components.Facemesh

Bases: SceneElement

Renders an oriented MediaPipe face mesh:

Parameters:

• points (MediaPipePoints, optional) – An array of 468+ keypoints as returned by google/mediapipe tasks-vision. Defaults to a sample face.

• face (MediaPipeFaceMesh, optional) – An face object as returned by tensorflow/tfjs-models face-landmarks-detection. This parameter is deprecated.

• width (int, optional) – Constant width of the mesh. Defaults to undefined.

• height (int, optional) – Constant height of the mesh. Defaults to undefined.

• depth (int, optional) – Constant depth of the mesh. Defaults to 1.

• verticalTri (Tuple[int, int, int], optional) – A landmarks tri supposed to be vertical. Defaults to [159, 386, 200]. See: https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection#mediapipe-facemesh-keypoints

• origin (Union[int, THREE.Vector3], optional) – A landmark index (to get the position from) or a vec3 to be the origin of the mesh. Defaults to undefined (i.e., the bbox center).

• facialTransformationMatrix (FacemeshDatas.SAMPLE_FACELANDMARKER_RESULT.facialTransformationMatrixes[0], optional) – A facial transformation matrix, as returned by FaceLandmarkerResult.facialTransformationMatrixes. See: https://developers.google.com/mediapipe/solutions/vision/face_landmarker/web_js#handle_and_display_results

• offset (bool, optional) – Apply position offset extracted from facialTransformationMatrix.

• offsetScalar (float, optional) – Offset sensitivity factor, less is more sensible.

• faceBlendshapes (FacemeshDatas.SAMPLE_FACELANDMARKER_RESULT.faceBlendshapes[0], optional) – Face blendshapes, as returned by FaceLandmarkerResult.faceBlendshapes. See: https://developers.google.com/mediapipe/solutions/vision/face_landmarker/web_js#handle_and_display_results

• eyes (bool, optional) – Whether to enable eyes (note: faceBlendshapes is required for this). Defaults to True.

Usage:

const faceLandmarkerResult = {
    "faceLandmarks": [
      [
        { "x": 0.5760777592658997, "y": 0.8639070391654968, "z": -0.030997956171631813 },
        { "x": 0.572094738483429, "y": 0.7886289358139038, "z": -0.07189624011516571 },
        // ...
      ],
      // ...
    ],
    "faceBlendshapes": [
      // ...
    ],
    "facialTransformationMatrixes": [
      // ...
    ]
  },
}

const points = faceLandmarkerResult.faceLandmarks[0]

<Facemesh points={points} />

tag: str = 'Facemesh'

Physics Components

class vuer.schemas.physics_components.SceneElement

Bases: BlockElement

class vuer.schemas.physics_components.MuJoCoProvider

Bases: SceneElement

tag: str = 'MuJoCoProvider'

class vuer.schemas.physics_components.MuJoCo

Bases: SceneElement

tag: str = 'MuJoCo'

src: str

workDir: str = None

assets: List[str] = []

speed: float = 1.0

pause: bool = False

fps: int = 60

timeout: float = 0.0

threshold: float = 0.001

keyFrames: List[str] = []

selfProvide = True

useLights = True

useMocap = True

gizmoScale = 0.3

class vuer.schemas.physics_components.HandActuator

Bases: SceneElement

tag: str = 'HandActuator'

ctrlId = -1

offset = 0.01

low = 0.01

high = 1.0

cond = 'right-squeeze'

value = 'right:thumb-tip,right:index-finger-tip'

scale = 1.0

class vuer.schemas.physics_components.MotionControllerActuator

Bases: SceneElement

MotionControllerActuator component for actuating the MuJoCo simulation based on motion controller inputs.

Parameters:

• ctrlId (int) – The control ID in the MuJoCo simulation to actuate.

• low (float) – The minimum value for actuation.

• high (float) – The maximum value for actuation.

• cond (str) – The condition for actuation, e.g., ‘right-trigger’.

• scale (float) – The scaling factor applied to the input value for actuation.

tag: str = 'MotionControllerActuator'

ctrlId: int = -1

low: float = 0.0

high: float = 1.0

cond: str = 'right-trigger'

scale: float = 1.0