Component Library
vuer.schemas#

vuer.schema contains the schemas for the various components of the Vuer system. These schemas are used to validate the data that is passed to the various components of the system.

For detailed view of how these components are implemented, please refer to the typescript source code at https://github.com/vuer-ai/vuer-ts/tree/master/src/schemas.

Example Usage:

from vuer import Vuer
from vuer.schemas import DefaultScene, Sphere

vuer = Vuer()
vuer.set @ DefaultScene(up=[0, 1, 0])
vuer.upsert @ Sphere(args=[0.1, 20, 20], position=[0, 0.1, 0], key="sphere")

HTML Components#

class vuer.schemas.html_components.BytesIO#

Bases: _BufferedIOBase

Buffered I/O implementation using an in-memory bytes buffer.

__new__(**kwargs)#
close()#

Disable all I/O operations.

closed#

True if the file is closed.

flush()#

Does nothing.

getbuffer()#

Get a read-write view over the contents of the BytesIO object.

getvalue()#

Retrieve the entire contents of the BytesIO object.

isatty()#

Always returns False.

BytesIO objects are not connected to a TTY-like device.

read(size=-1, /)#

Read at most size bytes, returned as a bytes object.

If the size argument is negative, read until EOF is reached. Return an empty bytes object at EOF.

read1(size=-1, /)#

Read at most size bytes, returned as a bytes object.

If the size argument is negative or omitted, read until EOF is reached. Return an empty bytes object at EOF.

readable()#

Returns True if the IO object can be read.

readinto(buffer, /)#

Read bytes into buffer.

Returns number of bytes read (0 for EOF), or None if the object is set not to block and has no data to read.

readline(size=-1, /)#

Next line from the file, as a bytes object.

Retain newline. A non-negative size argument limits the maximum number of bytes to return (an incomplete line may be returned then). Return an empty bytes object at EOF.

readlines(size=None, /)#

List of bytes objects, each a line from the file.

Call readline() repeatedly and return a list of the lines so read. The optional size argument, if given, is an approximate bound on the total number of bytes in the lines returned.

seek(pos, whence=0, /)#

Change stream position.

Seek to byte offset pos relative to position indicated by whence:

0 Start of stream (the default). pos should be >= 0; 1 Current position - pos may be negative; 2 End of stream - pos usually negative.

Returns the new absolute position.

seekable()#

Returns True if the IO object can be seeked.

tell()#

Current file position, an integer.

truncate(size=None, /)#

Truncate the file to at most size bytes.

Size defaults to the current file position, as returned by tell(). The current file position is unchanged. Returns the new size.

writable()#

Returns True if the IO object can be written.

write(b, /)#

Write bytes to file.

Return the number of bytes written.

writelines(lines, /)#

Write lines to the file.

Note that newlines are not added. lines can be any iterable object producing bytes-like objects. This is equivalent to calling write() for each element.

class vuer.schemas.html_components.Element[source]#

Bases: object

Base class for all elements

tag: str = 'div'#
serialize()[source]#

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.html_components.BlockElement[source]#

Bases: Element

serialize()[source]#

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.html_components.AutoScroll[source]#

Bases: BlockElement

tag: str = 'AutoScroll'#
class vuer.schemas.html_components.Markdown[source]#

Bases: BlockElement

tag: str = 'Markdown'#
class vuer.schemas.html_components.Page[source]#

Bases: BlockElement

A Page is an element that contains other elements. It is represented by a div element in the DOM.

tag: str = 'article'#
class vuer.schemas.html_components.div[source]#

Bases: BlockElement

tag: str = 'Div'#
class vuer.schemas.html_components.InputBox[source]#

Bases: Element

An InputBox is an element that allows the user to input text. It is represented by an input element in the DOM.

tag: str = 'Input'#
class vuer.schemas.html_components.Header1[source]#

Bases: BlockElement

A Text element is an element that displays text. It is represented by a text, or p element in the DOM.

tag: str = 'h1'#
vuer.schemas.html_components.Header#

alias of Header1

class vuer.schemas.html_components.Header2[source]#

Bases: Header1

Header 2

tag: str = 'h2'#
class vuer.schemas.html_components.Header3[source]#

Bases: Header1

Header 2

tag: str = 'h3'#
class vuer.schemas.html_components.Paragraph[source]#

Bases: BlockElement

A Text element is an element that displays text. It is represented by a text, or p element in the DOM.

tag: str = 'p'#
class vuer.schemas.html_components.Text[source]#

Bases: Element

A Text element is an element that displays text. It is represented by a text, or p element in the DOM.

tag: str = 'Text'#
class vuer.schemas.html_components.Bold[source]#

Bases: Text

class vuer.schemas.html_components.Italic[source]#

Bases: Text

Bases: Text

tag: str = 'a'#
class vuer.schemas.html_components.Button[source]#

Bases: Element

A Button element is an element that allows the user to click on it. It is represented by a button element in the DOM.

tag: str = 'Button'#
class vuer.schemas.html_components.Slider[source]#

Bases: Element

A Slider element is an element that allows the user to slide a value. It is represented by a slider element in the DOM.

tag: str = 'Slider'#
class vuer.schemas.html_components.Image[source]#

Bases: Element

An Image element is an element that displays an image. It is represented by an img element in the DOM.

tag: str = 'Img'#
class vuer.schemas.html_components.ImageUpload[source]#

Bases: Element

A ImageUpload element is an element that allows the user to upload a file. It is represented by a file upload element in the DOM.

tag: str = 'ImageUpload'#

3D Scene Components#

class vuer.schemas.scene_components.BlockElement[source]#

Bases: Element

serialize()[source]#

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.scene_components.Image[source]#

Bases: Element

An Image element is an element that displays an image. It is represented by an img element in the DOM.

tag: str = 'Img'#
class vuer.schemas.scene_components.Element[source]#

Bases: object

Base class for all elements

tag: str = 'div'#
serialize()[source]#

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.scene_components.Scene[source]#

Bases: BlockElement

tag: str = 'Scene'#
serialize()[source]#

Serialize the element to a dictionary for sending over the websocket.

Returns:

Dictionary representing the element.

class vuer.schemas.scene_components.SceneElement[source]#

Bases: BlockElement

class vuer.schemas.scene_components.Frustum[source]#

Bases: SceneElement

Camera Frustum

Parameters:
  • position (tuple[float, float, float]) – An optional tuple of three numbers representing the position.

  • rotation (tuple[float, float, float]) – An optional tuple of three numbers representing the rotation.

  • matrix (tuple[float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float]) – An optional tuple of sixteen numbers representing the matrix.

  • aspect (float) – An optional number representing the aspect.

  • focus (float) – An optional number representing the focus.

  • fov (float) – An optional number representing the field of view.

  • near (float) – An optional number representing the near field.

  • far (float) – An optional number representing the far field.

  • scale (float) – An optional number representing the scale.

  • upScale (float) – An optional number representing the up scale.

  • focalLength (float) – An optional number representing the focal length.

  • showUp (bool) – An optional boolean indicating whether to show up.

  • showFrustum (bool) – An optional boolean indicating whether to show the frustum.

  • showFocalPlane (bool) – An optional boolean indicating whether to show the focal plane.

  • showImagePlane (bool) – An optional boolean indicating whether to show the image plane.

  • src (str) – An optional string representing the source.

  • colorOrigin (ColorRepresentation) – An optional ColorRepresentation for the origin color.

  • colorFrustum (ColorRepresentation) – An optional ColorRepresentation for the frustum color.

  • colorCone (ColorRepresentation) – An optional ColorRepresentation for the cone color.

  • colorFocalPlane (ColorRepresentation) – An optional ColorRepresentation for the focal plane color.

  • colorUp (ColorRepresentation) – An optional ColorRepresentation for the up color.

  • colorTarget (ColorRepresentation) – An optional ColorRepresentation for the target color.

  • colorCross (ColorRepresentation) – An optional ColorRepresentation for the cross color.

tag: str = 'Frustum'#
class vuer.schemas.scene_components.CameraHelper[source]#

Bases: SceneElement

tag: str = 'CameraHelper'#
class vuer.schemas.scene_components.group[source]#

Bases: SceneElement

tag: str = 'group'#
children = []#
class vuer.schemas.scene_components.mesh[source]#

Bases: SceneElement

tag: str = 'mesh'#
children = []#
class vuer.schemas.scene_components.TriMesh[source]#

Bases: SceneElement

tag: str = 'TriMesh'#
children = []#
vertices: ndarray[Any, dtype[float16]] = None#
faces: ndarray[Any, dtype[uint32]] = None#
colors: ndarray[Any, dtype[uint8]] = None#
uv: ndarray[Any, dtype[float16]] = None#
class vuer.schemas.scene_components.PointCloud[source]#

Bases: SceneElement

PointCould element, highly optimized for payload size and speed.

Parameters:
  • vertices (NDArray[np.float16]) – An optional numpy array of shape (N, 3) containing the vertices of the pointcloud.

  • colors – An optional numpy array of shape (N, 3) containing the colors of the point cloud.

  • size (float) – An optional float that sets the size of the points.

  • key (str) – str An optional string that sets the key of the element.

Usage:

sess.upsert @ PointCloud(
    vertices=np.random.rand(1000, 3),
    colors=np.random.rand(1000, 3),
    size=0.01,
    key="pointcloud",
)
tag: str = 'PointCloud'#
vertices: ndarray[Any, dtype[float16]] = None#

An optional numpy array of shape (N, 3) containing the vertices of the point cloud.

colors: ndarray[Any, dtype[uint8]] = None#

An optional numpy array of shape (N, 3) containing the colors of the point cloud.

children = []#
vuer.schemas.scene_components.p#

alias of PointCloud

class vuer.schemas.scene_components.Box[source]#

Bases: SceneElement

tag: str = 'Box'#
class vuer.schemas.scene_components.Capsule[source]#

Bases: SceneElement

tag: str = 'Capsule'#
class vuer.schemas.scene_components.Cone[source]#

Bases: SceneElement

tag: str = 'Cone'#
class vuer.schemas.scene_components.Circle[source]#

Bases: SceneElement

tag: str = 'Circle'#
class vuer.schemas.scene_components.Cylinder[source]#

Bases: SceneElement

tag: str = 'Cylinder'#
class vuer.schemas.scene_components.Dodecahedron[source]#

Bases: SceneElement

tag: str = 'Dodecahedron'#
class vuer.schemas.scene_components.Edges[source]#

Bases: SceneElement

tag: str = 'Edges'#
class vuer.schemas.scene_components.Extrude[source]#

Bases: SceneElement

tag: str = 'Extrude'#
class vuer.schemas.scene_components.Icosahedron[source]#

Bases: SceneElement

tag: str = 'Icosahedron'#
class vuer.schemas.scene_components.Lathe[source]#

Bases: SceneElement

tag: str = 'Lathe'#
class vuer.schemas.scene_components.Octahedron[source]#

Bases: SceneElement

tag: str = 'Octahedron'#
class vuer.schemas.scene_components.Plane[source]#

Bases: SceneElement

tag: str = 'Plane'#
class vuer.schemas.scene_components.Polyhedron[source]#

Bases: SceneElement

tag: str = 'Polyhedron'#
class vuer.schemas.scene_components.Ring[source]#

Bases: SceneElement

tag: str = 'Ring'#
class vuer.schemas.scene_components.Shape[source]#

Bases: SceneElement

tag: str = 'Shape'#
class vuer.schemas.scene_components.Sphere[source]#

Bases: SceneElement

tag: str = 'Sphere'#
class vuer.schemas.scene_components.Tetrahedron[source]#

Bases: SceneElement

tag: str = 'Tetrahedron'#
class vuer.schemas.scene_components.Torus[source]#

Bases: SceneElement

tag: str = 'Torus'#
class vuer.schemas.scene_components.TorusKnot[source]#

Bases: SceneElement

tag: str = 'TorusKnot'#
class vuer.schemas.scene_components.Tube[source]#

Bases: SceneElement

tag: str = 'Tube'#
class vuer.schemas.scene_components.Fog[source]#

Bases: SceneElement

Fog is a scene element that adds fog to the scene. This can be used to approximate depth.

Args:

color: The color of the fog. near: The distance to the near plane. far: The distance to the far plane.

Example Usage:

Fog(color=”green”, near=3, far=7)

tag: str = 'fog'#
class vuer.schemas.scene_components.Wireframe[source]#

Bases: SceneElement

tag: str = 'Wireframe'#
class vuer.schemas.scene_components.Splat[source]#

Bases: SceneElement

tag: str = 'Splat'#
class vuer.schemas.scene_components.LumaSplats[source]#

Bases: SceneElement

tag: str = 'Splats'#
class vuer.schemas.scene_components.Pcd[source]#

Bases: SceneElement

tag: str = 'Pcd'#
class vuer.schemas.scene_components.CameraView[source]#

Bases: SceneElement

CameraView for rendering from arbitrary camera poses.

Parameters:
  • fov (float, optional) – The vertical field of view of the camera. Defaults to 50.

  • width (int, optional) – The width of the camera image. Defaults to 320.

  • height (int, optional) – The height of the camera image. Defaults to 240.

  • key (str, optional) – The key of the camera view. Defaults to “ego”.

  • position (List[float], optional) – The position of the camera. Defaults to [0, 0, 0].

  • rotation (List[float], optional) – The rotation of the camera. Defaults to [0, 0, 0]

  • stream (str, optional) – The stream of the camera. Defaults to “ondemand”.

  • fps (int, optional) – The frames per second of the camera. Defaults to 30.

  • near (float, optional) – The near field of the camera. Defaults to 0.1.

  • far (float, optional) – The far field of the camera. Defaults to 20.

  • renderDepth (bool, optional) – Whether to render depth. If set, returns value[“depthFrame”]. Defaults to True.

  • showFrustum (bool, optional) – Whether to show the frustum. Defaults to True.

  • downsample (int, optional) – The downsample rate. Defaults to 1.

  • distanceToCamera (float, optional) – The distance to the camera. Defaults to 2.

tag: str = 'CameraView'#
class vuer.schemas.scene_components.SceneBackground[source]#

Bases: Image, SceneElement

Sets the background of the scene to a static image. Does not work well with high frame rates. For displaying movies, use the ImageBackground element.

tag: str = 'SceneBackground'#
class vuer.schemas.scene_components.ImageBackground[source]#

Bases: Image, SceneElement

Sets the background of the scene to an image, Supports high frame rates.

We use a plane that is always facing the camera to display the image.

tag: str = 'ImageBackground'#
class vuer.schemas.scene_components.Gamepads[source]#

Bases: SceneElement

tag: str = 'Gamepads'#
class vuer.schemas.scene_components.DirectionalLight[source]#

Bases: SceneElement

tag: str = 'DirectionalLight'#
class vuer.schemas.scene_components.PointLight[source]#

Bases: SceneElement

tag: str = 'PointLight'#
class vuer.schemas.scene_components.SpotLight[source]#

Bases: SceneElement

tag: str = 'SpotLight'#
class vuer.schemas.scene_components.AmbientLight[source]#

Bases: SceneElement

tag: str = 'AmbientLight'#
class vuer.schemas.scene_components.Html[source]#

Bases: SceneElement

Usage:

as='div' // Wrapping element (default: 'div')
wrapperClass // The className of the wrapping element (default: undefined)
prepend // Project content behind the canvas (default: false)
center // Adds a -50%/-50% css transform (default: false) [ignored in transform mode]
fullscreen // Aligns to the upper-left corner, fills the screen (default:false) [ignored in transform mode]
distanceFactor={10} // If set (default: undefined), children will be scaled by this factor, and also by distance to a PerspectiveCamera / zoom by a OrthographicCamera.
zIndexRange={[100, 0]} // Z-order range (default=[16777271, 0])
portal={domnodeRef} // Reference to target container (default=undefined)
transform // If true, applies matrix3d transformations (default=false)
sprite // Renders as sprite, but only in transform mode (default=false)
calculatePosition={(el: Object3D, camera: Camera, size: { width: number; height: number }) => number[]} // Override default positioning function. (default=undefined) [ignored in transform mode]
occlude={[ref]} // Can be true or a Ref<Object3D>[], true occludes the entire scene (default: undefined)
onOcclude={(visible) => null} // Callback when the visibility changes (default: undefined)
{...groupProps} // All THREE.Group props are valid
{...divProps} // All HTMLDivElement props are valid
tag: str = 'Html'#
class vuer.schemas.scene_components.Pivot[source]#

Bases: SceneElement

tag: str = 'Pivot'#
class vuer.schemas.scene_components.Movable[source]#

Bases: SceneElement

tag: str = 'Movable'#
class vuer.schemas.scene_components.Hands[source]#

Bases: SceneElement

tag: str = 'Hands'#
class vuer.schemas.scene_components.Obj[source]#

Bases: SceneElement

tag: str = 'Obj'#
class vuer.schemas.scene_components.CoordsMarker[source]#

Bases: SceneElement

Coordinates Marker Component.

Args:

matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. position: A list of 3 numbers representing the position. rotation: A list of 3 numbers representing the rotation. matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. scale: 1.0 headScale: 1.0 lod: Level of detail. The number of segments for the cone and the stem.

tag: str = 'CoordsMarker'#
class vuer.schemas.scene_components.Arrow[source]#

Bases: SceneElement

Coordinates Marker Component.

Args:

matrix: A list of 16 numbers representing the matrix. Overrides position and rotation. position: A list of 3 numbers representing the position. rotation: A list of 3 numbers representing the rotation. scale: 1.0 headScale: 1.0 lod: Level of detail. The number of segments for the cone and the stem.

tag: str = 'Arrow'#
class vuer.schemas.scene_components.Ply[source]#

Bases: SceneElement

tag: str = 'Ply'#
class vuer.schemas.scene_components.Glb[source]#

Bases: SceneElement

Glb Component

# this follows the material type :param materialType: Literal[“basic”, …] :param material: {

side=0: inside, side=1: outsie, side=2: both.

}

tag: str = 'Glb'#
class vuer.schemas.scene_components.Urdf[source]#

Bases: SceneElement

tag: str = 'Urdf'#
class vuer.schemas.scene_components.Gripper[source]#

Bases: SceneElement

tag: str = 'Gripper'#
class vuer.schemas.scene_components.SkeletalGripper[source]#

Bases: SceneElement

tag: str = 'SkeletalGripper'#
class vuer.schemas.scene_components.Grid[source]#

Bases: SceneElement

tag: str = 'Grid'#
class vuer.schemas.scene_components.GrabRender[source]#

Bases: SceneElement

tag: str = 'GrabRender'#
key = 'DEFAULT'#

We do not want the client to set keys automatically since GrabRender is usually used a singleton component as default.

class vuer.schemas.scene_components.TimelineControls[source]#

Bases: SceneElement

tag: str = 'TimelineControls'#
class vuer.schemas.scene_components.PointerControls[source]#

Bases: SceneElement

tag: str = 'PointerControls'#
class vuer.schemas.scene_components.DefaultScene[source]#

Bases: Scene

Default Scene that includes a basic setup of ambient lights.

Parameters:
  • children (SceneElement, ...) – list of children elements to be rendered in the scene.

  • rawChildren – list of children elements to be rendered in the scene.

  • htmlChildren – list of children elements to be rendered in the scene.

  • bgChildren – list of children elements to be rendered in the scene.

  • show_helper – list of children elements to be rendered in the scene.

  • startStep – list of children elements to be rendered in the scene.

  • endStep – list of children elements to be rendered in the scene.

  • up – list of children elements to be rendered in the scene.

  • kwargs – list of children elements to be rendered in the scene.

Example Usage:

DefaultScene(
    # Ambient Light does not have helper because it is ambient.
    AmbientLight(intensity=1.0, key="default_ambient_light"),
    DirectionalLight(
        intensity=1, key="default_directional_light", helper=show_helper
    ),
    *children,
    rawChildren=rawChildren,
    htmlChildren=htmlChildren,
    bgChildren=[
        GrabRender(),
        *[
            # we use a key here so that we can replace the timeline controls via update
            TimelineControls(start=startStep, end=endStep, key="timeline")
            if endStep
            else None,
        ],
        PointerControls(),
        Grid(),
        *bgChildren,
    ],
    up=up,
    **kwargs,
)

Drei Components#

class vuer.schemas.drei_components.SceneElement[source]#

Bases: BlockElement

class vuer.schemas.drei_components.Html[source]#

Bases: SceneElement

tag: str = 'Html'#
class vuer.schemas.drei_components.Splat[source]#

Bases: SceneElement

tag: str = 'Splat'#
class vuer.schemas.drei_components.Line[source]#

Bases: SceneElement

Renders a THREE.Line2 or THREE.LineSegments2 (depending on the value of segments).

Usage:

<Line
  points={[[0, 0, 0], ...]}       // Array of points, Array<Vector3 | Vector2 | [number, number, number] | [number, number] | number>
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  segments                        // If true, renders a THREE.LineSegments2. Otherwise, renders a THREE.Line2
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>
tag: str = 'Line'#
class vuer.schemas.drei_components.QuadraticBezierLine[source]#

Bases: SceneElement

Renders a THREE.Line2 using THREE.QuadraticBezierCurve3 for interpolation.

Usage:

<QuadraticBezierLine
  start={[0, 0, 0]}               // Starting point, can be an array or a vec3
  end={[10, 0, 10]}               // Ending point, can be an array or a vec3
  mid={[5, 0, 5]}                 // Optional control point, can be an array or a vec3
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>

You can also update the line runtime.

code:

const ref = useRef()
useFrame((state) => {
  ref.current.setPoints(
    [0, 0, 0],
    [10, 0, 0],
    // [5, 0, 0] // Optional: mid-point
  )
}, [])
return <QuadraticBezierLine ref={ref} />
}
tag: str = 'QuadraticBezierLine'#
class vuer.schemas.drei_components.CubicBezierLine[source]#

Bases: SceneElement

Renders a THREE.Line2 using THREE.CubicBezierCurve3 for interpolation.

Usage:

<CubicBezierLine
  start={[0, 0, 0]}               // Starting point
  end={[10, 0, 10]}               // Ending point
  midA={[5, 0, 0]}                // First control point
  midB={[0, 0, 5]}                // Second control point
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>
tag: str = 'CubicBezierLine'#
class vuer.schemas.drei_components.CatmullRomLine[source]#

Bases: SceneElement

Renders a THREE.Line2 using THREE.CatmullRomCurve3 for interpolation.

Usage:

<CatmullRomLine
  points={[[0, 0, 0], ...]}       // Array of Points
  closed={false}                  // Default
  curveType="centripetal"         // One of "centripetal" (default), "chordal", or "catmullrom"
  tension={0.5}                   // Default (only applies to "catmullrom" curveType)
  color="black"                   // Default
  lineWidth={1}                   // In pixels (default)
  dashed={false}                  // Default
  vertexColors={[[0, 0, 0], ...]} // Optional array of RGB values for each point
  {...lineProps}                  // All THREE.Line2 props are valid
  {...materialProps}              // All THREE.LineMaterial props are valid
/>
tag: str = 'CatmullRomLine'#
class vuer.schemas.drei_components.Facemesh[source]#

Bases: SceneElement

Renders an oriented MediaPipe face mesh:

Parameters:
  • points (MediaPipePoints, optional) – An array of 468+ keypoints as returned by google/mediapipe tasks-vision. Defaults to a sample face.

  • face (MediaPipeFaceMesh, optional) – An face object as returned by tensorflow/tfjs-models face-landmarks-detection. This parameter is deprecated.

  • width (int, optional) – Constant width of the mesh. Defaults to undefined.

  • height (int, optional) – Constant height of the mesh. Defaults to undefined.

  • depth (int, optional) – Constant depth of the mesh. Defaults to 1.

  • verticalTri (Tuple[int, int, int], optional) – A landmarks tri supposed to be vertical. Defaults to [159, 386, 200]. See: https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection#mediapipe-facemesh-keypoints

  • origin (Union[int, THREE.Vector3], optional) – A landmark index (to get the position from) or a vec3 to be the origin of the mesh. Defaults to undefined (i.e., the bbox center).

  • facialTransformationMatrix (FacemeshDatas.SAMPLE_FACELANDMARKER_RESULT.facialTransformationMatrixes[0], optional) – A facial transformation matrix, as returned by FaceLandmarkerResult.facialTransformationMatrixes. See: https://developers.google.com/mediapipe/solutions/vision/face_landmarker/web_js#handle_and_display_results

  • offset (bool, optional) – Apply position offset extracted from facialTransformationMatrix.

  • offsetScalar (float, optional) – Offset sensitivity factor, less is more sensible.

  • faceBlendshapes (FacemeshDatas.SAMPLE_FACELANDMARKER_RESULT.faceBlendshapes[0], optional) – Face blendshapes, as returned by FaceLandmarkerResult.faceBlendshapes. See: https://developers.google.com/mediapipe/solutions/vision/face_landmarker/web_js#handle_and_display_results

  • eyes (bool, optional) – Whether to enable eyes (note: faceBlendshapes is required for this). Defaults to True.

Usage:

const faceLandmarkerResult = {
    "faceLandmarks": [
      [
        { "x": 0.5760777592658997, "y": 0.8639070391654968, "z": -0.030997956171631813 },
        { "x": 0.572094738483429, "y": 0.7886289358139038, "z": -0.07189624011516571 },
        // ...
      ],
      // ...
    ],
    "faceBlendshapes": [
      // ...
    ],
    "facialTransformationMatrixes": [
      // ...
    ]
  },
}

const points = faceLandmarkerResult.faceLandmarks[0]

<Facemesh points={points} />
tag: str = 'Facemesh'#