1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on RE Data In from Performers, Objects, and Controllers in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data). Performers may be captured by MoCap, one or more video cameras, and one or more microphones.

2. Functional Requirements

Performance Description AIM generates descriptors conveying information on:

  1. The scene as volume or mesh (Volumetric, e.g., point cloud).
  2. Performers
    1. Visual Description
      1. Performers attributes
        1. Position and Orientation
        2. Face and Gestures
      2. Scene attributes
        1. Position and Orientation of set pieces
        2. The scene as volume (Volumetric, e.g., point cloud)
    2. Audio behaviour
      1. Performers are captured by one or more microphones (on body/stage).
      2. Descriptors on a per-performer basis:
        1. Text uttered by performer.
        2. Performer’s audio activity:
          1. Speaking
          2. Laughing
          3. Clapping
          4. Booing
          5. Shouting
          6. Singing
      3. Biometric data:
        1. Heart rate and Heart rate variability (HRV).
        2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
        3. Galvanic Skin Response (Electrodermal Activity).
        4. Myoelectric intensity per electrode site.
        5. Skin temperature.
  3. Objects:
    2. Position and Orientation (Spatial Attitude) of set pieces.
    3. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).

3. Syntax

https://schemas.mpai.community/XRV1/V1.0/data/REPerformanceDescriptors.json

4. Semantics

Label Size Description
Header N1 Bytes RE Performance Descriptors Header
– Standard-REPerformanceDescriptors 9 Bytes The characters “XRV-RFD-V”
– Version N2 Bytes Major version – 1 or 2 characters
– Dot-separator 1 Byte The character “.”
– Subversion N3 Bytes Minor version – 1 or 2 characters
SpaceTime N6 Bytes  Space-Time info of RE Performance Descriptors
PerformanceDescriptions[] N7 Bytes Collection of Performance Descriptions.
– Scene N8 Bytes The Visual Scene
– PerformerDescription[] N9 Bytes Set of Performance Descriptions,
  – PerformerID N10 Bytes ID of Performer.
  – SpatialAttitude N11 Bytes Of Performer.
  – Descriptors N12 Bytes Set of Descriptors
    – FaceDescriptors N13 Bytes Of Performer.
    – BodyDescriptors N14 Bytes Of Performer.
    – UtteredSpeech 1 bit Of Performer.
    – VisualActivity 1 bit Moving/not moving
    – AudioActivity N15 Bytes Making sound/not making sound
    – BiometricData N16 Bytes Of Performer
– OtherObjects[] N17 Bytes Set of Descriptors of other Objects.
  – ObjectID N18 Bytes oneOf: knob, slider, button
  – Spatial Attitude N19 Bytes Object Spatial Attitude.
  – Object variables N20 Bytes Type of variable is defined by RE Venue Specification and signaling by DMX/Midi.
– ControllerData N21 Bytes From Performer
DescrMetadata N22 Bytes Descriptive Metadata