1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on RE Data In from Performers, Objects, and Controllers in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data). Performers may be captured by MoCap, one or more video cameras, and one or more microphones.

2. Functional Requirements

Performance Description AIM generates descriptors conveying information on:

  1. The scene as volume or mesh (Volumetric, e.g., point cloud).
  2.  Performers
    1. Visual Description
      1. Performers attributes
        1. Position and Orientation
        2. Face and Gestures
    2. Audio behaviour
      1. Performers are captured by one or more microphones (on body/stage).
      2. Descriptors on a per-performer basis:
        1. Text uttered by performer.
        2. Performer’s audio activity:
          1. Speaking
          2. Laughing
          3. Clapping
          4. Booing
          5. Shouting
          6. Singing
    3. Biometric data:
      1. Heart rate and Heart rate variability (HRV).
      2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
      3. Galvanic Skin Response (Electrodermal Activity).
      4. Myoelectric intensity per electrode site.
      5. Skin temperature.
  3. Objects:
    1. Position and Orientation (Spatial Attitude) of set pieces.
    2. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).

3. Syntax

https://schemas.mpai.community/XRV1/V1.0/data/REPerformanceDescriptors.json

4. Semantics

Label Description
Header RE Performance Descriptors Header
– Standard-REPerformanceDescriptors The characters “XRV-RFD-V”
– Version Major version – 1 or 2 characters
– Dot-separator The character “.”
– Subversion Minor version – 1 or 2 characters
SpaceTime  Space-Time info of RE Performance Descriptors
PerformanceDescriptions[] Collection of Performance Descriptions.
– AudioVisual Scene The Audio-Visual Scene of the stage.
– PerformerDescription[] Set of Performer Descriptions,
  – PerformerID ID of Performer.
  – SpatialAttitude Of Performer.
  – Descriptors Set of Descriptors
    – FaceDescriptors Of Performer.
    – BodyDescriptors Of Performer.
    – UtteredSpeech Of Performer.
    – VisualActivity Boolean: 1=moving, 0=not moving
    – AudioActivity Boolean: 1=making sound, 0=not making sound
    – BiometricObject Of Performer
– OtherObjects[] Set of Descriptors of other Objects.
  – ObjectID oneOf: knob, slider, button
  – Spatial Attitude Object Spatial Attitude.
  – Object variables Type of variable is defined by RE Venue Specification and signaling by DMX/Midi.
– ControllerData From Performer
DescrMetadata Descriptive Metadata