1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on RE Data In from Performers, Objects, and Controllers in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data). Performers may be captured by MoCap, one or more video cameras, and one or more microphones.

2. Functional Requirements

Performance Description AIM generates descriptors conveying information on:

  1. Visual Description
    1. Performers attributes
      1. Position and Orientation
      2. Face and Gestures
    2. Scene attributes
      1. Position and Orientation of set pieces
      2. The scene as volume (Volumetric, e.g., point cloud)
  2. Audio behaviour
    1. Performers are captured by one or more microphones (on body/stage).
    2. Descriptors on a per-performer basis:
      1. Text uttered by performer.
      2. Performer’s audio activity:
        1. Speaking
        2. Laughing
        3. Clapping
        4. Booing
        5. Shouting
        6. Singing
  3. Object data:
    1. The scene as volume or mesh (Volumetric, e.g., point cloud).
    2. Position and Orientation (Spatial Attitude) of set pieces.
    3. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).
  4. Biometric data:
    1. Heart rate and Heart rate variability (HRV).
    2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
    3. Galvanic Skin Response (Electrodermal Activity).
    4. Myoelectric intensity per electrode site.
    5. Skin temperature.

3. Syntax

4. Semantics

Label Size Description
Header N1 Bytes RE Participant Descriptors Header
– Standard-REPerformanceDescriptors 9 Bytes The characters “XRV-RFD-V”
– Version N2 Bytes Major version – 1 or 2 characters
– Dot-separator 1 Byte The character “.”
– Subversion N3 Bytes Minor version – 1 or 2 characters
SpaceTime N6 Bytes  Space-Time info of RE Performance Descriptors
PerformanceDescriptions[] N7 Bytes Collection of Performance Descriptions.
– PerformerDescription N8 Bytes Set of Performance Descriptions,
  – PerformerID N9 Bytes ID of Performer.
  – SpatialAttitude N10 Bytes Of Performer.
  – Descriptors N11 Bytes Set of Descriptors
    – FaceDescriptors N12 Bytes Of Performer.
    – BodyDescriptors N13 Bytes Of Performer.
    – UtteredSpeech N14 Bytes Of Performer.
    – VisualActivity 1 bit Moving/not moving
    – AudioActivity 1 bit Making sound/not making sound
    – BiometricData N15 Bytes Of Participant
– OtherObjects[] N16 Bytes Set of Descriptors of other Objects.
  – ObjectDescription N17 Bytes
    – ObjectID N18 Bytes one of: knob, slider, button
    – Spatial Attitude N19 Bytes
    – SpatialAttitude N20 Bytes
– ControllerData N21 Bytes From Participant
– AppData N22 Bytes From Participant
DescrMetadata N23 Bytes Descriptive Metadata