1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on performance data form the Real Environment and have a form that is suitable for interpretation (e.g., Face and Gestures) and immediate reproduction in the Virtual Environment (digital twin) in addition to providing Descriptors to determine cue points.

2. Functional Requirements

  1. Visual behaviour
    1. Performers attributes
      1. Position and Orientation (Spatial Attitude, see MPAI-OSD)
      2. Face and Gestures (See MPAI-PAF)
    2. Scene attributes
      1. Position and Orientation (Spatial Attitude) of set pieces
      2. The scene as volume (Volumetric, e.g., point cloud)
    3. Audio behaviour
      1. Performers are captured by one or more microphones (on body/stage).
      2. Descriptors on a per-performer basis:
        1. Text uttered by performer.
        2. Performer’s audio activity:
          1. Speaking
          2. Laughing
          3. Clapping
          4. Booing
          5. Shouting
          6. Singing
        3. Object data:
          1. The scene as volume or mesh (Volumetric, e.g., point cloud).
          2. Position and Orientation (Spatial Attitude) of set pieces.
          3. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).
        4. Biometric data:
          1. Heart rate and Heart rate variability (HRV).
          2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
          3. Galvanic Skin Response (Electrodermal Activity).
          4. Myoelectric intensity per electrode site.
          5. Skin temperature.

3. Syntax

4. Semantics