1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on performance data form the Real Environment and have a form that is suitable for interpretation (e.g., Face and Gestures) and immediate reproduction in the Virtual Environment (digital twin) in addition to providing Descriptors to determine cue points.

2. Functional Requirements

Performance Description AIM generates descriptors that are based on data form the Real Environment and have a form that is suitable for interpretation (e.g., Face and Gestures) and immediate reproduction in the Virtual Environment (digital twin) in addition to providing Descriptors to determine cue points.

  1. Visual behaviour
    1. Performers attributes
      1. Position and Orientation (Spatial Attitude, see MPAI-OSD)
      2. Face and Gestures (See MPAI-PAF)
    2. Scene attributes
      1. Position and Orientation (Spatial Attitude) of set pieces
      2. The scene as volume (Volumetric, e.g., point cloud)
    3. Audio behaviour
      1. Performers are captured by one or more microphones (on body/stage).
      2. Descriptors on a per-performer basis:
        1. Text uttered by performer.
        2. Performer’s audio activity:
          1. Speaking
          2. Laughing
          3. Clapping
          4. Booing
          5. Shouting
          6. Singing
        3. Object data:
          1. The scene as volume or mesh (Volumetric, e.g., point cloud).
          2. Position and Orientation (Spatial Attitude) of set pieces.
          3. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).
        4. Biometric data:
          1. Heart rate and Heart rate variability (HRV).
          2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
          3. Galvanic Skin Response (Electrodermal Activity).
          4. Myoelectric intensity per electrode site.
          5. Skin temperature.

3. Syntax

4. Semantics

Label Size Description
Header N1 Bytes Header
– Standard- 9 Bytes The characters “XRV-RFD-V”
– Version N2 Bytes Major version – 1 or 2 characters
– Dot-separator 1 Byte The character “.”
– Subversion N3 Bytes Minor version – 1 or 2 characters
MInstanceID N4 Bytes Identifier of M-Instance.
ID N5 Bytes Identifier of
SpaceTime N7 Bytes  Space-Time info of CogState.