1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

The focused status data obtained by observing the scene and individual and collective behaviour of Performers in the Real Environment by interpreting the Descriptors, per the current Cue Point, including behaviours, Personal Status (Emotion, Cognitive State, and Social Attitude), as needed by the Action Descriptor Generation AIM to service the current cue point.

2. Functional Requirements

  1. Audio-Visual Scene Descriptors
  2.  Individual
    1. Performer ID
    2. Performer’s visual behaviour:
      1. Visual Activity Activity
        1. Raising arms
        2. Jumping
        3. Pointing to a direction
        4. Gazing at a direction
        5. Dancing
        6. Standing or sitting
        7. Waving
        8. Clapping
      2. Supplementary data
        1. Relevant objects in gaze direction.
        2. Relevant objects in pointing direction.
        3. Dance Object
    3. Performer’s audio behaviour:
      1. Audio Activity: Booing, Clapping, Laughing, Shouting Singing, Speaking
      2. Supplementary data: Intensity, Uttered text
    4. Personal Status (Cognitive State, Emotion, Social Attitude).
  3. Collective
    1. Visual behaviour (performance)
      1. Visual status data
        1. From Proximity Performer A
        2. Gaze: performer A looking at Performer B
      2. Scene attributes
        1. Position and Orientation (Spatial Attitude) of set pieces
        2. The scene as volume (Volumetric, e.g., point cloud)
    2. Audio behaviour (performance)
      1. Status on a per-performer basis:
        1. Text uttered by performer.
        2. Performer’s audio activity:
          1. Performer doing a certain activity, e.g., Laughing, Clapping, Booing, Shouting, Singing
          2. Intensity of Performer’s activity
          3. Particular phrase/text uttered
    3. Personal Status.
  4. Object data:
    1. The scene as volume or mesh (Volumetric, e.g., point cloud).
    2. Position and Orientation (Spatial Attitude) of set pieces.
    3. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).

3. Syntax

https://schemas.mpai.community/XRV1/V1.0/data/REPerformanceStatus.json

4. Semantics

Label Description
Header RE Performance Status Header
– Standard-REPerformanceStatus The characters “XRV-RFS”
– Version Major version – 1 or 2 characters
– Dot-separator The character “.”
– Subversion Minor version – 1 or 2 characters
MInstanceID ID of M-Instance.
UEnvironmentID ID of U-Environment connect to M-Instance.
REPerformanceStatusID Identifier of RE Performance Status.
REPerformanceStatusTime Time of RE Performance Status.
AudioVisualSceneDescriptors DEescriptors of AV Scene.
REPerformanceStatusData Data of RE Performance Status.
– IndividualStatus[] Collection of Statuses of individual Performers.
  – PerformerID ID of one Performer.
  – AudioBehaviour Performer’s behaviour in terms of Audio.
  – VisualBehaviour Performer’s behaviour in terms of Visual.
  – Personal Status Of Performer
– CollectiveStatus[] Collection of Statuses of groups of Performers.
  PerfomerGroupID ID of the group of Performers.
  – AudioBehaviour Group of Performers’ behaviour in terms of Audio.
  – VisualBehaviour Group of Performers’ behaviour in terms of Visual.
  – GazingPerformers[] Configuration of Gazing Performers.
    – Performer1ID ID of Gazing Performer.
    – Performer2ID ID of Gazed Performer.
  – PerformerDistances[] Configuration of inter-Performers’ distances.
    – PerformerID ID of one Performer.
      – PerformerID ID of all other Performers.
– SetPieces[] Set Pieces of Stage.
  – ObjectID ID of Set Piece.
  – SpatialAttitude Set Piece’s Spatial Attitude.
  – ControlData[] Set of Data Generated by Set Pieces.
  – ControlDataID Data meaning anyOf: encoder, trigger, measuring device, motion sensor proximity sensor.
DescrMetadata Descriptive Metadata