Go to XRV-LTP Data Types

1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

The focused status data obtained by observing the scene and individual and collective behaviour of Performers in the Real Environment by interpreting their Descriptors, per the current Cue Point Conditions, including behaviours, and Personal Status (Emotion, Cognitive State, and Social Attitude), as required by the Action Descriptor Generation AIM to service the current Cue Point.

2. Functional Requirements

  1. Audio-Visual Scene Descriptors
  2.  Individual
    1. Performer ID
    2. Performer’s visual behaviour:
      1. Visual Activity Activity
        1. Raising arms
        2. Jumping
        3. Pointing to a direction
        4. Gazing at a direction
        5. Dancing
        6. Standing or sitting
        7. Waving
        8. Clapping
      2. Supplementary data
        1. Relevant objects in gaze direction.
        2. Relevant objects in pointing direction.
        3. Dance Object
    3. Performer’s audio behaviour:
      1. Audio Activity: Booing, Clapping, Laughing, Shouting Singing, Speaking
      2. Supplementary data: Intensity, Uttered text
    4. Personal Status (Cognitive State, Emotion, Social Attitude).
  3. Collective
    1. Visual behaviour (performance)
      1. Visual status data
        1. From Proximity Performer A
        2. Gaze: performer A looking at Performer B
      2. Scene attributes
        1. Position and Orientation (Spatial Attitude) of set pieces
        2. The scene as volume (Volumetric, e.g., point cloud)
    2. Audio behaviour (performance)
      1. Status on a per-performer basis:
        1. Text uttered by performer.
        2. Performer’s audio activity:
          1. Performer doing a certain activity, e.g., Laughing, Clapping, Booing, Shouting, Singing
          2. Intensity of Performer’s activity
          3. Particular phrase/text uttered
    3. Personal Status.
  4. Object data:
    1. The scene as volume or mesh (Volumetric, e.g., point cloud).
    2. Position and Orientation (Spatial Attitude) of set pieces.
    3. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).

3. Syntax

https://schemas.mpai.community/XRV1/V1.0/data/REPerformanceStatus.json

4. Semantics

Label Description
Header RE Performance Status Header. Shall have constant value “XRV-RFS-V1.0”.
MInstanceID Identifier of the M-Instance (Virtual Space) associated with this RE Performance Status instance.
UEnvironmentID ID of U-Environment connect to M-Instance.
REPerformanceStatusID Identifier of RE Performance Status.
REPerformanceStatusTime Time of RE Performance Status.
AVSceneDescriptors Descriptors of AV Scene.
REPerformanceStatusData Data of RE Performance Status.
– IndividualStatuses[] Collection of Statuses of individual Performers.
    – REPerformerID ID of one Performer.
    – SpatialAttitude Performer’s Spatial Attitude.
    – AudioBehaviour Performer’s behaviour in terms of Audio.
    – VisualBehaviour Performer’s behaviour in terms of Visual.
    – PersonalStatus Of Performer
    – Brainwaves Frequency, in Hz, of each EEG band, per Brainwaves (AIH-BRW).
    – BiometricData Biometric Data as defined in AIH-HSP V1.1.
– CollectiveStatuses[] Collection of Statuses of groups of Performers.
    – GroupID ID of the group of Performers.
    – CentroidPosition Position as defined in OSD V1.5.
    – PrincipalAxes Orientation as defined in OSD V1.5.
    – PrincipalAxisVariances[] Principal Axis Variances (number). Each variance is not less than 0, expressed in the square of the unit of the coordinates.
    – PerformerPercentage Performer Percentage (number).
    – Brainwaves Frequency, in Hz, of each EEG band, per Brainwaves (AIH-BRW).
    – BiometricSynchrony Biometric Synchrony as defined in XRV1 V1.0.
    – AudioBehaviour Group of Performers’ behaviour in terms of Audio.
    – VisualBehaviour Group of Performers’ behaviour in terms of Visual.
    – Performers[] Array of Performers.
        – REPerformerID ID of one Performer.
        – DistanceFromCentroid Distance From Centroid (number).
– Objects[] Array of Objects.
    – REObjectID ID of Set Piece.
    – REObject 3D Model Object as defined in OSD V1.5.
    – SpatialAttitude Spatial Attitude as defined in OSD V1.5.
    – ControlObject Control Object as defined in XRV1 V1.0.
– RECameraData[] Array of RE Camera Data.
    – REVisualObject Visual Object as defined in OSD V1.5.
    – FieldOfView Field of view, in degrees.
    – REAudioObject Audio Object as defined in OSD V1.5.
    – SpatialAttitude Spatial Attitude as defined in OSD V1.5.
DataXMData MPAI-PTF Data Exchange Metadata providing provenance, authorisation, legal, security, and confidence information associated with this RE Performance Status instance in a trusted data exchange context.
DescrMetadata Descriptive Metadata

Go to XRV-LTP Data Types