Go to XRV-LTP Data Types

1. Definition 2. Functional Requirements 3. Syntax 4. Semantics 5. Conformance Testing 6. Performance Assessment

1. Definition

The focused status data obtained by observing the individual and collective behaviour of Performers and Objects in the Virtual Environment by interpreting the descriptors, per the current Cue Point. including object and performer Behaviours and Personal Status (Emotion, Cognitive State, and Social Attitude), as needed by .he Action Generation AIM to service the current cue point.

2. Functional Requirements

Components from the virtual environment: (data describing any and all “objects” and avatars from the Virtual Environment)

  1. Multimodal Scene Descriptors.
  1.  Individual Performers
    1. Performer ID
    2. Avatar ID
    3. Visual attributes
      1. Position and Orientation
      2. Face and Gestures
    4. Performer Audio
    5. Performer Biometrics:
      1. Heart rate and Heart rate variability (HRV).
      2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
      3. Galvanic Skin Response (Electrodermal Activity).
      4. Myoelectric intensity per electrode site.
      5. Skin temperature.
  2. Collective Performers
    1. GroupID
    2. Centroid Position of group
    3. Principal Axes of group
    4. Principal Axis Variances
    5. Percentage of Performers in Group
    6. Composite Brainwaves
    7. Biometric Synchrony
    8. Composite Audio Behaviour
    9. Composite Visual Behaviour
    10. Distance From Centroid for each Performer.
  3.  Objects:
    1. ObjectID
    2. Visual Object Data
    3. Position and Orientation (Spatial Attitude) of set pieces.
    4. Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.)
  4. Cameras
    1. Audio/Visual
    2. Position and Orientation.
    3. Field of View.

3. Syntax

https://schemas.mpai.community/XRV1/V1.0/data/VEPerformanceStatus.json

4. Semantics

Label Description
Header VE Performance Status Header. Shall have constant value “XRV-VFS-V1.0”.
MInstanceID Identifier of the M-Instance (Virtual Space) associated with this VE Performance Status instance.
UEnvironmentID ID of U-Environment connect to M-Instance.
VEPerformanceStatusID Identifier of VE Performance Status.
VEPerformanceStatusTime Time of VE Performance Status.
MultimodalSceneDescriptors Multimodal Scene Descriptors as defined in OSD V1.5, describing the Virtual Scene.
VEPerformanceStatusData Data of VE Performance Status.
– IndividualStatuses[] Collection of Statuses of individual Performers.
    – VEPerformerID ID of one Performer.
    – SpatialAttitude Performer’s Spatial Attitude.
    – AvatarOrAvatarID[] Array of Portable Avatars (objects) or Portable Avatar IDs (string).
    – AudioBehaviour Performer’s behaviour in terms of Audio.
    – VisualBehaviour Performer’s behaviour in terms of Visual.
    – PersonalStatus Performer’s Personal Status.
    – Brainwaves Frequency, in Hz, of each EEG band, per Brainwaves (AIH-BRW).
    – BiometricData Per Biometric Data specification.
– CollectiveStatuses[] Collection of Statuses of groups of Performers.
    – GroupID ID of the group of Performers.
    – CentroidPosition Position of Group Centroid.
    – PrincipalAxes Principal Axes of Group.
    – PrincipalAxisVariances[] Variances along Principal Axes Each variance is not less than 0, expressed in the square of the unit of the coordinates.
    – PerformerPercentage Number of Performers in the Group by total number of Performers.
    – BiometricSynchrony Biometric Synchrony as defined in XRV1 V1.0.
    – Brainwaves Frequency, in Hz, of each EEG band, per Brainwaves (AIH-BRW).
    – AudioBehaviour Group of Performers’ behaviour in terms of Audio.
    – VisualBehaviour Group of Performers’ behaviour in terms of Visual.
    – Performers[] Array of Performers.
        – VEPerformerID ID of one Performer.
        – DistanceFromCentroid Distance From Centroid (number).
– Objects[] Set Pieces of Stage.
    – VEObjectID ID of Set Piece.
    – VEObject Actual Object.
    – SpatialAttitude Spatial Attitude as defined in OSD V1.5.
    – ControlObject Control Object as defined in XRV1 V1.0.
– VECameraData[] Array of VE Camera Data.
    – VEVisualObject 3D Model Object as defined in OSD V1.5.
    – FieldOfView Field of view, in degrees.
    – VEAudioObject Audio Object as defined in OSD V1.5.
    – SpatialAttitude Spatial Attitude as defined in OSD V1.5.
DataXMData MPAI-PTF Data Exchange Metadata providing provenance, authorisation, legal, security, and confidence information associated with this VE Performance Status instance in a trusted data exchange context.
DescrMetadata Descriptive Metadata

5. Conformance Testing

A Data instance conforms with VE Performance Status (XRV-VFS-V1.0) if:

  1. The Data validates against the JSON Schema of VE Performance Status (VEPerformanceStatus.json).
  2. All Data in the JSON Schema of VE Performance Status have the specified types.

6. Performance Assessment

Not part of this specification.

Go to XRV-LTP Data Types