Go to PGM-AUA V1.0 Data Types

Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment

Definition

Behavioural and Expressive Indicators (PGM-BEE): The indicators provided by the conduct of an Entity and by the expression of its body, produced by Behavioural and Expressive Analysis, including their agreement with the Linguistic-Paralinguistic Evidence. An indicator derived while the Entity was imperfectly observed is qualified.

Functional Requirements

  1. Header: The Data shall have Header “PGM-BEE-V1.0”.
  2. Identification: The Data shall carry the identifiers of the M‑Instance, of the U‑Environment and of the session, and a unique identifier and a time (BehaviouralExpressiveIndicatorsID, BehaviouralExpressiveIndicatorsTime).
  3. Entities: The Data shall include the Entities.
  4. Metadata: The Data shall include DataXMData and may include DescrMetadata.

Syntax

https://schemas.mpai.community/PGM1/V1.0/data/BehaviouralExpressiveIndicators.json

Semantics

Table 1 – Semantics of Behavioural and Expressive Indicators

Label Description
Header Header of the Data Type, with the version. (required)
MInstance Identifier of the M-Instance. (required)
UEnvironment Identifier of the U-Environment. (required)
SessionID Identifier of the A-User session. (required)
BehaviouralExpressiveIndicatorsID Identifier of this instance. (required)
BehaviouralExpressiveIndicatorsTime Time at which the Data was produced. (required)
Entities The Entities. (required)
– EntityID Identifier of the Entity concerned. (required)
– Interval Time interval, on the time base of the capture. (required)
–– StartTime Time at which the Data was produced.
–– EndTime Time at which the Data was produced.
– Expressive Indicators carried by the face, the gaze and the head.
–– FacialExpression Facial expression and its intensity.
––– Label Label.
––– Intensity Intensity.
–– Gaze Gaze direction and fixation.
––– Direction Direction.
––– Fixation True if the gaze is fixed.
–– Head Head pose, nodding and shaking.
––– Pose Head pose.
––– Nodding True if the head nods.
––– Shaking True if the head shakes.
–– Blink Blink and eye-opening behaviour.
– Behavioural Indicators carried by posture, gesture and movement.
–– Posture Posture and its orientation.
––– Label Label.
––– OrientationTowards Identifiers of the Entities towards which the posture is oriented.
–– Gesture Gesture and its referent.
––– Label Label.
––– Deictic True if the gesture is deictic.
––– ReferentIDs Identifiers of the Entities and Objects the gesture refers to.
–– Movement Movement in the Space. One of: approach, withdrawal, stillness, other.
–– Manipulation Manipulation of Objects in the Space.
––– ObjectIDs Identifiers of the Objects manipulated.
––– Affordances The affordances exercised.
–– ExchangeRhythm Rhythm of the exchange. One of: turn_taking, overlapping, waiting, other.
– AttentionTarget Entity or Object the Entity attends to.
– Engagement Degree of participation in the exchange.
– Disposition Interpersonal disposition towards the A-User.
– AgreementWithLinguistic Whether the bodily evidence reinforces the Linguistic-Paralinguistic Evidence, is at variance with it, or one of the two is absent or unusable. One of: reinforcing, at_variance, one_absent.
– ObservationQuality Qualification from the Enhanced Visual Scene Descriptors. One of: good, partial, occluded, out_of_frame.
– Confidence Confidence, 0 to 1.
DataXMData Data Exchange Metadata of MPAI-PTF: provenance, authorisation, security and confidence of the Data.
DescrMetadata Descriptive metadata (plain text, at most 2048 characters).

Conformance Testing

Data conforms with Behavioural and Expressive Indicators if it validates against the schema of Section 3. If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema.

Performance Assessment

Not part of this specification.

Go to PGM-AUA V1.0 Data Types