Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment
Definition
Cross-Modal Interpretative Evidence (PGM-CMI): The readings supported by the evidence of all modalities, in order of confidence, with the established reference bindings and the identified disagreements, produced by Cross-Modal Interpretation. Discounted readings are not discarded.
Functional Requirements
- Header: The Data shall have Header “PGM-CMI-V1.0”.
- Identification: The Data shall carry the identifiers of the M‑Instance, of the U‑Environment and of the session, and a unique identifier and a time (CrossModalInterpretativeEvidenceID, CrossModalInterpretativeEvidenceTime).
- Readings: The Data shall include readings in order of decreasing confidence.
- ReferenceBindings: The Data may include bindings of referring expressions to the Objects and Entities they denote.
- UnresolvedReferences: The Data may include referring expressions that remain unresolved.
- Disagreements: The Data may include disagreements between modalities.
- Metadata: The Data shall include DataXMData and may include DescrMetadata.
Syntax
https://schemas.mpai.community/PGM1/V1.0/data/CrossModalInterpretativeEvidence.json
Semantics
Table 1 – Semantics of Cross-Modal Interpretative Evidence
| Label | Description |
| Header | Header of the Data Type, with the version. (required) |
| MInstance | Identifier of the M-Instance. (required) |
| UEnvironment | Identifier of the U-Environment. (required) |
| SessionID | Identifier of the A-User session. (required) |
| CrossModalInterpretativeEvidenceID | Identifier of this instance. (required) |
| CrossModalInterpretativeEvidenceTime | Time at which the Data was produced. (required) |
| Readings | Readings in order of decreasing confidence. (required) |
| – EntityID | Identifier of the Entity concerned. |
| – Reading | What the evidence supports. (required) |
| – Confidence | Confidence, 0 to 1. (required) |
| – Persistence | Whether the reading is momentary or sustained. One of: momentary, sustained. |
| – ChangePoint | Change Point. |
| – SupportedBy | Evidence the reading rests on. Each one of: linguistic, paralinguistic, behavioural, expressive, domain, history. |
| ReferenceBindings | Bindings of referring expressions to the Objects and Entities they denote. |
| – Expression | The referring expression. (required) |
| – TargetIDs | Identifiers of the Objects and Entities denoted. |
| – AddresseeID | Addressee ID. |
| – By | The evidence on which the binding rests. Each one of: deictic_gesture, gaze, posture, domain. |
| UnresolvedReferences | Referring expressions that remain unresolved. |
| Disagreements | Disagreements between modalities. |
| – Modalities | The modalities that disagree. |
| – Interval | Time interval, on the time base of the capture. |
| –– StartTime | Time at which the Data was produced. |
| –– EndTime | Time at which the Data was produced. |
| – Kind | Kind of disagreement. One of: meaningful, apparent, unresolved. (required) |
| – Description | Description of the reason. |
| DataXMData | Data Exchange Metadata of MPAI-PTF: provenance, authorisation, security and confidence of the Data. |
| DescrMetadata | Descriptive metadata (plain text, at most 2048 characters). |
Conformance Testing
Data conforms with Cross-Modal Interpretative Evidence if it validates against the schema of Section 3. If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema.
Performance Assessment
Not part of this specification.
