Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment
Definition
Joint Salience (PGM-JSL): The ranking of the AV Entities by joint salience produced by Audio-Visual Salience Reconciliation, each rank with its ground and confidence. A divergence between the modalities is recorded, not removed.
Functional Requirements
- Header: The Data shall have Header “PGM-JSL-V1.0”.
- Identification: The Data shall carry the identifiers of the M‑Instance, of the U‑Environment and of the session, and a unique identifier and a time (JointSalienceID, JointSalienceTime).
- Ranking: The Data shall include the ranking, in order of rank.
- Metadata: The Data shall include DataXMData and may include DescrMetadata.
Syntax
https://schemas.mpai.community/PGM1/V1.0/data/JointSalience.json
Semantics
Table 1 – Semantics of Joint Salience
| Label | Description |
| Header | Header of the Data Type, with the version. (required) |
| MInstance | Identifier of the M-Instance. (required) |
| UEnvironment | Identifier of the U-Environment. (required) |
| SessionID | Identifier of the A-User session. (required) |
| JointSalienceID | Identifier of this instance. (required) |
| JointSalienceTime | Time at which the Data was produced. (required) |
| Ranking | The ranking, in order of rank. (required) |
| – AVEntityID | AVEntity ID. (required) |
| – Rank | Position in the ranking, starting from 1. (required) |
| – JointSalience | Joint salience of the AV Entity, 0 to 1. (required) |
| – Ground | The ground on which the rank holds. One of: concurrence, single_modality, divergence. (required) |
| – Modality | For Ground single_modality, the modality on which the rank rests. One of: audio, visual. |
| – AudioSalience | Audio salience, weighted by the conditions of observation. |
| – VisualSalience | Visual salience, weighted by the conditions of observation. |
| – Divergence | For Ground divergence, the difference between the qualified saliences. |
| – Confidence | Confidence, 0 to 1. |
| DataXMData | Data Exchange Metadata of MPAI-PTF: provenance, authorisation, security and confidence of the Data. |
| DescrMetadata | Descriptive metadata (plain text, at most 2048 characters). |
Conformance Testing
Data conforms with Joint Salience if it validates against the schema of Section 3. If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema.
Performance Assessment
Not part of this specification.
