Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment
Definition
Association Candidates (PGM-ASC): The admissible pairings of Audio Entities with Visual Entities proposed by Cross-Modal Candidate Association, each with its association score and the contribution of each kind of evidence. The association is many-to-many: no pairing is committed.
Functional Requirements
- Header: The Data shall have Header “PGM-ASC-V1.0”.
- Identification: The Data shall carry the identifiers of the M‑Instance, of the U‑Environment and of the session, and a unique identifier and a time (AssociationCandidatesID, AssociationCandidatesTime).
- Candidates: The Data shall include the admissible pairings of an Audio Entity with a Visual Entity.
- Metadata: The Data shall include DataXMData and may include DescrMetadata.
Syntax
https://schemas.mpai.community/PGM1/V1.0/data/AssociationCandidates.json
Semantics
Table 1 – Semantics of Association Candidates
| Label | Description |
| Header | Header of the Data Type, with the version. (required) |
| MInstance | Identifier of the M-Instance. (required) |
| UEnvironment | Identifier of the U-Environment. (required) |
| SessionID | Identifier of the A-User session. (required) |
| AssociationCandidatesID | Identifier of this instance. (required) |
| AssociationCandidatesTime | Time at which the Data was produced. (required) |
| Candidates | The admissible pairings of an Audio Entity with a Visual Entity. (required) |
| – AudioEntityID | Audio Entity ID. (required) |
| – VisualEntityID | Visual Entity ID. (required) |
| – AssociationScore | Score of the pairing, 0 to 1, combining the four kinds of evidence. (required) |
| – Evidence | The contribution of each kind of evidence, with whether it was available. |
| –– Spatial | Agreement between the direction of arrival and, where available, the distance of the Audio Entity and the position of the Visual Entity. |
| ––– Score | Score of this kind of evidence, 0 to 1. |
| ––– Available | True if this kind of evidence was available for the pairing. (required) |
| –– Temporal | Agreement between the onset, offset and activity of the Audio Entity and the motion of the Visual Entity. |
| ––– Score | Score of this kind of evidence, 0 to 1. |
| ––– Available | True if this kind of evidence was available for the pairing. (required) |
| –– MotionCorrelation | Agreement between the displacement of the Audio Entity and that of the Visual Entity over the capture interval. |
| ––– Score | Score of this kind of evidence, 0 to 1. |
| ––– Available | True if this kind of evidence was available for the pairing. (required) |
| –– Precedence | Whether the same pairing was established in earlier captures of the session. |
| ––– Score | Score of this kind of evidence, 0 to 1. |
| ––– Available | True if this kind of evidence was available for the pairing. (required) |
| – UnavailableEvidence | Kinds of evidence that were unavailable for the pairing. Each one of: spatial, temporal, motion, precedence. |
| DataXMData | Data Exchange Metadata of MPAI-PTF: provenance, authorisation, security and confidence of the Data. |
| DescrMetadata | Descriptive metadata (plain text, at most 2048 characters). |
Conformance Testing
Data conforms with Association Candidates if it validates against the schema of Section 3. If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema.
Performance Assessment
Not part of this specification.
