Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment
Definition
Domain (PGM-DOM): The Data Type exchanged between any AIM of the A-User and Domain Access. It has two modes, told apart by Mode: Request (RQ), that supplies the evidence and the candidate interpretations to be checked together with the operations requested, and Response (RS), that returns the outcome, the validated interpretations with updated confidences and an explanation. SourceAIM states which AIM or Sub-AIM sends the Request or receives the Response, and Subject which medium or User it relates to. A Response carries the QueryID of its Request.
Functional Requirements
- Header: The Data shall have Header “PGM-DOM-V1.0” and Mode equal to RQ for a Request and to RS for a Response; the properties that belong to the other mode shall not be present.
- Identification: The Data shall carry the identifiers of the M‑Instance, of the U‑Environment and of the session, and a unique identifier and a time (QueryID, QueryTime).
- Correlation: A Response shall carry the identifier (RequestID) of the Request that it answers.
- Source and Subject: The Data shall state in SourceAIM which AIM or Sub-AIM sends the Request or receives the Response, and in Subject which medium or User it relates to; for Subject user it may list the Users in UserIDs.
- Request: A Request shall carry a QueryID and may carry the evidence appropriate to the Subject (scene descriptors, User State, utterances, a temporal scope, a region of interest, a viewpoint), the candidate classes with their confidence and the operations requested.
- Response: A Response shall carry the QueryID of its Request and the ResponseStatus, and may carry the validated classes, the updated confidences, the resolved targets, the weight the domain gives to each modality, the classes the setting makes salient, and an explanation.
- Metadata: The Data shall include DataXMData and may include DescrMetadata.
Syntax
https://schemas.mpai.community/PGM1/V1.0/data/Domain.json
Semantics
Table 1 gives the properties common to the two modes, Table 2 those of the Request and Table 3 those of the Response.
Table 1 – Properties common to the Request and the Response of Domain
| Label | Description |
| Header | Header of the Data Type, with the version. (required) |
| Mode | RQ for a Request, RS for a Response. One of: RQ, RS. (required) |
| MInstance | Identifier of the M-Instance. (required) |
| UEnvironment | Identifier of the U-Environment. (required) |
| SessionID | Identifier of the A-User session. (required) |
| QueryID | Identifier of this Request or Response. (required) |
| QueryTime | Time at which the Data was produced. (required) |
| CaptureIndex | Position of the capture within the session to which the exchange refers. |
| SourceAIM | Header of the AIM or Sub-AIM that sends the Request or the Response (e.g., PGM-ASD-V1.0), or of the AIM to which a Response is sent. (required) |
| Subject | The medium or the entity the exchange relates to: the audio scene, the visual scene, the User, the alignment of audio and visual Entities, or the context, prompt, personality or deliberation of the A-User. One of: audio, visual, user, ava, context, prompt, personality, deliberation. (required) |
| UserIDs | Identifiers of the Users the exchange relates to, where the Subject is user. |
| DataXMData | Data Exchange Metadata of MPAI-PTF: provenance, authorisation, security and confidence of the Data. |
| DescrMetadata | Descriptive metadata (plain text, at most 2048 characters). |
Table 2 – Properties of the Request (Mode RQ)
| Label | Description |
| DomainID | Identifier of the domain the Request concerns. |
| Targets | Identifiers of the entities the Request concerns. |
| CandidateClasses | Candidate interpretations, each a Class (URI) with an optional Confidence. |
| – Class | Class. (required) |
| – Confidence | Confidence. |
| RequestedOperations | Operations requested. |
| – ResolveAmbiguities | Request: whether the production of the resolution of ambiguities is required, optional or not_requested. Response: fulfilment of the production of the resolution of ambiguities (produced, partial, missing or not_requested). |
| – ApplyConstraints | Request: whether the production of the application of the constraints is required, optional or not_requested. Response: fulfilment of the production of the application of the constraints (produced, partial, missing or not_requested). |
| – SemanticEnrichment | Request: whether the production of the semantic enrichment is required, optional or not_requested. Response: fulfilment of the production of the semantic enrichment (produced, partial, missing or not_requested). |
| – CrossModalResolution | Request: whether the production of the cross-modal resolution is required, optional or not_requested. Response: fulfilment of the production of the cross-modal resolution (produced, partial, missing or not_requested). |
| AudioSceneDescriptors | Evidence supplied, as appropriate to the Subject. |
| EnhancedAudioSceneDescriptors | Evidence supplied, as appropriate to the Subject. |
| VisualSceneDescriptors | Evidence supplied, as appropriate to the Subject. |
| EnhancedVisualSceneDescriptors | Evidence supplied, as appropriate to the Subject. |
| AudioVisualSceneDescriptors | Evidence supplied, as appropriate to the Subject. |
| UserState | Evidence supplied, as appropriate to the Subject. |
| Utterances | Identifiers of the utterances the Request concerns. |
| TemporalScope | The interval considered. |
| – StartTime | Time at which the Data was produced. |
| – EndTime | Time at which the Data was produced. |
| RegionOfInterest | The region the Request concerns. |
| Viewpoint | The viewpoint for the Request. |
Table 3 – Properties of the Response (Mode RS)
| Label | Description |
| RequestID | ID of the Request this Response answers. |
| ResponseStatus | Outcome. One of: Affirmed, Rejected, Conditional, Inconclusive. |
| ValidatedClasses | Classes confirmed by Domain Access. |
| UpdatedConfidences | Revised confidences. |
| – Class | Class. (required) |
| – Confidence | Confidence. |
| ResolvedTargets | Identifiers of the entities resolved. |
| ModalityWeights | The relative weight the domain gives to what is heard and to what is seen. |
| – Audio | Audio. |
| – Visual | Visual. |
| SalientClasses | The classes of Entity that the setting makes salient. |
| Explanation | Human-readable explanation of the outcome. |
Conformance Testing
Data conforms with Domain if it validates against the schema of Section 3 and, if the Mode is RQ, contains the properties required for a Request and none of those of a Response, and vice versa. If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema.
Performance Assessment
Not part of this specification.
