Go to MPAI-MMC V2.5 AI Modules
Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance
1 Functions
The Multimodal Access Control (MMC‑MAC) AIM recognises a User from the face and the speech and gives or refuses access. Face Identity Recognition and Speaker Identity Recognition read the gallery of enrolled Users, produced by Access Control Registration (MMC‑ACR), and the reference population of known people from Access; ID Reconciliation reconciles the two candidate identities and issues the verdict as a Response and its Personal Status. A Portable Avatar speaks the verdict; the verdict is also given as Text and as an identification flag.
2 Reference Model
Figure 1 depicts the Reference Model of the Multimodal Access Control (MMC‑MAC) AIM.
Figure 1 – Reference Model of the Multimodal Access Control (MMC‑MAC) AIM
The operation of the Multimodal Access Control (MMC‑MAC) AIM develops in the following way:
- The human provides an image of the face and a speech sample.
- The User Agent provides, optionally, a Prompt to be spoken, the Avatar Model that represents the machine and the Machine ID.
- The machine provides the verdict as Text, as an identification flag and as the Portable Avatar that speaks it.
3 I/O Data
Table 1 specifies the Input and Output Data of the Multimodal Access Control (MMC‑MAC) AIM.
| Input | Description |
|---|---|
| Face Object | Image of the face of the User to be recognised. |
| Speech Object | Speech of the User to be recognised. |
| Prompt | Text the avatar speaks to prompt the User. Optional. |
| Avatar Model | The 3D model of the avatar that represents the machine. |
| Machine ID | Identifier of the machine. Optional. |
| Output | Description |
| Response | The verdict in words. |
| Identification | Whether the User was identified (true or false). |
| Portable Avatar | The avatar that speaks the verdict, with lip-synchronised Face Descriptors. |
4 SubAIMs
4.1 Reference Model
Figure 2 depicts the Reference Model of the Multimodal Access Control (MMC‑MAC) Composite AIM.
Figure 2 – Reference Model of the Multimodal Access Control (MMC‑MAC) Composite AIM
4.2 Operation
The Multimodal Access Control (MMC‑MAC) AIM receives the Face Object and the Speech Object. Face Identity Recognition and Speaker Identity Recognition each give the Instance Identifier of the User they recognise, reading the gallery of enrolled Users and the Scene Descriptors from Shared Storage, which are not Ports, and the reference population of known people from Access (declared in the AccessSources of each AIM). ID Reconciliation reconciles the two Instance Identifiers and issues the Response and its Personal Status. Response and Scene Rendering speaks the Response and produces the lip-synchronised Machine Face Descriptors. The reconciled User ID stays inside the AIM. Acquisition and delivery are the User Agent’s, not SubAIMs.
4.3 Functions of SubAIMs
Table 2 provides the functions of the Multimodal Access Control (MMC‑MAC) SubAIMs.
| SubAIM | Function |
|---|---|
| Face Identity Recognition | Recognises the face against the gallery of enrolled Users and the reference population of known people, and gives the Instance Identifier of the User. |
| Speaker Identity Recognition | Recognises the speaker against the gallery of enrolled Users and the reference population of known people, and gives the Instance Identifier of the User. |
| ID Reconciliation | Reconciles the two Instance Identifiers and issues the Response and its Personal Status. |
| Response and Scene Rendering | Receives the Response, the Prompt, the Personal Status, the Avatar Model and the Machine ID and produces the Portable Avatar that speaks them with lip-synchronised Machine Face Descriptors. |
4.4 I/O Data of SubAIMs
Table 3 gives the Input and Output Data of the Multimodal Access Control (MMC‑MAC) SubAIMs.
| SubAIM | Input | Output |
|---|---|---|
| Face Identity Recognition | Face Object | Instance Identifier (from face) |
| Speaker Identity Recognition | Speech Object | Instance Identifier (from speech) |
| ID Reconciliation | Instance Identifier (from face) Instance Identifier (from speech) |
Response Personal Status |
| Response and Scene Rendering | Response Prompt Personal Status Avatar Model Machine ID |
Portable Avatar |
4.5 AIMs and JSON Metadata
Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.
| AIM1 | AIM2 | Name | JSON |
|---|---|---|---|
| MMC‑MAC | Multimodal Access Control | X | |
| PAF‑FIR | Face Identity Recognition | X | |
| MMC‑SIR | Speaker Identity Recognition | X | |
| OSD‑IDR | ID Reconciliation | X | |
| PAF‑RSR | Response and Scene Rendering | X |
5 JSON Metadata
https://schemas.mpai.community/MMC/V2.5/AIMs/MultimodalAccessControl.json
6 Profiles
No Profiles.
7 Reference Software
Under development.
8 Conformance Testing
Table 5 provides the Conformance Testing Method for the Multimodal Access Control (MMC‑MAC) AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.
| Receives | Face Object | Shall validate against Basic Visual Object schema. Visual Data shall conform with Visual Qualifier. |
| Speech Object | Shall validate against Basic Speech Object schema. Speech Data shall conform with Speech Qualifier. | |
| Prompt | Shall validate against Basic Text Object schema. Text Data shall conform with Text Qualifier. | |
| Avatar Model | Shall validate against Basic 3D Model Object schema. 3D Model Data shall conform with 3D Model Qualifier. | |
| Machine ID | Shall be a string. | |
| Produces | Response | Shall validate against Basic Text Object schema. Text Data shall conform with Text Qualifier. |
| Identification | Shall be true or false. | |
| Portable Avatar | Shall validate against Portable Avatar schema. |
9 Performance Assessment
Not part of this specification.
