| 1 Function | 2 Reference Model | 3 Input/Output Data |
| 4 SubAIMs | 5 JSON Metadata | 6 Profiles |
| 7 Reference Software | 8 Conformance Texting | 9 Performance Assessment |
1 Functions
The Video Lip Animation (MMC-VLA) AIM receives an input speech and an Emotion, it queries a video from a KB and produces a Video moving lips in sync with the text and displaying the input Emotion:
| Receives | Speech Object | E.g., from upstream AIM. |
| Machine Emotion | From upstream AIM. | |
| Video | From the KB of Videos of Faces. | |
| Produces | Face Object | Displaying lips animated by Speech on a face Emotion. |
2 Reference Model
Figure 1 depicts the Reference Model of the Video Lip Animation (MMC-VLA) AIM.
Figure 1 – The Video Lip Animation (MMC-VLA) AIM Reference Model
3 Input/Output Data
Table 1 specifies the Input and Output Data of the Video Lip Animation (MMC-VLA) AIM.
Table 1 – I/O Data of the Video Lip Animation (MMC-VLA) AIM
| Input data | Description |
| Speech Object | An input Speech that may have been produced by a Machine. |
| Emotion | Emotion in the input Speech. |
| Video Object | A video drawn from a KB of Faces. |
| Output data | Description |
| Emotion | Emotion in the input Speech. that is used to alter the Video drawn from the KB Faces. |
| Face Object | Synthetic speaking Face Object displaying Emotion. |
4 SubAIMs
No SubAIMs.
5 JSON Metadata
https://schemas.mpai.community/MMC/V2.4/AIMs/VideoLipAnimation.json
6 Profiles
No Profiles.
7 Reference Software
8 Conformance Testing
Table 2 provides the Conformance Testing Method for MMC-VLA AIM.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and conform with the Qualifier, if present.
Table 2 – Conformance Testing Method for MMC-VLA AIM
| Input | Machine Speech Object | Shall validate against Speech Object schema. Speech Data shall conform with Speech Qualifier. |
| Machine Emotion | Shall validate against Emotion schema. | |
| Video Object | Shall validate against Visual Object schema. Visual Data shall conform with Visual Qualifier. |
|
| Output | Face Object | Shall validate against Visual Object schema. Visual Data shall conform with Visual Qualifier. |
Table 3 provides an example of MMC-VLA AIM conformance testing.
Table 3 – An example MMC-VLA AIM conformance testing
| Input Data | Data Type | Input Conformance Testing Data |
| Machine Speech | .wav | All input Text files to be drawn from Text files. |
| Machine Emotion | JSON | All input JSON Emotion files to be drawn from Emotion JSON Files |
| Video of Face | AVC | All input Video files to be drawn from Video files. |
| Output Data | Data Type | Output Conformance Testing Criteria |
| Machine Face | AVC | All Video files produced shall conform with Video. |
