Go to MPAI-MMC V2.5 AI Modules
Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance
1 Functions
The Video Lip Animation (MMC‑VLA) AIM receives an input Speech and an Emotion, queries a video from a KB and produces a Video moving lips in sync with the Speech and displaying the input Emotion:
| Receives | Speech Object | E.g., from upstream AIM. |
| Machine Emotion | From upstream AIM. | |
| Video | From the KB of Videos of Faces. | |
| Produces | Face Object | Displaying lips animated by Speech on a face with Emotion. |
2 Reference Model
Figure 1 depicts the Reference Model of the Video Lip Animation (MMC‑VLA) AIM.
Figure 1 – The Video Lip Animation (MMC‑VLA) AIM Reference Model
3 I/O Data
Table 1 specifies the Input and Output Data of the Video Lip Animation (MMC‑VLA) AIM.
| Input | Description |
|---|---|
| Speech Object | An input Speech that may have been produced by a Machine. |
| Emotion | Emotion in the input Speech. |
| Video Object | A video drawn from a KB of Faces. |
| Output | Description |
| Face Object | Synthetic speaking Face Object displaying Emotion. |
4 SubAIMs
No SubAIMs.
5 JSON Metadata
https://schemas.mpai.community/MMC/V2.5/AIMs/VideoLipAnimation.json
6 Profiles
No Profiles.
7 Reference Software
Under development.
8 Conformance Testing
Table 2 provides the Conformance Testing Method for the Video Lip Animation (MMC‑VLA) AIM.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.
| Receives | Speech Object | Shall validate against Speech Object schema. Speech Data shall conform with Speech Qualifier. |
| Emotion | Shall validate against Emotion schema. | |
| Video Object | Shall validate against Visual Object schema. Visual Data shall conform with Visual Qualifier. | |
| Produces | Face Object | Shall validate against Visual Object schema. Visual Data shall conform with Visual Qualifier. |
9 Performance Assessment
Not part of this specification.
