Go To MPAI-MMC AI Modules

1     Function 2     Reference Model 3     Input/Output Data
4     SubAIMs 5     JSON Metadata 6     Profiles
7     Reference Software 8     Conformance Texting 9     Performance Assessment

1     Functions

Video Lip Animation (MMC-VLA):

Receives Machine Speech from upstream AIM.
Machine Emotion from upstream AIM.
Video from Video of Faces KB.
Produces Face Object displaying lips animated by Speech on a face Emotion.

2     Reference Model

Figure 1 depicts the Reference Model of the Video Lip Animation (MMC-VLA) AIM.

Figure 1 – The Video Lip Animation (MMC-VLA) AIM Reference Model

3    Input/Output Data

Table 1 specifies the Input and Output Data of the Video Lip Animation (MMC-VLA) AIM.

Table 1 – I/O Data of the Video Lip Animation (MMC-VLA) AIM

Input data Description
Speech The vocal response of a Machine.
Emotion Emotion in Speech.
Video From KB Faces.
Output data Description
Emotion Of Video in Video of KB Faces.
Face Object Synthetic speaking Face Object displaying Emotion.

4     SubAIMs

No SubAIMs.

5     JSON Metadata

https://schemas.mpai.community/MMC/V2.3/AIMs/VideoLipAnimation.json

6     Profiles

No Profiles.

7     Reference Software

8     Conformance Testing

Table 2 provides the Conformance Testing Method for MMC-VLA AIM.

Note that a schema may contain references to other schemas. In this case, validation of data for the primary schema implies that any data that refers to a secondary schema shall also validate.

Table 2 – Conformance Testing Method for MMC-VLA AIM

Input Machine Speech Object Shall validate against Speech Object schema.
Speech Data shall conform with Speech Qualifier.
Machine Emotion Shall validate against Emotion schema.
Video Visual Object Shall validate against Visual Object schema.
Visual Data shall conform with Visual Qualifier.
Output Face Visual Object Shall validate against Visual Object schema.
Visual Data shall conform with Visual Qualifier.

Table 3 provides an example of MMC-VLA AIM conformance testing.

Table 3 – An example MMC-VLA AIM conformance testing

Input Data Data Type Input Conformance Testing Data
Machine Speech .wav All input Text files to be drawn from Text files.
Machine Emotion JSON All input JSON Emotion files to be drawn from Emotion JSON Files
Video of Face AVC All input Video files to be drawn from Video files.
Output Data Data Type Output Conformance Testing Criteria
Machine Face AVC All Video files produced shall conform with Video.

9     Performance Assessment

Go To MPAI-MMC AI Modules