Go To MPAI-PAF AI Modules

1 Function 2 Reference Model 3 Input/Output Data
4 SubAIMs 5 JSON Metadata 6 Profiles
7 Reference Software 8 Conformance Texting 9 Performance Assessment

1 Functions

The Response and Scene Rendering (PAF-RSR) AIM produces the Avatar in an Audio-Visual Scene from AV Scene Descriptors, Point of View of the viewer, Speech or Text and Personal Status:

Receives Point of View Selected by viewer to see/hear the scene.
AV Scene Descriptors Describing the Audio-Visual Scene.
Avatar PoV The Point of View of the Avatar in the AV Scene.
Input Speech Speech alternative to Speech synthesised from Text Object.
Speech Model The Speech Model used to synthesise Speech.
Text Object To be used to:
1. Synthesise into Speech using a provided or an internal Speech Model
2. Generate Face Descriptors and Body Descriptors.
Personal Status To be used as additional information to synthesise Speech and generate Face and Body Descriptors.
Avatar Model When a specific Avatar Model is externally provided.
Produces Output Speech The output Speech Object .
Audio Speech The Audio of the Audio-Visual Scene Descriptors.
Output Visual The rendering of the animated avatar and the visual scene.

2 Reference Model

Figure 1depicts the Reference Model of the Avatar Animation and Speech (PAF-AAS).

Figure 1– The Response and Scene Rendering (PAF-RSR) AIM

3 Input/Output Data

Table 1 specifies the Input and Output Data of the Response and Scene Rendering (PAF-RSR) AIM.

Table 1– I/O Data of the Response and Scene Rendering (PAF-RSR) AIM

Input Description
Point of View Selected by viewer to see/hear the scene.
Audio-Visual Scene Descriptors Describing the Audio-Visual Scene.
Avatar PoV The Point of View of the Avatar in the AV Scene.
Input Speech Speech alternative to Speech synthesised from Text Object.
Speech Model The Speech Model used to synthesise Speech.
Text Object For speech synthesis and Face and Body Descriptors.
Personal Status additional information to synthesise Speech and generate Face and Body Descriptors.
Avatar Model When a specific Avatar Model is externally provided.
Output Description
Output Speech The output Speech Object .
Audio Speech The Audio of the Audio-Visual Scene Descriptors.
Output Visual The rendering of the animated avatar and the visual scene.

4 SubAIMs

A Response and Scene Rendering (PAF-RSR) AIM may be implemented as as a composite AIM depicted in Figure 2

Figure 2 – Response and Scene Rendering (PAF-RSR) Composite AIM Reference Model

The AIMs composing the Response and Scene Rendering (PAF-RSR) Composite AIM are:

AIM AIMs Names JSON
PAF-RSR Response and Scene Rendering X
PAF-PSD Personal Status Display X
MMC-TTS Text-To-Speech X
PAF-EFD Entity Face Description X
PAF-EBD Entity Body Description X
PAF-SAR Scene and Avatar Rendering X

5 JSON Metadata

https://schemas.mpai.community/PAF/V1.5/AIMs/AvatarAnimationAndSpeech.json

6 Profiles

No Profiles

7 Reference Software

8 Conformance Testing

Table 2 provides the Conformance Testing Method for PAF-PSRAIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and conform with the Qualifier, if present.

Table 2 – Conformance Testing Method for PAF-AAS AIM

Input Description
Input Speech Shall validate against Speech Object Schema.
Face Descriptors Shall validate against Face Descriptors Schema.
Body Descriptors Shall validate against Body Descriptors Schema.
Avatar Model Shall validate against 3D Model Object Schema.
Output Description
Output Speech Shall validate against Speech Object Schema.
Output Visual Shall validate against Visual Object Schema.

 

Input Description
Text Object Shall validate against Text Object Schema.
Personal Status Shall validate against Personal Status Schema.
Speech Model Shall validate against Machine Learning Schema.
Avatar Model Shall validate against 3D Model Object Schema.
Output Description
Speech Object Shall validate against Speech Object Schema.
Visual Object Shall validate against Visual Object Schema.

9 Performance Assessment

Go To MPAI-PAF AI Modules