1 Function | 2 Reference Model | 3 Input/Output Data |
4 SubAIMs | 5 JSON Metadata | 6 Profiles |
7 Reference Software | 8 Conformance Texting | 9 Performance Assessment |
1 Functions
The Response and Scene Rendering (PAF-RSR) AIM produces the Avatar in an Audio-Visual Scene from AV Scene Descriptors, Point of View of the viewer, Speech or Text and Personal Status:
Receives | Point of View | Selected by viewer to see/hear the scene. |
AV Scene Descriptors | Describing the Audio-Visual Scene. | |
Avatar PoV | The Point of View of the Avatar in the AV Scene. | |
Input Speech | Speech alternative to Speech synthesised from Text Object. | |
Speech Model | The Speech Model used to synthesise Speech. | |
Text Object | To be used to: 1. Synthesise into Speech using a provided or an internal Speech Model 2. Generate Face Descriptors and Body Descriptors. |
|
Personal Status | To be used as additional information to synthesise Speech and generate Face and Body Descriptors. | |
Avatar Model | When a specific Avatar Model is externally provided. | |
Produces | Output Speech | The output Speech Object . |
Audio Speech | The Audio of the Audio-Visual Scene Descriptors. | |
Output Visual | The rendering of the animated avatar and the visual scene. |
2 Reference Model
Figure 1depicts the Reference Model of the Avatar Animation and Speech (PAF-AAS).
Figure 1– The Response and Scene Rendering (PAF-RSR) AIM
3 Input/Output Data
Table 1 specifies the Input and Output Data of the Response and Scene Rendering (PAF-RSR) AIM.
Table 1– I/O Data of the Response and Scene Rendering (PAF-RSR) AIM
Input | Description |
Point of View | Selected by viewer to see/hear the scene. |
Audio-Visual Scene Descriptors | Describing the Audio-Visual Scene. |
Avatar PoV | The Point of View of the Avatar in the AV Scene. |
Input Speech | Speech alternative to Speech synthesised from Text Object. |
Speech Model | The Speech Model used to synthesise Speech. |
Text Object | For speech synthesis and Face and Body Descriptors. |
Personal Status | additional information to synthesise Speech and generate Face and Body Descriptors. |
Avatar Model | When a specific Avatar Model is externally provided. |
Output | Description |
Output Speech | The output Speech Object . |
Audio Speech | The Audio of the Audio-Visual Scene Descriptors. |
Output Visual | The rendering of the animated avatar and the visual scene. |
4 SubAIMs
A Response and Scene Rendering (PAF-RSR) AIM may be implemented as as a composite AIM depicted in Figure 2
Figure 2 – Response and Scene Rendering (PAF-RSR) Composite AIM Reference Model
The AIMs composing the Response and Scene Rendering (PAF-RSR) Composite AIM are:
AIM | AIMs | Names | JSON |
PAF-RSR | Response and Scene Rendering | X | |
PAF-PSD | Personal Status Display | X | |
MMC-TTS | Text-To-Speech | X | |
PAF-EFD | Entity Face Description | X | |
PAF-EBD | Entity Body Description | X | |
PAF-SAR | Scene and Avatar Rendering | X |
5 JSON Metadata
https://schemas.mpai.community/PAF/V1.5/AIMs/AvatarAnimationAndSpeech.json
6 Profiles
No Profiles
7 Reference Software
8 Conformance Testing
Table 2 provides the Conformance Testing Method for PAF-PSRAIM.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and conform with the Qualifier, if present.
Table 2 – Conformance Testing Method for PAF-AAS AIM
Input | Description |
Input Speech | Shall validate against Speech Object Schema. |
Face Descriptors | Shall validate against Face Descriptors Schema. |
Body Descriptors | Shall validate against Body Descriptors Schema. |
Avatar Model | Shall validate against 3D Model Object Schema. |
Output | Description |
Output Speech | Shall validate against Speech Object Schema. |
Output Visual | Shall validate against Visual Object Schema. |
Input | Description |
Text Object | Shall validate against Text Object Schema. |
Personal Status | Shall validate against Personal Status Schema. |
Speech Model | Shall validate against Machine Learning Schema. |
Avatar Model | Shall validate against 3D Model Object Schema. |
Output | Description |
Speech Object | Shall validate against Speech Object Schema. |
Visual Object | Shall validate against Visual Object Schema. |