MPAI-PAF V1.6 AIMs - Response and Scene Rendering

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

The Response and Scene Rendering (PAF‑RSR) AIM renders the response of a machine – a Text, and the Personal Status that goes with it – as a Portable Avatar of the input Avatar Model, ready to play. Whoever requested the Portable Avatar places it in a Scene and renders it:

Receives	Input Speech	The Speech the Avatar utters instead of the Machine Speech (optional).
	Speech Model	The Model with which the Machine Speech is made (optional).
	Text for Speech	The Text from which the Machine Speech is made (Port 1).
	Text for Formation	The Text carried by the Portable Avatar, from which its Face and Body behaviour is made (Port 2).
	Personal Status	The Personal Status the Avatar expresses (optional).
	Avatar Model	The Avatar Model, represented by a Basic 3D Model Object, that is animated.
	Machine ID	The identifier of the machine the Portable Avatar represents.
	Point of View	The Point of View of the Scene in which the Portable Avatar will be placed (optional). It goes through to the Output Point of View.
	Avatar Spatial Attitude	The position and orientation of the Avatar in the Scene (optional).
	AV Scene Descriptors	The Descriptors of the Audio-Visual Scene in which the Avatar is placed (optional).
Produces	Portable Avatar	The Avatar, the Speech and the Face and Body Descriptors, ready to play.
	Output Point of View	The Point of View received, passed through (optional).

2 Reference Model

Figure 1 depicts the Reference Model of the Response and Scene Rendering (PAF‑RSR) AIM.

Response and Scene Rendering PAF-RSR AIM

Figure 1 – The Response and Scene Rendering (PAF‑RSR) AIM

3 I/O Data

Table 1 specifies the Input and Output Data of the Response and Scene Rendering (PAF‑RSR) AIM.

Table 1 – I/O Data of the Response and Scene Rendering (PAF‑RSR) AIM

Input	Description
Input Speech or Input Speech	The Speech the Avatar utters instead of the Machine Speech (optional).
Speech Model	The Model with which the Machine Speech is made (optional).
Text for Speech or Text for Speech	The Text from which the Machine Speech is made (Port 1).
Text for Formation or Text for Formation	The Text carried by the Portable Avatar, from which its Face and Body behaviour is made (Port 2).
Personal Status	The Personal Status the Avatar expresses (optional).
Avatar Model	The Avatar Model, represented by a Basic 3D Model Object, that is animated.
Machine ID	The identifier of the machine the Portable Avatar represents.
Point of View	The Point of View of the Scene in which the Portable Avatar will be placed (optional). It goes through to the Output Point of View.
Avatar Spatial Attitude	The position and orientation of the Avatar in the Scene (optional).
AV Scene Descriptors	The Descriptors of the Audio-Visual Scene in which the Avatar is placed (optional).
Output	Description
Portable Avatar	The Avatar, the Speech and the Face and Body Descriptors, ready to play.
Output Point of View	The Point of View received, passed through (optional).

4 SubAIMs

4.1 Reference Model

The Response and Scene Rendering (PAF‑RSR) AIM is a Composite AIM depicted in Figure 2.

Response and Scene Rendering PAF-RSR Composite AIM

Figure 2 – The Response and Scene Rendering (PAF‑RSR) Composite AIM

4.2 Operation

Personal Status Demultiplexing splits the Personal Status into the Speech, Face and Body Personal Statuses that make the Avatar expressive. Text-To-Speech makes the Machine Speech from the Text, expressive of the Speech Personal Status, with the Speech Model when one is supplied. Portable Avatar Formation forms the Portable Avatar of the Avatar Model from the Machine Speech – or from the Input Speech, when one is supplied – the Text and the Face and Body Personal Statuses.

4.3 Functions of AI Modules

Table 2 specifies the Function of the AI Modules.

Table 2 – Functions of AI Modules

Personal Status Demultiplexing	Splits the Personal Status into the Speech, Face and Body Personal Statuses.
Text-To-Speech	Makes the Machine Speech from the Text, expressive of the Speech Personal Status.
Portable Avatar Formation	Forms the Portable Avatar of the Avatar Model.

4.4 I/O Data of AI Modules

Table 3 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.

Table 3 – AIMs and JSON Metadata of the Response and Scene Rendering (PAF‑RSR)

AIM1	AIM2	Name	JSON
PAF‑RSR		Response and Scene Rendering	X
	MMC‑PDX	Personal Status Demultiplexing	X
	MMC‑TTS	Text-To-Speech	X
	PAF‑SAF	Portable Avatar Formation	X

5 JSON Metadata

https://schemas.mpai.community/PAF/V1.6/AIMs/ResponseAndSceneRendering.json

6 Profiles

No Profiles.

7 Reference Software

Not part of this specification.

8 Conformance Testing

Table 4 provides the Conformance Testing Method for the PAF‑RSR AIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 4 – Conformance Testing Method for the PAF‑RSR AIM

Receives	Input Speech or Input Speech	Shall validate against Basic Speech Object or Speech Object schema.
	Speech Model	Shall validate against ML Model Object schema.
	Text for Speech or Text for Speech	Shall validate against Basic Text Object or Text Object schema.
	Text for Formation or Text for Formation	Shall validate against Basic Text Object or Text Object schema.
	Personal Status	Shall validate against Personal Status schema.
	Avatar Model	Shall validate against Basic 3D Model Object schema.
	Machine ID	Shall be a string.
	Point of View	Shall validate against Point of View schema.
	Avatar Spatial Attitude	Shall validate against Spatial Attitude schema.
	AV Scene Descriptors	Shall validate against Audio-Visual Scene Descriptors schema.
Produces	Portable Avatar	Shall validate against Portable Avatar schema.
	Output Point of View	Shall validate against Point of View schema.

9 Performance Assessment