Go to MPAI-PAF V1.6 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

In Virtual Meeting applications, such as in the Avatar-Based Videoconference Use Case, i.e., a video-conference where avatars participate realistically impersonating the human participants, the Virtual Secretary:

  1. Listens to the Speech of each avatar.
  2. Monitors their Personal Status.
  3. Drafts a Summary using the avatars’ Personal Status and Text obtained from Automatic Speech Recognition or directly via Text input in the meeting’s common language.
  4. The Summary can be made available to participants in two different ways:
    • Transferred to an external application so that participants can edit the Summary.
    • Displayed to avatars where:
      • Avatars make Speech or Text comments (e.g., offline via chat).
      • The Virtual Secretary edits the Summary by understanding Text, Speech, and the avatars’ Personal Statuses.
  5. Outputs the Summary and the Virtual Meeting Secretary’s Portable Avatar.

2 Reference Model

Figure 1 depicts the Reference Model of the Virtual Meeting Secretary (PAF‑VMS) AIM.

 

Figure 1 – Reference Model of the Virtual Meeting Secretary (PAF‑VMS) AIM

3 I/O Data

Table 1 gives the Input and Output Data of the Virtual Meeting Secretary (PAF‑VMS) AIM.

Table 1 – I/O Data of the Virtual Meeting Secretary (PAF‑VMS) AIM

Input From Description
Portable Avatar Server Participants’ Portable Avatars as re-multiplexed by Server (External Port).
Output To Description
VMS Portable Avatar Server VMS Portable Avatar to Server (External Port).
Summary Server Summary of avatars’ interventions, to Server (External Port).

4 SubAIMs

4.1 Reference Model

Figure 2 depicts the Reference Model of the Virtual Meeting Secretary (PAF‑VMS) Composite AIM.

 

Figure 2 – Reference Model of the Virtual Meeting Secretary (PAF‑VMS) Composite AIM

4.2 Operation

The Virtual Secretary processes one avatar at a time. Since the Summary goes from Summary Creation to Entity Dialogue Processing and back, the Virtual Secretary runs continuously.

  1. Portable Avatar Demultiplexing produces Input Text, Input Speech, Avatar ID and Personal Status.
  2. Automatic Speech Recognition extracts Recognised Text from Input Speech.
  3. Natural Language Understanding receives Recognised Text or Input Text, and produces Refined Text and Meaning.
  4. Summary Creation Module receives Avatar ID, Personal Status, Refined Text and Meaning, and the Edited Summary from Entity Dialogue Processing where there is one, and produces the Summary.
  5. Entity Dialogue Processing receives Input Text, Refined Text, Meaning, Personal Status and Summary; sends the Edited Summary back to Summary Creation Module and the Summary out; and, when the avatar addresses the Virtual Secretary, produces VMS Text, VMS Personal Status and VMS Avatar Model.
  6. Personal Status Display produces the VMS Portable Avatar from VMS Text, VMS Personal Status and VMS Avatar Model.

4.3 Functions of SubAIMs

Table 2 gives the functions of the Virtual Meeting Secretary (PAF-VMS) SubAIMs.

Table 2 – Functions of the Virtual Meeting Secretary (PAF-VMS) SubAIMs

SubAIM Function
Portable Avatar Demultiplexing Receives Portable Avatar and provides Input Text, Input Speech, Avatar ID and Personal Status.
Automatic Speech Recognition Receives Input Speech and provides Recognised Text.
Natural Language Understanding Refines Recognised Text or Input Text and extracts Meaning.
Summary Creation Module Receives Avatar ID, Personal Status, Refined Text, Meaning and Edited Summary, and produces and refines the Summary.
Entity Dialogue Processing Receives Input Text, Refined Text, Meaning, Personal Status and Summary; produces Edited Summary, Summary and, when addressed, VMS Text, VMS Personal Status and VMS Avatar Model.
Personal Status Display Receives VMS Avatar Model, VMS Personal Status and VMS Text, and produces the VMS Portable Avatar.

4.4 I/O Data of SubAIMs

Table 3 gives the Input and Output Data of the Virtual Meeting Secretary (PAF-VMS) SubAIMs.

Table 3 – I/O Data of the Virtual Meeting Secretary (PAF-VMS) SubAIMs

SubAIM Input Output
Portable Avatar Demultiplexing Portable Avatar Input Text
Input Speech
Avatar ID
Personal Status
Automatic Speech Recognition Input Speech Recognised Text
Natural Language Understanding Input Text
Recognised Text		 Refined Text
Meaning
Summary Creation Module Avatar ID
Personal Status
Refined Text
Meaning
Edited Summary		 Summary
Entity Dialogue Processing Input Text
Refined Text
Meaning
Personal Status
Summary		 Edited Summary
Summary
VMS Text
VMS Personal Status
VMS Avatar Model
Personal Status Display VMS Text
VMS Personal Status
VMS Avatar Model		 VMS Portable Avatar

4.5 AIMs and JSON Metadata

Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.

Table 4 – AIMs and JSON Metadata of the Virtual Meeting Secretary (PAF‑VMS)

AIM1 AIM2 Name JSON
PAF‑VMS Virtual Meeting Secretary X
PAF‑PDX Portable Avatar Demultiplexing X
MMC‑ASR Automatic Speech Recognition X
MMC‑NLU Natural Language Understanding X
MMC‑SCM Summary Creation Module X
MMC‑EDP Entity Dialogue Processing X
PAF‑PSD Personal Status Display X

5 JSON Metadata

https://schemas.mpai.community/PAF/V1.6/AIMs/VirtualMeetingSecretary.json

6 Profiles

No Profiles.

7 Reference Software

Under development.

8 Conformance Testing

Table 5 provides the Conformance Testing Method for the Virtual Meeting Secretary (PAF‑VMS) AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 5 – Conformance Testing Method for the Virtual Meeting Secretary (PAF‑VMS) AIM

Receives Portable Avatar Shall validate against Portable Avatar schema.
Produces VMS Portable Avatar Shall validate against Portable Avatar schema.
Summary Shall validate against Summary schema.

9 Performance Assessment

Not part of this specification.

Go to MPAI-PAF V1.6 AI Modules