Go to MPAI-MMC V2.5 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

In Virtual Meeting applications, such as in the Avatar Based Videoconference, i.e., a video-conference where avatars participate realistically impersonating the human participants, the Virtual Secretary:

  1. Listens to the Speech of each avatar.
  2. Monitors their Personal Status.
  3. Drafts a Summary using the avatars’ Personal Status and Text obtained from Automatic Speech Recognition or directly via Text input in the meeting’s common language.
  4. The Summary can be made available to participants in two different ways:
    • Transferred to an external application so that participants can edit the Summary.
    • Displayed to avatars where:
      • Avatars make Speech or Text comments (e.g., offline via chat).
      • The Virtual Secretary edits the Summary by understanding Text, Speech, and the avatars’ Personal Statuses.

2 Reference Model

Figure 1 depicts the Reference Model of the Virtual Meeting Secretary (MMC‑VMS) AIM.

Virtual Meeting Secretary MMC-VMS AIM

Figure 1 – Reference Model of the Virtual Meeting Secretary (MMC‑VMS) AIM

3 I/O Data

Table 1 gives the Input and Output Data of the Virtual Meeting Secretary (MMC‑VMS) AIM.

Table 1 – I/O Data of the Virtual Meeting Secretary (MMC‑VMS) AIM

Input From Description
Portable Avatar Server Participants’ Portable Avatars as re-multiplexed by Server.
Output To Description
VMS Portable Avatar Server VMS Portable Avatar to Server.
Summary Server Summary of avatars’ interventions.

4 SubAIMs

4.1 Reference Model

Figure 2 depicts the Reference Model of the Virtual Meeting Secretary (MMC‑VMS) Composite AIM.

Virtual Meeting Secretary MMC-VMS Composite AIM

Figure 2 – Reference Model of the Virtual Meeting Secretary (MMC‑VMS) Composite AIM

4.2 Operation

The Virtual Secretary processes one avatar at a time according to the following workflow:

  1. Portable Avatar De-multiplexing produces Input Text, Input Speech, Avatar ID, Body Descriptors, and Face Descriptors.
  2. Automatic Speech Recognition extracts Text from Avatar Speech.
  3. Natural Language Understanding:
    • Receives Recognised Text.
    • Produces Refined Text (of Recognised Text) and Meaning.
  4. Personal Status Extraction:
    • Receives Meaning, Speech, and Body and Face Descriptors.
    • Produces the Personal Status of the avatar it is interacting with.
  5. Summary Creation Module:
    • Receives Refined Text, Personal Status, and Meaning.
    • Produces Summary using Personal Status and Text in the meeting’s common language.
    • Receives Edited Summary from Entity Dialogue Processing.
  6. Entity Dialogue Processing:
    • Sends Summary to external application.
    • Sends Edited Summary produced from Refined Text (from Speech), Avatar’s Text (from chat), Meaning, and Summary back to Summary Creation Module.
    • Produces VMS Text and VMS Personal Status.
  7. Personal Status Display produces VMS Portable Avatar containing VMS Avatar Model, VMS Text, VMS Speech, and VMS Avatar Descriptors.

4.3 Functions of SubAIMs

Table 2 gives the functions of the Virtual Meeting Secretary (MMC‑VMS) SubAIMs.

Table 2 – Functions of the Virtual Meeting Secretary (MMC‑VMS) SubAIMs

SubAIM Function
Portable Avatar Demultiplexing Receives Portable Avatar and provides the Data required by Virtual Secretary’s SubAIMs.
Automatic Speech Recognition Receives Speech and provides Recognised Text.
Natural Language Understanding Refines Recognised Text and extracts Meaning.
Personal Status Extraction Receives Meaning, Input Speech, Body Descriptors, and Face Descriptors, and extracts Personal Status.
Summary Creation Module Receives Meaning, Refined Text, Avatar ID, Personal Status of Avatar ID, and Edited Summary from Entity Dialogue Processing, and produces and refines Summary.
Entity Dialogue Processing Receives Input Text, Refined Text, Meaning, Summary, and Personal Status, and produces VMS Text, VMS Personal Status, and Edited Summary.
Personal Status Display Receives VMS Avatar Model, VMS Personal Status, and VMS Text, and produces the VMS Portable Avatar.

4.4 I/O Data of SubAIMs

Table 3 gives the Input and Output Data of the Virtual Meeting Secretary (MMC‑VMS) SubAIMs.

Table 3 – I/O Data of the Virtual Meeting Secretary (MMC‑VMS) SubAIMs

SubAIM Input Output
Portable Avatar Demultiplexing Portable Avatar Input Text
Input Speech
Avatar ID
Body Descriptors Object
Face Descriptors Object
Automatic Speech Recognition Speech Recognised Text
Natural Language Understanding Recognised Text Refined Text
Meaning
Personal Status Extraction Meaning
Speech
Face Descriptors Object
Body Descriptors Object		 Personal Status
Summary Creation Module Meaning
Refined Text
Edited Summary		 Summary
Entity Dialogue Processing Refined Text
Portable Avatar
Meaning
Summary		 VMS Portable Avatar
VMS Text
Edited Summary
Personal Status Display VMS Text
VMS Avatar Model
VMS Personal Status		 VMS Portable Avatar

4.5 AIMs and JSON Metadata

Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.

Table 4 – AIMs and JSON Metadata of the Virtual Meeting Secretary (MMC‑VMS)

AIM1 AIM2 Name JSON
MMC‑VMS Virtual Meeting Secretary X
PAF‑PDX Portable Avatar Demultiplexing X
MMC‑ASR Automatic Speech Recognition X
MMC‑NLU Natural Language Understanding X
MMC‑PSE Personal Status Extraction X
MMC‑SCM Summary Creation Module X
MMC‑EDP Entity Dialogue Processing X
PAF‑PSD Personal Status Display X

5 JSON Metadata

https://schemas.mpai.community/MMC/V2.5/AIMs/VirtualMeetingSecretary.json

6 Profiles

No Profiles.

7 Reference Software

Under development.

8 Conformance Testing

Table 5 provides the Conformance Testing Method for the Virtual Meeting Secretary (MMC‑VMS) AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 5 – Conformance Testing Method for the Virtual Meeting Secretary (MMC‑VMS) AIM

Receives Portable Avatar Shall validate against Portable Avatar schema.
Produces VMS Portable Avatar Shall validate against Portable Avatar schema.
Summary Shall validate against Summary schema.

9 Performance Assessment

Not part of this specification.

Go to MPAI-MMC V2.5 AI Modules