<-Go to AI Workflows Go to ToC
| 1 Functions | 2 Reference Model | 3 I/O Data |
| 4 Functions of AI Modules | 5 I/O Data of AI Modules | 6 AIW, AIMs, and JSON Metadata |
| 7 Reference Software | 8 Conformance Texting | 9 Performance Assessment |
1 Functions
In Virtial Meeting applications, such as in the Avatar Based Videoconference, i.e., a videoconference where avatars participate realistically impersonating the human participants, the Virtual Secretary:
- Listens to the Speech of each avatar.
- Monitors their Personal Status.
- Drafts a Summary using the avatars’ Personal Status and Text obtained from Automatic Speech Recognition or directly via Text input in the meeting’s common language.
- The Summary can be made available to participants in two different ways:
- Transferred to an external application so that participants can edit the Summary.
- Displayed to avatars where:
- Avatars make Speech or Text comments (e.g., offline via chat).
- The Virtual Secretary edits the Summary by understanding Text, Speech, and the avatars’ Personal Statuses.
2 Reference Model
Figure 1 specifies the Reference Model of the Virtual Secretary AIW. It is assumed that Meaning represents both meaning of Input Text and meaning of Refined Text.
Figure 1 – Reference Model of the Virtual Meeting Secretary (MMC-VMS) AIW
The Virtual Secretary processes one avatar at a time according to the following workflow:
- Portable Avatar Demultiplexing produces Input Text, Input Speech, Avatar ID, Body Descriptors, and Face Descriptors.
- Automatic Speech Recognition extracts Text from Avatar Speech.
- Natural Language Understanding:
- Receives Recognised Text.
- Produces Refined Text (of Recognised Text) and Meaning.
- Personal Status Extraction:
- Receives Meaning, Speech, and Body and Face Descriptors.
- Produces the Personal Status of the avatar it is interacting with.
- Summary Creation Module:
- Receives Refined Text, Personal Status, and Meaning
- Produces Summary using Personal Status and Text in the meeting’s common language.
- Receives Edited Summary from Entity Dialogue Processing.
- Entity Dialogue Processing:
- Sends Summary to external application.
- Sends Edited Summary produced from Refined Text (from Speech), Avatar’s Text (from chat), Meaning, and Summary back to Summary Creation Module.
- Produces VMS Text and VMS Personal Status.
- Personal Status Display produces VMS Portable Avatar containing VMS Avatar Model, VMS Text, VMS Speech, and VMS Avatar Descriptors.
3 I/O Data
Table 1 gives the input/output data of Virtual Meeting Secretary.
Table 1 – I/O data of Virtual Meeting Secretary
| Input data | From | Description |
| Portable Avatar | Server | Participants’ Portable Avatars as re-multiplexed by Server |
| Output data | To | Descriptions |
| VMS Portable Avatar | Server | VMS Portable Avatar to Server |
| Summary | Server | Summary of avatars’ interventions |
4 Functions of AI Modules
Table 2 gives the functions of Virtual Meeting Secretary AIMs.
Table 2 – Functions of Virtual Meeting Secretary AI Modules
| AIM | Functions |
| Portable Avatar Demultiplexing | 1. Receives Portable Avatar. 2. Provides the Data required by Virtual Secretary’s AIMs. |
| Automatic Speech Recognition | 1. Receives Speech. 2. Provides Recognised Text. |
| Natural Language Understanding | 1. Refines Recognised Text. 2. Extracts Meaning. |
| Personal Status Extraction | 1. Receives Meaning, Input Speech, Body Descriptors, Face Descriptors. 2. Extracts Personal Status. |
| Summary Creation Module | 1. Receives Meaning, Refined Text, Avatar ID, Input Personal Status of Avatar ID, and Edited Summary (from Entity Dialogue Processing.) 2. Produces and refines Summary using Edited Summary. |
| Entity Dialogue Processing | 1. Receives Input Text, Refined Text, Meaning, Summary, Input Personal Status. 2. Produces Text, Virtual Secretary Personal Status, and Edited Summary. |
| Personal Status Display | 1. Receives Virtual Secretary’s Avatar Model, Personal Status, and Text. 2. Shows Virtual Secretary as Virtual Secretary Portable Avatar. |
5 I/O Data of AI Modules
Table 3 gives the AI Modules of the Virtual Secretary depicted in Figure 4.
Table 3 – AI Modules of Virtual Secretary
| AIM | Receives | Produces |
| Portable Avatar Demultiplexing | Portable Avatar | 1. Input Text 2. Input Speech 3. Avatar ID 4. Body Descriptors 5. Face Descriptors |
| Automatic Speech Recognition | Speech | Recognised Text |
| Natural Language Understanding | Recognised Text | 1. Refined Text 2. Meaning |
| Personal Status Extraction | 1. Meaning 2. Speech 3. Face Descriptors 4. Body Descriptors |
Personal Status |
| Summary Creation Module | 1. Meaning 2. Refined Text 3. Edited Summary |
Summary |
| Entity Dialogue Processing | 1. Refined Text 2. Portable Avatar 3. Meaning 4. Summary |
1. VMS Portable Avatar 2. VMS Text 3. Edited Summary |
| Personal Status Display | 1. VMS Text 2. VMS Avatar Model 2. VMS Personal Status |
VMS Portable Avatar |
6 AIW, AIMs, and JSON Metadata
Table 4 – AIMs and JSON Metadata
Note: AIM1/s are Composite AIMs, AIM/2s are Basic AIMs.
| AIW | AIMs/1 | AIMs/2 | Name | JSON |
| MMC-VMS | Virtual Meeting Secretary | X | ||
| PAF-PDX | Portable Avatar Multiplexing | X | ||
| MMC-ASR | Automatic Speech Recognition | X | ||
| MMC-NLU | Natural Language Understanding | X | ||
| MMC-PSE | Personal Status Extraction | X | ||
| MMC-ETD | Entity Text Description | X | ||
| MMC-ESD | Entity Speech Description | X | ||
| PAF-EFD | Entity Face Description | X | ||
| PAF-EBD | Entity Body Description | X | ||
| MMC-PTI | PS-Text Interpretation | X | ||
| MMC-PSI | PS-Speech Interpretation | X | ||
| PAF-PFI | PS-Face Interpretation | X | ||
| PAF-PGI | PS-Gesture Interpretation | X | ||
| MMC-PMX | Personal Status Multiplexing | X | ||
| MMC-SCM | Summary Creation Module | X | ||
| MMC-EDP | Entity Dialogue Processing | X | ||
| PAF-PSD | Personal Status Display | X | ||
| MMC-TTS | Text-To-Speech | X | ||
| PAF-EFD | Entity Face Description | X | ||
| PAF-EBD | Entity Body Description | X | ||
| PAF-PMX | Portable Avatar Multiplexing | X |
7 Reference Software
8 Conformance Testing
Table 5 provides the Conformance Testing Method for MMC-VMS AIM.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and conform with the Qualifier, if present.
Table 5 – Conformance Testing Method for MMC-VMS AIM
| Receives | Portable Avatars | Shall validate against Portable Avatar Schema. Speech Data shall conform with Summary Qualifier. |
| Produces | VMS Portable Avatar | Shall validate against Portable Avatar Schema. Speech Data shall conform with Summary Qualifier. |
| Summary | Shall validate against Summary Schema. Speech Data shall conform with Summary Qualifier. |
