| Function | Ref. Model | I/O Data | SubAIMs | JSON MData | Profiles | Ref. Software | Conformance | Performance |
1. Function
An Autonomous User (A-User) ha the functionalities specified by Technical Specification: MPAI Metaverse Model (MPAI-MMM) – Technologies (MMM-TEC) V2-2 and is implemented according to the AI Workflow of Technical Specification: AI Framework (MPAI-AIF), according to this PGM-AUA standard. An A-User has the following functions:
- Receives commands from a human responsible for it.
- Captures Perceptible Objects (Text Objects, Audio Objects, and Visual Objects) from an Audio-Visual Scene in an M-Instance. The latter includes any User with which the A-User interacts. This can be another Autonomous User or a Human User (H-User), i.e., a User that is under direct control of a human. Other objects may also be included.
- Processes the captured information.
- Produces a Portable Avatar rendered in the M-Instance and performs Actions or Process Action Requests. These are generated to achieve the results that the A-User concluded to be desirable in point 3.
- Receives Process Action Responses produced by the M-Instance in response to the Process Action Requests.
2. Reference Model
Figure 1 gives the Reference Model of the AI Workflow implementing the Autonomous User.
Figure 1 – Reference Model of Autonomous User Architecture (PGM-ATU)
3. Input/Output Data
Table 2 gives the Input/Output Data of the Autonomous User AIW.
Table 2 – Input/output data of the Autonomous User
| Input | Description |
| Human Command | A command from the responsible human overtaking or complementing the control of the A-User. |
| Process Action Response | Generated by the M-Instance Process in response to the A-User’s Process Action Request. |
| Text Object | User input as text. |
| Audio Object | The Audio component of the Scene where the User is embedded. |
| Visual Object | The Visual component of the Scene where the User is embedded. |
| Output | Description |
| Human Command Status | The A-User’s report about the execution of the Human Command. |
| Action | Action performed by the A-User. |
| Process Action Request | A-User’s Process Action Request. |
4. SubAIMs
4.1 Reference Model
Figure 1 gives the Reference Model of the Autonomous User (PGM-ATU) Composite AI Module implementing the Autonomous User.
Figure 1 – Reference Model of the Autonomous User (PGM-ATU) Composite AIM
4.2 Operation
The A-User operates as an instruction-driven system composed of several interacting sub-processes, orchestrated by a central controller (A-User Control).
The sub-processes are implemented as AI Modules (AIMs) executed in an AI Framework (AIF) according to Technical Specification: AI Framework (MPAI-AIF) V3.0.
A-User Control coordinates the activities of the A-User AIMs by issuing Directive messages and receiving Status messages.
Context Description produces Enhanced Descriptors by improving the understanding of the Space and by deriving an initial version of the User State from perceptual and contextual evidence. Domain information requested from Domain Access enables the A-User to understand the Audio and Visual Space and User State where required.
Prompt Creation uses the Enhanced Context, including the User State and any textual information, to construct and maintain a structured interaction representation capturing the current situation of the A-User vis-à-vis the Audio and Visual Space and the User. This structured representation is submitted to Basic Knowledge to initiate deliberative processing.
Basic Knowledge receives the structured interaction representation from Prompt Creation and determines the appropriate communicative message and associated behaviour of the A-User. To this end, Basic Knowledge may query Prompt Creation for reformulation, Domain Access for applicable rules and constraints, User State Refinement for improved understanding of the User State, and Personality Alignment to determine the A-User State, the stance of the A-User vis-à-vis the User State.
User State Refinement improves the A-User’s understanding of the User State, while Personality Alignment determines the A-User State that the A-User should assume to respond appropriately to the User.
Basic Knowledge produces the Final Response to be uttered by the A-User together with the corresponding A-User Entity State, ensuring congruence with the User Entity State and the context of interaction.
A-User Formation produces a Portable Avatar able to utter the Final Response produced by Basic Knowledge and expresses the A-User State through speech, facial expression, and gesture.
4.3 Functions of AI Modules
Table 3 gives the functions performed by PGM-ATU AIMs.
Table 3 – Functions of PGM-ATU AIMs
Note: The table does not analyse Directive/Status Data to and from A-User Control to PGM-AUA.
| Acronym | Name | Definition |
| PGM-AUC | A-User Control | The User Control AIM (PGM-AUC) governs the operational lifecycle of the A-User though its AIMs and orchestrates its interaction with both the M-Instance and the human. |
| PGM-CXE | Context Description | Captures and processes the Audio Objects and Visual Objects and accesses Domain Access to obtain Domain information allowing it to understand the Audio and Visual Space and the User and produce the Enhanced Audio Scene Descriptors, Enhanced Visual Scene Descriptors, and and initial version of the User State. |
| PGM-PRC | Prompt Creation | Transforms the acquired understanding of the Audio and Visual Space (Enhanced Audio Scene Descriptors and Enhanced Visual Scene Descriptors) and User State into a prompt (PR-Prompt) to Basic Knowledge. May receive queries from Basic Knowledge reasoning on PR-Prompt. |
| PGM-BKN | Basic Knowledge | A language model – not necessarily general-purpose – receiving PR-Prompts and interacting with Prompt Creation (PRC), Domain Access (DAC), User State Refinement (USR), and Personality Alignment (PAL)with the goal yo produce the Final Response. |
| PGM-DAC | Domain Access | Performs the following main functions: – Receives queries from Context Description. – Selects and activates domain-specific behaviours to deal with specific inputs from Context Enhancement. – Responds to Context Description and Basic Knowledge queries. |
| PGM-USR | User State Refinement | Interacts with Basic Knowledge to produce the final User State that it sends to Personality Alignment. |
| PGM-PAL | Personality Alignment | Receives the authoritative User State and interacts with Basic Knowledge to produce the final A-User State that it sends to A-User Formation . |
| PGM-AUR | A-User Formation | Receives the Final Response from Basic Knowledge, A-User State from Personality Alignment (PAL), and Command from A-User Control and produces a Portable Avatar. |
Note that all Autonomous User AIMs – save A-User Formation – may read/write information from/to A-User Storage as directed by A-User Control.
4.4 I/O Data of AI Modules
Table 4 provides acronyms, names, and links to the specification of the AI modules composing the PGM-AUA AIW and their input/output data. The current specification is tentative but is expected to evolve from input from Responses to the Call for Technologies.
Table 4 – Input/output Data of AI Modules
4.5 AIMs and JSON Metadata
Table 6 provides the links to the AIW and AIM specifications and to the JSON syntaxes.
Table 6 – AIW, AIMs, and JSON Metadata
| AIW | AIMs | Name | JSON |
| PGM-AUA | Autonomous User | X | |
| PGM-AUC | A-User Control | X | |
| PGM-CXT | Context Description | X | |
| PGM-PRC | Prompt Creation | X | |
| PGM-BKN | Basic Knowledge | X | |
| PGM-DAC | Domain Access | X | |
| PGM-USR | User State Refinement | X | |
| PGM-PAL | Personality Alignment | X | |
| PGM-AUR | A-User Formation | X |
5. JSON Metadata
https://schemas.mpai.community/PGM1/V1.0/Autonomous.json
6. Profiles
7. Reference Software
8. Conformance Testing
9. Performance Assessment
