| Function | Ref. Model | I/O Data | SubAIMs | JSON MData | Profiles | Ref. Software | Conformance | Performance |
1. Function
The A-User is a Module of the AI Framework (MPAI-AIF V3.0): its AIMs are listed in this specification. The Module does not contain the Controller, which builds and runs it, nor the User Agent, which is its interface with the worlds it lives between: the universe of the responsible human and the metaverse in which the A-User acts. The User Agent speaks only to the Controller.
An Autonomous User (A-User) ha the functionalities specified by Technical Specification: MPAI Metaverse Model (MPAI-MMM) – Technologies (MMM-TEC) V2-2 and is implemented according to the AI Workflow of Technical Specification: AI Framework (MPAI-AIF), according to this PGM-AUA standard. An A-User has the following functions:
- Receives commands from a human responsible for it, through the User Agent.
- Captures Perceptible Objects (Text Objects, Audio Objects, and Visual Objects) from an Audio-Visual Scene in an M-Instance. The latter includes any User with which the A-User interacts. This can be another Autonomous User or a Human User (H-User), i.e., a User that is under direct control of a human. Other objects may also be included.
- Processes the captured information.
- Produces a Portable Avatar rendered in the M-Instance and performs Actions or Process Action Requests. These are generated to achieve the results that the A-User concluded to be desirable in point 3.
- Receives Process Action Responses produced by the M-Instance in response to the Process Action Requests.
2. Reference Model
Figure 1 gives the Reference Model of the AI Workflow implementing the Autonomous User.
Figure 1 – Reference Model of Autonomous User Architecture (PGM-ATU)
3. Input/Output Data
Table 2 gives the Input/Output Data of the Autonomous User AIW.
Table 2 – Input/output data of the Autonomous User
| Input | Description |
| Human Command | A command from the responsible human, who lives in the universe. The User Agent of the Module acquires it (it may be expressed with audio, text or visual media) and the Controller passes it to the Module as a Human Command Request. |
| Process Action Response | Generated by the M-Instance Process in response to the A-User’s Process Action Request. |
| Text Object | User input as text. |
| Audio Object | The Audio component of the Scene where the User is embedded. |
| Speech Object | Speech of the human or of the User, when the User Agent has recognised that the audio it captured is speech. |
| AUC-AUS Response | Response of the Private Storage of the Module to the AUC-AUS Request of A-User Control, through the Controller. |
| Visual Object | The Visual component of the Scene where the User is embedded. |
| Output | Description |
| Human Command (Response) | The answer of the A-User to the Human Command (Mode RS): accepted, rejected, escalated or ignored. It goes to the human through the Controller and the User Agent. |
| Action | Action performed by the A-User. |
| Avatar Speech Object | The speech of the A-User, authorised by A-User Control, for the Speech Object Delivery Unit of the User Agent. |
| AUC-AUS Request | Request of A-User Control to the Private Storage of the Module, through the Controller: scopes, retention, life cycle and access rights. |
| Avatar Model | The model of the avatar, for the 3D Model Object Delivery Unit. |
| Face Descriptors | The animation of the face of the avatar, for the 3D Model Object Delivery Unit. |
| Body Descriptors | The animation of the body of the avatar, for the 3D Model Object Delivery Unit. |
| Process Action Request | A-User’s Process Action Request. |
4. SubAIMs
4.1 Reference Model
Figure 1 gives the Reference Model of the Autonomous User (PGM-ATU) Composite AI Module implementing the Autonomous User.
Figure 1 – Reference Model of the Autonomous User (PGM-ATU) Composite AIM
4.2 Operation
The A-User operates as an instruction-driven system composed of several interacting sub-processes, coordinated by A-User Control.
The sub-processes are implemented as AI Modules (AIMs) executed in an AI Framework (AIF) according to Technical Specification: AI Framework (MPAI-AIF) V3.0.
A-User Control coordinates the activities of the A-User AIMs by issuing Request messages and receiving Responses.
Context Description produces Enhanced Descriptors by improving the understanding of the Space and by deriving an initial version of the User State from perceptual and contextual evidence. Domain information requested from Domain Access enables the A-User to understand the Audio and Visual Space and User State where required.
Prompt Creation uses the Context Descriptors, including the User State and any textual information, to construct and maintain a structured interaction representation capturing the current situation of the A-User with respect to the Audio and Visual Space and the User. This structured representation is submitted to Basic Knowledge to initiate deliberative processing (i.e., processing based on semantic interpretation).
Basic Knowledge receives the structured interaction representation from Prompt Creation and determines the appropriate communicative message and associated behaviour of the A-User. To this end, Basic Knowledge may query Prompt Creation for reformulation; Domain Access for applicable rules and constraints; User State Refinement for improved understanding of the User State; and Personality Alignment to decide the A-User State, that is, the stance of the A-User with respect to the User State.
User State Refinement improves the A-User’s understanding of the User State, while Personality Alignment determines the A-User State that the A-User should assume to respond appropriately to the User.
Basic Knowledge produces the Final Response to be uttered by the A-User, together with the corresponding A-User Entity State, ensuring consistency with the User State and the context of interaction.
A-User Formation receives the Final Response from Basic Knowledge, and only from Basic Knowledge. It turns the Process Actions that concern the A-User as an entity into Motion Objects, produces a Portable Avatar able to utter the text and expresses the A-User State through speech, facial expression, and gesture. It gives the Portable Avatar, and the other Process Actions, to A-User Control.
A-User Control agrees or stops. If it agrees, it sends the Process Actions to the M-Instance and the Portable Avatar to Portable Avatar Demultiplexing, which separates the speech, the avatar model and the face and body descriptors; the Controller gives them to the User Agent, which delivers the speech and the animated avatar.
4.3 Functions of AI Modules
Table 3 gives the functions performed by PGM-ATU AIMs.
Table 3 – Functions of PGM-ATU AIMs
Note: The table does not analyse Request/Response Data to and from A-User Control to PGM-AUA.
| Acronym | Name | Definition |
| PGM-AUC | A-User Control | The A-User Control AIM (PGM-AUC) governs the operational lifecycle of the A-User through its AIMs and orchestrates its interaction with both the M-Instance and the human. |
| PGM-CXT | Context Description | Captures and processes the Audio Objects and Visual Objects and accesses Domain Access to obtain Domain information allowing it to understand the Audio and Visual Space and the User. Based on them, it produces the Enhanced Audio Scene Descriptors, Enhanced Visual Scene Descriptors, and and initial version of the User State. |
| PGM-PRC | Prompt Creation | Transforms the acquired understanding of the Audio and Visual Space (Enhanced Audio Scene Descriptors and Enhanced Visual Scene Descriptors) and User State into a prompt (PR-Prompt) addressed to Basic Knowledge. May receive queries from Basic Knowledge while it reasons about PR-Prompt. |
| PGM-BKN | Basic Knowledge | A language model – not necessarily general-purpose – receiving PR-Prompts and interacting with Prompt Creation (PRC), Domain Access (DAC), User State Refinement (USR), and Personality Alignment (PAL) with the goal of producing the Final Response. |
| PGM-DAC | Domain Access | Performs the following main functions: – Receives queries, and selects and activates domain-specific behaviours to deal with specific inputs from Context Description. – Responds to queries from Basic Knowledge. |
| PGM-USR | User State Refinement | Interacts with Basic Knowledge to produce the final User State to be sent to Personality Alignment. |
| PGM-PAL | Personality Alignment | Receives the authoritative User State and interacts with Basic Knowledge to produce the final A-User State to be sent to A-User Formation. |
| PGM-AUR | A-User Formation | Produces a Portable Avatar after receiving the Final Response from Basic Knowledge, A-User State from Personality Alignment (PAL), and Command from A-User Control. |
Note that all Autonomous User AIMs (except A-User Formation) may read/write information from/to Private Storage through the Controller as directed by A-User Control.
4.4 I/O Data of AI Modules
Table 4 provides acronyms, names, and links to the specification of the AI modules composing the PGM-AUA AIW and their input/output data. The current specification is tentative but is expected to evolve from input from Responses to the Call for Technologies.
Table 4 – Input/output Data of AI Modules
4.5 AIMs and JSON Metadata
Table 6 provides the links to the AIW and AIM specifications and to the JSON syntaxes.
Table 6 – AIW, AIMs, and JSON Metadata
| AIW | AIMs | Name | JSON |
| PGM-AUA | Autonomous User | X | |
| PGM-AUC | A-User Control | X | |
| PGM-CXT | Context Description | X | |
| PGM-PRC | Prompt Creation | X | |
| PGM-BKN | Basic Knowledge | X | |
| PGM-DAC | Domain Access | X | |
| PGM-USR | User State Refinement | X | |
| PGM-PAL | Personality Alignment | X | |
| PGM-AUR | A-User Formation | X |
5. JSON Metadata
https://schemas.mpai.community/PGM1/V1.0/AutonomousUser.json
6. Profiles
7. Reference Software
8. Conformance Testing
9. Performance Assessment
