Geneva, Switzerland – 14 June 2023. Today, the international, non-profit, and unaffiliated Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) organisation developing AI-based data coding standards has concluded its 33^rd General Assembly (MPAI-33) approving the Call for Technologies: MPAI Metaverse Model (MPAI-MMM) – Architecture. Responses are due by 10 July. Two online presentations of the Call will be made on 23 June at 8 and 15 UTC.

After publishing two Technical Reports on Functionalities and Functionality Profiles of the MPAI Metaverse Model, MPAI is now kicking off an ambitious plan to develop a Technical Specification on MPAI Metaverse Model – Architecture. This is a project that no standards body has ever attempted so far.

The first step of the plan is the publication of the Call for Technologies, as mandated by the MPAI standard development process. Note that the Call does not address data formats, only metaverse functionalities.

Three documents are attached to the Call: the first is Use Cases and Functional Requirements. It includes a reference to some thirty metaverse use cases explored by MPAI, a set of Functionalities that the Architecture should provide, and the functional requirements of its key elements: Processes, Items, Actions and Data Types.

The second document is the Framework Licence designed to facilitate the timely access to IP that is essential to implement the planned MPAI-MMM – Architecture standard. Finally, the third document is a Template for responses that respondents to the Call may wish to use in their responses.

Anybody may respond to the Call. However, non-members should join MPAI to participate in the development of the MPAI-MMM – Architecture standard.

MPAI is continuing its work plan comprising the development of the following Technical Specifications:

1. The AI Framework (MPAI-AIF) V2 Technical Specification will enable an implementer to establish a secure AIF environment to execute AI Workflows (AIW) composed of AI Modules (AIM).
2. The Avatar Representation and Animation (MPAI-ARA) V1 Technical Specification will support creation and animation of interoperable human-like avatar models able to understand and express a Personal Status.
3. The Multimodal Conversation (MPAI-MMC) V2 Technical Specification will generalise the notion of Emotion by adding Cognitive State and Social Attitude and specify a new data type called Personal Status.

The MPAI work plan also includes exploratory activities, some of which are close to becoming standard or technical report projects:

1. AI Health (MPAI-AIH). Targets an architecture where smartphones store users’ health data processed using AI and AI Models are updated using Federated Learning.
2. Connected Autonomous Vehicles (MPAI-CAV). Targets the Human-CAV Interaction Environment Sensing, Autonomous Motion, and Motion Actuation subsystems implemented as AI Workflows.
3. End-to-End Video Coding (MPAI-EEV). Extends the video coding frontiers using AI-based End-to-End Video coding.
4. AI-Enhanced Video Coding (MPAI-EVC). Improves existing video coding with AI tools for short-to-medium term applications.
5. Server-based Predictive Multiplayer Gaming (MPAI-SPG). Uses AI to train neural networks that help an online gaming server to compensate data losses and detects false data.
6. XR Venues (MPAI-XRV). Identifies common AI Modules used across various XR-enabled and AI-enhanced use cases where venues may be both real and virtual.

Legal entities and representatives of academic departments supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data can become MPAI members.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.

Abstract

1       Introduction. 1

2       A functional operation model 2

3       Actions. 3

4       Items. 5

5       Data Types. 6

6       Use cases. 7

6.1        AR Tourist Guide. 7

6.2        Virtual Dance. 8

7       Initial functionality profiles. 10

8       Conclusions. 11

9       References. 11

Abstract

MPAI has developed a roadmap for Metaverse interoperability. The published Technical Report – MPAI Metaverse Model – Functionalities [1] claims that, as standards for a market as vast as the one expected for the Metaverse are difficult to develop, functional profiles should be developed. The published draft Technical Report – MPAI Metaverse Model – Functionality Profiles [2] develops a Metaverse functional operation model, applies it to 7 Use Cases and proposes 4 initial functionality profiles. This document is a concise summary of [2].

1        Introduction

The MPAI Metaverse Model (MPAI-MMM) aims to provide Technical Reports and Technical Specifications that apply to as many kinds of Metaverse instances as possible and enable varied Metaverse implementations to interoperate.

At present, achieving interoperability is difficult because:

1. There is no common understanding of what a Metaverse is or should be, in detail.
2. There is an abundance of existing and potential Metaverse Use Cases.
3. Some independently designed Metaverse implementations are very successful.
4. Some important technologies enabling more advanced and even unforeseen forms of the Metaverse may be uncovered in the next several years.

MPAI has developed a roadmap to deal with this challenging situation. The first milestone of the roadmap is based on the idea of collecting the functionalities that potential Users expect the Metaverse to provide, instead of trying to define what the Metaverse is. The first Technical Report of this roadmap [1] includes definitions, makes assumptions for the MPAI-MMM project, identified sources that can generate functionalities, develops an organised list of commented functionalities, and analyses the main enabling technologies. Reference [3] provides a summary of [1].

Potential Metaverse Users with different needs might require different technologies to support these needs. Therefore, an approach that tried to achieve the goal of making every M-Instance be able to interoperate with every other M-Instance would force implementers to take technologies on board that are useless for their needs and potentially costly.

Reference [1] posits that Metaverse standardisation should be based on the notion of Profiles[1] and Levels[2] successfully adopted by digital media standardisation. A Metaverse Standard that includes Profiles and Levels would enable Metaverse developers to use only the technologies they need that are offered by whatever profile is most suitable to them.

The notion of profile can mitigate the impact of having many disparate Metaverse Users with diverse requirements. Unfortunately, that notion cannot be currently implemented because some key technologies are not yet available and at this time it is unclear which technologies, existing or otherwise, will eventually be adopted. To cope with this situation, [2] only targets Functionality Profiles, i.e., profiles that are defined by the functionalities they offer, not by technologies implementing them. Functionality Profiles are not meant to fully address the interoperability problem, but rather to allow a technology-independent definition of profiles based on the functional value they provide rather than on the “influence” of specific technologies.

2        A functional operation model

A Metaverse instance – called M-Instance – is composed of interacting Processes within and without the M-Instance. An M-Instance interoperates with another M-Instance to the extent its Processes interoperate.

MPAI has identified 3 classes of Process:

1. Users: Processes that represent humans using data from the real world (U-Locations) or autonomous agents (both need not be human-like).
2. Devices: Processes interconnecting U-Locations to M-Locations and vice-versa.
3. Services: Processes providing Functionalities.

A Process performs or requests another Process to perform Actions on Items. Item is data with attached metadata possibly including Rightss enabling a Process to perform Actions on. Rights express the ability to perform an Action on an Item.

The names of some Actions are prefixed depending on where Action is performed:

1. MM- if the interAction takes place in Metaverse.
2. UM- if the interAction is between Universe to Metaverse (Universe is the real world).
3. MU- if the interAction is between Metaverse to Universe.

An interAction of Process #1 with Process #2 unfolds as follows:

1. Process #1 requests Process #2 to performs Actions on Items.
2. Process #2 executes the request if Process #1 has Rightss to call Process #2.
3. Process #2 responds to Process #1’s request.

Note that Processes and Items need not be in the same M-Instance. However, the possibility of having Actions performed on Items may be more limited if they are not in the same M-Instance.

Figure 1 depicts a Metaverse configuration with interacting Processes.

Figure 1- A Metaverse configuration

Devices connect humans and objects in a U-Location (a location in the real world – called Universe) to one or more Metaverse Service generating M-Instances as depicted in Figure 2.

Figure 2 – The relationship between the real world (Universe) and M-Instances

A Device UM-Captures scenes with its Sensors and MU-Render Entities (i.e., Item that can be perceived) though their Actuators.

To enable its User(s) to perform Actions in an M-Instance, a human may be asked to Register and provide a subset of their User data, e.g., Persona(e) and ID(s) of Wallet. The M-Instance provides an Account that univocally associates a Registered human with their Items with the following features:

1. A human may have more than one Account in more than one M-Instance.
2. A User has Rightss to act in the M-Instance associated with the human’s Account.
3. A human may have more than one Account and more than one User per Account.
4. A User exists after a human Registers with an M-Instance.
5. Different Users of an Account may have different Rightss.

The Rules of an M-Instance express the obligations undertaken by the Registering human represented by the User and the terms and conditions under which a User exists in an M-Instance and operates either there or in another M-Instance. Depending on the Rules, a User of a human Registered on an M-Instance may or may not interact with another M-Instance and Rightss enforcement on some Actions performed on some Items may be forfeited.

Data entering an M-Instance, e.g., by Reading or MM-Capturing may include metadata and the Rightss granted to a Process to perform Actions on the data.

Items bear an Identifier is uniquely associated to that Item. However, an Item may bear more than one identifier. It is assumed that identifiers have the following structure:

[M-InstanceID] [ItemID]; [M-InstanceID] [M-LocationID] [ItemID].

3        Actions

A User can call a Process to perform Actions on Entities. An Entity can be:

1. Authored, i.e., the User calls an Authoring Tool Service with an accompanying request to obtain Rights to act on the authored Entity.
2. MM-Added, i.e., the User requests that an Entity be added to an M-Location with a Spatial Attitude.
3. MM-Enabled, i.e., the User requests that a Process be allowed to MM-Capture an Entity that is MM-Added at an M-Location.
4. MM-Embedded, i.e., MM-Added and MM-Enabled in one stroke.
5. MM-Captured, i.e., the User requests that an Entity MM-Embedded at an M-Location be MM-Sent to a Process.
6. UM-Animated, i.e., the User requests that a Process change the features of an Entity using a Stream of data.
7. MM-Disabled, i.e., the User requests that the MM-Enabling of the Entity be stopped.
8. Authenticated, i.e., the User calls a Service to make sure that an Item is what it claims to be.
9. Interpreted, i.e., the User calls a Service to obtain an interpretation of Items in an M-Instance, e.g., translate a Speech Object MM-Embedded at an M-Location into a specific language.
10. Informed, i.e., the User calls a Service to obtain data about an Item.

An example of composite Action, i.e., an Action that involves a plurality of Actions and a plurality of Items is Track that enables a User to request Services to:

1. MM-Add a Persona at an M-Location with a Spatial Attitude.
2. UM-Animate the Persona MM-Added at an M-Location.
3. MU-Render specified Entities at the M-Location to a U-Location with Spatial Attitudes.

So far, the following Actions have been identified:

Of course, the Actions identified are not intended to be the only ones that will be needed by all M-Instances.

A Process requests another Process to perform an Action sending the following payload.

The requested Process will respond by sending the following payload:

4        Items

An Item can be:

1. Created: from data and metadata. Metadata may include Rights.
2. Changed: its Rightss are modified.
3. Discovered by calling an appropriate Service.
4. Written, i.e., stored.
5. Posted as an Asset (i.e., an Item that can be the object of a Transaction to a Marketplace) to a marketplace.
6. Transacted as an Asset.

An Item can belong to one of six categories:

1. Items characterised by the fact that they can be MM-Captured by a User.
2. Items that can cause an Entity to change its perceptible features.
3. Items that have space and time attributes.
4. Items that are finance related.
5. Items that are non-perceptible.
6. Items that are Process-related.

Entities are Items that can be perceived. Here are some relevant Items.

1. Event: the set of Entities that are MM-Embedded at an M-Location from Start Time until End Time.
2. Experience: An Event as a User MM-Captured it and the User’s Interactions with the Entities belonging to the Entity that spawned the Event.
3. Object: the representation of an object and its features. Currently, only the following object types are considered: Audio, Visual, and Haptic.
4. Model: An Object that can be UM-Animated by a Stream or a Process. A Persona is the Model of a human.
5. Scene: a dynamic composition of Objects described by Times and Spatial Attitudes.

The finance-related Items are:

1. Asset: An Item that may be the object of a Transaction and is embedded at an M-Location or Posted to a Service.
2. Ledger: the list of Transactions executed on Assets.
3. Provenance: the Ledger of an Asset.
4. Transaction: Item representing the changed state of the Accounts and the Rights of a seller User and a buyer User on an Asset and optionally of the Service facilitating/enabling the Transaction.
5. Value: An Amount expressed in a Currency.
6. Wallet: A container of Currency units.

So far, the following Items have been identified:

Of course, more Items will be identified as more application domains will be taken into consideration.

Each Item is specified by the following table. It should be noted that, in line with the assumptions made by the MPAI Metaverse roadmap and the current focus on functionalities, the formats of the Item Data are not specified.

5        Data Types

Data Types are data that are referenced in Actions or in Items. Currently, the following data types have been identified.

1. Address
2. Amount
3. Coordinates
4. Currency
5. Personal status
   1. Cognitive state
   2. Emotion
   3. Social attitude
6. Point
7. Spatial attitude
   1. position
   2. orientation
8. Time

6        Use cases

So far 7 Use Cases have been developed to verify the completeness of Actions, Items and data types specified. Use cases are also a tool that facilitates the development of functionality profiles.

To cope with the fact that Use Cases may involve several locations and Items, a notation has been developed exemplified by the following:

1. User_i MM-Embeds Persona₁, Persona_i.2, etc.
2. User_i Calls Process₁, Process_i.2, etc.
3. User_i MM-Embeds Persona_j, at M-Location_i.1, M-Location_i.2, etc.
4. User_i MU-Renders Entity_j at U-Location_i.1, U-Location_i.2, etc.
5. User_i MM-Sends object₂ to User_j.

The following is to be noted:

Here only two of the seven Use Cases of [2] will be presented.

6.1       AR Tourist Guide

6.1.1      Description

This Use Case describes a human who intends to develop a tourist application through an App that alerts the holder of a smart phone where the App is installed and lets them view Entities and talk to autonomous agents residing at M-Locations:

1. User₁
   1. Buys M-Location₁ (parcel) in an M-Environment.
   2. Creates Entity₁ (landscape suitable for a virtual path through n sub-M-Locations).
   3. Embeds Entity₁ (landscape) on M-Location_1.1 (parcel).
   4. Sells Entity₁ (landscape) and M-Location_1.1 (parcel) to a User₂.
2. User₂
   1. Authors Entity₁ to Entity_2.n for the M-Locations.
   2. Embeds the Entities at M-Location₁ to M-Location_2.n.
   3. Sells the result to User₃.
3. human₄
   1. Develops
      1. Map recording the pairs M-Location_i – U-Location_2.i
      2. App alerting a human₅ holding the Device with the App installed that a key U-Location has been reached.
   2. Sells Map and App to human₃.
4. User₃ MM-Embeds one or more autonomous Personae at M-Location₁ to M-Location_2.n.
5. When human₅ gets close to a key U-Location:
   • App prompts Device to Request User₃ to MU-Render the Entity_i MM-Embedded at M-Location_2.i to the key U-Location_2.i.
   • human₅ interacts with MU-Rendered Entity_i that may include an MM-Animated Persona_2.i.

6.1.2      Workflow

Table 1 – AR Tourist Guide workflow.

6.1.3      Actions, Items, and Data Types

6.2       Virtual Dance

6.2.1      Description

1. User₂ (dance teacher)
   • Teaches dance at a virtual classroom.
   • Works at M-Location₁ where its digital twin Persona_2.1 is Audio-Visually MM-Embedded.
   • MM-Embeds and MM-Animates Persona₂ (A-V) (another of its Personae) at M-Location_2.2 as virtual secretary to attends to students coming to learn dance.
2. User₁ (dance student #1):
   • MM-Embeds its Persona₁ (A-V) at Location_1.1 (its “home”).
   • Audio-Visual-Haptically MM-Embeds Persona₁ (A-V-H) at Location_1.2 close to Location_2.2.
   • Sends Object₁ (A) to Persona_2.2 (greets virtual secretary).
3. Virtual secretary:
   • Sends Object₁ (A) to dance students #1 (reciprocates greeting).
   • Send Object₂ (A) to call regular dance teacher’s Persona_2.1.
4. Dance teacher MM-Embeds (A-V-H) Persona₁ at Location_2.3 (classroom) where it dances with Persona_1.1 (dance student #1).
5. While Persona₁ (student #1) and Persona_2.1 (teacher) dance, User₃ (dance student #2):
   • MM-Embeds (A-V) Persona₁ (its digital twin) at Location_3.1 (its “home”).
   • MM-Embeds (A-V-H) Persona₁ to Location_3.2 (close to Location_2.2 where the secretary is located).
6. After a while, User₂ (dance teacher):
   • MM-Embeds (A-V-H) Persona₁ at Location_2.4, (close to Location_3.2 of dance student #2).
   • MM-Disables Persona₁ from Location_2.3 where it was dancing with Persona_1.1 (student #1).
   • MM-Embeds (A-V-H) and MM-Animates an autonomous Persona₃ replacing Persona_2.1 from Location_2.3 so that student #1 can continue practising dance.
   • Dances with Persona₁ (student #2).

6.2.2      Workflow

Table 2 – Virtual Dance workflow.

6.2.3      Actions, Items, and Data Types

7       Initial functionality profiles

The structure of the Metaverse functionality profiles derived from the above includes hierarchical profiles and one independent profile. Profiles may have levels. As depicted in Figure 2, the currently identified profiles are baseline, management, finance, and high. The currently identified levels for baseline, management, and high profiles are audio only, audio-visual, and audio-visual-haptic.

Figure 3 – the currently identified Functionality Profiles

As an example, the baseline functionality profile enables a human equipped with a Device to allow their Users to:

1. Author Entities, e.g., object models.
2. Sense a scene at a U-Location:
   • UM-Capture a scene.
   • UM-Send data.
3. MM-Embed Personae and objects:
   • MM-Add Persona and object.
   • UM-Animate a Persona, with a stream (UM-Animate) or using a Process (MM-Animate).
4. MU-Render an Entity at an M-Location to a U-Location.
5. MM-Capture Entities at an M-Location
6. MM-Disable an Entity.

This Profile supports baseline lecture, meeting, and hang-out Use Cases. TransActions and User management are not supported.

Table 3 lists the Actions, Entities, and Data Types of the Baseline Functionality Profile.

Table 3 – Actions, Entities, and Data Types of the Baseline Functionality Profile

The identified four profiles serve well the needs conveyed by the identified functionalities. As more functionalities will be added, the number of profiles and potentially of levels, is likely to increase.

8       Conclusions

Reference [2] demonstrates the feasibility of the first two milestones of the proposed MPAI roadmap to Metaverse interoperability. Currently, four functionality profiles supporting the selected functionalities have been identified and specified. As more basic Metaverse elements are added, however, more functional profiles are likely to be found necessary. Functionality profiles can be extended and restructured as more Functionalities will be added.

The next step of the MPAI roadmap to Metaverse interoperability is the development of Technical Specification – MPAI Metaverse Model (MPAI-MMM) – Metaverse Architecture.

9        References

1. MPAI; Technical Report – MPAI Metaverse Model – Functionalities (MPAI-MMM); January 2023; https://mpai.community/standards/mpai-mmm/mpai-Metaverse-model/mmm-functionalities/
2. MPAI; Technical Report – MPAI Metaverse Model – Functionality Profiles (MPAI-MMM); March 2023; https://mmm.mpai.community/
3. MPAI; A roadmap to Metaverse Interoperability; February 2023; FGMV-I-012

[1] A Profile is sets of one or more base standards and, if applicable, chosen classes, subsets, options, and parameters of those standards that are necessary for accomplishing a particular function.

[2] A Level is a subdivision of a Profile indicating the completeness of the User experience.

1. Introduction

The metaverse is expected to create new jobs, opportunities, and experiences with transformational impacts on virtually all sectors of human interaction. Therefore, metaverse standards are important, but:

1. There is no common agreement on what a “metaverse” is or should be.
2. There are many potential users of the metaverse.
3. There are many successful independent implementations of “metaverse”.
4. Some important enabling technologies may be years away.

We need to agree on how to tackle metaverse standardisation, without necessarily:

1. Reaching an agreement on what a metaverse is.
2. Disenfranchising potential users.
3. Alienating existing initiatives
4. Dealing with technologies (for now)

2. The MPAI proposal

MPAI – Moving Picture, Audio, and Data Coding by Artificial Intelligence – believes that developing a (set of) metaverse standards is very challenging goal. Metaverse standardiosation requires that we should:

1. Start small and grow.
2. Be creative and devise a new working method.
3. Test the method.
4. Gather confidence in the method.
5. Gather a wide consensus on the method.

Then, we could develop a (set of) metaverse standards.

MPAI has developed an initial roadmap:

1. Build a metaverse terminology.
2. Agree on basic assumptions.
3. Collect metaverse functionalities.
4. Develop functionality profiles.
5. Develop a metaverse architecture.
6. Develop functional requirements of the metaverse architecture data types.
7. Develop a Table of Contents of Common Metaverse Specifications.
8. Map technologies to the ToC of the Common Metaverse Specifications.

Step #1 – develop a common terminology

We need no convincing of the importance of this step as many are developing metaverse terminologies. Unfortunately, there is no attempt at converging to an industry-wide terminology.

The terminology should:

• Have an agreed scope.
• Be technology- and business-agnostic.
• Not use one industry’s terms if they are used by more than one industry.

The terminology is:

• Intimately connected with the standard that will use it.
• Functional to the following milestones of the roadmap.

MPAI has already defined some 150 classified metaverse terms and encourages the convergence of existing terminology initiatives. The MPAI terminology can be found here.

Step #2 – Agree on basic assumptions

Assumptions are needed for a multi-stakeholder project because designing a roadmap depends on the goal and on the methods used to reach it.

MPAI has laid down 16 assumptions which it proposes for a discussion. Here the first 3 assumptions are presented. All the assumptions can be found here.

• Assumption #1: As there is no agreement on what a metaverse is, let’s accept all legitimate requests of “metaverse” functionalities.
  • Note: an accepted functionality does not imply that a Metaverse Instance shall support it.
• Assumptions#2: Common Metaverse Specifications (CMS) will be developed.
  • Note: they will provide the technologies supporting identified Functionalities.
• Assumption#3: CMS Technologies will be grouped into Technology Profiles in response to industry needs.
  • Note: A profile shall maximise the number technologies supported by specific groups of industries.

The notion of profile was well known and used in digital media standardisation and is defined here:

A set of one or more base standards, and, if applicable, their chosen classes, subsets, options and parameters, necessary for accomplishing a particular function.

Step #3 – Collect metaverse functionalities

The number of industries potentially interested in deploying metaverse is very large and MPAI has explored 18 of them. See here. A metaverse implementation is also likely to use external service providers – interfaces should be defined. See here.

MPAI has collected > 150 functionalities organised in: Areas – Subareas – Titles. See here.

This is not an accomplished task, but its beginning. Collecting metaverse functionalities should be a continuous task.

Step #4 – Which profiles?

The notion of profile is not currently implementable because some key technologies are not yet available and it is not clear which technologies, exisiting or otherwise, will eventually be selected.

MPAI proposes to introduce a new type of profile – functionality profile, characterised by the functionalities offered, not by the technologies implementing them. By dealing only with functionalities and not technologies, profile definition is not “contaminated” by technology considerations. MPAI is in the process of developing:

Technical Report – MPAI Metaverse Model (MPAI-MMM) – Functionality Profiles.

It is expected that the Technical Report will be published for Community Comments on the 26^th of March and finally adopted on the 19^th of April 2023. It will contain the following table of contents:

1. A scalable Metaverse Operational Model.
2. Actions (what you do in the metaverse):
   1. Purpose – what the Action is for.
   2. Payload – data to the Metaverse.
   3. Response – data from the Metaverse.
3. Items (on what you do Actions):
   1. Purpose – what the Item is for.
   2. Data – functional requirements.
   3. Metadata – functional requirements.
4. Example functionality profiles.

The Technical Report will not contain nor make reference to technologies.

3. The next steps of the MPAI poposal

The Technical Report will demonstrate that it is possible to develop metaverse functionality profiles (and levels) that do not make reference to technologies, only to functionalities.

Step #5 – Develop a metaverse architecture.

The goal is to specify of a Metaverse Architecture, including the main functional blocks and the data types exchanged between blocks.

Step #6 – Develop functional requirements of the metaverse architecture data types.

The goal is to develop the functional requirements of the data types exchanged between functional blocks of the metaverse architecture.

Step #7 – Develop the Table of Contents of the Common Metaverse Specifications.

The goal is to produce an initial Table of Contents (ToC) of Common Metaverse Specifications to have a clear understanding of which Technologies are needed for which purpose in which parts of the metaverse architecture to achieve interoperability.

Step #8 – Map technologies to the ToC of the Common Metaverse Specifications.

MPAI intends to map its relevant technologies and see how they fit in the Common Metaverse Specification architecture. Other SDOs are invited to join the effort.

4. Conclusions

Of course, step #8 there will not provide the metaverse specifications but a tested method to produce them. MPAI envisages to reach step #8 in December 2023. It is a good price to pay before engaging in a perilous project.

Geneva, Switzerland – 22 February 2023. Today the international, non-profit, unaffiliated Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) standards developing organisation has concluded its 29^th General Assembly (MPAI-29) approving a new version of its Audio Enhancement (MPAI-CAE) Technical Specification posted for Community Comments and the Neural Network Watermarking (MPAI-NNW) Reference Software.

Version 2 of the Context-based Audio Enhancement (MPAI-CAE) Technical Specification, besides supporting the functionalities of Version 1, specifies new technologies to enable a device to describe an audio scene in terms of audio objects and their directions. MPAI uses this Technical Specification to enable human interaction with autonomous vehicles, avatar-based videoconference and metaverse applications. The document is posted with a request for Community Comments to be sent to secretariat@mpai.community until the 20^th of March 2023.

The Reference Software of Neural Network Watermarking (MPAI-NNW) provides the means, including the software, to evaluate the performance of neural network-based watermarking solutions in terms of imperceptibility, robustness, and computational cost. The version of the software is specific for image classification but can be extended to other application areas.

MPAI is continuing its work plan including the development of the following Technical Specifications:

1. The AI Framework (MPAI-AIF) V2 Technical Specification will enable an implementer to establish a secure AIF environment to execute AI Workflows (AIW) composed of AI Modules (AIM).
2. The Avatar Representation and Animation (MPAI-ARA) V1 Technical Specification will support creation and animation of interoperable human-like avatar models expressing a Personal Status.
3. The Multimodal Conversation (MPAI-MMC) V2 Technical Specification will generalise the notion of Emotion by adding Cognitive State and Social Attitude and specify a new data type called Standard for Personal Status.

The MPAI work plan also includes exploratory activities, some of which are close to becoming standard or technical report projects:

1. AI Health (MPAI-AIH). Targets an architecture where smartphones store users’ health data processed using AI and AI Models are updated using Federated Learning.
2. Connected Autonomous Vehicles (MPAI-CAV). Targets the Human-CAV Interaction Environment Sensing, Autonomous Motion, and Motion Actuation subsystems implemented as AI Workflows.
3. End-to-End Video Coding (MPAI-EEV). Extends the video coding frontiers using AI-based End-to-End Video coding.
4. AI-Enhanced Video Coding (MPAI-EVC). Improves existing video coding with AI tools for short-to-medium term applications.
5. Server-based Predictive Multiplayer Gaming (MPAI-SPG). Uses AI to train neural networks that help an online gaming server to compensate data losses and detects false data.
6. XR Venues (MPAI-XRV). Identifies common AI Modules used across various XR-enabled and AI-enhanced use cases where venues may be both real and virtual.

It is still a good opportunity for legal entities supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data to join MPAI.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.