Highlights

 

From the 70th General Assembly

 

The last MPAI General Assembly (MPAI-70) has achieved several important results:

  • The announcement of a seminar the MPAI End-to-End Video Coding project to be held online on the 15th of July at 14 UTC.
  • The publication of the draft MPAI as a Service (MPAI-MAS) V1.0 standard with a request for Community Comments together with an exciting demonstration of a real MPAI as a Service running on a cloud.
  • The publication of the draft AI Framework (MPAI-AIF) V3.0 standard also with a request for Community Comments.
  • The publication of the draft Audio Object Rendering (CAE-AOR) V1.0 standard, also with a request for Community Comments.
  • Finally, MPAI-70 decided to publish MPAI Metaverse Model – Technologies V2.2 as an MPAI standard.

 

A Seminar on a future for Video Coding – The MPAI End-to-end Video Coding (EEV) Project

 

Video coding technology has been pushed into high drive in recent years. In the AI era, standards for video compression remain high priority with traditional data processing more and more complemented by AI-based tools.

Since 2022, MPAI is engaged in a project for an end-to-end video codec entirely based on end-to-end optimized AI models. This project has attracted increasing attention by both academic and industrial entities. The project – open to non-members because it is still exploratory – has gone through a series of phases called EEV-0.x.

Each phase has improved performance by enriching the network design, training strategies and advanced neural coding tools. The latest EEV-0.6 software introduced scaled hierarchical inter-predictive coding and spatial/bi-directional conditional coding. Prior EEV papers have been published in ACM MM/IEEE DCC/IEEE TCSVT/IJCAI/CVPR.

Currently, MPAI-EEV is working on an extended framework for the next EEV-0.7 model looking to another jump in both rate-distortion performance and model robustness. On this occasion, MPAI-EEV is holding an online seminar to review what has been learned from the last years of investigation that enabled the EEV-0.6 reference model to outperform the VVC standard in random access mode and explore promising avenues of development for the coming EEV-0.7 reference mode.

 

Register at https://us06web.zoom.us/meeting/register/4eAZjzHdSIKOPa4ZwlqncQ to join the online seminar on 15 July at 14 UTC.

 

From a standard for local processing to a standard for cloud processing

 

MPAI started with the idea of an organisation that would bring the notion and benefits of standards to the AI world. The ideas of replacing monolithic AI with applications having a known structure of processing elements called AI Modules (AIM) with standard interfaces made it possible to realise the MPAI vision. Thus, from its early days MPAI worked to develop a standard for a locally deployed environment in which AI Workflows could be initialised, dynamically configured, and controlled.

By the end of its 2nd year of activities, MPAI published Technical Specification: AI Framework (MPAI-AIF). This standard specifies the functions and characteristics of the elements of the AIF architecture with a significant set of APIs. It also specifies a JSON schema that could be used to inform an AIF instance of the structure of AIMs that were expected to run on the AIF.

MPAI-AIF has become the starting point of a large share of MPAI activities since then and, at the last MPAI-70 General Assembly, MPAI has approved publication of MPAI-AIF Version 3.0.

What is new in this version? One admittedly not fundamental but revealing element is the abandonment of the notion of AI Workflow that became a “Composite AIM”. MPAI has a portfolio of hundreds of such AIMs. Some of them have up to three layers of hierarchy.

More fundamental, is the introduction of four new APIs. However, we need a little digression to explain why those API are needed and for what.

The business of cloud services is estimated to be around 700B$ per year. Currently, AI services – defined as creation, training, deployment, execution, or management of AI models for perception, reasoning, prediction, generation, recommendation, decision support, or autonomous action – may affect only 5-10% of the cloud business, but those figures are due to increase in the next few years.

Full portability of cloud-based AI services is not the normal practice today, but what if there was a standard that ensures such portability? MPAI thought that it was the standard body that could set that goal, develop functional requirements, issue a Call for Technologies, and develop a standard.

The first deliverable for that is called Technical Specification: MPAI as a Service (MPAI-MAS). It was submitted to the MPAI-70 General Assembly and published with a request for Community Comments.

The scope of this draft standard specifies the Service Controller Instance (SCI) that enables a Remote Client Application (RCA) to request execution of AI Modules – whether Simple or Composite – on an AI Framework. The RCA may deliver input data to the Ports of the AI Modules – as specified by the MPAI-AIF standard – and receive output data resulting from their execution.

Let’s see more details how the MPAI-MAS can work:

  1. An implementer requests the MPAI Store – the not-for-profit entity collecting, testing, and distributing AI Modules – to be recognised as an Implementer of MPAI AIMs.
  2. If the implementer qualifies, the MPAI Store grants them an Implementer ID.
  3. The implementer who has developed an AI Module stores AI Module and JSON metadata at a URL and communicates the URL to the MPAI Store.
  4. If the MPAI Store finds that the assets at the URL are acceptable (e.g., they conform with the MPAI standard) it computes a fingerprint.
  5. Let’s assume now that a Remote Client Application issues a Create Controller request to the MAS API Service.
  6. This creates a new named Controller Instance and notifies the Remote Client Application of that name.
  7. Remote Client Application issues a Start Module request for a suitably identified AIM to the MAS API Service.
  8. MAS API Service forwards the Start Module request to the named Controller.
  9. The Controller
    1. requests the list of matching AIMs from the MPAI Store.
    2. retrieves the selected AIM Metadata from the MPAI Store.
    3. creates description and management structures for the resources required for the AIM launch.
    4. initiates provisioning and life cycle management of the resources required for the AIM.
  10. The Deployment Service of the cloud where the MPAI-MAS runs provisions the resources for the AIM and accomplishes installation, configuration and operation.
  11. The installation and configuration procedures of the provisioned resources retrieve the required information from Implementer.
  12. The received information is checked for validity, through MPAI Store, and finally activated.

 

The new APIs are

  1. Submit AIM Metadata: Implementer submits AIM Metadata of their Implementation to the MPAI Store for their review and subsequent availability for use by an SCI.
  2. Request Matching AIM: SCI requests the list of AIMs matching a standard AIM name.
  3. Get AIM Metadata: Controller retrieves AIM Metadata and uses it for selecting the AIM suitable for the current purpose, evaluating the required resources, and deciding on deployment.
  4. Verify Implementation: The MPAI Store verifies the integrity of an externally downloaded implementation against MPAI Store records upon request by the Controller running an AIM.

 

MPAI-MAS heralds the advent of an era where a given AIF implementation (an AI application) can be executed anywhere.

 

Looking inside the MPAI Metaverse Model – Technologies standard

 

A metaverse instance (M-Instance) is an Information and Communication Technologies (ICT) platform implementing this MMM-TEC specification. An M-Instance is populated by Processes.

  • Processes
  1. May be imported(UM-Sent) by a human Registering with the M-Instance. The human:
    1. Requeststo open an Account of a certain class.
    2. May be requestedto provide their Personal Profile and perform a Transaction.
    3. Obtainsa set of Rights that their Processes may exercise.
    4. May need to have their Processes Certified before they can be imported.
  2. Operatewith various degrees of autonomy and interactivity under the responsibility of the M-Instance Manager, Third-Party Service Providers, or humans residing in the Universe, i.e., the real world.
  3. Perform
    1. Actionswhose names may begin with:
      1. MM:indicating Actions performed inside the M-Instance, e.g., MM-Animate, which uses a stream or command to animate a 3D Model Object with a Spatial Attitude (defined as Position, Orientation, and their velocities and accelerations).
      2. MU:indicating Actions originating in the M-Instance but influencing the Universe, e.g., MU-Add to place a physical object (R-Item) at a Location with a Spatial Attitude, or MU-Move to move it from one U-Location to another.
      3. UM:indicating Actions originating in the Universe and influencing the M-Instance, e.g., UM-Capture to acquire Data by capturing a scene or an object at a U-Location using a Qualifier of the Data.
    2. Applied on:
      1. Processes, and/or
      2. Items, signed assemblies of Data each composed of:
        1. A unique Identifier.
        2. Data proper to the Item that has been Identifiedin, and thus recognised by, the M-Instance.
        3. A Qualifierof the Data, providing the format, transport, and other characteristics of the specific instance of the Data.
        4. Capabilitiesthat include the Rights held by identified Processes on the Item.
        5. The Identifier of the Item that spawned it.
        6. Trace– the name of the Process that produced the Item and the Time it was produced.
        7. A standard format specified as a JSON Schema.
      3. Based on:
        1. The M-Instance or M-Environment Rules and their own Rights.
        2. The security protocols of the Process Instance Trust Frameworkto establish trust, adding a Service Pricing Model to the content of messages sent to other Processes.
      4. Requestanother Process to perform an Action, either on their own initiative or driven by the actions of humans or machines in the Universe, using the Inter-Process Protocol. This may follow a Transaction of a Value (i.e., an Amount in a Currency) to the Wallet of the requested Process based on a Service Pricing Model, when the requesting Process cannot (lacks the technology) or may not (lacks the Rights) perform the Process Action itself.
        1. The request is called a Process ActionRequest when the Action is requested of another Process. A Process Action Request is expressed as: deontic verb + Action + set of Complements (Nil/At/From/To/With) + Item or ProcessID, where:
          1. Deontic verbs are May, May Not, and Must, corresponding to Permission, Prohibition, or Obligation.
          2. Complements (e.g., Nil, At, From, To, With) are applied to Items and Processes.
        2. The response is called a Process Action Response and consists of one or more Complements (Nil/At/From/To/With) followed by an Item or ProcessID and a PA Status, where:
          1. The first part conveys the result of the executed Process Action Request.
          2. PA Status reflects success or failure in the execution of the Process Action Request.
        3. May holda set of Rights on Items or Processes, i.e., may perform the set of Process Actions listed as Rights of the Process. Rights may also be applied to an Item to specify which Processes may perform which Process Actions on it. Rights have Levels indicating that a specific set of Rights is:
          1. Internal, e.g., assigned by the M-Instance at Registration time according to the M-Instance Rules and Account type.
          2. Acquired, e.g., obtained on the initiative of the Process.
          3. Grantedto the Process or Item by another Process.
        4. Exposetheir Capabilities. A slightly modified format is used to expose the Capabilities of an M-Instance and of an Item.
        5. Are characterisedas:
          1. Servicesproviding specific functionalities, such as content authoring. They are classified as:
            1. Centralised Services, acting at the M-Instance level.
            2. Distributed Services, typically executed in a subset of an M-Instance (M-Environment) in coordination with Centralised Services, but which may also be executed independently as Distributed Services.
            3. Third-Party Services, offered by third-party service providers.
          2. Devicesconnecting the Universe to the M-Instance and the M-Instance to the Universe.
          3. Appsrunning on Devices.
          4. Usersrepresenting and acting on behalf of human entities residing in the Universe, and possibly rendered as Personae, i.e., avatars.
  • Virtual Economy

MMM-TEC offers specific functionalities designed to support a virtual economy.

An Asset is an Item intended for Transaction that includes, in addition to the standard Item elements, the Identifier of the Item that spawned it, the Provenance – the list of all Transactions executed on an Asset, first and last included – and Economic Metadata, namely: Currency, Service Pricing Model, Market Classes, Value Metadata IDs, and Marketplace Policy IDs.

  • Rules

The Rules shall be made accessible to all Users having business with and within the M-Instance and may include:

  1. The carriage of Version ID and Effective Time condition.
  2. The procedure whereby the Rules may be changed and a new Version made effective.
  3. The procedures for the investigation or enforcement of compliance with the Rules of an M-Instance.
  4. The procedure used by a Process to modify its Capabilities, e.g., its own economic Rules.
  5. Statistical information about the number of Assets available in a Market Class, the relationship between supply and demand, the frequency of time-dependent Asset exchange, and average price movement.
  6. Disclosure of conflicts of interest affecting fees, rankings, or visibility.
  7. Procedures for pausing markets, rolling back affected Transactions, and communicating incident status to participants.
  8. The types and number of Messages that a Process may send to a Centralised or Distributed Service.
  9. The degree of control applied to the Actions performed by a Process (full, random, or no control) and, in the case of random selection, the selection criteria.
  10. How Activity Data is recorded and in which conditions an Action can be recalled.
  11. The sanctions for infringing Actions.
  12. The ability of a Process to view the Rights of a Process or an Item.
  13. The rendering of and access to the Identity associated with an activated Persona.
  14. The conditions under which access to the Personal Profile is permitted.
  • Federated M-Instances

Two M‑Instances may establish an active Resolution session using the Inter-Process Protocol for different M-Instances. If they are bound by a governance/trust agreement, each M‑Instance may treat Items originating from the other as if locally originated. In this federated model, interoperable use of Rights relies on:

  1. The foreign Item being a signed assembly with Capabilities including Rights held by identified Processes.
  2. Globally unique Identifiers for Items and Processes.
  3. The receiving M‑Instance’s verification of Item integrity and compliance with local Rules at the Effective Time.

MMM‑TEC does not require a portable Rights‑proof artefact in this case; interoperable recognition follows from the combination of Resolution, signed Items, and external governance.

  • M-Instance Manager

An M-Instance is managed by an M-Instance Manager. At the initial time, the M-Instance Manager holds Rights covering the M-Instance and may decide to define certain subsets inside the M-Instance – called M-Environments – on which it has Rights and to which it may attach Rights. The following are examples of functions that may be relevant to an M-Instance implementation and may be retained by an M-Instance Manager:

  1. Management of the M-Instance.
  2. Establishment of M-Instance Rules.
  3. Installation and maintenance of Services.
  4. Definition of Account Types.
  5. Monitoring of Process Actions.
  6. Sanctions against infringers.
  7. Resolution of conflicts.
  8. Certification of Processes and Data Types.

MMM-TEC does not specify how an M-Instance verifies that a Process’s Actions are compliant with its Rights or the M-Instance Rules. An M-Instance may choose to verify the full set of Activity Data (the log of Process Actions performed), to make verifications based on claims by another Process, to perform random verifications, or to make no verification at all. Accordingly, MMM-TEC does not specify how an M-Instance Manager may sanction transgressing Processes.

 

Meetings in the July-August 2026 cycle.