Highlights
- Online presentation: Server-based Predictive Multiplayer Gaming (MPAI-SPG) on 12 February
- Any time is a good time to become an MPAI Member
- How can we mitigate data loss effects in Server-based Predictive Multiplayer Gaming?
- Data standards need Qualifiers
- Meetings in the coming January/February 2025 meeting cycle
Online presentation: Server-based Predictive Multiplayer Gaming (MPAI-SPG) on 12 February
Technical Report: Server-based Predictive Multiplayer Gaming (MPAI-SPG) – Mitigation of Data Loss effects V1.0 will be presented online according to the schedule reported below.
Type | Register here for presentation of | Code | Date/Time (UTC) |
Technical Report | Server-based Predictive Multiplayer Gaming (MPAI-SPG) – Mitigation of Data Loss effects (SPG-MDL) V1.0 | SPG | 2025/02/12T15 |
Any time is a good time to become an MPAI Member
MPAI has started this year 2025 with a good plan of what it intends to do. Here is a summary list that should convince you that, as MPAI has already published 14 standards in different domains, it is in a good position to deliver on its plans and that you should be part of these efforts.
AIF | Work on a “dynamic” version of the AI Framework standard |
AIH | Publish the AI for Health specification of Health Secure Platform (AIH-HSP) V1.0 |
CAE | Publish MPAI-CAE specification for Six Degrees of Freedom audio (CAE-6DF) V1.0 |
CAV | Publish MPAI-CAV specification of Architecture CAV-ARC) V1.1 and (CAV-TEC) V1.0 |
CUI | Publish MPAI-CUI specification of Company Performance Prediction (CUI-CPP) V2.0 |
EEV | Develop Call for Technologies of MPAI-EEV when the reference model is mature. |
Develop EEV CfT | |
EVC | Publish MPAI-EVC specification of Up-sampling Filter for Video applications EVC-UFV) V1.0 |
GME | Publish MPAI-GME specification V2.0 |
MMC | Develop the Perceptive and Agentive AI specification. |
MMM | Publish the MPAI-MMM specification of Technologies (MMM-TEC) V2.0 |
Publish the open-source implementation of MMM-TEC V2.0 | |
NNW | Develop Neural Network Traceability and Security specification |
OSD | Develop specifications responding to needs of MPAI organisational units. |
PAF | Develop specifications responding to needs of MPAI organisational units. |
PRF | Develop Profiles for AI Workflows and Modules |
TFA | Develop specifications responding to needs of MPAI organisational units. |
XRV | Publish MPAI-XRV specification of Live Theatrical Performance (XRV-LTP) V1.0 |
How can we mitigate data loss effects in Server-based Predictive Multiplayer Gaming?
The 52nd MPAI General Assembly (MPAI-52) has approved Server-based Predictive Multiplayer Gaming (MPAI-SPG) – Mitigation of Data Loss effects (SPG-MDL) V1.0. This is a Technical Report that provides a methodology to predict the game state of an online gaming server when some controller data is lost. The Prediction is obtained by applying Machine Learning algorithms based on historical data of the online game.
An online Multiplayer Game is based on a server. When this maintains consistency among all clients’ game instances it is called authoritative. It updates and broadcasts the game state using the controller data of all the clients. This function is harmed when controller data are not correctly received or are maliciously modified.
There are several techniques currently used to cure this situation. In Client Prediction, client game state is updated locally using predicted or interpolated data while waiting for the server data; in Time Delay, the server buffers the game state updates to synchronise all clients; and in Time Warp the server rolls back the game state to when controller data was sent by a client and acts as if the action was taken then, reconciling this new game state with the current game state.
These three methods have shortcomings. Client Prediction causes perceptible delay, Time Delay affects responsiveness, and Time Warp disadvantages other players because the new game state likely differs from the previous one.
Figure 1 depicts the arrangement proposed by SPG-MDL. The right-hand side represents the online game server with the Game State Engine tasked to produce the game state using all clients’ controller data and specialised engines. The Engines of the figure are:
- Behaviour Engine, orchestrating actions from players and non-player entities.
- Rules Engine, ensuring adherence to game mechanics.
- Physics Engine, responsible for physical interactions within the game environment.
Figure 1 – Server Prediction
Whenever a client’s controller data is lost, the server requests the SPG-MDL module (the left-hand side of Figure 1) to compute the next game state. In this way, even if a client is experiencing network latency, the other clients maintain a continuous playing environment. The more accurate the predictions, the less noticeable the effect of the synchronisation process on the lagging client when the network resumes normal operations. A latency-affected client will still receive the results of its actions with a delay, but the server will send the new game state before receiving the action from the client, effectively halving the wait time. Of course, it is possible to further mitigate the effects of this problem by implementing additional client-side techniques, for example Client Prediction.
When some controller data is lost, the process begins with the last correct game state being fed into the SPG-MDL’s Game State Demultiplexer, which deconstructs it into discrete Game Messages* (). To differentiate the Game Messages inside the “twin” game server from the ones inside the game server, the ‘*’ symbol is attached to the SPG-MDL Game Messages. Each Game Message* is then processed by its respective Engine AI, leveraging a Neural Network Model to produce a predicted Game Message* (GMt+1). These predictions are assembled by the Predicted Game State Multiplexer into the predicted Game State (), which is then sent back to the game server for the next iteration of Game State computation.
While the SPG-MDL module operates, the server independently computes its updated Game State (pGSt+1) using the available Client Data. The server utilises the predicted Game State as follows. If any Client Data is missing, the server uses the predicted game state to compensate for the missing data shortfall from one or more clients. Note that the online game server architecture is a reference model and that the three engines are not a requirement for a specific game applying the MPAI-SPG methodology. For example, some games may not have a Physics Engine because physical-based behaviour is not required.
SPG-MDL provides a 10-step procedure to develop an SPG-MDL that is applicable to any authoritative game server.
The first 4 steps are required to outline the game setup to enable informed decisions for the implementation of SPG.
- Select the game.
- Define the Entities (to identify NN Model parameters): Environment and Human-controlled players (HPC) and Non-player characters (NPC).
- Define the Game State and relevant Entities.
- Design the training dataset.
- Collect the training dataset.
- Train prediction NN Models defining viable architectures and training parameters and comparing the training results of different architectures.
- Implement SPG-MDL.
- Evaluate SPG-MDL to select the model yielding the best predictions.
- Implement modules which simulate the disturbances.
- Evaluate the SPG-MDL-enabled game experience with human players.
For each of the ten steps, the Technical Report provides:
- High-level guidelines to outline the actions required.
- An example of how the guidelines are implemented using a car racing game (Figure 2).
Figure 2 – Modular tiles and an example of a racetrack
The following components are provided:
- The car racing game.
- Four different categories of Agent Players trained using Unity’s ML-Agents library.
- The dataset used for training generated by simulating game sessions played by the Agent Players.
- Jupyter Notebooks for training experiments and results.
- The trained models used by the AI-Behaviour Engine.
The software is available online. Registration is required to access the repository. Please inform the MPAI Secretariat before registration. Details on how to use the material are provided in the repository’s README page.
Data standards need Qualifiers
MPAI is about AI-enabled data coding across a very wide range of industries. Some industries are unable to negate dearly held traditions and make firm decisions, like MPEG standards did by specifying technology up to the last bit. MPAI must take this reluctance as a requirement for some of its standards.
How can we then make standards that are technology agnostic?
A standard should respond to a need and digging inside that need is a prerequisite for a meaningful standard. Most standards can be technology-neutral as a first step. Then there may be another phase where precise technologies are specified.
Let’s take the case of the XR Venue (MPAI-XRV) project where word Venue is used as a synonym for either Real or Virtual Environments. MPAI-XRV addresses a multiplicity of use cases enabled by Extended Reality (XR) – the combination of Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) technologies – enhanced by Artificial Intelligence (AI) technologies.
Live Theatrical Performance is one of the MPAI-XRV projects with a reference model depicted in Figure 3.
Figure 3 – Live Theatrical Performance (XRV-LTP)
XRV-LTP can acquire data from the Real Environment (real world) and from parallel Virtual Environments (metaverses). The data it can acquire includes Audio, Visual, MoCap, LiDAR, Biometrics, and more. Should it specify single formats for Audio, Volumetric, MoCap, and LiDAR? There is no way industry would follow that.
Another example is the MPAI-MMM (metaverse) project where the data types involved are manifold with new data types expected to come to the fore in the next year. Likewise, for the MPAI-AIH (health) project where the data types are many and with many formats each.
MPAI has devised the Data Types, Formats and Attributes (MPAI-TFA) standard, a growing collection of data types for each of which the Sub-Types, Formats and Attributes are specified. For instance, using MPAI-TFA, it is possible to indicate that an instance of the Visual Data Type:
- Sub-Type
- Uses the Rec. ITU-R BT.2020 colour space
- Has an alpha channel of 0.75
- Has a brightness of 1000 Nits
- Was 4:2:2 subsampled
- Is represented with 10 bits/pixel
- Format
- Is time sampled at 50 Hz
- Is space sampled at 100 dpi
- Is a static 3D visual object with a bounding box
- Has the OBJ format
- Attribute
- Has real attributes (captured from the real world)
- Uses Dublin Core metadata
- Contains objects with IDs
- There is a human expressing happiness
- The visual object was captured by a device with an ID
- The device had a Position and Orientation in the capture space
The example above illustrates capabilities of the MPAI-TFA standard and is not an exhaustive list.
MPAI designates Qualifier as the collection of these data elements and it defines Data Object as a Data Type with its Qualifier.
Most MPAI standards now reference the MPAI-TFA standard. The table below lists the data types for which Qualifiers have been (normal font) or are in the process of being (italic font) specified. The specification is available online.
Automotive | GNSS | LiDAR | Offline Map | RADAR | Ultrasound | ||
Health | EEG | ECG | Genomics | Medical Images | |||
Machine Learning | ML Model | ||||||
Media | 3D Model | Audio | Audio-Visual | MoCap | Speech | Text | Visual |
Metaverse | Contract | Discovery | Information | Interpretation | Program | Rules | |
Space-Time | Location | Time |