The Data Types are organised by the standards that specify them in categories and the definitions in two columns.

Currently, this page provides access to the Data Types specified by MPAI-CAE,  MPAI-CUI, MPAI-MMC, MPAI-OSD, and MPAI-PAF. Version 2.2 AIWs, AIMs, and Data Types are still not officially approved.

AIF CAE CAV CUI HMC MMC MMM NNW OSD PAF
ARC ARC
TEC TEC

Artificial Intelligence Framework(MPAI-AIF)

Data Types 2.1 2.2 AI Modules 2.1 2.2 AI Workflows 2.1 2.2
Neural Network Model X

Context-based Audio Enhancement (MPAI-CAE)

Data Types 2.1 2.2 AI Modules 2.1 2.2 AI Workflows 2.1 2.2
Audio X Audio Analysis for Preservation X Emotion-Enhanced Speech X
Audio Basic Scene Descriptors X Audio Analysis Transform X X Audio Recording Preservation X
Audio Basic Scene Geometry X Audio Basic Scene Description Speech Restoration System X
Audio Object X X Audio Descriptors Multiplexing X X Enhanced Audioconference Experience X
Audio Scene Descriptors X X Audio Description Packaging X
Audio Scene Geometry X X Audio Object Identification X X
Damaged List X X Audio Scene Description X X
Editing List X X Audio Separation and Enhancement X X
Enhanced Audio X Audio Source Localisation X X
Input Audio X Audio Synthesis Transform X X
Irregularity File X X Emotion Feature Production X X
Microphone Array Geometry X X Noise Cancellation Module X X
Multichannel Audio X Neural Emotion Insertion X X
Multichannel Audio Stream X Packaging for Audio Preservation X X
Output Audio X Prosodic Emotion Insertion X X
Preservation Files X Speech Detection and Separation X X
Transform Audio X Speech Feature Analysis 1 X X
Transform Enhanced Audio X Speech Feature Analysis 2 X X
Transform Multichannel Audio X Sound Field Description X X
Speech Model Creation X X
Speech Restoration Assembly X X
Speech Synthesis for Restoration X X
Tape Audio Restoration X X
Tape Irregularity Classification X X
Video Analysis for Preservation X X

Connected Autonomous Vehicle (MPAI-CAV) – Technologies (CAV-TEC)

Data Types 1.0 AI Modules 1.0 AI Workflows 1.0
Accelerometer Data X AMS Command Interpretation X Autonomous Motion Subsystem X
Alert X AMS Command Issuance X CAV-to-Everything X
AMS-HCI Message X AMS Decision Recording X Environment Sensing Subsystem X
Basic Environment Descriptors X Basic Environment Description X Human-CAV Interaction X
Brake Command X Full Environment Description X Motion Actuation Subsystem X
Brake Response X LiDAR Scene Description X
CAV Identifier X MAS Response Analysis X
Ego-Remote AMS Message X Motion Selection Planning X
Full Environment Descriptors X Online Map Scene Description X
Goal X Path Selection Planning X
HCI-AMS Message X RADAR Scene Description X
Input GNSS X Route Selection Planning X
Input LiDAR X Spatial Attitude Generation X
Input RADAR X Traffic Obstacle Avoidance X
Input Ultrasound X Traffic Signalisation Description X
Lidar Scene Descriptors X Ultrasound Scene Description X
MAS-AMS Message X
Motor Command X
Motor Response X
Odometer Data X
Offline Map Data X
Offline Map Scene Descriptors X
Other Environment Descriptors X
Path X
Pose X
RADAR Scene Descriptors X
Road State X
Route X
Speedometer Data X
Steering Command X
Steering Response X
Traffic Signal Descriptors X
Trajectory X
Ultrasound Scene Descriptors X

Compression and Understanding of Industrial Data (MPAI-CUI)

Data Types 1.1 AI Modules 1.1 AI Workflows 1.1
Prediction Horizon X Discontinuity and Default Prediction X Company Performance Prediction X
Financial Statement X Financial Data Assessment X
Governance X Governance Data Assessment X
Risk Assessment X Prediction Result Perturbation X
Financial Features X Risk Matrix Generation X
Governance Features X
Risk Matrix X
Default Probability X
Organisational Model Index X
Business Discontinuity Probability X

Human and Machine Communication (MPAI-HMC)

Data Types 1.0 1.1 AI Modules 1.0 1.1 AI Workflows 1.0 1.1
AV Scene Integration and Description X X Communicating Entities in Context X X
Entity and Context Understanding X X

Multimodal Conversation (MPAI-MMC)

Data Types 2.1 2.2 AI Modules 2.1 2.2 AI Workflows 2.1 2.2
Cognitive State X X Audio Segmentation X Bidirectional Speech Translation X
Emotion X X Automatic Speech Recognition X X Conversation About a Scene X
Intention X X Entity Dialogue Processing X X Conversation with Emotion X
Language Identifier X Input Speech Description X Conversation with Personal Status X
Meaning X X Input Text Description X Human-CAV Interaction X X
Personal Status X X Natural Language Understanding X X Multimodal Question Answering X
Recognised Text X Personal Status Extraction X One-to-Many Speech Translation X
Refined Text X Personal Status Multiplexing X Unidirectional Speech Translation X
Social Attitude X X PS-Speech Interpretation X Virtual Meeting Secretary X X
Speech X PS-Text Interpretation X
Speech Descriptors X X Speaker Identity Recognition X X
Speech Object X Summary Creation Module X
Speech Overlap X Text and Speech Translation X
Summary X X Text-To-Speech X X
Text X Text-to-Text Translation X
Text Descriptors X  X
Text Object X

MPAI Metaverse Model – Technologies

Data Types 1.0
Account X
Activity Data X
Amount X
Asset X
Basic M-Location X
Basic U-Location X
Contract X
Currency X
Discover X
E-Capabilities X
Identifier X
Inform X
Interpret X
M-Capabilities X
M-Environment X
Message X
M-Instance X
M-Location X
P-Capabilities X
Personal Data X
Personal Profile X
Process Action X
Program X
Provenance X
Request-Action X
Response-Action X
Rights X
Rules X
Stream X
Transaction X
U-Environment X
U-Location X
Universe-Metaverse Map X
Value X
Wallet X

Object and Scene Description (MPAI-OSD)

Data Types 1.0 1.1 AI Modules 1.0 1.1 AI Workflows 1.0 1.1
Anchored Direction X Audio-Visual Alignment X X Television Media Analysis X
Audio-Visual Basic Scene Descriptors X X Audio-Visual Event Description X  
Audio-Visual Basic Scene Geometry X Audio-Visual Scene Demultiplexing X X
Audio-Visual Event Descriptors X Audio-Visual Scene Description X X
Audio-Visual Object X X Audio-Visual Scene Multiplexing X
Audio-Visual Scene Descriptors X X Television Splitting X
Audio-Visual Scene Geometry X X Visual Change Detection X
Input Visual X X Visual Direction Identification X  
Instance Identifier X X Visual Object Extraction X
Output Visual X X Visual Object Identification X
Point of View X X Visual Scene Description X X
Selector X X
Space-Time X
Spatial Attitude X X
Time X X
Visual Basic Scene Descriptors X
Visual Basic Scene Geometry X
Visual Object X X
Visual Scene Descriptors X X
Visual Scene Geometry X X

Portable Avatar Format (MPAI-PAF)

Data Types 1.1 1.2 AI Modules 1.1 1.2 AI Workflows 1.1 1.2
Avatar X Audio-Visual Scene Rendering X X Avatar Videoconference Server X X
Avatar Model X Face Identity Recognition X X Videoconference Client Receiver X X
Body Descriptors X Input Body Description X Videoconference Client Transmitter X X
Body Object X Input Face Description X
Face Descriptors X Portable Avatar Demultiplexing X X
Face Object X PS-Face Interpretation X
Gesture Descriptors X PS-Gesture Interpretation X
Model X Personal Status Display X X
Portable Avatar X X Portable Avatar Multiplexing X X