The Data Types are organised by the standards that specify them in categories and the definitions in two columns.

Currently, this page provides access to the Data Types specified by MPAI-CAE,  MPAI-CUI, MPAI-MMC, MPAI-OSD, and MPAI-PAF. Version 2.2 AIWs, AIMs, and Data Types are still not officially approved.

CAE CAV CUI HMC MMC MMM NNW OSD PAF
ARC ARC
TEC TEC

Context-based Audio Enhancement (MPAI-CAE)

Data Types 2.1 2.2 AI Modules 2.1 2.2 AI Workflows 2.1 2.2
Audio X y Audio Analysis for Preservation X Emotion-Enhanced Speech X
Audio Object X X Audio Analysis Transform X X Audio Recording Preservation X
Audio Basic Scene Descriptors X Audio Basic Scene Description Speech Restoration System X
Audio Scene Descriptors X X Audio Description Multiplexing X X Enhanced Audioconference Experience X
Audio Scene Geometry X X Audio Description Packaging X X
Damaged List X X Audio Object Identification X X
Editing List X X Audio Scene Description X X
Enhanced Audio X y Audio Separation and Enhancement X X
Input Audio X y Audio Source Localisation X X
Irregularity File X X Audio Synthesis Transform X X
Microphone Array Geometry X X Emotion Feature Production X X
Multichannel Audio X X Noise Cancellation Module X X
Multichannel Audio Stream X X Neural Emotion Insertion X X
Output Audio X y Packaging for Audio Preservation X X
Preservation Files X y Prosodic Emotion Insertion X X
Transform Audio X y Speech Detection and Separation X X
Transform Enhanced Audio X y Speech Feature Analysis 1 X X
Transform Multichannel Audio X y Speech Feature Analysis 2 X X
Sound Field Description X X
Speech Model Creation X
Speech Restoration Assembly X
Speech Synthesis for Restoration X
Tape Audio Restoration X
Tape Irregularity Classification X
Video Analysis for Preservation X

Connected Autonomous Vehicle (MPAI-CAV) – Technologies (CAV-TEC)

Data Types 1.0 AI Modules 1.0 AI Workflows 1.0
Accelerometer Data X AMS Command Interpretation X Autonomous Motion Subsystem X
Alert X AMS Command Issuance X CAV-to-Everything X
AMS-HCI Message X AMS Decision Recording X Environment Sensing Subsystem X
Basic Environment Descriptors X Basic Environment Description X Human-CAV Interaction X
Brake Command X Full Environment Description X Motion Actuation Subsystem X
Brake Response X LiDAR Scene Description X
CAV Identifier X MAS Response Analysis X
Ego-Remote AMS Message X Motion Selection Planning X
Full Environment Descriptors X Online Map Scene Description X
Goal X Path Selection Planning X
HCI-AMS Message X RADAR Scene Description X
Input GNSS X Route Selection Planning X
Input LiDAR X Spatial Attitude Generation X
Input RADAR X Traffic Obstacle Avoidance X
Input Ultrasound X Traffic Signalisation Description X
Lidar Scene Descriptors X Ultrasound Scene Description X
MAS-AMS Message X
Motor Command X
Motor Response X
Odometer Data X
Offline Map Data X
Offline Map Scene Descriptors X
Other Environment Descriptors X
Path X
Pose X
RADAR Scene Descriptors X
Road State X
Route X
Speedometer Data X
Steering Command X
Steering Response X
Traffic Signal Descriptors X
Trajectory X
Ultrasound Scene Descriptors X

Compression and Understanding of Industrial Data (MPAI-CUI)

Data Types 1.1 AI Modules 1.1 AI Workflows 1.1
Prediction Horizon X Discontinuity and Default Prediction X Company Performance Prediction X
Financial Statement X Financial Data Assessment X
Governance X Governance Data Assessment X
Risk Assessment X Prediction Result Perturbation X
Financial Features X Risk Matrix Generation X
Governance Features X
Risk Matrix X
Default Probability X
Organisational Model Index X
Business Discontinuity Probability X

Human and Machine Communication (MPAI-HMC)

Data Types 1.0 1.1 AI Modules 1.0 1.1 AI Workflows 1.0 1.1
AV Scene Integration and Description X Communicating Entities in Context X X
Entity Context Understanding X X

Multimodal Conversation (MPAI-MMC)

Data Types 2.1 2.2 AI Modules 2.1 2.2 AI Workflows 2.1 2.2
Cognitive State X X Audio Segmentation X Bidirectional Speech Translation X
Emotion X X Automatic Speech Recognition X X Conversation About a Scene X
Intention X X Entity Dialogue Processing X X Conversation with Emotion X
Language Identifier X y Input Speech Description X Conversation with Personal Status X
Meaning X X Input Text Description X Human-CAV Interaction X X
Personal Status X X Natural Language Understanding X X Multimodal Question Answering X
Recognised Text X y Personal Status Extraction X One-to-Many Speech Translation X
Refined Text X y Personal Status Multiplexing X Unidirectional Speech Translation X
Social Attitude X X PS-Speech Interpretation X Virtual Meeting Secretary X X
Speech X y PS-Text Interpretation X
Speech Descriptors X X Speaker Identity Recognition X X
Speech Objext X Summary Creation Module X
Speech Overlap X Text and Speech Translation X
Summary X X Text-To-Speech X X
Text X y Text-to-Text Translation X
Text Descriptors X  X
Text Object X

MPAI Metaverse Model – Technologies

Data Types 1.0
Account X
Activity Data X
Amount X
Asset X
Basic M-Location X
Basic U-Location X
Contract X
Currency X
Discover X
E-Capabilities X
Identifier X
Inform X
Interpret X
M-Capabilities X
M-Environment X
Message X
M-Instance X
M-Location X
P-Capabilities X
Personal Data X
Personal Profile X
Process Action X
Program X
Provenance X
Request-Action X
Response-Action X
Rights X
Rules X
Stream X
Transaction X
U-Environment X
U-Location X
Universe-Metaverse Map X
Value X
Wallet X

Object and Scene Description (MPAI-OSD)

Data Types 1.0 1.1 AI Modules 1.0 1.1 AI Workflows 1.0 1.1
Anchored Direction X Audio-Visual Alignment X X Television Media Analysis X
Audio-Visual Object X X Audio-Visual Event Description X  
Audio-Visual Basic Scene Descriptors X X Audio-Visual Scene Multiplexing X
Audio-Visual Scene Descriptors X X Audio-Visual Alignment X X
Audio-Visual Event Descriptors X Audio-Visual Scene Demultiplexing X X
Audio-Visual Scene Geometry X y Audio-Visual Scene Description X X
Input Visual X X Television Splitting X
Instance Identifier X X Visual Change Detection X
Output Visual X X Visual Direction Identification X  
Point of View X X Visual Object Extraction X
Space-Time X Visual Object Identification X
Spatial Attitude X X Visual Scene Description X X
Time X X
Visual Basic Scene Descriptors X
Visual Object X X
Visual Scene Descriptors X X
Visual Scene Geometry X y

Portable Avatar Format (MPAI-PAF)

Data Types 1.1 1.2 AI Modules 1.1 1.2 AI Workflows 1.1 1.2
Avatar X Audio-Visual Scene Rendering X X Avatar Videoconference Server X X
Avatar Model X Face Identity Recognition X X Videoconference Client Receiver X X
Body Descriptors X Input Body Description X Videoconference Client Transmitter X X
Body Object X Input Face Description X
Face Descriptors X Portable Avatar Demultiplexing X X
Face Object X PS-Face Interpretation X
Gesture Descriptors X PS-Gesture Interpretation X
Model X Personal Status Display X X
Portable Avatar X X Portable Avatar Multiplexing X X
Selector X