The MPAI-HMC V2.0 specification

What MPAI-HMC is about

Description of the standard

Technical Specification: Human and Machine Communication (MPAI-HMC) is a project developing standards for advanced forms of human-machine communication. The current focus is the Communicating Entities in Context (CEC) AI Workflow enabling an Entity (human or Machine operating in a Context)  to have a multimodal communication with another Entity possibly in a different Context.

Watch the video recording of online presentation (YouTube, WimTV) Read PowerPoint presentation.

What MPAI-HMC is about

MPAI-HMC V2.0

Technical Specification – Human and Machine Communication (MPAI-HMC) V2.0 (pdf file)

MPAI is indebted to the following individuals for their contributions to MPAI-HMC V2.0: Miran Choi, Gérard Chollet, Philip Merrill, Mark Seligman, Jaime Yoon.

MPAI-HMC V1.1

Technical Specification – Human and Machine Communication (MPAI-HMC) V1.1 (pdf file)

MPAI is indebted to the following individuals for their contributions to MPAI-HMC V1.1: Andrea Basso, Marina Bosi, Andrea Bottino, Miran Choi, Gérard Chollet, Mert Burkay Çöteli, Alessandro D’Amelio, Huseyin Hacihabiboglu, Paolo Ribeca, Mark Seligman, Seunggeun Baek, Jaime Yoon.

Human and Machine Communication (MPAI-HMC) is an MPAI Technical Specification  that enables new forms of communication between an Entity – a human present or represented in a real or virtual space, or a machine represented in a virtual space as a speaking avatar, acting in a context – and another Entity using text, speech, face, gesture, and the audio-visual scene in which it is embedded, much as humans do. It integrates a wide range of technologies available from existing MPAI standards.

The Term “communication” is used to include the possibility for a Machine:

  1. To understand the semantics of the communicated information at different layers of depth.
  2. To produce a multimodal response expected to be congruent with the received information.

Material about MPAI-HMC V1.0

Technical Specification – Human and Machine Communication (MPAI-HMC) V1.0 integrates five MPAI Technical Specifications: NPAI-AIF, MPAI-CAE, MPAI-MMC, MPAI-OSD, and MPAI-PAF. Its development has required an unprecedented level of cooperation among different MPAI groups. MPAI is indebted to Marian Alexandru Ilies, Seunggeun Baek, Marina Bosi, Andrea Bottino, Miran Choi, Gérard Chollet, Mert Burkay Çöteli, Alessandro D’Amelio, Hüseyin Hacıhabiboğlu, Mark Seligman, Jaime Yoon for their efforts in the development of MPAI-HMC V1.0.

In the process of developing the MPAI-HMC Technical Specification, MPAI realised that only technologies that had been specified in existing standards – Context-based Audio Enhancement (NPAI-CAE) V2.1, Multimodal Conversation (MPAI-MMC) V2.1, Object and Scene Description (MPAI-OSD), and Portable Avatar Format (MPAI-PAF) – were needed. For this reason, no Call for MPAI-HMC Technologies has been published.

The MPAI-HMC working draft, was published as a linked web document with a request for Community Comments. Anybody was invited to submit WD comments to the MPAI Secretariat until 2023/02/19T23:59. The Framework Licence: Human and Machine Communication (MPAI-HMC) was published on that occasion. Comments to be considered when producing the MPAI-HMC Technical Specification.