Go to MPAI-PGM V1.0 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

The Automatic Speech Recognition (PGM‑ASR) AIM extracts the text conveyed by an utterance and derives the paralinguistic features of the speech signal. The input speech may be accompanied by an auxiliary text, the identifier of the speaker, the Speech Overlap data type, and the time indicating the portion of the input speech that should be recognised. Operation is governed by the User CXT Request, which specifies scope, depth, and policy constraints. Processing outcome is reported via the User CXT Response. Domain knowledge and interaction history may be queried by issuing User Domain and User Interaction History Requests and receiving the corresponding Responses.

PGM‑ASR extends its output with Speech Descriptors, providing downstream AIMs with paralinguistic information — including prosody, vocal affect, and speaker characteristics — derived from the same speech signal used for recognition.

Receives Language Selector Signalling the language of the speech.
Auxiliary Text Text that may be used to provide context information.
Speech Object Speech to be recognised.
Speaker ID ID of speaker uttering speech.
Speech Overlap Data type providing information on speech overlap.
Speaker Time Time during which the speech is to be recognised.
User CXT Request Contextual directive specifying scope, depth, and policy constraints for speech recognition.
User Domain Response Domain‑specific knowledge supporting recognition and descriptor derivation.
User Interaction History Response Interaction History response providing temporal context for recognition.
Produces Recognised Text Text transcript of the recognised speech.
Speech Descriptors Paralinguistic features of the recognised speech segment.
User CXT Response Response reporting the processing outcome of the AIM.
User Domain Request Request for domain‑specific knowledge issued to Domain Access.
User Interaction History Request Request for Interaction History issued to Private Storage.

2 Reference Model

Figure 1 depicts the Reference Model of the Automatic Speech Recognition (PGM‑ASR) AIM.

Automatic Speech Recognition PGM-ASR V1.0

Figure 1 – The Automatic Speech Recognition (PGM‑ASR) AIM

3 I/O Data

Table 1 specifies the Input and Output Data of the Automatic Speech Recognition (PGM‑ASR) AIM.

Table 1 – I/O Data of the Automatic Speech Recognition (PGM‑ASR) AIM

Input Description
Language Selector Selects input language.
Auxiliary Text Object Text Object with content related to Speech Object.
Speech Object Speech Object emitted by Entity.
Speaker Identifier Identity of Speaker.
Speech Overlap Times and IDs of overlapping speech segments.
Speaker Time Time during which Speech is recognised.
User CXT Request Contextual directive specifying scope, depth, and policy constraints for speech recognition.
User Domain Response Domain‑specific knowledge supporting recognition and descriptor derivation.
User Interaction History Response Interaction History response providing temporal context for recognition.
Output Description
Recognised Text Output of the Automatic Speech Recognition AIM — a Text Segment or just a string.
Speech Descriptors Paralinguistic features of the speech segment including prosody, vocal affect, speaker characteristics, and optional neural network descriptors.
User CXT Response Response reporting the processing outcome of the AIM.
User Domain Request Request for domain‑specific knowledge issued to Domain Access.
User Interaction History Request Request for Interaction History issued to Private Storage.

4 SubAIMs

No SubAIMs.

5 JSON Metadata

https://schemas.mpai.community/PGM1/V1.0/AIMs/AutomaticSpeechRecognition.json

6 Profiles

No Profiles.

7 Reference Software

Not part of this specification.

8 Conformance Testing

Table 2 provides the Conformance Testing Method for the PGM‑ASR AIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 2 – Conformance Testing Method for the PGM‑ASR AIM

Input Language Selector Shall validate against the Language Selector part of the schema.
Auxiliary Text Shall validate against the Text Object schema. Text Data shall conform with the Text Qualifier.
Speech Object Shall validate against the Speech Object schema. Speech Data shall conform with the Speech Qualifier.
Speaker ID Shall validate against the Instance ID schema.
Speech Overlap Shall validate against the Speech Overlap schema.
Speaker Time Shall validate against the Time schema.
User CXT Request Shall validate against the User CXT Request schema.
User Domain Response Shall validate against the User Domain schema.
User Interaction History Response Shall validate against the User Interaction History schema.
Output Recognised Text Shall validate against the Text Object schema. Text Data shall conform with the Text Qualifier, e.g. output language shall be that indicated by the Language Selector.
Speech Descriptors Shall validate against the Speech Descriptors schema.
User CXT Response Shall validate against the User CXT Response schema.
User Domain Request Shall validate against the User Domain schema.
User Interaction History Request Shall validate against the User Interaction History schema.

Table 3 provides an example of PGM‑ASR AIM Conformance Testing.

Table 3 – An example of PGM‑ASR AIM Conformance Testing

9 Performance Assessment

Performance Assessment of an ASR Implementation (ASRI) can be performed for a language for which there is a dataset of speech segments of various durations with corresponding Transcription Text. A PGM‑ASR AIM Performance Assessment Report shall be based on the following steps and specify the input dataset used.

For each Recognised Text produced by the ASRI being Assessed for Performance in response to a speech segment provided as input:

  1. Compare the Recognised Text with the Transcription Text.
  2. Compute the Word Error Rate (WER) defined as the sum of deletion, insertion, and substitution errors in the Recognised Text compared to the Transcription Text, divided by the total number of words in the Transcription Text.

This code can be used to compute the WER.

Performance Assessment of an ASRI for a language in a Performance Assessment Report is defined as “The WER computed on all speech segments included in the reported dataset”.

Go to MPAI-PGM V1.0 AI Modules