Go to PGM-AUA V1.0 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

The Linguistic‑Paralinguistic Analysis (PGM‑LPA) AI Module derives, from the utterances of the Harmonised Multimodal Context, the evidence carried by the utterance  and by the manner in which it was said, by

  • Segmenting the attributed utterances into units of analysis.
  • Deriving the linguistic evidence carried by the words: their content, the speech act each utterance performs, and the entities they refer to.
  • Deriving the paralinguistic evidence carried by the audio delivery: prosody, voice quality, rate, and disfluency. (See below concerning visual information.)
  • Recording where the words and the delivery agree and where they do not.
  • Producing the Linguistic‑Paralinguistic Evidence.

The module is analytical. It derives evidence about the utterances of an Entity; it should not construct a state of that Entity, which is the function of Entity State Construction, and it should not draw on the visual and behavioural evidence of the scene, which is the function of Behavioural and Expressive Analysis.

Each item of evidence produced carries the identifier of the Entity to which the Harmonised Multimodal Context attributed the utterance it derives from, and a measure of confidence. Evidence that cannot be attributed to an Entity is reported but not asserted of any Entity.

2 Reference Model

Figure 1 depicts the Reference Model of the Linguistic‑Paralinguistic Analysis (PGM‑LPA) AIM.

Figure 1 – The Linguistic‑Paralinguistic Analysis (PGM‑LPA) AIM

The Linguistic‑Paralinguistic Analysis (PGM‑LPA) AI Module operates as follows:

  1. Receives the Harmonised Multimodal Context and the Speech Descriptors
  2. Segments the attributed utterances into units of analysis, on the common time base of the Harmonised Multimodal Context
  3. Derives the linguistic evidence:
    • Propositional content of the utterance
    • Speech act the utterance performs (asserting, asking, requesting, promising, declining, acknowledging)
    • Referring expressions and the Entities of the scene they denote
    • Discourse role of the utterance in the exchange (opening, responding, continuing, closing, interrupting)
    • Sentiment and stance expressed by the words
    • Language, register, and formality
  4. Derives the paralinguistic evidence:
    • Prosody: pitch contour, stress, and intonation
    • Timing: speech rate, pausing, and hesitation
    • Voice quality: intensity, breathiness, tension, and tremor
    • Disfluency: repetition, repair, and filled pause
    • Non‑verbal vocalisation: laughter, sigh, and throat‑clearing
  5. Compares the two:
    • Marks the utterances whose delivery reinforces their content
    • Marks the utterances whose delivery is at variance with their content
    • Marks the utterances for which one of the two is absent or unusable
  6. Constructs a consistent Linguistic‑Paralinguistic Evidence
  7. Outputs the Linguistic‑Paralinguistic Evidence for use by downstream AI Modules

The module operates in real time, without recourse to the reasoning capability of Basic Knowledge and without Model Context Protocol interactions. Domain knowledge is obtained by query, and the Response to a query is in general received during a subsequent processing cycle.

The Domain Response supports the interpretation of what is said in the terms of the domain: the vocabulary and the conventions in use, the registers appropriate to the setting, and the expressions that denote domain entities.

3 I/O Data

Table 1 specifies the Input and Output Data of the Linguistic‑Paralinguistic Analysis (PGM‑LPA) AIM.

Table 1 – I/O Data of the Linguistic‑Paralinguistic Analysis (PGM‑LPA) AIM

Input Description
Harmonised Multimodal Context Audio, visual, textual, and paralinguistic evidence of the scene placed on a common time base and attributed to harmonised Entities.
Speech Descriptors Paralinguistic features of the speech segment produced by Automatic Speech Recognition.
User‑USD Directive Control directive specifying scope, depth, or policy constraints for linguistic and paralinguistic analysis.
User Domain Response Domain‑specific knowledge supporting the interpretation of utterances in the terms of the domain.
User Interaction History Response Prior interaction‑history content supporting the resolution of reference to Entities and utterances of earlier captures.
Output Description
Linguistic‑Paralinguistic Evidence Evidence borne by the content of the utterances of an Entity and by the manner of their delivery, with the agreement between the two.
User‑USD Status Status information describing the execution and outcome of Linguistic‑Paralinguistic Analysis processing.
User Domain Request Query to Domain Access for domain‑specific knowledge.
User Interaction History Request Query to A‑User Storage for prior interaction‑history content.

4 SubAIMs

No SubAIMs.

5 JSON Metadata

https://schemas.mpai.community/PGM1/V1.0/AIMs/LinguisticParalinguisticAnalysis.json

6 Profiles

No Profiles.

7 Reference Software

Not part of this specification.

8 Conformance Testing

Table 2 provides the Conformance Testing Method for the Linguistic‑Paralinguistic Analysis (PGM‑LPA) AIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 2 – Conformance Testing Method for the Linguistic‑Paralinguistic Analysis (PGM‑LPA) AIM

Receives Harmonised Multimodal Context Shall validate against Harmonised Multimodal Context schema.
Speech Descriptors Shall validate against Speech Descriptors schema.
User‑USD Directive Shall validate against User‑USD Directive schema.
User Domain Response Shall validate against User Domain schema.
User Interaction History Response Shall validate against Interaction History schema.
Produces Linguistic‑Paralinguistic Evidence Shall validate against Linguistic‑Paralinguistic Evidence schema.
User‑USD Status Shall validate against User‑USD Status schema.
User Domain Request Shall validate against User Domain schema.
User Interaction History Request Shall validate against Interaction History schema.

9 Performance Assessment

Not part of this specification.

Go to PGM-AUA V1.0 AI Modules