1     Definition 2     Functional Requirements 3     Syntax
4     Semantics 5    Conformance Testing 6     Performance Assessment

1      Definition

Cognitive State is a Personal Status Factor representing the internal state of an Entity such as “surprised” or “interested”.

2      Functional Requirements

Cognitive State can be expressed via several Modalities: Text, Speech, Face, and Gestures. (Other Modalities, such as body posture, may be handled in future MPAI Versions.)

Within a given Modality, Cognitive State can be analysed and interpreted via various Descriptors. For example, when expressed via Speech, the elements may be expressed through combinations of such features as prosody (pitch, rhythm, and volume variations); separable speech effects (such as degrees of voice tension, breathiness, etc.); and vocal gestures (laughs, sobs, etc.).

Cognitive State is represented by a standard set of labels and associated semantics by two tables:

  • A Label Set Table containing descriptive labels relevant to the Factor in a three-level format:
    • The CATEGORIES column specifies the relevant categories using nouns (e.g., “AN-GER”).
    • The GENERAL ADJECTIVAL column gives adjectival labels for general or basic la-bels within a category (e.g., “angry”).
    • The SPECIFIC ADJECTIVAL column gives more specific (sub-categorised) labels in the relevant category (e.g., “furious”).
  • A Label Semantics Table providing the semantics for each label in the GENERAL AD-JECTIVAL and SPECIFIC ADJECTIVAL columns of the Label Set Table. For example, for “angry” the semantic gloss is “emotion due to perception of physical or emotional damage or threat.”

Table 1 gives the standardised three-level Basic Cognitive State Label Set.

Table 1 – Basic Cognitive State Label Set

COGNITIVE CATEGORIES GENERAL ADJECTIVAL SPECIFIC ADJECTIVAL
AROUSAL aroused/excited/energetic cheerful
playful
lethargic
sleepy
ATTENTION attentive expectant/anticipating
thoughtful
distracted/absent-minded
vigilant
hopeful/optimistic
BELIEF credulous sceptical
INTEREST interested fascinated
curious
bored
SURPRISE surprised astounded
startled
UNDERSTANDING comprehending uncomprehending
bewildered/puzzled

Table 2 provides the semantics for each label in the GENERAL ADJECTIVAL and SPECIFIC ADJECTIVAL columns above.

Table 2 – Basic Cognitive State Semantics Set

ID Cognitive State Meaning
1 aroused/excited/energetic cognitive state of alertness and energy
2 astounded high degree of surprised
3 attentive cognitive state of paying attention
4 bewildered/puzzled high degree of incomprehension
5 bored not interested
6 cheerful energetic combined with and communicating happiness
7 comprehending cognitive state of successful application of mental models to a situation
8 credulous cognitive state of conformance to mental models of a situation
9 curious interest due to drive to know or understand
10 distracted/absent-minded not attentive to present situation due to competing thoughts
11 expectant/anticipating attentive to (expecting) future event or events
12 fascinated high degree of interest
13 interested cognitive state of attentiveness due to salience or appeal to emotions or drives
14 lethargic not aroused
15 playful energetic and communicating willingness to play
16 sceptical not credulous
17 sleepy not aroused due to need for sleep
18 surprised cognitive state due to violation of expectation
19 startled surprised by a sudden event or perception
20 surprised cognitive state due to violation of expectation
21 thoughtful attentive to thoughts
22 uncomprehending not comprehending

These sets have been compiled in the interests of basic cooperation and coordination among AIM submitters and vendors complemented by a procedure whereby AIM submitters may propose extended or alternate sets for their purposes.

An Implementer wishing to extend or replace a Label Set Table for one of the three Factors is requested to do the following:

  1. Create a new Label Set Table where:
    1. Proposed additions are clearly marked (in case of extension).
    2. b. All the elements of the target Cognitive State and levels (up to 3) are listed (in case of replacement).
  2. Create a new Label Semantics Table where the semantics of elements of the Cognitive State is:
    1. Added to the semantics of the existing Cognitive State (in case of extension).
    2. Provided (in case of replacement).
      The submitted semantics should have a level of detail comparable to the semantics given in the current Label Semantics Table.
  3. Submit both tables to the MPAI Secretariat.

The appropriate MPAI Development Committee will examine the proposed extension or replac­ement. Only the adequacy of the proposed new tables in terms of clarity and completeness will be considered. In case the new tables are not clear or complete, a revision of the tables will be requested.

The accepted Cognitive State Set will be identified as proposed by the submitter and reviewed by the appropriate MPAI Committee and posted to the MPAI web site.

The versioning system is based on a name – MPAI for MPAI-generated versions or “organisation name” for the proposing organisation – with a suffix m.n where m indicates the version and n indicated the subversion.

3      Syntax

https://schemas.mpai.community/MMC/V2.3/data/CognitiveState.json

4      Semantics

Label Size Description
Header N1 Bytes Entity Cognitive State Header
– Standard-EntityCognitiveState 9 Bytes The characters “MMC-ECS-V”
– Version N2 Bytes Major version – 1 or 2 characters
– Dot-separator 1 Byte The character “.”
– Subversion N3 Bytes Minor version – 1 or 2 characters
MInstanceID N4 Bytes Identifier of M-Instance.
EntityCognitiveStateID N5 Bytes Identifier of CogState.
EntityCognitiveStateSpaceTime N7 Bytes  Space-Time info of CogState.
EntityCognitiveStateData N8 Bytes Data associated to CogState.
– FusedCogState N9 Bytes Integrated CogState Value.
– TextCogState N10 Bytes Text CogState Value.
– SpeechCogState N11 Bytes Speech CogState Value.
– FaceCogState N12 Bytes Face CogState Value.
– GestureCogState N13 Bytes Gesture CogState Value.
DescrMetadata N14 Bytes Descriptive Metadata

5     Conformance Testing

A Data instance Conforms with MPAI-MMC V2.3 Entity Cognitive State (MMC-ECS) if:

  1. The Data validates against the Entity Cognitive State ’s JSON Schema.
  2. All Data in the  Entity Cognitive State ’s JSON Schema
    1. Have the specified type
    2. Validate against their JSON Schemas
    3. Conform with their Data Qualifiers if present.

6     Performance Assessment