| 1 Definition | 2 Functional Requirements | 3 Syntax |
| 4 Semantics | 5 Conformance Testing | 6 Performance Assessment |
Definition
The digital representation of temporal entities, events, and their relations – observable realisations of events – designed to support semantic annotation of dialogue, text, and multimodal content (according to ISO 24617-2:202).
Functional Requirements
- Temporal Semantics – MUST Represent temporal points, intervals, durations, and uncertainty (e.g., bounds, confidence) with normalisation to standard time scales (UTC offsets, calendars).
- Event Semantics – MUST Represent events with type, participants/actors, attributes (e.g., modality, polarity, state), and links to time and source media segments.
- Expression Semantics – MUST Represent expressions as observable realizations of events in communicative media (linguistic forms, gestures, vocalisations, multimodal composites), including structure, content, and association to the event(s) they realize.
- Expression–Event Linking – MUST Provide explicit links from expressions to the event(s) and time(s) they realise, supporting one‑to‑one, one‑to‑many, and many‑to‑one
- Expression Typing – MUST Support typing of expressions (e.g., utterance, token span, gesture segment, vocal cue, composite) and sub‑typing via controlled vocabularies.
- Multimodal Anchoring – MUST Provide anchoring mechanisms to reference spans/segments in text, audio, video, images, or sensor data (e.g., offsets, timestamps, frame indices) for both expressions and events.
- Identity and Referencing – MUST Ensure stable identifiers for Time, Event, and Expression entities, cross‑reference capability, and uniqueness constraints within an annotation instance.
- Trace – MUST Record provenance metadata (origin, timestamp, tool/user, version) for each entity and change operation.
- Extensibility – MUST Allow controlled extension points (namespaces, custom attributes, vocabularies) without breaking core conformance.
- Constraints and Profiles – SHOULD Support domain‑specific profiles (e.g., dialogue acts, clinical discourse, media production) with additional constraints and enumerations.
- Quality Measures – SHOULD Provide optional confidence scores, quality flags, and validation reports for expressions, events, and links.
- Queryability – SHOULD Facilitate efficient querying by time window, event type, expression type, participant, or modality.
Syntax
https://schemas.mpai.community/MMC/V2.5/data/TimeEventExpression.json
Semantics
| Label | Description |
| Header | Time Event Expression Header |
| Standard‑TEE | The characters ‘MMC‑TEE‑V’ marking the Time-Event-Expression data type. |
| Version | Major version – 1 or 2 characters. |
| Dot‑separator | The character ‘.’ separating version components. |
| Subversion | Minor version – 1 or 2 characters. |
| TEEAnnotationID | Unique identifier for this TEE annotation instance. |
| Time | Set of Time entities representing temporal points, intervals, or durations. |
| participants | Array of participant objects; items are unique. |
| – id | String identifier of the participant. |
| – role | Role of the participant: speaker | addressee | audience | system | other. |
| – type | Type of participant: human | agent | system. |
| – label | Human-readable label for the participant. |
| – metadata | Participant-specific metadata. |
| functionalSegments | Array of functional segment objects. |
| – id | String identifier of the functional segment. |
| – source | Reference to the primary data resource (e.g., document URI, audio/video file). |
| – modality | Segment modality: text | speech | gesture | multimodal | other. |
| – language | Language code for the segment. |
| – start | Numeric start position/time. |
| – end | Numeric end position/time. |
| – offsetType | Offset units: char | token | ms | frame | sample. |
| – text | Text content of the segment (if applicable). |
| communicativeFunctions | Array registry of communicative functions (may reference an external vocabulary). |
| – id | String identifier of the communicative function. |
| – label | Human-readable label of the communicative function. |
| – dimension | Dimension: Task | Feedback | TurnManagement | TimeManagement | DiscourseStructuring | OwnCommunicationManagement | PartnerCommunicationManagement | SocialObligationsManagement. |
| – definition | Definition text of the function. |
| – uri | URI of the function entry (if any). |
| dialogueActs | Array of dialogue act objects. |
| – id | String identifier of the dialogue act. |
| – segmentRef | Reference to a functionalSegments[id]. |
| – senderRef | Reference to the sender participants[id]. |
| – addresseeRefs | Array of references to participants[id] for addressees. |
| – functionRef | Reference to communicativeFunctions[id]. |
| – dimension | Dimension of the act; same enumeration as communicativeFunctions.dimension. |
| – semanticContent | Optional object container for domain semantics. |
| – qualifiers.sentiment | String sentiment qualifier. |
| – qualifiers.certainty | Certainty value in range 0.0–1.0. |
| – qualifiers.conditional | Boolean conditional flag. |
| – dependence.functional | DialogueAct[id] this act depends on (e.g., answer depends on question). |
| – dependence.feedback | DialogueAct[id] this act gives feedback to. |
| – rhetoricalRelations | Array of rhetorical relation objects attached to the act. |
| – type | Type of rhetorical relation (string). |
| – targetActRef | Target DialogueAct reference (string). |
| – trace | |
| – origin | Origin string for provenance. |
| – timestamp | Timestamp in date-time format. |
| – tool | Tool identifier string. |
| – version | Version string. |
| relations | Array of non-dependence relations among dialogue acts. |
| – id | String identifier of the relation. |
| – sourceActRef | Source DialogueAct reference (string). |
| – targetActRef | Target DialogueAct reference (string). |
| – relationType | Type of relation (string). |
| – confidence | Confidence value in range 0.0–1.0. |
| – trace.origin | Origin string for provenance. |
| – trace.timestamp | Timestamp in date-time format. |
5 Conformance Testing
A Data instance Conforms with MPAI-MMC Time Event Expression (MMC-TEE) if:
- Its JSON Object validates against its JSON Schema.
- Any included JSON Object validates against its JSON Schema.
- All Data in the JSON Object:
- Have the specified Data Types.
- Conform with the Qualifiers signaled in their JSON Schemas.
