1     Definition 2     Functional Requirements 3     Syntax
4     Semantics 5    Conformance Testing 6     Performance Assessment

1. Definition

The digital representation of temporal entities, events, and their relations – semantic links between two or more entities (events or times) that specify how they are connected – designed to support semantic annotation of multimodal and linguistic content (according to ISO 24617-1:2012).

2. Functional Requirements

  1. Temporal Semantics – MUST Represent temporal points, intervals, durations, and uncertainty (e.g., bounds, confidence) with normalisation to standard time scales.
  2. Event Semantics – MUST Represent events with type, participants/actors, attributes (e.g., modality, polarity, state), and links to time and source media segments.
  3. Relations – MUST Represent typed relations among Time and Event entities (e.g., BEFORE, OVERLAPS, CAUSES, PART_OF), including directionality, symmetry/asymmetry, transitivity where applicable, and constraint propagation.
  4. Multimodal Anchoring – MUST Provide anchoring mechanisms to reference spans/segments in text, audio, video, images, or sensor data (e.g., offsets, timestamps, frame indices).
  5. Identity and Referencing – MUST Ensure stable identifiers, cross‑reference capability, and uniqueness constraints within an annotation instance.
  6. Trace – MUST Record trace metadata (origin, timestamp, tool/user, version) for each entity and change operation.
  7. Versioning – MUST Support schema/model versioning and per‑instance version metadata.
  8. Extensibility – MUST Allow controlled extension points (namespaces, custom attributes, vocabularies).
  9. Constraints and Profiles – SHOULD Support domain‑specific profiles (e.g., clinical, media production) with additional constraints and enumerations.
  10. Quality Measures – SHOULD Provide optional confidence scores, quality flags, and validation reports for annotations and relations.
  11. Queryability – SHOULD Facilitate efficient querying of events, times, and relations (e.g., by time window, event type, participant).

3. Syntax

https://schemas.mpai.community/MMC/V2.5/data/TimeEventRelation.json

4. Semantics

Term Description
Header Schema version identifier (pattern MMC-TER-Vx.y).
– Standard‑TER The characters ‘MMC‑TER‑V’ marking the Time-Event-Relation data type.
– Version Major version – 1 or 2 characters.
– Dot‑separator The character ‘.’ separating version components.
– Subversion Minor version – 1 or 2 characters.
MInstanceID Identifier of the multimodal instance.
terAnnotationID Identifier of this Time–Event–Relation annotation.
time Array of Time objects representing temporal entities.
Time.timeID Unique identifier for a time entity.
Time.type Type of temporal entity: point | interval | duration.
Time.value Temporal value: timestamp (point), {start,end} (interval), or duration.
Time.timezone Time zone of the timestamp.
Time.calendar Calendar system used.
Time.uncertainty Uncertainty/tolerance around time (e.g., ±PT5S).
Time.confidence Confidence score for the time value (0.0–1.0).
Time.trace Provenance information via Trace.
event Array of Event objects representing events.
Event.eventID Unique identifier for an event.
Event.type Event type (string).
Event.participants Array of Participant objects involved in the event.
Event.attributes Additional event attributes, e.g. {  “mood”: “happy”,   “topic”: “greeting” }
Event.timeRefs Array of IDs referencing related Time entities.
Event.anchors Anchors linking the event to signals.
Event.trace Provenance information via Trace.
relation Array of Relation objects describing links between entities.
Relation.relationID Unique identifier for a relation.
Relation.source ID of the origin entity of the relation.
Relation.target ID of the destination entity of the relation.
Relation.relationType Type of relation: BEFORE | AFTER | DURING | OVERLAPS | CAUSES | ENABLES | PART_OF | SEQUENCE.
Relation.directional Whether the relation is directional (order matters).
Relation.symmetric Whether the relation is symmetric (order does not matter).
Relation.transitive Whether the relation is transitive.
Relation.confidence Confidence score for the relation (0.0–1.0).
Relation.trace Provenance information via Trace.
anchors Array of Anchor objects linking to external signals.
Anchor.id Unique identifier for an anchor.
Anchor.signalID Identifier of the external signal referenced by the anchor.
Anchor.modality Signal modality: text | speech | video | image | gesture | sensor | other.
Anchor.start Start offset within the signal (numeric).
Anchor.end End offset within the signal (numeric).
Anchor.offsetType Unit of offset: char | token | ms | frame | sample.
participant
Participant.id Unique identifier for a participant.
Participant.role Role of the participant in the event.
Participant.label Human-readable label for the participant.
Trace
Trace.origin Origin of the annotation (e.g., AIM instance).
Trace.timestamp Timestamp of creation/modification (date-time).
Trace.tool Tool that generated the annotation.
Trace.version Version of the tool.

5     Conformance Testing

A Data instance Conforms with MPAI-MMC Belief (MMC-EBL) if:

  1.  Its JSON Object validates against its JSON Schema.
  2. Any included  JSON Object validates against its JSON Schema.
  3. All Data in the JSON Object:
    1. Have the specified Data Types.
    2. Conform with the Qualifiers signaled in their JSON Schemas.

6     Performance Assessment