PGM-AUA V1.0 AIMs - Scene Geometry Construction

Go to PGM-AUA AI Modules

Function

Ref. Model

1. Function

The Scene Geometry Construction (PGM‑SGC) AIM constructs the AV Scene Descriptors: a single description of the captured Space in which each Entity appears once, whether it was perceived by both modalities or by one, placed in a single spatial frame and related to the other Entities of the Space.

Scene Geometry Construction:

Operates on the Entity Correspondences and the Unmatched Entities produced by Correspondence Resolution, the Enhanced Audio Scene Descriptors produced by Audio Scene Enhancement, the Enhanced Visual Scene Descriptors produced by Visual Scene Enhancement, and the AVA CXT Directive received from A‑User Control.
Constructs one AV Entity for each Entity Correspondence and for each Unmatched Entity, determines its Spatial Attitude in a single frame of reference, and determines the spatial relations holding between the AV Entities.
Produces the AV Scene Descriptors sent to AV Geometry Multiplexing, and the AVA CXT Status sent to AV Geometry Multiplexing.

The AV Scene Descriptors are to audio and vision together what the Audio Scene Descriptors and the Visual Scene Descriptors are to each modality: the description of the Space on which the AI Modules downstream of the A-User Frontend operate. They do not replace the Enhanced Audio Scene Descriptors and the Enhanced Visual Scene Descriptors, which Scene Geometry Construction does not modify. Each AV Entity refers to the Audio Entity, the Visual Entity, or both, from which it was constructed, so that the modality-specific properties of an Entity remain available where they were first described.

2. Reference Model

Figure 1 depicts the Reference Model of the Scene Geometry Construction (PGM‑SGC) AIM.

Figure 1 – Reference Model of the Scene Geometry Construction (PGM‑SGC) AIM

The Scene Geometry Construction (PGM‑SGC) AIM operates as follows:

Receives the Entity Correspondences and the Unmatched Entities of the capture, together with the Enhanced Audio Scene Descriptors and the Enhanced Visual Scene Descriptors from which they were derived.
Establishes the frame of reference of the Space, and expresses the positions estimated by each modality in it.
Constructs the AV Entities:
- One AV Entity for each Entity Correspondence, carrying the aligned identifier and the confidence established by Correspondence Resolution
- One AV Entity for each Unmatched Entity, marked as perceived by the audio modality alone or by the visual modality alone
- For each AV Entity, the references to the Audio Entity, the Visual Entity, or both, from which it was constructed
Determines the Spatial Attitude of each AV Entity:
- Where both modalities estimate the position, a reconciled estimate that weights each by its precision, the audio estimate weighing less under strong reverberation, as the Acoustic Profile reports it, and the visual estimate weighing less under occlusion or poor visibility, as the Visual Profile reports it
- The divergence between the two estimates, when it exceeds the tolerance the AVA CXT Directive sets
- Where one modality alone estimates the position, that estimate with its precision, an audio-only position being in general known in direction more precisely than in distance
Determines the spatial relations holding between the AV Entities: relative position, distance, relative orientation, and containment, such as that of a sound source within an object.
Relates each AV Entity to the A-User: its distance from the A-User and its orientation with respect to it.
Outputs the AV Scene Descriptors.

The AIM operates in real time. It does not engage the reasoning capability of Basic Knowledge and does not open Model Context Protocol sessions. It performs no query: the knowledge it needs has already been drawn into the Enhanced Scene Descriptors and the Entity Correspondences it receives.

The AV Scene Descriptors do not carry the Joint Salience. When the AVA CXT Directive requires it, the Joint Salience is produced by Audio-Visual Salience Reconciliation and joined to the AV Scene Descriptors by AV Geometry Multiplexing.

3. Input/Output Data

Table 1 lists the Input and Output Data of the Scene Geometry Construction (PGM‑SGC) AIM.

Table 1 – Input/Output Data of the Scene Geometry Construction (PGM‑SGC) AIM

Input	Description
Entity Correspondences	Correspondences established between Audio Entities and Visual Entities, produced by Correspondence Resolution.
Unmatched Entities	Audio Entities and Visual Entities left without counterpart, produced by Correspondence Resolution.
Enhanced Audio Scene Descriptors	Descriptors of the audio scene produced by Audio Scene Enhancement.
Enhanced Visual Scene Descriptors	Descriptors of the visual scene produced by Visual Scene Enhancement.
AVA CXT Directive	Directive from A‑User Control specifying scope, depth, and policy constraints for scene construction, including the tolerance on the divergence of position estimates.
Output	Description
AV Scene Descriptors	Description of the Space in which each Entity appears once as an AV Entity, with its Spatial Attitude in a single frame, its relations to the other AV Entities and to the A-User, and its references to the Audio and Visual Entities it was constructed from.
AVA CXT Status	Status information describing the execution and outcome of Scene Geometry Construction processing.

4. SubAIMs

No SubAIMs.

5. JSON Metadata

https://schemas.mpai.community/PGM1/V1.0/AIMs/SceneGeometryConstruction.json

6. Profiles

No Profiles.

7. Reference Software

Not part of this specification.

8. Conformance Testing

Table 2 provides the Conformance Testing Method for the Scene Geometry Construction (PGM‑SGC) AIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 2 – Conformance Testing Method for the Scene Geometry Construction (PGM‑SGC) AIM

Receives	Entity Correspondences	Shall validate against Entity Correspondences schema.
	Unmatched Entities	Shall validate against Unmatched Entities schema.
	Enhanced Audio Scene Descriptors	Shall validate against Enhanced Audio Scene Descriptors schema.
	Enhanced Visual Scene Descriptors	Shall validate against Enhanced Visual Scene Descriptors schema.
	AVA CXT Directive	Shall validate against AVA CXT Directive schema.
Produces	AV Scene Descriptors	Shall validate against AV Scene Descriptors schema.
	AVA CXT Status	Shall validate against AVA CXT Status schema.

9. Performance Assessment