Display & Audio Routing

TL;DR

Multiple OSs share physical displays and speakers on the same SoC. QNX renders the instrument cluster (safety-critical, always-on); AAOS renders the IVI center stack (infotainment). VirtIO-GPU lets the AAOS guest submit frames to a virtual GPU backed by QNX/hypervisor. Audio is harder than video: nav, media, phone, and chime sources compete with priority-based arbitration. Audio HAL handles stream I/O; Audio Control HAL handles routing, volume groups, and ducking. When cluster and IVI both glitch, suspect GPU contention or display pipeline config first.

Display and audio are where users feel the multi-OS architecture. Two operating systems, one windshield’s worth of pixels, one cabin full of speakers. Somehow it has to look seamless and never miss a chime.

Multiple OSs, Multiple Displays

Modern cockpits have 2–4 physical displays driven from one SoC:

Display	Typical Owner	Content
Instrument cluster (behind steering wheel)	QNX	Speed, RPM, telltales, ADAS icons, trip computer
Center stack / IVI	AAOS	Navigation, media, climate UI, apps
Passenger display	AAOS (second display)	Entertainment, co-pilot nav
HUD (if present)	QNX or dedicated renderer	Speed, nav arrows, ADAS alerts

The ownership split is deliberate: safety-adjacent pixels live on QNX. If AAOS reboots during a drive, the cluster keeps showing speed and warnings. Cluster is QNX; IVI is AAOS.

flowchart TB
    subgraph Displays["Physical Displays"]
        CLUSTER["Instrument Cluster<br/>(1920×720 typical)"]
        IVI["Center Stack IVI<br/>(1920×1080 typical)"]
        PASS["Passenger Display"]
    end

    subgraph QNX["QNX VM"]
        QCOMP["Cluster Compositor"]
        QGL["OpenGL / Screen Graphics"]
    end

    subgraph AAOS["AAOS Guest VM"]
        SURF["SurfaceFlinger"]
        VGPU["VirtIO-GPU Driver"]
        APPS["Apps / System UI"]
        APPS --> SURF --> VGPU
    end

    subgraph HW["Display Hardware"]
        DPU["Display Processing Unit (DPU)"]
        PHY["DSI / eDP PHY"]
    end

    QCOMP --> QGL --> DPU
    VGPU -->|"VirtIO-GPU"| DPU
    DPU --> PHY
    PHY --> Displays

VirtIO-GPU: Guest Rendering

AAOS doesn’t drive the display hardware directly. It renders through VirtIO-GPU, a paravirtualized GPU device:

Concept	What Happens
Guest (AAOS)	Apps render → SurfaceFlinger → OpenGL/Vulkan → VirtIO-GPU driver
VirtIO protocol	GPU commands and frame buffers sent across VM boundary
Host (QNX/Hypervisor)	VirtIO backend maps guest frames to hardware display layers
DPU compositing	Hardware blends QNX cluster layer + AAOS IVI layer → physical output

Benefits:

• AAOS gets full GPU acceleration without direct hardware access
• QNX controls which layer goes to which display
• Hypervisor can limit GPU memory and schedule GPU time between VMs

Tradeoffs:

• GPU contention: cluster and IVI share the same physical GPU
• Extra latency: one VM boundary crossing for every frame submission
• Debug complexity: rendering bugs could be guest driver, VirtIO backend, or DPU config

Mental Model

Think of the DPU as a stack of transparent sheets (hardware layers). QNX paints the bottom sheet (cluster). AAOS paints a sheet above it (IVI) via VirtIO-GPU — like slipping a drawing through a slot in the desk to the shared light table. The light table (DPU) blends all sheets and projects to the screen. Neither OS sees the other’s pixels directly — they see their assigned layer.

Display Routing: Who Gets Which Screen

Display assignment is configured at platform integration time, not runtime-discoverable:

Config Element	Set By	Example
Display IDs	BSP / device tree	Display 0 = cluster, Display 1 = IVI
Layer assignment	QNX display manager	QNX compositor → layer 0, VirtIO → layer 1
Resolution / timing	Panel specs	Cluster 1920×720@60, IVI 1920×1080@60
Multi-display AAOS	config.xml / overlay	Passenger display as display 2

Common Gotcha

“IVI works but cluster is black” and “cluster works but IVI is black” are different teams and different pipelines. Cluster black → QNX compositor, QNX display driver. IVI black → VirtIO-GPU, AAOS SurfaceFlinger, guest display config. Don’t assume one fix solves both.

Audio Routing: Multiple Sources, One Cabin

Audio is more contentious than video because speakers are shared. You can’t give nav and Spotify separate physical speakers (usually). Sources compete:

Source	Priority (typical)	Behavior When Higher Priority Speaks
Chime / warning	Highest	Mutes or ducks everything: collision alert, seatbelt
Phone call	High	Ducks media and nav
Navigation guidance	Medium-high	Ducks media, may duck with phone on hold
Voice assistant	Medium	Ducks media during response
Media (Spotify, radio)	Low	Ducked by everything above
System UI sounds	Low-medium	Brief, usually mixes or ducks media

Ducking means temporarily lowering volume of lower-priority streams (not muting entirely; nav might drop media from 100% to 30%). Chimes and calls win over everything else.

Audio HAL + Audio Control HAL

Android automotive splits audio into two HAL layers:

HAL	AIDL/HAL Name	Responsibility
Audio HAL	android.hardware.audio	Stream I/O: open output stream, write PCM buffers, capture from mic
Audio Control HAL	android.hardware.automotive.audiocontrol	Routing, volume groups, ducking policy, bus assignment

Audio HAL (Stream I/O)

Handles the mechanics:

• Open/close output streams per audio usage (media, navigation, voice, alarm)
• Write PCM data to the correct hardware audio bus (bus0_media, bus1_nav, etc.)
• Microphone capture for voice assistant and BT phone

Audio Control HAL (Routing & Policy)

Handles the intelligence:

• Map Android audio usages to physical output buses
• Define volume groups (media volume knob vs nav volume vs phone)
• Implement ducking rules: when nav speaks, tell Audio HAL to reduce media gain
• Coordinate with Car Service for occupant zone routing (driver vs passenger headphones)

flowchart TB
    subgraph AAOS["AAOS Audio Stack"]
        APP["Apps · Nav · Phone · Media"]
        AFW["AudioFlinger"]
        AH["Audio HAL"]
        ACH["Audio Control HAL"]
        APP --> AFW --> AH
        ACH -.->|"routing · ducking · volume"| AFW
        ACH -.->|"bus mapping"| AH
    end

    subgraph QNX["QNX Audio Path"]
        MIX["Audio Mixer / DSP"]
        AMP["Amplifier · Speakers"]
    end

    subgraph Sources["Priority Sources"]
        CHIME["Chimes / Warnings (QNX)"]
        NAV["Navigation (AAOS)"]
        MEDIA["Media (AAOS)"]
        PHONE["Phone (AAOS / BT)"]
    end

    AH -->|"PCM streams"| MIX
    CHIME -->|"Highest priority"| MIX
    MIX --> AMP

Note: safety chimes often originate on QNX. They bypass AAOS entirely and inject directly into the mixer. If AAOS crashes, you still hear the seatbelt chime.

Audio Bus Architecture (Automotive)

Automotive Audio HAL uses bus-based routing instead of phone-style deep buffer paths. Each logical bus maps to a physical output:

Android Bus	Typical Physical Output
bus0_media_out	Cabin speakers (media)
bus1_nav_out	Cabin speakers (nav; same physical, mixed)
bus2_phone_out	Cabin speakers or BT HFP
bus3_alarm_out	Chimes / warnings

The Audio Control HAL’s setBalanceTowardRight() and ducking APIs operate on these buses. Misconfigured bus mapping = “nav voice comes from wrong speaker.”

Display & Audio Routing Diagram

flowchart LR
    subgraph Video["Display Path"]
        QNX_D["QNX Cluster Renderer"]
        AAOS_D["AAOS SurfaceFlinger"]
        VGPU["VirtIO-GPU"]
        DPU["DPU Compositor"]
        QNX_D --> DPU
        AAOS_D --> VGPU --> DPU
        DPU --> PANELS["Cluster + IVI Panels"]
    end

    subgraph Audio["Audio Path"]
        QNX_A["QNX Chimes"]
        AAOS_A["AAOS AudioFlinger"]
        ACH["Audio Control HAL"]
        MIXER["Hardware Mixer / DSP"]
        SPK["Speakers"]
        QNX_A -->|"Priority 1"| MIXER
        AAOS_A --> ACH --> MIXER
        MIXER --> SPK
    end

Where You'll See This

Common display/audio tickets and first checks:

Ticket	First Look
Cluster flicker	QNX compositor logs, DPU underrun counters, GPU load
IVI black screen, cluster OK	adb shell dumpsys SurfaceFlinger, VirtIO-GPU dmesg, guest display config
Both displays glitch	GPU thermal throttling, shared memory pressure, hypervisor GPU scheduling
Nav voice silent	Audio Control HAL routing, bus1_nav_out mapping, nav app audio focus
Media doesn’t duck for nav	Ducking policy in Audio Control HAL, CarAudioService config
Chime too quiet / missing	QNX chime path (not AAOS), amplifier gain, mixer priority
Phone audio from wrong speaker	BT HFP routing, occupant zone config, bus2 mapping

Debug commands:

# AAOS display state
adb shell dumpsys SurfaceFlinger
adb shell dumpsys display

# AAOS audio routing
adb shell dumpsys media.audio_policy
adb shell dumpsys car_service --audio

# Active audio focus holders
adb shell dumpsys audio | grep -i focus

GPU contention between cluster and IVI often shows up as both stuttering simultaneously. That’s your clue it’s shared hardware, not one OS.

Connect the Dots

Module 6.1 (QNX) owns cluster rendering and chime injection. Module 6.2 (AAOS) owns IVI and media apps. Module 6.5 (Power Management) explains display/audio behavior during suspend: cluster may stay on while IVI sleeps. Module 0.1 (SoC Anatomy) introduced the GPU and DPU blocks these pipelines use.

Bottom line

1. Cluster = QNX, IVI = AAOS: safety pixels vs infotainment pixels.
2. VirtIO-GPU lets AAOS render without direct hardware access.
3. Audio uses priority arbitration: chimes > phone > nav > media.
4. Audio HAL = stream I/O; Audio Control HAL = routing, ducking, volume groups.
5. Safety chimes often bypass AAOS, injected from QNX directly.

Check Your Understanding

1. How does AAOS render graphics to the center stack display on a virtualized cockpit platform?

A. AAOS drives the display hardware directly via a bare-metal GPU driverB. QNX copies Android framebuffer files to the display each frameC. AAOS renders through VirtIO-GPU; the hypervisor/QNX backend maps frames to a DPU display layerD. All rendering is done by QNX; AAOS sends XML layout descriptions

2. Which audio source typically has the highest priority in automotive audio arbitration?

A. Media playback (Spotify, radio)B. Navigation turn-by-turn guidanceC. Safety chimes and collision warningsD. Voice assistant responses

3. What is the primary role of the Audio Control HAL (separate from Audio HAL)?

A. Writing PCM audio buffers to hardware output streamsB. Managing audio routing, volume groups, ducking policy, and bus assignmentC. Encoding Bluetooth A2DP streamsD. Rendering the instrument cluster audio visualizer