OTA Architecture

Advanced10 min read

Assumes: 8.4 Secure Boot

The OTA campaign pushed successfully to 10,000 vehicles, then bricked unit #10,001. How do you prevent that?

TL;DR

OTA (Over-The-Air) updates write to the inactive A/B slot while the active slot keeps the vehicle running, then reboot into the new slot on success. Virtual A/B reduces storage overhead by snapshotting blocks instead of duplicating entire partitions. UpdateEngine (AAOS) orchestrates download → verify → apply → mark pending → reboot → post-boot verification. Slot metadata (bootctrl in misc) tracks current slot, boot success, retry counts, and unbootable flags. Rollback is automatic if the new slot fails verification or post-boot health checks within a merge/rollback window. OTA is a state machine with cryptographic gates, not a simple file copy.

You trigger OTAs with lb campaigns. This page is the machinery behind the button: AVB gates, slot metadata, and the state transitions you’ll grep in update_engine logs. The goal: update without bricking. The reality: check the rollback plan three times.

Think About It

An OTA update writes to slot B and reboots. Slot B fails to boot 3 times. What happens next?

Reveal Answer

ABL decrements bootctrl retry count on each failed boot; after three failures, slot B is marked unbootable and ABL automatically falls back to slot A — the last known-good slot. The vehicle keeps running on slot A without a brick, assuming slot A still passes AVB verification. Debug slot B with adb shell bootctl dump-bootloader-control and serial logs for vbmeta / dm-verity mismatch on the failed apply.

A/B Update Scheme: Two Copies, One Active

Classic A/B OTA maintains duplicate physical GPT partitions:

Active:   boot_a  +  super (system_a, vendor_a, ...)  +  vbmeta_a
Inactive: boot_b  +  super (system_b, vendor_b, ...)  +  vbmeta_b

Phase	Active Slot	Inactive Slot
Normal operation	Running OS	Stale or empty
OTA download	Running OS	Unchanged (payload cached separately)
OTA apply	Running OS	Being written
Reboot pending	Running until reboot	Marked pending/active
First boot after OTA	Unused	Boot attempt
Success	Becomes fallback	Confirmed active
Failure	Fallback boot	Marked unbootable

OTA always builds on the inactive slot while the active slot keeps the vehicle running.

Mental Model

A/B OTA is changing tires while driving — the car runs on three wheels while the fourth is swapped. You only commit weight to the new tire (reboot) after it’s bolted on (apply complete + verified). If the new tire wobbles (boot failure), you put the old one back (slot fallback).

A/B slots = double-buffered update. Write to B while running on A. Switch only if B is verified. Safe as houses.

Virtual A/B: Storage-Efficient Evolution

Full A/B duplicates large partitions, expensive on 64 GB UFS programs. Virtual A/B (Android 11+) uses copy-on-write snapshots:

Classic A/B	Virtual A/B
Two full super copies	One super + snapshot COW overlay
Higher storage cost	~40–50% less space for same safety
Simple mental model	More complex merge phase
Direct partition write	snapshotd tracks changed blocks

Virtual A/B flow additions:

1. Create snapshot before apply
2. Apply payload to snapshot/COW layer
3. Boot into snapshot
4. Merge snapshot into base on success (background, may take minutes)
5. On failure → cancel snapshot, revert to base

Check your platform:

adb shell getprop ro.virtual_ab.enabled
# true = Virtual A/B

Common Gotcha

Beginner thinks OTA is just download and install. Reality: it’s a state machine with A/B slot switching, dm-verity hash verification, rollback protection via anti-rollback fuses, and post-boot health checks before the new slot is confirmed. A failed update falls back to the previous slot — it doesn’t leave the vehicle bricked (if configured correctly).

If Virtual A/B merge is in progress, starting another OTA will fail. If update_engine reports NEED_REBOOT or merge running, don’t start another campaign. You’ll get cryptic SNAPSHOT_MERGE errors in logs.

Full OTA Flow: Download to Confirmed Boot

stateDiagram-v2
    [*] --> Idle
    Idle --> Downloading: Update available
    Downloading --> DownloadComplete: Payload on disk
    Downloading --> Idle: Cancel / network fail
    DownloadComplete --> Verifying: Hash + signature check
    Verifying --> Applying: Valid payload
    Verifying --> Idle: Invalid — abort
    Applying --> Applied: Inactive slot written
    Applying --> Idle: Apply error
    Applied --> PendingReboot: Mark slot pending
    PendingReboot --> FirstBoot: User/system reboot
    FirstBoot --> PostVerify: New slot boots
    PostVerify --> Success: Health OK
    PostVerify --> Rollback: Boot fail / dm-verity fail
    Success --> Merge: Virtual A/B merge (if enabled)
    Merge --> Idle: Merge complete
    Rollback --> Idle: Fallback slot active
    Success --> Idle: Classic A/B — done

Phase Breakdown

Phase	Component	What Happens
1. Check	UpdateEngine / campaign agent	Query server or local package for new build
2. Download	HTTP(S) / gRPC stream	Payload to /data/ota_package/ or cache partition
3. Verify	UpdateEngine + OpenSSL	Signature, metadata, compatible version, rollback index
4. Apply	update_engine --payload	Binary diff or full payload → inactive slot partitions
5. Finalize	bootctrl / misc	Set slot priority, mark pending reboot
6. Reboot	Power manager / user consent	Automotive: often parked + ignition policy gated
7. Post-boot verify	ABL + init + UpdateEngine	dm-verity, slot success marker, crash detection
8. Merge	snapshotd (Virtual A/B)	COW merge into base partition

Rollback Mechanism: Safety Net

Rollback operates at multiple layers:

Layer	Trigger	Action
Pre-apply	Signature/rollback index fail	Abort before writing inactive slot
Apply failure	I/O error, hash mismatch mid-write	Abort; inactive slot marked incomplete
Boot verification	ABL AVB failure	Try alternate slot
Post-boot window	Crash loop, watchdog, explicit markBootSuccessful() timeout	Revert bootctrl to previous slot
User-initiated	Factory tools / recovery	Manual slot switch via fastboot

bootctrl / Slot Metadata

Stored in misc partition (or dedicated metadata region):

Field	Purpose
Current slot	a or b; which slot booted this session
Priority	Higher priority slot tried first at boot
Successful boot flag	Set after clean boot + health check
Retry count	Decremented on failed boot attempts
Unbootable flag	Slot skipped entirely when set

# From running Android
adb shell bootctl get-current-slot
adb shell bootctl get-suffix 1    # _b suffix for slot 1
adb shell bootctl dump-bootloader-control

A slot isn’t truly active until markBootSuccessful() runs. First boot after OTA is a probation period.

Delta vs Full Payloads

Full payloads replace entire partition contents: simpler, larger download. Delta payloads (binary diffs against source version) produce smaller OTA packages but require strict source version matching. Mismatch between installed build and delta source → apply failure. Campaign tools like lb must target exact build fingerprints when using deltas.

Automotive OTA Constraints

Cockpit OTAs add policy gates beyond phone Android:

Constraint	Reason
Vehicle speed = 0	Safety; no reboot while moving
Battery / power budget	Sufficient power for download + apply + reboot
Ignition state	Some programs require ACC ON or engine running for apply
Multi-ECU coordination	Cockpit + modem + ADAS may need synchronized campaigns
User consent UI	AAOS Automotive settings / OEM HMI prompt
Differential signing	Production keys, fleet certificate pinning

Where You'll See This

OTA debug grep list (save this):

adb logcat -s update_engine UpdateEngine
adb shell dumpsys update_engine
adb shell bootctl get-current-slot
adb shell getprop ro.build.fingerprint

States to recognize in logs:

Log Signal	Meaning
UpdateStatus::DOWNLOADING	Payload in progress
UpdateStatus::VERIFYING	Signature/hash gate
UpdateStatus::FINALIZING	Writing slot metadata
UpdateStatus::UPDATED_NEED_REBOOT	Apply done; reboot required
ErrorCode::kDownloadTransferError	Network issue, not a signing issue
ErrorCode::kPayloadHashMismatchError	Corrupt download or wrong payload
ErrorCode::kRollbackNotPermitted	Rollback index too low

After OTA reboot, confirm success before celebrating:

adb shell bootctl get-current-slot    # did slot flip?
adb shell getprop ro.build.display.id # correct build?

Connect the Dots

Page 8.1 defined A/B partition names. Page 8.4 explained AVB gates during apply and first boot. Page 8.6 maps this architecture to your lb tool’s gRPC + SOME/IP transport layer.

What matters

1. A/B OTA writes the inactive slot while the active slot runs. Reboot switches.
2. Virtual A/B trades snapshot complexity for storage savings.
3. Slot metadata in bootctrl governs fallback. First boot is probation until marked successful.
4. OTA is a state machine. Learn UpdateEngine statuses to debug failed campaigns.

Check Your Understanding

1. During a classic A/B OTA apply phase, which slot is being written?

Recall

A. The inactive slot — while the active slot continues serving the running OSB. Both slots simultaneouslyC. LUN 0 boot chain partitionsD. The currently running active slot

2. What marks an OTA slot transition as permanently successful (not subject to automatic rollback)?

Applied

A. bootctrl markBootSuccessful() after a healthy post-OTA bootB. Flashing vbmeta via fastbootC. SOME/IP service registrationD. Download complete notification

3. How does Virtual A/B differ from classic A/B?

Recall

A. Virtual A/B eliminates the need for verified bootB. Virtual A/B uses COW snapshots to reduce duplicate storage, keep rollback safeC. Virtual A/B only updates LUN 0 boot chain partitions, not system or vendorD. Virtual A/B requires a QFIL flash with matched rawprogram for every update

4. An OTA writes slot B and reboots. Slot B fails AVB verification three times. What happens next?

Applied

A. ABL marks slot B unbootable and falls back to slot A — the last known-good slotB. QFIL is invoked over SOME/IPC. Device stays in a permanent boot loop on slot BD. UpdateEngine re-downloads the payload automatically

5. A delta OTA fails at apply with a source-version mismatch error. Installed fingerprint is 2024.09.01 but the delta targets 2024.08.15. Which OTA design constraint applies?

Synthesis

Connects to: Module 1 — Why Cars Became Computers

A. Virtual A/B disables delta updatesB. Delta payloads only work on LUN 0C. Delta payloads need exact source build match — fingerprint must align to baselineD. Delta payloads skip AVB verification

6. During OTA apply, UpdateEngine verifies payload signature and rollback index before writing the inactive slot. Which component re-verifies AVB and dm-verity on first boot of the new slot?

Synthesis

Connects to: Module 9 — Safety & Diagnostics

A. SOME/IP ProgressChanged event handlerB. ABL at boot time, then kernel dm-verity driver on system/vendor readsC. Firehose programmer during EDLD. persist partition calibration daemon

Further Reading

• Over-the-Air Updates Overview — AOSP entry point for OTA architecture, payload formats, and the UpdateEngine-driven update pipeline.
• A/B System Updates — Slot metadata, boot control, and the inactive-slot apply model that this page’s state machine describes.
• Virtual A/B OTA — Snapshot and copy-on-write merge mechanics that reduce duplicate super storage on storage-constrained programs.
• update_engine Source — Reference implementation for download, verify, apply, and reboot orchestration; grep target for log status strings.
• Dynamic Partitions and OTA — How logical partitions inside super are targeted by OTA payloads and delta updates.
• Android Verified Boot and OTA — AVB gates that run during apply and first-boot verification before a slot is marked successful.