AIR RACE X

AR System Development — SUPER GT / AIR RACE X

AIR RACE X

Following the initial live deployment at a SUPER GT event (Phase 1), served as technical lead for the AR system at AIR RACE X, a global air racing competition (Phase 2). Designed and implemented a pipeline to precisely reproduce flight data as CG and composite it onto real urban environments, creating the illusion of aircraft competing simultaneously in the same sky.

AIR RACE X Official Post

A post from the AIR RACE X official account (@airrace_x).

What is AIR RACE X?

AIR RACE X is a new kind of air race that uses real flight data to recreate races between pilots from around the world in AR. Rather than gathering in one place, each pilot flies from their own location, and their data is composited together to generate race footage that looks as if they are competing simultaneously. This allows for a thrilling aerial race experience while maintaining safety.

Technologies

AR / Unity / Shader

UnityC#ARShaderHLSLVPSPLATEAUPICO

Streaming Infrastructure / API

TypeScriptNode.js

Infrastructure / Tools

JestGitHub ActionsAWSJira

Live Event

Live EventOn-site
Phase 1August 2023

SUPER GT — High-Speed AR Live Deployment

SUPER GT — High-Speed AR Live Deployment

Participated in a live demonstration synchronizing AR with actual aircraft in real-time at a SUPER GT event. AR elements such as pylons and aircraft ghosts were composited onto footage captured with professional cameras, and output to large venue screens and live streams. Through this effort, we validated and established the technical foundation leading to AIR RACE X (Phase 2).

Responsibilities & Technical Challenges

Distance-Based Fade Shader Design

Challenge

The AR-displayed pylons and aircraft ghosts needed to blend naturally with live footage. In particular, the depth inconsistency between distant aircraft and the composited AR elements created an unnatural look.

Technical Approach

  • Implemented an HLSL shader that adjusts transparency and color tone based on camera distance, reproducing aerial perspective
  • Achieved stable AR compositing even for subjects flying hundreds to thousands of meters away
  • Tuned for both visibility and natural appearance on large screens and streamed footage

On-Site AR Filming & Field Operations

Challenge

On race day at the venue, attached gyro sensor equipment to professional broadcast cameras and conducted real-time AR composite filming. Stable tracking and operation were required across multiple camera positions spread across a large venue.

Technical Approach

  • Mounted and adjusted gyro sensor equipment on broadcast cameras for stable acquisition of camera orientation data
  • Moved between multiple filming points across a race venue spanning over 4 km, handling setup, calibration, and operation
  • Coordinated with a filming team including camera operators met for the first time on-site, responding on the fly to ensure a successful live shoot and broadcast

Outcomes

  • Achieved real-time synchronization between high-speed aircraft and AR elements at an outdoor live event
  • Realized AR effects using aircraft ghosts (visualization of past flight data) and pylon representations
  • Event footage was broadcast on large venue screens and distributed via live streaming
  • Distance-based shader technology was carried forward to AIR RACE X (Phase 2)
Phase 2

AIR RACE X

Responsibilities & Technical Challenges

AR Streaming Infrastructure Design & Implementation

Challenge

Led the design and implementation of the API infrastructure that integrates real flight sensor data from around the world and delivers it synchronously to the AR rendering engine (Unity) on race day.

Technical Approach

  • Designed an intermediate layer to convert raw sensor data (position, velocity, acceleration, orientation) into AR rendering format
  • Defined a timeline structure for gate crossings, penalties, and time measurement, with data design aligned to rendering requirements
  • Built a data transformation layer in TypeScript (Node.js) with an intermediate format to absorb spec changes
  • Designed synchronized delivery for 50 simultaneous HMD connections aligned with venue audio

Cross-team Interface Design

Challenge

Led interface design and technical coordination with overseas teams (sensor / WebAPI).

Technical Approach

  • Organized and documented data specifications; led interface design with overseas teams (sensor / WebAPI)
  • Built extensible transformation processing to handle per-event variations in sensor specs and presentation requirements
  • Unified data specifications across multinational teams and standardized the collaboration flow

AR Rendering & Device Support

Challenge

Responsible for Unity-side AR rendering engine and device-specific optimization.

Technical Approach

  • Designed and validated outdoor city-scale AR using VPS and PLATEAU
  • Optimized performance for stable operation on PICO devices
  • Designed Unity-side reception and rendering to synchronize with streamed data

Outcomes

  • This project won the Pioneer Award at the Sports Innovation Studio Contest
  • Built a streaming infrastructure that has been in continuous use across the 2023–2025 competitions
  • Unified data specifications across multinational teams and standardized the collaboration flow

Technical Articles

AR Implementation in AIR RACE X

AR Implementation in AIR RACE X

Covers the definition of a timeline structure for gate crossings, penalties, and time measurement, along with data design aligned to rendering requirements.

ARUnitySystem DesignReal-time Data
Key Considerations When Operating Large Numbers of HMDs at Events

Key Considerations When Operating Large Numbers of HMDs at Events

Explains the delivery design and operational methods for smoothly running venue briefings and event operations in a 50-device simultaneous PICO HMD environment.

PICOVREvent OperationLBE