Philpax

https://fosdem.org/2026/schedule/event/T38AKM-monado-and-beyond/

• VR Hardware and Tracking Systems
  • VR started in 1968 with Ivan Sutherland's work, but this was basically useless in comparison
  • 2012: Oculus Rift DK1
    • Small, affordable
    • IMU-only; can track rotation/acceleration, no position
  • Tracking sytsems
    • LED ("Constellation")
      • Started with Oculus Rift DK2
      • Known pattern of LEDs on headset
      • Using a infrared webcam
      • Computer vision matching LED pattern to 3D pose
      • Using camera calibration to get exact poses
    • Lighthouse
      • No cameras
      • Lighthouses that sweep room with lasers
      • Sensors on the headset then register time of sweep hitting
      • Match timestamps to 3D pose
    • SLAM
      • Introduced by Windows Mixed reality
      • Cameras built into headset, no fixed external reference
      • Using computer vision to find and track features across the field of view
  • LED tracking systems have largely fallen out of favour, but they're still used for controllers by SLAM headsets
  • Standalone headsets
    • Typically bootloader locked
    • Lynx R-1 supports mainline Linux porting effort
    • Steam Frame?
• In 2013, the OpenHMD project was started
  • Now mostly unmaintained
• OSVR
  • HDK headset by Razer
  • SDK by Sensics
  • Sensic ceased operations in 2019
  • OSVR mostly unmaintained
• Monado
  • Finally a push to unify APIs in 2018
  • Trying to build Mesa for OpenXR
  • Lots of drivers for lots of hardware
• Monado supports all three tracking systems to varying success
  • LED tracking
    • Supports PSVR/PS Move, shipped by iVRy, but no further development
    • Rift DK2/CV1/controllers, started by thaytan, ongoing work
  • Lighthouse
    • Open-source library libsurvive, but not fully complete, and on maintenance mode by Collabora
    • Using closed-source steamvr_lh wrapper by most people
  • SLAM
    • Basalt is CPU-heavy, tracking quality is not quite commercial level
    • New SLAM frameworks are always coming out, some open-source
    • Quick demo of the SLAM system with a headset
• Hand-tracking implementation
  • Stereo cameras
  • onnx model trained on open + synthetic data
• Monado can be ported to standalone headsets
  • GNU/Linux: Linux headsets
  • Android: Currently 3DoF-only, SLAM/additional tracking required
• Contributing
  • Talk to the team/community before anything
  • GitLab + Matrix/Discord (bridged) + IRC (not particularly used)
• Future:
  • MR/AR/perception
  • Plane/object detection
  • Environment depth estimation
  • Mesh environment reconstruction
  • Semantic labelling
  • AI commodisation should make it possible to support all of this perception tech
• Desktop:
  • xrdesktop
  • WayVR
  • Stardust XR
  • Simula VR
• OSXR ecosystem:
  • Godot, LOVR, osgXR, StereoKit, Bevy
  • ParaView, Blender, FreeCad
• Web ecosystem is worse:
  • Firefox is OpenVR-only, Windows-only, no current XR effort
  • Chrome doesn't want to ship OpenXR Linux support
  • WebKit is being worked on by Igalia
  • Servo doesn't support WebXR on Linux yet
  • Android based with Lepton: Wolvic, maybe?