Skip to content

BeaconFlow

Low-latency global robot teleoperation through computer vision

BeaconFlow screen
Category
Robotics & AI
Platform
Android
Status
Prototype
Version
v0.5.0-alpha
Released
2025

About BeaconFlow

Global telesurgery was first validated scientifically in 'Operation Lindbergh', in 2001, proving it technically possible [1]. BeaconFlow takes the idea further by making it available through an app, using a hybrid architecture. The project opens doors for telemedicine, assistive technology (giving autonomy to people with limited mobility) and industry (safe operation in hazardous environments).

What it does

  • A mobile app that controls robotic hardware (ESP32) remotely, without depending on a local network
  • Real-time motion mimicry through the camera
  • Autonomous commands through AI (TonyAI)
  • Low latency in mimicry mode, which is critical for effective telerobotic control and stability [2]

Screens

  • Digital teleoperation cockpit
    Digital teleoperation cockpitLow-latency video streaming with a neural overlay detecting movement in real time.
  • Kotlin app
    Kotlin app
  • Test hardware
    Test hardware
  • Test hardware
    Test hardware

How itwas built

  • ESP32
  • OpenCV
  • TensorFlow
  • Kotlin
  • MQTT
  • Python
Telemetry latency
< 18ms
Detection accuracy
90.1%
  • Hardware

    An ESP32 microcontroller receives the commands and drives the robotic arm's servos.

  • Communication

    The global connection between the app and the ESP32 runs over the MQTT protocol.

  • App (vision, edge AI and AI commands)

    The app is written in Kotlin. Computer vision models (edge AI) process the mimicry locally, tackling latency, which is central to teleoperation and visual servoing [2]. A modern AI turns natural-language text commands into robot actions.