Deep Reinforcement Learning for Vision-Based UAV Tracking by an Autonomous Ground Vehicle

Gallery

Abstract

This paper addresses persistent tracking of an unmanned aerial vehicle (UAV) relative to an autonomous ground vehicle (AGV) using deep reinforcement learning (DRL), with body-frame command velocities as actions. Visual feedback is provided by a YOLOv8n detector trained on a custom dataset of 5000 annotated frames of a DJI Matrice~600 UAV, executed in real time on an upward-facing camera mounted on a Diadem AGV (RigBetel Labs); the centroid of the detected bounding box yields the planar tracking error on the image plane. A composite reward is proposed that retains the prescribed linear xy-error, angular-error, and actuation-energy terms and augments them with a bounded exponential proximity kernel, a potential-based progress term with guaranteed optimal-policy invariance, and an action-smoothness penalty. Five algorithms---DQN, TD3, SAC, TQC, and TD7---are benchmarked under an identical Markov decision process in terms of tracking error over time, three-dimensional trajectory geometry, and episodic return. TD7 attains the lowest steady-state error and the fastest convergence, followed by TQC and SAC, and a reward ablation shows that the proposed shaping accelerates convergence across all algorithm classes. Detector performance is quantified through a confusion matrix and precision--recall metrics, confirming real-time, high-recall UAV detection sufficient for closed-loop tracking.

Design Overview

This RL vision-based UAV–AGV tracking system combines onboard camera perception, UAV detection, and reinforcement learning control to maintain reliable closed-loop tracking on the ground vehicle.

Visual observations are processed by a detection module, which feeds into state estimation and a trained policy network that outputs body-frame velocity commands for the AGV. The architecture is designed for real-time operation with robust feedback from the image plane.

Dataset and code resources will be made available once the work is published.

Training Parameters

The training setup for our RL vision-based UAV–AGV tracking uses image-based observations and reward-driven policy optimization to learn robust control behavior. The system is trained to minimize haloed distance error while following circular and square trajectories in the AGV reference frame.

Key parameters include observation pre-processing, reward shaping for distance and angle alignment, and an action-smoothing term to prevent oscillatory control commands. This design prioritizes stable tracking and sample efficiency for deployment on embedded AGV hardware.

The diagram illustrates the learning pipeline and control interaction between perception, policy inference, and AGV motion execution.

Results

Detailed benchmark results for five RL algorithms on circle and square tasks. Use the arrows to cycle through each algorithm’s performance suite.

DQN

DQN performance on visual tracking tasks with stable but conservative control behavior. The plots show 3D trajectories and error traces for circle and square benchmarks.