A mixed reality testbed for design and validation of estimation and control strategies for unmanned systems research




Reyes, Gabriel

Journal Title

Journal ISSN

Volume Title




Unmanned Aircraft Systems (UAS) research and development groups validate their estimation and control strategies in simulations and real-life scenarios. However, various factors affect the time it takes to perform such tests as well as their quality, e.g. hardware malfunctions and physical limitations of the testing equipment and environment. This project focuses on generating and integrating virtual autonomous vehicles with real prototypes into a mixed reality space for evaluation of single and multi-agent UAS strategies. The calibration and alignment of the computer-generated world with respect to the inertial frame of the real world was accomplished based on the Umeyama algorithm. Making use of an entity existing in both real and virtual worlds, it was found that the pose error of the entity in the mixed reality environment with respect to the real world is always kept below 10%. A control strategy for enabling multiple virtual UAS for tracking a real ground vehicle was generated in the mixed reality environment. In our tests, a specific flight formation around the target vehicle was achieved by the virtual agents. Additionally, the flight formation around the moving target can be commanded online by a human user. Ultimately, the development of this project resulted in a modular solution that facilitates testing and evaluation of estimation and control strategies for autonomous agents, with the possibility of adaptation and expansion into different research domains.



Control, Mixed Reality, robotics, Test and Validation, Unmanned Aircraft Systems



Attribution-NonCommercial 4.0 International, This material is made available for use in research, teaching, and private study, pursuant to U.S. Copyright law. The user assumes full responsibility for any use of the materials, including but not limited to, infringement of copyright and publication rights of reproduced materials. Any materials used should be fully credited with its source. All rights are reserved and retained regardless of current or future development or laws that may apply to fair use standards. Permission for publication of this material, in part or in full, must be secured with the author and/or publisher.