Finger-based 3D human-swarm interaction interface: Design and human-subject evaluation

Abstract

Human-swarm interaction (HSI) in 3D environments faces critical challenges, including the high degrees of freedom (DOFs) of large swarms and limited operator spatial awareness. To address these issues, we introduce a novel finger-based HSI interface capable of managing 100 or more agents. The interface integrates three core interaction methods-Attraction, Repulsion, and Relaxed-mapping-along with auxiliary utilities and viewpoint controls, leveraging finger dexterity for expressive and responsive swarm manipulation. We conducted rigorous human-subject studies across three scenarios: pattern formation, collective exploration, and coordinated navigation. Results demonstrate that our interface significantly outperforms the baseline in performance and workload, primarily due to efficient, implicit viewpoint control. We also found strong evidence for scenario-dependent optimality, where the effectiveness of interaction methods varied by task demands. Scalability analysis revealed that performance in macro-management tasks remained constant regardless of swarm size, whereas micro-management tasks scaled linearly. Furthermore, while objective performance and perceived workload generally correlated, user preference sometimes diverged when performance gains were marginal, highlighting the importance of intuitiveness. This study provides empirical insights for designing adaptive, context-aware HSI systems for large-scale human-swarm collaboration.