For a delicate micromanipulation, subjects need to perform tasks while effectively minimizing their hand tremor. Recently, there is growing interest in creating virtual reality (VR) microsurgery systems to enable enhanced visual feedback for precise manipulation tasks as well as a training environment. For an accurate micro-location, it is needed the use of various sensors (sensor fusion) which encompasses different ways of combining data through algorithms in order to refine information quality or derive more information about it. To enable precise location tracking of surgical tools (e.g., forceps), we propose a VR microsurgical environment featuring a sensor fusion algorithm for location tracking system with an additional inertial measurement unit (IMU). The system reconstructs an image of surgical instruments (i.e., forceps) and 2D figures with a zoomed view of them, to help surgeons have a more precise sense in visual feedback. We hypothesize that merging two different sensors and matching them in a Kalman filter algorithm, the micro-localization will be more accurately. We develop and test two ways, that is, (1) location tracking without Kalman filter, and (2) location tracking with Kalman filter activated. A performance evaluation of location tracking in terms of error has been conducted in real and virtual experiments. The result shows that the condition with sensor fusion activated achieves more precise location tracking than without Kalman filter condition.
Table Of Contents
1. Introduction 1-- 1.1 Study Background 1-- 1.2 Previous Studies 2-- 1.2.1 Virtual reality applications 2-- 1.2.2 The importance of virtual reality 2-- 1.2.3 Visual stimuli in virtual reality to reduce hand tremor 3-- 1.3 Study Purpose 5-- 2. Sensor Fusion 6-- 2.1 Sensor Fusion Algorithms 6-- 2.1.1 Bayesian network 6-- 2.1.2 Dempster-Shafer 9-- 2.1.3 Convolutional neural networks 10-- 2.2 Sensor Fusion Methods 11-- 2.2.1 Smoothing 11-- 2.2.2 Filtering 12-- 2.2.3 Prediction 12-- 2.3 Sensor Fusion Example 12-- 2.4 Theory of Kalman Filter 14-- 2.5 System Model for Kalman Filter 18-- 3. Implementation of the VR System 18-- 3.1 System Architecture 18-- 3.2 Hardware System 19-- 3.3 Software System 19-- 3.4 General Schematic of the System 20-- 4. Experiments 21-- 4.1 Single Axis Dot by Dot 21-- 4.2 Motorized Trace 24-- 4.2.1 X axis 24-- 4.2.2 Y axis 26-- 4.3 2-D Axis Trace 27-- 4.3.1 2-D printed circle 28-- 4.3.2 VR circle 31-- 5. Discussion and Conclusion 35-- REFERENCES 36