To locate the different joints of the human body in 3D space when walking on a treadmill, a state-of-the-art 2D skeleton algorithm will be combined with depth imaging. During this project we will investigate different combinations of RGB and time-of-flight cameras to achieve the most accurate joint positions.
In the first part of this validation phase, we will use virtual scenes with a known ground truth. In the virtual scenes we will simulate both
the 2D and 3D imaging devices on different human bodies.
When all validation results based on virtual images are satisfying, we will benchmark the entire skeleton tracking system against a commercially available marker-based Vicon system.
During this stage of the research project, we will write software to control the treadmill and to link its data to a virtual environment. Also the skeleton tracking system developed earlier, will be combined with the treadmill.
To make the rehabilitation exercise more patient specific, some virtual environments will be created in Unity in combination with the HTC VIVE Pro virtual reality glasses. The physiotherapist now has the ability to change the surroundings (beach, north pole, city, ...), but also to add specific features such as birds, people, signs, etc. This makes it possible to trigger the patient to move certain body parts (head, arms, ...).
Finally, the gait of the patient walking on a treadmill, will be analysed and compared to situations where no virtual reality environment is used. Also we will gather some information on how patients perceive this new approach.