Dr. Marianne Black’s research reveals how AR can aid in surgical rehabilitation
Illustration by Sage Blackwell.
Augmented reality, sometimes known as AR, is a rapidly developing form of technology that integrates computer-generated digital images with the real world. AR has various applications, ranging from photo app filters to phone camera language translation to video games like Pokémon GO.
AR differs from virtual reality, better known as VR, in terms of a user’s vision. AR users are still able to see the real world around them, but VR relies entirely on digital objects and landscapes — such virtual applications include chatrooms like VR Chat, art-making software such as Tilt Brush, and training simulations like Axon Enterprise’s VR Training program.
However, AR software can also be used for a variety of medical purposes.
Dr. Marianne Black, former Canada Research Chair in Assistive Technologies at UVic, used AR software to support those with osteoarthritis — a degenerative joint condition. One of Black’s projects incorporated AR with rehabilitative treatment following surgery, using the technology to “[guide] individuals to remotely manage musculoskeletal disorders.”
The Martlet had the opportunity to sit down with Dr. Black and dive into the medical potential of AR technology.
This interview has been edited for clarity and concision.
Where is AR technology being used now?
“Some people have deployed it for some surgical procedures. I think there’s a group — I want to say they’re out of Seattle — that use it to help with spine surgery. People have started using it in manufacturing processes. You can imagine when a new person is learning how to create something — you’ve got a manual, you’ve got someone teaching you — putting on a headset and having the instructions delivered to you that interact with whatever machines are around you can be really helpful to guide them.”
Where does AR come into your projects?
“I was inspired because one of my friends in my lab was working on a really cool project for breast cancer surgery. One of the biggest issues with breast cancer surgery, if you’re trying to remove a tumour … is [that] the surgeon’s trying to carefully resect the margins of the tumour — they don’t want to over-resect because then things won’t feel right after, and then, if they under-resect, they have to go back in if there’s still cancerous tissue.
We were a medical imaging lab, so we had these really nice medical images to help us identify the location of the tumour, but then when the surgeon’s in the operating room, doing the mental fusion between those images they have and the person on the table is really challenging. So, my friend figured out how to overlay those images on the person with an AR headset, so the surgeon could see where the tumour is.… I started to think it would be really interesting to use this technology to guide people through physical therapy or physical rehabilitation.
With people who have osteoarthritis, one of the ways they can help improve their symptoms is exercise therapy. A lot of it is, like, doing squats in very specific ways. But people tend to not follow through on their physical therapy, because often they go home after being in the clinic and they don’t know if they’re doing it right. And it’s not because they’re lazy or anything negative there…. They really have a lack of confidence … when they’re not being observed.
So, what if we had a way to provide complete, 3D feedback for the people to guide them through the movements and let them know when they’re doing it wrong?
Our AR device is just like a really bulky pair of sunglasses. You can see the entire room around you and it’s slightly shaded, but you can display virtual objects.
What I’m trying to work on with my students is how can we display these virtual objects, and detect human motion, with the AR headset to guide people through the movements that they need for rehabilitation. You can imagine, even though I’m kind of taking this with my background of osteoarthritis, it can be helpful for any musculoskeletal disorders, and I also want to explore using it for optimizing sports performance, too.”
What were some past projects you worked on?
“We used a cadaveric model.… We took our cadavers we had — just mid-shin to mid-thigh — we put them up in this biomechanical rig, where you drive it through flexion and extension [bending and unbending the knee]. You’re pulling on the quadriceps tendon [which connects the kneecap to thigh muscles] to go through the flexion and extension.
Then we have sensors in the knee joint to look at the force being distributed, and then other sensors that were attached to all the bones to look at how the bones are moving, as well, in this location. So that just provided more insight into how can we change that osteotomy, or the cut that we’re putting into the bone, to affect changes and load distribution and movement.… It was super fun just to work with all the technology.”
Why is AR, among other technologies, important for the healthcare system?
“Tech giants will be pushing out this technology. We can expect — and they expect — that it’s going to replace our phones for all the activities that we typically do.… As a researcher who’s really interested in health, I want to get ahead of this mass deployment and make sure that we know how to use these devices to help people. And that they can be best used to improve human health — improve medicine –– in ways that we’re ready for, rather than waiting for the mass-market devices to come out and then spending years figuring out how we optimize them to help people.”
