About Vice-Principal Research“Someday soon when you enter a medical building, there will be a tablet that will automatically check and report whether you are healthy or not.”
Professor Farhana Zulkernine is describing a virtual visit to a hospital or a doctor’s office in the not-too-distant future. You don’t have to imagine that we are still in the grips of an enduring pandemic, just think of a doctor’s office in flu season. A quick scan of your face will give the nurses your temperature, blood pressure, oxygen, and a host of other readings, saving time and minimizing the chances that health-care workers and other patients get infected.
It’s not here yet, but a recent project named Veyetals that Dr. Zulkernine and the students working in her Bigdata Analytics and Management Laboratory at Queen’s University have undertaken with Canadian AI firm MarkiTech is helping to bring that day closer. A beta version for measuring heart rate, heart rate variability, oxygen saturation, blood pressure and stress level is currently being tried out by hundreds of users in Canada, the United States, Australia, the United Arab Emirates and Pakistan.
A professor in Queen’s School of Computing, Dr. Zulkernine’s particular field of interest is, she says, “analytics and management of any kind of large data.” She has a special interest “in multimodal streaming data from the Internet of Things,” as it’s known, with such “things” as wearable sensors, audio and video devices, and other digital sources generating text or hybrid data.
Dr. Zulkernine was connected to MarkiTech by a previous industry collaborator, Gnowit, due to MarkiTech’s interest in wearable technologies and remote health monitoring. MarkiTech had already developed what CEO Nauman Jaffar calls “a unique, contactless, remote patient monitoring system” designed for seniors that features a wifi device, a tablet camera intended to scan the face to check vital signs and a series of remote monitoring devices. But their target audience, people in their 70s, 80s and even older, are not always comfortable with wearing sensor devices all the time and had issues with the unit’s batteries as well. MarkiTech felt that a remote health-monitoring tablet coupled with a proactive, voice-enabled bot to measure vital signs using a smart phone camera might work better.
As is so often the case with any new innovation, developing it takes money, but in the world of startups raising money requires some sort of proof of concept. It is, says Jaffar, “a long, drawn-out process.” Dr. Zulkernine was able to help. She constantly looks for innovative private-sector projects that can engage the students in her lab in developing technology for real-life applications. To that end, she turned to Mitacs, a national not-for-profit organization. Mitacs seeks to advance industrial and social innovation in Canada by helping businesses benefit from academic research by jointly funding internships with interested companies for graduate and undergraduate students. “Many of my projects have been funded by Mitacs,” she says. “Mitacs is great because they give opportunities for students to get hands-on experience.” And because the students work so closely with private sector companies, Mitacs-enabled internships often smooth the path to future employment.
Supported by a Mitacs internship with MarkiTech as the industry partner, Donghao Qiao, one of Dr. Zulkernine’s PhD students, began developing an algorithm that would take the face video captured by a cell phone camera and use it to measure a patient’s heart rate, heart rate variability, oxygen saturation and stress level. In turn, MarkiTech was able to have potential users field test the new phone app.
Not surprisingly, there were a number of challenges – if the lighting was poor, the person wasn’t directly in front of the camera or it wasn’t held steady, the readings were affected. With continued support from Mitacs and MarkiTech, one of Dr. Zulkernine’s M.Sc students, Amtul Haq Ayesha, has been working on these problems as well as extending the range of the app’s diagnostic tools to monitor blood pressure and temperature. Ayesha’s work on the project will continue to the end of this year. “It has been really interesting,” she says. “This research has been so novel.”
“There is still a lot to be done,” says Dr. Zulkernine. For example, her lab has a great deal of visual data for people in the normal range for heart rate or oxygen levels, but very little for those lying outside that range. And it’s not easy to come by. “MarkiTech has been really active in getting this data from outside Canada,” she says. Ultimately, she can see the idea being extended beyond clinics, hospitals and a cell phone app. “I would like to combine this research with the work I have been doing on human activity recognition to facilitate home or long-term care.” For example, based on patients’ needs and health status, multiple wearable and camera-based monitoring could be combined to not only record vital signs but also remotely monitor patients’ daily activities and notify others of dangerous incidents such as falls.
The Queen’s Partnerships and Innovation (QPI) team helps companies and other external organizations find Queen’s researchers with whom to collaborate. QPI and Research Services colleagues work closely with federal and provincial funding agencies to leverage investments in research, help researchers and partners prepare proposals, and develop associated agreements.