PROJECT OVERVIEW

Main goal of Augmented Reality (AR) technology is that it extends what people can
see with additional data/information. People can get information without need of using any
input devices like mouse or keyboard. Until recently this technology was not evolving very
well as there were a lot of technological limitations. Now everything changed and many
different types of AR devices appeared on the market.
Currently the entire process of patient examination is done by doctors with many hi-tech
devices which still present results on separate screens/outputs. Each device contains a screen
where the physician needs to look for result/progress, without a unified layout. This pulls their
attention away from patient, which is unwanted especially at Intensive Care Unit’s (ICU) or
Critical Care Unit’s (CCU). A universal device, which would contain all the information in
one place would be greatly appreciated in these settings.
Main goal of this work is to show the capability of Augmented Reality to be used during
patient’s examination by looking at him without the need to look away to view important vital
signs on various devices around the bed, specifically to show how information from a medical
device can be collected and broadcast to a AR-capable device over WiFi. To make it work, a
dedicated interface between diagnostic devices and Augmented Reality Glasses was
developed. This was realized using a Raspberry Pi microcomputer connected to the HeartRate
sensor. Heart rate sensor returns very exact values from heart reads. Value range of sensor is
from 1 to 1024 which is very good for precision and future development of solution (Fig. 1).
The computer collects patients’ heart rate data and after analysis sends it over WiFi network
to AR device where the patient’s heart rate is displayed. Specifically, for every new value of
heart beats per second, application executes a linux-based command which opens a dedicated
UDP port and shares data from the algorithm over it. Data can be accessed by any
software/device which knows Raspberry Pi’s IP address and UDP port number. Next, we
have used AR device, running Windows-based operating system, to access the data sent from
Raspberry Pi. Applications running on AR device are written in Unity Engine, which allows
to use all advantages of Augmented Reality technology. In our case, the application connects
to a specific IP address and UDP port, and listens on them for incoming data. New information is automatically presented to the doctor in real time, whilst allowing continued
and undisturbed doctor-patient interaction.
The purpose of this solution was to show how easily it can be to relay information
from medical devices onto AR apparatus. For the first time the physician is not limited to the
place where the device screen is placed/installed. Thanks to this solution, one can imagine
that every medical information can be digitalized, sent and displayed on AR devices.