Virginia Tech® home

Sergio Ballivian: Localizing and tracking individuals in indoor environments using machine learning

The following story was written in April 2018 by Lauren Armstrong in ​ENGL ​4824​: Science Writing ​as part of a collaborative project that included the English department, the Center for Communicating Science, ​the ​Fralin Life Science​ Institute​, and ​Technology-enhanced Learning and Online ​Strategies (TLOS).

Perhaps the most striking take-away from meeting Sergio Ballivian, a graduate student conducting research at Virginia Tech, is his unwavering motivation to succeed. With only two weeks left in his undergraduate career in civil engineering in 2017, he decided to stay to get a master’s degree–-a decision he never anticipated making and one he has never looked back on.

Sergio Ballivian

    Ballivian, originally from Bolivia, came to America with his mother in 2012. When he arrived in Northern Virginia he spoke no English. He began learning through English as a Second Language classes while simultaneously taking engineering classes at Northern Virginia Community College (NOVA). He spent four years working and taking classes at NOVA before finishing his last two years at Virginia Tech.

    He thrived here in Blacksburg, including participating in “Concrete for kids,” an outreach program for schoolchildren, and attending weekly research meetings in an engineering laboratory. Before long, he became a graduate research assistant in the Vecellio Construction Engineering & Management Program (VCEMP) as a member of Dr. Jazizadeh’s research group. Ballivian now considers this lab in 315 Patton Hall a second home.

    The goal of his research is localizing and tracking individuals as they move throughout an indoor environment using passive detection methods and a machine learning system that requires minimal training. Passive detection methods are those that do not require an individual to wear or carry a tag or receiver to be localized. In this case, they are detected by depth sensors. The depth sensors are run on a Matlab code, which utilizes machine learning algorithms. The code is Ballivian’s primary task on the team. Machine learning algorithms are those which allow the system to learn and improve without the need for additional programming.

    The sensors find a person in the space, distinguish who the person is based on their unique body dimensions, and then track them as they move throughout the building and in and out of range of separately located sensors. 

Computer screen showing human tracking
The screen shows an image processed by the lab’s sensors of subjects moving throughout the room.

    While the sensors are good at distinguishing people and can work in both the light and the dark, they do have some limitations. These include only having the capability to track up to six people per sensor and only being able to recognize and track individuals accurately from up to 10 feet away. For this reason, multiple thoughtfully placed sensors would be required. 

Computer screen with grid and human form
This image shows a more sophisticated three dimensional representation of a subject in front of a depth sensor where thermal loads can also be observed.

    The research conducted in this lab could be used in a variety of applications. Ballivian's team focuses on two main functions. The first is for the healthcare market to track individuals in case of emergencies or disaster. The second is for heating, ventilation, and air conditioning (HVAC) implementation in sustainable building design. The sensors would be programmed to know how much thermal comfort is required for a person and then would track when people were in the space, including the number of people, to determine how much heating or cooling was required. Potential applications of this research could be important for energy savings.

    When observing Ballivian explore these applications in the lab, it was especially apparent how passionate he is about learning new things. With so much success already behind him, it is impressive to think about what contributions Ballivian and his research can make to society.