The old adage says “a picture is worth a thousand words,” but just exactly which words is the question.
While facial recognition and GPS-enabled cameras have made tagging digital snapshots with names and locations much easier, a team of students from Duke University and the University of South Carolina has developed a smartphone app called TagSense that takes advantage of the range of multiple sensors on a mobile phone to automatically apply a greater variety of tags to photos.
Looking to make it easier for people to search and retrieve specific photos from their ever-growing digital albums, the team set about creating an app that takes advantage of not just a smartphone’s GPS, but also its accelerometer, light sensor and microphone to provide additional details about a given photo. The app doesn’t just make use of the sensors on the phone taking the photo, but also collects information from the smartphone’s of subjects within the photo – with permission of course.
“Phones have many different kinds of sensors that you can take advantage of,” said Chuan Qin, a visiting graduate student from USC. “They collect diverse information like sound, movement, location and light. By putting all that information together, you can sense the setting of a photograph and describe its attributes.”
The developers say that through the use of a smartphone’s built-in accelerometer, the app can tell if a person is standing still for a posed photograph, bowling or even dancing, while light sensor’s in the phone’s camera can be used to tell if the picture is being taken indoors or out, or on a sunny or cloudy day. The app can even look up weather conditions at the time and location of the photograph. Additionally, the phone’s microphone can be used to detect whether the subject is laughing or quiet. All of these attributes are then assigned to each photograph.