Researchers develop 3D hand-sensing wristband
In a potential breakthrough in wearable sensing technology, researchers from Cornell and the University of Wisconsin, Madison, have designed a wrist-mounted device that continuously tracks the entire human hand in 3D.
The bracelet, called FingerTrak, can sense and translate into 3D the many positions of the human hand, including 20 finger joint positions, using three or four miniature, low-resolution thermal cameras that read contours on the wrist. The device could be used in sign language translation, virtual reality, mobile health, human-robot interaction and other areas, the researchers said.
“This was a major discovery by our team – that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are,” said Cheng Zhang, assistant professor of information science and director of Cornell’s new SciFi Lab, where FingerTrak was developed. “It’s the first system to reconstruct your full hand posture based on the contours of the wrist.”
Yin Li, assistant professor of biostatistics and medical informatics at the University of Wisconsin, Madison School of Medicine and Public Health, contributed to the software behind FingerTrak.
Their paper, “FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist,” was published in June in the Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. It also will be presented at the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing, taking place virtually Sept. 12-16.
Past wrist-mounted cameras have been considered too bulky and obtrusive for everyday use, and most could reconstruct only a few discrete hand gestures.
FingerTrak’s breakthrough is a lightweight bracelet, allowing for free movement. Instead of using cameras to directly capture the position of the fingers, the focus of most prior research, FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand. The bracelet’s four miniature, thermal cameras – each about the size of a pea – snap multiple “silhouette” images to form an outline of the hand.
A deep neural network then stitches these silhouette images together and reconstructs the virtual hand in 3D. Through this method, Zhang and his fellow researchers were able to capture the entire hand pose, even when the hand is holding an object.
“The most novel technical finding in this work is discovering that the contours of the wrist are enough to accurately predict the entire hand posture,” Zhang said. “This finding allows the reposition of the sensing system to the wrist, which is more practical for usability.”
The technology has a wide range of possible uses, the researchers said, but Zhang said the most promising is its potential application in sign language translation.
“Current sign language translation technology requires the user to either wear a glove or have a camera in the environment, both of which are cumbersome,” he said. “This could really push the current technology into new areas.”
FingerTrak could also have an impact on health care applications, specifically in monitoring disorders that affect fine-motor skills, Li said.
“How we move our hands and fingers often tells about our health condition,” Li said. “A device like this might be used to better understand how the elderly use their hands in daily life, helping to detect early signs of diseases like Parkinson’s and Alzheimer’s.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Wearable sensing technology
- Wearable Ozone Gas Sensor with Real Time Monitoring of Personal Exposure to Ozone by AerNoson October 6, 2020 at 2:50 am
Game changing Ground Level Ozone Gas Sensor detection from 1 ppb to 500 ppb with 1 ppb resolution now shipping. Scientists, physicians and researchers have long been aware of ...
- Development of haptic touch sensor that works by static electricityon October 5, 2020 at 4:35 am
Super-micro, low-power sensors and devices that can send and receive signals and information anytime, anywhere will become an integral part of people's lives in a hyper-connected world driven by the ...
- Amazing wearable could detect epileptic seizures an hour before they strikeon October 1, 2020 at 12:33 pm
Tech could help users to prepare for seizures such as falls or other injuries that might arise from epilepsy sufferers being caught unawares.
- 3Q Tech Report: New Ideas for a Strange Eraon September 30, 2020 at 2:18 pm
Innovations in construction technology are reconfigured for the COVID-19 pandemic as firms look to preserve the momentum behind digitization ...
- NextUp: The Philly Company Working on a Wearable Device to Detect COVID-19 Symptomson September 30, 2020 at 1:01 pm
RTM Vital Signs is working on monitoring symptoms that can be used to track everything from heart health to respiratory function.
Go deeper with Google Headlines on:
Wearable sensing technology
Go deeper with Bing News on:
- Next Time We Zoom, Show Me Your Handson September 8, 2020 at 11:59 pm
Technologies could be also used for hands-based communication: The recently launched FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand.
- Next Time We Zoom, Show Me Your Handson September 8, 2020 at 11:42 pm
Technologies could be also used for hands-based communication: the recently launched FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand.
- Wrist-mounted wearable tracks your hand in 3D using thermal sensorson July 29, 2020 at 8:22 pm
The team has developed a device they call the FingerTrak. The wearable employs four low-resolution thermal sensors, each the size of about a pea, to monitor the contours of the wrist. The cameras ...