Skip to content

Work / Sensor and behaviour ML

Who is holding the phone? Behavioural identification

Told which hand holds a phone, right 90%+ of the time over 30+ seconds, and who holds it, without deep learning.

Sensor and behaviour ML
Samsung R&D Institute India
Shipped
1 min read

90%+ over 30+ s

accuracy at telling which hand holds the phone, without deep learning

○ reported, from my account

87%

the same, about, over shorter windows

○ reported, from my account

1 of 6The hand shows in touch and motion

Used in one hand, a phone is worked by the thumb, and that leaves cues in touch and motion. One drawn here: a thumb swipe pivots on its joint, so its curve differs between the hands.

6 steps · 42 s

The hand shows in touch and motion. Used in one hand, a phone is worked by the thumb, and that leaves cues in touch and motion. One drawn here: a thumb swipe pivots on its joint, so its curve differs between the hands.

Taps and tilt. Where taps land and how the phone sits in the hand are cues of the same kind. An Android service works out the hand in real time.

Evidence adds up. Each swipe and tap adds a little evidence. Over 30+ seconds of use it names the hand 90%+ of the time; over shorter windows, about 87%.

Who is holding it?. The hand is the first signal. Swipes, taps, walking pattern and other sensor data describe how each person handles the phone.

Keep checking. It checks all the time. When the pattern stops matching the owner, access can be limited: against theft, or a colleague picking the phone up.

Small models, always on. No deep learning: the models are small enough to run all the time on the phone. Used in a deliberately odd way, it could still be fooled.

Illustration drawn by code, not a recording: the swipes, taps and meters are drawn to show the idea.

An Android service that works out in real time whether the phone is in the left or right hand, then combines that with swipes, taps, walking pattern and other sensor signals to keep checking who is using the phone. When the pattern looks wrong it can restrict access: against theft, or when a colleague or family member picks the phone up. It names the hand 90%+ of the time over 30+ seconds of use, and about 87% over shorter windows, without deep learning.

Problem

A phone checks who you are once, at unlock. After that, whoever holds it has the same access: a thief, or a colleague or family member who picks it up.

Approach

The first signal: the hand. An Android service tells, in real time, whether the phone is in the left or the right hand.
More signals from the same person. Swipes, taps, walking pattern and other sensor data, collected from several users, describe how each person handles the phone.
Keep checking. Identification runs continuously; when the pattern does not match the owner, access can be restricted.
No deep learning. The models are small enough to run all the time on the phone.

Results

90%+ accuracy at telling the hand over 30+ seconds of use, and about 87% over shorter windows, without deep learning. Used in a deliberately odd way, to break it, it could still be fooled in some cases.

More in sensor and behaviour ml

Phone drop detection in under 22 parametersMar 2023 – May 2023
Replaced a deep-learning drop detector of millions of parameters with sensor features and a model of under 22, saving over 95% of the power. · Samsung R&D Institute India · Shipped
Car crash detection with automatic SOSNov 2022 – Aug 2024
A phone that detects a car crash from its sensors and sends an SOS to chosen contacts. · Samsung R&D Institute India · Built