Control your mouse cursor and trigger mouse and keyboard actions with hand gestures captured through your webcam.
Turns your webcam into a virtual mouse controller. Hand detection runs on Google's MediaPipe Hands solution, which returns 21 hand landmarks per frame.
Gesture reference:
| Gesture | Action |
|---|---|
| Move index finger | Move cursor |
| Quick thumb curl into the palm | Left click |
| Hold thumb curled (0.35s+) | Click and hold (drag) |
| Closed fist (all 5 fingers curled), held ~0.4s | Copy (Ctrl+C) |
| Open palm (all 5 fingers extended), held ~0.4s | Paste (Ctrl+V) |
The index finger never takes part in the click gesture — it only drives the cursor. Curling the thumb toward the palm doesn't move the index fingertip, so clicking never nudges the cursor off target. Copy/paste require the whole hand to commit to a fist or an open palm, which keeps them clearly distinct from pointing (index finger only) and clicking (thumb only) — there's no natural real-world gesture for copy/paste (per HCI gesture-elicitation research), so this leans on the closest physical metaphor available: grab to take a copy, open the hand to release it. Both fire once per sustained pose and re-arm only once the hand returns to a neutral shape, so holding the pose doesn't repeat-fire. All gesture states are derived purely from landmark geometry (fingertip-to-wrist vs. knuckle-to-wrist distance), no external classifier involved.
Cursor movement is smoothed with an exponential moving average to reduce jitter. The app displays handedness, detection confidence, and the current action state on screen to help with calibration (src/hand_control/config.py).
main.py— thin entrypoint that runs the capture loopsrc/hand_control/— camera setup, hand detection (MediaPipe), gesture classification, action mapping, cursor smoothing, and config constants
pip install -r requirements.txt
python main.pyA webcam window will open. Place your hand in front of the camera to start controlling the mouse.
This project is licensed under the MIT License.
