3rd place worldwide, Huawei ICT Competition · Teknofest 2025 finalist
Conversign is an Android app that translates sign language gestures into text in real time, using the phone's camera. Hand tracking and classification both run on the device, so recognition works without sending video to a server.
Available on Huawei AppGallery.
This is a showcase repository. The source code is kept in a private repository; this page describes what the app does and how it is built.
- Reads the live camera feed and finds the hand in each frame.
- Extracts 21 hand landmarks with MediaPipe.
- Classifies the gesture with a model trained in PyTorch and exported to ONNX.
- Shows the recognized sign as text in the UI.
flowchart LR
A[Camera frame] --> B[MediaPipe<br/>21 hand landmarks]
B --> C[ONNX Runtime<br/>gesture classifier]
C --> D[ViewModel<br/>UI state]
D --> E[Jetpack Compose<br/>text on screen]
The model is trained offline in PyTorch on collected gesture data, exported to ONNX, and bundled with the app. At runtime, ONNX Runtime runs inference on the landmark coordinates rather than raw pixels, which keeps the model small enough for a phone.
The app follows Clean Architecture with MVVM:
| Layer | Responsibility |
|---|---|
| Presentation | Jetpack Compose screens and ViewModels that expose UI state |
| Domain | Use cases for recognizing a gesture from a set of landmarks |
| Data | Camera input, MediaPipe hand landmark detection, ONNX Runtime inference |
The first version was written in Flutter. It was rewritten in native Kotlin and Jetpack Compose to fix compatibility issues on Huawei devices.
Kotlin · Jetpack Compose · MVVM · Clean Architecture · MediaPipe · PyTorch · ONNX Runtime
The source code is not public, so the app cannot be built from this repository. To try Conversign, install it from Huawei AppGallery.
Ulaş Uçan · LinkedIn · uls.ucn@gmail.com