Solve a randomly generated maze hands-free, using only your voice. Say "left", "right", "up" or "down" to move and "stop" to quit. A convolutional neural network classifies each one-second clip in real time and moves the marker through the maze. The recogniser reaches 94.57% validation accuracy across six commands.
📄 Published at IEEE ICCCNT 2023. See Publication.
A live game frame. The title shows the recognised command, the black diamond is the current position, and the trail behind it is the path already travelled.
Voice control widens who can play a game, but only if recognition is fast enough that the game does not feel laggy — which is the real constraint, and the reason the model here is deliberately small. A maze is a good testbed for it: the vocabulary is closed and tiny, and a misrecognition is immediately visible on screen rather than buried in a metric.
The network was grown one convolutional block at a time until the accuracy stopped improving:
| Configuration | Accuracy |
|---|---|
| 1 convolutional block | 79.0% |
| 3 convolutional blocks | 94.52% |
| 5 convolutional blocks (final), training | 95.34% |
| 5 convolutional blocks (final), validation | 94.57% |
Most of the gain arrives by three blocks; the last two buy under a point. That is a useful thing to know for a model that has to run inside a one-second interaction budget.
Confusion matrix on the validation set. The four movement commands are recognised at 93–98%; most of the remaining confusion is with the catch-all
unknownclass rather than between two movement commands — the failure mode that matters least in play.
Training accuracy (blue) against validation accuracy (dashed) over 10 epochs.
flowchart LR
A[Microphone<br/>1-second clip] --> B[Auditory spectrogram<br/>50-band Bark, log-scaled]
B --> C[CNN<br/>5 conv blocks]
C --> D{Recognised command}
D -->|left / right / up / down| E[Move marker<br/>if the move is legal]
D -->|stop| F[Abort the game]
E --> G[Maze redrawn in MATLAB]
1. Audio to spectrogram (helperExtractAuditoryFeatures.m). Audio is never fed to the network
raw. Each clip is zero-padded to exactly one second and converted to a Bark auditory
spectrogram, then log-scaled to smooth the distribution:
| Parameter | Value |
|---|---|
| Segment duration | 1 s |
| Frame duration | 25 ms |
| Hop duration | 10 ms |
| Filter bands | 50 |
Fixing the clip length turns every utterance into a same-sized image, which is what makes a convolutional network the natural fit.
2. Command recognition (Model_Training_Evaluating.mlx). Five repeated
convolution → batch normalisation → ReLU → max pooling blocks over the spectrogram, with filter
counts scaling as numF, 2*numF, 4*numF (numF = 12), then time-dimension pooling, dropout,
a fully connected layer and softmax. Trained with Adam, learning rate 0.003, mini-batch size 128,
10 epochs, with background-noise augmentation mixed into the training set.
The class list is up, down, left, right, stop, go plus unknown and background. Those two
extra classes are what stop silence or an unrelated word from being forced into a movement
command and moving the marker when the player did not ask.
3. The maze game (maze.m). A random perfect maze is generated by growing a spanning tree
over the grid graph and removing the walls it crosses — every cell reachable, exactly one path
between any two. On each keypress the game records one second at 16 kHz, extracts the same
features used in training, classifies, and applies the move if no wall blocks it. You start at the
top-left and win by reaching the bottom-right.
- Language / Environment: MATLAB
- Toolboxes: Deep Learning Toolbox, Audio Toolbox
- Model: CNN over auditory-spectrogram input
- Dataset: Google Speech Commands v0.02
Playing-Maze-Using-Voice-Commands/
├── src/
│ ├── maze.m # maze generation, rendering, voice-driven game loop
│ ├── helperExtractAuditoryFeatures.m # audio to Bark auditory spectrogram
│ ├── Model_Training_Evaluating.mlx # CNN training and evaluation (live script)
│ └── model1net.mat # trained network, ready to use
├── docs/
│ ├── Report.docx # full paper
│ └── Presentation.pptx
├── assets/ # figures used in this README
├── DATASET.md # dataset download and layout
├── README.md
└── LICENSE
Requires MATLAB with the Audio Toolbox and Deep Learning Toolbox, plus a microphone. The trained model is included, so you can play immediately without retraining.
-
Set the MATLAB current folder to
src/, somaze.m, the feature helper and the model are all on the path. -
Start a maze of your chosen size:
maze(10, 10) % rows, columns
-
Press any key. When the title reads "Start Speaking", say one command within one second — the recording window is exactly that long, so speak promptly.
-
Reach the bottom-right cell to win. Say
stopto abort at any point.
To retrain from scratch, open src/Model_Training_Evaluating.mlx and run it. A parallel pool
speeds up feature extraction considerably. See DATASET.md for the data.
Google Speech Commands v0.02 — six command classes plus unknown and background-noise classes. Download details and the expected folder layout are in DATASET.md.
Presented at the 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), Delhi, India, and published by IEEE.
K. Vishnu Sainadh, K. Satwik, V. Ashrith, Susmitha Vekkot. "Playing Maze using Voice Commands." In: 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), Delhi, India, IEEE, 2023, pp. 1–7.
🔗 Read the paper (IEEE, DOI: 10.1109/ICCCNT56998.2023.10307127)
- Extend voice control to other games beyond the maze.
- Apply the same command recogniser to smart-home appliance control.
- Support continuous listening rather than press-a-key-then-speak.
Corrections, accessibility improvements, and reproducibility updates are welcome. Open a focused pull request with the MATLAB release, toolbox versions, dataset assumptions, and commands tested. Do not commit local recordings, generated models, or redistributed dataset copies.
Licensed under the MIT License.