Multi-Solutions Workshop — Profiling & optimizing AI models on the Qualcomm NPU

Companion material for the hands-on workshop on profiling and optimizing Edge Impulse models for the Qualcomm Dragonwing™ QCS6490 NPU. It applies to the Dragonwing RB3 Gen 2 Dev Kit, Thundercomm Rubik Pi 3, and other QCS6490-based boards.

This repo contains everything an instructor or participant needs to run the workshop: the slide deck, setup and profiling scripts, and worksheets.

What you'll learn

• How AI accelerators schedule work across the NPU, GPU, and CPU
• Why some operators fall back to the CPU
• Three ways to measure where a model runs
• How to optimize a model to increase NPU offload — the main showcase

Companion tutorials (Edge Impulse docs)

Tutorial	Purpose
Optimize a model for the Qualcomm NPU	Main showcase: the profile → optimize → re-profile loop
Profile model layer placement with Qualcomm AI Hub	Map each layer to NPU/GPU/CPU from Studio
Measure on-device NPU performance on QCS6490	Verify QNN acceleration and measure throughput

Repository structure

.
├── README.md
├── LICENSE
├── slides/
│   ├── slides.md          # Marp slide deck (source)
│   └── README.md          # how to export to PDF/HTML
├── scripts/
│   ├── 00-setup-rubikpi-ubuntu.sh
│   ├── 01-setup-rb3-qualcomm-linux.sh
│   ├── 02-verify-qnn.sh
│   ├── 03-run-inference.sh
│   ├── 04-run-gstreamer-pipeline.sh
│   ├── 05-monitor-utilization.sh
│   └── README.md
└── content/
    ├── agenda.md
    ├── prerequisites.md
    ├── profiling-worksheet.md
    ├── optimization-checklist.md
    └── instructor-notes.md

Prerequisites

• A QCS6490 board:
  • Qualcomm Dragonwing RB3 Gen 2 Dev Kit (Qualcomm Linux), or
  • Thundercomm Rubik Pi 3 (Ubuntu 24.04 or Qualcomm Linux)
• A free Edge Impulse account with a trained impulse
• Terminal access to the board over serial or SSH
• A USB or CSI camera for the vision pipeline

See content/prerequisites.md for the full checklist.

Quick start

Run these on the board, in order. Full details are in scripts/README.md.

# 1. Set up the board (pick one for your OS)
./scripts/00-setup-rubikpi-ubuntu.sh        # Rubik Pi 3 / Ubuntu 24.04
./scripts/01-setup-rb3-qualcomm-linux.sh    # RB3 Gen 2 / Qualcomm Linux

# 2. Confirm the QNN backends (including the Hexagon NPU/HTP) are reachable
./scripts/02-verify-qnn.sh

# 3. Run accelerated inference and watch per-frame timing
./scripts/03-run-inference.sh

# 4. Run the object detection GStreamer pipeline (from an IM SDK deployment)
./scripts/04-run-gstreamer-pipeline.sh /path/to/unzipped-imsdk-deployment

# 5. Sample CPU utilization while the pipeline runs (separate terminal)
./scripts/05-monitor-utilization.sh 60 1 cpu-utilization.csv

Slide deck

The deck is written in Marp Markdown at slides/slides.md. Export it to PDF or HTML with a single command — see slides/README.md.

Contributors

• Lukasz Grzymkowski — Arrow
• Eoin Jordan — Edge Impulse
• George Dickey

License

Licensed under the Apache License, Version 2.0. See LICENSE.