feat: add RK3576 airborne object detection (YOLOv8s, VisDrone) package

.github/workflows/docker-build.yml

@@ -18,6 +18,7 @@ on:
       - 'src/rk3576_yolov8_pose/**'
       - 'src/rk3588_yolov8_seg/**'
       - 'src/rk3576_yolov8_seg/**'
+      - 'src/rk3576_yolov8_airborne/**'
       - 'src/rk3588_whisper/**'
       - 'src/rk3576_whisper/**'
       - 'docker/**'
@@ -58,6 +59,7 @@ jobs:
           - rk3576_yolov8_pose
           - rk3588_yolov8_seg
           - rk3576_yolov8_seg
+          - rk3576_yolov8_airborne
           - rk3588_whisper
           - rk3576_whisper
 

README.md

@@ -58,6 +58,7 @@ sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-yolo11:latest
 sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolo11:latest
 sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-yolov8-obb:latest
 sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolov8-obb:latest
+sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolov8_airborne:latest
 ```
 
 #### Step C: Run with One Click

README_zh.md

@@ -58,6 +58,7 @@ sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-yolo11:latest
 sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolo11:latest
 sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-yolov8-obb:latest
 sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolov8-obb:latest
+sudo docker pull ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolov8_airborne:latest
 ```
 
 #### 步骤 C：一键运行

docker/rk3576/yolov8_airborne.dockerfile

@@ -0,0 +1,53 @@
+# Use Python 3.11 slim image as base (matching the wheel requirement)
+FROM python:3.11-slim
+
+# Set working directory
+WORKDIR /app
+
+# Install system dependencies for OpenCV and others
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    libgl1 \
+    libglib2.0-0 \
+    libgomp1 \
+    libsm6 \
+    libxext6 \
+    libxrender1 \
+    libxcb-cursor0 \
+    libxcb-xinerama0 \
+    libxcb-keysyms1 \
+    libxcb-image0 \
+    libxcb-shm0 \
+    libxcb-icccm4 \
+    libxcb-sync1 \
+    libxcb-xfixes0 \
+    libxcb-shape0 \
+    libxcb-randr0 \
+    libxcb-render-util0 \
+    libxkbcommon-x11-0 \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy requirements file
+COPY requirements.txt .
+
+# Install Python dependencies
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy the RKNN Toolkit Lite2 wheel
+COPY rknn-toolkit-lite2-packages/rknn_toolkit_lite2-2.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl .
+
+# Install the local wheel
+RUN pip install --no-cache-dir rknn_toolkit_lite2-2.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+
+# Copy librknnrt.so to /usr/lib/
+COPY lib/librknnrt.so /usr/lib/
+RUN chmod 755 /usr/lib/librknnrt.so
+
+# Copy the rest of the application code
+COPY . .
+
+# Expose port for Web Preview
+EXPOSE 8000
+
+# Set the default command to run the airborne detection script
+# This script now supports both GUI (if DISPLAY is available) and Web Preview (on port 8000)
+CMD ["python", "web_detection.py", "--model_path", "model/yolov8s_airborne.rknn", "--video_path", "video/test.mp4"]

src/rk3576_yolov8_airborne/README.md

@@ -0,0 +1,163 @@
+# RK3576 Airborne Object Detection (YOLOv8s) Deployment Guide
+
+[English] | [中文](./README_zh.md)
+
+This directory contains a YOLOv8s airborne object detection inference package optimized for RK3576, trained on the VisDrone dataset for drone/aerial view object detection.
+
+## Core Features
+- **Hardware Acceleration**: Optimized for RK3576's 6 TOPS NPU architecture.
+- **Latest Driver**: Integrates the 5th generation NPU runtime library supporting RK3576.
+- **Flexible Input**: Supports camera and local MP4 video input.
+- **VisDrone Classes**: Pre-configured with 11 VisDrone aerial categories (pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, motor, others).
+
+## Directory Structure
+- `lib/`: Contains `librknnrt.so` for RK3576.
+- `model/`: Stores the `.rknn` model converted for RK3576 (e.g., `yolov8s_airborne.rknn`).
+- `py_utils/`: Inference engine wrapper and post-processing utilities (letterbox + coordinate restoration).
+- `web_detection.py`: Main program (supports Web preview and API).
+- `video/`: Sample test video.
+
+## VisDrone Classes (11)
+
+| ID | Class |
+|----|-------|
+| 0 | pedestrian |
+| 1 | people |
+| 2 | bicycle |
+| 3 | car |
+| 4 | van |
+| 5 | truck |
+| 6 | tricycle |
+| 7 | awning-tricycle |
+| 8 | bus |
+| 9 | motor |
+| 10 | others |
+
+## Quick Start
+
+### 1. Run the Project (One command, dual-mode preview)
+
+This project supports simultaneous preview via **Local GUI** and **Web Browser**. The program automatically detects the display environment and downgrades to Web mode if no display is connected.
+
+#### Step A: Configure Display Permissions (Optional)
+If you have a monitor connected and want to see the window locally:
+```bash
+xhost +local:docker
+```
+
+#### Step B: One-click Run
+```bash
+sudo docker run --rm --privileged --net=host \
+    -e PYTHONUNBUFFERED=1 \
+    -e RKNN_LOG_LEVEL=0 \
+    --device /dev/video0:/dev/video0 \
+    --device /dev/dri/renderD128:/dev/dri/renderD128 \
+    -v /proc/device-tree/compatible:/proc/device-tree/compatible \
+    ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolov8_airborne:latest \
+    python3 web_detection.py --model_path model/yolov8s_airborne.rknn --video video/test.mp4
+```
+Access via: `http://<Board_IP>:8000`
+
+> **Note**: The program defaults to the 11 VisDrone classes. If you need custom classes, you can add `-v $(pwd)/class_config.txt:/app/class_config.txt \` mount and the `--class_path` parameter.
+
+Example:
+
+```bash
+sudo docker run --rm --privileged --net=host \
+    -e PYTHONUNBUFFERED=1 \
+    -e RKNN_LOG_LEVEL=0 \
+    -v $(pwd)/class_config.txt:/app/class_config.txt \
+    --device /dev/video0:/dev/video0 \
+    --device /dev/dri/renderD128:/dev/dri/renderD128 \
+    -v /proc/device-tree/compatible:/proc/device-tree/compatible \
+    ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolov8_airborne:latest \
+    python3 web_detection.py --model_path model/yolov8s_airborne.rknn --video video/test.mp4 --class_path class_config.txt
+```
+
+---
+
+## 🔌 API Documentation
+
+This project provides RESTful interfaces compatible with the Ultralytics Cloud API standard, supporting object detection via image, video uploads or direct camera calls.
+
+### 1. Model Inference Interface (Predict)
+
+**Endpoint:** `POST /api/models/yolov8/predict`
+
+#### Request Parameters (Multipart/Form-Data):
+- `file`: (Optional) Image file to be detected.
+- `video`: (Optional) MP4 video file to be detected.
+- `timestamp`: (Optional) Timestamp in the video file (seconds), returns detection results for the frame at that point. Default is 0.
+- `realtime`: (Optional) Boolean. If `true` or if no `file`/`video` parameters are provided, returns detection results for the current camera frame.
+- `conf`: (Optional) Confidence threshold for a single request, range 0.0-1.0.
+- `iou`: (Optional) NMS IOU threshold for a single request, range 0.0-1.0.
+
+#### Usage Examples:
+
+**1. Image Detection:**
+```bash
+curl -X POST "http://127.0.0.1:8000/api/models/yolov8/predict" -F "file=@/home/cat/001.jpg"
+```
+
+**2. Video Specific Frame Detection:**
+```bash
+curl -X POST "http://127.0.0.1:8000/api/models/yolov8/predict" -F "video=@/home/cat/test.mp4" -F "timestamp=5.5"
+```
+
+**3. Get Current Camera Frame Detection:**
+```bash
+curl -X POST "http://127.0.0.1:8000/api/models/yolov8/predict" -F "realtime=true"
+# Or without file parameters
+curl -X POST "http://127.0.0.1:8000/api/models/yolov8/predict"
+```
+
+#### Response Format (JSON):
+```json
+{
+  "success": true,
+  "source": "video frame at 5.5s",
+  "predictions": [
+    {
+      "class": "pedestrian",
+      "confidence": 0.92,
+      "box": { "x1": 100, "y1": 200, "x2": 300, "y2": 500 }
+    }
+  ],
+  "image": { "width": 1280, "height": 720 }
+}
+```
+
+### 2. System Configuration Interface (Config)
+
+Used to dynamically adjust thresholds for real-time video streams and default inference.
+
+#### Get Current Configuration
+- **Endpoint:** `GET /api/config`
+- **Response:** `{"obj_thresh": 0.25, "nms_thresh": 0.45}`
+
+#### Update System Configuration
+- **Endpoint:** `POST /api/config`
+- **Request Body (JSON):** `{"obj_thresh": 0.3, "nms_thresh": 0.5}`
+- **Response:** `{"status": "success"}`
+
+### 3. Real-time Video Stream Interface (Video Feed)
+
+Get real-time MJPEG video stream with detection boxes drawn, can be directly embedded in HTML `<img>` tags.
+
+- **Endpoint:** `GET /api/video_feed`
+- **Example Usage:** `<img src="http://<Board_IP>:8000/api/video_feed">`
+
+---
+
+## 🛠️ Developer Guide (Production Recommendations)
+### Code Description
+- `web_detection.py`:
+    - **Dual-mode Support**: Integrates FastAPI, supporting both local rendering and MJPEG streaming output.
+    - **Environment Adaptive**: Automatically detects the `DISPLAY` environment variable, silently skipping GUI initialization if not present.
+    - **RKNN Inference**: Encapsulates RKNN initialization, model loading, and multi-core inference logic.
+    - **Dynamic Loading**: Supports dynamic class configuration loading via `--class_path`.
+    - **Post-processing**: YOLOv8 specific Box decoding (DFL) and NMS logic.
+
+### Modifying Models
+1. Place the trained and converted .rknn model into the `model/` directory.
+2. Add the `--model_path` argument to the running command to point to the new model.