Skip to content

[BUG] DPDK VLAN-push flow is rejected by mlx5 under default HWS mode #305

Description

@dleshchev

Describe the bug

A raw DPDK configuration containing a TX vlan_push action cannot initialize on
ConnectX-7 under DAQIRI's default dv_flow_en=2 hardware-steering mode.

DAQIRI completes configuration parsing, memory registration, ethdev setup, and TX
queue creation. The mlx5 driver then rejects the VLAN-push action while creating the
TX flow. daqiri_init() returns failure and the benchmark cannot start.

Steps/Code to reproduce bug

Build a local container from main commit
57f3d8ebdd2e5410f654edea2e25b211cbca17de:

IMAGE_TAG=daqiri:vlan-repro \
BASE_TARGET=dpdk \
DAQIRI_ENGINE="dpdk ibverbs" \
scripts/build-container.sh

Start it with access to the GPU, NIC, and hugepages:

docker run --rm -it \
  --privileged \
  --gpus all \
  --network host \
  -v /dev/hugepages:/dev/hugepages \
  -v "$PWD:/workspace/daqiri" \
  -w /workspace/daqiri \
  daqiri:vlan-repro

Inside the container, save this as /tmp/vlan-dpdk.yaml:

%YAML 1.2
---
daqiri:
  cfg:
    version: 1
    stream_type: raw
    engine: dpdk
    master_core: 3
    debug: false
    log_level: info
    loopback: ''
    memory_regions:
    - name: Data_TX_GPU
      kind: device
      affinity: 0
      num_bufs: 16384
      buf_size: 1042
    - name: Data_RX_GPU
      kind: device
      affinity: 0
      num_bufs: 16384
      buf_size: 1042
    interfaces:
    - name: tx_port
      address: '0005:03:00.0'
      tx:
        queues:
        - name: tx_q_0
          id: 0
          batch_size: 1024
          cpu_core: 4
          memory_regions: [Data_TX_GPU]
          offloads: [tx_eth_src]
        flows:
        - name: vlan_encap
          id: 101
          actions:
          - type: vlan_push
            vlan_id: 100
            pcp: 0
            dei: 0
            ethertype: 33024
          match:
            udp_src: 48100
            udp_dst: 48100
    - name: rx_port
      address: '0005:03:00.1'
      rx:
        flow_isolation: true
        queues:
        - name: rx_q_0
          id: 0
          cpu_core: 5
          batch_size: 1024
          memory_regions: [Data_RX_GPU]
        flows:
        - name: vlan_decap
          id: 101
          match: {}
          actions:
          - type: vlan_pop
          - type: queue
            id: 0
bench_rx:
- interface_name: rx_port
  queue_id: 0
  cpu_core: 7
bench_tx:
- interface_name: tx_port
  queue_id: 0
  cpu_core: 6
  batch_size: 1024
  payload_size: 1000
  header_size: 42
  eth_dst_addr: 48:b0:2d:f4:04:24
  ip_src_addr: 1.1.1.1
  ip_dst_addr: 2.2.2.2
  udp_src_port: 48100
  udp_dst_port: 48100

Run:

/opt/daqiri/bin/daqiri_bench_raw_gpudirect \
  /tmp/vlan-dpdk.yaml --seconds 2

The DPDK arguments confirm that hardware steering is selected:

-a 0005:03:00.0,txq_inline_max=0,dv_flow_en=2
-a 0005:03:00.1,txq_inline_max=0,dv_flow_en=2

Flow creation then fails:

[INFO] ... Adding TX flow vlan_encap
mlx5_net: [mlx5dr_action_root_build_attr]: Found unsupported action type: 14
mlx5_net: [mlx5dr_bwc_rule_create_root]: BWC rule: failed creating rule on root tbl
[CRITICAL] ... Failed to create TX flow 'vlan_encap' on port 0:
                 fail to create rte flow (Invalid argument)
[CRITICAL] ... Failed to initialize DPDK
daqiri_init failed

Expected behavior

DAQIRI documents VLAN push/pop as supported by the raw DPDK engine, so the VLAN
flow should install under the engine's default steering configuration.

If mlx5 requires a different steering mode for this action, DAQIRI should either:

  1. select the compatible mode for interfaces containing VLAN actions, or
  2. reject the configuration before partial initialization with a clear message
    identifying the required mode.

Whether dv_flow_en=1 works on this hardware has not yet been tested.

Environment overview (please complete the following information)

  • Environment location: Docker on bare-metal IGX
  • Method of DAQIRI install: locally built Docker image
    • No docker pull command was used.
    • Build and run commands are included above.

Environment details

  • Host OS: Ubuntu 22.04.5 LTS
  • Host kernel: 5.15.0-1039-nvidia-tegra-igx, aarch64
  • Container base: CUDA 13.1 development image on Ubuntu 24.04
  • DAQIRI version: 2026.7.0
  • DAQIRI commit: 57f3d8ebdd2e5410f654edea2e25b211cbca17de
  • DPDK version: 25.11
  • Hardware: NVIDIA IGX with discrete RTX 6000 Ada GPU
  • NIC: NVIDIA ConnectX-7, ports 0005:03:00.0 and 0005:03:00.1
  • NIC firmware: 28.39.3004
  • mlx5 kernel driver: 24.07-0.6.1
  • MLNX_OFED/rdma-core: 24.07
  • NVIDIA GPU driver: 535.274.02

Additional context

VLAN/tunnel actions were introduced in PR #194 as the implementation for #196.
The recorded PR checks did not compile or exercise the VLAN flow on DPDK hardware.

The rejected action is emitted as RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN. Memory
registration and queue setup finish before this error, isolating the observed failure
to TX flow creation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions