Skip to content

refactor(ops): migrate paged caching to canonical InfiniOps API - #1467

Closed
voltjia wants to merge 1 commit into
mainfrom
refactor/migrate-paged-caching-infiniops
Closed

refactor(ops): migrate paged caching to canonical InfiniOps API#1467
voltjia wants to merge 1 commit into
mainfrom
refactor/migrate-paged-caching-infiniops

Conversation

@voltjia

@voltjia voltjia commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Replace the deprecated PagedCachingInfinilm dependency in the InfiniOps-backed paged-caching adapter with the canonical ReshapeAndCacheFlash API added by InfiniOps Issue/867: adjust paged_attention_prefill interface naming #883.
  • Preserve the existing InfiniCore paged_caching(k_cache, v_cache, key, value, slot_mapping) public API.
  • Present the existing HND cache tensors to InfiniOps as metadata-only NHD views; cache storage is not copied.
  • Use kv_cache_dtype = auto and one persistent scalar scale tensor, which the current auto path ignores.

API Migration

Layer Before After Alignment
InfiniCore public API paged_caching(k_cache, v_cache, key, value, slot_mapping) Unchanged Existing InfiniCore contract
InfiniOps adapter PagedCachingInfinilm::Call(key, value, slot_mapping, key_cache, value_cache) ReshapeAndCacheFlash::Call(key, value, slot_mapping, k_scale, v_scale, kv_cache_dtype, key_cache, value_cache) vLLM reshape_and_cache_flash, adapted to InfiniOps input/attribute/output grouping

Dependency: InfiniOps #883, merged as e733e325.

Validation

Remote environment: ssh nvidia, image accelerator-dev/nvidia:latest.

  • clang-format --dry-run --Werror src/infinicore/ops/paged_caching/paged_caching_infiniops.cc: passed.
  • git diff --check origin/main...HEAD: passed.
  • Full infinicore_cpp_api and _infinicore build/link/install: passed.
  • Installed-library ldd: libinfinicore_cpp_api.so resolves libinfiniops.so from python/infinicore/lib.
  • Existing test/infinicore/ops/paged_caching.py --nvidia, restricted to its eight equal-head-size configurations: 24/24 passed across FP16, BF16, FP32, HND, and NHD physical cache layouts.

The full integration build used an InfiniOps validation tree containing open PRs #887-#891 because current InfiniCore adapters also reference the providers covered by those PRs. This change itself pins only merged InfiniOps #883.

Known Limitation

The final DeepSeek MLA case in the existing test uses key/value head sizes 576/512. It runs after all 24 standard cases and is rejected by the canonical API's equal-shape constraint. This is not introduced by the migration: deprecated PagedCachingInfinilm also asserts equal key/value and cache shapes. Extending that behavior belongs in a separate operator/API change.

@voltjia

voltjia commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #1480, which consolidates the canonical InfiniOps adapter migrations while preserving each logical change as a separate commit.

@voltjia voltjia closed this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant