refactor(ops): migrate paged caching to canonical InfiniOps API - #1467
Closed
voltjia wants to merge 1 commit into
Closed
refactor(ops): migrate paged caching to canonical InfiniOps API#1467voltjia wants to merge 1 commit into
voltjia wants to merge 1 commit into
Conversation
This was referenced Aug 6, 2026
Collaborator
Author
|
Superseded by #1480, which consolidates the canonical InfiniOps adapter migrations while preserving each logical change as a separate commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PagedCachingInfinilmdependency in the InfiniOps-backed paged-caching adapter with the canonicalReshapeAndCacheFlashAPI added by InfiniOps Issue/867: adjust paged_attention_prefill interface naming #883.paged_caching(k_cache, v_cache, key, value, slot_mapping)public API.kv_cache_dtype = autoand one persistent scalar scale tensor, which the currentautopath ignores.API Migration
paged_caching(k_cache, v_cache, key, value, slot_mapping)PagedCachingInfinilm::Call(key, value, slot_mapping, key_cache, value_cache)ReshapeAndCacheFlash::Call(key, value, slot_mapping, k_scale, v_scale, kv_cache_dtype, key_cache, value_cache)reshape_and_cache_flash, adapted to InfiniOps input/attribute/output groupingDependency: InfiniOps #883, merged as
e733e325.Validation
Remote environment:
ssh nvidia, imageaccelerator-dev/nvidia:latest.clang-format --dry-run --Werror src/infinicore/ops/paged_caching/paged_caching_infiniops.cc: passed.git diff --check origin/main...HEAD: passed.infinicore_cpp_apiand_infinicorebuild/link/install: passed.ldd:libinfinicore_cpp_api.soresolveslibinfiniops.sofrompython/infinicore/lib.test/infinicore/ops/paged_caching.py --nvidia, restricted to its eight equal-head-size configurations: 24/24 passed across FP16, BF16, FP32, HND, and NHD physical cache layouts.The full integration build used an InfiniOps validation tree containing open PRs #887-#891 because current InfiniCore adapters also reference the providers covered by those PRs. This change itself pins only merged InfiniOps #883.
Known Limitation
The final DeepSeek MLA case in the existing test uses key/value head sizes 576/512. It runs after all 24 standard cases and is rejected by the canonical API's equal-shape constraint. This is not introduced by the migration: deprecated
PagedCachingInfinilmalso asserts equal key/value and cache shapes. Extending that behavior belongs in a separate operator/API change.