litellm patches

Local modifications to the LiteLLM proxy image (ghcr.io/berriai/litellm-non_root:main-stable), applied by mounting files into the container instead of maintaining a fork.

Compatibility: litellm v1.100.0. Every patch below is a diff against the stock v1.100.0 tag and was verified to apply cleanly to it. Per-item notes state the version each item was originally written and tested against; after an image upgrade, re-check each patch against the new stock source.

What's here

Two kinds of modification:

• patches/ — diffs against litellm source modules, one numbered directory per patch. Each directory holds the .diff plus a README describing the problem, the root cause, and how to apply it.
• callbacks/ — request-hook files loaded via litellm_settings.callbacks, one numbered directory per callback. These are complete, self-contained files you mount as-is; no litellm source is modified.

patches/

Patch	File modified	Base
001-alias_token_count	litellm/router.py	v1.97.0, re-verified on v1.100.0
002-anthropic_vllm_passthrough_params	litellm/types/llms/anthropic.py	v1.97.0, re-verified on v1.100.0
003-clamp_max_tokens_pre_call_skip	litellm/router.py	v1.100.0

The three patches are independent. Each applies cleanly to pristine stock v1.100.0 on its own, and applying all three in any order produces identical output. 001 and 003 both touch litellm/router.py, but in disjoint regions (get_configured_token_limits vs the pre-call-check helpers), so they do not conflict. Apply the subset you need.

To build a patched tree:

git clone --depth 1 --branch v1.100.0 https://github.com/BerriAI/litellm
cd litellm
git apply /path/to/patches/001-alias_token_count/alias_token_count.diff
git apply /path/to/patches/002-anthropic_vllm_passthrough_params/anthropic_vllm_passthrough_params.diff
git apply /path/to/patches/003-clamp_max_tokens_pre_call_skip/clamp_max_tokens_pre_call_skip.diff

patch -p1 works equivalently. Then mount the resulting files read-only over their module paths (see below).

callbacks/

Callback	Problem it fixes	Base
001_zai_thinking_fix	thinking / reasoning_effort dropped for Z.AI models	v1.97.0
002_team_prompt_params	per-team system prompt injection + parameter locking	v1.98.0
003_usage_details_patch	cached_tokens lost on live streaming usage	v1.98.x, verified v1.100.0
004_cache_hit_usage_details	usage details lost on cache-hit stream replay	v1.100.0

003 and 004 are complementary: 003 covers the live streaming path (usage arriving from the provider), 004 the cache-hit replay path (usage rebuilt from the cached dict). Both are needed for full cache information on streaming responses.

Mounting

Site-packages path depends on the image's Python version; adjust as needed.

volumes:
  - ./litellm/router.py:/app/.venv/lib/python3.13/site-packages/litellm/router.py:ro
  - ./litellm/types/llms/anthropic.py:/app/.venv/lib/python3.13/site-packages/litellm/types/llms/anthropic.py:ro
  - ./callbacks/:/app/callbacks/:ro

Mounted files take effect on container start — restart the proxy after changing them. Mounts shadow the image's files entirely: after pulling a new image, re-check each patch still applies, or drop the mount if upstream fixed the issue.

Deploying callbacks

1. Copy the hook file(s) into a directory mounted at /app/callbacks/ (read-only), keeping flat names.

2. Register each hook in the litellm config. Entries are callbacks.<module>.<instance> strings, resolved relative to the config file's directory:

   litellm_settings:
     callbacks:
       - "callbacks.zai_thinking_hook.zai_thinking_hook_instance"
       - "callbacks.team_prompt_params_hook.team_prompt_params_hook_instance"
       - "callbacks.usage_details_patch.usage_details_patch"
       - "callbacks.cache_hit_details_patch.cache_hit_details_patch"
   

   No __init__.py is needed; each hook file must be self-contained (litellm + stdlib imports only, no cross-imports between hook files). Callbacks load at startup — restart to activate.

Notes

• supported_openai_params shown by /v1/model/info comes from litellm's bundled catalog (advisory). Request-parameter forwarding is governed by allowed_openai_params on each deployment.
• Streaming usage with cached_tokens requires clients to send stream_options: {"include_usage": true}.
• debug/ holds diagnostic tooling for the proxy (on-demand asyncio task stack dumper, streaming-usage repro script). Not loaded by default.

Disclaimer

These patches were written by local LLMs with only overall guidance on how, where and what to patch. Use at your own risk.