Support Kimi K3 Tokenizer

[yijingl-nvidia]

What does this PR do?

Changed the code's check on tokenizer to accept Kimi K3 tokenizer.

Kimi K3 tokenizer does not inherit from PreTrainedTokenizerFast but it is implemented in Rust using tiktoken.core.Encoding. So it should be accepted by Endpoints to run in the benchmark.

Type of change

• Bug fix
• New feature
• Documentation update
• Refactor/cleanup

Testing

• Tests added/updated
• All tests pass locally
• Manual testing completed

Checklist

• Code follows project style
• Pre-commit hooks pass
• Documentation updated (if needed)

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[yijingl-nvidia]

recheck

[codecov-commenter]

Codecov Report

❌ Patch coverage is 88.63636% with 5 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@111cf76). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...utils/services/metrics_aggregator/token_metrics.py	87.80%	5 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #435   +/-   ##
=======================================
  Coverage        ?   81.71%           
=======================================
  Files           ?      146           
  Lines           ?    19373           
  Branches        ?        0           
=======================================
  Hits            ?    15831           
  Misses          ?     3542           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.