Skip to content

refactor(inference): move error text matchers, emoji, voice MIME, reply-speech parsing and external STT/TTS clients to tinyinference (wave 4) - #6840

Merged
senamakel merged 330 commits into
tinyhumansai:mainfrom
senamakel:vendor-extract-w4-inference
Oct 1, 2026
Merged

senamakel merged 330 commits into
tinyhumansai:mainfrom
senamakel:vendor-extract-w4-inference

Conversation

@senamakel

@senamakel senamakel commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

This is the last cluster of vendor-extraction wave 4. It moves web-chat error text matchers, emoji extraction, voice MIME mapping, reply-speech parsing and the external speech-to-text (STT) and text-to-speech (TTS) HTTP clients into tinyinference.

crates/openhuman-core/src: 594,536 → 593,405 lines (−1,131), measured on top of #6838.

Stacked on #6838, which is itself stacked on #6829. Merge those first.

Depends on:

What moved (tinyinference)

  • tinyinference-llm failure::text: the pure predicates from web_chat/web_errors/{response_predicates,retry,provider_detail}.rs.
    • This covers extract_provider_name, extract_provider_error_detail, with_provider_detail and is_fallback_chain_exhausted.
    • parse_retry_after_secs is added upstream as its own function, next to parse_retry_after_ms. It keeps the host's semantics: camelCase keys, quote stripping and rounding up.
    • User-facing copy stays in the host: malformed_history_user_message, retry_after_hint and the error classification table.
  • tinyinference-llm classification::emoji: extract_first_emoji, is_emoji_start and is_regional_indicator, from inference reactions.
  • tinyinference-voice mime: mime_for_extension and extension_for_mime.
  • tinyinference-voice reply:
    • AlignmentFrame, ReplySpeech and the viseme and alignment parsers, with a literal-JSON test of the shape.
    • VisemeFrame now uses the vendor type.
    • synthesize_reply stays in the host.
  • tinyinference-voice: the ExternalSttClient and ExternalTtsClient classes (OpenAI-compatible, Deepgram and ElevenLabs).
    • The caller passes in the reqwest::Client, so proxy and TLS stay under host control.
    • SttApiStyle and TtsApiStyle move upstream behind an optional schemars feature, which the host enables. The published config schema and the TOML shape are unchanged.
  • Cron: is_local_provider_no_model_loaded_message now calls body_indicates_no_model_loaded instead of carrying its own copy.
  • Deleted: TranscriptionResult, which had no users.

API note: openhuman::config::schema::{SttApiStyle, TtsApiStyle} no longer exists. The runtime-boundary check forbids re-exporting tinyinference types, so callers import them from tinyinference_voice directly. openhuman-cli gains a dev-dependency for its config e2e test.

Error strings are byte-identical.

Verification

  • Vendor:
    • tinyinference-llm: 448 tests pass.
    • tinyinference-voice: 41 tests pass, with and without schemars.
    • The workspace passes fmt and clippy -D warnings.
  • OpenHuman:
    • Every test target builds with product features, as do the app manifest and embed/tinyhumans.
    • Full lib suite: 9,570 passed, 1 failed. The failure, credential_scrub::browser_confirmation_token_survives_without_exempting_page_credentials, is known on base.
  • CI scripts:
    • Feature forwarding, the gated-test allowlist and submodule monotonic pass.
    • rust:layout and the runtime-boundary check fail only on the entries that already fail on base.
    • No module pins moved.

Summary by CodeRabbit

  • Refactor
    • Agent memory guidance, run controls, progress tracking, voice processing, network access, and usage reporting now use shared implementations. Existing workflows are routed through updated integrations.
    • Network tools retain host-level security checks, including privacy and approval policies.
  • Documentation
    • Updated technical guides to reflect current tool responsibilities and integrations.

senamakel and others added 30 commits September 30, 2026 19:46
Add README files for the cron and tinyflows modules to provide documentation and usage guidance for developers working with these components.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The pin exemption for the tinymemory submodule is updated from v1.17.0-6-gea7dee88 to v1.18.0-17-g2ad4c128 to reflect that the pinned commit now sits on top of tinymemory main (v1.18.0) and includes the additional pure-library crate tinymemory-guard and conformance seeds from tinymemory#173, alongside the previously exempted conversations and scheduler_gate_decide additions. The registry's v1.17.0 artifact remains valid because no module, contract, or bus source changes are introduced.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the pinned commit for the tinyagents vendored dependency to incorporate upstream fixes and improvements.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…Windows API bumps

The lockfile is updated to reflect the addition of the `tinyconnectors` crate and its transitive dependencies, including `cookie_store`, `document-features`, `fs4`, `litrs`, and `tinyconnectors-sync`. Several existing dependencies now use `windows-sys 0.61.2` instead of earlier versions, and `ureq` gains support for cookie stores and serialization.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The baseline count of ignored tests is reduced from 26 to 24, reflecting the removal or resolution of two tests that are no longer being skipped.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The legacy line count for `crates/openhuman-core/src/tools/ops.rs` was reduced from 1341 to 1339, reflecting a decrease in the file's size. This change keeps the CI check aligned with the current state of the codebase.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the tinyflows submodule to incorporate upstream changes, ensuring compatibility with the latest caps module implementation.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…enhuman-core/src/flows/ops_migr

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The flows/mod.rs entry was removed from the allowlist because it is no longer a gated test that needs special handling, simplifying the allowlist to only track tests that require explicit opt-in.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…cies

The Cargo.lock file is updated to include two new crate dependencies, tinymemory-conformance and tinymemory-guard, which are now required by the openhuman-app crate.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…erence_providers`

Updated all call sites across six files to use the `tinyinference_providers` crate directly instead of routing through `crate::backend::classify` for the `is_budget_exhausted_message` and `is_provider_config_rejection_message` functions. This removes the internal re-export dependency and aligns the codebase with the new provider classification module.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The classify function now returns an empty result when given an empty input string, preventing a panic that occurred when trying to process zero-length content. This ensures the function behaves gracefully for edge cases where no text is provided.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Fixed a bug where the wait tool and ask_clarification tool were being executed in the wrong order during orchestration, causing the agent to attempt clarification before waiting for required input. The execution sequence now correctly processes the wait tool first, ensuring the agent only asks for clarification after the necessary waiting period has elapsed.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the tinymcp submodule to point to a newer commit, incorporating upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The tool dispatch logic was incorrectly using a numeric index to select which tool to run, causing mismatches when tools were reordered or removed. Changed the dispatch to match on the tool's name string, ensuring stable and predictable tool selection regardless of their position in the collection.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the pinned commit for the tinymcp vendored subproject to incorporate upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Added a README file to document the agent tools module, providing an overview of available tools and their usage for developers working with the openhuman-core agent.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the pinned commit for the tinymcp vendored dependency to include recent changes. The new commit hash reflects the current state of the subproject.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the tinymcp submodule to a newer commit that includes uncommitted local changes, keeping the dependency in sync with the current development state.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the pinned commit for the tinymcp vendored subproject to incorporate upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the tinymcp submodule to point to a newer commit, incorporating upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the pinned commit for the tinymcp vendored subproject to incorporate upstream fixes or improvements.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the pinned commit of the tinymemory vendored dependency to a newer revision, incorporating upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the pinned commit of the tinymcp submodule to include recent changes, keeping the dependency in sync with the upstream repository.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the pinned commit of the tinymemory vendored dependency to incorporate upstream fixes or improvements.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the vendored tinymcp subproject to a newer commit, incorporating upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the pinned commit of the tinybox vendor dependency to incorporate upstream changes.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The reaper was incorrectly processing journal entries by using the reversed order from the journal's internal storage, causing entries to be processed in the wrong sequence. Changed the reaper to iterate over journal entries in chronological order instead of reversed.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
senamakel and others added 21 commits October 1, 2026 06:08
…r to merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…r to merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…-tinytools-net

refactor(tools): move network tools to tinytools-std behind NetGate (wave 4)
…r module

Removed leftover merge conflict markers from the tool implementation README and the browser module file, keeping the post-merge version of the content. The conflict in the README was resolved by adopting the updated tool tables that reflect the current registration gates and tool assignments, while the browser module conflict was resolved by removing the `image_info` and `security` module declarations that were part of the conflicting branches.

Auto-committed-on: dragonfly
Updated the vendored tinyagents dependency to incorporate upstream fixes and improvements. No local changes were made to the vendored code.

Auto-committed-on: dragonfly
Replace the direct call to `FixedRecallProvider::guarded` with the `guarded_fixed_recall` helper function in the unscored lane test, aligning with the updated API. The vendor submodule for tinyagents is also advanced to a newer commit.

Auto-committed-on: dragonfly
…-memory-guard

refactor(memory): move guard decorator to tinymemory-guard behind GuardPolicy (wave 4)
Update the pinned commits for the tinyagents and tinymemory vendor submodules to their latest versions.

Auto-committed-on: dragonfly
Updated the tinymemory subproject to point at a newer commit, incorporating upstream changes.

Auto-committed-on: dragonfly
…-pin

fix(vendor): pin tinyagents at merged main, not a local-only commit
Update the tinyconnectors crate from version 0.10.4 to 0.11.0 in Cargo.lock, along with the corresponding vendor submodule commits for both tinyagents and tinyconnectors, to pull in the latest upstream changes.

Auto-committed-on: dragonfly
…-connectors

refactor(composio): replace host duplicates with the tinyconnectors library (wave 4)
…ged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…-flows

refactor(flows): move cron store and engine helpers to tinyflows (wave 4)
…ed module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ed module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…-misc

refactor(core): move shell classifier, module artifact selection, tinyjuice seams and text paste to vendor (wave 4)
… merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
@senamakel
senamakel dismissed coderabbitai[bot]’s stale review October 1, 2026 03:30

The merge-base changed after approval.

senamakel and others added 7 commits October 1, 2026 06:32
… merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…-mcp-hosts

refactor(core): move hosting tools to tinyhosts and MCP bridge tools to tinymcp (wave 4)
…to merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…-memory-rest

refactor(memory): move safety, memory tools, readers, scheduler gate and importers to tinymemory (wave 4)
…o merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
…-tinyagents

refactor(agent): move journal, trace export, stash, command center, turn-state mirror and harness tools to tinyagents (wave 4)
… merged module mains

Co-authored-by: Medulla <medulla@tinyhumans.ai>
@senamakel
senamakel merged commit afb41f7 into tinyhumansai:main Oct 1, 2026
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants