-
Notifications
You must be signed in to change notification settings - Fork 10
feat: tinyinference-image and tinyinference-video (OpenRouter media generation) #27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
4c5da84
chore(deps): add image-rs dependency to tinyinference-image
senamakel ac958c6
fix(error): handle missing image dimensions in error conversion
senamakel 101811c
fix(transport): handle missing content-length header
senamakel 43c6f75
fix(reference): handle empty image reference in inference
senamakel 747e97e
fix(media): handle missing image dimensions in media type detection
senamakel b5e14af
feat(capabilities): add support for image generation capabilities
senamakel a13be74
fix(image): add missing serde attributes to image types
senamakel 98ed713
feat(openrouter): add OpenRouter image inference support
senamakel 0b51f52
fix(image): restore image decoding support
senamakel f6b60c4
chore: add mock inference implementation
senamakel ef427a9
chore: files changed crates/tinyinference-image/src/openrouter_test.rs
senamakel 94c9b91
fix(reference_test): restore missing test assertions
senamakel effcb48
chore(deps): add tinyinference-image package to lockfile
senamakel a6db20f
chore: reformat long method chains and nested expressions for readabi…
senamakel 7da3e34
fix(docs): use mod@ disambiguator for intra-doc links to reference mo…
senamakel 236bdc0
chore(tinyinference-video): add missing dependency for video processing
senamakel 3157f44
fix(error): handle missing video file path in error conversion
senamakel d20143d
fix(types): correct video type field name from "type" to "video_type
senamakel 558f418
chore: add video inference crate
senamakel 9ede102
fix(openrouter): handle missing video URL in response
senamakel 3c67097
chore: add mock video inference implementation
senamakel ae51156
fix(test): update test to reflect new inference behavior
senamakel eb2e484
fix(test): add openrouter test file
senamakel c234cec
chore(deps): add tinyinference-video package to lockfile
senamakel 75e3a9c
chore: format code with rustfmt
senamakel 05adc16
fix(video): box large error variant to reduce enum size
senamakel 3dac06a
feat(examples): add live OpenRouter image example
senamakel 5968ea8
feat(video): add live OpenRouter video example
senamakel 5982c05
chore(examples): reformat long method chains and function calls
senamakel 5c5e6c9
docs: document new image and video generation crates
senamakel 0e3081f
feat(transport): unwrap proxied backend success envelope
senamakel 0ed6625
fix(transport): unwrap envelope error message from nested JSON
senamakel f6ac355
feat(examples): include creation timestamp in persisted image filenames
senamakel c126ef9
docs(env): document OPENROUTER_API_KEY for the live media examples
senamakel d634551
fix(transport): handle missing content-length header in image response
senamakel fab7ca2
fix(transport): handle missing content-length header in response
senamakel 915394c
fix(transport): handle missing content-type header in response
senamakel 38a3878
fix(transport): handle missing content-length header for chunked resp…
senamakel 4859408
fix(transport): handle missing content-length header in response
senamakel 027db27
feat(openrouter): add OpenRouter video inference support
senamakel e44b052
fix(tyinference-video): correct video inference output dimensions
senamakel 8e5725c
refactor(tinyinference-video): remove timeout logic from wait_for_job
senamakel a982154
fix(error): handle missing video codec in error conversion
senamakel f822f6d
fix(error): handle missing image dimensions in error conversion
senamakel ab7ed87
fix(media): handle missing image dimensions in metadata
senamakel 2cc0327
chore: files changed crates/tinyinference-image/src/openrouter.rs
senamakel fc5d889
fix(openrouter): handle missing image URL in response
senamakel 57a1fae
fix(openrouter): handle missing video URL in response
senamakel 1d8b436
fix(video): handle missing video file path in error conversion
senamakel ec0ca22
fix(tyinference-video): correct video inference output dimensions
senamakel f61d29d
fix(tyinference-video): correct video inference output dimensions
senamakel e6830a5
fix(tinyinference-video): reformat download_all call arguments
senamakel 7d080fa
fix(scope): correct video inference output dimension calculation
senamakel 42fdff9
fix(video): handle empty frame list in inference
senamakel 3cc3f92
chore(tinyinference-video): add missing dependency
senamakel 479a407
chore(tinyinference-video): remove unused test file
senamakel 482fe5f
fix(test): update test to match new inference behavior
senamakel 76db6c1
chore(deps): add tokio-test and tokio-stream dependencies
senamakel efe152d
fix(reference): handle missing image reference gracefully
senamakel b4c1793
fix(reference): handle missing image metadata gracefully
senamakel 5f7bcbf
fix(reference): handle empty image list in reference resolution
senamakel eba85b9
fix(reference): handle missing image dimensions in metadata
senamakel 8e65e09
feat(reference): add support for image reference resolution
senamakel 79dacfa
fix(reference): handle missing image metadata gracefully
senamakel f58c1d1
fix(reference): inline debug formatting for MediaReference
senamakel 5b238ea
fix(transport): handle missing content-length header in response
senamakel 49a1503
fix(transport): handle missing content-length header in image response
senamakel 4e93582
fix(scope): correct video inference output format
senamakel 050d921
feat(capabilities): add support for image generation capabilities
senamakel ac5b640
feat(types): add support for image dimensions in type definitions
senamakel ac5bee9
fix(types): correct image type validation for empty inputs
senamakel 09a73e0
fix(transport): handle connection reset errors during inference
senamakel fb1551c
fix(reference): handle missing image dimensions in metadata
senamakel 61531a5
fix(reference): handle missing image dimensions in metadata
senamakel 66c4f75
fix(scope): correct video inference output handling
senamakel 0c9e3fe
fix(scope): correct video inference output handling
senamakel 6393465
fix(scope): correct video inference output handling
senamakel 91f6680
fix(video): reformat fallback download call for readability
senamakel a8c7cb2
fix(media): paused-clock deadlines, bounded image JSON, redacted base…
senamakel d0d329c
fix(media): inline typed local refs, precheck data URL size, one boun…
senamakel 24c7058
feat(video): handle sparse output indices from OpenRouter
senamakel 6081aef
test(openrouter): add tests for omitted enum capability and sparse ou…
senamakel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| [package] | ||
| name = "tinyinference-image" | ||
| version.workspace = true | ||
| edition.workspace = true | ||
| rust-version.workspace = true | ||
| license.workspace = true | ||
| repository.workspace = true | ||
| description = "Provider-neutral image generation, media references, and OpenRouter transport for Rust." | ||
| documentation = "https://docs.rs/tinyinference-image" | ||
| readme = "../../README.md" | ||
| keywords = ["image-generation", "inference", "openrouter", "media"] | ||
| categories = ["api-bindings", "asynchronous", "multimedia::images"] | ||
|
|
||
| [dependencies] | ||
| async-trait = { workspace = true } | ||
| base64 = "0.23" | ||
| bytes = { workspace = true } | ||
| reqwest = { workspace = true } | ||
| serde = { workspace = true } | ||
| serde_json = { workspace = true } | ||
| thiserror = { workspace = true } | ||
| tinyinference-core = { version = "0.3.0", path = "../tinyinference-core" } | ||
| tokio = { workspace = true, features = ["fs"] } | ||
| tracing = { workspace = true } | ||
|
|
||
| [dev-dependencies] | ||
| axum = { workspace = true } | ||
| tempfile = { workspace = true } | ||
| tokio = { workspace = true, features = ["fs", "net"] } | ||
|
|
||
| [lints] | ||
| workspace = true | ||
95 changes: 95 additions & 0 deletions
95
crates/tinyinference-image/examples/live_openrouter_image.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| //! Live smoke test: generate an image through OpenRouter and save it. | ||
| //! | ||
| //! Network- and credential-gated. Reads `OPENROUTER_API_KEY` from the | ||
| //! environment, falling back to the workspace `.env` file; exits cleanly | ||
| //! (status 0, "skipped") when neither provides one. | ||
| //! | ||
| //! ```sh | ||
| //! cargo run -p tinyinference-image --example live_openrouter_image | ||
| //! # image-to-image: pass a reference image (path or URL) | ||
| //! LIVE_REFERENCE=path/to/ref.png cargo run -p tinyinference-image --example live_openrouter_image | ||
| //! # optional overrides | ||
| //! LIVE_IMAGE_MODEL=google/gemini-3.1-flash-lite-image LIVE_PROMPT="…" cargo run … | ||
| //! ``` | ||
| //! | ||
| //! Output lands in `target/live-media/`. | ||
|
|
||
| use std::path::PathBuf; | ||
| use std::time::Instant; | ||
|
|
||
| use tinyinference_image::{ | ||
| ImageGenerator, ImageRequest, MediaAuth, MediaReference, OpenRouterImageGenerator, | ||
| }; | ||
|
|
||
| fn api_key() -> Option<String> { | ||
| if let Ok(key) = std::env::var("OPENROUTER_API_KEY") | ||
| && !key.trim().is_empty() | ||
| { | ||
| return Some(key.trim().to_owned()); | ||
| } | ||
| let env_file = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../.env"); | ||
| std::fs::read_to_string(env_file) | ||
| .ok()? | ||
| .lines() | ||
| .find_map(|line| { | ||
| let value = line.trim().strip_prefix("OPENROUTER_API_KEY=")?; | ||
| let value = value.trim().trim_matches('"').trim_matches('\''); | ||
| (!value.is_empty()).then(|| value.to_owned()) | ||
| }) | ||
| } | ||
|
|
||
| #[tokio::main] | ||
| async fn main() -> Result<(), Box<dyn std::error::Error>> { | ||
| let Some(key) = api_key() else { | ||
| println!("skipped: OPENROUTER_API_KEY is not set (env or workspace .env)"); | ||
| return Ok(()); | ||
| }; | ||
| let generator = OpenRouterImageGenerator::new(MediaAuth::ApiKey(key)); | ||
| let model = | ||
| std::env::var("LIVE_IMAGE_MODEL").unwrap_or_else(|_| generator.default_model().to_owned()); | ||
| let prompt = std::env::var("LIVE_PROMPT").unwrap_or_else(|_| { | ||
| "A four-panel anime comic of two cheerful engineers shaking hands in front of a glowing \ | ||
| server, bright colors, thick ink outlines" | ||
| .to_owned() | ||
| }); | ||
|
|
||
| let mut request = ImageRequest::new(prompt) | ||
| .with_model(&model) | ||
| .with_aspect_ratio("landscape") | ||
| .with_seed(42); | ||
| if let Ok(reference) = std::env::var("LIVE_REFERENCE") { | ||
| println!("image-to-image with reference {reference}"); | ||
| request = request.with_reference(MediaReference::parse(&reference)); | ||
| } | ||
|
|
||
| let started = Instant::now(); | ||
| let response = generator.generate(request).await?; | ||
| let out_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../target/live-media"); | ||
| for (index, image) in response.images.iter().enumerate() { | ||
| let path = image | ||
| .persist( | ||
| &out_dir, | ||
| &format!( | ||
| "image-{}-{}-{index}", | ||
| model.replace('/', "_"), | ||
| response.created.unwrap_or_default() | ||
| ), | ||
| "png", | ||
| ) | ||
| .await?; | ||
| println!( | ||
| "saved {} ({} bytes, {})", | ||
| path.display(), | ||
| image.data.len(), | ||
| image.media_type | ||
| ); | ||
| } | ||
| println!( | ||
| "model={} images={} cost_usd={:?} elapsed={:.1}s", | ||
| response.model, | ||
| response.images.len(), | ||
| response.cost_usd, | ||
| started.elapsed().as_secs_f64() | ||
| ); | ||
| Ok(()) | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.