Multimodal Generative Systems • Video Understanding • World Models • Conference Speaker
View architecture, source code and tests
Deployed a fine-tuned Stable Diffusion 1.5 image-generation model behind an authenticated FastAPI endpoint, with a separate Modal NVIDIA T4 GPU worker running PyTorch and Hugging Face Diffusers.
- Inference architecture: CPU API gateway validates requests and dispatches GPU generation; responses return PNG images and inference/VRAM measurements.
- Cost-aware serving: FP16 inference, persistent model-file cache, and GPU scale-to-zero when idle. Cold starts remain a latency trade-off.
- Reproducibility & quality: Pinned model/tokenizer commit revisions, dependency versions, GPU-free API tests, Ruff and GitHub Actions quality checks.
- Validation: Real image-generation smoke test completed on Modal separately from mocked CI tests.
| Initial measured GPU inference | End-to-end request | Peak GPU VRAM |
|---|---|---|
| 1.14–1.49 s | 18.22–32.64 s | 2.73–2.75 GiB |
Initial individual runs, not production p95/SLO metrics. Current next steps: request tracing, structured logs, load/latency and cost-per-image benchmarks.
Building next-generation generative AI systems that understand and create multimodal content from text to motion to photorealistic video.
|
🔹 Transformer-based architectures for text→video synthesis |
🔸 Distributed training pipelines (PyTorch DDP) |
┌─ Generative Video Systems │ ├─ 🧠 Transformer-based architectures │ ├─ 🎨 Diffusion model research │ ├─ ⏰ Temporal consistency modeling │ └─ 🎥 Text-to-motion-to-video synthesis │ ├─ World Models │ ├─ 🌍 Environment dynamics learning │ ├─ 🎮 Predictive modeling │ └─ 🔮 Temporal coherence │ └─ Multimodal AI ├─ 🔄 Cross-modal fusion ├─ 👁️ Vision-language models └─ 🎵 Audio-visual learning
┌─ Distributed Training │ ├─ ⚡ PyTorch DDP (8-GPU clusters) │ ├─ 📊 Large-scale dataset pipelines │ ├─ 🔬 Systematic experimentation │ └─ 📈 Mixed precision training │ ├─ Production ML │ ├─ 🚀 ONNX deployment │ ├─ ⚡ TensorRT optimization │ ├─ 📦 Model quantization │ └─ 🎯 Inference acceleration │ └─ Research Infrastructure ├─ 🔧 Experiment tracking ├─ 📉 Metric dashboards └─ 🛠️ Reproducible pipelines
| Domain | Technologies | Experience Level |
|---|---|---|
| 🧠 Generative Modeling | Diffusion • GANs • VAEs • Transformers • Temporal Consistency | ████████████ 95% |
| 🎬 Video & Motion | Pose Estimation • Temporal Modeling • Sequence Alignment • Motion Synthesis | ███████████░ 90% |
| 🔄 Multimodal Systems | Cross-modal Fusion • Vision-Language • Audio-Visual Learning | ██████████░░ 85% |
| ⚡ Distributed Training | PyTorch DDP • 8-GPU Clusters • Large-scale Pipelines • Experiment Tracking | ███████████░ 88% |
| 🚀 Model Deployment | ONNX • TensorRT • Quantization • Inference Optimization | ██████████░░ 82% |
Run | Keeping Models Reliable in Live Systems Tobacco Dock, London
Practical strategies for deploying and maintaining reliable AI systems in production, presented to delegates, AI leaders, researchers, and engineering leaders.
|
MSc Computer Vision, 📄 Dissertation: Cross-modal latent fusion for multimodal face generation |
2 Research Papers ✨ Generative AI Architectures 🔬 Computer Vision Systems |
AI Encode London Winner ⚡ Real-time AI prototype 🚀 Production-ready |
MSc Dissertation: Latent fusion for multimodal synthesis
Tech: VAEs • Cross-modal Fusion • PyTorch
Status: Published Research
A hands-on portfolio focused on model training, systems tradeoffs, reproducibility, and publishing deployable artifacts.
Repository: Deep-Learning-using-PyTorch
Case studies: ML Engineering Portfolio
Model artifacts: Hugging Face · pynk17
| Experiment | What I worked on | Notebook / Model |
|---|---|---|
| Custom DDPM / UNet | Built diffusion models from a small UNet baseline to a larger 256 × 256 UNet2D DDPM with attention, BF16, gradient accumulation and periodic sampling | Notebook · Model |
| Stable Diffusion systems comparison | Fine-tuned Stable Diffusion v1.5 across FP32, FP16, and FP16 + gradient checkpointing to compare precision and memory-oriented training configurations | Notebook · FP32 · FP16 · FP16 + GC |
| BLOOM-7B1 LoRA | Adapted a 7B-scale causal LM with PEFT / LoRA while training about 0.11% of total parameters, using FP16 and gradient checkpointing | Notebook · Adapter |
| PyTorch foundations | Worked through tensor fundamentals and deep learning building blocks used throughout the later experiments | Overview · Basics |
PyTorch · Diffusers · Transformers · PEFT / LoRA · Stable Diffusion · DDPM · Mixed Precision · Gradient Checkpointing · Hugging Face Hub · Safetensors · Experiment Documentation
I'm actively seeking opportunities to collaborate on cutting-edge research problems:
| Research Area | Specific Interests |
|---|---|
| 🌍 World Models | Environment dynamics • Predictive modeling • Spatial reasoning |
| 🎬 Video Generation | Temporal coherence • Motion synthesis • Controllable generation |
| 🔄 Multimodal AI | Cross-modal fusion • Unified representations • Audio-visual learning |
| ⏰ Temporal Reasoning | Long-range dependencies • Sequence modeling • Event prediction |
| 🎮 Interactive Systems | Real-time generation • Human-AI collaboration • Embodied AI |
📍 London, United Kingdom
🎓 MSc Computer Vision, Robotics & ML @ University of Surrey
🔬 Researching at the intersection of generative AI, video understanding, and world models
