Skip to content

About

This repository contains the complete 15-day hands-on engineering assignments completed during the Agentic AI Internship.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

93 Commits

Folders and files

Repository files navigation

Agentic AI: Production-Grade Autonomous Systems & Enterprise Capstone

Python 3.12 FastAPI Microservice CI/CD Pass Rate p50 Latency p95 Latency Durable State Security & HITL License: MIT

Author & Engineer: Harsh Ramesh Nerkar
Repository: harshrameshnerkar/Agentic-AI
Program: Agentic AI β€” Production Engineering Curriculum


πŸ“Œ Executive Summary

This repository contains the complete 20-day hands-on engineering assignments completed across the 20-Day Autonomous Agentic AI Engineering Curriculum.

Moving far beyond naive prompt wrappers and single-turn completions, this repository implements the full architectural spectrum of modern Autonomous Agent Engineering, Multi-Agent Swarms, Model Context Protocol (MCP), and LLMOps at Scale:

  • Deterministic ReAct execution loops & structured tool schemas
  • Multi-agent hierarchical supervisor patterns & worker swarms
  • Adversarial prompt injection defense & safety guardrails
  • Sub-millisecond intent routing & hybrid adaptive RAG
  • High-throughput asynchronous concurrency (asyncio.gather)
  • Durable transactional state machines with SQLite Write-Ahead Logging (WAL)
  • Human-in-the-Loop (HITL) approval gates with cryptographic SHA-256 audit chaining
  • Automated CI/CD regression gates with golden evaluation suites
  • Production containerization (Docker & docker-compose) and real-time observability

πŸ“š Complete 20-Day Topic-Wise Syllabus & Deliverables

Days Topic Name Technical Scope & Key Deliverables Code Link
1 LLM Fundamentals & API Basics β€’ Setup, first LLM API calls, tokenization mechanics
β€’ Temperature, top_p, frequency penalty parameters, context windows, cost models
day_01/
2 Advanced Prompting & Structured Output β€’ Chain-of-Thought (CoT), Task Decomposition, Few-Shot prompting
β€’ Pydantic schema validation, JSON mode parsing, prompt templates & versioning
day_02/
3 Embeddings, Vector DB & RAG β€’ Dense vector embeddings, cosine/dot similarity, chunking strategies
β€’ Vector database indexing, retrieval failure modes, semantic search
day_03/
4 Tool Calling & Function Execution β€’ OpenAI/Anthropic/Gemini function schemas, tool registration
β€’ Multi-step tool use, argument extraction, error handling loops
day_04/
6 Tool Calling & MCP β€’ Model Context Protocol (MCP) integration & tool design best practices
β€’ Blast-radius classification, safe read vs destructive write tiers, compensating rollbacks
day_06/
7 Agents: ReAct, Frameworks & Memory β€’ Pure Thought-Action-Observation ReAct execution loops
β€’ LangGraph StateGraph workflows, conversational & episodic checkpointed memory
day_07/
8 Multi-Agent Systems & Orchestration β€’ Centralized Supervisor router, collaborative worker swarms
β€’ Inter-agent communication protocols, context compaction & engineering
day_08/
9 Guardrails, Security & Agent Evaluation β€’ Prompt injection detection, jailbreak mitigation, Canary token leaks
β€’ Dual-LLM safety boundary guards, adversarial red-teaming test suite
day_09/
10 Capstone, Deployment & Final Review β€’ OpsSentinel AI Midterm Capstone: autonomous multi-tool SRE diagnostic copilot
β€’ Containerized deployment, interactive Streamlit UI, comprehensive review
day_10/
11 Advanced Retrieval Architectures β€’ Hybrid dense/sparse search (BM25 + vector), graph & structured metadata RAG
β€’ Query rewriting, sub-query decomposition, dynamic retrieval routing
day_11/
12 Fine-Tuning & Model Customisation β€’ Decision framework: RAG vs Fine-Tuning, SFT data synthesis & QC filters
β€’ Parameter-Efficient Fine-Tuning (PEFT/LoRA), multi-tier cost router
day_12/
13 Production Agent Architecture β€’ High-throughput async tool dispatch via asyncio.gather (3.2x speedup)
β€’ SQLite WAL ACID checkpointer, multi-tenant state isolation, circuit breakers
day_13/
14 LLMOps, Evaluation at Scale & Incidents β€’ Automated GitHub Actions CI regression gates with 100-case golden datasets
β€’ Canary/shadow trace evaluators, emergency kill switches & postmortems
day_14/
15 Production Capstone & Final Assessment β€’ Hardened autonomous SRE control center with sub-2ms intent router
β€’ FastAPI microservice, SHA-256 Merkle audit trail, HITL approval gate
β€’ 4-Page publication TDD, 10x cost projection, 20-min executive defense
day_15/
16 Scoping a Real Problem β€’ Session 1 (Discovery): Stakeholder problem statement in stakeholder's words, 7-stage workflow replacement analysis & Monte Carlo MTTR simulation
β€’ Session 2 (Constraints): Signed-off success metrics charter (95% accuracy, <=90s p95 latency, <=$0.15 cost ceiling), 3-tier blast radius classification, 5 Never-Automate red lines & cryptographic HMAC-SHA256 HITL gateway
β€’ Session 3 (Feasibility Spike): Data & documentation audit, 2-hour timeboxed retrieval spike (BM25 vs. Dense vs. Hybrid RRF), official Feasibility Note & 93% static retrieval ceiling analysis
β€’ Session 4 (Plan, Estimate & Eval Set First): Filled MoSCoW Sprint Planning Board (21 Must, 13 Should, 8 Could SPs), team capacity balancing (29h / 32h), pre-code 30-case golden evaluation dataset with 6 adversarial red-line scenarios
day_16/
17 Build Sprint 1 β€’ Session 1 (Standup & Riskiest Task): 15-minute standup (Done, Next, Blocked), Sprint 1 Kanban tracker moving TASK-1.1 to In Progress, concurrent async diagnostic tool dispatcher, 50k-line log anomaly compactor & circuit-breaker resilience
β€’ Session 2 (Simplest Thing That Works): Non-agent baseline evaluation (Plain Prompt 0% vs. Vanilla RAG 30%), golden 20-case evaluation dataset, empirical proof of ephemeral cluster gap
β€’ Session 3 (First Real Iteration): Scientific step-by-step evolution (+Retrieval 30%, +Cluster Tool 100%, +Conditional Routing 100% with 13.7% latency reduction)
β€’ Session 4 (Progress Update): Strict 5-line executive status update to mentor Dr. Rostova & Redis cluster state caching sign-off
day_17/
18 Build Sprint 2 & Code Review β€’ Session 1 (Standup & Replan): 15-minute standup & explicit scope cut (Slack bot moved from Must to Could), capacity rebalancing (13.5h load vs 16.0h cap), updated Sprint Board & Replan Manager
β€’ Session 2 (Integration & Safety): Universal REST/CLI delivery surface, regex PII scrubber (SSN, CC, email, JWT, API tokens), blast radius classifier, HMAC approval gateway & tamper-evident SHA-256 hash-chained audit logging
β€’ Session 3 (Code Review): Formal PR #18, line-by-line review addressing 5 comments without defensiveness (UsedNonceRegistry replay defense, ReDoS guards, Enum tiers, RLock thread-safety, @dataclass TriageResult)
β€’ Session 4 (Regression Suite): Automated one-command CI regression runner evaluating 35 golden cases (30 baseline + 5 regression bugs), 100.0% pass rate, sub-50ms latency, zero-tolerance non-zero exit gate
day_18/
19 Hardening, Documentation & Handover β€’ Session 1 (Standup & Freeze): 15-minute standup & formal feature freeze agreement (v1.0.0-rc1), zero-new-feature invariant, allowed/disallowed commit matrix & automated freeze auditor
β€’ Session 2 (Break Your Own System): 6-vector adversarial chaos test harness (prompt injections, malformed JSON schemas, empty RAG zero-matches, downstream socket crashes, upstream 429 outages, runaway recursive loops) & 8-column failure-mode matrix
β€’ Session 3 (Handover Pack): Complete 8-section production engineering manual (1-click setup, architecture, prompt catalog, on-call runbooks, unit economics model with $0.0028/query baseline, roadmap & walkthrough)
β€’ Session 4 (Clean-Clone Test): Independent peer review sandbox trial by Senior SRE Alex Chen with 0 author questions, resolution of 3 documentation bugs, and AST-verified headless non-interactive execution
day_19/
20 Demo, Retrospective & Final Assessment β€’ Session 1 (Stakeholder Demo): 20-minute executive presentation to VP of Infra & Lead SRE, leading with workflow impact ($42k/mo saved, 45ms MTTR), live incident walkthroughs & explicit refusal boundary disclosures
β€’ Session 2 (Results & Limitations): Formal 3-page technical results report evaluating stratified pass rates (100.0% on 35 cases), p95 latency (48.2ms), unit cost ($0.0028), scope cut defense & honest limitations
β€’ Session 3 (Retrospective): Engineering sprint retrospective analyzing estimation variance (+60% on retrieval spike, -100% on cut Slack bot) & deep-dives into 3 mistakes (HMAC replay nonces, ReDoS, and audit lock race condition)
β€’ Session 4 (Final Assessment & Conversion): 4-Week technical interview rubric (5.0/5.0 Exceeds Expectations), live multi-region Byzantine audit system-design defense, and unanimous Full-Time Conversion Approval
day_20/

πŸ† Final Capstone Spotlight: OpsSentinel AI Enterprise (Day 15)

The program culminates in OpsSentinel AI Enterprise (day_15/) β€” an autonomous Site Reliability Engineering (SRE) copilot designed to detect, triage, and remediate distributed cloud infrastructure incidents under enterprise SLAs and strict blast-radius controls.

                           +-------------------------------------------+
                           |         Inbound Operations Query          |
                           |   (Terminal CLI / REST API / Webhook)     |
                           +---------------------+---------------------+
                                                 |
                                                 v
                           +-------------------------------------------+
                           |        Sub-2ms Semantic Query Router      |
                           |  (Regex Pre-Filter + Tenant-Scoped SOPs)  |
                           +---------------------+---------------------+
                                                 |
             +--------------------+--------------+--------------+---------------------+
             |                    |                             |                     |
             v                    v                             v                     v
    [ INFO_SOP (RAG) ]   [ DIAGNOSTIC_READ ]          [ DESTRUCTIVE_WRITE ]   [ ADVERSARIAL_ATTACK ]
    Vector Runbook SOP   Parallel async tools         Tier 3 Blast Radius     Immediate Abort
    Direct response      (Metrics, Logs, Topology)    State: AWAITING_APV     Security Audit Log
             |                    |                             |                     |
             +--------------------+-----------------------------+---------------------+
                                  |
                                  v
        +---------------------------------------------------------------+
        |               Transactional Durable Checkpointer              |
        |              (SQLite WAL ACID - Crash-Safe State)             |
        +-------------------------------+-------------------------------+
                                        |
                   +--------------------+--------------------+
                   |                                         |
                   v                                         v
   +-------------------------------+         +-------------------------------+
   |   Human-In-The-Loop Gateway   |         | Cryptographic Audit Trail Log |
   |  (HMAC Token / Role Approval) |         | (SHA-256 Tamper-Evident Chain)|
   +---------------+---------------+         +-------------------------------+
                   |
                   v (Approved)
   +-------------------------------+
   | Executing Blast-Radius Action |
   | (Rollback Hook + Verification)|
   +-------------------------------+

Key Architectural Highlights

  1. Parallel Concurrency: 3.2x latency reduction over sequential tool dispatch using asyncio.gather for cluster telemetry, logs, and topology checks.
  2. Crash-Resilient State: 100% state recovery across mid-execution crashes with SQLite WAL checkpointing and idempotent replay.
  3. Cryptographic Blast-Radius Control: Any Tier-3 destructive operation (service restarts, scaling, cache flushes) is paused in AWAITING_APPROVAL. Authorized actions produce an immutable SHA-256 Merkle-style audit log.
  4. CI/CD Quality Gates: Fully wired into GitHub Actions (.github/workflows/day15_capstone_ci.yml) enforcing $\ge 90%$ pass rate, 0 critical regressions, and sub-250ms p95 latency.

πŸ“Š Empirical Benchmarks & Telemetry

Empirical test data measured on Python 3.12 running against the golden evaluation harness (12 stratified incident cases):

Metric Target SLA Measured Baseline Status
Overall Pass Rate $\ge 90.0%$ $100.0%$ PASS
Critical Regressions $0$ $0$ PASS
Adversarial Jailbreak Defense $100.0%$ $100.0%$ (Blocked) PASS
Sequential Diagnostics Latency β€” $482.4 \text{ ms}$ Baseline
Async Parallel Latency $&lt; 250 \text{ ms}$ $150.1 \text{ ms}$ (3.2x Speedup) PASS
p50 Latency (Overall) $&lt; 100 \text{ ms}$ $54.3 \text{ ms}$ PASS
p95 Latency (Overall) $&lt; 250 \text{ ms}$ $181.8 \text{ ms}$ PASS
Audit Chain Cryptographic Integrity 100% Intact SHA-256 Validated PASS

πŸš€ Quickstart Guide

1. Prerequisites & Virtual Environment

git clone https://github.com/harshrameshnerkar/Agentic-AI.git
cd Agentic-AI

# Create virtual environment
python -m venv .venv
# On Windows:
.venv\Scripts\activate
# On Linux/macOS:
source .venv/bin/activate

# Install dependencies
pip install -r day_15/requirements.txt

2. Run All Capstone Master Tests

python day_15/test_day_15_all.py

Expected: 16 integration tests passing in ~4 seconds with 100% evaluation pass rate.

3. Launch OpsSentinel REST Microservice

python day_15/session_1/api_server.py

Microservice starts at http://localhost:8000. Access Swagger UI docs at http://localhost:8000/docs.

4. Run CLI Operations

# Verify cryptographic audit chain integrity
python day_15/session_1/main.py --audit-check

# Run an infrastructure diagnostic query
python day_15/session_1/main.py -q "check cpu and error rate on auth-service"

# Trigger a destructive write requiring HITL approval
python day_15/session_1/main.py -q "restart pod auth-service-prod-0"

πŸ‘¨β€πŸ’» Author & Connect

Harsh Ramesh Nerkar
Autonomous AI Systems Engineer | Production LLMOps Architect

About

This repository contains the complete 15-day hands-on engineering assignments completed during the Agentic AI Internship.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages