ShieldScan — AI Content Moderation Platform

A full-stack AI-powered content moderation platform. Users submit images for automated policy compliance screening; administrators oversee verdicts, manage appeals, configure moderation policies per category, and monitor platform-wide analytics.

Quick Start

# 1. Clone and enter the project
git clone https://github.com/sheix-khizar/content-mod-platform
cd content-mod-platform

# 2. Copy and fill in environment variables
cp .env.example .env
# Set ANTHROPIC_API_KEY for real AI moderation (optional — runs in mock mode without it)

# 3. Start everything
docker-compose up --build

The app will be available at http://localhost:3000.

Demo Credentials

Role	Email	Password
Admin	admin@contentmod.io	Admin1234!
User	user@contentmod.io	User1234!

These are seeded automatically on first run.

---

Environment Variables

Variable	Required	Default	Description
JWT_SECRET	Yes	supersecret-...	Token signing secret — change in production
ANTHROPIC_API_KEY	No	(empty)	Claude API key. If absent, moderation runs in mock mode with randomized scores
PORT	No	3000	Host port for the web UI
CORS_ORIGIN	No	*	Allowed CORS origin — restrict in production

---

Architecture

docker-compose
├── mongo          MongoDB 7 — primary data store
├── backend        Node.js / Express REST API (port 5000, internal)
└── frontend       React SPA built with Vite, served by Nginx (port 3000, public)
                   Nginx also proxies /api/* → backend:5000

Backend (/backend)

• Express REST API; single entry point at src/index.js
• Mongoose ODM with four primary models:
  • User — credentials, role (user | admin)
  • Policy — active moderation config; per-category toggle, threshold, enforcement mode
  • Submission — one or more images with embedded verdicts (outcome + per-category results)
  • Appeal — linked to a specific image inside a submission; pending → accepted/rejected
• Auth — JWT via Authorization: Bearer <token>; authenticate middleware + requireAdmin guard
• Moderation service (src/services/moderationService.js) — sends images to Claude's vision API with a structured prompt; falls back to mock randomized results when no API key is set
• Seeder (src/utils/seed.js) — creates default admin, test user, and initial policy on startup

Frontend (/frontend)

• React 18 + React Router v6 + Tailwind CSS
• Recharts for analytics charts
• react-dropzone for multi-image drag-and-drop upload
• Context-based auth (AuthContext) with token storage in localStorage
• Role-aware routing: admin-only pages redirect non-admins

Key Design Decisions

Embedded verdicts: Image verdicts are embedded inside the Submission document rather than stored as a separate collection. This keeps the full audit trail — including the per-category breakdown and which policy version was active — in a single document fetch, avoiding joins and making the history immutable.

Policy snapshots: Each verdict stores a reference to the Policy document at time of screening. Policy changes are never retroactive; the stored reference gives admins a full audit trail of which rules applied to each decision.

Mock mode: Without an Anthropic API key the moderation service returns randomized confidence scores, making the platform fully runnable and demonstrable without external dependencies.

Single active policy: The system supports one active policy document at a time (queried via isActive: true). A more advanced implementation could version policies and maintain history, but the current design satisfies the brief's requirement that changes only affect future submissions.

---

API Reference

Auth

Method	Path	Description
POST	/api/auth/register	Create account
POST	/api/auth/login	Sign in, returns JWT
GET	/api/auth/me	Current user

Submissions

Method	Path	Description
POST	/api/submissions	Submit images for moderation
GET	/api/submissions	List user's submissions
GET	/api/submissions/:id	Get single submission

Appeals

Method	Path	Description
POST	/api/appeals	File an appeal
GET	/api/appeals/mine	User's appeals
GET	/api/appeals/queue	Admin: appeals queue
PATCH	/api/appeals/:id/review	Admin: accept or reject

Policies

Method	Path	Description
GET	/api/policies	Get active policy
PUT	/api/policies	Replace entire policy
PATCH	/api/policies/category/:cat	Update a single category

Admin

Method	Path	Description
GET	/api/admin/analytics	Platform analytics
GET	/api/admin/submissions	All submissions
GET	/api/admin/users	All users
PATCH	/api/admin/submissions/:sid/images/:iid/override	Override verdict