apache-spark-streaming

An event-driven System-of-Systems architecture for continuous, incremental knowledge distillation in compact LLMs using PostgreSQL CDC, Apache Kafka, Spark Streaming, and deterministic non-LLM evaluation.

Spark Streaming KPI Processing is a real-time data processing application built using Apache Spark Streaming

End-to-end batch + streaming earthquake data pipeline with visualization using Kafka, Spark (Structured Streaming), HDFS, PostgreSQL, MongoDB, Metabase/Kepler.gl and Docker.

Built a complete setup from streaming to storing and analyzing the data from sensors employed in different Victorian cities to predict and stop fire.

Analyze the real-time sales data related to invoices of orders of customers across the globe and computing various Key Performance Indicators (KPIs) for an e-commerce company, RetailCorp Inc. to analyze trends and current consumer behavioral patterns.