vedanthv/data-engineering-portfolio

Hello World! I'm Vedanth.

This is a complete portfolio of the projects I have designed with a major focus on implementing various data engineering tech and cloud services.

Feel Free to Connect with me 🤠

LinkedIn | GitHub

Quick Links

Here is an index of projects with the tech, domain and link.

Real Time Streaming and Analytics

Project	Tech
Realtime Vehicle Fleet Monitoring Analysis	Redpanda,Apache Flink,Clickhouse,Metabase,Docker,Kubernetes
InvestIQ Metrics	AWS EC2,Docker,Airflow,ReapidAPI,AWS Lambda,AWS S3,AWS CloudWatch,AWS Redshift,Prometheus,Grafana,PowerBI

Databricks End to End Products

Project	Description	Tech
Deal2Delivery (Live Demo)	Unified SAP + Salesforce demand intelligence platform on a Databricks Lakehouse - combines churn prediction and demand forecasting ML models with a Genie AI/BI space for internal analytics and a Next.js app for business stakeholders. 🏆 2nd Place Winner, Databricks DAIS 2026 Virtual Hackathon	Databricks (Serverless), Delta Lake, Unity Catalog, Lakeflow DLT, Databricks Asset Bundles, XGBoost, Genie AI/BI, Next.js, Vercel
Dataflix (Live Demo)	Netflix-themed, Databricks-native AI content-strategy platform - real TMDB catalog data plus synthetic licensing/engagement/marketing data, four tuned Genie spaces, a document-AI pipeline for contract and script ingestion, and a Multi-Agent Supervisor that orchestrates it all behind a chat interface, casting recommender, and auto-generated compliance memos.	Unity Catalog, Delta Lake, Databricks Jobs, Genie Spaces, Agent Bricks Multi-Agent Supervisor, MCP, Lakebase, Next.js, Vercel

Small Data Engineering And Analytics Projects

Project	Tech
Pt 1 : Streaming ETL with Airflow Orchestrator and Postgres DB	Near Realtime Streaming,SQL Database, Kafka, Cricket!
Pt 2 : Serving Postgres Data using FastAPI	Backend API Development, Streaming Data
Pt 3 : Frontend Web App Powered by Streaming Data with Streamlit	Frontend Application Development,MVC Architecture
Pt 4 : Realtime Ingestion and Transformation with Apache Druid as OLAP Database	Data Ingestion and Big Data Processing
User Mingle : Kafka Driven User Profile Streaming	Airflow, Zookeeper, Cassandra, Schema Registry, Spark
Grand Prix Data Odyssey	Azure Databricks,Spark SQL,Postman,Blob Storage,Unity Catalog,ADF,Azure Devops,Synapse Studio, Delta Lake,PowerBI
Medal Metrics: Tokyo Olympics Data Alchemy	ADF,Azure Data Lake Gen2,Blob Storage,Databricks,Synapse Analytics,PowerBI
Airflow with Postgres as OLTP Database [Batch Processing]	Beautiful Soup, Apache Airflow, Postgres, Docker