Hi, I'm
Sourav Halder
System Metrics & Operations
Telemetric ledger showing automated sync operations, pipeline efficiencies, and service level metrics.
0+
ETL Pipelines
Active production pipelines extracting SAP ERP, SQL Server, and REST API endpoints into a unified Redshift warehouse.
0.0%
Operational SLA
Availability verified via automated event-driven checks, retry logic, and event alert routing.
0.00%
Traffic Reduction
Migrating from legacy batch databases to a delta-load layer using Apache Iceberg, avoiding SAP database bottlenecks.
-0%
MTTR Improvement
Automated integration linking pipeline alerts to ServiceNow API, replacing manual triaging and routing.
Expertise & Services
Bridging the gap between raw data systems, machine learning models, and secure cloud orchestration.
Cloud Architecture
Designing and implementing highly scalable, resilient, and cost-effective cloud solutions. Expert in infrastructure automation, serverless computing, and secure networking layouts.
- Infrastructure as Code
- High Availability Clustering
- Serverless Design Patterns
- Cost Optimization Analytics
Data Engineering
Building production-grade ETL/ELT pipelines and delta lakehouses. Specializing in delta-load mechanics, data warehousing, and automated operational observability.
- Incremental Delta Loads
- Medallion Lakehouse Architectures
- Orchestration & Retry Logic
- Incident Telemetry Pipelines
Data Science
Extracting actionable insights from complex datasets using advanced mathematics, statistical validation, and intelligent machine learning pipeline orchestration.
- Predictive Modeling
- LLM & Agent Architectures
- Feature Engineering
- Neural Network Design
Professional Experience
Detailed operational domains and systems implemented during my consulting career.
Senior Associate Consultant
Infosys LimitedServing as a Senior Associate Consultant at Infosys Limited, specializing in enterprise data engineering pipeline architecture, serverless cloud orchestration, data lakehouse migrations, and automated systems telemetry.
Key Contributions & Achievements
- Developed and maintained 50+ production ETL pipelines using Python, SQL, and AWS Glue, orchestrating data extraction from multi source SAP ERP, SQL Server, and REST APIs into Redshift data warehouse supporting 14 data marts, ensuring timely data availability for business dashboards and analytics teams.
- Implemented robust pipeline orchestration with event-driven and schedule-based triggers (Lambda, EventBridge, S3), incorporating automated retry logic, data validation checks, eliminated synchronization failures by 85%, resolved data mismatches, and achieved 99.5% pipeline reliability.
- Optimized data delivery cycles to support monthly Sales & Operations Planning (S&OP) processes for leadership and regional sales heads, cutting data delays by 60% and ensuring accurate, timely insights for strategic sales planning and forecasting.
- Re-architected data extraction pipeline by migrating from direct SQL database to Apache Iceberg-based live layer with incremental delta loads, achieving 93.75% reduction in data extraction and eliminating performance bottlenecks caused by high-volume ERP replication traffic.
- Automated incident management by integrating ServiceNow API for pipeline monitoring, enabling automatic ticket creation and intelligent team assignment for job failures—decreased mean time to resolution (MTTR) by 40% and eliminated manual triaging overhead.
Education Records
Academic qualifications, specialized engineering domains, and formal credentials.
Technical Skills
Organized the way I think about engineering and automation — directly inside a simulated code editor.
Featured Projects
Architecture-first implementations focused on cloud automation, data pipelines, and machine learning structures.
Delta Lakehouse Pipeline
SAP ERP → Glue → Iceberg → Redshift
Optimized a mission-critical financial ledger ETL pipeline by introducing partition-aware incremental delta-load extraction, slashing data processing workloads dramatically.
ML-Powered Anomaly Detection
Real-time anomaly pipeline on streaming data
Designed and deployed a streaming telemetry ingest engine that processes high-frequency IoT readings to detect operational anomalies in real-time.
Cloud Infrastructure Automation
Terraform + ECS + CI/CD orchestration
Standardized cloud operations across production clusters using declarative infrastructure definitions and secure automated deployment gates.
AWS Three-Tier Web Architecture
Auto Scaled Load Balanced Infrastructure
Utilized a public Application Load Balancer to route client traffic to auto-scaling Nginx web servers, communicating with internal application servers and RDS databases.
Spotify Big Data Analytics Pipeline
Serverless Ingest & Aggregation ETL
Built a serverless data pipeline to extract, transform, and load Spotify top songs data into a visual analysis dashboard using AWS serverless analytics services.
DynamoDB Real-Time Audit Table
NoSQL Database Mutation Capturing
Designed an automated audit trail system for DynamoDB databases, tracking data mutations in real-time for compliance and operational analytics.
WildRydes Serverless App
Complete Serverless App Ingestion
Built an end-to-end serverless ride-sharing web application leveraging AWS serverless services to manage user authentication, ride requests, and backend processing.
Medallion Data Warehouse Schema
Layered Relational Database Design
A comprehensive data warehousing solution, cataloging and exploring relational datasets using SQL scripts to structure Bronze, Silver, and Gold layers.
Certifications & Recognition
Professional cloud architecture, data engineering, and governance credentials. Click any badge to verify.
Get In Touch
Let's collaborate on data pipelines, cloud architecture, or intelligent software automation.