Developed Data pipelines for Batch and Streaming Data ingestion using Airflow (Python) and Dataflow. Created custom Dataflow templates using Apache Beam library and Java.

Performed ETL operations in BigQuery and configured Scheduled Queries and External Transfers. Optimized SQL queries and Airflow DAG with parallel execution which lowered execution time by 35 percent for Data Pipelines.

Communicate effectively with vendors and Data customers to fine-tune Data Platform. Participated in Solutions design and Data modelling for new data sources and application enhancements.

Integrated with downstream applications for effective Campaigns, and solve data requirements for Mobile/Web applications. Performed Customer segmentation and Master Data Management that enhanced campaigns and helped increase returning customers by 18 percent.

Developed CI/CD pipelines using Jenkins for automated deployments and releases across environments.

Responsible for developing and maintaining on-premise Data Warehouse with Peta-bytes of data for Financial use-cases. Performed Security audit, Data Governance, and vulnerability closure of Data Warehouse and Analytical Data Mart.

Developed Batch Data ingestion pipeline using Sqoop and Hadoop Distributed File System (HDFS). Extensively used ETL to transfer and extract data from various sources (RDBMS, APIs, Flat files) to target Data Warehouse.

Created a distributed computing environment using Jupyter Hub and Spark for Data Science workloads with multi-user Active Directory authentication and authorization.

Implemented hybrid multi-cloud architecture and migrated applications to cloud using cloud native services.

Worked on Python and Django to create an in-house team work management application for tracking and planning all tasks efficiently.

Worked closely with Data Scientists to assist on feature engineering, model training frameworks, and model deployments at scale. Adopted Innovative architectural approaches to leverage in-house data integration capabilities consistent with architectural goals of the enterprise.

Performed Text Mining by Transformation. Cleansing, and term-document Matrix extraction using Python and Spark.

Used Statistical techniques or hypothesis testing to validate data and interpretations. Performed intuitive Data preparation using Rapidminer.

Partnered with Project managers and internal teams to address complex Business questions and provide insightful analysis and strategic recommendations through strong story telling capabilities to both technical and non-technical stakeholders.

Integrated backend database environments for production applications and developed stored procedures. Involved in Extract, Transform, Load, Model, and Reconcile of large volume of data across multiple system platforms and sources.