Loading profile…
Loading profile…
Senior Data Engineer
Senior Data Engineer with 10+ Years of Experience | Data Architecture | Big Data | ETL | Big Data | Cloud | AI
United KingdomSenior Data Engineer at MicroDev Solutions
Led a team of 5 data engineers designing and maintaining enterprise-scale ETL/ELT pipelines using Databricks and cloud data infrastructure across AWS and Azure. Architected cloud data platforms processing 5+ TB of data daily and led modernization initiatives migrating legacy ETL solutions to Azure Data Factory, Databricks, Snowflake, and Delta Lake.
DatabricksAzure Data FactorySnowflakeDelta LakePythonPySparkSparkNiFiAirflowDBTInformaticaKafkaEvent HubTerraformDockerKubernetes
Bachelor of Science in Computer Science from FAST National University, completed in May 2016.
Bachelor of Science in Computer Science from FAST National University, completed in May 2016.
Designed and optimized ETL/ELT pipelines using Spark, PySpark, Airflow, and Kafka to process billions of data records. Migrated legacy Teradata and Oracle workloads to Snowflake, AWS S3, and Redshift, reducing costs by 40%, and built cloud-native data solutions using AWS Glue, EMR, S3, Redshift, and Snowflake.
SparkPySparkAirflowKafkaSnowflakeS3RedshiftAWS GlueEMRAthenaHadoopPythonSQLPower BIGitCI CDData QualityData LineageCDC
Modernized legacy ETL pipelines and data warehouse workflows, achieving a 30% improvement in processing speed. Implemented Python, Kafka, Elasticsearch, FluentD, and GCP (GKE) solutions, supporting 5+ TB daily data ingestion, and assisted in migration from on-prem Oracle systems to AWS Redshift.
PythonKafkaElasticsearchFluentdGCPGKECloud StorageDockerKubernetesSQLLinuxELK StackData WarehouseETLData Lineage
Profile sourced from Himalayas.