Data Engineer – Data Platforms (GCP)

About this job

Experience: 5 – 7 Years

Required Skills

Google Cloud Platform (GCP), Python, SQL, PySpark, BigQuery, Cloud Composer (Apache Airflow), Dataproc, Pub/Sub, Cloud Storage, ETL/ELT, Batch & Streaming Pipelines, Data Warehousing, Data Modeling, Git, CI/CD

Job Summary

We are seeking an experienced Senior Data Engineer with strong Google Cloud Platform (GCP) expertise to design, develop, and maintain scalable, cloud-native data solutions. The ideal candidate should have strong hands-on experience in Python, SQL, PySpark, and GCP data services, with expertise in building enterprise-grade ETL/ELT pipelines and batch and streaming data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark.
• Build and optimize cloud-native data solutions using Google Cloud Platform services.
• Develop and optimize SQL queries, data transformations, and data processing workflows in BigQuery.
• Design and manage workflow orchestration using Cloud Composer (Apache Airflow).
• Develop, optimize, and troubleshoot Spark applications running on Dataproc.
• Build event-driven data ingestion pipelines using Pub/Sub.
• Implement batch and real-time data processing pipelines for data warehouses and data lakes.
• Ensure data quality through validation, reconciliation, monitoring, and exception handling.
• Optimize pipeline performance, data processing, and storage efficiency.
• Implement data warehouse concepts, including star and snowflake schemas, fact and dimension modeling, SCD, partitioning, and clustering.
• Follow engineering best practices using Git/GitHub, CI/CD, and code reviews.
• Collaborate with Product Owners, Business SMEs, and Engineering teams to deliver reliable data products.
• Create and maintain technical documentation for data pipelines and architecture.

Mandatory Skills

• 5+ years of experience in Data Engineering.
• Strong hands-on experience in Python, SQL, and PySpark.
• Hands-on experience with Google Cloud Platform (GCP).
• Experience with BigQuery, Dataproc, Pub/Sub, and Cloud Composer (Apache Airflow).
• Experience developing production-grade ETL/ELT pipelines.
• Knowledge of batch and streaming data processing.
• Experience with data warehouse concepts and data modeling.
• Understanding of Git/GitHub and CI/CD practices.

Preferred Skills

• Google Dataflow and Apache Beam.
• Bigtable, Cloud Spanner, Cloud SQL, and AlloyDB.
• Google Cloud Storage and Cloud Scheduler.
• dbt, Apache Spark, Hadoop, and Spark/Scala.
• Docker, data governance, and data quality practices.
• Experience with enterprise-scale cloud migration initiatives.
• Familiarity with AI-assisted development tools such as GitHub Copilot.

Ideal Candidate: A Senior Data Engineer with strong GCP data platform expertise, capable of building and optimizing scalable batch and streaming pipelines, enterprise ETL/ELT solutions, and cloud-native data architectures.

Pay: ₹1,100,000.00 - ₹1,300,000.00 per year

Work Location: In person