Data Engineer

About this job

Role Overview

We are looking for an experienced Data Engineer with 45 years of hands-on experience in building scalable data pipelines, managing large datasets, and designing efficient data architectures.Key Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and real-time processing.
• Build and optimize ETL/ELT workflows to integrate data from multiple sources.
• Develop and optimize complex SQL queries, database schemas, and data models.
• Work with analytical databases and data warehouses to support business intelligence and reporting.
• Implement data ingestion pipelines using APIs, databases, event streams, and files.
• Ensure data quality, consistency, reliability, and governance.
• Optimize pipeline performance, query execution, storage, and infrastructure costs.
• Collaborate with backend engineers, product teams, and business stakeholders.
• Implement monitoring, logging, error handling, and automated recovery for data pipelines.
• Maintain technical documentation and follow engineering best practices.

Required Skills

• Experience: 4–5 years in data engineering or a similar role.
• Programming: Strong proficiency in Python.
• Databases: Advanced SQL and hands-on experience with MySQL or PostgreSQL.
• Data Processing: Experience building ETL/ELT pipelines and data transformation workflows.
• Analytical Databases: Hands-on experience with ClickHouse.
• Cloud: Practical experience with AWS services such as S3, Glue, Lambda, Athena, or EMR.
• Big Data: Working knowledge of Apache Spark or PySpark.
• Streaming: Understanding of Apache Kafka or other event-streaming technologies.
• Orchestration: Experience with Apache Airflow or similar workflow orchestration tools.
• DevOps: Familiarity with Docker, Git, CI/CD, and monitoring tools.
• Strong understanding of data modelling, partitioning, indexing, and query optimization.

Preferred Qualifications

• Experience handling high-volume transactional and analytical datasets.
• Experience with CDC (Change Data Capture) and incremental data loading.
• Familiarity with lakehouse architecture and data formats such as Parquet.
• Experience building near-real-time analytics and reporting platforms.
• Knowledge of data security, access control, and data governance.

What We Expect

• Strong problem-solving and analytical skills.
• Ability to independently own data pipelines from design to production.
• Focus on reliability, scalability, performance, and maintainability.
• Ability to work effectively with cross-functional engineering teams.
• A proactive approach to identifying and resolving data quality and performance issues.

Success Metrics

• Reliability and freshness of data pipelines.
• Reduction in data processing time and infrastructure costs.
• Accuracy and consistency of analytical datasets.
• Performance and scalability of data infrastructure.
• Timely delivery of business-critical data requirements.