Lead AWS Data Engineer

This role is for one of the Weekday's clients

Salary range: Rs 1500000 - Rs 2500000 (ie INR 15-25 LPA)

Experience: 7+ yrs

Location: Bengaluru, Pune

Job Type: full-time

We are looking for a highly experienced Senior Data Engineer to design, build, and optimize scalable cloud-based data platforms that power enterprise analytics, reporting, and AI-driven initiatives. This role is ideal for a hands-on data engineering professional who combines deep technical expertise with strong business understanding to create reliable, secure, and high-performance data solutions. You will be responsible for developing modern data architectures, building robust ETL/ELT frameworks, and leveraging cloud-native technologies to support large-scale data processing and analytics. Working closely with business stakeholders, data scientists, analysts, and engineering teams, you will transform complex business requirements into scalable technical solutions while driving best practices in data engineering, governance, performance optimization, and platform reliability.

Requirements

Key Responsibilities

• Design, develop, and maintain scalable data pipelines, ETL/ELT processes, and data integration frameworks.
• Build cloud-native data solutions using AWS services such as Glue, Lambda, Redshift, Aurora, OpenSearch, Step Functions, SNS, and S3.
• Develop and optimize large-scale data processing workflows using Python, PySpark, SQL, and PL/SQL.
• Design and implement enterprise data warehouses, data marts, dimensional models, and reporting structures.
• Collaborate with business stakeholders, product teams, analysts, and data scientists to deliver data-driven solutions.
• Ensure data quality, governance, security, compliance, and lifecycle management across the data ecosystem.
• Optimize database performance, query execution, and high-volume data processing workloads.
• Implement monitoring, logging, alerting, and troubleshooting mechanisms to maintain platform reliability.
• Participate in architecture reviews, cloud modernization initiatives, and technical design discussions.
• Support AI, machine learning, and advanced analytics use cases through scalable data infrastructure.
• Drive continuous improvement initiatives focused on performance, scalability, and operational excellence.
• Mentor junior engineers and promote engineering best practices, code quality standards, and knowledge sharing.
• Support deployment activities, production issue resolution, and ongoing platform enhancements.

What Makes You a Great Fit

• 8–10 years of experience in Data Engineering, Data Warehousing, and Cloud Data Platform development.
• Strong expertise in AWS cloud services, including Glue, Lambda, Redshift, Aurora, OpenSearch, Step Functions, SNS, and S3.
• Advanced proficiency in Python and PySpark for large-scale data processing and transformation.
• Strong command of SQL and PL/SQL development and optimization.
• Extensive experience building ETL/ELT pipelines, data ingestion frameworks, and integration solutions.
• Deep understanding of data warehousing concepts, dimensional modeling, and database architecture.
• Experience working with structured, semi-structured, and unstructured datasets.
• Strong knowledge of modern data architectures, including Data Lakes, Data Warehouses, and Lakehouse environments.
• Familiarity with workflow orchestration, event-driven architectures, and distributed data processing systems.
• Experience implementing data governance, quality frameworks, and security best practices.
• Strong analytical thinking, problem-solving skills, and attention to detail.
• Excellent communication and stakeholder management abilities.
• Experience supporting AI, Machine Learning, Data Science, or Advanced Analytics initiatives is highly desirable.
• Exposure to CI/CD pipelines, DevOps practices, Infrastructure as Code (IaC), and Agile delivery methodologies is a plus.
• AWS certifications or equivalent cloud credentials are highly valued.
• A proactive, ownership-driven mindset with a passion for building scalable and reliable data platforms.