Python ETL Developer

Job Description

Our client is looking for a skilled ETL Developer with strong expertise in Python, PySpark, Pandas, NumPy, and Apache Airflow.

Experience 5-9 years

Roles & Responsibilities:

• Design, develop, and maintain ETL pipelines using Python, NumPy, Pandas, PySpark, and Apache Airflow to process and transform large volumes of data efficiently.
• Optimize and scale ETL workflows to enhance data processing performance, ensuring minimal latency and maximum reliability.
• Work with large-scale data processing frameworks, integrating various data sources, APIs, and databases to streamline data ingestion and transformation.
• Implement data validation, error handling, and logging mechanisms to ensure the integrity and consistency of data.
• Collaborate with data engineers, analysts, and business stakeholders to understand data requirements and deliver high-quality ETL solutions.
• Monitor, troubleshoot, and debug ETL failures and performance issues, ensuring smooth data operations.
• Enhance and automate existing ETL processes, adopting best practices in data engineering and software development.
• Ensure compliance with data governance, security, and regulatory requirements while handling sensitive business data.