Python Data Engineer

Albiorix Technology


Date: 2 weeks ago
City: Ahmedabad, Gujarat
Contract type: Full time
Requirements

  • Strong proficiency in Python (pandas, PySpark, or similar frameworks).
  • Solid knowledge of SQL (query optimization, joins, indexing, performance tuning).
  • Experience with ETL frameworks/workflow orchestration (Airflow, or custom pipelines).
  • Hands-on with cloud platforms (AWS S3, Redshift, Glue, GCP BigQuery, Azure Data Factory).
  • Knowledge of data warehousing concepts and schema design (Star/Snowflake).
  • Familiarity with file formats such as JSON, Parquet, Avro.
  • Exposure to big data ecosystems (Hadoop, Spark, Hive, Kafka) is a plus.
  • Experience with containerization and orchestration (Docker, Kubernetes).
  • Familiarity with CI/CD practicesfor data pipelines (GitHub Actions, Jenkins, etc.).
  • Exposure to real-time/streaming platforms (Kafka, Kinesis, Pub/Sub).

Roles and Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL workflows using Python and SQL.
  • Work with structured, semi-structured, and unstructured data to ensure efficient ingestion, transformation, and storage.
  • Optimize data processes for performance, scalability, and cost-efficiency.
  • Manage and monitor data workflows to ensure accuracy, consistency, and reliability.
  • Implement data quality checks, validation, and error-handling mechanisms in pipelines.
  • Integrate data from multiple sources (databases, APIs, cloud storage, streaming platforms).
  • Work with cloud environments (AWS/Azure/GCP) for data storage, processing, and orchestration.
  • Maintain clear documentation of data workflows, schemas, and business logic.
  • Support and troubleshoot production issues in data pipelines and recommend improvements.

Nice to Have

  • Awareness of data security and governance (IAM roles, encryption).
  • Knowledge of monitoring/logging tools (CloudWatch, Grafana, ELK stack).
  • Awareness of data security and governance (IAM roles, encryption).

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Senior Manager Project

Johnson Controls, Ahmedabad, Gujarat
2 days ago
Build your best future with the Johnson Controls teamAs a global leader in smart, healthy and sustainable buildings, our mission is to reimagine the performance of buildings to serve people, places and the planet. Join a winning team that enables you to build your best future! Our teams are uniquely positioned to support a multitude of industries across the globe....

Kanerika Software - AEM Backend Developer - Java

Kanerika Inc, Ahmedabad, Gujarat
4 days ago
RoleWe are looking for an experienced AEM Backend Developer to join our digital platforms team.The ideal candidate will have strong backend development skills in Java (primary) and/or Python, along with proven experience in designing and building scalable, maintainable Adobe Experience Manager (AEM) solutions.This role will focus on implementing backend logic, custom workflows, integrations, and supporting content managementResponsibilities Design, develop, and...

Executive - Debt Management Services - Urban GCL

Bajaj Finserv, Ahmedabad, Gujarat
1 week ago
Location Name: AhmedabadJob Purpose“This position is open with Bajaj Finance ltd.”Duties And ResponsibilitiesХ To achieve collection targets of the Respective Branch assigned, ensuring meeting PI Targets.Х Meeting the Target on Cost of Collections. Ensuring meeting PI Targets within cost limit specified on monthly basis.Х Ensuring legal guidelines are complied for entire collection structure in letter and sprits.Х Ensure that the...