Python Data Engineer

Albiorix Technology


Date: 5 hours ago
City: Ahmedabad, Gujarat
Contract type: Full time
Requirements

  • Strong proficiency in Python (pandas, PySpark, or similar frameworks).
  • Solid knowledge of SQL (query optimization, joins, indexing, performance tuning).
  • Experience with ETL frameworks/workflow orchestration (Airflow, or custom pipelines).
  • Hands-on with cloud platforms (AWS S3, Redshift, Glue, GCP BigQuery, Azure Data Factory).
  • Knowledge of data warehousing concepts and schema design (Star/Snowflake).
  • Familiarity with file formats such as JSON, Parquet, Avro.
  • Exposure to big data ecosystems (Hadoop, Spark, Hive, Kafka) is a plus.
  • Experience with containerization and orchestration (Docker, Kubernetes).
  • Familiarity with CI/CD practicesfor data pipelines (GitHub Actions, Jenkins, etc.).
  • Exposure to real-time/streaming platforms (Kafka, Kinesis, Pub/Sub).

Roles and Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL workflows using Python and SQL.
  • Work with structured, semi-structured, and unstructured data to ensure efficient ingestion, transformation, and storage.
  • Optimize data processes for performance, scalability, and cost-efficiency.
  • Manage and monitor data workflows to ensure accuracy, consistency, and reliability.
  • Implement data quality checks, validation, and error-handling mechanisms in pipelines.
  • Integrate data from multiple sources (databases, APIs, cloud storage, streaming platforms).
  • Work with cloud environments (AWS/Azure/GCP) for data storage, processing, and orchestration.
  • Maintain clear documentation of data workflows, schemas, and business logic.
  • Support and troubleshoot production issues in data pipelines and recommend improvements.

Nice to Have

  • Awareness of data security and governance (IAM roles, encryption).
  • Knowledge of monitoring/logging tools (CloudWatch, Grafana, ELK stack).
  • Awareness of data security and governance (IAM roles, encryption).

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Solvative - Frontend Lead - HTML/CSS/Javascript

Solvative, Ahmedabad, Gujarat
3 days ago
Position : Front End Lead.Headquarters : Dallas.Location : Remote.Company : Solvative.Experience Required : 6+ years.Job Type : Full-time.About UsSolvative is a dynamic and innovative company specializing in delivering top-notch digital solutions.We are looking for a talented web designer with a strong focus on HTML, CSS, and Core JavaScript to join our team remotely.The ideal candidate will be based in Ahmedabad...

Deputy Manager - PL - Prime - T1

Bajaj Finserv, Ahmedabad, Gujarat
1 week ago
Location Name: AhmedabadJob Purpose“This position is open with Bajaj Finance ltd.” &ltDuties And ResponsibilitiesХ Meeting & exceeding business goals/targets in different products ie. Loans & Insurance bundled/Non Bundled through Lead fulfillment & Field Activity on existing Data BaseХ Manage&ltRequired Qualifications And Experiencebr/&gt

Executive-QCC-INJ-IP-FP-STABILITY

Amneal Pharmaceuticals, Ahmedabad, Gujarat
1 week ago
Job DescriptionPerson having minimum experience of analysis and processing of In process and Finished product and handling of HPLC,UV,FTIR,K/F,AAS,IC,Dissolution (Sotax) and other instrumentsQualificationsQualification: Master's degree preferred (M.sc/ M.pharm / B.Pharm)Experience: 2-4 yearsPerson having minimum experience of analysis and processing of In process and Finished product and handling of HPLC,UV,FTIR,K/F,AAS,IC,Dissolution (Sotax) and other instruments