Data Engineer
Virtusa
Date: 17 hours ago
City: Chennai, Tamil Nadu
Contract type: Full time
We are looking for a highly skilled Senior Data Engineer to join our team.
The ideal candidate will have hands-on experience working with large-scale data platforms and a strong background in Python, PySpark, AWS, and modern data warehousing tools such as Snowflake and DBT. Familiarity with NoSQL databases like MongoDB and real-time streaming platforms like Kafka is essential.
Key Responsibilities
Design, build, and maintain scalable data pipelines using PySpark and Python.
Work with AWS cloud services (S3, Lambda, Glue, EMR, Redshift) for data ingestion, processing, and storage.
Implement and maintain ETL workflows using DBT and orchestration tools (e.g., Airflow).
Design and manage data models in Snowflake, ensuring performance and reliability.
Work with SQL for querying and optimizing datasets across different databases.
Integrate and manage data from MongoDB, Kafka, and other streaming or NoSQL sources.
Collaborate with data scientists, analysts, and other engineers to support advanced analytics and ML initiatives.
Ensure data quality, lineage, and governance through best practices and tools.
Required Skills & Qualifications
Strong programming skills in Python and PySpark.
Hands-on experience with AWS data services.
Proficiency in SQL and experience with DBT for data transformation.
Experience with Snowflake for data warehousing.
Knowledge of MongoDB, Kafka, and data streaming concepts.
Good understanding of data architecture, modeling, and data governance.
Experience with CI/CD and DevOps practices in a data engineering environment is a plus.
Excellent problem-solving skills and the ability to work independently or as part of a team.
The ideal candidate will have hands-on experience working with large-scale data platforms and a strong background in Python, PySpark, AWS, and modern data warehousing tools such as Snowflake and DBT. Familiarity with NoSQL databases like MongoDB and real-time streaming platforms like Kafka is essential.
Key Responsibilities
Design, build, and maintain scalable data pipelines using PySpark and Python.
Work with AWS cloud services (S3, Lambda, Glue, EMR, Redshift) for data ingestion, processing, and storage.
Implement and maintain ETL workflows using DBT and orchestration tools (e.g., Airflow).
Design and manage data models in Snowflake, ensuring performance and reliability.
Work with SQL for querying and optimizing datasets across different databases.
Integrate and manage data from MongoDB, Kafka, and other streaming or NoSQL sources.
Collaborate with data scientists, analysts, and other engineers to support advanced analytics and ML initiatives.
Ensure data quality, lineage, and governance through best practices and tools.
Required Skills & Qualifications
Strong programming skills in Python and PySpark.
Hands-on experience with AWS data services.
Proficiency in SQL and experience with DBT for data transformation.
Experience with Snowflake for data warehousing.
Knowledge of MongoDB, Kafka, and data streaming concepts.
Good understanding of data architecture, modeling, and data governance.
Experience with CI/CD and DevOps practices in a data engineering environment is a plus.
Excellent problem-solving skills and the ability to work independently or as part of a team.
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
Manager Supplier Quality
Larsen & Toubro,
Chennai, Tamil Nadu
3 days ago
Job DescriptionPosition Title: Manager Supplier Quality Location: Riyadh, Saudi Arabia Experience: 5 years to 10 Years Qualification: B.E in ElectricalKey Roles & ResponsibilitiesSupplier QualityoReview of Manufacturing quality plans /testing procedures /Type test reports for various packages.oVisit suppliers for witnessing factory acceptance tests (Need basis)oFormalized and establish TPIA agency contracts for inspection to ensure timely inspections within optimum cost.oVisit suppliers for...
Senior Executive - Finance
Ingersoll Rand,
Chennai, Tamil Nadu
4 days ago
Ingersoll Rand is committed to achieving workforce diversity reflective of our communities. We are an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion,...
Manager, Risk Data Strategy
Standard Chartered India,
Chennai, Tamil Nadu
5 days ago
Job SummaryIts techno-functional data programming role in the area of Risk&CFCC Data Store, Analysis & AI-backed product delivery. Job scope includes design, development & maintenance of Compliance and Financial Crime Data Store & Reporting as well as Providing AI backed solutions. Functions include both new development as well as BAU support for legacy data products. Responsibilities include accurate & thorough...
