Lead Data Engineer - Big Data & PySpark
UST
Date: 3 weeks ago
City: Thiruvananthapuram, Kerala
Contract type: Full time
Experience: 5+ years relevant experience.
Key Responsibilities:
- Big Data Architecture: Design, develop, and maintain scalable and distributed data architectures capable of processing large volumes of data.
- Data Storage Solutions: Implement and optimize data storage solutions using technologies such as Hadoop, Spark, and PySpark.
- PySpark Development: Develop and implement efficient ETL processes using PySpark to extract, transform, and load large datasets.
- Performance Optimization: Optimize PySpark applications for better performance, scalability, and resource management.
Qualifications:
- Proven experience as a Big Data Engineer with a strong focus on PySpark.
- Deep understanding of Big Data processing frameworks and technologies.
- Strong proficiency in PySpark for developing and optimizing ETL processes and data transformations.
- Experience with distributed computing and parallel processing.
- Ability to collaborate in a fast-paced, innovative environment.
Skills
Pyspark,Big Data, Python
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
Site Reliability Engineer
UST,
Thiruvananthapuram, Kerala
5 days ago
We are seeking a highly skilled Site Reliability Engineer (SRE) with strong expertise in Google Cloud Platform (GCP) and CI/CD automation to lead cloud infrastructure initiatives. The ideal candidate will design and implement robust CI/CD pipelines, automate deployments, ensure platform reliability, and drive continuous improvement in cloud operations and DevOps practices.Key Responsibilities:Design, develop, and optimize end-to-end CI/CD pipelines using Jenkins,...
Test Automation Lead - Java, Selenium, Cucumber and Linux
Quest Global,
Thiruvananthapuram, Kerala
2 weeks ago
Job RequirementsAt Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible. Our people are driven by the desire to make the world a better place—to make a...
Sr Test Automation Engineer
Quest Global,
Thiruvananthapuram, Kerala
2 weeks ago
Work ExperienceRequired Skills:Excellent communication, presentation and interpersonal skills. 4 - 6 years of experience in software testing.Experience in Verification and Validation of engineering applications preferably in Healthcare domain.Participate in requirement analysis, design, coding and testing (manual as well as automation)Experience in tools such as Jenkins, Polarion, Git.Experience in Behavior driven development tool (Squish / Cucumber / SpecFlow / JBehave)Solid understanding...