Lead Data Engineer - Big Data & PySpark
UST
Date: 6 days ago
City: Thiruvananthapuram, Kerala
Contract type: Full time
Experience: 5+ years relevant experience.
Key Responsibilities:
- Big Data Architecture: Design, develop, and maintain scalable and distributed data architectures capable of processing large volumes of data.
- Data Storage Solutions: Implement and optimize data storage solutions using technologies such as Hadoop, Spark, and PySpark.
- PySpark Development: Develop and implement efficient ETL processes using PySpark to extract, transform, and load large datasets.
- Performance Optimization: Optimize PySpark applications for better performance, scalability, and resource management.
Qualifications:
- Proven experience as a Big Data Engineer with a strong focus on PySpark.
- Deep understanding of Big Data processing frameworks and technologies.
- Strong proficiency in PySpark for developing and optimizing ETL processes and data transformations.
- Experience with distributed computing and parallel processing.
- Ability to collaborate in a fast-paced, innovative environment.
Skills
Pyspark,Big Data, Python
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
MATLAB-Senior LEAD Engineer
Quest Global,
Thiruvananthapuram, Kerala
5 hours ago
Job RequirementsThe primary responsibility for this role is analyzing the data recordings of the Automated Driving Systems, Infrastructure communications and Embedded platforms and recognizing the root cause of the bugs and failures in the system and potential solution for them.Job responsibility also requires analysis for CAN data recordings using PCAN tools, UDP data recordings using Wireshark Recordings, Embedded system recordings...
AI/ML Engineer-Automotive
Quest Global,
Thiruvananthapuram, Kerala
6 hours ago
Job RequirementsWe are looking for an experienced AI/ML Engineer to design, develop and deploy machine learning algorithms on embedded platforms for the next generation automotive infotainment systems. This engineer will work on Android / Linux based applications that will process various sensor data and external input at real-time to predict various vehicle conditions and adjust vehicle operations accordingly.Roles & ResponsibilitiesDesign,...
Unit Sales Manager
ICICI Lombard,
Thiruvananthapuram, Kerala
3 weeks ago
This job is provided by apna.co Job Responsibilities – Unit Sales Manager. (Field Sales)Recruit the agents and through them generate business by selling health products.Recruit, train agents and generate business through them in the assigned region.Responsible for licensing & tracking activation of the same regularly.Interdepartmental Coordination with other departments such as U/W, Claims, & Operations.
