Python-Pyspark Developer

Infosys


Date: 3 hours ago
City: Hyderabad, Telangana
Contract type: Full time
  • Primary skills:Python, Pyspark

We are looking for an experienced Python PySpark Developer to design, develop, and optimize large-scale data processing systems. The ideal candidate will work on big data platforms, build scalable ETL pipelines, and process high-volume datasets using Spark and Python. Key Responsibilities Data Engineering & Development Develop and maintain data pipelines using Python and PySpark Process and transform large datasets in distributed environments Build scalable ETL/ELT workflows Big Data Processing Work with Apache Spark (PySpark) for batch and real-time processing Optimize Spark jobs for performance and efficiency Handle structured and unstructured data Data Integration Ingest data from multiple sources: Databases (SQL/NoSQL) APIs Files (CSV, JSON, Parquet) Integrate with data platforms like: Hadoop (HDFS) Cloud (AWS, Azure, GCP) Performance Optimization Tune Spark jobs (partitioning, caching, parallelism) Optimize SQL queries and transformations Improve data processing efficiency and cost Collaboration & Support Work with data engineers, data scientists, and analysts Translate business requirements into technical solutions Participate in code reviews and agile development practices Monitoring & Troubleshooting Debug and resolve issues in data pipelines Monitor job execution and data quality Ensure reliability and availability of data workflows

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Motion Graphic Designer

InfoSwift Corp, Hyderabad, Telangana
3 hours ago
Tag/ShareShould have minimum 3yrs of experience in Motion Graphic designingGood experience in software tools like Adobe After effects, Premiere Pro, Photoshop, Canva etcGood exposure to 2D/3D Animation and Visual EffectsGood in communicationKey skills – Motion Graphics, Graphic designing, 2D/3D Animation, video Editing, Vector animation.Software tools – Adobe Photoshop, Adobe After Effects, Adobe Premiere Pro, Canva etc.

Consultant - Firm Tax - Indirect Tax (GST) - Hyderabad

Deloitte, Hyderabad, Telangana
3 days ago
Summary Position Summary Consultant, Indirect Tax - USI Firms Tax - Deloitte Consulting India Private Limited We are looking for professionals with sound working knowledge in Indian GST regulations including allied procedures.Work you'll doAs a part of this team, you will be expected to possess and demonstrate the following work skill sets: Assistance to the USI Firm Tax Leadership in...

Data Delivery lead/ Associate Director, Data and Analytics

HSBC Global Services Limited, Hyderabad, Telangana
4 days ago
Some careers shine brighter than others. If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of...