Data Engineer
HP
Date: 4 days ago
City: Bengaluru, Karnataka
Contract type: Full time
Job Summary
We are looking for a Senior Analytics Engineer to drive data excellence and innovation in our organization. As a thought leader in data engineering and analytics principles, you will be responsible for designing, building, and optimizing our data infrastructure while ensuring cost efficiency, security, and scalability. You will play a crucial role in managing Databricks and AWS usage, ensuring budget adherence, and taking proactive measures to optimize costs. This role also requires expertise in ETL processes, large-scale data processing, analytics, and data-driven decision-making, along with strong analytical and leadership skills.
Responsibilities
We are looking for a Senior Analytics Engineer to drive data excellence and innovation in our organization. As a thought leader in data engineering and analytics principles, you will be responsible for designing, building, and optimizing our data infrastructure while ensuring cost efficiency, security, and scalability. You will play a crucial role in managing Databricks and AWS usage, ensuring budget adherence, and taking proactive measures to optimize costs. This role also requires expertise in ETL processes, large-scale data processing, analytics, and data-driven decision-making, along with strong analytical and leadership skills.
Responsibilities
- Act as a thought leader in data engineering and analytics, driving best practices and standards.
- Oversee cost management of Databricks and AWS, ensuring resource usage stays within allocated budgets and taking corrective actions when necessary.
- Design, implement, and optimize ETL pipelines for incremental data loading, ensuring seamless data ingestion, transformation, and performance tuning.
- Lead migration activities, ensuring smooth transitions while maintaining data integrity and availability.
- Handle massive data loads efficiently, optimizing storage, compute usage, and query performance.
- Adhere to Git principles for version control, ensuring best practices for collaboration and deployment.
- Implement and manage DSR (Airflow) workflows to automate and schedule data pipelines efficiently.
- Ensure data security and compliance, especially when handling PII data, aligning with regulations like GDPR and HIPAA.
- Optimize query performance and data storage strategies to improve cost efficiency and speed of analytics.
- Collaborate with data analysts and business stakeholders to enhance analytics capabilities, enabling data-driven decision-making.
- Develop and maintain dashboards, reports, and analytical models to provide actionable insights for business and engineering teams.
- Four-year or Graduate Degree in Computer Science, Information Systems, or any other related discipline or commensurate work experience or demonstrated competence.
- 6-11 years of experience in Data Engineering, Analytics, Big Data, or related domains.
- Strong expertise in Databricks, AWS (S3, EC2, Lambda, RDS, Redshift, Glue, etc.), and cost optimization strategies.
- Hands-on experience with ETL pipelines, incremental data loads, and large-scale data processing.
- Proven experience in analyzing large datasets, deriving insights, and optimizing data workflows.
- Strong knowledge of SQL, Python, PySpark, and other data engineering and analytics tools.
- Strong problem-solving, analytical, and leadership skills.
- Experience with BI tools like Tableau, Looker, or Power BI for data visualization and reporting.
- Certified Software Systems Engineer (CSSE)
- Certified Systems Engineering Professional (CSEP)
- Effective Communication
- Results Orientation
- Learning Agility
- Digital Fluency
- Customer Centricity
- Impacts function and leads and/or provides expertise to functional project teams and may participate in cross-functional initiatives.
- Works on complex problems where analysis of situations or data requires an in-depth evaluation of multiple factors.
- This job description describes the general nature and level of work performed in this role. It is not intended to be an exhaustive list of all duties, skills, responsibilities, knowledge, etc. These may be subject to change and additional functions may be assigned as needed by management.
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
Director, Customer Care
Toast,
Bengaluru, Karnataka
9 hours ago
Toast is driven by building the all-in-one restaurant platform that helps restaurants operate their business, increase sales, engage guests, and keep employees happy.We are seeking a dynamic operational leader to join our Care leadership team. You will build and shape our India Customer Support teams partnering closely with existing global leadership teams. You will help shape the Care strategy that...
Sourcing Head
Ninjacart,
Bengaluru, Karnataka
4 days ago
SOURCING MANAGER - S&D Ninjacart - Pioneer. Challenge Yourself. Create Better Lives. Ninjacart is a pioneer in agritech. We believe in fostering a culture of innovation. We develop professionals who pioneer new ideas, simplify customer experiences, and impact the lives of millions. Our employees take complete ownership and truly belong.Ninjacart is India's leading agri-platform, which leverages technology and data to...
Software Engineer (C++, Mac/Linux)
Trellix,
Bengaluru, Karnataka
5 days ago
Job Title:Software Engineer (C++, Mac/Linux)About Trellix:Trellix, the trusted CISO ally, is redefining the future of cybersecurity and soulful work. Our comprehensive, GenAI-powered platform helps organizations confronted by today’s most advanced threats gain confidence in the protection and resilience of their operations. Along with an extensive partner ecosystem, we accelerate technology innovation through artificial intelligence, automation, and analytics to empower over...
