Data Engineer - Palantir Foundry

HashedIn by Deloitte


Date: 3 hours ago
City: Bengaluru, Karnataka
Contract type: Full time
The Core Responsibilities For The Job Include The Following

Palantir Data Engineering and Analytics;

  • Design, develop, and maintain scalable, modular data pipelines using Foundry Pipeline Builder (visual and code-based).
  • Ingest, integrate, and process data from diverse sources (S3 RDBMS, REST APIs, flat files, etc. ).
  • Implement advanced data processing techniques: incremental processing, anomaly detection, geospatial transformations, and time series analysis using PySpark, Python, and Foundry's no-code tools.
  • Parse and process various data formats (CSV, JSON, XML, Parquet, Avro) within Foundry.
  • Reuse and modularize functions and parameters for efficient, maintainable pipeline management.
  • Leverage LLMs for translation, classification, and data enrichment in pipelines via Palantir AIP Logic.

Ontology And Schema Management

  • Create and manage ontologies using Ontology Manager and Object Explorer to model business entities, relationships, and data lineage.
  • Implement property-level and object-level access controls for secure data modeling and compliance.

Data Quality, Validation, And Monitoring

  • Design and implement Master Data Management (MDM) and Reference Data Management solutions to ensure consistency and accuracy of key business entities across the organization.
  • Lead efforts in entity resolution, de-duplication, and golden record creation within Palantir or integrated MDM platforms.
  • Implement data validation, health checks, and monitoring for production-grade reliability.
  • Ensure data integrity, quality, and consistency across all stages of the data lifecycle.
  • Set up automated alerts and dashboards for pipeline health and anomaly detection.

Data Security And Governance

  • Enforce data privacy, security, and compliance standards (RBAC, audit logs, data masking) within Palantir and cloud environments.
  • Document data lineage, transformations, and access controls for auditability and governance.

Collaboration And Best Practices

  • Work closely with business analysts, data scientists, and product owners to translate requirements into robust data solutions.
  • Mentor junior engineers and analysts, contribute to code reviews, and champion best practices.
  • Document technical designs, workflows, and user guides for maintainability and knowledge transfer.

Data Analysis And Visualization

  • Perform data profiling, cleaning, joining, and enrichment to support business decision-making.
  • Conduct statistical and exploratory analysis to uncover trends, patterns, and actionable insights.

Dashboarding And Reporting

  • Develop and manage interactive dashboards and reports using Palantir Contour, Quiver, and other BI tools (Tableau, Power BI, Looker).
  • Build pivot tables, advanced reporting, and custom visualizations tailored to business needs.
  • Leverage Palantir's visualization modules for real-time and historical data analysis.

Cloud Platform Integration

  • Integrate AWS, Azure, or GCP data engineering and analytics services (Glue, Data Factory, BigQuery, Redshift, Synapse, etc. ) with Palantir workflows.
  • Design and implement end-to-end data pipelines that bridge Palantir and cloud-native ecosystems.

API And Microservices Integration

  • Develop and consume RESTful APIs, GraphQL endpoints, and microservices for scalable, modular data architectures.

DevOps And Best Practices

  • Implement CI/CD pipelines for data pipeline deployment and updates (Foundry, GitHub Actions, Jenkins, etc. ).
  • Apply containerization (Docker) and orchestration (Kubernetes) for scalable data processing.

Agile Collaboration

  • Work in Agile/Scrum teams, participate in sprint planning, and contribute to continuous improvement.

Requirements

  • Experience: 7-12 years in data engineering, data analysis, or related roles; 1-2 years on Palantir Foundry, Pipeline Builder, Contour, Quiver, or strong experience with AWS/Azure/GCP data engineering and analytics services.
  • Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, Mathematics, or a related field.

Certifications (Preferred But Not Mandatory)

  • Palantir Foundry Data Engineer/Analyst Certification.
  • AWS/Azure/GCP Data Engineering or Analytics Certifications.
  • Relevant BI/Visualization tool certifications.

Palantir Platform

  • Data pipeline development, transformation, and cleaning (Pipeline Builder, Code Workspaces).
  • Ontology creation, management, and data lineage (Ontology Manager, Object Explorer).
  • Data validation, health checks, and monitoring in production pipelines.
  • Data security, RBAC, audit logging, and compliance within Palantir.
  • Dashboarding and visualization (Contour, Quiver).
  • LLM integration for data enrichment (AIP Logic).

Data Engineering

  • Proficiency in SQL and Python; experience with PySpark is highly desirable.
  • Experience with data ingestion, integration, aggregation, and transformation from multiple sources.
  • Geospatial data processing, time series analysis, and anomaly detection.
  • Parsing and processing structured, semi-structured, and unstructured data.

Data Analysis

  • Data profiling, cleaning, joining, and enrichment.
  • Exploratory and statistical analysis.
  • Dashboarding, reporting, and advanced visualization (Contour, Quiver, Tableau, Power BI).

Cloud Platforms

  • Hands-on experience with AWS, Azure, or GCP data engineering and analytics services.
  • Integration of cloud services with Palantir workflows.

General Skills

  • Strong analytical, problem-solving, and communication skills.
  • Experience working in Agile/Scrum environments.
  • Ability to mentor and guide junior engineers and analysts.

Preferred Skills

  • Experience with Palantir's Pipeline Builder, Code Workspaces, and advanced data transformation modules.
  • Exposure to LLMs for data translation and enrichment.
  • Familiarity with data governance, security, and compliance best practices.
  • Prior experience in industries such as finance, healthcare, manufacturing, or government.
  • Experience with open-source data engineering tools (dbt, Great Expectations, Delta Lake, Iceberg).
  • Knowledge of CI/CD, DevOps, and automation tools.

This job was posted by Veronica A from Hashedin by Deloitte.

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Software Engineer 2, Java Backend

Okta, Bengaluru, Karnataka
3 hours ago
Get to know OktaOkta is The World’s Identity Company. We free everyone to safely use any technology, anywhere, on any device or app. Our flexible and neutral products, Okta Platform and Auth0 Platform, provide secure access, authentication, and automation, placing identity at the core of business security and growth.At Okta, we celebrate a variety of perspectives and experiences. We are...

Senior Analytics Manager

Meesho, Bengaluru, Karnataka
12 hours ago
About The TeamWith almost 4 Lakh entrepreneurs aboard the Meesho app, we are all set to democratise internet commerce for everyone in India. Our primary responsibility, as the Business org, is to actively drive this mission from the frontlines.As a Senior Manager - Business Analytics, you will play a huge role in driving growth across all categories. Our team holds...

Purchase Manager - DoubleTree by Hilton Bengaluru Airport

Hilton, Bengaluru, Karnataka
1 day ago
With thousands of hotels in over 100 countries and territories, Hilton offers countless opportunities to delight. From an open door to a welcoming smile and an exceptional experience, we offer the millions of travellers who stay with us every year a welcome they will never forget. In addition to our flagship brand, Hilton Hotels & Resorts, the family of brands...