Senior Data Engineer

Skillsheet.me


Date: 11 hours ago
City: Remote, Remote
Contract type: Full time
Remote

About the Company

Our customers discover the future of problem-solving with our AI Solutions. We

offer a suite of innovative artificial intelligence solutions designed to revolutionize

how businesses tackle challenges. Our tailored AI applications, powered by state-

of-the-art algorithms, bring unparalleled efficiency and insight to your operations.

As a partner to universities, this company specializes in developing and managing

workforce-focused online education programs that are both accessible and

affordable.


We do the following:

University Partnerships: Collaborates with over 100 public and private

universities to launch and manage online degree programs.

Student Support: Offers services like marketing, recruitment, retention, and

performance monitoring.

Technology & ROI: Implements tech solutions and conducts research on

student return on investment (ROI), aiming to make education more accessible

and impactful.


Impact & Reach

25,000+ courses launched

22,000+ faculty supported

Average student payback time: 1.6 years

Active in 40 U.S. states and 5 countries


Job Requirements

  • Bachelor's degree in Computer Science, Engineering, or a related field
  • 5+ years of experience in software/data engineering, including at least 2 years
  • working with Databricks and Apache Spark
  • Databricks Data Engineer 1Strong proficiency in Python, SQL, and PySpark
  • Deep understanding of AWS and Azure Cloud services
  • Experience with Databricks Data Lakehouse, Databricks Workflows, and
  • Databricks SQL
  • Solid grasp of data Lakehouse and warehousing architecture
  • Prior experience supporting AI/ML workflows, including training data pipelines
  • and model deployment support
  • Familiarity with infrastructure-as-code tools like Terraform or CloudFormation
  • Strong analytical and troubleshooting skills in a fast-paced, agile environment
  • Excellent collaboration skills for interfacing with both technical and non-
  • technical customer stakeholders
  • Clear communication skills with strong documentation habits
  • Comfortable leading discussions, offering strategic input, and mentoring
  • others


Job Responsibilities

  • The ideal candidate will have a strong background in building scalable data
  • pipelines, optimizing big data workflows, and integrating Databricks with cloud
  • services.
  • This role will play a pivotal part in enabling the customer's data engineering
  • and analytics initiatives—especially those tied to AI-driven solutions and
  • projects—by implementing cloud-native architectures that fuel innovation and
  • sustainability.
  • Partner directly with the customer's data engineering team to design and
  • deliver scalable, cloud-based data solutions.
  • Execute complex ad-hoc queries using Databricks SQL to explore large
  • lakehouse datasets and uncover actionable insights.
  • Leverage Databricks notebooks to develop robust data transformation
  • workflows using PySpark and SQL.
  • Databricks Data Engineer 2Design, develop, and maintain scalable data pipelines using Apache Spark on
  • Databricks.
  • Build ETL/ELT workflows with AWS and Azure Services.
  • Optimize Spark jobs for both performance and cost within the customer's
  • cloud infrastructure.
  • Collaborate with data scientists, ML engineers, and business analysts to
  • support AI and machine learning use cases, including data preparation,
  • feature engineering, and model operationalization.
  • Contribute to the development of AI-powered solutions that improve
  • operational efficiency, route optimization, and predictive maintenance in the
  • waste management domain.
  • Implement CI/CD pipelines for Databricks jobs using GitHub Actions, Azure
  • DevOps, or Jenkins.
  • Ensure data quality, lineage, and compliance through tools like Unity Catalog,
  • Delta Lake, and AWS Lake Formation.
  • Troubleshoot and maintain production data pipelines.
  • Provide mentorship and share best practices with both internal and customer teams

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Member of Technical Staff - Wind River Conductor

Wind River, Remote, Remote
11 hours ago
DescriptionPosition at Wind RiverJob Title: Member of Technical Staff – Wind River ConductorAbout The OpportunityWind River Systems is building Wind River Studio for Operators, delivering an integrated cloud platform, unifying infrastructure, orchestration, and analytics capabilities so operators can deploy and manage their intelligent 5G edge networks globally. Wind River Systems is seeking an experienced, high-performing Senior Engineer Software Engineer developing...

Director - Data Governance and AI Operations

SailPoint, Remote, Remote
1 day ago
SailPoint is the leader in identity security for the cloud enterprise. Our identity security solutions secure and enable thousands of companies worldwide, giving our customers unmatched visibility into the entirety of their digital workforce, ensuring workers have the right access to do their job – no more, no less.  Built on a foundation of AI and ML, our Identity Security...

Cambridge English Faculty

PW (PhysicsWallah), Remote, Remote
5 days ago
Job Title: Cambridge English FacultyLocation: Work from Home (WFH)Job Type: Full-TimeAbout the Role:We are looking for an enthusiastic and experienced Cambridge English Faculty member to join our team. This role is ideal for educators passionate about helping students achieve success in Cambridge English Exams (A1, A2, B1, B2, etc.). You will design and deliver engaging lessons, evaluate student progress, and...