Senior Data Engineer

Skillsheet.me


Date: 2 weeks ago
City: Remote, Remote
Contract type: Full time
Remote

About the Company

Our customers discover the future of problem-solving with our AI Solutions. We

offer a suite of innovative artificial intelligence solutions designed to revolutionize

how businesses tackle challenges. Our tailored AI applications, powered by state-

of-the-art algorithms, bring unparalleled efficiency and insight to your operations.

As a partner to universities, this company specializes in developing and managing

workforce-focused online education programs that are both accessible and

affordable.


We do the following:

University Partnerships: Collaborates with over 100 public and private

universities to launch and manage online degree programs.

Student Support: Offers services like marketing, recruitment, retention, and

performance monitoring.

Technology & ROI: Implements tech solutions and conducts research on

student return on investment (ROI), aiming to make education more accessible

and impactful.


Impact & Reach

25,000+ courses launched

22,000+ faculty supported

Average student payback time: 1.6 years

Active in 40 U.S. states and 5 countries


Job Requirements

  • Bachelor's degree in Computer Science, Engineering, or a related field
  • 5+ years of experience in software/data engineering, including at least 2 years
  • working with Databricks and Apache Spark
  • Databricks Data Engineer 1Strong proficiency in Python, SQL, and PySpark
  • Deep understanding of AWS and Azure Cloud services
  • Experience with Databricks Data Lakehouse, Databricks Workflows, and
  • Databricks SQL
  • Solid grasp of data Lakehouse and warehousing architecture
  • Prior experience supporting AI/ML workflows, including training data pipelines
  • and model deployment support
  • Familiarity with infrastructure-as-code tools like Terraform or CloudFormation
  • Strong analytical and troubleshooting skills in a fast-paced, agile environment
  • Excellent collaboration skills for interfacing with both technical and non-
  • technical customer stakeholders
  • Clear communication skills with strong documentation habits
  • Comfortable leading discussions, offering strategic input, and mentoring
  • others


Job Responsibilities

  • The ideal candidate will have a strong background in building scalable data
  • pipelines, optimizing big data workflows, and integrating Databricks with cloud
  • services.
  • This role will play a pivotal part in enabling the customer's data engineering
  • and analytics initiatives—especially those tied to AI-driven solutions and
  • projects—by implementing cloud-native architectures that fuel innovation and
  • sustainability.
  • Partner directly with the customer's data engineering team to design and
  • deliver scalable, cloud-based data solutions.
  • Execute complex ad-hoc queries using Databricks SQL to explore large
  • lakehouse datasets and uncover actionable insights.
  • Leverage Databricks notebooks to develop robust data transformation
  • workflows using PySpark and SQL.
  • Databricks Data Engineer 2Design, develop, and maintain scalable data pipelines using Apache Spark on
  • Databricks.
  • Build ETL/ELT workflows with AWS and Azure Services.
  • Optimize Spark jobs for both performance and cost within the customer's
  • cloud infrastructure.
  • Collaborate with data scientists, ML engineers, and business analysts to
  • support AI and machine learning use cases, including data preparation,
  • feature engineering, and model operationalization.
  • Contribute to the development of AI-powered solutions that improve
  • operational efficiency, route optimization, and predictive maintenance in the
  • waste management domain.
  • Implement CI/CD pipelines for Databricks jobs using GitHub Actions, Azure
  • DevOps, or Jenkins.
  • Ensure data quality, lineage, and compliance through tools like Unity Catalog,
  • Delta Lake, and AWS Lake Formation.
  • Troubleshoot and maintain production data pipelines.
  • Provide mentorship and share best practices with both internal and customer teams

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Vice President Of Products, IgniteTech (Remote) - $400,000/year USD

Trilogy, Remote, Remote
22 hours ago
Elite Product Visionaries Only: Lead the GenAI Revolution Across 100+ Enterprise Software ProductsIgniteTech is transforming B2B enterprise software with a GenAI-first approach that demands exceptional product leadership. We're seeking a strategic VP of Products who can harness the power of Generative AI to revolutionize our extensive portfolio and drive unprecedented customer retention.Working directly with our CEO, you'll architect the GenAI...

Customer Success Manager

Optimite, Remote, Remote
1 day ago
About OptimiteOptimite is the Email Ops-as-a-Service partner for high-growth eCommerce brands and agencies. We fully manage email strategy, copy, design, development, QA, and reporting - so our clients can step out of the inbox and scale faster. Trusted by over 500 teams globally, we deliver fast, conversion-focused email systems built on Figma and executed in 48 hours or less.TeamWith a...

Software Engineer

Zoom, Remote, Remote
4 days ago
What You Can ExpectAs a Fullstack Software Engineer, you will be responsible for designing and implement scalable, high-performance backend systems using Node.js, TypeScript, and Fastify. (You will need to have basic REACT experience.) You will collaborate with cross-functional teams to analyze user needs, create technical specifications, and write clean, efficient, and well-documented code. Your goal is to deliver high-quality software...