Data Engineer

Invictus


Date: 3 days ago
City: Chennai, Tamil Nadu
Contract type: Full time
Responsibilities

  • Design, develop, and implement ETL data pipelines that load data into an information product that helps the organization in reaching strategic goals.
  • Work on ingesting, storing, cleansing, processing, and analyzing large data sets from heterogeneous data sources.
  • Finalizing the scope of the system and delivering Big Data solutions.
  • Translate complex technical and functional requirements into detailed designs.
  • Schedule, orchestrate, and implement data pipelines and processes that scale with an increase in data volume.
  • Investigate and analyze alternative solutions to data storing, processing, etc., to ensure the most streamlined approaches are implemented.
  • Collaborate with business consultants, data scientists, and application developers to develop analytics solutions.
  • Help define data governance policies and support data versioning processes.
  • Maintain security and data privacy by working closely with the Data Protection Officer internally.
  • Analyze a vast number of data stores and uncover insights.
  • Create data tools for business team members that assist them in analysis that gives them the competitive edge.
  • Monitor data application performance for potential bottlenecks and resolve performance issues.
  • Identify and implement cost-saving strategies to reduce ongoing Big Data and Cloud costs/expenses.

Requirements

  • Strong understanding of Data Warehousing concepts and data modeling techniques, and columnar databases.
  • Strong hands-on experience in Data Analysis, preprocessing, and validation of the engineered dataset in SQL/NoSQL databases.
  • Hands-on Experience in data cleansing, processing, and loading using Big Data tools mainly (Py)Spark and Hive/Presto.
  • Exposure to any workflow scheduling and Orchestration tools Airflow/Prefect/SQS, would be helpful.
  • Comfortable working with any programming/scripting language like Python(Preferred)/Scala/Shell scripting.
  • Strong data engineering skills on anyCloud Platform are essential(Preferred: AWS).
  • Exposure to a data lake and a data lake house would be an added advantage. Solid Understanding of data structures and algorithms.
  • Knowledge of distributed/MPP systems of data storage and computing.
  • Ability to effectively communicate with both business and technical teams.
  • Good interpersonal skills and a positive attitude.
  • Experience in working with agile methodology in a fast-paced environment.

This job was posted by Subashini T from Invictus.

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Research Analyst, Data Analytics

Crisil, Chennai, Tamil Nadu
1 day ago
ResponsibilitiesSupport investment portfolio reporting and quantitative analyses for a leading alternative asset managerDevelop and improve workflow efficiency through automationAssist with preparation of materials for client and senior managementAssist with development of robust risk management framework and analytics across portfoliosAnalyse time series data to identify and report any trends or errors/exceptions, using quantitative techniques, machine learning models, in the form of...

Front-end developer (CSS HTML XML)

Verizon Data Services, Chennai, Tamil Nadu
2 days ago
When you join VerizonYou want more out of a career. A place to share your ideas freely even if theyre daring or different. Where the true you can learn, grow, and thrive. At Verizon, we power and empower how people live, work and play by connecting them to what brings them joy. We do what we love driving innovation, creativity,...

Director - Salesforce

TransUnion, Chennai, Tamil Nadu
3 days ago
TransUnion's Job Applicant Privacy NoticeWhat We'll BringWe are looking for an experienced and highly motivate senior leader to lead our B2B Salesforce Delivery in the Global Operations portfolio. The ideal candidate will have a strong background in Salesforce design, delivery and experience in managing a global Salesforce platform. As a Director of Salesforce B2B Delivery services, you should have Salesforce...