data engineer

Overview

Job-type Full-Time Job Category IT - Software Industries IT Salary MYR 10,000 - 11,000 /Month

Who you'll be working for

The client is a leading F&B company operating over 2,000 outlets.

What requirements you'll need to be eligible

• Bachelor’s degree holder in Computer Science, Data Science, Statistics, Mathematics or equivalent
• Minimum 5 years of experience in data engineering or a similar role.
• Strong proven expertise in Azure Synapse Analytics, Databricks, and other Azure cloud technologies.
• Proficiency in Python, PySpark, and other relevant programming languages for ETL automation.
• Experience in designing and implementing data models, including star schemas and other dimensional models.
• Experience in Data Management tools such as Data Lake, BI, ETL, Dashboards, Balanced Scorecards, etc.
• Excellent communication and presentation skills, to communicate complex analytical and technical content clearly and effectively.
• Demonstrated ability to work with large, complex datasets and build scalable data solutions.
• Track record of forming strong partnerships with cross-functional teams.
• Experience with data visualization tools and an understanding of how data engineering supports data analytics.

KNOWLEDGE, SKILLS & ABILITIES REQUIRED

• Highly proficient in data analysis and data mining skills (such as Python, Scala, R, T-SQL, MDX, DAX, etc.)
• Strong analytical skills with the ability to translate business needs into technical solutions.
• In-depth knowledge and understanding of data relationships and able to comprehend the data.
• Understand the reporting and business intelligence best practices.
• Ability to synthesize information from various sources to Data Management Platform to create a single and actionable point of view and direction for BI reporting requirements.
• Highly resourceful, with strong research skills and a working knowledge of digital marketing analytics.
• Possess a strong ability to query and use large, robust data sets across the latest Delta Lake and Databrick platforms.
• Strong understanding of data governance, security, and compliance best practices.
• Ability to manage multiple priorities and projects simultaneously.

OTHERS

• Fluent communication in English and interpersonal skills, both verbal and written.
• Meticulous and good team player who can work with various levels of stakeholders from different cultures and locations.
• Ability to work in a flexible, dynamic, and fast-paced environment managing multiple projects concurrently.
• Should be able to travel to other countries if needed.

What you'll be doing on the job

We are seeking a skilled Senior Data Engineer to join our team and contribute to the successful implementation of our regional Data & Analytics solutions across Southeast Asia Markets. This role will be pivotal in designing, building, and optimizing our data pipelines and infrastructure, ensuring seamless integration and processing of data from various sources, including CRM, sales, and D365 systems. The ideal candidate will have substantial experience in working with Azure technologies, particularly Databricks and Synapse Analytics, and will be proficient in automating ETL processes using SQL, Python or PySpark.

KEY RESPONSIBILITIES

• Design, develop, and maintain robust data pipelines and ETL processes to support data integration and transformation across multiple data sources.
• Design analytical solutions for strategic business problems through engagement and research.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and translate them into technical solutions in meeting milestones, timelines, and deliverables for projects.
• Implement and optimize data models, including star schemas, to facilitate efficient data querying and analysis.
• Leverage Azure Synapse Analytics, Databricks, and other Azure services to build scalable data infrastructure.
• Ensure data quality and consistency across all layers of the data lakehouse (bronze, silver, and gold layers).
• Automate ETL processes using SQL, Python, PySpark, or other relevant technologies to streamline data ingestion and transformation.
• Monitor and maintain the performance, security, and reliability of data systems.
• Provide technical leadership and guidance on best practices for data engineering within the organization.
• Document data pipelines, processes