Senior Data Engineer

2 days ago

Kuala Lumpur, Kuala Lumpur, Malaysia Talentbook Solutions Full time 60,000 - 120,000 per year

Urgent opening for Senior Data Engineer-Quality who's focused on testing ETL pipelines, data structure etc ... must be able to work EMEA hours 4pm to 1am (hybrid) ... salary up to RM15k

  1. Perm employment with top notch US Delivery Center in Malaysia (4 Positions)

Role Summary

We are looking for a detail-oriented and passionate Quality Engineer to ensure the reliability, quality, and integrity of modern cloud-based data platforms and software components. This role will test complex data workflows involving identity resolution, data tokenization, and privacy-compliant data sharing, with a strong focus on quality practices in cloud and data ecosystems like Databricks, Snowflake, AWS, Azure, and GCP. You will play a key role in validating ETL/ELT pipelines, data transformations, and integrations that power high-scale marketing, analytics, and identity systems.

Key Responsibilities


• Execute manual and automated QA test cases for

·       Batch and streaming data pipelines on Databricks, Snowflake, and Spark environments.

·       ETL/ELT workflows using orchestration tools like Airflow, dbt, or native cloud schedulers.


• Validate data ingestion from various sources (S3, ADLS, GCS, Kafka) into staging and curated zones.


• Perform field-level and schema-level validation between raw and transformed datasets.


• Validate business logic transformations, aggregations, joins, and data quality rules.


• Develop and maintain SQL- or Python-based test scripts to automate validation steps and data assertions.


• Collaborate with Data Engineers to ensure pipeline coverage for edge cases, data anomalies, and lineage tracking.


• Run data profile checks to detect duplicates, null values, incorrect mappings, and data drift.


• Test and validate secure data-sharing setups such as:

·       Snowflake Clean Rooms and Databricks Delta Sharing

·       Cross-cloud or multi-party identity joins and tokenization flows


• Perform QA of source-to-target mappings and documentation updates based on data lineage and metadata.


• Integrate data validations into CI/CD pipelines using tools like GitHub Actions, Azure DevOps, or GitLab.


• Develop and maintain test documentation, QA best practices, and aut

Requirements


• 3+ years of experience in QA, data validation, or data pipeline testing.


• Strong SQL skills with the ability to write data validation queries across complex joins and aggregations.


• Experience with scripting in Python for test automation.


• Familiarity with data formats such as Parquet, Avro, JSON, and Delta Lake.


• Familiarity with batch vs. stream data processing concepts.


• Familiarity with cloud storage and compute concepts in AWS (e.g., S3, Glue), Azure (e.g., Data Lake, Synapse), or GCP (e.g., BigQuery, Dataflow).


• Familiarity with QA practices: test planning, exploratory testing, regression, and bug tracking.


• Familiarity with one or more test frameworks (e.g., Mocha, Jasmine, Jest).


• Good communication skills, attention to detail, and a team-oriented mindset.


• Preferred Qualifications:

·       Exposure to data orchestration tools like Airflow, dbt, or cloud-native schedulers (e.g., AWS Step Functions, Azure Data Factory).

·       Experience using Databricks notebooks or Snowflake SQL worksheets for test development.

·       Basic understanding of distributed compute concepts (Spark, Delta Live Tables).

·       Familiarity with data quality frameworks (e.g., Great Expectations, Deequ) or metadata management tools (e.g., Collibra, Alation).

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia SRKay Consulting Group Full time 1,000,000 - 1,400,000 per year

    Role DescriptionThis is a contract on-site role for a Senior Data Engineer based in WP. Kuala Lumpur. The Senior Data Engineer will be responsible for designing and implementing scalable data engineering solutions, building efficient data models, and managing ETL pipelines. Additional responsibilities include optimizing existing data architectures, ensuring...

  • Senior Data Engineer

    2 weeks ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Plaza Premium Group Full time 120,000 - 240,000 per year

    Join Our Family Today.Together, We'll Make Travel Better.The senior data engineer will be responsible for building and maintaining scalable and reliable data infrastructure to support enterprise-wide analytics and business intelligence initiatives.This role leads junior data engineer in designing and implementing efficient data pipelines, integrating diverse...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Ploy Full time 120,000 - 180,000 per year

    What's on offer:Job Type: ContractLocation: Menara AIA CapSquareDuration: 1 year (highly renewable)Job Summary:As a Senior Data Engineer, you will be responsible for designing and implementing scalable and robust data infrastructure to support our client's data processing needs.Job Responsibilities:Design, develop, and maintain data pipelines and ETL...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia NERA GROUP Full time 90,000 - 120,000 per year

    Job Description:We are seeking a highly experiencedSenior Data Engineerwith a proven track record in designing, building, and managing complex data engineering solutions. The ideal candidate will have deep expertise indata pipelines, data warehousing,andinfrastructure management, especially withon-premises server environments.Key Responsibilities:Lead the...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia ZALORA SOUTH EAST ASIA PTE LTD Full time $900,000 - $1,100,000 per year

    ZALORA Group is looking for a Senior Data Engineer to join its Data Science Team. As a Senior Data Engineer, you will:Work daily with machine learning problems. For this purpose, you are expected to be involved in the process of collecting, storing, processing, and analyzing data generated from our database, in addition to joining our data scientists in...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Kamlax Global Technologies SDN BHD Full time 96,000 - 120,000 per year

    OverviewWe are seeking an experienced Senior Data Engineer to support the migration of legacy Cloudera applications to a modern Kubernetes-based data platform. The ideal candidate will possess strong hands-on development skills in data engineering and the ability to deliver high-quality, production-ready data pipelines.Key ResponsibilitiesDevelop and...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Quandatics Full time 120,000 - 240,000 per year

    The Senior Data Engineer plays a key role in designing, building, and optimizing data solutions. This role involves leading and guiding a team of data engineers while working on data architecture, ETL pipelines, and business rule transformation. Expertise in data integration, processing, and visualization ensures the delivery of high-quality data...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Lalamove Full time 120,000 - 240,000 per year

    At Lalamove, we believe in the power of community. Millions of drivers and customers use our technology every day to connect with one another and move things that matter. Delivery is what we do best and we ensure it is always fast and simple. Since 2013, we have tackled the logistics industry head on to find the most innovative solutions for the world's...

  • Senior Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia InfoTech Consultancy Full time 100,000 - 120,000 per year

    Position: Senior Data EngineerLocation: MalaysiaDuration: 12+ Months and ExtendableRole OverviewProjects where date is stored at-rest and in-flight encryptionRedshift – Role based access (RBAC), Row and column level accessGovernance using  AWS Lakeformationor  AWS Cloudwatch, AWS SNS, Cloudtrail4. Orchestration using Airflow and Glue workflows

  • Senior Data Engineering

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Lavu Tech Solutions Sdn Bhd Full time 120,000 - 240,000 per year

    Description:Role Summary :We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality pipelines under guidance from internal leads.Key...