Senior Data Engineer

1 month ago


Kuala Lumpur, Malaysia NES Fircroft Full time
Job Requirements:
The position entails designing, developing, and maintaining robust, scalable, and sustainable data products. This involves building and optimizing data pipelines and infrastructure using technologies like Synapse, Azure Data Factory (ADF), Apache Spark, Apache Kafka, or similar tools. Collaboration with stakeholders is crucial to understanding their data requirements and translate them into technical solutions. Implementing data quality monitoring and validation processes using these technologies ensures data integrity. Additionally, enforcing data governance best practices, including data lineage, documentation, and security, is essential. Supporting data analysts and architects, contributing to the organization's data strategy, and evaluating new technologies to enhance the data engineering ecosystem are also part of the role.

The role also involves troubleshooting and resolving data-related issues, performance bottlenecks, and scalability challenges specifically on the Azure platform. It requires a solid grasp of DevOps principles and expertise in infrastructure automation using tools like Terraform or AWS CloudFormation. Hands-on experience with Azure cloud platforms and related services such as Azure Synapse, Azure Data Lake, etc., is essential.
Additionally, familiarity with data warehousing concepts and best practices is necessary. Collaborating closely with stakeholders to grasp business requirements and translating them into data solution designs is a key aspect. A strong understanding of data architecture principles, data modeling techniques, and data integration patterns is also vital for success in this role.

Educational Qualification & Experience:
BA/BS degree in Computer Science, Computer Engineering, Electrical Engineering or related technical field4- 8 years to total IT experience preferably in field of data engineeringExperience with Azure services including IAM, Synapse, DataLake, SQL Server, ADF etc.Experience in creating and deploying docker containers on Kubernetes.Experience in supporting development teams on Kubernetes best practices, troubleshooting, and performance optimization.Experience with CI/CD pipelines toll such as Jenkins and GitHub ActionsExperience with Synapse data warehousing and data lake solutionsStrong programming skill in Python, PySpark and SQLExperience in scripting and automation using languages such as Bash, Python, or GoExperience with infrastructure-as-code tools such as Terraform, Ansible, or CloudFormation and containerization technologies (e.g., Docker, Kubernetes).Knowledge of Agile methodologies and software development lifecycle processesProven experience in designing and implementing large-scale data solutions, including data pipelines and ETL processes on Azure.

With over 90 years' combined experience, NES Fircroft (NES) is proud to be the world's leading engineering staffing provider spanning the Oil & Gas, Power & Renewables, Chemicals, Construction & Infrastructure, Life Sciences, Mining and Manufacturing sectors worldwide. With more than 80 offices in 45 countries, we are able to provide our clients with the engineering and technical expertise they need, wherever and whenever it is needed. We offer contractors far more than a traditional recruitment service, supporting with everything from securing visas and work permits, to providing market-leading benefits packages and accommodation, ensuring they are safely and compliantly able to support our clients.

  • Senior Data Engineer

    4 weeks ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companyboutique consulting data and analytics services and solutioning company, providing end to end services including data strategy, data engineering, data visualization, data analytics and application development. about the jobResponsible to design, build and maintain data pipelines Responsible to perform ETL process with data from various...

  • Senior Data Engineer

    4 weeks ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companycourier service provider that provides universal postal services, providing services to both end users and corporate clients. They also provide services within the logistics and aviation industry about the jobResponsible for creating and maintaining data pipeline architecture Tansforming structured and unstructured data that is extracted...

  • Data engineer

    6 days ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Unison Consulting Pte Ltd Full time

    Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in...

  • Data Engineer

    4 weeks ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companyleading public listed company with 5 subsidiaries specializing in property development, investment & trading rspectivelyabout the jobdesigning, development and maintaining data pipelines from various sources responsible for data ingestion and transformationwork closely with data scientist and data analyst to implement data transformations...

  • Data Engineer

    1 month ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Cognizant Technology Solutions Full time

    Role: Data Engineer (Cantonese Speaker)Location: KL, Malaysia Mandatory:Azure SQL, Azure Databricks, Azure DevOps, Azure Pipeline.Secondary:Java, Kafka, API, MS SQL Server, ADFDescription:Data Analysis of new and existing inhouse systems.Data analysis of heterogeneous external source systems.Query, analyse and transform complex data sets with the goal of...

  • Data Architect

    4 weeks ago


    Kuala Lumpur, Malaysia Hyppies Full time

    Calling All Data Lake Crusaders: Are You Ready to Build the Future of Our Data? **Tired of wading through data swamps?** Do you dream of building a **majestic data lake** that empowers our entire organization? If so, then **grab your data kayak and paddle on over**, because we're looking for a **Senior Data Lake Architect** to join our awesome team! **In...

  • Senior AI

    2 days ago


    Kuala Lumpur, Malaysia 2019 Full time

      Responsibilities:   Work in project teams using agile and traditional project approaches. Identify customer needs and develop innovative data integration scenarios in modern cloud and on-premises environments. Analyze and optimize ETL/ELT processes in hybrid system landscapes. Develop solutions for various industries for generative AI,...

  • Data engineer

    7 days ago


    Kuala Lumpur, Malaysia Growthfn Sdn Bhd Full time

    Job Purpose: • Should be able to troubleshoot errors on Informatica, Teradata, Hadoop platforms and supports applications built on top of it. • Strong problem solving knowledge on Databases like Hadoop, Teradata & SQL. • Monitoring Jobs Workflow and working with technical teams to derive permanent fix. • Hands on Experience in Building, Trouble...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companyPioneering marketing-tech in ASEAN, this firm's mission is to simplify and leverage data to connect with millions of ASEAN consumers. Specializing in Consumers, Media & E-Commerce, it leads innovation in the industry. Join the journey to redefine marketing-tech and unlock the full potential of data-driven insights about the roleThis...

  • Data engineer

    6 days ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Growthfn Sdn Bhd Full time

    Job Purpose: Should be able to troubleshoot errors on Informatica, Teradata, Hadoop platforms and supports applications built on top of it. Strong problem solving knowledge on Databases like Hadoop, Teradata & SQL. Monitoring Jobs Workflow and working with technical teams to derive permanent fix. Hands on Experience in Building, Trouble Shooting...


  • Kuala Lumpur, Malaysia Agensi Pekerjaan Btc Sdn Bhd Full time

    Open Position: Senior Network Engineer (IT Company) An IT Company is currently looking for Senior Network Engineer to join the team and be based in the Kuala Lumpur office. Key responsibilities include: Experience in supporting and administrating network (LAN WAN) infrastructure platforms including Router Switches Firewall VPN and etc Good technical skills...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Agensi Pekerjaan Btc Sdn Bhd Full time

    Open Position: Senior Network Engineer (IT Company) An IT Company is currently looking for Senior Network Engineer to join the team and be based in the Kuala Lumpur officeKey responsibilities include: Experience in supporting and administrating network (LAN WAN) infrastructure platforms including Router Switches Firewall VPN and etc Good technical skills...


  • Kuala Lumpur, Malaysia Standard Chartered Full time

    **Job***: Operations **Primary Location***: Asia-Malaysia-Bukit Jalil KL **Schedule***: Full-time **Employee Status***: Permanent **Posting Date***: 08/Aug/2023, 6:42:47 AM **Unposting Date***: Ongoing **Role Responsibilities** The senior data analyst will work closely with data to help the organization make better business decisions. Using techniques...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companya reputable EPCC contractor with extensive experience offering comprehensive solutions to fulfill the needs of clients in Malaysia's petrochemicals, chemical, and general industries. about the jobDevelop and implement standards and concepts to meet job requirements.Plan pipe routing and equipment layout studies.Carry out engineering design...

  • Marketing Executive

    1 month ago


    Kuala Lumpur, Malaysia Damai Senior Care Centre Sdn Bhd Full time

    Position Overview: As a Marketing Executive for Damai Senior Care Centre, you will play a vital role in promoting our senior care services and building strong relationships with potential clients and their families. Your primary responsibility will be to develop and implement effective marketing strategies to increase the center's visibility, attract new...


  • Kuala Lumpur, Malaysia Petroplan Full time

    CORE REQUIREMENTS (Must haves): Degree or higher qualification in relevant area (eg. Mechanical, Electrical, Instrument, Reliability) Minimum 10 years (Senior) or 15 years (Lead) Maintenance/Reliability Engineering experience 3 years or more of master data experience. Knowledge of SAP strongly preferred. Ability to communicate effectively in English in both...


  • Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companyPioneering marketing-tech in ASEAN, this firm's mission is to simplify and leverage data to connect with millions of ASEAN consumers. Specializing in Consumers, Media & E-Commerce, it leads innovation in the industry. Join the journey to redefine marketing-tech and unlock the full potential of data-driven insights! about the roleThis...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companyThe client is a publicly listed construction firm with a market presence spanning over five decades. Widely recognized and financially robust, they have earned a solid reputation for the successful completion of projects. With numerous ongoing and upcoming secured projects, the company continues to thrive in the industry.about the jobCompile...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the companyyou will be joining an established solution provider who specializes in offering software product services such as fullstack development and cloud migration to both local as well as overseas clientele across Asia. The company currently has over 50 enthusiastic tech specialists delivering quality products and is seeking to expand their...

  • Data engineer

    3 days ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Ambition Full time

    Role Overview:As the key player, you will manage the end-to-end software delivery process in the data analytics field. This includes everything from analyzing requirements to implementing solutions, conducting testing, and providing support in Finance & Risk Analytics.Analyze user needs and translate them into solution design documentsCommunicate technical...