Big Data Hadoop Developer

2 days ago

Kuala Lumpur, Kuala Lumpur, Malaysia Unison Group Full time 80,000 - 140,000 per year

Job Summary:
We are looking for a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The ideal candidate should have strong hands-on experience with the Hadoop ecosystem and integration with relational databases such as MariaDB or Oracle DB for analytics and reporting.

Key Responsibilities:

  • Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
  • Work with data ingestion frameworks to integrate data from MariaDB/Oracle DB into Hadoop (Sqoop, Apache Nifi, Kafka).
  • Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
  • Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
  • Create and maintain ETL pipelines for structured and unstructured data.
  • Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
  • Collaborate with BI, analytics, and data science teams for data provisioning.
  • Ensure data security, governance, and compliance in all solutions.

Technical Skills:

  • Big Data Ecosystem: Hadoop (HDFS, YARN), Hive, Spark, Sqoop, MapReduce, Oozie, Flume.
  • Databases: MariaDB and/or Oracle DB (SQL, PL/SQL).
  • Programming: Java, Scala, or Python for Spark/MapReduce development.
  • Data Ingestion: Sqoop, Kafka, Nifi (for integrating RDBMS with Hadoop).
  • Query Optimization: Hive tuning, partitioning, bucketing, indexing.
  • Tools: Ambari, Cloudera Manager, Git, Jenkins.
  • OS & Scripting: Linux/Unix shell scripting.

Soft Skills:

  • Strong analytical skills and problem-solving abilities.
  • Good communication skills for working with cross-functional teams.
  • Ability to manage priorities in a fast-paced environment.

Nice to Have:

  • Experience with cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).
  • Knowledge of NoSQL databases (HBase, Cassandra).
  • Exposure to machine learning integration with Hadoop/Spark.

  • BigData Hadoop Developer

    4 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Tata Consultancy Services (TCS) Full time 80,000 - 120,000 per year

    JD:Need to work as a developer in Cloudera Hadoop.Work on Hadoop, PySpark, Spark SQL, Hive, Bigdata Eco System Tools.Experience in working with teams in a complex organization involving multiple reporting lines.The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking...

  • SQL Developer

    4 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Adecco Full time 80,000 - 120,000 per year

    We are seeking aSQL Developerto design, build, and optimize data pipelines and data models for enterprise digital platforms. You will work on database development, ETL workflows, big data performance tuning, and production support.Key ResponsibilitiesDesign and implement scalable ETL workflows and data models for analytics platforms.Optimize SQL queries and...

  • Data Architect

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Two95 International Full time 150,000 - 250,000 per year

    SCOPE & AUTHORITY Data Architecture:Execute Enterprise data initiatives / programs to establish a governed, curated, and agile data ecosystem that enables business to make data-driven decisions. Translate strategic requirements into a usable Enterprise information architecture (e.g. enterprise data model, associated metamodel, common business...

  • Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Luxoft Full time 120,000 - 180,000 per year

    Project Description:Our Client a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and territories in Asia Pacific, Europe and North America are looking for Consultants to be part of the projectThe Technology and Operations function is comprised of five teams of specialists with distinct capabilities: business...

  • Data Engineer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Oliver James Full time 80,000 - 120,000 per year

    We're looking for aSenior Data Engineerto design, develop, and maintain modern data solutions that power insights across multiple markets. If you're passionate about big data, cloud platforms, and building scalable data systems, this is an opportunity to make an impact in a fast-growing regional technology hub.What You'll DoDesign, build, and implement...

  • 65097 Data Architect

    4 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Half the sky Full time 100,000 - 120,000 per year

    Job DescriptionResponsible for data lake design and providing technical direction to data engineer, data analysts, data integration developers and database administratorsProvide input, monitor, and coordinate the project solution team across infrastructure, technical, and functional work streamsResponsible for the quality assurance of all architecture and...

  • Cloudera System Administrator

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia MindGraph Full time 120,000 - 180,000 per year

    Responsibilities:Administer and maintain Cloudera CDH/CDP clusters, including installation, configuration, upgrades, and patching.Monitor cluster health and performance using Cloudera Manager and other tools.Manage and optimize Hadoop ecosystem components (HDFS, YARN, Hive, Impala, Spark, HBase, etc.).Implement and manage Kubernetes clusters for...

  • Senior Data Scientist

    4 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia AMK Technology Sdn Bhd Full time 132,000 per year

    Qualificaition:4–6 years of proven experience in Data Science, Machine Learning, or AI-related roles.Strong proficiency in Python (NumPy, Pandas, Scikit-learn, TensorFlow, PyTorch, etc.).Hands-on experience with statistical modeling, machine learning, and deep learning techniques.Solid understanding of data structures, algorithms, and model evaluation...

  • Data Science Intern

    4 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Datamicron Full time 30,000 - 60,000 per year

    DataMicron is a multi-award Big Data and AI company, with patented technologies in USA and China. We have extensive experience in developing products and full-scale solutions in the field of business intelligence, machine learning & AI, data engineering and front-end development. We have covered and implemented end-to-end Big Data Analytics solutions in...

  • SQL Developer

    2 days ago

    Kuala Lumpur, Kuala Lumpur, Malaysia Agensi Pekerjaan Career International FOS Sdn Bhd Full time 36,000 - 43,200 per year

    Job Description:We are looking for a skilled SQL Development Consultant to support data-related projects, including data modeling, system optimization, and maintenance. The ideal candidate will have strong database development experience, excellent problem-solving skills, and the ability to collaborate effectively with business teams to deliver efficient...