Data Engineer

DATA ENGINEER at a Data Consultancy & Data Solutions Provider

Salary: RM4000 - RM6000

Position: Senior Data Engineer

JOB OVERVIEW

Join Avinity Analytics where we focus on providing you with the best training and education in data engineering, as well as relevant data projects that will help you gain the necessary experience to advance your career.

There are multiple vacancies available for senior data engineers with 2 to 4 years of experience, to learn and execute data projects at a basic and advanced level. You will be developing essential data engineering know-how in our 3 year Data Development Program, whilst executing data solutions for mid-size to enterprise clients. Your data engineering skills will be applied in data analytics, data automation, and machine learning & Ai projects.

Working at a data consultancy with a flexible working environment and a continuous learning culture, you will gain significant improvements in your data engineering skillsets. Through solving real business challenges with data, you will be implementing the latest data software and technologies to drive data adoption and innovation.

If you have a great interest to be involved in a variety of data projects and would like to take your data skills to the next level, you are encouraged to apply.

RESPONSIBILITIES

• Create and maintain optimal data pipeline architecture via scheduling and authoring tools such as Airflow & dbt.
• Wrangling large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
• Build the data infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL including Google, Microsoft & AWS cloud data/serverless technologies.
• Build analytics models that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

REQUIREMENTS

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as query optimization.
• Strong analytic skills related to working familiarity with a variety of databases - MSSQL, MySQL, PostgreSQL, MongoDB.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing, and extracting value from large disconnected datasets.
• Knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Experience using one or more of the following software/tools:
• Big data storage & transform tools: Hadoop, Spark, Kafka, Snowflake, BigQuery.
• Relational SQL and NoSQL databases.
• Data pipeline and workflow management tools: Luigi, Airflow.
• AWS cloud services: EC2, Labda, EMR, RDS, Redshift
• Stream-processing systems: Kafka, Spark-Streaming, etc.
• Object-oriented/object function scripting languages: Python, R.

WHY JOIN US?

• If you're a data enthusiast, there's no better place than here.
• Work in a flexible, dynamic, and merit based work environment.
• Your personalized learning experience through the data development programme.
• Better than average employment benefits to satisfy your human needs.
• Bonuses, ESOP, and other performance benefits for productive employees.