Data Engineer Team Lead

Are you ready to revolutionise the way we build data?

At Deriv, we're looking for a talented Data Integration Engineer to spearhead our digital transformation initiatives. In this role, you'll lead the organisation in cultivating a data-driven culture as our company moves toward the future.

As a key technology team member, you'll be responsible to oversee the design, development, and optimization of our data pipelines, ensuring seamless integration and availability of data across all systems. You will lead a talented team of data engineers and spearhead the execution of data strategies that are integral to supporting critical business decisions and operations. A significant responsibility is to develop a unified data pipeline that effectively serves both internal systems and external applications, ensuring secure and real-time data flow across all platforms.

AI is being embedded into our company DNA, across every department. For further information, please read our guide: ' AI at Deriv: Your future fuelled by innovation '.

Your challenges:

• Team Leadership: Lead and mentor a team of data engineers to design, build, and optimize scalable data pipelines for analytics, reporting, and operational needs.
• Unified Data Pipeline Management: Create and manage a unified data pipeline that integrates multiple data sources, facilitating efficient access for both internal and external systems, including third-party tools and client-facing platforms.
• Utilization of Low-Code/No-Code Tools: Leverage low-code/no-code integration tools such as Fivetran or RudderStack to accelerate data workflows and improve integration efficiency.
• Cross-Functional Collaboration: Work closely with various teams to identify data needs and ensure smooth data flows between systems, enhancing high availability and integration.
• Data Governance and Quality Assurance: Implement and manage data governance and quality standards, ensuring data consistency and reliability throughout the organization.
• Real-Time and Batch Data Architecture: Architect and oversee the implementation of real-time and batch data pipelines using data streaming tools like Kafka or Pub/Sub, ensuring low-latency, high-performance data processing.
• Technical Guidance and Support: Provide technical guidance to your team, aiding in troubleshooting complex data issues and optimizing performance and cost-effectiveness while adhering to best practices.
• Continuous Improvement: Stay informed on the latest in data engineering technologies and best practices, especially concerning real-time processing and low-code/no-code integrations, to enhance our data infrastructure continually.
• Stakeholder Coordination: Communicate with stakeholders to ensure timely delivery of data solutions that effectively meet business requirements.

You may enjoy working with us if you:

• Experience in fintech, trading, or other data-heavy industries.
• Familiarity with API-based integrations and third-party system integrations.
• Knowledge of additional data streaming technologies and best practices.
• Experience with automating data flows using low-code platforms.

Requirements:

• Bachelor's degree in computer science, data engineering, or a related field.
• 10 years of experience in data engineering, with at least 4 years in a leadership role.
• Expertise in designing, building, and maintaining data pipelines and data warehouse solutions.
• Strong knowledge of cloud platforms such as Google Cloud or AWS, and tools like BigQuery, PostgreSQL, and Airflow.
• Proficiency in programming languages such as Python or similar, with experience in ETL/ELT processes.
• Hands-on experience with low-code/no-code data integration tools like Fivetran, RudderStack, or similar platforms.
• Deep understanding of real-time data streaming and data processing tools such as Kafka, Pub/Sub, or similar.
• Experience in building unified data pipelines that serve both internal and external systems, ensuring consistent, secure data flows.
• Experience working with both real-time and batch data processing systems.
• Strong understanding of data governance, security, and compliance practices.
• Excellent communication skills, with the ability to convey technical concepts to non-technical stakeholders and mentor junior team members.

Company culture:

• Join a diverse, inclusive workplace that values innovation and cross-cultural collaboration
• Enjoy competitive compensation, professional development opportunities
• Be part of a company recognised for excellence and committed to career growth opportunities

You'll be part of our flagship headquarters in Cyberjaya, home to 600 talented professionals driving innovation in online trading. Our purpose-built, five-storey building enhances collaboration and creativity, leveraging cutting-edge AI and no-code/low-code platforms to push boundaries. You'll join a vibrant workplace where weekly knowledge-sharing sessions, cross-functional collaborations, and cultural celebrations create an engaging atmosphere. You'll work in a stimulating, high-tech environment featuring modern conference rooms, a fully-furnished gym, and barista-made coffee, all designed to keep you motivated and inspired.

Our culture is unique, and we live by our values and leadership principles, so it's worth learning more about Deriv's culture .