Databricks Architect

2 days ago


Kuala Lumpur, Kuala Lumpur, Malaysia Avanade Full time

Our talented Data & AI Practice is made up of globally recognized experts - and there's room for more analytical and ambitious data professionals. If you're passionate about helping clients make better data-driven decisions to tackle their most complex business issues, let's talk. Take your skills to a new level and launch a career where you can truly do what matters.

Come join us
The Databricks Solution Architect is a senior technical leadership role responsible for defining, designing, and overseeing the implementation of scalable, secure, and high-performance data platforms using the Databricks Lakehouse Platform. This individual will translate business strategy into technical architecture, guide development teams, and ensure all data solutions align with enterprise standards for governance, security, and cost optimization.

Skills And Experiences
Required Skills & Experience:

  • 10+ years of experience in data architecture, data engineering, or a related senior technical role.
  • 5+ years of deep, hands-on experience designing and implementing large-scale data solutions on the Databricks Platform.
  • Expertise in Lakehouse Architecture, Delta Lake, Apache Spark, and performance tuning techniques.
  • Proven experience implementing Databricks Unity Catalog for centralized governance.
  • Deep proficiency in Python/PySpark or Scala and advanced SQL.
  • Extensive experience with one major cloud platform and its security and data services.
  • Strong experience with data modeling (dimensional, data vault) and data warehouse concepts.
  • Demonstrated ability to lead technical discussions, document architectural designs, and communicate complex concepts to both technical and non-technical audiences.

Preferred Skills & Certifications

  • Experience with real-time/streaming data architectures (Structured Streaming).
  • Experience with MLOps practices, including MLflow for model lifecycle management.
  • Expertise in Infrastructure-as-Code tools (e.g., Terraform) and automated CI/CD for Databricks.
  • Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional or equivalent cloud architecture certification.

About You
Characteristics that can spell success for this role

  • Consultative, collaborative, curious
  • Resilient, adaptable, flexible
  • Humble leader, strong negotiator, relationship builder
  • Passionate about tech and engaging with and advising clients

What You'll Do

  • Architectural Leadership: Define and own the end-to-end technical architecture and roadmap for the enterprise data platform on Databricks, including data ingestion, transformation, storage, and consumption layers.
  • Lakehouse Design: Design and champion the implementation of the Lakehouse architecture utilizing Delta Lake, Databricks Unity Catalog, and Databricks SQL Warehouse to support all data, analytics, and AI/ML initiatives.
  • Data Governance & Security: Architect and enforce enterprise-level data governance, security, and access control policies using Unity Catalog (e.g., fine-grained access, lineage tracking, auditing).
  • Technical Guidance & Mentorship: Provide technical leadership, guidance, and mentorship to a team of Databricks Engineers. Conduct architectural reviews, code audits, and ensure adherence to best practices and standards.
  • Performance and Cost Optimization: Define strategies for, and lead, major performance tuning and cost optimization initiatives across all Databricks workloads, clusters, and Delta Lake storage.
  • Cloud Integration: Lead the integration of Databricks with core cloud services (AWS, Azure, or GCP) and other enterprise systems (e.g., data catalogs, BI tools, ML platforms).
  • Stakeholder Collaboration: Engage with executive stakeholders, data scientists, and business leaders to understand complex requirements and translate them into robust, scalable technical designs and solution blueprints.
  • DataOps and Automation Strategy: Define the DevOps/DataOps strategy for the Databricks environment, including continuous integration/continuous delivery (CI/CD) pipelines and Infrastructure-as-Code (IaC) using tools like Terraform or Databricks Asset Bundles.
  • Innovation: Evaluate new Databricks features, open-source technologies (e.g., MLflow), and industry trends to drive continuous platform improvement and competitive advantage.

Learn more
To learn more about the types of projects our Data & AI team works on check out these case studies:

  • thyssenkrupp Materials Services uses data to help strike a delicate operational balance
  • What matters to SSE Renewables is innovating for a sustainable future
  • Hachette UK enables employees with machine learning-driven contract searches
  • Marston Holdings identifies significant cloud savings

Interested in knowing what's going on inside Avanade? Check out our blogs:

  • Avanade Insights – exchange ideas that drive tomorrow's innovation
  • Inside Avanade – explore what life is like working at Avanade

Enjoy your career
Some of the best things about working at Avanade

  • Opportunity to work for Microsoft's Global Alliance Partner of the Year, with exceptional development and training (minimum 80 hours per year for training and paid certifications)
  • Real-time access to technical and skilled resources globally
  • Dedicated career advisor to encourage your growth
  • Engaged and helpful coworkers genuinely interested in you

Find out more about some of our benefits Employee Benefits at Avanade | Avanade

A great place to work
As you bring your skills and abilities to Avanade, you'll get distinctive experiences, limitless learning, and ambitious growth in return. As we continue to build our diverse and inclusive culture, we become even more innovative and creative, helping us better serve our clients and communities. You'll join a community of smart, supportive collaborators to lift, mentor, and guide you, and to lean on your expertise. You get a company purpose-built for business-critical, leading-edge technology solutions, committed to improving the way humans work, interact, and live. It's all here, so take a closer look

We work hard to provide an inclusive, diverse culture with a deep sense of belonging for all our employees. Visit our Inclusion & Diversity page.

Create a future for our people that focuses on

  • Expanding your thinking
  • Experimenting courageously
  • Learning and pivoting

Inspire greatness in our people by

  • Empowering every voice
  • Encouraging boldness
  • Celebrating progress

Accelerate the impact of our people by

  • Amazing the client
  • Prioritizing what matters
  • Acting as one


  • Kuala Lumpur, Kuala Lumpur, Malaysia SoftwareOne Full time

    Why SoftwareOne?SoftwareOne and Crayon have come together to form a global, AI-powered software and cloud solutions provider with a bold vision for the future. With a footprint in over 70 countries and a diverse team of 13,000+ professionals, we offer unparalleled opportunities for talent to grow, make an impact, and shape the future of technology. At the...


  • Kuala Lumpur, Kuala Lumpur, Malaysia AIA Digital+ Full time 150,000 - 250,000 per year

    Job DescriptionSolution Architecture Experience on Azure Cloud and SaaS services.Experience on API management.Design and implement Azure Well Architected framework.Design Technology platforms on Azure cloud, by adopting PaaS services such as container platforms.Define and establish Application design best practice, standards, and guidelines to ensure...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time

    about the roleas a Senior Staff Data Scientist, you will lead the technical vision and roadmap for the company's next-generation segmentation engine. You will drive architectural decisions, mentor senior data scientists, and ensure the delivery of scalable, high-impact models. This is a hybrid leadership role: highly strategic, but still deeply hands-on in...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Black Pen Recruitment Full time

    Our client's Microsoft Business Solutions team is a Microsoft Gold Partner and leader in Microsoft software implementations for medium to large organisations, providing their clients with the ability to keep up with the ever-changing digital world by implementing Microsoft's Enterprise Resource Planning (ERP) and Customer Engagement (CE) software from the...

  • Data Engineer

    1 week ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Getronics Solutions (Malaysia) Sdn. Bhd. Full time 900,000 - 1,200,000 per year

    Job Description - Data EngineerSummaryResponsible for building and maintaining our data infrastructure including pipelines and data storage.Roles & responsibilitiesDesign, develop, and maintain data pipelines to ingest data from SAP ECC, SAP S4 Hana, and SAPCollaborate with data architects and other stakeholders to build efficient data integration...

  • Data Engineer

    1 week ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Getronics Full time 80,000 - 160,000 per year

    Job Description - Data EngineerSummaryResponsible for building and maintaining our data infrastructure including pipelines and data storage.Roles & responsibilitiesDesign, develop, and maintain data pipelines to ingest data from SAP ECC, SAP S4 Hana, and SAP.Collaborate with data architects and other stakeholders to build efficient data integration...


  • Kuala Lumpur, Kuala Lumpur, Malaysia Randstad Malaysia Full time $60,000 - $180,000 per year

    About The CompanyA technology-driven performance marketing agency that delivers solutions across media, data, and audience analytics, working to accelerate business growth for its clients through expertise in AI and experimentation. It unifies complex data and technology to optimize advertising effectiveness and provide actionable intelligence for global...

  • Data Engineer

    4 days ago


    Kuala Lumpur, Kuala Lumpur, Malaysia Atomic Recruitment SEA Full time

    Job Title: Data Engineer (Mid to Senior)Key ResponsibilitiesPartner with product owners, managers, and engineers to define project scopes and shape Minimum Viable Products (MVPs).Collaborate with cross-functional teams to design and implement big data solutions using technologies such as Databricks, Azure Data Factory, Spark, PySpark, Python , and ...


  • Kuala Lumpur, Kuala Lumpur, Malaysia AIA Australia Full time 120,000 - 180,000 per year

    Are you ready to shape a better tomorrow?AIA Digital+ is a Technology, Digital and Analytics innovation hub dedicated to powering AIA to be more efficient, connected and innovative as it fulfils its Purpose to help millions of people across Asia-Pacific live Healthier, Longer, Better Lives.If you are hungry and driven to play an active role in shaping a...

  • Data Engineer

    2 days ago


    Kuala Lumpur, Kuala Lumpur, Malaysia rhi Full time

    rhi is recruiting for 4xData Engineersto join our clients team in KL, Malaysia.Contract duration - minimum 12 months (possible extension)Working hours will be 9.00am - 6.00pm (flexibility given as and when there are morning and evening calls - primarily supporting ANZ with occasional support for the UK)Working arrangement will be hybrid (2 days in office, 3...