Principal Data Engineer

About the role

The Principal Data Engineer will own the design, development, and operationalisation of complex data pipelines and platform capabilities, mentor engineers across the team, and act as the primary technical authority for the lakehouse setting the bar for data quality, performance, and engineering best practices across all domains.

Key responsibilities:

• Design and build scalable data pipelines using Azure Databricks, Delta Live Tables, and Azure Data Factory
• Architect bronze-to-gold medallion layers for new domains and guide the evolution of existing ones
• Govern and extend Unity Catalog, including table ownership, lineage, access controls, and schema design
• Lead platform reliability engineering - SLA monitoring, cluster optimisation and cost governance
• Data Quality & Standards
• Define and enforce data engineering standards: naming conventions, testing frameworks, CI/CD pipelines, and documentation

Essential experience

• 8+ years of professional data engineering experience in complex, production environments
• Deep expertise in SQL, PySpark, and Python for large-scale data processing
• Hands-on experience with Azure-native services: ADF, Event Hubs, ADLS Gen2, Azure Synapse
• Azure Databricks (Unity Catalog, Delta Live Tables, Workflows, MLflow)
• Microsoft Fabric and Power BI semantic model integration
• CI/CD for data pipelines (Azure DevOps, Git-based workflows)
• Microsoft Certified: Azure Data Engineer Associate or Databricks certification

If you are interested in this opportunity, please appy via the link. All shortlisted candidates will be contacted this week.

Please note, this is a contract role with potential for extention. All candidates must be able to commit to 3 days onsite.

If you wish to apply for this position, please submit your resume by clicking the 'Apply Now' button. For further information please contact Olivia Costello at Clicks IT Recruitment on +61 419 740 399.

Apply Now