Lead Data Engineer

Lead Data Engineer 

📍QLD, WA, ACT, VIC, NSW, NT, SA, TAS
🕒 Until June 2027 with 12 months extension 
🔒 Baseline Clearance Required

We are seeking an experienced Lead Data Engineer to play a key role in the development of a nationally significant AI and data capability focused on workforce, education and skills intelligence.
Working within a multidisciplinary team of Data Scientists, Information Management specialists and policy experts, you will design, build and maintain scalable data and AI infrastructure that supports advanced analytics, machine learning, NLP and Generative AI solutions.

Key duties and responsibilities as a Lead Data Engineer: 

• Designing, developing and maintaining scalable data ingestion, transformation and processing pipelines for structured and unstructured data sources.
• Applying modern data engineering practices to support the collection, preparation, cleansing, validation and integration of skills-related datasets.
• Developing automated data quality, testing and validation capabilities that support reliable and reproducible analytical outcomes.
• Designing and implementing repeatable validation pipelines that support the assessment, monitoring and quality assurance of AI and NLP model outputs.
• Collaborating with Data Scientists to operationalise machine learning, NLP and Generative AI solutions within scalable production environments.
• Developing and maintaining model-serving, deployment and API integration capabilities that support analytical products and downstream systems.
• Implementing data governance, metadata management, lineage tracking and versioning practices to support transparency, traceability and reproducibility.
• Developing and maintaining cloud-based and distributed data processing environments capable of supporting large-scale data and AI workloads.

Skills & Experience Essential

• Demonstrated experience designing, building and maintaining scalable data pipelines and data platforms.
• Strong expertise in data ingestion, transformation, automation and bulk processing workflows.
• Experience working with cloud-based and distributed data environments.
• Advanced Python development skills.
• Proven experience deploying Machine Learning, NLP and/or Generative AI solutions into production environments.
• Experience supporting end-to-end delivery of AI-enabled products and services.

Please note you will need to have an existing AGSVA Baseline security Clearance to apply for this role


If you wish to apply for this position, please submit your resume by clicking the 'Apply Now' button. For further information please contact Abhi Naik at Clicks IT Recruitment on 
+61 427 997 851.
 
At Clicks we embrace diversity, inclusion and equal opportunity. First Nations people are encouraged to apply.
 
We provide reasonable adjustments, including alternate formats to the recruitment process for individuals with a disability. If you
require an adjustment to be made during the recruitment process, please call 1300 254 257 or email adjustments@clicks.com.au