Manager Data Engineering (Databricks)

Join an organisation with a focus on diversity and inclusion, ensuring that all our employees feel welcomed, recognised, and valued.

About us – Australian Reinsurance Pool Corporation (ARPC) 

ARPC’s purpose as a public financial corporation is to make a meaningful difference to Australian communities by providing sustainable and effective reinsurance for terrorism and cyclone events. 

Our vision is to support insurers to deliver affordable terrorism and cyclone insurance in Australia. At ARPC, our values of Integrity, Respect, Service and Wellbeing shape how we work and deliver outcomes.  

Opportunity to be involved 

The Manager Data Engineering (Databricks) is a key technical leadership role responsible for delivering ARPC’s data engineering capability and playing a central role in the implementation of ARPC’s modern data and analytics platform (Databricks). 

ARPC is implementing Databricks to strengthen data quality, governance, automation and insight generation across its reinsurance pools and broader enterprise reporting needs. This role acts as the ARPC-side technical leader, accountable for ensuring the platform is implemented effectively and delivers scalable, secure and high-quality data outcomes. 

Reporting to the Head of Actuarial, the role combines hands-on engineering with architectural judgement, data governance discipline and delivery leadership. It works closely across actuarial, business, IT, security and external partners. 

Role responsibilities 

• Lead technical delivery coordination across data engineering activities and stakeholders  

• Act as the ARPC technical lead for the Databricks platform implementation 

• Contribute hands-on to Databricks engineering and migration delivery, including:  

• Development of pipelines, workflows, notebooks/jobs and orchestration  

• Implementation of Unity Catalog and data governance controls  

• Performance optimisation and platform uplift activities  

• Support migration of priority data assets and analytics workloads into Databricks, including:  

• Translation of existing processing logic and analytics code  

• Data validation, reconciliation and quality assurance  

• Production hardening and operational readiness  

• Establish and uplift data engineering governance practices including version control, testing and documentation  

• Implement and maintain strong data governance controls including access management, data quality and auditability  

• Translate business and actuarial requirements into structured engineering tasks and delivery outcomes  

• Work collaboratively with internal teams and external partners to deliver high-quality technical solutions  

• Review and assure technical designs, code and release approaches  

• Support future data capability planning and continuous improvement of data platforms  

• Build internal capability through knowledge transfer, documentation and support for BAU operations  

Candidate Attributes 

Qualifications 

• Tertiary qualification in Computer Science, Data Engineering, Software Engineering, Data Science or related discipline — Mandatory 

• Relevant industry certifications (e.g. Databricks, Microsoft Fabric, CDMP) — Desirable 

Experience 

• Extensive experience delivering end-to-end data engineering or platform solutions in production environments — Mandatory 

• Strong hands-on Databricks / Spark experience including pipelines, orchestration and optimisation — Mandatory 

• Experience implementing data governance controls (e.g. access models, audit logging, data quality) — Mandatory 

• Experience working in regulated or high-sensitivity data environments — Mandatory 

• Proven ability to work with stakeholders and translate requirements into technical outcomes — Mandatory 

• Experience working with vendors and reviewing technical deliverables — Highly Desirable 

Technical capability 

• Strong SQL and Python (including PySpark) capability 

• Strong Databricks engineering capability including workflows, notebooks and optimisation 

• Experience with data engineering patterns (ingestion, transformation, validation, lineage and auditability) 

• Understanding of secure data practices and privacy considerations 

• Experience with version control, testing and CI/CD practices 

• Strong communication skills with the ability to translate technical concepts clearly 

• Ability to set and maintain engineering standards and best practice 

Note 

The role is based in Sydney CBD. This is a 12-month non-ongoing position. 

You must be an Australian citizen and be able to obtain the Australian Government’s baseline security clearance successfully. 

Please apply for this position with a resume and covering letter to [email protected]. Your application will be reviewed by a member of the HR team and suitable candidates will be invited to an interview for the role. 

The closing date for applications for this role is Tuesday, 5 May 2026.