Architect leading initiatives in data platform engineering at F5, solving challenges in unstructured data and enhancing governance frameworks.
About the role
- Lead Data Engineer architecting and building data platforms at an ASX-listed fintech. Spearheading MLOps initiatives and collaborating across teams in a hybrid work environment.
Responsibilities
- Lead the design, development, and maintenance of scalable data pipelines using Apache Airflow, dbt, and Azure Data Factory.
- Architect and optimise our Snowflake data warehouse, including schema design, performance tuning, and cost management.
- Implement and enforce data governance policies, including data masking, role-based access control, and compliance with regulatory standards.
- Drive MLOps initiatives: model deployment, monitoring, retraining pipelines, and integration with CI/CD workflows.
- Collaborate with cross-functional teams to operationalise ML models, including Generative AI solutions, into production systems.
- Partner with the Platforms team to oversee infrastructure-as-code deployments using Pulumi, Docker, Kubernetes and cloud-native services (Azure, AWS).
- Lead, coach and guide a team of engineers, fostering a culture of best practices in coding, testing, and documentation alongside a culture of continuous learning and psychological safety.
- Partner with stakeholders across the business such as Operations to translate business requirements into technical solutions.
Requirements
- 7+ years experience in data engineering, with at least 2 years experience leading a team, ideally in a fast paced, high growth environment.
- Strong proficiency in Apache Airflow, dbt, and Azure Data Factory.
- Expertise in Snowflake, including advanced SQL, performance optimisation, and security features.
- Experience with real-time data processing (Apache Flink, Kafka).
- Proven experience with MLOps tools and frameworks (MLflow, Kubeflow, or similar).
- Hands-on experience deploying and integrating Generative AI models (OpenAI, Hugging Face, or similar).
- Solid understanding of data governance frameworks and compliance requirements (GDPR, CCPA).
- Proficiency in Python and SQL; familiarity with PySpark or other distributed processing frameworks is a plus.
- Experience with cloud platforms (Azure, AWS) and infrastructure-as-code tools (Pulumi, Terraform).
- Strong CI/CD skills with Azure Pipelines or equivalent.
- Excellent communication skills, with the ability to interact effectively with technical and non-technical stakeholders
- Strong problem solving skills, with a continuous learning mindset.
Benefits
- Flexible and hybrid working
- $500 every year to spend on your wellbeing
- Take an extra Annual Leave day off on us every financial year, with A Day on Wisr
- Access via WHEREFIT to discounted gym memberships, corporate discounts for wellbeing products and more!
- Generous paid parental leave to support your transition to parenthood
- Regular social events and awesome team offsites
- Access to our Employee Assistance Program, Uprise with up to 6 coaching sessions per year
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Data Architect designing and implementing data architectures supporting analytics and ML for federal clients. Collaborating with teams to translate mission needs into robust data solutions.
IT Data Engineer developing data pipelines and integrations for Scanfil Group's global IT organization. Collaborating across teams to enhance data solutions and reporting capabilities.
Data Engineer developing Azure data solutions at PwC New Zealand. Responsibilities include data quality monitoring, pipeline development, and collaboration with stakeholders in a supportive environment.
Senior Data Engineer designing and implementing the Enterprise Data Platform at Stellix. Focusing on analytics and insights with a growth path to Principal Data Engineer or Data Architect.
R&D Data Engineer at DXC, transforming complex data into digital assets for global analytics and Smart Lab solutions. Collaborating on ELN and LIMS tools for enhanced data management.
Data Engineer role focusing on data pipelines and processing at 42dot, a mobility AI company. Responsibilities include data collection, schema management, and pipeline monitoring.
Senior Data Engineer at mobility AI company designing large - scale data processing pipelines. Leading technical decisions and mentoring junior engineers in data architecture.
Senior Data Engineer at Booz Allen building advanced tech solutions for mission - driven projects. Utilizing data engineering activities, pipelines, and platforms for impactful data insights.
Senior Software Engineer contributing to Workday's AI/MLOps cloud ops platform. Involves data ingestion, computation, and generation of curated data sets with modern technologies.