Hybrid Software Engineer – Data Engineering, Data Analytics and Reporting

Posted last month

Apply now

About the role

  • Work in a dedicated data reporting and analytics team building a world-class data platform to produce data-driven insights for Transact Campus and clients
  • Analyze, interpret and orchestrate complex data across disparate sources comprising unstructured, semi-structured and structured datasets in streaming and batch modes
  • Design and develop real-time data pipelines using Databricks and Delta Lake on Azure
  • Collaborate with data consumers (reporting, analysis, or data science) to provide metrics that meet their needs
  • Contribute to standards for data producers streaming data into the Lakehouse
  • Test commercial software products using both manual and automated testing processes
  • Support the application lifecycle during QA, UAT, and post-release phases
  • Comply with and contribute to consistent development guidelines (coding, change control, build, versioning)
  • Participate in peer code reviews

Requirements

  • Bachelor’s degree in Computer Science, IT or related field OR equivalent related work experience
  • 5+ years of experience in software enterprise-level Data Engineering
  • Expertise in big data workloads
  • Experience with data lakes and scale-out processing
  • Relational database design and best practices
  • Hands-on experience designing and developing Spark data pipelines
  • Strong SQL and Python skills
  • Understanding of cloud-based big data workloads
  • Experience with ETL/ELT patterns, preferably using Databricks jobs
  • Excellent technical documentation skills
  • Experience with data lakes (HDFS, Azure Data Lake or AWS S3)
  • Experience with source code management systems such as Git/TFS/SVN
  • Experience working in Agile teams (Scrum, XP, Kanban)
  • Ability to present ideas and insights to business stakeholders
  • Fluency in written and spoken English
  • Hybrid working with at least 3 days a week on site during probationary period (onboarding requirement)
  • Preferred: Good understanding of Azure Data Services (Azure Databricks, Azure Data Factory, Azure Data Lake Gen 2)
  • Preferred: Experience with Databricks Delta Lake, Delta Sharing, and Delta Live Tables
  • Preferred: Experience with Spark Structured Streaming
  • Preferred: Experience with NoSQL databases
  • Preferred: Experience with Infrastructure as Code technologies such as Terraform or ARM
  • Preferred: Experience in Data Science and ML methodologies
  • Preferred: Experience with Azure services for streaming data (EventHub, EventGrid)
  • Preferred: Understanding of Data strategy including Data Governance and Data management

Benefits

  • Work with cutting-edge data technologies and platforms
  • Collaborative and supportive work environment
  • Ongoing professional development and training opportunities
  • Hybrid working model (at least 3 days a week on site during probationary period)
  • Regular social, sporting, and community events
  • Private Health Insurance
  • Dental Insurance
  • Matched Pension Contribution
  • 25 Days Annual Leave

Job title

Software Engineer – Data Engineering, Data Analytics and Reporting

Job type

Experience level

Mid levelSenior

Salary

Not specified

Degree requirement

Bachelor's Degree

Location requirements

Report this job

See something inaccurate? Let us know and we'll update the listing.

Report job