Big Data Engineer working within the Big Data Team using cutting edge technologies to collate data across different sources. Supporting an event driven micro service lambda architecture for a Tier 1 Platform.
Responsibilities
Participate in all agile scrum meetings such as daily standups, refinement sessions, retro sessions, etc
Maintaining the data platform by means of daily reconciliations, data reloads, tackling support tickets, etc.
Being able to come up with new ideas to improve existing products and services.
Implementation of bug fixes and enhancements within the data platform.
Performing code reviews for other engineers.
Taking ownership of releases where necessary
Being able to communicate with the respective stakeholders, making sure that all information is transmitted correctly and to the right audience.
Take ownership of complex, large-scale initiatives, including data migration projects, ensuring successful planning, execution, and delivery.
Requirements
Object oriented programming (Java)
Data modelling using any database technologies
ETL processes (ETLs are oldschool, we transfer in memory now) and experience with Apache Spark or Apache NiFi
Applied understanding of CI\CD in change management
Dockerised applications
Used distributed version control systems
Hold a Bachelor’s Degree in Computer Science or equivalent
Can collaborate in scoping exercises
Mentoring skills
Have worked with both Linux and Windows environments.
Data Engineer managing and organizing datasets for AI models at Walaris, developing AI - driven autonomous systems for defense and security applications.
Data Engineer designing and maintaining data pipelines at Black Semiconductor. Collaborating with process, equipment, and IT teams to support manufacturing analytics and decision - making.
Junior Data Engineer role focusing on Business Intelligence and Big Data at Avanade. Collaborating on data analysis and SQL queries in a supportive learning environment.
GCP Data Engineer designing and developing data processing modules for Ki, an algorithmic insurance carrier. Working closely with multiple teams to optimize data pipelines and reporting.
Data Engineer at Securian Financial optimizing scalable data pipelines for AI and advanced analytics. Collaborating with teams to deliver secure and accessible data solutions.
IT Data Engineering Co‑Op at BlueRock Therapeutics supports development of scientific data systems. Collaboration on data workflows and foundational AWS data engineering tasks.
Data Engineer I building and operationalizing complex data solutions for Travelers' analytics using Databricks. Collaborating within teams to educate end users and support data governance.
Data Engineer shaping modern data architecture to drive golf’s digital transformation. Collaborating with teams to enhance data pipelines and insights for customer engagement and revenue growth.
Staff Data Engineer overseeing complex data systems for CITY Furniture. Responsible for architecting and optimizing data ecosystems in a hybrid work environment.
Data Engineer strengthening data platform team at Samba TV to improve data analytics and reporting capabilities. Building on AWS, Databricks, BigQuery, and Snowflake technology.