Data Engineer at Pruna AI merging data engineering, analytics engineering, and revenue operations. Working on intelligent automation and analytics for accessible and sustainable AI.
About the role
- Data Engineer creating and implementing Big Data applications at Absa with business stakeholders and technology leaders. Involves ETL/ELT pipeline design, data automation, and team mentorship.
Responsibilities
- Design, implement, and maintain scalable, high-performance ETL/ELT pipelines for structured and unstructured data
- Understand the technical landscape and bank wide architecture to effectively design & deliver data solutions (architecture, pipeline etc.)
- Create & Maintain CI / CD Pipelines (authoring & supporting CI/CD pipelines within Git Hub Actions and deploy to production)
- Automate data applications using orchestration tools
- Debug and improve existing source code
- Support the continuous optimisation, improvement & automation of data pipelines
- Coach & mentor other data engineers
- Conduct peer reviews, testing, problem solving within the team
- Identify technical risks and mitigate these (pre, during & post deployment)
- Update / Design all application documentation aligned to the organization technical standards and risk / governance frameworks
- Create business cases & solution specifications for various governance processes (e.g. CTO approvals)
- Participate in incident management & DR activity – applying critical thinking, problem solving & technical expertise to get to the bottom of major incidents
- Deliver on time & on budget (always)
Requirements
- BSc Honours, BCom Honours, BEng, BBusSc in Computer Science, Information Systems or any Information Technology qualification that is at NQF level 8 or higher
- 3 or more years of experience as a Data Engineer
- Understanding of and experience of using Big Data technologies (Hadoop) is essential
- Experience with designing and developing Scala/ Apache Spark data applications
- Understanding of Linux and Bash scripting
- Understanding of Git and GitHub Actions
- Experience in CA Wade or any other orchestration tool
- Great SQL skills
- Ability to work in either an Agile or project methodology to deliver tasks
- Datawarehouse experience is beneficial but not a must
- Cloud skills (AWS preferable) and Databricks are beneficial but not a must
Benefits
- Equal opportunity employer
- Preference to designated groups for demographics
- Contribution to diversity of workforce
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer developing modern cloud - based data pipelines at UK Biobank for research support. Collaborating within the Data & Technology team to create clean, scalable, secure code.
Senior Consultant SAP Data Migration leading projects in data migration for various clients. Collaborating with project teams to ensure successful strategy implementation and data quality management.
Data Engineering Team Lead engaging with enterprise - level organizations on data architecture using Google Cloud solutions. Leading a team of data engineers in a hybrid work model.
Senior Lead Data Engineer in Bangalore, Karnataka, India designing and building AWS data engineering solutions. Leading teams on scalable data pipelines using Spark, PySpark, and Python.
Senior Data Architect delivering, enhancing, and adopting enterprise data and analytics products for DoD organizations. Collaborating with teams to translate requirements into scalable solutions for national security outcomes.
Senior Data Engineer supporting the delivery and enhancement of enterprise data and analytics for DoD organizations. Collaborating with engineers and government partners on scalable, production - ready solutions.
Senior Data Engineer at Skillfield designing distributed data processing solutions using Apache Spark. Collaborates on cloud and on - prem solutions across enterprise levels in a hybrid work environment.
Senior Data Engineer optimizing ETL/ELT pipelines at Asahi Kasei. Evaluate programming concepts and support data science projects while ensuring solution stability in a hybrid work setup.
Data Engineer responsible for building data intelligence system for the public sector. Ensuring data ingestion, quality, correlation, and helping with analytics for decision making.