AI Data Pipeline Engineer designing and operating high - throughput systems for petabyte - scale data delivery. Collaborating across teams to ensure data flows into AI workloads efficiently.
About the role
- Data Architect designing and developing end-to-end data pipelines for actionable insights at Swiss Re. Collaborating with stakeholders to deliver scalable, efficient data solutions.
Responsibilities
- Design and implement data storage solutions including data lakes and data warehouses
- Develop and maintain robust data pipelines to extract, transform, and load data from various source systems
- Create and manage data models to support analytical and operational requirements
- Optimize data storage and processing for performance, cost, and reliability
- Collaborate with business stakeholders to understand requirements and provide efficient solutions
- Integrate data from multiple sources, including APIs, databases, and external data providers
- Ensure data quality and integrity through validation and transformation processes
- Document data engineering processes, systems, and standards
- Monitor and optimize the performance of data systems and pipelines
- Promote best practices for data management and engineering
- Partner with Product Owners and Engineering Leads to build target architectures and improve business processes
Requirements
- Bachelor's or Master's degree in a quantitative field such as Computer Science, Mathematics, Engineering, or Statistics
- 4-6 years of experience designing and implementing end-to-end data pipelines, data models, and analytical dashboards
- Strong programming skills with proficiency in Python, PySpark, SQL, and TypeScript
- Experience with data visualization tools like Palantir Workshop, Slate, and Contour
- Proficiency in Spark-based data lake design and operations
- Experience with integration technologies including REST/SOAP APIs and event-based architecture
- Palantir certification, Azure Data Engineering certification, or AI-102 Azure AI Engineer certification (nice to have)
- Experience with relational databases such as Oracle and Azure SQL
- Familiarity with cloud platforms (particularly Azure) and their data services
- Experience with data modeling and schema design
- Strong interpersonal, written, and verbal communication skills
- Ability to explain technical concepts to non-technical audiences
- Experience working in Agile/Scrum environments
Benefits
- None specified
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
AWS Data Engineer role focusing on AWS technologies in Gurugram, Haryana, India. Responsibilities include AWS data engineering tasks and collaboration with team members.
Data Engineer implementing innovative technology for various domains at Quantexa. Building data pipelines and providing insights to help clients solve complex business problems.
Principal Consultant Data Architecture leading complex Data and Analytics projects in a hybrid environment. Responsible for designing enterprise data architectures and mentoring technical teams.
Consultant / Senior Consultant in Data Engineering & Data Science contributing to data solutions. Collaborating with cross - functional teams in a hybrid environment in Germany.
Senior Data Engineer managing data platform strategy and analytics architecture at HALOS scaleup company. Owning design and implementation of analytical data platform.
Data Engineer building trusted data platforms for decision making at Lyrebird Health. Collaborating with teams to develop and maintain data pipelines and analytics - ready tables.
Data Engineer at leading online insurance platform for businesses. Delivering data pipelines and collaborating cross - functionally to enhance decision - making.
Senior Data Engineer developing and maintaining products for dentsu’s marketing using cloud technologies. Collaborating with teams to solve complex data problems and ensuring driving business value.
Senior Data Engineer responsible for designing and developing data products and solutions at Lloyds Banking Group. Collaborating with various teams to improve regulatory data processes and mentoring other engineers.