Tech Lead Data Engineering overseeing data engineering, ETL processes, and cloud technologies. Leading project delivery with strong hands - on experience in Informatica, Python, and GCP.
About the role
- Senior Staff AI Data Engineer focused on cloud modernization and data integration for an insurance company. Designing ETL pipelines and collaborating on AI projects to support analytics.
Responsibilities
- Design, develop, and optimize ETL/ELT pipelines for both structured and unstructured data.
- Mentor junior team members and engage in communities of practice to deliver high-quality data and AI solutions while promoting best practices, standards, and adoption of reusable patterns.
- Partner with architects and stakeholders to influence and implement the vision of the AI and data pipelines while safeguarding the integrity and scalability of the environment.
- Ingest and process large-scale datasets into the Enterprise Data Lake and downstream systems.
- Curate and publish Data Products to support analytics, visualization, and machine learning use cases.
- Collaborate with data analysts, data scientists, and BI teams to build data models and pipelines for research, reporting, and advanced analytics.
- Apply best practices for data modeling, governance, and security across all solutions.
- Partner with cross-functional teams to ensure alignment and delivery of high-value outcomes.
- Monitor and fine-tune data pipelines for performance, scalability, and reliability.
- Automate auditing, balance, reconciliation, and data quality checks to maintain high data integrity.
- Develop self-healing pipelines with robust re-startability mechanisms for resilience.
- Schedule and orchestrate complex, dependent workflows using tools like MWAA, Autosys, or Control-M.
- Leverage CI/CD pipelines to enable automated integration, testing, and deployment processes.
- Lead Proof of Concepts (POCs) and technology evaluations to drive innovation.
- Develop AI-driven systems to improve data capabilities, ensuring compliance with industry best practices.
- Implement efficient Retrieval-Augmented Generation (RAG) architectures and integrate with enterprise data infrastructure.
- Implement data observability practices to proactively monitor data health, lineage, and quality across pipelines, ensuring transparency and trust in data assets.
Requirements
- Bachelor’s or master’s degree in computer science or a related discipline
- 5+ years of experience in data analysis, transformation, and development, with ideally 2+ years in the insurance or a related industry
- 3+ years of experience developing and deploying large-scale data and analytics applications on cloud platforms such as AWS and Snowflake
- Strong proficiency in SQL, Python, and ETL tools such as Informatica IDMC for data integration and transformation (3+ years)
- Experience designing and optimizing data models for Data Warehouses, Data Marts, and Data Fabric, including dimensional modeling, semantic layers, metadata management, and integration for scalable, governed, and high-performance analytics (3+ years)
- 3+ years of hands-on experience in processing large-scale structured and unstructured data in both batch and near-real-time environments, leveraging distributed computing frameworks and streaming technologies for high-performance data pipelines
- Strong technical knowledge (AI solution leveraging Cloud and modern solutions)
- 3+ years of experience in Agile methodologies, including Scrum and Kanban frameworks
- 2+ years of experience in leveraging DevOps pipelines for automated testing and deployment, ensuring continuous integration and delivery of data solutions
- Proficient in data visualization tools such as Tableau and Power BI, with expertise in creating interactive dashboards, reports, and visual analytics to support data-driven decision-making
- Ability to analyze source systems, provide business solutions, and translate these solutions into actionable steps.
- Candidate must be authorized to work in the US without company sponsorship.
- The company will not support the STEM OPT I-983 Training Plan endorsement for this position.
Benefits
- short-term or annual bonuses
- long-term incentives
- on-the-spot recognition
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Data Engineer developing and maintaining data infrastructure for healthcare solutions in Portugal. Working with Oracle databases and Pentaho ETL pipelines in a flexible hybrid model.
Google Data Architect creating and optimizing data solutions using GCP technologies. Collaborating with teams to enhance enterprise data architecture across business functions.
Azure Lead Data Engineer designing and developing ETL/ELT pipelines with Azure Data Factory and Snowflake. Collaborating with cross - functional teams in a cloud - native environment.
Principal Data Engineer leading Azure platform designs and implementations for enterprise solutions at UBDS Group. Mentoring teams and driving high engineering standards in hybrid environments.
Data Engineer designing and maintaining the data systems for Skiffra’s AI - native orchestration platform. Collaborating closely with product and engineering teams for data integration and system design.
Data Engineer building data pipelines for AI Safety company. Handling petabytes of logs and creating a clean, reliable data environment.
Data Engineer at Kyndryl designing and maintaining data pipelines using AWS and Python. Optimizing ingestion, transformation workflows, and cloud solutions for large - scale data environments.
Data Architect responsible for the integrity and reliability of Patient Services data in Life Sciences. Ensuring analytics - ready data through strategic vendor collaboration and data stewardship.
Project & Data Engineer providing operational support and data management for utility service projects in the Greater Los Angeles area. Involves invoice processing, data accuracy, and system coordination.