Lead Data Engineer overseeing engineers and advancing the data platform at American Family Insurance. Creating tools and infrastructure to empower teams across the company.
About the role
- Data Engineer creating and operating data pipelines in SQL Server or Azure Databricks environments. Supporting business needs and enhancing data quality for healthcare outcomes with a focus on automation.
Responsibilities
- The Data Engineer creates, operates, and extends data pipelines and/or orchestration solutions built in SQL Server or Azure Databricks environments.
- This role develops solutions to support the business and operational needs of internal and external partners, commonly applying SQL and ETL tools to optimize outcomes.
- Design and Develop Data Pipelines: Creates and maintains robust data pipelines using Databricks, Spark, SQL, or SSIS.
- ETL/ELT Processes: Develops and manages efficient ETL/ELT processes for data extraction, transformation, and loading.
- Data Integration: Integrates data from various sources, ensuring data quality and consistency.
- Query Writing and Optimization: Writes, optimizes, and maintains complex SQL queries to enhance performance.
- Collaborate with Stakeholders: Works closely with analysts and stakeholders to understand data requirements and deliver solutions.
- Performance Tuning: Optimizes data pipelines, SQL queries, and Delta Tables to ensure high performance and scalability.
- Data Quality Assurance: Ensures the accuracy and reliability of data through rigorous testing and validation.
- Documentation: Maintains comprehensive documentation of data pipelines, processes, and query optimizations.
- Continuous Improvement: Identifies and implements opportunities for process improvements and automation.
Requirements
- Bachelor’s degree in computer science, Mathematics, Statistics, or a related field.
- Minimum 4 years of experience in data engineering, including:
- Advanced SQL
- Azure Data Factory or other ETL tools
- Python or other programming languages
- Azure Databricks or Spark
- Azure DevOps or other CI/CD tools
- Strong analytical and problem-solving skills.
- Excellent communication and interpersonal skills.
- Ability to work independently and as part of a team.
- Strong attention to detail and ability to maintain data accuracy and integrity.
- Preferred Qualifications:
- Experience in healthcare, particularly specialty pharmacy.
- Familiarity with healthcare data standards such as HIPAA and HITECH.
- Proficiency working with Azure Log Analytics.
- Knowledge of dimensional data modeling.
- Data Warehousing Experience
Benefits
- Hybrid, remote and flexible on-site work schedules are available, based on the position.
- PANTHERx Rare Pharmacy also affords an excellent benefit package, including but not limited to medical, dental, vision, health savings and flexible spending accounts, 401K with employer matching, employer-paid life insurance and short/long term disability coverage, and an Employee Assistance Program!
- Generous paid time off is also available to all full-time employees, as well as limited paid time off for part-time employees.
- Of course we offer paid holidays too!
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Working Student contributing to data engineering projects for AI/ML workflows while gaining hands - on experience with data pipelines and tools.
Data Architect designing end - to - end Snowflake data solutions and collaborating with technical stakeholders at Emerson. Supporting the realization of Data and Digitalization Strategy.
Manager of Data Engineering leading data assets and infrastructure initiatives at CLA. Collaborating with teams to enforce data quality standards and drive integration efforts.
Data Engineer building modern Data Lake architecture on AWS and implementing scalable ETL/ELT pipelines. Collaborating across teams for analytics and reporting on gaming platforms.
Chief Data Engineer leading Scania’s Commercial Data Engineering team for growing sustainable transport solutions. Focused on data products and pipelines for BI, analytics, and AI.
Entry - Level Data Engineer at GM, focusing on building large scale data platforms in cloud environments. Collaborating with data engineers and scientists while migrating systems to cloud solutions.
Data Engineer designing and building scalable ETL/ELT pipelines for enterprise - grade analytics solutions. Collaborating with product teams to deliver high - quality, secure, and discoverable data.
Data Engineer responsible for data integrations with AWS technology stack for Adobe's Digital Experience. Collaborating with multiple teams to conceptualize solutions and improve data ecosystem.
People Data Architect designing and managing people data analytics for Gen, delivering actionable insights for HR. Collaborating across teams to enhance data - driven decision - making.