Senior Lead Data Engineer designing and building scalable data solutions utilizing AI technology for a globally recognized financial institution. Serving sophisticated clients across the globe.
About the role
- Principal HR Data Engineer specializing in Microsoft Azure and Databricks Lakehouse platforms. Responsible for designing, implementing, and maintaining scalable data pipelines and architectures for analytics.
Responsibilities
- Design, develop, and maintain scalable, high-performance data pipelines and architectures leveraging Microsoft Azure and Databricks Lakehouse to enable data analytics and machine learning capabilities.
- Partner with data scientists, data analysts, business teams, and other stakeholders to understand data requirements and ensure high-quality data solutions are delivered in alignment with business needs.
- Implement and optimize ETL (Extract, Transform, Load) processes for the ingestion, transformation, and loading of data from diverse data sources into the Databricks Lakehouse environment.
- Optimize the performance and cost-efficiency of data storage solutions and data retrieval methods within the cloud ecosystem.
- Ensure data integrity, consistency, and security across all data pipelines, storage layers, and access points.
- Monitor data pipelines and associated infrastructure, troubleshooting and resolving any performance, data flow, or reliability issues.
- Stay current with emerging trends and best practices in cloud computing, big data technologies, and data engineering methodologies to drive continuous improvement.
Requirements
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field
- Relevant certifications—such as Microsoft Certified: Azure Data Engineer or Databricks Certified Data Engineer—are a plus.
- 7 or more years of experience as a Data Engineer with a focus on Microsoft Azure and Databricks Lakehouse technologies.
- Demonstrated experience working with HR or People data is strongly preferred, particularly within enterprise HCM, workforce analytics, or employee lifecycle reporting environments.
- Experience in designing, developing, and optimizing scalable data solutions in cloud environments.
- Working knowledge of machine learning and AI data pipelines, including how data engineering supports feature engineering, model training, and scalable analytics.
- Strong background in SQL, Python, PySpark, and other relevant programming languages.
- Extensive experience in ETL processes, data integration, and data warehousing.
- Data Management: Expertise in data modeling, data quality assurance, and applying industry-standard data governance practices.
- Cloud & Big Data Technologies: Familiarity with tools and frameworks like Azure Synapse, Azure Databricks, Apache Kafka, and Azure Data Factory.
- Problem Solving: Strong analytical and troubleshooting skills with an attention to detail.
- Collaboration: Excellent communication and collaboration skills to work effectively with cross-functional teams.
- Independence & Time Management: Ability to work autonomously, manage multiple projects, and prioritize tasks efficiently.
Benefits
- We prioritize your growth, development, and well-being to help you reach your full potential.
- Opportunities that fit your lifestyle and ambitions—whether you’re looking for part-time flexibility or full-time career advancement.
- Reasonable accommodations for applicants with disabilities.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer Consultant building and maintaining enriched data infrastructure for analytical thinking at Northwest Permanente. Involves data collection, cleansing, and transformation for business intelligence.
Vice President - Business, Data Architect role at TD Securities focusing on business data architecture and analytics capabilities. Collaborate with stakeholders to define and govern data models and ensure alignment with strategy.
Staff Data Engineer at Headspace building privacy - first data platforms for mental health support. Leading data engineering strategies and mentoring team members to enhance data - driven decision making.
Senior Data Engineer building and implementing data pipelines at Headspace. Collaborating with analytics and data science teams to enhance personalized mental health support.
Data Engineering Intern working on data pipelines and infrastructure in fast - growing fintech. Collaborating with data engineers, learning best practices and developing data solutions.
Senior Software Engineer building and maintaining data infrastructure for Gusto. Collaborating with Data Science and Business Intelligence teams to achieve their goals.
Data Engineer building and maintaining scalable data pipelines for AI Search Infrastructure at You.com. Collaborating across teams to ensure data quality and enable AI capabilities.
Data Engineer developing and managing technology - based data solutions for clients in different industries in Greece. Participating in software development lifecycle within Agile team setting.
Data Architect leading design and governance of high - quality data architectures for clients. Collaborating with engineering teams and stakeholders to transform business challenges into scalable data solutions.