Design and implement robust data pipelines using Databricks and integrate these with Delta Lake for efficient data storage and management.
Manage end-to-end data workflows from ingestion to insights, utilizing FiveTran for seamless data integration and GCP for scalable cloud solutions.
Administer Databricks environments, ensuring optimal configuration, security, and performance across all data operations.
Utilize Unity Catalog to manage data governance across all Databricks workspaces, ensuring compliance with data privacy and security policies.
Develop and maintain scalable and efficient data models and architecture, supporting the strategic goals of the organization.
Collaborate with data scientists and analysts to deploy machine learning models and complex analytical projects.
Requirements
Bachelor’s degree in computer science, engineering, information technology or related field or four (4) years of relevant experience in lieu of degree.
Minimum of five (5) years of experience [nine (9) years for non-degreed candidates] in a data engineering role with significant exposure to Databricks, Delta Lake, and cloud platforms like GCP.
Proven expertise in Databricks administration and managing large-scale data environments.
Strong experience with SQL, Python, and other scripting languages commonly used in data engineering.
Familiarity with Fivetran or similar data integration tools and understanding of ETL processes.
Data Engineer building solutions on AWS for high - performance data processing. Leading initiatives in data architecture and analytics for operational support.
Senior Data Engineer overseeing Databricks platform integrity, optimizing data practices for efficient usage. Leading teams on compliance while mentoring a junior Data Engineer.
Associate Data Engineer contributing to software applications development and maintenance using Python. Collaborating with teams for clean coding and debugging practices in Pune, India.
Lead Data Engineer responsible for delivering scalable cloud - based data solutions and managing cross - functional teams. Collaborating with global stakeholders and ensuring high - quality project execution in a fast - paced environment.
Data Engineer focusing on development and optimization of data pipelines in an insurance context. Ensuring data integrity and supporting data - driven decision - making processes.
Data Engineer designing and implementing data pipelines and services for Ford Pro analytics. Working with diverse teams and technologies to drive data - driven solutions.
Full Stack Data Engineer on a Central Engineering Portfolio Team in Chennai delivering curated data products and collaborating with data engineers and product owners.
Data Engineer developing best - in - class data platforms for ClearBank with a focus on data insights and automation. Collaborating closely with stakeholders and supporting data science initiatives.
Data Engineer operating cloud - based data platform for Business Intelligence and Data Science. Collaborating on data architectures and ETL processes for Sparkassen - Finanzgruppe.
Data Engineer at Love, Bonito optimizing data pipelines and ensuring data quality for analytics. Collaborating on data architecture, operations, and compliance for effective data management.