Data Architect needed to define and evolve data architecture supporting scientific compute at EIT. Collaborate and lead in large - scale research environments for transformative scientific challenges.
About the role
- Data Engineer developing scalable data pipelines at Daikin Applied. Collaborating with teams to support advanced analytics and AI use cases using Databricks.
Responsibilities
- Design, build, and maintain ETL/ELT pipelines to ingest, transform, curate and store data from multiple sources
- Optimize data processing workflows for performance, reliability, and scalability
- Implement real-time and batch data processing using technologies like Apache Spark, Kafka, and Databricks
- Work with structured and unstructured data
- Implement data validation, cleansing, and monitoring to ensure high-quality datasets
- Implement data governance, security, and compliance policies (e.g., GDPR, CCPA)
- Maintain metadata management, data lineage, and documentation for data assets
- Deploy and manage data solutions on cloud platforms (Azure, Databricks)
- Develop and maintain documentation, data models, and technical standards
- Optimize query performance, cost efficiency, and storage utilization
- Monitor, troubleshoot, and resolve issues in production data pipelines and environments
- Stay current with the latest advancements in data engineering, cloud computing, and analytics technologies on the Databricks ecosystem
- Partner with data analysts, and software engineers to support analytics initiatives
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field
- 8+ years of Data Engineering, with a strong understanding of cloud-based data solutions
- At least 3 years hands-on experience building and delivering data products on Databricks
- Proven experience in data engineering and pipeline development on Databricks
- Hands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumption
- Deep expertise with the Databricks platform (SQL, Python and PySpark, Delta Lake, Unity Catalog, MLflow)
- Strong SQL and Python skills for data processing and data manipulation
- Strong problem-solving skills and an analytical mindset
- Excellent verbal and written communication skills, with the ability to explain technical concepts to non-technical audiences
- Extensive experience with data ingestion methodology including ADF
- Proficiency in Python, SQL, or Scala for data processing
- Experience with cloud data services (Azure Data Factory, Databricks)
- Hands-on experience with big data frameworks (Databricks, Apache Spark)
- Strong knowledge of data modeling, database optimization, and API-based data integration
- Proficiency in designing and implementing the Medallion Architecture on Databricks
- Experience with code repositories, CI/CD processes and release management
- Work visa sponsorship is not available for this position
Benefits
- Multiple medical insurance plan options + dental and vision insurance
- 401K retirement plan with employer contributions matching 100% of the first 3% of employee contributions and 50% on the next 2% of employee contributions
- Company provided life insurance + optional employee paid voluntary life insurance, dependent life coverage and voluntary accident coverage
- Short term and long-term disability
- 3 weeks of paid time off for new employees + 11 company paid holidays
- Vacation accrues on a monthly basis, unless applicable federal, state and local law requires a faster accrual
- Paid sick time in accordance of the federal, state and local law
- Paid parental leave and tuition reimbursement after 6 months of continuous service
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Engineering Data Coordinator leading a data engineering team in Azure and Databricks at Deroyque. Focusing on project management, quality assurance, and team development based in Campinas.
Data Migration Specialist managing ongoing Salesforce data quality initiatives for Abby Care. Executing and validating data migrations while ensuring data accuracy.
Data Migration Analyst responsible for supporting data migration and data quality initiatives at Abby Care, tackling family caregiving challenges.
Lead Data Engineer overseeing and managing the Data Engineering team. Developing ETL pipelines and ensuring data integrity within Cloud (Azure) infrastructure.
Data Engineer designing and optimizing data solutions for Qualco Intelligent Finance. Focus on data integrity, consistency, and reusability in analytics deliverables within a hybrid environment.
Data Architect leading data architecture and design for LifeByte's technology ecosystem. Collaborating with multiple teams to ensure robust data governance, compliance, and innovative strategies.
Data Engineer developing data solutions for major UK consumer brands. Collaborating with the Product & Technology team to meet diverse data requirements.
Senior Data Engineer building and managing data pipelines at Immediate Media. Collaborating with cross - functional teams to deliver robust data solutions and ensure data quality.
Data Engineer Team Lead at iKnowHow S.A leading Data Engineering team for outsourced projects. Driving scalable data solutions and fostering team collaboration with technical expertise.