Senior Microsoft Data Consultant specializing in Business Intelligence and Data Warehousing for Azure practice. Seeking career - oriented individuals with a commitment to deliver high - quality solutions.
CS
Hybrid Senior Cloud Data Engineer
Posted 28 minutes ago
About the role
- Senior Cloud Data Engineer responsible for operating and optimizing cloud-based data environments. Collaborating with analytics teams and specializing in AWS, Databricks, and Spark technologies.
Responsibilities
- Responsible for operating and optimizing our cloud-based data processing environment.
- Work with Databricks, AWS services, Spark, Unity Catalog, and Delta Lake to ensure efficient, secure, and reliable data pipelines and analytics workloads.
- Refines data transformations using PySpark and Spark SQL within notebooks.
- Leverages orchestration tools like Apache Airflow to automate data workflows.
- Participates in code reviews, testing, and documentation.
- Supports and troubleshoots Databricks jobs, Spark workloads, and AWS-based data processes
- Optimizes Databricks clusters and jobs for performance and cost.
- Works closely with data engineering and analytics teams to improve data quality.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or similar
- Master’s Degree will be considered as an asset
- 5+ years of experience in big data operations or cloud-based data engineering.
- Strong hands-on experience with AWS, Databricks, Delta Lake, and Apache Spark
- Proficient in Python, SQL, and PySpark
- Experience with CI/CD, version control, and release processes (AWS CodePipeline, Git)
- Experience with monitoring, debugging, and optimizing ETL/ELT and Spark workloads
- Knowledge of data governance frameworks and exposure to enterprise security or regulated environments will be considered as an asset
- **Competencies**
- Excellent problem-solving skills and attention to detail
- Strong communication skills and the ability to work collaboratively in a team environment
- Effective time management with ability to multi-task and prioritize work
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer designing, building, and supporting data integration solutions at Starr & Associates. Role includes working with client delivery teams across multiple projects and industries.
Technical leader in Data & Analytics projects at Keyrus. Responsible for designing modern architectures and ensuring technical execution alignment.
Data Platform Lead at TF Bank responsible for designing a cloud - native data platform. Overseeing architecture, governance, and data quality across the bank in Málaga.
Lead Data Platform architecting data infrastructure for Ankorstore’s wholesale operations. Managing data engineers and ensuring reliable data flows for business intelligence and insights.
Data Engineer specializing in Azure Databricks and healthcare data pipelines. Responsible for designing high - throughput ingestion pipelines and ensuring compliance with data standards.
Lead Cloud Data Engineer developing applications on Azure data platform initiative at LSEG. Collaborating with data teams to implement scalable and efficient data solutions.
Senior Enterprise Data Architect shaping data flow across Lundbeck by unifying enterprise data platforms. Leading architectural design and governance for scalable and sustainable data solutions.
Data Engineer supporting the development and implementation of enterprise - wide data governance practices at a climate technologies company focused on sustainability. Collaborating cross - functionally to enhance data quality and compliance processes.
Data Engineer III at Hanger, Inc. designing and maintaining data solutions using Microsoft Azure. Collaborating with stakeholders and optimizing ETL processes for enterprise analytics.