Data Management professional at Kyndryl involved in creating innovative data solutions and ensuring the seamless operation of complex data systems. Collaborating with teams to transform requirements into scalable database solutions.
CS
Hybrid Senior Cloud Data Engineer
Posted last month
About the role
- Senior Cloud Data Engineer responsible for operating and optimizing cloud-based data environments. Collaborating with analytics teams and specializing in AWS, Databricks, and Spark technologies.
Responsibilities
- Responsible for operating and optimizing our cloud-based data processing environment.
- Work with Databricks, AWS services, Spark, Unity Catalog, and Delta Lake to ensure efficient, secure, and reliable data pipelines and analytics workloads.
- Refines data transformations using PySpark and Spark SQL within notebooks.
- Leverages orchestration tools like Apache Airflow to automate data workflows.
- Participates in code reviews, testing, and documentation.
- Supports and troubleshoots Databricks jobs, Spark workloads, and AWS-based data processes
- Optimizes Databricks clusters and jobs for performance and cost.
- Works closely with data engineering and analytics teams to improve data quality.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or similar
- Master’s Degree will be considered as an asset
- 5+ years of experience in big data operations or cloud-based data engineering.
- Strong hands-on experience with AWS, Databricks, Delta Lake, and Apache Spark
- Proficient in Python, SQL, and PySpark
- Experience with CI/CD, version control, and release processes (AWS CodePipeline, Git)
- Experience with monitoring, debugging, and optimizing ETL/ELT and Spark workloads
- Knowledge of data governance frameworks and exposure to enterprise security or regulated environments will be considered as an asset
- **Competencies**
- Excellent problem-solving skills and attention to detail
- Strong communication skills and the ability to work collaboratively in a team environment
- Effective time management with ability to multi-task and prioritize work
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Manager of Data Platform overseeing AWS cloud infrastructure and Snowflake data warehouses for Thomson Reuters. Leading the design and implementation of data processing applications in a hybrid role located in Bengaluru.
Software Engineer designing and developing scalable data processing applications on cloud infrastructure for Thomson Reuters. Collaborating with Data Analysts on AI - enabled solutions for data management and insight generation.
Senior Data Engineer designing scalable data pipelines and solutions for Enterprise Data Lake at Thomson Reuters. Collaborating across teams to ensure efficient data ingestion and accessibility.
Senior Data Engineer at Technis developing scalable data pipelines and solutions for innovative connected spaces products. Collaborating within a cross - functional team to deliver high - quality data - driven outcomes.
Architect leading initiatives in data platform engineering at F5, solving challenges in unstructured data and enhancing governance frameworks.
Data Architect designing and implementing data architectures supporting analytics and ML for federal clients. Collaborating with teams to translate mission needs into robust data solutions.
IT Data Engineer developing data pipelines and integrations for Scanfil Group's global IT organization. Collaborating across teams to enhance data solutions and reporting capabilities.
Data Engineer developing Azure data solutions at PwC New Zealand. Responsibilities include data quality monitoring, pipeline development, and collaboration with stakeholders in a supportive environment.
Senior Data Engineer designing and implementing the Enterprise Data Platform at Stellix. Focusing on analytics and insights with a growth path to Principal Data Engineer or Data Architect.