Data Engineer developing and evolving BI platform for Thomson Reuters in India. Building architectural solutions and collaborating in all development lifecycle aspects.
About the role
- Data Engineer responsible for designing and maintaining enterprise data warehouse for various projects. Collaborating with stakeholders to ensure efficient data flow and integration.
Responsibilities
- Design, develop, and maintain scalable Synapse pipelines for data ingestion, processing, and transformation in Azure data platform.
- Develop and optimize T-SQL and Python-based data processing scripts and integrate them with Synapse pipelines for ETL tasks and data transformations.
- Design and develop mapping dataflow solutions for complex data transformations.
- Develop and maintain data storage solutions, ensuring efficient data retrieval and storage on Azure Data Lake and databases.
- Implement and maintain API integrations, including REST APIs, for seamless data flow between systems.
- Design and test the end-to-end process for data ingestion, data transformation, data cleansing, data delivery, data quality and data timeliness.
- Implement and ensure data security and privacy measures, complying with industry standards and regulations.
- Monitor and optimize the performance of Azure data services, Azure Synapse pipelines, Function Apps, and Logic Apps.
- Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
Requirements
- Bachelor's degree or equivalent in the field of Computer Science or Engineering
- 3+ years of Proven experience as a Data Engineer, specifically working with Microsoft Azure technologies.
- Proficient in T-SQL, Pyspark, Mapping Dataflow and Python programming language
- Hands-on experience with Azure data services, including Azure Data Factory, Azure Data Lake, Azure Function App, Azure Logic App, Azure SQL Database, Azure Synapse Analytics, mapping dataflow, and Event Hub.
- Understanding of data warehousing concepts and best practices.
- Experience with Data modeling.
- Experience in working with structured, semi-structured and unstructured data.
- Excellent written and verbal communication skills
- Excellent multi-tasking and organizational skills
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer responsible for building and maintaining AWS Lakehouse infrastructure for trade contractors at Remarcable. Focused on clean data architecture and AI/ML data infrastructure.
Senior Data Engineer joining Reconomy to design, build and optimize data pipelines for an international circular economy specialist.
Data Engineer/Analyst maintaining and improving data infrastructure for Braiins. Collaborating with technical and business teams to ensure reliable data flows and insights.
Medior Data Engineer handling Azure migrations for a major urban mobility client. Focused on data pipeline development and ensuring platform reliability with cutting - edge technologies.
Developing ML and computer vision solutions for cutting - edge autonomous vehicle dataset pipeline at Mobileye. Collaborating across teams for data curation and advanced perception algorithms.
Data Migration Lead in a hybrid role managing data migration for a major transformation programme in the media sector. Collaborating with various teams to ensure data integrity and successful migration.
Consultant ML & DataOps at Smile integrating data science projects for major clients. Designing MLOps solutions and enhancing data governance in a collaborative environment.
Data Engineer developing and maintaining data pipelines for Coolbet’s analytical services. Working within an Agile framework to ensure data reliability and efficiency.
API Data Engineer developing innovative data - driven solutions and advancing data architecture for AI Control Tower. Building and integrating APIs and data pipelines to support organizational needs.