Senior Data Engineer responsible for migrating and modernising data platforms in banking. Rebuilding critical data platform with a focus on risk and core financial data flows.
About the role
- Senior Data Engineer designing complex data systems for Opea, focusing on cloud and big data technologies. Leading a team and optimizing data processing solutions.
Responsibilities
- Design and implement complex, scalable data systems using technologies such as cloud computing, big data, streaming, machine learning and AI, defining strategies for data storage, processing and analysis. Incorporate Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) and vector databases for intelligent data processing.
- Lead and inspire the data engineering team by sharing knowledge, defining standards and best practices, and fostering the team's technical growth.
- Identify and resolve complex problems related to data systems, architecture, performance and scalability, using monitoring tools, log analysis and code debugging.
- Create and implement innovative solutions to optimize data systems, leveraging emerging technologies such as machine learning, data lakes, real-time data streaming and big data analytics.
- Communicate technical solutions clearly and concisely to stakeholders, managers, developers and other teams, influencing strategic decisions and helping align the data strategy with company objectives.
Requirements
- Deep proficiency in Python and data processing tools such as Apache Spark, Kafka, Flink or equivalents.
- Knowledge of LLMs, fine-tuning AI models and using RAG to improve search and information retrieval. Proficiency with Amazon Bedrock and/or Azure OpenAI services.
- Experience with data system architecture, big data (Hadoop, Hive, HBase), streaming (Kafka, Kinesis) and databases (SQL, NoSQL).
- Advanced knowledge of data modeling, data processing and data visualization tools such as SQL, NoSQL, Tableau, Power BI, etc.
- Advanced experience building Data Lakes and Data Warehouses using tools like AWS Athena, Amazon Redshift, Amazon S3, AWS Glue, Airbyte, dbt and PostgreSQL.
- Deep understanding of machine learning concepts and applying ML techniques within data systems.
- Bachelor's degree in Computer Science, Statistics, Mathematics or a related field.
- Technical English for reading, writing and professional communication.
Benefits
- Health and Dental plan
- Meal and Food vouchers
- Transportation voucher
- TotalPass
- Life Insurance
- Childcare/Babysitter assistance
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineering Lead managing enterprise - scale data platforms using AWS, Snowflake, and Databricks in financial services. Leading data engineering teams and ensuring data governance.
AWS Data Engineer working in Gurugram to support data architecture and integration solutions. Collaborating and translating business needs into data models.
Senior Data Engineer handling data engineering responsibilities in hybrid setting for banking industry. Collaborating with cross - functional teams and maintaining data quality in Azure environments.
Data Management professional at Kyndryl involved in creating innovative data solutions and ensuring the seamless operation of complex data systems. Collaborating with teams to transform requirements into scalable database solutions.
Software Engineer designing and developing scalable data processing applications on cloud infrastructure for Thomson Reuters. Collaborating with Data Analysts on AI - enabled solutions for data management and insight generation.
Manager of Data Platform overseeing AWS cloud infrastructure and Snowflake data warehouses for Thomson Reuters. Leading the design and implementation of data processing applications in a hybrid role located in Bengaluru.
Senior Data Engineer designing scalable data pipelines and solutions for Enterprise Data Lake at Thomson Reuters. Collaborating across teams to ensure efficient data ingestion and accessibility.
Senior Data Engineer at Technis developing scalable data pipelines and solutions for innovative connected spaces products. Collaborating within a cross - functional team to deliver high - quality data - driven outcomes.
Architect leading initiatives in data platform engineering at F5, solving challenges in unstructured data and enhancing governance frameworks.