Lead Data Engineer responsible for designing and implementing enterprise data platform at Ness Digital Engineering. Collaborating across teams to optimize performance and ensure data integrity in engineering processes.
Responsibilities
Lead the data architecture and engineering strategy, ensuring alignment with business and technology roadmaps.
Design and implement data models, pipelines, and data integration frameworks across multiple platforms.
Partner with stakeholders to translate business requirements into scalable data solutions.
Performance Optimization: Optimize data pipelines for performance, scalability, and reliability, including query tuning and resource management within Snowflake.
Drive adoption of best practices in data engineering design patterns and modern cloud architectures.
Data Quality Assurance: Implement and monitor data validation procedures to ensure data accuracy and consistency across systems.
Collaboration and Communication: Work closely with project managers, data architects, and business analysts to align project milestones and deliverables with business goals.
Mentor and guide data engineering teams (onsite & offshore) to deliver high-quality outcomes.
Ensure compliance with data governance, security, and privacy standards.
Documentation: Create and maintain detailed documentation of data pipelines, data flow diagrams, and transformation logic.
Issue Resolution: Troubleshoot and resolve issues related to data pipelines, including job failures and performance bottlenecks.
Requirements
Bachelor’s degree in Computer Science, Information Technology, or a related field
8+ years of experience in data engineering with a strong focus on Data Architecture, Data Modelling (Conceptual, Logical, Physical), ELT processes and data pipeline development.
Hands-on experience with Snowflake cloud data platform, including data sharing, secure views, and performance optimization.
Proficiency in SQL and familiarity with data integration and ETL/ELT tools.
AWS Technologies (Glue, EMR)
Python for data engineering workflows
Strong understanding of data engineering design patterns
Strong problem-solving skills and the ability to work independently to meet deadlines.
Excellent communication skills for effectively interacting with technical and non-technical stakeholders.
Senior Data Engineer in Data Ingestion team at Novo Nordisk, designing scalable data solutions for analytics, AI, and research. Building robust applications and pipelines to support operational use cases.
Data Engineer delivering data for Financial Crime Prevention teams and supporting consistent data layer. Collaborating with multiple teams and defining expected solution details.
Senior consultant at Infosys designing enterprise data solutions and leading technical teams. Collaborating across business pillars in a high - growth consulting environment focused on analytics and data strategy.
Data Engineer I developing data services with Azure technology for global risk management insights. Collaborating with teams to optimize data processes and ensure quality standards.
Senior Data Engineer designing and overseeing data pipelines in Databricks on AWS. Responsible for data quality and performance for enterprise analytics and AI workloads.
AI Data Pipeline Engineer designing and operating high - throughput systems for petabyte - scale data delivery. Collaborating across teams to ensure data flows into AI workloads efficiently.
AWS Data Engineer role focusing on AWS technologies in Gurugram, Haryana, India. Responsibilities include AWS data engineering tasks and collaboration with team members.
Data Engineer implementing innovative technology for various domains at Quantexa. Building data pipelines and providing insights to help clients solve complex business problems.