Designs, builds, and maintains reliable, efficient and scalable data infrastructure for data collection, storage, transformation, and analysis.
Implements data orchestration pipelines, data sourcing, cleansing, augmentation, and quality control processes.
Works with business and technology collaborators to grasp current and future data infrastructure needs.
Designs, builds and maintains scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow; including those specifically tailored for machine learning workflows.
Builds, implements, and upholds current and upcoming data platforms such as data warehouses, repositories for structured and unstructured data.
Collaborates with Data Scientists and Engineers to create features and pre-process data for ML models and move data analysis models into production.
Designs and develops analytical tools, algorithms, data landscape modernization roadmaps, and programs to support Data Engineering activities like writing scripts and automating tasks.
Applies a variety of data interchange formats to ensure data requirements are met and continuously monitors data integrity across the organization.
Integrates machine learning algorithms into current production systems and workflows, taking into account compatibility with other systems, data sources, and APIs.
Builds and advocates for efficient utilization of data querying APIs to ensure seamless access to organizational data sources.
Evaluates, integrates, and manages tools and frameworks within the data engineering ecosystem, ensuring compatibility and efficiency in model development and deployment.
Designs and promotes data versioning and lineage tracking, including transparency and traceability for data used in ML model training and inference.
Requirements
Knowledge of database systems, data lakes, and NoSQL databases
Knowledge of data warehouse concepts and architectures (e.g., Synapse)
Familiarity with data quality and data modelling tools
Proficiency in using version control systems like Git for managing codebase
Experience with Cloud native data services such as PySpark, Scala, Azure Data Factory and Databricks
Practical experience with big data processing frameworks and techniques such as HDFS, MapReduce, Storage formats (Avro, Parquet), Stream processing
Experience with integrating to back-end/legacy environments
Knowledge of AI model deployment in production environments
Experience handling real-time data for AI Applications
Ability to build and deploy Data Ops and ML Ops Pipelines in Cloud-native environments
Benefits
health, dental, mental health, vision insurance
short- and long-term disability
life and AD&D insurance coverage
adoption/surrogacy and wellness benefits
employee/family assistance plans
retirement savings plans (including pension and employer matching contributions)
customizable benefits
paid time off including holidays, vacation, personal days, sick days
Senior Data Engineer at Clorox developing cloud - based data solutions. Leading data engineering projects and collaborating with business stakeholders to optimize data flows.
Data Engineer building solutions on AWS for high - performance data processing. Leading initiatives in data architecture and analytics for operational support.
Senior Data Engineer overseeing Databricks platform integrity, optimizing data practices for efficient usage. Leading teams on compliance while mentoring a junior Data Engineer.
Associate Data Engineer contributing to software applications development and maintenance using Python. Collaborating with teams for clean coding and debugging practices in Pune, India.
Lead Data Engineer responsible for delivering scalable cloud - based data solutions and managing cross - functional teams. Collaborating with global stakeholders and ensuring high - quality project execution in a fast - paced environment.
Data Engineer focusing on development and optimization of data pipelines in an insurance context. Ensuring data integrity and supporting data - driven decision - making processes.
Full Stack Data Engineer on a Central Engineering Portfolio Team in Chennai delivering curated data products and collaborating with data engineers and product owners.
Data Engineer designing and implementing data pipelines and services for Ford Pro analytics. Working with diverse teams and technologies to drive data - driven solutions.
Data Engineer developing best - in - class data platforms for ClearBank with a focus on data insights and automation. Collaborating closely with stakeholders and supporting data science initiatives.
Data Engineer operating cloud - based data platform for Business Intelligence and Data Science. Collaborating on data architectures and ETL processes for Sparkassen - Finanzgruppe.