Data Engineer designing, building, and maintaining scalable data platforms and pipelines at Kyndryl. Utilizes Azure cloud solutions and adheres to modern software development practices.
About the role
- Data Engineer building and operating data pipelines for Qloo's platform. Collaborating with teams on data integrity and accessibility processes.
Responsibilities
- Design, develop, and maintain batch data pipelines using Python, Spark (EMR), and AWS Glue, loading data from S3, RDS, and external sources into Hive/Athena tables.
- Model datasets in our S3/Hive data lake to support analytics (Hex), API use cases, Elasticsearch indexes, and ML models.
- Implement and operate workflows in Airflow (MWAA), including dependency management, scheduling, retries, and alerting via Slack.
- Build robust data quality and validation checks (schema validation, freshness/volume checks, anomaly detection) and ensure issues are surfaced quickly with monitoring and alerts.
- Optimize jobs for cost and performance (partitioning, file formats, join strategies, proper use of EMR/Glue resources).
- Collaborate closely with data scientists, ML engineers, and application engineers to understand data requirements and design schemas and pipelines that serve multiple use cases.
- Contribute to internal tooling and shared libraries that make working with our data platform faster, safer, and more consistent.
- Document pipelines, datasets, and best practices so the broader team can easily understand and work with our data.
Requirements
- Bachelor’s degree in Computer Science, Software Engineering, or a related field, or equivalent practical experience.
- Experience with Python and distributed data processing using Spark (PySpark) on EMR or a similar environment.
- Hands-on experience with core AWS data services, ideally including:
- • S3 (data lake, partitioning, lifecycle management)
- • AWS Glue (jobs, crawlers, catalogs)
- • EMR or other managed Spark platforms
- • Athena/Hive and SQL for querying large datasets
- • Relational databases such as RDS (PostgreSQL/MySQL or similar)
- Experience building and operating workflows in Airflow (MWAA experience is a plus).
- Strong SQL skills and familiarity with data modeling concepts for analytics and APIs.
- Solid understanding of data quality practices (testing, validation frameworks, monitoring/observability).
- Comfortable working in a collaborative environment, managing multiple projects, and owning systems end-to-end.
Benefits
- Competitive salary and benefits package, including health insurance, retirement plan, and paid time off.
- The opportunity to shape a modern cloud-based data platform that powers real products and ML experiences.
- A collaborative, low-ego work environment where your ideas are valued and your contributions are visible.
- Flexible work arrangements (remote and hybrid options) and a healthy respect for work-life balance.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Junior Data Engineer responsible for developing and maintaining software programs and scripts under guidance. Collaborating with a software engineer to ensure compliance with policies and security standards.
Data Engineer developing scalable data pipelines and ETL processes at Walmart. Collaborating with cross - functional teams to ensure seamless data integration and quality.
Software Developer Specialist contributing to data processing solutions in big data framework at Verafin. Collaborating with cross - functional teams to ensure data accuracy and pipeline reliability in a flexible work environment.
Principal Data Engineer designing, building, and maintaining data pipelines for finance analytics at Northrop Grumman. Collaborating with engineers and finance analysts to ensure data accuracy and availability.
Senior Data Engineer responsible for migrating and modernising data platforms in banking. Rebuilding critical data platform with a focus on risk and core financial data flows.
Data Engineering Lead managing enterprise - scale data platforms using AWS, Snowflake, and Databricks in financial services. Leading data engineering teams and ensuring data governance.
AWS Data Engineer working in Gurugram to support data architecture and integration solutions. Collaborating and translating business needs into data models.
Senior Data Engineer handling data engineering responsibilities in hybrid setting for banking industry. Collaborating with cross - functional teams and maintaining data quality in Azure environments.
Data Management professional at Kyndryl involved in creating innovative data solutions and ensuring the seamless operation of complex data systems. Collaborating with teams to transform requirements into scalable database solutions.