Linux/Middleware Analyst managing complex systems and optimizations. Collaborating in a hybrid environment to enhance performance and troubleshooting for operational excellence.
About the role
- Senior Data Tech Lead/Architect designing and implementing AWS-based streaming data pipelines. Collaborating with teams to improve data quality and reconciliation workflows in a high-impact data platform.
Responsibilities
- Provide technical solution discovery effort on any new capabilities or new functionality.
- Assist PO with technical user stories to ensure healthy backlog features
- Lead the development of **real-time data pipelines** using **AWS DMS**, **MSK**, Kafka or Glue Streaming and for CDC ingestion from multiple SQL Server sources (RDS/on-prem).
- Build and optimize **streaming and batch data pipelines** using **AWS Glue (PySpark)** to validate, transform, and normalize data to **Iceberg and DynamoDB**.
- Define and enforce **data quality, lineage, and reconciliation logic** with support for both streaming and batch use cases.
- Integrate with **S3 Bronze/Silver layers** and implement efficient **schema evolution and partitioning strategies** using Iceberg.
- Collaborate with architects, analysts, and downstream application teams to design **API and file-based egress** layers.
- Implement **monitoring, logging**, and **event-based alerting** using CloudWatch, SNS, and EventBridge.
- Mentor junior developers and enforce best practices for modular, secure, and scalable data pipeline development.
Requirements
- 6+ years of hands-on expert level data engineering experience in **cloud-based environments** (AWS preferred) with event driven implementation
- Strong experience with **Apache Kafka / AWS MSK** including topic design, partitioning, and Kafka Connect/Debezium
- Proficiency in **AWS Glue (PySpark)** and for both batch and streaming ETL
- Working knowledge of **AWS DMS**, **S3**, **Lake Formation**, **DynamoDB**, and **Iceberg**
- Solid grasp of **schema evolution**, **CDC patterns**, and **data reconciliation frameworks**
- Experience with infrastructure-as-code (CDK/Terraform) and DevOps practices (CI/CD, Git)
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Principal Engineer in Agentic AI creating AI agents and enhancing low - code development capabilities at VF Corporation. Leading technical architecture and governance standards for decentralized agent creation.
Software Engineer II at Leonardo DRS analyzing and developing fault isolation and troubleshooting procedures. Engaging in engineering investigations and design analysis with focus on reliability.
Software Engineer focusing on data infrastructure at Persona, building and maintaining data platforms for analytics applications. Collaborating closely with product and data science teams in a hybrid environment.
Senior Software Engineer developing resilience practices for a configurable identity platform. Collaborating with product teams to enhance performance and scalability in complex systems.
Engineering Lead Analyst responsible for architecting core Controls Technology platforms at Citi. Leading hands - on technical teams while driving innovation and engineering practices.
Join SCIO Automation as an Automation Software Engineer to develop and optimize Beckhoff TwinCAT systems. Collaborate on innovative solutions for Industry 4.0 processes based in Duisburg, Germany.
Software developer for process and logistics solutions using .NET technologies. Join SCIO Automation to advance smart Industry 4.0 initiatives.
Software Developer designing and implementing software solutions for Sentry, leveraging creativity and innovation in a full - stack environment.
Tech Lead driving technical direction and modern engineering practices at 77sol, a solar energy B2B2C platform in Brazil.