Senior Data Tech Lead/Architect designing and implementing AWS-based streaming data pipelines. Collaborating with teams to improve data quality and reconciliation workflows in a high-impact data platform.
Responsibilities
Provide technical solution discovery effort on any new capabilities or new functionality.
Assist PO with technical user stories to ensure healthy backlog features
Lead the development of **real-time data pipelines** using **AWS DMS**, **MSK**, Kafka or Glue Streaming and for CDC ingestion from multiple SQL Server sources (RDS/on-prem).
Build and optimize **streaming and batch data pipelines** using **AWS Glue (PySpark)** to validate, transform, and normalize data to **Iceberg and DynamoDB**.
Define and enforce **data quality, lineage, and reconciliation logic** with support for both streaming and batch use cases.
Integrate with **S3 Bronze/Silver layers** and implement efficient **schema evolution and partitioning strategies** using Iceberg.
Collaborate with architects, analysts, and downstream application teams to design **API and file-based egress** layers.
Implement **monitoring, logging**, and **event-based alerting** using CloudWatch, SNS, and EventBridge.
Mentor junior developers and enforce best practices for modular, secure, and scalable data pipeline development.
Requirements
6+ years of hands-on expert level data engineering experience in **cloud-based environments** (AWS preferred) with event driven implementation
Strong experience with **Apache Kafka / AWS MSK** including topic design, partitioning, and Kafka Connect/Debezium
Proficiency in **AWS Glue (PySpark)** and for both batch and streaming ETL
Working knowledge of **AWS DMS**, **S3**, **Lake Formation**, **DynamoDB**, and **Iceberg**
Solid grasp of **schema evolution**, **CDC patterns**, and **data reconciliation frameworks**
Experience with infrastructure-as-code (CDK/Terraform) and DevOps practices (CI/CD, Git)
Desenvolvedor Fullstack Java e Angular para atuar híbrido em soluções de produtos financeiros complexos. Integrando sistemas e desenvolvendo backend e frontend dentro de uma equipe ágil.
AI Software Engineer building AI - powered development tools in legal tech startup. Focused on enhancing automation and development workflows within specialized AI systems.
Product Engineer I at Lear Corporation supporting the launch of new seating systems components from conception to product launch. Involvement in integration of various components and engineering changes management.
Senior Full - Stack Engineer responsible for developing scalable web applications for DANA, leading frontend and backend engineering, optimizing performance, and mentoring junior developers.
Senior Full - Stack Developer building web solutions and APIs for Nordic customers. Collaborating in agile teams using modern technologies like Next.js and Azure.
Embedded Software Engineer developing and testing solutions for embedded systems in Pune, India. Collaborating with hardware engineers and utilizing C programming for software development.
Fullstack Staff Engineer shaping architecture and driving technical standards for DEUNA's payments platform. Collaborate across teams to ensure scalability and engineering excellence.
Lead .NET Software Developer guiding technical direction within financial services firm. Delivering high - quality applications for a growing team focused on Microsoft Azure technologies.