Design, build, and maintain data ingestion pipelines that consolidate customer profile data for a real-time AI-driven decisioning application
Develop and optimize ETL processes using Azure Databricks and Azure Data Factory to source, transform, and load data from multiple upstream systems and enterprise data lakes
Ensure data quality, consistency, and reliability across batch and near real-time data flows
Manage deployment and version control of pipelines using GitLab and collaborate on code development via VSCode with GitHub Co-pilot
Monitor, troubleshoot, and optimize scheduled jobs in Control-M
Support ingestion into Postgres (for decisioning reference tables) and Cassandra (for near real-time attributes)
Work closely with other engineers, analysts, and application teams to ensure timely delivery of accurate customer data
Requirements
Bachelor's Degree Computer Science or Engineering (Required)
2-4+ years technical engineering experience (Required)
Strong SQL skills with proven experience in relational database systems (preferably PostgreSQL)
Hands-on experience with Azure Databricks (PySpark, SparkSQL, Python) for building and maintaining data pipelines
Proficiency with Azure Data Factory for pipeline orchestration and data integration
Experience with batch ETL processes at scale, including data cleansing, transformation, and performance optimization
Familiarity with Cassandra or other NoSQL databases for handling near real-time data
Version control and CI/CD using GitLab (or similar tools)
Job scheduling and monitoring experience (Control-M or equivalent)
Proficient in Python for ETL scripting and automation
Experience working in agile teams using Jira for sprint planning, backlog tracking, and defect management
Ability to work independently while maintaining clear communication and collaboration with stakeholders
Software Engineer on the Motion Planning team at Zoox improving driving behavior of robots. Collaborating on algorithmic solutions for complex driving scenarios in urban environments.
Engenharia Specialist responsible for subsea maintenance planning and control at BIP. Fostering operational efficiency and compliance through strategic technical support and documentation management.
Senior iOS Software Engineer developing medical mobile applications at Insulet. Collaborating with teams to build world - class software for diabetes management.
Senior Automation Engineer leading automation projects at Etex. Focused on improving operational excellence and technological innovation in industrial processes.
Full Stack Engineer seeking to build the Opik product for AI applications at Comet. Engaging in fast - paced development and collaboration with AI researchers and ML engineers.
Software Developer for Kentro working on payroll data accuracy and compliance for military contracts. Required to write queries, create test plans, and ensure regulatory compliance.
Lead Full Stack Engineer integrating a team of engineers, developers and analysts for global operations. Focused on building tools and processes for decision making and technology integration.
Combat Coder developing full stack applications for Leidos' Sentinel program supporting the US Air Force. Integrating data sources and interfaces in resource constrained environments.