Senior Data Engineer developing data pipelines and infrastructure on Google Cloud Platform for WorkWhile's staffing marketplace. Collaborating with Data Science and Engineering teams to enhance data quality and availability.
About the role
- Senior Data Engineer developing high-impact data solutions in a collaborative financial team. Integrating data systems and ensuring performance with innovative technologies.
Responsibilities
- Design, develop and support robust data infrastructure solutions with a focus on scalability, security and efficiency.
- Lead the implementation and maintenance of ecosystems such as Trino, AWS Glue, Redshift, Airflow, Kafka and Spark on Kubernetes (K8s).
- Collaborate with engineering, data science and product teams to ensure seamless integration between systems.
- Diagnose and resolve performance, scalability and reliability incidents.
- Apply and promote best practices in architecture, data governance and monitoring.
Requirements
- Strong experience with data infrastructure ecosystems: Trino, AWS Glue, Terraform, CI/CD, Apache Airflow, Kafka and Spark on Kubernetes.
- Proven experience with cloud computing (AWS).
- Advanced proficiency in Python (including PySpark) and object-oriented programming.
- Demonstrated experience with distributed systems, data warehouses and large-scale ETL/ELT pipelines.
- Hands-on knowledge of data governance, security and storage (e.g., S3, Redshift).
- Analytical problem-solving skills with a focus on operational efficiency.
- Strong communication skills and experience working collaboratively.
- Preferred qualifications:
- Certifications such as AWS Certified Solutions Architect or similar.
- Familiarity with Trino for ad-hoc analytics.
- Experience with monitoring tools such as Datadog or New Relic.
- Knowledge of AI applied to data, especially RAG (Retrieval-Augmented Generation), for analytics and intelligent insight generation projects.
Benefits
- Competitive compensation based on experience
- Opportunities for career growth and participation in strategic projects
- Dynamic and challenging work environment
- Opportunity to work at a fast-growing company in the market.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Data Engineer developing data platforms for a consulting firm focused on quality solutions. Collaborating within a small team to deliver robust infrastructure and systems.
Senior Data Engineer designing and maintaining data pipelines within a fast - growing social impact startup. Collaborate cross - functionally to enhance products and analytics capabilities.
Senior Associate in data engineering at PwC focusing on designing robust data solutions. Leading complex data pipeline projects and collaborating with cross - functional teams to support automation and analytics.
Data Engineering & Warehousing Manager overseeing data engineering and warehousing operations at Hastings Insurance. Leading pipelines, platforms, and technical teams for enterprise data insights.
Senior Data Engineer delivering scalable data solutions in data engineering team at fintech startup. Building and maintaining data pipelines, collaborating with cross - functional teams for accurate data delivery.
Software Engineer contributing to Machine Learning initiatives and data infrastructure at AKASA. Working in a hybrid setup between South San Francisco and NYC.
Senior Enterprise Data Architect at Fresenius Kabi shaping data governance and architecture strategies across the enterprise for data - driven decision - making.
Database Engineer I focused on acquisition and integration of data into Data Lake. Responsible for management of databases and leading company’s data strategy.
Data Engineer at Logcomex, developing robust data pipelines and ensuring data quality for trade solutions. Collaborating with various teams to optimize data governance and performance.