Senior Data Architect responsible for building data infrastructure at Trexquant, integrating diverse datasets for research and simulation applications. Collaborating with teams to enhance data accessibility and quality.
About the role
- Data Engineer developing scalable data pipelines for RunBuggy's automotive logistics platform. Collaborate with cross-functional teams to unlock powerful insights and optimize data infrastructure.
Responsibilities
- Design, develop, and maintain scalable data pipelines and systems.
- Independently create and own new data capture/ETL’s for the entire stack and ensure data quality.
- Collaborate with data scientists, engineers, business leaders, and other stakeholders to understand data requirements and provide the necessary infrastructure.
- Create and contribute to frameworks that improve the effectiveness of logging data, triage issues, and resolution.
- Define and manage Service Level Agreements (SLA) for all data sets in allocated areas of ownership.
- Lead data engineering projects and determine the appropriate tools and libraries for each task.
- Implement data security and privacy best practices.
- Create and maintain technical documentation for data engineering processes.
- Work with cloud-based data storage and processing solutions (for example, Docker and Kubernetes).
- Build out and support a DAG orchestration cluster framework.
- Migrate workflows from batch processes to the DAG cluster via concurrent data flows.
- Data pipeline maintenance, including debugging code, monitoring, and incident response.
- Collaborate with engineering to enforce data collection and data contracts for API’s, databases, etc.
- Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field required; master’s degree preferred.
- 5+ years of experience in data engineering.
- Proficiency in Python and experience with data engineering libraries (e.g., Pandas).
- Experience with ETL processes and tools.
- Strong knowledge of relational and non-relational databases.
- Experience with cloud platforms (e.g., AWS, GCP, Azure).
- Excellent communication skills.
- Ability to work independently and lead projects.
- Experience with data warehousing solutions.
- Familiarity with data visualization tools (e.g., Tableau).
- Experience with building and managing DAG clusters (e.g. Airflow, Prefect).
- Ability to work with the following: JavaScript, Node.js, AngularJS, Java, and Java Spring Boot.
- Knowledge of machine learning and data science workflows.
- Ability to handle a variety of duties in a fast-paced environment.
- Excellent organizational skills, along with professionalism and diplomacy with internal and external customers/vendors.
- Ability to prioritize tasks and manage time.
- Ability to work under tight deadlines.
Benefits
- Highly competitive medical, dental, vision, Life w/ AD&D, Short-Term Disability insurance, Long-Term Disability insurance, pet insurance, identity theft protection, and a 401(k) retirement savings plan.
- Employee wellness program.
- Employee rewards, discounts, and recognition programs.
- Generous company-paid holidays (12 per year), vacation, and sick time.
- Paid paternity/maternity leave.
- Monthly connectivity/home office stipend if working from home 5 days a week.
- A supportive and positive space for you to grow and expand your career.
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Data Engineer responsible for developing data solutions and integrating systems for advanced analytics at Lilly. Focusing on data pipelines and solutions ensuring data quality and compliance.
Junior Data Engineer assisting with data - driven use - cases in the payment sector. Contributing to the establishment of a central data platform at S - Payment.
Senior Data Engineer leading tailored data - driven solutions delivery for public sector clients. Involves data transformation projects and building AI - powered tools for decision making.
Technical Lead in Data Engineering at Intentsify, building scalable applications for B2B marketing solutions. Leading a small team and making key technological decisions.
Working Student in Data Engineering supporting the development of an energy management app's data backbone across Europe. Collaborate with diverse teams to ensure data quality and optimization.
Senior Data Engineer at Minsait responsible for designing and maintaining data infrastructure. Ensuring efficient and secure data collection, storage, and processing across various sectors.
Senior Data Engineer developing and maintaining scalable data pipelines at Quality Digital. Ensuring data quality, security, and compliance with best practices while collaborating with data teams.
AI Data Engineer at Convatec designing and deploying data and AI workflows. Collaborating with AI Engineers and Data Scientists to maintain data pipelines and support analytics.
Data Engineer designing and developing data pipelines and infrastructure for processing and analyzing large data volumes at CIAL. Collaborating with data scientists to meet data requirements.