TR
TravelersSenior Data Engineer
Senior Associate Data Engineer contributing to Travelers' analytics landscape by building and operationalizing data solutions. Collaborating with teams to ensure reliable data delivery across the enterprise.
About the role
- Design, develop, and maintain robust and efficient ETL pipelines and processes on Databricks.
- Troubleshoot and resolve Databricks pipeline errors and performance issues.
- Maintain legacy SSIS packages for ETL processes.
- Troubleshoot and resolve SSIS package errors and performance issues.
- Optimize data flow performance and minimize data latency.
- Implement data quality checks and validations within ETL processes.
- Develop and maintain Databricks pipelines and datasets using Python, Spark and SQL.
- Migrate legacy SSIS packages to Databricks pipelines.
- Optimize Databricks jobs for performance and cost-effectiveness.
- Integrate Databricks with other data sources and systems.
- Participate in the design and implementation of data lake architectures.
- Implement DevOps best practices for data pipelines, including CI/CD, monitoring, observability, and automated testing.
- Integrate data ingestion from multiple sources (API, streaming, batch, databases) into centralized data platforms.
- Use Terraform/CloudFormation (or similar IaC tools) for provisioning Databricks clusters, cloud infrastructure, and networking components.
- Improve system performance and cost efficiency through monitoring, autoscaling, and cluster configurations.
- Provide mentorship and technical guidance to junior data engineers and collaborate with cross-functional teams.
- Participate in the design and implementation of data warehousing solutions.
- Support data quality initiatives and implement data cleansing procedures.
- Collaborate with business users to understand data requirements for department driven reporting needs.
- Maintain existing library of complex SSRS reports, dashboards, and visualizations.
- Troubleshoot and resolve SSRS report issues, including performance bottlenecks and data inconsistencies.
- Comfortable in entrepreneurial, self-starting, and fast-paced environment, working both independently and with our highly skilled teams.
- Communicate technical information clearly and concisely, both verbally and in writing.
- Document all development work and procedures thoroughly.
Requirements
- Bachelor's degree in computer science, Information Systems, or a related field.
- 7+ years of experience in data integration and reporting.
- Extensive experience with Databricks, including Python, Spark, and Delta Lake.
- Strong proficiency in SQL Server, including T-SQL, stored procedures, and functions.
- Experience with SSIS (SQL Server Integration Services) development and maintenance.
- Experience with SSRS (SQL Server Reporting Services) report design and development.
- Experience with data warehousing concepts and best practices.
- Experience with Microsoft Azure cloud platform and Microsoft Fabric desirable.
- Strong analytical and problem-solving skills.
- Excellent communication and interpersonal skills.
- Ability to work independently and as part of a team.
- Experience with Agile methodologies.
- Must be legally authorized to work in the United States.
Benefits
- Continuous learning opportunities
- Ability to work hybrid
- Mentorship programs
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
3I
3Core Systems, IncAWS Data Engineer
Data Engineer specializing in PySpark and Snowflake at 3Core Systems, Inc. Focused on building and optimizing data pipelines on AWS and Snowflake.
NL
NOVALINK SOLUTIONS LLCSalesforce Data Engineer
Salesforce Data Engineer serving as a subject matter expert in the State of Tennessee. Designing scalable data pipelines and collaborating on cross - agency initiatives.
FI
FindasenseSenior Data Engineer – Intelligence
Data Engineer Senior responsible for building data architecture and optimizing pipelines for Business Intelligence. Collaborating with analysts to develop insights using Power BI and Azure technologies.
Principal Data Engineer driving modernization from legacy systems to cloud - native platforms at Mastercard. Architecting and developing ETL platforms with AI integration and establishing data - driven strategies.
Principal Data Engineer modernizing cloud - native platforms for AI - powered solutions at Mastercard. Leading teams to enhance data processing efficiency and reliability across global operations.
IN
Data Engineer creating data pipelines for Santander's card transactions. Collaborating with an agile team in strategic projects involving Databricks and PySpark.
SA
Sabiá AdministraçãoSenior Data Engineer
Data Engineer designing, implementing, and maintaining data pipelines at Sabiá Gaming. Focused on high - quality data access and integration for enhanced decision - making.
MA
MassMutualQuantitative Data Engineer
Quantitative Data Engineer developing data solutions and automations for MassMutual's investment management. Working with data orchestration tools within a collaborative team environment.
Senior Data Engineer designing and scaling data infrastructure for analytics, machine learning, and business intelligence in a software supply chain security company.