Founding Staff Data Engineer building and leading data engineering team for AI - driven art valuation platform. Establishing architecture and standards for data systems and pipelines.
About the role
- Sr. Lead Data Engineer implementing data platforms with expertise in Databricks/AWS EMR. Collaborating with cross-functional teams to enhance data-driven initiatives.
Responsibilities
- Lead the design and implementation of distributed data processing workloads using Python, including data access from relational databases and cloud storage technologies.
- Ensure performance, security, and reliability; coordinate with application teams to define database design.
- Collaborate with cross-functional teams to support data-driven initiatives.
- Mentor junior team members and promote best practices.
- Oversee maintenance and troubleshooting of distributed data processing systems.
- Drive innovation by evaluating and integrating new technologies.
- Produce system design documents and participate in technical walkthroughs.
- Perform application and system performance tuning and troubleshoot performance issues.
- Effectively interact with global customers, business users, and IT staff.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent work experience.
- 10+ years of IT experience in application support or development.
- Design & build ingestion pipelines using ETL/ELT and schema-on-read in data lake technologies.
- Experience in designing transactional systems, data warehouses, data lakes, and data integrations within a big data ecosystem, particularly leveraging cloud technologies.
- Provide technical expertise in the design and implementation of data ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and large-scale data analytics tools.
- Strong experience in managing distributed data processing workloads, including configurations.
- Experience in working with multi-threaded, high-performance, low-latency messaging systems.
- Experience in AWS cloud-based technologies.
- Experience using system tools, source control systems, utilities, and third-party products.
- Excellent communication skills, with strong verbal and writing proficiencies.
Benefits
- Health & Wellness: Health care coverage designed for the mind and body.
- Flexible Downtime: Generous time off helps keep you energized for your time on.
- Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
- Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
- Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
- Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Data Engineer responsible for developing, maintaining ETL processes and integrating data solutions. Collaborating with teams on data quality and cloud migration initiatives.
Data Engineer optimizing data architectures and pipelines at Nexu. Focused on building reliable and efficient data flows while collaborating with cross - functional teams.
Senior Software Engineer designing and maintaining scalable data solutions for restaurant tech industry at SpotOn. Collaborating with cross - functional teams to enhance reporting and analytics platforms.
Data Architect needed to define and evolve data architecture supporting scientific compute at EIT. Collaborate and lead in large - scale research environments for transformative scientific challenges.
Engineering Data Coordinator leading a data engineering team in Azure and Databricks at Deroyque. Focusing on project management, quality assurance, and team development based in Campinas.
Data Migration Specialist managing ongoing Salesforce data quality initiatives for Abby Care. Executing and validating data migrations while ensuring data accuracy.
Data Migration Analyst responsible for supporting data migration and data quality initiatives at Abby Care, tackling family caregiving challenges.
Lead Data Engineer overseeing and managing the Data Engineering team. Developing ETL pipelines and ensuring data integrity within Cloud (Azure) infrastructure.
Data Engineer designing and optimizing data solutions for Qualco Intelligent Finance. Focus on data integrity, consistency, and reusability in analytics deliverables within a hybrid environment.