Data Engineer at Concept Reply developing innovative data - driven solutions in IoT. Collaborating with teams to unlock the potential of data and cloud computing.
About the role
- Data Engineer creating and managing data pipelines for critical data solutions at S&P Global. Collaborating on enterprise-scale data processing in a supportive, innovative environment.
Responsibilities
- Design, develop, and maintain robust, scalable, and reliable data pipelines using Python, PySpark, and Databricks.
- Develop complex SQL queries, optimize data models, and manage large datasets within our PostgreSQL databases.
- Build and deploy ETL/ELT processes to ingest, transform, and load data from a wide variety of sources.
- Implement and manage CI/CD pipelines (e.g., Azure DevOps, Jenkins) for automated testing and deployment of data applications.
- Ensure data quality, reliability, and observability by implementing comprehensive logging, monitoring, and alerting.
- Collaborate with data scientists, analysts, and other engineers to understand data needs and deliver effective solutions.
- Contribute to the evolution of our platform standards and best practices for data engineering, code quality, and testing.
- Create and maintain clear technical documentation for data pipelines, architecture, and processes.
Requirements
- Bachelor’s degree in computer science, Engineering, or a related technical field.
- 4+ years of professional experience in a data engineering role.
- Expert-level programming skills in Python and its data ecosystem.
- Demonstrated expertise in large-scale data processing using PySpark and the Databricks platform.
- Advanced proficiency in SQL, with hands-on experience in database design and performance tuning, specifically with PostgreSQL.
Benefits
- Health & Wellness: Health care coverage designed for the mind and body.
- Flexible Downtime: Generous time off helps keep you energized for your time on.
- Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
- Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
- Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in-class benefits for families.
- Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer supporting and evolving data environment in cloud migration. Maintain and optimize existing databases while designing modern data solutions with cross - functional collaboration.
Senior Data Engineer responsible for data pipeline projects at Suprema Gaming. Focus on batch and streaming data solutions while collaborating with business teams.
Senior data leader managing the enterprise data architecture at Breakthru Beverage. Leading high - performing teams in data engineering and defining modern data strategies.
Data Engineer at Equinix implementing data architecture solutions for scalability and analytics. Collaborating with teams to design data pipelines and maintain data models for business objectives.
Data Warehouse Architect developing and optimizing robust data warehouse environments on SAP BW/4HANA. Critical for enabling advanced analytics and reporting across the organization.
Data Engineering Manager leading a new Data Engineering team in Bengaluru. Shaping the design and scaling of core data engineering practices across the organization.
Sr. ETL/Data Warehouse Lead at Huntington designing, developing, and supporting ETL and Data Warehousing framework. Analyzing systems based on specifications and providing technical assistance.
Senior Google Data Architect designing and delivering scalable data solutions on Google Cloud Platform. Collaborating across teams to shape target - state data architectures and influence enterprise data strategy.
Data Engineer developing scalable data lake solutions and optimizing data pipelines at U.S. Bank. Collaborating with teams to manage data governance and cloud migration activities.