Data Engineer at Concept Reply developing innovative data - driven solutions in IoT. Collaborating with teams to unlock the potential of data and cloud computing.
About the role
- Senior Data Engineer designing and maintaining cloud data solutions using Spark and Databricks. Collaborating with teams and clients to optimize data infrastructure and insights delivery.
Responsibilities
- Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and Databricks.
- Gather and analyze data requirements from business stakeholders and identify opportunities for data-driven insights.
- Build and optimize data pipelines for data ingestion, processing, and integration using Spark and Databricks.
- Ensure data quality, integrity, and security throughout all stages of the data lifecycle.
- Collaborate with cross-functional teams to design and implement data models, schemas, and storage solutions.
- Optimize data processing and analytics performance by tuning Spark jobs and leveraging Databricks features.
- Provide technical guidance and expertise to junior data engineers and developers.
- Stay up-to-date with emerging trends and technologies in cloud computing, big data, and data engineering.
- Contribute to the continuous improvement of data engineering processes, tools, and best practices.
Requirements
- Bachelor's or Master's degree in computer science, engineering, or a related field.
- 10+ years of experience as a Data Engineer, Software Engineer, or similar role, with a focus on building cloud-based data solutions.
- Strong experience with cloud platforms such as Azure or AWS.
- Proficiency in Apache Spark and Databricks for large-scale data processing and analytics.
- Experience in designing and implementing data processing pipelines using Spark and Databricks.
- Strong knowledge of SQL and experience with relational and NoSQL databases.
- Experience with data integration and ETL processes using tools like Apache Airflow or cloud-native orchestration services.
- Good understanding of data modeling and schema design principles.
- Experience with data governance and compliance frameworks.
- Excellent problem-solving and troubleshooting skills.
- Strong communication and collaboration skills to work effectively in a cross-functional team.
- Relevant certifications in cloud platforms, Spark, or Databricks are a plus.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer creating and managing data pipelines for critical data solutions at S&P Global. Collaborating on enterprise - scale data processing in a supportive, innovative environment.
Data Engineer supporting and evolving data environment in cloud migration. Maintain and optimize existing databases while designing modern data solutions with cross - functional collaboration.
Senior Data Engineer responsible for data pipeline projects at Suprema Gaming. Focus on batch and streaming data solutions while collaborating with business teams.
Senior data leader managing the enterprise data architecture at Breakthru Beverage. Leading high - performing teams in data engineering and defining modern data strategies.
Data Engineer at Equinix implementing data architecture solutions for scalability and analytics. Collaborating with teams to design data pipelines and maintain data models for business objectives.
Data Warehouse Architect developing and optimizing robust data warehouse environments on SAP BW/4HANA. Critical for enabling advanced analytics and reporting across the organization.
Data Engineering Manager leading a new Data Engineering team in Bengaluru. Shaping the design and scaling of core data engineering practices across the organization.
Sr. ETL/Data Warehouse Lead at Huntington designing, developing, and supporting ETL and Data Warehousing framework. Analyzing systems based on specifications and providing technical assistance.
Senior Google Data Architect designing and delivering scalable data solutions on Google Cloud Platform. Collaborating across teams to shape target - state data architectures and influence enterprise data strategy.