Data Engineer managing and organizing datasets for AI models at Walaris, developing AI - driven autonomous systems for defense and security applications.
About the role
- Senior Software Engineer developing scalable data processing solutions for data engineering team. Implementing data models and building ETL pipelines with Python, Spark, and SQL.
Responsibilities
- Design and develop scalable data processing solutions using Spark and PySpark
- Implement robust data models and optimize SQL queries
- Collaborate with data analysts and stakeholders
- Build and maintain ETL pipelines leveraging Python, Hive, and SQL
- Conduct thorough code reviews, performance tuning, and debugging
- Monitor, troubleshoot, and resolve issues in production data workflows
- Document technical processes, data models, and workflow architectures
- Stay updated with industry trends in big data technologies
Requirements
- 2 - 4 years of experience in software development
- Advanced proficiency in Spark and PySpark
- Strong knowledge of SQL (basic and advanced)
- Expertise in Python programming for data processing
- Experience with Hive for data warehousing solutions
- Solid understanding of data modelling fundamentals
- Ability to design and optimize ETL pipelines
- Hands-on experience with large-scale data processing
- Proficient in performance tuning of SQL and Spark jobs
- Familiarity with distributed computing concepts
- Competence in debugging and troubleshooting data workflows
- Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field
- Relevant certifications in big data technologies or data engineering are advantageous
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Data Engineer designing and maintaining data pipelines at Black Semiconductor. Collaborating with process, equipment, and IT teams to support manufacturing analytics and decision - making.
Junior Data Engineer role focusing on Business Intelligence and Big Data at Avanade. Collaborating on data analysis and SQL queries in a supportive learning environment.
GCP Data Engineer designing and developing data processing modules for Ki, an algorithmic insurance carrier. Working closely with multiple teams to optimize data pipelines and reporting.
Data Engineer at Securian Financial optimizing scalable data pipelines for AI and advanced analytics. Collaborating with teams to deliver secure and accessible data solutions.
IT Data Engineering Co‑Op at BlueRock Therapeutics supports development of scientific data systems. Collaboration on data workflows and foundational AWS data engineering tasks.
Data Engineer I building and operationalizing complex data solutions for Travelers' analytics using Databricks. Collaborating within teams to educate end users and support data governance.
Data Engineer shaping modern data architecture to drive golf’s digital transformation. Collaborating with teams to enhance data pipelines and insights for customer engagement and revenue growth.
Staff Data Engineer overseeing complex data systems for CITY Furniture. Responsible for architecting and optimizing data ecosystems in a hybrid work environment.
Data Engineer strengthening data platform team at Samba TV to improve data analytics and reporting capabilities. Building on AWS, Databricks, BigQuery, and Snowflake technology.