Data Engineer designing, implementing, and optimizing data pipelines for DeepLight AI. Collaborating closely with a multidisciplinary team to analyze large - scale data.
About the role
- Data Engineer building scalable data systems for e-commerce use cases. Focus on automation, pipeline orchestration, and leveraging cloud services to improve data architecture.
Responsibilities
- Build and maintain scalable data pipelines to process and integrate e-commerce data from multiple sources
- Develop production-ready Python scripts for API consumption, web scraping, and data transformation
- Design and implement cloud-based data architecture using AWS services such as S3, EC2, and Lambda
- Orchestrate and monitor scheduled data collection workflows using Apache Airflow
- Evaluate and implement tools for high-performance data transformation and analysis (e.g., Polars, DuckDB, PySpark)
- Collaborate with data analysts to deliver accessible, well-structured datasets for reporting and advanced analytics
- Identify opportunities to improve data quality, reliability, and performance across all data collection endpoints
Requirements
- Proficiency in programming languages such as Java and Python
- Hands-on experience with SQL and database design
- Previous experience as a Data Engineer or in a similar role
- Strong technical knowledge of web data collection architectures, including proxy and header management
- Excellent numerical and analytical skills
- Willingness to learn and adapt to new tools and technologies
- Proactive, curious, and highly tenacious, with a strong drive to grow and stay ahead of industry trends
- Comfortable navigating a fast-paced, ambiguous environment with a high degree of independence
- A Bachelor’s degree in a quantitative field (e.g., Computer Science, Engineering, Information Systems) is preferred
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer designing and maintaining scalable ETL pipelines at Satori Analytics. Collaborating with teams to deliver high - quality analytics solutions across various industries.
Data Architect responsible for defining enterprise data architecture on AWS and Databricks Lakehouse platforms. Enabling scalable data lakes and enterprise analytics for financial services organizations.
Data Platform Operations Support leading data engineering strategy across projects for EXL. Driving innovation and optimization while collaborating with various teams in the organization.
Manager II leading data engineering projects at Navy Federal Credit Union. Overseeing data governance and quality initiatives while managing engineering teams in a hybrid work environment.
Senior Data Engineer building and maintaining data pipelines for cloud and AI solutions at Qodea. Collaborating with ML engineers and focusing on reliability and performance in a cloud - native environment.
Principal Data Engineer responsible for architecting scalable data pipelines and building high - quality data foundations. Collaborating closely with experts to ensure data readiness for advanced analytics.
Senior Data Engineer at Qodea designing scalable data pipelines and infrastructure. Delivering solutions utilizing cutting - edge tools and collaborating closely with teams for impactful results.
Senior Data Engineer designing and maintaining data pipelines for Qodea's global technology solutions. Collaborating with teams to ensure data quality and governance across platforms.
Product Director managing Target's Customer Data Platform. Leading strategy, financials, and team development to enhance guest experience through data - driven initiatives.