Data Engineer at Pruna AI merging data engineering, analytics engineering, and revenue operations. Working on intelligent automation and analytics for accessible and sustainable AI.
About the role
- Senior Data Engineer optimizing ETL/ELT pipelines at Asahi Kasei. Evaluate programming concepts and support data science projects while ensuring solution stability in a hybrid work setup.
Responsibilities
- Evaluate and improve T-SQL, MDX, DAX, HiveQL programming concepts such as queries, stored procedures, functions, temporary tables, parameterization, complex joins and groupings
- Develop and optimize ETL/ELT pipelines, to load data from on premise and online systems
- Ensure data solutions stability and performance optimization
- Conduct data warehouse model design, development and support
- Prepare, cleanse, validated datasets for data science purpose
- Assist data troubleshooting, data featuring and data discovery
- Develop tools to automate development and monitoring process
- Develop python algorithms for data processing
- Support Data science environment, assists data science projects
- Manage time effectively to ensure that projects are delivered on schedule
- Provide on-going maintenance and support of existing and new data solutions
- Support solution automation and CI/CD
Requirements
- Bachelor’s Degree in Computer Science / Information technology or related field AND five (5) years as Data Engineer, Big Data Engineer, Data Architect, SQL Developer, Database Developer, or related
- 5 years’ experience with: Developing business intelligence solutions including data integration, data schema development, data pipelines, modeling and reporting/analytics Database design principles, data modeling, partitioning, and data warehouse Python, and Shell scripting SQL writing and query tuning, and query performance optimization data analysis, data modeling, data migration, computer programming, and problem-solving
- 4 years’ experience with: Data validation, cleansing, featuring: Pandas, Spark dataframes, and DQ solutions
- 3 years’ experience with: CI/CD. CDC
- 2 years’ experience with: big data pipeline development, monitoring and support: ETL, SSIS, Hadoop, HDFS, Spark, Hive, RDD, and UDF. cloud data ecosystem: Spark API, Spark SQL, PySpark, Scala, Python, and data Streaming
- Demonstrative experience with: data science tools: Python ML, Scala, and Databricks.
Benefits
- Comprehensive benefits package
- Growth opportunities throughout our companies
- Diverse environment to enhance your career
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer developing modern cloud - based data pipelines at UK Biobank for research support. Collaborating within the Data & Technology team to create clean, scalable, secure code.
Senior Consultant SAP Data Migration leading projects in data migration for various clients. Collaborating with project teams to ensure successful strategy implementation and data quality management.
Data Engineering Team Lead engaging with enterprise - level organizations on data architecture using Google Cloud solutions. Leading a team of data engineers in a hybrid work model.
Senior Lead Data Engineer in Bangalore, Karnataka, India designing and building AWS data engineering solutions. Leading teams on scalable data pipelines using Spark, PySpark, and Python.
Data Engineer creating and implementing Big Data applications at Absa with business stakeholders and technology leaders. Involves ETL/ELT pipeline design, data automation, and team mentorship.
Senior Data Architect delivering, enhancing, and adopting enterprise data and analytics products for DoD organizations. Collaborating with teams to translate requirements into scalable solutions for national security outcomes.
Senior Data Engineer supporting the delivery and enhancement of enterprise data and analytics for DoD organizations. Collaborating with engineers and government partners on scalable, production - ready solutions.
Senior Data Engineer at Skillfield designing distributed data processing solutions using Apache Spark. Collaborates on cloud and on - prem solutions across enterprise levels in a hybrid work environment.
Data Engineer responsible for building data intelligence system for the public sector. Ensuring data ingestion, quality, correlation, and helping with analytics for decision making.