Data Engineer designing and maintaining data pipelines at Black Semiconductor. Collaborating with process, equipment, and IT teams to support manufacturing analytics and decision - making.
About the role
- Staff Data Engineer developing streaming transformation processes and managing data pipelines for a digital health startup. Collaborating with cross-functional teams to improve patient experience through technology.
Responsibilities
- Design and build a **Streaming Transformation** process to evolve an existing scheduled PySpark ETL batch transformation process.
- Reduce the time to ingest incoming data to where it shows up in our SaaS app & reports e2e, from hours to minutes or even seconds.
- Produce effective design documentation to help circulate design ideas & transition plans. Break down approved designs into granular tasks.
- Lead the implementation and take on the most challenging or escalated tasks personally while delegating less strategic work to a mix of onshore and offshore team members.
- Meet regularly with technical and business stakeholders to communicate the plan, status, risks and delivery timeline estimates.
- Drive the go-live of the streaming transformation and establish observability & operational SOP process to hand off to other engineers.
- After go-live handle any escalated operational issues based on severity within a reasonable timeline and take on enhancement requests to harden the system and prevent future issues.
- Foster innovation by designing and building solutions that solve business problems and by partnering with cross functional stakeholders and at senior levels.
- Develop secure and high-quality production code. Review / debug code written by others.
- Actively oversee live operations of our SaaS offering, monitoring KPI’s to find opportunities and threats.
- Organize work around a cross-functional team of Product Managers, BAs, Engineers, BI Developers and Quality Engineers.
- Stay up-to-date with the latest technologies and industry trends, leveraging them to drive platform innovation.
- **Participate in week-long quarterly planning and roadmap meetings in-person.**
- (For many cities) Participate in local office gatherings 1x every 2 months for Town Hall.
Requirements
- 5 years leading other software engineers
- Expertise in data architecture and business intelligence systems including ELT development, data transformation, data modeling, report design / development and BI dashboard design / development.
- 10+ years of professional software engineering experience
- 5 years of Python experience
- 5 years of Typescript experience
- 5 years of SQL experience including complex joins, query performance, triggers, views
- 5 years of experience with Infrastructure as Code tooling, preferably CDK
- Deep expertise & certifications on AWS
- **Preferred:**
- Experience with ETL tooling on AWS like PySpark, Lake Formation and Glue
- Familiarity with Golang
- Experience with Kinesis data streaming
- Experience working with visualization tools like Tableau
- Familiarity with HIPAA/SOC/HITRUST requirements
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Junior Data Engineer role focusing on Business Intelligence and Big Data at Avanade. Collaborating on data analysis and SQL queries in a supportive learning environment.
GCP Data Engineer designing and developing data processing modules for Ki, an algorithmic insurance carrier. Working closely with multiple teams to optimize data pipelines and reporting.
Data Engineer at Securian Financial optimizing scalable data pipelines for AI and advanced analytics. Collaborating with teams to deliver secure and accessible data solutions.
IT Data Engineering Co‑Op at BlueRock Therapeutics supports development of scientific data systems. Collaboration on data workflows and foundational AWS data engineering tasks.
Data Engineer I building and operationalizing complex data solutions for Travelers' analytics using Databricks. Collaborating within teams to educate end users and support data governance.
Data Engineer shaping modern data architecture to drive golf’s digital transformation. Collaborating with teams to enhance data pipelines and insights for customer engagement and revenue growth.
Staff Data Engineer overseeing complex data systems for CITY Furniture. Responsible for architecting and optimizing data ecosystems in a hybrid work environment.
Data Engineer strengthening data platform team at Samba TV to improve data analytics and reporting capabilities. Building on AWS, Databricks, BigQuery, and Snowflake technology.
Data Engineer focusing on secure ETL/ELT data pipelines and compliance in healthcare. Designing scalable ingestion frameworks and ensuring alignment with federal standards.