Responsable Technique R&D sur des innovations dans le domaine des hautes tensions. SuperGrid Institute facilite la transition énergétique avec des solutions avancées en collaboration avec des acteurs industriels.
About the role
- Staff Software Engineer at Blackbird.AI architecting and scaling data platforms. Collaborating on AI/ML processing and building intelligence into data pipelines.
Responsibilities
- Design and implement scalable data platform architecture on Databricks, supporting both batch and streaming ingestion
- Build robust, fault-tolerant data ingestion pipelines that integrate with multiple third-party APIs and data providers
- Design and implement AI-powered enrichment stages within pipelines—applying ML clustering, generative AI summarization, classification, and entity extraction to transform raw data into actionable intelligence
- Build analytical systems with full-text search capabilities using Elasticsearch for rapid querying and analysis of enriched data
- Work with AI/ML researchers to implement, integrate and scaling AI processing
- Expose data platform capabilities as APIs and other interfaces for downstream consumption by applications and services
- Optimize data lake and lakehouse architecture for performance, cost-efficiency, and scalability
- Design and implement data quality frameworks, monitoring, and alerting systems
- Design efficient architectures for calling external AI APIs and managing rate limits, costs, and reliability
- Architect solutions with cost-efficiency as a first-class concern, implementing monitoring and optimization strategies for compute and storage
- Make critical build-vs-buy decisions and establish architectural standards for the data organization
- Mentor engineers and elevate the team's technical capabilities through code reviews, design discussions, and knowledge sharing
Requirements
- 8+ years of software engineering experience with 5+ years focused on data platforms or data engineering
- Deep expertise with Databricks, Apache Spark, and data lakehouse architectures
- Strong experience building and operating data pipelines at scale (handling TBs+ of data)
- Experience integrating AI/ML capabilities into data pipelines (clustering, LLM APIs, classification, summarization)
- Proficiency in Python, DBT, and SQL for data processing and pipeline development
- Experience with both batch and streaming large scale data processing patterns
- Strong understanding of cloud platforms (AWS, Azure)
- Excellent communication skills and ability to mentor engineers
- **Preferred Qualifications:**
- Experience designing both batch and streaming/near real-time data architectures
- Proficiency with Elasticsearch for building analytical systems with full-text search capabilities
- Hands-on experience with LLM APIs and understanding of rate limiting and cost optimization
- Experience with Agentic AI, context engineering, and evaluation
- Background in trust & safety, security, or content moderation domains
- Experience with data observability tools and building comprehensive monitoring systems
- Prior experience at a startup or fast-paced environment
- Apply agentic coding tools for day to day development
- Familiarity with Databricks' Lakeflow, Agent Bricks, and vector databases
Benefits
- Competitive compensation package, 401(k), and equity - everyone has a stake in our growth!
- Comprehensive health benefits for you and your loved ones, including wellness days and monthly wellness reimbursements - an apple a day doesn't always keep the doctor away!
- Generous vacation policy, encouraging you to take the time you need - we trust you to strike the right work/life balance!
- A flexible work environment with opportunities to collaborate with your team in person - you can have it all!
- Inclusion and Impact - soar to new heights!
- Professional development stipend - never stop learning!
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Software Engineer designing scalable information retrieval infrastructure for Slack. Collaborating with teams to maintain high availability and build new features.
Software Engineer developing scalable, resilient offline indexing pipelines for Slack's search infrastructure. Collaborating with product engineering to build new features and ensure system reliability.
Senior Systems/Software Engineer designing and developing complex software solutions for HPE's edge - to - cloud offerings. Leading project teams and managing internal and outsourced development partners.
ETL/Data Validation QA professional responsible for validating Informatica - to - Oracle PL/SQL migrations and data accuracy in SAP Commissions. Execute manual and automated tests and manage test cases efficiently.
Senior Software Engineer responsible for designing scalable systems at GEICO. Collaborating across teams while guiding quality practices in a fast - paced environment.
Staff Software Engineer developing reliability software for GM Autonomous Vehicles, collaborating across teams to enhance multi - sensor systems and improve data quality.
Senior Software Engineer developing and implementing vehicle simulation components for General Motors. Collaborating with technical experts to optimize performance and maintainability in vehicle modeling.
Senior Software Engineer developing and maintaining datapath software components for F5’s cybersecurity innovations. Collaborating across teams to optimize hardware and software integration.
Software Engineer building tools that shape how Homebase engineers ship software. Contributing to AWS infrastructure while improving internal developer experience as part of a collaborative team.