On - board Mission Software Developer at Airbus involved in designing and developing mission software for space platforms. Collaborating with international teams on 5G technology via embedded systems in Toulouse.
About the role
- Web Crawling and Data Indexing Engineer at Mistral AI creating web data extraction tools. Join a team committed to AI innovation and excellence in engineering.
Responsibilities
- Develop and maintain web crawlers using Python libraries such as Beautiful Soup to extract data from target websites.
- Utilize headless browsing techniques, such as Chrome DevTools, to automate and optimize data collection processes.
- Collaborate with cross-functional teams to identify, scrape, and integrate data from APIs to support business objectives.
- Create and implement efficient parsing patterns using regular expressions, XPaths, and CSS selectors to ensure accurate data extraction.
- Design and manage distributed job queues using technologies such as Redis, Kubernetes, and Postgres to handle large-scale data processing tasks.
- Develop strategies to monitor and ensure data quality, accuracy, and integrity throughout the crawling and indexing process.
- Continuously improve and optimize existing web crawling infrastructure to maximize efficiency and adapt to new challenges.
Requirements
- Proficiency in Python, Java, or C++
- Strong understanding of HTTP/HTTPS protocols and web communication.
- Knowledge of HTML, CSS, and JavaScript for parsing and navigating web content.
- Mastery of queues, stacks, hash maps, and other data structures for efficient data handling.
- Ability to design and optimize algorithms for large-scale web crawling.
- Hands-on experience with web scraping libraries/frameworks (e.g., Scrapy, BeautifulSoup, Selenium, Playwright).
- Understanding of how search engines work and best practices for web crawling optimization.
- Experience with SQL and/or NoSQL databases (e.g., PostgreSQL, MongoDB) for storing and managing crawled data.
- Familiarity with data warehousing and scalable storage solutions.
- Knowledge of distributed systems (e.g., Hadoop, Spark) for processing large datasets.
- Proficiency in Pandas, NumPy, and Matplotlib for analyzing and visualizing scraped data.
- Experience applying Machine Learning to improve crawling efficiency or accuracy.
- Familiarity with cloud platforms (AWS, GCP) and containerization (Docker) for deployment.
Benefits
- 💰 Competitive salary and equity
- 🧑⚕️ Health insurance
- 🚴 Transportation allowance
- 🥎 Sport allowance
- 🥕 Meal vouchers
- 💰 Private pension plan
- 🍼 Parental : Generous parental leave policy
- 🌎 Visa sponsorship
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Full Stack Engineer at Doxel developing project management tools using computer vision and AI for the construction industry. Collaborating with teams to build full - stack applications for massive data handling.
Senior Advanced Software Engineer at Quantinuum focused on documentation platform and data engineering for quantum computing users. Collaborating with multi - disciplinary teams to enhance user engagement and analytics.
Lead Full Stack Engineer developing generative UI capabilities for AI - driven experiences at Salesforce. Drive innovation across the enterprise ecosystem while mentoring engineering teams.
Product Engineer responsible for delivering high - quality solutions on NPD projects in a hybrid setup. Engaging across teams to ensure manufacturability and compliance in engineering processes.
Responsible for training coordination and product technical training at GROHE France. Engaging with team members and partners to enhance technical knowledge and product usage.
Senior Fullstack Engineer at Bastion developing regulated stablecoin solutions for financial institutions. Responsible for end - to - end feature delivery and leading technical directions in a fast - paced environment.
Senior Software Engineer developing backend systems for Bastion's stablecoin infrastructure. Leading projects and collaborating with cross - functional teams in a fast - paced startup environment.
Software Engineer 2 designing and building ingestion pipelines at WEX. Responsible for integrating data from various internal and external systems into scalable solutions.
Softwareentwickler*in designing modern solutions for Doxis4 system, analyzing requirements and developing technical solutions using agile methods.