[ engineering ]
Software Engineer, Web Crawling
Exa- Location
- Not specified
- Work mode
- On-site
- Type
- full time
- Listed
Applications go directly to Exa. Ramai takes no fee for placement and no money for ranking.
[ About this role ]
Software Engineer, Web Crawling at Exa Labs. This role involves building and scaling Google-scale web crawlers capable of handling over 100 million pages per day. Responsibilities include optimizing crawl politeness, rate limiting, and designing systems to handle dynamic content and anti-bot measures.
Skills and focus areas
- web crawling
- scaling
- high performance languages
- c++
- rust
- typescript
- playwright
- modern web design
- chrome devtools protocol
- system optimization
- distributed systems
- crawl politeness
Summary written by Ramai from the original posting. Full details and requirements are on Exa’s careers page.
[ More at Exa ]
Web search rebuilt for LLMs
See all 61 open roles at Exa →[ Similar roles at other startups ]
- Software Engineer - SystemsDoe
- Site Reliability Engineer, Sr. Site Reliability EngineerProve
- Senior Software Engineer, ComputationAurora Solar
- Chief Technology OfficerArtsy
- Staff Technical Program Manager - MobileMX Technologies
- Software Engineer - Full-stackCardless
[ Not quite right? ]
Tell the assistant what you’re after — stage, location, stack, visa sponsorship — and it searches every open role for you. A few questions are free, no account needed.