Our client, a growing web 3 company that seeks to revolutionize crypto research and investing with AI. The team comprises of some of the top expertise in hedge fund, machine learning, and blockchain experience.
Description
As a Senior Web Scraping/Crawler Engineer, you are responsible for:
- Architecture design and research and development of distributed crawler and data acquisition system
- Design and development of distributed crawler module service architecture and data storage architecture
- Lead the effort to design and implement the architecture of a large-scale crawling system (100+ crawlers)
- Design, implement, and maintain various components of data acquisition infrastructure (building new crawlers, maintaining existing crawlers, data cleaners & loaders)
Profile
A successful candidate should possess min. 3 years in designing crawler, data wrangling, building and maintaining 100+ spiders, avoiding bot prevention techniques. He/she shall possess at least Bachelor's degree in quantitative field (e.g. Computer Science, Engineering, Mathematics, Statistics, Operations Research or other related field).
Job Offer
- Competitive salary & equity compensation
- Annual trip in some coolest places across the world
To apply online please click the 'Apply' button below. For a confidential discussion about this role please contact Wilson Tay on +65 6416 9838
As a senior Web scraping & crawler engineer, you will be working closely with co-founding team to define priorities and develop information sourcing roadmaps. You will also be in-charge of design and implement the architecture of a large-scale crawling system.