Avatar for Spoonshot

Helping CPG companies stay ahead of food trends.

Web Scraping Engineer Internship for 4-6 months

₹15,000 – ₹35,000
Apply now
We are looking for an ambitious individual who never gets tired of handling a large pool of unstructured data sources. Data is the most important fuel for the solutions we build at Spoonshot. Data is partially organized on any given webpage because of the page layout, this structure has to be preserved when data is extracted. We have internally developed a python-based framework to constantly scrape data at large scale from various domains. Our goal is to build capabilities to scale beyond 100K+ sources of stable data pipelines.

Job responsibilities:
- Improve the infrastructure in terms of throughput, transparency, failure rates, resilience, monitoring, unit testing, etc.
- Develop components for automating and maintaining a constant flow of data from multiple sources.
- Work with SQL and NoSQL databases to store raw data.
- Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrape, parse and store.
- Work independently with little supervision to research and test innovative solutions.
- Coordinate with other engineers and data strategy team.

Required skills & experience:
- Strong coding experience in Python.
- Knowledge of working with various front end technologies and how various websites are built.
- Experience with SQL and NoSQL databases.
- Experience with AWS / GCP.
- Previous experience with web crawling is a plus.

More jobs at Spoonshot

View all jobs

Tech Lead

Apply now

Data Engineer

Apply now

Data Scientist

Apply now

Data Science Internship for 6 months

Apply now