repeated get requests using Node.js (or python) language to specific website, without getting blocked
€8-30 EUR
Betales ved levering
I need to make a get request every 1-2 second (an interval between 1000ms and 2000ms, the shortest possible) to a page of this specific website: [login to view URL] (e.g. '[login to view URL]')
I already wrote down some code, but as of now I've encountered two problems:
a) the scraper gets recognized after x requests (or even at the first request) and gets blocked (bot protections returns an encrypted document)
B) I get IP banned (403) after x requests
I found out that the headers are really important and could be a key to bypass the protection.
SO, what you have to do would be just provide me the snippet to achieve this and prove that the script doesn't get blocked after running for hours.
Prosjekt-ID: #19992985
Om prosjektet
4 frilansere byr i gjennomsnitt €21 for denne jobben
Hello sir, I am an expert Python developer specialized in Web Scraping. You are right about the headers, they are the most important thing when scraping, and I am an expert in mimicing real requests headers, so I can m Mer
In order not to be banned, you need to check the requester and after receiving 403 change the proxy. I have a lot of experience in scraping and I can show you how to do this on Python with well-documented code.