Scraping Google Search (without getting caught)
Disclaimer: use ideas here under your own responsibility.
The IP blocking problem
If you are into web scraping you probably know that websites don’t like automated bots that pay a visit just to gather information. They have set up systems which can figure out that your program is not an actual person and, after a bunch of requests coming from your script, you usually get the dreadful HTTP 429 Too Many Requests…