24 November, 2024 - 04:26 PM
(23 November, 2024 - 10:38 AM)SpamAssassin Wrote: Show MoreA Premium version for Akamai will soon be released if I hit 500 Likes or 50 Popularity on this thread. Don't forget to rate my profile. Don't be a leacher and Like this post to encourage me to release the Akamai Version as well.
Cloudflare Bypass Scrapper.
A simple Python module to bypass Cloudflare's anti-bot page / I'm Under Attack Mode / IUAM). This will be useful if you wish to scrape or crawl a website protected with Cloudflare. Cloudflare's anti-bot page currently just checks if the client supports Javascript and Requests module, though they may add additional techniques in the future just as Akamai.
Due to Cloudflare continually changing and hardening their protection page, this bypass module requires a JavaScript interpreter to solve Javascript challenges. This allows the script to easily impersonate a regular web browser without explicitly de-obfuscating and parsing Cloudflare's Javascript.For reference, this is the default message Cloudflare uses for these sorts of pages:
Code:Checking your browser before accessing website.com. This process is automatic. Your browser will redirect to your requested content shortly. Please allow up to 5 seconds...
If you embed this python module with your script then it will sleep for ~5 seconds for the first visit to any site with Cloudflare anti-bots enabled, though no delay will occur after the first request. Once the first request is up, you won't need to worry about waiting to scrape/check on that website for a certain period of time, lets say, 180 Minutes or 3 Hours, then again you'll be halt for another ~5 seconds and the process continues.
If you feel appreciation for this module, then how about hitting up the like button, not asking much, or you can donate me some credits :)
Installation Process and Requirements
Code:
Python 3.x
Requests >= 2.9.2
requests_toolbelt >= 0.9.1
'python setup.py install'
The above command will install the Python dependencies automatically. The javascript interpreters and/or engines you decide to use are the only things you need to install yourself, excluding js2py which is part of the requirements as the default.
Download the files and run :
You use cloudflare bypass scraper module exactly the same way you use Requests. The python module works identically to a Requests Session object, just instead of calling requests.get() or requests.post(), you call scraper.get() or scraper.post().
For further uses doc you can visit the Request website doc at https://docs.python-requests.org/en/late...uickstart/
Have a good day and don't forget to Like!
very good