OPTION 2: Automated Crawling with Google Script
A standalone script that automatically crawls sites at set time intervals. This project will have 3 segments: a. spreadsheet containing website URLs and crawling info b. script for crawling websites and reporting issues c. Crawling automation using time triggers The script will be similar to its counterpart in the first project except it will not deliver live reports to the dashboard. This means that once we set it up, it will be crawling sites at set times and be delivering reports to the designated emails. So you can be travelling around the world and have your sites crawled and report delivered to mail and backup spreadsheets. This option will be ideal if you do not need to be visualizing or monitoring crawling progress. We could have the two setups. The first option for when you want to manually check the websites and do not have to wait for set times, and the second option for regular crawls. Lets see how it goes. Thank you and waiting for your feedback.
