Updating Totals for Each Domain Daily

To better understand how things are working, as well as the scope of each domain I'm profiling I wanted to regularly update the numbers for the number of URLs indexed, and how many of them have had their Facebook shares pulled. I run this hourly so that I can get updated number of URLs and shares on a regular basis (daily at least).

If the numbers work out, I should be able to pull the Facebook shares weekly, and eventually identify trending numbers for each URL, across all the domains. Eventually, I'll establish a way to stop pulling for URLs that aren't trending, do not have any shares, and are just too old, or not relevant--it will just take time and refinement.

These totals for each domain will drive a reporting tool that I have planned. I want to easily allow for comparison of one or many domains across both the news and propaganda repositories. Eventually, I'll develop a variety of visual tools to help make sense of the data being pulled across all domains targeted by the system.

Return to Main Page