Sorry for the delay, but was quite busy updating our website: webstatschecker.com with new functionality and find new and interesting ways to optimize MS SQL to store the data we keep on the database.
We currently hold stats for 750 000 Domains, over 1 000 000 keywords, and more that 1 000 000 000 Search results. Yes, that is a lot of Zeros, and these stats are increasing every day.
That said, we have also removed a lot of keywords from the system which is not appropriate, and which our advertisers will not approve off, is not as easy as you might think.
Anyway, in short, storing the information in normal Tables, just ain’t good enough. It involved some caching and redesigns of our database. We always try to keep our pages response time under a second. Please feel free to let us know if you are not experiencing the same speeds.
Obviously when requesting an update in the system (clicking on the “Update” button on some of the pages), will result in the cache being cleared, and then we retrieve the latest stats.
Anyway, hopefully we will be able to keep this up in the months to come.
Any feedback is welcome, and greatly appreciated.