Big data and monitoringSo Netasure has big data, and we are currently undergoing some major enhancements to better optimize this data for a more responsive view. If you have signed into the account dashboard lately, you have probably had to wait for twenty seconds or so. This is not ideal, and we are working hard to fix it.
I wanted to briefly describe why we are experiencing this issue, as well as what our current thinking is on a solid resolution.
First, due to the many thousands of websites that our monitors are tracking every minute of every day, we have a database and data set that grows rapidly. We are pleased with the ability of our website monitors to robustly monitor, we now just have a data aggregation issue as this data has accumulated.
Never fear, we are not without solutions. First will be to directly tabulate historical results for faster tabulation and calculation. Since any historical data will not be subject to change, we can optimize the storage mechanism quite a bit, by storing one record where many are currently.
Now you might wonder, how will this affect reporting? Well it will speed up reports, but won't change the granularity currently available. Basically we can show the same results in our reports as we are currently, and provide the same relevance with fewer rows.
As we transition the data, we'll be taking it in steps starting with the oldest data, and moving forward. We hop that this ensures maximum impact for our clients and their monitored sites. Upward and onward, and did we mention upward!
Last modified: 2013-01-30 21:08:26
After we had a few embarrassing hardware failures that we didn't know about for nearly a day, I signed us up for Netasure website monitoring. Two months later I began noticing that we had a server that was having issues, and it let me fix it proactively. Thanks Netasure.
Brady, SLC, UT