Drive Failures Affecting Some Customers’ Rankings and Reports
Late Sunday night through Monday, the harddrives responsible for most of our rankings system failed. These failures have resulted in slower rankings collection, but has not resulted in lost data.
Update for 5/5/2012-
Monday, Tuesday, Wednesday and Thursday rankings are done and Friday is over 80% complete. Almost caught up!
April monthly reports are re-generated, as are this weeks reports for Monday and Tuesday. Wednesday and Thursday are being generated now and we still expect to be back to normal by Sunday night.
We will update this post again tomorrow.
What does this mean for customers?
If you’ve set up a customized report, you might not have received your report as soon as you expected to receive it. If you did receive a report and it included a rankings section, one or more of the subsections might have been blank:
These features were intermittently failing because the data was on the affected drives, but as of right now, these should now be functioning, albeit possibly more slowly than normal:
- Campaign ranking history
- Campaign ranking history CSV exports
- The Keyword Analysis Tool
- Custom Reports that include rankings data
For more technical details, please see this post on the dev blog.
When will it be fixed?
We are currently prioritizing the collection of keywords in the order they would normally be collected, which means all rankings collection will be delayed for customers this week. We expect this catch-up process to complete by the end of the week, returning us to normal at that point. Our current expectation is that rankings will be delayed 2-3 days for all campaigns, until we can catch up this weekend.
We are still working to resolve this issue and will update this post every morning until the issue is resolved. We sincerely apologize if these failures affect your work as we know how much you depend on our tools. If you have any questions, please leave a comment below and we’ll try to respond quickly.