As the number of servers running at CV3 has grown, it’s grown harder and harder to push out new information to all of them. This was already becoming an issue in our old data center, and now that we are in cloud we have even more servers that need that information. We’ve already moved from a “push system” to a “pull” system, so that each web server in the cluster pulls it’s own updates from a central repository. This is a more reliable and scalable architecture, but it’s slow and there have been times when the number of changes has overloaded the system. Right now we have scripts that look for updates and “sweep” them in to the central repository.
We are currently working on implementing a queue for updates, so that when you publish a template it lines up to go into the central repository. This is a much faster and more reliable way to process them because it expects at all times to be overloaded so it works just fine whether busy or not. Once in the repository, changes are then pulled to each web node when it checks for updates.
This whole process will first and foremost give us a maximum “time to publish” so that you know when to expect your update to be completed, even if the system is busy and even if your browser happens to be refreshing a page on a web node that updated right before your change went in and has to wait a full cycle to pull your update. Right now our goal is to keep that “worst case” time under ten minutes. Once that timing is reliable and never missed, we can monitor the process for ways to make it even faster.
Please note that all of this refers only to the process of publishing changes from Staging to Live. When you click save while working on a template, your changes can be previewed immediately in your staging area with no delay.
