Messy! Why you should never have automated updates on production systems

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2060190

A pre-release version of vSphere Replication has been made available and if you have Automatic Check and Install updates selected the vSphere Replication appliance will automatically upgrade to version 5.5.
vCenter Update Manager may also have this version downloaded. VMware recommends not installing the vSphere 5.5 Replication version in your 5.1 environment.

I’m not blaming VM Ware…just dispensing some well learned experience. What I am saying is that when looking at new software (including updates) to put on your systems you need to be aware of unintended consequences. Here its an upgrade to a new version of vSphere Replication. Which will, by all accounts, leave you in a world of pain. This VM Ware technology is used to provide business continuity by replicating virtual machines.

vSphere Replication 5.5 is not compatible with vSphere 5.1, which results in an inability to manage vSphere Replication and initiate failover.

Using proper controls when releasing software and updates does tend to mitigate these kinds of issues. You should be looking to test the new software or update on a non-production environment to understand how it installs and to familiarise yourself with any interface and options or, in the case of updates, how the software has changed. You should then look to stage it in an environment that replicates your systems as close as possible to look for any issues with applications, operating systems, networks and storage. You also look to see how you roll back in the event of any serious issues, just in case.

Once you’ve done these things you should look to write up your findings and roll back techniques as part of your release management strategy (if you don’t have one…perhaps you should). If all is good and approval given in a change management meeting (aka Change Advisory Board) you go ahead and release into the production environment.

It really is not rocket science and saves you from major headaches.