Tuesday, May 11, 2010

Recoverpoint CAN corrupt your production data

I had to expand a production lun, and of course when you expand a lun that is replicated by recoverpoint you also need to expand the CDP replica volume as well as the CRR replica volume if you are using CRR and CDP. I followed the steps listed in priumus article emc148277, however the steps listed aren't correct, you can't destroy the CG and then detach the luns from the splitters. If you try to build a new CG at this point, recoverpoint will still see the original size of each lun, not recognizing that the lun has been expanded. What you need to do at this point is to reboot your RPA's all at the same time to flush the cache.

What I did (which they have now noted as a bug in the primus article, and they also now warn you NOT to do this thanks to my discovery) was removed the luns from the storage group in Navisphere. When i added the luns back to the storage group, i was then able to see the correct size in recoverpoint. A few hours later my exchange lun disappeared from the VM guest after it slowly started getting corrupted as shown in the windows event logs. The recoverpoint appliance mixed up the production lun with the CDP replica volume and started writing the replica directly on top of the production lun. BE VERY CAREFUL, and pay close attention to the notes that they have added to the primus article so that no one experiences the same issues that I had!

This is resolved in RecoverPoint 3.1.4 (3.1 SP4) and 3.2.3 (3.2 SP3).
See primus emc223955

No comments:

Post a Comment