- Posted by ploft on March 31, 2009
Admitted, this is not my favorite solution to this problem, but if the alternative is no backup at all, this is without doubt a workable resolution.
In our setup I've experienced backup problems when using DPM 2007 to backup some of my VM's running on our Microsoft Virtual Server 2005 host. More specific these the errors appearing:
DPM 2007 reported:
The replica of vm-client-001 on vm-host-xxx.local is inconsistent with the protected data source. All protection activities for this data source will fail until the replica is synchronized with consistency check.
DPM encountered a retryable VSS error.
Microsoft Virtual Server 2005 host reported:
(Both visible in Event log and Virtual Server 2005 R2 master status webpage)
The VSS writer for Virtual Server failed during the PostBackup phase, during the mounting of disk \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy290\Virtual Servers\vm-client-001.vhd for virtual machine vm-client-001. The disk did not come online within 5 minutes. Confirm that the vhd mount driver in installed correctly and is able to mount virtual hard disks.
It's worth mentioning the VM, DPM has been failing to backup of the type: Online backups. I have several other VM's setup on the VM-host, without backup problems, both Online and Offline backups.
My way of fixing the issue with DPM continuing to fail when backup is initiated is solved by changing the backup-mode from Online backup to Offline backup. This is done by the registry on the physical/host machine running the VMs.
1. Open Registry Editor/regedit
2. Navigate to "HKEY_LOCAL_MACHINE\Software\Microsoft\Virtual Server\1.0\Backup\BackupType"
3. Either change the existing REG_DWORD-record if your VM is located here, or create a new REG_DWORD with the same name as your VM appears in Virtual Server 2005 R2 web-interface.
4. Set the value of appropriate record to: 1 This would changes the Backup mode to Offline backups.
Just for the kicks, here is the value definitions:
0/1: Offline backup
2: Online backup
Beware that DPM still reports it's as a Online Backup, but underneath the bonnet, it is Offline backup, and your DPM should no longer experience the mentioned problem. It's by the way not necessary to reboot any of the machines involved for the changes to take effect.
- Posted by ploft on March 20, 2009
I've been running with Windows 2008 for quite some time now, and at the time where DPM 2007 was released I guess Windows 2008 still was kind' a new to people, so no resolution or scenario of this issue was right at hand at the time.
System:
HP ProLiant ML 110 powered by Intel Xeon CPU with 8 Gb memory running 64-bit version of Windows Server 2008 Enterprise (Hyper-V role enabled) with SP1 and all latest updates.
Used quite some time searching the web for a resolution to this problem, and most people a who I "ran into" suggested that I should wait for the SP1 for DPM 2007. Doing some more digging it turns out that the Virtual Disk Service (VDS) on the older 2000 and 2003 system had some memory leaks, and kb-articles referring to these scenarios looked a lot like the problems I have on my Windows 2008 system. Chances for an error reoccurring could therefore be somewhat expected. Not key on manually overwriting important system files on a semi-working system which handles backup, I first of all turn to my patience willing to give the SP1 a chance to might fix this issue since Microsoft at this time doesn't have any fixes ready yet.
SP1 came and a lot of improvement in speed generally, but I still see the VDS service consume way to much memory most of the time, but the applications is more responsive and therefore gives a better user experience speed-wise.
In the lack of better suggestions I turn to the physical hardware and installs further 4 Gb memory, maxing out the total memory possible for this machine 8 Gb. Once again it helps - but only for some time. Hence the amount of backup jobs and data being stored, the system slowly grinds back to the same sluggish state that it had in the beginning.
Getting more and more errors and failed backup messages, and I'm guessing the VDS memory leak is affecting the DPM and its ability to successfully backup things.
Today I stumbled over this kb-article:
http://support.microsoft.com/kb/958387
"On a computer that is running Windows Vista or Windows Server 2008, a memory leak may occur in the Virtual Disk Service. This problem may occur in the following scenarios.
A memory leak occurs in the Virtual Disk Service when an application uses the Virtual Disk Service to enumerate disk resources. In this scenario, you notice that the memory consumption of the Virtual Disk Service (Vds.exe) increases continually."
Once again - sounds exactly like what I'm experiencing, and this time it's for Windows 2008 - yes baby!
Installed the fix (kb958387) and have been running with this update for a week now, backup jobs are hitting a much higher successful rate than before, and whereas the machine often was slow after just one day in companionship with memory-leaking-VDS. VDS and DPM seems to be better friends now and I don't see any huge unexpected memory usage anymore.
I've been running a couple of weeks with this fix, and yes no more memory slaughtering from the VDS service. Thought I'd just share this with you, so none of you should experience the same annoying scenario with a sluggish DPM server or other backup products using Volume Shadow Service (VSS).