One of my Customers is using DPM to backup multiple SharePoint farms. One of the Farms is still a MOSS 2007 farm. Since the SLA for the farm only requires a backup to be available for a month, they choose only to use a disk based backup, and skip the long term retention of DPM (Moving the data from disk to tape). They used the DPM storage size Calculator in order to determine the amount of disk space they needed.
The customer monitored the Database in order to determine the Change Rate in SharePoint. They have a lot of content databases and they noticed that they grew on average 1 % per day. So they used this number and inserted it into the DPM Storage calculator (Available here). The total size of the Farm in question is about 4 Terabytes.
DPM works on a block level of the file system. It utilizes the VSS (Volume Shadow Copy Service) and keeps track of all changes on the file system. The DPM Server downloads an initial snapshot of all databases, and after that he will keep doing an incremental (continuous) backup. This means that every changed block on the file system is transferred to the DPM Server. This is actually is pretty smart way of handling it, and the total amount of space needed for the backups is greatly reduced. SQL Server for example will always backup all changes since the last full backup when you do an incremental backup (Meaning each SQL differential in sequence will grow in size, since it will also contain the changes of the last SQL differential backup (or all changes since the last full backup))
Since they were monitoring the DPM Server, they noticed that the values they used differ from the actual values. With the current rate of change they would not be able to meet their SLA. They opened a case with Microsoft in order to determine why their calculations were of by so much.
Only accounting for the growth is not considering the whole picture. You also need to backup any block on the filesystem that has changed.
So what are possible changes?
- Additions
- Modifications
- Deletions
- Maintenance
The first 3 are the most obvious ones. But there is more that can cause a DB to change the layout on the drives. One thing that bit my customer hard was the maintenance jobs of SharePoint. In MOSS2007 those jobs where quite aggressive, and tried to keep the DBs “far too optimized”. This caused a lot of change on the disk level (daily index rebuilds), and increased our change rate to a huge amount (The Jobs are way better in SP2010, and this is not an issue anymore).
But my colleague Patrick Heyde already had a solution for my Problem. Disable the Timer job that does the DB-defrag, and replace it with a “smarter” implementation (only defragment when a certain threshold is reached).
Once we implemented the changes from Patrick’s Blog, we saw an immediate decrease in DPM storage. The required space for our daily snapshots of the MOSS2007 farm went down:
This graph shows the daily growth of the disk space needed by the DPM for the daily backups. At day 5 we disabled the SharePoint index rebuild job. At the end of the graph we see some negative values, since the additional space is released once the old backups phase out.
Before we disabled the maintenance Job, the daily change was approx. 270 GB/Day (9% of the Farm size). The next 13 days only have an average growth of about 56 GB/Day (1.9% of the Farm size). That’s only 21% of the original space that was required before the change. Now we are able to meet our SLAs, and even have some spare room for “huge changes” – like installing a Service pack ;)
Disclaimer: Your rates may vary depending on farm usage and various other unpredictable metrics.