People have been asking me for some time about the search post in this series. Sorry it took so long but as promised here it is!
Backup and Recovery of SharePoint is complicated at the best of times, what with Config DB dependencies, customisations and a whole load of other things to worry about, we don’t really want to have to worry about SSP’s, Search and the dependencies there either. In fact a lot of people don’t worry and just choose to re-crawl all their content. However, for some this may not be acceptable. The good news is that as of DPM SP1, SharePoint SSP’s and Search can be easily protected with DPM out of the box.
This post aims to demystify protecting and recovering SharePoint Search with DPM in a language that SharePoint and DPM IT Pros understand.
Let’s dive in…
Pre DPM 2007 SP1, protection of SSP’s and Search for SharePoint was a manual and complicated affair either by using stsadm or by using DPM with custom scripts and scheduled tasks as documented in this whitepaper. With SP1 we can now protect SSP’s and Search out of the box in a 100% supported and recoverable fashion.
The challenges surrounding backup and recovery of SSP’s and Search are there because the index files on the file system, and the associated search database must be in-sync. Since enterprise search is made up of these 2 core components, it is only possible to backup and restore these in a supported way by using built-in tools (Central Administration and STSADM) or a SharePoint VSS aware application such as DPM 2007. More on this here: http://technet.microsoft.com/en-us/library/cc261903.aspx#section3.
These tools ensure a crawl is not in progress whilst we take the backup. We can do this Pre DPM 2007 SP1 by pausing the crawler, completing the backup and then resuming it. The whitepaper and utilities given in the above link help to achieve this, but you must create a custom protection group in DPM, manually create scheduled tasks and edit XML. All this before you even come to perform a backup or recovery. This is by no means ideal and is where SP1 makes life a lot easier!
How it Works
The first point to note is that DPM 2007 SP1 does NOT protect SSP’s or search as part of the SharePoint Farm single-click selection. In order to do this you must first enable protection of Search (and ultimately SSP’s since Enterprise Search is provided as a shared service by an SSP). We must run the ConfigureSharePoint.exe on the WFE used to protect the farm with the –EnableSPSearchProtection parameter:
This adds a registry key ‘SharePointSearchEnumerationEnabled’ under ‘HKLM\Software\Microsoft\ Microsoft Data Protection Manager\Agent\2.0\’ on the WFE that enables protection of SSP’s and search. Once the operation has completed we will see a few new options when choosing to edit our protection group for SharePoint or when protecting SharePoint for the first time (provided we have an SSP created and search setup):
It is important to note that any item starting ‘SSPComponent’ refers to the ENTIRE SSP (and inherently search). Any item starting ‘SPSearchComponent’ refers to the WSS Search service.
Selecting the first checkbox below the farm checkbox in the example above will protect WSS Search only. When using MOSS 2007 this usually means the help content. (Note: In MOSS 2007, the WSS 3.0 Search service is used to index SharePoint help by default, therefore it is arguably not a service that requires restoration and could be ignored). If using DPM 2007 with WSS 3.0 you will only see the WSS Search option, therefore to protect your search content you will need to select this box. You will not see the SSP option as SSP’s are part of the MOSS 2007 product.
OK, so all the background aside, let’s look at the backup and restore process… for the rest of this post we will focus on restoring SSP’s (and enterprise search), the procedure for WSS Search is the same.
When DPM takes a snap of the SharePoint Search components it must ensure Search is in a consistent state. In order to do this, crawls must be temporarily paused. You will notice an event with ID 2442 is generated as part of this process. You will likely see multiple entries representing these pauses for enterprise search, WSS Search and profile imports as denoted by ‘1’ in the screenshot below:
The backup process will then be started and you will see MSSQLSERVER entries with IDs 3197, 3198 and 18264 on your SQL Server. Respectively, these represent database I/O being frozen, resumed and then notification of a successful backup. You should see an entry for the SSP database and the enterprise search database as denoted by ‘2’ in the screenshot above. Note: You will not see an entry for the index file backups.
Finally you will see multiple entries with ID 2444. This indicates the gatherer being resumed and is denoted by ‘3’ above.
You are also likely to see events with ID 4164 and 4103 (‘4’ above). These indicate a search master merge starting and completing.
The SharePoint ULS logs provide little extra information when set to their default logging levels:
Since VSS is used by DPM, all of the above operations will have no impact on search in your farm. Once a snapshot is complete you will see the items in the protection group they were added to. You will also able to select the SSP or WSS Search component for restore from within the DPM recovery tab.
Note: If you need to protect more than one SSP, you must have the post-SP1 update for WSS 3.0 installed: http://support.microsoft.com/kb/941422.
The restore procedure is a little different to a standard SharePoint content database when you restore an SSP or WSS Search. You must first select the entire SSP or WSS Search instance. It is not possible to restore individual search databases and index files to their original location. If you attempt this you will be presented with the following options and message in the recovery wizard:
To select all components, select ‘All Protected SharePoint Data’ from the Recoverable data column on the left. You can then choose the recoverable item from the list below the calendar:
Once you click Recover, you will be presented with the DPM recovery wizard and can now choose to recover the components to their original location:
It is important to note the message that states you must ensure the SSP, associated databases and index files do not exist in the original location before you start the recovery. Failure to remove these will result in recovery failure.
Note: It is not possible to remove the last SSP by using Central Administration (this will apply if you only have the one). You will need to use the stsadm command: stsadm –o deletessp –title <SSP name> –deletedatabases –force.
Once the recovery is complete you still need to complete one more manual step. You must run the stsadm –o restoressp operation on a WFE and ensure you use the KeepIndex parameter. This ‘hooks up’ the SSP you restored by using DPM to the SharePoint farm.
You can find out more about the command here: http://technet.microsoft.com/en-us/library/cc262163.aspx. This is a long command and susceptible to many input errors, therefore I suggest you have it pre-written and saved off as a script!
You will see event log entries for restores also. These will be 10045, 10044 and 2444 on the WFE. Respectively, they represent the successful import of search configuration settings into the registry, storage of them in the database and the gatherer index successfully resuming. Your SQL Server will contain multiple event IDs corresponding to the restoration and configuration of the SSP databases.
At this point your SSP and search is back up and running…… Phew, we got there in the end! :-)