Sign In
 
 
Go Search
 
RAID is not backup

In my last couple of posts, I’ve been spending some time talking about how to set up your front-end physical disks so that you can maintain high availability. In one of my diagrams, I mentioned RAID 1 (mirroring) for the physical disks. I wanted to spend a few minutes on this topic and hopefully help dispel the myth that RAID is backup, just in case you didn’t know that.

RAID = Redundant Arrays of Independent Disks. The keyword here is Redundant. RAID helps you solve two important problems: (1) you get faster read performance because you have multiple physical disks serving your requests, and (2) depending on your configuration, you can survive a hard disk failure with no downtime.

There are tons of RAID standards and the goal of this post is not to cover the info that you can read elsewhere on RAID 0, 1, 5, or 10. Instead, the goal is to talk about the myth that RAID = Backup. For some folks, the appeal of RAID 1, 5, and 10, all of which offer some form of redundant data copies, is that they “don’t need to backup their data anymore because you can lose a physical hard drive.” While that’s true, redundancy only solves one aspect of backup: you still have your data in the light of a physical hard drive loss. But, here’s some of the ways you can suffer data loss for which RAID will not help you:

  • Multiple hard disk failures. With the exception of RAID 6, which is pretty uncommon in practice, most forms of RAID can only suffer a single hard drive failure before you incur data loss. Because heat is the most common cause of hard disk failure, you want to insure that your disks are adequately cooled. Another important consideration: if at all possible, buy drives from multiple manufacturers or in separate production lots. It is more common for drives produced in the same lot, by the same manufacturer, to fail at or very close to the same time when running in similar conditions.
  • Oops… deletion. Unless you have a backup strategy, you cannot recover from mistakes where the wrong files are deleted.
  • Data corruption. The worst cause of failure without backup is data corruption. As far as any user or admin can tell, all of their data is present and available. Only when you try to access the content do you realize that it is ruined. Again, without backup, you are lost.
  • Physical theft, fire, or damage. Finally, most RAID arrays are physically located together. Any situation that wrecks the physical location of your drives also wrecks your data. Offsite backup is a must if you care about your data.

So, why use RAID at all then!? Because it still has value from redundancy (which is what it was meant for). But, this is what always causes me to chuckle when people say “Never use RAID 0… you can lose everything!” Well, yeah… and you can too with 1, 5, and 10, under the right conditions.

Bottom line: backup is a necessary evil if the data you are storing is meaningful. This last part is important, though, too: the data must be meaningful! For example, it’s quite alright to have no backup of your System drive and Program files, so long as you have a way to rebuild the same setup in a reasonable period of time. Similarly, as we will talk about later with respect to database backups, if you can afford to lose “short term data” – like a backup you just started a few minutes ago, and your primary copy is safe, then you may want to opt for RAID 0 configurations that will suffer total data loss during failure, but won’t result in a perceivable outage to a customer, or a risk to your service’s SLA. When the system is working properly though, you will get the fastest read/write performance possible. More on that later…

M4D H4><0R: Redirecting IIS7 and SharePoint 2007 logging in PowerShell

In my last post, I showed you how to redirect your IIS logs and SharePoint trace logs to a separate logical hard drive. This is important to prevent system instability if you let your logs get really long (as in up to the size of the remaining free space of your system disk). The thing is: I showed you how to do this via the UI. As we all know, doing this via the UI can be prone to human error. It’s SO much easier to just to put this stuff in a script, so that you can ensure that each setting is committed the right way every time.

For IIS7, the way to control the application is via AppCmd.exe, which the IIS team ships. For SharePoint 2007, STSADM is normally the way to do this. If you go and explore STSADM’s endless collection of operations, however, you will find that the ability to set the log location is not available. Bummer! Particularly if you are trying to script the deployment of your farms and want to ensure that it’s always done the same way.

Never fear! PowerShell is here. If you haven’t already acquired a taste for Windows PowerShell, now is the time to start digging in! Every Microsoft server product and technology will be required to natively support PowerShell in the future. In fact, it’s pretty safe to say (since it’s written in the Windows Server Common Engineering Criteria for FY09) that any product shipping after July 1, 2008 is required to have this support or else explain themselves to our internal executives that closely ensure we follow our internal rules for first-class software engineering. So, even though SharePoint 2007 doesn’t include built-in support for PowerShell, it turns out that you can take advantage of our managed object model and get the benefits right now. The scripts below will show you how.

Note: You must run these scripts within an elevated Windows PowerShell instance. One way to do this is to right-click on the Windows PowerShell icon and select Run as Administrator and approve the resulting UAC prompt. Failure to do so will cause the IIS7 components to fail. For the SharePoint part, you must be running as a user with permission to set this setting within Central Administration, regardless of whether the Windows PowerShell instance is elevated.

Automating IIS7 Log Locations

Continuing with our example from my last post, where we want One log per: Site and we want it located at E:\logs\iis, here’s the simple PowerShell script you can run one every machine in your SharePoint farm:

#
# SCRIPTING THE CHANGE OF THE IIS 7 LOGS

& $env:windir\system32\inetsrv\appcmd set config "-section:system.applicationHost/sites" "-siteDefaults.logfile.directory:""E:\logs\iis"""
& $env:windir\system32\inetsrv\appcmd set config "-section:system.applicationHost/log" "-centralBinaryLogFile.directory:""E:\logs\iis"""
& $env:windir\system32\inetsrv\appcmd set config "-section:system.applicationHost/log" "-centralW3CLogFile.directory:""E:\logs\iis"""

Automating SharePoint 2007 Trace Log Locations

Because SharePoint manages the notion of a “farm,” which comprises multiple machines, you only need to run the script below on a single machine within your farm (though it wouldn’t hurt to run this on every machine, it will just result in setting the property multiple times):

#
# SCRIPTING THE CHANGE OF THE SHAREPOINT 2007 TRACE LOGS

[System.Reflection.Assembly]::Load("Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c");
[Microsoft.SharePoint.Administration.SPDiagnosticsService]::Local.LogLocation = "E:\logs\uls";
[Microsoft.SharePoint.Administration.SPDiagnosticsService]::Local.Update();

The magic behind how this script above works is that it is leveraging PowerShell’s ability to run against any managed code objects. SharePoint 2007 has an entirely managed object model, so even without built-in cmdlets, you can still get a bunch of benefit from using PowerShell. The first line loads the SharePoint assemblies into the Windows PowerShell instance. Once we’ve done that, we can get to singleton objects easily using the [] syntax. We can then use the :: operator to call methods and properties off of the object. If you look at the SharePoint object model documentation, you will find that the SPDiagnosticsService object with the LogLocation and Update() methods are the keys to setting the trace log’s location. Once these lines have run, you can go to Central Admin and look at Central Administration > Operations > Diagnostic Logging page and you will see E:\logs\uls in the Path box of the Trace Log section.

Pretty cool, huh?

References

I wanted to give credit where credit was due for this post. Without the following blog posts, I wouldn't have been able to figure out everything I showed here:

SharePoint 101: It’s log… it’s log… it’s better than bad… it’s good!

I must admit, I was never much of a Ren and Stimpy fan, but if you haven’t seen their ridiculous “fake commercial” for “Log!!”, then you owe it to yourself to head over to YouTube and do a quick search for “it’s log ren stimpy”… It’s a takeoff on the 70’s ads for Slinky, but taken to a new level of absurdity. Sadly, this was the first thing that popped in my mind when I sat down to write this post.

So, logs. We love ‘em, but as I mentioned in my last post on partitioning hard drives for your SharePoint servers, you want to have a single, consistent location on every machine where your log files can be placed. Additionally, this should be a separate logical partition from your OS and swap files so that when it gets too big, you don’t lose your system.

So, what should you do? Here’s what we do: First, let’s assume your logical log drive is E:. The first thing to do is to create two folders: E:\logs\iis and E:\logs\uls. Then you will redirect your IIS and ULS (SharePoint Trace Logs) into these folders. Here’s how: Open up Internet Information Services (IIS) Manager (these instructions assume IIS 7) and click on your server name (you must do this for each and every server in your farm). In the main body of the dialog, select Logging:

IIS 7 configuration options

When the Logging page appears, you’ll want to configure it as follows:

IIS 7 logging options

Then, using the drop downs set:

  • One log file per: Site
  • Format: IIS
  • Directory: E:\logs\iis

Next you’ll want to redirect the SharePoint Trace Logs (ULS Logs). To do this, go to Central Administration > Operations > Diagnostic Logging:

SharePoint 2007 Diagnostic Logging

Here, at the bottom of the page, click in the Path: edit box and change it to:

  • E:\logs\uls

Click OK.

Because this is set once in Central Admin, there are two consequences: It doesn’t propagate instantly to every server, it will instead kick off as a job in a short time. Secondly, the path that you specify must exist and be the same on every machine in the farm. Failure to do this will result in problems.

SharePoint 101: Setting up your local disks for optimal performance

How much up front thought did you give to the local disks in your SharePoint web front-ends (WFE) and application servers (APP)? It turns out that it matters quite a bit, particularly if you are trying to offer high availability.

One of my colleagues, Mark Harmsworth on our product team, is responsible for running our internal SharePoint deployments for the Office, SharePoint, and Exchange product teams and has spent quite a bit of time trying different options. I thought I'd share what we've learned and you can consider implementing it (or sharing with us, if you have an option that you think is even better).

Before I go into details, I want to be clear that this post is not about the disks that you use for your SQL storage, as there are separate considerations for those disks, based on the tempdb, temp db logs, and the content, which I will share in a later post. This post is really about how many disks you should have in each of your WFE and APP servers and I'll assume you are putting your SQL storage elsewhere.

The bottom line best practice that we've found is that you should have at least 3 logical drives and at least four physical disks in each server. In an ideal config, you would have 3 logical drives and 6 physical drives.

SharePoint 2007 hard drive configuration

The set up is such that you want to separate where your software is installed (your OS / Program Files disk) from your OS Swap File (which is where your system turns to under memory pressure) from the place where IIS and SharePoint's log files are written. The reason for this separation is pretty simple: under heavy loads, you want to have separate physical hardware handling the three separate places that can receive pressure. If one of your servers gets under memory pressure (because of lots of concurrent users or large requests), the OS will need to start paging out to virtual memory (the OS Swap File). If this is on the same physical disks as where your OS / Program Files are installed, then you've introduced latency when the system needs to (for any reason) load any system file into memory. Separating these places relieves that issue and makes your system more responsive. Similarly, log files can get very large, very fast and the last thing you want is for your logs to accidentally take up all remaining space on your drives with the OS or the Swap file, because that will kill the system. Regardless of how many physical disks you can afford, you want to have the three logical partitions, because then your logs filling up will not prevent normal system operation.

If you must "go cheap", then consider using the physical disks for your OS / Program Files also for your Logs (but still on separate logical drives). If at all possible, you should have separate physical disks for your OS Swap file. The picture I've shown here recommends RAID 1 and this is because you can afford to lose a disk without having to rebuild your entire system. You should also consider the speed of the drives that you are buying. Standard off-the-shelf, cheap commodity SATA drives are typically large and slow (7200 RPM). If you want to stay with SATA, you should consider bumping up your OS Swap disks to 10,000 RPM drives. Still better, though much more pricy would be to consider 15,000 RPM SAS (Serial Attached SCSI) drives. These configuration choices will make a big difference in the overall performance and reliability of your systems.

Welcome to my SharePoint Blog!

Hello and welcome!

Thanks for taking the time to visit my corner of the world. I’ve never really been much of a blogger in the past, but now that we have a hosted SharePoint Server, I figured the time was right start.

I am a Program Manager on the SharePoint engineering team leading up our investments in platform infrastructure and running “SharePoint as a Service.” My responsibilities include the areas of deployment, administration, command-line scripting, SSPs, and hosting. I work on helping support both our in-market product (MOSS 2007) as well as working on future generations of our software, which I’m not at liberty to discuss yet. I also work closely with our internal service teams: Microsoft Office Live Small Business, Microsoft Office Live Workspace, and Microsoft Office SharePoint Online, all of which use SharePoint Products and Technologies as their core platform.

I’ve been a full-time employee at Microsoft since May, 1997 and during my tenure I’ve had the privilege to work on some of the coolest products and technologies, including: Microsoft Excel, Xbox, Windows Rights Management and the IRM feature set in Office 2003 and Office 2007, and Records Management in MOSS 2007. I love designing software and using my technical skills to solve complex business problems.

In the coming weeks and months, I’d like to begin sharing information that I’ve learned about running SharePoint as both and Enterprise- and Service-oriented product. My focus is really aimed as network- and farm-level admins, but I’ll occasionally discuss other topics as well. Thanks for stopping by!

 ‭(Hidden)‬ Admin Links