Administration - From The Field

11/2/2009

Does this SQL Server Host a SharePoint Farm ?

I recently responded to a thread on Twitter (http://www.twitter.com) offering to describe a method of detecting whether a SQL server hosts a SharePoint farm. Several folks followed up asking for a blog on how to do this so here goes. Note: This is the same for 2007 and for 2010

Consider the way PSConfig identifies a Configuration Database when you try to join a server to an existing farm.

Once you choose to join an existing farm, you will then be asked to specify a database server. At that point you have the option of retrieving the database names of every configuration database on that server so that you can choose the farm you would like to join.

The database name field is populated by issuing the following SQL command from the WFE to the SQL Server:

SELECT name FROM sysdatabases WHERE has_dbaccess (name) = 1

The server responds with a list of all databases to which the logged in user has access.

In order to determine which databases are configuration databases, the following SQL query is used:

SELECT @Version=Version FROM [dbo].[Versions] WHERE VersionId=@VersionId

In this case the VersionId that SharePoint is trying to match is:

60B1F2BE-5130-45AB-AF1D-EDD34E626B5D

Only a configuration database will have a row that matches this GUID although you will find a Versions table in all SharePoint databases.
Once the information requested is provided, clicking Next will perform the following actions.
The database id is determined using the friendly name for the database.

SELECT @dbid = db_id(@databaseName)

Next, SharePoint will read in the application settings that are related to the farm to be connected to. Once that is complete, the configuration wizard will display a final confirmation before performing the configuration.

Posted at 4:06 PM by Neil 'The Doc' Hodgkinson | Category: Administration; SharePoint 2010; Troubleshooting | Permalink | Email this Post | Comments (3)

10/19/2009

Single Server Complete Install of SharePoint 2010 using local accounts

UPDATED 19 Nov 2009 - It is suggested the reader refer to MSDN article -

Setting Up the Development Environment for SharePoint Server

- http://msdn.microsoft.com/en-us/library/ee554869(office.14).aspx

Before commencing with implementing this approach to a single server installation.

Something that was possible in SharePoint Server 2007 has become tricky in SharePoint Server 2010. The complete installation on a single server using non-domain accounts. Something most developers, demonstrators and testers do a lot of suddenly requires the use of domain accounts instead of local machine accounts. Or does it....

The recommendation for a single server build of SharePoint 2010 is to use the Stand Alone installation giving you SQL express and a default configuration. But what if you want to use SQL Server 2008 and to have more control over the build, and to use local service accounts. In this case you need to use complete install and either PowerShell alone or a combination of Windows PowerShell and PSCONFIG(UI).EXE

To begin with carry out your SharePoint 2010 installation using the advanced option and complete as the server type. This is recommended in a farm configuration.

Next you would think to use either PSCONFIG or PSCONFIGUI to create the farm. Well you would be wrong

PSCONFIG.EXE -cmd configdb -create -server neilhw2k8r2 -database sharepoint_2010_config -user neilhw2k8r2\administrator -password ******** -passphrase ********
-admincontentdatabase sharepoint2010_admincontent

Results in

The specified user neilhw2k8r2\administrator is a local account.

Local accounts should only be used in stand alone mode.

So how do we get around this limitation without using the corporate domain or else promoting the server to a domain controller.

Windows PowerShell is your friend, New-SPConfigurationDatabase allows you to specify none domain credentials for the farm.

To execute this command launch the SharePoint 2010 management shell (in the same location as the central admin link) and simply type the command at the cursor and press enter.

The beauty of the Windows PowerShell approach is you get prompted for the missing command line attributes instead of the rather horrible error dialog that PSConfig throws at you

After this completes you will find in SQL a new configuration database and an admin content database (unfortunately the GUID is back but that can be fixed if necessary)

Next the simplest way to complete the installation is revery back to PSConfigUI as you now are starting with the server already joined to the farm.

Follow the wizard through the same options as you had with SharePoint 2007 and complete the installation/configuration with the following screen

Clicking Fiinish launches the central admin website and after agreeing to report back customer experiences to Microsoft (or not) you get your first look at SharePoint 2010 central admin and the configuration wizards.

So wizard - or - manual configuration. That's a subject for another post so come back for more...

Posted at 4:36 PM by Neil 'The Doc' Hodgkinson | Category: SharePoint 2010; PowerShell; Administration; Central Administration | Permalink | Email this Post | Comments (6)

9/11/2009

mergecontentdbs gotcha

Back in SharePoint 2007 Service Pack 1, a new command “stsadm –o mergecontentdbs” was released to help administrators move a site collection from one content database to another. (Despite its name it doesn’t actually merge content databases, just moves the data from one content database to another.) This was fine as long as the size of the site collection being moved was not too big.

It might take several hours to perform the whole move for a large site collection. Plus if for any reason the job crashed out during its run you could be left with orphans in either of your content databases or neither of them in working state. See http://support.microsoft.com/kb/969242/ for more information.

Recently I helped a customer upgrade their SharePoint system to SP2 with the June 09 Cumulative Update and once this was done, to move a 20Gb site collection to a new content database.

The move worked correctly and the “stsadm –o mergecontentdbs” command returned without error. Looking in the new content database I could see the data and an entry in the “sites” table which showed that this contentdb did in fact contain the moved site collection. Looking in the old contentdb at its “sites” table indeed showed that the site collection was gone.

So what was the problem?

It became clear that once the move was over, we were unable to shrink the size of the old content database. It still contained over 20Gb of data and I had no idea why it was still there.

Running a “stsadm –o databaserepair” command against the old content database showed 1000’s of orphans and upon closer inspection these where all individual sites (SPWeb objects) which had been part of the moved site collection. Running “stsadm –o databaserepair” this time again with the “-deletecorruption” switch did the job of removing the orphans after which we could shrink the old content database.

Therefore it would seem that the way the “stsadm –o mergecontentdbs” command works between SP1 and SP2 + June 09 CU seems to have changed. Indeed it seems to deliberately create orphans as part of the move.

After speaking to some of the guys on my team (hat tip to Andy D) the following blog posting came to light from the team at Microsoft who maintains the documentation you see on TechNet.

http://blogs.technet.com/tothesharepoint/archive/2009/05/21/3244169.aspx

It makes mention of the way the mergecontentdbs command was changed in the April 09 Cumulative Update and that it is now the default behaviour to leave the data in the old content database and rely on a new timer job to delete the data automatically in the background. The role of this job is to simply clean up orphans of a particular type which are generated as part of the call to mergecontentdbs.

In summary: Systems which have at least Service Pack 2 and the April 09 Cumulative Update will by default rely on a timer job to remove data from old content databases when the “stsadm –o mergecontentdbs” command is used to move a site collection from one database to another.

Posted at 3:20 AM by David Little | Category: Administration; Configuration; General; SP2; SP1; stsadm; Troubleshooting | Permalink | Email this Post | Comments (0)

7/16/2009

Blocking specific traffic to your Web Servers – using Microsoft URLScan 3.1

Ever wanted to control what applications or urls get passed (or more importantly blocked by) your MOSS Web front-end Servers.

I recently came across a MOSS 2007 customer who wanted to block certain request coming from specific applications (e.g. MSIE Crawler, MS Access), The client was using Windows 2003 OS with IIS 6.0.

There are usually several ways to ‘skin-a-cat’in this case URLScan 3.1 seemed like a perfect candidate for the job, and it is a free download:

Statement on the tin:
"UrlScan 3.1 screens all incoming requests to the server by filtering the requests based on rules that are set by the administrator. Filtering requests helps secure the server by ensuring that only valid requests are processed." http://www.iis.net/extensions/UrlScan

Note: Most of the features of URLScan 3.1 have been incorporated in IIS 7.0, so if you’re using IIS 7.0 you should be able to easily achieve the above from the GUI, see technet documentation on IIS 7.0

From the looks of it all we need to do is download and install URLScan3.1 and then add a few entries in the urlscan.ini file i.e. by identifying and denying specific user agents. (urlscan.ini file is located in the C:\Windows\System32\inetsrv\urlscan folder)

To test if this function actually worked we created the below entries in the .ini file to block Access to all ‘Mozilla’ requests; (n.b: User agent Strings can be taken from IIS log entries);

[Options]
RuleList=DenyUserAgent

[DenyHeaders]
Transfer-Encoding:

[DenyUserAgent]
DenyDataSection=AgentStrings
ScanHeaders=User-Agent

[AgentStrings]
Mozilla

These entries will need to be made on all of the servers assigned the WFE role, for testing you can just amend your Hosts file to direct you’re traffic to one specific front end, as we did.

To our delight, when we tried browsing to any sight hosted on our web server, we received the error page as described in;

Error page returned when URLScan rejects request

http://technet.microsoft.com/en-us/library/cc751375.aspx

Now that we have the format specified in the urlscan.ini all we need to do is identify which user agents we want to block and add these under [AgentStrings] (btw don’t forget to remove Mozilla otherwise no one will get through J)

To identify your Agent Strings use a combination of the entries from your IISLogs, which will look something like: Microsoft Office/11.0 (Windows NT 5.1; Microsoft Office Access 11.0.8221; Pro), (don’t forget to replace the + sign with spaces) and the list of User Agent Strings from this site: http://www.useragentstring.com/pages/useragentstring.php

We managed to successfully block the following Applications responding with unable to connect to server and unable to connect to site messages;

[AgentStrings]

;Browsers

Opera/9.64 (Windows NT 5.1; U; en) Presto/2.1.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1) Gecko/20090624 Firefox/3.5

;Applications

Microsoft Office/12.0 (Windows NT 5.1; Microsoft Office Access 12.0.6320; Pro)

You should also see the corresponding entries in the IIS Logs;

Success:

2009-07-15 12:49:15 10.162.74.223 GET /Rejected-By-UrlScan ~/default.aspx 80 GLOBAL\9739330 10.174.146.69 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-GB;+rv:1.9.1)+Gecko/20090624+Firefox/3.5 200 0 0 346 579 31

We blocked MS Access and got the messages unable to connect to server and unable to connect to site, as expected. You can probably modify these to redirect to something more appropriate, but these were sufficient for us.

Posted at 8:00 PM by Zaheer | Category: Administration; Authentication; IIS; Security | Permalink | Email this Post | Comments (2)

5/2/2009

Crawler Impact Rules - What not to do

In a recent engagement I was asked to troubleshoot an indexing problem for a customer. Essentially the SharePoint indexer despite running on 64bit with 12GB ram and having real fast network links to the Database and remote File Shares was crawling very slowly.

So this first thing I did was take a look at these performance counters to assess the state of play:

What was obvious from this was that the Documents Filtered rate was extremely low at an average of 2 per second. Moss Indexer at full throttle should be indexing considerably faster than this, 80 docs per second is not uncommon in well performing systems.

What was also apparent was that the Threads Accessing Network was a similar figure to the Filtering Threads total. This is indicative of threads waiting on response from the data source.

So where to go from here?

The thread count was so high in these counters that I wanted to see what 'tuning' had been applied to the Indexer. Two places to check.

The SSP Search Settings on the Indexer for the Indexer Performance Setting.
Any Crawler Impact Rules that may have been set

In this case the Indexer Performance Level was set to Maximum which is fine with a dedicated Indexing Server.

When Checking the Crawler Impact Rules though I discovered a whole world of problems.

Crawler Impact Rules

What I found here was over twenty crawler impact rules had been configured for the search service but each one had been setup to use the maximum number of requests for the crawl - Sixty Four.

Best Practice and Technet provide guidance for crawler impact rules as follows.

For crawling internal content in your organization, you can set crawler impact rules based on the performance and capacity of the crawled servers. For example, you might try to avoid crawling internal servers at peak load times. However, for crawling external sites, this kind of coordination is usually not feasible. Therefore, it is best to configure crawl requests to minimize consumption of external site resources and bandwidth so that external site administrators are less inclined to restrict your future access.

During initial deployment, set your crawler impact rules to minimize impact on crawled servers while crawling them frequently enough to ensure relatively fresh results. Later, during the operations phase, you can adjust crawler impact rules based on your experience and the data from your crawl logs.

With many impact rules and all set to sixty four the target servers were simply overwhelmed with requests resulting in a major bottleneck in the search service and reduced performance as we have seen.

Testing the search service by reducing the crawler impact rule maximum requests to the default of eight resulted in immediate improvements in the document filtered rate to around thirty documents per second and the threads accessing the network : total filtering threads ratio improved enormously.

This was by no means the end of the story and the next task involved a lot of trial and error to determine the optimum configuration for the impact rules (or deleting them entirely).

The moral of this tale though is to get the message out about what Crawler Impact Rules are all about. They are not there to squeeze more output from the search engine, they are there to reduce the impact the crawler has on the sources being crawled. There is almost never a reason to increase this number beyond the default and in many cases, such as this one, reducing the number actually improves performance.

Posted at 2:54 PM by Neil Hodgkinson | Category: Administration; Search; SSP | Permalink | Email this Post | Comments (0)

4/29/2009

A Common Alternate Access Mapping (AAM) Mistake Revisited....

A question often asked when exposing an internal site out to the Internet is, "why do I have to extend my original web app into a new zone with an intermediary internal URL to publish to a public URL, can't I simply add a public URL to a new zone for the already existing web app?"

The answer is no as described in mistake #3 in the Plan alternate access mappings article.

In short, if a web application was created with a host header then IIS will only listen on that host header and the request for the new URL will never get to SharePoint. The recommended approach would be to make the web application listen on a different URL by extending the web application into a different zone which creates a second IIS site and provides the opportunity to configure a host header for the extended site to listen on (this would form the internal URL, and a public URL in the same zone should be defined).

Although technically speaking, if a host header wasn't defined when the web application was originally created, and IIS was blindly listening on port 80 this wouldn't be a problem and we could simply add a public URL in the same zone as the original web application.....so we may believe.....

Whilst a fellow PFE was onsite with a customer, some strange behaviour in SharePoint was experienced.

The customer was reporting that even though they had multiple web applications serving content and that all the web applications were hosted under their own application pools, only one w3wp.exe process was on each of their web front end servers.

We had initially thought that this was because of their AAM configuration, as they had entries pointing to specific servers. In other words, we thought we would find that only certain servers would have multiple w3wp.exe processes as the AAM redirected the users to a specific server and not to NLB.

However, after digging into AAM, DNS name resolution, application pool configuration and so on, it turns out that in fact, only one w3wp.exe process was serving content for all the web applications in the environment.

The following diagram explains the configuration of web applications, worker processes, AAMs and Content DBs in the environment:

The problem was that although users were hitting content hosted on WEB APPLICATION 2 we could only see 1 w3wp.exe process with PID (6148). That worker process corresponded to WEB APPLICATION 1, not WEB APPLICATION 2 – Don’t get me wrong, users were requesting data from WEB APPLICATION 1, but they were also requesting data from CONTENT DB 2 which was attached to WEB APPLICATION 2.

Long story short, it turns out that the AAM mapping for WEB APPLICATION 2 (INTERNET ZONE) was created in the AAM page. They never extended the web application to use the loadbalancedurl2.net address. They just created the new zone via AAM page. Because it was never extended, when the client resolved the loadbalancedurl2.net address from WEB APPLICATION 2, the address was resolved to their WFEs IP address, but IIS would accept the request via the website on PORT 80 (http://loadbalancedurl2.net:80) – which corresponds to WEB APPLICATION 1. This essentially meant that WEB APPLICATION 1, listening on port 80 was accepting all requests from users that used the load balanced URLs for all the web applications in the environment.

WSS_Content2 is only actually served thanks to the absence of host header on WEB APPLICATION 1 thus permitting PID 6148 to receive all incoming requests on port 80 and route as “best effort”. And as you may have guessed by now, this only works because both web applications are using an application pool with the same login. If each web application’s application pool had a unique login, then WEB APPLICATION 1 would not be able to access the content database for WEB APPLICATION 2 and vice versa.

As outlined in the second paragraph of this post, the proposed resolution and recommended approach would be to have an extension of each of the applications into another zone, ideally having hostheader mapped to public url (or whatever it might be http://mossserver:85 in this case), if the reverse proxy device exposing the web application is NOT forwarding the host header. This will isolate user / proxy traffic on a specific process and no longer serve the multiple Content DBs through the single web application.

Many thanks to Leandro Iacono for the useful customer example.

Posted at 1:14 AM by samhassa | Category: Administration; Authentication; Central Administration; Configuration; General; IIS; Permissions; Security; Troubleshooting | Permalink | Email this Post | Comments (0)