Sign In
 
 
Go Search
 
Zach Rosenfield's SharePoint Blog > Categories
SP+PS 2010: PowerShell to Create a Service Application

One of the principles that PowerShell offers is the ability to “discover” what commands you need to run to do your tasks.  This principle mostly holds up—with on caveat: you have to “Know” the concepts about the task upfront for the commands to make sense.  I’m going to both demonstrate what this discovery means at the same time as showing how to create a service application.  The idea behind the discovery concept is that you should be able to take the knowledge behind creating 1 specific type of service application and re-use it to create any other type of service application*.   Note that Search can be a bit more difficult because of all the flexibility built into 2010—the more flexible the system gets the more detailed setup can be.

First, you need to understand the concept of what is required to create a Service Application (I’m only going to discuss non-federated, so running on the same farm that is using it).  In the most simplified form: the following items are required for every service application:

1.       The SPServiceApplication: this is the actual definition in the configuration database that will contain information like related databases and settings.  Note that this requires providing a web service Application Pool (More on this later).

2.       The SPServiceApplicationProxy: This is the “service connection” that is used for Web Applications to “talk” to the Service Application

3.       Start at least one SPServiceInstance: This is the actual “process” that will be used to run the Service Application code.  This can run on multiple servers to share load and provide high-availability—but only 1 is “required”.

4.       Attach the service to a SPServiceApplicationProxyGroup: This is the ‘group’ of service applications that a web application can be set to use.  There is a “Default” group that you can add services to that all web apps will be associated with by default.

Each service obviously has it’s own requirements (for example, you need to create databases for the User Profile service), but PowerShell will actually Prompt you for the “required” parameters and will give defaults for as many as it can.  Feel free to provide alternates to the defaults.

So, with that, let’s use the four requirements above to create a service app!  Metadata Service is a good starting point…

1.       Start one or more service instances.  Note that these exist regardless of whether they have any service applications to actually run.  We also want to wait for this to fully provision before continuing.

$MetadataServiceInstance = (Get-SPServiceInstance |Where{$_.TypeName -eq "Managed Metadata Web Service"})   #Get a pointer to the Service Instance

      if($MetadataserviceInstance.Status -eq "Disabled"){  $MetadataserviceInstance | Start-SPServiceInstance  }  #Start the Service Instance

      while(-not ($MetadataServiceInstance.Status -eq "Online")){                                                 #wait for provisioning to finish

            write-host -ForegroundColor Yellow "Waiting for Metadata service to provision"; sleep 5;

}

     

Add-SPServiceApplicationProxyGroupMember -Identity $ProxyGroup -Member $MetaDataServiceAppProxy

2.       Get or Create an Application Pool.  If you already have one running other services you can get an existing one using the GET command, or create a new one like this.

$ManagedAccount = Get-SPManagedAccount | select -First 1  #Get a managed account. I RECOMMEND YOU CHANGE THIS LINE TO GET THE RIGHT ACCOUNT FOR YOU!
$ApplicationPool = New-SPIisWebServiceApplicationPool "SharePoint Hosted Services" -account $ManagedAccount

 

*Note: this is NOT a Web Application “app pool”.  Do NOT create one of these for a web application –it will NOT work.

3.       Create the Service Application

$MetaDataServiceApp = New-SPMetadataServiceApplication -Name "Metadata Service Application" -ApplicationPool $ApplicationPool

 

4.       Create the proxy.  I also “attach it to the default proxy group using the -DefaultProxyGroup flag.

   $MetaDataServiceAppProxy = New-SPMetadataServiceApplicationProxy -Name "Metadata Service Application Proxy" -ServiceApplication $MetaDataServiceApp –DefaultProxyGroup

 

That’s it!  It’s a little basic, and can’t be part of a larger script that is going to be “re-run” very easily, so here’s a full script with some error handling to wrap the action…

try{

      #Managed Account

      $ManagedAccount = Get-SPManagedAccount | select -First 1

      if ($ManagedAccount -eq $NULL) { throw "No Managed Accounts" }

     

      #App Pool

      $ApplicationPool = Get-SPIisWebServiceApplicationPool "SharePoint Hosted Services" -ea SilentlyContinue

      if($ApplicationPool -eq $null){

            $ApplicationPool = New-SPIisWebServiceApplicationPool "SharePoint Hosted Services" -account $ManagedAccount

            if (-not $?) { throw "Failed to create an application pool" }

      }

     

      #Create a Taxonomy Service Application

      if((Get-SPServiceApplication |?{$_.TypeName -eq "Managed Metadata Service"})-eq $null){     

            Write-Progress "Creating Taxonomy Service Application" -Status "Please Wait..."

            #get the service instance

            $MetadataServiceInstance = (Get-SPServiceInstance |?{$_.TypeName -eq "Managed Metadata Web Service"})

            if (-not $?) { throw "Failed to find Metadata service instance" }

           

             #Start Service instance

            if($MetadataserviceInstance.Status -eq "Disabled"){ 

                  $MetadataserviceInstance | Start-SPServiceInstance 

                  if (-not $?) { throw "Failed to start Metadata service instance" }

            }

            #Wait

            while(-not ($MetadataServiceInstance.Status -eq "Online")){ #wait for provisioning

                  write-host -ForegroundColor Yellow "Waiting for Metadata service to provision"; sleep 5;

            }

            #Create Service App

            $MetaDataServiceApp  = New-SPMetadataServiceApplication -Name "Metadata Service Application" -ApplicationPool $ApplicationPool

            if (-not $?) { throw "Failed to create Metadata Service Application" }

           

            #create proxy

            $MetaDataServiceAppProxy  = New-SPMetadataServiceApplicationProxy -Name "Metadata Service Application Proxy" -ServiceApplication $MetaDataServiceApp -DefaultProxyGroup

            if (-not $?) { throw "Failed to create Metadata Service Application Proxy" }

      }

}

catch

    {

            Write-Output $_

      }

 

SP+PS 2010: Understanding Features

SharePoint PowerShell was designed with Administrators in mind—so those of you who understand the difference between a “Feature” and a “FeatureDefiniton” might be a little concerned when you only see one feature related “noun”: “SPFeature”.   This is really because we could simplify this concept for PowerShell to only need one set of commands to manage functionality in your farms.    The important thing to realize is that no command will return an “SPFeature”, but rather an SPFeatureDefinition.  I’ll try to expect the rest of the concepts by example… Here’s what you can expect from the following commands:

Starting with Get-SPFeature; this cmdlet will behave differently depending on the way it’s called.  Simply calling the command with no parameters will return all features installed in the farm.

PS> Get-SPFeature

If you wanted to see what features are installed at a specific scope, use “Client Side Filtering”:

PS> Get-SPFeature |Where{ $_.Scope –eq “Web” }

To get information about a specific feature, provide the Name or ID to the identity parameter of Get-SPFeature:

PS> Get-SPFeature BaseWeb | select *

If you wanted to get the list of features that are Enabled at a specific scope, you have to provide the Scope and object of interest.  The farm scoped features only require a flag—here’s the command to get all Enabled features at an SPFarm level:

PS> Get-SPFeature -Farm

To get the features enabled on a site collection use this:

PS> Get-SPFeature –Site “http://sitename”

Or on a specific SPWeb:

PS> Get-SPFeature –Web “httP://sitename/blog”

For those you who really wanted to know the details, the three previous commands are returning the SPFeatureDefinition for all the SPFeatures (which is really all Enabled features) at the provided scope.

The other important commands are:

·         Install-SPFeature:  Use this command to install an SPFeatureDefinition into the farm. The files should already be deployed in the proper location (FEATURE directory) so you can just provide the “name” of the feature to be installed (Name is just the containing folder’s name)

·         Uninstall-SPFeature: removes an installed SPFeatureDefinition from the farm.  Use just like Install

·         Enable-SPFeature: turns on an SPFeature on the given Web, Site, Web Application, or the local Farm.  All that is required is the feature identity and the desired URL for the site, web or web app (for Farm level features you do not need to provide any URL)

·         Disable-SPFeature: turns off an SPFeature at the given scope.  Use just like Enable.

As a final bonus—here’s a quick demo of how to use this knowledge to compare the features enabled on a given SPSite to the list of Site-Scoped features in the local SPFarm:

$s1 = Get-SPFeature -site "http://mysitetolookat"  #Get the list of features on our site

$s2 = Get-SPFeature |?{$_.Scope -eq "Site"}  | sort DisplayName  #Get the list of available features (sorted)

$results = Compare-Object $s1 $s2  #Compare them

 

foreach($row in $results){   #Print the features in the master list not enabled on the site.

  switch ($row.SideIndicator){

    =>{ $row.InputObject }

  }

}

*Note that the above script only looks at “=>”, but there could be Sandboxed Solutions that are unique to the site.  These would show with a “<=” SideIndicator instead.

SP 2010: Intro to PowerShell Part 4

A quick post today on a rather important new functionality: Server-Side Filtering.  This functionality is all about letting administrators quickly get the sites and site collections they were looking for—something that in STSADM could take hours! 

Before we discuss the new feature, we need to discuss the problem that this is helping to solve.  The root issue is that iterating over every site collection looking at properties could take a really long time.  For example, what if you wanted to enable a new blogging feature on every “blog” site in a very large farm?  Well, first you have to look at all sites that were deployed in the farm to figure out which ones are Blog sites.  With STSADM this would require custom code—but with PowerShell you might say, why wouldn’t the following line work?

PS> Get-SPSite –Limit ALL | Get-SPWeb –Limit ALL |?{ $_.WebTemplate –eq “BLOG#0”}

The answer is: it does, but slowly!  And while it might work fine on a small farm—it will take a very long time on a large farm as you are essentially opening every website in the farm, one at a time.  The better way to do this is to let SQL do the searching for you with Server-Side filtering! While only subsets of object properties are available, these filters make sorting and searching your farm very achievable.  Let’s do some demos of the various types of filters:

1.       Wildcard URLs.  These can be used on the Get-SPSite and Get-SPSiteAdministration* (see NOTE) cmdlets.  Here I am getting all sites under the “My” managed path in the “MySharePoint” web application
PS> Get-SPSiteAdministration “http://MySharePoint/my/*”

2.       Regular Expression URLS  (supported by Get-SPSite and Get-SPSiteAdministration).  By provided this basic Regular Expression in the “identity” field and then giving the RegEx flag I will get all sites named “Demo1” under both the sites and teams managed path. 
PS> Get-SPSite “http://MySharePoint/(sites|teams)/demo1 -RegEx

3.       Filter Parameter for “special” properties (Supported by Get-SPSite, Get-SPSiteAdministration and Get-SPWeb).  First I will get all Site Collections with “Contoso\JDoe” as the owner:
PS> Get-SPSite -Filter {$_.Owner –eq “CONTOSO\JDoe”}

Note that the above command uses the same syntax as “Where-Object”, just it runs these server-side to make them almost instantaneous.  Here’s another example where I solve the original problem we discussed—getting all blog sites in the farm:
PS> Get-SPSite -Limit All | Get-SPWeb -Limit All -Filter {$_.Template -eq "BLOG#0"}


Here’s a real version of the previous examples running in a 2010 farm…



The filter parameter support is as follows:

a.       SPWeb supports:

                                                               i.      Title and Template

b.      SPSite and SPSiteAdministration supports:

                                                               i.      Owner, SecondaryContact, and LockState

Any of the above get commands can be piped to additional cmdlets to perform operations.  Remember to use WhatIf first to make sure you got your filter right!

*NOTE: I used the Site Administration cmdlets in this post—and you might be saying to yourself, what’s this?!  These are cmdlets that allow farm administrators to manage SPSites that they don’t have access to explicitly.  While Get-SPSite will work for all sites, if you (the admin) do not have explicit access to the SPSite then you will be unable to see anything other than the URL of the SPSite.  Also, in this situation all set cmdlets will fail.  The SPSiteAdministration cmdlets will always allow a farm admin to configure some basic properties (owner, quota, etc) on a SPSite.  Remove-SPSite will still work in all cases as this does not require accessing content inside the site collection (spsite and spsiteadministration cmdlets can be piped to each other).  Note that the SPSiteAdministration object is what you’re using managing SPSites in Central Admin—so that’s a good reference for the configurations that can be done

SharePoint PowerShell “Remoting” Requirements

In a brief “lapse” from the “Intro to PowerShell” sections I wanted to outline some important background for those of you anxious to run your SharePoint administration commands from a remote console.  Keep in mind that just running Enable-PSRemoting is not enough! There are a few unique requirements that the SharePoint environment adds to running remote commands:

1.        You MUST use CredSSP authentication.  Any command that talks to a SharePoint cmdlets that itself talks to SQL (which is most commands) will need to call SQL “as you”.  This means you need the ability to “double hop”—which CredSSP provides.  This is enabled using the “Enable-WSmanCredSSP” cmdlets.    (if you don’t use this you’ll most likely see a message saying the farm does not exist or you do not have enough privledges).

2.       You SHOULD increase the MaxMemoryPerShellMB value on the remote boxes which essentially limits the amount of memory that any single remote process can use.  I would not recommend doing this on every box—but rather an “admin machine” that is not externally available.  The default value is 150MB, which will often fail for Site Collection creation and other long-running commands. You can change this value to a larger value (in this example 1000) using the Set-Item cmdlet:

Set-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB 1000

You may hit other “WSman” specific configuration issues along the way—they are often specific to your domain.  However, these error messages are usually quite detailed and I have found over time to always include enough information to solve the problem right away.

Happy remoting!

SP 2010: Intro to PowerShell Part 3

This is another installment in the intro to PowerShell “series”.  I’m going to start using the "Intro to PowerShell" header to differentiate these PowerShell intro topics from other 2010 discussions.

This particular posts is still on using the “pipeline” with SharePoint 2010 PowerShell cmdlets… More advanced stuff will be coming in future posts.

1.       Get all sites from all content db’s in a web application with a particular name
PS>  Get-SPWebApplication "SharePoint - 80" | Get-SPContentDatabase | Get-SPSite –limit all



2.       Get all sites from the 1st database attached to the web application
PS> Get-SPWebApplication | Get-SPContentDatabase |select -First 1 | Get-SPSite –limit all

 

3.       You can also mix SharePoint cmdlets and Windows PowerShell cmdlets together in a single pipeline.  Here we get a sorted list of all site collections from all content databases that start with a particular phrase (WSS_):
PS> Get-SPContentDatabase | Where{ $_.Name –like "WSS_*"}  | Get-SPSite –limit all | Sort URL –Descending

 

4.       The Foreach-Object cmdlet is useful for commands where something other than the Identity is being piped into the object.  For example, when enabling a feature on one or more sites you can use the “Foreach-Object” (which can be aliased with a percent sign ‘%’) to iterate through sites and call a specific command.  For example I can run this command to enable Ratings on one site collection (I’ve left “whatif” on to have it show the behavior—a good thing to always run first!):

PS> Get-SPSite "http://MySharePoint" | ForEach-Object { Enable-SPFeature "Ratings" -url $_.url -whatif }



Or I can run this on all site collections:

PS> Get-SPSite -limit ALL | ForEach-Object { Enable-SPFeature "Ratings" -url $_.url -whatif }

 

5.       Last item for this post: calculated properties.  This is a functionality of the “Select-Object” cmdlets from PowerShell—but very useful for getting data out of SharePoint that may exist at multiple “levels” of SharePoint.  Getting the Usage data from a site collection is a good example—here’s a demonstration:

#Get Usage for a set of site collections—shows a collection for usage

Get-SPSite | Select URL, Usage


#Use a calculate property to “reach inside” the Usage property to get a specific value

Get-SPSite | Select URL, @{Expression={$_.Usage.Storage}}


#Make it cleaner with titles and formatting

Get-SPSite | Select URL, @{Name="Storage"; Expression={"{0:N2} GB" -f ($_.Usage.Storage/1000000)}}


#Then do something with the data!  This new "object" is still pipelineable and actionable (here with GridView)

Get-SPSite | Select URL, @{Name="Storage"; Expression={"{0:N2} GB" -f ($_.Usage.Storage/1000000)}}, @{Name="Quota"; Expression={"{0:N2} GB" -f ($_.Quota.StorageMaximumLevel/1000000)} } | Out-GridView -Title "Sites w/Usage"

 

And below you can see this all in action (I use a limit to make this fit on one screen. Use “–Limit ALL” to have the script display all site collections):

PowerShell 2010: Basic SharePoint Cmdlets

Two important notes for all future PS+SP posts before i begin to give some SharePoint specific cmdlets:

 

First, make sure if you've added any of my "V3/2007" Scripts to your environment to remove them! Functions "Outweigh" cmdlets so you'll be getting "strange" behavior if you load these!

 

Second, i am doing examples to show you possibilities.  Unless i say "always do things this way" or "never do things this way"--then I’m not demonstrating a 'best practice'.  Please don't assume that I will always show the "best" way--just one option.  I'll happily add any alternatives people suggest.

 

With that in mind, here are some starter "SP+PS" samples...

 

1. Get the local farm:

PS> Get-SPFarm

 

2. Get All details of the farm (default is a limited "formatted" view):

PS> Get-SPFarm | Select *

 

3. Get all web applications in the farm:

PS> Get-SPWebApplication

 

4. Get all web applications including Central Administration (CA) web applicaiton:

PS> Get-SPWebApplication -IncludeCentralAdministration

 

5. Get all site collections in the farm (across all web applications, but NOT including CA):

PS> Get-SPSite -Limit All

 

*Note the "limit" parameter.  By default we limit to 20 site collections as "listing all cmdlets" can be a 'large' operation that you should be very aware of doing...

 

6. Use WhatIf to "see what will happen".  No more "run and hope"!

PS> Get-SPSite -Limit All | Remove-SPSite -whatif

 

7. "Destructive" cmdlets will prompt by default (provided the Environment's "Prompt" defaults have not been lowered!)

PS>Remove-SPSite http://contoso

 

8. Prompts can be "suppressed" using "-Confirm:$False" (not recommended!):

PS>Remove-SPSite http://contoso -Confirm:$False

 

*NOTE: "Success" in PowerShell is "No" output.  When an error or warning occurs it will print to screen--ErrorAction, ErrorVariable, WarningAction, and WarningVariable can be used to "script" handling these outputs.

 

As usual, more advanced stuff coming soon!

PowerShell 2010: Basic Discovery

And on the tone of my previous post--no time to start like the present!  First we should start on SharePoint PowerShell "101".  As announced at SPC, we have over 500 cmdets (and even more if you include FAST and PerformancePoint)--but no need to get started memorizing all those; PowerShell offers great "discovery" tools. 

 

Below are some basic samples to get everyone started.  Please note that i'm going to skip some "basic" PowerShell concepts (like "Piping") as there is some great information on the web about these; however, some of these first few posts will be "repeat" for PowerShell regulars.

 

(Note all these commands assume that you have started your PowerShell in the "SharePoint Management Shell" shown below)

 

 

1. This command will return all SharePoint cmdlets available on the local farm:

 

PS> Get-Command -pssnapin Microsoft.SharePoint.PowerShell

 

 

2. This command will get help for the given command (we have "beta" documentation for over 90% of our commands!):

 

PS> Get-Help Get-SPSite

 

 

3. We can even get a set of Examples from cmdlet help:

 

PS> Get-Help Get-SPSite -Examples

 

 

4. This command will give you a list of all the "Service" related cmdlets in the local SharePoint install:

 

PS> Get-Command -noun "SPService*" | Sort Noun

 

 

5. This command will simplify #3 by only showing the unique "objects" (aka. Nouns) that are available in PowerShell:

 

PS> Get-Command -noun "SPService*" | Sort Noun | Select Noun -Unique

 

 

 

6. Need to figure out what properties or methods a .NET SharePoint object has on it?  Instead of MSDN, try:

 

PS> Get-SPSite http://webapplication | Get-Member

 

 

That’s all for now!  More demos—from the simple to the most advanced—coming soon!

SPC09 Session: PowerShell
I hope you're all at the SharePoint Conference, but even those who are not are probably enjoying all the excited new features.  Among these--PowerShell Integration! 
 
If you're at the conference and you want to see how PowerShell for SharePoint really works, you should come see my session Thursday @ 9am.  We'll do a wide range of tasks--from the most basic to the truly exciting!
 
Hope to see you there...
SharePoint 101: What’s a Host Header Site Collection?

I often get asked to differentiate the different ways to configure URLs for site collections—most often how Host Header (HH) Site Collections fit into the mix—so I thought I’d try to disambiguate this for everyone.  In the simplest explanation, there are three types of Site Collections:

 

·    Load-Balanced URL web application:   A single root URL that is shared amongst all contained site collections.  For example, you could have http://sharepoint, http://my, or http://teams.  There is one root site collection (matches web application URL) and additional site collections can be created at Managed Paths under this URL (http://sharepoint/sites/zach).

·    Host header web applications: Similar to a load-balanced URL web application, except the URLs all use a single host header (e.g.: http://www.foo.com).

·    Host Header Site Collection: These are site collections that each have their own unique host header. For example, http://foo.com and http://bar.com.  These sites are by far the easiest to rename—use STSADM –o RenameSite to change a HH Site Collection host header.  These site collections must still be hosted in a web application—but the Web App URL or Host Header is ignored for these sites.

 

While host header site collections give the most flexibility for offering multiple unique FQDNs, there are some limitations:

 

·    Only 1 site collection can use a unique Host Header

·    Managed Paths do no work

·    AAMs are not used as part of site lookup for HH Site Collections, so URL re-writing will not work

·    SSL Termination at the load balancer will not work  (due to lack of AAM support)

 

It’s important to note that Host Header Site Collections can only be created from the command line—in the STSADM command createsite, provide the Host Header for the SPSite in the URL parameter and then provide the url of the web application that will host the HH Site Collection.   The good news is that for those of you familiar with previous versions of SharePoint, there is no longer a “Host Header Mode”; HH Site Collections can be created at anytime on existing web applications.

SharePoint+PowerShell is (Usually) a Perfect Match

Today I’m posting about a rather dense technology issue—so please bear with me!  Those of you who read my blog know that I am a huge fan of PowerShell and on occasion talk about various ways to use PowerShell for managing SharePoint deployments.  Additionally I’ve seen projects and people around the web discuss it.  However, it wasn’t until learning about some of the most in-depth inner-workings of PowerShell did an important realization come about.  In short: There was a potential for PowerShell users to constantly leak memory without knowing it!  While this is not a common situation and it would be hard to do so in such large volume to really cause real harm—it is possible, and thus, everyone should know about it.

 

So first, what’s the issue?  Well, first of all it is important to know that SharePoint does an amazing amount of work in relation to the SPSite and SPWeb objects to make sure that they are running for optimal performance—which in terms of native code means working to improve memory allocation.  Thus there is some special code in these objects, but this also comes with some unique requirements—one that should not be new to SharePoint developers: the need to Dispose of these objects.  So what’s changed you ask?  Well—did you know that SPSite and SPWeb objects should never be used across threads?

 

Usually this is not important.  Multiple threads can safely access the same SPSite or SPWeb simultaneously, so there’s no reason for a developer to open a reference to one of these objects and pass it to another thread.  However, if you did try this, the unmanaged Heap (which is thread specific) will be “lost” and a new Heap will be allocated in the new thread.  So if you did this endlessly your machine will eventually run out of memory (though this memory is freed if you close the PowerShell process).

 

So on to the specific PowerShell concern.  I only recently learned that every time a new ‘pipe’ in PowerShell is executed (essentially every time you press ‘enter’), that command is run in a unique thread.  So what does this mean for SharePoint?  Well, it means that saving an SPSite or an SPWeb to a variable will cause memory leaks—no matter what you do. 

 

As an example, let’s take the following syntax:

 

PS>  $S = New-Object Microsoft.SharePoint.SPSite(“http://sharepoint”)

PS>  $S.Title

My SharePoint Site

PS> $S.Dispose()

 

Looks fine, right?  Unfortunately, since each “PS>” line is executed in a new thread, the final $S.Dispose() does not dispose of the unmanaged heap (before you go try to prove me wrong, realize that the managed code is properly disposed—so you will recover some memory).

 

Well, as it turns out this is a simple problem to mitigate.  Since each PowerShell ‘sentence’ is run in a single thread, any loaded function or script is run as a single thread!  So while the syntax above was not healthy, the exact same lines of code run in script work fine.  Or even this is line works great:

 

PS>  $S = New-Object Microsoft.SharePoint.SPSite(“http://sharepoint”);  $S.Title; $S.Dispose()

 

Keep your eye out on this blog for other potential workarounds in the future. 

 

With that in mind, happy scripting!

 

Zach Rosenfield
Program Manager, Microsoft Office SharePoint Server

1 - 10 Next
 
 
   
Real Time Web Analytics