Skip to main content

From The Field

Go Search
From The Field
  

Lessons from the field by SharePoint's Premier Field Engineers
Explorer View performance - watch that load balancer!

I recently did some investigation into an explorer view performance issue that a customer was experiencing, end users were reporting that connecting to SharePoint via Explorer View was taking forever (well a few minutes). I did some basic testing and found that it was taking in excess of three minutes to connect using Explorer View – this is painfully slow and understandably end users were getting frustrated.

To give you some background to configuration of the farm, it was running MOSS 2007 and had two load balanced WFE’s. Load balancing was being performed by a hardware device rather than WNLB.

The first thing I always like to do with performance issues is to eradicate load balancing from the equation. This was fairly simple to do, by updating the hosts file on a client machine to point the hostname of the SharePoint site directly to the IP address of one of the WFE’s, this caused the client to connect directly to the WFE.

I then tried to connect using Explorer View, this time it was lightning fast and the performance problem had gone away - bingo, I’d found the problem! Obviously the customer couldn’t disable the load balancer to fix this issue so I had to find a solution to the performance problem.  The next thing that I did was to run Network Monitor from a test machine to capture network traffic whilst making a connection to Explorer View to see what was happening on the network layer. The network trace was very interesting (well as interesting as a network trace will ever be)

What I discovered was that the client machine was making a request to the IP address of the load balancer on port 445 and then 139 (these are the ports that are used for file sharing). It tried to make a connection on these ports a total of 6 times before eventually giving up and connecting to SharePoint on port 80, which it should have done from the start J

Each time a connection to these ports failed it doubled the time it waited between the next connection attempt, for example it attempted at 3 seconds into the trace and then 6, 12, 24, 48, 96 before finally giving up at 192 seconds and connecting on port 80.

As it turns out when Explorer View is initiated it attempts to connect to the WFE using port 139 and 445, if it fails for example if the port isn’t open or is blocked by a firewall it backs off and tries again. In this particular case it re-tried a total 5 times. You may be thinking, how does the load balancer tie into this? Well the load balancer had been configured to only listen on port 80 and distribute traffic to the WFE’s, which is perfectly reasonable and expected, but this meant that connection made to any other ports would be automatically discarded. The fix is to configure the load balancer to listen on port 139 and 445 and distribute this traffic to the WFE's (as it is currently doing for port 80 traffic) this ensures that the initial connection that the client makes to the IP address is successful. An alternative solution is to block ICMP requests (Ping) to the load balanced IP address.

To summarise, when a client uses Explorer View it does the following:

1.       Sends an ICMP (Ping) request to the IP address of the server (or load balancer if used) if this fails it goes straight to step 3. If it is successful it goes to step 2.

2.       It attempts a connection on port 139 and/or 445 (Explorer View doesn’t actually use these ports) if this fails it re-tries multiple times before going to step 3 (hence the delay in rendering Explorer View)

3.       It then makes a connection to the IP address of the server (or load balancer if used) on port 80 (or other port the Web application is configured to listen on) and Explorer View is rendered on the client.

Brendan Griffin

DPM and SharePoint - Part 7 - OK, you've convinced me, where do I find out more?

 

It took a while to get here, but here’s the last post in this series! I’d like to think of this as a brain-dump of links and summary of all the important ones given in the previous posts in the series. Rather than list every TechNet article (!), I’ve linked to the index for the important topics.

If you have any further questions about DPM 2007 and MOSS 2007, please post a comment. Look out for more on DPM 2010 and SharePoint 2010 from us in the near future.

Thanks for reading!

 

General DPM:

·         The Official DPM Website - http://www.microsoft.com/systemcenter/dataprotectionmanager/en/us/default.aspx

·         The DPM TechNet TechCenter - http://technet.microsoft.com/en-us/dpm/default.aspx - An awesome starting point for e-learning, troubleshooting and support info, links to recent KB’s, blog posts and all the usual TechNet goodies.

·         DPM Product Group Blog - http://blogs.technet.com/dpm/

·         Jason Buffington’s Blog - http://blogs.technet.com/jbuff/

·         DPM 2010 Beta Overview & Download Link - http://www.microsoft.com/systemcenter/dataprotectionmanager/en/us/2010beta-overview.aspx

·         DPM 2007 Health Model - http://technet.microsoft.com/en-us/dpm/dd218051.aspx

·         DPM 2007 Error Code Catalog - http://technet.microsoft.com/en-us/library/bb795681.aspx

·         How VSS works - http://technet.microsoft.com/en-us/library/cc785914(WS.10).aspx, http://msdn.microsoft.com/en-us/library/bb417449.aspx,

·         How does DPM really work (Video) - http://edge.technet.com/Media/DPM-2007-SP1-How-does-DPM-really-work

·         DPM 2007 SP1 Licencing (Video) - http://edge.technet.com/Media/DPM-2007-sp1-Licensing/

·         DPM Newsgroups - http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.dataprotectionmanager

 

SharePoint Specific:

·         Managing Protected Servers Running SharePoint (TechNet DPM Operations pages) - http://technet.microsoft.com/en-us/library/bb795864.aspx

·         SharePoint Protection & Recovery using DPM 2007 (Windows Server Core Team Blog) - http://blogs.technet.com/askcore/archive/2008/06/24/sharepoint-protection-and-recovery-using-dpm-2007-part-i.aspx, http://blogs.technet.com/askcore/archive/2008/07/16/sharepoint-protection-and-recovery-using-dpm-2007-part-ii.aspx

·         Using ConfigureSharePoint.exe - http://technet.microsoft.com/en-us/library/dd441708.aspx

·         The SharePoint VSS Writer explained - http://msdn.microsoft.com/en-us/library/bb417449.aspx

·         Creating a Recovery Farm - http://technet.microsoft.com/en-us/library/dd180789.aspx

·         How to Protect SharePoint with DPM 2007 SP1 (Video) - http://edge.technet.com/Media/How-to-protect-SharePoint-with-DPM-2007-sp1

DPM and SharePoint - Part 6 - What about Search?

 

Intro

 

People have been asking me for some time about the search post in this series. Sorry it took so long but as promised here it is!

 

Backup and Recovery of SharePoint is complicated at the best of times, what with Config DB dependencies, customisations and a whole load of other things to worry about, we don’t really want to have to worry about SSP’s, Search and the dependencies there either. In fact a lot of people don’t worry and just choose to re-crawl all their content. However, for some this may not be acceptable. The good news is that as of DPM SP1, SharePoint SSP’s and Search can be easily protected with DPM out of the box.

 

This post aims to demystify protecting and recovering SharePoint Search with DPM in a language that SharePoint and DPM IT Pros understand.

 

Let’s dive in…

 

 

Background

 

Pre DPM 2007 SP1, protection of SSP’s and Search for SharePoint was a manual and complicated affair either by using stsadm or by using DPM with custom scripts and scheduled tasks as documented in this whitepaper. With SP1 we can now protect SSP’s and Search out of the box in a 100% supported and recoverable fashion.

 

The challenges surrounding backup and recovery of SSP’s and Search are there because the index files on the file system, and the associated search database must be in-sync. Since enterprise search is made up of these 2 core components, it is only possible to backup and restore these in a supported way by using built-in tools (Central Administration and STSADM) or a SharePoint VSS aware application such as DPM 2007. More on this here: http://technet.microsoft.com/en-us/library/cc261903.aspx#section3.

 

These tools ensure a crawl is not in progress whilst we take the backup. We can do this Pre DPM 2007 SP1 by pausing the crawler, completing the backup and then resuming it. The whitepaper and utilities given in the above link help to achieve this, but you must create a custom protection group in DPM, manually create scheduled tasks and edit XML. All this before you even come to perform a backup or recovery. This is by no means ideal and is where SP1 makes life a lot easier!

 

 

How it Works

 

The first point to note is that DPM 2007 SP1 does NOT protect SSP’s or search as part of the SharePoint Farm single-click selection. In order to do this you must first enable protection of Search (and ultimately SSP’s since Enterprise Search is provided as a shared service by an SSP). We must run the ConfigureSharePoint.exe on the WFE used to protect the farm with the –EnableSPSearchProtection parameter:
 
 
This adds a registry key ‘SharePointSearchEnumerationEnabled’ under ‘HKLM\Software\Microsoft\ Microsoft Data Protection Manager\Agent\2.0\’ on the WFE that enables protection of SSP’s and search. Once the operation has completed we will see a few new options when choosing to edit our protection group for SharePoint or when protecting SharePoint for the first time (provided we have an SSP created and search setup):
 
 

It is important to note that any item starting ‘SSPComponent’ refers to the ENTIRE SSP (and inherently search). Any item starting ‘SPSearchComponent’ refers to the WSS Search service.

 

Selecting the first checkbox below the farm checkbox in the example above will protect WSS Search only. When using MOSS 2007 this usually means the help content. (Note: In MOSS 2007, the WSS 3.0 Search service is used to index SharePoint help by default, therefore it is arguably not a service that requires restoration and could be ignored). If using DPM 2007 with WSS 3.0 you will only see the WSS Search option, therefore to protect your search content you will need to select this box. You will not see the SSP option as SSP’s are part of the MOSS 2007 product.

 

OK, so all the background aside, let’s look at the backup and restore process… for the rest of this post we will focus on restoring SSP’s (and enterprise search), the procedure for WSS Search is the same.

 

 

Backup

 

When DPM takes a snap of the SharePoint Search components it must ensure Search is in a consistent state. In order to do this, crawls must be temporarily paused. You will notice an event with ID 2442 is generated as part of this process. You will likely see multiple entries representing these pauses for enterprise search, WSS Search and profile imports as denoted by ‘1’ in the screenshot below:
 
 

The backup process will then be started and you will see MSSQLSERVER entries with IDs 3197, 3198 and 18264 on your SQL Server. Respectively, these represent database I/O being frozen, resumed and then notification of a successful backup. You should see an entry for the SSP database and the enterprise search database as denoted by ‘2’ in the screenshot above. Note: You will not see an entry for the index file backups.

 

Finally you will see multiple entries with ID 2444. This indicates the gatherer being resumed and is denoted by ‘3’ above.

 

You are also likely to see events with ID 4164 and 4103 (‘4’ above). These indicate a search master merge starting and completing.

 

The SharePoint ULS logs provide little extra information when set to their default logging levels:
 
 

Since VSS is used by DPM, all of the above operations will have no impact on search in your farm. Once a snapshot is complete you will see the items in the protection group they were added to. You will also able to select the SSP or WSS Search component for restore from within the DPM recovery tab.

 

 

Note: If you need to protect more than one SSP, you must have the post-SP1 update for WSS 3.0 installed: http://support.microsoft.com/kb/941422.

 

 

Recovery

 

The restore procedure is a little different to a standard SharePoint content database when you restore an SSP or WSS Search. You must first select the entire SSP or WSS Search instance. It is not possible to restore individual search databases and index files to their original location. If you attempt this you will be presented with the following options and message in the recovery wizard:

 

 

To select all components, select ‘All Protected SharePoint Data’ from the Recoverable data column on the left. You can then choose the recoverable item from the list below the calendar:

 

 

Once you click Recover, you will be presented with the DPM recovery wizard and can now choose to recover the components to their original location:

 
 

It is important to note the message that states you must ensure the SSP, associated databases and index files do not exist in the original location before you start the recovery. Failure to remove these will result in recovery failure.

Note: It is not possible to remove the last SSP by using Central Administration (this will apply if you only have the one). You will need to use the stsadm command: stsadm –o deletessp –title <SSP name> –deletedatabases –force.

Once the recovery is complete you still need to complete one more manual step. You must run the stsadm –o restoressp operation on a WFE and ensure you use the KeepIndex parameter. This ‘hooks up’ the SSP you restored by using DPM to the SharePoint farm.
 
 

You can find out more about the command here: http://technet.microsoft.com/en-us/library/cc262163.aspx. This is a long command and susceptible to many input errors, therefore I suggest you have it pre-written and saved off as a script!

You will see event log entries for restores also. These will be 10045, 10044 and 2444 on the WFE. Respectively, they represent the successful import of search configuration settings into the registry, storage of them in the database and the gatherer index successfully resuming. Your SQL Server will contain multiple event IDs corresponding to the restoration and configuration of the SSP databases.

At this point your SSP and search is back up and running…… Phew, we got there in the end! :-)

DPM and SharePoint - Part 5 - What's coming next?

 

Intro

 

I chose to delay this blog series so I could speak about DPM 2010 and SharePoint 2010 openly once the betas were made available!

 

By now everyone should know the pros and cons of SharePoint and DPM 2007 when it comes to backup and recovery, not least the need to have a recovery farm. I’m pleased to say that the requirement for a recovery farm in SharePoint 2010 is now gone! The real technical specifics of this can be found on MSDN. This also means DPM no longer needs a recovery environment and can recover SharePoint 2010 content faster and easier than ever before.

 

 

Enhancements

 

Since this blog is not aimed at marketing, I will just list the key enhancements below and let you decide for yourself if the new product enhancements are cool:

 

·         No need for a recovery farm (SharePoint 2010 feature).

·         Automatic protection of new content databases (without the need for a consistency check – DPM 2010 feature).

·         Scheduling of the SharePoint catalog job (which enables item level recovery – DPM 2010 feature).

·         There are also many non-SharePoint enhancements all of which can be found on the DPM product groups blog here: http://blogs.technet.com/dpm/archive/2009/09/29/DPM-2010-beta-is-available-now.aspx!

 

And yes you can still protect previous versions of SharePoint with DPM 2010!

 

 

2010 Item-Level Recovery Walkthrough

 

Since the protection group and recovery point creation process is the same as it was in DPM 2007, I’ll avoid showing that here. In addition recovery of databases and high level items like the entire farm is the same. Therefore I will focus on the item level recovery process for SharePoint 2010 by using DPM 2010.

 

The first difference in the recovery wizard for SharePoint items is that you now get to choose whether you want to perform a recovery with or without a recovery farm.
 
DPM2010 Recovery Options
 

Note: You do not get this option for a MOSS 2007/WSS 3.0 farm, as unattached database recovery is a new feature in SharePoint 2010.

 

You should choose the ‘without using a recovery farm’ option if you are restoring from a backup that is the same version as the production farm. If however you were restoring content that was backed-up prior to an update (service pack or cumulative update for example), you should choose the option to use a recovery farm.

 

You may be wondering why you still need a recovery farm when recovering content from an old SharePoint build. This is because the unattached database recovery method will restore a copy of the database with the content you wish to restore and then use the new API allowing unattached extraction of content from a SharePoint content database. As a result, the content is NOT upgraded to the production farm version and may cause corruption if restored.

 

When using a recovery farm (the old way), the database is actually attached to a SharePoint farm and part of that process involves upgrading the database and all its content to the current version of the farm (and as you know recovery farms must be the same version of the production farm). This is a lot slower but guarantees the content is at the correct version level before being exported out of the database and re-imported back into the production farm.

 

It may be easier to visualise this:
 
DPM 2010 Data Flow
 

It is worth noting that this diagram does not show exact communication paths and ordering but is merely an aid to show data flow during the 2010 item-level recovery process. If you can’t remember how data flows when using a recovery farm, see part 3 of this series.

 

Once you have chosen the recovery process you wish to use, you will be asked to specify a temporary server which is used as shown in the diagram above:

 

Temp Location

 

Notice how you are not asked for a recovery farm Web front end server when you choose to perform an unattached database restore (recovery without a recovery farm). You are just asked for a SQL instance and the location to store the database files during the recovery process.

 

Next you are asked for a staging location on the production WFE. This will be used to extract the files from the recovered database and then import them into the production database.

 

Temp Location 2

 

You are then given the standard recovery options window where you can choose whether to restore security settings, use network bandwidth throttling, enable SAN based recovery and provide notification upon recovery completion. This has not changed from DPM 2007. The recovery summary window is given and the recovery starts when you click on Recover. I have not shown these stages as they do not differ from DPM 2007.

 

Finally, it is worth mentioning that DPM 2010 now includes the export and import logs that SharePoint can generate when using the content migration API. These are stored in the temporary folder that you specify when performing a recovery. This makes troubleshooting much easier than described in part 4 of this series.

 

You also get much better detail about what DPM is actually doing with SharePoint in the DPM Jobs window:

 

DPM SharePoint Jobs

 

That’s it; you should now be able to recover SharePoint content faster than ever before. Roll on DPM 2010 and SharePoint 2010!

Migrategroup is here!

Some people reading this post may be familiar with the stsadm operation migrateuser; this command was introduced back in the good old days of WSS 2.0/SPS 2003 SP2 and is also available in WSS 3.0 and MOSS 2007.

The Technet definition of the operation is:

“Migrates a user account in Microsoft Office SharePoint Server 2007 to a new login name and binary ID. If an entry for the new login name already exists, the entry is marked for deletion to make way for the migration. Migrates user access from one domain user to another”

This operation therefore provides the ability to migrate the SharePoint permissions an AD user account has to another AD account, for example it would be useful in the following scenarios:

·         Domain Migration – If a single or multiple user accounts are being migrated to a new AD domain and you want to avoid re-adding all of the users into SharePoint using their AD new accounts, assigning permissions, removing any references to their old AD accounts and testing etc etc then migrateuser can be used to tell SharePoint, hey! Could you update all references to domain1\user1 with domain2\user1.

·         Migrating permissions – If a user leaves a company and their permissions need to be migrated to another users AD account, for example to provide a new employee with their predecessor’s exact permissions.

·         AD user account re-created – If for any reason a user’s AD account is deleted and then re-created with the same username the SID will not match and SharePoint will treat this as a new user therefore all permissions will be lost. Migrateuser can be used to migrate the permissions from the original to the new account. Keith Richie has an excellent Blog post on how to do this - http://blog.krichie.com/2008/06/27/using-stsadm-o-migrateuser-on-a-re-created-account/

A reference for the stsadm –o migrateuser can be found here - http://technet.microsoft.com/en-us/library/cc262141.aspx

One of the common questions I’ve always had from customers is, is there an alternative command for dealing with AD groups? Unfortunately there wasn’t anything available out of the box until the August cumulative update; this added the stsadm operation migrategroup.  The KB article that discusses this can be found here - http://support.microsoft.com/kb/974550

Have fun!

THE Script To Retrieve Effective SharePoint Permissions Has Arrived!
The SharePoint Administration Toolkit contains some useful tools for obtaining effective permissions, but seeing the whole picture may still be quite a challenge...
 
Now, to make up for this we've come up with this PowerShell script that retrieves effective permissions for a specified Windows user/group account or all accounts for any part of a SharePoint web application (list item, list, site, site collection or entire web application).
 
You can find the script along with quite a few other useful bits in the CodePlex SharePoint Management PowerShell Scripts project - look for Get-EffectiveSPPermissions!
Parse IIS Logs to Report on Page Performance
Once again - this is not strictly SharePoint but rather general IIS, however it may prove very handy for you SharePoint guys out there. This script has helped a customer of mine to identify some serious (and evasive!) application bottlenecks - so check it out in the Technet Script Center Gallery
Does this SQL Server Host a SharePoint Farm ?
I recently responded to a thread on Twitter (http://www.twitter.com) offering to describe a method of detecting whether a SQL server hosts a SharePoint farm. Several folks followed up asking for a blog on how to do this so here goes. Note: This is the same for 2007 and for 2010
 
Consider the way PSConfig identifies a Configuration Database when you try to join a server to an existing farm.
 
Once you choose to join an existing farm, you will then be asked to specify a database server.  At that point you have the option of retrieving the database names of every configuration database on that server so that you can choose the farm you would like to join.
 
The database name field is populated by issuing the following SQL command from the WFE to the SQL Server:

 
SELECT name FROM sysdatabases WHERE has_dbaccess (name) = 1
 
The server responds with a list of all databases to which the logged in user has access.
In order to determine which databases are configuration databases, the following SQL query is used:

SELECT @Version=Version FROM [dbo].[Versions] WHERE VersionId=@VersionId

In this case the VersionId that SharePoint is trying to match is:

60B1F2BE-5130-45AB-AF1D-EDD34E626B5D

Only a configuration database will have a row that matches this GUID although you will find a Versions table in all SharePoint databases.
Once the information requested is provided, clicking Next will perform the following actions.
The database id is determined using the friendly name for the database. 

SELECT @dbid = db_id(@databaseName)

Next, SharePoint will read in the application settings that are related to the farm to be connected to. Once that is complete, the configuration wizard will display a final confirmation before performing the configuration.
SharePoint Content Deployment Wizard to the rescue

I’ve been assisting a customer migrate some sites from a pilot MOSS 2007 farm to their production farm, the plan was to use stsadm export/import to move a handful of sites across. The export of the sites worked fine, the problems started when we tried to import them into production.

When attempting to import each of the sites the following error was received – Fatal error " Allowautomaticaspxpageindexing attribute is not declared" it turned out that they had kept their pilot environment up to date and it was running SP2 J the problem was that their production environment was still running SP1 L. It’s not recommended to use stsadm export/import between farms that are at different build levels. The simple solution would be to upgrade production to SP2 and retry the import, unfortunately this wasn’t an option. It was also critical that the content was migrated. After much scratching my head and coming up with a totally un-supported solution of hacking the export CMP file (which is basically just a CAB file) I had to find a realistic approach to migrate the content that wouldn't get me in trouble!

I then thought that I’d give SharePoint Content Deployment Wizard  (SPCDW) a whirl - http://www.codeplex.com/SPDeploymentWizard. Although it’s possible to migrate entire sites using this, as it uses the same method as export/import to migrate content (PRIME API) this wasn’t an option as it would generate the same errors. The approach that I took was to use the tool to export each of the lists from the sites that needed to be migrated – these were primarily Wiki Page libraries and then imported them into production and it worked like a dream. The only issue is that things such as web part pages and the customised quick launch that they had were not brought across, the effort involved in re-creating these far outweighed the pain in having the sites on a pilot system that was starting to creak!

The process that I used was as follows.

·         Create a site in production with the same template as pilot

·         Used SPCDW in the pilot environment to export all of the lists that contained data that needed to be migrated for a site. The tool allows you to select specific lists from the source site – cool!

·         Used SPCDW in production to import all of the lists

·         Repeat process for each site to be migrated

·         Manually re-create customisations to the look and feel

Not an ideal approach I know but if you're desperate it’s always an option!

Single Server Complete Install of SharePoint 2010 using local accounts
 
UPDATED 19 Nov 2009 - It is suggested the reader refer to MSDN article -
Setting Up the Development Environment for SharePoint Server
 - http://msdn.microsoft.com/en-us/library/ee554869(office.14).aspx
 
Before commencing with implementing this approach to a single server installation.
 
Something that was possible in SharePoint Server 2007 has become tricky in SharePoint Server 2010. The complete installation on a single server using non-domain accounts. Something most developers, demonstrators and testers do a lot of suddenly requires the use of domain accounts instead of local machine accounts. Or does it....
 
The recommendation for a single server build of SharePoint 2010 is to use the Stand Alone installation giving you SQL express and a default configuration. But what if you want to use SQL Server 2008 and to have more control over the build, and to use local service accounts. In this case you need to use complete install and either PowerShell alone or a combination of Windows PowerShell and PSCONFIG(UI).EXE
 
To begin with carry out your SharePoint 2010 installation using the advanced option and complete as the server type. This is recommended in a farm configuration.
 
Next you would think to use either PSCONFIG or PSCONFIGUI to create the farm. Well you would be wrong
 
PSCONFIG.EXE -cmd configdb -create -server neilhw2k8r2 -database sharepoint_2010_config -user neilhw2k8r2\administrator -password ******** -passphrase ********
-admincontentdatabase sharepoint2010_admincontent
 
Results in
 
SharePoint Products Configuration Wizard version 14.0.4514.1009. Copyright (C) Microsoft Corporation 2010. All rights reserved.
The specified user neilhw2k8r2\administrator is a local account.
Local accounts should only be used in stand alone mode.
 
So how do we get around this limitation without using the corporate domain or else promoting the server to a domain controller.
 
Windows PowerShell is your friend, New-SPConfigurationDatabase allows you to specify none domain credentials for the farm.
 
To execute this command launch the SharePoint 2010 management shell (in the same location as the central admin link) and simply type the command at the cursor and press enter.
 
CreateNewFarm
 
The beauty of the Windows PowerShell approach is you get prompted for the missing command line attributes instead of the rather horrible error dialog that PSConfig throws at you
 
After this completes you will find in SQL a new configuration database and an admin content database (unfortunately the GUID is back but that can be fixed if necessary)
 
Next the simplest way to complete the installation is revery back to PSConfigUI as you now are starting with the server already joined to the farm.
 
 
 
Follow the wizard through the same options as you had with SharePoint 2007 and complete the installation/configuration with the following screen
 
 
Clicking Fiinish launches the central admin website and after agreeing to report back customer experiences to Microsoft (or not) you get your first look at SharePoint 2010 central admin and the configuration wizards.
 
 
So wizard - or - manual configuration. That's a subject for another post so come back for more...
 
 
 
 
1 - 10 Next

 ‭(Hidden)‬ Admin Links