Today we are announcing two related changes to the way we describe data storage in SharePoint. First, by taking advantage of performance and reliability improvements in SP1 and by defining specific requirements for large data storage in SharePoint, Microsoft is able to increase the supported limits for data storage in SharePoint.
The SharePoint Content Database Data Size Limit
Please review the full TechNet Article SharePoint Server 2010 capacity management: Software boundaries and limits document. We have also published a guide on SharePoint 2010 scalability here: http://go.microsoft.com/fwlink/?LinkId=223599. In the near future we will publish a test report of large scale testing that supports these new size limits.
· RBS enables SharePoint Foundation 2010 running on SQL Express to store more data than the SQL Express limit of 4 GB. In SQL Express 2008 R2 this limit was increased to 10 GB.
· Some operations can be performance optimized with average blob sizes over 1Mb. This result is from tests with the SQL RBS Provider. Ref: http://msdn.microsoft.com/en-us/library/cc949109(SQL.100).aspx
· There could be storage optimizations with potential disk space and disk cost savings from differential backups or tiered storage.
· We have completed testing on the SQL RBS FILESTREAM provider which can enable iSCSI connected storage for RBS use. Using iSCSI allows for the use of lower cost NAS storage.
· Other potential data optimizations may be developed by ISV’s using the supported public RBS APIs and SharePoint APIs.
· Backup strategy must be carefully considered. Both document metadata and document BLOBs must be backed up at exactly the same point in time. This means any third party backup solution needs to be capable of restoring both the SQL database used by SharePoint and the BLOBs used by SharePoint as a set where no variance occurs which would have the database reference BLOBs that are not available from the same backup.
· RBS is most likely to be used for document archive scenarios where documents are written and not updated. BLOBs in RBS are never updated once they are written; instead a new BLOB is created for any update. BLOBs are immutable, old BLOBs are garbage collected later. You can read more about RBS garbage collection in this article: http://technet.microsoft.com/en-us/library/ff628583.aspx
· RBS providers are required to return the first byte of data in a request in 20ms. This applies for all requests between SharePoint and the RBS provider storage layer.
· The SharePoint database is not intended to be read from or written to except by SharePoint. RBS providers don’t have separate access to the data. This includes direct access to blobs. Ref: http://support.microsoft.com/kb/841057/en-us
· Performance may decrease for smaller BLOB sizes when using RBS. This is also shown in the “FILESTREAM Storage in SQL Server 2008” article referenced above.
· There are many RBS providers available and customers should evaluate them for suitability for their implementations.
· Plan for RBS (SharePoint Server 2010) [http://technet.microsoft.com/en-us/library/ff628583.aspx]
· Overview of RBS (SharePoint Server 2010) [http://technet.microsoft.com/en-us/library/ee748649.aspx]
· Maintain RBS (SharePoint Server 2010) [http://technet.microsoft.com/en-us/library/ff943565.aspx]
· Q: How come you couldn’t provide these increased data limits when SharePoint 2010 launched?
· A: We have learned more about how customers implement document archive solutions on SharePoint in the past 12 months. Now by providing specific guidance around data size scaling and focusing supportability around those we can have an increased data size limit for SharePoint and avoid having a data size limit for the document archive scenario.
· Q: What is the new data size limit for document archives on SharePoint
· A: There isn’t a data size limit, though the new guidance factors for building supportable large scale systems must be followed. If the additional factors are not properly addressed then the lower supportability limit applies.
· Q: What if I really need more than 4 TB on a SharePoint farm and it isn’t document archive?
· A: You should use a scale out topology. This involves having multiple content databases in a single farm and spread sites out amongst them. Each content database can grow to 4 TB by following the guidance.
· Q: What if I had incorrectly assumed the 200 GB limit could be avoided by moving BLOBs to a Remote Blob Storage provider thereby reducing the amount of SQL Server stored data for SharePoint?
· A: We recommend that you upgrade to SharePoint 2010 SP1 and follow the new guidance for the total size you have. Consult the company you purchased your RBS provider from to ensure they are tested with SharePoint 2010 SP1. If you have a deployment that falls outside of the new and old limits we recommend you to contact Microsoft Support and request a supportability review. This is a paid support review and the support engineer will be able to tell you if your current implementation can be supported or if changes to reduce the data per content database are recommended.
· Q: Since NAS is supported, does the SQL Server RBS FILESTREAM provider allow use of a network share to store BLOBs on?
· A: No, NAS must be connected using iSCSI and appear as a local drive on the SQL Server machine.
· Q: Will the content database size limit or the 20mS TTFB limit be enforced in the software?
· A: No. These are support limits that we recommend customers stay within for best performance and in order to get the best support from Microsoft. They are not hard boundaries that are measured by the SharePoint software.
· Q: Where was the old 200 GB limit detailed on TechNet?
· A: It was listed on the SharePoint Capacity Planning Boundaries and Limits page on TechNet. Whilst RBS and BLOBs were not previously specifically called out, the limit of 200 GB was clearly stated for a SharePoint Content Database which includes metadata and BLOBs. This article has been updated for the new limits and to list RBS to be more explicit and to avoid any future misinterpretation.
· Q: Can a large document archive have multiple SharePoint sites collections?
· A: Yes. However our guidance is that if you have a site collection over 100 GB, it should be the only site collection in a content database.
· Q: Can a large document archive have multiple document libraries?
· A: Yes. You can have multiple document libraries with different permissions set.
· Q: Is SharePoint 2010 SP1 required to take advantage of these new content database limits?
· A: No. The limits apply to SharePoint 2010 regardless of whether SP1 is applied. However due to improvements in SharePoint 2010 SP1 you are strongly encouraged to install it.