Russ' SharePoint Blog

In my previous post, I suggested that I don’t like to plug KnowledgeLake on my personal blog because I want this blog to be neutral in order to best serve the SharePoint community.  In regard to my technical posts, I will always strive to maintain this neutrality.  But this is not a technical post!  So I’m going to make an exception because I think this information could really help a few people out there.

I have been proudly serving KnowledgeLake for more than 6 years and I have been a developer/consultant for over 15 years.  I can honestly say that I have never an experienced a company that values their employees more than KnowledgeLake.  They really do want their employees to succeed in their careers and they provide all of the tools to get this done.  They also do this while encouraging their employees to maintain a good work/life balance.  They have figured out that happy employees lead directly to happy customers which leads to a happy profit margin.  These are foundational philosophies that work their way UP from the incredible servant leadership group that drives KnowledgeLake.

KnowledgeLake is looking for a few seasoned developers that know SharePoint Workflow inside and out and also have InfoPath experience to join the Professional Services crew (where I work) at KnowledgeLake.  Your current geographical location doesn’t really matter.  We have PS folks that work in various cities around the US.

Do you have these skills or know someone who does?  Are you tired of grinding it out for some giant company with only the company benefiting from your hard work?  Are you tired of working on the same code project for years on end with the same company?  Are you tired of your employer abusing your skills and not properly compensating you for them?  Would you like to have a SharePoint MCM mentor you as you progress through a career in consulting? (Ok, maybe that last one was a little weak ;-)  )

Wow. I totally just sounded like a head hunter with those questions! 

Anyway, if you answered yes to ANY of these questions, do not pass GO, do not collect $200, proceed directly to the “Contact” link just to the right of this blog post and get in touch with me.  I would be happy to talk with you about my experiences at this impossibly amazing company.

How many people do you know who literally have nothing but praises to sing of their employer after 6+ years of service?

I’m off my shameless plug soapbox now.  Carry on!

Last fall I was asked by the Microsoft SQL Server RBS Dev team to author a whitepaper on the topic of Remote BLOB Storage (RBS).  The whitepaper has now been published.

Here is a direct download link to the RBS Whitepaper.

The document reference was originally featured on this page. 

RBS is a powerful SQL Server 2008 R2 technology that helps to get binary content out of the database.  It is extremely beneficial to SharePoint 2010.  By keeping the file binaries out of the content databases, it becomes easily possible to store 10 or 20 million items without having to worry too much about database size.

Since SharePoint 2010 has been released, Microsft has done a good job updating TechNet with procedures for installing and configuring RBS for SharePoint content databases.  But the “why” behind the “how” was missing.  I wanted to know things like:

Before this whitepaper, I suspected that the lack of documentation meant that Microsoft wasn’t fully vested in the technology and they just put it out there to keep the vendors happy and maybe even the FILESTREAM RBS provider was just sort of an “example” provider.  One day I innocently expressed this opinion in a forum post.  This really got the attention of the RBS Dev Team.  They wanted me to clearly understand that not only was the FILESTREAM provider extensively tested for efficiency and reliability, but it was absolutely production ready.  I stand corrected!

I have since implemented the FILESTREAM provider many times and I’ve worked through most of the issues that you might encounter.  I can also say that if the TechNet procedures are followed closely, you shouldn’t have any trouble.

Anyway, after the dialog with the RBS Dev Team, we thought that perhaps a whitepaper was needed to get some additional information out there regarding how SQL Server 2008 R2 RBS really functions.  With this additional knowledge, perhaps SharePoint Architects and Administrators won’t fear deploying it.  This whitepaper is the result of that effort.

I also posted some additional thoughts regarding how RBS benefits ECM and KnowledgeLake based solutions at my company blog here.  I prefer to keep my personal BLOG a bit more neutral unless the subject matter absolutely lends itself to a comment about KnowledgeLake.  So I saved my KL specific comments for that blog post if you’re interested.

I hope this whitepaper helps out both SQL and SharePoint commmunities!

I am honored to once again be invited to speak at the European SharePoint Best Practices Conference in London.  The conference is being held from April 11th to the 13th   Last year a pesky little volcano in Iceland completely wrecked international travel between the US and Europe for several week.

God willing, the volcano will remain at rest this year and I will be delivering an Information Worker session titled “The Content Organiser: A practical guide to distributing content in SharePoint 2010″.

This session will start with a brief overview of the Content Organizer, Content Type Syndication and “Send To Connections”. The remainder of the session will be an extended demonstration of a content distribution solution. The flavor of the demo is that an end user can send documents to a single known location while SharePoint dynamically routes them either to local libraries or to other site collections for long term storage, records management or workflow. I’ll be highlighting the ability of SharePoint to automatically create dynamic folder structures based on document counts in a folder or based on metadata property values. I’ll also discuss the scalability benefits of routing content to other content databases. The demo will include high speed loading of several hundred documents which SharePoint will then shuffle out to multiple site collections and libraries.

I look forward to seeing you at my session.  I’ve got some fun stuff to show off and I’m sure you won’t be disappointed!

Cheers.

Wow.  Long break from blog posting!  Sorry about that.  Once again, duty calls.  I was booked solid from March until mid-summer.  Then it was vacation time.  So there was little time to generate meaningful blog post content.

What happened in the meantime?  Well, Microsoft released this tiny little server product called SharePoint Server 2010 and a tiny little client application called Office 2010!  Yeah, yeah… I know.  I’m a bit behind.  Well, I have been playing with SP2010 and over the coming months, I’ll be posting on some interesting things around Enterprise Search in SP2010.  Our scalable architecture options have grown considerably and I’d like to cover some of that.

I will be presenting on a couple of topics at SharePoint Saturday in Denver this Saturday (August 7th).  “Architecting for Scale in SP2010″ will cover some of those new scale out topology options we have in SP2010.  “Remote BLOB Storage Deep Dive” will get into some of the nuts and bolts of RBS.  Both of these were queued up for the SharePoint Evolution / Portugal TechDays conferences earlier this year.  Unfortunately, the Iceland volcano had other plans for me.  Fortunately, SharePoint Saturday has provided a new forum for me to deliver this content.

In preparing for SharePoint Saturday this weekend, I realized that the RTM version of SharePoint 2010 had a few minor changes to the PowerShell commands for enabling RBS.  I’ve updated my earlier post to reflect the proper commands.

Hope to see all of you SharePoint types in Denver this weekend!

Otherwise, I’m back in the saddle with a laundry list of research topics.  So stay tuned…

Over the holiday break I spent quite a bit of time trying to get a real good handle on exactly how SharePoint 2010 and RBS interact in order to externalize content to a BLOB store. As part of that effort, I implemented the CodePlex RBS Provider as an exercise to learn more about the roles of SQL Server 2008, the RBS Framework, and SharePoint 2010.

So why would you want to try out this exercise? Well, there are some limitations to the FILESTREAM provider. First, it only works with local volumes on the SQL Server. So if you want to store your binaries out on a file share that consists of cheap storage, you’re out of luck. Also, by default, FILESTREAM provider seems to put all binaries in a single folder. NTFS used to have problems when you had too many files in a single folder. That may have been mitigated in newer platforms, so I can’t testify to this specifically. But in general, I’m not a big fan hundreds of thousands of files in single folder.

I will be doing an RBS Deep Dive session at the SharePoint 2010 Evolution conference on April 21st and there are going to be some real juicy tidbits in that session. But in the meantime, I thought I would provide a guide to installing the CodePlex RBS Provider in SharePoint 2010 (beta). Note that some of these steps may be subject to change as I am currently working with the beta bits.

In order for the CodePlex provider to work, it has to be tweaked a bit because SharePoint 2010 requires the provider to properly dispose the BlobStore object in the framework. So I’ve recompiled the provider appropriately. You’ll find the link in the steps below.

Also, before we get started, the steps below assume you have a single WFE in your farm. If you have more than one WFE, steps 2 – 9 will need to be executed on each WFE.

So here we go:

1. Prepare your content database. This script must be executed in SQL Management Studio in the context of the content database that will have RBS installed:
   IF NOT EXISTS (SELECT * FROM sys.symmetric_keys WHERE NAME = N’##MS_DatabaseMasterKey##’) CREATE MASTER KEY ENCRYPTION BY PASSWORD = N’Admin Key Password !2#4′
2. Download rbs_x64.msi from here and save it to your WFE.
3. Execute rbs_x64.msi on your WFE to begin installation of the RBS Framework.
   • Buzz through the standard agreement and registration forms.
   • All features should be installed except FILESTREAM Provider is not necessary. So your feature options should look like this:
     
   • Configure the connection to your content database. Use “default” for the Filegroup name. Test the connection to ensure your good before proceeding. Should look something like this:
     
   • On the database configuration form, no need to check the “Request connecting clients…” box.
   • On the Maintainer Task form, scheduling the Maintainer is optional. I typically do because it’s easier to let the installer help you set up the task. It’s important to configure the “Run As” account as well as the task schedule which is disabled by default.
     
   • On the Client Configuration form, I typically set all log settings to “Warning”.
     
   • You should be ready to “Install” at this point. Once you kick it off, the installer will go to town setting up the framework binaries and running RBS SQL scripts on your content database. If you told the installer to configure a scheduled task for the Maintainer, the Schedule Task configuration box will pop up for you.
   • Once the installer has completed, your content database should have some new tables in it that look like this:
     
4. Download the updated CodePlex RBS Provider that I updated here.
5. For the sake of this guide, create a directory to permanently contain the RBS Provider binaries at c:\CodePlex_RBS_Provider
6. Copy the contents of the “Binaries” directory in the rbs.zip file to c:\CodePlex_RBS_Provider
7. Now you need to use Notepad to edit c:\CodePlex_RBS_Provider\InstallProvider.cmd
   • The “RootDir” parameter needs to be c:\CodePlex_RBS_Provider
     • If you choose a different RBS Provider location for the RBS binaries, this value needs to reflect that location
   • The “ProviderName” parameter needs to be FileStoreProvider_1 for the purposes of this example.
   • The “DataLocation” parameter contains the location of your BLOB store. This will typically be a network share somewhere but it could be a local directory.
     • Note that the RBS provider will be invoked in the context of the SharePoint content access (Application Pool) account specified for the Web Application that the content database is associated with. So that service account needs to have FULL Permissions to this folder.
     • I typically create a subdirectory in the file share for each BLOB Store which I create 1 to 1 for each Content Database that I enable for RBS.
   • The “ServerName” parameter contains the name of your SQL Server instance.
   • The “DatabaseName” parameter contains the name of the content database that has had the RBS resources already installed (new RBS tables, etc).
   • Save the InstallProvider.cmd which should now looks something like this:
     NOTE!!! The application pool account for the SharePoint web application that contains the content database that will have RBS enabled, must have full permission to the final BLOB store folder as well as read access to the location of the RBS provider DLL (if it’s not installed to the GAC).
8. Launch a command prompt with Run As Administrator:
   
9. Run the InstallProvider command: C:\CodePlex_RBS_Provider\InstallProvider.cmd
   • When it runs correctly, the result looks like this:
     
10. Run the SharePoint 2010 Management Shell as Administrator.
11. Execute the following commands to enable the CodePlex RBS Provider for the content database:$site = get-spsite http://siteurl
    $rbss = $site.ContentDatabase.RemoteBlobStorageSettings
    $rbss.Enable()
    $rbss.SetActiveProviderName($rbss.GetProviderNames()[0])
    $rbss
    • When you execute “$rbss” you should see a result that looks like this:
      
    • Congratulations, you just enabled RBS! If all goes well, all future binaries added to your content database will be sent to the BLOB Store.
    • In order to externalize existing BLOB content, you need to run one more PowerShell command.$rbss.Migrate()
      • This command will send all binaries currently in the database to the active RBS Provider BLOB Store.
    • What if I want to pull all my BLOBS back into the database? Simple. Just execute:$rbss.SetActiveProviderName(“”)
      $rbss.Migrate()
      • This will pull all the BLOBs back in-line in the content database. This is handy when you need to move your BLOB store or if you need to change your RBS Provider all together.

Ok, so that was all pretty cool, so how do I know that it worked? Well, before you enable the RBS Provider, the AllDocStreams table in your content database will show values in the “Content” varbinary(max) column like this:

After you enable the RBS Provider, the AllDocStreams table in your content database will show NULL values in the “Content” varbinary(max) column and instead show values in the “RbsId” varbinary(64) column like this:

Also, if all goes well, then the BLOB Store directory that you specified in the InstallProvider.cmd configuration file will have BLOB files showing up like this:

Finally, I’ll say that while this is a great exercise to show you what’s possible with an RBS Provider other than FILESTREAM, the CodePlex provider should probably be used at your own risk in production. In my mind, this is an extremely useful but elaborate teaching example. It should be thoroughly tested before being deployed in mission critical systems.

It’s also important to know that Microsoft ISV Partners are working on several different RBS Providers that will provide feature support for file shares, BLOB encryption, BLOB compression, REST, and Cloud Storage as well as support for specific storage platforms such as Fujitsu Eternus, EMC Centera, and Hitatchi HCAP.

I’m convinced the flexible RBS Provider options in the works will squash any remaining FUD out there regarding SharePoint storing binaries in the database. The scalability story was already great, and it is so much better for SP2010 that it makes my job as a SharePoint evangelist that much easier!

Cheers.

So I’ve been spending the last month or so disecting the RBS framework in SharePoint 2010.  I’ve already succeeded in enabling the CodePlex RBS example provider in SP2010.  I’m sure I’ll be posting a “how to” on that in the coming weeks.

But I’m also cooking something else.  The CodePlex RBS SP2010 example is the foundation for an RBS Deep Dive session that I’m assembling for the SharePoint 2010 Evolution Conference in London.  I’m currently scheduled to speak on Wednesday (4/21/2010).

The session will have just a few overview slides on the concepts of RBS and how RBS affects scalable architecture and then I’ll dive right in to demos and show you how all the pieces and parts of the RBS subsystems in SharePoint 2010 and SQL Server 2008 R2 work together!

Come check it out!

I’m finally getting around to posting the slide deck that presented at SharePoint Saturday KC.  The topic was “Architecting for Scale in SharePoint 2010″

It’s a pretty juicy deck with lots of detail that will help you with storage architecture and SQL tuning.  It also provides good background on Remote BLOB Storage (RBS) and the improvements to the search subsystem in SP2010.

You’ll find the slides here:

Architecting for Scale in SharePoint 2010