Move Content Between Envs: SharePoint Features & Solutions

I get this question all the time so I finally am taking the time to document the answer I repeat over and over. First the question, then my answer:

• Question: How do I move the content I’ve created in my development environment, such as master pages, page layouts, cascading style sheets, JavaScript, etc. using Office SharePoint Designer 2007 to another environment, such as QA, staging, and ultimately production?
• Answer: Use SharePoint Features & Solutions!

Now to explain, let’s first take a look at the challenge (note: I’m not calling it a problem, it is a challenge).

When you create files in your Publishing site (aka: WCM site) in a development environment, the files are stored within the site’s content database, not on the file system. We call these types of pages “customized” (WSS v2 folk know this as “unghosted”). While this is fine and good for development, you’re going to have some challenges in a real MOSS environment where you’ll have developers working on developer machines, a shared development environment (aka: build server), a test environment and a production environment. Why? Because you will only get those files in the content database into the different environments in one of two ways:

1. Open each file in your development environment in SharePoint Designer, copy the contents, create a new file in your target environment (such as production), paste the contents on the clipboard, save, check-in, publish, and approve. Ah… very tedious if you have more than… oh… two files. Yes, I know you can save the file from development locally and upload it into the target environment, but I consider that just a slight variation on the copy-paste technique.
2. Use MOSS’ Content Deployment capabilities to package the files up and deploy them to the new environment. No, I’m not going to turn this post, or the comments, into a discussion on Content Deployment… for more info on this, check Tyler Butler’s (PM on the SharePoint WCM Team) excellent post on the SharePoint Team Blog earlier this year .

Either way you go, the files still live in the database, not in the file system. I’d argue this is sub-optimal (not bad, not good, just not ideal). Why? Consider the following reasons:

• No true source control: No, I don’t mean version control or audit tracking which SharePoint lists give you OOTB, I’m talking real source control… things like branching, tagging, atomic check-ins, etc. Things like VSS (yuck), Team Foundation Server, or my favorite, Subversion give you.
• No easy way to change one file used across MANY sites: Nope… only way to do this is to do a variation of #1 above.
• No easy way to package up many changes and deploy them at once: Nope… see #1 above again… its a one-by-one operation.
• Making changes to many files already deployed to production is manual, thus tedious: It’s a one-by-one operation (again, mass uploads is a variation of this).

So, how do SharePoint solutions & features fix this? You can create a feature that provisions files into the lists & libraries upon activation. Not only will it provision one file, but it can provision many files at once and even supply additional properties for each file such as those preview images you see when you pick a new master page or page layout in a Publishing site.

I demonstrated how to do this in part 2 of my Building an Internet Site with Web Content Management in MOSS 2007 session at SharePoint Connections in November 2006… you can also get the sample code from that session that includes a sample feature that once installed & activated in a Publishing site, you’ll find a new master page & page layout in the Master Page Gallery. The concept is simple: create your master pages, page layouts, CSS files, JavaScript, etc… create it all in your development environment using SharePoint Designer where you have a rich development experience. Once you have a working site and you’re ready to move stuff, save all these changes to the local file system. You will then take these files, include them in a SharePoint feature, and once the feature is installed, when activated on a Publishing site, it will add these files to your desired libraries (such as the Master Page Gallery).

The magic behind the feature is in the element manifest file. Take a look at this sample elements manifest file:

This file is broken into two parts: two nodes. Each one is responsible for adding a file to the Master Page Gallery. In the first node (lines 3-16), you’ll see that it specifies the location to upload the master page to in the Url attribute and where the file is within the feature in the Path attribute. The inner node (line 7) tells SharePoint what the name of the file is in the Url attribute, that it’s Type is GhostableInLibrary (more on this in a moment), and then proceeds to set a few properties like the content type, URL where it will find the preview image, and the title of the master page. The type GhostableInLibrary tells SharePoint to put an entry in the Master Page Library for this file, but that the file’s contents will not be stored in the database… they’ll stay on the file system. The second node (lines 18-25) follow the syntax to add the preview image to the Preview Images folder in the Master Page Gallery.

Once you have a feature built that provisions files into your Publishing sites, you’re left with quite a few advantages over the two methods described above (in no particular order):

• Ability to leverage a rich source control management solution: Because everything is now on the file system, this is no different than your non-SharePoint ASP.NET 2.0 Web projects, or class libraries, etc… they are just files on the file system so you can easily add them to your source control management system of choice! This is HUGE!
• Ability to package up the feature (and all it’s files) in a SharePoint solution: See where I’m going? If you package up your feature and it’s files into a SharePoint solution file (*.WSP) which you can then use to…
• Deploy to all SharePoint web front end (WFE) servers in your farm at one time: Solution deployment can be scheduled to run at a future time… and the best part: SharePoint’s solution framework will automatically deploy the feature to ~all~ your WFE’s in the farm at that time. But that’s not the only advantage of solutions…
• Ability to retract deployed solutions: Ever had that “oh crap, we gotta get that off the servers NOW!” moment? The solution framework has the ability to yank a deployed solution back at a scheduled time as well (or immediately if you have the “oh crap” moment).
• No need for developers to have access to your production environment: Since the files are in a feature that’s packaged in a single solution (*.WSP) file, your production administrators can easily add the files to your production environment without developers having to open SharePoint Designer to make changes!
• All files remain ghosted (uncustomized) on the file system: Remember, they are provisioned as Type=GhostableInLibrary which means the content doesn’t really reside within the content database, they are on the file system. This is HUGE! This way…
• Easier to make changes to existing files going forward: Everyone has changes to files once deployed into production. Using this mechanism, you can just upgrade the solution with an updated feature (don’t change the solution or feature ID’s) and every site that’s activated the feature will automatically get the changes applied to their site! This is HUGE!

I could go on, but you get the picture (I hope). I highly advise you at the very least consider this approach over the two others mentioned previously in this post. The subject of uncustomized/customized (or ghosted/unghosted in WSS v2 parlance) and how significant it can be in your environment is one of a very controversial nature, so I don’t want to take this post (or the comments) in that direction… I just wanted to share a technique for moving your site’s files from one environment to another.

One last thing… before you ask “uh, is this supported by Microsoft?” because 90% of the people that I explain this to who hear it for the first time always ask, I want to point out one thing. How do you think Microsoft got all those default master pages (BlueBand.master, BlackBand.master, etc) and page layouts (ArticleLeft.aspx, ArticleRight.aspx, etc), images, and style sheets into the Master Page Gallery in the first place? Check out the feature PublishingLayouts ([..]\12\TEMPLATES\FEATURES\PublishingLayouts)… specifically the ProvisionedFiles.xml file which does all the provisioning. So, this is how Microsoft actually got all the content into these sites you’re already using!

Enough hand waiving! Don’t you just hate the posts that talk at this high level, but never really show implementation? Me too! In my next post, I’ll show you how I use a single Visual Studio 2005 project to create my feature and package it up in a solution… all the while staying in-the-box with respect to Visual Studio with only a few, yet very easy, steps.