This post is part of a series on Office365/SharePoint Online, Windows Azure and Authentication. The other posts in this series are as follows:

• O365/SPO + Azure + AuthN - About this Series
• O365/SPO + Azure + AuthN - Overview of Challenges with O365, SPO & MSO
• O365/SPO + Azure + AuthN - How to Authenticate with O365, SPO & MSO
• O365/SPO + Azure + AuthN - Workarounds and Fixes for Claims-Based Auth Sites
• O365/SPO + Azure + AuthN - Critical Path Training's Office365 AuthN Helper Library

In this last post in my Office365/SharePoint Online + Windows Azure + Authorization blog series, I want to introduce a little helper project I am using. To make life easier I created a little O365 authorization helper library that does a lot of the heavy lifting for you. It covers two of the three things I outlined in my series as workarounds.

Code samples I show in this post were taken from a code sample CPT's Office365/SharePoint Online Claims Authentication Helper Library… you can get the code from the Critical Path Training site’s Members section… look in the Code Samples section.

Introducing the Claims Auth Friendly ClientContext: ClaimsClientContext

First it creates a special Client Site Object Model (CSOM) ClientContext object. This object has a few properties needed for authenticating with Microsoft Online (MSO) to obtain the SAML token. It then rewrires the ClientContext so that every request includes the SAML token:

   1: namespace CriticalPathTraining.Office365.AuthLibrary {

   2:   public class ClaimsClientContext : ClientContext {

   3:     public string MsoUsername { get; set; }

   4:     public string MsoPassword { get; set; }

   5:     public string MsoRootSiteCollectionUrl { get; set; }

   6:  

   7:     public ClaimsClientContext(string webFullUrl) : base(webFullUrl) { }

   8:     public ClaimsClientContext(Uri webFullUrl) : base(webFullUrl) { }

   9:  

  10:     private MsOnlineClaimsHelper _claimsHelper;

  11:     /// <summary>

  12:     /// Microsoft Online claims helper used to authenticate to 

  13:     /// SharePoint Online.

  14:     /// </summary>

  15:     private MsOnlineClaimsHelper MsftOnlineClaimsHelper {

  16:       get {

  17:         if (_claimsHelper == null) {

  18:           _claimsHelper = new MsOnlineClaimsHelper(

  19:                                     MsoUsername, 

  20:                                     MsoPassword, 

  21:                                     MsoRootSiteCollectionUrl);

  22:         }

  23:         return _claimsHelper;

  24:       }

  25:     }

  26:  

  27:     /// <summary>

  28:     /// Rewire event for client context so that 

  29:     /// every request includes authenticated cookies.

  30:     /// </summary>

  31:     protected override void OnExecutingWebRequest(WebRequestEventArgs args) {

  32:       args.WebRequestExecutor.WebRequest.CookieContainer = 

  33:         MsftOnlineClaimsHelper.CookieContainer;

  34:     }

  35:   }

  36: }

Usage is very simple… the downloadable library includes a test project that shows the usage:

   1: [TestMethod]

   2: public void CliamsClientContextTest() {

   3:   using (var context = new ClaimsClientContext(MSO_SPSITE_URL) {

   4:     MsoUsername = MSO_USERNAME,

   5:     MsoPassword = MSO_PASSWORD,

   6:     MsoRootSiteCollectionUrl = MSO_ROOT_SPSITE_URL

   7:   }) {

   8:     // get the web

   9:     var web = context.Web;

  10:     context.Load(web, w => w.Title);

  11:     context.ExecuteQuery();

  12:  

  13:     Assert.IsNotNull(web.Title);

  14:     Assert.IsTrue(web.Title.Length > 0);

  15:  

  16:     Console.WriteLine("Retrieved site title:" + web.Title);

  17:   }

  18:  

  19: }

Introducing the Claims Friendly Web Client: ClaimsWebClient

The other thing I give you is a special version of the WebClient class that’s makes working with claims a bit easier. It has a single property where you specify the CookieContainer that will contain the SAML token. The library exposes the samples Wictor Wilen provided on to do the authentication for you and generate the CookieContainer:

   1: namespace CriticalPathTraining.Office365.AuthLibrary {

   2:   public class ClaimsWebClient : WebClient {

   3:     /// <summary>

   4:     /// Cookies that should be included on every Web request.

   5:     /// </summary>

   6:     public CookieContainer CookieContainer { get; set; }

   7:  

   8:     /// <summary>

   9:     /// Override base GetWebRequest() method to always include 

  10:     /// cookies if they were specified.

  11:     /// </summary>

  12:     protected override WebRequest GetWebRequest(Uri address) {

  13:       var request = base.GetWebRequest(address);

  14:       if (request is HttpWebRequest && CookieContainer != null)

  15:         ((HttpWebRequest)request).CookieContainer = CookieContainer;

  16:  

  17:       return request;

  18:     }

  19:   }

  20: }

In the associated test project you’ll also find the usage for this as well:

   1: [TestMethod]

   2: public void ClaimsWebClientTest() {

   3:   // file to download

   4:   string fileToDownload = "/_layouts/images/siteIcon.png";

   5:  

   6:   var claimsHelper = new MsOnlineClaimsHelper(

   7:                             MSO_USERNAME, 

   8:                             MSO_PASSWORD, 

   9:                             MSO_ROOT_SPSITE_URL);

  10:   using (var webClient = new ClaimsWebClient() {

  11:     CookieContainer = claimsHelper.CookieContainer

  12:   }) {

  13:     // get the file

  14:     var fileStream = ((ClaimsWebClient)webClient).OpenRead(

  15:         string.Format("{0}{1}", MSO_SPSITE_URL, fileToDownload)

  16:     );

  17:         

  18:     // download & write local        

  19:     string tempFilePath = Path.GetTempFileName();

  20:     var tempFile = File.Open(tempFilePath, FileMode.OpenOrCreate);

  21:     fileStream.CopyTo(tempFile);

  22:     fileStream.Close();

  23:     tempFile.Close();

  24:  

  25:     Console.WriteLine("Downloaded file to:" + tempFilePath);

  26:  

  27:     // make sure file exists & bigger than 0 bytes

  28:     Assert.IsTrue(File.Exists(tempFilePath));

  29:     var fileInfo = new FileInfo(tempFilePath);

  30:     Assert.IsTrue(fileInfo.Length > 0);

  31:   }

  32: }

Last but not least, for completeness I threw in a test for working with any of the SharePoint *.ASMX or *.SVC Web services. You don’t need any special helpers here as they include a CookieContainer class already:

   1: [TestMethod]

   2: public void WebServiceTest() {

   3:   XmlNode results;

   4:  

   5:   var claimsHelper = new MsOnlineClaimsHelper(

   6:                             MSO_USERNAME, 

   7:                             MSO_PASSWORD, 

   8:                             MSO_ROOT_SPSITE_URL);

   9:   using (var client = new Lists() {

  10:     Url = string.Format("{0}_vti_bin/Lists.asmx", MSO_SPSITE_URL),

  11:     UseDefaultCredentials=true,

  12:     CookieContainer = claimsHelper.CookieContainer

  13:   }) {

  14:     results = client.GetList("Shared Documents");

  15:   }

  16:  

  17:   Assert.IsNotNull(results);

  18: }

This post is part of a series on Office365/SharePoint Online, Windows Azure and Authentication. The other posts in this series are as follows:

• O365/SPO + Azure + AuthN - About this Series
• O365/SPO + Azure + AuthN - Overview of Challenges with O365, SPO & MSO
• O365/SPO + Azure + AuthN - How to Authenticate with O365, SPO & MSO
• O365/SPO + Azure + AuthN - Workarounds and Fixes for Claims-Based Auth Sites
• O365/SPO + Azure + AuthN - Critical Path Training's Office365 AuthN Helper Library

Now let's see how we can address the authentication fixes for each of the different ways you can access SharePoint remotely. In this post I'll cover each of the specific tools (REST or OData / CSOM / Web Services / WebClient) and how to address each of the tricks. Each one has it's pros & cons, hence why I had to use all four tools in my demo in my breakout session Out of the Sandbox and into the cloud: Build your next SharePoint app on Azure at the Microsoft SharePoint Conference 2011 (see that link for where you can download the sample).

Any code samples I show in this post were taken from my session Out of the Sandbox and into the cloud: Build your next SharePoint app on Azure at the Microsoft SharePoint Conference 2011… you can get the demo code from the Critical Path Training site’s Members section… look for the AC's SharePoint Conference 2011 Sessions download in the Presentations section.

For all the samples below, I created a private property in my class called MsftOnlineClaimsHelper that creates a local instance of the MSO helper and automatically authenticates.

   1: private MsOnlineClaimsHelper _claimsHelper;

   2: /// <summary>

   3: /// Microsoft Online claims helper used to authenticate to SharePoint Online.

   4: /// </summary>

   5: private MsOnlineClaimsHelper MsftOnlineClaimsHelper {

   6:   get {

   7:     if (_claimsHelper == null) {

   8:       _claimsHelper = new MsOnlineClaimsHelper(

   9:                 RoleEnvironment.GetConfigurationSettingValue("SharePointUsername"),

  10:                 RoleEnvironment.GetConfigurationSettingValue("SharePointPassword"),

  11:                 RoleEnvironment.GetConfigurationSettingValue("SharePointRootSiteUrl"));

  12:     }

  13:     return _claimsHelper;

  14:   }

  15: }

CSOM Client Context & CBA Challenges

One of the most common ways to work with SharePoint 2010 from off the SharePoint server is using the CSOM. Authentication with the CSOM is pretty straight forward using the ClientContext object. The trick comes into play with claims based authentication (CBA).

When you want to switch to FBA it's a simple property switch on the ClientContext, but as I previously stated there is no such way to do this for CBA. What you need to do is rewire the ClientContext so that every request it makes to a site collection includes a SAML token to authenticate the request. You do this by trapping the ExecutingWebRequest event of the ClientContext and injecting the cookie container generated by the MSO helper into all requests:

   1: private ClientContext _clientContext;

   2: /// <summary>

   3: /// CSOM client context.

   4: /// </summary>

   5: private ClientContext CsomClientContext {

   6:   get {

   7:     if (_clientContext == null) {

   8:       _clientContext = new ClientContext(

   9:         RoleEnvironment.GetConfigurationSettingValue("SharePointSiteUrl")

  10:       );

  11:  

  12:       // wire up claim helper to include SAML tokens (cookies) in all requests

  13:       _clientContext.ExecutingWebRequest += 

  14:             (webRequestSender, args) => {

  15:                 args.WebRequestExecutor.WebRequest.CookieContainer 

  16:                     = MsftOnlineClaimsHelper.CookieContainer;

  17:             };

  18:     }

  19:     return _clientContext;

  20:   }

  21: }

Web Request & CBA Challenges

The way you can address the lack of passing along the SAML token to SPO when you try to open and download a file using the File.OpenBinaryDirect() method is to simply create a simple Web request that will download the file. However this process also needs a little bit of work to pass along the SAML token. What I did was create a custom version of the WebClient class that did this for you as follows:

   1: namespace AndrewConnell.ACsCichlids.StoreFront {

   2:   public class ClaimsFriendlyWebClient : WebClient {

   3:     private CookieContainer _cookieContainer;

   4:     public CookieContainer CookieContainer {

   5:       get { return _cookieContainer; }

   6:       set { _cookieContainer = value; }

   7:     }

   8:  

   9:     protected override WebRequest GetWebRequest(Uri address) {

  10:       var request = base.GetWebRequest(address);

  11:       if (request is HttpWebRequest)

  12:         ((HttpWebRequest)request).CookieContainer = _cookieContainer;

  13:  

  14:       return request;

  15:     }

  16:   }

  17: }

This method is handy when you want to download a file from a site collection. To use it you simply pass in the MSO helper's cookies and they will be included on all requests:

   1: using (var webclient = new ClaimsFriendlyWebClient() 

   2:     { CookieContainer = MsftOnlineClaimsHelper.CookieContainer }

   3:   ) {

   4:       // download file into a memory stream

   5:       var fileStream = ((ClaimsFriendlyWebClient)webclient).OpenRead(cichlidPicture.OriginalUri);

   6:  

   7:       // create & save blob

   8:       var blob = AzureStorageContainer.GetBlobReference(cichlidPicture.ImportedFilename);

   9:       blob.UploadFromStream(fileStream);

  10: }

REST / OData / Web Services & CBA Challenges

My preferred way to read/write data to SharePoint lists is using the RESTful OData service ListData.svc. This service, like all the other Web services that are included with SharePoint 2010 (*.ASMX & *.SVC), don't understand claims by default. When you want to authenticate for a Windows or FBA site you have to create a network credential object and set it as a property on the service proxy.

However this isn't available when it comes to authenticating with CBA. Like the ClientContext, you need to rewire the calls to make sure they include the SAML token in each request. This is pretty simple as most services, like the Lists.asmx service, includes a CookieContainer property:

   1: XmlNode results;

   2:  

   3: // call lists.asmx web service to get attachments for each item 

   4: //    use same "cookie container" technique to authenticate

   5: using (var client = new Lists() {

   6:   Url = siteUrl + "/_vti_bin/Lists.asmx",

   7:   UseDefaultCredentials = true,

   8:   CookieContainer = MsftOnlineClaimsHelper.CookieContainer

   9: }) {

  10:   results = client.GetAttachmentCollection(

  11:     RoleEnvironment.GetConfigurationSettingValue("SharePointManagementListName")

  12:     , listItemId.ToString()

  13:   );

  14: }

There you have it... hopefully this series & code samples will help you authenticate into your site collections in SharePoint Online!

This post is part of a series on Office365/SharePoint Online, Windows Azure and Authentication. The other posts in this series are as follows:

• O365/SPO + Azure + AuthN - About this Series
• O365/SPO + Azure + AuthN - Overview of Challenges with O365, SPO & MSO
• O365/SPO + Azure + AuthN - How to Authenticate with O365, SPO & MSO
• O365/SPO + Azure + AuthN - Workarounds and Fixes for Claims-Based Auth Sites
• O365/SPO + Azure + AuthN - Critical Path Training's Office365 AuthN Helper Library

In my previous post in this series I talked about how authentication works with Office365 (O365) & SharePoint Online (SPO). You first authenticate with Microsoft Online (MSO) and then your browser passes along the SAML token with each request to site collections in SPO.

The first thing we need to do when accessing SPO programmatically is to understand how we authenticate with MSO and get the SAML token. Thankfully Microsoft provides a sample to help with the authentication piece. What you need is to make sure you have the Windows Identity Framework Runtime installed first. The main class that Wictor Wilen provides for authenticating to MSO is MsOnlineClaimsHelper. This class takes three parameters in the constructor:

• MSO Username of the account to authenticate with
• MSO Password of the account to authenticate with
• URL of the root site collection for your SPO account - note this is the root site collection which may not necessarily be the site collection you are interested in, but it is needed to authenticate and get the key

The MsOnlineClaimsHelper class has a public property, CookieContainer (which is also of type System.Net.CookieContainer), that you'll use to get the SAML token from MSO to attach to future requests. When the CookieContainer property is accessed, if null, the MsOnlineClaimsHelper will go authenticate and parse the results to extract the SAML token and store it in the cookie container.

Check out the code sample I mentioned previously in this post. What you're looking for are two files in the MSDN folder of the ACsCichlids.StoreFront project & the CSOM redistributable. These two files are the aforementioned MsOnlineClaimsHelper.cs & a file it depends on, WcfClientContracts.cs. Copy them into your project and that's all you need in order to authenticate.

Here's just a little code from a simple console app that shows me authenticating into my SharePoint Online site collection:

   1: static void Main(string[] args) {

   2:  

   3:   string msoUsername = "jack.sparrow@aconn.onmicrosoft.com";

   4:   string msoPassword = "[password]";

   5:   string msoRootSpSite = "https://aconn.sharepoint.com/";

   6:  

   7:   var msoHelper = new MsOnlineClaimsHelper(

   8:                           msoUsername,

   9:                           msoPassword,

  10:                           msoRootSpSite);

  11:  

  12:   CookieContainer cookieJar = msoHelper.CookieContainer;

  13:  

  14: }

This post is part of a series on Office365/SharePoint Online, Windows Azure and Authentication. The other posts in this series are as follows:

• O365/SPO + Azure + AuthN - About this Series
• O365/SPO + Azure + AuthN - Overview of Challenges with O365, SPO & MSO
• O365/SPO + Azure + AuthN - How to Authenticate with O365, SPO & MSO
• O365/SPO + Azure + AuthN - Workarounds and Fixes for Claims-Based Auth Sites
• O365/SPO + Azure + AuthN - Critical Path Training's Office365 AuthN Helper Library

Creating applications in SharePoint 2010, specifically the hosted environment Office365/SharePoint Online, when you want to communicate with applications and services in Windows Azure you are limited to client-side communications or using Business Connectivity Services (BCS). This is because all server-side code must run in the sandbox and that blocks all external web service calls.

What's the Problem?

However when you create an application in Windows Azure, or anything that wants to talk to SharePoint Online, regardless of the tools you use to connect to SharePoint Online (REST or OData / CSOM / Web Services / WebClient) you'll quickly find you are having authentication issues. The reason for this is because SharePoint Online is configured for claims authentication and use Microsoft Online (MSO) for authentication. That's why when you go to your SharePoint Online site, you are sent to what looks similar to a LiveID login page but it is branded for Office 365:

So…. what's the problem? The root problem is that none of the tools I mentioned above (REST or OData / CSOM / Web Services / WebClient) understand how to authenticate to a system that's configured for claims based authentication (CBA). Thankfully each one can be rewired a bit as to make it work but it takes a bit of work.

Quick Primer on Claims Based Authentication

If you aren't versed in CBA, let me point to two great links where you can get more information. Check out  MSDN's Patterns & Practices group: An Introduction to Claims but my favorite explanation one is by Donovan Follette's real world  example here. Basically here's how it works, borrowing a picture from the P&P article:

The user tries to hit a resource that's protected by CBA such as a site collection in SharePoint Online (SPO). Because the user isn't authenticated, the user is sent to a login page which in this case is Microsoft Online (MSO). The user then logs (#1) in and is given a  token (#2) that's in the format of Security Assertion Markup Language (SAML). The user puts this token in a cookie in the HTTP header and goes back to the SPO site collection (#3). Because the site collection gets the authentication token and the hosting SharePoint installation trusts the source that created the token (using some digital signatures & trust configurations that were setup ahead of time by administrators), the user is allowed in.

OK… So… How Do you Fix the Tools for CBA?

Here's the root of the problem: none of the four tools know how to authenticate with MSO and how to pass the SAML token along to the target system. The light bulb might have just gone off for a few of you… you may have been trying to programmatically authenticate with MSO and then make your request only to get a 401 - Access Denied. The reason you get that is because you aren't passing along your token in the header of your requests… you're being denied because you're trying to hit the resource as an unauthenticated (aka: anonymous) user.

1. Programmatically authenticate with MSO
2. Parse the response to get the SAML token
3. Attach the token to all requests to site collections in SharePoint Online

• MSDN:
  • Remote Authentication in SharePoint Online Using Claims-Based Authentication
• Steve Peshka: Mr. Claims for SharePoint
  • Retrieving Data from a Multi Auth Site Using the Client OM and Web Services in SharePoint 2010
  • Using the Client Object Model with a Claims Based Auth Site in SharePoint 2010
  • Retrieving REST Data in a Claims Based Auth Site in SharePoint 2010
• Wictor Wilen: SharePoint MCM & MVP who created some great code samples
  • Presentations and code for Office 365 and Windows Azure sessions from TechDays 2011
  • How to do active authentication to Office 365 and SharePoint Online
• Chris Johnson: Former Microsoft Sr. Technical Project Manager who created a nice rollup app demonstrating what you learned from above links
  • Authentication with SharePoint Online and the Client Side Object Model
  • Part 2: “Headless” Authentication with SharePoint Online and the Client Side Object Model

Up Next…

Now that you understand how the authentication works & the challenges each of our client tools & techniques has with it, we need to take a moment to get past the first step: authenticate with MSO to get a SAML token!

In the previous series I talked about how you can leverage both SharePoint 2010 and Windows Azure to create compelling & powerful integrated solutions. In the introduction post I talked about how I did a session at Microsoft's SharePoint Conference 2011 on integrating the two. In building out the demo for that I ran into some challenges when I tried to have my custom Windows Azure solution talk to and interact with a hosted SharePoint 2010 site collection living in my Office 365 SharePoint Online space. I ran into a few challenging spots when I tried using any of the four remote data access techniques available to developers:

• REST / OData / WCF Data Services (ListData.svc)
• Client Side Object Model (CSOM)
• Web Services (ASP.NET ASMX / WCF SVC)
• Web Requests (WebClient)

This blog series will talk about some of these challenges and how to overcome them. The last post will introduce a little helper library you can get for free from the Critical Path Training Member's only download site. Let's get to it! 

• O365/SPO + Azure + AuthZ - About this Series
• O365/SPO + Azure + AuthZ - Overview of Challenges with O365, SPO & MSO
• O365/SPO + Azure + AuthZ - How to Authenticate with O365, SPO & MSO
• O365/SPO + Azure + AuthZ - Workarounds and Fixes for Claims-Based Auth Sites
• O365/SPO + Azure + AuthZ - Critical Path Training's Office365 AuthN Helper Library