Access Denied Error with App Only Access Token When Reading Profile Info

This is yet another rather strange error that I ran across and couldn’t find any info out on the interwebs about it so I though I would document it here.  Suppose you have a SharePoint App that needs to access some User Profile information.  You will probably use the PeopleManager class and ask for user profile properties using the PersonProperties class or one of the methods off of the PeopleManager class.  You write your code up using the standard TokenHelper semantics to get a user + app key to retrieve this information, i.e. something like var clientContext = spContext.CreateUserClientContextForSPHost().  In your AppManifest file you ask for (at a minimum) Read rights to User Profiles (Social).   Works great, okay, good start.

Now you determine that you need to retrieve that same information but use an App Only token.  So you use whatever method you want to get an App Only token.  You use the same code but now you get an Access Denied error message.  Why is that – App Only tokens are supposed to have the same or greater rights than user + app tokens.  Well…for right now…I don’t know why not.  NOTE:  I DO understand needing to be a tenant admin to install an app that requires access to User Profiles, but this is different; it happens after the app is installed.  But I do know how I fixed it.  I added Tenant…Read rights to my AppManifest file.  Now my App Only token is able to read properties from the User Profile in o365.  Just thought I would share this “not at all obvious” tip so that if you get stuck hopefully your favorite search engine will find this post.  Happy coding!

Updating Trust Between OnPrem Farms and ACS for Apps When Your SharePoint STS Token Signing Certificate Expires

For those of you who are “in the app way” with SharePoint 2013 (no, not a lot different from being “in the pregnant way”, as they say), you’ll reach that point sooner or later where the token signing certificate for your SharePoint STS expires if you are using low trust apps on premises.  Why is that?  Because in order to establish the trust with ACS that is necessary to use low trust apps on prem, you had to replace the token signing certificate that comes out of the box.  Most folks simply use a self-signed certificate, which when you get it from the IIS Manager snap-in is good for one year.  Even if you get it from a public root authority, they are typically good for a year or two, although there are exceptions.

All that being said, when it comes time to change your STS token signing cert then you will need to re-establish that trust between your on premises farm and ACS.  Having just beat my head against the wall for the better part of a day to get this to work I thought I’d share some tips that I used to get me through it.  Here are some things you should be looking at:

  1. Get the latest version of the PowerShell modules for MSOL.  For some reason this is always harder than it should be, so again, to save you some time, here’s where I went to get the latest versions:  http://msdn.microsoft.com/en-us/library/azure/jj151815.aspx .  This is the page that describes managing Azure Active Directory using PowerShell and it seems to consistently have the “right” links.  On this page you will find a link to the Microsoft Online Services Sign-On Assistant, which you must get and install first. Right below that you will find a link for the 32-bit and 64-bit version of the PowerShell module for Azure Active Directory.  I use this one because it has all the MSOL goodness we require.  One other thing to note – you must first uninstall your existing version, and it will tell you that you need to reboot after doing so.  Yeah, you need to do this.  You may or may not get prompted to reboot again after installing it.  The net of all this is that I tried to skate by and skip the reboots a couple times and then the Azure AD module just flat out wouldn’t install.  One reboot later everything installed just fine and using the new modules cleared the first set of roadblocks I had been hitting.
  2. Start with the scripts for establishing trust that we previously published here:  http://msdn.microsoft.com/en-us/library/dn155905.aspx.
  3. There are two modifications to the scripts you will need to make; without them you will get errors and stuff will still be broken.  Here are the changes:

    1. Find this line of in the main script:  [Guid[]] $ExistingKeyIds = Get-MsolServicePrincipal -AppPrincipalId $SP_APPPRINCIPALID | Get-MsolServicePrincipalCredential | % KeyId.  Comment it out (by putting a # sign in front of it) as well as the line that follows it:  Remove-MsolServicePrincipalCredential -AppPrincipalId $SP_APPPRINCIPALID -KeyIds $ExistingKeyIds.  At some point you may end up with key ID that is null, and that causes this part of the script to break.  I fixed it with some other PowerShell I wrote up, so that chunk of the script now looks like this:

#Remove existing connection to an Office 365 SharePoint site if the switch is specified.
    if ($RemoveExistingAADCredentials.IsPresent -and $RemoveExistingAADCredentials -eq $true) {
        #[Guid[]] $ExistingKeyIds = Get-MsolServicePrincipal -AppPrincipalId $SP_APPPRINCIPALID | Get-MsolServicePrincipalCredential | % KeyId
        #Remove-MsolServicePrincipalCredential -AppPrincipalId $SP_APPPRINCIPALID -KeyIds $ExistingKeyIds

    [string[]] $ExistingKeyIds = get-MsolServicePrincipalCredential -ReturnKeyValues $returnIds -AppPrincipalID $SP_APPPRINCIPALID | % KeyId

    foreach($id in $ExistingKeyIds)
    {
    If ($id)
    {
        Remove-MsolServicePrincipalCredential -AppPrincipalId $SP_APPPRINCIPALID -KeyIds $id
    }
    }
    }

b.  There is a second script that enumerates all of your web apps and adds each one as a ServicePrincipalName (SPN) to the AppPrincipal.  If you’ve already added those SPNs before though, then it causes all sorts of problems and you will end up getting 400 Bad Request errors anyways (as I described here:  http://blogs.technet.com/b/speschka/archive/2013/07/29/security-in-sharepoint-apps-part-3.aspx).  To fix it you just need to add one line of code to the PowerShell so it will remove the SPN before adding it.  So if the SPN already exists it will be removed before adding, and if it doesn’t exist it just gets added.  Again, the net is you end up with a working script and one SPN for each web app in your farm.  Here is the section of that script with the new line of code:

Write-Host “Adding SPN for” $NewSPN

#here’s the new line of code
$SPAppPrincipal.ServicePrincipalNames.Remove($NewSPN)


$SPAppPrincipal.ServicePrincipalNames.Add($NewSPN)
Set-MsolServicePrincipal -AppPrincipalId $SPAppPrincipal.AppPrincipalId -ServicePrincipalNames $SPAppPrincipal.ServicePrincipalNames

After doing ALL of these things, my low trust apps on premise are working again.  Hope your sailing goes a little smoother.


A Few Additional Notes

You may still experience some issues after using this code.  Here are a few suggestions:

  1. Try using the /_layouts/15/appregnew.aspx page to create a new app principal.  If things are messed up then you will see an error there, like invalid JWT token.
  2. If appregnew.aspx is not working look in the ULS logs.  You should be able to track down the invalid JWT token, and you may see a subsequent error along the lines of request aborted.  I call this part of the error out specifically because if you are just trying to deploy a new app from Visual Studio you will also see this error in there – request aborted, request was cancelled, something like that (sorry I don’t have the exact wording handy).
  3. If you are getting the invalid JWT tokens I recommend you do this:Also, I recommend when you run the script to create the trust you include the -SharePointWeb parameter.  That parameter should be the Url to a web app on your on prem farm.  If you don’t include it then you will get one randomly selected and I actually ran into a problem with that scenario in one of the farms where I was doing this fix up.
    1. Manually remove the SPServiceApplication “ACS”; use this PowerShell script to do it:  $p = Get-SPServiceApplicationProxy | ? DisplayName -EQ “ACS”, then Remove-SPServiceApplicationProxy $p
    2. Manually remove the SPServiceApplication “ACS-STS”; the PowerShell script is the same as the previous step, but use -EQ “ACS-STS”.
    3. Rerun the script to create the trust but DO NOT include any of the -Remove* parameters

  4. When you run the script to create the trust include the -SharePointWeb parameter.  This should be the Url to a web application on your local farm.  If you don’t include this parameter then you will have a web app randomly selected, and this scenario actually caused another issue in one of the farms where I was applying these fixes.

Create a List in the Host Web When Your SharePoint App is Installed…and Remove it from the Recent Stuff List

This is one of those questions that I’ve seen asked a lot but for some reason I’ve never really seen anyone post an answer to it.  I did a quick Bing search before writing this post but didn’t really see anything up there so I’m going to go out on a bit of a limb and guess that maybe there’s still some use for this kind of information.  In my particular scenario I am creating a SharePoint app and when it’s installed I want it to create a list in the host web, because I’ll be using it to store data for my application.  I’m using a provider hosted app so I’m not going to have an app web, but even if I DID have an app web, I’d rather have this list in the host web in case my app is ever uninstalled and then reinstalled later, my application data will still be there and I can pick right up where I left off.

So to get started on this little adventure, I created a new SharePoint application in Visual Studio 2013.  It began as just your basic out of the box application – I just wanted to get it up and working first to verify that all the plumbing is in place.  With that ready to go, the next thing I wanted to do is have something similar to a feature receiver that you have for full trust code that would fire when my app is installed.  Fortunately the cloud app model provides just such a thing.  I just needed to select my SharePoint App project in Visual Studio and in the properties windows there is a property called HandleAppInstalled; just double-click that to change it from it’s default value of false to true and when you do a new WCF endpoint is added to your provider hosted app.  It’s all fleshed out with a basic stub implementation, and then you can add your code in there.  That’s what we’ll begin with.

The stub implementation looks something like this:

public SPRemoteEventResult ProcessEvent(SPRemoteEventProperties properties)
{
     SPRemoteEventResult result = new SPRemoteEventResult();

     using (ClientContext ctx = TokenHelper.CreateAppEventClientContext(properties, false))
     {
          //blah blah blah
     }

     return result;
}

Now the challenge is that the CreateAppEventClientContext is geared towards creating a context for an app web, which we don’t want.  So instead we’re going to create a client context the way that Visual Studio 2012 did when you create a provider hosted app, in the default.aspx code behind.  That means we’re going to need the Url to the host web, a context token, and the host name of our application.  Here’s how we’re going to get all of that stuff.

To begin with, you can get the Url to the host web in the SPRemoteEventProperties parameter that is passed into our method.  The properties parameter has an object property called AppEventProperties that contains a URI property called HostWebFullUrl.  So we can get the first parameter we need – the host web Url – like this:  Uri hostUrl = properties.AppEventProperties.HostWebFullUrl.  The context token is actually very easy to get; it’s a property of the properties parameter, so we can get it like this:  properties.ContextToken.  The last thing we need – the host name of our application – is more difficult to get in a WCF.  That’s because you don’t have an HttpContext in a WCF, so you have to reach back into the ServiceContext to get the information you’re looking for.  Fortunately the ServiceContext contains a Host property that has a collection of URIs in the BaseAddresses property.  Here’s a chunk of code to pull out that information:

System.ServiceModel.OperationContext oc = System.ServiceModel.OperationContext.Current;

Uri localUrl = null;

//enumerate through the Host base addresses and look for the SSL connection
foreach (Uri u in oc.Host.BaseAddresses)
{
     if (u.Scheme.ToLower() == “https”)
     {
          localUrl = u;
          break;
     }
}

UPDATE:  The code above looking through the BaseAddresses works great when you’re using it with the IIS Express web site Visual Studio adds to your project when you create a new SharePoint App.  HOWEVER…it blows up when you move your code to the full blown version of IIS.  What I found in trying to get this working is that the you still get two BaseAddresses; the first one uses the HTTP scheme but IS the correct host name for your WCF service.  The second one uses the HTTPS scheme, but is the fully qualified domain name of the server on which the code is running.  As a result, the code above picks the second host name, but that is NOT the host name registered for the endpoint with SharePoint.  As a result, the code below to get a client context fails and everything blows up at that point.  NOTE:  This is because my IIS web name is not the same as my machine name, which would generally be the case in a production environment.  The code I’m using now that works with both IIS Express as well as IIS is just this:

if (oc.Host.BaseAddresses.Count > 0)
     localUrl = oc.Host.BaseAddresses[0];

I’m looking for the host that is using HTTPS, since that’s what our apps should always communicate over.  It is arguably unlikely that you would have two different host names for HTTP and HTTPS, but you never know so why tempt fate?  Now that I have the three parameters I need I can create my client context for the host web; note that I always FIRST check to make sure that localUrl is not null, which would happen if I had no SSL endpoints on my provider hosted app.  Getting my client context now looks like this:

//this is what was originally here
//using (ClientContext ctx = TokenHelper.CreateAppEventClientContext(properties, false))
using (ClientContext ctx = TokenHelper.GetClientContextWithContextToken(hostUrl.ToString(), properties.ContextToken, localUrl.Authority))

Awesome!  Now that I have a client context for my host web, as long as my user / application has sufficient rights I can create a new list in my host web.  I’m not really going to cover that in great detail because I think creating lists via CSOM is all over the interwebs.  But for completeness here’s a shortened version of my code to create a new list:

Web web = ctx.Web;

ListCreationInformation ci = new ListCreationInformation();
ci.Title = LIST_NAME;
ci.TemplateType = (int)ListTemplateType.GenericList;
ci.QuickLaunchOption = QuickLaunchOptions.Off;
                                
l = web.Lists.Add(ci);

//add a description and some fields in here  
//blah                              
                                
l.Hidden = true;
l.Update();

//this creates the list
ctx.ExecuteQuery();

Okay, all of that is great, I now have my list in my host web.  There is one important aspect to this scenario though, and that is that my list should be hidden from users.  Now you might think that I have that covered with my ListCreationInformation (ci.QuickLaunchOption) and the Hidden property of the list…but that is not enough.  Unfortunately SharePoint still throws it in the Recent bucket that shows up on the Quick Launch navigation.  In fact the Recent list is NOT a list, it’s a collection of NavigationNode items.  So if you REALLY want your list to be invisible, you need to remove it from over there as well.  Doing that is three step process (NOTE: I’m leaving out all the error handling and try…catch blocks for read-ability; you would of course want that in any code you write…but you knew that, I know…  🙂  ):


Step 1:  Get the quick launch navigation node collection

In this step you’re going to make a call back to get the root web of the site to get the quick launch navigation nodes.  This code is just continued from the create list code above, so it’s still in the “using” statement for my ClientContext (ctx) shown above:

//get the site and root web, where the navigation lives
Site s = ctx.Site;
Web rw = s.RootWeb;

//get the QuickLaunch navigation, which is where the Recent nav lives
ctx.Load(rw, x => x.Navigation, x => x.Navigation.QuickLaunch);
ctx.ExecuteQuery();

//now extract the Recent navigation node from the collection
var vNode = from NavigationNode nn in rw.Navigation.QuickLaunch
     where nn.Title == “Recent”
     select nn;

NavigationNode nNode = vNode.First<NavigationNode>();

Step 2:  Get the Child Nodes of the “Recent” Navigation Node

Okay, now that I have the “Recent” navigation node, I need to populate it’s child property, which is where I should find a node for the list I just created.  This gets simpler now, here is the code to retrieve that:

//now we need to get the child nodes of Recent, that’s where our list should be found
ctx.Load(nNode.Children);
ctx.ExecuteQuery();

Step 3:  Find The Node for the New List and Delete It

Now that I have the collection of items in the Recent navigation, I can find my item and delete it like you would any other item via CSOM.  Here’s the code:

var vcNode = from NavigationNode cn in nNode.Children
     where cn.Title == LIST_NAME
     select cn;

//now that we have the node representing our list, delete it
NavigationNode cNode = vcNode.First<NavigationNode>();
cNode.DeleteObject();

ctx.ExecuteQuery();

And there you have it.  You’ve now created a new list in the host web, made it hidden and not shown on the quick launch bar, and removed it from the Recent navigation list so it is REALLY out of sight.

 

Missing Context Token in Low Trust App with SharePoint 2013 – Part 2

As I noted in my first post on this topic – http://blogs.technet.com/b/speschka/archive/2013/05/28/missing-context-token-in-low-trust-app-with-sharepoint-2013.aspx – you may find yourself in a low trust app without a context token, which basically leaves you up the creek without a paddle.  I ran across another scenario where this was happening so thought I would add it to my list of app model troubleshooting tips I’m keeping here on Share-n-Dipity.  In this case, I had added a AppInstalled receiver to my application, and some code within it was failing. There weren’t any unhandled exceptions, but nonetheless when the page in my provider hosted app was loaded, it contained no context token.

The way I finally started narrowing this problem down was to remove the Xml for the AppInstalled receiver from the AppManifest.xml file for my application.  Specifically, I commented out this:

<InstalledEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</InstalledEventEndpoint>

With that commented out, my page began getting the context token again.  That of course then allowed me to redirect my troubleshooting efforts towards my event receiver.  Overall it fits with your debugging 101 concepts of always trying to reduce scope to the smallest reproducible scenario.  Just add this to the list of things to check off if you find yourself missing a context token with a low trust app.

 

Getting the Current User Identity in a Low Trust App in SharePoint 2013

Today’s post comes primarily because it’s a common question, but not because it’s a particularly brilliant answer; maybe just a bit unexpected.  If you followed the some of the differences between low trust and high trust apps in SharePoint, you will know that in a low trust app, SharePoint knows who the user is, versus a high trust app where the app tells SharePoint who the user is – see Security in SharePoint Apps Part 3 for more details (http://blogs.technet.com/b/speschka/archive/2013/07/29/security-in-sharepoint-apps-part-3.aspx).

The common misconception here though is that you can look at the context token that SharePoint sends over to determine who the user is that is making the request.  I explain more about the context token in Part 4 of the Security in SharePoint Apps series:  http://blogs.technet.com/b/speschka/archive/2013/07/30/security-in-sharepoint-apps-part-4.aspx.  If you look at the contents of the context token you’ll see that there really isn’t a single thing in there that uniquely identifies an individual.  The solution to this problem then, is that you need to make a CSOM call to actually get the user’s identity.  Fortunately this actually pretty easy; what I’ve got here is a slightly modified version of the basic code that Visual Studio generates for you when you create a new low trust SharePoint App project in a provider hosted web:

var contextToken = TokenHelper.GetContextTokenFromRequest(Page.Request);
var hostWeb = Page.Request[“SPHostUrl”];

using (var clientContext = TokenHelper.GetClientContextWithContextToken(hostWeb, contextToken, Request.Url.Authority))
{
 clientContext.Load(clientContext.Web, web => web.Title, user => user.CurrentUser);
 clientContext.ExecuteQuery();
 Microsoft.SharePoint.Client.User curUser = clientContext.Web.CurrentUser;
 Response.Write(“Current user is ” + curUser.Email);
}

I highlighted the parts there to illustrate the main difference with the code that you get out of the box.  As you can see, you can just use the CurrentUser of the context web in order to get information about which user is actually using your application.  As I said, not difficult, but maybe not the answer you were expecting.

 

Another Tip if Visual Studio Times Out Trying to Deploy Your App in SharePoint 2013

I’ve had this happen to me, seemingly randomly, a number of times before.  You open Visual Studio and create a new SharePoint App.  You go through the wizard and then hit F5 to run, deploy and debug your app.  Something happens along the way though…you watch the output window and it just keeps counting time while it’s trying to deploy your app.  A few seconds, becomes a few minutes, which becomes several minutes, and then finally it just quits and says sorry, there was a deployment error.  Now no more walking away, looking at the sky and shouting “SERENITY NOW” friends – instead, here’s a tip that will hopefully help you.

When I had this happen to me (again) today, I changed the SharePoint site collection to which I was deploying the app to one I had successfully deployed before.  Sure enough, it worked like a champ.  Hmm, that got me to wondering.  Rather than looking at the whole side loading apps feature (which you really should not try and use as a work-around), I went to the problematic site collection and just went into the appregnew.aspx page (i.e. https://yourSiteCollection/_layouts/15/appregnew.aspx).  I generated a new client ID in there and registered my app, then I took that client ID and pasted it into my appmanifest.xml and web.config file.  I tried deploying to that site collection again and – voila! – sure as I’m the master of my own domain, the app deployed successfully and started running.

I’ve heard of many other folks hitting this issue…this is nearly as frustrating as those who can take the reservation, but can’t hold the reservation, so hopefully this will buy you some peace and happiness.

How To Control App Token Lifetimes in SharePoint 2013

Today’s post is the first selection from the little twitter contest I announced on the Share-n-Dipity blog a few days ago:  http://blogs.technet.com/b/speschka/archive/2013/09/04/use-social-tools-to-tell-me-what-you-want-to-see-here-next.aspx.  Shariq wanted to know more about the lifetime for high trust app tokens, as he tweeted here:

So we’ll look at where you can control that, but we’ll also see that at the end of the day it really doesn’t matter.  The keys to controlling the token lifetime for apps in a high trust app is in TokenHelper.  If you look at the IssueToken method you’ll see that there are one or two JWT tokens created – one if you are using an app only token, and a second if you are using tokens for the app and the current user.  In both cases it creates a JsonWebSecurityToken, and the fourth parameter used to create that token defines how long the token is valid for.  For the app token, TokenHelper uses a constant called TokenLifetimeMinutes that is set to a value of 1,000,000 by default.  Definitely not a value you should need to worry about in any scenario, unless of course you wanted to limit it (which is really the more interesting scenario).  For the user token, it is hard-coded by default to use a lifetime of 10 minutes.  So if you drill into the IssueToken method you can find those values and change them to whatever you want.  So why doesn’t it really matter?

Well, if you dig further into the code, you see that TokenHelper includes a delegate that’s invoked in the GetClientContextWithAccessToken method.  It’s whole role in life is to add a bearer token to any request that comes from your app into SharePoint.  You can see it here:

clientContext.ExecutingWebRequest +=
                delegate(object oSender, WebRequestEventArgs webRequestEventArgs)
                {
                    webRequestEventArgs.WebRequestExecutor.RequestHeaders[“Authorization”] =
                        “Bearer ” + accessToken;
                };

If you set a breakpoint on that line of code and then step through a request from your app, you’ll see that none of the token “setup” functions – i.e. those that create the ClientContext – actually result in an HTTP call.  It’s only when you call the ExecuteQuery method on the ClientContext object that a request goes over the wire, and then the delegate fires and the token is added to the request.  So getting back to really the intent of Shariq’s original question, while you CAN cache the access token you get in a high trust app, there’s really not a good reason to do so (at least that I can think of so far), because it doesn’t generate a call to SharePoint anyways.  Just for proving this out I wrote a little sample app where I played with setting the token lifetime for the user to 5 minutes, but I cached the access token for 1 year.  Sure enough, after 5 minutes had passed all of my app calls started failing with 401 – unauthorized errors, which is exactly what you would expect to see.  I’ve attached a zip file to this posting with that sample application.

Now…where caching app tokens DOES help is when you are using low trust apps.  In that case you absolutely do save one or more roundtrips to ACS to get an access token if you are caching them.  If you have lots of users and/or lots of requests this can really add up.  I won’t bother going into the details of how to cache tokens for low trust apps here, because I’ve already posted about that in the series on app security.  For more details, see part 4 of that series at http://blogs.technet.com/b/speschka/archive/2013/07/30/security-in-sharepoint-apps-part-4.aspx.

Hopefully that explains the concepts in a little more detail and will be useful to you in your application security designs.  That was a great question Shariq, and thanks for responding to the twitter ‘test.  Feel free to continue to pile your requests in there.

You can download the attachment here:

Security in SharePoint Apps – Part 8

PREVIOUS:  Security in SharePoint Apps – Part 7

For this, the very last in the series, I just wanted to briefly talk at a high level about the process you should be doing when developing an App for a SharePoint site that uses SAML (or FBA for that matter).  I’ve already written a detailed post on the “how” you develop an application for this scenario, I just want to cover the high level overview because I’ve seen some folks getting stuck trying to wrap their head around this and figure out how to even get started.  So here are the basic steps when you need to build this out.

The first caveat here is I’m going to assume that you already have SharePoint up and running, and one or more web applications configured to use SAML authentication.  Also…I strongly recommend that at least at first, you use both Windows and SAML authentication on the web site where you will be doing your development.  It will greatly simplify the process while you are trying to get everything up and running.  Now…knowing that, here’s how you can think about tackling this:

  • Phase 1 – A working SharePoint App.  During this phase you’re going to create a new SPTrustedSecurityTokenIssuer, if you don’t already have one.  That’s because in order to use Apps with SAML you must use high trust; low trust will not work.  Once you have that in place you should create a new SharePoint App using the provider hosted option with high trust.  At this point you should just hit F5 to validate that you can trust the App and the App security is configured correctly so that you can use CSOM to retrieve the web site title.  NOTE:  I have periodically experienced problems when trying to use Visual Studio to deploy an App to a SharePoint site that uses SAML.  If that happens, you should use appregnew.aspx to create a client ID for your application, then hard-code that client ID into your AppManifest.xml and web.config files in your solution.  You can get to appregnew.aspx in the _layouts directory; for example, if your site is at https://sps, then you would navigate to https://sps/_layouts/15/appregnew.aspx.
  • Phase 2 – A functional provider hosted app.   What I mean by “functional” is that you should create an ASP.NET application that is going to be your provider hosted app.  You should go through all of the steps necessary to configure it to use SAML authentication, including setting up a relying party for it in ADFS.  You should be able to independently navigate to that ASP.NET site and successfully authenticate using SAML authentication before you ever try and integrate with SharePoint.  Generally speaking you should find it easier to configure your provider hosted web to use SAML authentication if you use the Identity and Access tool for Visual Studio.  You can download it from http://visualstudiogallery.msdn.microsoft.com/e21bf653-dfe1-4d81-b3d3-795cb104066e.   Once this phase is completed, you should have a working SharePoint site, a working App, and a working ASP.NET site that will become your provider hosted app – it just isn’t hooked up to SharePoint yet.
  • Phase 3 – A working provider hosted app.  At this point you’re going to hook up your ASP.NET app to SharePoint so you can use it to start making CSOM calls.  I’ve actually already written a fairly detailed set of instructions about how you can take the web application that Visual Studio creates for you and port it over to a full blown ASP.NET app; look at this post for more details:  http://blogs.technet.com/b/speschka/archive/2013/06/12/converting-a-vs-net-web-for-a-provider-hosted-sharepoint-app-to-a-full-blown-web-or-console-application.aspx.   Follow the guidance in there for getting your ASP.NET application hooked up as a provider hosted app.  Then you can use my original post on building Apps for SharePoint sites that use SAML and FBA:  http://blogs.technet.com/b/speschka/archive/2012/12/07/using-sharepoint-apps-with-saml-and-fba-sites-in-sharepoint-2013.aspx.   At this point you should have end-to-end connectivity, authorization and authentication and can access content in your SharePoint site.

With that, I’ll conclude this series.  This topic is one that I expect will continue to rapidly evolve over the next few years because let’s face it – SharePoint Apps are really a v1 release, so they’re going to keep getting better and open up many new opportunities.  I hope you find this series helpful as you build your SharePoint Apps and figure out how make the security model work for you.

Security in SharePoint Apps – Part 7

PREVIOUS:  Security in SharePoint Apps – Part 6

In this part of the series, I’m going to shift gears a bit and talk about high trust apps and the plumbing that goes along with them.  As I’ve explained somewhat earlier in this series, one of the big differences between low trust and high trust apps is that high trust apps do not get a context token.  The context token contains information about the App and the user, so without that, a high trust app identifies both itself and the user to SharePoint.

The first issue to overcome is to get SharePoint to trust your App when it comes asking for content and says “hey I’m Contoso App and I’m asking for data on behalf of Steve”.  Without checks in place to make sure the request is coming from a trusted source, your farm would be wide open to information leaks.  In order to prevent that, just as you do with low trust Apps, high trust Apps have to use a trust broker.  For a high trust scenario, that “broker” is really just an X509 certificate.  I’ve been asked a few times about what kind of certificate it needs to be.  In all honestly, I haven’t really found one that doesn’t work yet.  My only recommendation here is that you create one specifically for this purpose – i.e. I would not try and take, for example, a wildcard SSL cert that I’m using on my web site and use it as a trust broker.  You can create a self-signed certificate, you can create a domain-issued SSL certificate, etc…but I would create one specifically for this purpose.

Once you have the certificate you want to use, you need to configure SharePoint to trust it and to use it – these are two different things!  The first thing you need to do is configure SharePoint to trust it.  This is the same process as you go through when you create an SPTrustedIdentityTokenIssuer for SAML claims – you get the token signing certificate and you add it to the list of trusted root authorities.  When you are creating a new trust broker you need to do the same thing – add your certificate to the list of trusted root authorities.  Also, just like with an SPTrustedIdentityTokenIssuer, if your certificate has parent certificates in its certificate hierarchy, you need to add ALL of them to the list of trusted root authorities.  I’ve explained this concept in more detail here:  http://blogs.technet.com/b/speschka/archive/2010/02/13/root-of-certificate-chain-not-trusted-error-with-claims-authentication.aspx.  

After you’ve done this, then you need to create your trust broker, which in SharePoint world is an SPTrustedSecurityTokenIssuer.  When you create an SPTrustedSecurityTokenIssuer, you are basically saying that you are going to have Apps submitting requests to SharePoint, and those requests are going to be signed with the public key of this certificate.  The only way you can sign something with the public key, is if you possess the private key.  That’s how SharePoint knows if the request comes in signed by this certificate, then it could have only come from someone who possesses the keys to that certificate.  In looking through some of my blog postings I realized that it doesn’t really seem like I’ve ever shared the PowerShell for creating one based on a certificate, so here goes:

#NOTE:  You can use the .CER file here, i.e. without the private key

$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(“C:\spapps.cer”)

New-SPTrustedRootAuthority -Name “Apps Certificate” -Certificate $certificate

#NOTE:  Create your own GUID and put it here, make sure

#all alpha characters are lower case

$issuerId = “e9134021-0180-4b05-9e7e-0a9e5a524965”

$spurl =”https://foo.contoso.com&#8221;

$spsite = Get-SPSite $spurl

$realm = Get-SPAuthenticationRealm -ServiceContext $spsite

$fullAppIdentifier = $issuerId + ‘@’ + $realm

New-SPTrustedSecurityTokenIssuer -Name “High Trust Apps” -Certificate $certificate -RegisteredIssuerName $fullAppIdentifier -IsTrustBroker

 

They key to this bit of PowerShell is the -IsTrustBroker flag when you create your SPTrustedSecurityTokenIssuer.  If you don’t include that flag, then that certificate can only be used with a single App, and that App would need to use the $issuerId as both the clientId and issuerId in the web.config file.  That’s not to say that there aren’t scenarios where you might want to do that, but it’s certainly not the mainstream case.  I’ll talk more about certificate management issues later in this post.

Once you’ve created your SPTrustedSecurityTokenIssuer, you’re ready to create your applications.  That process is somewhat more complicated than when you build a low trust App.  You’ll open up Visual Studio and select new SharePoint App, then in the first page of the wizard you’ll pick the site you want to deploy to during development and configure it as a provider hosted app.  On the second page of the wizard though, instead of selecting a client secret you’ll need to browse and select the certificate – the .pfx version – that is configured to be the SPTrustedSecurityTokenIssuer.  You’ll need to provide the password to open up the PFX file, as well as the issuerId associated with that certificate.  Here’s what the second page of the wizard looks like when you create a high trust app – again, you MUST use only lower case for your alpha characters in the Issuer ID field or you will get 401 unauthorized errors when access SharePoint:

There is one thing that’s very important to note here about the location of your token signing certificate…I’m seeing this issue come up more and more frequently as folks deploy their apps so make sure you are on top of this.  When you create your provider hosted app, Visual Studio will create a new web app project that uses IIS Express.  When you run that App, IIS Express is effectively running in the context of your own credentials.  So when IIS Express tries to load the pfx certificate to use for signing your application request, it all just works.  Where the problems tend to occur is when you deploy your application to production.  Now it’s running on IIS and it’s doing so in the context of an app pool.  I’ve seen many times where the app pool account does NOT have rights to the directory where the pfx file is kept.  As a result you will get all sorts of nasty untrapped errors (bad TokenHelper class, bad TokenHelper).  Make sure you have at least Read rights to the directory for your app pool account before you deploy.

Now that all your plumbing is in place and you’re ready to write your app, you need to remember what “high trust” really means.  As I’ve said before it means it’s up to YOU to tell SharePoint who the App is and who the user is to SharePoint.  You tell it who the App is by the clientId value in your web.config.  How you tell it the user identity is completely up to you.  If you use the out of the box TokenHelper class, it’s configured to use the current user’s Windows identity.  It doesn’t have to work that way though – you can modify TokenHelper and have it send an identifier for whatever user you want.  I say “identifier”, because really you are just sending over a claim value – nameid, SMTP, SIP or UPN – that SharePoint will use to “rehydrate” the user when the request is received.  For a more complete discussion on what rehydrate means, you can read this post:  http://blogs.technet.com/b/speschka/archive/2012/08/15/oauth-and-the-rehydrated-user-in-sharepoint-2013-how-d-they-do-that-and-what-do-i-need-to-know.aspx.   The net of it is you need to have an up to date User Profile Application (UPA) so when your request arrives at SharePoint, it can look in the UPA and find a profile that matches the claim type and value you sent in.  As a high trust app you can tell SharePoint that you’re the Queen of England or the Mayor of Steveville and it will just take your word for it.

Since I keep pointing this fact out to people, I thought that perhaps it might help to actually dig into TokenHelper or ClaimsTokenHelper and show you exactly where and how you do that.  In fact I’m going to show ClaimsTokenHelper here, because that is the class that I wrote to do this “stuff” for SharePoint sites that use SAML or FBA so I had a delightful time playing with this very fun feature.  J  If you open up ClaimsTokenHelper.cs, go into the IssueToken method (the TokenHelper class has this same method as well).  Here’s an abbreviated version of what it looks like:

List<JsonWebTokenClaim> actorClaims = new List<JsonWebTokenClaim>();

actorClaims.Add(new JsonWebTokenClaim(samlClaimType, samlClaimValue));

 

In this case I’m passing some variable so I can plug in whatever claim type and value I want.  However you could just as easily replace this line of code with something like this:

 

JsonWebTokenClaim(“http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress&#8221;,queen@england.com));

 

Congratulations – all of your users are now the Queen of England…as long as you have a user profile with an SMTP address of queen@england.com.

There’s one other point worth making about the clientId, which identifies the App.  Remember that when you are building your App in Visual Studio, VS will automatically generate a clientId and update the web.config, as well as configure the deployment site to ensure that everything works when you press F5.  Once you’re ready to deploy your App to production you need to take care of that yourself.  That means you need to go to a site and use appregnew.aspx to generate a new clientId for your application, republish your application using that clientId, deploy your App to the App Catalog, then install it in the site collection.  The use of appregnew.aspx is also covered in a little more detail in this post (as well as TechNet):  http://blogs.technet.com/b/speschka/archive/2013/02/18/when-do-your-apps-need-appregnew-aspx-in-sharepoint-2013.aspx.

Now that we’ve gone end-to-end on the high trust plumbing and application deployment, it’s worth taking a few minutes to come back to the topic of certificate management.  Generally speaking, as I’ve said, you want to use the –IsTrustBroker flag when you create your new SPTrustedSecurityTokenIssuer.  However you also need to understand the flip side of that arrangement – while every app can use that certificate as the trust broker, what happens if you want to stop trusting one particular App?  The only way to really do that is to remove the SPTrustedSecurityTokenIssuer.  When you do that of course, ALL of your Apps are broken.  To fix it, you would need to create a new certificate, create a new SPTrustedSecurityTokenIssuer, distribute the pfx version of the new certificate to all of your Apps, and then have all of the Apps change their issuerId to match the new SPTrustedSecurityTokenIssuer.  That’s potentially a lot of work.  Another option would be to create separate certificates based on your organizational hierarchy.  For example, you could create a separate certificate and SPTrustedSecurityTokenIssuer for each division, or each department, or however you organize your company.  That will reduce the impact when or if you want to stop trusting an App, but of course it also introduces some overhead you have to plan for – how are we going to track, store, and safeguard these certificates?  That’s an aspect of SharePoint Apps development that I really haven’t seen anyone address so far, which is kind of interesting in and of itself.  For more discussion on managing certificates you can also take a look at this article:  http://msdn.microsoft.com/en-us/library/jj945118.aspx.

That’s going to wrap up this discussion of high trust apps.  The next part will be the last part in this series, and it will be kind of short.  I’m going to talk briefly about developing Apps for SharePoint sites that use SAML authentication.  I already have a blog post with code that covers this in some detail, so in this last post I’m really going to look at it from a higher level.  The idea will be to set you up with the pieces you’ll need to put into place, what your high level plan should look like, etc. and not really the coding aspect since that’s covered in my other post.

NEXT:  Security in SharePoint Apps – Part 8

Security in SharePoint Apps – Part 5

PREVIOUS:  Security in SharePoint Apps – Part 4

As promised in Part 4, there’s actually a very interesting type of application that you don’t even install in a site collection.  How can this be you ask…well let’s talk about it. 

With most apps, you have to deploy it somewhere to be installed – the SharePoint Store or the app catalog.  A user then has to install that application by going to the Site Contents menu, finding the application they want, and installing it.  Once it’s installed, you will typically see a link (if it’s an App) somewhere in the site to launch it, or maybe it shows up looking like a web part (if it’s an App Part) or on the ribbon.  The key though is that they users go to their SharePoint site and then launch the application from there.

The exception to that is the focus of this posting, which is an app that uses “permissions on demand” (AKA “permissions on the fly”).  One of the biggest differences in the use case for it is that these applications typically initiate the connection to SharePoint, rather than the other way around.  For example maybe you have an ASP.NET web site and it needs to go out and pull in data for a user from SQL, from Facebook, and from a SharePoint list.  That’s a great use case for this kind of application.  There are a few steps and things to be aware of though, so let’s go through the process.

First, it’s important to understand that this kind of application can ONLY be used with low trust.  You cannot use a permissions on demand application with high trust.

Next, even though you don’t have to “install” the app per se, you need to start out by registering it with at least one site collection in your farm.  You can do that by going to any site collection, and navigating to the appregnew.aspx page.  For example, if you have a site collection at https://sps, then you would go to https://sps/_layouts/15/appregnew.aspx.  When you do the application registration there’s one important difference between how you register an application that’s going to be using permissions on demand: you need to provide a Redirect URI in addition to all of the other fields on the page.  The reason you provide this is because what is going to happen is that your app is going to make a request to SharePoint and say hey – I need to access this site for this user and I need this set of permissions.  The App won’t have an access token at first, so SharePoint is going to provide it with a code that it can use to get one.  The way SharePoint does that is that it puts the code in the query string and then redirects the request back to the Redirect URI that you have specified for the App.

The next thing that’s important to understand is that you decide when you make the request to SharePoint, what permissions you are going to ask for.  So how do you do that?  Well remember above I said that your App initiates the request to SharePoint, to get a code that it can use to get an access token.  When it makes that initial request it also tells SharePoint what rights it wants.  It does this by passing a parameter in a query string, where that parameter contains a set of one or more rights that are space delimited.  For example, if you wanted Read rights on a web, then you would specify Web.Read.  If you wanted rights to both read from the web and write to a list then you would ask for “Web.Read List.Write”.  You can get a complete list of all of the rights you can request at http://msdn.microsoft.com/en-us/library/jj687470.aspx.

There’s one more very important thing to understand about these kinds of Apps.  The first time it makes a request to a SharePoint site, you will get an App trust dialog – just like you do when you install an App from the App Catalog.  This is going to be as close as you get to “installing” the application – what it’s really doing is just creating an App Principal (which I described in Part 1).  The trust UI is slightly but not significantly different from a standard App trust dialog when you do this:

 

What’s especially important to remember here is that you must be a Site Owner or better in order to successfully trust this app.  Normally you might say hey, that’s okay, whenever I install an App you have to be a Site Owner to trust the app and then once that’s done everyone can use it.  That is not the case here.  EVERY user who uses the app will need to be able to trust the app.  In fact, if you do NOT cache the refresh token and/or access token for the user, that user will be asked to trust the App EVERY time they use the app.  This is important!  What that should tell you is that this kind of app is only going to be useful when you know your users are going to be Site Owners or better (like site collection administrator).  That obviously won’t make sense for every type of site collection out there, but it does for example work quite nicely with My Sites where you are always a site collection administrator by default. 

From a coding perspective, if you were to use the same page for both your code as well as your Redirect URI, it would look like this at a very high level:

if (Request.Cookies[myAccessTokenKey] != null)

{

//get your access token out of the cookie

}

else if (Request.Cookies[myRefreshTokenKey] != null)

{

               //get my refresh token out of the cookie

               //use the refresh token to get an access token

               //store the access token in the cookie (or wherever you want)

}

else if (Request.QueryString[“code”] != null)

//you don’t have a refresh token OR an access token; if you have “code” in the

//query string then you must have come from the SharePoint authorize page

//get a refresh token from the authorization code

//store the refresh token in the cookie (or wherever you want)

               //use the refresh token to get an access token

               //store the access token in the cookie (or wherever you want)

}

else if (Request.QueryString[“error”] != null)

{

               //if the query string contains an error key, then that means that the

               //SharePoint authorize page encountered a problem when we asked

               //it to issue a code.  A common cause would be if the current user

               //is not a Site Owner on the site we’re hitting up for data

               //do your error handling for this scenario here

}

else

{

//we have no access token, no refresh token, no errors and no

               //authorization codes that we can use to get one, so we need to

               //go ask for an authorization code.  Remember that every time we

               //do this, the user will be prompted to trust the application

 

//here is where we define as well what rights we want when we hit

//the site; you can use a space-delimited

Response.Redirect(TokenHelper.GetAuthorizationUrl(hostWeb, “Web.Read”));

}

I think the pseudo-code above should illustrate pretty clearly what you need to do, but here are a couple of points to add:

  • I’ve attached a sample program to this posting that has a complete implementation for you to review
  • You can get more details on permissions on demand and the specific steps you need to go through to configure at http://msdn.microsoft.com/en-us/library/jj687470.aspx.  My goal here was not to provide a step-by-step set of instructions because they already exist.  It was really to describe the concept and make you aware of the pros and cons, and generally how it works.

As I mentioned above, the “install” is not like a typical application install.  Normally when you install an application in a site, you can go into the Site Contents and you will see all of your Apps, including those that you have installed from the SharePoint Store or App Catalog.  Even after you trust a permissions on demand App, you will not see it in this list.  Remember I said though that it really just creates an App Principal?  Well the way you CAN see that is if you go to the appprincipals.aspx page in the _layouts directory.  So for example, if my SharePoint site is at https://sps, you would go to https://sps/_layouts/15/appprincipals.aspx.  That page also supports a query string variable called “Scope”.  If you wanted to see the App Principals that have Web-scoped rights, you would enter https://sps/_layouts/15/appprincipals.aspx?Scope=Web, and so on.  Here’s an example of what mine looks like after I’ve trusted my permissions on demand application, which is called Dynamo App:

That wraps up this part.  In the next part in this series we’ll look at some of the heavy hitters of SharePoint Apps – those that can access any content, anywhere.

NEXT:  Security in SharePoint Apps – Part 6