Even More “Things to Do When you Change the SharePoint STS Token Signing Certificate”

I recently changed the token signing certificate in my farm…it having expired from when I first changed it, since that is required to set up the trust with ACS in SharePoint Online for low trust apps and SharePoint hybrid features.  Assume the longer the sentence the more unhappy I am about having to do it…  🙂

Well, of course as luck would have it once I finished all that up, all of my low trust and high trust apps broke.  I expected my high trust apps to break, since I’ve blogged about this before.  I wasn’t really expecting the full breakdown of my low trust apps as well though, even though I’ve also had to blog about that scenario before as well:  https://samlman.wordpress.com/2015/03/02/updating-trust-between-onprem-farms-and-acs-for-apps-when-your-sharepoint-sts-token-signing-certificate-expires/.

Unfortunately, even using the tips in the previous blog post above did not fully solve my problems.  Amazingly, I was getting 401 Unauthorized errors in my low trust apps, which I traced back to a “certificate is not from a trusted root authority” in the ULS logs.  This is one of my favorite errors with the app model (ridiculously heavy sarcasm here), since they are so routinely generic and difficult to resolve.  Not only that, but the error is a little looney tunes in this scenario if you ask me.  What certificate is not trusted, and by whom?  Does my local SharePoint farm not trust it’s own SharePoint STS token signing cert?  That seems rather impossible.  Does it not trust the Microsoft SharePoint Online ACS certificate?  Equally illogical.  Does SharePoint Online ACS not trust my local farm’s STS token signing cert?  Seems like the only possible scenario.  Without going through a bunch of boring PowerShell I will just say that I ran the code (for a second and then third time) to create a New-MsolServicePrincipalCredential, without success.

Finally, I just ran the Connect-SPFarmToAAD cmdlet again, but this time with every possible “Remove*” option it offers:  -RemoveExistingACS -RemoveExistingSTS -RemoveExistingSPOProxy -RemoveExistingAADCredentials.  After doing that and trying my low trust application again, it started working.  Yay.  Yet another Apps for SharePoint tip to keep handy.

Changes To Know About When Your SharePoint STS Token Signing Certificate Expires – the Impact to Your High Trust Applications

This is a another follow up to my earlier post on changing the token signing certificate for the SharePoint STS as described here:  http://blogs.technet.com/b/speschka/archive/2014/05/06/updating-trust-between-onprem-farms-and-acs-for-apps-when-your-sharepoint-sts-token-signing-certificate-expires.aspx.  As I mentioned in that post, you will undoubtedly wind up in this situation if you configure your farm to use low trust apps, because as part of creating the trust with ACS you need to change the STS’ token signing certificate.  As part of that process, one of the side effects is that it changes the realm associated with your SharePoint farm.  The reason why that’s important to you as it relates to high trust apps is that the farm’s realm is part of the identifier used with the SPSecurityTokenIssuer used for high trust apps, as well as the identifier for individual apps.  For example, here’s what the identifier looks like for one of my SPTrustedSecurityTokenIsssuers:

9e000da5-800a-4f60-a855-057f5bf1d8ff@c98f41a0-e9d7-4d31-9ac4-813b81680177

The first part of the identifier (before the @sign) is the issuer ID for it; the part after the @sign is the realm for the farm.  The impact of this is that all of your high trust apps will no longer work after you change the token signing certificate for your farm’s STS.  The work-around to get things going is as follows:

  1. Delete your existing SPTrustedSecurityTokenIssuers (that you created, where you are using a certificate for the token issuer).  Use the Remove-SPTrustedSecurityTokenIssuer cmdlet.
  2. Re-create your SPTrustedSecurityTokenIssuers; you can use the same certificate and the same issuer ID as you had originally.  Use the New-SPTrustedSecurityTokenIssuer cmdlet.
  3. Re-create the client ID / application ID / App ID (these terms mean the same thing but are used fairly interchangeably) for each high trust app.  For example, go to the sites where you are using your apps, use appregnew.aspx to create a new App ID, then update your application to use the new App ID.

Once you do these steps your high trust applications should start working again.

SAML Support for SharePoint-Hosted Apps with ADFS 3.0

This is another case where I’m just passing information along here, based on the great work of others.  As you probably know, we did not have a good story for SharePoint-hosted apps in web application that uses SAML authentication with ADFS 2.0.  However, I have had reports from a couple of different teams now that they ARE working with ADFS 3.0.  The main differences that are needed to make this work include:

  • In ADFS you need to define a wildcard WS-Fed endpoint.  For example, normally for a SharePoint web application, in ADFS you create a relying party and set the WS-Fed endpoint to be something like https://www.foo.com/_trust/.  To do the same thing with apps, you take your apps namespace – assume it’s “contosoapps.com” – and add a WS-Fed endpoint like this:  https://*.contosoapps.com/_trust/.
  • Configure the SharePoint STS to send the wreply parameter.  You can do that with PowerShell that looks like this:

$sts = Get-SPTrustedIdentityTokenIssuer
$sts.UseWReplyParameter = $true
$sts.Update() 

One other thing to note – the behavior to use the wreply parameter is supposed to be turned on by default in an upcoming CU.  I heard it was the April 2014 CU actually but have not had a chance to see if that is really in there or not.  It won’t hurt to run the PowerShell above though.

This is good news, thanks for those of you that shared your experiences!

Debugging SharePoint Apps That Are Hosted In Windows Azure Web Sites

Today, I’m going to be the lazy human I’m so frequently accused of being by my somewhat faithful dog Shasta, and bring together two posts written by two other folks into one uber “ain’t it cool how this all works together post” by me.  Here are the two concepts we’re combining today:

Now, once our SharePoint App has been published to a Windows Azure web site, the error prone and/or forward-thinking amongst you may be wondering…um, great…so what do I do to track down bugs?  Well that’s where the second piece of brilliant advice that I had nothing to do with comes in.

Now, let’s briefly walk through the steps to combine these two nuggets of goodness:

  1. Create a SharePoint provider hosted app and verify that it works.
  2. Create an Azure web site and download publishing profile. (in Vesa’s video)
  3. Use appregnew.aspx to get a client ID and client secret. (in Vesa’s video)
  4. Publish the App to your Windows Azure site using the publishing profile, client ID and client secret retrieved in the previous steps. (in Vesa’s video)
  5. Create the app package, install it to your app catalog, and add it to your site. (in Vesa’s video)
  6. Open Server Explorer in Visual Studio 2013, right-click on the Windows Azure node and select Connect to Windows Azure…
  7. Expand to see all the Azure services, and then expand the collection of Web Sites.
  8. Right-click on the Azure web site where you published your provider-hosted app and select Attach Debugger.
  9. The browser opens to your Azure web site, and VS.NET starts up in debugging mode.  Set your breakpoints in your code and start debugging!

See the remotely debugging Azure web sites post for the details on pre-requisites, but in short you need Visual Studio 2013 and the Azure 2.2 SDK for VS 2013; you will find a link to that in the blog post. (NOTE:  that same post also describes how to do this with Visual Studio 2012 but I have not tried that)  This actually works pretty great and I was able to get a first-hand experience using it when I went through the steps for this blog post.  As it turns out, the SharePoint site where I installed my sample application uses the Url https://sps2.  Well, the problem of course is that in my Azure Web site, my code was trying to make a CSOM call to an endpoint at “sps2”.  That works great when I’m in my lab environment, but out in the interwebs that Azure lives in of course it cannot resolve to a simple NetBIOS name (remember, this code is running server side, not client side).  So as a result it was blowing up.  By using this cool new debugging feature I was able to find my issue, appropriately for this debugging post.  Here’s a screenshot of it in action:

 

Another 401 Unauthorized Tip for Working with SharePoint Apps

I’ve tried to update the related postings to this, but sometimes you need a call out to make sure it catches your attention so…here’s another troubleshooting tip for when you get a 401 unauthorized error when your app tries to access SharePoint content.  Suppose you have gotten everything working and then create a provider hosted app hosted in IIS instead of IIS Express (see here for more details on this process:  http://blogs.technet.com/b/speschka/archive/2013/06/12/converting-a-vs-net-web-for-a-provider-hosted-sharepoint-app-to-a-full-blown-web-or-console-application.aspx).  Despite that, when the code runs in your app you get access denied again.  Here’s the tip – set a breakpoint in TokenHelper.cs in the GetClaimsWithWindowsIdentity method, and then look at the identity.User.Value.  If you see a very short SID value, like S-1-5-17, then that probably means it’s the anonymous account for IIS.  The SID for a “real” user is much longer, something like S-1-5-21-1644491937-1935655697-1957994488-2138.  Remember that the way OAuth works (in the most common case with SharePoint Apps) is that it checks to make sure BOTH the app AND the user have rights to the content.  In most cases the anonymous user account will not have rights, and so you will get an access denied error message.  To fix this you need to go into IIS and find your provider hosted app, then disable Anonymous access and enable Windows authentication.  I recommend restarting the IIS virtual server for your hosted app, then try again.

Converting a VS.NET Web for a Provider Hosted SharePoint App to a Full Blown Web or Console Application

This post is yet another in my series of “how long of a post title I can come up with and still not convey the actual essence of the post” contest.  I’m sure you’ve all been following along at home and are thrilled to see another entry, but it was either this or create a title with like 63 words in it.  So…the point of this post is to take you through the typical development scenario for a SharePoint App – you open up Visual Studio 2012, you create a new SharePoint App project, and you configure it to be provider hosted using high trust (i.e. a certificate).  Now you have this SharePoint App, which is great, but you also have this little application web that VS.NET created for you and that uses a local IIS Express instance.  That’s actually also great to get you started and running, but of course you are going to have to move away from that as you go into QA and production (and honestly many times even in dev – in fact I almost always do this even in dev).  The process can be a little wonky to get down so I thought I’d document the steps for doing so.  I’m also going to show you how to make it work with a console application as well, because that can be a super useful scenario for doing things like timer jobs, etc. (perhaps the source of another post in the future…we’ll see, I’ve got kind of a cool solution going here).

So, how do we get from IIS Express to a SharePoint App as a full blown ASP.NET or console application?  Well, here goes:

  1. Create your application using the Visual Studio wizard.  It creates the SharePoint App as well as web application project that uses IIS Express.  From this point forward this web application project that VS.NET creates shall be referred to as the “original web app”.  Verify that everything is working correctly.
  2. Create your new application you are going to use – a web project or a console project.  Remember to use SSL if creating a web project.
  3. Copy the configuration information from the original web app; for:
    1. Web Project – copy the items in the <appSettings> of web.config to web.config of your new web project.
    2. Console App – Go into the Properties of the application, click on the Settings link on the left, then click on the link in the middle of the page that says “This project does not contain a default settings file.  Click here to create one.”  Create properties for each of the appSetting properties – copy in both the key (as the setting Name) and value (as the setting Value).  Make sure you make each setting an Application scope property (it is User by default).
  4. Create an App_Code folder and copy into it the TokenHelper.cs file from the original web project.  For a console application you can just create a new folder; for an ASP.NET application you should use the Add…Add ASP.NET Folder…App_Code menu option.
  5. Remove the “namespace” declaration from the TokenHelper.cs file.
  6. Add references so your application will compile; the exact ones will vary slighlty between an ASP.NET and console application, but here is a complete list so just make sure whichever application type you choose, you have all of these references:
    1. Microsoft.SharePoint.Client
    2. Microsoft.SharePoint.Client.Runtime
    3. Microsoft.IdentityModel
    4. Microsoft.IdentityModel.Extensions
    5. System.ServiceModel
    6. System.Web
    7. System.Web.Extensions
    8. System.IdentityModel

At this point, the steps diverge depending on what type of project you created.  If you created a web project:

  1. Create a default.aspx page (if you did not already – this assumes you are using ASP.NET web forms by the way and not MVC).
  2. Copy and paste the code from the code-behind in default.aspx in the original web app to your default.aspx.  You should not need to add a “using” statement for the App_Code folder.
  3. Open the AppManifest.xml file in your SharePoint App project and change the Start Page property to point to the default.aspx page you created in your new web project.
  4. In the IIS Manager, make sure that for your provider hosted site anonymous is disabled and Windows authentication is enabled.
  5. Press F5 to launch the Visual Studio .NET debugger, install the app and validate that the code works.

If you created a console app:

  1. Open the TokenHelper.cs file and edit it; find these lines of code:

        private static readonly string ClientId = string.IsNullOrEmpty(WebConfigurationManager.AppSettings.Get(“ClientId”)) ? WebConfigurationManager.AppSettings.Get(“HostedAppName”) : WebConfigurationManager.AppSettings.Get(“ClientId”);
        private static readonly string IssuerId = string.IsNullOrEmpty(WebConfigurationManager.AppSettings.Get(“IssuerId”)) ? ClientId : WebConfigurationManager.AppSettings.Get(“IssuerId”);

(and then a little below)

        private static readonly string ClientSigningCertificatePath = WebConfigurationManager.AppSettings.Get(“ClientSigningCertificatePath”);
        private static readonly string ClientSigningCertificatePassword = WebConfigurationManager.AppSettings.Get(“ClientSigningCertificatePassword”);

and replace them with these; note that you must replace “yourAppsNamespace” with the namespace for your application:

        private static readonly string ClientId = yourAppsNamespace.Properties.Settings.Default.ClientId;
        private static readonly string IssuerId = yourAppsNamespace.Properties.Settings.Default.IssuerId;

        private static readonly string ClientSigningCertificatePath = yourAppsNamespace.Properties.Settings.Default.ClientSigningCertificatePath;
        private static readonly string ClientSigningCertificatePassword = yourAppsNamespace.Properties.Settings.Default.ClientSigningCertificatePassword;

2.  The code you run in the console app is slightly different from a web application.  To simplify, I’m pasting here the code you get out of the box in default.aspx, and I’ve indicated with a comment above the lines of code that were changed to make this work in a console app.  Sorry, only C# here folks, if you want VB though this stuff should be easy to convert.

                // – NOTE:  You need to put the Url to the site collection you want to work with here
                Uri hostWeb = new Uri(“https://mySharePointUrl“);

                // – here we are using WindowsIdentity.GetCurrent instead of from an HttpRequest
                using (var clientContext = TokenHelper.GetS2SClientContextWithWindowsIdentity(hostWeb, WindowsIdentity.GetCurrent()))
                {
                    clientContext.Load(clientContext.Web, web => web.Title);
                    clientContext.ExecuteQuery();
                    // – here we are using Console.Write instead of Response.Write
                    Console.Write(clientContext.Web.Title);
                }

3.  Add a using statement to the program.cs (or .vb) file for System.Security.Principal
4.  Right-click on your console project and select the Set as Start Up Project option.
5.  Press F5 to launch console app and validate that the code works.

You should be good to go at this point.  I actually walked through this process with a console app while writing this post, so as every good developer says if you have problems…”well, it works on my machine!”  🙂  I’ve also used this method MANY times previously for web apps so this should be solid as well.

Missing Context Token in Low Trust App with SharePoint 2013

Should you decide that you want to write low trust apps for an on-premises SharePoint 2013 farm, there are a number of hoops you should expect to jump through (NOTE:  this assumes the SPNs for your web apps are already configured in the MsolServicePrincipal for your o365 tenant):

  1. Create a new app in Visual Studio; in the wizard you select provider hosted app on the first page, and then client secret on the second page.
  2. Add a New Web Site to your solution, configure it to use SSL, and add all the necessary references to it (usually Microsoft.IdentityModel, Microsoft.IdentityModel.Extensions, Microsoft.SharePoint.Client, and Microsoft.SharePoint.Client.Runtime).
  3. Copy the TokenHelper.cs file from the app web created by the Visual Studio wizard and paste it into your new web site – create an ASP.NET Folder in your project for App_Code and copy it in there, then remove the “namespace” from the file.
  4. Copy all the goo from the web.config in the app web that the Visual Studio wizard created, and paste it into the web.config for your new web site.
  5. Add a new web form to your new web site called Default.aspx.
  6. Copy the code from the default.aspx.cs file in the app web that the Visual Studio wizard created, and paste it into the default.aspx.cs file in your new web site.

Now…that seems like a lot of good stuff, and it is, but when you try and run the application you’ll probably get an error that looks like this:

Along with that, you’ll see the error message about the parameter ‘token’ cannot be a null or empty string (yeah, I just added that in for the search engines…you’re welcome  🙂  ).  If you try debugging it you’ll see that when the call is made in the code-behind to GetContextTokenFromRequest, the return value is null.  So this is basically saying you’re not getting the context token you need from SharePoint.  What this means is that in all likelihood you don’t have a synced up ClientId value between your new web app’s web.config, the AppManifest.xml file for the SharePoint App, and the SharePoint site. 

After trying a few different ways around this, I honestly found the best way to resolve this problem is to do the following – go into your site into the /_layouts/15/appregnew.aspx page.  Use that to generate a new ClientId and ClientSecret.  Complete that page to register those values for the App Principal in the site.  Now, paste the ClientId into the AppManifest.xml file (in the RemoteWebApplication element), and paste the ClientId AND the ClientSecret into the web.config for your new web application.

You will need to redeploy your SharePoint App at this point, which you can still do by pressing F5 in Visual Studio.  After that it *should* work, however you may get an Access Denied error message too (I had this happen once).  Just to make sure everything was squared away, I went into the Site Contents page, clicked the ellipsis next to my application, and then clicked on the Permissions link.  That page contains a link that says “If there’s something wrong with the app’s permissions, click here to trust it again”, and click it I did.  In all honesty, things get wrapped up enough that it may very well have worked if I had just tried it again, but it seemed like an interesting link.  🙂

Hopefully this gives you heads up and maybe a little background as you look to use low trust apps in provider hosted apps on premises.

Using REST Services with a Provider Hosted App in SharePoint 2013

Today’s topic seems straightforward enough, right?  There’s lots of documentation all over the interwebs about how to do this so should be a piece of cake.  Well, as it turns out, when I did this a couple of weeks ago I found that there is a lot of missing information, misleading information, and in some cases incorrect information out there about how to do this.  So I’m going to describe here the main points of how to get this to work and try and clear up a few gaps and what I call “tribal knowledge” about what is needed to make this work.

Let’s start by considering the scenario – you want to use a provider-hosted app to connect to SharePoint and retrieve data via search.  Now, by definition a provider hosted app is going to exist in a different domain from your SharePoint site where you are going to execute the query.  This introduces the first problem:  there are several docs out there that show REST calls being made using jQuery.  Guess what – that’s not going to work.  Why not?  Because your app is in a different domain from your SharePoint site, jQuery will fail by default with a “no transport” error.  Steve’s Seemingly Random Yet Useful Tip #1 – don’t bother configuring the cross domain property of your AJAX query with jQuery; it will still fail when you get over to SharePoint land.

Okay, so we determine that we’re in a different domain, then that must mean that we need to use the cross domain library that SharePoint offers in order to make this query work, right?  So you follow the instructions to modify your AppManifest.xml, you change your javascript to use the SP.RequestExecutor object, and yet your request fails again.  What’s the problem here?  Well, you can’t use the host web Url with SP.RequestExecutor, you must use an App Web.  But wait, provider-hosted apps don’t have App Webs, so what gives?  Well, the fact of the matter is you need to have your provider-hosted app create an App Web so that you can hook it up with the SP.RequestExecutor object.  Steve’s Seemingly Random Yet Useful Tip #2 – add an artifact to your SharePoint Application (not the web application that is your provider-hosted web app) that will force an App Web to be created.  In my case I just added a custom site column.  I don’t do anything with it, I don’t refer to it ever again, I just add it to the project, and that forces an App Web to be created.  That allows me to use this tiny piece of code to get the App Web Url and pass it to my SP.RequestExecutor constructor:  var appweburl = decodeURIComponent(getQueryStringParameter(“SPAppWebUrl”));

There’s also one other thing worth noting here.  When you want to use the cross domain library, you need to modify the AppManifest.xml by changing this:

  <AppPrincipal>
               <RemoteWebApplication ClientId=”*” />
  </AppPrincipal>
 

To this:

<AppPrincipal>
               <Internal AllowedRemoteHostUrl=”~remoteAppUrl”/>
</AppPrincipal>

Well guess what happens when you do that?  If you also want to use CSOM in your code behind with a high-trust app, you will no longer be able to do so. Your app won’t have the clientID it requires to make the connection to your SPTrustedSecurityTokenIssuer, so all of your CSOM calls will fail with authentication errors.  Steve’s Seemingly Random Yet Useful Tip #3 – it really does you no good in this case to make those changes to your AppManifest.xml to add support for the cross domain library; you’re better off just leaving it as is so you can also use CSOM and server-side code if needed.

Okay, so now we’re getting somewhere.  The last important thing to understand here though is that when you go to run your code, you are going to have a very sub-optimal user experience by default.  What do I mean by that?  Well when your code executes, your REST call is going to a different domain.  What you should expect to happen is that the end user then will get prompted for their credentials, even when using Windows auth.  This of course is a terrible thing to have happen – a user launches an app, they see a blank page and suddenly it’s asking for their Windows username and password.  Not only is it a jarring experience, but if you’re paranoid at all, the last thing you’re going to do is give your credentials when a blank browser window randomly pops up in front of you.  Steve’s Seemingly Random Yet Useful Tip #4 – (this assumes Windows auth because it’s simpler for now) – remember that your REST call is going to be processed by your App Web; to avoid those authentication prompts, add *.yourAppWebDomain.com to your list of Intranet Sites in IE.  That stops the authentication prompts from occurring and you get a seamless end to end user experience.

Finally, here’s an example of what my javascript actually ended up looking like to make the call to my app web from my provider-hosted app:

 

function getSearchInfo()
{
    try
    {
        //use the app web for the search endpoint
        var appweburl = decodeURIComponent(getQueryStringParameter(“SPAppWebUrl”));

 

        //create the Url for getting the info, needs to look like this:
        //https://sp5.vbtoys.com/_api/search/query?querytext=’Author:”Steve+Peschka”‘
        var searchHost = appweburl + “/_api/search/query”;

 

        //create the query terms
        var qry = “‘Author:\”” + $(“#authorName”).html() + “\”‘”;

 

        //create the concatenated query url
        var qryUrl = searchHost + “?querytext=” + qry;

 

        //initialize the RequestExecutor with the app web Url
        var executor = new SP.RequestExecutor(appweburl);

 

        //issue the query
        executor.executeAsync(
            {
                url: qryUrl,
                method: “GET”,
                headers: { “Accept”: “application/json; odata=verbose” },
                success: onSearchSuccess,
                error: onSearchError
            }
        );

 

    } catch (e)
    {
        //do something here
    }
}

More TroubleShooting Tips for High Trust Apps on SharePoint 2013

Hey, I’m an app guy, I like doing dev, but honestly – I may go hoarse screaming at my computer if I have to track down one more “The issuer of the token is not a trusted issuer” problem with my new SharePoint apps.  To try and help you save your own voice (and sanity) I’m going to start a list here of things that I look for when I get this issue.  As I discover new and exciting ways of both invoking this error and resolving it, I will just update the post here and throw an “UPDATED!” doo hickey below. 

It’s important to remember when I say “high trust app”, that means that you are NOT using ACS as the trust broker for your SharePoint app; instead your are creating the OAuth token and signing it with your own certificate.  I know we have this whole process documented out there somewhere so I’m not going to try and describe that here again.  I’m going to assume you read it, you’ve been trying it, and now you are ready to give the one-finger salute to your monitor.  So, that being said, here are some of the ways I’ve seen this error occur:

  1. Add to the SPTrustedRootAuthority list – when you use a certificate to sign your OAuth tokens, you need to create a New-SPTrustedRootAuthority.  Just like the New-SPTrustedIdentityTokenIssuer in SharePoint 2010, you need to add the token signing certificate and every certificate in the chain to SharePoint’s list of trusted authorities.  You can follow the same steps for this process as I described in this post:  http://blogs.technet.com/b/speschka/archive/2010/02/13/root-of-certificate-chain-not-trusted-error-with-claims-authentication.aspx.   Just ignore the stuff about exporting the certificate from ADFS – I’m assuming you created your certificate for your high trust apps via some other means – public root CA like GoDaddy, VeriSign, etc., self-signed cert, or domain issued cert.
  2. Client ID uses all uppercase characters – as I described in another post, make sure that you do NOT use uppercase letters when you plug in the client ID in the AppManifest.xml file of your application, or in the web.config of your web application if you are building a self-hosted solution.  For more details see this:  http://blogs.technet.com/b/speschka/archive/2012/07/28/an-important-tip-about-client-id-values-for-s2s-apps-in-sharepoint-2013.aspx.
  3. Token issuer not configured as trust broker – I also described this problem in another post here:  http://blogs.technet.com/b/speschka/archive/2012/09/27/another-apps-for-sharepoint-tip-with-the-error-quot-the-issuer-of-the-token-is-not-a-trusted-issuer-quot.aspx.  The net of this is make sure when you create your New-SPTrustedSecurityTokenIssuer that you include the -IsTrustBroker flag.
  4. UPDATED!:  Missing the IssuerId key in web.config – In order to use the trust broker feature of apps in SharePoint 2013, your application has to know what the IssuerId of the trust broker is when it creates the JWT token it sends to SharePoint.  The way it knows this is by looking in web.config for an app property called IssuerId.  It goes the same place in your application’s web.config as ClientId, ClientSecret, etc.  It just looks like this:  <add key=”IssuerId” value=”e9134021-0180-4b05-9e7e-0a9e5a524965″/>.
  5. UPDATED!:  Using the Office Tools RTM Preview for Visual Studio – there actually is a little bug in the RTM Preview bits, which is fixed in Preview 2.  The code that sends that authorization token to SharePoint doesn’t look for the IssuerId element in web.config, instead it sends a different value.  What it sends and why is not really important; what IS important is that the way you work around this with that version of the tools is to always use the IssuerId value for the SPTrustedSecurityTokenIssuer in the ClientId key in your web.config.  As you get the Preview 2 bits please install them right away and change the ClientId to a unique GUID that you create and register (with Register-SPAppPrincipal as explained below).  You don’t want the ClientIds to all be the same because it identifies what application signed the OAuth token, and is used throughout the SharePoint UI.  If you ever need to do any troubleshooting or auditing it will be a problem if all apps use the same value.  UPDATED Part 2!:  The Preview 2 bits are now available!  You can find them at http://www.microsoft.com/web/handlers/WebPI.ashx?command=GetInstallerRedirect&appid=OfficeToolsForVS2012GA.
  6. UPDATED!:  Authentication not Configured on Provider Hosted App – you may have gotten everything working and then create a provider hosted app hosted in IIS instead of IIS Express (see here for more details on this process:  http://blogs.technet.com/b/speschka/archive/2013/06/12/converting-a-vs-net-web-for-a-provider-hosted-sharepoint-app-to-a-full-blown-web-or-console-application.aspx).  Despite that, when the code runs in your app you get access denied again.  Here’s a tip – set a breakpoint in TokenHelper.cs in the GetClaimsWithWindowsIdentity method, and then look at the identity.User.Value.  If you see a very short SID value, like S-1-5-17, then that probably means it’s the anonymous account for IIS.  The SID for a “real” user is much longer, something like S-1-5-21-1644491937-1935655697-1957994488-2138.  Remember that the way OAuth works (in the most common case with SharePoint Apps) is that it checks to make sure BOTH the app AND the user have rights to the content.  In most cases the anonymous user account will not have rights, and so you will get an access denied error message.  To fix this you need to go into IIS and find your provider hosted app, then disable Anonymous access and enable Windows authentication.  I recommend restarting the IIS virtual server for your hosted app, then try again.

Now, there’s also a related issue worth noting:  suppose you “think” you’ve gotten past this error, but then you get an Access Denied error when trying to retrieve content from a SharePoint site in your self-hosted application?  Well what that can mean is:

  1. The ClientId value in your AppManifest.xml file for your SharePoint app does not match the ClientId value in the web.config for your self-hosted app.  We’re making improvements to the Visual Studio tools that should alleviate this problem going forward.

Now a nearly equally good question is how do I track down stuff like this when it happens?  Well if it were easy I wouldn’t be hoarse and saluting my monitor with one finger.  But here’s the best data sources I’ve found so far to use when this problem happens.  Again, as I find new things I will add to the list:

  1. ULS logs – a perennial favorite, especially at holiday parties, I like to crank open the ULS logs and just admire the sheer volume of data.  Okay, that was sarcasm.  But what you can really do is go into Central Admin, Monitoring, Configure diagnostic logging.  Expand the SharePoint Foundation category and select the following items:  App Auth, Application Authentication, Authentication Authorization and Claims Authentication.  Setting the logging and trace level to Verbose for these guys and save your changes, then go and try and launch your application again.  Grab one of the many ULS log viewing tools to see your request come in and being processed.  It’s a good way to gather hints about your app authentication issues.
  2. Fiddler – also a fan favorite, Fiddler is very useful for these situations too.  What you will most frequently see is a 401 unauthorized error (like whenever the underlying issue is the “issuer of the token is not a trusted issuer”).  If you look at the last frame in the request and click on the Headers tab in the Response frame you will see a WWW-Authenticate cookie that looks something like this:  Bearer realm=”8a96481b-6c65-4e78-b2ef-a446adb79b59″,client_id=”00000003-0000-0ff1-ce00-000000000000″,trusted_issuers=”e9134021-0180-4b05-9e7e-0a9e5a524965@8a96481b-6c65-4e78-b2ef-a446adb79b59,00000003-0000-0ff1-ce00-000000000000@8a96481b-6c65-4e78-b2ef-a446adb79b59 .  So what can you take away from this?  Well I know from looking at this that it expects my ClientId value to be e9134021-0180-4b05-9e7e-0a9e5a524965 and it expects my realm to be 8a96481b-6c65-4e78-b2ef-a446adb79b59.  The ClientId value is easy enough to check – I can go look in my AppManifest.xml file and my web.config for my self-hosted app.  It’s extremely unlikely that your realm would be wrong, but you can always verify by running this PowerShell:

$spurl =”https://foo
$spsite = Get-SPSite $spurl
$realm = Get-SPAuthenticationRealm -ServiceContext $spsite
$realm

That will output to the screen whatever your realm is.  Finally, there’s one other thing you can do to verify – make sure that you have an appPrincipal created for the ClientId you are using.  Again here’s some PowerShell you can use to check that, using my WWW-Authenticate header info from above:

Get-SPAppPrincipal -NameIdentifier e9134021-0180-4b05-9e7e-0a9e5a524965@8a96481b-6c65-4e78-b2ef-a446adb79b59 -Site https://foo

If you get an error or no results then you know you don’t have a valid SPAppPrincipal so you need to create one using PowerShell.  For completeness, here’s an example of that:

$clientId = “some guid you create”
$spurl =”https://foo
$spsite = Get-SPSite $spurl
$realm = Get-SPAuthenticationRealm -ServiceContext $spsite
$fullAppIdentifier = $clientId + ‘@’ + $realm
$appPrincipal = Register-SPAppPrincipal -NameIdentifier $fullAppIdentifier -Site $spsite.OpenWeb() -DisplayName “My Cool App”

Okay, and with that, my list of high trust app troubleshooting tips is exhausted for today, as am I.  When or if I have more news I will update this post.

 

 

 

 

Planning the Infrastructure Required for the new App Model in SharePoint 2013

SharePoint 2013 brings with it a brand new application model, which we euphemistically refer to as the “app model” or “cloud app model”.  While it brings a whole new set of opportunities from a development perspective, it also carries with it infrastructure requirements that you need to plan and build to support them.  When I think about the app model in fact there are really three big pieces that I look at:

 

  1. The development model – this all about the how I’m going to develop my application, what new features am I going to take advantage of, etc.  This is primarily a developer activity.
  2. The security model – the security model for these new applications is an interesting combination of “stuff”.  You have the new concept of an app-principal, which is how you define rights for your applications.  There’s the OAuth model which defines what content you can get at, as well as a mechanism for presenting SharePoint with a token that describes who you are – it’s very much like claims for applications.  The security model and means by which these tokens are created and presented varies significantly depending on whether your application is hosted in SharePoint or somewhere else, and whether you’re using Access Control Services (ACS) as a trust broker or not.  Lots of stuff there, and not something I’ll be covering in this blog post, but it’s something that is both a developer and an IT Pro activity.
  3. The infrastructure – here we’re talking about all of the “goo” that makes applications work within an organization.  It includes the service applications, application domain and prefix, DNS, SSL, and web application configurations.  It is primarily an IT Pro activity and is the basis of this blog post.  This posting is also going to be focused on applications that are hosted in SharePoint itself, versus an application that is hosted externally in another site or cloud like Windows Azure.  When you install an application hosted in SharePoint, a new SPWeb is created for it.  That way each application gets its own set of data and it cannot write over the data for another application.  It also provides a layer of security so that someone who has rights to one application cannot have malicious code that uses the rights of the current user to access data in another site in the farm.  This is an important basic principal of applications based on the new app model to remember – when they are hosted in SharePoint, one app = one SPWeb.

Now let’s start working through the list of these different infrastructure components.  First and foremost, you need to make sure that you have an instance of the App Management and Subscription Settings service applications created and running in the farm.  The App Management service is used for things like keeping track of applications and deployments, licensing, etc.  The Subscription Settings service is the same service application that is used for multi-tenant deployments, and it is also used by the app model to create the unique Urls that are used by applications.

So how are these Urls created?  Well it starts with the Apps configuration that you do in Central Admin.  When you go into Central Admin you will find a new section called Apps.  If you click on it you will see an option to Configure App Urls.  In there you will find two values that are used to create an application Url – the App Domain and App Prefix.  The App Domain is analogous to the host name that is going to be used for all applications in the farm.  Here are three things to consider when you plan for this name:

  1. Because it is used for the whole farm it should get the same level of planning and thought as you use for your other web applications. 
  2. In addition to that it needs to be a fully-qualified domain name – you will find trying to use a NetBIOS style name for this doesn’t work well for name resolution. 
  3. Finally, it should NOT be a child domain of your web applications.

So let’s look at a specific example.  Suppose your web applications are something like team.contoso.com, my.contoso.com and portal.contoso.com.  First of all, you probably don’t want to use just “contoso.com” for your App Domain, because doing so will require you to create a wildcard DNS entry for ALL of contoso.com that points to your apps endpoint (more on that in a minute).  Also you don’t want to use something like “apps.contoso.com” because that is a child domain to what you are using for your web applications (which are all rooted in “contoso.com”).  So a better choice based on these rules would be something like “contosoapps.com”. 

The App Prefix value can be anything you want – it’s plugged into the first part of the application Url, which we’ll talk through next.

I mentioned using a wildcard DNS entry above, and that’s the next part to cover.  When an application is installed and that application is hosted in SharePoint it will have a Url that is something like this:  https://apps87e90ada14c175.contosoapps.com/myapp/pages/default.aspx.   The elements of the Url break down like this:

  • apps” – this is the App Prefix value I mentioned above that you entered in central admin.
  • 87e90ada14c175” – this is a unique value that is based on the site collection and web where the application is installed
  • contosoapps.com” – this is the App Domain that you entered in central admin.
  • myapp” – this is the name of the application that has been installed.  The rest of the Url – pages/default.aspx – is what is contained in the SPWeb where the application is installed.

As you look at the Url and how it’s formed, hopefully you understand why you will want to use a wildcard DNS entry.  Because the first part of the Url is going to be different for every single application instance that is installed – even if the same application is installed in multiple site collections – it is not feasible to continually update your DNS for every single instance.  So that being said, that means for DNS for the scenario I’ve described here, you are going to want a wildcard DNS entry for *.contosoapps.com.  Now the IP address that it points to is something we’ll discuss in just a bit.

Related to the wildcard DNS entry is a wildcard SSL entry.  Just so we’re perfectly clear on this point – if you’re using apps, you should be using SSL.  The app model involves passing around OAuth access tokens, which describe who the current user and application are.  Just like if you were passing around SAML tokens for a user, you want to encrypt the channel that these tokens flow over to prevent any kind replay style attack that would grant access to content by someone who might be out there sniffing your network traffic.  Using SSL prevents those scenarios from happening, and since you’ve seen the Urls for these applications are going to be different for every installed instance, it also means that in order to be manageable you will need a wildcard SSL certificate.  Again, in our scenario, we would get a wildcard SSL certificate for *.contosoapps.com.

Okay, so far we’ve talked about the service applications that are required, the configuration that is required for the App Domain and Prefix, how the Urls are formed, and the DNS and SSL entries that are needed.  Now to tie everything off we need to talk about how application requests get routed.  Generally speaking, you will need a separate SharePoint web application just to do the routing of application requests.  Let’s take a look at why that is.  Suppose again that you have three web applications that are defined as follows; they are all using SSL so we are using unique IP addresses for each one:

  • team.contoso.com – 192.168.1.10
  • my.contoso.com – 192.168.1.11
  • portal.contoso.com – 192.168.1.12

Now as described above you can only have one App Domain in the farm, and it should not be a child domain.  This opens up two very real problems with the applications above:

  • If we install applications in each of these web apps, how do we get the right application request routed to the right web app?  We only have one App Domain but three web apps.
  • If we try and route an application request to one of these existing web apps, then SSL will break.  Why?  Because since we are using SSL on all our web apps (since we’re using Apps), then they all have an SSL certificate that is something like *.contoso.com.  So we can’t route application requests to them anyways, because you can’t get an SSL wildcard that works on both *.contoso.com and *.contosoapps.com, which is what our apps are going to be using.

The solution then is to create a fourth web application.  SIDE NOTE:  Technically you could create a second IP and SSL binding on an existing web application as long as that web application does not use a host header.  You would of course then need to bind the SSL wildcard certificate for your apps namespace to that SSL binding.  I do not cover that in the description below, but if you are doing a PLA-style deployment (i.e. one web app and only use host header site collections) that would be another option.  This is already fairly complicated so I don’t want to try and work that into the rest of this content, but wanted to point it out for completeness.  You can create it without a host header name and assign it a shared IP of 192.168.1.13.  In DNS then your wildcard entry for *.contosoapps.com will point to 192.168.1.13.  What ends up happening is that your apps web application “listens” on that IP address and the SharePoint http module that is responsible for routing will pick up the request for the application.  It will then use the App Management service application to determine what web application is actually hosting that application and request all of the information about the app, security, etc..  The request is then served from the web application that was listening, which is why it needs to be using the same app pool account as the other web applications are using – so it can access the content databases for those web apps.

Now our farm configuration looks like this:

  •  Apps Configuration
    • App Domain – contosoapps.com
    • App Prefix – apps
  • Web Applications
    • team.contoso.com
      • IP Address:  192.168.1.10
      • SSL Certificate:  *.contoso.com
    • my.contoso.com

      • IP Address:  192.168.1.11
      • SSL Certificate:  *.contoso.com
    • portal.contoso.com

      • IP Address:  192.168.1.12
      • SSL Certificate:  *.contoso.com
    • No host header, or you could use something like contosoapps, etc. – it doesn’t really matter since we aren’t routing based on host header, we’re routing based on IP address. 

      • IP Address:  192.168.1.13
      • SSL Certificate:  *.contosoapps.com
  • DNS
    • team.contoso.com – 192.168.1.10
    • my.contoso.com – 192.168.1.11
    • portal.contoso.com – 192.168.1.12
    • *.contosoapps.com – 192.168.1.13

Now, that we have all of the infrastructure configuration aspects of apps covered, there are a couple of other considerations you need to remember when planning for the rollout of new apps for SharePoint 2013:

  • SharePoint-hosted Apps do not work on web applications that use SAML; they DO work if you use a provider-hosted app – for more details see http://blogs.technet.com/b/speschka/archive/2012/12/07/using-sharepoint-apps-with-saml-and-fba-sites-in-sharepoint-2013.aspx.  If you are using SAML authentication then you will not be able to use SharePoint-hosted Apps on those web applications.  This might be resolved in a future service pack or some other type of patch, but this is a limitation right now.
  • SharePoint Apps do not support multiple zones (i.e. AAM); all requests are served out of the default zone.  UPDATE:  The March PU eliminates this restriction AS LONG AS you have a load balancer that can redirect to a different port number; Windows Load Balancing will not do this.  If you are using AAM to support multiple zones then you will likely not be able to use SharePoint apps.  All apps get served out of the default zone, so in theory if you had the default zone exposed so that everyone could reach it, and you were using the same exact authentication mechanism on every zone (with some possible limitations that are too detailed to go into here and now), then it’s possible that it would work.  But then again, if you are using zones it’s usually because you either a) don’t want every endpoint exposed to everyone or b) you DO want to use different authentication methods.  Again, this is something that may change in the future, but for now this what is available for SharePoint 2013 RTM.
  • UPDATE – I just found out today that if the web application that receives the request for an app is different from the web application that the app actually lives in, the web app that receives the request must be using the same app pool as the web app where the app lives.  Alternatively, its app pool must use the same identity as the app pool where the application lives.  So if you are following the guidance above and you have a separate web application that all of your application requests get routed to, the identity that its application pool uses must have rights to all the content databases for all of the web apps.  There are a few different ways you can do that, but the simplest is to use the same app pool for all your web apps.  The second easiest is to use the same identity for all your app pools.  This also is something that may change in the future, but this is how it currently works for SharePoint 2013 RTM.  UPDATE:  Again, the March PU can help in this scenario, to the extent that you can redirect to different web apps.  Explaining how is beyond the scope of this update note, there is a link to the latest docs explaining this in the comments below (thank you for that folks!)

 

One other important, final note:  this seems like a lot of configuration, and it is.  However, you should keep in mind that if you are using Office 365 then all of this is taken care of for you – no muss, no fuss, just easy to install and use applications.  You should give that some consideration when weighing your options.

So there you have it – SharePoint Apps are a big part of SharePoint 2013, but they require some planning and design work up front to ensure the infrastructure is in place to support them.