Even More “Things to Do When you Change the SharePoint STS Token Signing Certificate”

I recently changed the token signing certificate in my farm…it having expired from when I first changed it, since that is required to set up the trust with ACS in SharePoint Online for low trust apps and SharePoint hybrid features.  Assume the longer the sentence the more unhappy I am about having to do it…  🙂

Well, of course as luck would have it once I finished all that up, all of my low trust and high trust apps broke.  I expected my high trust apps to break, since I’ve blogged about this before.  I wasn’t really expecting the full breakdown of my low trust apps as well though, even though I’ve also had to blog about that scenario before as well:  https://samlman.wordpress.com/2015/03/02/updating-trust-between-onprem-farms-and-acs-for-apps-when-your-sharepoint-sts-token-signing-certificate-expires/.

Unfortunately, even using the tips in the previous blog post above did not fully solve my problems.  Amazingly, I was getting 401 Unauthorized errors in my low trust apps, which I traced back to a “certificate is not from a trusted root authority” in the ULS logs.  This is one of my favorite errors with the app model (ridiculously heavy sarcasm here), since they are so routinely generic and difficult to resolve.  Not only that, but the error is a little looney tunes in this scenario if you ask me.  What certificate is not trusted, and by whom?  Does my local SharePoint farm not trust it’s own SharePoint STS token signing cert?  That seems rather impossible.  Does it not trust the Microsoft SharePoint Online ACS certificate?  Equally illogical.  Does SharePoint Online ACS not trust my local farm’s STS token signing cert?  Seems like the only possible scenario.  Without going through a bunch of boring PowerShell I will just say that I ran the code (for a second and then third time) to create a New-MsolServicePrincipalCredential, without success.

Finally, I just ran the Connect-SPFarmToAAD cmdlet again, but this time with every possible “Remove*” option it offers:  -RemoveExistingACS -RemoveExistingSTS -RemoveExistingSPOProxy -RemoveExistingAADCredentials.  After doing that and trying my low trust application again, it started working.  Yay.  Yet another Apps for SharePoint tip to keep handy.

Bug Alert for April CU and Migrating Users

Just heard about a nasty little bug in the April CU from my friend Syed.  He was using the SPWebApplication.MigrateUsers method to migrate accounts from one claim value to another (i.e. like if you were migrating from Windows claims to SAML claims, or in his case, changing identity claim values).  Turns out after doing the migration the actual claim value in the content databases was not getting updated.  He managed to track this down to a bug in the April CU, but found that it was fixed again in the June CU.  So…if you are doing any kind of claims migrations – including updating an identity claim value – make sure that you are NOT on the April CU, but instead apply one after that.

Thanks Syed for passing this info along.

A Couple of Be Prepared Actions When Changing the SharePoint STS Token Signing Certificate for Hybrid Features

I’ve been working recently with some Microsoft Services and their customers who are changing the SharePoint STS token signing certificate.  We are doing this as part of the set of steps required to set up the SharePoint Hybrid features for integrating search and BCS between on premises SharePoint farms and Office 365.  I’ve had a couple of these folks get a little concerned because after making this change they had a seemingly random scattering of people that were unable to authenticate into SharePoint afterwards.  The answer as it turns out was just that those users still had a valid fedauth cookie from a previous authentication into SharePoint and that cookie became invalid when the STS token signing certificate was changed.  So just in case, here are a couple of things to remember before you undertake this operation:

  • You can always back up the SharePoint STS token signing certificate before you change it.  You can use the PowerShell I posted about here to export the certificate:  http://blogs.technet.com/b/speschka/archive/2011/12/21/some-easy-powershell-to-export-the-token-signing-certificate-from-sharepoint-2010.aspx.  Never hurts to have that stored away some way for a rainy day until you’ve done a complete check of your services.  NOTE:  the link above does not include exporting the private key, which you will need to do as well if you want to reuse that cert in the future.  You can modify the PowerShell to get the private key or you can just export it out the Certificates MMC snap in.
  • If after you change it and you have some users who are unable to log in, you can have them a) delete their cookies (which most users are loathe to do, including me), or b) have them use an In Private session to access SharePoint.  Their fedauth cookie is only good for as long as you configured your Identity Provider to make it, so in most cases the cookie expires in a few hours.  That makes this work around only temporary until that cookie expires then you should be good to go again.

Access Denied Error with App Only Access Token When Reading Profile Info

This is yet another rather strange error that I ran across and couldn’t find any info out on the interwebs about it so I though I would document it here.  Suppose you have a SharePoint App that needs to access some User Profile information.  You will probably use the PeopleManager class and ask for user profile properties using the PersonProperties class or one of the methods off of the PeopleManager class.  You write your code up using the standard TokenHelper semantics to get a user + app key to retrieve this information, i.e. something like var clientContext = spContext.CreateUserClientContextForSPHost().  In your AppManifest file you ask for (at a minimum) Read rights to User Profiles (Social).   Works great, okay, good start.

Now you determine that you need to retrieve that same information but use an App Only token.  So you use whatever method you want to get an App Only token.  You use the same code but now you get an Access Denied error message.  Why is that – App Only tokens are supposed to have the same or greater rights than user + app tokens.  Well…for right now…I don’t know why not.  NOTE:  I DO understand needing to be a tenant admin to install an app that requires access to User Profiles, but this is different; it happens after the app is installed.  But I do know how I fixed it.  I added Tenant…Read rights to my AppManifest file.  Now my App Only token is able to read properties from the User Profile in o365.  Just thought I would share this “not at all obvious” tip so that if you get stuck hopefully your favorite search engine will find this post.  Happy coding!

SAML in a Box – Inviting External Users to Your SharePoint Farm

Those of you who follow the Share-n-Dipity blog know that I don’t really do much in the way of product endorsements.  However, you have probably also figured out that I’m a big SAML fan, so when a friend of mine recently released a new product for this market it really caught my eye.  If you’ve followed the blog for a while you may have seen me reference a friend mine by the name of Israel Vega, Jr.  He worked with me at Microsoft for many years and last year decided to head out on his own.  “Iz”, as we call him, has done a ton of work with SharePoint over the years and was really one of the go to guys for it in Microsoft Services when he left.

He has just released a new product called CloudExtra.NET, which is obviously a play on “cloud extranet”.  What I really like about his solution is that it’s SAML in a box – basically a turn-key product that brings a lot of things customers frequently ask me about to a packaged product.  He uses SAML authentication and a custom claims provider to enable you to use any number of cloud-based directories out of the box.  That includes Azure Active Directory for internal users, and anything you can connect to ACS for your external users.  That part is pretty cool because he has basically built “invited users” for on-premises SharePoint farms, a feature that currently is only available to Office 365. For those of you who aren’t ready to do that yet, this is a pretty slick way to do it with your own farms now too. 

He’s added some nice features that round out his solution.  He has a Terms and Conditions agreement for it so you can make invited users agree to your terms of use before gaining access to your farm.  He also has a product that’s cool in its own right and is bundled with CloudExtra.NET, which is a business impact application.  It lets you assign an impact rating to sites (like High, Medium and Low) and then also use that to apply a site policy to it when you do so.  He ties it all together with farm-wide reporting on sites based on business impact.  There are also a bunch of reporting and management features built in for invited users that make it a very solid offering.

I gleaned all this from about a 30 minute demo that Iz gave me, and I’m hoping to spend some time in the not too distant future doing a hands-on review of the product to check it out for myself.  If it sounds like something that might be interested to you then you should go visit his web site at http://cloudextra.net.  Thanks Iz for the SAML love!

Remote SharePoint Index, Hybrid, and SAML

Today’s post combines a few different topics, that dare I say, started as a dare (I think).  The gauntlet thrown out was whether or not Remote SharePoint Index would work for SAML users as well as Windows users.  For those of you not completely familiar with Remote SharePoint Index, I covered it in some detail in my blog post here:  https://samlman.wordpress.com/2015/03/01/setting-up-an-oauth-trust-between-farms-in-sharepoint-2013/.  The gist of it is that you set up an OAuth trust between two SharePoint farms, and it lets you get search results from both farms into a single search center results page.  All of our examples in this area have been around using Windows users, but when the question was posed my best guess was “yeah, seems like that should work”.  As it turns out, it’s true, it really does.  In order to get this to work it requires the following:

  • Two SharePoint farms
  • Both farms have an SPTrustedIdentityTokenIssuer configured (ideally using the same Name, but in theory should not be required.  In theory.)
  • Both farms have at least one web app configured to use the SPTrustedIdentityTokenIssuer
  • Both farms have UPA configured and running
  • Both farms are doing a profile import from the same Active Directory forest.
  • Both farms have the profile import Authentication Provider Type configured to be Trusted Claims Provider Authentication and Authentication Provider Instance configured to be their SPTrustedSecurityToken Issuer.  When you’re done it should look like this:    

After you’ve set all of that up, you also need to into Manage User Properties in the UPA and map both the Claim User Identifier and (usually) the Work email properties.  In my case I used email address as the identity claim, so I mapped both of those properties to the “mail” attribute in Active Directory.  If you are using a different identity claim then you would map the Claim User Identifier accordingly and would not need to map Work email.  By default, the Claim User Identifier is mapped to samAccountName, so you’ll want to Remove that mapping, then add one for mail.  There’s nothing to helpful about doing this, you literally just type in “mail” in the Attribute field and then click the Add button.  When you’re done it will look like this:  .

You’ll probably want to do a full import after that (unless you have a really huge directory).  The biggest gotcha here is that you can really only set up one profile connection per Active Directory domain (at least when using Active Directory import instead of FIM).  The reason that may be a problem is that most folks do an import to get all their Windows users into the UPA.  However if you want to use the same domain to import users for both Windows and SAML auth into the UPA, it won’t work.  You’ll get an error when you try to create a second connection to the same domain.  So you basically have to be all in – all SAML or all Windows; otherwise you’ll end up getting all sorts of seeming random results.

Once you’ve get everything configured and you’ve run a profile import in both farms, then you can set up the first part of this scenario, which is doing cross farm searches by SAML users using Remote SharePoint Index.  I literally used the same exact steps that I described in my previous blog post I referenced at the start of this posting (setting up an oauth trust between farms). Once that trust is set up then it’s just a matter of creating a new result source that is configured as a Remote SharePoint Index, using the Url to the web application in the other farm (the one you used when creating the trust between the farms), and then creating a query rule to execute your query against the remote farm.  When it’s all said and done you end up being able to authenticate into your local farm as a SAML user and search results from the remote farm.  In the screenshot here I’m logged in as a SAML user; you see his full display name because it was brought in when I did the profile sync:

Now, the real beauty of this solution is that we can actually get this to work when using the SharePoint 2013 hybrid search features with Office 365.  In this case I have a farm where I already had the hybrid features configured and working.  When I created my SPTrustedIdentityTokenIssuer, I made sure to include 3 of the 4 claim types (the only ones I populate) that are used in rehydrating users for OAuth calls – email, Windows account, and UPN.  Since those values are also synchronized up to my Office 365 tenant (and thus put into the UPA for my tenant up there), I’m able to rehydrate the user in Office 365 and get search results from there as well, even though I’m logged in locally as a SAML user.  Very cool!!  Here’s a screen shot of the same SAML user getting both local search results as well as hybrid search results from Office 365:

Hopefully this helps to clear up some of the mystery about SAML users and OAuth, as well as how it interacts with some of the built-in features of SharePoint 2013 like Remote SharePoint Index and Hybrid Search.  Clearly the complicating factor in all of this is the UPA and managing your user profile imports.  That would be the first planning item I would be thinking about if you want to use SAML authentication on site, in addition to SharePoint Hybrid.  It’s also clearly a sticking point if you wanted to use both Windows and SAML auth in the same farm with the same users from the same domain.

Updating Trust Between OnPrem Farms and ACS for Apps When Your SharePoint STS Token Signing Certificate Expires

For those of you who are “in the app way” with SharePoint 2013 (no, not a lot different from being “in the pregnant way”, as they say), you’ll reach that point sooner or later where the token signing certificate for your SharePoint STS expires if you are using low trust apps on premises.  Why is that?  Because in order to establish the trust with ACS that is necessary to use low trust apps on prem, you had to replace the token signing certificate that comes out of the box.  Most folks simply use a self-signed certificate, which when you get it from the IIS Manager snap-in is good for one year.  Even if you get it from a public root authority, they are typically good for a year or two, although there are exceptions.

All that being said, when it comes time to change your STS token signing cert then you will need to re-establish that trust between your on premises farm and ACS.  Having just beat my head against the wall for the better part of a day to get this to work I thought I’d share some tips that I used to get me through it.  Here are some things you should be looking at:

  1. Get the latest version of the PowerShell modules for MSOL.  For some reason this is always harder than it should be, so again, to save you some time, here’s where I went to get the latest versions:  http://msdn.microsoft.com/en-us/library/azure/jj151815.aspx .  This is the page that describes managing Azure Active Directory using PowerShell and it seems to consistently have the “right” links.  On this page you will find a link to the Microsoft Online Services Sign-On Assistant, which you must get and install first. Right below that you will find a link for the 32-bit and 64-bit version of the PowerShell module for Azure Active Directory.  I use this one because it has all the MSOL goodness we require.  One other thing to note – you must first uninstall your existing version, and it will tell you that you need to reboot after doing so.  Yeah, you need to do this.  You may or may not get prompted to reboot again after installing it.  The net of all this is that I tried to skate by and skip the reboots a couple times and then the Azure AD module just flat out wouldn’t install.  One reboot later everything installed just fine and using the new modules cleared the first set of roadblocks I had been hitting.
  2. Start with the scripts for establishing trust that we previously published here:  http://msdn.microsoft.com/en-us/library/dn155905.aspx.
  3. There are two modifications to the scripts you will need to make; without them you will get errors and stuff will still be broken.  Here are the changes:

    1. Find this line of in the main script:  [Guid[]] $ExistingKeyIds = Get-MsolServicePrincipal -AppPrincipalId $SP_APPPRINCIPALID | Get-MsolServicePrincipalCredential | % KeyId.  Comment it out (by putting a # sign in front of it) as well as the line that follows it:  Remove-MsolServicePrincipalCredential -AppPrincipalId $SP_APPPRINCIPALID -KeyIds $ExistingKeyIds.  At some point you may end up with key ID that is null, and that causes this part of the script to break.  I fixed it with some other PowerShell I wrote up, so that chunk of the script now looks like this:

#Remove existing connection to an Office 365 SharePoint site if the switch is specified.
    if ($RemoveExistingAADCredentials.IsPresent -and $RemoveExistingAADCredentials -eq $true) {
        #[Guid[]] $ExistingKeyIds = Get-MsolServicePrincipal -AppPrincipalId $SP_APPPRINCIPALID | Get-MsolServicePrincipalCredential | % KeyId
        #Remove-MsolServicePrincipalCredential -AppPrincipalId $SP_APPPRINCIPALID -KeyIds $ExistingKeyIds

    [string[]] $ExistingKeyIds = get-MsolServicePrincipalCredential -ReturnKeyValues $returnIds -AppPrincipalID $SP_APPPRINCIPALID | % KeyId

    foreach($id in $ExistingKeyIds)
    If ($id)
        Remove-MsolServicePrincipalCredential -AppPrincipalId $SP_APPPRINCIPALID -KeyIds $id

b.  There is a second script that enumerates all of your web apps and adds each one as a ServicePrincipalName (SPN) to the AppPrincipal.  If you’ve already added those SPNs before though, then it causes all sorts of problems and you will end up getting 400 Bad Request errors anyways (as I described here:  http://blogs.technet.com/b/speschka/archive/2013/07/29/security-in-sharepoint-apps-part-3.aspx).  To fix it you just need to add one line of code to the PowerShell so it will remove the SPN before adding it.  So if the SPN already exists it will be removed before adding, and if it doesn’t exist it just gets added.  Again, the net is you end up with a working script and one SPN for each web app in your farm.  Here is the section of that script with the new line of code:

Write-Host “Adding SPN for” $NewSPN

#here’s the new line of code

Set-MsolServicePrincipal -AppPrincipalId $SPAppPrincipal.AppPrincipalId -ServicePrincipalNames $SPAppPrincipal.ServicePrincipalNames

After doing ALL of these things, my low trust apps on premise are working again.  Hope your sailing goes a little smoother.

A Few Additional Notes

You may still experience some issues after using this code.  Here are a few suggestions:

  1. Try using the /_layouts/15/appregnew.aspx page to create a new app principal.  If things are messed up then you will see an error there, like invalid JWT token.
  2. If appregnew.aspx is not working look in the ULS logs.  You should be able to track down the invalid JWT token, and you may see a subsequent error along the lines of request aborted.  I call this part of the error out specifically because if you are just trying to deploy a new app from Visual Studio you will also see this error in there – request aborted, request was cancelled, something like that (sorry I don’t have the exact wording handy).
  3. If you are getting the invalid JWT tokens I recommend you do this:Also, I recommend when you run the script to create the trust you include the -SharePointWeb parameter.  That parameter should be the Url to a web app on your on prem farm.  If you don’t include it then you will get one randomly selected and I actually ran into a problem with that scenario in one of the farms where I was doing this fix up.
    1. Manually remove the SPServiceApplication “ACS”; use this PowerShell script to do it:  $p = Get-SPServiceApplicationProxy | ? DisplayName -EQ “ACS”, then Remove-SPServiceApplicationProxy $p
    2. Manually remove the SPServiceApplication “ACS-STS”; the PowerShell script is the same as the previous step, but use -EQ “ACS-STS”.
    3. Rerun the script to create the trust but DO NOT include any of the -Remove* parameters

  4. When you run the script to create the trust include the -SharePointWeb parameter.  This should be the Url to a web application on your local farm.  If you don’t include this parameter then you will have a web app randomly selected, and this scenario actually caused another issue in one of the farms where I was applying these fixes.