YARBACS or Yet Another RBAC Solution

RBAC is all the rage (as it should be) these days, but it’s not really new.  The concepts have been in use for many years, it’s just a case of bringing it into modern cloud-based scenarios.  Azure continues to invest in this concept so that we can increasingly lock down the control and use of various cloud-based services they offer.  There’s other “interesting” aspects to these scenarios though that can impact the availability and usefulness of it – things such as existing systems and identities, cross tenant applications, etc.

It’s these interesting aspects that bring me to this post today.  At Office365Mon we are ALL about integration with Azure Active Directory and all of the goodness it provides.  We also have had from day 1 the concept of allowing users from different tenants to be able to work on a set of resources using a common security model.  That really means that when you create Office365Mon subscriptions, you can assign subscription administrators to anyone with an Azure AD account.  It doesn’t require Azure B2C, or Azure B2B, or any kind of trust relationship between organizations.  It all “just works”, which is exactly what you want.  IMPORTANT DISCLAIMER:  the RBAC space frequently changes.  You should check in often to see what’s provided by the service and decide for yourself what works best for your scenario.

“Just works” in this case started out being really just a binary type operation – depending on who you are, you either had access or you didn’t.  There was no limited access based on one or more roles that you had.  As we got into more complex application scenarios it became increasingly important to develop some type of RBAC support capabilities, but there really wasn’t an out of the box solution for us.  That led us to develop this fairly simple framework of YARBACS as I call it, or “Yet Another RBAC Solution”.  It’s not a particularly complicated approach (I don’t think) so I thought I’d share the high level details here in case it may help those of you who are living with the same sort of constraints that we have.  We have a large existing user base, everyone is effectively a cloud user to us, we don’t have any type of trust relationship with other Azure AD tenants, but we need to be able to support a single resource (Office365Mon subscription) to be managed by users from any number of tenants and with varying sets of permissions.

With that in mind, these are the basic building blocks that we use for our YARBACS:

  1. Enable support for Roles in our Azure AD secured ASP.NET application
  2. Create application roles in our AD tenant that will be used for access control
  3. Store user and role relationships so that it can be used in our application
  4. Add role attributes to views, etc.
  5. Use a custom MVC FilterAction to properly populate users from different tenants with application roles from our Azure AD tenant

Let’s walk through each of these in a little more detail.

 

Enable Support for Roles

Enabling support for roles in your Azure AD secured application is something that has been explained very nicely by Dushyant Gill, who works as a PM on the Azure team.  You can find his blog explaining this process in more detail here:  http://www.dushyantgill.com/blog/2014/12/10/roles-based-access-control-in-cloud-applications-using-azure-ad/.   Rather than trying to plagiarize or restate his post here, just go read his.  J   The net effect is that you will end up with code in your Startup.Auth.cs class in your ASP.NET project that looks something like this:

app.UseOpenIdConnectAuthentication(

new OpenIdConnectAuthenticationOptions

{

ClientId = clientId,

Authority = Authority,

TokenValidationParameters = new System.IdentityModel.Tokens.TokenValidationParameters

{

//for role support

RoleClaimType = “roles”

},

… //other unrelated stuff here

Okay, step 1 accomplished – your application supports the standard use of ASP.NET roles now – permission demands, IsInRole, etc.

 

Create Application Roles in Azure AD

This next step is also covered in Dushyant’s blog post mentioned above.  I’ll quickly recap here, but for complete steps and examples see Dushyant’s blog post.  Briefly, you’ll go to the application for which you want to use YARBACS and create applications roles.  Download the app manifest locally and add roles for the application – as many as you want.  Upload the manifest back to Azure AD, and now you’re ready to start using them.  For your internal users, you can navigate to the app in the Azure management portal and click on the Users tab.  You’ll see all of the users that have used your app there.  You can select whichever user(s) you want to assign to an app role and then click on the Assign button at the bottom.  The trick of course is now adding users to these groups (i.e. roles) when the user is not part of your Azure AD tenant.

 

Store User and Role Relationships for the Application

This is another step that requires no rocket science.  I’m not going to cover any implementation details here because it’s likely going to be different for every company in terms of how and what they implement it.  The goal here is simple though – you need something – like a SQL database – to store the user identifier and the role(s) that user is associated with.

In terms of identifying your user in code so you can look up what roles are associated with it, I use the UPN.  When you’ve authenticated to Azure AD you’ll find the UPN is in the ClaimsPrincipal.Current.Identity.Name property.  To be fair, it’s possible for a user’s UPN to change, so if you find that to be a scenario that concerns you then you should use something else.  As an alternative, I typically have code in my Startup.Auth.cs class that creates an AuthenticationResult as part of registering the application in the user’s Azure AD tenant after consent has been given.  You can always go back and look at doing something with the AuthenticationResult’s UserInfo.UniqueId property, which is basically just a GUID that identifies the user in Azure AD and never changes, even if the UPN does.

Now that you have the information stored, when we build our MVC ActionFilter we’ll pull this data out to plug into application roles.

 

Add Role Attributes to Views, Etc.

This is where the power of the ASP.NET permissions model and roles really comes into play.  In step 1 after you follow the steps in Dushyant’s blog, you are basically telling ASP.NET that anything in the claim type “roles” should be considered a role claim.  As such, you can start using it the way you would any other kind of ASP.NET role checking.  Here are a couple of examples:

  • As a PrincipalPermission demand – you can add a permission demand to a view in your controller (like an ActionResult) with a simple attribute, like this:  [PrincipalPermission(SecurityAction.Demand, Role = “MyAppAdmin”)].  So if someone tries to access a view and they have not been added to the MyAppAdmin role (i.e. if they don’t have a “roles” claim with that value), then they will get denied access to that view.
  • Using IsInRole – you can use the standard ASP.NET method to determine if a user is in a particular role and then based on that make options available, hide UI, redirect the user, etc. Here’s an example of that:  if (System.Security.Claims.ClaimsPrincipal.Current.IsInRole(“MyAppAdmin “))…  In this case if the user has the “roles” claim with a value of MyAppAdmin then I can do whatever – enabling a feature, etc.

Those are just a couple of simple and the most common examples, but as I said, anything you can do with ASP.NET roles, you can now do with this YARBACS.

 

Use a Custom FilterAction to Populate Roles for Users

This is really where the magic happens – we look at an individual and determine what roles we want to assign to it.   To start with, we create a new class and have it inherit from ActionFilterAttribute.  Then we override the OnActionExecuting event and plug in our code to look for the roles for the user and assign them.

Here’s what the skeleton of the class looks like:

public class Office365MonSecurity : ActionFilterAttribute

{

public override void OnActionExecuting(ActionExecutingContext filterContext)

{

//code here to assign roles

}

}

Within the override, we plug in our code.  To begin with, this code isn’t going to do any good unless the user is already authenticated.  If they haven’t then I have no way to identify them and look up the roles (short of something like a cookie, which would be a really bad approach for many reasons).  So first I check to make sure the request is authenticated:

if (filterContext.RequestContext.HttpContext.Request.IsAuthenticated)

{

//assign roles if user is authenticated

}

 

Inside this block then, I can do my look up and assign roles because I know the user has been authenticated and I can identify him.  So here’s a little pseudo code to demonstrate:

SqlHelper sql = new SqlHelper();

List<string> roles = sql.GetRoles(ClaimsPrincipal.Current.Identity.Name);

 

foreach(string role in roles)

{

Claim roleClaim = new Claim(“roles”, role);

ClaimsPrincipal.Current.Identities.First().AddClaim(roleClaim);

}

 

So that’s a pretty good and generic example of how you can implement it.  Also, unlike application roles that you define through the UI in the Azure management portal, you can assign multiple roles to a user this way.  It works because they just show up as role claims.  Here’s an example of some simple code to demonstrate:

 

//assign everyone to the app reader role

Claim readerClaim = new Claim(“roles”, “MyAppReader”);

ClaimsPrincipal.Current.Identities.First().AddClaim(readerClaim);

 

//add a role I just made up

Claim madeUpClaim = new Claim(“roles”, “BlazerFan”);

ClaimsPrincipal.Current.Identities.First().AddClaim(madeUpClaim);

This code actually demonstrates a couple of interesting things.  First, as I pointed out above, it adds multiple roles to the user.  Second, it demonstrates using a completely “made up” role.  What I mean by that is that the “BlazerFan” role does not exist in the list of application roles in Azure AD for my app.  Instead I just created it on the fly, but again it all works because it’s added as a standard Claim of type “roles”, which is what we’ve configured our application to use as a role claim.  Here’s a partial snippet of what my claims collection looks like after running through this demo code:

yarbacs

To actually use the FilterAction, I just need to add it as an attribute on the controller(s) where I’m going to use it.  Here’s an example – Office365MonSecurity is the class name of my FilterAction:

[Office365MonSecurity]

public class SignupController : RoutingControllerBase

There you have it.  All of the pieces to implement your own RBAC solution when the current service offering is not necessarily adequate for your scenario.  It’s pretty simple to implement and maintain, and should support a wide range of scenarios.

Bug Alert for April CU and Migrating Users

Just heard about a nasty little bug in the April CU from my friend Syed.  He was using the SPWebApplication.MigrateUsers method to migrate accounts from one claim value to another (i.e. like if you were migrating from Windows claims to SAML claims, or in his case, changing identity claim values).  Turns out after doing the migration the actual claim value in the content databases was not getting updated.  He managed to track this down to a bug in the April CU, but found that it was fixed again in the June CU.  So…if you are doing any kind of claims migrations – including updating an identity claim value – make sure that you are NOT on the April CU, but instead apply one after that.

Thanks Syed for passing this info along.

SAML in a Box – Inviting External Users to Your SharePoint Farm

Those of you who follow the Share-n-Dipity blog know that I don’t really do much in the way of product endorsements.  However, you have probably also figured out that I’m a big SAML fan, so when a friend of mine recently released a new product for this market it really caught my eye.  If you’ve followed the blog for a while you may have seen me reference a friend mine by the name of Israel Vega, Jr.  He worked with me at Microsoft for many years and last year decided to head out on his own.  “Iz”, as we call him, has done a ton of work with SharePoint over the years and was really one of the go to guys for it in Microsoft Services when he left.

He has just released a new product called CloudExtra.NET, which is obviously a play on “cloud extranet”.  What I really like about his solution is that it’s SAML in a box – basically a turn-key product that brings a lot of things customers frequently ask me about to a packaged product.  He uses SAML authentication and a custom claims provider to enable you to use any number of cloud-based directories out of the box.  That includes Azure Active Directory for internal users, and anything you can connect to ACS for your external users.  That part is pretty cool because he has basically built “invited users” for on-premises SharePoint farms, a feature that currently is only available to Office 365. For those of you who aren’t ready to do that yet, this is a pretty slick way to do it with your own farms now too. 

He’s added some nice features that round out his solution.  He has a Terms and Conditions agreement for it so you can make invited users agree to your terms of use before gaining access to your farm.  He also has a product that’s cool in its own right and is bundled with CloudExtra.NET, which is a business impact application.  It lets you assign an impact rating to sites (like High, Medium and Low) and then also use that to apply a site policy to it when you do so.  He ties it all together with farm-wide reporting on sites based on business impact.  There are also a bunch of reporting and management features built in for invited users that make it a very solid offering.

I gleaned all this from about a 30 minute demo that Iz gave me, and I’m hoping to spend some time in the not too distant future doing a hands-on review of the product to check it out for myself.  If it sounds like something that might be interested to you then you should go visit his web site at http://cloudextra.net.  Thanks Iz for the SAML love!

Remote SharePoint Index, Hybrid, and SAML

Today’s post combines a few different topics, that dare I say, started as a dare (I think).  The gauntlet thrown out was whether or not Remote SharePoint Index would work for SAML users as well as Windows users.  For those of you not completely familiar with Remote SharePoint Index, I covered it in some detail in my blog post here:  https://samlman.wordpress.com/2015/03/01/setting-up-an-oauth-trust-between-farms-in-sharepoint-2013/.  The gist of it is that you set up an OAuth trust between two SharePoint farms, and it lets you get search results from both farms into a single search center results page.  All of our examples in this area have been around using Windows users, but when the question was posed my best guess was “yeah, seems like that should work”.  As it turns out, it’s true, it really does.  In order to get this to work it requires the following:

  • Two SharePoint farms
  • Both farms have an SPTrustedIdentityTokenIssuer configured (ideally using the same Name, but in theory should not be required.  In theory.)
  • Both farms have at least one web app configured to use the SPTrustedIdentityTokenIssuer
  • Both farms have UPA configured and running
  • Both farms are doing a profile import from the same Active Directory forest.
  • Both farms have the profile import Authentication Provider Type configured to be Trusted Claims Provider Authentication and Authentication Provider Instance configured to be their SPTrustedSecurityToken Issuer.  When you’re done it should look like this:    

After you’ve set all of that up, you also need to into Manage User Properties in the UPA and map both the Claim User Identifier and (usually) the Work email properties.  In my case I used email address as the identity claim, so I mapped both of those properties to the “mail” attribute in Active Directory.  If you are using a different identity claim then you would map the Claim User Identifier accordingly and would not need to map Work email.  By default, the Claim User Identifier is mapped to samAccountName, so you’ll want to Remove that mapping, then add one for mail.  There’s nothing to helpful about doing this, you literally just type in “mail” in the Attribute field and then click the Add button.  When you’re done it will look like this:  .

You’ll probably want to do a full import after that (unless you have a really huge directory).  The biggest gotcha here is that you can really only set up one profile connection per Active Directory domain (at least when using Active Directory import instead of FIM).  The reason that may be a problem is that most folks do an import to get all their Windows users into the UPA.  However if you want to use the same domain to import users for both Windows and SAML auth into the UPA, it won’t work.  You’ll get an error when you try to create a second connection to the same domain.  So you basically have to be all in – all SAML or all Windows; otherwise you’ll end up getting all sorts of seeming random results.

Once you’ve get everything configured and you’ve run a profile import in both farms, then you can set up the first part of this scenario, which is doing cross farm searches by SAML users using Remote SharePoint Index.  I literally used the same exact steps that I described in my previous blog post I referenced at the start of this posting (setting up an oauth trust between farms). Once that trust is set up then it’s just a matter of creating a new result source that is configured as a Remote SharePoint Index, using the Url to the web application in the other farm (the one you used when creating the trust between the farms), and then creating a query rule to execute your query against the remote farm.  When it’s all said and done you end up being able to authenticate into your local farm as a SAML user and search results from the remote farm.  In the screenshot here I’m logged in as a SAML user; you see his full display name because it was brought in when I did the profile sync:


Now, the real beauty of this solution is that we can actually get this to work when using the SharePoint 2013 hybrid search features with Office 365.  In this case I have a farm where I already had the hybrid features configured and working.  When I created my SPTrustedIdentityTokenIssuer, I made sure to include 3 of the 4 claim types (the only ones I populate) that are used in rehydrating users for OAuth calls – email, Windows account, and UPN.  Since those values are also synchronized up to my Office 365 tenant (and thus put into the UPA for my tenant up there), I’m able to rehydrate the user in Office 365 and get search results from there as well, even though I’m logged in locally as a SAML user.  Very cool!!  Here’s a screen shot of the same SAML user getting both local search results as well as hybrid search results from Office 365:


Hopefully this helps to clear up some of the mystery about SAML users and OAuth, as well as how it interacts with some of the built-in features of SharePoint 2013 like Remote SharePoint Index and Hybrid Search.  Clearly the complicating factor in all of this is the UPA and managing your user profile imports.  That would be the first planning item I would be thinking about if you want to use SAML authentication on site, in addition to SharePoint Hybrid.  It’s also clearly a sticking point if you wanted to use both Windows and SAML auth in the same farm with the same users from the same domain.

SAML Support for SharePoint-Hosted Apps with ADFS 3.0

This is another case where I’m just passing information along here, based on the great work of others.  As you probably know, we did not have a good story for SharePoint-hosted apps in web application that uses SAML authentication with ADFS 2.0.  However, I have had reports from a couple of different teams now that they ARE working with ADFS 3.0.  The main differences that are needed to make this work include:

  • In ADFS you need to define a wildcard WS-Fed endpoint.  For example, normally for a SharePoint web application, in ADFS you create a relying party and set the WS-Fed endpoint to be something like https://www.foo.com/_trust/.  To do the same thing with apps, you take your apps namespace – assume it’s “contosoapps.com” – and add a WS-Fed endpoint like this:  https://*.contosoapps.com/_trust/.
  • Configure the SharePoint STS to send the wreply parameter.  You can do that with PowerShell that looks like this:

$sts = Get-SPTrustedIdentityTokenIssuer
$sts.UseWReplyParameter = $true
$sts.Update() 

One other thing to note – the behavior to use the wreply parameter is supposed to be turned on by default in an upcoming CU.  I heard it was the April 2014 CU actually but have not had a chance to see if that is really in there or not.  It won’t hurt to run the PowerShell above though.

This is good news, thanks for those of you that shared your experiences!

More Info on an Old Friend – the Custom Claims Provider, Uri Parameters and EntityTypes in SharePoint 2013

Back to oldie but a goodie – the custom claims provider for SharePoint.  I believe this applies to SharePoint 2010 as well but honestly I have only tested what I’m about to describe on SharePoint 2013 and don’t have the bandwidth to go back and do a 2010 test as well.  What I wanted to describe today is the values you may expect to get, and the values you actually get, in a custom claims provider method for FillClaimsForEntity, FillResolve and FillSearch.  Chances are they may not be what you expect.

  • FillClaimsForEntity – this method is invoked at login time and is used for claims augmentation.  The Uri parameter that you get (“context”) will NOT tell you what site a person is authenticating into.  The value you will get back is the root of the web application zone.  So if a user is trying to get into https://www.contoso.com/sites/accounting, the value for the “context” parameter will be https://www.contoso.com.
  • FillResolve – this method has two different overrides, but both include a Uri parameter (“context”).  The value you get for this parameter varies.  When you’re in a site collection the Uri will be for the site collection you are in.  When you are in central admin and you are invoked when picking a site collection administrator, “it varies”.  When you are first selecting a site collection administrator, the Uri value will be of the root site collection again – NOT the site collection you are trying to set the admin for.  Unfortunately, this gets a bit more convoluted yet – when you click on the OK button to save your changes, the FillResolve method will get invoked again.  This time however, the Uri value WILL BE the actual site collection Url for which you just changed the site collection admin.
  • FillSearch – this method is invoked when someone is doing a search in people picker.  It exhibits the same behavior as FillResolve above – when you’re in a site collection you get the correct site collection Uri.  When you’re in central admin and you are trying to search for an entity to add as a site collection admin, the value for the Uri parameter is the root site collection – NOT the site collection for which you are setting the admin.

All of the details above apply to the Uri parameter in those methods.  There’s one other thing to be aware of as well however, and that’s the array of entity types that are provided in these methods.  The string[] entityTypes parameter is important to understand because that lets you know what type of result (i.e. PickerEntity) you should return when FillResolve or FillSearch is invoked.  The main scenario where it matters is when you are setting the site collection administrator.  To this day, SharePoint has a limitation that only an individual user (or better stated really – only an identity claim) can be added as a site collection administrator.  What I’ve found is that when you are trying to set the site collection administrator and your custom claims provider is invoked from within central admin, the entityType array correctly contains only one value – SPClaimEntityTypes.User.  HOWEVER…if you are in a site collection and you try and change the site collection administrators from within it, the entityType is returning five different entity types, instead of just User.  At this point I don’t know of a good way to distinguish that from a legitimate request within the site collection where all of those entity types would be appropriate – such as when you’re adding a user / claim to a SharePoint group or permission level.

I don’t know if or how much any of this may change in the future (i.e. are they considered bugs or not), so for now I just wanted to document the behavior so as you’re creating your designs around your farm implementations you know better exactly what info you’ll have within your custom claims provider to help make fine grained security decisions.

Claim Type Exceptions with Custom Claims Providers in SharePoint 2013

This issue applies to SharePoint 2010 as well but…suppose you have created a custom claims provider and one of the things you want to do is to have some custom claim types that you use.  What I mean by custom claim types is just that they are not one of the standard out of the box types like email, upn, role, etc.  Now in this posting here: http://blogs.technet.com/b/speschka/archive/2011/04/02/how-to-add-additional-claims-in-adfs-2-0-that-can-be-consumed-in-sharepoint-2010.aspx – from way back in 2011, I described the format that you need to use for your claim type in order to be accepted by SharePoint.

Suppose that you have built and deployed your custom claims provider though and you decide you really need another claim type – we’ll call it http://www.foo.com/bar. So you start using the claim by creating a new SPClaim and that works just fine.  But then you want to use the claim with something where you require an encoded value – like adding it to a web app policy or a SharePoint group or whatever.  So you create an instance of the SPClaimManager and you try the EncodeClaim method on the value of your SPClaim.  Instead of happiness, what you get is an error that says you have an ArgumentException, the param name is claimType, and in the stack trace you see it happened in the EncodeClaimIntoFormsSuffix method.

You may think hey – I just need to add this claim type to the list of claims my provider supports. I do that by modifying my code in the FillClaimTypes override of my custom claims provider.  That actually is correct – you DO need to do that.  So you do that, recompile and redeploy the assembly…and continue to get the same error.  Well the fact of the matter is that you’ve done everything correctly from a coding standpoint.  The problem is that this is a different flavor of the same problem that SharePoint SPTrustedIdentityTokenIssuers have, which is the claims collection is immutable.  That just means that after you create the claims mappings for an SPTrustedIdentityTokenIssuer, you can not go back and change it afterwards.  Yes, I know people have posted code on how to do that; it is also unsupported to do so.

So, how do you fix this?  If you are NOT the default claims provider, you should be able to just uninstall and remove the custom claims provider feature, remove all instances of it from the GAC and deploy all over again.  If you ARE the default claims provider, then unfortunately the fix is the same way you would as if you needed to change the claims mappings for your SPTrustedIdentityTokenIssuer.  You need to change the configuration for any web apps that are using your SPTrustedIdentityTokenIssuer so that they no longer use it, and then remove your custom claims provider.  Then you need to remove the SPTrustedIdentityTokenIssuer, redeploy your custom claims provider, recreate your SPTrustedIdentityTokenIssuer, make your custom claims provider the default provider, then go back and add the SPTrustedIdentityTokenIssuer to the web apps that need it.  It wears me out just typing out that sentence, let alone doing the work.

Once you’ve done your redeployment though, SharePoint should pick up on your new collection of claims that you will be using.  Until you need to change them again.

Programmatically Adding A Trusted Identity Token Issuer to a Web Application Zone in SharePoint 2010 and 2013

Seems like I haven’t had a chance to write a good SharePoint / SAML claims kind of article in a while.  I was happily plugging away on some code this weekend for a scenario I haven’t done before so I thought I would go ahead and post it here for the search engines.  The whole topic in general has kind of reached a point where I’m not really sure if anyone even needs these kinds of postings anymore, but ultimately I decided it’s worth jotting down because there will probably be someone, somewhere that needs to do this in a hurry so perhaps this post will help.

So the topic today is changing the configuration of a web application zone so that it uses an SPTrustedIdentityTokenIssuer – i.e. I want to configure my web app to use SAML claims.

The first challenge I typically face with folks is getting them to restate the problem – more precisely, in this case we’re not configuring a web app to use SAML, we’re configuring a zone in the web app to use SAML.  This is a key to remember because you need to know that as you start plugging through the object model to find and change these settings.  So let’s start working through some code.

The first step of course is going to be to get a reference to the web application you want to work with.  You can use the ContentService to do that, or I frequently just use a site.  Here’s an example:

using (SPSite s = new SPSite(https://foo))

{

SPWebApplication wa = s.WebApplication;

 }

Okay, so now that I have my web application, I need to get the IIS settings for the zone.  In this case I’m just going to hard-code it to look in the default zone:

Dictionary<SPUrlZone, SPIisSettings> allSettings = wa.IisSettings;

SPUrlZone z = SPUrlZone.Default;

SPIisSettings settings = allSettings[z];

As you can probably tell from the code, you can get the settings for any zone, but we’re grabbing them for the default zone.  Once I have that, I can look at what authentication providers the zone is using with this:

List<SPAuthenticationProvider> providers = settings.ClaimsAuthenticationProviders.ToList<SPAuthenticationProvider>();

So for example if you wanted to see if your provider was being used before trying to add it, this is where you would look.  In our case we’ll just assume it’s not being used by this zone yet, so next we need to get the SPTrustedClaimProvider that represents our SPTrustedIdentityTokenIssuer.  We’ll use this code to do that:

//get the local claim provider manager, which tracks all providers and get all trusted claim providers into a list

SPClaimProviderManager cpm = SPClaimProviderManager.Local;

List<SPTrustedClaimProvider> claimProviders = cpm.TrustedClaimProviders.ToList<SPTrustedClaimProvider>();

//look to see if ours exists

const string ISSUER_NAME = “MyTokenIssuer”;

var cp = from SPTrustedClaimProvider pd in claimProviders

where pd.DisplayName == ISSUERI_NAME

select pd;

SPTrustedClaimProvider ceProvider = null;

 

try

 {

ceProvider = cp.First<SPTrustedClaimProvider>();

}

catch

{

//it will throw an exception if it doesn’t exist

}

Okay, so at this point our provider should be in the ceProvider variable.  Once we have that, we have a couple of different options to configure  the zone to use it.  If you just want to add it to the collection of authentication providers the zone is using, use the AddClaimsAuthenticationProvider method of the IisSettings instance.  If you want to replace the list of authentication providers so it only uses your SAML provider (which also means it can’t be crawled, since that requires NTLM), use the ReplaceClaimsAuthenticationProviders method.  Here’s some code that demonstrates both methods, you would obviously only use one of them:

 

if (ceProvider != null)

{

SPAuthenticationProvider newProvider = new SPTrustedAuthenticationProvider(ceProvider.Name);

 

//this works to add it to the collection of claims providers

settings.AddClaimsAuthenticationProvider(newProvider);

 

//this can be used to replace it so you only have the trusted provider

providers = new List<SPAuthenticationProvider>();

providers.Add(newProvider);

settings.ReplaceClaimsAuthenticationProviders(providers);

 

//update the web application to save the authentication provider settings

wa.Update();                 

 }

So there you have it, hopefully if you ever need to do this now you can grab this code and run with it.

Integrating SharePoint 2013 with Azure Active Directory – Part 2 The Custom Claims Provider

In Part 1 of this series, we went through how to configure SharePoint to use ACS and Azure Active Directory (AAD) as our Identity Provider.  Once that is complete you will have a working end to end solution in which you can authenticate, get authorized and work in the site.  What you also have is the standard out of the box experience, which means you have the “echo chamber” people picker experience (i.e. whatever you type in, people picker will “echo” back out that it is a valid value for every claim you have mapped).

One of the things we love about AAD though is the Graph API – an out of the box API and REST endpoint that we can use to query the directory.  This is really one of the big value propositions for using AAD with SharePoint; unlike a lot of other SAML directories, this one has a queryable directory ready to go.  In this post I’ll cover some of the Graph API programming that I did, but I won’t focus exclusively on that because there are or will be lots of Graph API code samples out there (try starting here if you are lost:  http://msdn.microsoft.com/en-us/library/windowsazure/hh974476.aspx).  Of more importance I think is for you to understand some of the features of the Graph API and how that impacts how one might typically write a custom claims provider.  With that background, I’ll explain then the reasoning I used in deciding how to implement the different features of the custom claims provider.

Let’s talk first about some of those features of the Graph API and AAD because it should provide better context as to why I made certain implementation choices.  The first release has these constraints that you need to be aware of (some of which I covered in Part 1):

  • When a user is authenticated through ADFS to AAD to ACS, the role claims for the user are not sent back.
  • The claims sent back for a user are part of a fixed schema.  You cannot assume that every claim you define in ADFS will be sent back.  Instead, only the properties in the AAD schema are returned; you can see the complete schema here:  http://msdn.microsoft.com/en-us/library/windowsazure/dn195587.aspx.
  • The Graph API only supports “AND” queries, not “OR” queries.  In SharePoint a user typically enters their query, and we return results that match across any of the mapped claims.  Since you can’t create a query that says were foo=”James” OR bar=”James”, it means that you have to issue a separate query to Graph for every attribute you want to query on, which can obviously lead to performance / latency issues.
  • There is not any way to do wildcard searches with Graph API; it supports equals, less than or equal to, and greater than or equal to.  Of course, users are used to typing “Ste” and getting back “Steve”, “Stephen”, “Stephanie”, etc. so this can be a tough one, but I do have one thought on this problem and have incorporated it into the custom claims provider I’m going to be describing here.
  • You are effectively forced to use UPN as the identity claim.  This is because all queries for user attributes to the Graph API must include either the user’s UPN or objectID, and the objectID is not useful if you are human, so I will focus here on using UPN.

Okay, now that we know the parameters under which we will be working, let’s talk about the implementation of the custom claims provider.   What I would suggest first is that you get your development environment where you will be creating your custom claims provider set up and ready to go to build Graph applications.  In order to do that there are a few steps you need to do:

  1. Download the WCF Data Services for OData 3 from http://www.microsoft.com/download/en/details.aspx?id=29306.
  2. Get the CreateServicePrincipal.ps1 PowerShell script that is included with the sample application here:  http://code.msdn.microsoft.com/Write-Sample-App-for-79e55502
  3. Download and install the PowerShell tools described here:  http://technet.microsoft.com/en-us/library/jj151815.aspx
  4. Run the CreateServicePrincipal.ps1 PowerShell script to create your ServicePrincipal.  The script will output a set of values at the end like TenantDomainName, AppPrincipalId, Password, etc. that will be required for getting the access token you need to query the Graph API.  Once you run the script you should see output like this:

TenantDomainName: dreamswirls.onmicrosoft.com
TenantContextId: 06c83d0b-d384-4dcd-a8fc-a347d83a37b4
AppPrincipalId: 1f438108-ba1b-427f-9ce9-3a99d1aa2103
Password: e+XalkeKfs6Ax1Fqj++OP4mcS8PKQDHwzeG7rB7LiM=
Audience URI: 1f3cd808-21b-4d7f-9ee9-8298f1aa2103@06832a0b-d3dd-4e3d-a0fc-a3478ad2b7b4

Now that you have details needed to connect to the Graph API, we can set about the first task, which is to address the fact that the role claims for the user don’t come through.  The way we’ll tackle solving that is with claims augmentation, by implementing the FillClaimsForEntity method in our provider.  What I’ll need to do is to get the user’s UPN claim; as I explained in Part 1 of this series, we created a claim rule in ACS that sends us the UPN claim.  Unfortunately, even in SharePoint 2013 we still can’t get the user’s claims directly from the parameters provided when the FillClaimsForEntity method is invoked.  Instead then we’ll use the method I previously described here:  http://blogs.technet.com/b/speschka/archive/2011/03/29/how-to-get-all-user-claims-at-claims-augmentation-time-in-sharepoint-2010.aspx.  This method works in SharePoint 2013 as well as SharePoint 2010.   That may make this a good time to call out one other point – this entire scenario works both for SharePoint 2010 and SharePoint 2013.  In the attachment that accompanies this post, the zip file will contain a Word document with this posting, a complete solution compiled for SharePoint 2010 and Visual Studio 2010, and a complete solution compiled for SharePoint 2013 and Visual Studio 2012.

To get back on track now…the first thing you need to keep in mind is that this code should only fire if the current user is a SAML claims user, so I use code like this to make sure that is the case:

string upn = string.Empty;
 
//get the claim provider manager
SPClaimProviderManager cpm = SPClaimProviderManager.Local;
 
//get the current user so we can get to the “real” original issuer
SPClaim curUser = SPClaimProviderManager.DecodeUserIdentifierClaim(entity);

//get the original issuer for the user
SPOriginalIssuerType loginType = SPOriginalIssuers.GetIssuerType(curUser.OriginalIssuer);
 
//we only need to do this for SAML users, so see if that’s what our user is
if ((loginType == SPOriginalIssuerType.TrustedProvider) ||
 (loginType == SPOriginalIssuerType.ClaimProvider))
{
       //run the code to get the UPN

Okay, now that I know I’m working with a SAML user, here’s how I’ll get their UPN from the token sent to SharePoint:

//get the request envelope with the claims information
 
string rqst = System.ServiceModel.OperationContext.Current.RequestContext.RequestMessage.ToString();
 
//create an Xml document for parsing the results and load the data
XmlDocument xDoc = new XmlDocument();
xDoc.LoadXml(rqst);
 
//create a new namespace table so we can query
XmlNamespaceManager xNS = new XmlNamespaceManager(xDoc.NameTable);
 
//add the namespaces we’ll be using
xNS.AddNamespace(“s”, “http://www.w3.org/2003/05/soap-envelope“);
xNS.AddNamespace(“trust”, “http://docs.oasis-open.org/ws-sx/ws-trust/200512“);
xNS.AddNamespace(“saml”, “urn:oasis:names:tc:SAML:1.0:assertion”);
 
//get the list of claim nodes
XmlNode xNode = xDoc.SelectSingleNode(“s:Envelope/s:Body/trust:RequestSecurityToken/trust:OnBehalfOf/saml:Assertion/saml:AttributeStatement/saml:Attribute[@AttributeName = ‘upn’]”, xNS);

//get the match and pull in the UPN
if (xNode != null)
 upn = xNode.FirstChild.InnerText;

Now that I’ve got the UPN, I can go ahead and query the Graph API to get a list of groups the user belongs to; I’ll add each matching group as a Role claim for the user.  I’ll show the code here and then walk through it:

//see if we got the upn
if (!string.IsNullOrEmpty(upn))
{
 
 //make a request now to get the list of groups for this user
 DataClasses.UsersAndGroups groups = GetUsersAndGroups(upn, AadObjectType.GroupMembership);
 
 //add each group we got back as a role claim
 foreach (DataClasses.Group g in groups.AllGroups.GroupList)
 {
  claims.Add(new SPClaim(ROLE_CLAIM, g.DisplayName,
   Microsoft.IdentityModel.Claims.ClaimValueTypes.String,
   SPOriginalIssuers.Format(SPOriginalIssuerType.TrustedProvider,
   SPTrustedIdentityTokenIssuerName)));
 }
}

Let’s look at the code a little more closely now.  The first thing to call out is the code to retrieve the list of groups for this user.  In this case I’m calling a method called GetUsersAndGroups and I’m basically saying for this UPN I want a list of all the group memberships.  This is all code I wrote myself so it’s included in my provider.  Within that code it uses the ServicePrincipal information I described earlier in this post to query the Graph API.  In this case I’m providing the Url to the specific user and adding the memberOf attribute; the resulting Url looks something like this:  https://graph.windows.net/dreamswirls.onmicrosoft.com/users/speschka@dreamswirls.com/memberOf?api-version=2013-04-05 (the “api-version” is an additional attribute required for all calls into Graph).

That leads into the second point, which is that I’m just using the REST endpoint directly to retrieve all my data from AAD.  I also have an access token that I need to include in my request headers.  I have a class-level variable in which I store that, so I start out by seeing if it’s been retrieved yet and if not, I go get it so I don’t have to make that additional web call each time. 

I also have code in the method that GetUsersAndGroups calls that will empty out that class-level variable for the access token if I get an error that suggests the access token is expired.  In the GetUsersAndGroups method I just look after I call that method to see if the access token is empty; if so then I do a one-time retry to get the data again.  When the code executes again it will go out and request a new access token and then get back to work.  It’s not fool proof of course, but will work adequately for most cases.

When I do get data back, I store it a custom class that I created to store results for both Users and Groups called DataClasses.UsersAndGroups.  I enumerate through all the groups and add them as role claims to the user.  Here’s the final thing to point out in this chunk of code – note how I am using the new SPClaim method to create the claim I’m augmenting, instead of the CreateClaim method.  The reason for that is that this provider must be the default because it is issuing identity claims.  The way to do that is with the new SPClaim method, as I’ve documented previously here:  http://blogs.technet.com/b/speschka/archive/2010/05/25/replacing-the-out-of-box-name-resolution-in-sharepoint-2010-part-2.aspx.  When users use the people picker to add users and groups to SharePoint groups, my custom claim provider is adding the claims in this way to, so when I add claims via augmentation I need to add them the same way.  If I didn’t, a role claim for “foo” added with CreateClaim would not have the same claim value as one added with new SPClaim.  As a result, someone who got the “foo” role claim added via augmentation would NOT have access to content.

Finally, remember one of the things I pointed out above, that you don’t get all claims back, just the fixed schema that is defined by AAD.  In this case I decided that all I really need for my farm is going to be the identity claim and role claims.  I got the identity claim at authentication time and I got the role claims in the code above.  If had some other set of claims though that I really needed for my farm, then I would also grab them in my FillClaimsForEntity method and add them there.  The number of claims you use though also has an impact on the Fill… methods that I will describe below.

Okay, there was some additional explanation in there that is relevant to all of the Graph API queries I’m using, so the explanation for the FillClaimsForEntity was a little longer than the rest will be.  Now I’ve mentioned that I am of course the default claims provider in this scenario, and that means I’m going to also need to implement both FillResolve methods and the FillSearch method.  I’ve also implemented other methods like FillHierarchy, FillClaimTypes, etc., but there’s nothing specific to using AAD about them so I won’t cover those here.  You can always look in the source code included with this post if you are curious about what’s in them.

Let’s talk about FillSearch next.  Now in this case, some user is going to type in a string and expect to find every claim that has a value that starts with what they typed in; using the example I gave above, they type in “Ste” and they would be shown identities like “Steve”, “Stephen”, “Stephanie”, etc.  Of course though, there are the problems I described above – we can’t issue “OR” queries and there isn’t support for wildcard queries…so what do we do?  Well first of all, as I mentioned earlier, I decided to limit the claims I am going to use in this farm to two:  identity claim and role claim.  The fact that I can’t issue an OR query means that whenever FillSearch is called, I’m going to have to fire off two queries to the Graph API – one to look for a matching identity and one for a matching group.

This is also why it’s impactful if you need more claims – the more you have, the more queries you have to execute each time someone types a new letter in the people picker.  Make sure you understand the behavior in SharePoint 2013 – after you type the 3rd character or more and pause, SharePoint will search for names.  Each character you type after that and pause, SharePoint will do another search.  You can imagine the number of queries you could end up firing off here, especially if you have multiple claim types to query against.

I also made another design choice here, in that I decided to make these queries look at displayName for matches.  In a perfect world I could look at first name, last name, displayName, emailAddress, UPN, etc., but last time I looked things aren’t perfect, so I chose what I think is the most common use case.  A user can still type in “Ste”, and I can hopefully bring back “Steve Peschka”, “Stephen Jackson”, “Stephanie Phillips”, etc.

Now the next problem is the wildcard issue – how can I find all those users I just listed if a user only types in “Ste”?  Well, as I hinted above, I ended up coding my own solution for this.  I will tell you now that it is not perfect, but, you should know my feelings on perfection by now.  So instead I wrote a function that I called GetWildcardQuery.  It takes a field name and query criteria, and spits out a string that can be used with the $filter operator in the Graph API.  So how does it work?  Well it’s what I call a classic Steve string munging routine.  It takes the input string and figures out what the logical “end” of that string is and adds one character. 

This is of course easier explained with some examples.  So suppose a user typed in “Ste”; the logical end of that string would be “Stz”, so when I add one it returns “Stf” (i.e. I want everything between “Ste” and “Stf”).  That allows me to create a query that says greater than or equal to “Ste”, and less than or equal to “Stf”.  The most obvious problem here is that anything that starts with “Stf” would also be a match, even though it should not be.  However, you can’t just say less than or equal to “Stz”, because then it would not match on “Stza”, “Stzaa”, and so forth and so on.  So the lesser of all evils is “Stf”.  Other examples (that you will find in the source code as my test cases) include “sz” should return “t”, “srz” should return “ss”, “sar” should return “sas”, “frzz” should return “fs”, “frizz” should return “frj”.  It’s not beyond my imagination that someone will say “hey, that’s wrong, here’s why”, but at least you know how I developed this logic and why.

So now that you understand how I chose to deal with the a) lack of an OR query and b) lack of a wildcard query, let’s get back to the SharePoint stuff.  It’s pretty simple really – in my FillSearch method I just execute two queries, and if I find matches then I add them to the list.  The relevant here code looks like this:

//query for users
DataClasses.UsersAndGroups people = QueryDirectory(searchPattern, AadObjectType.Users);
 
if (people.AllUsers.UserList.Count > 0)
{
 //add results to picker
}
 
//query for groups
DataClasses.UsersAndGroups groups = QueryDirectory(searchPattern, AadObjectType.Groups);
 
if (groups.AllGroups.GroupList.Count > 0)
{
 //add results to picker
}

The only thing worth calling out here is that the QueryDirectory is just a little stub for querying based on displayName.   It uses that field name with the wildcard filter method and then queries the directory as shown previously.  Essentially just two lines of code that looks like this:

string criteria = GetWildcardQuery(“displayName”, filter);
results = GetUsersAndGroups(criteria, objectType);

Here’s an example then of what the picker looks like using my wildcard logic:

There’s more entries than I can show in this screenshot but you get the idea – I typed in “fre” and it found a user called Freddy, a group called French Club, and a group called Frequent Flyers.  So, not perfect, but not bad either.

Now let’s talk about the two FillResolve methods.  They are somewhat similar to FillSearch but not exactly.  The first FillResolve method I’ll talk to is the one that’s called when you use the Search dialog and includes the SPClaim as a parameter.  Now what’s nice about this FillResolve method is that by looking at the SPClaim parameter, I can tell whether this is an identity claim or a role claim, so I only have to send off one query to the Graph API.  Other than that it looks very much like search; the main difference though is that I know this should only ever return one entity, since it represents a selection made from the picker so I always just pick the first match if there is one (which there always should be):

if (resolveInput.ClaimType == USER_CLAIM)
{
 DataClasses.UsersAndGroups users = GetUsersAndGroups(resolveInput.Value, AadObjectType.User);
 
 if (users.AllUsers.UserList.Count > 0)
 {
  //add a PickerEntity to the list
  }
}
else
{
 DataClasses.UsersAndGroups groups = QueryDirectory(resolveInput.Value, adObjectType.Groups);

 if (groups.AllGroups.GroupList.Count > 0)
 {
  //add a PickerEntity to the list
 }
}

For the last FillResolve it’s really just like search all over again…someone types in some assortment of characters and clicks the Resolve button.  It could be a user, it could be a group, who knows.  So I just treat it exactly like search:

//query for users
DataClasses.UsersAndGroups people = QueryDirectory(resolveInput, AadObjectType.Users);
 
if (people.AllUsers.UserList.Count > 0)
{
 //add results to list
}
 
//query for groups
DataClasses.UsersAndGroups groups = QueryDirectory(resolveInput, AadObjectType.Groups);

if (groups.AllGroups.GroupList.Count > 0)
{
 //add results to list
}

So that really covers the gist of the custom claims provider implementation.  There are a few other methods implemented and an event receiver to install the provider, but those are all just standard custom claims provider code snippets, nothing specific to AAD so I didn’t bother adding them to this already very long post.  Once it’s been compiled and deployed I just went back and modified the SPTrustedIdentityTokenIssuer to make this the default claims provider, as I’ve previously described here:  http://blogs.technet.com/b/speschka/archive/2010/04/28/how-to-override-the-default-name-resolution-and-claims-provider-in-sharepoint-2010.aspx.   After that you’re good to go. 

I tested this provider in what I believe are all the important scenarios – selecting a site collection administrator, using both an identity claim and role claim in a web app policy, using an identity and role claim in SharePoint groups, and with the permission checker dialog.  All have passed with flying colors, so this is now ready for you to try out if you wish – enjoy!

You can download the attachment here:

Checklist for Issues with Custom Claims Providers in SharePoint 2010 and 2013

As I was going round and round a few weeks ago trying to figure out why my custom claims provider was not working as I anticipated, one of our great developers (Chris R.) gave me a list of things to look at to try and diagnose the issue.  After spending about 5 minutes on his list I realized where the problem was, so I thought I would share his list as food for thought, and throw out a couple of other things I’ve seen and do as well.  This will be a living list, so as Chris or I come up with other suggestions we’ll just update this posting.  Also, if you folks have tips that you find useful I would encourage you to add them in the comments below.  I can’t offer any prizes other than a “thanks” from your fellow SharePoint developers!  🙂 

So, that being said, here’s the list:

  1. Are you logging the call to FillClaimsForEntity?  This is a great way to make sure that you can ensure all the claims you think should be getting added, are getting added.  I wrote a post about a pretty straightforward way to add logging to your applications some time ago, and it can be found here:  http://blogs.technet.com/b/speschka/archive/2011/02/04/tips-for-uls-logging-part-2.aspx.
  2. Look at a list of what claims are being added by your custom claims provider by emitting the encoded claim values.  For example, after you add your claims in FillClaimsForEntity you can enumerate through all of the claims for the user with code that looks something like this:  foreach (SPClaim c in claims)  {  Debug.WriteLine(c.ToEncodedString());  }.  Once you have the encoded claims that have been added, how can you use them?  See tip #3 for that.   
  3. What claims have been added to the site?  Go into your Site Settings and find a claim that you have added to a Site Group.  Click on it so you can get to the claim details, and it will show you the encoded claim value.  Does that encoded claim value that has been added to the Site Group match one of the encoded claims you have added via augmentation (that you’ve captured in tip #2)?  If not, then that may be why users cannot access the site.
  4. Are you sure your claim provider is active and enabled?  Use the Get-SPClaimProvider cmdlet to look at your provider.  If it IsEnabled is true, but IsUsedByDefault is false, then you should check to make sure it is used in the zone where you are expecting to use it.  You can use the ClaimProviderActivation tool for this, which I originally blogged about here:  http://blogs.technet.com/b/speschka/archive/2010/06/03/configuring-a-custom-claims-provider-to-be-used-only-on-select-zones-in-sharepoint-2010.aspx and then updated for SharePoint 2013 here:  http://blogs.technet.com/b/speschka/archive/2012/09/07/important-change-for-custom-claims-providers-in-sharepoint-2013-and-refresh-of-some-favorite-claims-tools.aspx.
  5. You can also check ALL of the claims that the current user has by using my HttpModule that writes out the users claims to the ULS logs.  Again, it provides all the encoded claim values even if a person is denied access to the site, you can see what claims they presented and use tips 2 and 3 above to try and reconcile why access has been denied.  This was originally posted here:  http://blogs.technet.com/b/speschka/archive/2010/02/13/figuring-out-what-claims-you-have-in-sharepoint-2010.aspx, and was updated for SharePoint 2013 here:  http://blogs.technet.com/b/speschka/archive/2012/09/07/important-change-for-custom-claims-providers-in-sharepoint-2013-and-refresh-of-some-favorite-claims-tools.aspx.
  6. A fairly common practice in custom claims providers is to check in the FillResolve methods to see if the entityTypes collection contains FormsRole and if not, to exit.  However, if you do this then your custom claims provider will not work when SharePoint is expecting an identity claim only – for example, when selecting a Site Collection Owner or using the Check Perms page in Site Settings.  So instead of checking for FormsRole only, IF you have developed a custom claims provider that is the default claims provider, so you are issuing identity claims, then you should check to see if the entityType contains either FormsRole OR User.
  7. Make sure you have all your custom claim provider code in try blocks because an untrapped failure it will result in your and other providers not resolving names at all.  If you start seeing that claim values aren’t resolving at all, for any provider, it’s a good bet that this is the problem.  Note that you are most likely to see this when trying to resolve an individual user.  Not always, but that’s the more common case.
  8. If you get an error along the lines of “user does not exist or is not unique” it usually means that your FillResolve method is returning 0 or 2 or more claims when SharePoint is expecting only one.  This one is more difficult to track down, and most times requires a debugger (or really good logging) to figure out what the issue is.  This is also an example of an error that you will generally only see when trying to add a user – that’s why I try and test so thoroughly on the Site Collection Owner and Check Perms pages whenever I develop a custom provider.

That’s the list for now – as I said, we’ll keep updating this as we have more things to add.  Also, please add your suggestions in the comments too!  Hope this helps someone, somewhere, out there.