CloudTopia: Connecting o365, Apps, Azure and Cortana – Part 1

This is going to be a multi-part series to dissect and tear apart an application I tweeted about using the #yammerofjuly hashtag.  This is an application I developed a couple of months ago as a means of illustrating some of the different complexities and options when building applications that span several cloud services.  Through the course of this series I’ll start out by looking at the application overall, and then start breaking down some of the different components of it and how everything is tied together in the hopes that you can use it to integrate some of these same service connection points within your own applications.

NOTE:  You can find ALL of the source code for everything in this series on GitHub at https://github.com/OfficeDev/CloudTopia-Code-Sample

Here’s some quick links to the whole series:

  1. Intro
  2. Open Graph
  3. Web API
  4. Azure Integration
  5. Office 365 Integration
  6. Cortana, Speech and Windows Phone Integration

We’ll start by looking at the business requirements for the application.  In our scenario we are working with a company that manages public events for their customers.  They have the basic collaboration needs – for each event they create brochures, flyers, calendars, budgets, schedules, etc.  In addition to that they want to be able to have a discussion forum around the event so they can talk through different aspects of the event with the team.  They also want to have some way to get a feel for the buzz about the event that’s happening out in the public.  They’re just looking for some way to get a feel for how their marketing of the event is working.

Given those requirements, I built the application I call CloudTopia.  It uses several services and technologies that will be described in subsequent posts:

  • Office 365
  • Yammer Open Graph
  • SharePoint Cloud App Model in a provider hosted app
  • JavaScript and jQuery
  • Web API 2.1
  • Azure web sites
  • SQL Azure
  • SQL Scheduler service
  • Twitter

So as you can see, a lot of things working together here to make our solution.  To start things off, I created a demo video of the application – you can watch it here:  https://onedrive.live.com/redir?resid=96D1F7C6A8655C41!18616&authkey=!AC4csVj2oAU6B7M&ithint=video%2cmp4.  After you’ve watched the video come back here and let’s talk a little bit about what you saw and how it was built.

 

Design and Architecture

Here’s a screenshot of the home page of the CloudTopia application with callouts for the different technologies and services being used:

 

 

Here’s a brief summary of each component:

  • Office 365 – the whole application starts out in an Office 365 site.  I just created a new site in o365 tenant and made it the home page for the application.
  • Cloud App Model – the SharePoint 2013 Cloud App Model (CAM) was the starting point that I used to create this application.  It ultimately relied upon several other services and technologies, but it was all delivered through CAM.  One of the main features of the application – the UI you see in the o365 site – is just the standard iFrame that you get with a provider hosted application.  From where it says “Upcoming Social Events” on down is all part of the app.
  • Azure Web Sites – one of the great things about CAM and provider hosted apps is that you can host them pretty much anywhere.  In my case I decided to use one of the free Azure web sites that I get with my Azure subscription to be the “host” in my provider hosted application.  It also underscores an important theme with this application: it’s called “CloudTopia” because it is 100% running in the cloud; there are NO on premises components to this application.
  • SharePoint List Data – all of the events and some related metadata is stored in a hidden list in the o365 Events site.  The list is created by a remote event receiver when the application is installed.  The event receiver creates the list, makes it hidden, and removes the list from the quick navigation links on the left side of a standard team site page.  For more information on this process of creating a list this way in the host web of an application see my previous post here:  http://blogs.technet.com/b/speschka/archive/2014/05/07/create-a-list-in-the-host-web-when-your-sharepoint-app-is-installed-and-remove-it-from-the-recent-stuff-list.aspx.
  • More o365 sites and Yammer Open Graph Items – each time a new event is added, a new o365 site is created for the event and a Yammer Open Graph item is created that is used or discussions on the event.  More details follow below where I describe what all happens when you create a new site.  In terms of working with Yammer Open Graph from .NET, I have already covered that in previous posts on my blog.  You can go to the Open Graph specific post at http://blogs.technet.com/b/speschka/archive/2014/05/29/using-yammer-open-graph-in-net.aspx; it’s also part of the bigger Yammer toolkit for .NET that I’ve been building over time that started with this post:  http://blogs.technet.com/b/speschka/archive/2013/10/05/using-the-yammer-api-in-a-net-client-application.aspx.
  • SQL Azure – the application uses SQL Azure to store certain data that is necessary to query twitter and add discussion items to Yammer Open Graph objects.  In a later part in this series I’ll explain why I chose to use SQL Azure instead of SharePoint list data to store this information.  As an added bonus, I used a free SQL Azure data instance that comes with an MSDN subscription.  I’ll also cover that a little later.
  • Azure Scheduler – as you saw in the demo video, one of the things the application does is go find tweets that match the hashtags for our event and then add those tweets to the Open Graph discussion for the event.  That work happens once a day and is triggered using a job in the Azure Scheduler service.  Due to the low volume of these requests it falls within the number of free jobs that can be scheduled with Azure.
  • Web API 2.1 – a key component to the entire CloudTopia application is the use of Web API.  It’s primary use case is that it does all of the hard work for all of the features of the application – creating o365 sites, creating Yammer Open Graph items, searching Twitter, etc.  One of the big reasons why it’s so valuable for SharePoint Apps is that it allows me to control the UI entirely client side, but still use all the power of CSOM and any other .NET API.  You saw a lot of activity in the demo video and all of that was done through JavaScript and jQuery calling the custom REST endpoint that was created for the CloudTopia application.  In addition to that, it allowed me to create a REST endpoint just for the Azure Scheduler job.  A job can call an HTTP or HTTPS endpoint to do something, but there needs to be a listener that can “do something” when invoked.  Adding an endpoint in my REST controller provides the interface for just that kind of automated integration with other applications.

 

What Happens When You Create a New Event

There are quite a few things that get triggered when you go through the seemingly simple process of creating a new event.  Let’s take a look at what happens.

  1. A new o365 site is created for the event.
  2. The out of the box Site Feed web part is removed from the new o365 site.
  3. A new Open Graph (OG) item is created in Yammer.
  4. The Url link for the OG item is set to be the Url of the new o365 site
  5. This also means you can go into Yammer and just search for your o365 site Url and you’ll find the Open Graph object.  Pretty cool.
  6. A welcome post is added to the Open Graph discussion.
  7. A script editor web part is added to the new o365 site; it uses Yammer Embed to display the feed for the Open Graph item.
  8. A new item is added to the hidden SharePoint list in the host site where the App part is installed; that’s how it shows up in the event list in the host site.
  9. A new item is added to SQL Azure with the Open Graph ID along with the new Site Url and Twitter tags.

That’s a lot of stuff!  That’s also good for the intro of this series.  In Part 2 of this series I’ll talk about some of the work I did around Yammer Open Graph objects in CloudTopia.

CloudTopia: Connecting o365, Apps, Azure and Cortana – Part 2

In Part 1 of this series, I introduced you to the CloudTopia app.  In Part 2 we’re going to look at some of the work we did with Open Graph items in CloudTopia.

Here’s some quick links to the whole series:

  1. Intro
  2. Open Graph
  3. Web API
  4. Azure Integration
  5. Office 365 Integration
  6. Cortana, Speech and Windows Phone Integration

As I described in Part 1, Yammer Open Graph (OG) items are used in CloudTopia in a couple of different ways:  1) to provide a forum for discussion for the team working on an event and 2) as a way to bring in external discussions from Twitter to the OG item so the team can get a good sense of the external buzz that’s happening around their event…like #yammerofjuly (inside joke for those of you who have been following me on Twitter).  As I also described in Part 1, I covered the basic details of working with OG items in a previous post here:  http://blogs.technet.com/b/speschka/archive/2014/05/29/using-yammer-open-graph-in-net.aspx.  I won’t be covering that all over again, but I will be covering some of the other implementation details and things you should be aware of when working with your own OG items.  And that’s really the point of this entire series – not how to write this application per se, but cover some general purpose implementation details and things you should be aware of.

Okay, so we’ll use the blog post above as a starting point for working with OG items, now let’s look at some of the implementation details.  In my previous posts on OG I talked about the object model that I created over an OG item and how to use it to create a new OG item in Yammer.   For review, here’s what my object model looks like:

 

To use it you need to send a chunk of JSON to Yammer to create the OG item.  What I did to facilitate this is let you create an OG item using my object model, and then when you call the ToString() method I’ve overridden that so that it produces the JSON you need.  With that in hand you can use one of the methods I included in my original Yammer and .NET posting to create the item, by calling the MakePostRequest method.  The net of this is that your code looks pretty straightforward:

YammerGraphObject go = new YammerGraphObject();

go.Activity.Action = “create”;

 

go.Activity.Actor = new YammerActor(“Steve Peschka”, “speschka@yammo.onmicrosoft.com”);

 

go.Activity.Message = “This is the discussion page for the ” + eventName + ” event on ” + eventDate;

 

go.Activity.Users.Add(new YammerActor(“Anne Wallace”, “annew@yammo.onmicrosoft.com”));

go.Activity.Users.Add(new YammerActor(“Garth Fort”, “garthf@yammo.onmicrosoft.com”));

 

YammerGraphObjectInstance jo = new YammerGraphObjectInstance();

jo.Url = Url;

jo.Title = eventName; 

jo.Description = “This is the discussion page for the ” + eventName + ” event on ” + eventDate; 

jo.Image = “https://socialevents.azurewebsites.net/images/eventplanning.png”;

jo.Type = “document”;

 

go.Activity.Object = jo;

 

string postData = go.ToString();

 

string response = MakePostRequest(postData, graphPostUrl, yammerAccessToken, “application/json”);

 

The other thing I mentioned in part 2 of that series on OG items is how important it is to have the OG ID; it is required whenever you want to read from or write to the discussions for the OG.  As I mentioned in that post there are basically three ways to get the OG ID:

  1. Capture it when the OG item is created.  You’ll get some JSON back if the create is successful and we can use that to extract out the ID of the newly created item.
  2. Search for an OG item.  You can search using the Url property of the OG item; however this only returns results if there is at least one discussion item created for the OG.  This is another reason why the CloudTopia app creates an initial discussion item when creating the OG.
  3. Do a “fake” update to the OG item.  What I mean by fake is that I just pass in my own username and email address as the actor, I change the Action to “follow”, and I set the Private property of the Activity to true.  When I do that I get back the same chunk of JSON as I do when I create an OG, so again I can extract out the ID from there.

 

In the case of CloudTopia I simply capture the ID at the time I create the item; that’s clearly the best option if you can do so.  In CloudTopia I take the ID of the newly created OG item and I save it to SQL Azure so I can use it later when creating new discussion items for the OG based on tweets I found for the event.  To extract the ID I built another class to serialize the JSON into and it looks like this:

 

 

Using it is quite simple:

//create the OGO

string response = MakePostRequest(postData, graphPostUrl, yammerAccessToken, “application/json”);

 

if (!string.IsNullOrEmpty(response))

{

      YammerGraphObjectItem gi = JsonConvert.DeserializeObject<YammerGraphObjectItem>(response);

}

Finally, remember that you always need the OG ID when reading or writing discussion items for it.  Also, the Url to use to read OG discussion items is currently undocumented “officially”, but I have covered all of this in greater detail in part 2 of my series on working with Yammer Open Graph from .NET (http://blogs.technet.com/b/speschka/archive/2014/05/29/using-yammer-open-graph-in-net-part-2.aspx).

Now let’s bring all the discussion of this back around to a concrete example – what we did in the CloudTopia app.  Remember that we create a new OG item when a new o365 site is created, and we use the Url for the new o365 site as the key for the OG item.  Here’s what that code looks like in CloudTopia:

//NOTE:  WILL EXPLAIN MORE OF THIS CODE

//IN A SUBSEQUENT PART OF THIS SERIES

//create a new site

string newUrl = AddNewSite(se.accessToken, se.hostUrl, se.eventName, se.eventDate);

 

//if it works, plug in the new site url

if (!string.IsNullOrEmpty(newUrl))

{

//create a new GraphObject item

YammerGraphObjectItem gi = CreateOpenGraphItem(newUrl, se.eventName, se.eventDate, se.twitterTags);

 

The CreateOpenGrahpItem method looks like this:

YammerGraphObjectItem gi = null;

 

YammerGraphObject go = new YammerGraphObject();

go.Activity.Action = “create”;

go.Activity.Actor = new YammerActor(“Steve Peschka”, “speschka@yammo.onmicrosoft.com”);

go.Activity.Message = “This is the discussion page for the ” + eventName + ” event on ” + eventDate;

 

go.Activity.Users.Add(new YammerActor(“Anne Wallace”, “annew@yammo.onmicrosoft.com”));

go.Activity.Users.Add(new YammerActor(“Garth Fort”, “garthf@yammo.onmicrosoft.com”));

 

YammerGraphObjectInstance jo = new YammerGraphObjectInstance();

jo.Url = Url;

jo.Title = eventName; 

jo.Description = “This is the discussion page for the ” + eventName + ” event on ” + eventDate; 

jo.Image = “https://socialevents.azurewebsites.net/images/eventplanning.png&#8221;;

jo.Type = “document”;

 

go.Activity.Object = jo;

 

string postData = go.ToString();

 

string response = MakePostRequest(postData, graphPostUrl, yammerAccessToken, “application/json”);

 

//serialize the results into an object with the OG ID

if (!string.IsNullOrEmpty(response))

{

gi = JsonConvert.DeserializeObject<YammerGraphObjectItem>(response);

string newMsg = “Welcome to the Yammer discussion for the ” + eventName +

” event, happening on ” + eventDate + “.  We’ll also be tracking external ” +

“discussions about this event on Twitter by using the tags ” + twitterTags + “.”;

CreateOpenGraphPost(gi.object_id, newMsg);

}

One quick note about the code above.  You may notice that I made the OG object Type a “document”.  You can find the complete list of Types that are supported at http://developer.yammer.com/opengraph/#og-schema.  There isn’t a type for “web site” so I just used document, but you can obviously choose one that works for you.  Finally, assuming our OG was created successfully then we serialize the return JSON to get the YammerGraphObjectItem so we have the object_id we need to update it later on.  Then we create the first discussion post to the OG item in the CreateOpenGraphPost method.  It is extraordinarily simple, just the way I like it:

string msg = “body=” + Message + “&attached_objects[]=open_graph_object:” +

ID + “&skip_body_notifications=true”;

 

//try adding the message

string response = MakePostRequest(msg, messageUrl + “.json”, yammerAccessToken);

That’s it!  We’ve talked about the semantics of working with Yammer Open Graph items, and we’ve looked at the specific implementation we used in the CloudTopia application.  In the next post in this series we’ll look at adding a Web API component to your standard out of the box SharePoint App project.  This can be extremely simple, I regularly use them in my SharePoint Apps now so I hope you’ll tune in and read more about it.

CloudTopia: Connecting o365, Apps, Azure and Cortana – Part 3

In Part 2 of this series we looked at some of the details of working with Yammer Open Graph items in the CloudTopia app.  In Part 3 we’re going to talk about adding and using Web API 2.1 functionality to a standard out of the box SharePoint App, and then look at what we do with that in CloudTopia.

Here’s some quick links to the whole series:

  1. Intro
  2. Open Graph
  3. Web API
  4. Azure Integration
  5. Office 365 Integration
  6. Cortana, Speech and Windows Phone Integration

 

In CloudTopia I use a Web API REST endpoint for my implementation of virtually everything.  It allows me to create an end user experience free of postbacks, but still access all the functionality of CSOM as well as .NET.  I can use JavaScript and jQuery to manage the front end interface because the code in my Web API controller is doing all the work.  In addition to that, by adding a REST endpoint to my application I open it up to be consumed and/or provide services to other applications.  A good example of this is the Twitter integration.  That process is kicked off by an Azure Scheduler job that makes a GET request on a REST endpoint I set up.  That’s a pretty key use case for Web API in your SharePoint Apps – without it, all of your application functionality is wrapped up in the fairly narrow confines of some browser based application.  By adding a Web API endpoint on top of it, now I can integrate that same functionality into many other applications across my organization or even, as demonstrated with the Azure Scheduler, outside my organization if I wish.

Now, adding the plumbing to support Web API 2.1 is not necessarily easy to find, so let me give you the steps here:

  1. Add the following two NuGet packages to your web application project:  Microsoft ASP.NET Web API 2.2 Core Libraries and Microsoft ASP.NET Web API 2.2 Web Host (NOTE:  the “2.2” refers the current version at the time of this blog post; you may find a more current version).
  2. Add a new class file to the root of your project and call it WebApiConfig.cs.
  3. Add the following code to your WebApiConfig.cs file (NOTE: You can add a different default route for your REST endpoints if you wish, I’m just following common convention here):

 

    public class WebApiConfig

    {

        public static void Register(HttpConfiguration config)

        {

            config.MapHttpAttributeRoutes();

 

            config.Routes.MapHttpRoute(“API Default”, “api/{controller}/{id}”,

                new { id = RouteParameter.Optional });

        }

    }

 

  1. Add this code to Application_Start in Global.asax:

protected void Application_Start(object sender, EventArgs e)

{

//for WebApi support

GlobalConfiguration.Configure(WebApiConfig.Register);

}

UPDATE 12/29/2014:  IMPORTANT!  Make sure you call the GlobalConfiguration.Configure method BEFORE you do any other configuration calls in the Start event.  For example, if you are adding this to a full blown MVC app then you will already have this in you Start code:  RouteConfig.RegisterRoutes(RouteTable.Routes);.  If you add the call for WebApiConfig.Register AFTER RouteConfig.RegisterRoutes then your Web API routes will not be found and your controller actions not hit.

Once you’ve done the configuration above, you can add a new Web API Controller Class v2 to your project and start creating your REST endpoints.  Before you get started writing code here a few tips:

  • Make sure the name of your class contains the word “Controller” (i.e. EventsController.cs); otherwise it won’t be found.  That one can drive you a little crazy if you don’t write much Web API and so aren’t familiar with this little nuance.
  • To create an overload inside your controller add a Route attribute and optionally an HTTP verb; that is how I added multiple POST endpoints to my single Web API controller class.  For example I do a POST request to api/events/currentevents to get the list of current events, a POST to api/events to create a new social event, etc.  Here’s an example of what the currentevents route looks like:

 

[Route(“api/events/currentevents”)]

[HttpPost]

public List<SocialEvent> Get([FromBody]string value)

{

//code goes here

}

  • Make sure your Route attribute uses the same path as a route you defined in WebApiConfig.cs.  For example, if your route definition uses “api/{controller}/{id}, your Route attribute should also start with “api/”.

 Once you have your controller added and configured as described above, calling it from jQuery is quite simple.  Here’s an abbreviated look at the jQuery in my SharePoint App that gets the list of events: 

//some SharePoint vars I get and will explain next

formData = JSON.stringify(formData);

 

$.post(“/api/events/currentevents”, { ”: formData })

.success(function (data)

{

       //code goes here

}).fail(function (errMsg)

{

alert(“Sorry, there was a problem and we couldn’t get your events: ” +

errMsg.responseText);

});

 

One of the biggest challenges when using a REST endpoint as part of your SharePoint App is getting an access token to work with (assuming you are doing more than just app only calls).  I looked three different options for this when I was writing CloudTopia:

  1. Persist token info to storage, like we recommend for the CAM “permissions on the fly” scenario
  2. Persist token info to ViewState
  3. Write out tokens to hidden fields on page

Of these options #1 is the safest and most enterprise-worthy of the bunch.  It also takes the most time and investment to do it right.  Because of the limited time I had to build the CloudTopia application I did not take this approach.  If I were though, I would consider having a method in my REST endpoint called something like RegisterRequest.  I could imagine calling a method like that and passing in a SharePointContextToken as I described in this post here:  http://blogs.technet.com/b/speschka/archive/2013/07/30/security-in-sharepoint-apps-part-4.aspx.  With the SharePointContextToken I have a) a guaranteed unique cache key for it and b) a refresh token that I can use to obtain an access token.  So when my RegisterRequest method was called I could then store that somewhere, like SQL Azure or whatever.  Then I could require that any calls into my REST endpoints provide the cache key, and all I have to do is look up what I’ve stored and if there’s something there, use the refresh token to get an access token and go to work.  This is just one idea I had, you may have others of your own.

Given my limited time, I chose to try both option 2 and 3.  Option 2 was really something I just wanted to play with to see if there would be any issues in using it.  So in that case I took the access token I got and wrote it to ViewState, and then I used it for one button in the page, which cleans the app up (i.e. deletes the hidden list I use to track new events).  I’m happy to report that it worked fine so if you want to go that way and you are doing your code in post back events it should work fine.  That’s really the key – you switch to a post back model if you want to use values from ViewState.

Primarily what I did was option 3.  I wrote both the SharePoint site Url as well as the access token to the page to some hidden fields, and then I pass those variables in to my REST calls.  It all travels over SSL, the access token has a limited lifetime, etc. so it was a reasonable choice given what I had to work with. 

The end-to-end implementation of if then went something like this:

 

  • I had client side script that looks like this:

//get the hiddens

var hostUrl = $(“#hdnHostWeb”).val();

var accessToken = $(“#hdnAccessToken”).val();

 

//create the JSON string to post

var formData = “{hostUrl:” + hostUrl + “,” +

“accessToken:” + accessToken + “}”;

 

//make it POST ready

formData = JSON.stringify(formData);

 

//call my REST endpoint

$.post(“/api/events/currentevents”, { ”: formData })

 

  • In my REST endpoint I parsed out the JSON that was sent in like this (NOTE:  ParseJson is just a custom method I wrote for my SocialEvent class):

SocialEvent se = SocialEvent.ParseJson(value);

 

  • The SocialEvent class has a hostUrl and accessToken property, so I just used them when calling methods that used CSOM to work with SharePoint:

List<SocialEvent> results = GetEventListItems(se.hostUrl, se.accessToken);

 

  • In my code that uses CSOM I created the ClientContext using the hostUrl and accessToken like this:

using (ClientContext ctx = TokenHelper.GetClientContextWithAccessToken(hostUrl, accessToken))

 

There you have it – that’s a roadmap for how to add Web API 2.x support to your SharePoint Apps.  I’m becoming a pretty big fan of this approach because it provides so much flexibility for making a client experience exactly like you want it, plus it opens up the capability to integrate all of your application functionality across other applications.  Now that we’ve wrapped up this piece of plumbing, in Part 4 of this series we’ll take a look at the Azure integration points.

CloudTopia: Connecting o365, Apps, Azure and Cortana – Part 4

In Part 3 of this series we looked at the plumbing required to add support for Web API 2.x to your SharePoint Apps, as well as some of the integration needed to have it work with SharePoint and CSOM.  In Part 4 we’re going to look at the integration with various Azure services in the CloudTopia app.

Here’s some quick links to the whole series:

  1. Intro
  2. Open Graph
  3. Web API
  4. Azure Integration
  5. Office 365 Integration
  6. Cortana, Speech and Windows Phone Integration


To begin with I started the CloudTopia project like any other SharePoint App – I cracked open Visual Studio 2013 Update 2, and then I created a new SharePoint App.  I made it a provider-hosted app, and it created two projects for me – one with the application manifest needed for app registration and the other a web site project.  Deployment was straightforward but I’ll cover that in the next post; I just wanted to set the stage for how we get things started.  Now let’s look at how we took this application and integrated with a variety of Azure services.

Azure Web Sites

CloudTopia is deployed to an Azure web site.  You get 10 free web sites with an Azure subscription so I used one of those to deploy my app.  The process for doing so is quite simple of course – you go to the Azure Management Portal and click the Web Sites link in the navigation and Add a new one.  There is some subtlety to adding a new one for this project, but I’ll cover that in the SQL Azure section (it will make sense why when we get there).  Once my web site is created I just downloaded the publishing profile from the Azure management portal page for the web site, and then in Visual Studio I chose the option to publish my site.  When the publish wizard ran I gave it the location of the publishing profile file that I had downloaded and away we went.  Whenever I made changes I just republished the site and about 30 seconds later my latest code was up and running in the Azure web site.  Good stuff.

One other thing worth noting here is debugging.  Debugging is possible for web sites hosted in Windows Azure, you just do it a little differently than if you were running your web site locally.  I’ve previously posted about this process for debugging your SharePoint Apps that are hosted in an Azure web site – you can find that post here:  http://blogs.technet.com/b/speschka/archive/2013/11/25/debugging-sharepoint-apps-that-are-hosted-in-windows-azure-web-sites.aspx.

SQL Azure

I used SQL Azure in the CloudTopia app primarily to simplify the process of the daily task to go out and find matching tweets for social events.  As I described earlier, I was able to take advantage of a free 20MB SQL Azure database that I get with my Azure subscription.  You actually create and/or connect it to your application at the time you create your Azure web site – that’s why we’re covering site creation here in the SQL Azure topic.  To connect these up you want to first do a custom create for your new Azure web site:

When you do that you’ll have the option of selecting a database.  Click the drop down and if you have an MSDN subscription you should see an option to create a free 20MB database (assuming you have not created it already; if you have then you can just select the instance you already created):

Now I’m going to take what might seem like a brief detour but I’ll bring it back around when I’m done.  One of the features of the CloudTopia app is that it will take a set of Twitter tags that have been defined for an event and go do a search to find tweets in the previous 24 hours that have used them.  Every tweet that is found is added to the discussion on the Yammer Open Graph item that’s associated with the event.   That’s how we get this nice integrated discussion in our events:

We’re just running this code once a day, so the process to gather these matching tweets is kicked off when an Azure Scheduler job makes a GET request to our REST endpoint that runs this code.  So why am I sharing this information here?  Because way back in Part 1 of this series I mentioned that I was using SQL Azure to store some of the CloudTopia data versus just keeping everything in a SharePoint list.  Understanding this piece of functionality should help explain why SQL Azure.

As I also mentioned previously in this series, you always need the ID of a Yammer Open Graph item in order to read or write to the discussion that’s associated with it.  Also, as I described above, this process kicks off once a day from an Azure Scheduler job.  The distinction in this scenario is that there is no human present.  That means that I don’t have a user context in order to make a call back into SharePoint.  So if I wanted to store ALL of the CloudTopia metadata in a SharePoint list, I would need to configure my app to use an app only request.  While I could certainly do that, it requires an elevated level of permissions versus a simple user-driven request and that was something I did not want to do.  That’s how I landed on using SQL Azure for this purpose.  Not only is it free for my application, I’m able to use it without any user context at all – I just use a connection string with a set of credentials for a SQL Azure user that has rights to my CloudTopia database.  It’s also significantly easier for most developers to create SQL queries then the “sometimes mystical, sometimes magical, sometimes maddening” world of SharePoint CAML queries.  SQL Azure makes it easy to retrieve the information needed for the CloudTopia app, and also doesn’t require a high level of permission from the application itself.  Score one for SQL Azure!

Azure Scheduler

The final Azure service I used on CloudTopia is the Azure Scheduler service.  This is a pretty straightforward service to use and configure so I’m not going to spend a ton of time talking about it.  There are always several options when you are looking to schedule tasks; for CloudTopia though, as the name implies, I wanted it to be 100% hosted in the cloud – and cheap.  The Azure Scheduler service is a great solution for these requirements.  You get a set number of job iterations for free, and since I’m only making one job run a day – to get the tweets from the last 24 hours – this fits the bill perfectly.  When you create your job you can choose an HTTP or HTTPS endpoint to invoke, and you can define whether you want to do a GET, POST, PUT or DELETE.  For POST and PUT you can optionally provide a Body to send along with the request; for all of them you can add one to many custom Http headers to send as well.  After you configure your job endpoint you set up your schedule – a one time run to something that reoccurs on a regular basis.  That is basically it, but here’s some pictures to show you the UI that was used to create the CloudTopia Scheduler job:

When the Scheduler job runs here’s the REST endpoint that it invokes:

public async Task<HttpResponseMessage> Get()

{

HttpResponseMessage result = Request.CreateResponse(HttpStatusCode.OK);

 

try

{

await Task.Run( () => UpdateYammerWithTwitterContent());

}

catch (Exception ex)

{

result = Request.CreateErrorResponse(

HttpStatusCode.BadRequest, ex.Message, ex);

}

 

return result;

}

So I just go off and run my code to update the Yammer Open Graph item with any tweets from the last 24 hours.  If it works I return an HTTP status code 200, and if it fails I decided to return an HTTP status code 400.  Yeah, it’s not really a bad request, but I’ve always wanted to return that to someone else for a change.

Here’s an abbreviated version of the code to actually go out and get the tweets and write them to Yammer.  First I connect to SQL Azure and get the list of events and their associated Twitter tags and Open Graph IDs:

using (SqlConnection cn = new SqlConnection(conStr))

{

SqlCommand cm = new SqlCommand(“getAllEvents”, cn);

cm.CommandType = CommandType.StoredProcedure;

SqlDataAdapter da = new SqlDataAdapter(cm);

 

DataSet ds = new DataSet();

da.Fill(ds);

 

With my dataset of events I enumerate through each one and go get the event tweets.  I start out by getting an access token for Twitter:

if (string.IsNullOrEmpty(TWT_ACCESS_TOKEN))

{

//create the authorization key

string appKey = Convert.ToBase64String(

System.Text.UTF8Encoding.UTF8.GetBytes(

(HttpUtility.UrlEncode(TWT_CONSUMER_KEY) + “:” +

HttpUtility.UrlEncode(TWT_CONSUMER_SECRET))));

 

//set the other data for our post

string contentType = “application/x-www-form-urlencoded;charset=UTF-8”;

string postData = “grant_type=client_credentials”;

 

//need to get the oauth token first

response = MakePostRequest(postData, TWT_OAUTH_URL, null,

contentType, appKey);

 

//serialize it into our class

TwitterAccessToken accessToken =

TwitterAccessToken.GetInstanceFromJson(response);

 

//plug the value into our local AccessToken variable

TWT_ACCESS_TOKEN = accessToken.AccessToken;

}

 

Now that I’m sure I have a Twitter access token I can go ahead and query twitter for the tags I’m interested in:

//now that we have our token we can go search for tweets

response = MakeGetRequest(TWT_SEARCH_URL +

HttpUtility.UrlEncode(query), TWT_ACCESS_TOKEN);

 

//plug the data back into our return value, which is just a

//custom class with a list of SearchResult so I can work

//with it easily from my code

results = SearchResults.GetInstanceFromJson(response);

 

//trim out any tweets older than one day, which is how frequently

//this task should get invoked

if (results.Results.Count > 0)

{

//retrieve items added in the last 24 hours

var newResults = from SearchResult oneResult in results.Results

where DateTime.Now.AddDays(-1) <

DateTime.Parse(oneResult.Published)

select oneResult;

 

results.Results = newResults.ToList<SearchResult>();

}

Once I get my search results back, I can add each one to the discussion on the Yammer Open Graph item:

foreach (SearchResult sr in queryResults.Results)

{

string newPost = “From Twitter: ” +

sr.User.FromUser + ” says – ” + sr.Title + “.  See the post and more at ” +

https://twitter.com/&#8221; + sr.User.FromUser + “.  Found on ” +

DateTime.Now.ToShortDateString();

 

CreateOpenGraphPost(objectGraphID, newPost);

}

You may notice that I’m calling the same CreateOpenGraphPost that I described earlier in this series – I used it previously to create the initial post for new Open Graph items.

That’s it for this post.  In Part 5 of the series we’ll look at all of the integration that was done with Office 365.  It was a lot, so stay tuned.

CloudTopia: Connecting o365, Apps, Azure and Cortana – Part 5

In Part 4 of this series we looked at the integration with various Azure services in the CloudTopia app.  In this part we are going to explore all of the integration that was done with Office 365 and how we did it. 

Here’s some quick links to the whole series:

  1. Intro
  2. Open Graph
  3. Web API
  4. Azure Integration
  5. Office 365 Integration
  6. Cortana, Speech and Windows Phone Integration


Let’s start by looking at all of the different integration that was done with o365 and then we’ll examine each one in more detail:

  • SharePoint App that runs in the Events host site
  • Create hidden SharePoint list to track all of the events
  • Create new sites for each new event
  • Remove the out of box Site Feed web part as each new event site is created
  • Add script editor web part with JS to render Yammer Open Graph item discussion
  • Create, read, update and delete entries from the hidden SharePoint list of events

 

SharePoint App

Of course the piece that drives CloudTopia is the SharePoint App.  Let’s look first at how everything got package, deployed and installed.  I started out by going to one of the existing o365 sites I had and just navigating to the /_layouts/15/appregnew.aspx page to create a new client ID and secret for my application.  I then updated my AppManifest.xml file with the client ID, and added the client ID and secret to the web.config file for my web project.

With that configuration data in hand, I deployed my web project to its Azure web site using the publishing profile, as I described in Part 4 of this series.  I then published my SharePoint App, which really just created a .app file for me.  I uploaded my .app file to the App Catalog for my o365 tenant and installed into my Events site.  My permissions in the app were for the full enchilada – Full rights in the Tenant.  This is because I need to be able to create new site collections in the tenant, add and remove web parts, etc.

 

Create Hidden List

When the app installed it fired a remote event receiver that created the list for tracking events, made it hidden, and removed it from the left Quick Nav bar in o365.  The complete steps and relevant code for doing that were described in my previous post here:  http://blogs.technet.com/b/speschka/archive/2014/05/07/create-a-list-in-the-host-web-when-your-sharepoint-app-is-installed-and-remove-it-from-the-recent-stuff-list.aspx.

 

Create New Sites

The code for creating new sites borrowed liberally from the Office365 Development Patterns and Practices project (OfficeDev PnP) at https://github.com/OfficeDev/PnP.  It’s part of the bigger process that I alluded to in Part 1 of this series that occurs when someone clicks a couple of buttons and says create me a new event site.  Under the covers that makes a request to the REST endpoint we created for this action and these four activities happen:

  • Create a new site
  • Get the Url and create a new Open Graph item
  • Record the Url, event name, event date, and TwitterTag data in the SharePoint list
  • Record the Yammer Open Graph ID, o365 Url, and Twitter tags data in SQL Azure

I’ve already covered creating the new Open Graph item in Part 2 of this series, so I’ll just focus on the other items in the list.  In terms of creating a new site, it’s probably better to just review the content from the OfficeDev PnP repo.  In CloudTopia the primary modification I made to their code was to check to ensure that the Url was available, and if not add an incrementally larger number to the end of it until I found a Url that is available.  For example if you had 10 “Crazy Sell-a-thon” events they couldn’t all have the same Url.  In addition to that, even after a site is deleted it’s not really deleted – not at first – so you need to check deleted sites for Url availability as well.  Here’s the code I used to ensure a uniquely available Url: 

//create the client context for the admin site

//the token is obtained in code not shown here,

//webUrl is a parameter to this method, and

//tenantAdminUri is from code in this method

//that is not shown here, but represents the admin

//site for the tenant, i.e.

// https://yourTenantName-admin.sharepoint.com

//uniqueUrl is just an integer initialized to 0

//baseUrl is initialized to webUrl

using (var adminContext =

TokenHelper.GetClientContextWithAccessToken(

tenantAdminUri.ToString(), token))

{

var tenant = new Tenant(adminContext);

 

//look to see if a site already exists at that Url; if it does then create it

bool siteExists = true;

 

while (siteExists)

{

try

{

//look for the site

Site s = tenant.GetSiteByUrl(webUrl);

adminContext.Load(s);

adminContext.ExecuteQuery();

 

//if it exists then update the webUrl and

//do the while loop again;

                     //if it doesn’t exist it will throw an exception                           

uniqueUrl += 1;

webUrl = baseUrl + uniqueUrl.ToString();

}

catch

{

try

{

//doesn’t exist, need to check deleted sites too

DeletedSiteProperties dsp =

                                  tenant.GetDeletedSitePropertiesByUrl(webUrl);

adminContext.Load(dsp);

adminContext.ExecuteQuery();

 

//if it exists then update the webUrl

//and do the while loop again                           

uniqueUrl += 1;

webUrl = baseUrl + uniqueUrl.ToString();

}

catch

{

//okay it REALLY doesn’t exist, so go ahead

//and grab this url and set the flag to

//exit the while loop

siteExists = false;

}

}

}

 

//now we can create the site using the webUrl

//follow OfficeDev PnP and use

//SiteCreationProperties here

 

Once the site is created I can go ahead and add the Url and other information to the hidden list in SharePoint so that it shows up in my list of events.  That code is pretty simple and looks like this:

using (ClientContext ctx = TokenHelper.GetClientContextWithAccessToken

(hostUrl, accessToken))

{

//get our event list

List eventsList = ctx.Web.Lists.GetByTitle(LIST_NAME);

 

//create the list item

ListItemCreationInformation ci = new ListItemCreationInformation();

ListItem newItem = eventsList.AddItem(ci);

 

newItem[“Title”] = eventName;

newItem[“SiteUrl”] = siteUrl;

newItem[“EventName”] = eventName;

newItem[“EventDate”] = eventDate;

newItem[“TwitterTags”] = twitterTags;

newItem[“ObjectGraphID”] = objectGraphID;

 

//update the list item

newItem.Update();

 

//add the item to the list

ctx.ExecuteQuery();

}

Now, finally, I’ll go ahead and add the data to SQL Azure.  There’s absolutely nothing new here, this is basically ADO.NET code from earlier this century…but, for completeness here you go:

//record the OG ID, OG URL, and TwitterTag data in SQL

using (SqlConnection cn = new SqlConnection(conStr))

{

cn.Open();

 

SqlCommand cm = new SqlCommand(“addEvent”, cn);

cm.CommandType = CommandType.StoredProcedure;

 

cm.Parameters.Add(new SqlParameter(“@ObjectGraphID”, Double.Parse(gi.object_id)));

cm.Parameters.Add(new SqlParameter(“@ObjectGraphUrl”, newUrl));

cm.Parameters.Add(new SqlParameter(“@TwitterTags”, se.twitterTags));

cm.Parameters.Add(new SqlParameter(“@EventName”, se.eventName));

cm.Parameters.Add(new SqlParameter(“@EventDate”, se.eventDate));

 

cm.ExecuteNonQuery();

 

cn.Close();

}

 

Remove Site Feed Web Part

The code to remove the Site Feed web part was really pretty much just pulled from the OfficeDev PnP project.  I’ll include an abbreviated version of it here so you get an idea of how it looks:

//create the client context

using (ClientContext ctx =

TokenHelper.GetClientContextWithAccessToken(SiteUrl, token))

{

ctx.Load(ctx.Web, w => w.RootFolder, w => w.RootFolder.WelcomePage,

w => w.ServerRelativeUrl);

ctx.ExecuteQuery();

 

Microsoft.SharePoint.Client.File webPage =

ctx.Web.GetFileByServerRelativeUrl(ctx.Web.ServerRelativeUrl +

ctx.Web.RootFolder.WelcomePage);

 

ctx.Load(webPage);

ctx.Load(webPage.ListItemAllFields);

ctx.ExecuteQuery();

 

string wikiField = (string)webPage.ListItemAllFields[“WikiField”];

 

LimitedWebPartManager wpm =

              webPage.GetLimitedWebPartManager(

Microsoft.SharePoint.Client.WebParts.PersonalizationScope.Shared);

 

//remove the OOB site feeds web part

WebPartDefinitionCollection allParts = wpm.WebParts;

ctx.Load(allParts);

ctx.ExecuteQuery();

 

for (int i = 0; i < allParts.Count; i++)

{

WebPart almostDeadPart = allParts[i].WebPart;

ctx.Load(almostDeadPart);

ctx.ExecuteQuery();

 

if (almostDeadPart.Title == “Site Feed”)

{

allParts[i].DeleteWebPart();

ctx.ExecuteQuery();

break;

}

}

Add Script Editor Web Part to Render Yammer Open Graph Discussion

This chunk of code was pretty nice and really speaks to the flexibility that you get with the Yammer JavaScript Embed library.  I actually created the code for it in a simple HTML page, and then copied into a script editor web part I added to a page in a SharePoint site.  Once I validated that it was all working there I exported the web part and copied everything out and plugged it into my CloudTopia code.  This code runs in the same abbreviated code block I showed above for removing the web part, so the steps are remove the Site Feed web part, then add the Script Editor web part and populate it with JavaScript that pulls from the Open Graph item discussion.  Here’s what that looks like (it got it’s ClientContext from the code shown above):

const string URL_REPLACE = “$URL$”;

 

string partTxt = @”<webParts>

  <webPart xmlns=””http://schemas.microsoft.com/WebPart/v3″”&gt;

    <metaData>

      <type name=””Microsoft.SharePoint.WebPartPages.ScriptEditorWebPart, Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”” />

      <importErrorMessage>Cannot import this Web Part.</importErrorMessage>

    </metaData>

    <data>

      <properties>

        <property name=””ExportMode”” type=””exportmode””>All</property>

        <property name=””HelpUrl”” type=””string”” />

        <property name=””Hidden”” type=””bool””>False</property>

        <property name=””Description”” type=””string””>Allows authors to insert HTML snippets or scripts.</property>

        <property name=””Content”” type=””string””>    &lt;script type=””text/javascript””

        src=””https://assets.yammer.com/assets/platform_embed.js””&gt;&lt;/script&gt;

 

    &lt;div id=””embedded-feed”” style=”height:400px;width:500px;”&gt;&lt;/div&gt;

 

    &lt;script&gt;

 

        yam.connect.embedFeed({

            container: “”#embedded-feed””,

            network: “”yammo.onmicrosoft.com””,

            feedType: “”open-graph””,

            objectProperties: {

                url: “”$URL$””

            }

        });

    &lt;/script&gt;

</property>

        <property name=””CatalogIconImageUrl”” type=””string”” />

        <property name=””Title”” type=””string””>Script Editor</property>

        <property name=””AllowHide”” type=””bool””>True</property>

        <property name=””AllowMinimize”” type=””bool””>True</property>

        <property name=””AllowZoneChange”” type=””bool””>True</property>

        <property name=””TitleUrl”” type=””string”” />

        <property name=””ChromeType”” type=””chrometype””>None</property>

        <property name=””AllowConnect”” type=””bool””>True</property>

        <property name=””Width”” type=””unit”” />

        <property name=””Height”” type=””unit”” />

        <property name=””HelpMode”” type=””helpmode””>Navigate</property>

        <property name=””AllowEdit”” type=””bool””>True</property>

        <property name=””TitleIconImageUrl”” type=””string”” />

        <property name=””Direction”” type=””direction””>NotSet</property>

        <property name=””AllowClose”” type=””bool””>True</property>

        <property name=””ChromeState”” type=””chromestate””>Normal</property>

      </properties>

    </data>

  </webPart>

</webParts>”;

 

partTxt = partTxt.Replace(URL_REPLACE, SiteUrl);

 

Microsoft.SharePoint.Client.File webPage =

ctx.Web.GetFileByServerRelativeUrl(ctx.Web.ServerRelativeUrl +

ctx.Web.RootFolder.WelcomePage);

 

ctx.Load(webPage);

ctx.Load(webPage.ListItemAllFields);

                   

string wikiField = (string)webPage.ListItemAllFields[“WikiField”];

 

LimitedWebPartManager wpm =

webPage.GetLimitedWebPartManager(

Microsoft.SharePoint.Client.WebParts.PersonalizationScope.Shared);

 

//add the new part

WebPartDefinition wpDef = wpm.ImportWebPart(partTxt);

WebPartDefinition wp = wpm.AddWebPart(wpDef.WebPart, “wpz”, 1);

 

ctx.Load(wp);

ctx.ExecuteQuery();

 

//run some other clean up code from OfficeDev PnP to get the

//web part displaying correctly; see their project for those

//details

 

Read, Update and Delete Items from the Hidden SharePoint List

The code in the other REST controller methods are really so generic and mirror the code I’ve already shown so closely that there’s not really a point to show them as well.  If you’re really interested then please go to the repo on GitHub for this project and review it up there.

Okay, we’ve now wrapped up all of the code in the CloudTopia application.  If you’ve read everything so far, congratulations – good job!  As it turns out though there is one other piece of application integration I did for CloudTopia, but it’s not in the cloud – it’s on a phone.  In the next and final part of this series we’ll look at writing a Windows Phone app that uses speech, voice recognition, and of course, Cortana!

 

CloudTopia: Connecting o365, Apps, Azure and Cortana – Part 6

In Part 5 of this series we looked at all of the different ways in which CloudTopia is integrated with Office 365 through our custom Web API REST controller.  We’re going to wrap up this series with a little sizzle, and look at the custom Windows Phone app we wrote to work with CloudTopia using voice recognition, speech, and Cortana.

Here’s some quick links to the whole series:

  1. Intro
  2. Open Graph
  3. Web API
  4. Azure Integration
  5. Office 365 Integration
  6. Cortana, Speech and Windows Phone Integration

 

You heard it here first – a picture’s worth a thousand words.  J  To start with I’d recommend watching the demo video of the Cortana Social Events app that you will find here:  http://1drv.ms/1kPjjKf. 

Okay, hopefully you watched the video and now your appetite has been whetted a little.  So let’s talk Windows Phone 8.1, using voice and integrating with Cortana.  The implementation itself consists of two parts:

  • Your custom app that does voice recognition and optionally speech.
  • Integration with Cortana

All of the coding and logic happens in your app.  Integration with Cortana actually just happens via an Xml file, that is both surprisingly powerful and easy.  In fact that’s my theme for this entire post – the folks that have been doing the work on speech in Windows Phone need to take a friggin’ bow: their stuff works pretty dang good and the implementation is much simpler than I expected.

So how do we do it?  Well first and foremost I would recommend that you download the MSDN sample app that demonstrates all this functionality from http://aka.ms/v4o3f0.  When I first started looking around I didn’t find any kind of handy tutorial to get me on my way so I relied heavily on this app.  With that as a resource for you to lean on, here are the basic steps to integrating voice recognition, speech and Cortana into your Windows Phone apps:

  • Create a new Windows Phone Silverlight App
  • Add a SpeechRecognizer instance in code to listen; add a SpeechSynthesizer to speak
  • Initialize the Recognizer
  • Tell the Recognizer to start listening
  • When the Recognizer completed action fires, take the recognized words and do “something”

 

Create New Windows Phone Silverlight App

This should be pretty obvious but just want to make sure I call this out.  As of the time I am writing this post all of this voice goodness is not integrated with Windows Universal Apps, so when you create a new project create a Windows Phone Silverlight app project type as shown here:

 

 

Add Speech Recognizer and Synthesizer

Start by adding instances of SpeechRecognizer, SpeechSynthesizer, plus a few other classes that are used for maintaining state to the “main” page of your application (typically mainpage.xaml).  This is the page where you’re going to do your voice recognition, and eventually connect to Cortana.  The instances you add should look like this:

// State maintenance of the Speech Recognizer

private SpeechRecognizer Recognizer;

private AsyncOperationCompletedHandler<SpeechRecognitionResult> recoCompletedAction;

private IAsyncOperation<SpeechRecognitionResult> CurrentRecognizerOperation;

 

// State maintenance of the Speech Synthesizer

private SpeechSynthesizer Synthesizer;

private IAsyncAction CurrentSynthesizerAction;

 

In order to get these all to resolve you may need to add some using statements for Windows.Phone.Speech.Recognition, Windows.Phone.Speech.Synthesis, and Windows.Phone.Speech.VoiceCommands.  Okay, step 2 complete.

Initialize the Recognizer

The next step is to initialize the Recognizer – this is what’s going to do the speech recognition for us.  To start that process we’ll create a new instance of the Recognizer like so:

this.Recognizer = new SpeechRecognizer();

 

Once the Recognizer has been created we need to add grammar sets.  The grammar sets are the collection of words the Recognizer is going to use to recognize what it hears.  While this could potentially be a daunting task, it is – surprise – pretty easy in most cases.  Here’s what I did to populate my grammar set:

this.Recognizer.Grammars.AddGrammarFromPredefinedType(“search”,       SpeechPredefinedGrammar.WebSearch);

await this.Recognizer.PreloadGrammarsAsync();

 

Boom – there you go – done adding my grammar set.  There are a couple of grammar sets that ship out of the box, and so far I have found the WebSearch set to do everything I need.  You can also create your own grammar sets if you like; dev.windowsphone.com has documentation on how to do that.  The last thing you want to do for the initialization is to define what to do when speech is actually recognized.  To do that we’re going to add a handler for when speech recognition is completed.  Remember above where I made this declaration – private AsyncOperationCompletedHandler<SpeechRecognitionResult> recoCompletedAction?  We’re going to use that recoCompletedAction variable now, like this:

recoCompletedAction = new

AsyncOperationCompletedHandler<SpeechRecognitionResult>

((operation, asyncStatus) =>

{

Dispatcher.BeginInvoke(() =>

{

this.CurrentRecognizerOperation = null;

switch (asyncStatus)

{

case AsyncStatus.Completed:

SpeechRecognitionResult result =

operation.GetResults();

 

//use the recognized text here     

LaunchSearch(result.Text);

break;

case AsyncStatus.Error:

//respond to error; often user

//hasn’t accepted privacy policy

break;

}

});

});

So what we’re saying here is that when the speech recognition event happens, if we completed recognition successfully then we’re going to extract the collection of recognized words by getting a SpeechRecognitionResult and looking at its Text property.  If there was an error, then we’ll need to do something else.  Where I found the error condition happening was after building my app, when I tried the app out on the emulator.  You always have to accept the privacy policy before it will do voice recognition and it’s an easy thing to forget when you start up a new emulator session.  Other than that though, you are set – that’s all you need to do to initialize the Recognizer before you start listening for speech.

Use the Recognizer to Start Listening

Now that the Recognizer is configured you can start listening.  It’s pretty easy – just start listening asynchronously and configure the listening completed handler you want to use, which will be the recoCompletedAction variable I was describing above:

this.CurrentRecognizerOperation = this.Recognizer.RecognizeAsync();

this.CurrentRecognizerOperation.Completed = recoCompletedAction;

As far as speech recognition goes, that’s it – you’re done!  Pretty easy, huh?  It’s amazingly simple now to add voice recognition to your apps so I recommend you go out and give it a whirl.  For the CloudTopia application, when speech recognition was completed you’ll notice that I invoked a method called LaunchSearch.  In that method I create a query term that I want to use for CloudTopia and then I send it off to another REST endpoint I created in my SharePoint App project.  See, those Web API REST endpoints are very valuable! 

 

The way I decided to implement the search capability was to make the spoken query search for Twitter tags that have been configured for an event.  So after recognizing some spoken words I get a list of all the words that I think are not noise words.  To do that, I just keep a list of words that I consider to be noise and I load those into a List<string> when my Windows Phone app starts up.  It includes words like “event”, “Cortana”, “find”, etc.  After I eliminate all the noise words I see what’s left and assuming there is still one or more words there, I concatenate them together.  They get concatenated because we’re going to be searching for hashtags, and hashtags are always going to be a single word.  That makes it easy to say things like “yammer of july” and then have it concatenated into a the hashtag “yammerofjuly” (if you read about this series on Twitter you’ll know exactly what I’m talking about).

If I have a hashtag to search for I use the SpeechSynthesizer to speak back to the person what I’m going to search for (I’ll cover that in more detail in just a bit) and then I go send off my query to my REST endpoint.  The chunk of code to do all of that is here:

//we really only want to look for a single phrase, which could be several words

//but in a tag are represented as a single word

//in order to do that we’ll take all the words we’re given,

//extract out the noise words, and then create a single word

//that’s a concatenation of all the remaining non-words into a single phrase

string searchTerm = GetSearchTerm(queryTerms);

 

if (!string.IsNullOrEmpty(searchTerm))

{

//create the template for speaking back to us

string htmlEncodedQuery = HttpUtility.HtmlEncode(searchTerm);

StartSpeakingSsml(String.Format(

AppResources.SpokenSearchShortTemplate, htmlEncodedQuery));

 

//update UI

Dispatcher.BeginInvoke(() =>

{

WaitTxt.Text = “Searching for \”” + searchTerm + “\” events…”;

});

 

//execute the query

QueryEvents(searchTerm);

}

else

{

WaitTxt.Text = “Sorry, there were only noise words in your search request”;

}

 

That code sets us up to send the query to the REST endpoint, and here’s where we actually do the query; one of the things it hopefully demonstrates is just how easy it is to use a REST endpoint.  Within the REST controller it takes the search term that was passed in and looks for a match against any of the Twitter tags in SQL Azure.  It uses a LIKE comparison so you don’t need to have the exact tag to find a match.  If it does find one or more events then it also uses the Yammer Search REST endpoint to query Yammer for matches as well.  It then sends back both the events and hits from Yammer for us to display in our Windows Phone app.  This is coolness…querying the Yammer cloud service by talking to my phone.  Love it.

string searchUrl =

https://socialevents.azurewebsites.net/api/events/search?tagName=&#8221; + searchTerms;

 

HttpClient hc = new HttpClient();

string data = await hc.GetStringAsync(searchUrl);

 

//if we got some data back then try and load it into our set of search results of

//social events and Yammer messages

if (!string.IsNullOrEmpty(data))

{

CortanaSearchResult csr = CortanaSearchResult.GetInstanceFromJson(data);

 

Dispatcher.BeginInvoke(() =>

{

//if we found some events plug them

//into our UI by databinding to the

//lists in the Panorama control

if ((csr != null) && ((csr.Events.Count > 0) ||

(csr.YammerMessages.Count > 0)))

{

                        EventsLst.DataContext = csr.Events;

                        YammerLst.DataContext = csr.YammerMessages;

}

else

{

//update the UI to show that there were no search results found

WaitTxt.Text = “Sorry, I couldn’t find any results for \”” +

searchTerms + “\””;

 

}

});

}

 

Add Speech to Your App

Adding speech to your application is even easier than adding voice recognition.  The first thing you’re going to do is create a new instance of the SpeechSynthesizer, like this:

this.Synthesizer = new SpeechSynthesizer();

 

That’s all you do to set it up.  To actually have my app say something I created a simple method for it that looks like this:

private void StartSpeakingSsml(string ssmlToSpeak)

{

//Begin speaking using our synthesizer, wiring the

//completion event to stop tracking the action

//when it finishes.

           

this.CurrentSynthesizerAction = this.Synthesizer.SpeakSsmlAsync(ssmlToSpeak);

this.CurrentSynthesizerAction.Completed = new AsyncActionCompletedHandler(

(operation, asyncStatus) =>

{

Dispatcher.BeginInvoke(() =>

{ this.CurrentSynthesizerAction = null; });

});

}

There are different methods and overloads that you can use to have the phone speak.  I chose to use SpeakSsml because of the control it gives you over the text that’s spoken.  SSML stands for Speech Synthesis Markup Language, and it’s really just Xml that uses a schema to control things like pitch and rate at which words are spoken.  Here’s an example of an Xml template I use to say back to the user what it is we are searching for:

<speak version=’1.0′ xmlns=’http://www.w3.org/2001/10/synthesis&#8217; xml:lang=”en-US”> <prosody pitch=’+35%’ rate=’-10%’> Searching </prosody> <prosody pitch=’-15%’> for </prosody> {0} events </speak>

The way I use that is to create the string that it will say like this:

StartSpeakingSsml(String.Format(

AppResources.SpokenSearchShortTemplate, searchTerm));

 

So if the search term is “yammerofjuly” what is spoken back to the user is “Searching for yammerofjuly”, but it’s said with different pitch and rate around the words “Searching” and “for”.  VERY cool stuff.

 

Cortana Integration

Finally the last thing I did was “integrate” my app with Cortana.  What does that mean exactly?  Well I wanted someone to be able to use Cortana in Windows Phone 8.1 to execute a query using my Social Events app.  The way you do that is you create a VoiceCommandDefinition file, which is just another Xml file. In the file you can configure things like the name of your app, examples that can be shown users who aren’t sure how to use voice recognition with your app, the things to listen for with your app, etc.

The first and most important thing I defined in my file is the CommandPrefix; this is really just the name by which my app will be known.  The Xml looks like this:

<!– The CommandPrefix provides an alternative to your full app name for invocation –>

        <CommandPrefix> Social Events </CommandPrefix>

What this means now is when someone says something to Cortana that starts with “Social Events”, Cortana knows that it needs to use my app to get the results.  For example, if I say “Social Events find yammer of july events”, Cortana will figure out that “Social Events” means my app, and it’s going to let my app know that the recognized words were “find yammer of july events”.  The way it lets my app know is that it’s going to launch my app and redirect me to a page in the application (you can configure in the VoiceCommandDefinition file the page in your app where it redirects to as well; mine gets sent to mainpage.xaml).  In the code behind for your page then you can override the OnNavigatedTo event and look at the query string.

If Cortana sent the user to your app, it will have used a query string variable that you have also defined in your Xml file, in the ListenFor element:

<!– ListenFor elements provide ways to say the command as well as [optional] words –>

           <ListenFor> Search [for] {dictatedSearchTerms} </ListenFor>

Note the “dictatedSearchTerms” in the curly brackets.  There are actually multiple uses for it, but for now I’ll just explain the use case I have.  When Cortana redirects a request to my application it will use “dictatedSearchTerms” as the query string variable that contains the recognized words.  That means in my override of OnNavigatedTo I can look to see if the QueryString collection contains the key “dictatedSearchTerms”.  If it does, I’ll extract the value and then just call my same LaunchSearch method that I described earlier, and pass in the value from the query string.

One of the other really useful aspects of the Xml file is the ability to create examples of how your app can be used with Cortana.  When you say “what can I say” to Cortana, it will display a list of all the apps that it knows about that can use voice recognition; if you’ve registered your VoiceCommandDefinition file (more on how to do that in a moment), then your app will be display along with an example of what can be said with your app.   The example that it displays is what you configured in your VoiceCommandDefinition file, like this:

<!– The CommandSet Example appears in the global help alongside your app name –>

        <Example> find blazer party events </Example>

 

So in this case it will show the small icon for my application, it display my application name in big bold letters, and then it will say find blazer party events underneath it.  Again, very cool!

If you click the icon of the application it brings it all together and will display something like this:

 

The last step now is to install your definition file.  You can install it as many times as you want, and it just overwrites whatever copy it had previously.  Because of that, I just install my definition file every time my Windows Phone app starts up.  It’s one simple line of code:

await VoiceCommandService.InstallCommandSetsFromFileAsync(“ms-appx:///MyVoiceDefinitionFile.xml”);

 

Well there you go – that’s it.  That is a wrap on this six part series on the CloudTopia app.  Don’t forget to go grab the code from GitHub.Com (the exact location is included in Part 1 of this series).  I hope you can use this to help navigate your way through some of the very interesting connections you can make in a pure cloud-hosted world with SharePoint Apps and many other cloud services.  When you get the chance, you can also add some dazzle to your apps with some voice recognition and Cortana integration.  Finally…time to stop writing blog posts and go back to building apps.