Stay Informed with New Malware Monitoring from Office365Mon.Com

It seems like organizations of all types and sizes are under digital attack these days.  Using email to transmit malware and then compromise an organization is a common way in which these kinds of attacks strike.  Today Office365Mon is launching a new service to help keep you in the know of when and where these attacks are directed at your organization.  In conjunction with the Threat Intelligence features of Office 365, we have a new feature we call Threat Intelligence Monitoring.

The Threat Intelligence features in Office 365 are included for those users that have an E5 license.  The Office 365 E5 license includes numerous additional features beyond the basic email and SharePoint, and Threat Intelligence is one of them.  The Threat Intelligence feature in Office 365 is a collection of insights used in analyzing your tenant to help you find and eliminate threats, proactively.  The Threat Intelligence Monitoring feature in Office365Mon builds on that in some important ways.  For example, you can:

  • Get notified the first time a new malware is sent to your organization. Know when a new type of malware has been targeted at your company so you can make sure you have the tools and plans in place to defend yourself.
  • Get notified when you get more than a certain number of malware within a given time period. Set thresholds for malware volume so you know if you are being targeted for broader malware attacks.
  • Get notified when any user gets more than a certain number of malware in any given day. Be in the know and in control if any of your users are being singled out and specifically targeted with malware attacks so you quarantine and limit the potential damage.

Configuring these options, like all features in Office365Mon, is super simple.  A few mouse clicks and you are ready to go:

ticonfig

Once configured, you’ll have all of the standard Office365Mon notification options to keep you in the know when there’s a problem:  email messages, text messages, and our webhook feature.  In addition to the notifications, there are a number of interesting reports that we provide with Threat Intelligence Monitoring to help you analyze the nature of these attacks against your organization.

For example, here you can get the trend of malwares entering your organization during the current month:

CurrentTrends

In addition to the trend for the current month, there’s a similar chart that shows you a rolling two-month period so you can see what’s being targeted at you over a longer period of time.

You can also get an overview of the top 10 targeted users within your organization, so you can ensure that they are following security best practices:

TargetedUsers

There’s other reports that show you both for the current month as well as historically, data for different ways in which malware has been targeted at your organization.  For example, here’s one that shows the different malware file names that were sent into your organization:

MonthlyCount2

In addition to this, you can view this kind of summary data based on who sent malware infected messages, summaries of the Senders’ IP address, summaries based on the email Subject so you can look for patterns there, summaries on file type and file name as shown above, and also information on when the malware was detected.

We’re also taking this information and have added it into our Microsoft Cloud Command Center.  For those of you who aren’t familiar with it, the Cloud Command Center brings together information that previously existed as islands of data and loaded up all of the key metrics that you need about everything that’s going on with your Microsoft cloud services.  We’ve plugged in the malware trend report and user targeting report into the Cloud Command Center for a really great overview of the health of your organization and its cloud services:

ticloudcommand

We think features like Threat Intelligence Monitoring really expand and strengthen the base of important information you need to be in the know and in control of your organization and its cloud software services.  It all starts in Office 365, so you can help yourself get connected with this information by incorporating the E5 license in your organization.

The Threat Intelligence Monitoring service in Office365Mon is available in Preview today for everyone.  As with all new Office365Mon features, all existing customers have had this feature turned on for the next 90 days to try it out.  All new Office365Mon customers will also have this feature enabled for 90 days so they can see it working in their environment.  As always, we would still love to get feedback on how we can improve it and make it more useful to you, so please feel free to send it our way.  Licensing and pricing is not yet available for the Threat Intelligence Monitoring service; that will be set in Q1 of 2018.

We really have a wide and expansive set of tools to help you with your Microsoft cloud services now.  For monitoring Office 365 performance and availability, go to https://office365mon.com.  For monitoring Azure performance and availability, go to https://azureservicemon.com.  To monitor malware attacks using Threat Intelligence, go Office365Mon.Com and create your Office365Mon subscription, then you can configure Threat Intelligence monitoring at https://www.office365mon.com/Configure/Threats.

Thanks, and I hope everyone has a great holiday season!

From Sunny Phoenix,

Steve

Advertisements

Office365Mon Now Offers Monitoring Out of Regional Data Centers Starting with Germany

Office365Mon

Be in the know. Be in control

Office365Mon is now available to be run out of regional Azure data centers around the globe.  In addition to the US data centers, Office365Mon is available today via Germany data centers.

 logo

At Office365Mon.Com we’ve heard from customers of our Office 365 monitoring service that had certain data sovereignty restrictions placed upon them.  This means that in some areas, there are government regulations that require data for a company in that region, to be stored in that region.

In addition to that, today Office365Mon issues its health probes out of a US-based data center.  Customers can always (and in fact are encouraged) to augment that with our Distributed Probes and Diagnostics feature, which lets you issue probes from any location where you have users.  However, some folks were hoping to have our cloud-based probes issued from a data center closer to where their users live.

Today we’re happy to announce an option that solves both problems – you can now create Office365Mon subscriptions in regional data centers.  Our first data center is available now, and is located in Germany.  This means not only is your subscription created in Germany, and your health probes are issued out of Germany, but also all of the data we capture is also stored in that same German data center.  That allows you to meet data sovereignty requirements in that region, as well as potentially getting cloud-based probes closer to your users.

One of the first things to call out is that all of the features you get from our US-based service, you will also find in our German-based service.  So no matter where you go, you will find feature parity.  If you have users spread across the globe and want to create Office365Mon subscriptions in different geographies, that’s okay too.  One of the cool things we do is pull together data from all of your subscriptions in every data center when we show you our Power BI reports.  For example, here’s a screenshot the Outages report with data from every subscription in every region for our tenant:

powerbiall

Of course, one of the cool things about Power BI is that you also have a slew of filtering and report building features available to you.  Here’s a screenshot of that same report, only this time it’s filtered to show only data from the Office365Mon subscription that we have in Germany:

powerbifiltered

The net of this is that if you know Office365Mon today, it will look, work, and act exactly the same no matter what data center your subscription is located.

The next question we expect to get of course is – that’s great, but what if I want Office365Mon in a different geography?  The answer is…that’s great!  Email us and let us know what you are interested in, and we can provide you with an idea of when it can be done.  As long as the different regions have parity in the Azure features that Office365Mon requires, it should be possible to deploy Office365Mon to virtually any regional Azure data center.

There’s only one real difference with using Office365Mon in a regional data center – you have to work with our support team to enable you to get going.  If you contact us at support@office365mon.com, we will give you all of the details about the on-boarding process, pricing, etc.

We know that this has been in important issue for many of our European friends and customers, so I hope you’ll take a few minutes to take a look at our PDF from our US site or our German site.  Of course, you can also visit our US site directly at https://office365mon.com, or our new German site at https://office365mon.de.

Thanks as always for your great suggestions and ideas!
From Sunny Phoenix,

Steve

Now You Can Monitor OneDrive and Power BI with Office365Mon

Today we’ve released two new features that customers have been asking us about – monitoring for OneDrive for Business and Power BI.  These are just the latest set of services in an ever increasing stable of monitored features and services from Office365Mon.  Let’s take a quick look at each.

OneDrive for Business

Office365Mon has actually been able to monitor OneDrive for Business since our first release over two years ago.  However, many customers weren’t really aware of that and didn’t realize that when you configure monitoring for SharePoint Online, you could just tell it to go monitor a OneDrive site.  What we’ve done is to break OneDrive out as a separate monitored item on your Office365Mon subscription.  This gets you a couple of things – first, now you can monitor both a SharePoint Online site and a OneDrive site with the same subscription.  Second, it also allows us to give you a more granular breakdown of both performance and availability with your SharePoint Online tenant.  For example, here’s a report with the average performance across all Office365Mon customers for the primary Office 365 Services:

overallservice

As you can see with the yellow bar, now you can see what performance is like just on OneDrive sites vs. SharePoint Online sites from all of our customers.  Of course, you’ll also be able to see what performance is like just for your tenant in OneDrive.  We think breaking OneDrive out in this way will give you an even deeper view into the health and performance of your tenant, which is a great thing.

Power BI in Office 365

The second feature we added is brand new, and something that we’ve heard customers ask us about for a while now.  Power BI is increasingly becoming a critical part of the toolset that customers are using, so of course it’s important to know how well it’s performing and when it goes down.  Our new monitoring for Power BI does all of that, just as you’ve come to expect from Office365Mon.  The configuration of Power BI monitoring is drop dead simple, which is also something you’ve come to expect from us.  You literally click a single button – that’s it – as shown in this screenshot below:

powerbiconfig

Once you’ve enabled it, health probes begin a minute or two later; can’t get much easier than that!  Once we start collecting the data, details on performance and availability of Power BI show up in all of the same reports that you’ve already been using.  For example, here’s a screenshot of the recent health probes report, showing you the performance of your monitored resources:

perfmonchart

As you can see, you get the performance of your individual resources that are being monitored – including both Power BI and OneDrive – and that is overlaid on top of a bar chart with the overall numbers from all Office365Mon customers with the average performance for Exchange Online, SharePoint Online, OneDrive and Power BI.  Our goal is always to show you as much relevant data as possible and then give you the opportunity to drill down and filter as best suits your requirements.  Also, like all of our reports, you can add this to your own corporate intranet for anyone you want to be able to see, by using our Dashboard Reports feature.  Meanwhile, should Power BI go down, you can also take comfort in the fact that you’ll get all of the same notifications from us as you do today for all of your other monitored services.  Power BI is just another one of “the team” now, and is monitored just like all of the other features and services we’re keeping an eye on for you.

 

Try it Out

Both the OneDrive and Power BI monitoring features are available for you to try out now.  All existing Office365Mon customers as well as all new customers will have access to these features for 90 days.  New customers can go create an Office365Mon subscription at https://www.office365mon.com.  No payment information is required up front, and if you choose not to use it there’s nothing you need to do – after 90 days we just stop monitoring stuff for you.

For all Office365Mon paid subscriptions, we will continue to monitor OneDrive for you.  Power BI monitoring requires the Enterprise Platinum license.  If you have that, monitoring Power BI will continue for you; if you don’t have that license, you can upgrade to it at any time by going to our Payments page at https://www.office365mon.com/Products/Payments.

 

Thanks Again

As always, thanks again for the many terrific suggestions you all have provided.  We’re always looking for ways in which we can do a better job meeting your needs, and the best way to do that is for you to let us know what’s important to you.  Office365Mon really has a wide range of monitoring features now, so I hope if you haven’t tried it yet, or maybe if it’s been a while since you’ve looked at it, you’ll come by and take it out for a spin for 90 days.  We think you’ll find that it’s well worth the 2 minutes of your time to get things set up and going.

 

From Sunny Phoenix,

Steve

Monitoring Exchange Online Errors You Might Not Otherwise Notice with Office365Mon

We introduced monitoring for the email transport for Office 365 customers back in February of this year after strong demand from our customers (https://samlman.wordpress.com/2017/02/14/know-when-your-office-365-email-gets-stuck-in-transit-from-office365mon/).  We had provided monitoring of the email service itself since Day 1, but if your mailbox is up but the messages you send don’t go out, or the message people send you don’t make it in, then email is of pretty limited value.  As we moved the email transport monitoring service out into production and began gathering more and more data, we started finding a few common errors that many tenants were experiencing, but which are innocuous enough you might not even know that they are happening.

We decided to create some new reports around the email transport monitoring feature to help customers understand better when these situations occur.  Sometimes they point out problems that customers can resolve themselves, and other times it’s just good to understand better when and where the hiccups are in your email service.  In a nutshell, as we capture more data around errors in the transport, we are bubbling up the common errors into these new reports so you can see for yourself when they happen and how frequently they happen.

If you go into the Advanced Reports gallery on our site at https://www.office365mon.com/Reports/advreports, there are new “Recent Email Transport Errors” and “Monthly Email Transport Errors” reports.  The Recent Email Transport Errors report is a simple tabular list of the 100 most recent instances of these common errors described above.  Here’s a sample report:

etRecentErrors

As you can see, we indicate whether the issue was with an inbound or outbound email, when it happened (most recent first), and what the problem was.  This report has already paid off big for one of our customers, because with the information in it they were able to determine that they had an old MX record in DNS, and that MX record pointed to a server that was no longer available.  This showed up in our reports every time that server was selected for us to send mail to it, and as a result they were able to clean that up in their environment.  Some of the other common problems we see are things like Outlook API service is temporarily unavailable, the anti-spam features have incorrectly marked an outbound message as spam (and as a result it gets stuck in the Drafts folder)…happens most frequently with .onmicrosoft.com email accounts by the way, and requests to the service are unauthorized (which could be your access token has expired or Azure Active Directory is temporarily unavailable).  Some things you may be able to fix yourself; other things it’s just good information to have so you are aware of how well your transport features are performing.

The Monthly Email Transport Errors report helps you stay on top of this by presenting a bar chart with a count of each type of error, so you can see how frequently each type of error is occurring in your tenant each month.  Here’s an example of that:

etMonthlyErrors

At Office365Mon our position is you can never have too much good information.  Based on the early results of this reporting, we’re already seeing good outcomes and actionable data for our customers.  If you aren’t set up for Office 365 monitoring yet, please visit our web site at https://www.office365mon.com to get started.  Once you’ve configured basic monitoring, you can turn on email transport monitoring at https://www.office365mon.com/Configure/EmailTransport to get these reports yourself.  We’ll be continuing to mine through the data and expand out the collection of common errors as we see them.

As always, please feel free to send us your feedback at support@office365mon.com and thank you for all of the great ideas you’ve sent us already.

From Sunny Phoenix,

Steve

Monitoring Large Lists in SharePoint Online with Office365Mon

One of the enduring realities I saw over and over in my many years working at Microsoft with SharePoint, is that customers LOVE the lists they can create in SharePoint.  They’re super easy to set up, you can quick do all kinds of sorting, filtering and searching on them, and it requires no technical expertise to get up and running and with them.  This led to another enduring reality, which is that many customers started loving lists TOO much.  I saw many, many customers over the years that had lists that had just exploded in size.  As these lists grew larger, the performance in using them tends to get worse and worse.  This problem was also compounded by the fact that many developers saw SharePoint lists as a quick and easy place to store data for their application.  That meant even bigger lists sizes, and more people using them more often.

Over the years, we developed lots of documentation and options for improving the performance of your lists.  As customers have moved to SharePoint Online in Office 365 though, we would occasionally hear people ask if it had the same large list limitations as SharePoint on premises does…and the answer is yes, it does.  Now as more customers are moving their SharePoint on premises data to SharePoint Online, we see increasing concern about how the lists they do have are going to perform once it’s all up in Office 365.  Fortunately, at Office365Mon, we’ve just released a new feature designed specifically to help you stay on top of this issue.

List Monitoring is a feature that lets you select one or more lists in SharePoint Online for us to monitor.  For the lists that we’re monitoring, we will do a couple of things:  first, we’ll issue health probes for each list that we’re monitoring and render the default view for it to see what the performance is like.  That’s typically one of the first places where you’ll see performance issues with a large list.  You can configure List Monitoring so that it will send you a notification if it takes longer than “x” seconds to render the default view, where “x” is a number that you decide.

The second thing we’ll do is keep tabs on how large the list is, i.e. how many items it contains.  Again, you can set a threshold for us to look for, and when a monitored list gets bigger than that threshold, we’ll send you a notification to alert you to it.  So, for example, if you’re worried about a large list approaching that magic 5,000 item limit, you can have us notify you when it’s getting close.  Here’s a screen shot of where you configure the monitoring thresholds:

MonLargeLists1

Selecting the lists to be monitored is also quite simple – we provide you with a collection of all of the lists in the SharePoint Online site that we’re monitoring, and you can just check boxes next to the lists you want us to monitor for you.  It can be any of the lists that come out of the box with SharePoint, or any custom list that you’ve created:

MonLargeLists2

Once we’ve started monitoring lists for you, not only will we notify you according to the thresholds you’ve configured, but as you’ve come to expect from Office365Mon, we also have a nice set of reports you can use to see where you stand.  To begin with, you can see the performance of the recent health probes we’ve issued against monitored lists in our Average Response Time report.  It shows the performance of all of the resources that we’re monitoring for you, including monitored lists.  We also have a new report that shows you the average performance each month just for your monitored lists:

MonLargeLists3

In addition to that, we have a report that shows you the size of your monitored lists each day, so you can visualize any growth trends that might be happening that you need to get in front of:

MonLargeLists4

We also provide a monthly view of the average size of each monitored list, so you have a longer-term view of how rapidly your lists are growing:

MonLargeLists5

Being aware of large lists and their impact on performance is one of the best ways to ensure a good experience for your users.  I’ve heard many, many times from customers that say “our site is slow”.  There are lots of reasons why that might be, but a couple of the most common reasons are slow query times and large lists.  At Office365Mon we’ve provided monitoring for your query execution time for nearly a year now.  With the new List Monitoring feature, now you can also know when you have large list performance problems.  Once you know that, you can start working on a mitigation strategy – splitting the data out into multiple lists, creating customized views of the data, etc., etc., etc.  There are a lot of different things you can do to work on improving the performance, but if you don’t know you have a problem then you’ll forever be stuck wondering why your users keep telling you that your “site is slow”.  Take advantage of features like these to stay in the know and stay in control of your Office 365 tenant, and keep everyone happy.  Start by visiting us at https://www.office365mon.com and clicking the Configure…List Monitoring menu.

This is yet another feature at Office365Mon that was driven from feedback by our customers.  I hope you’ll take a look at this feature and as always, let us know how we can make it better as well as ways in which we might be able to help you to do Office 365 monitoring even better.

From Sunny Phoenix,

Steve

 

Migrating an Office Task Pane App to a Toolbar Add In

If you’re one of those less-informed people, like myself apparently, you may have recently received an email or otherwise found out that the Office task pane app you published in the Office Store a while ago is no longer in compliance.  It is now a requirement that any task pane apps that were previously submitted – back when that was the only app form factor available – need to be refactored to support the Add In toolbar model.  This announcement was easy to miss since it only appeared rather randomly once on some Office team blog.  Rather ironic given that this comes from the team that provides so many office communications tools.

But I digress…once you find out you need to update your app to support the new model (as well as the old model, since not every single customer is on Office 365 and/or Office 2016), how do you do it?  Unfortunately the fine folks that made this policy change neglected to provide a guide on how to do so.  They merely provide links to very general and fairly vague documentation, so since I just went through this process I thought I would capture it for others who may need to do the same thing.

Here are the steps I went through to move my Outlook task pane app to the Add In toolbar model.

Step 1 – Get the latest version of the Office Tools for Visual Studio

The actual location to get this may vary depending on your locale and version of Visual Studio, so I’ll just leave it at “go get it and install it”.

Step 2 – Start Working with Your Application’s Manifest File

Interestingly enough, once the latest tools are installed, it’s actually gone a step backwards from a tooling perspective.  When I double-click on my app manifest in Visual Studio 2015 now, it no longer opens the visual designer that lets you set attributes in your manifest.  Instead it just opens the underlying Xml file and you get to edit Xml by hand.  I could swear I’ve seen this before…oh wait – I did – like 15 years ago.  Visual Studio does at least provide some editing advantages over Notepad, so there is a certain amount of forward progress.

Step 3 – Update the top-level OfficeApp Element

The main change needed to the manifest file is going to be to add a VersionOverrides node and populate it with the correct data.  Note that you can keep all of your existing content in your Xml file, which you’ll actually want to do to ensure that it is supported on clients that do not have Office 2016.  In order to add the VersionOverrides node though, you first need to update the top-level OfficeApp element.  This is to add some namespaces that are needed to support the content in VersionOverrides.  Here’s an example of my new OfficeApp element:

xml0

One important point to note:  my element includes the “mailappor” namespace and is of type “MailApp”.  If your app is not an Outlook add-in, then the values you will need to add will be different.  While I could try and point you to some documentation, I actually recommend doing this to make sure you have it exactly right (I found the documentation to be a bit painful):  just create a new project in Visual Studio for an Office Add In that is the same type as the one you are updating.  Then you can grab the correct namespaces and types from the Xml file for the manifest it creates.  Much quicker and easier than searching for and through sketchy documentation.

Step 4 – Add the VersionOverrides Node at the END of Your Existing Content

Here’s something that isn’t at all obvious – you can’t just add the VersionOverrides node anywhere in your app’s manifest.  You need to add it at the end – after any Rules you may have (like in a Rule node), which is after your Permissions node.  Your VersionOverrides node may be perfectly formed, but if you place it any higher in the manifest, you will get an Xml validation error and your life will suck.

Step 5 – Add Your VersionOverrides Node and Start Editing

For this step, I’m just going to paste in a big bunch of Xml and then start talking through what it is and how it’s all linked together.  So to begin, here’s the Xml – both some stuff from my original manifest file and the new VersionOverrides node:

  xml1

Okay, let’s start breaking some of this down.

Step 5.1 – The VersionOverrides Node

My VersionOverrides node looks like this:

xml3

The main thing I want to point out here is the namespace.  Again, my app is an Add In for Outlook, so it uses that namespace.  If your app is not for Outlook, it will be different.  Again, look at the content in your sample application you created above to get the correct value.

Step 5.2 – Set All of the Common Property Values

Within the VersionOverrides node, there’s a chunk of properties that will likely be used in different form factors – for example reading a message, composing a message, reading an appointment, etc.   So let’s look at this chunk of Xml:

 xml2

As you can see, these are the resources for your app – icons, icon titles, tool tips, that sort of thing.  You’ll notice that you need icons that are 16×16, 32×32 and 80×80.  Add those to your Add In’s web site and then plug in the Url.

For the functionFile I cheated…just meaning I don’t have any UI-less triggers, so I just plugged in the home page for my Add In’s site.  If you have a javascript file for this, then you would put the Url here.

The ShortStrings and LongStrings section are where you put the strings that will be used for the icon titles and tool tips.  Plug in whatever is appropriate for your app.

Now, here’s an important one – look at the item with the id “messageReadTaskPaneUrl” – THIS is where you plug in the Url that you used before in your original task pane app.  For example, I’m going to paste in side by side here a couple of chunks of Xml from above.  The first one is for my original task pane definition, and the second one is my messageReadTaskPaneUrl node:

xml6

Hey look – the Urls are the same!  Finally, starting to make some sense.

Here’s the big thing that is not clearly called out in the documentation – look at the id’s all of the elements in the resources section.  You’re going to plug these same id’s in when you declare the various form factors your application supports – such as message read, message compose, etc.  Let’s see how that works next.

Step 5.3 – Add Your Form Factor Implementations

This step is what ties it all together.  You create a definition for each form factor you want to support, like Message Read, Message Compose, etc.  I’m going to paste in the Xml for one of my form factors below and then talk through it:

xml7

Let’s look at some of these individual elements now:

  • ExtensionPoint xsi:type – this is what defines the form factor, in this case Message Read
  • Label resid=”groupLabel” – this corresponds to the ShortString I have in the resources section with the id “groupLabel”
  • Label resid=”paneReadButtonLabel” – this corresponds to the ShortString I have in the resources section with the id “paneReadButtonLabel”
  • Title resid=”paneReadSuperTipTitle” – you probably get this by now – corresponds to the id in ShortStrings
  • Description resid=”paneReadSuperTipDescription” – again, guessing you understand here. What you’ll see is the icons you’ve defined show up in the toolbar (in the Outlook 2016 client) or in the Outlook Web Access UI.  In the toolbar the icon will have the title I put in the “paneReadSuperTipTitle”, and when you hover over it you will get a tooltip that contains the paneReadSuperTipDescription.
  • Icon node – you’ll see the three different icons, with each one pointing to the different id’s that were added in the resource section.
  • Action – this is the money node – the SourceLocation has a resid value that points to the id in our Resources section with the Url that we used in our task pane app; in my example that’s https://weatheroutlook.azurewebsites.net/AppRead/Home/HomeRead.html.

That’s really it – once you understand how all the pieces relate to each other in the manifest file, then it’s really pretty simple to add other form factors (or ExtensionPoint nodes as they’re called in the Xml).  After that, you just need to click the Publish item on the shortcut menu for your App project and click the Package the add-in button.  It’s a little silly because all it does is open a window with your manifest’s Xml file.  Then you can click the button at the bottom of the page to Visit the Seller Dashboard, update your manifest, and you’re done!  Just wait for it to get through the validation process successfully and your Office Store application will be updated.

That’s it – hopefully you can use this guide to walk through the process of updating your own applications.  I’ve uploaded my own manifest file so you can see the entire thing and hopefully it will help you see how it all plugs in together – you can get it from https://1drv.ms/u/s!AEFcZajG99GWlIYw.  In case it doesn’t automatically download for you, look for the WeatherOutlook.zip file. Good luck!!

 

Using the Office 365 Batch Operations API

As I was looking around for a way to batch certain operations with the Office 365 API the other day, I stumbled upon a Preview of just such a thing, called “Batch Outlook REST Requests (Preview)” – https://msdn.microsoft.com/en-us/office/office365/api/batch-outlook-rest-requests.  The fundamentals of how it works is fairly straightforward, but it’s completely lacking implementation details for those using .NET.  So, I decided to write a small sample application that demonstrates using this new API / feature / whatever you want to call it.

First, let’s figure out why you might want to use this.  The most common reason is you are doing a bunch of operations and don’t want to go through the overhead of creating, establishing, and tearing down an HTTP session for each operation.  That can slow down quickly and burn up a lot of resources.  Now when I was first looking at this, I was also interested in how it might impact throttling limits that Office 365 imposes.  Turns out I had a little misunderstanding of that, but fortunately Abdel B. and Venkat A. explained Exchange throttling to me, and so now I will share with you.

My confusion about impact on throttling that batch operations might have was borne out of the fact that SharePoint Online has an API throttling limit that has been somewhat ubiquitously defined as no more than 1 REST call per second over an extended time.  So…kind of specific, but also a little vague.  Exchange Online throttling is arguably even less specific, but they do have some good information about how to know when it happens and what to do about it.

In Exchange Online, different operations may have a different impact on the system, and it may also be impacted by demands from other clients.  So when making REST API calls to Exchange Online your code should account getting a throttling response back.  A throttled response in Exchange Online returns a standard http status code 429 (Too many requests).  The service also returns a Retry-After header with the number of seconds to resubmit the request.  Now that you know what a throttled response from Exchange Online looks like, you can develop your code to include a process for retry and resubmission.

The batching feature lets you work around the overhead of multiple calls by allowing you to send in up to 20 operations in a single request.  That means 1 connection to create, establish and tear down instead of 20.  This is goodness.

The basic process of doing batch operations using this feature is to create what I’ll call a “container” operation.  In it, you will put all of the individual operations you want to perform against a particular mailbox.  Note that I said mailbox – this is important to remember for two reasons:  1) the batch feature only works today with Outlook REST APIs and 2) the individual operations should all target the same mailbox.  That makes sense as well when you consider that you have to authenticate to do these operations, and since they are all wrapped up in this “container” operation, you’re doing so in the context of that operation.

The “container” operation that I’m talking about is POST’ed to the $batch endpoint in Outlook:  https://outlook.office.com/api/beta/$batch.  The Url is hard-coded to the “beta” path for now because this API is still in preview.  In order for you to POST to the $batch endpoint you need to provide an access token in the authorization header, the same way as you would if you were making each of the individual calls contained in your container operation.  I’m not going to cover the process of getting an access token in this post because it’s not really in scope, but if you’re curious you can just look at the sample code included with this post or search my blog for many posts on that type of topic.

While I’m not going to cover getting an access token per se, it’s important to describe one higher level aspect of your implementation, which is to create an application in your Azure Active Directory tenant.  Generally speaking, you don’t access an Office 365 REST API directly; instead, you create an application and configure it with the permissions you need to execute the various Outlook REST APIs you’ll be using.  In my case, I wanted to be able to read emails, send emails and delete emails, so in my application I selected the following permissions:

batchop1

So with that background, here are the basic steps you’ll go through; I’ll include more details on each one below:

  1. If you don’t have an access token, go get one.
  2. Create your “container” operation – this is a MultipartContent POST.
  3. Create your individual operations – add each one to your MultipartContent.
  4. POST the “container” operation to the $batch endpoint.
  5. Enumerate the results for each individual operation.

 

Step 1 – Get an Access Token

As I described above, I’m not going to cover this in great detail here.  Suffice to say, you’ll need to create an application in Azure Active Directory as I briefly alluded to above.  As part of that, you’ll also need to do “standard Azure apps for Office 365” stuff in order to get an access token.  Namely, you’ll need to create a client secret, i.e. “Key”, and copy it along with the client ID to your client application in order to convert the access code you get from Azure into an AuthenticationResult, which contains the access token.  This assumes you are using ADAL; if you are not, then you’ll have your own process to get the access token.

 

Step 2 – Create Your Container Operation

The “container” operation is really just a MultipartContent object that you’ll POST to the $batch endpoint.  Unfortunately, there is scarce documentation on how to create these, which is in large part why I wrote this post.  The code to get you started though is just this simple:

 

//create a new batch ID

string batchId = Guid.NewGuid().ToString();

//create the multipart content that is used for a batch process

MultipartContent mpc = new MultipartContent(“mixed”, “batch_” + batchId);

The main thing to note here is just that each “container” operation requires a unique batch identifier.  A Guid is perfect for this, so that’s what I’m using to identify my batch operation.

 

Step 3 – Create Individual Operations and Add to the Container Operation

The actual code you write here will vary somewhat, depending on what your operation is.  For example, a request to send an email message is going to be different from one to get a set of messages.  The basic set of steps though are similar:

  1. Create a new HttpRequestMessage. This is going to be how you define whether the individual operation is a GET, a POST, or something else, what Url to use, etc.  Here’s the code I used for the operation to send a new email:  HttpRequestMessage rqMsg = new HttpRequestMessage(HttpMethod.Post, BATCH_URI_BASE + “me/sendmail”);  It’s worth noting that you ALWAYS send your individual operations to the $batch endpoint to be included in the batch process.  For example, if you were using v2 of the Outlook API, to send a message you would use the Url https://outlook.office.com/api/v2.0/me/sendmail.  However, to use the $batch endpoint, since it’s in beta, you use the Url https://outlook.office.com/api/beta/me/sendmail.
  2. Create the content for your operation. In my case I used a custom class I created to represent a mail message, I “filled it all out”, and then I serialized it to a JSON string.  I then took my string to create the content for the operation, like this:  StringContent sc = new StringContent(msgData, Encoding.UTF8, “application/json”);  So in this case I’m saying I want some string content that is encoded as UTF8 and whose content type is application/json.
  3. Add your content to the HttpRequestMessage: Content = sc;
  4. Wrap up your HttpRequestMessage into an instance of the HttpMessageContent class. Note that you’ll need to add a reference to System.Net.Http.Formatting in order to use this class.  Here’s what it looks like:  HttpMessageContent hmc = new HttpMessageContent(rqMsg);  We’re doing this so that we can set the appropriate headers on this operation when it’s executed as part of the batch.
  5. Set the headers on the HttpMessageContent object: Headers.ContentType = new MediaTypeHeaderValue(“application/http”); and also hmc.Headers.Add(“Content-Transfer-Encoding”, “binary”);  You now have a single operation that you can add to the “container” operation.
  6. Add your individual operation to the “container” operation: Add(hmc);  That’s it – now just repeat these steps for each operation you want to execute in your batch.

Side note:  I realize some of this code may be difficult to follow when it’s intertwined with comments like I’ve done above.  If you’re get squinty eyed, just download the ZIP file that accompanies this post, and you can see all of the code end to end.

 

Step 4 – Post the Container Operation to the $Batch Endpoint

There’s not a lot to step 4.  You can just POST it now, but there’s one other point I want to make.  Your “container” operation may contain many individual operations.  There are a couple of points about that worth remembering.  First, the individual operations are not guaranteed to be performed in any specific order.  If you need them to be performed in a specific order, then either don’t do them in a batch or do them in separate batches.  Second, by default, at the point that any individual operation encounters an error, execution stops and no further operations in the batch will be executed.  However, you can override this behavior by setting a Prefer header in your “container” operation.  Here’s how you do that:

mpc.Headers.Add(“Prefer”, “odata.continue-on-error”);

With that done (or not, depending on your requirements), you can go ahead and POST your “container” operation to the $batch endpoint, like this:

HttpResponseMessage hrm = await hc.PostAsync(BATCH_URI_BASE + “$batch”, mpc);

With that done, it’s time to look at the results, which is covered in the next step.

 

Step 5 – Enumerate the Results for Each Individual Operation

At a high level, you can see if the overall batch operation worked the same way you would if it were just one operation:

if (hrm.IsSuccessStatusCode)

The important thing to understand though, is that even though the “container” POST may have worked without issue, one or more of the individual operations contained within may have had issues.  So how do you pull them all out to check?  Well, using the MultipartMemoryStreamProvider class is how I did it.  This is another class that requires a reference to System.Net.Http.Formatting in order to use, but you should already have it from the other steps above so that shouldn’t be a problem.

So we start out by getting all of the responses from each individual operation back like this:

MultipartMemoryStreamProvider responses = await hrm.Content.ReadAsMultipartAsync();

You can then enumerate over the array of HttpContent objects to look at the individual operations.  The code to do that looks like this:

for(int i=0; i < responses.Contents.Count;i++)

{

string results = await responses.Contents[i].ReadAsStringAsync();

}

It’s a little different from having an HttpResponseMessage for each one in that you have to do a little parsing.  For example, in my sample batch I sent two emails and then got the list of all of the emails in the inbox.  As I enumerate over the content for each one, here’s what ReadAsStringAsync returns for sending a message:

HTTP/1.1 202 Accepted

Okay, so you get to parse the return status code…should be doable.  It can get a little more cumbersome depending on the operation type.  For example, here’s what I got back when I asked for the list of messages in the inbox as part of the batch:

HTTP/1.1 200 OK

OData-Version: 4.0

Content-Type: application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8

{“@odata.context”:”https://outlook.office.com/api/beta/$metadata#Me/MailFolders(‘Inbox&#8217;)/Messages”,”value”:[{“@odata.id”:”https://outlook.office.com/api/beta/Users(’05d6cc47-5a79-4906-88e6-c39fcd595e15@b098aeb9-ce11-43ce-a49f-ee4b5a4b0a71&#8242;)/Messages(‘AAMkADEyMzQ3MzllLWM2NmItNGY2ZS04MWE1LTQwNjdiZDc1ZGYxNwBGAAAAAADRpmW4I…}}]}

Okay, so I trimmed a bunch of detail out of the middle there, but the gist is this – you would have to parse out your HTTP status code that was returned, and then parse out where your data begins.  Both quite doable, I just kind of hate having to do the 21st century version of screen scraping, but it is what it is.  The net is you can at least go look at each and every individual operation you submitted and figure out if they worked, retrieve and process data, etc.

Summary

That’s the short tour of using the Outlook batch API.  There are a handful of things you need to know about how it works and what it’s limitations are, and I’ve pointed out all of the ones I know about in this post.  The trickier part by far is understanding how to create a batch request using the .NET framework, as well as how to parse the results, and I covered both of those aspects of it as well.

As I mentioned a few times in this post, I just zipped up my entire sample project and have attached it to this post so you can download it and read through it to your heart’s content.  It does contain the details specific to my application in Azure AD, so you’ll need to create your own and then update the values in the app if you want to run this against your own tenant.  The ZIP file with the code is below: