Monitoring Large Lists in SharePoint Online with Office365Mon

One of the enduring realities I saw over and over in my many years working at Microsoft with SharePoint, is that customers LOVE the lists they can create in SharePoint.  They’re super easy to set up, you can quick do all kinds of sorting, filtering and searching on them, and it requires no technical expertise to get up and running and with them.  This led to another enduring reality, which is that many customers started loving lists TOO much.  I saw many, many customers over the years that had lists that had just exploded in size.  As these lists grew larger, the performance in using them tends to get worse and worse.  This problem was also compounded by the fact that many developers saw SharePoint lists as a quick and easy place to store data for their application.  That meant even bigger lists sizes, and more people using them more often.

Over the years, we developed lots of documentation and options for improving the performance of your lists.  As customers have moved to SharePoint Online in Office 365 though, we would occasionally hear people ask if it had the same large list limitations as SharePoint on premises does…and the answer is yes, it does.  Now as more customers are moving their SharePoint on premises data to SharePoint Online, we see increasing concern about how the lists they do have are going to perform once it’s all up in Office 365.  Fortunately, at Office365Mon, we’ve just released a new feature designed specifically to help you stay on top of this issue.

List Monitoring is a feature that lets you select one or more lists in SharePoint Online for us to monitor.  For the lists that we’re monitoring, we will do a couple of things:  first, we’ll issue health probes for each list that we’re monitoring and render the default view for it to see what the performance is like.  That’s typically one of the first places where you’ll see performance issues with a large list.  You can configure List Monitoring so that it will send you a notification if it takes longer than “x” seconds to render the default view, where “x” is a number that you decide.

The second thing we’ll do is keep tabs on how large the list is, i.e. how many items it contains.  Again, you can set a threshold for us to look for, and when a monitored list gets bigger than that threshold, we’ll send you a notification to alert you to it.  So, for example, if you’re worried about a large list approaching that magic 5,000 item limit, you can have us notify you when it’s getting close.  Here’s a screen shot of where you configure the monitoring thresholds:

MonLargeLists1

Selecting the lists to be monitored is also quite simple – we provide you with a collection of all of the lists in the SharePoint Online site that we’re monitoring, and you can just check boxes next to the lists you want us to monitor for you.  It can be any of the lists that come out of the box with SharePoint, or any custom list that you’ve created:

MonLargeLists2

Once we’ve started monitoring lists for you, not only will we notify you according to the thresholds you’ve configured, but as you’ve come to expect from Office365Mon, we also have a nice set of reports you can use to see where you stand.  To begin with, you can see the performance of the recent health probes we’ve issued against monitored lists in our Average Response Time report.  It shows the performance of all of the resources that we’re monitoring for you, including monitored lists.  We also have a new report that shows you the average performance each month just for your monitored lists:

MonLargeLists3

In addition to that, we have a report that shows you the size of your monitored lists each day, so you can visualize any growth trends that might be happening that you need to get in front of:

MonLargeLists4

We also provide a monthly view of the average size of each monitored list, so you have a longer-term view of how rapidly your lists are growing:

MonLargeLists5

Being aware of large lists and their impact on performance is one of the best ways to ensure a good experience for your users.  I’ve heard many, many times from customers that say “our site is slow”.  There are lots of reasons why that might be, but a couple of the most common reasons are slow query times and large lists.  At Office365Mon we’ve provided monitoring for your query execution time for nearly a year now.  With the new List Monitoring feature, now you can also know when you have large list performance problems.  Once you know that, you can start working on a mitigation strategy – splitting the data out into multiple lists, creating customized views of the data, etc., etc., etc.  There are a lot of different things you can do to work on improving the performance, but if you don’t know you have a problem then you’ll forever be stuck wondering why your users keep telling you that your “site is slow”.  Take advantage of features like these to stay in the know and stay in control of your Office 365 tenant, and keep everyone happy.  Start by visiting us at https://www.office365mon.com and clicking the Configure…List Monitoring menu.

This is yet another feature at Office365Mon that was driven from feedback by our customers.  I hope you’ll take a look at this feature and as always, let us know how we can make it better as well as ways in which we might be able to help you to do Office 365 monitoring even better.

From Sunny Phoenix,

Steve

 

Know When Your Office 365 Email Gets Stuck in Transit From Office365Mon

One of the things that we’ve done from day one when monitoring Office 365 is to let you know if your Exchange mailboxes are working or not.  While that’s an absolute necessity and quite valuable, it still left part of the story untold – until now.  One of the things we heard from customers is “my mailbox seems fine, but I sent an email 30 minutes ago and it hasn’t arrived yet”.  Or “I was told they sent me an email a couple of hours ago but I haven’t seen it yet”.  For those that have witnessed this behavior before, you know that sometimes email gets stuck in transit.  Your mailbox may be working fine, but if messages aren’t going out or coming in, then it can be a real problem.

At Office365Mon.Com, we’re addressing this with our new Email Transport Monitoring feature.  The Email Transport Monitoring feature lets you monitor both outbound messages – those that you send – and inbound messages – those that you receive.  You can set a delivery time for each direction, and if our health probe messages don’t arrive within that timeframe, we’ll send you a notification.  Like all of our other notifications at Office365Mon, that can be emails, text messages, and/or our webhook service.

Configuring monitoring for your email transport is incredibly simple.  Go to our web site and start here:  https://www.office365mon.com/Configure/EmailTransport.  This is a screenshot:

configemailtransport

It’s pretty simple – if you want us to monitor email transport, just check the box and tell us how long to wait before notifying you when a message doesn’t show up.  If you don’t check the box, then we won’t monitor it for you.

Once we’ve started monitoring it for you, there are some very nice reports you get.  To begin with, you’ll see the message delivery times for everything we’ve been monitoring over the previous 12 hours:

emailtransportrecent

In addition to that, you get monthly summary numbers so you can see what your delivery times have been:

emailtransportmonthly

Also, we have a report that shows you every message that was so slow it triggered a notification, so you can always review the history of when you had issues.  In addition, you will see email transport performance information rolled up into our Power BI reports.  The raw data for these reports is also available for download from our site if you have the Report Data feature enabled, and you can also programmatically retrieve it via our reporting REST API.

This is another in a long line of customer suggested features we’ve implemented.  Like the others, it’s a super scalable cloud-only monitoring feature that doesn’t require you to install any software, anywhere.  It’s available today as a Preview feature on our site at http://www.office365mon.com.  Just go to the Configure menu and click on Email Transport Monitoring – then check the boxes for the directions you wish to monitor.  All customers get this feature free to try for 90 days so check it out and let us know what you think.  After 90 days you can get this feature with our new Enterprise Platinum license.  See our site at https://www.office365mon.com/Products/Pricing for more details on pricing.

As always, thanks so much for all of the great suggestions.  We listen and continue to build on your needs.

From Sunny Phoenix,

Steve

Office365Mon Now Provides Reports Anywhere with Dashboard Reports

One of the most common requests we receive at Office365Mon has to do with the different types of reporting we provide on the availability and performance of a customer’s Office 365 tenant.  There are dozens of reports out of the box, as well as snapshot reports with the up to the minute availability of these services, and many of our customers have asked how they could get these reports for their own internal web sites.  Today we are announcing a feature to do exactly that – the Dashboard Reports feature from Office365Mon.

The Dashboard Reports feature is designed to let you take the virtually all of the reports available in our site that we get when we monitor Office 365 for you, and add them to any web site you wish with literally a single HTML tag.  In addition to being simple though, it’s also secure.  In order to use Dashboard Reports, you need to go to the Office365Mon site and get a secret key that is required to display Dashboard Reports.  That prevents anyone else from displaying your Office365Mon data.  In addition, you can remove and add keys as needed, so if a key is compromised at some point you can quickly and easily remove it.

In addition to the reports that we’ve had in our report gallery, the two most common requests we had from customers who want to display data in their own internal sites has been the current status information widgets from the My Info page (https://www.office365mon.com/Features/MyInfo).  To enable the display of these widgets in the Dashboard Reports features, we created two new reports and added them to the report gallery.  They display the same information as you find on the My Info page, and look like this:

Service Status

serviceinforpt

Tenant Status

tenantinforpt

Complete instructions for adding reports to your site using the Dashboard Reports feature can be found here:  https://www.office365mon.com/Configure/Dashboard.  It takes just a couple of minutes to create a key to use with your Dashboard Reports, and then to create the HTML tag to actually render it.  The documentation also describes how you can do things like color the report background as well as select which report to show.  Here’s an example of a Dashboard Report being shown in a completely different site from Office365Mon:

dashboardreport

The new Dashboard Reports feature makes it super easy and simple to build your own custom Office 365 health dashboards now with just a few lines of HTML.  Visit our site at https://www.office365mon.com to learn about it and all of the other Office 365 monitoring features available from Office365Mon.Com.

From Sunny Phoenix,

Steve

Monitor for Changes in Your Version of Exchange Online with Office365Mon

At Office365Mon we monitor all sorts of things about Office 365.  One of the early customer requests that we incorporated into our service was the ability to monitor for changes in the version of SharePoint that have been pushed out to your SharePoint Online tenant.  We have had several customers ask for the same type of monitoring and notification when changes occur in Exchange Online, and today we’re happy to announce the availability of just such a feature.

New versions can bring with them changes in the user interface and potential issues for customizations and apps.  That’s why for many customers, it’s critical for them to know when the version of SharePoint Online or Exchange Online changes.  Enabling Office365Mon to stay on top of these changes is quite simple.  When you go into configure your Office365Mon subscription, there is a box that you can check to monitor for version changes:

Monitor for Exchange Online Version Changes

exoversionchange

Monitor for SharePoint Online Version Changes

spoversionchange

One of the main differences between the two is that to monitor for changes in version for Exchange Online, we need a higher level of access to the mailbox being monitored than if we’re just monitoring for availability and performance.  As a result, we strongly recommend that you set up a separate mailbox just for monitoring if you want to have us check for version changes.  This is one of our standard security best practices in any case for Least Privileged Access, and the extra permissions needed to monitor for version changes only reinforces operating with that philosophy.

One of the other changes that we made as part of this feature release is to add webhook notifications for changes in version for both SharePoint Online and Exchange Online.  A webhook allows you go get a programmatic notification, and then you can execute your own workflows when you receive it.  For example, when you get a webhook notification that the version has changed, you can kick off a process to do validation of the features in your tenant or custom applications to ensure that they are all still working as expected with the new version.

We hope you find these new capabilities useful to the management of your Office 365 tenant.  It’s yet another feature set that has been driven entirely based on feedback from our customers.  Please try it out and let us know what you think!

From Sunny Phoenix,

Steve

 

Introducing Monitoring for Azure Secured Web Sites from Office365Mon

Today we’re excited to announce another new monitoring feature at Office365Mon.  Beginning today, we are now offering you the capability to monitor virtually any web site or REST API using the same proven, enterprise grade monitoring capabilities of Office365Mon.  The same service we use for Office 365 monitoring can now be used to monitor your sites that you deploy to Azure web sites, or your SharePoint hosted apps for on-prem or Office 365 sites, or virtually any other site!  You get the power of a monitoring infrastructure that sends out 10 to 20 million health probes a month to keep you in the know of your own web sites and REST APIs now.

All of this will come with the same kind of integration that you’re used to seeing at Office365Mon.  Setup will be extremely quick and easy, all done in your browser as well as via our Office365Mon Management API.  You’ll get the same sort of alert notifications as you do with the other resources we monitor for you – text messages, emails, and webhook notifications.  You’ll also see all of the data we capture about the health and performance of these sites in the same exact reports you use today, whether that’s one of our Standard or Advanced reports, you download your report data from our site, or you use Power BI with the Office365Mon Content Pack.  Here’s an example of a performance report that’s monitoring both Office 365 sites as well as other sites we have hosted in Microsoft Azure:

websitemonitoring

As you can see from the chart above, we’re monitoring:

Your sites and REST APIs can be hosted anywhere of course, as long as it has a public endpoint we can connect to.  It can be an anonymous site, or it can be secured with Azure Active Directory.  We can also monitor REST APIs as long as they don’t require any parameters.

This feature is available in Preview today and ready for you to begin trying out.  Get started by creating your Office365Mon subscription and then adding some sites to monitor here.  Pricing and licensing has not been set yet, but the good news is that like all new features, all existing customers will get 90 days to try it out for free.  Especially while this is in Preview, it’s a great opportunity to take a look and give us any feedback you have so we can fine-tune it to meet your needs.  Like many of the things you see at Office365Mon, this is another feature that was created based on feedback from our customers.

We hope you enjoy this new feature and will take the time to try it out.

From Sunny Phoenix,

Steve

 

Cloud Solution Services Now Available from Office365Mon and TechStar Group

Today I’m very excited to announce a new offering from Office365Mon and our newest Gold Star Partner, TechStar Group.  We are teaming together to take the skills and experience we’ve built while creating one of the most scalable and extensible cloud service solutions you’ll find for monitoring Office 365 – Office365Mon of course!

At Office365Mon we send out tens of millions of health probe queries every single month.  Now we’re bringing those skills out to help you with your move the cloud as well as part of our partnership with TechStar Group.   Our architecture and design demonstrates superb quality at a scale that is beyond what most of even the largest enterprises do. We send out tens of millions of health probe queries every month at Office365Mon, and we do so utilizing all of the leading edge technologies needed to accomplish such a huge task – Azure Active Directory, Office 365 APIs, Azure web apps, Azure cloud services, Azure storage and queueing, SQL Azure storage, and more.

Now, we’ve teamed up with our Gold Star Partner TechStar Group to unleash our team of experts on your cloud projects. We cover everything from architecture and design for new Azure and Office 365 projects, development, cloud migration, and much more. All projects are given oversight by former Microsoft Senior Principal Architect and Office365Mon founder Steve Peschka, so you can rest assured that the work being done will meet the highest quality bar. In addition, TechStar Group brings a number of former Microsoft employees and others with many years of successful technology delivery projects to the table. Together we offer a shared team that’s capable of delivering on the biggest and most challenging cloud projects.

We hope if you are working on cloud applications, services, migrations, etc. that you’ll consider contacting us to find out how our battle tested team of cloud savvy architects and engineers can help you make your own move to the cloud a successful one.  Please contact us to talk to us about your needs:

  • Office365Mon:  Steve Peschka, speschka@office365mon.com
  • TechStar Group:  Garrison Walls, garrison@techstargroup.com

Thanks as always.

From Sunny Phoenix,

Steve

 

Expanding the Office365Mon Subscription Management API

Today we’re happy to announce a slew of new APIs that have been added to our Subscription Management API tool set.  The Subscription Management API at Office365Mon has long been a market differentiator with other solutions in the Office 365 monitoring space.  Our first releases allowed you to manage the basics of the core monitoring features of Office365Mon.  Based on customer demand, we have just released a significant expansion of those APIs.  Our total feature set for managing your Office365Mon subscription has gone from 28 APIs to 46.

The new API support allows you to do things like configure the cloud service for your Distributed Probes and Diagnostics deployments (https://www.office365mon.com/Configure/OnPremProbes), which allows you to issue health probes in conjunction with our cloud service from any geographic location where you have users.  You can also configure the integration with the Office 365 Service Communication API (https://www.office365mon.com/Signup/Status), which allows you to stay up to date with any changes in the status of all of the services and features you have in your Office 365 tenant.  You can also manage Office365Mon’s monitoring of the SharePoint Online search service (https://www.office365mon.com/Configure/SearchMon).  This is critical for virtually all SharePoint customers, since so much of the content is driven by search – such as Content by Query web parts, search-based navigation, etc. – in addition to being used by many, many custom applications.  With these new APIs, you can now manage virtually every single thing using our API that you can do in the browser on our site.

Today’s announcement marks another set of innovative features that were developed based on feedback from you, our customers.  We hope you’ll find these to be valuable additions to the management of your Office365Mon subscriptions.  As always, if you have other requests or ideas for features you would like to see in our service, please just send us a note at support@office365mon.com.

From Sunny Phoenix,

Steve

New Geographic and Notification Features for Distributed Probes from Office365Mon

We’ve just released a significant update to the Office365Mon Distributed Probes and Diagnostics feature.  For those of you who aren’t familiar with this feature, it was originally released a little over a year ago.  From the beginning it was designed to do two things:

  1. Work in conjunction with the Office365Mon cloud service to issue health probes from different geographic regions where you have users. That allows you to check the availability and performance not only from our cloud service, but also from all of the locations where you have users.
  2. When there’s a problem connecting to Office 365, it runs a series of diagnostics on the local network to try and determine if there are any issues. That includes things like checking local network cards, DNS, gateway and a non-Office 365 Internet site.

In addition to the tasks above, it also allows you to set a performance threshold – for example, let me know when it takes longer than x seconds to connect to and get data back from Office 365.  You can set “x” to whatever value you want, so it allows you to set different minimum performance thresholds for each location where you have users.  One of the big reasons we did this is because we got a lot of feedback from our enterprise customers that they had situations where performance may be great in the US for example, but poor or completely down for users in another region, like Europe.

In the previous version of Distributed Probes and Diagnostics, any issues with health probes, performance, or any of the items in the diagnostics checklist was written to the local event log.  You could then monitor the event log in each location where you have it installed to find out when there are issues at a particular location.  That also proved to be pretty helpful if you had to open a support case with Microsoft because of connectivity issues.  They will typically try and triage the issue by looking to see if there are local network issues, versus an issue with the Office 365 service.  By using the Distributed Probes and Diagnostics feature, you can quickly check the event log on the machine(s) where it’s running and if any local network issues were found, it will be logged in there.  That notifies you if an issue is found, saving you a call and allowing you to focus on the real problem, or else validate with support that your local network is fine.

Our new update has all of the same features I’ve described above you’ve come to depend upon, but we’ve also built some very important new pieces to complement it.  Now, in addition to logging data to the local event log, as long as you have a working Internet connection it also reports and sends out alerts through our cloud service.  This opens up some very interesting data points, both from a reporting perspective as well as notifications.

New Reports

When you configure the Distributed Probes and Diagnostics feature now, you are asked to enter the ZIP code where the computer is located on which you installed it.  We use that data for both local and regional geographic data that feeds into the new reports that have been built for the service.  Overall we added 10 new reports to the service to accommodate this new data stream – two Basic reports and eight Advanced reports.

During the beta phase for this release we had the feature running in 8 different countries and more than a dozen locations.  From that data we can create a performance heat map across the globe from all of our customers that are running this service:

 

The picture above shows data from locations in the UK, India and Australia.  You can tell based on the intensity of the color around the push pin which locations are performing worse than others.  For example, Australia has the most intense colors and France has some of the lightest colors, so you can tell at a glance that you have much worse performance in Australia than France.  That’s going to be pretty important to know when supporting your Australian users.

We also create bubble maps to represent the performance in different locations for your Office365Mon subscription.  This gives you another “at a glance” snapshot of what how things are going in different locations.  The key distinction here is that in the report above, you get to see what the data looks like across the globe for all Office365Mon customers; the bubble map lets you see the performance just for the locations associated with your Office365Mon subscription.  That gives you the capability to compare how others are doing in a particular region relative to your users.  If you see a negative difference between them then that may indicate that you have problems in your network in those locations that should be addressed.

Here’s a screenshot of that report, where we’ve drilled down to see just locations in the US:

 

Here we can see that folks out West are getting much better performance than their counterparts in the East.

The graphical maps are a great way to use an “at a glance” view of the performance for your user base, where ever they may be located.  We also offer more traditional views of this data as well though, so you can quickly compare performance on each computer where you’ve installed the Distributed Probes and Diagnostics agent, as shown here:

 

In our case, we had the agent installed in a LOT of locations, so you see a lot of data there.  Again, the number of locations in which it’s installed is completely up to you.

Of course just as important as performance, we definitely have seen scenarios where the service as a whole may be up, but individual regions may be down.  A good example of this is the handful of times a few months ago when there were problems with Azure Active Directory in certain European regions.  Since our cloud service currently runs out of data centers in the US, it did not have any issues connecting to the service because the regional Azure AD services it uses were working.  However, our customers that had the Distributed Probes and Diagnostics agent running in Europe were able to find out first that there was an issue over there, because the probe and authentication process occurred there, where their users are.

We also saw this occur at times during the beta for this release, and you can see that reflected in the new availability reports.  They show availability based on the agents where the Distributed Probes and Diagnostics feature is installed; here’s a screenshot of that:

 

 

New Notification Capabilities

While we’ve added a bunch of new reports, we’ve also vastly improved upon the notification capabilities.  As I was describing earlier, in the previous release of Distributed Probes and Diagnostics, all notifications went exclusively to the local event log.  We still do that, but now these events are also wired up to go out to our cloud service as long as you have a working Internet connection.  Just like you might expect, you get notifications for the same kinds of things you get from our cloud monitoring service – when outages start and end.  But now you are getting those notifications from a specific location, so you can know right away if the service overall is up, but just one or two locations are down.

We also send notifications when the performance for a health probe doesn’t meet the threshold you had defined.  So for example, you could define a threshold of 15 seconds from Melbourne, Australia and 8 seconds from Glasgow, Scotland.  If it takes longer than the threshold you’ve defined for that location, then you’ll get notifications to all of the “channels” that you’ve configured for your Office365Mon subscription – emails, text messages, and webhook data if you have that configured – that indicate the issue and where it’s occurring.  You really will have an up-to-date, around-the-world view of your users’ ability to connect to Office 365 in a reasonable time frame.

Get Started Now

This feature is available to use now for all Office365Mon customers that are either in their 90-day trial period, or that have the Enterprise Premium license.  We hope that you’ll give it a try and, as always, let us know how we can improve upon it.  The features in this update were all driven by feedback from our customers so it DOES matter when you make suggestions.

To get more information on this feature, see our original post about it here: https://samlman.wordpress.com/2015/09/28/announcing-the-availability-of-office-365-on-premise-health-probes-for-office365mon-customers/.  To get get the documentation and agent, visit the Distributed Probes and Diagnostics configuration page on our site here:  https://www.office365mon.com/Configure/OnPremProbes.

Thanks from sunny Phoenix,

Steve

Preview Now Available to Monitor Skype for Business at Office365Mon

Today is a day that we’ve been waiting on for a while. In the past 15 months we’ve been building out a pretty comprehensive service centered around monitoring SharePoint Online and Exchange Online in Office 365. Thanks to some new APIs from Microsoft, we are now happy to announce that we’re adding Skype for Business (SfB) to the suite of products you can monitor with Office365Mon.

While we’re still in preview with SfB you may notice an occasional glitch here or there, but it’s been running in our labs for well over a month now and we’ve had pretty good luck with it. It fits into the same proven architecture that Office365Mon has been using since launch. That means – as always – that we don’t ask you for a username and password to monitor SfB. You simply log in through Azure Active Directory, and when you’re done it hands off an access token to us that we can use. At this time we will be providing monitoring for Skype Presence and Skype Instant Messaging. As the scope of the APIs that Microsoft has for SfB expands, we will also expand our offering into other features of the service, such as online meetings and voice.

Although we’ve always recommended a separate service account(s) to use for monitoring Office 365, with SfB it’s really a must. Because of the way we use the APIs to check presence and instant messaging, if you try using the same account you use at work every day to monitor these services, you likely will end up with a bunch of “stuff” going on that would be quite annoying, plus it would interfere with our ability to accurately monitor the service. To that end, we recommend you use the same sort of process that we outlined in our blog post for monitoring multiple sites and mailboxes: https://samlman.wordpress.com/2015/11/02/how-to-monitor-multiple-mailboxes-and-sites-with-office365mon/. In short you will a) create a new account for monitoring, b) give it a SfB license, c) add it as an admin to your Office365Mon subscription, and d) log into Office365Mon with that account and enable the Skype for Business monitoring.

Enabling monitoring for SfB is about as simple as it gets; here’s a screenshot from the configuration page:

SfBPreview

As you can see, all you have to do is click the Enable button to get things going – that’s it. This is also in line with how we’ve built our solutions at Office365Mon – as simple as possible, with nothing to download and install. There is one thing to remember when you click Enable the first time – you may get prompted by Azure Active Directory two times instead of the normal one, to consent to allow Office365Mon to have access to Skype resources for the account you are using for monitoring. That’s okay, it’s just because of the way the Skype team designed their service.

After that you’re off and running. We’ll automatically add the data to the reports you see for things like outages and recent health checks. You’ll also see the data show up in your My Info page next to all of the other resources we’re monitoring for you:

SkypeMyInfoReport

Finally, there is one other thing worth pointing out. Because of the way the SfB service is designed, there are times when it will be unavailable for monitoring. As we deploy monitoring for it as a Preview feature, we’re continuing to work on alternatives to minimize the alerting and configuration changes that may be needed as a result of SfB changing to an unmonitorable state. This is something that we’ll continue to work on over time, as well as await changes in the SfB architecture that will eliminate these issues.

This feature is available in Preview now for all of our customers to try. Also remember that all new customers get this along with every other feature we offer free for 90 days. So give it whirl and send your feedback our way.

From Sunny Phoenix,

Steve

 

 

Office 365 Search Monitoring at Office365Mon

Today we’re announcing a new feature at Office365Mon that our customers have been asking about for quite some time.  We’ve added the ability to monitor the Search service in your SharePoint Online tenant using our well-established health probe architecture.  This has been frequently requested because Search in SharePoint plays such a pivotal role in content delivery.  There are many out of the box web parts that depend on successfully executing queries to generate the content for the page.  On top of that there are many, many custom applications that are dependent upon the Search service working correctly.  From a developer perspective this has been an approach advocated by Microsoft for several years (including myself when I was in that role) because of the reduction in load it puts on the SharePoint farm as well the ability to pull data from a variety of sources.

When we were first designing this feature it primarily focused on the query aspect of the Search service.  However, after giving a sneak peek at what we were planning with several of my former colleagues at Microsoft, they felt just as passionate (if not more so) that we should also see what we could do about monitoring crawl performance.  Apparently quite a few of them have had customers frustrated about not seeing the content they expected when using Search and in many cases found out that their content had not actually been crawled yet.  There were some challenges in managing this request, but we found a solution that we’re happy with and causes a minimal amount of friction for our customers.  Best of all, like all our other features, I think you’ll find this extremely easy to set up and use.

 

What Does It Do?

Here’s a screenshot of the configuration page for the Office 365 Search Monitoring feature so you can see just how easy it is to configure what it does for you:

The Office 365 Search Monitoring feature allows you to provide a custom Keyword Query Language (KQL) query that we will execute against the site we’re monitoring for you with Office365Mon.  We have a link to the KQL guide when you go in and configure Search monitoring, and it can be something as simple as querytext=’sharepoint’.  As you’ll see if you look at the KQL reference though, the beauty of this is that you can actually get quite sophisticated in your KQL.  You can do things like control how many search results are returned, select a set of properties to return, use different ranking models, enable stemming, enable phonetics, etc.  One of the big reasons why we chose to use KQL directly is because so many of you have written us about custom applications you have built on Search and you want some means to monitor them.  By letting you use KQL, you can use any query that’s relevant to what your app does, and we’ll use that as the basis for monitoring.

Once you have your KQL, we take over from there.  We really monitor three things around the KQL query you’ve provided, based on the feedback and requests we’ve received the last several months:

  1. Query latency – you define a maximum query time, and if it takes longer than that for us to get results back from running your KQL, we send out a notification to all of your configured notification options. That includes emails, text messages, and of course – now – webhooks.  If you have apps that are based on the Search service it can be critically important to know when queries are running slowly – based on a latency you decide is needed – so you know whether there are issues with your app, or issues with the Search service that your app is using.
  2. Search results change – you can receive notifications when search results change. What this means is that if the set of results changes from the last time we ran your KQL, we’ll send out notifications.  If the only thing that changes is the raw rank value of the items in the search results, we will NOT send out notifications.  However, if all of the search results are the same, but the order of them changes, we DO send out notifications.  It’s also important to underscore that this reflects only on the search results themselves – it has nothing directly to do with the underlying items in the search results.  What that means is that if you change a document that’s included in the search results, we don’t detect and notify for that.  However, if that change caused a change in the search results – either the document no longer shows up, it shows up higher or lower in the search results, etc. – then we would send out notifications.

There are a couple of other things worth noting here as well.  First, we don’t store your actual search results.  This is consistent with how we do things here at Office365Mon – we don’t store usernames and passwords, and we don’t store your data.  What we DO store is a one-way hash of the search results.  When we do another search, we create the hash on the latest results and compare it to the hash we had before, and if they’re different we know the search results have changed.  It’s a one-way hash which means that it cannot be “reverse engineered” or otherwise tampered with to rehydrate the actual search results.

The second thing worth noting here is that in our experience, you may see the search results change quite frequently, even if there have not been any changes to the site content.  This is not a bug or issue with the Office365Mon Search Monitor feature.  We find that there may be hours and sometimes days when the search results come back exactly the same.  Then there are times when you may get different search results on five or six queries in a row.  Again – using the same KQL and the site content has not changed – but the search results ordering changes.  This is something you should be aware of when developing your applications based on the Search service.  If you need to know when it happens, now you can use our Search monitor feature to be alerted to it.  To help you parse through the changes, you can also elect to check the box to Include results in notifications when results change.  When that option is selected, you’ll get the actual search results we received in both emails and webhooks – just not text messages.  Again – these results are not saved anywhere, we are simply forwarding to you what we received.

  1. No search results – the other option you have is to get notified when no search results are returned from your KQL. Again, whether you’re using out of the box web parts or a custom application, if you have content in your site that’s dependent upon getting search results, this can be a critical notification to have.

Finally, as described above, the other thing we monitor is content indexing.  With a simple checkbox, you can have us monitor your search index to see how long it is taking to crawl new content.

With all of the features described above, there are several reports that you can also use to stay on top of how things are doing.  More details regarding the reports are described below.  Also, if you want detailed step-by-step instructions for configuring Search monitoring you can get them from our site at https://www.office365mon.com/configuresearchmon.pdf.  One final point worth making – you can also do all of the configuration programmatically using our Subscription Management API.  See our latest API documentation for details and code samples.

 

How Does It Do It

The Office 365 Search Monitor uses the same super-scale health probe infrastructure that Office365Mon has been using since Day One.  That enables us to issue and track query responses to your tenant.  To support crawl monitoring though, we had to come up with something a little different, and here’s why.  The monitoring applications we use are all defined in Azure Active Directory, and as part of that definition we describe what rights our applications need.  We always use the least invasive permissions possible to get the job done, so all of our apps are configured with the smallest amount of Read Only rights that we can get away with.  To do crawl monitoring though, we needed a way to determine how long it takes to get new content indexed – so how do we do that?  Well, we need to write a small amount of data to the site we’re monitoring, and then start issuing queries for that content.  We look at when we wrote the content into the site, and how long it takes until it starts showing up in search results, and that’s how we calculate the time it takes to index the content.

As described above though, all of our applications are configure to only have Read rights, so how can we write content to a site?  That’s where we had to add a new item to our toolbox to make it happen, and what we decided to do is to write a SharePoint App.  Yes, the same SharePoint Apps many of you develop to bring your site to life, we built one as well.  We wanted to limit our scope as much as possible, so the app only has rights in the current site it’s installed in – not any other site (i.e. SPWeb) in the site collection, nor any other site collection in the tenant.  When you first configure the Office 365 Search Monitoring feature, if you elect to Monitor search index freshness, the first check we do is to ensure the application is installed in the site being monitored.  If it’s not, we let you know so you can go install the app and try saving your Search monitoring configuration changes again.

The SharePoint App is in (or soon will be in) the Office Store under the name Office365Mon Search Monitor.  You can install it from there into the site being monitored.  In addition to that, since some folks turn off access to the Office Store, when you go to our Configure Office 365 Search Monitor page, we have a link you can use to download a zip file containing the SharePoint App.  If you go that route you’ll need extract the .app file out the zip file, upload it to the App Catalog in your tenant, and then you can install it in the monitored site.

When the app is installed and you save your search monitoring configuration, we’ll look for a custom list we use to store the data used for monitoring the search index freshness.  If it doesn’t exist, we’ll create it.  After that we use the app to create new content in the list so we can monitor how quickly it’s being indexed.

 

Reporting on Search Monitoring

We’ve built a half-dozen new reports based on the Office 365 Search Monitoring feature.  We have a very full list of Advanced Reports now with your Office365Mon subscription, as you can see in the Report Gallery here:

You can get recent data (i.e. from the last few hours) on query and crawl latency.  Here’s an example of the recent crawl latency report:

You can get daily stats on query and crawl latency – here’s an example of the daily query latency report:

Don’t be fooled by the numbers either – they show exactly why you want to monitor your query latency.  You can look at the graph and see overall the queries are returning data in sub second time.  However, it’s not uncommon to see this, yet still get notifications about queries that have taken 15 or 20 seconds or longer.  That’s what you want to know – your queries normally perform one way, but when they are significantly different it may be impacting the content and performance in your site.

We also have reports that show your monthly query and crawl latency averages, and they are overlaid on top of our service-wide averages so you can see how your performance is compared to your peers that are being monitored by Office365Mon.  In addition to that, in the Basic Reports you can see data just on the average crawl and query latency across our entire service:

In addition to our out of the box reports, you can also use our Report Data API to programmatically retrieve this data via our REST endpoints.  For those of you using our Power BI integration, you will also automatically see monthly crawl and query freshness data show up in Power BI after you refresh your data in there.

How Do I Get It

All existing Office365Mon customers always get all new features free to try for 90 days; everyone has had the feature turned on.  In addition, all new customers also always get ALL our features free to try for 90 days.  The feature is currently in beta, but rolled out now so everyone can begin using it.

That’s it – a quick run-down on a feature that we believe many of you will find extremely useful.  It has great value whether you are using out of the box web parts, or you’ve developed your own custom applications built on Office 365.  As always, if you have suggestions or ideas on how to improve this or any other feature at Office365Mon, please just drop me a note at support@office365mon.com.

From Sunny Phoenix,

Steve