[ #ESPC15 ] part 3 : SharePoint Online performances…

Third article of a series on what we learnt at [ #ESPC15 ]: Keynote and 2016 Roadmap Delve Analytics worldwide announcement SharePoint Online performances More sessions on PowerShell and OneDrive, and see you next year !

This was the whole subject on the session we presented with Serge Luca my fellow MVP from Belgium.

Here are the slides:

ESPC15 T8 – SharePoint Online performances – Serge Luca & Patrick Guimonet

I wanted here to come back to some of interesting points about SharePoint Online performances.

First, you have to understand the way to your data and Microsoft did a great work crafting a very good MVA course on that subject: Office 365 Performance management. You will find here this schema:

But I found that schema rather Microsoft centric. From a customer perspective I would rather represent things like that:

 

As reminded or stated in this slide, Microsoft network is one of the top 3 in the world with public peering in 23 countries with 1,500 ISPs.

So, from a performance point of view it’s of primary importance that your packets reach this network as soon as possible. This is what I called the rule #1 : the power of the backbone.

A quick demo can help us to understand how fast the backbone is. As a demo, I uploaded three files from two locations : one in Stockholm on a standard Internet access, the other from a VM hosed in Azure and in the same datacenter as my Office 365 tenant. The results are rather clear: the bigger the file is, the bigger the performance improvement.

Hence come rule #2: you should know your nearest Office 365 Datacenter and the fastest way to it (endpoint). For that, you can use several tools and sites from the simplest to the most sophisticated.

Microsoft Azure Speed site at http://azurespeedtest.azurewebsites.net is great visual tool to help identify the nearest (Azure) datacenter visually:

It’s for Azure but Office 365 datacenters and Azure datacenters are usually collocated.

Ping is less sexy but can show us some interesting infos: ping outlook.office365.com return the name of the Office 365 edge, below it’s outlook-emeasouth.office365.com which should be fine for my current location. Whereas ping to your tenant give a more cryptic server name.

Actually ping is usually blocked so psping from Sysinternals toolbox is a much better choice. It also gives histogram of the results showing clearly some variations in the results.

Tracert is the first tool to use to understand where your packets go and where they enter Microsoft network. On the following traces, we can see where they are entering the Microsoft network (nodes in the blue squares). It is in the par02 node which is probably for Paris. Interestingly, the first way (for Outlook/exchange) is going to Amsterdam nodes (am2 and amb) whereas the second way (for SharePoint) is going to Dublin (db3 node).

Microsoft do not publish a map of the edge points but Priasoft did a great job publishing a testing tool for all Office 365 Endpoints and testing them using TCP Ping.

On the below run, the tool detected that the default Office 365 Endpoint from local DNS is outlook-emeasouth.office365.com but that some other endpoint responded more quickly: outlook-emeacenter.office365.com and outlook-emeaeast.office365.com. This should be corrected if this is not a temporary situation.

 

Azure CDN are better documented here : https://azure.microsoft.com/en-us/documentation/articles/cdn-pop-locations/

With all of this, we have enough information to measure Round Trip Time (RTT) from our location to Office 365. In case of bad response times, Paul Collinge did a fantastic job describing and explaining the most common problems.

This is rule #3: if needed go to this superb resource, Top 10 tips for optimizing and troubleshooting your Office 365 network connectivity by Paul Collinge.

It should also be understood that Exchange and SharePoint do not handle loads the same way as depicted clearly on these schemas.

In both case the data are located in your “home” datacenter. For Exchange access, the Exchange front-end used is the nearest from the client, and it can be far from the back-end.

In SharePoint case, the front-end used is always situated on the same DC as the back-end. This is precisely described on the Client connectivity page here.

 

As we are (most of us at least) not dedicated network admins, it is good to know that there is a great tool that can embed most of the previous tests and measures, it’s Office 365 Client Performance Analyzer (OCPA).

Here are the results provided from Stockholmsmässan Wifi:

by clicking on “show trace route details” you can even have a copy of the tracert results for the current location :

a great thing is that it can be used to check and optimize SharePoint and Exchange connectivity:

This was rule #4 of my presentation: If you should use only one tool, use OPCA

This was the end of my part. After this first part on tuning and optimizing outside SharePoint Online, Serge Luca presented several ways to improve performances inside SharePoint online.

When we are inside SharePoint Online, optimizations are similar to what we used to do on SharePoint on-premises. So the tools you can use are standard F12 in the browser for example. The two most useful values for diagnosing performance issues are:

o SPRequestDuration

This is the amount of time that the request took on the server to be processed. This is the best insight you have into how much work the server is doing to serve the page.

o X-SharePointHealthScore

This indicates the health of the server

But a major difference between on-premises and online is that due to the fact that there are more than 30,000 VMs in Office 365 handling SharePoint workloads (see Keynote for details on that point), you can’t rely on object cache as it is clearly depicted on these schema from Using the object cache with SharePoint Online:

 

So optimizations can be put in place in different areas as described here Tune SharePoint Online performance. Especially:

• Navigation options for SharePoint Online
• Using Content Search Web Part instead of Content Query Web Part to improve performance in SharePoint Online
• Using content delivery networks
• Image optimization for SharePoint Online
• Delay loading images and JavaScript in SharePoint Online
• Minification and bundling in SharePoint Online

Last but not least and this is great difference between on-line and on premises: Throttling

SharePoint Online uses throttling to maintain optimal performance and reliability of the SharePoint Online service.

• Throttling limits the number of user actions or concurrent calls (by script or code) to prevent overuse of resources.
• For requests that a user performs directly in the browser, SharePoint Online redirects you to the throttling information page, and the requests fail.
• For all other requests, including CSOM or REST calls, SharePoint Online returns HTTP status code 429 (“Too many requests”), and the requests fail
• If the offending process continues to exceed usage limits, SharePoint Online might completely block the process; in this case, you may see HTTP status code 503 (“Service unavailable”), and we’ll notify you of the block in the Office 365 Message Center. The error message is:

Here is a great schema of this coming from How to: Avoid getting throttled or blocked in SharePoint Online

Here is a great piece of code coming from Office 365 Developer PnP (Patterns and Practices) https://github.com/OfficeDev/PnP/tree/dev/Samples/Core.Throttling that can be used to avoid throttling:

 

We can see that each time an error is trapped, the thread will sleep. And the sleeping time is multiplied by 2 each time.

This ultimately dictates how we should deploy in SharePoint online without performing traditional load testing since it is not permitted. From the doc Capacity planning and load testing SharePoint Online:

 

With SharePoint Online you don’t have to do capacity planning, as this is done for you as part of our service offering. With on-premises environments, load testing is used to validate scale assumption and ultimately find the breaking point of a farm; by saturating it with load. With SharePoint Online we need to do things differently. Being a multi-tenant environment, we have to protect all tenants in the same farm, so we will automatically throttle any load tests. This means you will receive disappointing and potentially misleading results if you attempt to load test your environment.

 

 

One of the main benefits of SharePoint Online over an on-premises deployment is the elasticity of the cloud. Our large scale environment is set up to service millions of users on a daily basis so it is important that we handle capacity effectively by automatically expanding farms, as and when needed. This article explains how we plan for capacity growth and scale out in place. The article also covers approaches for you to use that don’t involve load testing.

To summarize it in a few words, you have to roll out to all users in waves.

 

By the way, I will have the chance to present all of this again at Köln (Germany) at the end of this week at

The title of the session is: INTRODUCTION TO PERFORMANCE OPTIMIZATION FOR SHAREPOINT 2013/2016 AND OFFICE 365