Speed Index – how it works and what it means
We recently enhanced Performance Analyser to include Speed Index, a measure of how quickly the visible parts of a web page are displayed.
This is really useful because Speed Index is an easy to consume metric that indicates how quickly a visitor would see a page’s content, in a way that other metrics just can’t.
However, it can be difficult to understand exactly what Speed Index means and how it works, so this post will go into a little more detail.
An abstract number
The first thing to note is that Speed Index is an abstract score, not a timing metric, such as render start time or time to onload. It’s also important to remember that a smaller score is better than a large one.
So if Speed Index doesn’t represent a point in time, what is it? And how is it calculated?
A frame by frame view
Speed Index is worked out by looking at video frames of a page as it renders.
Each frame is given a score for visual incompleteness above the fold. So the score is 100 per cent for a blank screen and 0 per cent for a visually complete page.
The next step is to multiply that score by the number of milliseconds since the last frame (or since the beginning of the test if it’s the first frame).
The final step is very easy: add these numbers together. The total is your Speed Index score!
Example 1
In this a simple example, we’ve reduced the number of frames for illustration purposes. In reality, we’d be examining ten frames per second.
The chart below shows how quickly the page in this example becomes visually complete.
Frame 1 – 500 milliseconds
As you can see, the page is blank when the first frame is captured, at 500 milliseconds.
We multiply this time by how incomplete the page is at this point. Since this is a blank page, it’s 100 per cent incomplete.
500 x 100% = 500
Frame 2 – 1,000 milliseconds
Another 500 milliseconds pass, and we capture another frame.
There’s some content on the page now. Let’s say it’s 10 per cent visually complete. In other words, it’s 90 per cent incomplete.
500 x 90% = 450
Frame 3 – 1,500 milliseconds
In the next frame, after a further 500 milliseconds, the page is 50 per cent complete (and therefore 50 per cent incomplete too):
500 x 50% = 250
Frame 4 – 2,000 milliseconds
The page is virtually finished now, at 90 per cent complete (10 per cent incomplete):
500 x 10% = 50
Frame 5 – 2,500 milliseconds
By frame 5, the page is visually complete, so we don’t add anything to the overall score (500 x 0% = 0!).
Adding up the figures
Getting Speed Index is a simple matter of adding these figures together:
500 + 450 + 250 + 50 = 1,250
It’s also easy to see on the graph showing visual completeness over time. Speed Index is the area above this graph:
Example 2
In our second example the page is going to start rendering at the same time as it did in the first. It’s also going to be visually complete at 2,500 milliseconds, again just like the first page.
However, it’s not going to display quite as much quite as early. This means we should expect it to have a poorer (higher) Speed Index score.
Frame 1 – 500 milliseconds
As with the first example, the page is blank at this point:
500 x 100% = 500
Frame 2 – 1,000 milliseconds
In the second frame, there’s hardly any content on the page. In our first example, the first page was 10 per cent finished at this point, but this time the page is just 1 per cent complete (99 per cent incomplete):
500 x 99% = 495
Frame 3 – 1,500 milliseconds
The page is getting there, but it’s still lagging behind the first page – 20 per cent complete (80 per cent incomplete) by the third frame:
500 x 80% = 400
Frame 4 – 2,000 milliseconds
At 2,000 milliseconds, the page has finally caught up with the one in the first example and is 90 per cent complete (10 per cent incomplete):
500 x 10% = 50
Frame 5 – 2,500 milliseconds
Again, just like the first page, the second page is visually complete at this point.
Adding up the figures
So how does the Speed Index score look this time?
500 + 495 + 400 + 50 = 1,445
Sure enough, because this page didn’t display as much content early on, its Speed Index score is poorer (a higher number). Again, this is easy to see on a graph:
Why is this so useful?
It’s important to remember that in these two examples both pages started rendering at exactly the same time and they both finished rendering at exactly the same time.
The difference was that the first page displayed more content sooner, delivering a better user experience and getting a better (lower) Speed Index score as a result. That’s what makes Speed Index so useful and so powerful.
What do we mean by “visually complete”?
For Speed Index, visual completeness is based on the distribution of colours on the visible part of a page in the final frame, rather than the positions of individual pixels.
This means that Speed Index isn’t overly sensitive to small layout changes. For example, imagine that all the visible content on a page were to shift down by 20 pixels in the final frame (perhaps to accommodate a cookie notification). If visual completeness were based on the positions of each pixel, the final frame would be dramatically different from the preceding one, even though it was virtually identical from the user’s perspective.
By looking at the colours on a page, we get a much better approximation of visual completeness as the user experiences it.
What are the limitations of Speed Index?
Speed Index has no insight into what’s important to the people visiting your site – it can’t distinguish between critical content and non-critical content. It can’t therefore tell you whether you’re displaying content in the right order.
What’s more, pages featuring carousels that rotate automatically may be penalised because they continue to change after they’re visually complete from the user’s point of view.
Who came up with Speed Index?
Speed Index was developed by Pat Meenan, who currently works on the Chrome team at Google. He is also the creator of WebPagetest, and you can read his own, more detailed guide to Speed Index here.
What’s a “good” score?
This is really hard to answer, not least because Speed Index is an abstract number rather than a point in time. It also depends on a whole range of factors, such as the browser and the viewport size.
However, to give you some idea of what you could aim for, in a recent test of top 50 UK retailer home pages (tested in Performance Analyser with Internet Explorer 11 at 8Mbps), the best Speed Index score was 819. The average was 3,658 (median 3,106), while the poorest had a score of 8,582. We’ll look at some more examples in a future post.
Whether or not you aim for an absolute figure, what you can (and should!) do is take Speed Index into account when making changes to your site. For example, you might want to think twice about changes that cut 0.25 seconds from total load time but add 300 to your Speed Index score.
It’s also important to say that Speed Index complements rather than replaces other metrics. You’ll still need to look at render start, when the page becomes interactive and when it finishes loading. But Speed Index does give you another perspective on the kind of user experience your site delivers.
Published date: 17 June 2015
Written by: Alex Painter
