How we've been working to make your life easier – web performance updates 2017

We’ve been busy this year. A couple of our products have seen a big overhaul. But away from the fanfare surrounding the major enhancements, there are a host of other smaller updates and bug fixes that largely fly under the radar. They are, however, key to ensuring that we can continue to deliver a great service to our customers.

So here’s a list of some of the changes we’ve made over the past 12 months, including some of those updates you might not have heard about:

Real User Monitoring

  • Performance impact prediction (read more)
    • Enables you to predict the impact a change of speed would have on conversions, revenue, bounces and pages per session
  • Device name added to session processing
    • Session data includes the make and model of a visitor’s device, providing a deeper level of detail beyond simply smartphone, tablet or desktop
  • RegEx capabilities in URL filter
    • Grouping URLs is much easier thanks to support for Lucene Regular Expressions in the URL filter
  • Impact over time report (read more)
    • This gives you a single heads up view of five key metrics, so you can quickly see the impact a change in one has on the others
  • Performance trends updates
    • The usability of the report has been improved, with a cleaner layout and a focus on a single percentile
  • Performance trends expected range (read more)
    • By analysing historical performance for a given period, we're now able to present an expected range of load times to give you more context around the performance data you’re viewing
    • We also show you when performance falls outside the expected range
  • Session length report (read more)
    • This allows you to map out (and break down) session length in terms of number of pages consumed versus average load time
  • Session performance distribution
    • The average page load time in a session can now be viewed in a distribution
  • Compare mode for session length
    • This release means you can compare session lengths for different user populations
  • Device make and model filters and breakdown
    • You can now filter report data by device make and model across all reports in the portal
    • Device make and model options were added to the main breakdown menu
  • What's changed analysis – speed (read more)
    • In the performance trends report, anomalies are analysed automatically and the elements or timing points that changed the most are displayed, saving hours of manual analysis
  • Impact over time comparison
    • A compare function was added to the impact over time report to allow you to compare two different populations across all five metrics
  • Beta – session metric trends
    • Leveraging the capabilities of the performance trends report, including expected range and anomaly detection, you can now view actual versus expected performance for conversions, revenue, sessions and bounces.
  • Front-end visibility and usability changes (read more)
    • Based on feedback from various sources, we delivered a large number of enhancements to make the portal simpler to use and the data easier to consume
  • Experiments landing page
    • This is where we release reports that are not fully production ready, so you can play with them and provide feedback at an earlier stage.
  • Device make and model added to what's changed analysis and significant elements
    • The what's changed analysis and the significant elements report were updated to include device make and model when conducting their analysis, improving their accuracy
  • Capture and processing of multi-variate testing (MVT) in session processing (read more)
    • If you have MVT on your website, RUM can now capture the data for you to use across the portal
    • The newly captured MVT information is surfaced in the portal as a filter, so you can filter performance down to specific experiments and variants
  • Alerting
    • Customers subscribed to alerts are notified if there is an anomaly in performance across the site or in statistically significant categories
  • Normalisation of distributions as percentages
    • This enabled users of our performance alerting alpha to estimate how many alerts they were likely to receive based on their configured sensitivity
  • Daily ForEx file update
    • International currency conversion rates are updated on a daily basis, allowing for a more accurate representation of revenue
  • GDPR compliance changes
    • Removing personally identifiable information from our portal by obfuscating parts of IP addresses and de-sensitising geo information


Performance Monitoring

  • Screen shots from our latest monitoring agent
    • The agent can take a full page (including below the fold) screen shot of the page
    • This can be enabled for any test, but is typically configured to capture screen shots for all steps of a user journey in error, as well as any manual test run
    • Seeing exactly what web page is loaded makes it much easier to see why a monitor has failed
  • Retrospective diagnostics
    • We now store much more information, including headers and HTML, for all steps of a failing user journey, not just for the failed steps
    • This is because a failure on a step is often caused by an earlier part of a user journey – having retrospective diagnostics makes it easier to pinpoint the cause of a failure
  • New HTML (responsive) email alerts/recoveries
    • The new format is responsive, working on any kind of email client – particularly important, as more of our customers view alerts on mobile devices
    • Improved information architecture puts context specific information into the alert in order of priority
  • CORS headers added to the monitoring API
    • This allows you to build custom dashboards that run on (for example) internal intranets that display monitoring data, such as performance or availability
    • Without these, you would need to build a client application and not be able to host it as a web application
  • New Portal Hub login page
    • Improved design gives you easier access to other services, as well as to our latest blog posts
  • Interactive Dashboard (read more)
    • Enhanced the view of the Interactive Dashboard makes it much clearer to see the latest status of your monitors
    • This is particularly valuable if you are using the Interactive Dashboard on a large ‘NOC’ screen


Performance Analyser

  • More granular scheduler (read more)
    • Prior to this, it was only possible to repeat a job every day – now it’s possible to schedule a job to run as frequently as every 5 minutes
    • This makes it much easier to understand how site performance changes through the day
  • API (read more)
    • A rich API for Performance Analyser that allows you to run new or existing jobs, access results, including object level data
    • Screen shots (both thumbnail and full size) can also be retrieved at 100ms intervals
    • Customers have used this to extract performance data and user experience data to present on internal intranets and reporting
    • The API allows you to integrate Performance Analyser data into your automated build chain, baking performance budgets into your pipelines


Load Testing

  • Overhaul of the scenario wizard (read more)
    • Improvements to make it easier for you to set up profiles and scenarios
    • It’s now also possible to configure more advanced scenarios directly from the wizard, so that you do not need to go in and edit them after they have been created
  • Smooth ramp up
    • Improved the ramp up rate by which we inject load, in particular for smaller load engagements, where websites under test can typically only support lower concurrencies
    • Removes the need for you to use custom injector selection, simplifying how you run load tests
  • Agent updates
    • More granular timing points
    • Enables retrospective diagnostics (where needed) for all steps of a failed user journey
  • Migration to AWS
    • Elastic platform allows us to deliver far bigger load tests than ever before
  • Flexible testing window
    • Offers more granularity when booking a testing window
    • Easier management of platform usage and scheduling


We have much more planned for 2018, so watch this space for more big improvements in the new year.

Published date:  18 December 2017

Written by:  Alex Painter

comments powered by Disqus

Filter By Service

Filter By Date