Eggplant AI from Testplant: First impressions

Disruption. A word used to describe Testplant’s latest offering throughout its official launch at Digital Automation Intelligence Roadshow, an event hosted by Testplant in the impressive surroundings of Altitude London (Figure 1).

Figure 1: Altitude London

Figure 1: Altitude London

The gaps in modern testing

In the opening presentation by Dr. John Bates, Testplant CEO, he talked about a desire to create testing disruption, challenging the way in which test automation is done and moving towards a technology that learns and adapts its strategy over time. It was stated that almost 1 in 4 people who download an app only use it once and only 4% of apps downloaded from Google Play Store and Apple’s App Store are used after a month. In the view of Testplant there are five key gaps that are contributing to these issues (Figure 2):

  1. UX gap – Current testing is too focused on compliance rather than looking at how we can delight users.
  2. Productivity gap – An increase in application complexity alongside shrinking timescales, particularly for agile and DevOps, is making it difficult for test teams to keep up.
  3. Automation gap – Also due to the increase in system complexity, existing test tool solutions are unable to test everything that we need them too.
  4. Visibility gap – The user perception of the end product does not fulfil expectations.
  5. Confidence and Predictability gap – Quality throughout software development is often reported using metrics such as number of tests passed and failed or open defects. This level of reporting can make it challenging to interpret how shippable a product is, how it would be received by a user or whether the product is improving over time.

Figure 2: Testing needs to change - from testplant.com

Figure 2: Testing needs to change - from testplant.com

 

From a personal perspective, there isn’t anything here that I would fundamentally disagree with.  However, what I was keen to understand further at this early stage is how Eggplant’s Digital Automation Intelligence (DAI) suite (the new collective name for the Eggplant products) would address all of these gaps.

From compliance to profit

Next up at the roadshow we had Antony Edwards, Testplant CTO, who talked us through the five principles of DAI which will help to transform testing from “a compliance function to a profit centre”.

The five principles are:

  1. Test through the eyes of the user
  2. Test all aspects of user experience
  3. Expand automation beyond test execution with AI, machine learning and analytics
  4. Use predictive analytics to report quality status in terms of UX
  5. Take a coherent approach to monitoring and testing

While I can see that there are links between some of the five gaps and the five principles, I’m keen to hear more from Testplant on this connection. In addition, it was also not clear from the roadshow whether all five principles could be implemented through products within the DAI suite.

The main concern lies with principles four and five. I am personally not convinced that the products at present can effectively deal with these principles, or at least not in a way which is immediately obvious.

However there are some recognisable solutions already in place, as principle one is covered by Eggplant Functional with scripts which are recorded from a UI perspective, using image capture and OCR to drive scripts rather than the underlying HTML. Similarly, principle two is possible due to the testing tools being easier to use, meaning that non-technical resources across the project can get involved with test automation.

Eggplant AI

There is now also confidence for principle three to be resolved, as that is what Eggplant AI, the new product within the DAI suite, is all about. It enables the user (whether that be a tester, business analyst or product owner) to model an application on a Visio-like canvas, with the focus being on the states that an application can be in and the actions that can be done against those states. By mapping out the various states and actions within the application it is effectively creating a view of all the paths through the application on a single page. I should add that none of this is automated; this is a manual activity done by someone who understands the flows through the application, with the automation being managed by Eggplant Functional. Essentially, the tester will record the flows that have been modelled using Eggplant Functional and then proceed to assign snippets of automated code (generated in SenseTalk language) to each of the actions within the model.

It is at this point where Eggplant AI comes to the forefront. Rather than a standard Eggplant Functional suite which would follow a linear flow for test script execution, AI introduces variation based upon the model that has been derived for the application. Variation is executed in a number of ways it seems, including:

  • Weightings – the user can assign a weighting to specific actions which will increase the frequency of that action being exercised by the suite.
  • Previous runs – AI will look at previous test runs, and use this data to determine the coverage for subsequent runs. 
  • Defect History – AI will be able to link defects to certain areas of the application and will focus in on these depending on how the application has changed recently.

There are some bold claims in the above points and it will be interesting to see how this comes through in the product over time.

Figure 3: Eggplant AI model example

 

By providing this modelling layer and the underlying AI algorithms, Testplant are claiming to be automating aspects of the test preparation and scripting phase by automatically generating test cases continuously during execution. I can see where they are coming from here and it will be interesting to see how effective the modelling functionality is and how it could potentially be used to describe test scope to a group of stakeholders.

Testplant also claim that by introducing this variation and intelligence then they are effectively automating exploratory testing. Again, I see where they are coming from here and I agree that in part, they are. However, in my view, there will always be a need for manual exploratory testing and I’m sure that Testplant would tend to agree with me on that.

Overall I think that Testplant need to be clearer on how exactly they see DAI addressing the gaps that they have highlighted and how their products align with the principles stated. Saying that, I think they deserve real credit for trying to disrupt the market and developing a tool which does something different. I’m also excited to see how our team at NCC Group can implement Eggplant AI to help our clients reap the benefits that it offers.

Eggplant AI trial at NCC Group

Since attending the roadshow, NCC Group has been offered participation in the Early Access Programme for Eggplant AI, providing us with the opportunity to use the tool internally. Lauren Garner, one of our trainee Automation Test Analysts who has spent a week using Eggplant AI with Eggplant Functional, was enthused by her initial experience with the product.

“Eggplant AI is an incredibly user friendly piece of software and easy to get to grips with. You can start to build models and link to scripts written using Eggplant Functional very quickly, as soon as you have AI, Functional, AI Agent and the relevant gateways installed.

"Knowledge of scripting in Eggplant Functional is a must, as it’s here you develop the automated scripts that integrate with AI. Within only five days I was able to learn from scratch how to script in Functional, build models in AI and link the two to produce a basic model against the TripAdvisor Android app.

"I did of course encounter a few technical challenges along the way but found the Testplant support team very responsive. If an issue couldn’t be resolved over email, they would set up a WebEx to get a closer look at the issue and walk me through the resolution steps required.”

Published date:  10 November 2017

Written by:  Paul Bell

comments powered by Disqus

Filter By Service

Filter By Date