TEST AUTOMATION

SELENIUM AND IT'S FUTURE

Selenium is the leading framework of test automation if you’re already familiar with test automation you’ve at least heard the name before. If you haven’t, Selenium is a free and open source testing framework for web applications. In their own words ‘Selenium automates browsers”. It has the backing of the major players in the browser market such as Firefox and Chrome. A great feature of Selenium is that it is not limited to a specific language. Officially they support Java, C#, Ruby, Python, and Node. Although being open source many more languages can unofficially be used to create your automated tests.

Now as you may know software and programming languages are rapidly evolving. There’s a large competition between many of these, to become a new industry standard and or dethrone the current standard. Selenium is no exception to this and that’s what I'd like to talk about.

The only way forward, if we are going to improve the quality of the environment, is to get everybody involved. - Richard Rogers

There are a couple of new kids on the block gaining momentum. Cypress and TestCafe, the first releasing its first version in 2018 and the latter early 2019. Using GitHub stats as an indicator of popularity, both are quickly gaining on Selenium. As of today, Selenium has 14.9k stars while Cypress has already grown to 13.2k, and in only a handful of months, TestCafe has 7k.

Being an older technology is no reason to make a switch, just take a look at MySQL or Java both from ‘95 or C# in 2002. There has to be more to the rise of these Selenium competitors, what is it?

While selenium is still maintained and used by large corporations, many believe there exists a better alternative. Why do they think this? Well, let’s start with some of the cons of Selenium.

    CONS
  • Demands a higher technological skill set; you need to be fairly familiar with one of the coding languages supported.
  • It’s not an all in one test suite platform. Depending on the language you could need an assertion library, a reporting tool, and other ALM solutions.
  • Limited element locators
  • Slow performance
  • With its open-source status, there’s nobody providing reliable technical support.
  • Configuration can be a hassle. Think Getting/installing drivers for browser support.

Before diving into the specifics of each, let's take a high-level look at some of the similarities between them.

    Similarities
  • Open source but with dedicated and responsive developers
  • Written in Javascript
  • No manual timeouts/transparently retry assertions
  • Under heavy development by responsive developers
  • The configuration is fast (under a minute)
  • All in one automated test solution, no 3rd party dependencies required.
  • Blazingly fast

Now, these are just the similarities between them, and already they provide solutions to many of the cons of Selenium. Let's take a look at each of them individually. We’ll look at how they differ from Selenium, take a look at how they work, and any cons.

Cypress

Straight from the documentation, Cypress describes itself as “A next generation front end testing tool built for the modern web.”, and it shows. When trying out Cypress my initial first impressions were great. It’s intuitive and the UI component is beautiful. The platform has been adopted by major organizations like NASA, DHL, Autodesk, and many more. The set up and configuration is incredibly easy. It comes with an optional GUI dashboard allowing for easier debugging and screenshots. The biggest con for me is the lack of browser support.

Here's an example of a test written in Cypress
describe('My First Test', function() {
  it('Gets, types and asserts', function() {
    cy.visit('https://example.cypress.io')

    cy.contains('type').click()

     // Should be on a new URL which includes '/commands/actions'
     cy.url().should('include', '/commands/actions')

    // Get an input, type into it and verify that the value has been updated
    cy.get('.action-email')
      .type('fake@email.com')
      .should('have.value', 'fake@email.com')
   })
})

TestCafe

With version 1.0 only being released in February 2019, TestCafe is extremely new, but moving at a rapid pace. The development team is incredibly responsive, and it shows with its 7k+ stars as of this writing on Github. The team has a great feature roadmap, that shows great progression. Part of what I like about TestCafe is the setup and test creation, both are stupidly simple. No need to jump deep into documentation, although I have to mention how great it is laid out. I’ve yet to not find a solution to a problem. Did I mention it’s framework-specific selectors? TestCafe provides general purpose selectors based on CSS and JS, however, they also provide selector libraries for popular front-end frameworks such as React, Vue, and Aurelia.

Here's an example of a test written in TestCafe
import { Selector } from 'testcafe';

fixture `Getting Started`
    .page `http://devexpress.github.io/testcafe/example`;

test('My first test', async t => {
    await t
        .typeText('#developer-name', 'John Smith')
        .click('#submit-button')

        // Use the assertion to check if the actual header text is equal to the expected one
        .expect(Selector('#article-header').innerText).eql('Thank you, John Smith!');
});

Head-to-Head Comparison

  Selenium Cypress TestCafe
Simple test creation
Simple setup/configuration
Cross Browser support
Requires 3rd party libraries
Multiple element selector libraries
Fast test execution
Open source
Responsive Development team
Write tests in multiple languages
No manual timeouts required
Optional GUI
Major organization adoption
Easily Understandable/Laid out Documentation
Easy integration with cloud-based emulators (Saucelabs, Browserstack)

Conclusion

In conclusion, It’s my opinion that you’ll see the popularity of both Cypress and TestCafe continue to skyrocket. In frontend testing, Selenium just can’t compete with their speed in both test creation and runtime. Having tried both, I’d give my recommendation to TestCafe. The biggest reason being the cross-browser support, including its ease of connecting to services like Saucelabs. The libraries it has for different front-end framework selectors is a huge plus, and while it’s totally my opinion I find the code to be much more intuitive. Its command line debugging is also great.

That said Selenium will always have its place. In reality, it can be used for more than testing and was designed for multiple languages. I myself have used Selenium on an experimental web scraper, something that has nothing to do with testing. Another time I built Selenium into an Electron app. The point being at this moment in time there are valid use cases to still use Selenium.