Testing tools in 2018: Which one is the best?

In the last decade, the term beta user has become very popular – even as a business, and in some cases it has become a job – for the beta user, which refers to the test user between the programmer and the end user.

Introduction to Testing Tools

Testing tools have a funny story; since the 1980s, game programmers like John Romero have had special codes to quickly pass levels or to go straight to the final stage of programming. We consider that they were, in fact, very rudimentary testing tools, but we must bear in mind that the cost and capacity of computers at that time (we have witnessed this) prevented, in practice, the automation of testing.

In this 21st century, computers soon began to exceed the capabilities of their owners and the computing power allowed us to use them beyond ordinary use. One of these tasks is the execution of field-oriented testing tools in monitoring. Testing tools are used, for example, in White and Black Box Testing in Monitoring, and in the article on web performance optimization we listed some of them (“Selenio, TestingWhiz and TestComplete”). Of course, Pandora FMS and the flexibility that makes it unique combines all of this in the UX PWR Monitoring. We invite you to read these publications and to return to this article for more in-depth testing tools.

What are testing tools?

With the arrival of software development automation, it became clear that we would need more than just a beta user to achieve international “maturity” standards. We need a programmer beta user, in other words, a person who has the ability to “speak” various programming languages as a basis and then choose the most appropriate testing tools for each case, in order to automate software testing to keep up with the ongoing development of software.
Please note that in this article we will not recommend any particular software, because even our homemade tool needs Pandora FMS to deliver the data, so that it can be converted into information and generate the necessary alerts, if necessary, from the monitoring point of view. So each case and need is very particular.In honor of justice, we will list them according to the year their first version was released to the public, and I am sure you will like our list.

The best testing tools in 2018

Unified Functional Testing® (UFT), year 1998

“Micro Focus®, a company founded in 1976 by Brian Reynolds in the UK, launched this tool in 1998, which still uses ” innovative ” technology: VBScripting. It is proprietary software that runs on “Microsoft Windows”® (GNU/Linux had been released just 4 years ago and still had a long way to go), but let’s not underestimate this company for that reason, instead, it has been modernized according to the passage of time. In the same year, “Micro Focus”® acquired Intersolv®, in 2009 it acquired the legendary Borland® and in 2014 it acquired “Atachmate Group”®, who in turn owned Novell®, NetIQ®, and SUSE®. To top it all off, in 2017 it merged with HP Enterprise®, a division that serves only business software needs – a huge powerhouse with many pounds sterling under its belt!

We are not here to make financial evaluations of companies; however, the experience of the software company is unavoidable and we could not help but mention it. This experience is very well charged on their product and just as we pay money for their license, we will “spend” time on their installation and user training, both complex tasks. As an extenuating factor, the beta user should not have great programming skills; VBscript is a technology with years of experience in the market and a lot of trained people who handle it; the offer is large and it lowers our employee budget. It has a web user interface, API, desktop version and web application. It has reusable test components, automated documentation and is integrated with continuous development tools such as Jenkis (which we recently introduced in another article). Additionally, as a complement, UFT offers Lean Functional Testing (LeanFT) for test creation with Visual Studio (C#) or Eclipse (Java).

TestComplete®, year 1999

This is another proprietary tool, along with “SmartBear Software”®, which also uses VBScript and also handles C#Script, C++Script, DelphiScript, JScript, JavaScript and Python. In spite of all the interpreted languages it supports, extraordinary programming skills are still not required and its installation is relatively simple. It has three main modules: a desktop, web and mobile application. In many ways it looks like your UFT opponent, as it also integrates with Jenkins and keeps scripts or recorded objects up to date when changing any of them, which saves time and effort.

In this last point we must say that TestComplete offers keyword-driven testing. In keyword-based testing, each keyword corresponds to an individual test action, such as a mouse click, selection of a menu item, keyboard clicks, opening or closing a window, or other actions. These are the initial steps, and little by little, we will have to configure, and then create a keyword test, which is a sequence of operations, in keyword format, that simulates the user’s actions in the tested application. Basically, to perform any test action, evaluators simply drag and drop the keyword that corresponds to the desired operation or they can simply record their actions and the test directed by the keyword is built for them. These keywords can be reused endlessly to prevent this “Don’t repeat yourself” principle . It also offers flexibility when changing the application to be tested and for new beta users who come to work it will be more intuitive to maintain the created tests, to gain experience and to create new keyword-driven tests.

Apache JMeter, year 2001

One of the tools for testing databases that work with web pages, written entirely in Java language and aimed directly at programmers. If we need to test the massive data load and perform stress tests to the servers, we must use JMeter with a monitoring software like Pandora FMS (do not hesitate to contact us). Apache JMeter is not a web browser (it does not execute JavaScript in HTML code, for example) but works at the level of HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET, among others), FTP, SMTP(S), POP3(S) and IMAP(S) protocols. It also offers options for recording results in a simple text file, since the monitoring functions were removed from version 3.2.

Rational Functional Tester, year 2002

The company Rational® Software initially launched it as “Robotj 1.0” and when IBM® acquired the company in 2004 it simply renamed it “Rational” ® (although its logo includes the word “software” in reference to the parent company). It is the deepest and most complex of the testing tools and we dare to say that it is ineffable. The scripts are saved in Java and Visual Basic.NET, although from version 8.1 onwards you can interact with the screenshots that are recorded on a storyboard (something like WYSIWYG). It is available for both Windows and GNU/Linux vertically with “IBM Jazz integration”, like all the products of this giant company.

Selenium, year 2004

Here we go with our favorite: it runs on Microsoft Windows, GNU/Linux and OS X; it supports Java, Groovy, Python, C#, PHP, Ruby, and Perl; it is one of the web application testing tools that integrates with Chrome, Firefox, Microsoft IE and even browsers without a graphical interface, and as if that were not enough, it is free software!

Selenium is so advanced – full programmer skills are needed to use it – that it became a development platform for other test tools such as Katalon Studio, Watir, Protractor, and Robot Framework – what’s its downside? Expensive labor: to install, program and to adapt it to the needs, you need highly trained and difficult personnel. For anyone who has any doubt about our claim, Selenium has its own scripting language called Selenese (they inexplicably also used that name for the “Selenium Remote Control” protocol), so we can either save our scripts in their original Selenium supported language or in their specific Selenese language for compatibility. The components of Selenium are: IDE, Client API, Remote Control, WebDriver and Grid. Supports integration with Jenkins and Cruise Control.

Robot framework, year 2005

Presented as a Master’s work by Pekka Klärck and developed in the company “Nokia Siemens Networks” that same year, since version 2.0 was changed as free software (year 2008) and its version 3.0 came last year. Written in Python and with Apache 2.0 license, it is possible to use it mainly with Java and.NET (with plugins you can use Perl, Javascript and PHP). Like many other Python libraries, it is installed via PIP, and Selenium Web Driver libraries must be installed in a second step; then we can install additional libraries for MongoDB, FTP, Android, Appium (an application specialized in running scripts under Android and iOS) and many more. Robot framework also uses keyword-guided testing.

Watir, year 2008

As we mentioned, Watir uses Selenium and inherits its license and many of its features; however, it only supports Ruby language. Initially developed by Bret Pettichord and Paul Rogers and an enthusiastic community of developers, it focuses on automating web browsers as a normal user would and on manipulating all HTML elements. As with Selenium, deep programming knowledge is required and its installation takes the form of “RubyGems” as a package manager.

Katalon Studio, year 2015

Its parent company, KMS Technology, was jointly founded in 2009 in the United States and Vietnam, and after three years of development based on Selenium and Appium with support for Groovy (Java) was released under the name “Katalon Studio”. The highlight of Katalon is that instead of downloading and installing Selenium and Appium separately, Katalon offers it in a single package and also enhances the user experience, allowing for fast learning. Once we’ve mastered it, we can go all the way with Selenium, which runs in the background, dormant. It also supports ongoing software development with Teamcity, qTest, JIRA, Jenkins, and Git. You can read about the many benefits, actually, in this link in detail.

Conclusion

This is a list that obviously does not include all existing software – nor does it pretend to be exhaustive – but reflects the current situation of the software market, which is in excellent health with a range of prices and features – which one will be the most appropriate? We recommend the use of virtual machines, because of their unique features, so that you can carefully evaluate each of the programs and have a complete picture based on specific tests. Once you have chosen the best candidates it will be time to present them to the development teams on real computers and then select one or more, so that your company can acquire a service and/or use license, leave your comment here below and we will answer any questions, doubts or comments!