Regression nightmare
If you’re a tester, then you are probably familiar with this situation when dev informs you that he made a small change in the feature you were just finishing testing. You try to acknowledge it with dignity, but after thinking about all those regression tests that await you, on the inside you feel like this:
Regression tests are very time-consuming, especially when the product constantly grows bigger and bigger. There’s also the risk that you may miss something if you don’t have scripted test scenarios that cover almost all use cases. Wouldn’t it be great if the majority of those tests would be done automatically? Well, here’s some good news for you – it can be done! And it’s easier than you may think.
What are automated tests?
Automated testing is a powerful concept that allows you – by using separate software – to perform self-acting tests and compare their actual outcome with expected results. This kind of test can automate some repetitive, low-level UI checks, but this approach also works perfectly with more sophisticated end-to-end tests.
Automated tests have several advantages over manual testing. One of the greatest things is that once they have been developed, they can be run quickly and repeatedly. They can also save a lot of money in the long term because more tests can be performed in a short time (compared to manual testing) and potential defects can be found quicker. In my opinion, automated tests are more interesting, because you can focus your attention on creating and upgrading already existing test scenarios (e.g. by randomizing test data or adding more complex use cases), instead of on performing the same test manually over and over again. One more thing worth mentioning is that tests’ results can be seen by every team member involved in the project, as you can automatically create test logs – for example, you can include what each suite is testing, what test data was used or what type of error occurred and failed the test.
Testing web applications with Selenium
Automated tests can be run on any type of application – desktop, web or mobile. I would like to focus more on testing web applications and briefly describe how to easily create the first automated test by using Selenium IDE – a simple tool that allows you to record, edit and debug tests.
After successfully installing the Selenium IDE plugin in Firefox browser and launching it, you should see a new frame appear with a red button in the upper right corner turned on by default. It means that from now on all your actions performed in the browser will be recorded and they can be recreated step-by-step later on. For example, if you now type “google.com” and try to search for our company’s blog, then click on the first link and on any article available on the blog, you should get something like this:
Now you can just press the Run button (make sure to slow down the test execution by moving the slider in the upper left corner of the interface) and… voilà! Congratulations, you’ve just performed your first automated test. Nothing fancy, but it’s something. If everything went as expected, you probably noticed that the browser reproduced your previous actions and each row in Selenium has a green background which means that the test was passed.
Selenium has plenty of useful features, like the Select button, a dropdown list with available selectors for each element, a vast number of commands that can be performed, stack trace or options in the contextual menu that help you create tests effectively. You could stop at this point and make, upgrade or maintain all tests via the Selenium plugin. However, I’m sure that after some time you would notice a couple of hindrances that will prevent you from using the full potential of automated tests. First of all, you’re limited to only one browser. Secondly, you cannot create any conditionals or loops. Another issue is that you cannot easily randomize test data and on top of that, maintaining the tests can be really painful.
That’s when exporting test cases comes into play.
Exporting test case to Python
One of the coolest things offered by Selenium is that you can export recorded tests to one of the following languages: Java, C#, Ruby or Python. Below you can find a screen to demonstrate where to look for this option. In this case, I selected Python 2 / unittest / WebDriver which means that the exported file will use Python unit test framework to run the test and it will be performed by the WebDriver. If you are interested in the differences between WebDriver and Remote Control, please refer to SeleniumHQ site. TL;DR: use WebDriver, it’s better.
Configuring the environment
In order to run automated tests using Python, you have to first download and install Python 2.7. The installation process should introduce paths to the language interpreter and scripts folder so that you can use “python” and “pip” commands as environment variables. You can check that quickly by running the following commands:
python --version
and
pip --version
If either of the commands is not found, you will have to add the path(s) manually.
The last thing you need to do is to install Python bindings for Selenium. Run the following command:
pip install selenium
After a successful installation, you’re ready to go – you have all the basic and necessary configuration elements. In other words, you can now launch your first automated test in Python.
Let’s run it!
If you want to run the test, you have to go to the directory with the file exported from Selenium. Then simply run the following command:
python <file_name>.py
In just a few seconds, Firefox should launch and all the steps included in the exported file should be recreated. When the test is done, the browser should automatically close and you should see something like this in your command window:
But there’s also a great chance that the test will fail and it won’t execute all the steps. If the error was thrown, somewhere in the middle of the error message you should be able to find the following text:
NoSuchElementException: Message: Unable to locate element: {"method":"link text","selector":"Blog About Software Development, UX Design ... - DeSmart"}
It means that the link directing to the blog couldn’t be found on Google search results page. But why is that? It’s definitely there! The answer is simple – the test was executed too quickly. Nowadays, the content of many websites changes dynamically when users interact with it; the same goes for Google search site. In order to adjust the pace of test execution to the web application behavior, you have to indicate where the test scenario should “take a break”. How to do it? In the Selenium plugin, there’s this slider that allows you to change the speed of execution. In Python, you have to modify the code of the script. So let’s open the file and see what the test looks like.
What’s inside?
Each Python test file that runs on unit test module, consists of several important parts that need to be in place in order to successfully run the test. At the very top, there are a couple of import statements – those are always set automatically when you export tests from Selenium. Not all of them are used in our example, but there’s no harm in leaving it as it is.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re
Next, there is test class definition, its name is created automatically based on the exported file name. The first method, setUp, creates an instance of a browser (in this case, it’s Firefox) and sets a couple of basic attributes, like base URL address. For now, you shouldn’t change anything here.
def setUp(self):
self.driver = webdriver.Firefox()
self.base_url = "https://www.google.com"
self.verificationErrors = []
self.accept_next_alert = True
The next method, test_desmart_blog, contains all the steps to reproduce the test case. One very important thing here: the names of all the methods that contain test cases must start with “test_” in order to be identified by unittest module. If a method doesn’t have this “prefix”, it will be simply omitted.
Basically, all commands have high-level names and are quite self-explanatory. You should notice that they have a specific structure – first, there is a webdriver variable, then there’s an element you want to interact with (notice the different types of selectors) and then you have a specified action you want to perform on this element.
def test_desmart_blog(self):
driver = self.driver
driver.get(self.base_url + "/")
driver.find_element_by_id("lst-ib").clear()
driver.find_element_by_id("lst-ib").send_keys("desmart blog")
driver.find_element_by_name("btnG").click()
driver.find_element_by_link_text("Blog About Software Development, UX Design ... - DeSmart").click()
driver.find_element_by_xpath("(//a[contains(text(),'Why Do We Create Bad APIs')])[2]").click()
The next three methods presented concern different types of exceptions, you shouldn’t change anything there. Finally, there’s a tearDown the method that closes the browser after the test is done.
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
At the very bottom of the file, you can also find a statement that triggers the test execution by using unittest module.
if __name__ == "__main__":
unittest.main()
Upgrade your test
Now that we are more familiar with the overall test structure, we can upgrade it a little bit. First of all, we need to define places where the script should pause, in order to wait for the browser to load changes in the dynamic elements. The easiest way to do so is to add implicitly_wait() a command in the setUp method. For example, self.driver.implicitly_wait(2) would pause the script execution for up to two seconds, if it cannot instantly perform the next action.
You can also create new variables that will be referring to website elements. For instance, the search input field is mentioned twice – once for clearing the field, and the second time for sending the search phrase. If you store this element as a variable, you can use it later as many times as you like, but the best part of it is that if the selector ID would change, you only need to modify one line in your script. So let’s make a little alteration in the test:
search_input = driver.find_element_by_id("lst-ib")
search_input.clear()
search_input.send_keys("desmart blog")
You probably noticed that when you run the test, the browser window is not maximized. If you want to change that, you can put the following command at the beginning of the test:
driver.maximize_window()
Assertions
The main purpose of testing is obviously to check things – whether something works as expected, is something present or not, do all the elements have proper labels etc. Thus far my exemplary test consists of simple actions and it doesn’t really verify anything. Let’s change that and insert a couple of assertions.
The first thing you might want to check is to verify the website title. For example, let’s add an assertion statement after arriving at the DeSmart blog:
blog_title = "Blog About Software Development, UX Design and Project Management"
self.assertEqual(blog_title, driver.title)
You can also verify if any given element is currently present on the website. For instance, I would like to make sure that the company logo is displayed. In order to quickly get a valid selector, I recommend using the Select feature in Selenium plugin and then click on the desired element or just click the right mouse button on it and get the selector from the contextual menu. Notice that this statement has a slightly different structure, is_element_present the method takes two arguments: selector type and the name of the targeted element.
company_logo = (By.CSS_SELECTOR, "img[alt="'DeSmart'"]")
self.assertTrue(self.is_element_present(company_logo[0], company_logo[1]))
You can go the other way round and check if something is false or not equal. For example, let’s inspect the header image after opening any post on our blog. You can notice that this element has class=”promoted” CSS property. We can verify if it’s not called, let’s say, “not-promoted”.
header_image = driver.find_element_by_css_selector("div.promoted")
self.assertNotEqual("not-promoted", header_image.value_of_css_property("class"))
Summary
The guidelines discussed in this article are just the basics of creating automated tests by using Selenium IDE and Python, a tip of the iceberg. But I believe that it’s sufficient to start creating simple tests that are relatively easy to maintain. If you want to find out a little bit more about Selenium and the available methods in Python, please refer to these pages in docs: Selenium.
Should you have any questions or problems with setting up the environment or running your tests, feel free to contact us. I’m also curious if you have any tips & tricks for people who are about to start their adventure with automated tests.
