nav-left cat-right
cat-right

10 Tips for Building Selenium Integration Tests

I've been developing an API for our testers to use when writing GUI integration tests and acceptance tests for our main AJAXified web product. Having chosen Selenium Remote Control as our test engine, and writing many tests by hand and with the Selenium IDE Firefox Plugin, I've delved deeply into Selenium this past several weeks. (I've also managed to not only isolate it from our testers, but make our tests portable to any machine that you'd like to use, with no installs or executable paths, but I'll save that magic for another post ;))

So from the last few weeks, I thought I'd share some tips I've learned to make writing Selenium tests easier on yourself.

  1. Make a GUI Component map
  2. The first thing you should consider is mapping all your GUI elements to strongly typed objects, or at least a dictionary of named XPath strings. It is much easier on everyone if they can read your code and see something like
    Selenium.Click(UIElements["LoginButton"])
    rather then something like
    Selenium.Click("//div[@id='LoginContainer']/table[1]/tbody[1]
    /tr[3]/td[2]/a[1]/img[1]")

    Hand in hand with this tip, I'll remind you to use Best Practices and get your developers to put in an ID for any web page element that will be interacted with ;) Also, you can check out the Selenium extension called GUI_Map if you're looking to extend the Selenium core for your environment.

  3. Fix the timer code!
  4. The code that the Selenium IDE generates for a timer loop is terrible! It loops from 0 to your timeout number, checking the condition and then sleeping for one second. This assumes that the loop itself has a negligible execution time, which is untrue in many cases. I've run into situations where for whatever reason, the timer loop runs very slowly depending on the XPath you're checking against. This means your 60 second timeout actually takes wayyyy longer, which is very annoying. Use this code instead to more reliably measure elapsed time:

    [csharp]
    DateTime start = DateTime.Now;
    TimeSpan ts = DateTime.Now.Subtract(start);
    bool bFound = false;
    do
    {
    try
    {
    if (_selenium.IsElementPresent(elementName))
    {
    bFound = true;
    break;
    }
    }
    catch (Exception)
    { }
    Thread.Sleep(1000);
    ts = DateTime.Now.Subtract(start);
    }
    while (ts.TotalMilliseconds < _waittime) ;

    if (!bFound) Assert.Fail("timeout");
    [/csharp]

  5. MouseDown() + MouseUp() is better than Click()
  6. I've found that in many situations a call to Click() won't work, even if the Selenium IDE generated the code. This is especially true for block elements like table cells that you've styled to be clickable. Just give up and use a call to MouseDown("element") followed by MouseUp("element"). This works great in almost every situation.

  7. Simulate Click-Drag with MouseDownAt + MouseUpAt
  8. I also searched long and hard for a Click-drag method for Selenium until realizing that this can also simply be done with MouseDownAt("elementName", "x1,y1") followed by a MouseUpAt("elementName", "x2,y2"). Handy if you're testing on-screen drawing or selection tools using click-drags.

  9. Use the Command Pattern
  10. One of the things the testers appreciate most is that my API handles all the overhead in their tests. Typically you'll want to split your integration tests into multiple parts, so any given test run would be composed of several tests, each of which runs a sequence of selenium commands. In order to reset our test environment for each test, I made heavy use of the Command pattern to implement file undo stacks, reversible web-steps, etc. That way, testers can go deeply into a sequence of commands and simply call the reverse of each command in the stack to back them out to where they started from. Very useful when testing anything that modifies configurations, test data, etc.

  11. Modal Dialogs are a show-stopper
  12. There's no getting around this one. Browser dialogs such as the file download dialog are not testable in Selenium. There is work being done to allow this in the future, but currently you're out of luck.

  13. Access Javascript Variables in your page with getCurrentWindow()
  14. If you're using WaitForCondition(), you can use the following snippet to access the DOM for the page that Selenium is running:
    selenium.WaitForCondition(
    "this.browserbot.getCurrentWindow().imgloaded == 1", "15000"
    );

    This is handy for executing existing javascript functions as well as checking variable values.

  15. Selenium IDE is not your friend: Re-use code to reduce fragility
  16. A fundamental drawback to using a GUI direct-testing tool is the reliance on locators for GUI elements. If your locators change, your tests break, even though functionally the app might still be in perfect working order. As well as using Best Practices to make your tests as robust as possible, you need to centralize your locators (see Tip #1) and build a library of utility functions to re-use. Having 15 wait-for loops in a test or having the same navigation routines in multiple tests is hideous, not maintainable, and very fragile.

  17. Use Firebug + Selenium IDE for XPath finding
  18. While the Selenium IDE will choose the shortest XPath it can find when it generates your test code, often it isn't what you want. If you don't want to edit the default behaviours of the IDE tool (which requires some javascript tinkering), I find it helpful to just use the Firebug extension for Firefox and hit F12 to bring up Firebug when the Selenium IDE won't generate a good locator for you. Use firebug to view the element and determine the XPath you need, replace it in the Selenium IDE, and continue along your merry way.

  19. Use the following code to test for all finished callbacks
  20. If you're making use of a framework that is using the ASP.Net 2.0 Callback architecture, this little snippet can save you a bundle. The key to AJAX testing with Selenium is making use of the various WaitFor…() methods. For a normal AJAX request, you would just do a regular WaitForCondition or WaitForElementPresent - so what's different about some callbacks? If you have a Callback chain -- i.e., one callback has a result that sets off another callback - you can't use WaitFors to tell when they're done. A WaitForElementPresent on something that is changed later on in the callback chain will not work, since the element is present and sitting there while the first callbacks are executing. A WaitForCondition might work, depending on the specific situation, but generally it's not easy to inject a semaphore around some callback object without making the test very fragile, and having to write code for multiple situations in your app. And of course using Thread.Sleep() slows down your test unnecessarily and isn't reliable. What we really need is a reliable generic method to check that all Callbacks are complete before we start Asserting that everything worked out. So use this:

    WaitForCondition("var finished = 1; for (var i = 0; i < selenium.browserbot.getCurrentWindow().__pendingCallbacks.length; i++) { if (selenium.browserbot.getCurrentWindow().__pendingCallbacks[i] != null){ finished = 0; } }; finished == 1", "15000");

    This is a semaphore around the ASP.Net __pendingCallbacks array that waits for everything to finish up and returns true when all callbacks are complete. It times out in 15 seconds (you can change the 15000 parameter to any timeout length you'd like).

There it is - 10 tips for happier test writing. Hope it helps some people out.

kick it on DotNetKicks.com

Be Sociable, Share!

12 Responses to “10 Tips for Building Selenium Integration Tests”

  1. Joakim Nilsson says:

    Very good tips! Thanks!

  2. Thanks a lot for this article, this is exactly what I was looking for!!

  3. John says:

    A question about tip 7.

    I can't get this to work. Selenium cannot find the variable i am trying to access. This is a variable that I create myself on page load.

    It seems like selenium is able to access DOM variables, but not my own variables.

    Any thoughts?

  4. Anoop says:

    thnx ..This article helped me a lot

  5. Shae Erisson says:

    I've also found that the XPath and XPather add-ons for Firefox can make it easier to find a good XPath selector for an element.

    https://addons.mozilla.org/en-US/firefox/addon/1095 XPath

    https://addons.mozilla.org/en-US/firefox/addon/1192 XPather

  6. Rafael Ribeiro says:

    The problem with the variable it seems that it because of the wrapper that selenium uses around the window object to make it more "secure". See this link: http://crschmidt.net/blog/348/selenium-ide-getcurrentwindow-problems/

  7. Anonymous says:

    Good info!

  8. TAB says:

    Regarding point #9: I found XPather (https://addons.mozilla.org/en-US/firefox/addon/1192) to be a really nice addition to firebug when testing complex XPath expressions in Firefox.

  9. suresh says:

    Does WaitForElementPresent command is implemented in selenium or do we need to implement a wrapper around using selenium Wait.wait method? I am using selenium 1.0.2 snapshot and I don't see that method(command) at all to use it Java/C# langugage(not in IDE). Am I missing something here?

  10. Thanks for the tips.

    I have been trying to test a click and drag interaction on my project using tip 4 (Simulate Click-Drag with MouseDownAt + MouseUpAt) however this does not seem to work. It only simulates the click and release not the actual change of cursor position, if I moved my mouse into the spawned browser and dragged down in between the two events it worked, but not in an automated fashion. However I found that introducing mouse_move_at() in between resulted in the intended behavior.

    Example:
    selenium.mouse_down_at("//div[@id='events']", "8,200")
    selenium.mouse_move_at("//div[@id='events']", "8,400")
    selenium.mouse_up_at("//div[@id='events']", "8,400")

    I think this ought to be abstracted as a standalone function in selenium.

  11. mdma says:

    A great read, showing that developing tests (or a test support) is real software development. We don't need to put up with using tools that generate fragile code.

    Solid design for strong, maintainable tests!

  12. Aniket says:

    gui_map for selenium is a discovery for me. I was not aware of such thing. Thanks a lot for sharing, this will ease out life a lot

    Aniket

Leave a Reply