Implementing HTTP Status Codes in WebDriver, Part 2: Achievement Unlocked


UPDATE (21 August 2013): In response to a comment by Eric Lawrence (author of Fiddler and all around awesome chap), I’ve updated the code sample for the redirect case. Thanks Eric for taking the time to comment and point out where I could make improvements.

In Part 1 of this series, we looked at the beginnings of implementing HTTP status codes in WebDriver the correct way. That is to say, by using a proxy server to monitor traffic for the information we want. To recap, we’re using Fiddler as our proxy, the .NET bindings to execute our WebDriver code, and we’re running against Mozilla’s website as our test destination. At the end of the last blog post, we successfully had a proxy hooked up, which will log resources to the console as they are requested by the browser. Now it’s time to actually extract the HTTP status codes from the information that the proxy is able to collect. As a reminder, here’s what our WebDriver execution looks like:

private static void TestStatusCodes(IWebDriver driver)
{
// Using Mozilla's main page, because it demonstrates some of
// the potential problems with HTTP status code retrieval, and
// why there is not a one-size-fits-all approach to it.
string url = "http://www.mozilla.org/";
driver.Navigate().GoToUrl(url);

string elementId = "firefox-promo-link";
IWebElement element = driver.FindElement(By.Id(elementId));
element.Click();

// Demonstrates navigating to a 404 page.
url = "http://www.mozilla.org/en-US/doesnotexist.html";
driver.Navigate().GoToUrl(url);
}

So the first thing we are doing in our WebDriver code is navigating to http://www.mozilla.org/. So let’s create a method that will perform the navigation, and return us the status code. As we saw last time, Fiddler lets us hook up an event delegate to respond every time a resource is retrieved by the browser, and analyze that response. The nice thing about event delegates in .NET is that we don’t need to leave them hooked up any longer than necessary. Here’s our first stab at a method that will hook and unhook the delegate for the navigation:

public static int NavigateTo(IWebDriver driver, string targetUrl)
{
int responseCode = 0;
SessionStateHandler responseHandler = delegate(Session targetSession)
{
responseCode = targetSession.responseCode;
};

FiddlerApplication.AfterSessionComplete += responseHandler;
driver.Url = targetUrl;
while (responseCode == 0)
{
System.Threading.Thread.Sleep(100);
}

FiddlerApplication.AfterSessionComplete -= responseHandler;
return responseCode;
}

Astute readers will see that this has a couple of issues with it. First, how do we know what behavior we want for redirects? Our base URL to which we’re navigating has just such a redirect. Do we expect to return a 300-level response, or follow the navigations through until we receive a 200-level or 400-level response? This is a perfect example of why there’s no one-size-fits-all approach to HTTP status codes that will work for every WebDriver user, and a reason why, in turn, this feature is out of scope in the WebDriver API. In our case, if the URL redirects for navigation, we’re going to return the redirect response code. In your implementation, if you decide on another approach, you’ll want to modify the event handler delegate to meet your own needs.

The second issue is that we aren’t guaranteed that we’re returning the response code for the proper resource. So we want a modification that will validate that. Also, we’ll probably want to create a timeout so that we don’t inadvertently loop infinitely in the while loop. Making these modifications, you’ll get a method that looks something like this:

public static int NavigateTo(IWebDriver driver, string targetUrl)
{
int responseCode = 0;
SessionStateHandler responseHandler = delegate(Session targetSession)
{
if (targetSession.fullUrl == targetUrl)
{
responseCode = targetSession.responseCode;
}
};

FiddlerApplication.AfterSessionComplete += responseHandler;

// Yes, we're hard-coding a 10 second timeout here. Don't worry, we'll
// make that configurable before we're done.
DateTime endTime = DateTime.Now.Add(TimeSpan.FromSeconds(10));
driver.Navigate().GoToUrl(targetUrl);
while (responseCode == 0 && DateTime.Now < endTime)
{
System.Threading.Thread.Sleep(100);
}

FiddlerApplication.AfterSessionComplete -= responseHandler;
return responseCode;
}

Okay, so now we have a method that will return us the status code on explicit navigation to a URL. What about on a click that navigates to a new location? Clicks are a little trickier, because a click might trigger a navigation, or it might not. In my opinion, you should know what type of click you’ll be performing, so I’ll create a method that we will explicitly call when we want to perform a click that will navigate, and return the HTTP status code of that navigation. I’ll also take this opportunity to demonstrate a way to handle redirects, since the link we’re clicking on in our test code also causes a redirect. Again, we’ll hook up a delegate for the duration of the time we need it, and unhook it after we’re done.

public static int ClickNavigate(IWebElement element)
{
int responseCode = 0;
string targetUrl = string.Empty;
SessionStateHandler responseHandler = delegate(Session targetSession)
{
// For the first session of the click, the URL should be the initial
// URL requested by the element click.
if (string.IsNullOrEmpty(targetUrl))
{
targetUrl = targetSession.fullUrl;
}

// This algorithm could be much more sophisticated based on your
// needs. In our case, we'll only look for responses where the
// content type is HTML, and that the URL of the session matches
// our current target URL. Note that we also only set the response
// code if it's not already been set.
if (targetSession.oResponse["Content-Type"].Contains("text/html") &&
targetSession.fullUrl == targetUrl &&
responseCode == 0)
{
// If the response code is a redirect, get the URL of the
// redirect, so that we can look for the next response from
// the session for that URL.
if (targetSession.responseCode >= 300 &&
targetSession.responseCode < 400)
{
// Use GetRedirectTargetURL rather than examining the
// "Location" header, as some sites (illegally) might
// use a relative URL for the header (per Eric Lawrence).
targetUrl = targetSession.GetRedirectTargetURL();
}
else
{
responseCode = targetSession.responseCode;
}
}
};

// Note that we're using the ResponseHeadersAvailable event so
// as to avoid a race condition with the browser (per Eric
// Lawrence).
FiddlerApplication.ResponseHeadersAvailable += responseHandler;

// Yes, we're hard-coding a 10 second timeout here. Don't worry, we'll
// make that configurable before we're done.
DateTime endTime = DateTime.Now.Add(TimeSpan.FromSeconds(10));
element.Click();
while (responseCode == 0 && DateTime.Now < endTime)
{
System.Threading.Thread.Sleep(100);
}

FiddlerApplication.ResponseHeadersAvailable -= responseHandler;
return responseCode;
}

All that remains is to modify our WebDriver code to call our new methods instead of the standard WebDriver ones, and add some console logging to prove that we get actual status codes returned from our methods. That modifies our TestStatusCodes method to look like this:

private static void TestStatusCodes(IWebDriver driver)
{
// Using Mozilla's main page, because it demonstrates some of
// the potential problems with HTTP status code retrieval, and
// why there is not a one-size-fits-all approach to it.
string url = "http://www.mozilla.org/";
int responseCode = NavigateTo(driver, url);
Console.WriteLine("Navigation to {0} returned response code {1}",
url, responseCode);

string elementId = "firefox-promo-link";
IWebElement element = driver.FindElement(By.Id(elementId));
responseCode = ClickNavigate(element);
Console.WriteLine("Element click returned response code {0}",
responseCode);

// Demonstrates navigating to a 404 page.
url = "http://www.mozilla.org/en-US/doesnotexist.html";
responseCode = NavigateTo(driver, url);
Console.WriteLine("Navigation to {0} returned response code {1}",
url, responseCode);
}

Running our console application from last time, we now will receive output that looks like the following:

Starting Fiddler proxy
Fiddler proxy listening on port 62594
Navigating to http://www.mozilla.org/
Navigation to http://www.mozilla.org/ returned response code 301
Clicking on element with ID firefox-promo-link
Element click returned response code 200
Navigating to http://www.mozilla.org/en-US/doesnotexist.html
Navigation to http://www.mozilla.org/en-US/doesnotexist.html returned response code 404
Shutting down Fiddler proxy
Complete! Press <Enter> to exit.

Now we have a fully functioning example for Firefox. Next time, we’ll add the code to make it cross-browser aware, and add a few more tricks to make it more elegant for use with WebDriver.

Implementing HTTP Status Codes in WebDriver, Part 1: Challenge Accepted

A while back, I wrote a post that discussed at length why HTTP status codes are not present in the WebDriver API. Furthermore, the post went on to explain why I believe they’re not needed in the API, and that there are other tools better suited to retrieving this particular piece of esoterica. Since I wrote that article, other Selenium contributors have written about the same topic. The general premise of those blog posts and mine is that using a proxy is the proper way to capture the status code if you actually require it.

Nevertheless, the issue in the Selenium issue tracker that was the inspiration for those blog posts continues to receive comments, most of them vehemently opposed to the decision of the Selenium development team. The decision has been called “complete nonsense,” “silly,” “condescending,” and “simply defective,” among other choice phrases. My colleagues on the Selenium project have posted code samples that show how to effectively use a proxy with Selenium to solve this problem, and the response to those samples has been that they aren’t detailed enough.

Alright, fine. Time to put my proverbial money where my mouth is. I recently looked into what it would take to actually implement a proxy solution, with correct return of HTTP status codes, including writing all of the code necessary to extract it. It doesn’t take that much, as it turns out. Once I’d settled on a technology to use, I had a full working example in about 4 hours. Let’s take a look at how this would work.

In my example, I decided to use the Mozilla website, http://www.mozilla.org/, as my test. I settled on this because the site isn’t likely to disappear anytime soon, and as currently written, it nicely demonstrates some of the issues inherent in getting HTTP status codes. Please note that I don’t own the website, so it’s possible that these examples could break at any time after this writing; at some point, I’ll look at creating a standalone website that illustrates the same concepts. I’m also going to be using the WebDriver .NET bindings, and specifically, version 2.35.0 of the .NET bindings. For the proxy component, I decided to use Eric Lawrence’s (now Telerik’s) excellent Fiddler proxy.

Let’s talk for a moment about why I chose Fiddler. First, I’m a .NET guy, and I try to look for solutions that don’t require another runtime (like Java, Ruby, or Python) if possible. Second, Fiddler offers me the FiddlerCore component, which allows me to use an API-only version of Fiddler, and programmatic access to all of the proxy’s settings and data. The API may be a little less polished than other .NET component APIs, but it does use an event-driven model, which appeals to me as a .NET developer. While Fiddler isn’t open-source, it is free to use, with no feature restrictions based on free vs. paid use. With all of that in mind, let’s begin. Here’s the basic code that I want to run, using the standard WebDriver API:

private static void TestStatusCodes(IWebDriver driver)
{
// Using Mozilla's main page, because it demonstrates some of
// the potential problems with HTTP status code retrieval, and
// why there is not a one-size-fits-all approach to it.
string url = "http://www.mozilla.org/";
driver.Navigate().GoToUrl(url);

string elementId = "firefox-promo-link";
IWebElement element = driver.FindElement(By.Id(elementId));
element.Click();

// Demonstrates navigating to a 404 page.
url = "http://www.mozilla.org/en-US/doesnotexist.html";
driver.Navigate().GoToUrl(url);
}

I’ll be running this method from within a console application, with the main method looking something like this:

static void Main(string[] args)
{
// Eventually, we will use different browsers to prove this
// solution works cross-browser, but for now, we will use
// Firefox only.
IWebDriver driver = new FirefoxDriver();

TestStatusCodes(driver);

driver.Quit();

Console.WriteLine("Complete! Press <Enter> to exit.");
Console.ReadLine();
}

Let’s look at how to integrate Fiddler in this solution. Starting the proxy server couldn’t be easier. We’ll create a method to do this. One thing to note in the method is that we can either specify a port for the proxy to listen on, or let Fiddler pick one for us.

private static int StartFiddlerProxy(int desiredPort)
{
// We explicitly do *NOT* want to register this running Fiddler
// instance as the system proxy. This lets us keep isolation.
Console.WriteLine("Starting Fiddler proxy");
FiddlerCoreStartupFlags flags = FiddlerCoreStartupFlags.Default &
~FiddlerCoreStartupFlags.RegisterAsSystemProxy;

FiddlerApplication.Startup(desiredPort, flags);
int proxyPort = FiddlerApplication.oProxy.ListenPort;
Console.WriteLine("Fiddler proxy listening on port {0}", proxyPort);
return proxyPort;
}

Technically speaking, we probably don’t need to shut down the proxy, since it’s the last thing we do before our main method exits, but we’re going to be a good citizen and shut it down anyway.

private static int StopFiddlerProxy()
{
Console.WriteLine("Shutting down Fiddler proxy");
FiddlerApplication.Shutdown();
}

All that remains is to hook up an event handler so that we can analyze the traffic that comes through the proxy, and to make the Firefox driver aware of the proxy. We can do those things within the context of our main method. After all of these, the final main method looks like this:

static void Main(string[] args)
{
// Note that we're using a desired port of 0, which tells
// Fiddler to select a random available port to listen on.
int proxyPort = StartFiddlerProxy(0);

// Hook up the event for monitoring proxied traffic.
FiddlerApplication.AfterSessionComplete += delegate(Session targetSession)
{
Console.WriteLine("Requested resource from URL {0}",
targetSession.fullUrl);
};

// We are only proxying HTTP traffic, but could just as easily
// proxy HTTPS or FTP traffic.
OpenQA.Selenium.Proxy proxy = new OpenQA.Selenium.Proxy();
proxy.HttpProxy = string.Format("127.0.0.1:{0}", proxyPort);

// Eventually, we will use different browsers to prove this
// solution works cross-browser, but for now, we will use
// Firefox only.
FirefoxProfile profile = new FirefoxProfile();
profile.SetProxyPreferences(proxy);
IWebDriver driver = new FirefoxDriver(profile);

TestStatusCodes(driver);

driver.Quit();

Console.WriteLine("Complete! Press <Enter> to exit.");
Console.ReadLine();
}

When we run our console application, we see something like this:

Starting Fiddler proxy
Fiddler proxy listening on port 62492
Navigating to http://www.mozilla.org/
Requested resource from URL http://www.mozilla.org/
Requested resource from URL http://mozorg.cdn.mozilla.net/media/css/tabzilla-min.css?build=c2a3f7a
Requested resource from URL http://mozorg.cdn.mozilla.net/media/js/site-min.js?build=c2a3f7a
Requested resource from URL http://mozorg.cdn.mozilla.net/media/css/responsive-min.css?build=c2a3f7a
Requested resource from URL http://mozorg.cdn.mozilla.net/media/img/favicon.ico
Requested resource from URL http://www.mozilla.org/en-US/

[...
Many resources deleted for brevity
...]

Clicking on element with ID firefox-promo-link
Requested resource from URL http://mozorg.cdn.mozilla.net/media/fonts/Vollkorn-Regular-webfont.woff
Requested resource from URL http://mozorg.cdn.mozilla.net/media/fonts/Vollkorn-Bold-webfont.woff
Requested resource from URL http://mozorg.cdn.mozilla.net/media/img/home/promo/flicks/760.jpg
Requested resource from URL http://mozorg.cdn.mozilla.net/media/img/home/promo/android/760.jpg?2013-06
Requested resource from URL http://mozorg.cdn.mozilla.net/media/img/home/promo/makerparty/760.jpg
Navigating to http://www.mozilla.org/en-US/doesnotexist.html
Requested resource from URL http://www.mozilla.org/firefox/
Requested resource from URL http://www.mozilla.org/en-US/firefox/
Requested resource from URL http://mozorg.cdn.mozilla.net/media/css/firefox_fx-min.css?build=c2a3f7a
Requested resource from URL http://mozorg.cdn.mozilla.net/media/img/firefox/template/header-logo.png?2013-06
Requested resource from URL http://www.mozilla.org/en-US/firefox/fx/ 

[...
Many resources deleted for brevity
...]

Shutting down Fiddler proxy
Complete! Press <Enter> to exit. 

Obviously, this particular example doesn’t get us to our desired goal just yet. However, it does allow us to hook up a proxy. Next time, I’ll show you how we can refine this solution to actually extract those status codes.

Revisiting Native Events in the IE Driver

A little over six months ago, I wrote a blog post that concentrated on so-called “native events” in the IE driver. To refresh your memory, native events are using OS-level mechanisms for simulating mouse and keyboard input in the browser. This is distinct from “simulated events”, which rely on using JavaScript to trigger the events on the elements in a page. Since I wrote that blog post, there have been some developments in the IE driver that have changed the landscape a little, and it’s probably a good time to revisit the topic.

If you’ll recall, there were two major issues that people have had challenges with using native events with the IE driver. The first is mouse clicks being swallowed up if the IE window isn’t the foreground window. The second is hovering over elements, where the elements displayed on hover would appear for a fraction of a second, then disappear. This second issue was particularly vexing, since it only happens when the physical mouse cursor is within the bounds of the IE window. See the aforementioned previous blog post if you want more details about what exactly causes these issues, and why they’re particularly hard to solve. Having said that, let’s look at the newer approaches that have come available in the last six months.

Persistent Hovers

The first approach is using what’s called “persistent hovers.” By default, the IE driver simulates native mouse movements by using the Windows SendMessage API to send WM_MOUSEMOVE messages to the IE window. This can work, but if the IE window is in the foreground, and the physical mouse cursor is within the bounds of the IE window, the hover won’t persist; it’ll flash, and disappear. To solve this, the IE driver launches a separate thread and continually throws WM_MOUSEMOVE messages at the IE window for the coordinates that your test code specifies.

This means that the hover doesn’t flash once and disappear, now it flickers, but at least it sticks around long enough to do something with. With the flickering, there’s a far better chance that the hidden element you’re attempting to display with your hover will be visible when you try to interact with it. Careful readers will note that there’s still a race condition here, which your code might lose. Also, using persistent hovers renders that particular instance of IE unusable if you want to do something with it manually after you’ve finished with your test. Finally, notice that they only attempt to address the hover problem, not the “no focus click swallowing” problem.

Persistent hovers are available now with native events in the IE driver. They’re enabled by default. If, for whatever reason, you need to revert to the behavior without them, they can be controlled by the “enablePersistentHover” capability.

Simulated Events

Over the last six months or so, there have been some strong improvements made in the simulated events implementation in IE. Simulated events have the advantage that they’re not reliant on window focus. They’re also a bit faster than the default native events implementation. However, these advantages are not without their costs.

Simulated events are limited to the JavaScript sandbox. There are certain effects that can’t be activated via JavaScript, like a hover triggered solely by CSS. Blocking JavaScript events, like using alert(), confirm() or prompt(), especially in an onclick or onsubmit event handler, can cause your JavaScript execution (and therefore your WebDriver code) to hang.

If your application doesn’t run afoul of the issues endemic to pure JavaScript input simulation, simulated events may be a good choice for you. You can use simulated events by setting the “nativeEvents” capability to false.

Requiring Window Focus

One of the guiding principles of the WebDriver project is that a driver should not require the browser window to be in focus to work properly. This principle is so important to the project that it’s been codified as part of the draft W3C specification for WebDriver. However, there is some subset of users that don’t care about needing window focus in their environment, they just want the most accurate test utility possible. Often these people mention that they’re running their tests in a test lab or on dedicated virtual machines, and don’t need to worry that focus is taken by the browser window. Until now, the IE driver had no way to accommodate such a requirement.

However, with a recent change set in the IE driver code base, it is now possible to ask the IE driver to bring the window currently focused in the driver (as determined by the WebDriver.getWindowHandle) to be the window in the foreground. This attempt is not guaranteed to succeed, because making a window from another process the foreground window is a Bad Thing, but it’s the only way to make simulated input work properly. Additionally, instead of using the flawed SendMessage/PostMessage approach, this change introduces use of the Windows SendInput API.

This functionality is brand new. It should be considered very experimental. All input using the WebDriver Advanced User Interactions API (the “Actions” class) will go through this code path if set correctly, but mouse clicks using WebElement.click() will not yet do so. Additionally, there’s no mechanism (yet!) in the wire protocol to enforce that a sequence of actions should be handled as an atomic unit, though the new internal classes added to the IE driver will make it easy to implement such a feature. To enable this functionality, set the “requireWindowFocus” capability to true; it is defaulted to false, to maintain existing functionality. Note that requireWindowFocus and enablePersistentHover are mutually exclusive.

Hopefully, these methods will give you the most flexibility with how you perform keyboard and mouse interaction with Internet Explorer. Most of these are hacks at best, but that’s the best we can do until Microsoft begins producing the IE driver.

Seeing "Info" Messages in a Log Does Not Automatically Imply "Error"

Here’s what seems to be a common question among Java language bindings users of the IE driver (tl;dr):

Whenever I use the IE driver, I see an error in my log. The log output is below. How do I stop it?

Started InternetExplorerDriver server (32-bit)
2.28.0.0
Listening on port 48630
Sep 23, 2012 9:12:21 AM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond
Sep 23, 2012 9:12:21 AM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request



Follow-up questions like, “What happens after you receive this message?” or, “Does the rest of your code run?” usually get a response like, “No, the rest of my code works fine with IE, I just see this error message, and I don’t think I should be seeing any errors.”

There are two answers to this question, a short one and a long one. The short one is, “Read the log message. It’s clearly tagged as ‘INFO’, as in an informational message, and not indicative of any problem with the code?” I find that this question often comes from users of Eclipse, and that the Eclipse console has colored the message red, and people are so conditioned to see “red == bad” that they react to the format of the message rather than the content. The content of the message is flagged at a level that means, “Hey, nothing is wrong, we’re just telling you about it.”

If all you care about is how to get around the message, you can stop reading here. The longer answer to the question involves an explanation of why you might receive this message. The explanation requires a little knowledge of the architecture of the WebDriver language bindings in general, and the IE driver in particular. Remember that in the language bindings for most browsers, communication with the browser happens via a “server component”, and the language bindings use a JSON-over-HTTP protocol to communicate with that server component. The server component could be an instance of a remote WebDriver server (whether written in Java, or another language); or it could be the Firefox browser extension that the FirefoxDriver installs into an anonymous profile and uses; or it could be an instance of chromedriver.exe; or, and this is the important bit for our purposes here, it could be an instance of IEDriverServer.exe. Each of these server components starts up an HTTP server with which the language bindings can communicate.

Let’s think about what happens when you ask the Java language bindings to create a new instance of the InternetExplorerDriver class.

  1. Locate the IEDriverServer.exe, either from the system PATH or from a Java system property
  2. Launch IEDriverServer.exe, including its internal HTTP server
  3. Use the org.apache.http.client.HttpClient class to send HTTP requests to the IEDriverServer’s internal HTTP server to launch IE and establish a WebDriver session


Now looking at those steps, can you see the potential flaw in the execution? If you said “race condition,” you earn a gold star. The Java method for launching a process will return before the IEDriverServer’s HTTP server has had a chance to fully initialize itself and be ready to receive HTTP requests. However, since IEDriverServer.exe is running in a separate process, there’s no way for the Java language bindings to know when the HTTP server has fully initialized. The only way to handle the situation is to poll for the HTTP server, repeatedly sending requests until you get a valid response, and that’s the message you’re seeing in the log. All the bindings are telling you is that the HTTP server hasn’t yet been fully initialized, so it’s trying the request again, nothing to see here, move along.

The bottom line is that the message is informational, not indicative of an error, or else it would have a different logging level. Remember, the format of a log message is incidental; the important thing is its content.

Not Providing an HTML Page? Think of the Kittens!

Consider the following request for help with Selenium WebDriver:

I’m having trouble getting something working in WebDriver. I’m getting a NoSuchElementException when I try to execute the following code:

driver.get("http://xxxxxx/");
driver.findElement(By.id("userName")).sendKeys("user1234");

What could be causing this exception, and how do I fix it?

Obviously, this code could have several potential problems. The ID attribute of the element may be wrong. The element the user is looking for could be inside a frame or iframe. The element may be added to the page DOM via some JavaScript library, and the browser hasn’t had time to process the JavaScript yet when the findElement call is made. Or it could be something else. What’s missing from this request for help, whether the request is made in a post to one of the user-facing mailing lists, or in an issue report, is context. Anyone reading the request won’t know the structure of the page the user is automating, so any attempt at help is going to be pure speculation.

Most of the time, people wanting to help the requester will respond with something along the lines of, “Hey, we really can’t give you any guidance unless we can see a page that reproduces the problem. Can you share the page, or a URL to a page that will demonstrate the issue?” Surprisingly often, this simple request for more context is met with a very large amount of resistance, along with reasons why it isn’t possible. Almost equally as often, these reasons are bogus. Let’s look at a few of these reasons, and see why they’re bogus.

Evans Bogus Reason 1: “It’s hosted on an intranet.”

Just because a webpage happens to be hosted on an intranet doesn’t mean it can’t be shared. Yes, it will require some effort on your part to make it public. Such applications usually rely on some sort of database connection, or connection to internal resources, or LDAP integration, or some other such thing. Here’s a little secret: Because WebDriver works only on the UI, your WebDriver problem most likely has nothing whatsoever to do with those underlying technologies. Yes, that means you’ll probably have to stub out anything like that your application relies on before you can make it public, but it can be done.

Evans Bogus Reason 2: “We’re so afraid someone’s gonna steal our super-secret site concept/source code”

This reason is often imposed from on high by an upper management drone. It’s penny-wise and pound-foolish. You’re paying someone good money to develop automated code against your web application. The people you’re paying are encountering difficulty to the degree that they need outside help to figure out why. Yet you’re so scared that someone is going to steal your precious code or idea that you refuse to let them get help, and causing that money to be wasted because your automators aren’t being productive.

I direct this next bit to those short-sighted nincompoops who refuse to get out of their people’s way. If you’re going to insist on making it hard on your WebDriver coders who are caught between a complex web application and your tunnel-vision, I’ll make an offer here and now to sign whatever NDA is required to get them some help; however, I’ll charge a fee for it. A big one. Much, much more than if you had just let your coders get the help they need in the first place.

Evans Bogus Reason 3: “Do you know how complicated my site’s JS framework/CSS/HTML code is?”

The snarky jerk in me always wants to respond to this reason with something like, “Well, no, I don’t know how complicated it is because I can’t see your code from here, but it’s highly likely that complexity is exactly your problem.” Simplify the page down to plain, vanilla HTML if you can. It’s not like WebDriver is entirely untested. There’s a CI server that runs a bunch of tests (600+ at the time of this writing) on every single commit, against 5 version of IE, 4 versions of Firefox, Chrome, and Opera. If you’re doing something reasonably common in your WebDriver code, it’s probably a safe bet that it’s not going to be globally broken for everybody, everywhere, against every web page.

Also, there’s no requirement that someone needs to use the exact same site that’s giving them the problem. One that is similar will do the trick, which is why we who try to provide help often say ask for “an HTML page, or a public URL to a page” that demonstrates the problem. Here’s a tip, most of the JavaScript UI frameworks provide public demo pages. See if you can replicate the behavior from your WebDriver code against the demo page.

 

Evans Bogus Reason 4: “But I’m just a lowly tester, and I’m not allowed to touch the application code”

I can’t believe this is actually put forth as a reason, since the solutions are many and obvious. Work on a private fork of the application code. Enlist a developer to help you. Try to reproduce your problem against a different, public site.

Remember, We’re All Volunteers Here

The people who develop and provide support for WebDriver are unpaid volunteers. If you’re having an issue with running WebDriver against your web application, and you need to ask for help why it’s failing for you, it’ll go a long way toward establishing goodwill (and getting you a productive answer) if you provide everything someone needs to reproduce your problem easily.