Everyone keeps talking about unit-testing and TDD,
but to change paradigm of thinking thoroughly along with megabytes of code you do not have enough will, money and time? System blackbox testing with Selenium Server (RC) could help you.
It means that we are not testing every single class from within, how unit-tests and white-box techniques do, but instead we only test UI that end-user sees
In my opinion it is better to start integrating automatic QA in company from this kind of testing, simply because it can be first signal that something went wrong before deployment with less money and effort spent on writing all of the tests (compare results with unit-tests where they tell you wether everything is done correctly)
If you have already seen Mozilla Firefox plugin that allows to record all of your actions, creating macro-command list in the process, then you are in the right place, because that is Selenium IDE, which by now can transform commends to ruby, python, java, perl, c# and PHP code. And here is why..
After creating macro, copying php code (see Options → Clipboard format) and making php-file, you can see that for test to run, it needs
- PHPUnit and local php installed
- Selenium Server - a proxy written in java, that runs browser and executes your code
- PHPStorm or console that runs phpunit with relevant params
Second point is done by
installing and running from console
java -jar selenium-server.jar -interactive
Making cases
By extending PHPUnit_Extensions_SeleniumTestCase, we get some of the parent methods that customize functionality in setUp(), in particular
- Browser - firefox, googlechrome, iexplore, safari, konqueror
$this->setBrowser("*chrome");
- Pause between commands
$this->setSleep(1);
But that is not enough, since tests can be commited to SVN for other develpers, who can have different URLs, login/password for testing or even DB access for graybox testing.. Thats why before running tests, you can include bootstrap files. XML-file for PHPUnit is quite empty, and simply links to php file for each developer
<phpunit bootstrap="bootstrap.php"></phpunit>
You can see settings for PHPUnit-tests on the same image.
Not its enough to click Run and enjoy the show in browser
Now lets talk about test grouping.. If test case is elementary set of instructions for a single page (or at least I tend to think about it that way), then test suite is more global grouping, intented to test some process. For example..
Lets say we have an order made by client, which is processed by priviledged user. To test entire flow with status changes, we group single test methods into groups.
Unfortunately, Selenium does not have «Export Test Suite As → PHP» support yet, thats why we have to create inner methods, that would be used in one test "case" (which is actually more like test suite). Obvious disadvantage is that this grouping is usually rather long (several minutes), and its crash does not really tell where it fell.
Another problem with test sets, is data that gets generated as a result. If you are running user registration test, then you probably have validation somewhere in the middle for existing email or login. Naturally that after one successful registration, the second will fail. Adding user deletion in the end is a bad idea. So as an exit strategy you either need a web-access to manual user deletion from adminpanel, or direct access to DB
(hello graybox testing with DB-structure dependency)
If you go deeper into documentation, you may come across PHPUnit_Framework_TestSuite wich as I understood, only groups tests by topics, like «all tests with orders» or «all tests with payment» are successfull, without taking process into consideration.
More to come
Althoug not everything is white as i described, because writing test for Selenium is not easy as you may think, because you have to coupe with DOM changing, popup windows, periodic ajax. There are tonns of obstacles like capthas and input checks, social network authorizations, plugins like flash and java, scroll wheel.
Some things can be avoided by waitForElementPresent
Another, more company related problem is that writing tests should be obligatory, because if you change dom in templates or business logic and forget about running and changing tests, they will become obsolete. And since running all tests with Selenium Server may take minutes, developers may postpone it and commit changes before the results. Thats why you may need Selenium Grid,
that runs tests in parallel with different browsers and platforms, and you may need Hudson to run the tests all the time, to monitor project health as continuous integration.
To conclude, I'd like to remind you about reasonability. You can't test everything, and even if you do, it will cost much more to sustain all of the tests, thats why you need to start with most important and complex functionality — especially where money and payment is involved.
