Presented by John Paul Ashenfelter.

What is testing?

• Verify the software works.
• Validate the reliability and stability. May work on your dev machine but what about when getting hit by 500 people?
• Ensure the software solves the problem it is supposed to solve.
• Make sure the user can use the software (usability).

People remember bad software a lot longer than they remember late software. So don’t skip the testing stage. Better to test it and get it right. May take a number of versions to undo perception of bad software that was just released too soon.

Good testers are developers, QA staff, help desk, internal staff. Bad testers are users, customers, the public.

When test? All the time. When you write code, change code, fix a bug, integrate code, deploy code, think of a new edge case. Click “Run” on your test suite when you’re taking a break.

How should I test? (1) Code tests (Unit testing). (2) Application tests (functional testing). (3) System (load/performance testing). This can be a nasty black art. (4) If possible, also do user acceptance testing.

Functional Testing

Tests verify the application is working from the users’ piont of view. A “Test Case” is then a single step with a single outcome. You need at least one test per requirement/use case. You will have preconditions and postconditions, known inputs and expected outputs.

Selenium. Can help with functional testing. Especially when used with Firefox, there is a plugin that provides a nice GUI for setting up tests. Really easy. But even if you don’t use Firefox, you just need to type some simple HTML to generate the tests. Selenium will then run through the tests through its HTML/JavaScript interface.

Really neat trick: Your Selenium test case file can be a ColdFusion file, and you can embed bits of ColdFusion to do various things. For instance, John used the ColdFusion Rand() function when generating the text to enter into some of the app’s form fields, to make the fields unique (for instance, in a field that requires a unique value, perhaps an email field). Could also embed ColdFusion to do queries or CFC calls that would autodelete any database records before running your tests.

Unit Testing

A developer writes a unit test to verify an extremely specific area (unit) of functionality (code) behaves as they expect. One unit test isn’t good enough. Have tests with good data, bad data, edge cases (zero, etc). Takes a little time, but can be addicting and extremely useful during development and maintenance phase of an application. Also tests for: Negative numbers, duplicates, normal sets of values (the “good” data), empty arrays, single values.

Test cases will eventually become test suites. A bunch of individual but related test suites. You can then automate the tests with Ant. Even can do commit hooks in your source control (i.e. Subversion). Then there are commercial “continuous integration” tools like CruiseControl. This will improve the quality of your code, because everything is constantly being tested.

Spend your time setting up good automated repository and testing tools rather than running your tests manually over and over again. Subversion commit hooks set up to send emails, including emails with diffs. Can also do automated testing on commit.

Load Testing

“Premature optimization is the root of all evil in programming.” — C.A.R. Hoare

Don’t spend too much time on writing efficient right off the bat. Write for code readability and maintainability. If you really do have a problem in some areas once load testing, start the optimizations. But don’t bother load testing until after you have done functional and unit tests.

Flavors of load testing: Load test expected usage patterns; performance testing known workloads; stress testing to the breaking point. Load testing is WYTIWYM (what you test is what you measure).  But recognize that results rarely (i.e. never) scale as expected to different hardware configurations. So you’re really not sure what the results will be until it goes on the live server.

User Acceptance Testing

Very useful and provides insight on things. If users cannot figure out your app, the problem is somewhere on your end. We need to make the app simple and intuitive enough for them to use it with little or no guidance.

Final Thoughts

Testing is part of our job. Needs to be integrated into the entire development process, otherwise we won’t keep up on it and it won’t be meaningful.