WritingTestsHowTo
Selecting the right plugin
There are mainly two different kinds of test cases: manual and automated test cases. The former require a human intervention to decide if the test was successful or not while the latter just requires a script that executes the tests and provides a return code that will be interpreted by checkbox. Following the unix convention, the script will return 0 on success and any other code on failure.
Manual test cases are implemented by the manual plugin, while automated test cases are implemented by the shell plugin. So when adding a test case that needs someone to decide if the output was correct or not, use the manual plugin; and, when the test is fully automated, use the shell plugin and checkbox will take care of collecting the results for you.