MobileUNRKarmicComplianceAutotesting

Summary

Create an automated test suite for UNR to check screen size compliance

Rationale

Testing in previous versions of UNR has uncovered many applications where the main window, or even other dialog boxes the application uses fail to display correctly on small screens commonly used by netbooks. Some of these have already been fixed, but it is very time intensive to go through every application and visually inspect each window on a small screen to see if it has changed in a way that makes it too large for netbook screens. Having a testsuite may shift some of that work up front, and require a smaller amount of maintenance, but will provide us with a regression test suite we can use for checking if the applications remain compliant with smaller screens. It will also provide a basic smoke test, touching most of user visible applications in UNR. Automating this testing speeds up the process of testing it so that it can be performed more frequently, and makes it less error prone than following a script.

User stories

Isaac would like to QA UNR on a netbook, but doesn't have a lot of time. He would like an automated testsuite to test for application window size problems that he can launch and leave running while he does something else.

Miriam has a netbook running UNR, running at the default resolution for her device of 1024x576. All of the applications display in such a way that they fit correctly on her screen, because they were tested throughout the cycle and any bugs that might have made them display incorrectly were caught and fixed early.

Assumptions

  • Tests will be written only for applications that ship with UNR by default, and can be launched from the default user interface
  • Applications to be tested can be controlled with ldtp
  • Target screen size is 1024x576

Design

Create a list of applications that will be included in UNR, including such things as GNOME tools that can be run from the launcher.

Perform an initial manual test of each application, determine steps needed to get to any other windows or dialogs launched by the application and the names of the windows that are launched.

Care should be taken to check that there are no components that can be opened (tree views, twisties, etc) that would extend a windows size beyond the original. If this is the case, those need to be accessed by the test.

Implementation

Write a test that performs the same steps, and checks the size of each window as they appear.

This will likely be done in either Mago, or LDTP. The tests themselves should be fairly simple to write for most apps, once the research is done. LDTP has a function called getwindowsize() that can be used to determine the size of the window once it is launched. Note: even if the window can be resized, if the initial size comes up larger than 1024x576 on a netbook, the test should fail. I expect something like that to be rare though, due to maximus.

Test/Demo Plan

Test the test by ensuring that the results match up with what was discovered during manual testing.

If an application is added, or changed to add a new window, the test will need to be modified to deal with it. Resolution related UNR bugs should be examined to determine if they were caught by the automated test, or if there is a gap that needs to be filled in the automation.

Testing will be performed at the minimum supported testing of 1024x576 on supported netbooks.

Unresolved issues

The final list of applications for UNR in this cycle needs to be determined. This is listed as a dependency but work can still begin on this in parallel for apps we know will still be there.

Application List

This is the initial list of applications as reflected by what currently exists in UNR. As was mentioned in the Unresolved issues section, the final list of applications is TBD, so this list will be revisited to determine if there are applications that need to be added, or removed to reflect what will be in UNR. 

Favorites
---------
Cheese
Evolution
Firefox
Yelp
Pidgin

Accessories
-----------
add/remove applications (gnome-app-install)
calculator (gcalctool)
Character map (gucharmap)
Disk usage analyzer (baobab)
manage print jobs
passwords and encryption keys (seahorse)
take screenshot (gnome-screenshot)
terminal (gnome-terminal)
text-editor (gedit)
tomboy

all games from gnome-games package

Graphics
--------
f-spot
openoffice.org drawing

Office
------
Dictionary
Ebook-reader
Openoffice.org database
Openoffice.org presentation
Openoffice.org spreadsheet
Openoffice.org Word processor

Sound and video
---------------
Movie Player
Rhythmbox
sound recorder

System tools
------------
cd/dvd creator (nautilus)

Universal Access
----------------
cellwriter

Preferences
-----------
about me
appearance
assistive technologies
bluetooth
default printer
display keyboard
keyboard shortcuts
main menu
mouse
network connections
network proxy
palmos devices
power management
preferred applications
remote desktop
scim input method setup
screensaver
sound
startup applications
switch desktop mode (desktop-switcher)
windows

Administration
--------------
authorizations
calibrate touchscreen
hardware drivers
language support
log file viewer
login window
network tools
printing
services
software sources
synaptic package manager
system monitor
* system testing (this is checkbox and should be skipped for purposes of this testing)
time and date
update manager
update manager
users and groups

CategorySpec

Specs/MobileUNRKarmicComplianceAutotesting (last edited 2009-06-26 21:51:20 by 75-27-137-193)