Ubuntu Wiki

• Launchpad Entry: mobile-qa-karmic-arm

• Created: 2009-04-20

• Contributors: PaulLarson

• Packages affected:

Summary

We need to have testplans and testcases in, place for QA efforts on ARM. These should pull from other sources where possible but may also have pieces that are specific to ARM.

Release Note

No user visible change in the distribution.

Rationale

Having a documented, repeatable process is useful for understanding the current state of test coverage, creating test schedules, and improving the effectiveness of the testing efforts.

User stories

Edmund would like to test Ubuntu on ARM and would like guidance and a repeatable process for doing so Susan has some tests that would be useful for testing Ubuntu on ARM and would like to see if they are already utilized in the testing process and add them if they are not.

Assumptions

Testcases already exist for Ubuntu that are not architecture specific, but will be applicable to testing on ARM and can be reused.

Design

Test and execution plans may be documented on wiki

Some testcases may be manual to begin with, but work should be done to either automate them, or add a guided version of the manual test to checkbox

Implementation

The testplan will be documented on a wiki page, until another test tracking system is established by QA.

Automated testcases added will be implemented in checkbox, or other automation suites that can be executed from checkbox.

Target new tests to be added for ARM specific components (see UDS notes)

Unresolved issues

BoF agenda and discussion

Jaunty image mirrors x86 desktop with additional ARM specific pieces New features to test include: video playback and graphics performance as drivers become available Some packages available for desktop not available due to build issues on arm

• test for parity with i386 desktop

Ensure checkbox and LDTP functionality for test automation cgregan to produce test script for listing all the current mobile testcases on the wiki more focus should be placed on drivers as they come in, less on applications work with OEM to improve mobile QA test methodologies

• Provide OEM team with test documentation

Targets for testing:

• hardware itself
• kernel
• installation
• upgrading kernel - for flash kernel and such

might be a unr image to test in addition to desktop

performance testing - boot time - graphics performance - watch videos - play music

package testing? seeds should be the same, manifest should be exacly same if there are packages not building on arm, we need to put in a bug for now, this is a good test

Testing sata also sd card reader usb ports sound, video ports there is a wiki page about all the ports it has anything not working is a bug -audio not working is not a hardware problem, but not so important for babbage 1 anymore, retest on babbage 2

order things happen when generating kernel could end up with old initramfs for instance, if you upgrade udev, then kernel, or reverse, or do at same time triggers are done at end of package installation could stomp on each other new tool to generate list of kernels

new drivers for many things need good coverage for drivers - somehow test multiple applications that hit those drivers as testcases, even if we don't care about those applications

cgregan: 50 device specific tests being added to checkbox that could help with this

CategorySpec