OOoCon 2005

Doing QA @ OpenOffice.org

Joost Andrae

Sun Microsystems GmbH

Agenda

Speaker Introduction

Overview

How can you help?

confirming issues

submitting issues

gather feedback from various sources (Users mailing list, discussion forums, etc.)

Child workspaces from QA perspective

Information about OpenOffice.org automation

About crash reporting and how it helps OpenOffice.org development

QA project statistics

Localization (l10n) team collaboration with QA

Q & A

About the author

Biography:

Born in 1965

Since 1995 I'm working on the StarOffice product which later open sourced as OpenOffice.org

Started as a QA engineer doing QA & testing in Calc, Chart, Math and on Sun ONE Webtop.

Later doing QA & Coordination within StarOffice development at Sun Microsystems in Hamburg

Current position is StarOffice Program Manager as part of the StarOffice Operations team within StarOffice.

This year I took over the role as Co-Lead of the QA team on OpenOffice.org

How can you help

register at OpenOffice.org QA project list at
dev-subscribe@qa.openoffice.org

read documentation at qa.openoffice.org

http://qa.openoffice.org/helping.html

http://qa.openoffice.org/issue_handling/index.html

http://qa.openoffice.org/priorities.html

ask questions within the mailing list

if you want to automate then read

http://qa.openoffice.org/qatesttool/index.html

http://qa.openoffice.org/qadevOOo_doc/index.html

test localizations and read

http://qa.openoffice.org/localized/index.html

Confirming issues

login into http://qa.openoffice.org

load http://qa.openoffice.org/issuelinks.html and
select one of the links of unconfirmed issues

select each issue of type 'defect' one by one and try to reproduce the issue. Please take care about the operating system for which the issue has been written. Often the OS is not important to reproduce the issue but sometimes it is.

If the issue is reproducible within the current build then please add the keyword 'oooqa' and change the status to 'new'. Please write some words about the confirmation and if perhaps additional items are necessary to reproduce the issue. If the issue is reproducible in an old build then this info isn't needed

Submitting issues

What to do if you find an issue:

login into http://qa.openoffice.org

select the 'Issue tracker' link

select the link 'defect'

select the module (eg. Word Processor:)

enter the summary, description, 'found in version', platform and OS

for the subcomponent it's mostly 'ui' for user interface and 'code' for the core functionality

the priority unlikely is P1 because this would mean a breakage of the build or a core component (like Calc) which doesn't start. P2 can be taken for every kind of crash and main functionality regressions. P3 can be everything else. Details can be taken from http://qa.openoffice.org/scdocs/ddIssues_EnterModify.html#priority

Gathering feedback

Developers don't have much time for endless discussions within high traffic newsgroups and mailing lists. You can really help by providing feedback to users who seem to have a problem with the program. Sometimes you can get valuable information about a reproducible scenario for issues which you can file into IssueTracker.

Child workspaces from
QA perspective

Development uses copies of a CVS master to work on bugs and features. Such copies are called child workspaces. For collaboration purpose some of these CWS are uploaded to http://ooomisc.services.openoffice.org/pub/OpenOffice.org/cws/upload/. Currently most CWS integrations are QAed by Sun QA engineers but it's likely to get involved into CWS QA but the amount of data would be too much to have all CWS builds to be put into the CWS download area. At http://eis.services.openoffice.org there is a tool where information about CWS is published and where you can see the state of a CWS.

Child workspaces

The status phase of a CWS usually is

planned, new, ready for QA, approved by QA and integrated.

If a CWS contains new features then the specification needs to be documented. Depending on the complexity of a CWS at least first- and topten autotests need to be finished successfully to ensure main functionality. Some CWS need additional automation before the CWS can be approved by QA. After approval by QA each CWS gets a review before the code gets an approval for integration by a release engineer. If a QA engineer encounters a regression then the CWS will get a reset to 'New' and the approval process starts at the beginning.

About OOo automation

There are two types of OOo automation:

API based automation:

http://qa.openoffice.org/qadevOOo_doc/index.html

There are scripts for C/C++, Java, Python or Basic which can be used to ensure the functionality and reliability of specified APIs

GUI based automation:

http://qa.openoffice.org/qatesttool/index.html

A test tool is used to test the functionality which is available via graphical user interface. The programming language used is a BASIC dialect. Almost everything within the office can be accessed and tested.

About error reporting

If the application crashes then the error reporting module starts. In OpenOffice.org 2.0 this functionality has been combined with a document recovery dialog.

Besides other information an error report contains information about language, operating system, build ID and it contains a call stack which includes register information about the last steps that the soffice.bin process did before it crashed.

If you're interested into this then have a look at the preview dialog within the error reporter application when the soffice.bin process crashes. Please send an error report if the application crashes because it helps development a lot

QA project statistics

unconfirmed

confirmed

QA project statistics

today

last year

QA project statistics

today

last year

QA project statistics

today

last year

QA project statistics

today

last year

incoming

fixed

QA project statistics

today

last year

Collaboration with l10n project

Last year we started collaboration with the l10n team. In the mean time there were several localizations which have passed the QA tests of their localizations. If you're interested into getting involved then get in contact with your localization project team lead and please read http://qa.openoffice.org/localized/index.html

If you have questions regarding l10n tests then please ask within the qa mailing list and/or within the l10n mailing list to get in contact with people who can help you.

Q&A

Joost Andrae

2005-09-21T12:57:32