Procedures for Testing New GMAT Features

Conversations pertaining to the testing of GMAT

Procedures for Testing New GMAT Features

Postby shughes » Wed May 06, 2009 2:35 pm

This is a first cut at a list of procedures for testing new GMAT features. The goal is to develop a high level process for rigorous validation and verification of new features or modifications. Any feedback on missing items, better process, different organization or approach is welcome!


Procedures for Testing New Feature Testing
  1. Review the list of work items categories and add any special areas required for the new feature
  2. Develop detailed list of work items for each category.
  3. Perform review of test items and modify according to recommendations
  4. Use Wide-Band Delphi to generate estimates for all work items.
  5. Create the schedule
  6. Do the work and monitor/update the schedule.
  7. Work with Developers to aid in debugging.
  8. Perform review of test results and close out if review determines the testing is complete.
  9. At completion, review how progress went when compared to initial schedule and determine lessons learned.

Testing Work Items Categories for New Features (not necessarily in order to be performed)
  • Update the Wiki users guide to contain all new fields and their ranges for the new feature or updates. (GUI range testing will be based on this documentation.)
  • Update the Wiki users guide to contain new screen captures for all new panels/tabs.
  • Write/update GUI panel tests for all panels/tabs and perform the tests
  • Write acceptance test scripts, generate truth data, and perform the tests
  • Write system test scripts, generate truth data, and perform the tests
  • Write range test scripts and perform the tests
  • Update the test matrix with new functionality and show coverage of new tests
  • Update test plans with descriptions of new tests
shughes
 
Posts: 443
Joined: Mon Jun 09, 2008 6:27 pm

Re: Procedures for Testing New GMAT Features

Postby shughes » Thu May 07, 2009 3:09 pm

Here is an update to ensure we're addressing the feature requirements

shughes wrote:
Procedures for Testing New Feature
  1. Obtain the list of detailed requirements for the feature
  2. Review the list of work items categories and add any special areas required for the new feature
  3. Develop detailed list of work items for each category.
  4. Perform review of test items and modify according to recommendations. Ensure test suite verifies all feature requirements.
  5. Use Wide-Band Delphi to generate estimates for all work items.
  6. Create the schedule
  7. Do the work and monitor/update the schedule.
  8. Work with Developers to aid in debugging.
  9. Perform review of test results. Verify that requirements are met and document unsatisfied requirements and defects.
  10. Close out if review determines the testing is complete.
  11. Add new tests to the regression test suite.
  12. At completion, review how progress went when compared to initial schedule and determine lessons learned.

Testing Work Items Categories for New Features (not necessarily in order to be performed)
  • Update the Wiki users guide to contain all new fields and their ranges for the new feature or updates. (GUI range testing will be based on this documentation.)
  • Update the Wiki users guide to contain new screen captures for all new panels/tabs.
  • Write/update GUI panel tests for all panels/tabs and perform the tests
  • Write acceptance test scripts, generate truth data, and perform the tests
  • Write system test scripts, generate truth data, and perform the tests
  • Write range test scripts and perform the tests
  • Update the test matrix with new functionality and show coverage of new tests
  • Update test plans with descriptions of new tests
shughes
 
Posts: 443
Joined: Mon Jun 09, 2008 6:27 pm

Proposed Test Procedure Breakdown

Postby shughes » Tue Jun 16, 2009 6:34 pm

Wiki
  • Update/add gui screen captures
  • Update/add field descriptions
(GUI range testing will be based on this documentation.)

GUI Panel Tests
  • Update/write gui validation tests

Test Scripts
  • Write acceptance test scripts
  • Write system test scripts
  • Write range test scripts and perform tests

Generate Truth Data
  • Use external system to generate truth

Test Matrix
  • Update the system test matrix

Test Plan
  • Update the system test plan
Last edited by space_egd on Thu Jul 02, 2009 6:35 pm, edited 1 time in total.
Reason: Renamed Duplicate Test Matrix to Test Plan and renamed GUI category
shughes
 
Posts: 443
Joined: Mon Jun 09, 2008 6:27 pm

Work Items Template

Postby space_egd » Thu Jul 02, 2009 8:35 pm

Below is a template that can be used when posting work items to the forum. Just cut and paste the contents of the phpbb code below into your new post and add work items to your list.

Code: Select all
[list]
[*] Wiki
[list]
[*]Update [insert object/command name] GUI screen captures
[*]Update [insert object/command name] wiki content[/list]

[*] GUI Panel Tests
[list]
[*]Write/Update [insert object/command name] GUI panel tests for all panels/tabs
[*]Perform the [insert object/command name] GUI tests[/list]

[*]Test Scripts
[list]
[*]Acceptance Test Scripts
[list]
[*]Insert Text[/list]

[*]System Test Scripts
[list]
[*]Insert Test Scripts[/list]

[*]Validation/Range Test Scripts
[list]
[*] Insert Test Scripts
[/list][/list]

[*]Test Matrix
[list]
[*]Update the test matrix with new functionality and show coverage of new tests[/list]

[*]Test Plan
[list]
[*]Update test plans with descriptions of new tests[/list][/list]
Last edited by space_egd on Tue Jul 07, 2009 9:58 pm, edited 1 time in total.
Reason: Updated text so users know to replace certain lines and removed erroneous text in validation section
space_egd
 
Posts: 357
Joined: Sun Jun 08, 2008 9:02 pm
Location: MD

Creating Work Item Spreadsheet on Wiki

Postby space_egd » Wed Jul 15, 2009 12:44 am

Below is the process to create work item spreadsheets and wiki pages on the GMAT wiki:
  1. Log onto the wiki (Must be part of the GMAT team group to successfully perform all the items in this list)
  2. Download the attached exportWorkItems2.tws file to your machine
  3. Scroll down the page until you see the TikiSheet folder in the left pane of the window
  4. Click the folder icon to expand the folder's contents to reveal the 'List TikiSheets' link
  5. Click the 'List TikiSheets' link
  6. Click the 'Create New Sheet' button, located in the top left portion of the main right pane of the wiki page
  7. In the title input field, type the name of your spreadsheet. (i.e. AttitudeWorkItems)
  8. Fill out the description input field if you'd like. Filling out this field is optional.
  9. Change the 'Header Row' value from 0 to 1
  10. Leave the rest of the input fields as their default value
  11. Click the Save button
  12. Click the TikiSheet link, located in the top left portion of the main right pane of the wiki page
  13. Type in the name of your new sheet into the Find input field and click the Find button
  14. Click on the Title link for your spreadsheet.
  15. Click the Import button
  16. Leave all other input fields the default
  17. Click the browse button and select the exportWorkItems2.tws file you downloaded to your machine. Click the Import button.
  18. Click the Edit button and add in the work items to the spreadsheet and Save your spreadsheet.
  19. Record what the sheet ID for your new spreadsheet is. It will be in the URL.
  20. Navigate to the following page to add a new wiki:
    http://gmat.ed-pages.com/wiki/tiki-index.php?page=Testing+Status
  21. Edit the Testing Status wiki and add a link to the new wiki page which will house the spreadsheet.
    To add a new link to an internal wiki page use the following syntax:
    Code: Select all
    ((Name Of New Wiki Page Here))
  22. Save the wiki page edit
  23. Now you will see that the link you added to your new wiki page contains a red question mark to the right of the name.
  24. Click the red question mark to open the editor for that page.
  25. Add the following wiki syntax to allow the spreadsheet you created to appear on the wiki page:
    Replace MyTikiSheetIDhere with you tiki sheet Id
    Code: Select all
    {SHEET(id=MyTikiSheetIDhere)}{SHEET}
    ---
  26. Ta Da!!! now everyone on the testing team can update the spreadsheet easily
  27. Contact the wiki admin to add a link to your new wiki page on the left pane for easy access
Attachments
exportWorkItems2.tws
TikiWiki TikiSheet file
(34.97 KiB) Downloaded 341 times
Last edited by space_egd on Wed Jul 15, 2009 6:57 pm, edited 2 times in total.
Reason: changed forum to wiki
space_egd
 
Posts: 357
Joined: Sun Jun 08, 2008 9:02 pm
Location: MD

Re: Creating Work Item Spreadsheet on Wiki

Postby space_egd » Thu Jul 16, 2009 12:27 am

The instructions above on creating a spreadsheet on the wiki couldn’t be followed until now. There was a known bug for TikiWiki where non-admins couldn’t create TikiSheets.
It bothered me so I investigated how to fix the issue and reported my findings to TikiWiki’s existing bug for the issue (http://dev.tikiwiki.org/tiki-view_track ... temId=1920).

Long story short, I implemented the fix and you will now be able to follow the above instructions.
Last edited by space_egd on Thu Jul 16, 2009 12:28 am, edited 1 time in total.
Reason: Changed title
space_egd
 
Posts: 357
Joined: Sun Jun 08, 2008 9:02 pm
Location: MD


Return to Testing

Who is online

Users browsing this forum: No registered users and 2 guests

cron