Dynamics Model V&V

Topics related to QA and V&V of GMAT features

Dynamics Model V&V

Postby shughes » Thu Mar 08, 2012 10:10 pm

From inspecting the GUI, Script Interface, Requirements, and Draft Spec, here is my feature audit for dynamics models.

Initial Status of the Feature

We've spent a lot of time on the dynamics model base code and tests. With a few minor issues documented below, the numerics and tests are in good condition. We intentionally neglected the GUI, in hopes that we could refactor or rewrite the entire panel, now we will have to decide what to do. Given the number of issues I've identified during the initial audit, I'd like to review them as a group and decide on our initial approach to resolving each of them. I put in recommendations as a staring point for conversation. I'll check in the issues into JIRA after we agree on an initial approach to handling them. . Some issues are dependent and I'd like us all to look at the big picture before moving forward:

Here are resources to documentation for this feature

Tall Poles
  • Lot's force models implemented in the script and not in the GUI that we need to decide how to handle. Should we only support in the script? This feature is used by every GMAT user so it needs to be extra solid. I'd prefer implementing at least the easy, high value models in the GUI. For example, Relativistic Correction is just a check box.
  • Command mode settings are not working. This may be easy to fix, but because it works in GMAT functions and not in command mode I'm worried there is a design issue.

Issues

Issue 1: STM partials are not implemented for all forces
Recommendation: Throw a warning if STM is requested for forces that don't have partials implemented, add to user docs.
(Priority: P1, FixBy: 2012a)

Issue 2: MarsGRAMM not implemented in GUI or tested.
Recommendation: Don't include in production release, include as plugin. (Priority: P3, FixBy: Someday)

Issue 3: MSISE00 and MSISE86 are not avialable in GUI.
Recommendation: Add them to the list of Earth models, this is trivial (Priority: P1, FixBy: 2012a)

Issue 4: Tide model tests are meter level different from STK truth
Recommendation: Submit a minor bugand change tolerance in tc files. We don't even know if our models are the same and
the differences are very small. (Priority: P3, FixBy: Someday)

Issue 5: SPK propagator has many bugs.
Recommendation: Move to a plugin. If we have time, we can do V&V after all P1 features are ready. If not, we'll turn it off and call it a beta. I think we should put this early in the schedule because we need to know if pulling into a plugin is going to cause problems in base and GUI code.(Priority: P1, FixBy: 2012a)

Issue 6: STM tests don't meet tolerance.
Recommendation: Get FreeFlyer files and rerun with more accurate integration tolerances. I believe the truth data
was not generated with tight integration tolerances. (Priority: P2, FixBy: Production)

Issue 7: Setting ForceModel fields in Command mode does not work.
Recommendation: Look at the design and see why this is occurring. I can execute a complex force model test in a GMAT function, which is all executed in command mode, and the answers are correct. So I don't see why the behavior is wrong in when using a script. I hope we can fix this with some design and refactoring. As a last resort, if there is too much work, then we can throw an exception if setting fields in command mode. (Priority: P1, FixBy: 2012a)

Issue 8: Many dynamics model tests are failing because there are different number of rows in the truth files. This is most likely due to a
bug in STK that causes it to output too many rows in a report.
Recommendation: We should talk to STK rep to see when this bug was fixed and try rerunning the truth scenarios in that version. (Priority: P1, FixBy: 2012a)

Issue 9: Tide model tests are failing. They used to pass.
Recommendation: JIRA bug is checked in. Fix them. (Priority: P1, FixBy: 2012a)

Issue 10: Relativistic correction not available in GUI.
Recommendation: Add a check box for it. (Priority: P1, FixBy: 2012a)

Issue 11: Tide model not available in GUI.
Recommendation: Not sure, one possiblity is to only support via script. However, if command mode settings for propagator don't
support this (see issue above) then GUI user's won't be able to use this model. In that case, we should add to the GUI. (Priority: P2, FixBy: Production)

Issue 12: Drag Model Setup panel says "File Input: to be implemented"
Recommendation: Remove the text and uneditable widgets. (Priority: P1, FixBy: 2012a)

Issue 13: Propagator dialog box code is a mess and needs to be either refactored or thrown out and start from scratch.
Recommendation: Sigh. Wait on this and get what we have working. (Priority: P3, FixBy: Someday)

Issue 14: We currently only allow one Primary Body and it must be the same as the central body. However, the GUI let's you set them differently and then throws and a run-time exception.
Recommendation: Make one of the fields read only and dependent upon the other. Work closely with Bez because this change will break many regression tests. My suggestion is to make central body dependent upon Primary body because there is a lot of complexity with changing central body and interactions with other fields in the GUI. (Priority: P1, FixBy: 2012a)

Issue 15: SRP fields SRP.Flux, SRP.Nominal_Sun, and SRP.Flux_Pressure are not settable from the GUI. These two settings are currently missing from the spec as well. If my understanding is correct, SRP.Flux_Pressure is an obsolete parameter that is dependent upon SRP.Flux and SRP.NominalSun. We need to clarify what the SRP.Flux_Pressure is for and if it is obsolete then we should deprecate it.
Recommendation: Document the fields and show script examples. I'm torn over the GUI, given the command mode issue above, and the fact that every user of GMAT will use a propagator, I think we should consider putting in a setup panel for this. I suggest triaging as (Priority: P2, FixBy: Production)

Issue 16: Many error and warning messages are crytpic and not end-user oriented.
Recommendation: Time box this issue, organize output from all validation tests, and SPH and DJC spend a few days fixing the most cryptic error messages. (Priority: P3, FixBy: Someday)

Issue 17: SRP variational terms partially complete: the partial derivative when the spacecraft is in partial shadow is not included.
Recommendation: Document then change status to (Priority: P3, FixBy: Someday)

Issue 18: Some script fields are "turned off" when other script "flags" are switched causing you to comment out chunks of script to get a buildable script. For example if you are using MSISE90 drag and have flux fields set up, if you change Drag to None, the script won't build unless you comment out the flux fields.
Recommandation: Determine if there is a design reason for doing this and if it the behavior can be changed so that the fields are settable even when drag is None. If this is more than a few hours work, check in as (Priority: P3, FixBy: Someday).
Recommendation:
shughes
 
Posts: 443
Joined: Mon Jun 09, 2008 6:27 pm

Re: Dynamics Model V&V

Postby dev-wcs » Mon Mar 12, 2012 6:50 pm

I concur on issues 1, 2, 4, 6, 7, 8, 9, 10, 12, 13 (bummer), 14, 15, 16, 17, 18


Issues 3: This is only trivial if you hard-code them in (i.e. if you don't want to do it the right way). The panel should really be able to detect what atmosphere models are available (and then which celestial body goes with each) dynamically.

Issue 5: Not sure how much effort this is (seems like a lot).

Issue 11: Couldn't this just be implemented as a combo box? The GravityField is just expecting a string value for that.

Wendy
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Views expressed are my own and
do not necessarily reflect those
of my employer, or anyone else.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
dev-wcs
 
Posts: 97
Joined: Tue Jun 10, 2008 1:01 pm

Re: Dynamics Model V&V

Postby shughes » Tue Mar 13, 2012 5:34 pm

Regarding:

dev-wcs wrote:I concur on issues 1, 2, 4, 6, 7, 8, 9, 10, 12, 13 (bummer), 14, 15, 16, 17, 18
Issues 3: This is only trivial if you hard-code them in (i.e. if you don't want to do it the right way). The panel should really be able to detect what atmosphere models are available (and then which celestial body goes with each) dynamically.
Wendy


We should do it correctly or not do it.

Regarding:

dev-wcs wrote:
Issue 11: Couldn't this just be implemented as a combo box? The GravityField is just expecting a string value for that.

Wendy


This is a fairly simple field but may be coupled to how Earth is used? Most fields that are tied to a specific object are only active if that object is selected as the primary body. In that case, should we put the field inside of the PrimaryBody list box? I think so. IF a user changes PrimaryBody to Moon, do we leave the setting as is but grey out the field or leave active at all times? I'll think about this and clarify in the spec.
shughes
 
Posts: 443
Joined: Mon Jun 09, 2008 6:27 pm

Re: Dynamics Model V&V

Postby DJCinSB » Tue Mar 13, 2012 5:47 pm

I agree with the priorities on 1, 3, 4, 5, 7, 8, 10, 11, 12, 14, 16 (sort of)

On item 2, if we fix 3 by asking the engine for the list, it should just come along for the ride -- if the plugin is in place, it shows up; if not, it doesn't.

On 6: I elevated to P1 because we don't know what the issue is. But if it is understood, then P2 is fine.

On 9: Do we have a user that needs this fixed by the May release?

On 13: Every user interacts with this dialog box. If we are going to fix it, let's fix it early in the next set of work, so I made it P2. Unless we move to Qt!

On 15: I'd put doc'ing it P1, with a panel fix P2.

On 16: If there are frequently encountered messages, let's clean them up sooner than P3.

On 17: I said P2 but can live with P3

On 18: I said P2 at least to spec a design for it. We need a place to buffer unused fields for this one.
DJCinSB
 
Posts: 274
Joined: Mon Jun 09, 2008 3:57 pm

Re: Dynamics Model V&V

Postby shughes » Wed Mar 14, 2012 2:41 pm

Thanks for the responses! I will check in JIRA tickets for these based on our initial triage.

Darrel, Regarding Issue 7: Setting ForceModel fields in Command mode does not work. I spoke with Linda and she explained why we don't see this issue inside of GMAT functions. It is because Prop Setups are global in functions as specified in the requirements. So changes made to the "core" prop setup are automatically used by all Propagate commands.
shughes
 
Posts: 443
Joined: Mon Jun 09, 2008 6:27 pm


Return to QA and V&V

Who is online

Users browsing this forum: No registered users and 2 guests