Project 01-05SysTest

SYSTEST - Project Status


Return to Project 01-05 description

Summary

SYSTEST is a three-year project that started in March 2002. The project successfully completed its second of three years in March 2004. The project is on schedule and has been successful in developing usable methods and tools for system verification, validation, and testing (VVT). The primary products of SYSTEST are:

  • VVT Methodology Guidelines (VVTMG) - a 400+ page document containing general and specific guidelines for VVT.
  • VVT Process Model (VVTPM) - a software set that allows evaluating a VVT strategy in terms of its impact on cost, schedule, and risk.
  • Dissemination papers - a series of professional papers and presentations describing the SYSTEST results.

The VVTMG and VVTPM will be released for public use following the completion of the project. They are currently being evaluated in pilot projects as part of Work Package 6 (WP6) within each of the commercial partners. The figure below shows the project plan and status. All project tasks are on schedule.


Milestones and Accomplishments

Major milestones and accomplishments are:

Date Due
Ident
Task or Deliverable
Status
Mar 02 M0.1 Project Kickoff Completed
Aug 02 D1.1 Report on VVT state of the art Completed
Aug 02 D1.2 SYSTEST Project Technical Requirements Completed
Mar 03 M4.1 Evaluation of COTS tools Completed
Mar 03 D2.1 Methodology Assessment Criteria Completed
Mar 03 D2.2a VVT Methodology Guidelines (First Draft) Completed
Mar 03 M5.1 VVTMG Assimilation Kit (for pilot projects) Completed
May 03 D3.1 VVT Process Model Completed
Aug 03 M6.2 Start use of VVTMG and VVTPM on pilot projects Completed
Sep 03 M4.2 Integrated platform for VVT Process Model Completed
Sep 03 M6.1 First Internal SYSTEST Users Group Meeting Completed
Mar 04 M5.2 Assessment/Evaluation of the Assimilation Process (on pilot projects) Completed
Jan 04 D2.2b VVT Methodology Guidelines (Second Draft) Completed
Jun 04 M8.3 European SYSTEST Users Group Meeting (at INCOSE symposium, Toulouse) Completed
Dec 04 M6.2 Complete pilot projects evaluation  
Mar 05 D2.2c VVT Methodology Guidelines (final)  
Mar 05 D3.1a VVT Process Model (final)  

Papers and Presentations

More technical information on SYSTEST and its results can be found in the following extensive list of papers and presentations that have resulted from the project:

  • SYSTEST Project was presented at the 3rd European Systems Engineering Conference in Toulouse, France on 22-24, May 2002. Same presentation has been given at the INCOSE 2002, the 12th International Symposium in Las Vegas, Nevada, USA, on 28 Jul - 1 Aug. 2002.
  • VOLLERTHUN A., ENGEL A., HASKINS C. “Methodology Development for Systems Verification, Validation & Testing”, Conference paper presented at the 12th Annual International Symposium of INCOSE, Las Vegas, USA, 2002.
  • ENGEL A. "A Methodology for Modeling VVT Risk & Cost”. Conference paper presented at the 13th Annual International Symposium of INCOSE in Washington, DC, USA. June 29 - July 3, 2003.
  • HOPPE M., LÉVÁRDY V., VOLLERTHUN A., WENZEL S. "Interfacing the Verification, Validation, and Testing Process Model with Product Development Methods". Conference paper presented at the 13th Annual International Symposium of INCOSE in Washington, USA. July 1-3, 2003.
  • BUEHLER O., WEGENER J. “Evolutionary Functional Testing of an Automated Parking System”. International Conference on Computer, Communication and Control Technologies: CCCT '03 and The 9th. International Conference on Information Systems Analysis and Synthesis: ISAS '03, Orlando, Florida, USA. July 31- August 2, 2003.
  • HOPPE M., LÉVÁRDY V., IGENBERGS, E., "Das VVT Process Model für die Verifikations- und Validierungsplanung". Paper published in the Conference proceedings of the 15th Annual Spring Academy of the TU Munich and TU Budapest Balatonfüred, Hungary. May 17-21, 2003.
  • BARESEL A., POHLHEIM H., SADEGHIPOUR S., “Structural and Functional Sequence Test of Dynamic and State-Based Software with Evolutionary Algorithms”. Proceedings of the Genetic and Evolutionary Computation Conference - GECCO 2003, Chicago, Illinois, USA; pp. 2428 - 2441. July, 2003.
  • BARESEL A., STHAMER H. “Evolutionary Testing of Flag Conditions”. Proceedings of the Genetic and Evolutionary Computation Conference - GECCO 2003, Chicago, Illinois, USA; pp. 2442 - 2454. July, 2003.
  • ENGEL A., BARAD M. "A Methodology for Modeling VVT Risks and Costs”. Systems Engineering, Volume 6, issue 3, pp. 135 – 151, Wiley InterScience, Online ISSN: 1520-6858, Print ISSN: 1098-1241. July, 2003.
  • LÉVÁRDY V., HOPPE M., WENZEL S., VOLLERTHUN A. “Process Modeling Procedure for Verification, Validation and Testing Planning”. Conference paper was presented at the ASME 2003 Design Engineering Technical Conferences and Computers and Information in Engineering Conference (DETC2003/DTM-48682) Chicago. September 2-6, 2003.
  • LÉVÁRDY V., HOPPE M., “Systemtechnische Methodik zur Verifikations- und Validierungsplanung - Zwischenergebnisse des EU-Projekts SysTest”. Presented at the 3rd German Day of Systems Engineering, Munich, Germany. November 28, 2003.
  • BARESEL A., CONRAD M., SADEGHIPOUR S., WEGENER J. “The Interplay between Model Coverage and Code Coverage”. Paper to appear in the proceedings of the International European Conference on Software Testing, Analysis and Review. EuroSTAR 03.
  • SYSTEST project was presented by Avner Engel from IAI at the 4th Israeli System Engineering Conference in Haifa, Israel on 7 January 2004, as part of “The Modeling Systems’ VVT Risks and Costs” presentation.
  • BUEHLER, J., WEGENER, J. “Automatic Testing of an Autonomous Parking System using Evolutionary Computation”. SAE World Congress, Detroit, USA, March 8 – 11, 2004.
  • WEGENER, J.: “Test Case design by means of the Classification-Tree Method and the Classification-Tree Editor”. Proceedings of Test and Analysis of Component-based Systems (TACoS 2004), Barcelona, Spain, March 27 – April 04, 2004.
  • LÉVÁRDY V., HOPPE M. “Verification, Validation and Testing Strategy Planning Supported by a Process Model”. Conference paper published in the proceedings of the Tools and Methods for Competitive Engineering (TMCE) 2004 conference, Lausanne, Switzerland. April 12-16, 2004.
  • ENGEL A., BOGOMOLNI I., SHACHAR S., GRINMAN A. “Using historical cost and risk data to optimize the VVT process” (The 14th Annual International Symposium of INCOSE, Toulouse, France. June 20-24, 2004).
  • HOPPE M., LÉVÁRDY V., LEARDI C., MENDIKOA I., DE ABAJO N., GONZALEZ J., LOBATO V., PEREGRINA S. ”Application Experiences of the VVT Process Modeling Procedure at the Verification and Validation Planning”. Conference published in the proceedings of the 14th Annual International Symposium of INCOSE, Toulouse, France. June 20-24, 2004.
  • HOPPE M., LÉVÁRDY V., HASKINS C., BOGOMOLNI I., IGENBERGS E. “Generic Verification, Validation and Testing Methodology”. Tutorial presented at the 14th Annual International Symposium of INCOSE in Toulouse, France. June 20-24, 2004.
  • LÉVÁRDY V., HOPPE M., HONOUR E. ”Verification, Validation & Testing Strategy and Planning Procedure”. Conference paper published in the proceedings of the 14th Annual International Symposium of INCOSE, Toulouse, France. June 20-24, 2004.
  • LAMMERMANN F., BARESEL A., WEGENER J. “Evaluating Evolutionary Testability with Software Measurements”. Proceedings of the Genetic and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.
  • WEGENER J., BÜHLER O. “Evaluation of Different Fitness Functions for the Evolutionary Testing of an Autonomous Parking System”. Proceedings of the Genetic and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.
  • BARESEL A., STHAMER H., WEGENER J. “Applying Evolutionary Testing to Search for Critical Defects“. Proceedings of the Genetic and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.

Page last modified 25 Sep 04

Copyright 2003-2005 International Council on Systems Engineering