Towards an Effective Software Component Certification Process Advisor Silvio Lemos Meira [email protected] Student Alexandre Alvaro [email protected]

Towards an Effective Software Component Certification

Process

AdvisorSilvio Lemos Meira

[email protected]

StudentAlexandre Alvaro

[email protected]

Agenda

• Introduction– Reuse– RiSE {Reuse in Software Engineering}– CBD {Component-Based Development}– Component– Quality– Component Certification

• Component Certification History• Future

Introduction {Reuse}

• [Frakes, 1995]•“Software reuse is the use of existing software knowledge or artifacts to build new software artifacts.”

•“Everything that was done, should not be done again“

Introduction {Reuse}

• Advantages• Increase

• Quality• Productivity

• Decrease • Time to market • Manutenability

• Reduce life-cycle of development

• Software Reuse…– Not simple– Systematic Software Reuse

Introduction {RiSE}

• [Almeida et al., 2004]• The IEEE International Conference on Information Reuse and Integration.

http://www.cin.ufpe.br/~rise

Introduction {CBD}

• Software reuse idea’s is not new• [McIlroy, 1969]

“Mass Produced Software Components”

• “To (re)use”, instead of “to develop”• To keep a set of reused components

•Repository systems•[Mili, 1998]

• 50 proposed solutions to this problem

Introduction {Component}

“ A software component is a unit of composition with contractually specified

interfaces and explicit context dependencies only. A software component

can be independently deployed and is subject to third-party composition”.

“ A software component is a unit of composition with contractually specified

interfaces and explicit context dependencies only. A software component

can be independently deployed and is subject to third-party composition”.

[Szyperski, 2002]

• The exactly concept of component in CBD is not yet a consensus...


• [Bass et al., 2000]• CMU/SEI’s report

• Inhibitors:• Lack of available components;• Lack of stable standards for component technology;• Lack of certified components;• Lack of an engineering method to consistently

produce quality systems from components.


• Besides CMU/SEI...

• [Heineman, 2000]• [Councill, 2001]• [Crnkovic, 2001]• [Wallnau, 2003]

Introduction {Quality}

ISO 9000

CMM

[Weber & Nascimento, 2002]

Introduction {Component Certification}

• Concept...

“Third-party certification is a method to ensure that software components conform to well-defined standards; based on this

certification, trusted assemblies of components can be constructed.”

“Third-party certification is a method to ensure that software components conform to well-defined standards; based on this

certification, trusted assemblies of components can be constructed.”

[Councill, 2001]



• [Frakes et al., 1996]

History of Component Certification

•History of the Software Component Certification

• Decade of 90– Mathematical models– Test-Based models

• [Poore et al., 1993] “Planning and certifying software system reliability”

– Three mathematics model• Test cases • Report the failures• Data are analyzed to achieve a reliability index

– Reliability of system• Considering how the components affects this

reliability


• [Wohlin & Runeson, 1994]“Certification of Software Components”

– Method that consist:• Usage model• Usage profile

– Test cases based on this models– Collection the failure data– Certification of reliability

– Hypothesis certification• Certify a specific reliability level -> given degree of

confidence

– Reutilization degree of the models


• [Rohde et al., 1996]“Certification of Reusable Software Components”

– Rome Laboratory of the Air Force, NY


• [Rohde et al., 1996]“Certification of Reusable Software Components”

– Certification process:1. Readiness Assessment

– Compile without error and execute the code

2. Static Analysis– Automatic tool

3. Code Inspection– Manual technique

4. Testing – Other tests

– Analysis of the certification process…


• [TCI Initiative, 1998]1

– Affiliation of researchers• Formal interface specification

– Supports compositional reasoning • A restricted set of behavioral properties of

assemblies

– Difficult to find real contributions…1 http://www.trusted-components.org


• [Voas, 1998]“Certifying Off-the-Shelf Software Components”

– Automated technologies• Black-Box testing and fault injection

– Methodology:• Black-box component testing • System-level fault injection • Operational system testing

– Certify components to a determined environment...


• [Wohlin & Regnell, 1998]“Reliability Certification of Software Components”– Extend [Wohlin & Runeson, 1994] work

– Certification process1. Usage specification (usage model and usage

profile)2. Certification procedure• Three approaches:

– Certification Process – Reliability Certification of Component and Systems – Certify or Derive System Reliability






?

• [Voas & Payne, 2000]“Dependability Certification of Software

Components”

– Metrics framework– Create a tests methodology…

• Component testability score– Mathematical models– Statistics approaches

• Estimates the number of test cases necessary • Consider:

– The number of tests that a component received; – The “fault revealing” ability of those test cases.


• [Morris et al., 2001]“Software Component Certification”

– Four steps:• Tests Specification• Specification Document• Specified Results • Test-Pattern Verificator

– Limitations…


• However...– Testing is not enough...

• [Sametinger, 1997]– Component certification levels

• Level 1: A component is described with keywords and a summary and is stored for automatic search. No tests are performed; the degree of completeness is unknown;

• Level 2: A source code component must be compiled and metrics are determined;

• Level 3: Testing, test data, and test results are added;• Level 4: A reuse manual is added.


• [Heineman et al., 2000]– Panel presented in ICSE’2000– Discuss the necessity of trust assurance in

component

– Considerable CBD researchers participate:• Heineman {organizations}• Councill {software development}• Flynt {benefits to the customers}• Shaw {reutilization of the components}


• Workshops…

– 4th ICSE Workshop on Component-Based Software Engineering (CBSE): Component Certification and System Prediction, 2001.

– 5th ICSE Workshop on Component- Based Software Engineering (CBSE): Benchmarks for Predictable Assembly, 2002.


• Long time considering just test...• [Stafford & Wallnau, 2001]

“Is Third Party Certification Necessary?”

– Define a process model…• Support prediction of system properties prior

to component selection

– Introduce “credentials” concept• <property,value,credibility>

– Active component dossier• A dossier is an abstract component that

defines certain credentials


• Long time considering just test...• [Stafford & Wallnau, 2001]

“Is Third Party Certification Necessary?”


• [Stafford & Wallnau, 2001]“Is Third Party Certification Necessary?”

– Some open questions:•What level of trust is required? •Are there other mechanisms that might

be used to support trust?

•How to certify measurement techniques?


• Other authors…•How certification should be carried out?

(Goulao & Abreu, 2002)•What does it mean to trust a component?

(Hissam et al., 2003)•What characteristics of a component

make it certifiable, and what kinds of component properties can be certified? (Wallnau, 2003)


• [Councill, 2001]“Third-Party Certification and Its Required

Elements”

– Other aspect of component certification…• Human• Industrial• Business

– Certification is the components future…


• [Woodman et al., 2001]“Issues of CBD Product Quality and Process

Quality”

– Analyze some process in various CBD approaches

– Examine 11 potential CBD quality attributes

Reusability Maintainability Accuracy Clarity

Replaceability Interoperability Scalability Performance

Flexibility Adaptability Reliability


Reusability Maintainability Accuracy Clarity

Replaceability Interoperability Scalability Performance

Flexibility Adaptability Reliability

• [Hissam & Wallnau, 2003]“Enabling Predictable Assembly”– Extends the [Stafford & Wallnau, 2001]work

– Introduced Prediction-Enabled Component Technology (PECT)

– Component technology with analysis technology• Prediction of assembly properties

• Identify required component properties

• Certifiable properties



– Component technology and analysis technology

• Component model• Component runtime environment• Assembly environment


• Defines a property theory

• Parameters of this theory• Component properties

Increased accuracy prediction both

More abstract, less acurate

Increased accuracy prediction


– Validation:• Empirical

– Predictions made, conform to observations

– Limitations• Two prediction technology may be incompatible• How are non-resource attributes, such as

security, to be empirically validated?• Industrial component certification ?


• [Meyer, 2003]“The Grand Challenge of Trusted Components”

– Two complementary roads:•Low Road

– Qualification of existing components

•High Road– Production of components with fully proved correctness

properties.


• [Meyer, 2003]“The Grand Challenge of Trusted Components”


• [McGregor, 2003]“Measuring Component Reliability”

– Support prediction of assemblies reliabilities based on properties of the components

– Method to measuring and communicating the reliability of the component• Component’s services

– Component’s documentation

• Test plan is created, based on component’s services• Provide the reliability of each service

– This method is a fundamental element of PECT


• [Wallnau, 2003]“Volume III: A Technology for Predictable Assembly

from Certifiable Components”• CMU/SEI’s report

– How component technology can be extended in order to achieve Predictable Assembly from Certifiable Components (PACC).

• Runtime behavior of software components assemblies

• Component’s property– Component’s proprieties need rigorously defined and trusted; and

– It can be certified by independent third-party developers




– SEI’s approach to PACC is PECT.




– Status:• On going work

– PECT is relatively immature

• One or more certification properties…• Tools are being developed• Functional certification complements the PECT [Meyer, 2003]

– Precondition to PECT

• Non-functional properties


• Two failures case....– National Information Assurance

Partnership (NIAP) • Together with NIST and NSA• From 1993 until 1996• Defines criteria for certifying security features of

components • Restricted set of behavioral assembly properties.

– IEEE• 1997• The initiative was suspended, in this same year…


Summary

Future – RiSE Context

• [Almeida et al., 2004]• The IEEE International Conference on Information Reuse and Integration.

http://www.cin.ufpe.br/~rise

Future work

1. Key CBD Requirements• What are the requirements for a certification

process? • [Woodman et al., 2001]

• 11 CBD requirements

• [Simao,2003]• 124 CBD requirements

• [Larson, 2004]• 72 CBD requirements

2. Component Quality Model• What requirements are more important ?• [Meyer, 2003]

Future work

3. Certification Method• How certify components ?

4. A Metrics Framework• How to measure the component certification

processes ?

Future work

• Write a Paper– “On the Software Component

Certification Process”• The history• The proposal

Future work

References

• [Frakes, 1995] Frakes, W., B., Fox, C., J. Sixteen Questions about Software Reuse. Communications of the ACM, June, 1995.

• [Szyperski, 2002] Szyperski, C., 2002. Component Software: Beyond Object-Oriented Programming. Addison-Wesley, USA. ISBN 0-201-74572-0.

• [Mcllroy, 1968] Mcllroy, M. D., 1968. Mass Produced Software Components. NATO Software Engineering Conference Report, October, pp. 79-85.

• [Mili et al., 1998] Mili, A., Mili, R., Mittermeir, R., 1998. A Survey of Software Reuse Libraries. Annals Software Engineering, Vol. 05, pp. 349–414.

• [Heineman & Councill, 2001] Heineman, G. T., Councill, W. T., 2001. Component-Based Software Engineering: Putting the Pieces Together. Addison-Wesley, USA. ISBN: 0-201-70485-4.

• [Heineman et al., 2000] Heineman, G. T., Councill, W. T., Flynt, J. S., Mehta, A., Speed, J. R., Shaw, M., 2000. Component-Based Software Engineering and the Issue of Trust. The IEEE Proceedings of the 22nd International Conference on Software Engineering (ICSE), Canada, pp. 661-664.

• [Crnkovic, 2001] Crnkovic, I., 2001. Component-based software engineering - new challenges in software development. Software Focus, Vol. 2, No. 4, pp. 27-133.

References

• [Wallnau, 2003] Wallnau, K. C., 2003. Volume III: A Technology for Predictable Assembly from Certifiable Components. Software Engineering Institute (SEI), Technical Report, Vol. III, April.

• [Frakes & Terry, 1996] Frakes, W., Terry, C., 1996. Software Reuse: Metrics and Models. ACM Computing Survey, Vol. 28, No. 2, June, pp. 415-435.

• [Poore et al., 1993] Poore, J., Mills, H., Mutchler, D., 1993. Planning and certifying software system reliability. IEEE Computer, Vol. 10, No. 1, January, pp. 88-99.

• [Wohlin & Runeson, 1994] Wohlin, C., Runeson, P., 1994. Certification of Software Components. IEEE Transactions on Software Engineering, Vol. 20, No. 6, June, pp. 494-499.

• [Rohde et al., 1996] Rohde, S. L., Dyson, K. A., Geriner, P. T., Cerino, D. A., 1996. Certification of Reusable Software Components: Summary of Work in Progress. The IEEE Proceedings of the 2nd International Conference on Engineering of Complex Computer Systems (ICECCS), Canada, pp. 120-123.

References

• [Voas, 1998] Voas, J. M., 1998. Certifying Off-the-Shelf Software Components. IEEE Computer, Vol. 31, No. 6, June, pp. 53-59.

• [Wohlin & Regnell, 1998] Wohlin, C., Regnell, B., 1998. Reliability Certification of Software Components. The IEEE Proceedings of the 5th International Conference on Software Reuse (ICSR), Canada, pp 56-65.

• [Voas & Payne, 2000] Voas, J. M., Payne, J., 2000. Dependability Certification of Software Components. Journal of Systems and Software, Vol. 52, No.2-3, June, pp. 165-172.

• [Morris et al., 2001] Morris, J., Lee, G., Parker, K., Bundell, G. A., Lam, C. P., 2001. Software Component Certification. IEEE Computer, Vol. 34, No. 9, September, pp. 30-36.

• [Sametinger, 1997] Sametinger, J., 1997. Software Engineering with Reusable Components. Springer Verlag, USA. ISBN 3-540-62695-6.

References

• [Stafford & Wallnau, 2001] Stafford, J., Wallnau, K. C., 2001. Is Third Party Certification Necessary?. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May, pp. 13–17.

• [Councill, 2001] Councill, B., 2001. Third-Party Certification and Its Required Elements. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May.

• [Woodman et al., 2001] Woodman, M., Benebiktsson, O., Lefever, B., Stallinger, F., 2001. Issues of CBD Product Quality and Process Quality. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May.

• [Hissam et al., 2003] Hissam, S. A., Moreno, G. A., Stafford, J., Wallnau, K. C., 2003. Enabling Predictable Assembly. Journal of Systems and Software, Vol. 65, No. 3, March, pp. 185-198.

• [Meyer, 2003] Meyer, B., 2003. The Grand Challenge of Trusted Components. The IEEE Proceedings of 25th International Conference on Software Engineering (ICSE), USA, pp. 660–667.

References

• [McGregor et al., 2003] McGregor, J. D., Stafford, J. A., Cho, I. H., 2003. Measuring Component Reliability. The IEEE Proceedings of the 6th ICSE Workshop on Component-Based Software Engineering (CBSE), USA, May, pp. 13-24.

• [Schmidt, 2003] Schmidt, H., 2003. Trustworthy components: compositionality and prediction. Journal of Systems and Software, Vol. 65, No. 3, March, pp. 215-225.

• [Simão, 2003] R. Simao, A. Belchior, Quality Characteristics for Software Components: Hierarchy and Quality Guides. Lecture Notes in Computer Science, pp. 188-211, June. Springer-Verlag. 2003.

• [Larson, 2004] M. Larson, Predicting Quality Attributes in Component-based Software Systems, PhD Thesis, Malardalen University, 2004.

Documents

Towards an Effective Software Component Certification Process Advisor Silvio Lemos Meira [email protected] Student Alexandre Alvaro [email protected]