Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
1
M. Franke, A. M. Franke, A. WodrichWodrich
Internet Internet AidedAided SoftwarevalidationSoftwarevalidation
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Sou
rce:
ZE
ISS
, Klin
geln
berg
, Hex
agon
, Jen
optik
, Mah
rSoftware in Software in coordinatecoordinate metrologymetrology
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
2
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Sou
rce:
ZE
ISS
, Klin
geln
berg
, Hex
agon
, Jen
optik
, Mah
r
Evaluation Evaluation softwaresoftware
MeasurandMeasurand
Software in Software in coordinatecoordinate metrologymetrology
�� HardwareHardware�� MachineMachine--oriented softwareoriented software
��Physical standardsPhysical standards
�� AlgorithmsAlgorithms
��Numerical standardsNumerical standards
Point Point cloudcloud
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Software Software bugsbugs
The first one?The first one?
Found in the electromechanical computer Mark IIFound in the electromechanical computer Mark II
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
3
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Software Software bugsbugs
�� On 4 June 1996 the maiden flight of the On 4 June 1996 the maiden flight of the ArianeAriane 5 5 launcher ended in a failurelauncher ended in a failure
�� Reason: software bugReason: software bug
�� Conversion from float to integer failedConversion from float to integer failed
�� CostsCosts: : ~125 000 000~125 000 000€€ startingstarting costscosts~425 000 000~425 000 000€€ destroyeddestroyed cargocargo
loadload (4 (4 satelitessatelites))~600 000 000~600 000 000€€ followfollow--upup costscosts
horizontal_veloc_bias := integer(horizontal_veloc_sensor);
�� Only a single line of code leads to a catastrophe!Only a single line of code leads to a catastrophe!
One of the most expensive:One of the most expensive:
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
MotivationMotivation
Software is an essential component in coordinate Software is an essential component in coordinate metrologymetrology
�� Existing solutions (Softwaretest for Gaussian assoc iated features PTB, Existing solutions (Softwaretest for Gaussian assoc iated features PTB, NIST, NPL …, ISO 10360NIST, NPL …, ISO 10360--6, B89.4) cover only a very small part of modern 6, B89.4) cover only a very small part of modern software software
�� Existing process is manually and therefore prone to errorExisting process is manually and therefore prone to error
�� Test not reproducible for end users, e.g. after sof tware updatesTest not reproducible for end users, e.g. after sof tware updates
The correctness of the evaluation The correctness of the evaluation algorithms usually not been provedalgorithms usually not been proved
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
4
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
EU Project „Testing Coordinate Measuring Machine Al gorithms, Phase II“, 1991EU Project „Testing Coordinate Measuring Machine Al gorithms, Phase II“, 1991
�� GaußianGaußian best fitbest fit�� Geometrical elements:Geometrical elements:
Straight line, plane, circle, cylinder, cone, sphereStraight line, plane, circle, cylinder, cone, sphere�� Extend: 44 datasetsExtend: 44 datasets�� 17 participants17 participants�� 37%37% of the submitted solutions are acceptable of the submitted solutions are acceptable �� 18% 18% are questionableare questionable�� 45%45% are not tolerableare not tolerable�� only 2 participants delivered completely acceptable solutions for all datasetsonly 2 participants delivered completely acceptable solutions for all datasets
Regular test service by PTB: Regular test service by PTB: SoftwaretestSoftwaretest based on Phase II based on Phase II (Figures from 2007(Figures from 2007--2010)2010)
�� 20 Tests performed20 Tests performed�� 7 not passed at the first try7 not passed at the first try�� Problems experienced with each element typeProblems experienced with each element type
Failures / Type of element
Cone (6/7) IIIII I
Cylinder (5/7) IIIII
Plane (2/7) II
Line (4/7) IIII
3D-Circle (3/7) III
Sphere (1/7) ISubject matter is still relevant!Subject matter is still relevant!
Experience Experience ofof PTBPTB
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
SolutionSolution
IdeaIdea�� Supply a service for Supply a service for online validation online validation of metrological softwareof metrological software
VisionVision�� Test the correctness of mathematical algorithms integrated in Test the correctness of mathematical algorithms integrated in
metrological software metrological software –– at any time at any time and and any placeany place
PrinciplePrinciple�� A A central servercentral server holds or generates validated reference dataholds or generates validated reference data�� By By well defined protocols well defined protocols the software under test requests the the software under test requests the
data and submits test resultsdata and submits test results�� The server generates a The server generates a digital certificatedigital certificate , confirming the , confirming the
succesfulsuccesful testtest�� Possible access: online (via Possible access: online (via webservicewebservice) or offline (via ) or offline (via webformwebform))
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
5
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
AdvantagesAdvantages
�� AutomatedAutomated softwaresoftware testingtesting forfor algorithmsalgorithms
�� ImplementationImplementation accordingaccording toto international international standardsstandards andand guidelinesguidelines::
�� ISO 10360ISO 10360--6, ASME B89.4,…6, ASME B89.4,…
�� ExtensibleExtensible
�� GearGear softwaresoftware (GDE(GDE--REL)REL)
�� FreeformFreeform
�� Form Form measurementmeasurement
�� Filter Filter algorithmalgorithm
�� DirectDirect accessaccess toto validatedvalidated referencereference datadata
�� ForFor softwaresoftware manufacturersmanufacturers duringduring developmentdevelopment
�������� Quality Quality assuranceassurance
�� ForFor end end usersusers after after softwaresoftware updatesupdates
�������� ConfidenceConfidence buildingbuilding
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
CMM software
Sample pointsBest-fit geometriclal elements
Combine Report
Testing body
Reference data Test parameter
Reference parameter
Comparison & judgement
Converted test parameter
certificate
ManuellManuell workflowworkflow
Manual action needed!
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
6
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
CMM software
Sample pointsBest fit geometrical elements
Combine Report
IASV*
Reference data Test parameter
Reference parameter
Comparison & judgement
Converted test parameter
Digital certificate
Digital certificate
* Internet-Aided Software Validation system
Interfaces to be defined
Workflow using internet based serviceWorkflow using internet based service
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
NMIsNMIs ServerServer
Overview on the infrastructureOverview on the infrastructure
ReferenceReference--softwaresoftware
Data generatorData generator
CustomerCustomer
�� Expert knowledgeExpert knowledge�� Cross checking of Cross checking of
different reference different reference softwaresoftware�������� validated test datavalidated test data
�� Test tasksTest tasks�� Reference dataReference data�� Reference parameterReference parameter�� Permissible deviationPermissible deviation
�� User of measuring User of measuring equipmentequipment
�� Vendor of evaluation Vendor of evaluation softwaresoftware
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
7
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Webservice-Client Software under testApplication serverXML-Webservice
Data base
Referencedata
Parameter:Reference +
Test
Administrativedata
App
licat
ion
logi
c
ApplicationSoftware I
Stand-alone-webclient
Webform
ApplicationSoftware III
ApplicationSoftware II
Web
serv
ice
System layoutSystem layout
IASV-components◄ ► vendor / customer
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Validation processValidation process
ComponentsComponents
Administrative:Administrative:�� Type Type andand scopescope ofof thethe testtest ((productproduct: PTB Gauß, : PTB Gauß, GearGear, ISO 10360, ISO 10360--6, 6,
ASME B89.4)ASME B89.4)�� Customer Customer datadata ((namename, , adressadress, , softwaresoftware, ….), ….)�� Order Order datadata ((trackingtracking numbernumber, …), …)
Technical:Technical:�� DeliverDeliver testtest datadata ((cloudcloud ofof pointspoints))�� Software Software calculatescalculates thethe requestedrequested testtest parameterparameter andand sendssends resultsresults
back back toto thethe serverserver�� Test Test resultsresults areare comparedcompared withwith referencereference datadata on on thethe serverserver
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
8
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Content of data structureContent of data structure
Test task
Reference parameter
Test parameter
Performance parameter
Test set
1..n
Test data
(e.g. 3D koordinates)
1..n
Identification of test task,
Versioning
Defintion of task
(computational aim)
= input for software under test = Response of the IASV-Server= output of software under test
Identification of client
PTB Phase II, Best-Fit Gauß
Sphere, Best-Fit Gauß
Sphere3D.PI44.L2.XXX-YYY-ASD
Centre ( -39.850, 4.140, 8.180) Radius: 74.8830
Centre ( -39.8593, 4.1413, 8.1878) Radius: 74.8838
Deviation:Centre: 0.7764±0.0001Radius: 0.0008±0.0001
CloudFit 4.0.1.02.2006
-43.0804 -0.5598 82.7332-17.3040 24.4077 76.5901-29.5830 42.7305 71.5822-56.2379 18.3333 79.9582-10.4244 48.2501 60.9937
: : :
To
m
iib Rzyxbd ),,,(,min 00
2
2 =∑=
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
4.0022.9990.0
0.00.01.0
548.010
coordinate
normal vector
radius
84.70472075.0215 994.4039
0.00.00.0
….
…
coordinate
normal vector
ExampleExample ofof XML XML structurestructure
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
9
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Screenshot example programScreenshot example program
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
WebWeb--frontend of the IASV test serverfrontend of the IASV test server
URI of IASV server
OrderID
File to upload
NIMTech International Workshop22nd February 2011, Braunschweig, Germany_______________________________________________________________________________________
10
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Actual stateActual state
Realised components Realised components �� Transmission protocol (XML) Transmission protocol (XML) �� Database (Database (mySQLmySQL))�� Test serverTest server�� Test data for the current PTB test (Phase II)Test data for the current PTB test (Phase II)�� Some test data for the test according ISO 10360Some test data for the test according ISO 10360--66�� Example program (demonstrator) can be downloaded at:Example program (demonstrator) can be downloaded at:
http://iasv.ptb.de:8080/index.html http://iasv.ptb.de:8080/index.html
�� MAHR, Hexagon, MAHR, Hexagon, MitutoyoMitutoyo, , FrencoFrenco, ZEISS are testing the , ZEISS are testing the new servicenew service
Advances in Coordinate Measurement Techniques for Industrial ApplicationsNIMTech International Workshop
Thank you for your attention.Thank you for your attention.