Testing and Validating Machine Learning Classifiers by Metamorphic Testing

Machine Learning algorithms have provided core functionality to many application domains - such as bioinformatics, computational linguistics, etc. However, it is difficult to detect faults in such applications because often there is no "test oracle" to verify the correctness of the compute...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:The Journal of systems and software. - 1998. - 84(2011), 4 vom: 01. Apr., Seite 544-558
1. Verfasser: Xie, Xiaoyuan (VerfasserIn)
Weitere Verfasser: Ho, Joshua W K, Murphy, Christian, Kaiser, Gail, Xu, Baowen, Chen, Tsong Yueh
Format: Aufsatz
Sprache:English
Veröffentlicht: 2011
Zugriff auf das übergeordnete Werk:The Journal of systems and software
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM207854068
003 DE-627
005 20250212172724.0
007 tu
008 231224s2011 xx ||||| 00| ||eng c
028 5 2 |a pubmed25n0693.xml 
035 |a (DE-627)NLM207854068 
035 |a (NLM)21532969 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Xie, Xiaoyuan  |e verfasserin  |4 aut 
245 1 0 |a Testing and Validating Machine Learning Classifiers by Metamorphic Testing 
264 1 |c 2011 
336 |a Text  |b txt  |2 rdacontent 
337 |a ohne Hilfsmittel zu benutzen  |b n  |2 rdamedia 
338 |a Band  |b nc  |2 rdacarrier 
500 |a Date Revised 20.10.2021 
500 |a published: Print 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Machine Learning algorithms have provided core functionality to many application domains - such as bioinformatics, computational linguistics, etc. However, it is difficult to detect faults in such applications because often there is no "test oracle" to verify the correctness of the computed outputs. To help address the software quality, in this paper we present a technique for testing the implementations of machine learning classification algorithms which support such applications. Our approach is based on the technique "metamorphic testing", which has been shown to be effective to alleviate the oracle problem. Also presented include a case study on a real-world machine learning application framework, and a discussion of how programmers implementing machine learning algorithms can avoid the common pitfalls discovered in our study. We also conduct mutation analysis and cross-validation, which reveal that our method has high effectiveness in killing mutants, and that observing expected cross-validation result alone is not sufficiently effective to detect faults in a supervised classification program. The effectiveness of metamorphic testing is further confirmed by the detection of real faults in a popular open-source classification program 
650 4 |a Journal Article 
700 1 |a Ho, Joshua W K  |e verfasserin  |4 aut 
700 1 |a Murphy, Christian  |e verfasserin  |4 aut 
700 1 |a Kaiser, Gail  |e verfasserin  |4 aut 
700 1 |a Xu, Baowen  |e verfasserin  |4 aut 
700 1 |a Chen, Tsong Yueh  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t The Journal of systems and software  |d 1998  |g 84(2011), 4 vom: 01. Apr., Seite 544-558  |w (DE-627)NLM098175688  |x 0164-1212  |7 nnns 
773 1 8 |g volume:84  |g year:2011  |g number:4  |g day:01  |g month:04  |g pages:544-558 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 84  |j 2011  |e 4  |b 01  |c 04  |h 544-558