Filtering TOBIAS combinatorial test suites

Filtering TOBIAS combinatorial test suites Y. Ledru, L. du Bousquet, O. Maury, P. Bontron Laboratoire Logiciels, Systèmes, Réseaux - IMAG B.P. 72 - F-38402 - Saint Martin d'Hères Cedex – France Yves.Ledru, [email protected] Abstract TOBIAS is a combinatorial testing tool, aimed at the production of large test suites. In this paper, TOBIAS is applied to conformance tests for model-based specifications (expressed with assertions, pre and post-conditions) and associated implementations. The tool takes advantage of the executable character of VDM or JML assertions which provide an oracle for the testing process. Executing large test suites may require a lot of time. This paper shows how assertions can be exploited at generation time to filter the set of test cases, and at execution time to detect inconclusive test cases. Keywords: combinatorial testing, model-based specifications, VDM, JML Y. Ledru, L. du Bousquet, O. Maury, and P. Bontron. Filtering TOBIAS combinatorial test suites. In Proceedings of ETAPS/FASE’04 - Fundamental Approaches to Software Engineering, pp. 281-294, Vol. 2984 of LNCS, Springer-Verlag, Barcelona, 2004

LNCS proceedings are available at : http://www.springer.de/comp/lncs/index.html

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. ©2004 Springer-Verlag. The copyright for this contribution is held by Springer.

"!# $%& $&')(*(,+-.( /%0213)451687:9?6A@B1%689#CED2983:FG981HI;J.KL@L0?@L17 MONP987Q>G;8R@L17Q7:1&CED2SGHQ7:D2S T=UWV8XYQU Z[X\]YQ^%T_XI`W\ba:\b^:cbdegfihid[Z^j k%^de?lnôm d[ÛWp?qsrutwvyx{z |E} ~}8 r_r I rfiUW\]2Z{vyUWYQZ\]8 )^:j YQ^d{^2^q%rYQUGao^ Wi2i[i i 5% i8 2W? ?W¡ ?¢2£ ¤G¦¥?§ ¨g© ¬ª «_®¯°2±i® ²%³{É|tµx fs\bd)UnaXIkV2\]GUIZ[XWY[\bUc Z[^d[Z\]?`yZ[XiXWcµe=UW\bk%ô¶UIZ Z·G^)¸YQXr ipGaZ\bX¹Xº,cbUYQ` ^yZ[^d[Z)dp2\bZ[^d}-t»¼Z·\bd)¸GU¸8^:YenD2SAKB> ë=1yH:D2HNiKLKL>Oä29=NP7oNPS?H:116¹ëG>¶âD27:çêNiKå87:DGDiâo;1Wï?æ_Ni983:HQ@L021yH:13QH:@BSAä¼D7¬N#éDçyë8@LS=NiH:@BDS#Diâ ë=DHQænHQ1éæ8SA@BFG981W3IîGð)SÍâD27:HQ98S=NPHQ1KL>G; 3Q98éæH:1éæ8S8@LFG9813 Ni7:1 DiâH:1Sy0217:>¬6A@bñéW98KLH-H:D&Niå8å8KL>Gî ò[Só3:98éæôé Ni3Q1W3I;H:13QH:@LS8ä¹çêN >OéWD2SGHQ7:@BëA98HQ1#HQDõ@LS8é7:1INi3:1êH:æ81#éD2SÍì=681S8éW1OHQæ=NPHyHQæ81#@Lçyö å8KL1ç1WS?HNiH:@BDS&éDSGâD27:ç3H:D&@LHQ3E3Qå_1é@íì=éINPHQ@LD2S*î8÷ED2SGì=6A1S8é1.çêN >&7:13Q98KLH,â7QDçøéDi0217Niä1 ç1INP3Q987:1ç1WS?H:3I;Aâ7:D2çùHQæA1yå87Q@LS8é@Lå8KL13sDiâ{H:æ81yHQ13:H&3Q>GSGHQæA13Q@L3nD27%3Q1KL1éWHQ@LD2S¹H:1éæ8S8@LFG9812; â7QDçúHQæA1Ä3:@Lû1&DiâHQæ81sHQ13:H 3Q98@LHQ1;8D27Eâ7QD2çÉHQæ81&1Wï?å_17:HQ@L3Q1nDiâHQæ81%HQ1W3QH)1WS8ä2@LS811W7Q3Iî ò[S868983:HQ7Q@µNPK81ïGå=1W7Q@Lç1SGH:3½ü]ýþ=æ=N 021E3QæADiã S¬H:æ=NiHH:13QH,éINi3:13{ã @LH:æ8@LS¬N.K»NP7Qä1{H:13QHE3Q98@LH:1 DiâH:1S¾â1INPHQ987:1ÿNnæ8@LäæêKL101KGî2MONPSG>HQ1W3QH)é Ni3Q1W3ÄéWD27Q7:13Qå_D2SA6¹HQDyHQæA1&3oNPç1 3Q1FG981WS8é1nDiâç1H:æ8DG6éINPKLKB3 ;Gã @LHQæ#68@ g1W7Q1SGH%å=Ni7Niç¾1HQ17:3Iî8R7:DG6898é@LS8ä¾HQæ813:1¬H:13QH.éINi3:13 @L3½Nn7:1å=1WHQ@LHQ@L021sHoNi3 H:æ=NiH 7:1021INPKL3 HQæ81%S811W6#âD27)NPå8å87:D2å87:@»NPHQ1%HQDGDK3Q98å8å_D27:HIî 87:D2çH:æ813Q1õD2ëA3Q17:02NiHQ@LDS83I;,ã{1#æ=N 01O68101KLD2å_16 HQæA1 )J.CEò á HQ13:Hä1S817NiHQD7 ã æ8@Léæ¼@L3&Ni@Lç1W6¼NiH%HQæA1yå87QDG6898éWHQ@LD2S¹DPâN¾KµNi7:ä21n3Q1H&DPâ½3:@Lç@LKµNi7 HQ1W3QH&éINP3Q13 î )J.C{ò á 3QHNi7QH:3 â7:D2çùNnHQ13:H%å=NiH:HQ17:S¼NiS86õNn6813QéW7Q@Lå8HQ@LDSODiâ@LHQ3.@BSA3QHoNPSGHQ@µNiH:@BDS83Iî æA1&HQDGD2K*HQæ81WS 98SGâDKL683{H:æ81%å=NiH:HQ17:SO@LSGHQDyNnK»NP7Qä1)3Q1WH)DiâHQ13:H)éINP3Q13.ã æ8@Léæêé NiS#ë=1%D98HQå89AH)NiééD7Q68@LS8ä HQDnHQæ81.âD7Qç#NiHDPâ-3Q1W0217NiKGå8@LéINiK1WïÍNPçå8KL1.Diâ-éWD2çyë8@LS=NPHQD27:@µNiK8H:13QH:@BSAäyHQDGD2K[îGò[HQ3 D7Q@Lä2@LS=NPKL@BH[>n@L3 HQD%681INPKGã @BH:æ¬3:1FG981S8éW13 Diâç1H:æ8DG6&éINPKBKL3 ;I@LS83:HQ1INP6yDiâ=DS8KL>&éDçyë8@LS=NiH:@BDS)DPâgå=NP7oNPç1H:17 02NiKL9813 î æ8@L3 NiKLKLDiã 3-H:D&983:1ÄH:æ81)H:DGD2K8ã @LHQæ¾3Q>G3QH:1ç3,HQæ=NPH{7:1FG98@L7Q1)3:1021W7oNiK@LSGHQ1W7oNiéWHQ@LD2S83 ë=1ÔâD27:1¬7:1INiéæ8@LS8ä#3QDç 1 :@LS?H:17Q1W3QHQ@LS8ä #3QHoNPHQ13 îò[H&NiKL3QD#NiKLKLDiã 3 HQD#6813:@Lä2SõHQ13:H&éINi3:13&@LS HQ17:ç3EDiâHQæ81&ë_1æ=N 0G@LD27 H:æ=NiH æ_Ni3 HQDnë_1&1ïG17QéW@B3:16?3:HQ1çõ;Ni686A3¬H[ãED¼1WKB1Wç1SGHQ3yNiSA6ô7:1çDi013&DS81#Diâ HQæ81Wç¼î , =¢ 2 3 i- ) 2 < W¥? 2 Niéæ¹D2å_17NiHQ@LDS¼éINPKLKçêN >¾KB1 Ni6¹HQDON ~8x f2f ; x½twT D27 t ½´ QT ½ftµÇEË 017Q6A@BéWHIî )3%3QDGDS NP3ÃN x½twT D27 t ½´ QT ½ftµÇEË 021W7Q68@LéH&æ_Niå8å_1S83 ;ã{1yéæ8DGD3Q1yHQD¾3QH:D2å¹HQæA1 HQ13:H éINi3:1%1ïG1é98H:@BDSêNPS86ç#Ni7 n@LH{ã @LH:æyHQæ8@L3E0217:68@LéHIî H:13QH éINP3Q1%HQæ_NiHE@B3EéINP7Q7Q@L16#D98H éD2ç¾å8KL1HQ1WKB>n7Q1Wé1@L0213&N ~8x f2f 017Q6A@BéWHIî 8D7&1ï8Niç¾å8KL12;_@BS¹H:æ81éWD2SGHQ1ïGHnDiâ)H:æ81Nië_Di021y3:å=1éW@bì_éINiH:@BDS@@ î æA1Äå_NiHQH:17QS * @L3 98SGâDKL6816@LSGHQDyýnHQ1W3QH)3:1FG981S8éW13 , =-¢ 2 3 i- , =-¢ 2 3 i- ) ii # , =-¢ 2 3 i- )# %7QD98å83nçêN >¼NPKB3:DO@LSG02D2KL01y3Q1017oNPK D2å_17oNPHQ@LD2S83 î41H + iN S86 + ë_1H[ãED¼DHQæ817 1ï8Niç¾å8KL13Diâ3:éoæA1çêNP3 %

>@A@/),+.-

%

%

% % ++ 0

7 9 ;3=CB6F 'K@@@CB b F 54 76 c 8 /8V/8 8 .:: K@@A@ 2 2 HIOJKL-I

ã @LHQæ

:ý b c V ý @L3¬N#3:1H¬DPâ bý `5 c O@LS83:HoNiSGH:@»NPHQ@LD2SA3Iî æ81y1Wï?åA7Q13:3Q@LD2S * + ç1INPS83 K6@ ; HQæ=NPH)H:æ81&ä27:D298å¶@B3.7Q1å_1INPHQ16 &H:D yH:@Lç13 î æ81¬å_NiHQH:17QSA3 + NiS86 + NP7Q1n98SGâDKL6816 @LS?H:D `Mb] ) c &H:13QH.3Q1FG981WS8é13 i , =¢- 2 3 i- ii i , =¢- 2 < W¥? 2 )# i , =-¢ 2 3 i- 2 3 i- ii i , =-¢ 2 < W¥? 2 )# 2 < W¥? 2 )# iiiiiiii iIii i iiIii ! , =-¢ 2 3 i- ) 2 3 i- ii ! , =-¢ 2 3 i- ) 2 < W¥? 2 )# 2 < W¥? 2 )# %7QD98å¹681ì_S8@LHQ@LD2S83yç#N >Oë_1ê7:1983:16@BS'3Q1017oNPK)3:éoæA1çêNP3I;*KB1 Ni68@LS8äõHQDõ3QDç1yKL1021WK Diâ çDG6898KµNP7éD2SA3QHQ7:98éH:@BDS