Transcript

MODEL BASED TEST GENERATION

WHAT¶S A MODEL?
A model: ´ Is an abstraction or simplified representation of the system from a particular perspective ´ Supports investigation, discovery, explanation, prediction, or construction ´ May be expressed as a description, table, graphical diagram, or quantitative mathematical model ´ Is not necessarily comprehensive

2

MODELS IN EVERYDAY LIFE

3

EXAMPLES OF MODELS
´ ´ ´ ´ ´ ´ ´ ´ ´

Flow Charts Data Flow Diagrams Entity- Relationship Diagrams State Diagrams Deployment Diagrams Class Diagrams Use Cases Activity Diagrams State Transition Tables
4

MODEL BASED TEST GENERATION
Formal Model set of

Generator
Test Generation Directives

Test Cases
‡ sequence of stimuli ‡ expected responses

5

MODELING
Product Idea / Specification What product are you Modeling building? Validation Are you building the right product?

Specification AsmL Refinement

Verification Are you building the product right ?
6

Implementation C/C++/C#

FSM: START STATE & ACTION

Start state (Entry Point)

Files Shown: 0 File Select: Not Selected Exit Point: False

7

start

OTHER STATES

8

SIMPLE FSM MODEL (STATE MODEL)

9

WALK EVERY STATE-CHANGE

Create Invert Selection Invert Selection Delete
10

TEST EVERY WALK < 5 STEPS

A: Invert Selection B: Create; Delete C: Create; Invert Selection; Invert Selection; Delete

11

MODEL BASED TEST GENERATION TOOLS
T-VEC(Test Vector Generation System) ´ TestMaster ´ UniTesk (Unified Testing and spec toolkit) ´ GOCHA Techbeans ´ ASML Etc«..
´

12

THE ASML MODELING TOOLKIT
Abstract state machine Language is a powerful modeling language AsmL provides
« « « « «

Authoring via MS Word and XML Simulation and interoperability via COM and .NET Automatic test oracle Semi-automatic test case generation Integration with Microsoft·s test platform

13

ASML: GETTING STARTED, µSORT¶
EXAMPLE
var A as Seq of Integer = []
Non-determinism Swap() choose i in indices(A), j in indices(A) where i < j and A(i) > A(j) A(i) := A(j) Parallelism A(j) := A(i)

A = [2,3,1]

A = [1,3,2]

A = [2,1,3]

A = [1,2,3]

sort() step until fixpoint swap()
14

ASML: PREDEFINED DATA TYPES 

Sets menu = {"ham", "cheese", "blt"} Sequences
random = [3, 6, 7, 1, 5] //as Seq of Integer //as

Maps
directory as Map of String to Integer = {³police´ -> 112 , ³info´ -> 118} 118}

Tuples
nameAndAge = (³John´,39) //(String,Integer) ³John´,39) Integer)
15

USING MODELS FOR TEST
Model Generates User Info Provides expected results for Test Cases Are run by Pass Test Oracle No pass Provides actual results for Implementation

16

ASML-BASED TEST HARNESS
Any client

Side-by-side runs Discrepancies automatically flagged

Test harness

AsmL model

Implementation under test
17

ASML TEST CASE GENERATION - 1
Solves two problems How to find ´interestingµ parameter values and combinations How to find ´interestingµ call sequences Approach Use information derived from the model to optimize the number of test cases
18

ASML TEST CASE GENERATION 2
Divide state space according to state predicates System ´coversµ the state predicates 


Driven by input generator (parameter selection) or Driven by enabledness (call sequences)

Tester provides inputs, gives interesting state predicates, directs the search
19

THE DUAL ROLE OF ASM MODELS
Testing harness
AsmL Model M Call next action IUT Get state
view

Guided generation
(Param Select with Feedback)

Reduce FSM FSM tool

Pass

S
Is S valid according to M?

AsmL Model M
view

Test cases

Fail: witness

view
20

ASML TEST CASE GENERATION - 3
Generate only as many test cases as needed to have full coverage of the specification
Generator Input Feedback Model

Coverage criteria  Branch: Every ³branch´ of the spec must be taken  Modified condition: independent atomic conditions  Off-by-One: off-by-one errors Off-byoff-by Hyperstate coverage: Every abstract state Hyperstate must be visited
21

ASML TEST (ASMLT)
asmlT can be used to: 1. Find interesting sequences of method calls
¹

finds the shortest route in a generated FSM that uses every edge to traverse the states

2.

Find interesting parameters for each method call Access Driven Filtering (ADF)
¹

you write a predicate (boolean function) to decide if a given parameter is interesting

3.

Perform a conformance test Compare Model with Implementation

22

REFERENCES
´

Model-Based Test Generation Tools, Alan Hartman, AGEDIS CONSORTIUM http://www.agedis.de/documents/ModelBasedTestGenerationTools_cs.pdf Finite State Model-Based Testing on a Shoestring, Harry Robinson http://www.model-based-testing.org/shoestring.htm Model Based Testing, L. Apfelbaum, et al, Software Quality Week Conference in May, 1997. http://www.model-based-testing.org/sqw97.pdf Abstract State Machine Language, Microsoft Research http://research.microsoft.com/fse/asml/
The AsmL Test Generator tool, Microsoft Research http://research.microsoft.com/fse/AsmL/doc/AsmLTester.html

´

´

´

´

´

Intelligent Test Automation, Harry Robinson

http://www.model-based-testing.org/robinson.pdf

Software Testing & Quality Engineering (STQE) magazine Sept/Oct 2000

23

Queries..??

24