SDSU Client-Server Programming
Spring Semester, 2005
Testing
Previous     Lecture Notes Index     Next     
© 2005, All Rights Reserved, SDSU & Roger Whitney
San Diego State University -- This page last updated 25-Jan-05

CS 580 Client-Server Programming Spring Semester, 2005 Doc 2 Testing

Contents

Testing

Johnson's Law

Why Unit Testing

When to Write Unit Tests

What to Test

SUnit & JUnit

How to Use SUnit

TestCase methods of interest

Using JUnit

Running JUnit Using Eclipse

assert Methods

Testing the Tests

Test Fixtures

Suites – Multiple Test Classes

How to Test Exceptions

Testing and Hidden Methods/State

How to Test Hidden Methods Directly?

Method 1: Relax the protection level

Method 2: Use inner classes

Method 3: Use reflection

Running JUnit without Eclipse

Copyright ©, All rights reserved. 2005 SDSU & Roger Whitney, 5500 Campanile Drive, San Diego, CA 92182-7700 USA. OpenContent ( http://www.opencontent.org/opl.shtml) license defines the copyright on this document

.

References

JUnit Cookbook http://junit.sourceforge.net/doc/cookbook/cookbook.htm

JUnit Test Infected: Programmers Love Writing Tests http://junit.sourceforge.net/doc/testinfected/testing.htm

JUnit Javadoc: http://www.eli.sdsu.edu/java-SDSU/docs/

Brian Marick’s Testing Web Site: http://www.testing.com/

Testing for Programmers, Brian Marick, Available at: http://www.testing.com/writings.html

Reading Assignment

Test Infected - Programmers Love Writing Tests, http://junit.sourceforge.net/doc/testinfected/testing.htm

JUnit FAQ, http://junit.sourceforge.net/doc/faq/faq.htm

Programming with Assertions, http://java.sun.com/j2se/1.4/docs/guide/lang/assert.html

Testing

Johnson's Law

If it is not tested it does not work

Types of tests

Tests individual code segments

Test functionality of an application

Why Unit Testing

If it is not tested it does not work

The more time between coding and testing

Without unit tests

Why Automated Tests?

What wrong with:

Repeatability & Scalability

Need testing methods that:

Practices that work in a school project may not be usable in industry

Standard industry practices may seem overkill in a school project

Work on building good habits and skills

We have a QA Team, so why should I write tests?

How long does it take QA to test your code?

How much time does your team spend working around bugs before QA tests?

How easy is it to find & correct the errors after QA finds them?

Most programmers have an informal testing process

With a little more work you can develop a useful test suite

When to Write Unit Tests

First write the tests

Then write the code to be tested

Writing tests first saves time

What to Test

Everything that could possibly break

Test values

Inside valid range Outside valid range On the boundary between valid/invalid

GUIs are very hard to test

Keep GUI layer very thin Unit test program behind the GUI, not the GUI

Common Things that Programs Handle Incorrectly

Adapted with permission from “A Short Catalog of Test Ideas” by Brian Marick, http://www.testing.com/writings.html

Strings

Test using empty String

Collections

Test using:

Numbers

Test using:

SUnit & JUnit

Free frameworks for Unit testing

SUnit originally written by Kent Beck 1994

JUnit written by Kent Beck & Erich Gamma

Ports are available in

.NET

Ada

AppleScript

C

C#

C++

Curl

Delphi

Eiffel

Eiffel

Flash

Forte 4GL

Gemstone/S

Haskell

HTML

Jade

LISP

Objective-C

Oracle

Palm

Perl

Php

PowerBuilder

Python

Ruby

Scheme

Smalltalk

Visual Basic

XML

XSLT



See http://www.xprogramming.com/software.htm to download ports

How to Use SUnit

1. Make test class a subclass of TestCase

2. Make test methods

The framework treats methods starting with 'test' as test methods

3. Run the tests

      TestRunner open

Broswer SUnit Extensions

Sample TestClass

Smalltalk.CS535 defineClass: #TestCounter
   superclass: #{XProgramming.SUnit.TestCase}
   indexedType: #none
   private: false
   instanceVariableNames: 'counter '
   classInstanceVariableNames: ''
   imports: ''
   category: 'Course-Examples'

CS535.TestCounter methodsFor: 'testing'

setUp
   counter := Counter new.
   
tearDown
   counter := nil.
   
testDecrease
   counter decrease.
   self assert: counter count = -1.
   
testDecreaseWithShould
   "Just an example to show should: syntax"
   counter decrease.
   self should: [counter count = -1].
   
testIncrease
   self deny: counter isNil.
   counter increase.
   self assert: counter count = 1.
   
testZeroDivide
   "Just an example to show should:raise: syntax"
   self 
      should: [1/0]
      raise: ZeroDivide.
   
   self 
      shouldnt: [1/2]
      raise: ZeroDivide

TestCase methods of interest

Methods to assert conditions:

   assert: aBooleanExpression
   assert: aBooleanExpression description: aString
   deny: aBooleanExpression
   deny: aBooleanExpression description: aString
   
   should: [aBooleanExpression]
   should: [aBooleanExpression] raise: AnExceptionClass
   
   shouldnt: [aBooleanExpression]
   shouldnt: [aBooleanExpression] raise: AnExceptionClass
   
   signalFailure: aString

setUp

Called before running each test method in the class.

Used to:

Open files Open database connections Create objects needed for test methods

tearDown

Called after running each test method in the class.

Used to:

Close files Close database connections Nil out references to objects



Using JUnit

Example

Goal: Implement a Stack containing integers.

Tests:

First tests for the constructors:

public class  TestStack extends TestCase {
   
   
   public void testDefaultConstructor() {
      Stack test = new Stack();
      assertTrue("Default constructor", test.isEmpty() );
   }
      
   public void testSizeConstructor() {
      Stack test = new Stack(5);
      assertTrue( test.isEmpty() );
   }
}

First part of the Stack

package example;
   
public class Stack  {
   int[] elements;
   int topElement = -1;
   
   public Stack() {
    this(10); 
   }
   
   public Stack(int size) { 
      elements = new int[size]; 
   }
   
   public boolean isEmpty() {
      return topElement == -1;
   }
}

Running JUnit Using Eclipse

Select JUnit Test Case

There seems to be at least three different menus one can use to create a test case in eclipse. In the Java perspective after creating the class you create test case class by selecting the “JUnit Test Case” item in the “New” submenu of the “File” menu. Or you can select the “JUnit Test Case” item in the “New Class” popup menu obtained by clicking on the new class icon (see below)

When you ask eclipse to create a test case you get dialog window like:

Run the Test

The default settings in the above dialog are good enough for this example. Click on the finish button at the bottom of the window. Add the two methods to the test class. Then run the class as a JUnit test. You do this by in the “Run” menu select the “Run as...” item and then selecting the “JUnit Test” item in the submenu. After the test case runs you will see a new panel that displays the results of the run.

assert Methods

assertTrue()
assertFalse()
assertEquals()
assertNotEquals()
assertSame()
assertNotSame()
assertNull()
assertNotNull()
fail()

For a complete list of the assert methods & arguments see

http://www.junit.org/junit/javadoc/3.8/index.htm/allclasses-frame.html/junit/junit/framework/Assert.html/Assert.html

JUnit, Java 1.4 & assert

JUnit had a method called assert()

Java 1.4 makes assert a reserved word

JUnit starting 3.7 replaces assert() with assertTrue()

Use JUnit 3.7 or later with JDK 1.4

To use JDK 1.4 asserts:

   java -source 1.4 programFile.java

   java -ea programFile

Testing the Tests

If can be useful to modify the code to break the tests

package example;
   
public class Stack  {
   int[] elements;
   int topElement = -1;
   
   etc.
   
   public boolean isEmpty() {
      return topElement == 1;
   }
}

Why Test the Tests?

One company had an automatic build and test cycle that ran at night. The daily build was created and all the tests were run at night. The test results were available first thing in the morning. One night the build process crashed, so the daily build was not made. Hence there was no code to test. Still 70% of the tests passed. If they had tested their tests, they would have discovered immediately that their tests were broken.

Test Fixtures

Before each test setUp() is run

After each test tearDown() is run

package example;
   
import junit.framework.TestCase;
   
public class  StackTest extends TestCase {
   Stack test;
      
   public void setUp() {
      test = new Stack(5);
      for (int k = 1; k <=5;k++)
         test.push( k);   
   }
   
   public void testPushPop() {
   
      for (int k = 5; k >= 1; k--)
         assertEquals( "Pop fail on element " + k,  test.pop() , k);
   }
}

Suites – Multiple Test Classes

Multiple test classes can be run at the same time

Add Queue & TestQueue to Stack classes

package example;
   
import junit.framework.TestCase;
   
public class TestQueue extends TestCase{
   
   public void testConstructor() {
      Queue test = new Queue();
      assertTrue( test.isEmpty());
   }
}

package example;
   
import java.util.Vector;
   
public class  Queue{
   Vector elements = new Vector();
   public boolean isEmpty() {
      return elements.isEmpty();
   }
}

Using a Suite to Run Multiple Test Classes

Running AllTests runs the tests in

package example;
import junit.framework.Test;
import junit.framework.TestSuite;
   
public class AllTests  {
   static public void main(String[] args) {
       TestRunner.run( example.AllTests.suite());;
   }
   
   public static Test suite(){
      TestSuite suite = new TestSuite("Test for example");
      suite.addTestSuite(StackTest.class);
      suite.addTestSuite(QueueTest.class);
      return suite;
   }
}

Eclipse will generate the above class for you. In the file menu select “New”, then in the submenu select “other”. In the dialog window select “JUnit Test Suite” which is under Java and then under JUnit.



How to Test Exceptions

At times you may wish to test input to methods that will cause an exception to be thrown

Here is an example of a test that

 

Example is from the JUnit FAQ

public void testIndexOutOfBoundsException() {
      
    ArrayList list = new ArrayList(10);
      
    try {
   
        Object o = list.get(11);
   
        fail("Should raise an IndexOutOfBoundsException");
   
    } catch (IndexOutOfBoundsException success) {}
}

Testing and Hidden Methods/State

Issues:

Testing Hidden Methods One Position Don't Do it

Pro:

The basic idea is to work smarter not harder. One cannot completely test each class, and one does not have infinite time two write tests one should write the most effective tests possible. Tests of the public interface of class will also test hidden methods of the class. Bugs in hidden methods that never affect the public methods are not a problem. Since the most important thing is that the pubic interface works correctly, concentrate your tests on the public interface.

Con:

My experience is that the more code I write without tests, the more time I spend on finding and correcting bugs. How many times have you spent hours (days?) tracking down a bug that turned out to a simple bug in some simple untested method, which would have been easy to test? The argument that one cannot test everything and must make effective use of ones testing time is correct. Given the differences in programmer skill level, programmer experience, etc. everyone has to work out their own solution to this. The XP solution is to try to test everything that could possible break. Since most students are not used to testing, you have to fight the habit of not testing and testing after you have completely finished coding. Given the current state of affairs in commercial software, the industry has a lot to learn about testing.

How to Test Hidden Methods Directly?

Method 1: Relax the protection level

In Java one can

Pro:

Con:

You should comment the method to inform the clients that the method is not to be used

How to Test Hidden Methods Directly?

Method 2: Use inner classes

import junit.framework.TestCase;
   
public class Foo {
    private int value;
   
    private void bar() {
        value = 10;
    }
   
    public static class FooTest extends TestCase {
        public FooTest(String name) {
            super(name );
        }
   
        public void testBar() {
            Foo a = new Foo();
            a.bar();
            assert( 10 == a.value );
        }
    }
}

Pro:

Con:

How to Test Hidden Methods Directly?

Method 3: Use reflection

Pro:

Con:

See http://www.eli.sdsu.edu/courses/fall98/cs596/notes/reflection/reflection.html for more information about reflection

Running JUnit without Eclipse

JUnit has three interfaces

Fastest to run

Can reload class files so you can Run TestRunner once Recompile program until it passes tests

Starting Swingui TestRunner

Make sure your classpath includes the code to tested

On Rohan use:

java  junit.swingui.TestRunner
   

You get a window similar to that on the next page


Enter the full name of the test class

Click on the Run button

If there are errors/failures select one

You will see a stack trace of the error


The “...” button will search for all test classes in your classpath

Swing version of JUnit TestRunner


Running the textui TestRunner

Sample Program using main

public class Testing {
   public static void main (String args[]) {
      junit.textui.TestRunner.run( example.TestStack.class);
   }
}

Output

..
Time: 0.067
   
OK (2 tests)
   
java has exited with status 0.

Previous     visitors since 25-Jan-05     Next