Unit Testing

Why Test The AOT?

Testing the AOT functionality is perhaps the most important area of improvement for our project. The AOT is the backbone of the project, and due to its high level of complexity, it should be tested as thoroughly as possible.

Why Is Testing The AOT Complicated?

Because of the unique nature of the SharpOS project, we have unique conditions under which our AOT must be tested. Because the AOT generates code that has to be booted on a computer or in a computer emulator, conventional unit testing through unit testing suites such as NUnit is not directly possible.

How Does It Work?

For the time being, the closest we can do, is write our individual tests, and then auto-generate a kernel which, when booted, calls each test, and then prints to the screen indication of any test failures. The kernel that is auto-generated to execute the tests is called the “Test Kernel”, and along with the tests that it is auto-generated to execute, is located in /trunk/SharpOS.Kernel.Tests/ of the project’s Subversion repository.

How Are The Tests Written?

Tests can be written in C# or in IL. A test takes the form of a publicly accessible static method, whose name is prefixed with “CMP”, that returns an Int32. (And obviously, the method needs to be on a publicly accessible type.) A test should return 1 for success, and 0 for failure.

For example, the following C# test checks to make sure addition (of two 32-bit integer constants) is performed properly:

namespace SharpOS.Kernel.Tests.CS {
    public class Addition {        
        public static uint CMP_Int32_Constants ()
        {
            if ((1+2) == 3)
                return 1;

            return 0;
        }
    }
}

Or, if you are brave, in IL:

.assembly extern mscorlib
{
}

.namespace SharpOS.Kernel.Tests.IL
{
    .class public auto ansi abstract sealed beforefieldinit Addition extends [mscorlib]System.Object
    {
        .method public static hidebysig default int32 CMP_Int32_Constants () cil managed 
        {
            .maxstack 3

            ldc.i4 1
            ldc.i4 2
            add

            ldc.i4 3
            ceq
            brtrue ok

            ldc.i4 0
            ret

            ok:
            ldc.i4 1
            ret
        }
    }
}

Where Do Tests Go?

Tests written in C# should be placed in /trunk/SharpOS.Kernel.Tests/CS/, and tests written in IL should be placed in /trunk/SharpOS.Kernel.Tests/IL/. The NAnt build script automatically generates a kernel that will call all tests from both of the input assemblies, (one for the C# tests, and one for the IL tests).

The AOT Compiler doesn’t compile my test case at all!

Comment out the code in your test method and make it return 0,
that way we can see that the test fails, uncomment it, and debug it.

I Thought You Said No NUnit!?

There is an NUnit folder because the Test Kernel’s NAnt build script creates an NUnit assembly to wrap all of our tests. This assembly doesn’t serve for actually testing the AOT, but actually for verifying results from running the normal tests. Because NUnit, running on top of Mono or .NET on your normal OS, is pretty standard – any valid test for AOT sanity should always pass in NUnit, even if it unfortunately doesn’t in the test kernel. So if you see a test fail, and the test confuses you, and then you try it out in NUnit and it still fails – then it is probably a bad test!