Can anyone explain the absence of functions when testing

See the below ...

I've even gone so far as to attempt to 'addPath' to the directory which includes the convert_date.m file. I'm stumped on why I can call convert date from the command line, but not test it in the unit testing framework.

Does anyone have any ideas ?

 >> convert_date('today')
ans =
2015-09-08
>> runMyTests('',false)
Running convert_date_tests
================================================================================
Error occurred in convert_date_tests/testToday and it did not run to completion.

    --------------
    Error Details:
    --------------
    Undefined function 'convert_date' for input arguments of type 'char'.
    Error in convert_date_tests/testToday (line 8)
                aDate = convert_date('today');
================================================================================

For bonus points on a second note, does anyone have an example of assertError?

I need that too. In a different test, the test always fails because there is an error... there should be. I should have expected assertError to not fail this test... but it always does.

[Andy Campbell]

Hi Simon,

Can you show a full minimum test that is failing in this way? When I say "full minimum" I mean try to get the test down to as small as possible demonstrating the issue but its a full complete self contained test. For example, does this fail?

classdef SimpleTest < matlab.unittest.TestCase
    methods(Test)
        function callIt(testCase)
            convert_date('today');
        end
    end
end

If it doesn't then perhaps there is something in your test causing this problem. Also, while we are at it, please show how you are running the test.

[Simon Parten]

Hi Andy,

Thanks for the response, yes ... this test was failing. The other answers have pointed me toward the reason why.

For the sake of project hygiene I am separating out the tests from the 'main' code by having them in a sub directory. Adding this;

% 
testDir = [pwd '\tests'];
sourceDir = [pwd '\m-files'];
addpath(testDir);
addpath(sourceDir);

To my 'runMyTests' file solved the problem. The reasoning and hints to get there are in your longer answer below.

Answer 1

[Andy Campbell]

When you create a test suite "fromFile" or "fromFolder", the test runner changes to that folder and adds it to the path when run. This is because the test class needs to remain on the path at all times for deterministic behavior.

If your test classes (or their containiing packages) are always on the path you can create the suite "fromPackage" or "fromClass" and the runner does not change directory.

Ultimately, both the code under test as well as the tests themselves need to be added to to path in order to run. Running the tests does not remove any folders from the path that are explicitly listed. However, if the code under test is in the current folder (and therefore implicitly on the path) then you may lose accessibility to it when the framework changes to the test folder.

How did you addpath? Te following will not work:

   >> addpath .

because it si jsut adding "." to the path which changes everytime you change folders. However the following will work:

>> addpath(pwd)

because it resolves the current folder and actually adds it to the path explicitly.

Here are some examples of verifyError which has the same API as assertError. Also, more than likely you probably want to use verifyError anyway if this is the main verification point of your test method. Verifications are recommended to use in this scenario because they trigger failures but also let you know that the full test content ran to completion (and thus there are no more failures hiding behind the known failure). Assertions are best used when you are validating a precondition, and you know that a failure would only produce subsequent failures as well.

[Simon Parten]

this is exactly it.

Thanks. To anyone reading this, if you read through the other comments the explicit solutions (code) to both questions are there.

Answer 2

[Simon Parten]

The answer is apparently that the test runner somehow has it's own 'path'.

Can add the right directories using addPath ... but in the test routine somewhere.

Answer 3

[Adam]

Not sure I understand the main part of your question, but you seem to have answered that.

As for the other part, I have done a handful of error assertions although not as many as I ought to. Here is an (slightly altered to remove confidential stuff) example of a simple one I did. If I remember correctly (and judging by the error id) this one checks for something thrown by the validateattributes function that I use all over the place. use the same method checking for my own custom errors though:

function testGetDataWithIndexVectorLongerThan2Throws( obj )
  dataProvider = VolumeDataProvider3D( filename );

  f = @() dataProvider.getDataByIndexRange( 3:22, [4 5], : );

  obj.assertError( f, 'MATLAB:incorrectNumel' );
end

This test passes as it should by throwing the expected error caused by me passing a vector of 20 values as an argument where a vector of length 2 is expected.

[Simon Parten]

thanks. That's very helpful.

I've added another code example in the other comments.

Answer 4

[Simon Parten]

classdef convert_date_tests < matlab.unittest.TestCase
%Mostly for proof of concept testing and example useage of the %convert_date function methods (Test)
        function testToday(testCase)
            aDate = convert_date('today');
            testCase.assertEqual(aDate, datestr(now, 29));
        end  

        function shouldFail(testCase)                  
           testCase.assertError(@() convert_date('today1') , ?MException )
        end        
  end    
end

[Adam]

I would recommend testing for a specific error personally. The fact that you are writing a test asserting the error means you clearly know there should be one and therefore what its ID should be.

Testing just a generic error is dangerous because it will also catch syntax errors and simply pass the test because there was garbage code that didn't run at all!

[Steven Lord]

Can you display the output of these two commands both in the Command Window at the prompt and inside your test, immediately before you invoke convert_date?

 which -all convert_date
 pwd

The reason I'm asking you to display PWD is because I suspect runMyTests is trying to run a test in a different directory. I also suspect that you're only able to call convert_date because it's in your current directory (NOT on your path) and some setup code in runMyTests changes to a different directory so convert_date is no longer in scope.

[Simon Parten]

That's exactly the problem. I've added a comment to Andy's answer above which sets out exactly what the solution is.

Thankyou very much for the answer.

[Simon Parten]

@Adam - you're right. Given your hints and Andy's help below, my test now looks like this;

        function shouldGenerateException(testCase)       
           testCase.assertError(@() convert_date('nonsense_date') , 'MATLAB:datenum:ConvertDateString' )
        end