[wpkg-users] Extended hosts matches

[Falko Trojahn]

Hi Daniel,

thank you for doing the tests.
>> please test wpkg.js with your patch applied using "wpkg-test" from
>>
>> http://wpkg.svn.sourceforge.net/viewvc/wpkg/wpkg/wpkg-test/
>> rf.
>> |svn co http://wpkg.svn.sourceforge.net/svnroot/wpkg wpkg/wpkg-test|
>>
>> If the tests run well, it would be nice if you write an additional test
>> set with all possible ways of using your new features to make sure your
>> patch does the new things right.
>>
>>
>> The explanations in README.txt are way too short - had no time yet to
>> explain wpkg-test in the wiki. So, please, if you have questions e.g.
>> about how to create the test sets feel free to drop me a mail.
>>     
> With a mainline 1.1.3-RC2 the following tests failed:
>   
That seems really strange to me. I did test all using "testall.cmd
113RC3" before I had wrote my mail to you, using Windows XP 32-bit,
running as administrator.

Just again I did "testall 113RC2" with all tests running fine. (For
this, I have "wpkg-113RC3.js" and "wpkg-113RC2.js" etc. within wpkg-test
dir)
> - test 3: 23, 24, 25, 26, 31, 32, 33, 34, 35, 36
>   
22 - has check command with version information, so check must fail
23 - same
24 - test for expression matching to uninstall key scanner
25 - test for _not_ matching uninstall key regex
26 - test for expression matching in uninstall key

30 - completely remove all packages
31 - msitest1: test uninstall and version check, variables in install cmd
        wpkgtest-1.0.0.msi: {63E8D196-C3B6-48F3-BA4D-D1EE4775AE37}
32 - msitest2: test uninstall version check (versionequalto must be false),
33 - msitest2: test uninstall version check (versiongreaterorequal must
be false,
     msiexec returns 1619 for wrong file)
34 - msitest2: test uninstall version check (versiongreaterorequal must
be true: 1.1.0=1.1.0,
        variables in  upgrade wpkgtest-1.1.0.msi
{D30B7CC9-C6F3-4E35-9B3F-58CE845E8210}
35 - msitest2: test uninstall version check (versiongreaterorequal must
be true: 1.1.0>1.0.0,
36 - msitest3: downgrade to wpkgtest-1.0.0.msi


It's possible that some tests don't work on your system, as there is a
hard coded path to c:\wpkgtest-inno\. May be we should change this.

> - test 1: 10, 12
>   
10 downgrade package test from 100 to 99, installation of test0,test1,test2
   according to dependancies
12 test2 upgraded

> Same results with my patched[1] wpkg.js, looks like I don't break
> anything ;-).
>   
Nope, first you should have a look why the tests with original wpkg.js
fail with your system.
I suspect Wpkg didn't have enough rights and couldn't install the test
programs?

Please have a look into the results/ and logs/ directories to see full
output and logs of wpkg.js to find out what's wrong. If unsure, please
mail these directories zipped off-list to me.

> I'll look more closely at wpkg-test, looking at reliable full tests
> like:
>
> - Does it compile ? (cscript wpkg.js --help with no error)
>
> - Test each command line (maybe looking at debug output to see if options are
>   changed with command line switches)
>
> - Extending tests set with my uses cases
>
> wpkg-test seems to focus on working tests, I think we should add tests
> that are OK if they fail (error handling).
>   
As you may see further up, e.g. tests 32 and 33 use such failing tests.
> I confess that doing this in DOS batch is quite repulsive to me[2].
>   
I think I should explain some details. You have nothing to do with the
batch files at all - just use 'em - , since they just call wpkg.js using
different configurations/xml/packages.

Each test suite (test1_dependencies,test2_checkor,test3_cmdsetup)
consists of 1..99 test cases, whereas test case 99 has to clean the
system fully to be clean for the next test suite. In the file
"Checks.txt" you have a short explanation to see which test case does what.

For each test, wpkg does use the corresponding directory. Each has a
full blown set of files for a wpkg base. For example,
"test2_checkor/10/" contains config.xml, hosts.xml, profiles.xml,
packages.xml. So, from one test (directory) to the next, you change
these packages/profiles a bit, so you can do all what you could do with
Wpkg.

The comparison, if a test has failed or not, is done by comparing the
actual "wpkg.xml" file in the wpkg-test directory against  compare/[test
suite]/[test case]/wpkg.xml.

You can call the command files testone/testset/testall for a single
test/test suite - see README.txt.

> Regards.
>
> Footnotes: 
> [1]  I tested the extended-hosts and factorise series.
>   
I think you have to write corresponding tests first.

When a new test is run the first time ever, a corresponding
  compare/[test suite]/[test case]/wpkg.xml
is created automagically, so sometimes you have to remove that file if
your test case wasn't correct, yet.

> [2]  I like perl Test::Harness[3] ;-)
>
> [3]  http://search.cpan.org/~andya/Test-Harness-3.21/lib/Test/Harness.pm
>   
When I wrote the (quick 'n dirty) wpkg-test suite I had in mind, that
everyone can check Wpkg on every Win** system without installing extra
software. And it shouldn't interfere with real wpkg installations on the
test system by using it's own wpkg.xml and  not %windir%\system32\wpkg.xml.

When developing wpkg.js, of course you would use other development
tools, too.

Best regards,
Falko