[sheepdog] [PATCH 0/3] Introducing unittest framework to sheepdog

[Kai Zhang]

On Jun 3, 2013, at 7:49 PM, Liu Yuan <namei.unix at gmail.com> wrote:

> On 06/03/2013 07:32 PM, Kai Zhang wrote:
>> As we all known, unittest is quite important to any project which is aimed to be used in production environment.
>> 
>> This series of patches is to introduce 'check', a popular unittest framework for C language.
>> 
>> These patches are quite initial implemented.
>> But it still shows the abilities of:
>> - adding more unittests easily
>> - runing unittests selectively
>> - showing code coverage after running unittest
>> - generating html report by lcov and genhtml
>> - integrating with other CI (continuous integration) systems like Jenkins
>> 
>> In addition, we can also benifit from unittest in the following ways:
>> - help us write more testable code
>> - improve the robustness of sheepdog
>> - new contributors can be quickly involved by writing unittest code
>> 
> 
> Thanks for this framework. Besides above information, it would be nice
> if you shed more lights on the different between this framework and our
> current 'tests/' framework.
> 
> are these two framework completely complementary? Or they are somewhat
> overlapped in some areas? If I want to add a test, what I should
> consider to choose one over another?
> 

Generally saying, current 'tests/' is functional testing, it is quite different from unit test.
The fowling is copied from wikipedia. For more information, please check https://en.wikipedia.org/wiki/Software_testing

Functional testing refers to activities that verify a specific action or function of the code. These are usually found in the code requirements documentation, although some development methodologies work from use cases or user stories. Functional tests tend to answer the question of "can the user do this" or "does this particular feature work."
 

Unit testing, also known as component testing, refers to tests that verify the functionality of a specific section of code, usually at the function level. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors.[31]
These types of tests are usually written by developers as they work on code (white-box style), to ensure that the specific function is working as expected. One function might have multiple tests, to catch corner cases or other branches in the code. Unit testing alone cannot verify the functionality of a piece of software, but rather is used to assure that the building blocks the software uses work independently of each other.
Unit testing is a software development process that involves synchronized application of a broad spectrum of defect prevention and detection strategies in order to reduce software development risks, time, and costs. It is performed by the software developer or engineer during the construction phase of the software development lifecycle. Rather than replace traditional QA focuses, it augments it. Unit testing aims to eliminate construction errors before code is promoted to QA; this strategy is intended to increase the quality of the resulting software as well as the efficiency of the overall development and QA process.
Depending on the organization's expectations for software development, unit testing might include static code analysis, data flow analysis metrics analysis, peer code reviews, code coverage analysis and other software verification practices.

In my opine, unit test should be added very time you add a new function for testing each cases (branches) it could occur. And each unit test should just focus on one function. 
But functional test should be added only when you add a new feature that could be aware by user. (for example, a collie command)

In another way, unit test is white-box testing and functional testing is black-box testing.


>> Usage:
>> - $make check # run all unittests
>> - $make check CK_RUN_SUITE=xxx_suite CK_RUN_CASE=xxx_case # run selective unittest
>> - $make coverage # generate coverage report in html to directory 'coverage'
>> 
>> There are also many things that we can improve:
>> - more tests of cause
>> - more functional mock programs (inspired by gmock)
>> 
>> Another thing that I wanna share with you is that, current unittests in implemented under 'check', a C unittest framework.
>> There is another test framework, google testing framework (including gtest and gmock), which is more popular and powerful.
>> However, they are written in C++.
>> I have tried to use gtest and gmock to write unittests in C++, but failed to compile with our code.
>> We need to refine our code to be compatible with C++ compiler if we wanna adopt gtest and gmock.
> 
> I don't think we need follow the fashion, if 'check' serves sheepdog
> well, just use it. Adjusting code just for a C++ test system looks weird
> to me. Unless there is big win for gtest over check, I'd rather use check.
> 

I agree. I prefer to migrate some useful gmock code to c.

Thanks,
Kyle

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog/attachments/20130603/567682e9/attachment-0004.html>