3.3. How to do regression testing using CVS

A problem that can happen sometimes is 'it used to work before, now it doesn't anymore...'. Here is a step by step procedure to try to pinpoint when the problem occurred. This is NOT for casual users.

1. Get the "full CVS" archive from winehq. This archive is the CVS tree but with the tags controlling the versioning system. It's a big file (> 40 meg) with a name like full-cvs-<last update date> (it's more than 100mb when uncompressed, you can't very well do this with small, old computers or slow Internet connections).

2. untar it into a repository directory:

   cd /home/gerard
   tar -zxf full-cvs-2003-08-18.tar.gz
   mv wine repository
   	    

3. extract a new destination directory. This directory must not be in a subdirectory of the repository else cvs will think it's part of the repository and deny you an extraction in the repository:

   cd /home/gerard
   mv wine wine_current (-> this protects your current wine sandbox, if any)
   export CVSROOT=/home/gerard/repository
   cvs -d $CVSROOT checkout wine
   	    

   Note that it's not possible to do a checkout at a given date; you always do the checkout for the last date where the full-cvs-xxx snapshot was generated.

   Note also that it is possible to do all this with a direct CVS connection, of course. The full CVS file method is less painful for the WineHQ CVS server and probably a bit faster if you don't have a very good net connection.

4. you will have now in the ~/wine directory an image of the CVS tree, on the client side. Now update this image to the date you want:

   cd /home/gerard/wine
   cvs update -PAd -D "2004-08-23 CDT"   
   	    

   The date format is YYYY-MM-DD HH:MM:SS. Using the CDT date format ensure that you will be able to extract patches in a way that will be compatible with the wine-cvs archive http://www.winehq.org/hypermail/wine-cvs

   Many messages will inform you that more recent files have been deleted to set back the client cvs tree to the date you asked, for example:

   cvs update: tsx11/ts_xf86dga2.c is no longer in the repository
   	    

   cvs update is not limited to upgrade to a newer version as I have believed for far too long :-(

5. Now proceed as for a normal update:

   ./configure
   make depend && make
   	  

   If any non-programmer reads this, the fastest method to get at the point where the problem occurred is to use a binary search, that is, if the problem occurred in 1999, start at mid-year, then is the problem is already here, back to 1st April, if not, to 1st October, and so on.

   If you have lot of hard disk free space (a full compile currently takes 400 Mb), copy the oldest known working version before updating it, it will save time if you need to go back. (it's better to make distclean before going back in time, so you have to make everything if you don't backup the older version)

   When you have found the day where the problem happened, continue the search using the wine-cvs archive (sorted by date) and a more precise cvs update including hour, minute, second:

   cvs update -PAd -D "2004-08-23 15:17:25 CDT"
   	    

   This will allow you to find easily the exact patch that did it.

6. If you find the patch that is the cause of the problem, you have almost won; report about it to Wine Bugzilla or subscribe to wine-devel and post it there. There is a chance that the author will jump in to suggest a fix; or there is always the possibility to look hard at the patch until it is coerced to reveal where is the bug :-)