[FFmpeg-devel] [PATCH] Modular test sytem

[Stefano Sabatini]

On date Saturday 2009-12-26 12:46:21 +0100, Diego Biurrun encoded:
> On Sat, Dec 26, 2009 at 02:01:07AM +0100, Stefano Sabatini wrote:
> > Sample patch attached, which I tested it with lavf (I created a
> > tests/lavf.regression.refd/ dir, and I put a file for each test with
> > the corresponding lines taken from lavf.regression.ref).
> > 
> > With the patch applied it is possible to edit the regression.list file
> > (eliminting or commenting in the corresponding line) for disabling
> > tests.
> 
> I'm not sure I understand the point of this.

The idea is to be able to disable some test without to edit the
regression.ref + Makefile.

With this you only need to eliminate / comment the tests listed in the
regression.list file and the regression test will run only the
enabled tests.

> > --- ffmpeg.orig/Makefile	2009-12-26 01:15:08.000000000 +0100
> > +++ ffmpeg/Makefile	2009-12-26 02:00:43.000000000 +0100
> > @@ -156,6 +157,13 @@
> >  
> > +$(CODEC_VSYNTH1_REFFILE) = tests/codec.regression.list
> > +	cat $(patsubst regtest-%,$(SRC_PATH)/tests/codec-vsynth1.regression.refd/%, $(LAVF_TESTS)) > $@
> > +$(CODEC_VSYNTH2_REFFILE) = tests/codec.regression.list
> > +	cat $(patsubst regtest-%,$(SRC_PATH)/tests/codec-vsynth2.regression.refd/%, $(LAVF_TESTS)) > $@

ouch of course that was:
$(CODEC_VSYNTH1_REFFILE) = tests/codec.regression.list
	cat $(patsubst regtest-%,$(SRC_PATH)/tests/codec-vsynth1.regression.refd/%, $(CODEC_TESTS)) > $@
$(CODEC_VSYNTH2_REFFILE) = tests/codec.regression.list
	cat $(patsubst regtest-%,$(SRC_PATH)/tests/codec-vsynth2.regression.refd/%, $(CODEC_TESTS)) > $@

> > +$(LAVF_REFFILE): tests/lavf.regression.list
> > +	cat $(patsubst regtest-%,$(SRC_PATH)/tests/lavf.regression.refd/%,          $(LAVF_TESTS)) > $@
> 
> Is the ".refd" instead of ".ref" intentional?  If yes, I don't see where
> that directory is created and am confused.

Since this patch involves a lot of tedious editing work I didn't
posted a complete patch, before to do that I want to be sure that the
idea is going to be accepted.

The lavf.regression.refd directory needs to contain one file with the
corresponding test name containing the regression test reference
output.

I named it lavf.regression.refd because I don't know how SVN deals
with this:
svn remove lavf.regression.ref
svn mkdir  lavf.regression.ref
svn add    lavf.regression.ref
#create tests file ...
svn add    lavf.regression.ref/{TESTS}
svn commit lavf.regression.ref

This has some warts, re-creating the ref files will lose the history
of the lavf.regression.ref file, on the other hand it will make lot
easier in the future to check the history of each ref file simply
doing svn log TEST_REF on the specific file.

I'll rename lavf.regression.refd -> lavf.regression.ref in a
successive step.

Regards.
-- 
FFmpeg = Fantastic and Faithful Mythic Pitiless Exciting Guru