mirrors/haproxy
1
0
Fork 0
mirror of https://git.haproxy.org/git/haproxy.git/ synced 2025-08-06 23:27:04 +02:00
Code Issues Projects Releases Wiki Activity

REGTEST: Adapt reg test doc files to vtest.

Browse Source 
This is a first patch to switch from varnishtest to new standalone
varnish cache reg testing program: vtest.

More information may be found here:

https://github.com/vtest/VTest
https://varnish-cache.org/docs/trunk/reference/varnishtest.html
https://varnish-cache.org/docs/trunk/reference/vtc.html
This commit is contained in:
Frédéric Lécaille 2019-01-11 10:10:22 +01:00 committed by Willy Tarreau
parent 4336123b5c
commit 617d4f5276
2 changed files with 79 additions and 69 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

78
doc/regression-testing.txt
View File

@ -1,74 +1,82 @@
+---------------------------------------------+ +---------------------------------------+
| HAProxy regression testing with varnishtest | | HAProxy regression testing with vtest |
+---------------------------------------------+ +---------------------------------------+
The information found in this file are a short starting guide to help you to The information found in this file are a short starting guide to help you to
write VTC (Varnish Test Case) scripts (or VTC files) for haproxy regression testing. write VTC (Varnish Test Case) scripts (or VTC files) for haproxy regression testing.
Such VTC files are currently used to test Varnish cache application developed by Such VTC files are currently used to test Varnish cache application developed by
Poul-Henning Kamp. A very big thanks you to him for having helped you to add Poul-Henning Kamp. A very big thanks you to him for having helped you to add
our haproxy C modules to varnishtest tool. our haproxy C modules to vtest tool. Note that vtest was formally developed for
varnish cache reg testing and was named varnishtest. vtest is an haproxy specific
version of varnishtest program which reuses the non varnish cache specific code.
A lot of general information about how to write VTC files may be found in 'man/vtc.7' A lot of general information about how to write VTC files may be found in 'man/vtc.7'
manual. It is *highly* recommended to read this manual before asking. This manual of varnish cache sources directory or directly on the web here:
documentation only deals with the varnishtest support for haproxy.
https://varnish-cache.org/docs/trunk/reference/vtc.html
It is *highly* recommended to read this manual before asking to haproxy ML. This
documentation only deals with the vtest support for haproxy.
varnishtest installation vtest installation
------------------------ ------------------------
To use varnishtest you will have to download and compile the recent Varnish cache To use vtest you will have to download and compile the recent vtest
sources found at https://github.com/varnishcache/varnish-cache. sources found at https://github.com/vtest/VTest.
To compile Varnish cache : To compile vtest:
$ ./autogen.sh $ cd VTest
$ ./configure $ make vtest
$ make
The varnishtest sources may be found in 'bin/varnishtest' directory. Note that varnishtest may be also compiled but not without the varnish cache
'bin/varnishtest/tests' is plenty of VTC files for Varnish cache. After having sources already compiled:
compiled these sources, the varnishtest executable location is
'bin/varnishtest/varnishtest'.
varnishtest is able to search for the haproxy executable file it is supposed to $ VARNISH_SRC=<...> make varnishtest
launch in the PATH environment variable. To force the executable to be used by
varnishtest, the HAPROXY_PROGRAM environment variable for varnishtest may be
typically set as follows:
$ HAPROXY_PROGRAM=~/srcs/haproxy/haproxy varnishtest ... After having compiled these sources, the vtest executable location is at the
root of the vtest sources directory.
varnistest exectution vtest exectution
--------------------- ---------------------
varnishtest program comes with interesting options. The most interesting are: vtest is able to search for the haproxy executable file it is supposed to
launch thanks to the PATH environment variable. To force the executable to be used by
vtest, the HAPROXY_PROGRAM environment variable for vtest may be
typically set as follows:
$ HAPROXY_PROGRAM=~/srcs/haproxy/haproxy vtest ...
vtest program comes with interesting options. The most interesting are:
-t Timeout in seconds to abort the test if some launched program -t Timeout in seconds to abort the test if some launched program
-v By default, varnishtest does not dump the outputs of processus it launched -v By default, vtest does not dump the outputs of processus it launched
when the test passes. With this option the outputs are dumped even when the test passes. With this option the outputs are dumped even
when the test passes. when the test passes.
-L to always keep the temporary VTC directories. -L to always keep the temporary VTC directories.
-l to keep the temporary VTC directories only when the test fails. -l to keep the temporary VTC directories only when the test fails.
About haproxy when launched by varnishtest, -d option is enabled by default. About haproxy, when launched by vtest, -d option is enabled by default.
How to write VTC files How to write VTC files
---------------------- ----------------------
A VTC file must start with a "varnishtest" command line followed by a descriptive A VTC file must start with a "varnishtest" or "vtest" command line followed by a
line enclosed by double quotes. This is not specific to the VTC files for haproxy. descriptive line enclosed by double quotes. This is not specific to the VTC files
for haproxy.
The VTC files for haproxy must also contain a "feature ignore_unknown_macro" line The VTC files for haproxy must also contain a "feature ignore_unknown_macro" line
if any macro is used for haproxy in this file. This is due to the fact that if any macro is used for haproxy in this file. This is due to the fact that
varnishtest parser code for haproxy commands generates macros the varnishtest vtest parser code for haproxy commands generates macros the vtest
parser code for varnish has no knowledge of. This line prevents varnishtest from parser code for varnish cache has no knowledge of. This line prevents vtest from
failing in such cases. Since bbc34e2 varnish commit, as a "cli" macro is failing in such cases. As a "cli" macro automatically generated, this
automatically generated, this "feature ignore_unknown_macro" is mandatory "feature ignore_unknown_macro" is mandatory for each VTC file for haproxy.
for each VTC file for haproxy.
To make varnishtest capable of testing haproxy, two new VTC commands have been To make vtest capable of testing haproxy, two new VTC commands have been
implemented: "haproxy" and "syslog". "haproxy" is used to start haproxy processus. implemented: "haproxy" and "syslog". "haproxy" is used to start haproxy processus.
"syslog" is used to start syslog servers (at this time, only used by haproxy). "syslog" is used to start syslog servers (at this time, only used by haproxy).
@ -99,7 +107,7 @@ the configuration file is temporarily written.
A 'stats.sock' UNIX socket is also created in this directory. There is no need A 'stats.sock' UNIX socket is also created in this directory. There is no need
to declare such stats sockets in the -conf {...} section. The name of the parent to declare such stats sockets in the -conf {...} section. The name of the parent
directory of the haproxy instances working directories is stored in 'tmpdir'. In directory of the haproxy instances working directories is stored in 'tmpdir'. In
fact this is the working directory of the current varnishtest processus. fact this is the working directory of the current vtest processus.
There also exists 'testdir' macro which is the parent directory of the VTC file. There also exists 'testdir' macro which is the parent directory of the VTC file.
It may be useful to use other files located in the same directory than the current It may be useful to use other files located in the same directory than the current

70
reg-tests/README
View File

@ -1,63 +1,65 @@
* Regression testing for HAProxy with varnishtest * * Regression testing for HAProxy with VTest *
This little README file is about how to compile and run varnishtest test case files (VTC files) to test HAProxy for any regression. This little README file is about how to compile and run vtest test case files (VTC files)
to test HAProxy for any regression.
To do so, you will have to compile vtest program sources which depends on
Varnish cache application sources. vtest, formerly varnishtest, is a very useful
program which has been developed to test Varnish cache application. vtest has been
modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support
HAProxy in addition to Varnish cache.
To do so, you will have to compile varnishtest program sources which comes with
Varnish cache application. varnishtest is a very useful program which has been
developed to test Varnish cache application. varnishtest has been modified in
collaboration with Varnish cache conceptor Poul-Henning Kamp to support HAProxy in
addition to Varnish cache.
See also: doc/regression-testing.txt See also: doc/regression-testing.txt
* varnishtest compilation * * vtest compilation *
- Clone recent Varnish cache sources which may be found in this GitHub repository $ git clone https://github.com/vtest/VTest
https://github.com/varnishcache/varnish-cache:
$ git clone https://github.com/varnishcache/varnish-cache $ cd VTest
- Compile Varnish cache sources: $ make vtest
$ cd varnish-cache && ./autogen.sh && ./configure && make Then vtest program may be found at the root directory of vtest sources directory.
The Varnish cache manuals are located in 'man' directory of Varnish cache sources
Then varnishtest program may be found in bin/varnishtest directory. directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to
VTC files for Varnish cache may be found in bin/varnishtest/tests directory. use vtest.
The Varnish cache manuals are located in 'man' directory. You will have to
have a look at varnishtest(7) and vtc(7) manuals.
Some information may also be found in doc/regression-testing.txt in HAProxy Some information may also be found in doc/regression-testing.txt in HAProxy
sources. sources.
Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory
of Varnish cache sources directory which may be found here:
https://github.com/varnishcache/varnish-cache
* varnishtest execution *
* vtest execution *
You must set HAPROXY_PROGRAM environment variable to give the location You must set HAPROXY_PROGRAM environment variable to give the location
of the HAProxy program to test to varnishtest: of the HAProxy program to test to vtest:
$ HAPROXY_PROGRAM=<my haproxy program> varnishtest [-Dno-htx=] ... $ HAPROXY_PROGRAM=<my haproxy program> vtest [-Dno-htx=] ...
The HAProxy VTC files found in HAProxy sources may be run with the reg-tests The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
Makefile target. You must set the VARNISHTEST_PROGRAM environment variable to Makefile target. You must set the VTEST_PROGRAM environment variable to
give the location of the varnishtest program which has been previously compiled. give the location of the vtest program which has been previously compiled.
$ VARNISHTEST_PROGRAM=<my varnishtest program> HAPROXY_PROGRAM=<my haproxy program> make reg-tests $ VTEST_PROGRAM=<my vtest program> make reg-tests
Note that varnishtest is run with -t5 and -l option. -l option is to keep "reg-tests" Makefile target run scripts/run-regtest.sh script.
keep varnishtest temporary directory in case of failed test cases. core files To get more information about this script run it with --help option.
Note that vtest is run with -t10 and -l option. -l option is to keep
keep vtest temporary directory in case of failed test cases. core files
may be found in this directory (if enabled by ulimit). may be found in this directory (if enabled by ulimit).
* varnishtest patches for HAProxy VTC files * * vtest patches for HAProxy VTC files *
When producing a patch to add a VCT regression testing file to reg-tests directory, When producing a patch to add a VTC regression testing file to reg-tests directory,
please follow these simple rules: please follow these simple rules:
- Add the commit number on the first line. - If your VTC file needs others files, if possible, use the same basename as that
- Then, copy and paste the commit log. of the VTC file,
- If your VTC file needs others files, use the same basename as that of the VTC
file.
- Put these files in a directory with the same name as the code area concerned - Put these files in a directory with the same name as the code area concerned
by the bug ('peers', 'lua', 'acl' etc). by the bug ('peers', 'lua', 'acl' etc).