Commit Graph

14734 Commits

Author SHA1 Message Date
Willy Tarreau
e1465c1e46 REGTESTS: disable inter-thread idle connection sharing on sensitive tests
Some regtests involve multiple requests from multiple clients, which can
be dispatched as multiple requests to a server. It turns out that the
idle connection sharing works so well that very quickly few connections
are used, and regularly some of the remaining idle server connections
time out at the moment they were going to be reused, causing those random
"HTTP header incomplete" traces in the logs that make them fail often. In
the end this is only an artefact of the test environment.

And indeed, some tests like normalize-uri which perform a lot of reuse
fail very often, about 20-30% of the times in the CI, and 100% of the
time in local when running 1000 tests in a row. Others like ubase64,
sample_fetches or vary_* fail less often but still a lot in tests.

This patch addresses this by adding "tune.idle-pool.shared off" to all
tests which have at least twice as many requests as clients. It proves
very effective as no single error happens on normalize-uri anymore after
10000 tests. Also 100 full runs of all tests yield no error anymore.

One test is tricky, http_abortonclose, it used to fail ~10 times per
1000 runs and with this workaround still fails once every 1000 runs.
But the test is complex and there's a warning in it mentioning a
possible issue when run in parallel due to a port reuse.
2021-05-09 14:41:41 +02:00
Willy Tarreau
6b86d9e485 BUILD: errors: include stdarg in errors.h
It's needed for va_list as defined in ha_vdiag_warning().
2021-05-09 12:11:41 +02:00
Willy Tarreau
2a8a2f0223 BUILD: ssl: define HAVE_CRYPTO_memcmp() based on the library version
The build fails on versions older than 1.0.1d which is the first one
introducing CRYPTO_memcmp(), so let's have a define for this instead
of enabling it whenever USE_OPENSSL is set. One could also wonder why
we're relying on openssl for such a trivial thing, and a simple local
implementation could also allow to restore lexicographic ordering.
2021-05-09 12:10:36 +02:00
Willy Tarreau
48584645fb BUILD: http_fetch: address a few aliasing warnings with older compilers
gcc-4.4 complains about aliasing in smp_fetch_url_port() and
smp_fetch_url_ip() because the local addr variable is casted to sturct
sockaddr_in before being checked. The family should be checked on the
sockaddr_storage and we have a function to retrieve the port.
The compiler still sees some warnings but these ones are OK now.
2021-05-09 10:32:54 +02:00
Willy Tarreau
b2475a139e MINOR: tools/rnd: compute the result outside of the CAS loop
ha_random64() uses a DWCAS loop to produce the random, but it computes
the resulting value inside the loop while it doesn't change upon success,
so this is a needless overhead inside the critcal path that participates
to making threads fail the race and try again. Let's take the value out
of the loop.
2021-05-09 10:26:14 +02:00
Willy Tarreau
714f34580e DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
Some of the Lua doc and a few places still used "Haproxy" or "HAproxy".
There was even one "HA proxy". A few of them were in an example of VTest
output, indicating that VTest ought to be fixed as well. No big deal but
better address all the remaining ones so that these inconsistencies stop
spreading around.
2021-05-09 06:50:46 +02:00
Willy Tarreau
64975cf2a4 MEDIUM: mailers: use "HAProxy" nor "HAproxy" in the subject of messages
It seems to be the last visible casing inconsistency, but better address
it for completeness otherwise we'll always have to deal with some
exceptions.
2021-05-09 06:45:16 +02:00
Willy Tarreau
6613a3df8c ADMIN: netsnmp: report "HAProxy" and not "Haproxy" in output descriptions
In the SNMP fields descriptions, "Haproxy" was mentioned, let's make it
consistent with the rest.
2021-05-09 06:41:37 +02:00
Willy Tarreau
84c4e7451e DOC: peers: fix the protocol tag name in the doc
The peers protocol has been using "HAProxyS" as a binary tag sent on the
wire since day one in 1.5-dev3 with commit 2b920a1af ("[MAJOR] Add new
files src/peer.c, include/proto/peers.h and include/types/peers.h for
sync stick table management"), regardless, the doc says the protocol
identifier is "HaproxyS". It is likely this got fixed in the code before
merging and not in the doc.

This should be backported to any release as the doc is wrong.
2021-05-09 06:38:07 +02:00
Willy Tarreau
58000fe16d DOC: remove last occurrences of "HA-Proxy" syntax
There were only a few more used as output examples and comments in a few
docs, it was the right moment to get rid of them. The file intro.txt
which explains how to parse the version also got a hint about the possible
presence of a hyphen in the name in older versions.
2021-05-09 06:29:40 +02:00
Willy Tarreau
a5357cdfa5 MINOR: version: report "HAProxy" not "HA-Proxy" in the version output
When running "haproxy -v", we still get "HA-Proxy" which is the last
place where this confusing oddity happens. Being so used to it I didn't
even notice it until it was reported to me just after 2.2 but it never
got fixed, despite the PRODUCT_NAME macro that is used to report the
name in the stats page and in "show info" being already set to "HAProxy"
15 years ago in 1.2.14 with commit e03312613. It's about time to
uniformize everything.
2021-05-09 06:14:25 +02:00
Willy Tarreau
5f3448f6de REGTESTS: run-regtests: match both "HAProxy" and "HA-Proxy" in the version
The script checks the HAProxy version and was looking for "HA-Proxy" which
is hard-coded in haproxy.c, let's relax the test to also accept "HAProxy".
2021-05-09 06:12:41 +02:00
Daniel Corbett
9f0843f4e2 DOC: Fix a few grammar/spelling issues and casing of HAProxy
This patch fixes a few grammar and spelling issues in configuration.txt.
It was also noted that there was a wide range of case usage
(i.e. haproxy, HAproxy, HAProxy, etc... ). This patch updates them
all to be consistently "HAProxy" except where a binary is mentioned.
2021-05-09 05:59:49 +02:00
Willy Tarreau
1d14262a69 BUILD: makefile: reorder object files by build time
This is the per-release reordering to improve build parallelism.
It didn't change much, mostly dns+resolvers inflated this time.
Nowadays build times are mostly dominated by the long dependencies
of include files, no less than 170MB of preprocessed code has to be
built, and half of this is SSL support is disabled. Includes should
likely be reworked to be smaller with less dependencies each,
possibly splitting what's the core of each of them and what is used
to interface with other ones. Each split of a .C file in two adds
0.3s of build time just because of this.
2021-05-08 23:40:47 +02:00
Willy Tarreau
a219ec5cb2 BUILD: config: do not include proxy.h nor errors.h anymore in cfgparse.h
These ones induce a long dependency chain and are not needed anymore.
2021-05-08 20:35:39 +02:00
Willy Tarreau
c28aab05d8 BUILD: fd: include log.h from fd.c
It's needed for ha_alert() and the header was missing.
2021-05-08 20:35:39 +02:00
Willy Tarreau
202f93d885 BUILD: comp: include proxy.h from flt_http_comp.c
It's used for proxy_type_str() but the header was missing.
2021-05-08 20:35:39 +02:00
Willy Tarreau
11bd6f7296 BUILD: thread: include log.h from thread.c
It's needed for ha_alert(). Probably that a separate file for error
reporting at boot would be useful.
2021-05-08 20:35:39 +02:00
Willy Tarreau
d1dd2500f2 BUILD: http-rules: include proxy.h from http_rules.c
Many proxy functions are called there but the include was missing and
inherited via cfgparse.h.
2021-05-08 20:35:39 +02:00
Willy Tarreau
5958c43271 BUILD: listener: include proxy.h from listener.c
Many proxy functions are called there but the include was missing and
inherited via cfgparse.h.
2021-05-08 20:35:39 +02:00
Willy Tarreau
c5396bd673 BUILD: mux-fcgi: include proxy.h from mux-fcgi.c
proxy_capture_error() was called there without the include, which was
inherited via cfgparse.h.
2021-05-08 20:35:39 +02:00
Willy Tarreau
adc0240147 BUILD: mux-h1: include proxy.h from mux-h1.c
proxy_capture_error() was called there without the include, which was
inherited via cfgparse.h.
2021-05-08 20:35:39 +02:00
Willy Tarreau
3d6ee407e7 BUILD: hlua: include proxy.h from hlua.c
Many proxy functions are called there but the include was missing and
inherited via cfgparse.h.
2021-05-08 20:35:39 +02:00
Willy Tarreau
32840b77a5 BUILD: connection: stop including listener-t.h
listener-t comes with openssl just due to the SSL_CTX type that is
declred as a typedef in openssl hence cannot be abstracted at this
level. However connection-t.h doen't need all that just to know that
bind_conf is a struct. Let's declare it with other external types
instead..
2021-05-08 20:27:08 +02:00
Willy Tarreau
e08f4bf27f MINOR: task: stop including stream.h from task.c
This one comes with a very deep dependency hell, only to know that
process_stream() is a function. Dropping it reduces the preprocessed
output from 1.5MB to 640kB.
2021-05-08 20:27:08 +02:00
Willy Tarreau
c79e89853b BUILD: task: remove unused includes from task.c
freq_ctr.h and time.h are not used, let's drop them.
2021-05-08 20:27:08 +02:00
Willy Tarreau
08138612a4 REORG: config: uninline warnifnotcap() and failifnotcap()
These ones are used by virtually every config parser. Not only they
provide no benefit in being inlined, but they imply a very deep
dependency starting at proxy.h, which results for example in task.c
including openssl.

Let's move these two functions to cfgparse.c.
2021-05-08 20:27:08 +02:00
Willy Tarreau
6ec1f25bc5 REORG: stick-table: move composite address functions to stick_table.h
These caddr_* functions were once placed into tools.h in the hope they
would be useful but nobody knows they exist. They could deserve being
moved to their own file with other pointer manipulation functions maybe,
but for now they're the only reason left for stick_table.h to include
tools.h, so let's move them directly there since it's its only user.
This allows to remove tools.h from stick_table.h and slightly reduce
the overall build time.
2021-05-08 20:24:09 +02:00
Willy Tarreau
3b63ca20f4 REORG: stick-table: uninline stktable_alloc_data_type()
This function has no business being inlined in stick_table.h since it's
only used at boot time by the config parser. In addition it causes an
undesired dependency on tools.h because it uses parse_time_err(). Let's
move it to stick_table.c.
2021-05-08 20:24:09 +02:00
Willy Tarreau
e59b5169b3 BUILD: connection: move list_mux_proto() to connection.c
No idea why this was put inlined into connection.h, it's used only once
for haproxy -vv, and requires tools.h, causing an undesired dependency
from connection.h. Let's move it to connection.c instead where it ought
to have been.
2021-05-08 20:24:09 +02:00
Willy Tarreau
03f839d0ea BUILD: fcgi-app: include proxy.h in fcgi-app.c
It's needed for proxies_list and used to be inherited via cfgparse.h.
2021-05-08 20:24:09 +02:00
Willy Tarreau
daa6f1a33d BUILD: filters: include proxy.h in filters.c
It's needed for proxies_list and used to be inherited via cfgparse.h.
2021-05-08 20:24:09 +02:00
Willy Tarreau
7c6685770d BUILD: mworker: include proxy.h in mworker.c
It's needed for proxies_list and used to be inherited via cfgparse.h.
2021-05-08 20:24:09 +02:00
Willy Tarreau
817538e397 BUILD: sink: include proxy.h in sink.c
It's needed for proxies_list but was missing.
2021-05-08 20:24:09 +02:00
Willy Tarreau
b00a8e30f1 BUILD: server: include missing proxy.h in server.c
It's needed for a number of functions and definitions but was missing.
2021-05-08 20:24:09 +02:00
Willy Tarreau
ba6300ea62 BUILD: server: include tools.h from server.c
A lot of functions from tools.h are used there but the file was only
inherited via other ones.
2021-05-08 19:37:41 +02:00
Willy Tarreau
5703a38a06 BUILD: stick-table: include freq_ctr.h from stick_table.h
It's needed for update_freq_ctr_period() which is used there.
2021-05-08 19:37:41 +02:00
Willy Tarreau
ce65cbec38 BUILD: udp: include tools.h from proto_udp.c
A few functions are used from there for address conversion but the
file wasn't included.
2021-05-08 13:59:56 +02:00
Willy Tarreau
c1a689f2eb BUILD: queue: include tools.h from queue.c
It uses memprintf() without including the file because it inherited
it from other ones.
2021-05-08 13:59:05 +02:00
Willy Tarreau
745e98ce79 BUILD: mworker: include tools.h from mworker.c
It needs it for memprintf() but didn't include the file.
2021-05-08 13:58:19 +02:00
Willy Tarreau
c624da06c6 BUILD: compression: include tools.h in compression.c
It needs it for memprintf() but it wasn't included.
2021-05-08 13:57:19 +02:00
Willy Tarreau
67046bfc50 BUILD: vars: include tools.h in vars.c
A number of functions from tools.h are used there but the file was not
included.
2021-05-08 13:56:31 +02:00
Willy Tarreau
485261beab BUILD: payload: include tools.h in payload.c
It needs it for memprintf() but used to inherit it via other include files.
2021-05-08 13:55:40 +02:00
Willy Tarreau
9f9e9fc20c BUILD: dns: include tools.h in dns.c
It is used for get_addr_len() without being included. It could be worth
splitting address manipulation functions to a different set of files.
2021-05-08 13:09:46 +02:00
Willy Tarreau
bf1ae1a4b1 BUILD: server-state: include tools.h from server_state.c
Many functions from tools.h are called there without the file being
included.
2021-05-08 13:08:34 +02:00
Willy Tarreau
908908ef2a BUILD: connection: include tools.h in connection.c
Several functions from tools.h are called there without the file being
included.
2021-05-08 13:07:31 +02:00
Willy Tarreau
4bad5e2080 BUILD: sink: include tools.h in sink.c
Several functions from tools.h are used in sink.c without tools.h being
included.
2021-05-08 13:05:30 +02:00
Willy Tarreau
ce6700aec5 BUILD: cache: include tools.h in cache.c
cache.c uses a lot of functions from tools.h without including it.
2021-05-08 13:03:55 +02:00
Willy Tarreau
523ca9d102 BUILD: session: include tools.h in session.c
The file session.c calls plenty of functions from tools.h but did not
include it.
2021-05-08 13:03:04 +02:00
Willy Tarreau
e684483ec5 BUILD: proxy: include tools.h in proxy.c
Many functions are used from tools.h but the file wasn't included and
was inherited through others.
2021-05-08 13:02:07 +02:00