Commit Graph

882 Commits

Author SHA1 Message Date
Willy Tarreau
2b71810cb3 CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
The current "ADD" vs "ADDQ" is confusing because when thinking in terms
of appending at the end of a list, "ADD" naturally comes to mind, but
here it does the opposite, it inserts. Several times already it's been
incorrectly used where ADDQ was expected, the latest of which was a
fortunate accident explained in 6fa922562 ("CLEANUP: stream: explain
why we queue the stream at the head of the server list").

Let's use more explicit (but slightly longer) names now:

   LIST_ADD        ->       LIST_INSERT
   LIST_ADDQ       ->       LIST_APPEND
   LIST_ADDED      ->       LIST_INLIST
   LIST_DEL        ->       LIST_DELETE

The same is true for MT_LISTs, including their "TRY" variant.
LIST_DEL_INIT keeps its short name to encourage to use it instead of the
lazier LIST_DELETE which is often less safe.

The change is large (~674 non-comment entries) but is mechanical enough
to remain safe. No permutation was performed, so any out-of-tree code
can easily map older names to new ones.

The list doc was updated.
2021-04-21 09:20:17 +02:00
Willy Tarreau
c4c80fb4ea MINOR: time: move the time initialization out of tv_update_date()
The time initialization was made a bit complex because we rely on a
dummy negative argument to reset all fields, leaving no distinction
between process-level initialization and thread-level initialization.
This patch changes this by introducing two functions, one for the
process and the second one for the threads. This removes ambigous
test and makes sure that the relevant fields are always initialized
exactly once. This also offers a better solution to the bug fixed in
commit b48e7c001 ("BUG/MEDIUM: time: make sure to always initialize
the global tick") as there is no more special values for global_now_ms.

It's simple enough to be backported if any other time-related issues
are encountered in stable versions in the future.
2021-04-11 23:45:48 +02:00
Willy Tarreau
f459640ef6 MINOR: global: declare a read_mostly section
Some variables are mostly read (mostly pointers) but they tend to be
merged with other ones in the same cache line, slowing their access down
in multi-thread setups. This patch declares an empty, aligned variable
in a section called "read_mostly". This will force a cache-line alignment
on this section so that any variable declared in it will be certain to
avoid false sharing with other ones. The section will be eliminated at
link time if not used.

A __read_mostly attribute was added to compiler.h to ease use of this
section.
2021-04-10 19:27:41 +02:00
Willy Tarreau
185157201c CLEANUP: atomic: add a fetch-and-xxx variant for common operations
The fetch_and_xxx variant is often missing for add/sub/and/or. In fact
it was only provided for ADD under the name XADD which corresponds to
the x86 instruction name. But for destructive operations like AND and
OR it's missing even more as it's not possible to know the value before
modifying it.

This patch explicitly adds HA_ATOMIC_FETCH_{OR,AND,ADD,SUB} which
cover these standard operations, and renames XADD to FETCH_ADD (there
were only 6 call places).

In the future, backport of fixes involving such operations could simply
remap FETCH_ADD(x) to XADD(x), FETCH_SUB(x) to XADD(-x), and for the
OR/AND if needed, these could possibly be done using BTS/BTR.

It's worth noting that xchg could have been renamed to fetch_and_store()
but xchg already has well understood semantics and it wasn't needed to
go further.
2021-04-07 18:18:37 +02:00
Willy Tarreau
1db427399c CLEANUP: atomic: add an explicit _FETCH variant for add/sub/and/or
Currently our atomic ops return a value but it's never known whether
the fetch is done before or after the operation, which causes some
confusion each time the value is desired. Let's create an explicit
variant of these operations suffixed with _FETCH to explicitly mention
that the fetch occurs after the operation, and make use of it at the
few call places.
2021-04-07 18:18:37 +02:00
Amaury Denoyelle
5a6926dcf0 MINOR: diag: create cfgdiag module
This module is intended to serve as a placeholder for various
diagnostics executed after the configuration file has been fully loaded.
2021-04-01 18:03:37 +02:00
Amaury Denoyelle
7b01a8dbdd MINOR: global: define diagnostic mode of execution
Define MODE_DIAG which is used to run haproxy in diagnostic mode. This
mode is used to output extra warnings about possible configuration
blunder or sub-optimal usage. It can be activated with argument '-dD'.

A new output function ha_diag_warning is implemented reserved for
diagnostic output. It serves to standardize the format of diagnostic
messages.

A macro HA_DIAG_WARN_COND is also available to automatically check if
diagnostic mode is on before executing the diagnostic check.
2021-04-01 18:03:37 +02:00
Amaury Denoyelle
27fefa1967 MINOR: proxy: implement a free_proxy function
Move all liberation code related to a proxy in a dedicated function
free_proxy in proxy.c. For now, this function is only called in
haproxy.c. In the future, it will be used to free the lua proxy.

This helps to clean up haproxy.c.
2021-03-26 15:28:33 +01:00
Amaury Denoyelle
68fd7e43d3 REORG: global: move free acl/action in their related source files
Move deinit_acl_cond and deinit_act_rules from haproxy.c respectively in
acl.c and action.c. The name of the functions has been slightly altered,
replacing the prefix deinit_* by free_* to reflect their purpose more
clearly.

This change has been made in preparation to the implementation of a free
proxy function. As a side-effect, it helps to clean up haproxy.c.
2021-03-26 15:28:33 +01:00
Amaury Denoyelle
ce44482fe5 REORG: global: move initcall register code in a dedicated file
Create a new module init which contains code related to REGISTER_*
macros for initcalls. init.h is included in api.h to make init code
available to all modules.

It's a step to clean up a bit haproxy.c/global.h.
2021-03-26 15:28:33 +01:00
Amaury Denoyelle
828adf0121 REORG: server: add a free server function
Create a new server function named free_server. It can be used to
deallocate a server and its member.
2021-03-18 15:37:05 +01:00
Eric Salama
5ba8335186 MINOR: mworker/cli: alert the user if we enabled a master CLI but not the master-worker mode
Declaring a master CLI socket without activating the master-worker mode
is likely a user error, so we issue a warning.

This patch can be backported as far as 1.8.
2021-03-18 09:08:33 +01:00
Willy Tarreau
4975d1482f CLEANUP: cli: rename the last few "stats_" to "cli_"
There were still a very small list of functions, variables and fields
called "stats_" while they were really purely CLI-centric. There's the
frontend called "stats_fe" in the global section, which instantiates a
"cli_applet" called "<CLI>" so it was renamed "cli_fe".

The "alloc_stats_fe" function cas renamed to "cli_alloc_fe" which also
better matches the naming convention of all cli-specific functions.

Finally the "stats_permission_denied_msg" used to return an error on
the CLI was renamed "cli_permission_denied_msg".

Now there's no more "stats_something" that designates the CLI.
2021-03-13 11:04:35 +01:00
Willy Tarreau
060a761248 OPTIM: task: automatically adjust the default runqueue-depth to the threads
The recent default runqueue size reduction appeared to have significantly
lowered performance on low-thread count configs. Testing various values
runqueue values on different workloads under thread counts ranging from
1 to 64, it appeared that lower values are more optimal for high thread
counts and conversely. It could even be drawn that the optimal value for
various workloads sits around 280/sqrt(nbthread), and probably has to do
with both the L3 cache usage and how to optimally interlace the threads'
activity to minimize contention. This is much easier to optimally
configure, so let's do this by default now.
2021-03-10 11:15:34 +01:00
Willy Tarreau
430bf4a483 MINOR: server: allocate a per-thread struct for the per-thread connections stuff
There are multiple per-thread lists in the listeners, which isn't the
most efficient in terms of cache, and doesn't easily allow to store all
the per-thread stuff.

Now we introduce an srv_per_thread structure which the servers will have an
array of, and place the idle/safe/avail conns tree heads into. Overall this
was a fairly mechanical change, and the array is now always initialized for
all servers since we'll put more stuff there. It's worth noting that the Lua
code still has to deal with its own deinit by itself despite being in a
global list, because its server is not dynamically allocated.
2021-03-05 15:00:24 +01:00
Willy Tarreau
198e92a8e5 MINOR: server: add a global list of all known servers
It's a real pain not to have access to the list of all registered servers,
because whenever there is a need to late adjust their configuration, only
those attached to regular proxies are seen, but not the peers, lua, logs
nor DNS.

What this patch does is that new_server() will automatically add the newly
created server to a global list, and it does so as well for the 1 or 2
statically allocated servers created for Lua. This way it will be possible
to iterate over all of them.
2021-03-05 15:00:24 +01:00
Willy Tarreau
61cfdf4fd8 CLEANUP: tree-wide: replace free(x);x=NULL with ha_free(&x)
This makes the code more readable and less prone to copy-paste errors.
In addition, it allows to place some __builtin_constant_p() predicates
to trigger a link-time error in case the compiler knows that the freed
area is constant. It will also produce compile-time error if trying to
free something that is not a regular pointer (e.g. a function).

The DEBUG_MEM_STATS macro now also defines an instance for ha_free()
so that all these calls can be checked.

178 occurrences were converted. The vast majority of them were handled
by the following Coccinelle script, some slightly refined to better deal
with "&*x" or with long lines:

  @ rule @
  expression E;
  @@
  - free(E);
  - E = NULL;
  + ha_free(&E);

It was verified that the resulting code is the same, more or less a
handful of cases where the compiler optimized slightly differently
the temporary variable that holds the copy of the pointer.

A non-negligible amount of {free(str);str=NULL;str_len=0;} are still
present in the config part (mostly header names in proxies). These
ones should also be cleaned for the same reasons, and probably be
turned into ist strings.
2021-02-26 21:21:09 +01:00
Ilya Shipitsin
98a9e1b873 BUILD: SSL: introduce fine guard for RAND_keep_random_devices_open
RAND_keep_random_devices_open is OpenSSL specific function, not
implemented in LibreSSL and BoringSSL. Let us define guard
HAVE_SSL_RAND_KEEP_RANDOM_DEVICES_OPEN in include/haproxy/openssl-compat.h
That guard does not depend anymore on HA_OPENSSL_VERSION
2021-02-22 10:35:23 +01:00
Willy Tarreau
e90904d5a9 MEDIUM: proxy: store the default proxies in a tree by name
Now default proxies are stored into a dedicated tree, sorted by name.
Only unnamed entries are not kept upon new section creation. The very
first call to cfg_parse_listen() will automatically allocate a dummy
defaults section which corresponds to the previous static one, since
the code requires to have one at a few places.

The first immediately visible benefit is that it allows to reuse
alloc_new_proxy() to allocate a defaults section instead of doing it by
hand. And the secret goal is to allow to keep multiple named defaults
section in memory to reuse them from various proxies.
2021-02-12 16:23:46 +01:00
Willy Tarreau
7d0c143185 MINOR: cfgparse: move defproxy to cfgparse-listen as a static
We don't want to expose this one anymore as we'll soon keep multiple
default proxies. Let's move it inside the parser which is the only
place which still uses it, and initialize it on the fly once needed
instead of doing it at boot time.
2021-02-12 16:23:46 +01:00
Willy Tarreau
144289b459 REORG: move init_default_instance() to proxy.c and pass it the defproxy pointer
init_default_instance() was still left in cfgparse.c which is not the
best place to pre-initialize a proxy. Let's place it in proxy.c just
after init_new_proxy(), take this opportunity for renaming it to
proxy_preset_defaults() and taking out init_new_proxy() from it, and
let's pass it the pointer to the default proxy to be initialized instead
of implicitly assuming defproxy. We'll soon be able to exploit this.
Only two call places had to be updated.
2021-02-12 16:23:46 +01:00
Christopher Faulet
f5ea269723 CLEANUP: deinit: release global and per-proxy server-state variables on deinit
The global server-state base directory and file name are now released on
deinit, as well as per-proxy server-state file name.
2021-02-12 16:04:52 +01:00
Amaury Denoyelle
f232cb3e9b MEDIUM: connection: replace idle conn lists by eb trees
The server idle/safe/available connection lists are replaced with ebmb-
trees. This is used to store backend connections, with the new field
connection hash as the key. The hash is a 8-bytes size field, used to
reflect specific connection parameters.

This is a preliminary work to be able to reuse connection with SNI,
explicit src/dst address or PROXY protocol.
2021-02-12 12:33:05 +01:00
Amaury Denoyelle
69c5c3ab33 BUG/MINOR: config: fix leak on proxy.conn_src.bind_hdr_name
Leak for parsing of option usesrc of the source keyword.

This can be backported to 1.8.
2021-01-26 14:48:39 +01:00
Christopher Faulet
4e36682d51 BUG/MINOR: init: Use a dynamic buffer to set HAPROXY_CFGFILES env variable
The HAPROXY_CFGFILES env variable is built using a static trash chunk, via a
call to get_trash_chunk() function. This chunk is reserved during the whole
configuration parsing. It is far too large to guarantee it will not be
reused during the configuration parsing. And in fact, it happens in the lua
code since the commit f67442efd ("BUG/MINOR: lua: warn when registering
action, conv, sf, cli or applet multiple times"), when a lua script is
loaded.

To fix the bug, we now use a dynamic buffer instead. And we call memprintf()
function to handle both the allocation and the formatting. Allocation errors
at this stage are fatal.

This patch should fix the issue #1041. It must be backported as far as 2.0.
2021-01-13 17:45:25 +01:00
Jerome Magnin
50f757c5fd BUG/MINOR: init: enforce strict-limits when using master-worker
The strict-limits global option was introduced with commit 0fec3ab7b
("MINOR: init: always fail when setrlimit fails"). When used in
conjuction with master-worker, haproxy will not fail when a setrlimit
fails. This happens because we only exit() if master-worker isn't used.

This patch removes all tests for master-worker mode for all cases covered
by strict-limits scope.

This should be backported from 2.1 onward.
This should fix issue #1042.

Reviewed by William Dauchy <wdauchy@gmail.com>
2021-01-13 13:17:11 +01:00
Thayne McCombs
4fb255df03 BUG/MINOR: server: Memory leak of proxy.used_server_addr during deinit
GitHub Issue #1037 Reported a memory leak in deinit() caused by an
allocation made in sa2str() that was stored in srv_set_addr_desc().

When destroying each server for a proxy in deinit, include freeing the
memory in the key of server->addr_node.

The leak was introduced in commit 92149f9a8 ("MEDIUM: stick-tables: Add
srvkey option to stick-table") which is not in any released version so
no backport is needed.

Cc: Tim Duesterhus <tim@bastelstu.be>
2021-01-10 07:22:15 +01:00
Thayne McCombs
8f0cc5c4ba CLEANUP: Fix spelling errors in comments
This is from the output of codespell. It's done at once over a bunch
of files and only affects comments, so there is nothing user-visible.
No backport needed.
2021-01-08 14:56:32 +01:00
Willy Tarreau
421ed3952d [RELEASE] Released version 2.4-dev5
Released version 2.4-dev5 with the following main changes :
    - BUG/MEDIUM: mux_h2: Add missing braces in h2_snd_buf()around trace+wakeup
    - BUILD: hpack: hpack-tbl-t.h uses VAR_ARRAY but does not include compiler.h
    - MINOR: time: increase the minimum wakeup interval to 60s
    - MINOR: check: do not ignore a connection header for http-check send
    - REGTESTS: complete http-check test
    - CI: travis-ci: drop coverity scan builds
    - MINOR: atomic: don't use ; to separate instruction on aarch64.
    - IMPORT: xxhash: update to v0.8.0 that introduces stable XXH3 variant
    - MEDIUM: xxhash: use the XXH3 functions to generate 64-bit hashes
    - MEDIUM: xxhash: use the XXH_INLINE_ALL macro to inline all functions
    - CLEANUP: xxhash: remove the unused src/xxhash.c
    - MINOR: sample: add the xxh3 converter
    - REGTESTS: add tests for the xxh3 converter
    - MINOR: protocol: Create proto_quic QUIC protocol layer.
    - MINOR: connection: Attach a "quic_conn" struct to "connection" struct.
    - MINOR: quic: Redefine control layer callbacks which are QUIC specific.
    - MINOR: ssl_sock: Initialize BIO and SSL objects outside of ssl_sock_init()
    - MINOR: connection: Add a new xprt to connection.
    - MINOR: ssl: Export definitions required by QUIC.
    - MINOR: cfgparse: Do not modify the QUIC xprt when parsing "ssl".
    - MINOR: tools: Add support for QUIC addresses parsing.
    - MINOR: quic: Add definitions for QUIC protocol.
    - MINOR: quic: Import C source code files for QUIC protocol.
    - MINOR: listener: Add QUIC info to listeners and receivers.
    - MINOR: server: Add QUIC definitions to servers.
    - MINOR: ssl: SSL CTX initialization modifications for QUIC.
    - MINOR: ssl: QUIC transport parameters parsing.
    - MINOR: quic: QUIC socket management finalization.
    - MINOR: cfgparse: QUIC default server transport parameters init.
    - MINOR: quic: Enable the compilation of QUIC modules.
    - MAJOR: quic: Make usage of ebtrees to store QUIC ACK ranges.
    - MINOR: quic: Attempt to make trace more readable
    - MINOR: quic: Make usage of the congestion control window.
    - MINOR: quic: Flag RX packet as ack-eliciting from the generic parser.
    - MINOR: quic: Code reordering to help in reviewing/modifying.
    - MINOR: quic: Add traces to congestion avoidance NewReno callback.
    - MINOR: quic: Display the SSL alert in ->ssl_send_alert() callback.
    - MINOR: quic: Update the initial salt to that of draft-29.
    - MINOR: quic: Add traces for in flght ack-eliciting packet counter.
    - MINOR: quic: make a packet build fails when qc_build_frm() fails.
    - MINOR: quic: Add traces for quic_packet_encrypt().
    - MINOR: cache: Refactoring of secondary_key building functions
    - MINOR: cache: Avoid storing responses whose secondary key was not correctly calculated
    - BUG/MINOR: cache: Manage multiple headers in accept-encoding normalization
    - MINOR: cache: Add specific secondary key comparison mechanism
    - MINOR: http: Add helper functions to trim spaces and tabs
    - MEDIUM: cache: Manage a subset of encodings in accept-encoding normalizer
    - REGTESTS: cache: Simplify vary.vtc file
    - REGTESTS: cache: Add a specific test for the accept-encoding normalizer
    - MINOR: cache: Remove redundant test in http_action_req_cache_use
    - MINOR: cache: Replace the "process-vary" option's expected values
    - CI: GitHub Actions: enable daily Coverity scan
    - BUG/MEDIUM: cache: Fix hash collision in `accept-encoding` handling for `Vary`
    - MEDIUM: stick-tables: Add srvkey option to stick-table
    - REGTESTS: add test for stickiness using "srvkey addr"
    - BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11
    - BUG/MINOR: sink: Return an allocation failure in __sink_new if strdup() fails
    - BUG/MINOR: lua: Fix memory leak error cases in hlua_config_prepend_path
    - MINOR: lua: Use consistent error message 'memory allocation failed'
    - CLEANUP: Compare the return value of `XXXcmp()` functions with zero
    - CLEANUP: Apply the coccinelle patch for `XXXcmp()` on include/
    - CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/
    - MINOR: qpack: Add static header table definitions for QPACK.
    - CLEANUP: qpack: Wrong comment about the draft for QPACK static header table.
    - CLEANUP: quic: Remove useless QUIC event trace definitions.
    - BUG/MINOR: quic: Possible CRYPTO frame building errors.
    - MINOR: quic: Pass quic_conn struct to frame parsers.
    - BUG/MINOR: quic: Wrong STREAM frames parsing.
    - MINOR: quic: Drop packets with STREAM frames with wrong direction.
    - CLEANUP: ssl: Remove useless loop in tlskeys_list_get_next()
    - CLEANUP: ssl: Remove useless local variable in tlskeys_list_get_next()
    - MINOR: ssl: make tlskeys_list_get_next() take a list element
    - Revert "BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11"
    - BUG/MINOR: cfgparse: Fail if the strdup() for `rule->be.name` for `use_backend` fails
    - CLEANUP: mworker: remove duplicate pointer tests in cfg_parse_program()
    - CLEANUP: Reduce scope of `header_name` in http_action_store_cache()
    - CLEANUP: Reduce scope of `hdr_age` in http_action_store_cache()
    - CLEANUP: spoe: fix typo on `var_check_arg` comment
    - BUG/MINOR: tcpcheck: Report a L7OK if the last evaluated rule is a send rule
    - CI: github actions: build several popular "contrib" tools
    - DOC: Improve the message printed when running `make` w/o `TARGET`
    - BUG/MEDIUM: server: srv_set_addr_desc() crashes when a server has no address
    - REGTESTS: add unresolvable servers to srvkey-addr
    - BUG/MINOR: stats: Make stat_l variable used to dump a stat line thread local
    - BUG/MINOR: quic: NULL pointer dereferences when building post handshake frames.
    - SCRIPTS: improve announce-release to support different tag and versions
    - SCRIPTS: make announce release support preparing announces before tag exists
    - CLEANUP: assorted typo fixes in the code and comments
    - BUG/MINOR: srv: do not init address if backend is disabled
    - BUG/MINOR: srv: do not cleanup idle conns if pool max is null
    - CLEANUP: assorted typo fixes in the code and comments
    - CLEANUP: few extra typo and fixes over last one ("ot" -> "to")
2021-01-06 17:41:32 +01:00
Tim Duesterhus
e5ff14100a CLEANUP: Compare the return value of XXXcmp() functions with zero
According to coding-style.txt it is recommended to use:

`strcmp(a, b) == 0` instead of `!strcmp(a, b)`

So let's do this.

The change was performed by running the following (very long) coccinelle patch
on src/:

    @@
    statement S;
    expression E;
    expression F;
    @@

      if (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
      )
    (
      S
    |
      { ... }
    )

    @@
    statement S;
    expression E;
    expression F;
    @@

      if (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
      )
    (
      S
    |
      { ... }
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G &&
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G ||
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    && G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    || G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G &&
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G ||
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    && G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    || G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )
2021-01-04 10:09:02 +01:00
David Carlier
2d0493af49 BUILD/MINOR: haproxy DragonFlyBSD affinity build update.
sched_setaffinity supported by this platform.
2020-12-02 22:43:57 +01:00
Christopher Faulet
bb9fb8b7f8 MINOR: config: Deprecate and ignore tune.chksize global option
This option is now ignored because I/O check buffers are now allocated using the
buffer pool. Thus, it is marked as deprecated in the documentation and ignored
during the configuration parsing. The field is also removed from the global
structure.

Because this option is ignored since a recent fix, backported as fare as 2.2,
this patch should be backported too. Especially because it updates the
documentation.
2020-11-27 10:30:23 +01:00
Ilya Shipitsin
d9a16dc0f2 BUILD: SSL: add BoringSSL guarding to "RAND_keep_random_devices_open"
"RAND_keep_random_devices_open" is OpenSSL specific, does not present
in other OpenSSL variants like LibreSSL or BoringSSL. BoringSSL recently
"updated" its internal openssl version to 1.1.1, we temporarily set it
back to 1.1.0, as we are going to remove that hack, let us add proper
guarding.
2020-11-24 09:54:44 +01:00
Tim Duesterhus
c8d19702f4 BUILD: Show the value of DEBUG= in haproxy -vv
Previously this was not visible after building.
2020-11-21 18:27:33 +01:00
Christopher Faulet
83fefbcdff MINOR: init: Fix the prototype for per-thread free callbacks
Functions registered to release memory per-thread have no return value. But the
registering function and the function pointer in per_thread_free_fct structure
specify it should return an integer. This patch fixes it.

This patch may be backported as far as 2.0.
2020-11-13 16:26:10 +01:00
Christopher Faulet
d5bd824b81 BUG/MINOR: proxy/server: Skip per-proxy/server post-check for disabled proxies
per-proxy and per-server post-check callback functions must be skipped for
disabled proxies because most of the configuration validity check is skipped for
these proxies.

This patch must be backported as far as 2.1.
2020-11-03 10:23:00 +01:00
Willy Tarreau
ac66d6bafb MINOR: proxy; replace the spinlock with an rwlock
This is an anticipation of finer grained locking for the queues. For now
all lock places take a write lock so that there is no difference at all
with previous code.
2020-10-22 17:32:28 +02:00
Willy Tarreau
f42d794d96 MEDIUM: config: report that "nbproc" is deprecated
As previously discussed, nbproc usage is bad, deprecated, and scheduled
for removal in 2.5.

If "nbproc" is found with more than one process while nbthread is not
set, a warning will be emitted encouraging to remove it or to migrate
to nbthread instead. This makes sure the user has an opportunity to
both see the message and silence it.
2020-10-20 11:54:49 +02:00
Willy Tarreau
cd10def825 MINOR: backend: replace the lbprm lock with an rwlock
It was previously a spinlock, and it happens that a number of LB algos
only lock it for lookups, without performing any modification. Let's
first turn it to an rwlock and w-lock it everywhere. This is strictly
identical.

It was carefully checked that every HA_SPIN_LOCK() was turned to
HA_RWLOCK_WRLOCK() and that HA_SPIN_UNLOCK() was turned to
HA_RWLOCK_WRUNLOCK() on this lock. _INIT and _DESTROY were updated too.
2020-10-17 18:51:41 +02:00
Willy Tarreau
a74cb38e7c MINOR: protocol: register the receiver's I/O handler and not the protocol's
Now we define a new sock_accept_iocb() for socket-based stream protocols
and use it as a wrapper for listener_accept() which now takes a listener
and not an FD anymore. This will allow the receiver's I/O cb to be
redefined during registration, and more specifically to get rid of the
hard-coded hacks in protocol_bind_all() made for syslog.

The previous ->accept() callback in the protocol was removed since it
doesn't have anything to do with accept() anymore but is more generic.
A few places where listener_accept() was compared against the FD's IO
callback for debugging purposes on the CLI were updated.
2020-10-15 21:47:56 +02:00
Willy Tarreau
1a3770cbc7 BUG/MEDIUM: deinit: check fdtab before fdtab[fd].owner
When running a pure config check (haproxy -c) we go through the deinit
phase without having allocated fdtab, so we can't blindly dereference
it. The issue was added by recent commit ae7bc4a23 ("MEDIUM: deinit:
close all receivers/listeners before scanning proxies"), no backport is
needed.
2020-10-14 12:13:51 +02:00
Willy Tarreau
2bd0f8147b BUG/MINOR: init: only keep rlim_fd_cur if max is unlimited
On some operating systems, RLIM_INFINITY is set to -1 so that when the
hard limit on the number of FDs is set to unlimited, taking the MAX
of both values keeps rlim_fd_cur and everything works. But on other
systems this values is defined as the highest positive integer. This
is what was observed on a 32-bit AIX 5.1. The effect is that maxsock
becomes 2^31-1 and that fdtab allocation fails.

Note that a simple workaround consists in manually setting maxconn in
the global section.

Let's ignore unlimited as soon as we retrieve rlim_fd_max so that all
systems behave consistently.

This may be backported as far as 2.0, though it doesn't seem like it
has annoyed anyone.
2020-10-13 15:36:08 +02:00
Willy Tarreau
0a002df2c2 BUG/MINOR: proxy: respect the proper format string in sig_pause/sig_listen
When factoring out the pause/resume error messages in commit 775e00158
("MAJOR: signals: use protocol_pause_all() and protocol_resume_all()")
I forgot that ha_warning() and send_log() take a format string and not
just a const string. No backport is needed, this is 2.3-dev.
2020-10-09 19:26:27 +02:00
Willy Tarreau
775e00158a MAJOR: signals: use protocol_pause_all() and protocol_resume_all()
When temporarily pausing the listeners with SIG_TTOU, we now pause
all listeners via the protocols instead of the proxies. This has the
benefits that listeners are paused regardless of whether or not they
belong to a visible proxy. And for resuming via SIG_TTIN we do the
same, which allows to report binding conflicts and address them,
since the operation can be repeated on a per-listener basis instead
of a per-proxy basis.

While in appearance all cases were properly handled, it's impossible
to completely rule out the possibility that something broken used to
work by luck due to the scan ordering which is naturally different,
hence the major tag.
2020-10-09 11:27:30 +02:00
Willy Tarreau
337c835d16 MEDIUM: proxy: merge zombify_proxy() with stop_proxy()
The two functions don't need to be distinguished anymore since they have
all the necessary info to act as needed on their listeners. Let's just
pass via stop_proxy() and make it check for each listener which one to
close or not.
2020-10-09 11:27:30 +02:00
Willy Tarreau
43ba3cf2b5 MEDIUM: proxy: remove start_proxies()
Its sole remaining purpose was to display "proxy foo started", which
has little benefit and pollutes output for those with plenty of proxies.
Let's remove it now.

The VTCs were updated to reflect this, because many of them had explicit
counts of dropped lines to match this message.

This is tagged as MEDIUM because some users may be surprized by the
loss of this quite old message.
2020-10-09 11:27:30 +02:00
Willy Tarreau
c3914d4fff MEDIUM: proxy: replace proxy->state with proxy->disabled
The remaining proxy states were only used to distinguish an enabled
proxy from a disabled one. Due to the initialization order, both
PR_STNEW and PR_STREADY were equivalent after startup, and they
would only differ from PR_STSTOPPED when the proxy is disabled or
shutdown (which is effectively another way to disable it).

Now we just have a "disabled" field which allows to distinguish them.
It's becoming obvious that start_proxies() is only used to print a
greeting message now, that we'd rather get rid of. Probably that
zombify_proxy() and stop_proxy() should be merged once their
differences move to the right place.
2020-10-09 11:27:30 +02:00
Willy Tarreau
b50bf046e8 MINOR: startup: don't rely on PR_STNEW to check for listeners
Instead of looking at listeners in proxies in PR_STNEW state, we'd
rather check for listeners in those not in PR_STSTOPPED as it's only
this state which indicates the proxy was disabled. And let's check
the listeners count instead of testing the list's head.
2020-10-09 11:27:30 +02:00
Willy Tarreau
ae7bc4a237 MEDIUM: deinit: close all receivers/listeners before scanning proxies
Because of the zombie state, proxies have a skewed vision of the state
of listeners, which explains why there are hacks switching the state
from ZOMBIE to INIT in the proxy cleaning loop. This is particularly
complicated and not needed, as all the information is now available
in the protocol list and the fdtab.

What we do here instead is to first close all active listeners or
receivers by protocol and clean their protocol parts. Then we scan the
fdtab to get rid of remaining ones that were necessarily in INIT state
after a previous invocation of delete_listener(). From this point, we
know the listeners are cleaned, the can safely be freed by scanning the
proxies.
2020-10-09 11:27:29 +02:00
Willy Tarreau
02b092f006 MEDIUM: init: stop disabled proxies after initializing fdtab
During the startup process we don't have any fdtab nor fd_updt for quite
a long time, and as such some operations on the listeners are not
permitted, such as fd_want_*/fd_stop_* or fd_delete(). The latter is of
particular concern because it's used when stopping a disabled frontend,
and it's performed very early during check_config_validity() while there
is no fdtab yet. The trick till now relies on the listener's state which
is a bit brittle.

There is absolutely no valid reason for stopping a proxy's listeners this
early, we can postpone it after init_pollers() which will at least have
allocated fdtab.
2020-10-09 11:27:29 +02:00