Commit Graph

16085 Commits

Author SHA1 Message Date
Aurelien DARRAGON
33a8c2842b BUG/MINOR: hlua_fcn/queue: use atomic load to fetch queue size
In hlua_queue_size(), queue size is loaded as a regular int, but the
queue might be shared by multiple threads that could perform some
atomic pushing or popping attempts in parallel, so we better use an
atomic load operation to guarantee consistent readings.

This could be backported in 2.8.
2023-07-11 16:04:39 +02:00
Aurelien DARRAGON
c38cf3cf98 BUG/MINOR: sink/log: properly deinit srv in sink_new_from_logsrv()
When errors are encountered in sink_new_from_logsrv() function,
incompetely allocated ressources are freed to prevent memory leaks.

For instance: logsrv implicit server is manually cleaned up on error prior
to returning from the function.

However, since 198e92a8e5 ("MINOR: server: add a global list of all known
servers") every server created using new_server() is registered to the
global list, but unfortunately the manual srv cleanup in
sink_new_from_logsrv() doesn't remove the srv from the global list, so the
freed server will still be referenced there, which can result in invalid
reads later.

Moreover, server API has evolved since, and now the srv_drop() function is
available for that purpose, so let's use it, but make sure that srv is
freed before the proxy because on older versions srv_drop() expects the
srv to be linked to a valid proxy pointer.

This must be backported up to 2.4.

[For 2.4 version, free_server() must be used instead of srv_drop()]
2023-07-11 10:26:09 +02:00
Aurelien DARRAGON
b58bd9794f MINOR: hlua_fcn/mailers: handle timeout mail from mailers section
As the example/lua/mailers.lua script does its best to mimic the c-mailer
implementation, it should support the "timeout mail" directive as well.

This could be backported in 2.8.
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
b2f7069479 BUG/MINOR: server: set rid default value in new_server()
srv->rid default value is set in _srv_parse_init() after the server is
succesfully allocated using new_server().

This is wrong because new_server() can be used independently so rid value
assignment would be skipped in this case.

Hopefully new_server() allocates server data using calloc() so srv->rid
is already set to 0 in practise. But if calloc() is replaced by malloc()
or other memory allocating function that doesn't zero-initialize srv
members, this could lead to rid being uninitialized in some cases.

This should be backported in 2.8 with 61e3894dfe ("MINOR: server: add
srv->rid (revision id) value")
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
9859e00981 BUG/MINOR: sink: fix errors handling in cfg_post_parse_ring()
Multiple error paths (memory,IO related) in cfg_post_parse_ring() were
not implemented correcly and could result in memory leak or undefined
behavior.

Fixing them all at once.

This can be backported in 2.4
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
a26b736300 BUG/MINOR: sink: invalid sft free in sink_deinit()
sft freeing attempt made in a575421 ("BUG/MINOR: sink: missing sft free in
sink_deinit()") is incomplete, because sink->sft is meant to be used as a
list and not a single sft entry.

Because of that, the previous fix only frees the first sft entry, which
fixes memory leaks for single-server forwarders (this is the case for
implicit rings), but could still result in memory leaks when multiple
servers are configured in a explicit ring sections.

What this patch does: instead of directly freeing sink->sft, it iterates
over every list members to free them.

It must be backported up to 2.4 with a575421.
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
9f9d557468 BUG/MINOR: log: free errmsg on error in cfg_parse_log_forward()
When leaving cfg_parse_log_forward() on error paths, errmsg which is local
to the function could still point to valid data, and it's our
responsibility to free it.

Instead of freeing it everywhere it is invoved, we free it prior to
leaving the function.

This should be backported as far as 2.4.
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
21cf42f579 BUG/MINOR: log: fix multiple error paths in cfg_parse_log_forward()
Multiple error paths were badly handled in cfg_parse_log_forward():
some errors were raised without interrupting the function execution,
resulting in undefined behavior.

Instead of fixing issues separately, let's fix the whole function at once.
This should be backported as far as 2.4.
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
d1af50c807 BUG/MINOR: log: fix missing name error message in cfg_parse_log_forward()
"missing name for ip-forward section" is generated instead of "missing
name name for log-forward section" in cfg_parse_log_forward().

This may be backported up to 2.4.
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
47ee036e5f BUG/MEDIUM: log: improper use of logsrv->maxlen for buffer targets
In e709e1e ("MEDIUM: logs: buffer targets now rely on new sink_write")
we started using the sink API instead of using the ring_write function
directly.

But as indicated in the commit message, the maxlen parameter of the log
directive now only applies to the message part and not the complete
payload. I don't know what the original intent was (maybe minimizing code
changes) but it seems wrong, because the doc doesn't mention this special
case, and the result is that the ring->buffer output can differ from all
other log output types, making it very confusing.

One last issue with this is that log messages can end up being dropped at
runtime, only for the buffer target, and even if logsrv->maxlen is
correctly set (including default: 1024) because depending on the generated
header size the payload can grow bigger than the accepted sink size (sink
maxlen is not mandatory) and we have no simple way to detect this at
configuration time.

First, we partially revert e709e1e:

  TARGET_BUFFER still leverages the proper sink API, but thanks to
  "MINOR: sink: pass explicit maxlen parameter to sink_write()" we now
  explicitly pass the logsrv->maxlen to the sink_write function in order
  to stop writing as soon as either sink->maxlen or logsrv->maxlen is
  reached.

This restores pre-e709e1e behavior with the added benefit from using the
high-level API, which includes automatically announcing dropped message
events.

Then, we also need to take the ending '\n' into account: it is not
explicitly set when generating the logline for TARGET_BUFFER, but it will
be forcefully added by the sink_forward_io_handler function from the tcp
handler applet when log messages from the buffer are forwarded to tcp
endpoints.

In current form, because the '\n' is added later in the chain, maxlen is
not being considered anymore, so the final log message could exceed maxlen
by 1 byte, which could make receiving servers unhappy in logging context.

To prevent this, we sacrifice 1 byte from the logsrv->maxlen to ensure
that the final message will never exceed log->maxlen, even if the '\n'
char is automatically appended later by the forwarding applet.

Thanks to this change TCP (over RING/BUFFER) target now behaves like
FD and UDP targets.

This commit depends on:
 - "MINOR: sink: pass explicit maxlen parameter to sink_write()"

It may be backported as far as 2.2

[For 2.2 and 2.4 the patch does not apply automatically, the sink_write()
call must be updated by hand]
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
b6e2d62fb3 MINOR: sink/api: pass explicit maxlen parameter to sink_write()
sink_write() currently relies on sink->maxlen to know when to stop
writing a given payload.

But it could be useful to pass a smaller, explicit value to sink_write()
to stop before the ring maxlen, for instance if the ring is shared between
multiple feeders.

sink_write() now takes an optional maxlen parameter:
  if maxlen is > 0, then sink_write will stop writing at maxlen if maxlen
  is smaller than ring->maxlen, else only ring->maxlen will be considered.

[for haproxy <= 2.7, patch must be applied by hand: that is:
__sink_write() and sink_write() should be patched to take maxlen into
account and function calls to sink_write() should use 0 as second argument
to keep original behavior]
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
901f31bc9a BUG/MINOR: log: LF upsets maxlen for UDP targets
A regression was introduced with 5464885 ("MEDIUM: log/sink: re-work
and merge of build message API.").

For UDP targets, a final '\n' is systematically inserted, but with the
rework of the build message API, it is inserted after the maxlen
limitation has been enforced, so this can lead to the final message
becoming maxlen+1. For strict syslog servers that only accept up to
maxlen characters, this could be a problem.

To fix the regression, we take the final '\n' into account prior to
building the message, like it was done before the rework of the API.

This should be backported up to 2.4.
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
c103379847 BUG/MINOR: ring: maxlen warning reported as alert
When maxlen parameter exceeds sink size, a warning is generated and maxlen
is enforced to sink size. But the err_code is incorrectly set to ERR_ALERT

Indeed, being a "warning", ERR_WARN should be used here.

This may be backported as far as 2.2
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
30ff33bd9b BUG/MINOR: ring: size warning incorrectly reported as fatal error
When a ring section defines its size using the "size" directive with a
smaller size than the default one or smaller than the previous one,
a warning is generated to inform the user that the new size will be
ignored.

However the err_code is returned as FATAL, so this cause haproxy to
incorrectly abort the init sequence.

Changing the err_code to ERR_WARN so that this warning doesn't refrain
from successfully starting the process.

This should be backported as far as 2.4
2023-07-10 18:28:08 +02:00
Aurelien DARRAGON
a5754219e7 BUG/MINOR: sink: missing sft free in sink_deinit()
Adding missing free for sft (string_forward_target) in sink_deinit(),
which resulted in minor leak for each declared ring target at deinit().
(either explicit and implicit rings are affected)

This may be backported up to 2.4.
2023-07-06 15:41:17 +02:00
Aurelien DARRAGON
5028a6e50b BUG/MINOR: http_ext: unhandled ERR_ABORT in proxy_http_parse_7239()
_proxy_http_parse_7239_expr() helper used in proxy_http_parse_7239()
function may return ERR_ABORT in case of memory error. But the error check
used below is insufficient to catch ERR_ABORT so the function could keep
executing prior to returning ERR_ABORT, which may cause undefined
behavior. Hopefully no sensitive handling is performed in this case so
this bug has very limited impact, but let's fix it anyway.

We now use ERR_CODE mask instead of ERR_FATAL to check if err_code is set
to any kind of error combination that should prevent the function from
further executing.

This may be backported in 2.8 with b2bb9257d2 ("MINOR: proxy/http_ext:
introduce proxy forwarded option")
2023-07-06 15:41:17 +02:00
Aurelien DARRAGON
999699a277 BUG/MEDIUM: sink: invalid server list in sink_new_from_logsrv()
forward proxy server list created from sink_new_from_logsrv() is invalid

Indeed, srv->next is literally assigned to itself. This did not cause
issues during syslog handling because the sft was properly set, but it
will cause the free_proxy(sink->forward_px) at deinit to go wild since
free_proxy() will try to iterate through the proxy srv list to free
ressources, but because of the improper list initialization, double-free
and infinite-loop will occur.

This bug was revealed by 9b1d15f53a ("BUG/MINOR: sink: free forward_px on deinit()")

It must be backported as far as 2.4.
2023-07-06 15:41:17 +02:00
Remi Tricot-Le Breton
ca4fd73938 BUG/MINOR: cache: A 'max-age=0' cache-control directive can be overriden by a s-maxage
When a s-maxage cache-control directive is present, it overrides any
other max-age or expires value (see section 5.2.2.9 of RFC7234). So if
we have a max-age=0 alongside a strictly positive s-maxage, the response
should be cached.

This bug was raised in GitHub issue #2203.
The fix can be backported to all stable branches.
2023-07-04 22:15:00 +02:00
Frédéric Lécaille
6aeaa73d39 BUG/MINOR: quic: Possible crash in "show quic" dumping packet number spaces
This bug was introduced by this commit:

    MEDIUM: quic: Release encryption levels and packet number spaces asap

Add some checks before derefencing pointers to packet number spaces objects
to dump them from "show quic" command.

No backport needed.
2023-07-03 17:20:55 +02:00
Aurelien DARRAGON
4f0e0f5a65 MEDIUM: sample: introduce 'same' output type
Thierry Fournier reported an annoying side-effect when using the debug()
converter.

Consider the following examples:

[1]  http-request set-var(txn.test) bool(true),ipmask(24)
[2]  http-request redirect location /match if { bool(true),ipmask(32) }

When starting haproxy with [1] example we get:

   config : parsing [test.conf:XX] : error detected in frontend 'fe' while parsing 'http-request set-var(txn.test)' rule : converter 'ipmask' cannot be applied.

With [2], we get:

  config : parsing [test.conf:XX] : error detected in frontend 'fe' while parsing 'http-request redirect' rule : error in condition: converter 'ipmask' cannot be applied in ACL expression 'bool(true),ipmask(32)'.

Now consider the following examples which are based on [1] and [2]
but with the debug() sample conv inserted in-between those incompatible
sample types:

[1*] http-request set-var(txn.test) bool(true),debug,ipmask(24)
[2*] http-request redirect location /match if { bool(true),debug,ipmask(32) }

According to the documentation, "it is safe to insert the debug converter
anywhere in a chain, even with non-printable sample types".
Thus we don't expect any side-effect from using it within a chain.

However in current implementation, because of debug() returning SMP_T_ANY
type which is a pseudo type (only resolved at runtime), the sample
compatibility checks performed at parsing time are completely uneffective.
(haproxy will start and no warning will be emitted)

The indesirable effect of this is that debug() prevents haproxy from
warning you about impossible type conversions, hiding potential errors
in the configuration that could result to unexpected evaluation or logic
while serving live traffic. We better let haproxy warn you about this kind
of errors when it has the chance.

With our previous examples, this could cause some inconveniences. Let's
say for example that you are testing a configuration prior to deploying
it. When testing the config you might want to use debug() converter from
time to time to check how the conversion chain behaves.

Now after deploying the exact same conf, you might want to remove those
testing debug() statements since they are no longer relevant.. but
removing them could "break" the config and suddenly prevent haproxy from
starting upon reload/restart. (this is the expected behavior, but it
comes a bit too late because of debug() hiding errors during tests..)

To fix this, we introduce a new output type for sample expressions:
  SMP_T_SAME - may only be used as "expected" out_type (parsing time)
               for sample converters.

As it name implies, it is a way for the developpers to indicate that the
resulting converter's output type is guaranteed to match the type of the
sample that is presented on the converter's input side.
(converter may alter data content, but data type must not be changed)

What it does is that it tells haproxy that if switching to the converter
(by looking at the converter's input only, since outype is SAME) is
conversion-free, then the converter type can safely be ignored for type
compatibility checks within the chain.

debug()'s out_type is thus set to SMP_T_SAME instead of ANY, which allows
it to fully comply with the doc in the sense that it does not impact the
conversion chain when inserted between sample items.

Co-authored-by: Thierry Fournier <thierry.f.78@gmail.com>
2023-07-03 16:32:01 +02:00
Aurelien DARRAGON
4136aba830 MEDIUM: tree-wide: fetches that may return IPV4+IPV6 now return ADDR
Historically, the ADDR pseudo-type did not exist. So when IPV6 support
was added to existing IPV4 sample fetches (e.g.: src,dst,hdr_ip...) the
expected out_type in related sample definitions was left on IPV4 because
it was required to declare the out_type as the lowest common denominator
(the type that can be casted into all other ones) to make compatibility
checks at parse time work properly.

However, now that ADDR pseudo-type may safely be used as out_type since
("MEDIUM: sample: add missing ADDR=>? compatibility matrix entries"), we
can use ADDR for fetches that may output both IPV4 and IPV6 at runtime.

One added benefit on top of making the code less confusing is that
'haproxy -dKsmp' output will now show "addr" instead of "ipv4" for such
fetches, so the 'haproxy -dKsmp' output better complies with the fetches
signatures from the documentation.

out_ip fetch, which returns an ip according to the doc, was purposely
left as is (returning IPV4) since the smp_fetch_url_ip() implementation
forces output type to IPV4 anyway, and since this is an historical fetch
I prefer not to touch it to prevent any regression. However if
smp_fetch_url_ip() were to be fixed to also return IPV6 in the future,
then its expected out_type may be changed to ADDR as well.

Multiple notes in the code were updated to mention that the appropriate
pseudo-type may be used instead of the lowest common denominator for
out_type when available.
2023-07-03 16:32:01 +02:00
Aurelien DARRAGON
67544fdd1c MINOR: sample: fix ipmask sample definition
Since 1478aa7 ("MEDIUM: sample: Add IPv6 support to the ipmask converter")
ipmask converter may return IPV6 as well, so the sample definition must
be updated. (the output type is currently explicited to IPV4)

This has no functional impact: one visible effect will be that
converter's visible output type will change from "ipv4" to "addr" when
using 'haproxy -dKcnv'.
2023-07-03 16:32:01 +02:00
Aurelien DARRAGON
a635a1779a MEDIUM: sample: add missing ADDR=>? compatibility matrix entries
SMP_T_ADDR support was added in b805f71 ("MEDIUM: sample: let the cast
functions set their output type").

According to the above commit, it is made clear that the ADDR type is
a pseudo/generic type that may be used for compatibility checks but
that cannot be emitted from a fetch or converter.

With that in mind, all conversions from ADDR to other types were
explicitly disabled in the compatibility matrix.

But later, when map_*_ip functions were updated in b2f8f08 ("MINOR: map:
The map can return IPv4 and IPv6"), we started using ADDR as "expected"
output type for converters. This still complies with the original
description from b805f71, because it is used as the theoric output
type, and is never emitted from the converters themselves (only "real"
types such as IPV4 or IPV6 are actually being emitted at runtime).

But this introduced an ambiguity as well as a few bugs, because some
compatibility checks are being performed at config parse time, and thus
rely on the expected output type to check if the conversion from current
element to the next element in the chain is theorically supported.

However, because the compatibility matrix doesn't support ADDR to other
types it is currently impossible to use map_*_ip converters in the middle
of a chain (the only supported usage is when map_*_ip converters are
at the very end of the chain).

To illustrate this, consider the following examples:

  acl test str(ok),map_str_ip(test.map) -m found                          # this will work
  acl test str(ok),map_str_ip(test.map),ipmask(24) -m found               # this will raise an error

Likewise, stktable_compatible_sample() check for stick tables also relies
on out_type[table_type] compatibility check, so map_*_ip cannot be used
with sticktables at the moment:

  backend st_test
    stick-table type string size 1m expire 10m store http_req_rate(10m)
  frontend fe
    bind localhost:8080
    mode http
    http-request track-sc0 str(test),map_str_ip(test.map) table st_test     # raises an error

To fix this, and prevent future usage of ADDR as expected output type
(for either fetches or converters) from introducing new bugs, the
ADDR=>? part of the matrix should follow the ANY type logic.

That is, ADDR, which is a pseudo-type, must be compatible with itself,
and where IPV4 and IPV6 both support a backward conversion to a given
type, ADDR must support it as well. It is done by setting the relevant
ADDR entries to c_pseudo() in the compatibility matrix to indicate that
the operation is theorically supported (c_pseudo() will never be executed
because ADDR should not be emitted: this only serves as a hint for
compatibility checks at parse time).

This is what's being done in this commit, thanks to this the broken
examples documented above should work as expected now, and future
usage of ADDR as out_type should not cause any issue.
2023-07-03 16:32:01 +02:00
Aurelien DARRAGON
30cd137d3f MINOR: sample: introduce c_pseudo() conv function
This function is used for ANY=>!ANY conversions in the compatibility
matrix to help differentiate between real NOOP (c_none) and pseudo
conversions that are theorically supported at config parse time but can
never occur at runtime,. That is, to explicit the fact that actual related
runtime operations (e.g.: ANY->IPV4) are not NOOP since they might require
some conversion to be performed depending on the input type.

When checking the conf we don't know the effective out types so
cast[pseudo type][pseudo type] is allowed in the compatibility matrix,
but at runtime we only expect cast[real type][(real type || pseudo type)]
because fetches and converters may not emit pseudo types, thus using
c_none() everywhere was too ambiguous.

The process will crash if c_pseudo() is invoked to help catch bugs:
crashing here means that a pseudo type has been encountered on a
converter's input at runtime (because it was emitted earlier in the
chain), which is not supported and results from a broken sample fetch
or converter implementation. (pseudo types may only be used as out_type
in sample definitions for compatibility checks at parsing time)
2023-07-03 16:32:01 +02:00
Aurelien DARRAGON
58bbe41cb8 MEDIUM: acl/sample: unify sample conv parsing in a single function
Both sample_parse_expr() and parse_acl_expr() implement some code
logic to parse sample conv list after respective fetch or acl keyword.

(Seems like the acl one was inspired by the sample one historically)

But there is clearly code duplication between the two functions, making
them hard to maintain.
Hopefully, the parsing logic between them has stayed pretty much the
same, thus the sample conv parsing part may be moved in a dedicated
helper parsing function.

This is what's being done in this commit, we're adding the new function
sample_parse_expr_cnv() which does a single thing: parse the converters
that are listed right after a sample fetch keyword and inject them into
an already existing sample expression.

Both sample_parse_expr() and parse_acl_expr() were adapted to now make
use of this specific parsing function and duplicated code parts were
cleaned up.

Although sample_parse_expr() remains quite complicated (numerous function
arguments due to contextual parsing data) the first goal was to get rid of
code duplication without impacting the current behavior, with the added
benefit that it may allow further code cleanups / simplification in the
future.
2023-07-03 16:32:01 +02:00
Aurelien DARRAGON
461fd2e296 BUG/MINOR: tcp_sample: bc_{dst,src} return IP not INT
bc_{dst,src} sample fetches were introduced in 7d081f0 ("MINOR:
tcp_samples: Add samples to get src/dst info of the backend connection")

However a copy-pasting error was probably made here because they both
return IP addresses (not integers) like their fc_{src,dst} or src,dst
analogs.

Hopefully, this had no functional impact because integer can be converted
to both IPV4 and IPV6 types so the compatibility checks were not affected.

Better fix this to prevent confusion in code and "haproxy -dKsmp" output.

This should be backported up to 2.4 with 7d081f0.
2023-07-03 16:32:01 +02:00
Frdric Lcaille
f4d9fa4089 BUILD: quic: Compilation fixes for some gcc warnings with -O1
This is to prevent the build from these gcc warning when compiling with -O1 option:

  willy@wtap:haproxy$ sh make-native-quic-memstat src/quic_conn.o CPU_CFLAGS="-O1"
    CC      src/quic_conn.o
  src/quic_conn.c: In function 'qc_prep_pkts':
  src/quic_conn.c:3700:44: warning: potential null pointer dereference [-Wnull-dereference]
   3700 |                                 frms = &qel->pktns->tx.frms;
        |                                         ~~~^~~~~~~
  src/quic_conn.c:3626:33: warning: 'end' may be used uninitialized in this function [-Wmaybe-uninitialized]
   3626 |                         if (end - pos < QUIC_INITIAL_PACKET_MINLEN) {
        |                             ~~~~^~~~~
2023-07-03 14:50:54 +02:00
Frédéric Lécaille
7405874555 BUG/MINOR: quic: Missing QUIC connection path member initialization
This bug was introduced by this commit:
  MINOR: quic: Remove pool_zalloc() from qc_new_conn().

If ->path is not initialized to NULL value, and if a QUIC connection object
allocation has failed (from qc_new_conn()), haproxy could crash in
quic_conn_prx_cntrs_update() when dereferencing this QUIC connection member.

No backport needed.
2023-07-03 10:51:12 +02:00
Frédéric Lécaille
0e53cb07a5 BUG/MINOR: quic: Possible leak when allocating an encryption level
This bug was reported by GH #2200 (coverity issue) as follows:

*** CID 1516590:  Resource leaks  (RESOURCE_LEAK)
/src/quic_tls.c: 159 in quic_conn_enc_level_init()
153
154             LIST_APPEND(&qc->qel_list, &qel->list);
155             *el = qel;
156             ret = 1;
157      leave:
158             TRACE_LEAVE(QUIC_EV_CONN_CLOSE, qc);
>>>     CID 1516590:  Resource leaks  (RESOURCE_LEAK)
>>>     Variable "qel" going out of scope leaks the storage it points to.
159             return ret;
160     }
161
162     /* Uninitialize <qel> QUIC encryption level. Never fails. */
163     void quic_conn_enc_level_uninit(struct quic_conn *qc, struct quic_enc_level *qel)
164     {

This bug was introduced by this commit which has foolishly assumed the encryption
level memory would be released after quic_conn_enc_level_init() has failed. This
is no more possible because this object is dynamic and no more a static member
of the QUIC connection object.

Anyway, this patch modifies quic_conn_enc_level_init() to ensure this is
no more leak when quic_conn_enc_level_init() fails calling quic_conn_enc_level_uninit()
in case of memory allocation error.

quic_conn_enc_level_uninit() code was moved without modification only to be defined
before quic_conn_enc_level_init()

There is no need to backport this.
2023-07-03 10:50:08 +02:00
Willy Tarreau
1f60133bfb BUILD: debug: avoid a build warning related to epoll_wait() in debug code
Ilya reported in issue #2193 that the latest Fedora complains about us
passing NULL to epoll_wait() in the "debug dev fd" code to try to detect
an epoll FD. That was intentional to get the kernel's verifications and
make sure we're facing a poller, but since such a warning comes from the
libc, it's possible that it plans to replace the syscall with a wrapper
in the near future (e.g. epoll_pwait()), and that just hiding the NULL
(as was confirmed to work) might just postpone the problem.

Let's take another approach, instead we'll create a new dummy FD that
we'll try to remove from the epoll set using epoll_ctl(). Since we
created the FD we're certain it cannot be there.  In this case (and
only in this case) epoll_ctl() will return -ENOENT, otherwise it will
typically return EINVAL or EBADF. It was verified that it works and
doesn't return false positives for other FD types. It should be
backported to the branches that contain a backport of the commit which
introduced the feature, apparently as far as 2.4:

  5be7c198e ("DEBUG: cli: add a new "debug dev fd" expert command")
2023-07-02 11:01:37 +02:00
Frdric Lcaille
36e6c8aa4b BUILD: quic: Add a DISGUISE() to please some compiler to qc_prep_hpkts() 1st parameter
Some compiler could complain with such a warning:
  src/quic_conn.c:3700:44: warning: potential null pointer dereference [-Wnull-dereference]
   3700 |                                 frms = &qel->pktns->tx.frms;

It could not figure out that <qel> could not be NULL at this location.
This is fixed calling qc_prep_hpkts() with a disguise 1st parameter.
2023-06-30 18:48:52 +02:00
Frédéric Lécaille
7f3c1bef37 MINOR: quic: Drop packet with type for discarded packet number space.
This patch allows the low level packet parser to drop packets with type for discarded
packet number spaces. Furthermore, this prevents it from reallocating new encryption
levels and packet number spaces already released/discarded. When a packet number space
is discarded, it MUST NOT be reallocated.

As the packet number space discarding is done asap the type of packet received is
known, some packet number space discarding check may be safely removed from qc_try_rm_hp()
and qc_qel_may_rm_hp() which are called after having parse the packet header, and
is type.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
b97de9dc21 MINOR: quic: Move the packet number space status at quic_conn level
As the packet number spaces and encryption level are dynamically allocated,
the information about the packet number space discarded status must be kept
somewhere else than in these objects.

quic_tls_discard_keys() is no more useful.
Modify quic_pktns_discard() to do the same job: flag the quic_conn object
has having discarded packet number space.
Implement quic_tls_pktns_is_disarded() to check if a packet number space is
discarded. Note the Application data packet number space is never discarded.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
75ae0f7bbc CLEANUP: quic: Remove a useless test about discarded pktns (qc_handle_crypto_frm())
There is no need to check that the packet number space associated to the encryption
level to handle the CRYPTO frames is available when entering qc_handle_crypto_frm().
This has already been done by the caller: qc_treat_rx_pkts().
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
f1bfbf24cd MEDIUM: quic: Release encryption levels and packet number spaces asap
Release the memory allocated for the Initial and Handshake encryption level
under packet number spaces as soon as possible.

qc_treat_rx_pkts() has been modified to do that for the Initial case. This must
be done after all the Initial packets have been parsed and the underlying TLS
secrets have been flagged as to be discarded. As the Initial encryption level is
removed from the list attached to the quic_conn object inside a list_for_each_entry()
loop, this latter had to be converted into a list_for_each_entry_safe() loop.

The memory allocated by the Handshake encryption level and packet number spaces
is released just before leaving the handshake I/O callback (qc_conn_io_cb()).

As ->iel and ->hel pointer to Initial and Handshake encryption are reset to null
value when releasing the encryption levels memory, some check have been added
at several place before dereferencing them. Same thing for the ->ipktns and ->htpktns
pointer to packet number spaces.

Also take the opportunity of this patch to modify qc_dgrams_retransmit() to
use packet number space variables in place of encryption level variables to shorten
some statements. Furthermore this reflects the jobs it has to do: retransmit
UDP datagrams from packet number spaces.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
53caf351a9 CLEANUP: quic: Remove server specific about Initial packet number space
Remove a code section about the QUIC client Initial packet number space
dropping.

Should be backported as far as 2.6 to ease future backports to come.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
2d6a234af5 MINOR: quic: Remove call to qc_rm_hp_pkts() from I/O callback
quic_conn_io_cb() is the I/O handler callback used during the handshakes.
quic_conn_app_io_cb() is used after the handshakes. Both call qc_rm_hp_pkts()
before parsing the RX packets ordered by their packet numbers calling qc_treat_rx_pkts().
qc_rm_hp_pkts() is there to remove the header protection to reveal the packet
numbers, then reorder the packets by their packet numbers.

qc_rm_hp_pkts() may be safely directly called by qc_treat_rx_pkts(), which is
itself called by the I/O handlers.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
081f5002eb MEDIUM: quic: Handle the RX in one pass
Insert a loop around the existing code of qc_treat_rx_pkts() to handle all
the RX packets by encryption level for all the encryption levels allocated
for the connection, i.e. for all the encryption levels with secrets derived
from the ones provided by the TLS stack through ->set_encryption_secrets().
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
f7749968d6 CLEANUP: quic: Remove two useless pools a low QUIC connection level
Both "quic_tx_ring" and "quic_rx_crypto_frm" pool are no more used.

Should be backported as far as 2.6.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
3097be92f1 MEDIUM: quic: Dynamic allocations of QUIC TLS encryption levels
Replace ->els static array of encryption levels by 4 pointers into the QUIC
connection object, quic_conn struct.
    ->iel denotes the Initial encryption level,
    ->eel the Early-Data encryption level,
    ->hel the Handshaske encryption level and
    ->ael the Application Data encryption level.

Add ->qel_list to this structure to list the encryption levels after having been
allocated. Modify consequently the encryption level object itself (quic_enc_level
struct) so that it might be added to ->qel_list QUIC connection list of
encryption levels.

Implement qc_enc_level_alloc() to initialize the value of a pointer to an encryption
level object. It is used to initialized the pointer newly added to the quic_conn
structure. It also takes a packet number space pointer address as argument to
initialize it if not already initialized.

Modify quic_tls_ctx_reset() to call it from quic_conn_enc_level_init() which is
called by qc_enc_level_alloc() to allocate an encryption level object.

Implement 2 new helper functions:
  - ssl_to_qel_addr() to match and pointer address to a quic_encryption level
    attached to a quic_conn object with a TLS encryption level enum value;
  - qc_quic_enc_level() to match a pointer to a quic_encryption level attached
    to a quic_conn object with an internal encryption level enum value.
This functions are useful to be called from ->set_encryption_secrets() and
->add_handshake_data() TLS stack called which takes a TLS encryption enum
as argument (enum ssl_encryption_level_t).

Replace all the use of the qc->els[] array element values by one of the newly
added ->[ieha]el quic_conn struct member values.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
25a7b15144 MINOR: quic: Add a pool for the QUIC TLS encryption levels
Very simple patch to define and declare a pool for the QUIC TLS encryptions levels.
It will be used to dynamically allocate such objects to be attached to the
QUIC connection object (quic_conn struct) and to remove from quic_conn struct the
static array of encryption levels (see ->els[]).
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
6635aa6a0a MEDIUM: quic: Dynamic allocations of packet number spaces
Add a pool to dynamically handle the memory used for the QUIC TLS packet number spaces.
Remove the static array of packet number spaces at QUIC connection level (struct
quic_conn) and add three new members to quic_conn struc as pointers to quic_pktns
struct, one by packet number space as follows:
     ->ipktns for Initial packet number space,
     ->hpktns for Handshake packet number space and
     ->apktns for Application packet number space.
Also add a ->pktns_list new member (struct list) to quic_conn struct to attach
the list of the packet number spaces allocated for the QUIC connection.
Implement ssl_to_quic_pktns() to map and retrieve the addresses of these pointers
from TLS stack encryption levels.
Modify quic_pktns_init() to initialize these members.
Modify ha_quic_set_encryption_secrets() and ha_quic_add_handshake_data()  to
allocate the packet numbers and initialize the encryption level.
Implement quic_pktns_release() which takes pointers to pointers to packet number
space objects to release the memory allocated for a packet number space attached
to a QUIC connection and reset their address values.

Modify qc_new_conn() to allocation only the Initial packet number space and
Initial encryption level.

Modify QUIC loss detection API (quic_loss.c) to use the new ->pktns_list
list attached to a QUIC connection in place of a static array of packet number
spaces.

Replace at several locations the use of elements of an array of packet number
spaces by one of the three pointers to packet number spaces
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
411b6f73b7 MINOR: quic: Implement a packet number space identification function
Implement quic_pktns_char() to identify a packet number space from a
quic_conn object. Usefull only for traces.
2023-06-30 16:20:55 +02:00
Frédéric Lécaille
6593ec6f5e MINOR: quic: Move QUIC TLS encryption level related code (quic_conn_enc_level_init())
quic_conn_enc_level_init() location is definitively in QUIC TLS API source file:
src/quic_tls.c.
2023-06-30 16:20:55 +02:00
Willy Tarreau
56f15298d9 MINOR: compression/slz: add support for a pure flush of pending bytes
While HTTP makes no promises of sub-message delivery, haproxy tries to
make reasonable efforts to be friendly to applications relying on this,
particularly though the "option http-no-delay" statement. However, it
was reported that when slz compression is being used, a few bytes can
remain pending for more data to complete them in the SLZ queue when
built on a 64-bit little endian architecture. This is because aligning
blocks on byte boundary is costly (requires to switch to literals and
to send 4 bytes of block size), so incomplete bytes are left pending
there until they reach at least 32 bits. On other architecture, the
queue is only 8 bits long.

Robert Newson from Apache's CouchDB project explained that the heartbeat
used by CouchDB periodically delivers a single LF character, that it used
to work fine before the change enlarging the queue for 64-bit platforms,
but only forwards once every 3 LF after the change. This was definitely
caused by this incomplete byte sequence queuing. Zlib is not affected,
and the code shows that ->flush() is always called. In the case of SLZ,
the called function is rfc195x_flush_or_finish() and when its "finish"
argument is zero, no flush is performed because there was no such flush()
operation.

The previous patch implemented a flush() operation in SLZ, and this one
makes rfc195x_flush_or_finish() call it when finish==0. This way each
delivered data block will now provoke a flush of the queue as is done
with zlib.

This may very slightly degrade the compression ratio, but another change
is needed to condition this on "option http-no-delay" only in order to
be consistent with other parts of the code.

This patch (and the preceeding slz one) should be backported at least to
2.6, but any further change to depend on http-no-delay should not.
2023-06-30 16:12:36 +02:00
Willy Tarreau
90d18e2006 IMPORT: slz: implement a synchronous flush() operation
In some cases it may be desirable for latency reasons to forcefully
flush the queue even if it results in suboptimal compression. In our
case the queue might contain up to almost 4 bytes, which need an EOB
and a switch to literal mode, followed by 4 bytes to encode an empty
message. This means that each call can add 5 extra bytes in the ouput
stream. And the flush may also result in the header being produced for
the first time, which can amount to 2 or 10 bytes (zlib or gzip). In
the worst case, a total of 19 bytes may be emitted at once upon a flush
with 31 pending bits and a gzip header.

This is libslz upstream commit cf8c4668e4b4216e930b56338847d8d46a6bfda9.
2023-06-30 16:12:36 +02:00
Frédéric Lécaille
17eaee31c3 BUG/MINOR: quic: Wrong endianess for version field in Retry token
This field must be sent in network byte order.

Must be backported as far as 2.6.
2023-06-30 14:57:30 +02:00
Frédéric Lécaille
5997d18c78 BUG/MINOR: quic: Wrong Retry paquet version field endianess
The 32-bits version field of the Retry paquet was inversed by the code. As this
field must be in the network byte order on the wire, this code has supposed that
the sender of the Retry packet will always be little endian. Hopefully this is
often the case on our Intel machines ;)

Must be backported as far as 2.6.
2023-06-30 14:41:31 +02:00
Frédéric Lécaille
6c9bf2bdf5 BUG/MINOR: quic: Missing random bits in Retry packet header
The 4 bits least significant bits of the first byte in a Retry packet must be
random. There are generated calling statistical_prng_range() with 16 as argument.

Must be backported as far as 2.6.
2023-06-30 12:17:36 +02:00
Patrick Hemmer
bce0ca696c BUG/MINOR: config: fix stick table duplicate name check
When a stick-table is defined within a peers section, the name is
prefixed with the peers section name. However when checking for
duplicate table names, the check was using the table name without
the prefix, and would thus never match.

Must be backported as far as 2.6.
2023-06-30 10:27:16 +02:00
William Lallemand
593c895eed MINOR: ssl: allow to change the client-sigalgs on server lines
This patch introduces the "client-sigalgs" keyword for the server line,
which allows to configure the list of server signature algorithms
negociated during the handshake. Also available as
"ssl-default-server-client-sigalgs" in the global section.
2023-06-29 14:11:46 +02:00
William Lallemand
717f0ad995 MINOR: ssl: allow to change the server signature algorithm on server lines
This patch introduces the "sigalgs" keyword for the server line, which
allows to configure the list of server signature algorithms negociated
during the handshake. Also available as "ssl-default-server-sigalgs" in
the global section.
2023-06-29 13:40:18 +02:00
Emeric Brun
f473eb7206 BUG/MEDIUM: quic: error checking buffer large enought to receive the retry tag
Building a retry message, the offset of the tag was checked instead of the
remaining length into the buffer.

Must be backported as far as 2.6.
2023-06-27 18:54:10 +02:00
Willy Tarreau
e12e202f6a BUILD: mux-h1: silence a harmless fallthrough warning
This warning happened in 2.9-dev with commit 723c73f8a ("MEDIUM: mux-h1:
Split h1_process_mux() to make code more readable"). It's the usual gcc
crap that relies on comments to disable the warning but which drops these
comments between the preprocessor and the compiler, so using any split
build system (distcc, ccache etc) reintroduces the warning. Use the more
reliable and portable __fallthrough instead. No backport needed.
2023-06-27 16:08:13 +02:00
William Lallemand
3388b23465 BUG/MINOR: ssl: SSL_ERROR_ZERO_RETURN returns CO_ER_SSL_EMPTY
Return a more acurate error than the previous patch, CO_ER_SSL_EMPTY is
the code for "Connection closed during SSL handshake" which is more
precise than CO_ER_SSL_ABORT ("Connection error during SSL handshake").

No backport needed.
2023-06-26 19:10:24 +02:00
William Lallemand
e8e5762389 MEDIUM: ssl: handle the SSL_ERROR_ZERO_RETURN during the handshake
During a SSL_do_handshake(), SSL_ERROR_ZERO_RETURN can be returned in case
the remote peer sent a close_notify alert. Previously this would set the
connection error to CO_ER_SSL_HANDSHAKE, this patch sets it to
CO_ER_SSL_ABORT to have a more acurate error.
2023-06-26 18:52:53 +02:00
Frédéric Lécaille
1231810963 BUG/MINOR: quic: Prevent deadlock with CID tree lock
This bug was introduced by this commit which was not sufficient:
      BUG/MINOR: quic: Possible endless loop in quic_lstnr_dghdlr()

It was revealed by the blackhole interop runner test with neqo as client.

qc_conn_release() could be called after having locke the CID tree when two different
threads was creating the same connection at the same time. Indeed in this case
the last thread which tried to create a new connection for the same an already existing
CID could not manage to insert an already inserted CID in the connection CID tree.
This was expected. It had to destroy the newly created for nothing connection calling
qc_conn_release(). But this function also locks this tree calling free_quic_conn_cids() leading to a deadlock.
A solution would have been to delete the new CID created from its tree before
calling qc_conn_release().

A better solution is to stop inserting the first CID from qc_new_conn(), and to
insert it into the CID tree only if there was not an already created connection.
This is whas is implemented by this patch.

Must be backported as far as 2.7.
2023-06-26 14:09:58 +02:00
William Lallemand
117b03ff4a BUG/MINOR: mworker: leak of a socketpair during startup failure
Aurelien Darragon found a case of leak when working on ticket #2184.

When a reexec_on_failure() happens *BEFORE* protocol_bind_all(), the
worker is not fork and the mworker_proc struct is still there with
its 2 socketpairs.

The socketpair that is supposed to be in the master is already closed in
mworker_cleanup_proc(), the one for the worker was suppposed to
be cleaned up in mworker_cleanlisteners().

However, since the fd is not bound during this failure, the fd is never
closed.

This patch fixes the problem by setting the fd to -1 in the mworker_proc
after the fork, so we ensure that this it won't be close if everything
was done right, and then we try to close it in mworker_cleanup_proc()
when it's not set to -1.

This could be triggered with the script in ticket #2184 and a `ulimit -H
-n 300`. This will fail before the protocol_bind_all() when trying to
increase the nofile setrlimit.

In recent version of haproxy, there is a BUG_ON() in fd_insert() that
could be triggered by this bug because of the global.maxsock check.

Must be backported as far as 2.6.

The problem could exist in previous version but the code is different
and this won't be triggered easily without other consequences in the
master.
2023-06-21 09:44:18 +02:00
Aurelien DARRAGON
d35cee972b BUG/MINOR: http_ext: fix if-none regression in forwardfor option
A regression was introduced in 730b983 ("MINOR: proxy: move 'forwardfor'
option to http_ext")

Indeed, when the forwardfor if-none option is specified on the frontend
but forwardfor is not specified at all on the backend: if-none from the
frontend is ignored.

But this behavior conflicts with the historical one, if-none should only
be ignored if forwardfor is also enabled on the backend and if-none is
not set there.

It should fix GH #2187.

This should be backported in 2.8 with 730b983 ("MINOR: proxy: move
'forwardfor' option to http_ext")
2023-06-20 15:32:56 +02:00
Christopher Faulet
a150cfcfec CLEANUP: mux-h1: Remove useless __maybe_unused statement
h1_append_chunk_size() and h1_prepend_chunk_crlf() functions were marked as
possibly unused to be able to add them in standalone commits. Now these
functions are used, the __maybe_unused statement can be removed.
2023-06-20 13:59:24 +02:00
Christopher Faulet
c6ca6db034 MEDIIM: mux-h1: Add splicing support for chunked messages
When the HTX was introduced, we have lost the support for the kernel
splicing for chunked messages. Thanks to this patch set, it is possible
again. Of course, we still need to keep the H1 parser synchronized. Thus
only the chunk content can be spliced. We still need to read the chunk
envelope using a buffer.

There is no reason to backport this feature. But, just in case, this patch
depends on following patches:

  * "MEDIUM: filters/htx: Don't rely on HTX extra field if payload is filtered"
  * "MINOR: mux-h1: Add function to prepend the chunk crlf to the output buffer"
  * "MINOR: mux-h1: Add function to append the chunk size to the output buffer"
  * "REORG: mux-h1: Rename functions to emit chunk size/crlf in the output buffer"
  * "MEDIUM: mux-h1: Split h1_process_mux() to make code more readable"
2023-06-20 13:34:49 +02:00
Christopher Faulet
8bd835b2d2 MEDIUM: filters/htx: Don't rely on HTX extra field if payload is filtered
If an HTTP data filter is registered on a channel, we must not rely on the
HTX extra field because the payload may be changed and we cannot predict if
this value will change or not. It is too errorprone to let filters deal with
this reponsibility. So we set it to 0 when payload filtering is performed,
but only if the payload length can be determined. It is important because
this field may be used when data are forwarded. In fact, it will be used by
the H1 multiplexer to be able to splice chunk-encoded payload.
2023-06-20 13:34:46 +02:00
Christopher Faulet
05fe76b540 MINOR: mux-h1: Add function to prepend the chunk crlf to the output buffer
h1_prepend_chunk_crlf() function does the opposite of
h1_append_chunk_crlf(). It emit the chunk size in front of the output
buffer.
2023-06-20 13:33:59 +02:00
Christopher Faulet
a07c85c5df MINOR: mux-h1: Add function to append the chunk size to the output buffer
h1_append_chunk_size() function does the opposite of
h1_prepend_chunk_size(). It emit the chunk size at the end of the output
buffer.
2023-06-20 13:33:53 +02:00
Christopher Faulet
e081efd448 REORG: mux-h1: Rename functions to emit chunk size/crlf in the output buffer
h1_emit_chunk_size() and h1_emit_chunk_crlf() functions were renamed,
respectively, h1_prepend_chunk_size() and h1_append_chunk_crlf().
2023-06-20 13:33:23 +02:00
Christopher Faulet
723c73f8a7 MEDIUM: mux-h1: Split h1_process_mux() to make code more readable
h1_process_mux() function was pretty huge a quite hard to debug. So, the
funcion is split in sub-functions. Each sub-function is responsible to a
part of the message (start-line, headers, payload, trailers...). We are
still relying on a HTTP parser to format the message to be sure to detect
errors.  Functionnaly speaking, there is no change. But the code is now more
readable.
2023-06-20 13:33:01 +02:00
Frédéric Lécaille
a55acf993a BUG/MINOR: quic: ticks comparison without ticks API use
Replace a "less than" comparison between two tick variable by a call to tick_is_lt()
in quic_loss_pktns(). This bug could lead to a wrong packet loss detection
when the loss time computed values could wrap. This is the case 20 seconds after
haproxy has started.

Must be backported as far as 2.6.
2023-06-19 19:05:45 +02:00
William Lallemand
e6051a04ef BUG/MEDIUM: mworker: increase maxsock with each new worker
In ticket #2184, HAProxy is crashing in a BUG_ON() after a lot of reload
when the previous processes did not exit.

Each worker has a socketpair which is a FD in the master, when reloading
this FD still exists until the process leaves. But the global.maxconn
value is not incremented for each of these FD. So when there is too much
workers and the number of FD reaches maxsock, the next FD inserted in
the poller will crash the process.

This patch fixes the issue by increasing the maxsock for each remaining
worker.

Must be backported in every maintained version.
2023-06-19 17:32:32 +02:00
Frédéric Lécaille
98b55d1260 BUG/MINOR: quic: Missing transport parameters initializations
This bug was introduced by this commit:

     MINOR: quic: Remove pool_zalloc() from qc_new_conn()

The transport parameters was not initialized. This leaded to a crash when
dumping the received ones from TRACE()s.

Also reset the lengths of the CIDs attached to a quic_conn struct to 0 value
to prevent them from being dumped from traces when not already initialized.

No backport needed.
2023-06-19 08:49:04 +02:00
Frédéric Lécaille
30254d5e75 MINOR: quic: Remove pool_zalloc() from quic_dgram_parse()
Replace a call to pool_zalloc() by a call to pool_malloc() into quic_dgram_parse
to allocate quic_rx_packet struct objects.
Initialize almost all the members of quic_rx_packet struct.
->saddr is initialized by quic_rx_pkt_retrieve_conn().
->pnl and ->pn are initialized by qc_do_rm_hp().
->dcid and ->scid are initialized by quic_rx_pkt_parse() which calls
quic_packet_read_long_header() for a long packet. For a short packet,
only ->dcid will be initialized.
2023-06-16 16:56:08 +02:00
Frédéric Lécaille
9f9bd5f84d MINOR: quic: Remove pool_zalloc() from qc_conn_alloc_ssl_ctx()
pool_zalloc() is replaced by pool_alloc() into qc_conn_alloc_ssl_ctx() to allocate
a ssl_sock_ctx struct. ssl_sock_ctx struct member are all initiliazed to null values
excepted ->ssl which is initialized by the next statement: a call to qc_ssl_sess_init().
2023-06-16 16:56:08 +02:00
Frédéric Lécaille
ddc616933c MINOR: quic: Remove pool_zalloc() from qc_new_conn()
qc_new_conn() is ued to initialize QUIC connections with quic_conn struct objects.
This function calls quic_conn_release() when it fails to initialize a connection.
quic_conn_release() is also called to release the memory allocated by a QUIC
connection.

Replace pool_zalloc() by pool_alloc() in this function and initialize
all quic_conn struct members which are referenced by quic_conn_release() to
prevent use of non initialized variables in this fonction.
The ebtrees, the lists attached to quic_conn struct must be initialized.
The tasks must be reset to their NULL default values to be safely destroyed
by task_destroy(). This is all the case for all the TLS cipher contexts
of the encryption levels (struct quic_enc_level) and those for the keyupdate.
The packet number spaces (struct quic_pktns) must also be initialized.
->prx_counters pointer must be initialized to prevent quic_conn_prx_cntrs_update()
from dereferencing this pointer.
->latest_rtt member of quic_loss struct must also be initialized. This is done
by quic_loss_init() called by quic_path_init().
2023-06-16 16:55:58 +02:00
Frédéric Lécaille
4ae29be18c BUG/MINOR: quic: Possible endless loop in quic_lstnr_dghdlr()
This may happen when the initilization of a new QUIC conn fails with qc_new_conn()
when receiving an Initial paquet. This is done after having allocated a CID with
new_quic_cid() called by quic_rx_pkt_retrieve_conn() which stays in the listener
connections tree without a QUIC connection attached to. Then when the listener
receives another Initial packet for the same CID, quic_rx_pkt_retrieve_conn()
returns NULL again (no QUIC connection) but with an thread ID already bound to the
connection, leading the datagram to be requeued in the same datagram handler thread
queue. And so on.

To fix this, the connection is created after having created the connection ID.
If this fails, the connection is deallocated.

During the race condition, when two different threads handle two datagrams for
the same connection, in addition to releasing the newer created connection ID,
the newer QUIC connection must also be released.

Must be backported as far as 2.7.
2023-06-16 16:10:58 +02:00
Frédéric Lécaille
c02d898cd1 BUG/MINOR: quic: Possible crash in quic_conn_prx_cntrs_update()
quic_conn_prx_cntrs_update() may be called from quic_conn_release() with
NULL as value for ->prx_counters member. This is the case when qc_new_conn() fails
when allocating <buf_area>. In this case quic_conn_prx_cntrs_update() BUG_ON().

Must be backported as far as 2.7.
2023-06-14 18:09:54 +02:00
Aurelien DARRAGON
4e17a3f5b1 BUG/MINOR: namespace: missing free in netns_sig_stop()
On soft-stop, netns_sig_stop() function is called to purge the shared
namespace tree by iterating over each entries to free them.

However, once an entry is cleaned up and removed from the tree, the entry
itself isn't freed and this results into a minor leak when soft-stopping
because entry was allocated using calloc() in netns_store_insert() when
parsing the configuration.

This could be backported in every stable versions.
2023-06-14 11:27:29 +02:00
Aurelien DARRAGON
19b5a7c7a5 BUG/MINOR: server: inherit from netns in srv_settings_cpy()
When support for 'namespace' keyword was added for the 'default-server'
directive in 22f41a2 ("MINOR: server: Make 'default-server' support
'namespace' keyword."), we forgot to copy the attribute from the parent
to the newly created server.

This resulted in the 'namespace' keyword being parsed without errors when
used from a 'default-server' directive, but in practise the option was
simply ignored.

There's no need to duplicate the netns struct because it is stored in
a shared list, so copying the pointer does the job.

This patch partially fixes GH #2038 and should be backported to all
stable versions.
2023-06-14 11:27:29 +02:00
Frédéric Lécaille
4d56b725fb BUG/MINOR: quic: Address inversion in "show quic full"
The local address was dumped as "from" address by dump_quic_full() and
the peer address as "to" address. This patch fixes this issue.

Furthermore, to support the server side (QUIC client) to come, it is preferable
to stop using "from" and "to" labels to dump the local and peer addresses which
is confusing for a QUIC client which uses its local address as "from" address.

To mimic netstat, this is "Local Address" and "Foreign Address" which will
be displayed by "show quic" CLI command and "local_addr" and "foreign_addr"
for "show quic full" command to mention the local addresses and the peer
addresses.

Must be backported as far as 2.7.
2023-06-14 09:33:28 +02:00
Frédéric Lécaille
9b1f91fde8 BUG/MINOR: quic: Wrong encryption level flags checking
This bug arrived with this commit which was supposed to fix another one:

     BUG/MINOR: quic: Wrong Application encryption level selection when probing

The aim of this patch was to prevent the Application encryption to be selected
when probing leading to ACK only packets to be sent if the ack delay timer
had fired in the meantime, leading to crashes when no 01-RTT had been sent
because the ack range tree is empty in this case.

This statement is not correct (qc->pktns->flags & QUIC_FL_PKTNS_PROBE_NEEDED)
because qc->pktns is an array of packet number space. But it is equivalent
to (qc->pktns[QUIC_TLS_PKTNS_INITIAL].flags & QUIC_FL_PKTNS_PROBE_NEEDED).

That said, the patch mentionned above is not more useful since this following
which disable the ack time during the handshakes:

    BUG/MINOR: quic: Do not use ack delay during the handshakes

This commit revert the first patch mentionned above.

Must be backported as far as 2.6.
2023-06-14 08:54:51 +02:00
William Lallemand
0c9ff0cde4 BUG/MINOR: ssl: log message non thread safe in SSL Hanshake failure
It was reported in issue #2181, strange behavior during the new SSL
hanshake failure logs.

Errors were logged with the code 0, which is unknown to OpenSSL.

This patch mades 2 changes:

- It stops using ERR_error_string() when the SSL error code is 0
- It uses ERR_error_string_n() to be thread-safe

Must be backported to 2.8.
2023-06-12 16:35:57 +02:00
Christopher Faulet
28d17e26b8 BUG/MEDIUM: hlua: Use front SC to detect EOI in HTTP applets' receive functions
When an HTTP applet tries to get request data, we must take care to properly
detect the end of the message. It an empty HTX message with the SC_FL_EOI
flag set on the front SC. However, an issue was introduced during the SC
refactoring performed in the 2.8. The backend SC is tested instead of the
frontend one.

Because of this bug, the receive functions hang because the test on
SC_FL_EOI flag never succeeds. Of course, by checking the frontend SC (the
opposite SC to the one attached to the appctx), it works.

This patch should fix the issue #2180. It must be backported to the 2.8.
2023-06-12 09:16:29 +02:00
Aurelien DARRAGON
b7f8af3ca9 BUG/MINOR: proxy/server: free default-server on deinit
proxy default-server is a specific type of server that is not allocated
using new_server(): it is directly stored within the parent proxy
structure. However, since it may contain some default config options that
may be inherited by regular servers, it is also subject to dynamic members
(strings, structures..) that needs to be deallocated when the parent proxy
is cleaned up.

Unfortunately, srv_drop() may not be used directly from p->defsrv since
this function is meant to be used on regular servers only (those created
using new_server()).

To circumvent this, we're splitting srv_drop() to make a new function
called srv_free_params() that takes care of the member cleaning which
originally takes place in srv_drop(). This function is exposed through
server.h, so it may be called from outside server.c.

Thanks to this, calling srv_free_params(&p->defsrv) from free_proxy()
prevents any memory leaks due to dynamic parameters allocated when
parsing a default-server line from a proxy section.

This partially fixes GH #2173 and may be backported to 2.8.

[While it could also be relevant for other stable versions, the patch
won't apply due to architectural changes / name changes between 2.4 => 2.6
and then 2.6 => 2.8. Considering this is a minor fix that only makes
memory analyzers happy during deinit paths (at least for <= 2.8), it might
not be worth the trouble to backport them any further?]
2023-06-06 15:15:17 +02:00
Aurelien DARRAGON
9be9225ef2 BUG/MINOR: proxy: add missing interface bind free in free_proxy
bind->settings.interface hint is allocated when "interface" keyword
is specified on a bind line, but the string isn't explicitly freed in
proxy_free, resulting in minor memory leak on deinit paths when the
keyword is being used.

It partially fixes GH #2173 and may be backported to all stable versions.

[in 2.2 free_proxy did not exist so the patch must be applied directly
in deinit() function from haproxy.c]
2023-06-06 15:15:17 +02:00
Aurelien DARRAGON
c49224a29d BUG/MINOR: cfgparse-tcp: leak when re-declaring interface from bind line
When interface keyword is used multiple times within the same bind line,
the previous value isn't checked and is rewritten as-is, resulting in a
small memory leak.

Ensuring the interface name is first freed before assigning it to a new
value.

This may be backported to every stable versions.

[Note for 2.2, the fix must be performed in bind_parse_interface() from
proto_tcp.c, directly within the listener's loop, also ha_free() was
not available so free() must be used instead]
2023-06-06 15:15:17 +02:00
Christopher Faulet
2c29d1f524 BUG/MINOR: peers: Improve detection of config errors in peers sections
There are several misuses in peers sections that are not detected during the
configuration parsing and that could lead to undefined behaviors or crashes.

First, only one listener is expected for a peers section. If several bind
lines or local peer definitions are used, an error is triggered. However, if
multiple addresses are set on the same bind line, there is no error while
only the last listener is properly configured. On the 2.8, there is no crash
but side effects are hardly predictable. On older version, HAProxy crashes
if an unconfigured listener is used.

Then, there is no check on remote peers name. It is unexpected to have same
name for several remote peers. There is now a test, performed during the
post-parsing, to verify all remote peer names are unique.

Finally, server parsing options for the peers sections are changed to be
sure a port is always defined, and not a port range or a port offset.

This patch fixes the issue #2066. It could be backported to all stable
versions.
2023-06-05 08:24:34 +02:00
Christopher Faulet
bc9fb64623 BUG/MINOR: spoe: Only skip sending new frame after a receive attempt
When a SPOE appctx is processing frames in sync mode, we must only skip
sending a new frame if it is still waiting for a ACK frame after a receive
attempt. It was performed before the receive attempt. As a consequence, if
the ACK frame was received, the SPOE appctx did not try to process queued
messages immediately. This could increase the queue time and thus slow down
the processing time of the stream.

Thanks to Daniel Epperson for his help to diagnose the bug.

This patch must be backported to every stable versions.
2023-06-05 08:24:34 +02:00
Frédéric Lécaille
29a1d3679b BUG/MINOR: quic: Possible crash when SSL session init fails
This is due to the fact that qc->conn is never initialized before calling
qc_ssl_sess_init().

Must be backported as far as 2.6.
2023-06-02 18:12:48 +02:00
Willy Tarreau
4ad1c9635a BUG/MINOR: stream: do not use client-fin/server-fin with HTX
Historically the client-fin and server-fin timeouts were made to allow
a connection closure to be effective quickly if the last data were sent
down a socket and the client didn't close, something that can happen
when the peer's FIN is lost and retransmits are blocked by a firewall
for example. This made complete sense in 1.5 for TCP and HTTP in close
mode. But nowadays with muxes, it's not done at the right layer anymore
and even the description doesn't match what is being done, because what
happens is that the stream will abort the whole transfer after it's done
sending to the mux and this timeout expires.

We've seen in GH issue 2095 that this can happen with very short timeout
values, and while this didn't trigger often before, now that the muxes
(h2 & quic) properly report an end of stream before even the first
sc_conn_sync_recv(), it seems that it can happen more often, and have
two undesirable effects:
  - logging a timeout when that's not the case
  - aborting the request channel, hence the server-side conn, possibly
    before it had a chance to be put back to the idle list, causing
    this connection to be closed and not reusable.

Unfortunately for TCP (mux_pt) this remains necessary because the mux
doesn't have a timeout task. So here we're adding tests to only do
this through an HTX mux. But to be really clean we should in fact
completely drop all of this and implement these timeouts in the mux
itself.

This needs to be backported to 2.8 where the issue was discovered,
and maybe carefully to older versions, though that is not sure at
all. In any case, using a higher timeout or removing client-fin in
HTTP proxies is sufficient to make the issue disappear.
2023-06-02 16:33:40 +02:00
Tim Duesterhus
33a4461fa9 BUG/MINOR: stats: Fix Lua's get_stats function
Lua's `get_stats` function stopped working in
4cfb0019e6, due to the addition a new field
ST_F_PROTO without a corresponding entry in `stat_fields`.

Fix the issue by adding the entry, like
a46b142e88 did previously for a different field.

This patch fixes GitHub Issue #2174, it should be backported to 2.8.
2023-06-02 08:29:25 +02:00
Willy Tarreau
6ccc8625b4 MINOR: quic/cli: clarify the "show quic" help message
Make it clear what is expected in the "<format>" field on the help line.
This should be backported to 2.7.
2023-05-31 16:15:24 +02:00
Frdric Lcaille
a73563bfa7 MINOR: quic: Add QUIC connection statistical counters values to "show quic"
Add the total number of sent packets for each QUIC connection dumped by
"show quic".  Also add the remaining counter values only if not null.

Must be backported to 2.7.
2023-05-31 15:56:19 +02:00
Willy Tarreau
f279a2f148 BUG/MINOR: mux-h2: refresh the idle_timer when the mux is empty
There's a rare case where on long fat pipes, we can see the keep-alive
timeout trigger before the end of the transfer of the last large object,
and the connection closed a bit quickly after the end of the transfer
because a GOAWAY is queued. The data are not destroyed, except that
the WINDOW_UPDATES from the client arriving late while the last data
are being drained by the socket buffers may at some point trigger a
reset, and some clients might choke a bit too early on these. Let's
make sure we only arm the idle_start timestamp once the output buffer
is empty. Of course it will still not cover for the data pending in the
socket buffers but it will at least let those in the buffer leave in
peace. More elaborate options can be used to protect the data in the
kernel buffers, such as the one described in GH issue #5.

It's very likely that this old issue was emphasized by the following
commit in 2.6:
  15a4733d5 ("BUG/MEDIUM: mux-h2: make use of http-request and keep-alive timeouts")

and the behavior probably changed again with this one in 2.8, which
was backported to 2.7 and scheduled for 2.6:
  d38d8c6cc ("BUG/MEDIUM: mux-h2: make sure control frames do not refresh the idle timeout")

As such this patch should be backported to 2.6 after some observation
period.
2023-05-31 10:45:30 +02:00
Amaury Denoyelle
d68f8b5a4a CLEANUP: mux-quic: rename internal functions
This patch is similar to the previous one but for QUIC mux functions
used inside the mux code itself or application layer. Replace all
occurences of qc_* prefix by qcc_* or qcs_*. This should help to better
differentiate code between quic_conn and MUX.

This should be backported up to 2.7.
2023-05-30 15:45:55 +02:00
Amaury Denoyelle
0f61e4f6d3 CLEANUP: mux-quic: rename functions for mux_ops
Rename all QUIC mux function exposed through mux_ops structure. Use the
prefix qmux_* or qmux_strm_*. The objective is to remove qc_* prefix
which should only be used in quic_conn layer.

This should be backported up to 2.7.
2023-05-30 15:44:53 +02:00
Willy Tarreau
8fc7073906 BUG/MEDIUM: threads: fix a tiny race in thread_isolate()
Aurlien found a tiny race in thread_isolate() that can allow a thread
that was running under isolation to continue running while another one
enters isolation. The reason is that the check for harmless is only
done before winning the CAS, but since the previously isolated thread
doesn't wait for !rdv_request in thread_release(), it can effectively
continue its activities while the next one believes it's isolated. A
proper solution consists in looping once again in thread_isolate() to
recheck (and wait) for all threads to be isolated once the CAS is won.

The issue was introduced in 2.7 by commit 598cf3f22 ("MAJOR: threads:
change thread_isolate to support inter-group synchronization") so the
fix needs to be backported there.
2023-05-27 13:53:46 +02:00
Amaury Denoyelle
bfddb42c05 BUG/MEDIUM: mux-quic: only set EOI on FIN
Recently stconn flags were reviewed for QUIC mux to be conform with
other HTTP muxes. However, a mistake was made when dealing with a proper
stream FIN with both EOI and EOS set. This was done as RESET_STREAM
received after a FIN are ignored by QUIC mux and thus there is no
difference between EOI or EOI+EOS. However, analyzers may interpret EOS
as an interrupted request which result in a 400 HTTP error code.

To fix this, only set EOI on proper stream FIN. EOS is set when input is
interrupted (RESET_STREAM before FIN) or a STOP_SENDING is received
which prevent transfer to complete. In this last case, EOS must be
manually set too if FIN has been received before STOP_SENDING to go
directly from ERR_PENDING to final ERROR state.

This must be backported up to 2.7.
2023-05-26 17:17:25 +02:00
Amaury Denoyelle
6d6ee0dc0b MINOR: quic: fix stats naming for flow control BLOCKED frames
There was a misnaming in stats counter for *_BLOCKED frames in regard to
QUIC rfc convention. This patch fixes it to prevent future ambiguity :

- STREAMS_BLOCKED -> STREAM_DATA_BLOCKED
- STREAMS_DATA_BLOCKED_BIDI -> STREAMS_BLOCKED_BIDI
- STREAMS_DATA_BLOCKED_UNI -> STREAMS_BLOCKED_UNI

This should be backported up to 2.7.
2023-05-26 17:17:00 +02:00
Amaury Denoyelle
087c5f041b MINOR: mux-quic: remove nb_streams from qcc
Remove nb_streams field from qcc. It was not used outside of a BUG_ON()
statement to ensure we never have a negative count of streams. However
this is already checked with other fields.

This should be backported up to 2.7.
2023-05-26 17:17:00 +02:00
Amaury Denoyelle
7b41dfd834 CLEANUP: mux-quic: remove unneeded fields in qcc
Remove fields from qcc structure which are unused.

This should be backported up to 2.7.
2023-05-26 17:17:00 +02:00
Aurelien DARRAGON
33bbeecde3 BUILD: init: print rlim_cur as regular integer
haproxy does not compile anymore on macOS+clang since 425d7ad ("MINOR:
init: pre-allocate kernel data structures on init"). This is due to
rlim_cur being printed uncasted using %lu format specifier, with rlim_cur
being stored as a rlim_t which is a typedef so its size may vary depending
on the system's architecture.

This is not the first time we need to dump rlim_cur in case of errors,
there are already multiple occurences in the init code. Everywhere this
happens, rlim is casted as a regular int and printed using the '%d'
format specifier, so we do the same here as well to fix the build issue.

No backport needed unless 425d7ad gets backported.
2023-05-26 14:29:52 +02:00
eaglegai
ef667b1ad8 BUG/MINOR: thread: add a check for pthread_create
preload_libgcc_s() use pthread_create to create a thread and then call
pthread_join to use it, but it doesn't check if the option is successful.
So add a check to aviod potential crash.
2023-05-26 12:08:23 +02:00