This patch introduces the haload target to the Makefile. It defines
the HALOAD_OBJS list, which includes the standard objects along with
the specific haload_init.o, haload.o, and hbuf.o files.
The build process follows the same pattern as the haterm tool,
allowing haload to be compiled as a standalone binary.
Export _srv_parse_kw() and srv_postinit() so they can be called from
haload (to come), which needs to configure servers using HAProxy's configuration
parser keywords.
This patch exports sc_new() by removing its static storage class and
adding its prototype to include/haproxy/stconn.h.
This is required to allow external modules, such as the upcoming haload
benchmarking tool, to allocate and initialize new stream connectors
from a stream endpoint descriptor (sedesc).
This patch introduces the sc_hastream() and __sc_hastream() inline
helpers to retrieve a haload stream context (struct hastream) from
a stream connector.
These functions allow the stconn layer to safely access haload-specific
stream data when the application type is OBJ_TYPE_HXLOAD.
This patch introduces the OBJ_TYPE_HXLOAD object type to represent
haload stream objects (struct hastream).
It also adds the associated inline helper functions objt_hastream()
and __objt_hastream() to allow safe casting and retrieval of
hastream contexts from a generic object pointer, following the
standard container_of pattern.
haload is a client-side HTTP benchmarking tool designed to manage
concurrent HTTP streams.
This patch defines the hldstream C structure, which serves as the
core object to represent a haload HTTP stream for all the HTTP protocol.
It will be used by the upcoming haload module to handle specialized
stream contexts.
haload is the successor to the h1load HTTP benchmarking tool.
This patch adds haload stream definitions as arguments for the TRACE API.
These will be used by the upcoming haload module, which will handle
hldstream struct objects instead of regular stream structs.
Introduce the new <client_mode> global variable to define the operating mode
of haproxy. This variable can be set to 1 to allow haproxy to start without
any listeners.
During the initialization cycle, setting this flag ensures that the lack of
configured listeners is no longer treated as a fatal error. This allows
programs based on haproxy source code to initialize the stack and use its
features even without a frontend.
Add a new lightweight hbuf API to buffer formatted strings, similar to the
existing buffer API (struct buffer). This is required by haterm to build
its configuration in memory (fileless mode).
Update haterm to use this new API.
Note: hstream_str_buf_append() has been renamed to hbuf_str_append().
Unlike the detection performed during sendto() for an unreachable peer,
ECONNREFUSED was not handled when received via recvmsg() as an ICMP
"host unreachable" message.
This patch tracks ECONNREFUSED errors on the receive path.
Note that this detection is entirely dependent on the remote host effectively
sending an ICMP "host unreachable" message and on the absence of any network
filtering (e.g., firewalls) that would drop such ICMP packets. Without
receiving this ICMP signal, the connection state cannot be updated through
this mechanism.
At a higher level, similar to how this error is handled on sendto(),
the connection is now terminated as soon as possible by calling
qc_kill_conn(). This triggers a call to qc_notify_err(). When the mux
does not exist, it attempts to create one via conn_create_mux(). While
the latter systematically fails if the connection is flagged with
CO_FL_ERROR, it has the useful side effect of waking the stconn stream
attached to the connection during a session opening without a mux
(e.g., for H3).
This issue was caught by haload (upcoming tool).
Must be backported as far as 2.6 because it impacts both the QUIC
frontends and backends.
QMux implements a record layer which is used to encapsulate QUIC frames.
This patch implements reception of an incomplete record in
qcc_qstrm_recv(). BUG_ON() failures are removed and now reading will
continue until the whole record is received or a fatal error occurs.
Several adjustments were made in the logic for read operation.
Previously, read syscall was only performed if either data buffer was
empty or current record was incomplete. An extra condition is added to
perform read if there is data in the buffer but not enough to decode a
record header. Another change is that buffer realign is also performed
in this latter case and if buffer wrapping position has been reached.
Remove BUG_ON() related to connection errors when invoking XPRT
snd_buf/rcv_buf in QMux operations. Such errors are now converted in
QC_CF_ERR_CONN flag, which will disable any I/O operations and close the
connection as soon as possible.
Note that this error management is pretty crude. In particular, it could
lead to truncated data when dealing with unidirectional connection
closure from the remote peer. However, it is considered sufficient for
now to continue interop testing without being disturbed by BUG_ON()
assertion crashes.
Support reception via QMux of flow control MAX-STREAMS frame for
bidirectional streams. This is similar to the QUIC with shared
qcc_recv_max_streams() function.
When xprt_qstrm layer is completed, MUX layer is started. Rx buffer from
the XPRT layer is transferred to the MUX so that it can handle any extra
data following the transport parameters first frame.
Since previous commit, QCC Rx buffer is dynamically allocated only when
needed. However, qmux_init() must still allocate it when there is data
to be transferred from the XPRT layer. As a result, code has been over
extended to continue to support this case.
This patch simplifies xprt_qstrm API for the Rx buffer transfer. Buffer
content and remaining record length can now be retrieved via the single
function xprt_qstrm_xfer_rxbuf(). If the buffer is empty, nothing is
performed and XPRT layer will release it. If not empty, MUX will take
ownership of the buffer from the XPRT layer.
Allocate and release as needed the QCC buffers used for QMux protocol.
This should reduce the memory consumption of QMux. This is performed
both for send and receive buffers. Along with this, always free these
buffers in qcc_release() to prevent a memory leak.
Improve QMux memory usage at the QCS level in accordance with the
haproxy model. The tx buffer is now allocated only when used and
released as soon as it is empty.
This change requires to extend qcc_get_stream_txbuf() for QMux. Code
part related to qc_stream_desc is protected via conn_is_quic(). A
dedicated QMux bloc is added. Similarly to QUIC, a small buf can be
allocated first.
This also requires to adapt qcc_realloc_stream_txbuf() in a similar
fashion.
Clean up qcc_qstrm_send_frames(). The main change is that now return
value is clearly specified at the end of the function, depending if
everything was sent or not.
Implement close callback for xprt_qstrm layer. This is called when a
connection is prematurely closed following a connect failure. Its
purpose is to clean up all xprt resources.
A special care is required for the frontend side. Indeed,
xprt_qstrm_io_cb() can call close callback via conn_create_mux() on the
latter failure. The tasklet should then immediately be stopped as the
whole xprt layer has been freed as well.
Function conn_create_mux() has different behavior for frontend and
backend connections. In particular, on FE side, there is a risk that the
connection is freed.
Write a comment to explain these differences clearly.
Recently, an extra check has been added so that a dead connection is
immediately release on at the end of qcc_recv() operation. This is
useful when a GOAWAY frame is received from a server, so that the
backend connection is released if idle.
This step is in fact only necessary for QUIC, as qcc_recv() is called
directly from the lower transport layer. It causes issues with QMux as
in this case qcc_recv() is called via qcc_io_recv(). A crash in this
context will occur as qcc_recv() does not indicate that a release has
been performed.
To fix this, simply disable the extra check at the end of qcc_recv() for
QMux. This is fine as in this case receive operation is always followed
by qcc_io_process() which is able to release the connection in a safe
way.
No need to backport.
When QMux XPRT has successfully been able to process to transport
parameters exchange, the MUX is initialized and immediately woken up to
start transfers. However, if the connection is in an unusable state, the
latter operation will instead release the connection and all of its
network stack.
A crash would occur in case of release when finalizing the XPRT tasklet
completion. To fix this, first free every XPRT resources. MUX wake is
now conducted in a safe way as the last operation before the tasklet is
completely released.
No need to backport.
Complete initialization of xprt_qstrm layer by setting local parameters
to zero. This should prevent to emit random values to the peer.
No backport needed.
qc_frm_free() is a helper used to clean up a QUIC frame object. It is
used by MUX layer both for QUIC and QMux protocols.
This function takes a pointer to the underlying quic_conn, used only for
trace purpose. This patch fixes its usage for QMux to ensure that in
this case a NULL value is used.
No need to backport.
When dealing with EOH block, we must be sure to force the close mode for
message with no payload but annoncing a non-null content-length.
It is mainly an issue on the server side but it could be encountered on
client side too. Without this fix, a request can be switched to the DONE
state while the server is still expecting the payload. In an ideal world,
this case should not happen. But in conjunction with other bugs, it may lead
to a desynchro between haproxy and the server.
Now, when a non-null content-length is announced but we know we reached the
end of the message, we force the close mode. The only exception is for
bodyless responses (204s, 304s and responses to head requests).
Thanks to Martino Spagnuolo (r3verii) for his detailed report on this issue.
This patch must be backported to all stable version.
Checks are already made on H2 to detect inconsistencies between
advertised content-length and transferred data (excess of data or
premature END_STREAM flag on DATA frame). However, as found by
Martino Spagnuolo (r3verii), a subtle case remains: if the END_STREAM
appears on the HEADERS frame (i.e. a regular request for example),
then the check is not made. In this case it is possible to advertise
more contents than will really be transferred. If the other side uses
HTTP/1.1, and the server responds before the end of the transfer,
this means that the number of advertised bytes that will never be
transferred and that the server will drain will be taken from the
next request, effectively hiding a part of the header.
In practice this can be used to force subsequent requests to fail, or
when running with "http-reuse never" or when running with a totally
idle server, to perform a request smuggling by constructing specially
crafted request pairs where the first one is used to trigger an early
response and hide parts of or all headers of the second one, to
instead use a second embedded one that was not subject to analysis.
The risk remains moderate given the low prevalence of "http-reuse never"
in production environments, and of idle servers.
The fix consists in detecting if advertised content-length remains when
processing an END_STREAM flag on a HEADERS frame. It also does it for
trailers, which turn out to be another way to abuse the bug. However it
takes great care not to break bodyless responses (204, 304 and responses
to HEAD requests) that may present a content-length that doesn't reflect
the presence of a body in the response.
A temporary alternative to the fix is to disable HTTP/2 by specifying
"alpn http/1.1" on "bind" lines, and adding "option disable-h2-upgrade"
in HTTP frontends.
This must be backported to all stable versions.
In 3.4-dev6, commit de5fc2f515 ("BUG/MINOR: server: set auto SNI for
dynamic servers") allowed to properly set the SNI, and return an error
message. However the error message is leaked after being printed on the
CLI.
This should be backported to 3.3.
In 3.4-dev7, commit e1738b665d ("MINOR: debug: read all libs in memory
when set-dumpable=libs") reads dependencies into memory to store them as
a tar archive for later debugging. There was an attempt to mark the whole
archive read-only, except that the size passed in argument to mprotect()
is wrong: lib_size is only assigned after the operation and is still zero
at the moment this is done. new_size ought to be used instead.
This needs to be backported wherever the commit above is backported, at
least 3.2.
3 new enum values and a mask were added in latest -dev with commit
24e05fe33a ("MINOR: stream: Use a pcli transaction to replace pcli_*
members"), unfortunately the entries needed by the "flags" command were
forgotten.
No backport is needed.
Since commit 0af603f46f ("MEDIUM: threads: change the default
max-threads-per-group value to 16"), it was written "Tha minimum" instead
of "The minimum". No backport needed, this is only in latest -dev.
In 3.4-dev8, commit e264523112 ("MINOR: servers: Don't update last_sess
if it did not change") adjusted the last_sess date to avoid writing to
the same cache line all the time, however a typo makes it pick the wrong
second because it uses now_ms instead of now_ns (so the date would roughly
change every 12 days).
No backport needed.
In 2.8, commit ead43fe4f2 ("MEDIUM: compression: Make it so we can
compress requests as well.") added the ability to independently enable
compression on request and/or response. However there's a bug in the
"compression direction response" case, which preserves only the request
flag and adds the response direction instead of clearing the request
flag, so this directive would clear offload and make it impossible to
disable request if it was already previously enabled.
This can be backported to stable releases as far as 2.8.
When a http request is sent during an http healthcheck, if an error is
triggered while the output buffer is a small buffer, another attempt is made
with a larger one. When this happens, the temporary chunk used to format
headers must be released.
No backport needed.
Ruleset are stored in a global tree, released on deinit staged. All errors
are fatal and abort the configuration parsing. So the current ruleset must
not be released here.
