On sendto() transient error, prior to this patch sending was simulated
and we relied on retransmission to retry sending. This could hurt
significantly the performance.
Thanks to quic-conn owned socket support, it is now possible to improve
this. On transient error, sending is interrupted and quic-conn socket FD
is subscribed on the poller for sending. When send is possible,
quic_conn_sock_fd_iocb() will be in charge of restart sending.
A consequence of this change is on the return value of qc_send_ppkts().
This function will now return 0 on transient error if quic-conn has its
owned socket. This is used to interrupt sending in the calling function.
The flag QUIC_FL_CONN_TO_KILL must be checked to differentiate a fatal
error from a transient one.
This should be backported up to 2.7.
This patch improves the 'hold' parameter description in the 'resolvers'
section to make it clearer. It really explains differences between all
status. Thanks to Nick Ramirez for this update.
This patch should solve the issue #1694. It could be backported to all
stable versions.
This patch adds support from HAPROXY_BRANCH environment variable.
It can be useful is some resources are loaded from different
locations when migrating from one version to another.
Signed-off-by: Sébastien Gross <sgross@haproxy.com>
HAPROXY_STARTUP_VERSION: contains the version used to start, in
master-worker mode this is the version which was used to start the
master, even after updating the binary and reloading.
This patch could be backported in every version since it is useful when
debugging.
Implement a way to test if some options are enabled at run-time. For now,
following options may be detected:
POLL, EPOLL, KQUEUE, EVPORTS, SPLICE, GETADDRINFO, REUSEPORT,
FAST-FORWARD, SERVER-SSL-VERIFY-NONE
These options are those that can be disabled on the command line. This way
it is possible, from a reg-test for instance, to know if a feature is
supported or not :
feature cmd "$HAPROXY_PROGRAM -cc '!(globa.tune & GTUNE_NO_FAST_FWD)'"
This global option is documented but it is not in the list of supported
options for the global section. So let's add it.
This patch could be backported to all stable versions.
The option was renamed to only permit to disable the fast-forward. First
there is no reason to enable it because it is the default behavior. Then it
introduced a bug because there is no way to be sure the command line has
precedence over the configuration this way. So, the option is now named
"tune.disable-fast-forward" and does not support any argument. And of
course, the commande line option "-dF" has now precedence over the
configuration.
No backport needed.
For server connections, both the frontend and backend were considered to
enable the httpclose option. However, it is ambiguous because on client side
only the frontend is considerd. In addition for 2 frontends, one with the
option enabled and not for the other, the HTTP connection mode may differ
while it is a backend setting.
Thus, now, for the server side, only the backend is considered. Of course,
if the option is set for a listener, the option will be enabled if the
listener is the backend's connection.
Since the HTX, the decription of options about HTTP connection modes is
wrong. In fact, it is worst, all the documentation about HTTP connection
mode is wrong. But only options will be updated for now to be backported.
So, documentation of "option httpclose", "option "http-keep-alive", "option
http-server-close" and "option "http-pretend-keepalive" was reviewed. First,
it is specify these options only concern HTT/1.x connections. Then, the
descriptions were updated to reflect the HTX implementation.
The main changes concerns the fact that server connections are no longer
attached to client connections. The connection mode on one side does not
affect the connection mode on the other side. It is especially true for
t"option httpclose". For client connections, only the frontend option is
considered and for server ones, both frontend and backend options are
considered.
This patch should be backported as far as 2.2.
To prevent data loss for QUIC connections, haproxy global variable jobs
is incremented each time a quic-conn socket is allocated. This allows
the QUIC connection to terminate all its transfer operation during proxy
soft-stop. Without this patch, the process will be terminated without
waiting for QUIC connections.
Note that this is done in qc_alloc_fd(). This means only QUIC connection
with their owned socket will properly support soft-stop. In the other
case, the connection will be interrupted abruptly as before. Similarly,
jobs decrement is conducted in qc_release_fd().
This should be backported up to 2.7.
The -dF option can now be used to disable data fast-forward. It does the
same than the global option "tune.fast-forward off". Some reg-tests may rely
on this optim. To detect the feature and skip such script, the following
vtest command must be used:
feature cmd "$HAPROXY_PROGRAM -cc '!(globa.tune & GTUNE_NO_FAST_FWD)'"
The new global option "tune.fast-forward" can be set to "off" to disable the
data fast-forward. It is an debug option, thus it is internally marked as
experimental. The directive "expose-experimental-directives" must be set
first to use this one. By default, the data fast-forward is enable.
It could be usefull to force to wake the stream up when data are
received. To be sure, evreything works fine in this case. The data
fast-forward is an optim. It must work without it. But some code may rely on
the fact the stream will not be woken up. With this option, it is possible
to spot some hidden bugs.
Released version 2.8-dev4 with the following main changes :
- BUG/MINOR: stats: fix source buffer size for http dump
- BUG/MEDIUM: stats: fix resolvers dump
- BUG/MINOR: stats: fix ctx->field update in stats_dump_proxy_to_buffer()
- BUG/MINOR: stats: fix show stats field ctx for servers
- BUG/MINOR: stats: fix STAT_STARTED behavior with full htx
- MINOR: quic: Update version_information transport parameter to draft-14
- BUG/MINOR: stats: Prevent HTTP "other sessions" counter underflows
- BUG/MEDIUM: thread: fix extraneous shift in the thread_set parser
- BUG/MEDIUM: listener/thread: bypass shards setting on failed thread resolution
- BUG/MINOR: ssl/crt-list: warn when a line is malformated
- BUG/MEDIUM: stick-table: do not leave entries in end of window during purge
- BUG/MINOR: clock: do not mix wall-clock and monotonic time in uptime calculation
- BUG/MEDIUM: cache: use the correct time reference when comparing dates
- MEDIUM: clock: force internal time to wrap early after boot
- BUILD: ssl/ocsp: ssl_ocsp-t.h depends on ssl_sock-t.h
- MINOR: ssl/ocsp: add a function to check the OCSP update configuration
- MINOR: cfgparse/server: move (min/max)conn postparsing logic into dedicated function
- BUG/MINOR: server/add: ensure minconn/maxconn consistency when adding server
- BUG/MEDIUM: stconn: Schedule a shutw on shutr if data must be sent first
- BUG/MEDIUM: quic: fix crash when "option nolinger" is set in the frontend
- MINOR: quic: implement a basic "show quic" CLI handler
- MINOR: quic: display CIDs and state in "show quic"
- MINOR: quic: display socket info on "show quic"
- MINOR: quic: display infos about various encryption level on "show quic"
- MINOR: quic: display Tx stream info on "show quic"
- MINOR: quic: filter closing conn on "show quic"
- BUG/MINOR: quic: fix filtering of closing connections on "show quic"
- BUG/MEDIUM: stconn: Don't needlessly wake the stream on send during fast-forward
- BUG/MINOR: quic: fix type bug on "show quic" for 32-bits arch
- BUG/MINOR: mworker: fix uptime for master process
- BUG/MINOR: clock/stats: also use start_time not start_date in HTML info
- BUG/MEDIUM: stconn: stop to enable/disable reads from streams via si_update_rx
- BUG/MEDIUM: quic: Buffer overflow when looking through QUIC CLI keyword list
- DOC: proxy-protocol: fix wrong byte in provided example
- MINOR: ssl-ckch: Stop to test CF_WRITE_ERROR to commit CA/CRL file
- MINOR: bwlim: Remove useless test on CF_READ_ERROR to detect the last packet
- BUG/MINOR: http-ana: Fix condition to set LAST termination flag
- BUG/MINOR: mux-h1: Don't report an H1C error on client timeout
- BUG/MEDIUM: spoe: Don't set the default traget for the SPOE agent frontend
- BUG/MINOR: quic: Wrong datagram dispatch because of qc_check_dcid()
- BUG/CRITICAL: http: properly reject empty http header field names
There was a mistake in the example of proxy-proto frame
provided, it cannot end with 0x02 but only 0x20 or 0x21
since the version is in the upper 4 bits and the lower ones
are 0 for LOCAL or 1 for PROXY, hence the example should be:
\x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A\x20
Thanks to Bram Grit for reporting this mistake.
Reduce default "show quic" output by masking connection on
closing/draing state due to a CONNECTION_CLOSE emission/reception. These
connections can still be displayed using the special argument "all".
This should be backported up to 2.7.
Implement a basic "show quic" CLI handler. This command will be useful
to display various information on all the active QUIC frontend
connections.
This work is heavily inspired by "show sess". Most notably, a global
list of quic_conn has been introduced to be able to loop over them. This
list is stored per thread in ha_thread_ctx.
Also add three CLI handlers for "show quic" in order to allocate and
free the command context. The dump handler runs on thread isolation.
Each quic_conn is referenced using a back-ref to handle deletion during
handler yielding.
For the moment, only a list of raw quic_conn pointers is displayed. The
handler will be completed over time with more information as needed.
This should be backported up to 2.7.
Released version 2.8-dev3 with the following main changes :
- BUG/MINOR: sink: make sure to always properly unmap a file-backed ring
- DEV: haring: add a new option "-r" to automatically repair broken files
- BUG/MINOR: ssl: Fix leaks in 'update ssl ocsp-response' CLI command
- MINOR: ssl: Remove debug fprintf in 'update ssl ocsp-response' cli command
- MINOR: connection: add a BUG_ON() to detect destroying connection in idle list
- MINOR: mux-quic/h3: send SETTINGS as soon as transport is ready
- BUG/MINOR: h3: fix GOAWAY emission
- BUG/MEDIUM: mux-quic: fix crash on H3 SETTINGS emission
- BUG/MEDIUM: hpack: fix incorrect huffman decoding of some control chars
- BUG/MINOR: log: release global log servers on exit
- BUG/MINOR: ring: release the backing store name on exit
- BUG/MINOR: sink: free the forwarding task on exit
- CLEANUP: trace: remove the QUIC-specific ifdefs
- MINOR: trace: add a TRACE_ENABLED() macro to determine if a trace is active
- MINOR: trace: add a trace_no_cb() dummy callback for when to use no callback
- MINOR: trace: add the long awaited TRACE_PRINTF()
- MINOR: h2: add h2_phdr_to_ist() to make ISTs from pseudo headers
- MEDIUM: mux-h2/trace: add tracing support for headers
- CLEANUP: mux-h2/trace: shorten the name of the header enc/dec functions
- DEV: hpack: fix `trash` build regression
- MINOR: http_htx: add http_append_header() to append value to header
- MINOR: http_htx: add http_prepend_header() to prepend value to header
- MINOR: sample: add ARGC_OPT
- MINOR: proxy: introduce http only options
- MINOR: proxy/http_ext: introduce proxy forwarded option
- REGTEST: add ifnone-forwardfor test
- MINOR: proxy: move 'forwardfor' option to http_ext
- MINOR: proxy: move 'originalto' option to http_ext
- MINOR: http_ext: introduce http ext converters
- MINOR: http_ext: add rfc7239_is_valid converter
- MINOR: http_ext: add rfc7239_field converter
- MINOR: http_ext: add rfc7239_n2nn converter
- MINOR: http_ext: add rfc7239_n2np converter
- REGTEST: add RFC7239 forwarded header tests
- OPTIM: http_ext/7239: introduce c_mode to save some space
- MINOR: http_ext/7239: warn the user when fetch is not available
- MEDIUM: proxy/http_ext: implement dynamic http_ext
- MINOR: cfgparse/http_ext: move post-parsing http_ext steps to http_ext
- DOC: config: fix option spop-check proxy compatibility
- BUG/MINOR: fcgi-app: prevent 'use-fcgi-app' in default section
- DOC: config: 'http-send-name-header' option may be used in default section
- BUG/MINOR: mux-h2: Fix possible null pointer deref on h2c in _h2_trace_header()
- BUG/MINOR: http_ext/7239: ipv6 dumping relies on out of scope variables
- BUG/MEDIUM: h3: do not crash if no buf space for trailers
- OPTIM: h3: skip buf realign if no trailer to encode
- MINOR: mux-quic/h3: define stream close callback
- BUG/MEDIUM: h3: handle STOP_SENDING on control stream
- BUG/MINOR: h3: reject RESET_STREAM received for control stream
- MINOR: h3: add missing traces on closure
- BUG/MEDIUM: ssl: wrong eviction from the session cache tree
- BUG/MINOR: h3: fix crash due to h3 traces
- BUG/MINOR: h3: fix crash due to h3 traces
- BUG/MEDIUM: thread: consider secondary threads as idle+harmless during boot
- BUG/MINOR: stats: use proper buffer size for http dump
- BUILD: makefile: fix PCRE overriding specific lib path
- MINOR: quic: remove fin from quic_stream frame type
- MINOR: quic: ensure offset is properly set for STREAM frames
- MINOR: quic: define new functions for frame alloc
- MINOR: quic: refactor frame deallocation
- MEDIUM: quic: implement a retransmit limit per frame
- MINOR: quic: add config for retransmit limit
- OPTIM: htx: inline the most common memcpy(8)
- CLEANUP: quic: no need for atomics on packet refcnt
- MINOR: stats: add by HTTP version cumulated number of sessions and requests
- BUG/MINOR: quic: Possible stream truncations under heavy loss
- BUG/MINOR: quic: Too big PTO during handshakes
- MINOR: quic: Add a trace about variable states in qc_prep_fast_retrans()
- BUG/MINOR: quic: Do not ignore coalesced packets in qc_prep_fast_retrans()
- MINOR: quic: When probing Handshake packet number space, also probe the Initial one
- BUG/MAJOR: quic: Possible crash when processing 1-RTT during 0-RTT session
- MEDIUM: quic: Remove qc_conn_finalize() from the ClientHello TLS callbacks
- BUG/MINOR: quic: Unchecked source connection ID
- MEDIUM: listener: move the analysers mask to the bind_conf
- MINOR: listener: move maxseg and tcp_ut to bind_conf
- MINOR: listener: move maxaccept from listener to bind_conf
- MINOR: listener: move the backlog setting from listener to bind_conf
- MINOR: listener: move the maxconn parameter to the bind_conf
- MINOR: listener: move the ->accept callback to the bind_conf
- MINOR: listener: remove the useless ->default_target field
- MINOR: listener: move the nice field to the bind_conf
- MINOR: listener: move the NOLINGER option to the bind_conf
- MINOR: listener: move the NOQUICKACK option to the bind_conf
- MINOR: listener: move the DEF_ACCEPT option to the bind_conf
- MINOR: listener: move TCP_FO to bind_conf
- MINOR: listener: move the ACC_PROXY and ACC_CIP options to bind_conf
- MINOR: listener: move LI_O_UNLIMITED and LI_O_NOSTOP to bind_conf
- MINOR: listener: get rid of LI_O_TCP_L4_RULES and LI_O_TCP_L5_RULES
- CLEANUP: listener: remove the now unused options field
- MINOR: listener: remove the now useless LI_F_QUIC_LISTENER flag
- CLEANUP: config: remove test for impossible case regarding bind thread mask
- MINOR: thread: add a simple thread_set API
- MEDIUM: listener/config: make the "thread" parser rely on thread_sets
- CLEANUP: config: stop using bind_tgroup and bind_thread
- CLEANUP: listener/thread: remove now unused bind_conf's bind_tgroup/bind_thread
- CLEANUP: listener/config: remove the special case for shards==1
- MEDIUM: config: restrict shards, not bind_conf to one group each
- BUG/MEDIUM: quic: do not split STREAM frames if no space
- BUILD: thread: fix build warnings with older gcc compilers
Instead of reading and storing a single group and a single mask for a
"thread" directive on a bind line, we now store the complete range in
a thread set that's stored in the bind_conf. The bind_parse_thread()
function now just calls parse_thread_set() to complete the current set,
which starts empty, and thread_resolve_group_mask() was updated to
support retrieving thread group numbers or absolute thread numbers
directly from the pre-filled thread_set, and continue to feed bind_tgroup
and bind_thread. The CLI parsers which were pre-initialized to set the
bind_tgroup to 1 cannot do it anymore as it would prevent one from
restricting the thread set. Instead check_config_validity() now detects
the CLI frontend and passes the info down to thread_resolve_group_mask()
that will automatically use only the group 1's threads for these
listeners. The same is done for the peers listeners for now.
At this step it's already possible to start with all previous valid
configs as well as extended ones supporting comma-delimited thread
sets. In addition the parser already accepts large ranges spanning
multiple groups, but since the underlying listeners infrastructure
is not read, for now we're maintaining a specific check against this
at the higher level of the config validity check.
The patch is a bit large because thread resolution is performed in
multiple steps, so we need to adjust all of them at once to preserve
functional and technical consistency.
Define a new configuration option "tune.quic.max-frame-loss". This is
used to specify the limit for which a single frame instance can be
detected as lost. If exceeded, the connection is closed.
This should be backported up to 2.7.
Both doc and code agree on the fact that 'http-send-name-header' option
could be used in default section, but the keyword compatibility matrix
in configuration.txt reported the opposite.
This could be backported to all stable versions.
The doc mentioned that spop-check option may only be used for backends.
However, option may be used in default and listen sections as well
according to the code. Let's fix the doc so that doc and code are
consistent to each other.
This could be backported to all stable versions.
Adding new http converter: rfc7239_n2np.
Takes a string representing 7239 forwarded header node (extracted from
either 'for' or 'by' 7239 header fields) as input and translates it
to either unsigned integer or ('_' prefixed obfuscated identifier),
according to 7239RFC.
Example:
# extract 'by' field from forwarded header, extract node port from
# resulting node identifier and store the result in req.fnp
http-request set-var(req.fnp) req.hdr(forwarded),rfc7239_field(by),rfc7239_n2np
#input: "by=\"127.0.0.1:9999\""
# output: 9999
#input: "by=\"_name:_port\""
# output: "_port"
Depends on:
- "MINOR: http_ext: introduce http ext converters"
Adding new http converter: rfc7239_n2nn.
Takes a string representing 7239 forwarded header node (extracted from
either 'for' or 'by' 7239 header fields) as input and translates it
to either ipv4 address, ipv6 address or str ('_' prefixed if obfuscated
or "unknown" if unknown), according to 7239RFC.
Example:
# extract 'for' field from forwarded header, extract nodename from
# resulting node identifier and store the result in req.fnn
http-request set-var(req.fnn) req.hdr(forwarded),rfc7239_field(for),rfc7239_n2nn
#input: "for=\"127.0.0.1:9999\""
# output: 127.0.0.1
#input: "for=\"_name:_port\""
# output: "_name"
Depends on:
- "MINOR: http_ext: introduce http ext converters"
Adding new http converter: rfc7239_field.
Takes a string representing 7239 forwarded header single value as
input and extracts a single field/parameter from the header according
to user selection.
Example:
# extract host field from forwarded header and store it in req.fhost var
http-request set-var(req.fhost) req.hdr(forwarded),rfc7239_field(host)
#input: "proto=https;host=\"haproxy.org:80\""
# output: "haproxy.org:80"
# extract for field from forwarded header and store it in req.ffor var
http-request set-var(req.ffor) req.hdr(forwarded),rfc7239_field(for)
#input: "proto=https;host=\"haproxy.org:80\";for=\"127.0.0.1:9999\""
# output: "127.0.0.1:9999"
Depends on:
- "MINOR: http_ext: introduce http ext converters"
Adding new http converter: rfc7239_is_valid.
Takes a string representing 7239 forwarded header single value as
input and returns bool:TRUE if header is RFC compliant and
bool:FALSE otherwise.
Example:
acl valid req.hdr(forwarded),rfc7239_is_valid
#input: "for=127.0.0.1;proto=http"
# output: TRUE
#input: "proto=custom"
# output: FALSE
Depends on:
- "MINOR: http_ext: introduce http ext converters"
Introducing http_ext class for http extension related work that
doesn't fit into existing http classes.
HTTP extension "forwarded", introduced with 7239 RFC is now supported
by haproxy.
The option supports various modes from simple to complex usages involving
custom sample expressions.
Examples :
# Those servers want the ip address and protocol of the client request
# Resulting header would look like this:
# forwarded: proto=http;for=127.0.0.1
backend www_default
mode http
option forwarded
#equivalent to: option forwarded proto for
# Those servers want the requested host and hashed client ip address
# as well as client source port (you should use seed for xxh32 if ensuring
# ip privacy is a concern)
# Resulting header would look like this:
# forwarded: host="haproxy.org";for="_000000007F2F367E:60138"
backend www_host
mode http
option forwarded host for-expr src,xxh32,hex for_port
# Those servers want custom data in host, for and by parameters
# Resulting header would look like this:
# forwarded: host="host.com";by=_haproxy;for="[::1]:10"
backend www_custom
mode http
option forwarded host-expr str(host.com) by-expr str(_haproxy) for for_port-expr int(10)
# Those servers want random 'for' obfuscated identifiers for request
# tracing purposes while protecting sensitive IP information
# Resulting header would look like this:
# forwarded: for=_000000002B1F4D63
backend www_for_hide
mode http
option forwarded for-expr rand,hex
By default (no argument provided), forwarded option will try to mimic
x-forward-for common setups (source client ip address + source protocol)
The option is not available for frontends.
no option forwarded is supported.
More info about 7239 RFC here: https://www.rfc-editor.org/rfc/rfc7239.html
More info about the feature in doc/configuration.txt
This should address feature request GH #575
Depends on:
- "MINOR: http_htx: add http_append_header() to append value to header"
- "MINOR: sample: add ARGC_OPT"
- "MINOR: proxy: introduce http only options"
Released version 2.8-dev2 with the following main changes :
- CLEANUP: htx: fix a typo in an error message of http_str_to_htx
- DOC: config: added optional rst-ttl argument to silent-drop in action lists
- BUG/MINOR: ssl: Fix crash in 'update ssl ocsp-response' CLI command
- BUG/MINOR: ssl: Crash during cleanup because of ocsp structure pointer UAF
- MINOR: ssl: Create temp X509_STORE filled with cert chain when checking ocsp response
- MINOR: ssl: Only set ocsp->issuer if issuer not in cert chain
- MINOR: ssl: Release ssl_ocsp_task_ctx.cur_ocsp when destroying task
- MINOR: ssl: Detect more OCSP update inconsistencies
- BUG/MINOR: ssl: Fix OCSP_CERTID leak when same certificate is used multiple times
- MINOR: ssl: Limit ocsp_uri buffer size to minimum
- MINOR: ssl: Remove mention of ckch_store in error message of cli command
- MINOR: channel: Don't test CF_READ_NULL while CF_SHUTR is enough
- REORG: channel: Rename CF_READ_NULL to CF_READ_EVENT
- REORG: channel: Rename CF_WRITE_NULL to CF_WRITE_EVENT
- MEDIUM: channel: Use CF_READ_EVENT instead of CF_READ_PARTIAL
- MEDIUM: channel: Use CF_WRITE_EVENT instead of CF_WRITE_PARTIAL
- MINOR: channel: Remove CF_READ_ACTIVITY
- MINOR: channel: Remove CF_WRITE_ACTIVITY
- MINOR: channel: Remove CF_ANA_TIMEOUT and report CF_READ_EVENT instead
- MEDIUM: channel: Remove CF_READ_ATTACHED and report CF_READ_EVENT instead
- MINOR: channel: Stop to test CF_READ_ERROR flag if CF_SHUTR is enough
- MINOR: channel/applets: Stop to test CF_WRITE_ERROR flag if CF_SHUTW is enough
- DOC: management: add details on "Used" status
- DOC: management: add details about @system-ca in "show ssl ca-file"
- BUG/MINOR: mux-quic: fix transfer of empty HTTP response
- MINOR: mux-quic: add traces for flow-control limit reach
- MAJOR: mux-quic: rework stream sending priorization
- MEDIUM: h3: send SETTINGS before STREAM frames
- MINOR: mux-quic: use send-list for STOP_SENDING/RESET_STREAM emission
- MINOR: mux-quic: use send-list for immediate sending retry
- BUG/MINOR: h1-htx: Remove flags about protocol upgrade on non-101 responses
- BUG/MINOR: hlua: Fix Channel.line and Channel.data behavior regarding the doc
- BUG/MINOR: resolvers: Wait the resolution execution for a do_resolv action
- BUG/MINOR: ssl: Remove unneeded pointer check in ocsp cli release function
- BUG/MINOR: ssl: Missing ssl_conf pointer check when checking ocsp update inconsistencies
- DEV: tcploop: add minimal support for unix sockets
- BUG/MEDIUM: listener: duplicate inherited FDs if needed
- BUG/MINOR: ssl: OCSP minimum update threshold not properly set
- MINOR: ssl: Treat ocsp-update inconsistencies as fatal errors
- MINOR: ssl: Do not wake ocsp update task if update tree empty
- MINOR: ssl: Reinsert updated ocsp response later in tree in case of http error
- REGTEST: ssl: Add test for 'update ssl ocsp-response' CLI command
- OPTIM: global: move byte counts out of global and per-thread
- BUG/MEDIUM: peers: make "show peers" more careful about partial initialization
- BUG/MINOR: promex: Don't forget to consume the request on error
- MINOR: http-ana: Add a function to set HTTP termination flags
- MINOR: http-ana: Use http_set_term_flags() in most of HTTP analyzers
- BUG/MINOR: http-ana: Report SF_FINST_R flag on error waiting the request body
- MINOR: http-ana: Use http_set_term_flags() when waiting the request body
- BUG/MINOR: http-fetch: Don't block HTTP sample fetch eval in HTTP_MSG_ERROR state
- MAJOR: http-ana: Review error handling during HTTP payload forwarding
- CLEANUP: http-ana: Remove HTTP_MSG_ERROR state
- BUG/MEDIUM: mux-h2: Don't send CANCEL on shutw when response length is unkown
- MINOR: htx: Add an HTX value for the extra field is payload length is unknown
- BUG/MINOR: http-ana: make set-status also update txn->status
- BUG/MINOR: listeners: fix suspend/resume of inherited FDs
- DOC: config: fix wrong section number for "protocol prefixes"
- DOC: config: fix aliases for protocol prefixes "udp4@" and "udp6@"
- DOC: config: mention the missing "quic4@" and "quic6@" in protocol prefixes
- MINOR: listener: also support "quic+" as an address prefix
- CLEANUP: stconn: always use se_fl_set_error() to set the pending error
- BUG/MEDIUM: stconn: also consider SE_FL_EOI to switch to SE_FL_ERROR
- MINOR: quic: Useless test about datagram destination addresses
- MINOR: quic: Disable the active connection migrations
- MINOR: quic: Add "no-quic" global option
- MINOR: sample: Add "quic_enabled" sample fetch
- MINOR: quic: Replace v2 draft definitions by those of the final 2 version
- BUG/MINOR: mux-fcgi: Correctly set pathinfo
- DOC: config: fix "Address formats" chapter syntax
- BUG/MEDIUM: jwt: Properly process ecdsa signatures (concatenated R and S params)
- BUILD: ssl: add ECDSA_SIG_set0() for openssl < 1.1 or libressl < 2.7
- Revert "BUILD: ssl: add ECDSA_SIG_set0() for openssl < 1.1 or libressl < 2.7"
- BUG/MINOR: ssl: Fix compilation with OpenSSL 1.0.2 (missing ECDSA_SIG_set0)
- BUG/MINOR: listener: close tiny race between resume_listener() and stopping
- BUG/MINOR: h3: properly handle connection headers
- MINOR: h3: extend function for QUIC varint encoding
- MINOR: h3: implement TRAILERS encoding
- BUG/MINOR: bwlim: Check scope for period expr for set-bandwitdh-limit actions
- MEDIUM: bwlim: Support constants limit or period on set-bandwidth-limit actions
- BUG/MINOR: bwlim: Fix parameters check for set-bandwidth-limit actions
- MINOR: h3: implement TRAILERS decoding
- BUG/MEDIUM: fd/threads: fix again incorrect thread selection in wakeup broadcast
- BUG/MINOR: thread: always reload threads_enabled in loops
- MINOR: threads: add a thread_harmless_end() version that doesn't wait
- BUG/MEDIUM: debug/thread: make the debug handler not wait for !rdv_requests
- BUG/MINOR: mux-h2: make sure to produce a log on invalid requests
- BUG/MINOR: mux-h2: add missing traces on failed headers decoding
- BUILD: hpack: include global.h for the trash that is needed in debug mode
- BUG/MINOR: jwt: Wrong return value checked
- BUG/MINOR: quic: Do not request h3 clients to close its unidirection streams
- MEDIUM: quic-sock: fix udp source address for send on listener socket
It is now possible to set a constant for the limit or period parameters on a
set-bandwidth-limit actions. The limit must follow the HAProxy size format
and is expressed in bytes. The period must follow the HAProxy time format
and is expressed in milliseconds. Of course, it is still possible to use
sample expressions instead.
The documentation was updated accordingly.
It is not really a bug. Only exemples were written this way in the
documentation. But it could be good to backport this change in 2.7.
The section on "Address formats" doesn't provide the dot (.) after the
chapter numbers, which breaks parsing within the HTML converter.
This commit adds the dot (.) after each chapter within Section 11.
This should be backported to versions 2.4 and above.
This sample fetch returns a boolean. True if the support for QUIC transport
protocol was built and if this protocol was not disabled by "no-quic"
global option.
Must be backported to 2.7.
Add "no-quic" to "global" section to disable the use of QUIC transport protocol
by all configured QUIC listeners. This is listeners with QUIC addresses on their
"bind" lines. Internally, the socket addresses binding is skipped by
protocol_bind_all() for receivers with <proto_quic4> or <proto_quic6> as
protocol (see protocol struct).
Add information about "no-quic" global option to the documentation.
Must be backported to 2.7.
While we do support quic4@ and quic6@ for listening addresses, it was
not possible to specify that we want to use an FD inherited from the
parent with QUIC. It's just a matter of making it possible to enable
a dgram-type socket and a stream-type transport, so let's add this.
Now it becomes possible to write "quic+fd@12", "quic+ipv4@addr" etc.
Released version 2.8-dev1 with the following main changes :
- MEDIUM: 51d: add support for 51Degrees V4 with Hash algorithm
- MINOR: debug: support pool filtering on "debug dev memstats"
- MINOR: debug: add a balance of alloc - free at the end of the memstats dump
- LICENSE: wurfl: clarify the dummy library license.
- MINOR: event_hdl: add event handler base api
- DOC/MINOR: api: add documentation for event_hdl feature
- MEDIUM: ssl: rename the struct "cert_key_and_chain" to "ckch_data"
- MINOR: quic: remove qc from quic_rx_packet
- MINOR: quic: complete traces in qc_rx_pkt_handle()
- MINOR: quic: extract datagram parsing code
- MINOR: tools: add port for ipcmp as optional criteria
- MINOR: quic: detect connection migration
- MINOR: quic: ignore address migration during handshake
- MINOR: quic: startup detect for quic-conn owned socket support
- MINOR: quic: test IP_PKTINFO support for quic-conn owned socket
- MINOR: quic: define config option for socket per conn
- MINOR: quic: allocate a socket per quic-conn
- MINOR: quic: use connection socket for emission
- MEDIUM: quic: use quic-conn socket for reception
- MEDIUM: quic: move receive out of FD handler to quic-conn io-cb
- MINOR: mux-quic: rename duplicate function names
- MEDIUM: quic: requeue datagrams received on wrong socket
- MINOR: quic: reconnect quic-conn socket on address migration
- MINOR: quic: activate socket per conn by default
- BUG/MINOR: ssl: initialize SSL error before parsing
- BUG/MINOR: ssl: initialize WolfSSL before parsing
- BUG/MINOR: quic: fix fd leak on startup check quic-conn owned socket
- BUG/MEDIIM: stconn: Flush output data before forwarding close to write side
- MINOR: server: add srv->rid (revision id) value
- MINOR: stats: add server revision id support
- MINOR: server/event_hdl: add support for SERVER_ADD and SERVER_DEL events
- MINOR: server/event_hdl: add support for SERVER_UP and SERVER_DOWN events
- BUG/MEDIUM: checks: do not reschedule a possibly running task on state change
- BUG/MINOR: checks: make sure fastinter is used even on forced transitions
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: mworker: display an alert upon a wait-mode exit
- BUG/MEDIUM: mworker: fix segv in early failure of mworker mode with peers
- BUG/MEDIUM: mworker: create the mcli_reload socketpairs in case of upgrade
- BUG/MINOR: checks: restore legacy on-error fastinter behavior
- MINOR: check: use atomic for s->consecutive_errors
- MINOR: stats: properly handle ST_F_CHECK_DURATION metric
- MINOR: mworker: remove unused legacy code in mworker_cleanlisteners
- MINOR: peers: unused code path in process_peer_sync
- BUG/MINOR: init/threads: continue to limit default thread count to max per group
- CLEANUP: init: remove useless assignment of nbthread
- BUILD: atomic: atomic.h may need compiler.h on ARMv8.2-a
- BUILD: makefile/da: also clean Os/ in Device Atlas dummy lib dir
- BUG/MEDIUM: httpclient/lua: double LIST_DELETE on end of lua task
- CLEANUP: pools: move the write before free to the uaf-only function
- CLEANUP: pool: only include pool-os from pool.c not pool.h
- REORG: pool: move all the OS specific code to pool-os.h
- CLEANUP: pools: get rid of CONFIG_HAP_POOLS
- DEBUG: pool: show a few examples in -dMhelp
- MINOR: pools: make DEBUG_UAF a runtime setting
- BUG/MINOR: promex: create haproxy_backend_agg_server_status
- MINOR: promex: introduce haproxy_backend_agg_check_status
- DOC: promex: Add missing backend metrics
- BUG/MAJOR: fcgi: Fix uninitialized reserved bytes
- REGTESTS: fix the race conditions in iff.vtc
- CI: github: reintroduce openssl 1.1.1
- BUG/MINOR: quic: properly handle alloc failure in qc_new_conn()
- BUG/MINOR: quic: handle alloc failure on qc_new_conn() for owned socket
- CLEANUP: mux-quic: remove unused attribute on qcs_is_close_remote()
- BUG/MINOR: mux-quic: remove qcs from opening-list on free
- BUG/MINOR: mux-quic: handle properly alloc error in qcs_new()
- CI: github: split ssl lib selection based on git branch
- REGTESTS: startup: check maxconn computation
- BUG/MINOR: startup: don't use internal proxies to compute the maxconn
- REGTESTS: startup: change the expected maxconn to 11000
- CI: github: set ulimit -n to a greater value
- REGTESTS: startup: activate automatic_maxconn.vtc
- MINOR: sample: add param converter
- CLEANUP: ssl: remove check on srv->proxy
- BUG/MEDIUM: freq-ctr: Don't compute overshoot value for empty counters
- BUG/MEDIUM: resolvers: Use tick_first() to update the resolvers task timeout
- REGTESTS: startup: add alternatives values in automatic_maxconn.vtc
- BUG/MEDIUM: h3: reject request with invalid header name
- BUG/MEDIUM: h3: reject request with invalid pseudo header
- MINOR: http: extract content-length parsing from H2
- BUG/MEDIUM: h3: parse content-length and reject invalid messages
- CI: github: remove redundant ASAN loop
- CI: github: split matrix for development and stable branches
- BUG/MEDIUM: mux-h1: Don't release H1 stream upgraded from TCP on error
- BUG/MINOR: mux-h1: Fix test instead a BUG_ON() in h1_send_error()
- MINOR: http-htx: add BUG_ON to prevent API error on http_cookie_register
- BUG/MEDIUM: h3: fix cookie header parsing
- BUG/MINOR: h3: fix memleak on HEADERS parsing failure
- MINOR: h3: check return values of htx_add_* on headers parsing
- MINOR: ssl: Remove unneeded buffer allocation in show ocsp-response
- MINOR: ssl: Remove unnecessary alloc'ed trash chunk in show ocsp-response
- BUG/MINOR: ssl: Fix memory leak of find_chain in ssl_sock_load_cert_chain
- MINOR: stats: provide ctx for dumping functions
- MINOR: stats: introduce stats field ctx
- BUG/MINOR: stats: fix show stat json buffer limitation
- MINOR: stats: make show info json future-proof
- BUG/MINOR: quic: fix crash on PTO rearm if anti-amplification reset
- BUILD: 51d: fix build issue with recent compilers
- REGTESTS: startup: disable automatic_maxconn.vtc
- BUILD: peers: peers-t.h depends on stick-table-t.h
- BUG/MEDIUM: tests: use tmpdir to create UNIX socket
- BUG/MINOR: mux-h1: Report EOS on parsing/internal error for not running stream
- BUG/MINOR:: mux-h1: Never handle error at mux level for running connection
- BUG/MEDIUM: stats: Rely on a local trash buffer to dump the stats
- OPTIM: pool: split the read_mostly from read_write parts in pool_head
- MINOR: pool: make the thread-local hot cache size configurable
- MINOR: freq_ctr: add opportunistic versions of swrate_add()
- MINOR: pool: only use opportunistic versions of the swrate_add() functions
- REGTESTS: ssl: enable the ssl_reuse.vtc test for WolfSSL
- BUG/MEDIUM: mux-quic: fix double delete from qcc.opening_list
- BUG/MEDIUM: quic: properly take shards into account on bind lines
- BUG/MINOR: quic: do not allocate more rxbufs than necessary
- MINOR: ssl: Add a lock to the OCSP response tree
- MINOR: httpclient: Make the CLI flags public for future use
- MINOR: ssl: Add helper function that extracts an OCSP URI from a certificate
- MINOR: ssl: Add OCSP request helper function
- MINOR: ssl: Add helper function that checks the validity of an OCSP response
- MINOR: ssl: Add "update ssl ocsp-response" cli command
- MEDIUM: ssl: Add ocsp_certid in ckch structure and discard ocsp buffer early
- MINOR: ssl: Add ocsp_update_tree and helper functions
- MINOR: ssl: Add crt-list ocsp-update option
- MINOR: ssl: Store 'ocsp-update' mode in the ckch_data and check for inconsistencies
- MEDIUM: ssl: Insert ocsp responses in update tree when needed
- MEDIUM: ssl: Add ocsp update task main function
- MEDIUM: ssl: Start update task if at least one ocsp-update option is set to on
- DOC: ssl: Add documentation for ocsp-update option
- REGTESTS: ssl: Add tests for ocsp auto update mechanism
- MINOR: ssl: Move OCSP code to a dedicated source file
- BUG/MINOR: ssl/ocsp: check chunk_strcpy() in ssl_ocsp_get_uri_from_cert()
- CLEANUP: ssl/ocsp: add spaces around operators
- BUG/MEDIUM: mux-h2: Refuse interim responses with end-stream flag set
- BUG/MINOR: pool/stats: Use ullong to report total pool usage in bytes in stats
- BUG/MINOR: ssl/ocsp: httpclient blocked when doing a GET
- MINOR: httpclient: don't add body when istlen is empty
- MEDIUM: httpclient: change the default log format to skip duplicate proxy data
- BUG/MINOR: httpclient/log: free of invalid ptr with httpclient_log_format
- MEDIUM: mux-quic: implement shutw
- MINOR: mux-quic: do not count stream flow-control if already closed
- MINOR: mux-quic: handle RESET_STREAM reception
- MEDIUM: mux-quic: implement STOP_SENDING emission
- MINOR: h3: use stream error when needed instead of connection
- CI: github: enable github api authentication for OpenSSL tags read
- BUG/MINOR: mux-quic: ignore remote unidirectional stream close
- CI: github: use the GITHUB_TOKEN instead of a manually generated token
- BUILD: makefile: build the features list dynamically
- BUILD: makefile: move common options-oriented macros to include/make/options.mk
- BUILD: makefile: sort the features list
- BUILD: makefile: initialize all build options' variables at once
- BUILD: makefile: add a function to collect all options' CFLAGS/LDFLAGS
- BUILD: makefile: start to automatically collect CFLAGS/LDFLAGS
- BUILD: makefile: ensure that all USE_* handlers appear before CFLAGS are used
- BUILD: makefile: clean the wolfssl include and lib generation rules
- BUILD: makefile: make sure to also ignore SSL_INC when using wolfssl
- BUILD: makefile: reference libdl only once
- BUILD: makefile: make sure LUA_INC and LUA_LIB are always initialized
- BUILD: makefile: do not restrict Lua's prepend path to empty LUA_LIB_NAME
- BUILD: makefile: never force -latomic, set USE_LIBATOMIC instead
- BUILD: makefile: add an implicit USE_MATH variable for -lm
- BUILD: makefile: properly report USE_PCRE/USE_PCRE2 in features
- CLEANUP: makefile: properly indent ifeq/ifneq conditional blocks
- BUILD: makefile: rework 51D to split v3/v4
- BUILD: makefile: support LIBCRYPT_LDFLAGS
- BUILD: makefile: support RT_LDFLAGS
- BUILD: makefile: support THREAD_LDFLAGS
- BUILD: makefile: support BACKTRACE_LDFLAGS
- BUILD: makefile: support SYSTEMD_LDFLAGS
- BUILD: makefile: support ZLIB_CFLAGS and ZLIB_LDFLAGS
- BUILD: makefile: support ENGINE_CFLAGS
- BUILD: makefile: support OPENSSL_CFLAGS and OPENSSL_LDFLAGS
- BUILD: makefile: support WOLFSSL_CFLAGS and WOLFSSL_LDFLAGS
- BUILD: makefile: support LUA_CFLAGS and LUA_LDFLAGS
- BUILD: makefile: support DEVICEATLAS_CFLAGS and DEVICEATLAS_LDFLAGS
- BUILD: makefile: support PCRE[2]_CFLAGS and PCRE[2]_LDFLAGS
- BUILD: makefile: refactor support for 51DEGREES v3/v4
- BUILD: makefile: support WURFL_CFLAGS and WURFL_LDFLAGS
- BUILD: makefile: make all OpenSSL variants use the same settings
- BUILD: makefile: remove the special case of the SSL option
- BUILD: makefile: only consider settings from enabled options
- BUILD: makefile: also list per-option settings in 'make opts'
- BUG/MINOR: debug: don't mask the TH_FL_STUCK flag before dumping threads
- MINOR: cfgparse-ssl: avoid a possible crash on OOM in ssl_bind_parse_npn()
- BUG/MINOR: ssl: Missing goto in error path in ocsp update code
- BUG/MINOR: stick-table: report the correct action name in error message
- CI: Improve headline in matrix.py
- CI: Add in-memory cache for the latest OpenSSL/LibreSSL
- CI: Use proper `if` blocks instead of conditional expressions in matrix.py
- CI: Unify the `GITHUB_TOKEN` name across matrix.py and vtest.yml
- CI: Explicitly check environment variable against `None` in matrix.py
- CI: Reformat `matrix.py` using `black`
- MINOR: config: add environment variables for default log format
- REGTESTS: Remove REQUIRE_VERSION=1.9 from all tests
- REGTESTS: Remove REQUIRE_VERSION=2.0 from all tests
- REGTESTS: Remove tests with REQUIRE_VERSION_BELOW=1.9
- BUG/MINOR: http-fetch: Only fill txn status during prefetch if not already set
- BUG/MAJOR: buf: Fix copy of wrapping output data when a buffer is realigned
- DOC: config: fix alphabetical ordering of http-after-response rules
- MINOR: http-rules: Add missing actions in http-after-response ruleset
- DOC: config: remove duplicated "http-response sc-set-gpt0" directive
- BUG/MINOR: proxy: free orgto_hdr_name in free_proxy()
- REGTEST: fix the race conditions in json_query.vtc
- REGTEST: fix the race conditions in add_item.vtc
- REGTEST: fix the race conditions in digest.vtc
- REGTEST: fix the race conditions in hmac.vtc
- BUG/MINOR: fd: avoid bad tgid assertion in fd_delete() from deinit()
- BUG/MINOR: http: Memory leak of http redirect rules' format string
- MEDIUM: stick-table: set the track-sc limit at boottime via tune.stick-counters
- MINOR: stick-table: implement the sc-add-gpc() action
This action increments the General Purpose Counter at the index <idx> of
the array associated to the sticky counter designated by <sc-id> by the
value of either integer <int> or the integer evaluation of expression
<expr>. Integers and expressions are limited to unsigned 32-bit values.
If an error occurs, this action silently fails and the actions evaluation
continues. <idx> is an integer between 0 and 99 and <sc-id> is an integer
between 0 and 2. It also silently fails if the there is no GPC stored at
this index. The entry in the table is refreshed even if the value is zero.
The 'gpc_rate' is automatically adjusted to reflect the average growth
rate of the gpc value.
The main use of this action is to count scores or total volumes (e.g.
estimated danger per source IP reported by the server or a WAF, total
uploaded bytes, etc).
The number of stick-counter entries usable by track-sc rules is currently
set at build time. There is no good value for this since the vast majority
of users don't need any, most need only a few and rare users need more.
Adding more counters for everyone increases memory and CPU usages for no
reason.
This patch moves the per-session and per-stream arrays to a pool of a size
defined at boot time. This way it becomes possible to set the number of
entries at boot time via a new global setting "tune.stick-counters" that
sets the limit for the whole process. When not set, the MAX_SESS_STR_CTR
value still applies, or 3 if not set, as before.
It is also possible to lower the value to 0 to save a bit of memory if
not used at all.
Note that a few low-level sample-fetch functions had to be protected due
to the ability to use sample-fetches in the global section to set some
variables.
This patch adds the support of following actions in the http-after-response
ruleset:
* set-map, del-map and del-acl
* set-log-level
* sc-inc-gpc, sc-inc-gpc0 and set-inc-gpc1
* sc-inc-gpt and sc-set-gpt0
This patch should solve the issue #1980.
This patch provides a convenient way to override the default TCP, HTTP
and HTTP log formats. Instead of having a look into the documentation
to figure out what is the appropriate default log format three new
environment variables can be used: HAPROXY_TCP_LOG_FMT,
HAPROXY_HTTP_LOG_FMT and HAPROXY_HTTPS_LOG_FMT. Their content are
substituted verbatim.
These variables are set before parsing the configuration and are unset
just after all configuration files are successful parsed.
Example:
# Instead of writing this long log-format line...
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
%CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r \
lr=last_rule_file:last_rule_line"
# ..the HAPROXY_HTTP_LOG_FMT can be used to provide the default
# http log-format string
log-format "${HAPROXY_HTTP_LOG_FMT} lr=last_rule_file:last_rule_line"
Please note that nothing prevents users to unset the variables or
override their content in a global section.
Signed-off-by: Sébastien Gross <sgross@haproxy.com>
In order to simplify maintenance and long-term evolutions, now the
feature remains enabled by setting USE_51DEGREES=1 and the version
is set in 51DEGREES_VER (3 or 4 are supported only). The default
version remains 3. All 51DEGREES flags are shared between both
versions and only use the "51DEGREES_" prefix.
The related CFLAGS and LDFLAGS can now be overridden using
51DEGREES_CFLAGS and 51DEGREES_LDFLAGS, both of which are automatically
collected into the respective OPTIONS_*. The USE_51DEGREES_V4 option is
now removed, and the doc was updated.
The same change was already performed for the cli. The stats applet and the
prometheus exporter are also concerned. Both use the stats API and rely on
pool functions to get total pool usage in bytes. pool_total_allocated() and
pool_total_used() must return 64 bits unsigned integer to avoid any wrapping
around 4G.
This may be backported to all versions.
The new "update ssl ocsp-response <certfile>" CLI command allows to
update the stored OCSP response for a given certificate. It relies on
the http_client which is used to send an HTTP request to the OCSP
responder whose URI can be extracted from the certificate.
This command won't work for a certificate that did not have a stored
OCSP response yet.
Till now it was only possible to change the thread local hot cache size
at build time using CONFIG_HAP_POOL_CACHE_SIZE. But along benchmarks it
was sometimes noticed a huge contention in the lower level memory
allocators indicating that larger caches could be beneficial, especially
on machines with large L2 CPUs.
Given that the checks against this value was no longer on a hot path
anymore, there was no reason for continuing to force it to be tuned at
build time. So this patch allows to set it by tune.memory-hot-size.
It's worth noting that during the boot phase the value remains zero so
that it's possible to know if the value was set or not, which opens the
possibility that we try to automatically adjust it based on the per-cpu
L2 cache size or the use of certain protocols (none of this is done yet).
Since the massive pools cleanup that happened in 2.6, the pools
architecture was made quite more hierarchical and many alternate code
blocks could be moved to runtime flags set by -dM. One of them had not
been converted by then, DEBUG_UAF. It's not much more difficult actually,
since it only acts on a pair of functions indirection on the slow path
(OS-level allocator) and a default setting for the cache activation.
This patch adds the "uaf" setting to the options permitted in -dM so
that it now becomes possible to set or unset UAF at boot time without
recompiling. This is particularly convenient, because every 3 months on
average, developers ask a user to recompile haproxy with DEBUG_UAF to
understand a bug. Now it will not be needed anymore, instead the user
will only have to disable pools and enable uaf using -dMuaf. Note that
-dMuaf only disables previously enabled pools, but it remains possible
to re-enable caching by specifying the cache after, like -dMuaf,cache.
A few tests with this mode show that it can be an interesting combination
which catches significantly less UAF but will do so with much less
overhead, so it might be compatible with some high-traffic deployments.
The change is very small and isolated. It could be helpful to backport
this at least to 2.7 once confirmed not to cause build issues on exotic
systems, and even to 2.6 a bit later as this has proven to be useful
over time, and could be even more if it did not require a rebuild. If
a backport is desired, the following patches are needed as well:
CLEANUP: pools: move the write before free to the uaf-only function
CLEANUP: pool: only include pool-os from pool.c not pool.h
REORG: pool: move all the OS specific code to pool-os.h
CLEANUP: pools: get rid of CONFIG_HAP_POOLS
DEBUG: pool: show a few examples in -dMhelp
Activate QUIC connection socket to achieve the best performance. The
previous behavior can be reverted by tune.quic.socket-owner
configuration option.
This change is part of quic-conn owned socket implementation.
Contrary to its siblings patches, I suggest to not backport it to 2.7.
This should ensure that stable releases behavior is perserved. If a user
faces issues with QUIC performance on 2.7, he can nonetheless change the
default configuration.
Define global configuration option "tune.quic.socket-owner". This option
can be used to activate or not socket per QUIC connection mode. The
default value is "listener" which disable this feature. It can be
activated with the option "connection".
This change is part of quic-conn owned socket implementation.
It may be backported to 2.7 after a period of observation.
This is an initial work for the dedicated
event handler API internal documentation.
The file is located at doc/internals/api/event_hdl.txt
event_hdl feature has been introduced with:
MINOR: event_hdl: add event handler base api
This patch also adds a set of new global options:
- 51degrees-use-performance-graph { on | off }
- 51degrees-use-predictive-graph { on | off }
- 51degrees-drift <number>
- 51degrees-difference <number>
- 51degrees-allow-unmatched { on | off }
To build using the latest 51Degrees V4 engine with Hash algorithm, set
USE_51DEGREES_V4=1.
Other supported build options are 51DEGREES_INC, 51DEGREES_LIB and
51DEGREES_SRC which needs to be set to the directory that contains
headers and C files. For example:
make TARGET=<target> USE_51DEGREES_V4=1 51DEGREES_SRC='51D_REPO_PATH'/src
Released version 2.7.0 with the following main changes :
- MINOR: ssl: forgotten newline in error messages on ca-file
- BUG/MINOR: ssl: shut the ca-file errors emitted during httpclient init
- DOC: config: provide some configuration hints for "http-reuse"
- DOC: config: refer to section about quoting in the "add_item" converter
- DOC: halog: explain how to use -ac and -ad in the help message
- DOC: config: clarify the fact that SNI should not be used in HTTP scenarios
- DOC: config: mention that a single monitor-uri rule is supported
- DOC: config: explain how default matching method for ACL works
- DOC: config: clarify the fact that "retries" is not just for connections
- BUILD: halog: fix missing double-quote at end of help line
- DOC: config: clarify the -m dir and -m dom pattern matching methods
- MINOR: activity: report uptime in "show activity"
- REORG: activity/cli: move the "show activity" handler to activity.c
- DEV: poll: add support for epoll
- DEV: tcploop: centralize the polling code into wait_for_fd()
- DEV: tcploop: add support for POLLRDHUP when supported
- DEV: tcploop: do not report an error on POLLERR
- DEV: tcploop: add optional support for epoll
- SCRIPTS: announce-release: add a link to the data plane API
- CLEANUP: stick-table: fill alignment holes in the stktable struct
- MINOR: stick-table: store a per-table hash seed and use it
- MINOR: stick-table: show the shard number in each entry's "show table" output
- CLEANUP: ncbuf: remove ncb_blk args by value
- CLEANUP: ncbuf: inline small functions
- CLEANUP: ncbuf: use standard BUG_ON with DEBUG_STRICT
- BUG/MINOR: quic: Endless loop during retransmissions
- MINOR: mux-h2: add the expire task and its expiration date in "show fd"
- BUG/MINOR: peers: always initialize the stksess shard value
- REGTESTS: fix peers-related regtests regarding "show table"
- BUG/MEDIUM: mux-h1: Close client H1C on EOS when there is no output data
- MINOR: stick-table: change the API of the function used to calculate the shard
- CLEANUP: peers: factor out the key len calculation in received updates
- BUG/MINOR: peers: always update the stksess shard number on incoming updates
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: mux-h1: add the expire task and its expiration date in "show fd"
- MINOR: debug: improve error handling on the memstats command parser
- BUILD: quic: allow build with USE_QUIC and USE_OPENSSL_WOLFSSL
- CLEANUP: anon: clarify the help message on "debug dev hash"
- MINOR: debug: relax access restrictions on "debug dev hash" and "memstats"
- SCRIPTS: run-regtests: add a version check
- MINOR: version: mention that it's stable now
Stick-tables support sharding to multiple peers but there was no way to
know to what shard an entry was going to be sent. Let's display this in
the "show table" output to ease debugging.
There's regularly some confusion about them (do they match at the
beginning, end ? do they support multiple components etc). Tim
suggested to improve the doc in issue #61, it's never too late, so
let's do it now wih a few examples.
In issue #412 it was rightfully reported that the wording in "retries"
still exclusively speaks about connection attempts, while since L7
retries with "retry-on" it's no longer a limitation. Let's update the
text.
In issue #698, it's made apparent that the default matching method for
ACL keywords can be confusing when a converter is applied, because
depending on the converters used, users may think that the default
matching method from the sample fetch name might apply to the whole
expression. It's easier to understand that this doesn't make sense
when thinking about converters turning to completely different types
(e.g. hdr_beg(host),do_resolve() returns an IP, thus it's obvious
that _beg makes no sense at all). This patch states this in the
doc to avoid future confusion.
It was reported in issue #1059 that when multiple monitor-uri rules are
specified, only the last one is used. While this was done on purpose
since a single URI is used, it was not clearly mentioned in the doc,
possibly leading to confusion or wasted time trying to establish a
working setup. Let's clarify this point.
As reported by Tim in issue #1373 some warnings are deserved to explain
why using the frontend SNI for routing or connecting to a server is
usually not correct, especially since it can be tempting and used to
make sense in pure TCP scenarios.
As requested by Nick in issue #1719, let's add a reference to the section
about quoting there, since add_item() will often be used with commas and
it's easy to mess up.
This adds some configuration hints regarding various workloads that do
not manage to achieve high reuse rates due to too low a global maxconn
or thread groups.
This fixes github issue #1472.
Released version 2.7-dev10 with the following main changes :
- MEDIUM: tcp-act: add parameter rst-ttl to silent-drop
- BUG/MAJOR: quic: Crash upon retransmission of dgrams with several packets
- MINOR: cli: print parsed command when not found
- BUG/MAJOR: quic: Crash after discarding packet number spaces
- CLEANUP: quic: replace "choosen" with "chosen" all over the code
- MINOR: cli/pools: store "show pools" results into a temporary array
- MINOR: cli/pools: add sorting capabilities to "show pools"
- MINOR: cli/pools: add pool name filtering capability to "show pools"
- DOC: configuration: fix quic prefix typo
- MINOR: quic: report error if force-retry without cluster-secret
- MINOR: global: generate random cluster.secret if not defined
- BUG/MINOR: resolvers: do not run the timeout task when there's no resolution
- BUG/MINOR: server/idle: at least use atomic stores when updating max_used_conns
- MINOR: server/idle: make the next_takeover index per-tgroup
- BUILD: listener: fix build warning on global_listener_rwlock without threads
- BUG/MAJOR: sched: protect task during removal from wait queue
- BUILD: sched: fix build with DEBUG_THREAD with the previous commit
- DOC: quic: add note on performance issue with listener contention
- BUG/MINOR: cfgparse-listen: fix ebpt_next_dup pointer dereference on proxy "from" inheritance
- BUG/MINOR: log: fix parse_log_message rfc5424 size check
- CLEANUP: arg: remove extra check in make_arg_list arg escaping
- CLEANUP: tools: extra check in utoa_pad
- MINOR: h1: Consider empty port as invalid in authority for CONNECT
- MINOR: http: Considere empty ports as valid default ports
- BUG/MINOR: http-htx: Normalized absolute URIs with an empty port
- BUG/MINOR: h1: Replace authority validation to conform RFC3986
- REG-TESTS: http: Add more tests about authority/host matching
- BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action
- BUG/MEDIUM: mux-h1: Don't release H1C on timeout if there is a SC attached
- BUG/MEDIUM: mux-h1: Subscribe for reads on error on sending path
- BUILD: http-htx: Silent build error about a possible NULL start-line
- DOC: configuration.txt: add default_value for table_idle signature
- BUILD: ssl-sock: Silent error about NULL deref in ssl_sock_bind_verifycbk()
- BUG/MEDIUM: mux-h1: Remove H1C_F_WAIT_NEXT_REQ flag on a next request
- BUG/MINOR: mux-h1: Fix handling of 408-Request-Time-Out
- MINOR: mux-h1: Remove H1C_F_WAIT_NEXT_REQ in functions handling errors
- MINOR: mux-h1: Avoid useless call to h1_send() if no error is sent
- DOC: configuration.txt: fix typo in table_idle signature
- BUILD: stick-tables: fix build breakage in xxhash on older compilers
- BUILD: compiler: include compiler's definitions before ours
- BUILD: quic: global.h is needed in cfgparse-quic
- CLEANUP: tools: do not needlessly include xxhash nor cli from tools.h
- BUILD: flags: really restrict the cases where flags are exposed
- BUILD: makefile: minor reordering of objects by build time
- BUILD: quic: silence two invalid build warnings at -O1 with gcc-6.5
- BUILD: quic: use openssl-compat.h instead of openssl/ssl.h
- MEDIUM: ssl: add minimal WolfSSL support with OpenSSL compatibility mode
- MINOR: sample: make the rand() sample fetch function use the statistical_prng
- MINOR: auth: silence null dereference warning in check_user()
- CLEANUP: peers: fix format string for status messages (int signedness)
- CLEANUP: qpack: fix format string in debugging code (int signedness)
- CLEANUP: qpack: properly use the QPACK macros not HPACK ones in debug code
- BUG/MEDIUM: quic: fix datagram dropping on queueing failed
An extra ',' was mistakenly added in table_idle converter signature
with commit ed36968 ("DOC: configuration.txt: add default_value for
table_idle signature").
table_idle converter takes optional default_value argument.
The documentation correctly describes this usage but <default_value> was
missing in the converter signature.
It should be backported with bbeec37b3 ("MINOR: stick-table:
Add table_expire() and table_idle() new converters")
An abosulte URI is marked as normalized if it comes from an H2 client. This
way, we know we can send a relative URI to an H1 server. But, after a
set-uri action, the URI must no longer be considered as normalized.
Otherwise there is no way to send an absolute URI on the server side.
If it is important to update a normalized absolute URI without altering this
property, the host, path and/or query-string must be set separatly.
This patch should fix the issue #1938. It should be backported as far as
2.4.
Complete quic4/quic6 bind lines by a note on performance issues due to
receiver socket contention. Suggest to use sharding to improve the
situation.
This should be backported up to 2.6.
If no cluster-secret is defined by the user, a random one is silently
generated.
This ensures that at least QUIC Retry tokens are generated if abnormal
conditions are detected. However, it is advisable to specify it in the
configuration for tokens to be valid even after a reload or across LBs
instances in the same cluster.
This should be backported up to 2.6.
QUIC Retry generation relies on global cluster-secret to produce token
valid even after a process restart and across several LBs instances.
Before this patch, Retry is automatically deactivated if no
cluster-secret is provided. This is the case even if a user has
configured a QUIC listener with quic-force-retry. Change this behavior
by now returning an error during configuration parsing. The user must
provide a cluster-secret if quic-force-retry is used.
This shoud be backported up to 2.6.
The "show pools" command is used a lot for debugging but didn't get much
love over the years. This patch brings new capabilities:
- sorting the output by pool names to ese their finding ("byname").
- sorting the output by reverse item size to spot the biggest ones("bysize")
- sorting the output by reverse number of allocated bytes ("byusage")
The last one (byusage) also omits displaying the ones with zero allocation.
In addition, an optional max number of output entries may be passed so as
to dump only the N most relevant ones.
The silent-drop action was extended with an additional optional parameter,
[rst-ttl <ttl> ], causing HAProxy to send a TCP RST with the specified TTL
towards the client.
With this behaviour, the connection state on your own client-
facing middle-boxes (load balancers, firewalls) will be purged,
but the client will still assume the TCP connection is up because
the TCP RST packet expires before reaching the client.
Released version 2.7-dev9 with the following main changes :
- BUILD: quic: QUIC mux build fix for 32-bit build
- BUILD: scripts: disable tests build on QuicTLS build
- BUG/MEDIUM: httpclient: segfault when the httpclient parser fails
- BUILD: ssl_sock: fix null dereference for QUIC build
- BUILD: quic: Fix build for m68k cross-compilation
- BUG/MINOR: quic: fix buffer overflow on retry token generation
- MINOR: quic: add version field on quic_rx_packet
- MINOR: quic: extend pn_offset field from quic_rx_packet
- MINOR: quic: define first packet flag
- MINOR: quic: extract connection retrieval
- MINOR: quic: split and rename qc_lstnr_pkt_rcv()
- MINOR: quic: refactor packet drop on reception
- MINOR: quic: extend Retry token check function
- BUG/MINOR: log: Preserve message facility when the log target is a ring buffer
- BUG/MINOR: ring: Properly parse connect timeout
- BUG/MEDIUM: httpclient/lua: crash when the lua task timeout before the httpclient
- BUG/MEDIUM: httpclient: check if the httpclient was released in the IO handler
- REGTESTS: httpclient/lua: test the lua task timeout with the httpclient
- CI: github: dump the backtrace of coredumps in the alpine container
- BUILD: Makefile: add "USE_SHM_OPEN" on the linux-musl target
- DOC: lua: add a note about compression w/ httpclient
- CLEANUP: mworker/cli: rename the status function to loadstatus
- MINOR: mworker/cli: does no try to dump the startup-logs w/o USE_SHM_OPEN
- MINOR: list: fixing typo in MT_LIST_LOCK_ELT
- DOC/MINOR: list: fixing MT_LIST_LOCK_ELT macro documentation
- MINOR: list: adding MT_LIST_APPEND_LOCKED macro
- BUG/MINOR: mux-quic: complete flow-control for uni streams
- BUG/MEDIUM: compression: handle rewrite errors when updating response headers
- MINOR: quic: do not crash on unhandled sendto error
- MINOR: quic: display unknown error sendto counter on stat page
- MINOR: peers: Support for peer shards
- MINOR: peers: handle multiple resync requests using shards
- BUG/MINOR: sink: Only use backend capability for the sink proxies
- BUG/MINOR: sink: Set default connect/server timeout for implicit ring buffers
- MINOR: ssl: add the SSL error string when failing to load a certificate
- MINOR: ssl: add the SSL error string before the chain
- MEDIUM: ssl: be stricter about chain error
- BUG/MAJOR: stick-table: don't process store-response rules for applets
- MINOR: quic: remove unnecessary quic_session_accept()
- BUG/MINOR: quic: fix subscribe operation
- BUG/MINOR: log: fixing bug in tcp syslog_io_handler Octet-Counting
- MINOR: ssl: dump the SSL string error when SSL_CTX_use_PrivateKey() failed.
- MINOR: quic: add counter for interrupted reception
- BUG/MINOR: quic: fix race condition on datagram purging
- CI: add monthly gcc cross compile jobs
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: ssl: remove dead code in ssl_sock_load_pem_into_ckch()
- BUG/MINOR: httpclient: fixed memory allocation for the SSL ca_file
- BUG/MINOR: ssl: Memory leak of DH BIGNUM fields
- BUG/MINOR: ssl: Memory leak of AUTHORITY_KEYID struct when loading issuer
- BUG/MINOR: ssl: ocsp structure not freed properly in case of error
- CI: switch to the "latest" LibreSSL
- CI: enable QUIC for LibreSSL builds
- BUG/MEDIUM: ssl: Verify error codes can exceed 63
- MEDIUM: ssl: {ca,crt}-ignore-err can now use error constant name
- MINOR: ssl: x509_v_err_str converter transforms an integer to a X509_V_ERR name
- CLEANUP: cli: rename dynamic error printing state
- MINOR: cli: define usermsgs print context
- MINOR: server: clear prefix on stderr logs after add server
- BUG/MINOR: ssl: bind_conf is uncorrectly accessed when using QUIC
- BUILD: ssl_utils: fix build on gcc versions before 8
- BUILD: debug: remove unnecessary quotes in HA_WEAK() calls
- CI: emit the compiler's version in the build reports
- IMPORT: xxhash: update xxHash to version 0.8.1
- IMPORT: slz: declare len to fix debug build when optimal match is enabled
- IMPORT: slz: mention the potential header in slz_finish()
- IMPORT: slz: define and use a __fallthrough statement for switch/case
- BUILD: compiler: add a macro to detect if another one is set and equals 1
- BUILD: compiler: add a default definition for __has_attribute()
- BUILD: compiler: define a __fallthrough statement for switch/case
- BUILD: sample: use __fallthrough in smp_is_rw() and smp_dup()
- BUILD: quic: use __fallthrough in quic_connect_server()
- BUILD: ssl/crt-list: use __fallthrough in cli_io_handler_add_crtlist()
- BUILD: ssl: use __fallthrough in cli_io_handler_commit_{cert,cafile_crlfile}()
- BUILD: ssl: use __fallthrough in cli_io_handler_tlskeys_files()
- BUILD: hlua: use __fallthrough in hlua_post_init_state()
- BUILD: stream: use __fallthrough in stats_dump_full_strm_to_buffer()
- BUILD: tcpcheck: use __fallthrough in check_proxy_tcpcheck()
- BUILD: stats: use __fallthrough in stats_dump_proxy_to_buffer()
- BUILD: peers: use __fallthrough in peer_io_handler()
- BUILD: hash: use __fallthrough in hash_djb2()
- BUILD: tools: use __fallthrough in url_decode()
- BUILD: args: use __fallthrough in make_arg_list()
- BUILD: acl: use __fallthrough in parse_acl_expr()
- BUILD: spoe: use __fallthrough in spoe_handle_appctx()
- BUILD: logs: use __fallthrough in build_log_header()
- BUILD: check: use __fallthrough in __health_adjust()
- BUILD: http_act: use __fallthrough in parse_http_del_header()
- BUILD: h1_htx: use __fallthrough in h1_parse_chunk()
- BUILD: vars: use __fallthrough in var_accounting_{diff,add}()
- BUILD: map: use __fallthrough in cli_io_handler_*()
- BUILD: compression: use __fallthrough in comp_http_payload()
- BUILD: stconn: use __fallthrough in various shutw() functions
- BUILD: prometheus: use __fallthrough in promex_dump_metrics() and IO handler()
- CLEANUP: ssl: remove printf in bind_parse_ignore_err
- BUG/MINOR: ssl: crt-ignore-err memory leak with 'all' parameter
- BUG/MINOR: ssl: Fix potential overflow
- CLEANUP: stick-table: remove the unused table->exp_next
- OPTIM: stick-table: avoid atomic ops in stktable_requeue_exp() when possible
- BUG/MEDIUM: stick-table: fix a race condition when updating the expiration task
- MEDIUM: http-ana: remove set-cookie2 support
- BUG/MEDIUM: wdt/clock: properly handle early task hangs
- MINOR: deinit: add a "quick-exit" option to bypass the deinit step
- OPTIM: ebtree: make ebmb_insert_prefix() keep a copy the new node's pfx
- OPTIM: ebtree: make ebmb_insert_prefix() keep a copy the new node's key
- MINOR: ssl: ssl_sock_load_cert_chain() display error strings
- MINOR: ssl: reintroduce ERR_GET_LIB(ret) == ERR_LIB_PEM in ssl_sock_load_pem_into_ckch()
- BUG/MINOR: http-htx: Fix error handling during parsing http replies
- BUG/MINOR: resolvers: Don't wait periodic resolution on healthcheck failure
- BUG/MINOR: resolvers: Set port before IP address when processing SRV records
- BUG/MINOR: mux-fcgi: Be sure to send empty STDING record in case of zero-copy
- BUG/MEDIUM: mux-fcgi: Avoid value length overflow when it doesn't fit at once
- BUG/MINOR: ssl: SSL_load_error_strings might not be defined
- MINOR: pool/debug: create a new pool_alloc_flag() macro
- MINOR: dynbuf: switch allocation and release to macros to better track users
- BUG/MINOR: mux-h1: Do not send a last null chunk on body-less answers
- REG-TESTS: cache: Remove T-E header for 304-Not-Modified responses
- DOC: config: fix alphabetical ordering of global section
- MINOR: trace: split the CLI "trace" parser in CLI vs statement
- MEDIUM: trace: create a new "trace" statement in the "global" section
- BUG/MEDIUM: ring: fix creation of server in uninitialized ring
- BUILD: quic: fix dubious 0-byte overflow on qc_release_lost_pkts
- BUILD: makefile: mark poll and tcploop targets as phony
- BUILD: makefile: properly pass CC to sub-projects
- BUILD: makefile: move default verbosity settings to include/make/verbose.mk
- BUILD: makefile: use $(cmd_MAKE) in quiet mode
- BUILD: makefile: move the compiler option detection stuff to compiler.mk
- DEV: poll: make the connect() step an action as well
- DEV: poll: strip the "do_" prefix from reported function names
- DEV: poll: indicate the FD's side in front of its value
- BUG/MINOR: pool/cli: use ullong to report total pool usage in bytes
- MINOR: mux-h1: Remove usless code inside shutr callback
- CLEANUP: mux-h1; Rename H1S_F_ERROR flag into H1S_F_ERROR_MASK
- REORG: mux-h1: Reorg the H1C structure
- CLEANUP: mux-h1: Rename H1C_F_ST_ERROR and H1C_F_ST_SILENT_SHUT flags
- MINOR: mux-h1: Add a dedicated enum to deal with H1 connection state
- MEDIUM: mux-h1: Handle H1C states via its state field instead of H1C_F_ST_*
- MINOR: mux-h1: Don't handle subscribe for reads in h1_process_demux()
- CLEANUP: mux-h1: Rename H1C_F_ERR_PENDING into H1C_F_ABRT_PENDING
- MINOR: mux-h1: Add flag on H1 stream to deal with internal errors
- MEDIUM: mux-h1: Rely on the H1C to deal with shutdown for reads
- CLEANUP: mux-h1: Reorder H1 connection flags to avoid holes
- MEDIUM: mux-h1: Don't report a final error whe a message is aborted
- MEDIUM: mux-pt: Don't always set a final error on SE on the sending path
- MEDIUM: mux-h2: Introduce flags to deal with connection read/write errors
- CLEANUP: mux-h2: Remove unused fields in h2c structures
- MEDIUM: mux-fcgi: Introduce flags to deal with connection read/write errors
- MINOR: sconn: Set SE_FL_ERROR only when there is no more data to read
- MINOR: mux-h1: Rely on a H1S flag to know a WS key was found or not
- DOC: lua-api: Remove warning about the lua filters
- BUG/MEDIUM: listener: Fix race condition when updating the global mngmt task
- CLEANUP: listener: Remove useless task_queue from manage_global_listener_queue
- BUG/MINOR: mux-h1: Fix error handling when H1S allocation failed on client side
- DOC: internal: commit notes about polling states and flags
- DOC: internal: commit notes about polling states and flags on connect()
- CLEANUP: mux-h1: Don't test h1c in h1_shutw_conn()
- BUG/MINOR: http_ana/txn: don't re-initialize txn and req var lists
- BUG/MEDIUM: raw-sock: Don't report connection error if something was received
- BUG/MINOR: ssl: don't initialize the keylog callback when not required
- BUILD: Makefile: enable USE_SHM_OPEN by default on freebsd
- BUG/MEDIUM: peers: messages about unkown tables not correctly ignored
- MINOR: cfgparse: Always check the section position
- MEDIUM: thread: Restric nbthread/thread-group(s) to very first global sections
- BUILD: peers: Remove unused variables
- MINOR: ncbuf: complete doc for ncb_advance()
- BUG/MEDIUM: quic: fix unsuccessful handshakes on ncb_advance error
- BUG/MEDIUM: quic: fix memleak for out-of-order crypto data
- MINOR: quic: complete traces/debug for handshake
Let's keep these notes as references for later use. Polling on connect()
can sometimes return a few unexpected state combinations that such tests
illustrate. They can serve as reminders for special error handling.
The exact same commands as those from the CLI may be pre-loaded at boot
time by passing them one per line after the "trace" keyword in the global
section; i.e. just copy-pasting all commands directly there will do the
job. Note that if a ring is mentioned, it needs to be declared before the
global section. Another option is to append another global section after
"ring".
For now the keyword is marked as experimental to discourage its broad
adoption by default. "expose-experimental-directives" needs to be placed
in the global section to expose it.
the global section keywords were seriously misordered, and it's visible
that some mistakes have induced other ones over time, so it was about
time to fix this. Roughly 20% of the keywords were misplaced.
This commit only reordered the keywords index and their description,
nothing else was changed. It might be backported because it's a real
pain to find certain options there.
Once in a while we spot a bug in the deinit code that is complex,
especially when it has to deal with incomplete initializations, and the
ability to bypass this step has regularly been raised. In addition for
fast-reloading setups it could theoretically save some time. Tests have
shown that very large configs can barely save ~100-150ms by skipping the
deinit step. However the ability not to crash if a bug is encountered can
occasionally help.
This patch adds an option to do exactly this. It's obviously not enabled
by default and the documentation discourages from using it, but this might
be useful in the future.
The ca-ignore-err and crt-ignore-err directives are now able to use the
openssl X509_V_ERR constant names instead of the numerical values.
This allow a configuration to survive an OpenSSL upgrade, because the
numerical ID can change between versions. For example
X509_V_ERR_INVALID_CA was 24 in OpenSSL 1 and is 79 in OpenSSL 3.
The list of errors must be updated when a new major OpenSSL version is
released.
Add "shards" new keyword for "peers" section to configure the number
of peer shards attached to such secions. This impact all the stick-tables
attached to the section.
Add "shard" new "server" parameter to configure the peers which participate to
all the stick-tables contents distribution. Each peer receive the stick-tables updates
only for keys with this shard value as distribution hash. The "shard" value
is stored in ->shard new server struct member.
cfg_parse_peers() which is the function which is called to parse all
the lines of a "peers" section is modified to parse the "shards" parameter
stored in ->nb_shards new peers struct member.
Add srv_parse_shard() new callback into server.c to pare the "shard"
parameter.
Implement stksess_getkey_hash() to compute the distribution hash for a
stick-table key as the 64-bits xxhash of the key concatenated to the stick-table
name. This function is called by stksess_setkey_shard(), itself
called by the already implemented function which create a new stick-table
key (stksess_new()).
Add ->idlen new stktable struct member to store the stick-table name length
to not have to compute it each time a stick-table key hash is computed.
Released version 2.7-dev8 with the following main changes :
- BUG/MINOR: checks: update pgsql regex on auth packet
- DOC: config: Fix pgsql-check documentation to make user param mandatory
- CLEANUP: mux-quic: remove usage of non-standard ull type
- CLEANUP: quic: remove global var definition in quic_tls header
- BUG/MINOR: quic: adjust quic_tls prototypes
- CLEANUP: quic: fix headers
- CLEANUP: quic: remove unused function prototype
- CLEANUP: quic: remove duplicated varint code from xprt_quic.h
- CLEANUP: quic: create a dedicated quic_conn module
- BUG/MINOR: mux-quic: ignore STOP_SENDING for locally closed stream
- BUG/MEDIUM: lua: Don't crash in hlua_lua2arg_check on failure
- BUG/MEDIUM: lua: handle stick table implicit arguments right.
- BUILD: h1: silence an initiialized warning with gcc-4.7 and -Os
- MINOR: fd: add a new function to only raise RLIMIT_NOFILE
- MINOR: init: do not try to shrink existing RLIMIT_NOFIlE
- BUG/MINOR: http-fetch: Update method after a prefetch in smp_fetch_meth()
- BUILD: http_fetch: silence an uninitiialized warning with gcc-4/5/6 at -Os
- BUG/MINOR: hlua: hlua_channel_insert_data() behavior conflicts with documentation
- MINOR: quic: limit usage of ssl_sock_ctx in favor of quic_conn
- MINOR: mux-quic: check quic-conn return code on Tx
- CLEANUP: quic: fix indentation
- MEDIUM: quic: retrieve frontend destination address
- CLEANUP: Reapply ist.cocci (2)
- CLEANUP: Reapply strcmp.cocci
- CLEANUP: quic/receiver: remove the now unused tx_qring list
- BUG/MINOR: quic: set IP_PKTINFO socket option for QUIC receivers only
- MINOR: hlua: some luaL_checktype() calls were not guarded with MAY_LJMP
- DOC: configuration: missing 'if' in tcp-request content example
- MINOR: hlua: removing ambiguous lua_pushvalue with 0 index
- BUG/MAJOR: stick-tables: do not try to index a server name for applets
- MINOR: plock: support disabling exponential back-off
- MINOR: freq_ctr: use the thread's local time whenever possible
- MEDIUM: stick-table: switch the table lock to rwlock
- MINOR: stick-table: do not take an exclusive lock when downing ref_cnt
- MINOR: stick-table: move the write lock inside stktable_touch_with_exp()
- MEDIUM: stick-table: only take the lock when needed in stktable_touch_with_exp()
- MEDIUM: stick-table: make stksess_kill_if_expired() avoid the exclusive lock
- MEDIUM: stick-table: return inserted entry in __stktable_store()
- MEDIUM: stick-table: free newly allocated stkess if it couldn't be inserted
- MEDIUM: stick-table: switch to rdlock in stktable_lookup() and lookup_key()
- MEDIUM: stick-table: make stktable_get_entry() look up under a read lock
- MEDIUM: stick-table: do not take a lock to update t->current anymore.
- MEDIUM: stick-table: make stktable_set_entry() look up under a read lock
- MEDIUM: stick-table: requeue the expiration task out of the exclusive lock
- MINOR: stick-table: split stktable_store() between key and requeue
- MEDIUM: stick-table: always use atomic ops to requeue the table's task
- MEDIUM: stick-table: requeue the wakeup task out of the write lock
- BUG/MINOR: stick-table: fix build with DEBUG_THREAD
- REORG: mux-fcgi: Extract flags and enums into mux_fcgi-t.h
- MINOR: flags/mux-fcgi: Decode FCGI connection and stream flags
- BUG/MEDIUM: mux-h1: Add connection error handling when reading/sending on a pipe
- BUG/MEDIUM: mux-h1: Handle abort with an incomplete message during parsing
- BUG/MINOR: server: make sure "show servers state" hides private bits
- MINOR: checks: use the lighter PRNG for spread checks
- MEDIUM: checks: spread the checks load over random threads
- CI: SSL: use proper version generating when "latest" semantic is used
- CI: SSL: temporarily stick to LibreSSL=3.5.3
- MINOR: quic: New quic_cstream object implementation
- MINOR: quic: Extract CRYPTO frame parsing from qc_parse_pkt_frms()
- MINOR: quic: Use a non-contiguous buffer for RX CRYPTO data
- BUG/MINOR: quic: Stalled 0RTT connections with big ClientHello TLS message
- MINOR: quic: Split the secrets key allocation in two parts
- CLEANUP: quic: remove unused rxbufs member in receiver
- CLEANUP: quic: improve naming for rxbuf/datagrams handling
- MINOR: quic: implement datagram cleanup for quic_receiver_buf
- MINOR: ring: ring_cast_from_area() cast from an allocated area
- MINOR: buffers: split b_force_xfer() into b_cpy() and b_force_xfer()
- MINOR: logs: startup-logs can use a shm for logging the reload
- MINOR: mworker/cli: reload command displays the startup-logs
- MEDIUM: quic: respect the threads assigned to a bind line
- DOC: management: update the "reload" command of the master CLI
- BUILD: ssl_sock: bind_conf uninitialized in ssl_sock_bind_verifycbk()
- BUG/MEDIUM: httpclient: Don't set EOM flag on an empty HTX message
- MINOR: httpclient/lua: Don't set req_payload callback if body is empty
- DOC/CLEANUP: lua-api: some minor corrections
- DOC: lua-api: updating toolbox link
- DOC/CLEANUP: lua-api: removing duplicate core.proxies attribute
- DOC: management: add forgotten "show startup-logs"
- DOC: management: "show startup-logs" for master CLI
- CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in matrix.py
- CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in workflow definition
Link to lua toolbox was dead (project has been deprecated).
Adding a legacy link to get old toolbox source code as well as
a link to luarocks that seems to have superseded it.
An example given for tcp-request content rule with lua
was missing 'if' keyword. Using it "as is" makes haproxy unhappy.
The example was introduced with 579d83b05.
So it may be backported as far as 1.6, but it is a really minor typo.
The username is required in the Start-up message. Thus, since the 2.2, when
this health-check was refactored, the user parameter is mandatory. On prior
versions, when no username is provided, no pgsql check is performed but only
a basic tcpcheck.
This patch should be backported as far as 2.2.
Released version 2.7-dev7 with the following main changes :
- BUG/MEDIUM: mux-quic: fix nb_hreq decrement
- CLEANUP: httpclient: deleted unused variables
- MINOR: httpclient: enabled the use of SNI presets
- OPTIM: hpack-huff: reduce the cache footprint of the huffman decoder
- BUG/MINOR: mux-quic: do not keep detached qcs with empty Tx buffers
- REORG: mux-quic: extract traces in a dedicated source file
- REORG: mux-quic: export HTTP related function in a dedicated file
- MINOR: mux-quic: refactor snd_buf
- BUG/MEDIUM: mux-quic: properly trim HTX buffer on snd_buf reset
- BUG/MINOR: mux-h1: Account consumed output data on synchronous connection error
- BUG/MINOR: log: improper behavior when escaping log data
- CLEANUP: tools: removing escape_chunk() function
- MINOR: clock: split local and global date updates
- MINOR: pollers: only update the local date during busy polling
- MINOR: clock: do not update the global date too often
- REGTESTS: 4be_1srv_smtpchk_httpchk_layer47errors: Return valid SMTP replies
- MINOR: smtpchk: Update expect rule to fully match replies to EHLO commands
- BUG/MINOR: smtpchk: SMTP Service check should gracefully close SMTP transaction
- MINOR: list: documenting mt_list_for_each_entry_safe() macro
- CLEANUP: list: Fix mt_list_for_each_entry_safe indentation
- BUG/MINOR: hlua: Remove \n in Lua error message built with memprintf
- MINOR: hlua: Allow argument on lua-lod(-per-thread) directives
- BUG/MINOR: anon: memory illegal accesses in tools.c with hash_anon and hash_ipanon
- MEDIUM: mworker/cli: keep the connection of the FD that ask for a reload
- BUG/MINOR: hlua: fixing ambiguous sizeof in hlua_load_per_thread
- MINOR: mworker/cli: replace close() by fd_delete()
- MINOR: mworker: store and shows loading status
- MINOR: mworker: mworker_cli_proxy_new_listener() returns a bind_conf
- MINOR: mworker: stores the mcli_reload bind_conf
- MINOR: mworker/cli: the mcli_reload bind_conf only send the reload status
- DOC: management: describe the new reload command behavior
- CLEANUP: list: fix again some style issues in the recent comments
- BUG/MINOR: stream: Perform errors handling in right order in stream_new()
- BUG/MEDIUM: stconn: Reset SE descriptor when we fail to create a stream
- BUG/MEDIUM: resolvers: Remove aborted resolutions from query_ids tree
- DOC: management: add timeout on the "reload" command
- BUG/MINOR: ring: fix the size check in ring_make_from_area()
- BUG/MINOR: config: don't count trailing spaces as empty arg
- Revert "BUG/MINOR: config: don't count trailing spaces as empty arg"
- BUG/MINOR: hlua: fixing hlua_http_msg_del_data behavior
- BUG/MINOR: hlua: fixing hlua_http_msg_insert_data behavior
- MINOR: cli: Add anonymization on a missed element for 'show sess all'
- MINOR: cli: remove error message with 'set anon on|off'
- MINOR: tools: modify hash_ipanon in order to use it in cli
- MINOR: cli: use hash_ipanon to anonymized address
- MINOR: cli: Add an anonymization on a missed element in 'show server state'
- MINOR: config: correct errors about argument number in condition in cfgparse.c
- MINOR: config: Add other keywords when dump the anonymized configuration file
- MINOR: config: Add option line when the configuration file is dumped
- MINOR: cli: correct commentary and replace 'set global-key' name
- MINOR: tools: Impprove hash_ipanon to support dgram sockets and port offsets
- MINOR: tools: Impprove hash_ipanon to not hash FD-based addresses
- BUG/MINOR: hlua: _hlua_http_msg_delete incorrect behavior when offset is used
- DOC: management: httpclient can resolve server names in URLs
- BUG/MINOR: hlua: prevent crash when loading numerous arguments using lua-load(per-thread)
- DOC/CLEANUP: lua-api: removing duplicate date functions doc
- MINOR: hlua: ambiguous lua_pushvalue with 0 index
- BUG/MINOR: config: don't count trailing spaces as empty arg (v2)
- BUG/MEDIUM: config: count line arguments without dereferencing the output
- BUG/MAJOR: conn-idle: fix hash indexing issues on idle conns
- BUG/MINOR: config: insufficient syntax check of the global "maxconn" value
- BUG/MINOR: backend: only enforce turn-around state when not redispatching
Correct a commentary in in include/haproxy/global-t.h and include/haproxy/tools.h
Replace the CLI command 'set global-key <key>' by 'set anon global-key <key>' in
order to find it easily when you don't remember it, the recommandation can guide
you when you just tap 'set anon'.
No backport needed, except if anonymization mechanism is backported.
Released version 2.7-dev6 with the following main changes :
- MINOR: Revert part of clarifying samples support per os commit
- BUILD: makefile: enable crypt(3) for NetBSD
- BUG/MINOR: quic: Retransmitted frames marked as acknowledged
- BUG/MINOR: quic: Possible crash with "tls-ticket-keys" on QUIC bind lines
- MINOR: http-check: Remove support for headers/body in "option httpchk" version
- BUG/MINOR: h1: Support headers case adjustment for TCP proxies
- BUG/MINOR: quic: Possible crash when verifying certificates
- BUILD: quic: add some ifdef around the SSL_ERROR_* for libressl
- BUILD: ssl: fix ssl_sock_switchtx_cbk when no client_hello_cb
- BUILD: quic: temporarly ignore chacha20_poly1305 for libressl
- BUILD: quic: enable early data only with >= openssl 1.1.1
- BUILD: ssl: fix the ifdef mess in ssl_sock_initial_ctx
- BUILD: quic: fix the #ifdef in ssl_quic_initial_ctx()
- MINOR: quic: add QUIC support when no client_hello_cb
- MINOR: quic: Add traces about sent or resent TX frames
- MINOR: quic: No TRACE_LEAVE() in retrieve_qc_conn_from_cid()
- BUG/MINOR: quic: Wrong connection ID to thread ID association
- BUG/MINOR: task: always reset a new tasklet's call date
- BUG/MINOR: task: make task_instant_wakeup() work on a task not a tasklet
- MINOR: task: permanently enable latency measurement on tasklets
- CLEANUP: task: rename ->call_date to ->wake_date
- BUG/MINOR: sched: properly account for the CPU time of dying tasks
- MINOR: sched: store the current profile entry in the thread context
- BUG/MINOR: stream/sched: take into account CPU profiling for the last call
- MINOR: tasks: do not keep cpu and latency times in struct task
- MINOR: tools: add generic pointer hashing functions
- CLEANUP: activity: make memprof use the generic ptr_hash() function
- CLEANUP: activity: make taskprof use ptr_hash()
- MINOR: debug: add struct ha_caller to describe a calling location
- CLEANUP: debug: use struct ha_caller for memstat
- DEBUG: task: define a series of wakeup types for tasks and tasklets
- DEBUG: task: use struct ha_caller instead of arrays of file:line
- DEBUG: applet: instrument appctx_wakeup() to log the caller's location
- DEBUG: task: simplify the caller recording in DEBUG_TASK
- CLEANUP: task: move tid and wake_date into the common part
- CLEANUP: sched: remove duplicate code in run_tasks_from_list()
- CLEANUP: activity: make the number of sched activity entries more configurable
- DEBUG: resolvers: unstatify process_resolvers() to make it appear in profiling
- DEBUG: quic: export the few task handlers that often appear in task dumps
- MEDIUM: tasks/activity: combine the called function with the caller
- MINOR: tasks/activity: improve the caller-callee activity hash
- MINOR: activity/cli: support aggregating task profiling outputs
- MINOR: activity/cli: support sorting task profiling by total CPU time
- BUG/MINOR: signals/poller: set the poller timeout to 0 when there are signals
- BUG/MINOR: quic: Speed up the handshake completion only one time
- BUG/MINOR: quic: Trace fix about packet number space information.
- BUG/MINOR: h3: Crash when h3 trace verbosity is "minimal"
- MINOR: h3: Add the quic_conn object to h3 traces
- MINOR: h3: Missing connection argument for a TRACE_LEAVE() argument
- MINOR: h3: Send the h3 settings with others streams (requests)
- MINOR: dev/udp: Apply the corruption to both directions
- BUILD: udp-perturb: Add a make target for udp-perturb tool
- BUG/MINOR: signals/poller: ensure wakeup from signals
- CI: cirrus-ci: bump FreeBSD image to 13-1
- DEV: flags: fix usage message to reflect available options
- DEV: flags: add missing CO_FL_FDLESS connection flag
- MINOR: flags: add a new file to host flag dumping macros
- MINOR: flags: implement a macro used to dump enums inside masks
- MINOR: flags/channel: use flag dumping for channel flags and analysers
- MINOR: flags/connection: use flag dumping for connection flags
- MINOR: flags/stconn: use flag dumping for stconn and sedesc flags
- MINOR: flags/stream: use flag dumping for stream error type
- MINOR: flags/stream: use flag dumping for stream flags
- MINOR: flags/task: use flag dumping for task state
- MINOR: flags/http_ana: use flag dumping for txn flags
- DEV: flags: remove the now unused SHOW_FLAG() definition
- DEV: flags: remove the now useless intermediary functions
- MINOR: flags/htx: use flag dumping to show htx and start-line flags
- MINOR: flags/http_ana: use flag dumping to show http msg states
- BUG/MEDIUM: proxy: ensure pause_proxy() and resume_proxy() own PROXY_LOCK
- MINOR: listener: small API change
- MINOR: proxy/listener: support for additional PAUSED state
- BUG/MINOR: stats: fixing stat shows disabled frontend status as 'OPEN'
- BUILD: flags: fix build warning in some macros used by show_flags
- BUILD: flags: fix the fallback macros for missing stdio
- CLEANUP: pollers: remove dead code in the polling loop
- BUG/MINOR: mux-h1: Increment open_streams counter when H1 stream is created
- REGTESTS: healthcheckmail: Relax matching on the healthcheck log message
- CLEANUP: listener: function comment typo in stop_listener()
- BUG/MINOR: listener: null pointer dereference suspected by coverity
- MINOR: flags/fd: decode FD flags states
- REORG: mux-h2: extract flags and enums into mux_h2-t.h
- MINOR: flags/mux-h2: decode H2C and H2S flags
- REGTESTS: log: test the log-forward feature
- BUG/MEDIUM: sink: bad init sequence on tcp sink from a ring.
- REGTESTS: ssl/log: test the log-forward with SSL
- MEDIUM: httpclient: httpclient_create_proxy() creates a proxy for httpclient
- MEDIUM: httpclient: allow to use another proxy
- DOC: fix TOC in starter guide for subsection 3.3.8. Statistics
- MINOR: httpclient: export httpclient_create_proxy()
- MEDIUM: quic: separate path for rx and tx with set_encryption_secrets
- BUG/MEDIUM: mux-quic: fix crash on early app-ops release
- REORG: mux-h1: extract flags and enums into mux_h1-t.h
- MINOR: flags/mux-h1: decode H1C and H1S flags
- CLEANUP: mux-quic: remove stconn usage in h3/hq
- BUG/MINOR: mux-quic: do not remotely close stream too early
- CLEANUP: exclude udp-perturb with .gitignore
- BUG/MEDIUM: server: segv when adding server with hostname from CLI
- CLEANUP: quic,ssl: fix tiny typos in C comments
- BUG/MEDIUM: captures: free() an error capture out of the proxy lock
- BUILD: fd: fix a build warning on the DWCAS
- MINOR: anon: add new macros and functions to anonymize contents
- MINOR: anon: store the anonymizing key in the global structure
- MINOR: anon: store the anonymizing key in the CLI's appctx
- MINOR: cli: anonymize commands 'show sess' and 'show sess all'
- MINOR: cli: anonymize 'show servers state' and 'show servers conn'
- MINOR: config: add command-line -dC to dump the configuration file
- SCRIPTS: announce-release: update some URLs to https
This commit adds a new command line option -dC to dump the configuration
file. An optional key may be appended to -dC in order to produce an
anonymized dump using this key. The anonymizing process uses the same
algorithm as the CLI so that the same key will produce the same hashes
for the same identifiers. This way an admin may share an anonymized
extract of a configuration to match against live dumps. Note that key 0
will not anonymize the output. However, in any case, the configuration
is dumped after tokenizing, thus comments are lost.
In order to allow users to dump internal states using a specific key
without changing the global one, we're introducing a key in the CLI's
appctx. This key is preloaded from the global one when "set anon on"
is used (and if none exists, a random one is assigned). And the key
can optionally be assigned manually for the whole CLI session.
A "show anon" command was also added to show the anon state, and the
current key if the users has sufficient permissions. In addition, a
"debug dev hash" command was added to test the feature.
Add a uint32_t key in global to hash words with it. A new CLI command
'set global-key <key>' was added to change the global anonymizing key.
The global may also be set in the configuration using the global
"anonkey" directive. For now this key is not used.
This subsection has been moved from 3.4.9 to 3.3.8 somewhere along
2.4, but the TOC has not been updated - resulting in a invalid
anchor in the HTML version.
Needs to be backported to 2.4+
By default we now dump stats between caller and callee, but by
specifying "aggr" on the command line, stats get aggregated by
callee again as it used to be before the feature was available.
It may sometimes be helpful when comparing total call counts,
though that's about all.
This trick is deprecated since the health-check refactoring, It is now
invalid. It means the following line will trigger an error during the
configuration parsing:
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
It must be replaced by:
option httpchk OPTIONS * HTTP/1.1
http-check send hdr Host www
Released version 2.7-dev5 with the following main changes :
- BUG/MINOR: mux-quic: Fix memleak on QUIC stream buffer for unacknowledged data
- BUG/MEDIUM: cpu-map: fix thread 1's affinity affecting all threads
- MINOR: cpu-map: remove obsolete diag warning about combined ranges
- BUG/MAJOR: mworker: fix infinite loop on master with no proxies.
- REGTESTS: launch http_reuse_always in mworker mode
- BUG/MINOR: quix: Memleak for non in flight TX packets
- BUG/MINOR: quic: Wrong list_for_each_entry() use when building packets from qc_do_build_pkt()
- BUG/MINOR: quic: Safer QUIC frame builders
- MINOR: quic: Replace MT_LISTs by LISTs for RX packets.
- BUG/MEDIUM: applet: fix incorrect check for abnormal return condition from handler
- BUG/MINOR: applet: make the call_rate only count the no-progress calls
- MEDIUM: peers: limit the number of updates sent at once
- BUILD: tcp_sample: fix build of get_tcp_info() on OpenBSD
- BUG/MINOR: resolvers: return the correct value in resolvers_finalize_config()
- BUG/MINOR: mworker: does not create the "default" resolvers in wait mode
- BUG/MINOR: tcpcheck: Disable QUICKACK only if data should be sent after connect
- REGTESTS: Fix prometheus script to perform HTTP health-checks
- MINOR: resolvers: shut the warning when "default" resolvers is implicit
- Revert "BUG/MINOR: quix: Memleak for non in flight TX packets"
- BUG/MINOR: quic: Leak in qc_release_lost_pkts() for non in flight TX packets
- BUG/MINOR: quic: Stalled connections (missing I/O handler wakeup)
- CLEANUP: quic: No more use ->rx_list MT_LIST entry point (quic_rx_packet)
- CLEANUP: quic: Remove a useless check in qc_lstnr_pkt_rcv()
- MINOR: quic: Remove useless traces about references to TX packets
- Revert "MINOR: quic: Remove useless traces about references to TX packets"
- DOC: configuration: do-resolve doesn't work with a port in the string
- MINOR: sample: add the host_only and port_only converters
- BUG/MINOR: httpclient: fix resolution with port
- DOC: configuration.txt: do-resolve must use host_only to remove its port.
- BUG/MINOR: quic: Null packet dereferencing from qc_dup_pkt_frms() trace
- BUG/MINOR: quic: Frames added to packets even if not built.
- BUG/MEDIUM: spoe: Properly update streams waiting for a ACK in async mode
- BUG/MEDIUM: peers: Add connect and server timeut to peers proxy
- BUG/MEDIUM: peers: Don't use resync timer when local resync is in progress
- BUG/MEDIUM: peers: Don't start resync on reload if local peer is not up-to-date
- BUG/MINOR: hlua: Rely on CF_EOI to detect end of message in HTTP applets
- BUG/MEDIUM: mux-h1: do not refrain from signaling errors after end of input
- BUG/MINOR: epoll: do not actively poll for Rx after an error
- MINOR: raw-sock: don't try to send if an error was already reported
- BUG/MINOR: quic: Missing header protection AES cipher context initialisations (draft-v2)
- MINOR: quic: Add a trace to distinguish the datagram from the packets inside
- BUG/MINOR: ssl: fix deinit of the ca-file tree
- BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free()
- BUG/MINOR: tcpcheck: Disable QUICKACK for default tcp-check (with no rule)
- BUG/MEDIUM: ssl: Fix a UAF when old ckch instances are released
- BUG/MINOR: ssl: revert two wrong fixes with ckhi_link
- BUG/MINOR: dev/udp: properly preset the rx address size
- BUILD: debug: make sure debug macros are never empty
- MINOR: quic: Move traces about RX/TX bytes from QUIC_EV_CONN_PRSAFRM event
- BUG/MINOR: quic: TX frames memleak
- BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free() v2
- MINOR: sink/ring: rotate non-empty file-backed contents only
- BUG/MINOR: regex: Properly handle PCRE2 lib compiled without JIT support
- REGTESTS: http_request_buffer: Add a barrier to not mix up log messages
- BUG/MEDIUM: mux-h1: always use RST to kill idle connections in pools
- MINOR: backend: always satisfy the first req reuse rule with l7 retries
- BUG/MINOR: quic: Do not ack when probing
- MINOR: quic: Add TX frames addresses to traces to several trace events
- MINOR: quic: Trace typo fix in qc_release_frm()
- BUG/MINOR: quic: Frames leak during retransmissions
- BUG/MINOR: h2: properly set the direction flag on HTX response
- BUG/MEDIUM: httpclient: always detach the caller before self-killing
- BUG/MINOR: httpclient: only ask for more room on failed writes
- BUG/MINOR: httpclient: keep-alive was accidentely disabled
- MEDIUM: httpclient: enable ALPN support on outgoing https connections
- BUG/MINOR: mux-h2: fix the "show fd" dest buffer for the subscriber
- BUG/MINOR: mux-h1: fix the "show fd" dest buffer for the subscriber
- BUG/MINOR: mux-fcgi: fix the "show fd" dest buffer for the subscriber
- DEBUG: stream: minor rearrangement of a few fields in struct stream.
- MINOR: debug: report applet pointer and handler in crashes when known
- MINOR: mux-h2: extract the stream dump function out of h2_show_fd()
- MINOR: mux-h2: extract the connection dump function out of h2_show_fd()
- MINOR: muxes: add a "show_sd" helper to complete "show sess" dumps
- MINOR: mux-h2: provide a "show_sd" helper to output stream debugging info
- MINOR: mux-h2: insert line breaks in "show sess all" output for legibility
- MINOR: mux-quic: provide a "show_sd" helper to output stream debugging info
- MINOR: mux-h1: split "show_fd" into connection and stream
- MINOR: mux-h1: provide a "show_sd" helper to output stream debugging info
- BUG/MINOR: http-act: initialize http fmt head earlier
If the service is rechecked before a reload, that may cause the config
to be parsed twice and file-backed rings to be lost.
Here we make sure that such a ring does contain information before
deciding to rotate it. This way the first process starting after some
writes will cause a rotate but not subsequent ones until new writes
are applied.
An attempt was also made to disable rotations on checks but this was a
bad idea, as the ring is still initialized and this causes the contents
to be lost. The choice of initializing the ring during parsing is
questionable but the config check ought to be as close as possible to a
real start, and we could imagine that the ring is used by some code
during startup (e.g. lua). So this approach was abandonned and config
checks also cause a rotation, as the purpose of this rotation is to
preserve latest information against accidental removal.
Fix the documentation about do-resolve to handle the case where a port
is associated to the hostname in the Host header.
Must be backported as far as 2.0.
As seen in GH issue #1770, peers synchronization do not cope well with
very large buffers because by default the only two reasons for stopping
the processing of updates is either that the end was reached or that
the buffer is full. This can cause high latencies, and even rightfully
trigger the watchdog when the operations are numerous and slowed down
by competition on the stick-table lock.
This patch introduces a limit to the number of messages one may send
at once, which now defaults to 200, regardless of the buffer size. This
means taking and releasing the lock up to 400 times in a row, which is
costly enough to let some other parts work.
After some observation this could be backported to 2.6. If so, however,
previous commits "BUG/MEDIUM: applet: fix incorrect check for abnormal
return condition from handler" and "BUG/MINOR: applet: make the call_rate
only count the no-progress calls" must be backported otherwise the call
rate might trigger the looping protection.
Released version 2.7-dev4 with the following main changes :
- BUG/MEDIUM: quic: Wrong packet length check in qc_do_rm_hp()
- MINOR: quic: Too much useless traces in qc_build_frms()
- BUG/MEDIUM: quic: Missing AEAD TAG check after removing header protection
- MINOR: quic: Replace pool_zalloc() by pool_malloc() for fake datagrams
- MINOR: debug: make the mem_stats section aligned to void*
- MINOR: debug: store and report the pool's name in struct mem_stats
- MINOR: debug: also store the function name in struct mem_stats
- MINOR: debug/memstats: automatically determine first column size
- MINOR: debug/memstats: permit to pass the size to free()
- CLEANUP: mux-quic: remove loop on sending frames
- MINOR: quic: replace custom buf on Tx by default struct buffer
- MINOR: quic: release Tx buffer on each send
- MINOR: quic: refactor datagram commit in Tx buffer
- MINOR: quic: skip sending if no frame to send in io-cb
- BUG/MINOR: mux-quic: open stream on STOP_SENDING
- BUG/MINOR: quic: fix crash on handshake io-cb for null next enc level
- BUG/MEDIUM: quic: always remove the connection from the accept list on close
- BUG/MEDIUM: poller: use fd_delete() to release the poller pipes
- BUG/MEDIUM: task: relax one thread consistency check in task_unlink_wq()
- MEDIUM: quic: xprt traces rework
- BUILD: stconn: fix build warning at -O3 about possible null sc
- MINOR: quic: Remove useless lock for RX packets
- BUG/MINOR: quic: Possible infinite loop in quic_build_post_handshake_frames()
- CLEANUP: quic: Remove trailing spaces
- MINOR: mux-quic: adjust enter/leave traces
- MINOR: mux-quic: define protocol error traces
- CLEANUP: mux-quic: adjust traces level
- MINOR: mux-quic: define new traces
- BUG/MEDIUM: mux-quic: fix crash due to invalid trace arg
- BUG/MEDIUM: quic: Possible use of uninitialized <odcid> variable in qc_lstnr_params_init()
- BUG/MEDIUM: ring: fix too lax 'size' parser
- BUG/MEDIUM: quic: Wrong use of <token_odcid> in qc_lsntr_pkt_rcv()
- BUILD: ring: forward-declare struct appctx to avoid a build warning
- MINOR: ring: support creating a ring from a linear area
- MINOR: ring: add support for a backing-file
- DEV: haring: add a simple utility to read file-backed rings
- DEV: haring: support remapping LF in contents with CR VT
- BUG/MINOR: quic: memleak on wrong datagram receipt
- BUILD: sink: replace S_IRUSR, S_IWUSR with their octal value
- MINOR: ring: archive a previous file-backed ring on startup
- BUG/MINOR: mux-quic: fix crash with traces in qc_detach()
- BUG/MINOR: quic: MIssing check when building TX packets
- BUG/MINOR: quic: Wrong status returned by qc_pkt_decrypt()
- MINOR: memprof: export the minimum definitions for memory profiling
- MINOR: pool/memprof: report pool alloc/free in memory profiling
- MINOR: pools/memprof: store and report the pool's name in each bin
- MINOR: chunk: inline alloc_trash_chunk()
- MINOR: stick-table: Add table_expire() and table_idle() new converters
- CLEANUP: exclude haring with .gitignore
- MINOR: quic: adjust quic_frame flag manipulation
- MINOR: h3: report error on control stream close
- MINOR: qpack: report error on enc/dec stream close
- BUG/MEDIUM: mux-quic: reject uni stream ID exceeding flow control
- MINOR: mux-quic: adjust traces on stream init
- MINOR: mux-quic: add missing args on some traces
- MINOR: quic: refactor application send
- BUG/MINOR: quic: do not notify MUX on frame retransmit
- BUG/MEDIUM: http-ana: fix crash or wrong header deletion by http-restrict-req-hdr-names
- BUG/MINOR: quic: Missing initializations for ducplicated frames.
- BUG/MEDIUM: quic: fix crash on MUX send notification
- REORG: h2: extract cookies concat function in http_htx
- REGTESTS: add test for HTTP/2 cookies concatenation
- MEDIUM: h3: concatenate multiple cookie headers
- MINOR: applet: add a function to reset the svcctx of an applet
- BUG/MEDIUM: cli: always reset the service context between commands
- BUG/MEDIUM: mux-h2: do not fiddle with ->dsi to indicate demux is idle
- MINOR: mux-h2/traces: report transition to SETTINGS1 before not after
- MINOR: mux-h2: make streams know if they need to send more data
- BUG/MINOR: mux-h2: send a CANCEL instead of ES on truncated writes
- BUG/MINOR: quic: Possible crashes when dereferencing ->pkt quic_frame struct member
- MINOR: quic: Add frame addresses to QUIC_EV_CONN_PRSAFRM event traces
- BUG/MINOR: quic: Wrong splitted duplicated frames handling
- MINOR: quic: Add the QUIC connection to mux traces
- MINOR: quic: Trace fix in qc_release_frm()
- BUG/MAJOR: log-forward: Fix log-forward proxies not fully initialized
- BUG/MAJOR: log-forward: Fix ssl layer not initialized on bind even if configured
- MINOR: quic: Add reusable cipher contexts for header protection
- BUG/MINOR: ssl/cli: error when the ca-file is empty
- MINOR: ssl: handle ca-file appending in cafile_entry
- MINOR: ssl/cli: implement "add ssl ca-file"
In ticket #1805 an user is impacted by the limitation of size of the CLI
buffer when updating a ca-file.
This patch allows a user to append new certificates to a ca-file instead
of trying to put them all with "set ssl ca-file"
The implementation use a new function ssl_store_dup_cafile_entry() which
duplicates a cafile_entry and its X509_STORE.
ssl_store_load_ca_from_buf() was modified to take an apped parameter so
we could share the function for "set" and "add".
table_expire() returns the expiration delay for a stick-table entry associated
to an input sample. Its counterpart table_idle() returns the time the entry
remained idle since the last time it was updated.
Both converters may take a default value as second argument which is returned
when the entry is not present.
In order to ensure that an instant restart of the process will not wipe
precious debugging information, and to leave time for an admin to archive
a copy of a ring, now upon startup, any previously existing file will be
renamed with the extra suffix ".bak", and any previously existing file
with suffix ".bak" will be removed.
This mmaps a file which will serve as the backing-store for the ring's
contents. The idea is to provide a way to retrieve sensitive information
(last logs, debugging traces) even after the process stops and even after
a possible crash. Right now this was possible by connecting to the CLI
and dumping the contents of the ring live, but this is not handy and
consumes quite a bit of resources before it is needed.
With a backing file, the ring is effectively RAM-mapped file, so that
contents stored there are the same as those found in the file (the OS
doesn't guarantee immediate sync but if the process dies it will be OK).
Note that doing that on a filesystem backed by a physical device is a
bad idea, as it will induce slowdowns at high loads. It's really
important that the device is RAM-based.
Also, this may have security implications: if the file is corrupted by
another process, the storage area could be corrupted, causing haproxy
to crash or to overwrite its own memory. As such this should only be
used for debugging.
Released version 2.7-dev3 with the following main changes :
- BUILD: makefile: Fix install(1) handling for OpenBSD/NetBSD/Solaris/AIX
- BUG/MEDIUM: tools: avoid calling dlsym() in static builds (try 2)
- MINOR: resolvers: resolvers_destroy() deinit and free a resolver
- BUG/MINOR: resolvers: shut off the warning for the default resolvers
- BUG/MINOR: ssl: allow duplicate certificates in ca-file directories
- BUG/MINOR: tools: fix statistical_prng_range()'s output range
- BUG/MINOR: quic: do not send CONNECTION_CLOSE_APP in initial/handshake
- BUILD: debug: Add braces to if statement calling only CHECK_IF()
- BUG/MINOR: fd: Properly init the fd state in fd_insert()
- BUG/MEDIUM: fd/threads: fix incorrect thread selection in wakeup broadcast
- MINOR: init: load OpenSSL error strings
- MINOR: ssl: enhance ca-file error emitting
- BUG/MINOR: mworker/cli: relative pid prefix not validated anymore
- BUG/MAJOR: mux_quic: fix invalid PROTOCOL_VIOLATION on POST data overlap
- BUG/MEDIUM: mworker: proc_self incorrectly set crashes upon reload
- BUILD: add detection for unsupported compiler models
- BUG/MEDIUM: stconn: Only reset connect expiration when processing backend side
- BUG/MINOR: backend: Fallback on RR algo if balance on source is impossible
- BUG/MEDIUM: master: force the thread count earlier
- BUG/MAJOR: poller: drop FD's tgid when masks don't match
- DEBUG: fd: detect possibly invalid tgid in fd_insert()
- BUG/MINOR: sockpair: wrong return value for fd_send_uxst()
- MINOR: sockpair: move send_fd_uxst() error message in caller
- Revert "BUG/MINOR: peers: set the proxy's name to the peers section name"
- DEBUG: fd: split the fd check
- MEDIUM: resolvers: continue startup if network is unavailable
- BUG/MINOR: fd: always remove late updates when freeing fd_updt[]
- MINOR: cli: emit a warning when _getsocks was used more than once
- BUG/MINOR: mworker: PROC_O_LEAVING used but not updated
- Revert "MINOR: cli: emit a warning when _getsocks was used more than once"
- MINOR: cli: warning on _getsocks when socket were closed
- BUG/MEDIUM: mux-quic: fix missing EOI flag to prevent streams leaks
- MINOR: quic: Congestion control architecture refactoring
- MEDIUM: quic: Cubic congestion control algorithm implementation
- MINOR: quic: New "quic-cc-algo" bind keyword
- BUG/MINOR: quic: loss time limit variable computed but not used
- MINOR: quic: Stop looking for packet loss asap
- BUG/MAJOR: quic: Useless resource intensive loop qc_ackrng_pkts()
- MINOR: quic: Send packets as much as possible from qc_send_app_pkts()
- BUG/MEDIUM: queue/threads: limit the number of entries dequeued at once
- MAJOR: threads/plock: update the embedded library
- MINOR: thread: provide an alternative to pthread's rwlock
- DEBUG: tools: provide a tree dump function for ebmbtrees as well
- MINOR: ebtree: add ebmb_lookup_shorter() to pursue lookups
- BUG/MEDIUM: pattern: only visit equivalent nodes when skipping versions
- BUG/MINOR: mux-quic: prevent crash if conn released during IO callback
- CLEANUP: mux-quic: remove useless app_ops is_active callback
- BUG/MINOR: mux-quic: do not free conn if attached streams
- MINOR: mux-quic: save proxy instance into qcc
- MINOR: mux-quic: use timeout server for backend conns
- MEDIUM: mux-quic: adjust timeout refresh
- MINOR: mux-quic: count in-progress requests
- MEDIUM: mux-quic: implement http-keep-alive timeout
- MINOR: peers: Add a warning about incompatible SSL config for the local peer
- MINOR: peers: Use a dedicated reconnect timeout when stopping the local peer
- BUG/MEDIUM: peers: limit reconnect attempts of the old process on reload
- BUG/MINOR: peers: Use right channel flag to consider the peer as connected
- BUG/MEDIUM: dns: Properly initialize new DNS session
- BUG/MINOR: backend: Don't increment conn_retries counter too early
- MINOR: server: Constify source server to copy its settings
- REORG: server: Export srv_settings_cpy() function
- BUG/MEDIUM: proxy: Perform a custom copy for default server settings
- BUG/MINOR: quic: Missing in flight ack eliciting packet counter decrement
- BUG/MEDIUM: quic: Floating point exception in cubic_root()
- MINOR: h3: support HTTP request framing state
- MINOR: mux-quic: refresh timeout on frame decoding
- MINOR: mux-quic: refactor refresh timeout function
- MEDIUM: mux-quic: implement http-request timeout
- BUG/MINOR: quic: Avoid sending truncated datagrams
- BUG/MINOR: ring/cli: fix a race condition between the writer and the reader
- BUG/MEDIUM: sink: Set the sink ref for forwarders created during ring parsing
- BUG/MINOR: sink: fix a race condition between the writer and the reader
- BUG/MINOR: quic: do not reject datagrams matching minimum permitted size
- MINOR: quic: Add two new stats counters for sendto() errors
- BUG/MINOR: quic: Missing Initial packet dropping case
- MINOR: quic: explicitely ignore sendto error
- BUG/MINOR: quic: adjust errno handling on sendto
- BUG/MEDIUM: quic: break out of the loop in quic_lstnr_dghdlr
- MINOR: threads: report the number of thread groups in build options
- MINOR: config: automatically preset MAX_THREADS based on MAX_TGROUPS
- BUILD: SSL: allow to pass additional configure args to QUICTLS
- CI: enable weekly "m32" builds on x86_64
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MEDIUM: fix DH length when EC key is used
- REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
- REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
- REGTESTS: ssl: fix grep invocation to use extended regex in ssl_generate_certificate.vtc
- BUILD: cfgparse: always defined _GNU_SOURCE for sched.h and crypt.h
As it could be interesting to be able to choose the QUIC control congestion
algorithm to be used by listener, add "quic-cc-algo" new keyword to do so.
Update the documentation consequently.
Must be backported to 2.6.
Released version 2.7-dev2 with the following main changes :
- BUG/MINOR: qpack: fix build with QPACK_DEBUG
- MINOR: h3: handle errors on HEADERS parsing/QPACK decoding
- BUG/MINOR: qpack: abort on dynamic index field line decoding
- MINOR: qpack: properly handle invalid dynamic table references
- MINOR: task: Add tasklet_wakeup_after()
- BUG/MINOR: quic: Dropped packets not counted (with RX buffers full)
- MINOR: quic: Add new stats counter to diagnose RX buffer overrun
- MINOR: quic: Duplicated QUIC_RX_BUFSZ definition
- MINOR: quic: Improvements for the datagrams receipt
- CLEANUP: h2: Typo fix in h2_unsubcribe() traces
- MINOR: quic: Increase the QUIC connections RX buffer size (upto 64Kb)
- CLEANUP: mux-quic: adjust comment on qcs_consume()
- MINOR: ncbuf: implement ncb_is_fragmented()
- BUG/MINOR: mux-quic: do not signal FIN if gap in buffer
- MINOR: fd: add a new FD_DISOWN flag to prevent from closing a deleted FD
- BUG/MEDIUM: ssl/fd: unexpected fd close using async engine
- MINOR: tinfo: make tid temporarily still reflect global ID
- CLEANUP: config: remove unused proc_mask()
- MINOR: debug: remove mask support from "debug dev sched"
- MEDIUM: task: add and preset a thread ID in the task struct
- MEDIUM: task/debug: move the ->thread_mask integrity checks to ->tid
- MAJOR: task: use t->tid instead of ffsl(t->thread_mask) to take the thread ID
- MAJOR: task: replace t->thread_mask with 1<<t->tid when thread mask is needed
- CLEANUP: task: remove thread_mask from the struct task
- MEDIUM: applet: only keep appctx_new_*() and drop appctx_new()
- MEDIUM: task: only keep task_new_*() and drop task_new()
- MINOR: applet: always use task_new_on() on applet creation
- MEDIUM: task: remove TASK_SHARED_WQ and only use t->tid
- MINOR: task: replace task_set_affinity() with task_set_thread()
- CLEANUP: task: remove the unused task_unlink_rq()
- CLEANUP: task: remove the now unused TASK_GLOBAL flag
- MINOR: task: make rqueue_ticks atomic
- MEDIUM: task: move the shared runqueue to one per thread
- MEDIUM: task: replace the global rq_lock with a per-rq one
- MINOR: task: remove grq_total and use rq_total instead
- MINOR: task: replace global_tasks_mask with a check for tree's emptiness
- MEDIUM: task: use regular eb32 trees for the run queues
- MEDIUM: queue: revert to regular inter-task wakeups
- MINOR: thread: make wake_thread() take care of the sleeping threads mask
- MINOR: thread: move the flags to the shared cache line
- MINOR: thread: only use atomic ops to touch the flags
- MINOR: poller: centralize poll return handling
- MEDIUM: polling: make update_fd_polling() not care about sleeping threads
- MINOR: poller: update_fd_polling: wake a random other thread
- MEDIUM: thread: add a new per-thread flag TH_FL_NOTIFIED to remember wakeups
- MEDIUM: tasks/fd: replace sleeping_thread_mask with a TH_FL_SLEEPING flag
- MINOR: tinfo: add the tgid to the thread_info struct
- MINOR: tinfo: replace the tgid with tgid_bit in tgroup_info
- MINOR: tinfo: add the mask of enabled threads in each group
- MINOR: debug: use ltid_bit in ha_thread_dump()
- MINOR: wdt: use ltid_bit in wdt_handler()
- MINOR: clock: use ltid_bit in clock_report_idle()
- MINOR: thread: use ltid_bit in ha_tkillall()
- MINOR: thread: add a new all_tgroups_mask variable to know about active tgroups
- CLEANUP: thread: remove thread_sync_release() and thread_sync_mask
- MEDIUM: tinfo: add a dynamic thread-group context
- MEDIUM: thread: make stopping_threads per-group and add stopping_tgroups
- MAJOR: threads: change thread_isolate to support inter-group synchronization
- MINOR: thread: add is_thread_harmless() to know if a thread already is harmless
- MINOR: debug: mark oneself harmless while waiting for threads to finish
- MINOR: wdt: do not rely on threads_to_dump anymore
- MEDIUM: debug: make the thread dumper not rely on a thread mask anymore
- BUILD: debug: fix build issue on clang with previous commit
- BUILD: debug: re-export thread_dump_state
- BUG/MEDIUM: threads: fix incorrect thread group being used on soft-stop
- BUG/MEDIUM: thread: check stopping thread against local bit and not global one
- MINOR: proxy: use tg->threads_enabled in hard_stop() to detect stopped threads
- BUILD: Makefile: Add Lua 5.4 autodetect
- CI: re-enable gcc asan builds
- MEDIUM: mworker: set the iocb of the socketpair without using fd_insert()
- MINOR: fd: Add BUG_ON checks on fd_insert()
- CLEANUP: mworker: rename mworker_pipe to mworker_sockpair
- CLEANUP: mux-quic: do not export qc_get_ncbuf
- REORG: mux-quic: reorganize flow-control fields
- MINOR: mux-quic: implement accessor for sedesc
- MEDIUM: mux-quic: refactor streams opening
- MINOR: mux-quic: rename qcs flag FIN_RECV to SIZE_KNOWN
- MINOR: mux-quic: emit FINAL_SIZE_ERROR on invalid STREAM size
- BUG/MINOR: peers/config: always fill the bind_conf's argument
- BUG/MEDIUM: peers/config: properly set the thread mask
- CLEANUP: bwlim: Set pointers to NULL when memory is released
- BUG/MINOR: http-check: Preserve headers if not redefined by an implicit rule
- BUG/MINOR: http-act: Properly generate 103 responses when several rules are used
- BUG/MEDIUM: thread: mask stopping_threads with threads_enabled when checking it
- CLEANUP: thread: also remove a thread's bit from stopping_threads on stop
- BUG/MINOR: peers: fix possible NULL dereferences at config parsing
- BUG/MINOR: http-htx: Fix scheme based normalization for URIs wih userinfo
- MINOR: http: Add function to get port part of a host
- MINOR: http: Add function to detect default port
- BUG/MEDIUM: h1: Improve authority validation for CONNCET request
- MINOR: http-htx: Use new HTTP functions for the scheme based normalization
- BUG/MEDIUM: http-fetch: Don't fetch the method if there is no stream
- REGTEESTS: filters: Fix CONNECT request in random-forwarding script
- MEDIUM: mworker/systemd: send STATUS over sd_notify
- BUG/MINOR: mux-h1: Be sure to commit htx changes in the demux buffer
- BUG/MEDIUM: http-ana: Don't wait to have an empty buf to switch in TUNNEL state
- BUG/MEDIUM: mux-h1: Handle connection error after a synchronous send
- MEDIUM: epoll: don't synchronously delete migrated FDs
- BUILD: debug: silence warning on gcc-5
- BUILD: http: silence an uninitialized warning affecting gcc-5
- BUG/MEDIUM: mux-quic: fix server chunked encoding response
- REORG: mux-quic: rename stream initialization function
- MINOR: mux-quic: rename stream purge function
- MINOR: mux-quic: add traces on frame parsing functions
- MINOR: mux-quic: implement qcs_alert()
- MINOR: mux-quic: filter send/receive-only streams on frame parsing
- MINOR: mux-quic: do not ack STREAM frames on unrecoverable error
- MINOR: mux-quic: support stream opening via MAX_STREAM_DATA
- MINOR: mux-quic: define basic stream states
- MINOR: mux-quic: use stream states to mark as detached
- MEDIUM: mux-quic: implement RESET_STREAM emission
- MEDIUM: mux-quic: implement STOP_SENDING handling
- BUG/MEDIUM: debug: fix possible hang when multiple threads dump at once
- BUG/MINOR: quic: fix closing state on NO_ERROR code sent
- CLEANUP: quic: clean up include on quic_frame-t.h
- MINOR: quic: define a generic QUIC error type
- MINOR: mux-quic: support app graceful shutdown
- MINOR: mux-quic/h3: prepare CONNECTION_CLOSE on release
- MEDIUM: quic: send CONNECTION_CLOSE on released MUX
- CLEANUP: mux-quic: move qc_release()
- MINOR: mux-quic: send one last time before release
- MINOR: h3: store control stream in h3c
- MINOR: h3: implement graceful shutdown with GOAWAY
- BUG/MINOR: threads: produce correct global mask for tgroup > 1
- BUG/MEDIUM: cli/threads: make "show threads" more robust on applets
- BUG/MINOR: thread: use the correct thread's group in ha_tkillall()
- BUG/MINOR: debug: enter ha_panic() only once
- BUG/MEDIUM: debug: fix parallel thread dumps again
- MINOR: cli/streams: show a stream's tgid next to its thread ID
- DEBUG: cli: add a new "debug dev deadlock" expert command
- MINOR: cli/activity: add a thread number argument to "show activity"
- CLEANUP: applet: remove the obsolete command context from the appctx
- MEDIUM: config: remove deprecated "bind-process" directives from frontends
- MEDIUM: config: remove the "process" keyword on "bind" lines
- MINOR: listener/config: make "thread" always support up to LONGBITS
- CLEANUP: fd: get rid of the __GET_{NEXT,PREV} macros
- MEDIUM: debug/threads: make the lock debugging take tgroups into account
- MEDIUM: proto: stop protocols under thread isolation during soft stop
- MEDIUM: poller: program the update in fd_update_events() for a migrated FD
- MEDIUM: poller: disable thread-groups for poll() and select()
- MINOR: thread: remove MAX_THREADS limitation
- MEDIUM: cpu-map: replace the process number with the thread group number
- MINOR: mworker/threads: limit the mworker sockets to group 1
- MINOR: cli/threads: always bind CLI to thread group 1
- MINOR: fd/thread: get rid of thread_mask()
- MEDIUM: task/thread: move the task shared wait queues per thread group
- MINOR: task: move the niced_tasks counter to the thread group context
- DOC: design: add some thoughts about how to handle the update_list
- MEDIUM: conn: make conn_backend_get always scan the same group
- MAJOR: fd: remove pending updates upon real close
- MEDIUM: fd/poller: make the update-list per-group
- MINOR: fd: delete unused updates on close()
- MINOR: fd: make fd_insert() apply the thread mask itself
- MEDIUM: fd: add the tgid to the fd and pass it to fd_insert()
- MINOR: cli/fd: show fd's tgid and refcount in "show fd"
- MINOR: fd: add functions to manipulate the FD's tgid
- MINOR: fd: add fd_get_running() to atomically return the running mask
- MAJOR: fd: grab the tgid before manipulating running
- MEDIUM: fd/poller: turn polled_mask to group-local IDs
- MEDIUM: fd/poller: turn update_mask to group-local IDs
- MEDIUM: fd/poller: turn running_mask to group-local IDs
- MINOR: fd: make fd_clr_running() return the previous value instead
- MEDIUM: fd: make thread_mask now represent group-local IDs
- MEDIUM: fd: make fd_insert() take local thread masks
- MEDIUM: fd: make fd_insert/fd_delete atomically update fd.tgid
- MEDIUM: fd: quit fd_update_events() when FD is closed
- MEDIUM: thread: change thread_resolve_group_mask() to return group-local values
- MEDIUM: listener: switch bind_thread from global to group-local
- MINOR: fd: add fd_reregister_all() to deal with boot-time FDs
- MEDIUM: fd: support stopping FDs during starting
- MAJOR: pollers: rely on fd_reregister_all() at boot time
- MAJOR: poller: only touch/inspect the update_mask under tgid protection
- MEDIUM: fd: support broadcasting updates for foreign groups in updt_fd_polling
- CLEANUP: threads: remove the now unused all_threads_mask and tid_bit
- MINOR: config: change default MAX_TGROUPS to 16
- BUG/MEDIUM: tools: avoid calling dlsym() in static builds
This will allows nbtgroups > 1 to be declared in the config without
recompiling. The theoretical limit is 64, though we'd rather not push
it too far for now as some structures might be enlarged to be indexed
per group. Let's start with 16 groups max, allowing to experiment with
dual-socket machines suffering from up to 8 loosely coupled L3 caches.
It's a good start and doesn't engage us too far.
The principle remains the same, but instead of having a single process
and ignoring extra ones, now we set the affinity masks for the respective
threads of all groups.
The doc was updated with a few extra examples.
It was deprecated, marked for removal in 2.7 and was already emitting a
warning, let's get rid of it. Note that we've kept the keyword detection
to suggest to use "thread" instead.
This was already causing a deprecation warning and was marked for removal
in 2.7, now it happens. An error message indicates this doesn't exist
anymore.
The output of "show activity" can be so large that the output is visually
unreadable on a screen. Let's add an option to filter on the desired
column (actually the thread number), use "0" to report only the first
column (aggregated/sum/avg), and use "-1", the default, for the normal
detailed dump.
Released version 2.7-dev1 with the following main changes :
- BUG/MINOR: ssl_ckch: Free error msg if commit changes on a cert entry fails
- BUG/MINOR: ssl_ckch: Free error msg if commit changes on a CA/CRL entry fails
- BUG/MEDIUM: ssl_ckch: Don't delete a cert entry if it is being modified
- BUG/MEDIUM: ssl_ckch: Don't delete CA/CRL entry if it is being modified
- BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a cert entry
- BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a CA/CRL entry
- BUG/MEDIUM: ssl_ckch: Rework 'commit ssl cert' to handle full buffer cases
- BUG/MEDIUM: ssl_ckch: Rework 'commit ssl ca-file' to handle full buffer cases
- BUG/MEDIUM: ssl/crt-list: Rework 'add ssl crt-list' to handle full buffer cases
- BUG/MEDIUM: httpclient: Don't remove HTX header blocks before duplicating them
- BUG/MEDIUM: httpclient: Rework CLI I/O handler to handle full buffer cases
- MEDIUM: httpclient: Don't close CLI applet at the end of a response
- MEDIUM: http-ana: Always report rewrite failures as PRXCOND in logs
- CLEANUP: Re-apply xalloc_size.cocci (2)
- REGTESTS: abortonclose: Add a barrier to not mix up log messages
- REGTESTS: http_request_buffer: Increase client timeout to wait "slow" clients
- CLEANUP: ssl_ckch: Use corresponding enum for commit_cacrlfile_ctx.cafile_type
- MINOR: ssl_ckch: Simplify I/O handler to commit changes on CA/CRL entry
- BUG/MINOR: ssl_ckch: Use right type for old entry in show_crlfile_ctx
- BUG/MINOR: ssl_ckch: Dump CRL transaction only once if show command yield
- BUG/MINOR: ssl_ckch: Dump CA transaction only once if show command yield
- BUG/MINOR: ssl_ckch: Dump cert transaction only once if show command yield
- BUG/MINOR: ssl_ckch: Init right field when parsing "commit ssl crl-file" cmd
- CLEANUP: ssl_ckch: Remove unused field in commit_cacrlfile_ctx structure
- MINOR: ssl_ckch: Simplify structure used to commit changes on CA/CRL entries
- MINOR: ssl_ckch: Remove service context for "set ssl cert" command
- MINOR: ssl_ckch: Remove service context for "set ssl ca-file" command
- MINOR: ssl_ckch: Remove service context for "set ssl crl-file" command
- BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cert I/O handler
- BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cafile I/O handler
- BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_crlfile I/O handler
- BUILD: ssl_ckch: Fix build error about a possible uninitialized value
- BUG/MINOR: ssl_ckch: Fix another possible uninitialized value
- REGTESTS: http_abortonclose: Extend supported versions
- REGTESTS: restrict_req_hdr_names: Extend supported versions
- MINOR: connection: support HTTP/3.0 for smp_*_http_major fetch
- MINOR: h3: add h3c pointer into h3s instance
- MINOR: mux-quic: simplify decode_qcs API
- MINOR: mux-quic/h3: adjust demuxing function return values
- BUG/MINOR: h3: fix return value on decode_qcs on error
- BUILD: quic: fix anonymous union for gcc-4.4
- BUILD: compiler: implement unreachable for older compilers too
- DEV: tcploop: reorder options in the usage message
- DEV: tcploop: make the current address the default address
- DEV: tcploop: make it possible to change the target address of a connect()
- DEV: tcploop: factor out the socket creation
- DEV: tcploop: permit port 0 to ease handling of default options
- DEV: tcploop: add a new "bind" command to bind to ip/port.
- DEV: tcploop: add minimal UDP support
- BUG/MINOR: trace: Test server existence for health-checks to get proxy
- BUG/MINOR: checks: Properly handle email alerts in trace messages
- BUG/MEDIUM: mailers: Set the object type for check attached to an email alert
- REGTESTS: healthcheckmail: Update the test to be functionnal again
- REGTESTS: healthcheckmail: Relax health-check failure condition
- BUG/MINOR: h3: fix incorrect BUG_ON assert on SETTINGS parsing
- MEDIUM: mux-h2: try to coalesce outgoing WINDOW_UPDATE frames
- OPTIM: mux-h2: increase h2_settings_initial_window_size default to 64k
- BUG/MINOR: h3: fix frame type definition
- BUG/MEDIUM: h3: fix SETTINGS parsing
- BUG/MINOR: cli/stats: add missing trailing LF after JSON outputs
- BUG/MINOR: server: do not enable DNS resolution on disabled proxies
- BUG/MINOR: cli/stats: add missing trailing LF after "show info json"
- DOC: design: update the notes on thread groups
- BUG/MEDIUM: mux-quic: fix flow control connection Tx level
- MINOR: mux-quic: complete BUG_ON on TX flow-control enforcing
- BUG/MINOR: mux-quic: fix memleak on frames rejected by transport
- BUG/MINOR: tcp-rules: Make action call final on read error and delay expiration
- CLEANUP: check: Remove useless tests on check's stream-connector
- BUG/MEDIUM: stconn: Don't wakeup applet for send if it won't consume data
- BUG/MEDIUM: cli: Notify cli applet won't consume data during request processing
- BUG/MEDIUM: mux-quic: fix segfault on flow-control frame cleanup
- MINOR: task: move profiling bit to per-thread
- CLEANUP: quic: use task_new_on() for single-threaded tasks
- MINOR: tinfo: remove the global thread ID bit (tid_bit)
- CLEANUP: hlua: check for at least 2 threads on a task
- MINOR: thread: get rid of MAX_THREADS_MASK
- OPTIM: task: do not consult shared WQ when we're already full
- DOC: design: update the task vs thread affinity requirements
- MINOR: qpack: add comments and remove a useless trace
- MINOR: qpack: reduce dependencies on other modules
- BUG/MINOR: qpack: support header litteral name decoding
- MINOR: qpack: add ABORT_NOW on unimplemented decoding
- BUG/MINOR: h3/qpack: deal with too many headers
- MINOR: qpack: improve decoding function
- MINOR: qpack: implement standalone decoder tool
- BUG/BUILD: h3: fix wrong label name
- BUG/MINOR: quic: Stop hardcoding Retry packet Version field
- MINOR: quic: Add several nonce and key definitions for Retry tag
- BUG/MINOR: quic: Wrong PTO calculation
- MINOR: quic: Parse long packet version from qc_parse_hd_form()
- CLEANUP: quid: QUIC draft-28 no more supported
- MEDIUM: quic: Add QUIC v2 draft support
- MINOR: quic: Released QUIC TLS extension for QUIC v2 draft
- MEDIUM: quic: Compatible version negotiation implementation (draft-08)
- CLEANUP: quic: Remove any reference to boringssl
- BUG/MINOR: task: fix thread assignment in tasklet_kill()
- BUG/MEDIUM: stream: Properly handle destructive client connection upgrades
- MINOR: stream: Rely on stconn flags to abort stream destructive upgrade
- CLEANUP: stconn: Don't expect to have no sedesc on detach
- BUG/MINOR: log: Properly test connection retries to fix dontlog-normal option
- MINOR: hlua: don't dump empty entries in hlua_traceback()
- MINOR: hlua: add a new hlua_show_current_location() function
- MEDIUM: debug: add a tainted flag when a shared library is loaded
- MEDIUM: debug: detect redefinition of symbols upon dlopen()
- BUILD: quic: Wrong HKDF label constant variable initializations
- BUG/MINOR: quic: Unexpected half open connection counter wrapping
- BUG/MINOR: quic_stats: Duplicate "quic_streams_data_blocked_bidi" field name
- BUG/MINOR: quic: purge conn Rx packet list on release
- BUG/MINOR: quic: free rejected Rx packets
- BUG/MINOR: qpack: abort on dynamic index field line decoding
- BUG/MEDIUM: ssl/cli: crash when crt inserted into a crt-list
- REGTESTS: ssl: add the same cert for client/server
- BUG/MINOR: quic: Acknowledgement must be forced during handshake
- MINOR: quic: Dump version_information transport parameter
- BUG/MEDIUM: mworker: use default maxconn in wait mode
- MINOR: intops: add a function to return a valid bit position from a mask
- TESTS: add a unit test for one_among_mask()
- BUILD: ssl_ckch: fix "maybe-uninitialized" build error on gcc-9.4 + ARM
- BUG/MINOR: ssl: Do not look for key in extra files if already in pem
- BUG/MINOR: quic: Missing acknowledgments for trailing packets
- BUG/MINOR: http-ana: Set method to HTTP_METH_OTHER when an HTTP txn is created
- BUG/MINOR: http-fetch: Use integer value when possible in "method" sample fetch
- MINOR: freq_ctr: Add a function to get events excess over the current period
- BUG/MINOR: stream: only free the req/res captures when set
- CLEANUP: pool/tree-wide: remove suffix "_pool" from certain pool names
- MEDIUM: debug: improve DEBUG_MEM_STATS to also report pool alloc/free
- BUG/MINOR: quic: Wrong reuse of fulfilled dgram RX buffer
- BUG/MAJOR: quic: Big RX dgrams leak when fulfilling a buffer
- BUG/MAJOR: quic: Big RX dgrams leak with POST requests
- BUILD: quic+h3: 32-bit compilation errors fixes
- MEDIUM: bwlim: Add support of bandwith limitation at the stream level
This patch adds a filter to limit bandwith at the stream level. Several
filters can be defined. A filter may limit incoming data (upload) or
outgoing data (download). The limit can be defined per-stream or shared via
a stick-table. For a given stream, the bandwith limitation filters can be
enabled using the "set-bandwidth-limit" action.
A bandwith limitation filter can be used indifferently for HTTP or TCP
stream. For HTTP stream, only the payload transfer is limited. The filter is
pretty simple for now. But it was designed to be extensible. The current
design tries, as far as possible, to never exceed the limit. There is no
burst.
It looks like we'll need:
- one share timers queue for the rare tasks that need to wake up
anywhere
- one private timers queue per thread
- one global queue per group
- one local queue per thread
And may be we can simply get rid of any global/shared run queue as
we don't seem to have any task bound to a subset of threads.
This macro was used both for binding and for lookups. When binding tasks
or FDs, using all_threads_mask instead is better as it will later be per
group. For lookups, ~0UL always does the job. Thus in practice the macro
was already almost not used anymore since the rest of the code could run
fine with a constant of all ones there.
Instead of having a global mask of all the profiled threads, let's have
one flag per thread in each thread's flags. They are never accessed more
than one at a time an are better located inside the threads' contexts for
both performance and scalability.
This changes the default from RFC 7540's default 65535 (64k-1) to avoid
avoid some degenerative WINDOW_UPDATE behaviors in the wild observed with
clients using 65536 as their buffer size, and have to complete each block
with a 1-byte frame, which with some servers tend to degenerate in 1-byte
WU causing more 1-byte frames to be sent until the transfer almost only
uses 1-byte frames.
More details here: https://github.com/nghttp2/nghttp2/issues/1722
As mentioned in previous commit (MEDIUM: mux-h2: try to coalesce outgoing
WINDOW_UPDATE frames) the issue could not be reproduced with haproxy but
individual WU frames are sent so theoretically nothing prevents this from
happening. As such it should be backported as a workaround for already
deployed clients after watching for any possible side effect with rare
clients. As an added benefit, uploads from curl now use less DATA frames
(all are 16384 now). Note that the previous patch alone is sufficient to
stop the issue with curl in case this one would need to be reverted.
[wt: edited commit messaged, updated doc]
Released version 2.6.0 with the following main changes :
- DOC: Fix formatting in configuration.txt to fix dconv
- CLEANUP: tcpcheck: Remove useless test on the stream-connector in tcpcheck_main
- CLEANUP: muxes: Consider stream's sd as defined in .show_fd callback functions
- MINOR: quic: Ignore out of packet padding.
- CLEANUP: quic: Useless QUIC_CONN_TX_BUF_SZ definition
- CLEANUP: quic: No more used handshake output buffer
- MINOR: quic: QUIC transport parameters split.
- MINOR: quic: Transport parameters dump
- DOC: quic: Update documentation for QUIC Retry
- MINOR: quic: Tunable "max_idle_timeout" transport parameter
- MINOR: quic: Tunable "initial_max_streams_bidi" transport parameter
- MINOR: quic: Clarifications about transport parameters value
- MINOIR: quic_stats: add QUIC connection errors counters
- BUG/MINOR: quic: Largest RX packet numbers mixing
- MINOR: quic_stats: Add transport new counters (lost, stateless reset, drop)
- DOC: quic: Documentation update for QUIC
- MINOR: quic: Connection TX buffer setting renaming.
- MINOR: h3: Add a statistics module for h3
- MINOR: quic: Send STOP_SENDING frames if mux is released
- MINOR: quic: Do not drop packets with RESET_STREAM frames
- BUG/MINOR: qpack: fix buffer API usage on prefix integer encoding
- BUG/MINOR: qpack: support bigger prefix-integer encoding
- BUG/MINOR: h3: do not report bug on unknown method
- SCRIPTS: add make-releases-json to recreate a releases.json file in download dirs
- SCRIPTS: make publish-release try to launch make-releases-json
- MINOR: htx: add an unchecked version of htx_get_head_blk()
- BUILD: htx: use the unchecked version of htx_get_head_blk() where needed
- BUILD: quic: use inttypes.h instead of stdint.h
- DOC: internal: remove totally outdated diagrams
- DOC: remove the outdated ROADMAP file
- DOC: add maintainers for QUIC and HTTP/3
- MINOR: h3: define h3 trace module
- MINOR: h3: add traces on frame recv
- MINOR: h3: add traces on frame send
- MINOR: h3: add traces on h3s init/end
- EXAMPLES: remove completely outdated acl-content-sw.cfg
- BUILD: makefile: reorder objects by build time
- DOC: fix a few spelling mistakes in the docs
- BUG/MEDIUM: peers/cli: fix "show peers" crash
- CLEANUP: peers/cli: stop misusing the appctx local variable
- CLEANUP: peers/cli: make peers_dump_peer() take an appctx instead of an stconn
- BUG/MINOR: peers: set the proxy's name to the peers section name
- MINOR: server: indicate when no address was expected for a server
- BUG/MINOR: peers: detect and warn on init_addr/resolvers/check/agent-check
- DOC: peers: indicate that some server settings are not usable
- DOC: peers: clarify when entry expiration date is renewed.
- DOC: peers: fix port number and addresses on new peers section format
- DOC: gpc/gpt: add commments of gpc/gpt array definitions on stick tables.
- DOC: install: update supported OpenSSL versions in the INSTALL doc
- MINOR: ncbuf: adjust ncb_data with NCBUF_NULL
- BUG/MINOR: h3: fix frame demuxing
- BUG/MEDIUM: h3: fix H3_EXCESSIVE_LOAD when receiving H3 frame header only
- BUG/MINOR: quic: Fix QUIC_EV_CONN_PRSAFRM event traces
- CLEANUP: quic: remove useless check on local UNI stream reception
- BUG/MINOR: qpack: do not consider empty enc/dec stream as error
- DOC: intro: adjust the numbering of paragrams to keep the output ordered
- MINOR: version: mention that it's LTS now.
The HTML version appeared with sections in a different order where
3.3.10..3.3.16 were placed between 3.3.1 and 3.3.2. This patch just slips
them into an intermediary section so that we now have "basic features",
"standard features", and "advanced features".
Some users misunderstood that the parameter of gpc() gpt()
store types on the table line presents the number of elements
of the array to store and not an index of gpt/gpc tag/counter.
This patch adds some explanations.
This patch addresses github issue #1630
It should be backorted in until branch 2.5.
This patch fix the port number and addresses on the example
to match those of the old format.
This patch address the github issue #1492
This patch should be backported until version 2.0
This patch add some details to know which rules are updating
the expiration timer of an entry.
It also adds a comment to know how to fetch a value without renewing
this timer.
This patch addresses github issue #615
This patch should be backported on all still supported branches
Let's make it clear in the peers documentation that not all server
parameters may be used, as there is some confusion around this, and
the doc was even misleading by saying that all parameters were
supported.
This should address github issue #919.
The "sequence" and "entities" diagrams have become so much outdated that
they are at best confusing, but more generally wrong. Let's simply remove
them.
Rename "tune.quic.conn-buf-limit" to "tune.quic.frontend.conn-tx-buffers.limit"
to reflect the stream direction (TX) and the objects (frontends) which are
concerned.
Add tunable "tune.quic.frontend.max_streams_bidi" setting for QUIC frontends
to set the "initial_max_streams_bidi" transport parameter.
Add some documentation for this new setting.
Add two tunable settings both for backends and frontends "max_idle_timeout"
QUIC transport parameter, "tune.quic.frontend.max-idle-timeout" and
"tune.quic.backend.max-idle-timeout" respectively.
cfg_parse_quic_time() has been implemented to parse a time value thanks
to parse_time_err(). It should be reused for any tunable time value to be
parsed.
Add the documentation for this tunable setting only for frontend.
Released version 2.6-dev12 with the following main changes :
- CLEANUP: tools: Clean up non-QUIC error message handling in str2sa_range()
- BUG/MEDIUM: tools: Fix `inet_ntop` usage in sa2str
- CLEANUP: tools: Crash if inet_ntop fails due to ENOSPC in sa2str
- BUG/MEDIUM: mux-quic: adjust buggy proxy closing support
- Revert "MINOR: quic: activate QUIC traces at compilation"
- Revert "MINOR: mux-quic: activate qmux traces on stdout via macro"
- CLEANUP: init: address a coverity warning about possible multiply overflow
- BUG/MEDIUM: http: Properly reject non-HTTP/1.x protocols
- MEDIUM: h1: enlarge the scope of accepted version chars with accept-invalid-http-request
- BUG/MEDIUM: resolvers: Don't defer resolutions release in deinit function
- BUG/MEDIUM: peers: fix segfault using multiple bind on peers sections
- BUG/MEDIUM: peers: prevent unitialized multiple listeners on peers section
- BUG/MINOR: task: Don't defer tasks release when HAProxy is stopping
- MINOR: h3: mark ncbuf as const on h3_b_dup
- MINOR: mux-quic: do not alloc quic_stream_desc for uni remote stream
- MINOR: mux-quic: delay cs_endpoint allocation
- MINOR: mux-quic: add traces in qc_recv()
- MINOR: mux-quic: adjust return value of decode_qcs
- CLEANUP: h3: rename struct h3 -> h3c
- CLEANUP: h3: rename uni stream type constants
- BUG/MINOR: h3: prevent overflow when parsing SETTINGS
- MINOR: h3: refactor h3_control_send()
- MINOR: quic: support CONNECTION_CLOSE_APP emission
- MINOR: mux-quic: disable read on CONNECTION_CLOSE emission
- MINOR: h3: reject too big frames
- MINOR: mux-quic: emit STREAM_STATE_ERROR in qcc_recv
- BUG/MINOR: mux-quic: refactor uni streams TX/send H3 SETTINGS
- MINOR: h3/qpack: use qcs as type in decode callbacks
- MINOR: h3: define stream type
- MINOR: h3: refactor uni streams initialization
- MINOR: h3: check if frame is valid for stream type
- MINOR: h3: define non-h3 generic parsing function
- MEDIUM: quic: refactor uni streams RX
- CLEANUP: h3: remove h3 uni tasklet
- MINOR: h3: abort read on unknown uni stream
- MINOR: h3: refactor SETTINGS parsing/error reporting
- Revert "BUG/MINOR: task: Don't defer tasks release when HAProxy is stopping"
- DOC: configuration: add a warning for @system-ca on bind
- CLEANUP: init: address another coverity warning about a possible multiply overflow
- BUG/MINOR: ssl/lua: use correctly cert_ext in CertCache.set()
- BUG/MEDIUM: sample: Fix adjusting size in word converter
- REGTESTS: Do not use REQUIRE_VERSION for HAProxy 2.5+ (2)
- CLEANUP: conn_stream: remove unneeded exclusion of RX_WAIT_EP from RXBLK_ANY
- CLEANUP: conn_stream: rename the cs_endpoint's context to "conn"
- MINOR: conn_stream: add new sets of functions to set/get endpoint flags
- DEV: coccinelle: add cs_endp_flags.cocci
- CLEANUP: conn_stream: apply cs_endp_flags.cocci tree-wide
- DEV: coccinelle: add endp_flags.cocci
- CLEANUP: conn_stream: apply endp_flags.cocci tree-wide
- CLEANUP: conn_stream: rename the stream endpoint flags CS_EP_* to SE_FL_*
- CLEANUP: conn_stream: rename the cs_endpoint's target to "se"
- CLEANUP: conn_stream: rename cs_endpoint to sedesc (stream endpoint descriptor)
- CLEANUP: applet: rename the sedesc pointer from "endp" to "sedesc"
- CLEANUP: conn_stream: rename the conn_stream's endp to sedesc
- CLEANUP: conn_stream: rename cs_app_* to sc_app_*
- CLEANUP: conn_stream: tree-wide rename to stconn (stream connector)
- CLEANUP: mux-h1: add and use h1s_sc() to retrieve the stream connector
- CLEANUP: mux-h2: add and use h2s_sc() to retrieve the stream connector
- CLEANUP: mux-fcgi: add and use fcgi_strm_sc() to retrieve the stream connector
- CLEANUP: mux-pt: add and use pt_sc() to retrieve the stream connector
- CLEANUP: stdesc: rename the stream connector ->cs field to ->sc
- CLEANUP: stream: rename "csf" and "csb" to "scf" and "scb"
- CLEANUP: stconn: tree-wide rename stream connector flags CS_FL_* to SC_FL_*
- CLEANUP: stconn: tree-wide rename stconn states CS_ST/SB_* to SC_ST/SB_*
- MINOR: check: export wake_srv_chk()
- MINOR: conn_stream: test the various ops functions before calling them
- MEDIUM: stconn: merge the app_ops and the data_cb fields
- MINOR: applet: add new wrappers to put chk/blk/str/chr to channel from appctx
- CLEANUP: applet: use applet_put*() everywhere possible
- CLEANUP: stconn: rename cs_{i,o}{b,c} to sc_{i,o}{b,c}
- CLEANUP: stconn: rename cs_{check,strm,strm_task} to sc_strm_*
- CLEANUP: stconn: rename cs_conn() to sc_conn()
- CLEANUP: stconn: rename cs_mux() to sc_mux_strm()
- CLEANUP: stconn: rename cs_conn_mux() to sc_mux_ops()
- CLEANUP: stconn: rename cs_appctx() to sc_appctx()
- CLEANUP: stconn: rename __cs_endp_target() to __sc_endp()
- CLEANUP: stconn: rename cs_get_data_name() to sc_get_data_name()
- CLEANUP: stconn: rename cs_conn_*() to sc_conn_*()
- CLEANUP: stconn: rename cs_conn_get_first() to conn_get_first_sc()
- CLEANUP: stconn: rename cs_ep_set_error() to se_fl_set_error()
- CLEANUP: stconn: make a few functions take a const argument
- CLEANUP: stconn: use a single function to know if SC may send to SE
- MINOR: stconn: consider CF_SHUTW for sc_is_send_allowed()
- MINOR: stconn: remove calls to cs_done_get()
- MEDIUM: stconn: always rely on CF_SHUTR in addition to cs_rx_blocked()
- MEDIUM: stconn: remove SE_FL_RXBLK_SHUT
- MINOR: stconn: rename SE_FL_RXBLK_CONN to SE_FL_APPLET_NEED_CONN
- MEDIUM: stconn: take SE_FL_APPLET_NEED_CONN out of the RXBLK_ANY flags
- CLEANUP: stconn: rename cs_rx_room_{blk,rdy} to sc_{need,have}_room()
- CLEANUP: stconn: rename cs_rx_chan_{blk,rdy} to sc_{wont,will}_read()
- CLEANUP: stconn: rename cs_rx_buff_{blk,rdy} to sc_{need,have}_buff()
- MINOR: stconn: start to rename cs_rx_endp_{more,done}() to se_have_{no_,}more_data()
- MINOR: stconn: add sc_is_recv_allowed() to check for ability to receive
- CLEANUP: stconn: rename SE_FL_RX_WAIT_EP to SE_FL_HAVE_NO_DATA
- MEDIUM: stconn: move the RXBLK flags to the stream connector
- CLEANUP: stconn: rename SE_FL_WANT_GET to SE_FL_WILL_CONSUME
- CLEANUP: stconn: remove cs_tx_blocked() and cs_tx_endp_ready()
- CLEANUP: stconn: rename cs_{want,stop}_get() to se_{will,wont}_consume()
- CLEANUP: stconn: rename cs_cant_get() to se_need_more_data()
- CLEANUP: stconn: rename cs_{new,create,free,destroy}_* to sc_*
- CLEANUP: stconn: rename remaining management functions from cs_* to sc_*
- CLEANUP: stconn: rename cs{,_get}_{src,dst} to sc_*
- CLEANUP: stconn: rename cs_{shut,chk}* to sc_*
- CLEANUP: stconn: rename final state manipulation functions from cs_* to sc_*
- CLEANUP: quic: drop the name "conn_stream" from the pool variable names
- REORG: rename cs_utils.h to sc_strm.h
- REORG: stconn: rename conn_stream.{c,h} to stconn.{c,h}
- CLEANUP: muxes: rename "get_first_cs" to "get_first_sc"
- DEV: flags: use "sc" for stream conns instead of "cs"
- CLEANUP: check: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: connection: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: stconn: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: quic/h3: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: stream: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: promex: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: stats: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: cli: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: applet: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: cache: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: dns: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: spoe: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: hlua: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: log-forward: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: http-client: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: mux-fcgi: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: mux-h1: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: mux-h2: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: mux-pt: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: peers: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: sink: rename all occurrences of stconn "cs" to "sc"
- CLEANUP: sslsock: remove only occurrence of local variable "cs"
- CLEANUP: applet: rename appctx_cs() to appctx_sc()
- CLEANUP: stream: rename stream_upgrade_from_cs() to stream_upgrade_from_sc()
- CLEANUP: obj_type: rename OBJ_TYPE_CS to OBJ_TYPE_SC
- CLEANUP: stconn: replace a few remaining occurrences of CS in comments or traces
- DOC: internal: update the muxes doc to mention the stconn
- CLEANUP: mux-quic: rename the "endp" field to "sd"
- CLEANUP: mux-h1: rename the "endp" field to "sd"
- CLEANUP: mux-h2: rename the "endp" field to "sd"
- CLEANUP: mux-fcgi: rename the "endp" field to "sd"
- CLEANUP: mux-pt: rename the "endp" field to "sd"
- CLEANUP: stconn: rename a few "endp" arguments and variables to "sd"
- MINOR: stconn: turn SE_FL_WILL_CONSUME to SE_FL_WONT_CONSUME
- CLEANUP: stream: remove unneeded test on appctx during initialization
- CLEANUP: stconn: remove the new unneeded SE_FL_APP_MASK
- DEV: flags: fix "siet" shortcut name
- DEV: flags: rename the "endp" shortcut to "sd" for "stream descriptor"
- DEV: flags: reorder a few SC/SE flags
- DOC: internal: add a description of the stream connectors and descriptors
The "layers" mini-doc shows how streams, stconn, sedesc, conns, applets
and muxes interact, with field names, pointers and invariants. It should
be completed but already provides a quick overview about what can be
guaranteed at any step and at different layers.
The stream connector replaced the conn_stream and the sc_conn_io_cb()
function appeared. There's no place there to mention the endpoint
descriptor, but a separate diagram showing the relation between stream
and endpoint via the connector would be nice.
We used to support both RTSP and HTTP protocol version names with and
without accept-invalid-http-request, but since this is based on the
characters themselves, any protocol made of chars {0-9/.HPRST} was
possible and not others. Now that such non-standard protocols are
restricted to accept-invalid-http-request, there's no reason for not
allowing other letters. With this patch, characters {0-9./A-Z} are
permitted when the option is set.
Released version 2.6-dev11 with the following main changes :
- CI: determine actual LibreSSL version dynamically
- BUG/MEDIUM: ncbuf: fix null buffer usage
- MINOR: ncbuf: fix warnings for testing build
- MEDIUM: http-ana: Add a proxy option to restrict chars in request header names
- MEDIUM: ssl: Delay random generator initialization after config parsing
- MINOR: ssl: Add 'ssl-propquery' global option
- MINOR: ssl: Add 'ssl-provider' global option
- CLEANUP: Add missing header to ssl_utils.c
- CLEANUP: Add missing header to hlua_fcn.c
- CLEANUP: Remove unused function hlua_get_top_error_string
- BUILD: fix build warning on solaris based systems with __maybe_unused.
- MINOR: tools: add get_exec_path implementation for solaris based systems.
- BUG/MINOR: ssl: Fix crash when no private key is found in pem
- CLEANUP: conn-stream: Remove cs_applet_shut declaration from header file
- MINOR: applet: Prepare appctx to own the session on frontend side
- MINOR: applet: Let the frontend appctx release the session
- MINOR: applet: Change return value for .init callback function
- MINOR: stream: Export stream_free()
- MINOR: applet: Add appctx_init() helper fnuction
- MINOR: applet: Add a function to finalize frontend appctx startup
- MINOR: applet: Add function to release appctx on error during init stage
- MEDIUM: dns: Refactor dns appctx creation
- MEDIUM: spoe: Refactor SPOE appctx creation
- MEDIUM: lua: Refactor cosocket appctx creation
- MEDIUM: httpclient: Refactor http-client appctx creation
- MINOR: sink: Add a ref to sink in the sink_forward_target structure
- MEDIUM: sink: Refactor sink forwarder appctx creation
- MINOR: peers: Add a ref to peers section in the peer structure
- MEDIUM: peers: Refactor peer appctx creation
- MINOR: applet: Add API to start applet on a thread subset
- MEDIUM: applet: Add support for async appctx startup on a thread subset
- MINOR: peers: Track number of applets run by thread
- MEDIUM: peers: Balance applets across threads
- MINOR: conn-stream/applet: Stop setting appctx as the endpoint context
- CLEANUP: proxy: Remove dead code when parsing "http-restrict-req-hdr-names" option
- REGTESTS: abortonclose: Fix some race conditions
- MINOR: ssl: Add 'ssl-provider-path' global option
- CLEANUP: http_ana: Make use of the return value of stream_generate_unique_id()
- BUG/MINOR: spoe: Fix error handling in spoe_init_appctx()
- CLEANUP: peers: Remove unreachable code in peer_session_create()
- CLEANUP: httpclient: Remove useless test on ss_dst in httpclient_applet_init()
- BUG/MEDIUM: quic: fix Rx buffering
- OPTIM: quic: realign empty Rx buffer
- BUG/MINOR: ncbuf: fix ncb_is_empty()
- MINOR: ncbuf: refactor ncb_advance()
- BUG/MINOR: mux-quic: update session's idle delay before stream creation
- MINOR: h3: do not wait a complete frame for demuxing
- MINOR: h3: flag demux as full on HTX full
- MEDIUM: mux-quic: implement recv on io-cb
- MINOR: mux-quic: remove qcc_decode_qcs() call in XPRT
- MINOR: mux-quic: reorganize flow-control frames emission
- MINOR: mux-quic: implement MAX_STREAM_DATA emission
- MINOR: mux-quic: implement MAX_DATA emission
- BUG/MINOR: mux-quic: support nul buffer with qc_free_ncbuf()
- MINOR: mux-quic: free RX buf if empty
- BUG/MEDIUM: config: Reset outline buffer size on realloc error in readcfgfile()
- BUG/MINOR: check: Reinit the buffer wait list at the end of a check
- MEDIUM: check: No longer shutdown the connection in .wake callback function
- REORG: check: Rename and export I/O callback function
- MEDIUM: check: Use the CS to handle subscriptions for read/write events
- BUG/MINOR: quic: break for error on sendto
- MINOR: quic: abort on unlisted errno on sendto()
- MINOR: quic: detect EBADF on sendto()
- BUG/MEDIUM: quic: fix initialization for local/remote TPs
- CLEANUP: quic: adjust comment/coding style for TPs init
- BUG/MINOR: cfgparse: abort earlier in case of allocation error
- MINOR: quic: Dump initial derived secrets
- MINOR: quic_tls: Add quic_tls_derive_retry_token_secret()
- MINOR: quic_tls: Add quic_tls_decrypt2() implementation
- MINOR: quic: Retry implementation
- MINOR: cfgparse: Update for "cluster-secret" keyword for QUIC Retry
- MINOR: quic: Move quic_lstnr_dgram_dispatch() out of xprt_quic.c
- BUILD: stats: Missing headers inclusions from stats.h
- MINOR: quic_stats: Add a new stats module for QUIC
- MINOR: quic: Attach proxy QUIC stats counters to the QUIC connection
- BUG/MINOR: quic: Fix potential memory leak during QUIC connection allocations
- MINOR: quic: QUIC stats counters handling
- MINOR: quic: Add tune.quic.retry-threshold keyword
- MINOR: quic: Dynamic Retry implementation
- MINOR: quic/mux-quic: define CONNECTION_CLOSE send API
- MINOR: mux-quic: emit FLOW_CONTROL_ERROR
- MINOR: mux-quic: emit STREAM_LIMIT_ERROR
- MINOR: mux-quic: close connection on error if different data at offset
- BUG/MINOR: peers: fix error reporting of "bind" lines
- CLEANUP: config: improve address parser error report for unmatched protocols
- CLEANUP: config: provide cleare hints about unsupported QUIC addresses
- MINOR: protocol: replace ctrl_type with xprt_type and clarify it
- MINOR: listener: provide a function to process all of a bind_conf's arguments
- MINOR: config: use the new bind_parse_args_list() to parse a "bind" line
- CLEANUP: listener: add a comment about what the BC_SSL_O_* flags are for
- MINOR: listener: add a new "options" entry in bind_conf
- CLEANUP: listener: replace all uses of bind_conf->is_ssl with BC_O_USE_SSL
- CLEANUP: listener: replace bind_conf->generate_cers with BC_O_GENERATE_CERTS
- CLEANUP: listener: replace bind_conf->quic_force_retry with BC_O_QUIC_FORCE_RETRY
- CLEANUP: listener: store stream vs dgram at the bind_conf level
- MINOR: listener: detect stream vs dgram conflict during parsing
- MINOR: listener: set the QUIC xprt layer immediately after parsing the args
- MINOR: listener/ssl: set the SSL xprt layer only once the whole config is known
- MINOR: connection: add flag MX_FL_FRAMED to mark muxes relying on framed xprt
- MINOR: config: detect and report mux and transport incompatibilities
- MINOR: listener: automatically select a QUIC mux with a QUIC transport
- MINOR: listener: automatically enable SSL if a QUIC transport is found
- BUG/MINOR: quic: Fixe a typo in qc_idle_timer_task()
- BUG/MINOR: quic: Missing <conn_opening> stats counter decrementation
- BUILD/MINOR: cpuset fix build for FreeBSD 13.1
- CI: determine actual OpenSSL version dynamically
When loading providers with 'ssl-provider' global options, this
ssl-provider-path option can be used to set the search path that is to
be used by openssl. It behaves the same way as the OPENSSL_MODULES
environment variable.
When HAProxy is linked to an OpenSSLv3 library, this option can be used
to load a provider during init. You can specify multiple ssl-provider
options, which will be loaded in the order they appear. This does not
prevent OpenSSL from parsing its own configuration file in which some
other providers might be specified.
A linked list of the providers loaded from the configuration file is
kept so that all those providers can be unloaded during cleanup. The
providers loaded directly by OpenSSL will be freed by OpenSSL.
This option can be used to define a default property query used when
fetching algorithms in OpenSSL providers. It follows the format
described in https://www.openssl.org/docs/man3.0/man7/property.html.
It is only available when haproxy is built with SSL support and linked
to OpenSSLv3 libraries.
The "http-restrict-req-hdr-names" option can now be set to restrict allowed
characters in the request header names to the "[a-zA-Z0-9-]" charset.
Idea of this option is to not send header names with non-alphanumeric or
hyphen character. It is especially important for FastCGI application because
all those characters are converted to underscore. For instance,
"X-Forwarded-For" and "X_Forwarded_For" are both converted to
"HTTP_X_FORWARDED_FOR". So, header names can be mixed up by FastCGI
applications. And some HAProxy rules may be bypassed by mangling header
names. In addition, some non-HTTP compliant servers may incorrectly handle
requests when header names contain characters ouside the "[a-zA-Z0-9-]"
charset.
When this option is set, the policy must be specify:
* preserve: It disables the filtering. It is the default mode for HTTP
proxies with no FastCGI application configured.
* delete: It removes request headers with a name containing a character
outside the "[a-zA-Z0-9-]" charset. It is the default mode for
HTTP backends with a configured FastCGI application.
* reject: It rejects the request with a 403-Forbidden response if it
contains a header name with a character outside the
"[a-zA-Z0-9-]" charset.
The option is evaluated per-proxy and after http-request rules evaluation.
This patch may be backported to avoid any secuirty issue with FastCGI
application (so as far as 2.2).
Released version 2.6-dev10 with the following main changes :
- MINOR: ssl: ignore dotfiles when loading a dir w/ ca-file
- MEDIUM: ssl: ignore dotfiles when loading a dir w/ crt
- BUG/MINOR: ssl: Fix typos in crl-file related CLI commands
- MINOR: compiler: add a new macro to set an attribute on an enum when possible
- BUILD: stats: conditionally mark obsolete stats states as deprecated
- BUILD: ssl: work around bogus warning in gcc 12's -Wformat-truncation
- BUILD: debug: work around gcc-12 excessive -Warray-bounds warnings
- BUILD: listener: shut report of possible null-deref in listener_accept()
- BUG/MEDIUM: ssl: fix the gcc-12 broken fix :-(
- DOC: install: update gcc version requirements
- BUILD: makefile: add -Wfatal-errors to the default flags
- BUG/MINOR: server: Make SRV_STATE_LINE_MAXLEN value from 512 to 2kB (2000 bytes).
- BUG/MAJOR: dns: multi-thread concurrency issue on UDP socket
- BUG/MINOR: mux-h2: mark the stream as open before processing it not after
- MINOR: mux-h2: report a trace event when failing to create a new stream
- DOC: configuration: add the httpclient keywords to the global keywords index
- MINOR: quic: Add a debug counter for sendto() errors
- BUG/MINOR: quic: Dropped peer transport parameters
- BUG/MINOR: quic: Wrong unit for ack delay for incoming ACK frames
- MINOR: quic: Congestion controller event trace fix (loss)
- MINOR: quic: Add correct ack delay values to ACK frames
- MINOR: config: Add "cluster-secret" new global keyword
- MINOR: quic-tls: Add quic_hkdf_extract_and_expand() for HKDF
- MINOR: quic: new_quic_cid() code moving
- MINOR: quic: Initialize stateless reset tokens with HKDF secrets
- MINOR: qc_new_conn() rework for stateless reset
- MINOR: quic: Stateless reset token copy to transport parameters
- MINOR: quic: Send stateless reset tokens
- MINOR: quic: Short packets always embed a trailing AEAD TAG
- CLEANUP: quic: wrong use of eb*entry() macro
- CLEANUP: quic: Useless use of pointer for quic_hkdf_extract()
- CLEANUP: quic_tls: QUIC_TLS_IV_LEN defined two times
- MINOR: ncbuf: define non-contiguous buffer
- MINOR: ncbuf: complete API and define block interal abstraction
- MINOR: ncbuf: optimize storage for the last gap
- MINOR: ncbuf: implement insertion
- MINOR: ncbuf: define various insertion modes
- MINOR: ncbuf: implement advance
- MINOR: ncbuf: write unit tests
- BUG/MEDIUM: lua: fix argument handling in data removal functions
- DOC/MINOR: fix typos in the lua-api document
- BUG/MEDIUM: wdt: don't trigger the watchdog when p is unitialized
- MINOR: mux-h1: Add global option accpet payload for any HTTP/1.0 requests
- CLEANUP: mux-h1: Fix comments and error messages for global options
- MINOR: conn_stream: make cs_set_error() work on the endpoint instead
- CLEANUP: mux-h1: always take the endp from the h1s not the cs
- CLEANUP: mux-h2: always take the endp from the h2s not the cs
- CLEANUP: mux-pt: always take the endp from the context not the cs
- CLEANUP: mux-fcgi: always take the endp from the fstrm not the cs
- CLEANUP: mux-quic: always take the endp from the qcs not the cs
- CLEANUP: applet: use the appctx's endp instead of cs->endp
- MINOR: conn_stream: add a pointer back to the cs from the endpoint
- MINOR: mux-h1: remove the now unneeded h1s->cs
- MINOR: mux-h2: make sure any h2s always has an endpoint
- MINOR: mux-h2: remove the now unneeded conn_stream from the h2s
- MINOR: mux-fcgi: make sure any stream always has an endpoint
- MINOR: mux-fcgi: remove the now unneeded conn_stream from the fcgi_strm
- MINOR: mux-quic: remove the now unneeded conn_stream from the qcs
- MINOR: mux-pt: remove the now unneeded conn_stream from the context
- CLEANUP: muxes: make mux->attach/detach take a conn_stream endpoint
- MINOR: applet: replace cs_applet_shut() with appctx_shut()
- MINOR: applet: add appctx_strm() and appctx_cs() to access common fields
- CLEANUP: applet: remove the unneeded appctx->owner
- CLEANUP: conn_stream: merge cs_new_from_{mux,applet} into cs_new_from_endp()
- MINOR: ext-check: indicate the transport and protocol of a server
- BUG/MEDIUM: mux-quic: fix a thinko in the latest cs/endpoint cleanup
- MINOR: tools: improve error message accuracy in str2sa_range
- MINOR: config: make sure never to mix dgram and stream protocols on a bind line
- BUG/MINOR: ncbuf: fix coverity warning on uninit sz_data
- MINOR: xprt_quic: adjust flow-control according to bufsize
- MEDIUM: mux-quic/h3/hq-interop: use ncbuf for bidir streams
- MEDIUM: mux-quic/h3/qpack: use ncbuf for uni streams
- CLEANUP: mux-quic: remove unused fields for Rx
- CLEANUP: quic: remove unused quic_rx_strm_frm
Valerio Pachera explained [1] that external checks would benefit from
having a variable indicating if SSL is being used or not on the server
being checked, and the discussion derived to also indicating the protocol
in use.
This patch adds two environment variables for external checks:
- HAPROXY_SERVER_SSL: equals "0" when SSL is not used, "1" when it is
- HAPROXY_SERVER_PROTO: contains one of the following words to describe
the protocol used with this server:
- "cli": the haproxy CLI. Normally not seen
- "syslog": this is a syslog TCP server
- "peers": this is a peers TCP server
- "h1": this is an HTTP/1.x server
- "h2": this is an HTTP/2 server
- "tcp": this is any other TCP server
The patch is very simple, and may be backported to recent versions if
needed. This closes github issue #1692.
[1] https://www.mail-archive.com/haproxy@formilux.org/msg42233.html
Since the 2.5, for security reason, HTTP/1.0 GET/HEAD/DELETE requests with a
payload are rejected (See e136bd12a "MEDIUM: mux-h1: Reject HTTP/1.0
GET/HEAD/DELETE requests with a payload" for details). However it may be an
issue for old clients.
To avoid any compatibility issue with such clients,
"h1-accept-payload-with-any-method" global option was added. It must only be
set if there is a good reason to do so because it may lead to a request
smuggling attack on some servers or intermediaries.
This patch should solve the issue #1691. it may be backported to 2.5.
It could be usefull to set a ASCII secret which could be used for different
usages. For instance, it will be used to derive QUIC stateless reset tokens.
Released version 2.6-dev9 with the following main changes :
- MINOR: mux-quic: support full request channel buffer
- BUG/MINOR: h3: fix parsing of unknown frame type with null length
- CLEANUP: backend: make alloc_{bind,dst}_address() idempotent
- MEDIUM: stream: remove the confusing SF_ADDR_SET flag
- MINOR: conn_stream: remove the now unused CS_FL_ADDR_*_SET flags
- CLEANUP: protocol: make sure the connect_* functions always receive a dst
- MINOR: connection: get rid of the CO_FL_ADDR_*_SET flags
- MINOR: session: get rid of the now unused SESS_FL_ADDR_*_SET flags
- CLEANUP: mux: Useless xprt_quic-t.h inclusion
- MINOR: quic: Make the quic_conn be aware of the number of streams
- BUG/MINOR: quic: Dropped retransmitted STREAM frames
- BUG/MINOR: mux_quic: Dropped packet upon retransmission for closed streams
- MEDIUM: httpclient: remove url2sa to use a more flexible parser
- MEDIUM: httpclient: http-request rules for resolving
- MEDIUM: httpclient: allow address and port change for resolving
- CLEANUP: httpclient: remove the comment about resolving
- MINOR: httpclient: handle unix and other socket types in dst
- MINOR: httpclient: rename dash by dot in global option
- MINOR: init: exit() after pre-check upon error
- MINOR: httpclient: cleanup the error handling in init
- MEDIUM: httpclient: hard-error when SSL is configured
- MINOR: httpclient: allow to configure the ca-file
- MINOR: httpclient: configure the resolvers section to use
- MINOR: httpclient: allow ipv4 or ipv6 preference for resolving
- DOC: configuration: httpclient global option
- MINOR: conn-stream: Add mask from flags set by endpoint or app layer
- BUG/MEDIUM: conn-stream: Only keep app layer flags of the endpoint on reset
- BUG/MEDIUM: mux-fcgi: Be sure to never set EOM flag on an empty HTX message
- BUG/MEDIUM: mux-h1: Be able to handle trailers when C-L header was specified
- DOC: config: Update doc for PR/PH session states to warn about rewrite failures
- MINOR: resolvers: cleanup alert/warning in parse-resolve-conf
- MINOR: resolvers: move the resolv.conf parser in parse_resolv_conf()
- MINOR: resolvers: resolvers_new() create a resolvers with default values
- BUILD: debug: unify the definition of ha_backtrace_to_stderr()
- BUG/MINOR: tcp/http: release the expr of set-{src,dst}[-port]
- MEDIUM: resolvers: create a "default" resolvers section at startup
- DOC: resolvers: default resolvers section
- BUG/MINOR: startup: usage() when no -cc arguments
- BUG/MEDIUM: resolvers: make "show resolvers" properly yield
- BUG/MEDIUM: cli: make "show cli sockets" really yield
- BUG/MINOR: proxy/cli: don't enumerate internal proxies on "show backend"
- BUG/MINOR: map/cli: protect the backref list during "show map" errors
- BUG/MINOR: map/cli: make sure patterns don't vanish under "show map"'s init
- BUG/MINOR: ssl/cli: fix "show ssl ca-file/crl-file" not to mix cli+ssl contexts
- BUG/MINOR: ssl/cli: fix "show ssl ca-file <name>" not to mix cli+ssl contexts
- BUG/MINOR: ssl/cli: fix "show ssl crl-file" not to mix cli+ssl contexts
- BUG/MINOR: ssl/cli: fix "show ssl cert" not to mix cli+ssl contexts
- CLEANUP: ssl/cli: do not loop on unknown states in "add ssl crt-list" handler
- MINOR: applet: reserve some generic storage in the applet's context
- CLEANUP: applet: make appctx_new() initialize the whole appctx
- CLEANUP: stream/cli: take the "show sess" context definition out of the appctx
- CLEANUP: stream/cli: stop using appctx->st2 for the dump state
- CLEANUP: stream/cli: remove the unneeded init state from "show sess"
- CLEANUP: stream/cli: remove the unneeded STATE_FIN state from "show sess"
- CLEANUP: stream/cli: remove the now unneeded dump state from "show sess"
- CLEANUP: proxy/cli: take the "show errors" context definition out of the appctx
- CLEANUP: stick-table/cli: take the "show table" context definition out of the appctx
- CLEANUP: stick-table/cli: stop using appctx->st2 for the dump state
- CLEANUP: stick-table/cli: remove the unneeded STATE_INIT for "show table"
- CLEANUP: map/cli: take the "show map" context definition out of the appctx
- CLEANUP: map/cli: stop using cli.i0/i1 to store the generation numbers
- CLEANUP: map/cli: stop using appctx->st2 for the dump state
- CLEANUP: map/cli: always detach the backref from the list after "show map"
- CLEANUP: peers/cli: take the "show peers" context definition out of the appctx
- CLEANUP: peers/cli: stop using appctx->st2 for the dump state
- CLEANUP: peers/cli: remove unneeded state STATE_INIT
- CLEANUP: cli: initialize the whole appctx->ctx, not just the stats part
- CLEANUP: promex: make the applet use its own context
- CLEANUP: promex: stop using appctx->st2
- CLEANUP: stats/cli: take the "show stat" context definition out of the appctx
- CLEANUP: stats/cli: stop using appctx->st2
- CLEANUP: hlua/cli: take the hlua_cli context definition out of the appctx
- CLEANUP: ssl/cli: use a local context for "show cafile"
- CLEANUP: ssl/cli: use a local context for "show crlfile"
- CLEANUP: ssl/cli: use a local context for "show ssl cert"
- CLEANUP: ssl/cli: use a local context for "commit ssl cert"
- CLEANUP: ssl/cli: stop using appctx->st2 for "commit ssl cert"
- CLEANUP: ssl/cli: use a local context for "set ssl cert"
- CLEANUP: ssl/cli: use a local context for "set ssl cafile"
- CLEANUP: ssl/cli: use a local context for "set ssl crlfile"
- CLEANUP: ssl/cli: use a local context for "commit ssl {ca|crl}file"
- CLEANUP: ssl/cli: stop using appctx->st2 for "commit ssl ca/crl"
- CLEANUP: ssl/cli: stop using ctx.cli.i0/i1/p0 for "show tls-keys"
- CLEANUP: ssl/cli: add a new "dump_entries" field to "show_keys_ref"
- CLEANUP: ssl/cli: make "show tlskeys" not use appctx->st2 anymore
- CLEANUP: ssl/cli: make "show ssl ocsp-response" not use cli.p0 anymore
- CLEANUP: ssl/cli: make "{show|dump} ssl crtlist" use its own context
- CLEANUP: ssl/cli: make "add ssl crtlist" use its own context
- CLEANUP: ssl/cli: make "add ssl crtlist" not use st2 anymore
- CLEANUP: dns: stop abusing the sink forwarder's context
- CLEANUP: sink: use the generic context to store the forwarder's context
- CLEANUP: activity/cli: make "show profiling" not use ctx.cli anymore
- CLEANUP: debug/cli: make "debug dev fd" not use ctx.cli anymore
- CLEANUP: debug/cli: make "debug dev memstats" not use ctx.cli anymore
- CLEANUP: ring: pass the ring watch flags to ring_attach_cli(), not in ctx.cli
- CLEANUP: ring/cli: use a locally-defined context instead of using ctx.cli
- CLEANUP: resolvers/cli: make "show resolvers" use a locally-defined context
- CLEANUP: resolvers/cli: remove the unneeded appctx->st2 from "show resolvers"
- CLEANUP: cache/cli: make use of a locally defined context for "show cache"
- CLEANUP: proxy/cli: make use of a locally defined context for "show servers"
- CLEANUP: proxy/cli: get rid of appctx->st2 in "show servers"
- CLEANUP: proxy/cli: make "show backend" only use the generic context
- CLEANUP: cli: make "show fd" use its own context
- CLEANUP: cli: make "show env" use its own context
- CLEANUP: cli: simplify the "show cli sockets" I/O handler
- CLEANUP: cli: make "show cli sockets" use its own context
- CLEANUP: httpclient/cli: use a locally-defined context instead of ctx.cli
- CLEANUP: httpclient: do not use the appctx.ctx anymore
- CLEANUP: peers: do not use appctx.ctx anymore
- CLEANUP: spoe: do not use appctx.ctx anymore
- BUILD: applet: mark the CLI's generic variables as deprecated
- BUILD: applet: mark the appctx's st2 variable as deprecated
- CLEANUP: cache: take the context out of appctx.ctx
- MEDIUM: lua: move the cosocket storage outside of appctx.ctx
- MINOR: lua: move the tcp service storage outside of appctx.ctx
- MINOR: lua: move the http service context out of appctx.ctx
- CLEANUP: cli: move the status print context into its own context
- CLEANUP: stats: rename the stats state values an mark the old ones deprecated
- DOC: internal: document the new cleaner approach to the appctx
- MINOR: tcp: socket translate TCP_KEEPIDLE for macOs equivalent
- DOC: fix typo "ant" for "and" in INSTALL
- CI: dynamically determine actual version of h2spec
When an HTTP header rewrite failure is triggered, and 500-internal-error
response is returned. A "PR" termination state is logged if the error
occurred on the request and "PH" if the error is reported for the response.
The documentation was updated accordingly.
This patch is related to issue #1597.
Documentation about the 4 options in the global section for the
httpclient:
- httpclient.ssl.verify
- httpclient.ssl.ca-file
- httpclient.resolvers.id
- httpclient.resolvers.prefer
Released version 2.6-dev8 with the following main changes :
- BUG/MINOR: quic: fix use-after-free with trace on ACK consume
- BUG/MINOR: rules: Forbid captures in defaults section if used by a backend
- BUG/MEDIUM: rules: Be able to use captures defined in defaults section
- BUG/MINOR: rules: Fix check_capture() function to use the right rule arguments
- BUG/MINOR: http-act: make release_http_redir() more robust
- BUG/MINOR: sample: add missing use_backend/use-server contexts in smp_resolve_args
- MINOR: sample: don't needlessly call c_none() in sample_fetch_as_type()
- MINOR: sample: make the bool type cast to bin
- MEDIUM: backend: add new "balance hash <expr>" algorithm
- MINOR: init: add global setting "fd-hard-limit" to bound system limits
- BUILD: pollers: use an initcall to register the pollers
- BUILD: xprt: use an initcall to register the transport layers
- BUILD: thread: use initcall instead of a constructor
- BUILD: http: remove the two unused constructors in rules and ana
- CLEANUP: compression: move the default setting of maxzlibmem to defaults
- MINOR: tree-wide: always consider EWOULDBLOCK in addition to EAGAIN
- BUG/MINOR: connection: "connection:close" header added despite 'close-spread-time'
- MINOR: fd: add functions to set O_NONBLOCK and FD_CLOEXEC
- CLEANUP: tree-wide: use fd_set_nonblock() and fd_set_cloexec()
- CLEANUP: tree-wide: remove 25 occurrences of unneeded fcntl.h
- REGTESTS: fix the race conditions in be2dec.vtc ad field.vtc
- REGTESTS: webstats: remove unused stats socket in /tmp
- MEDIUM: httpclient: disable SSL when the ca-file couldn't be loaded
- BUG/MINOR: httpclient/lua: error when the httpclient_start() fails
- BUG/MINOR: ssl: free the cafile entries on deinit
- BUG/MINOR: ssl: memory leak when trying to load a directory with ca-file
- MEDIUM: httpclient: re-enable the verify by default
- BUG/MEDIUM: ssl/cli: fix yielding in show_cafile_detail
- BUILD: compiler: properly distinguish weak and global symbols
- MINOR: connection: Add way to disable active connection closing during soft-stop
- BUG/MEDIUM: http-ana: Fix memleak in redirect rules with ignore-empty option
- CLEANUP: Destroy `http_err_chunks` members during deinit
- BUG/MINOR: resolvers: Fix memory leak in resolvers_deinit()
- MINOR: Call deinit_and_exit(0) for `haproxy -vv`
- BUILD: fd: disguise the fd_set_nonblock/cloexec result
- BUG/MINOR: pools: make sure to also destroy shared pools in pool_destroy_all()
- MINOR: ssl: add a new global option "tune.ssl.hard-maxrecord"
- CLEANUP: errors: also call deinit_errors_buffers() on deinit()
- CLEANUP: chunks: release trash also in deinit
- CLEANUP: deinit: release the pre-check callbacks
- CLEANUP: deinit: release the config postparsers
- CLEANUP: listeners/deinit: release accept queue tasklets on deinit
- CLEANUP: connections/deinit: destroy the idle_conns tasks
- BUG/MINOR: mux-quic: fix build in release mode
- MINOR: mux-quic: adjust comment on emission function
- MINOR: mux-quic: remove unused bogus qcc_get_stream()
- BUG/MINOR: mux-quic: fix leak if cs alloc failure
- MINOR: mux-quic: count local flow-control stream limit on reception
- BUG/MINOR: h3: fix incomplete POST requests
- BUG/MEDIUM: h3: fix use-after-free on mux Rx buffer wrapping
- MINOR: mux-quic: partially copy Rx frame if almost full buf
- MINOR: h3: change frame demuxing API
- MINOR: mux-quic: add a app-layer context in qcs
- MINOR: h3: implement h3 stream context
- MINOR: h3: support DATA demux if buffer full
- MINOR: quic: decode as much STREAM as possible
- MINOR: quic: Improve qc_prep_pkts() flexibility
- MINOR: quic: Prepare quic_frame struct duplication
- MINOR: quic: Do not retransmit frames from coalesced packets
- MINOR: quic: Add traces about TX frame memory releasing
- MINOR: quic: process_timer() rework
- MEDIUM: quic: New functions for probing rework
- MEDIUM: quic: Retransmission functions rework
- MEDIUM: quic: qc_requeue_nacked_pkt_tx_frms() rework
- MINOR: quic: old data distinction for qc_send_app_pkt()
- MINOR: quic: Mark packets as probing with old data
- MEDIUM: quic: Mark copies of acknowledged frames as acknowledged
- MEDIUM: quic: Enable the new datagram probing process
- MINOR: quic: Do not send ACK frames when probing
- BUG/MINOR: quic: Wrong returned status by qc_build_frms()
- BUG/MINOR: quic: Avoid sending useless PADDING frame
- BUG/MINOR: quic: Traces fix about remaining frames upon packet build failure
- MINOR: quic: Wake up the mux to probe with new data
- BUG/MEDIUM: quic: Possible crash on STREAM frame loss
- BUG/MINOR: quic: Missing Initial packet length check
- CLEANUP: quic: Rely on the packet length set by qc_lstnr_pkt_rcv()
- MINOR: quic: Drop 0-RTT packets if not allowed
- BUG/MINOR: httpclient/ssl: use the correct verify constant
- BUG/MEDIUM: conn-stream: Don't erase endpoint flags on reset
- BUG/MEDIUM: httpclient: Fix loop consuming HTX blocks from the response channel
- BUG/MINOR: httpclient: Count metadata in size to transfer via htx_xfer_blks()
- MINOR: httpclient: Don't use co_set_data() to decrement output
- BUG/MINOR: conn_stream: do not confirm a connection from the frontend path
- MEDIUM: quic: do not ACK packet with STREAM if MUX not present
- MEDIUM: quic: do not ack packet with invalid STREAM
- MINOR: quic: Drop 0-RTT packets without secrets
- CLEANUP: quic: Remaining fprintf() debug trace
- MINOR: quic: moving code for QUIC loss detection
- BUG/MINOR: quic: Missing time threshold multiplifier for loss delay computation
- CI: github actions: update LibreSSL to 3.5.2
- SCRIPTS: announce-release: add URL of dev packages
Low footprint client machines may not have enough memory to download a
complete 16KB TLS record at once. With the new option the maximum
record size can be defined on the server side.
Note: Before limiting the the record size on the server side, a client should
consider using the TLS Maximum Fragment Length Negotiation Extension defined
in RFC6066.
This patch fixes GitHub issue #1679.
If the "close-spread-time" option is set to "infinite", active
connection closing during a soft-stop can be disabled. The 'connection:
close' header or the GOAWAY frame will not be added anymore to the
server's response and active connections will only be closed once the
clients disconnect. Idle connections will not be closed all at once when
the soft-stop starts anymore, and each idle connection will follow its
own timeout based on the multiple timeouts set in the configuration (as
is the case during regular execution).
This feature request was described in GitHub issue #1614.
This patch should be backported to 2.5. It depends on 'MEDIUM: global:
Add a "close-spread-time" option to spread soft-stop on time window'.
On some systems, the hard limit for ulimit -n may be huge, in the order
of 1 billion, and using this to automatically compute maxconn doesn't
work as it requires way too much memory. Users tend to hard-code maxconn
but that's not convenient to manage deployments on heterogenous systems,
nor when porting configs to developers' machines. The ulimit-n parameter
doesn't work either because it forces the limit. What most users seem to
want (and it makes sense) is to respect the system imposed limits up to
a certain value and cap this value. This is exactly what fd-hard-limit
does.
This addresses github issue #1622.
Almost all of our hash-based LB algorithms are implemented as special
cases of something that can now be achieved using sample expressions,
and some of them have adopted some options to adapt their behavior in
ways that could also be achieved using converters.
There are users who want to hash other parameters that are combined
into variables, and who set headers from these values and use
"balance hdr(name)" for this.
Instead of constantly implementing specific options and having users
hack around when they want a real hash, let's implement a native hash
mode that applies to a standard sample expression. This way, any
fetchable element (including variables) may be used to construct the
hash, even modified by any converter if desired.
Released version 2.6-dev7 with the following main changes :
- BUILD: calltrace: fix wrong include when building with TRACE=1
- MINOR: ssl: Use DH parameters defined in RFC7919 instead of hard coded ones
- MEDIUM: ssl: Disable DHE ciphers by default
- BUILD: ssl: Fix compilation with OpenSSL 1.0.2
- MINOR: mux-quic: split xfer and STREAM frames build
- REORG: quic: use a dedicated module for qc_stream_desc
- MINOR: quic-stream: use distinct tree nodes for quic stream and qcs
- MINOR: quic-stream: add qc field
- MEDIUM: quic: implement multi-buffered Tx streams
- MINOR: quic-stream: refactor ack management
- MINOR: quic: limit total stream buffers per connection
- MINOR: mux-quic: implement immediate send retry
- MINOR: cfg-quic: define tune.quic.conn-buf-limit
- MINOR: ssl: Add 'show ssl providers' cli command and providers list in -vv option
- REGTESTS: ssl: Update error messages that changed with OpenSSLv3.1.0-dev
- BUG/MEDIUM: quic: Possible crash with released mux
- BUG/MINOR: mux-quic: unsubscribe on release
- BUG/MINOR: mux-quic: handle null timeout
- BUG/MEDIUM: logs: fix http-client's log srv initialization
- BUG/MINOR: mux-quic: remove dead code in qcs_xfer_data()
- DEV: stream: Fix conn-streams dump in full stream message
- CLEANUP: conn-stream: Rename cs_conn_close() and cs_conn_drain_and_close()
- CLEANUP: conn-stream: Rename cs_applet_release()
- MINOR: conn-stream: Rely on endpoint shutdown flags to shutdown an applet
- BUG/MINOR: cache: Disable cache if applet creation fails
- BUG/MINOR: backend: Don't allow to change backend applet
- BUG/MEDIUM: conn-stream: Set back CS to RDY state when the appctx is created
- MINOR: stream: Don't needlessly detach server endpoint on early client abort
- MINOR: conn-stream: Make cs_detach_* private and use cs_destroy() from outside
- MINOR: init: add the pre-check callback
- MEDIUM: httpclient: change the init sequence
- MEDIUM: httpclient/ssl: verify required
- MINOR: httpclient/mworker: disable in the master process
- MEDIUM: httpclient/ssl: verify is configurable and disabled by default
- BUG/MAJOR: connection: Never remove connection from idle lists outside the lock
- BUG/MEDIUM: mux-quic: fix stalled POST requets
- BUG/MINOR: mux-quic: fix POST with abortonclose
- MINOR: task: add a new task_instant_wakeup() function
- MEDIUM: queue: use tasklet_instant_wakeup() to wake tasks
- DOC: remove my name from the config doc
I was surprised to notice that my name was still present as the author
at the top of the config manual. It turns out that this line and a few
other ones in this file remained unchanged since commit 6a06a40501 that
added this doc 15 years ago! It's long been time to get rid of this!
This adds a call to function <fct> to the list of functions to be called at
the step just before the configuration validity checks. This is useful when you
need to create things like it would have been done during the configuration
parsing and where the initialization should continue in the configuration
check.
It could be used for example to generate a proxy with multiple servers using
the configuration parser itself. At this step the trash buffers are allocated.
Threads are not yet started so no protection is required. The function is
expected to return non-zero on success, or zero on failure. A failure will make
the process emit a succinct error message and immediately exit.
Starting from OpenSSLv3, providers are at the core of cryptography
functions. Depending on the provider used, the way the SSL
functionalities work could change. This new 'show ssl providers' CLI
command allows to show what providers were loaded by the SSL library.
This is required because the provider configuration is exclusively done
in the OpenSSL configuration file (/usr/local/ssl/openssl.cnf for
instance).
A new line is also added to the 'haproxy -vv' output containing the same
information.
DHE ciphers do not present a security risk if the key is big enough but
they are slow and mostly obsoleted by ECDHE. This patch removes any
default DH parameters. This will effectively disable all DHE ciphers
unless a global ssl-dh-param-file is defined, or
tune.ssl.default-dh-param is set, or a frontend has DH parameters
included in its PEM certificate. In this latter case, only the frontends
that have DH parameters will have DHE ciphers enabled.
Adding explicitely a DHE ciphers in a "bind" line will not be enough to
actually enable DHE. We would still need to know which DH parameters to
use so one of the three conditions described above must be met.
This request was described in GitHub issue #1604.
Released version 2.6-dev6 with the following main changes :
- CLEANUP: connection: reduce the with of the mux dump output
- CI: Update to actions/checkout@v3
- CI: Update to actions/cache@v3
- DOC: adjust QUIC instruction in INSTALL
- BUG/MINOR: stats: define the description' background color in dark color scheme
- BUILD: ssl: add USE_ENGINE and disable the openssl engine by default
- BUILD: makefile: pass USE_ENGINE to cflags
- BUILD: xprt-quic: replace ERR_func_error_string() with ERR_peek_error_func()
- DOC: install: document the fact that SSL engines are not enabled by default
- CI: github actions: disable -Wno-deprecated
- BUILD: makefile: silence unbearable OpenSSL deprecation warnings
- MINOR: sock: check configured limits at the sock layer, not the listener's
- MINOR: connection: add a new flag CO_FL_FDLESS on fd-less connections
- MINOR: connection: add conn_fd() to retrieve the FD only when it exists
- MINOR: stream: only dump connections' FDs when they are valid
- MINOR: connection: use conn_fd() when displaying connection errors
- MINOR: connection: skip FD-based syscalls for FD-less connections
- MEDIUM: connection: panic when calling FD-specific functions on FD-less conns
- MINOR: mux-quic: properly set the flags and name fields
- MINOR: connection: rearrange conn_get_src/dst to be a bit more extensible
- MINOR: protocol: add get_src() and get_dst() at the protocol level
- MINOR: quic-sock: provide a pair of get_src/get_dst functions
- MEDIUM: ssl: improve retrieval of ssl_sock_ctx and SSL detection
- MEDIUM: ssl: stop using conn->xprt_ctx to access the ssl_sock_ctx
- MEDIUM: xprt-quic: implement get_ssl_sock_ctx()
- MEDIUM: quic: move conn->qc into conn->handle
- BUILD: ssl: fix build warning with previous changes to ssl_sock_ctx
- BUILD: ssl: add an unchecked version of __conn_get_ssl_sock_ctx()
- MINOR: ssl: refine the error testing for fc_err and fc_err_str
- BUG/MINOR: sock: do not double-close the accepted socket on the error path
- CI: cirrus: switch to FreeBSD-13.0
- MINOR: log: add '~' to frontend when the transport layer provides SSL
- BUILD/DEBUG: lru: fix printf format in debug code
- BUILD: peers: adjust some printf format to silence cppcheck
- BUILD/DEBUG: hpack-tbl: fix format string in standalone debug code
- BUILD/DEBUG: hpack: use unsigned int in printf format in debug code
- BUILD: halog: fix some incorrect signs in printf formats for integers
- BUG/MINOR: h3: fix build with DEBUG_H3
- BUG/MINOR: mux-h2: do not send GOAWAY if SETTINGS were not sent
- BUG/MINOR: cache: do not display expired entries in "show cache"
- BUG/MINOR: mux-h1: Don't release unallocated CS on error path
- MINOR: applet: Make .init callback more generic
- MINOR: conn-stream: Add flags to set the type of the endpoint
- MEDIUM: applet: Set the appctx owner during allocation
- MAJOR: conn-stream: Invert conn-stream endpoint and its context
- REORG: Initialize the conn-stream by hand in cs_init()
- MEDIUM: conn-stream: Add an endpoint structure in the conn-stream
- MINOR: conn-stream: Move some CS flags to the endpoint
- MEDIUM: conn-stream: Be able to pass endpoint to create a conn-stream
- MEDIUM: conn-stream: Pre-allocate endpoint to create CS from muxes and applets
- REORG: applet: Uninline appctx_new function
- MAJOR: conn-stream: Share endpoint struct between the CS and the mux/applet
- MEDIUM: conn-stream: Move remaning flags from CS to endpoint
- MINOR: mux-pt: Rely on the endpoint instead of the conn-stream when possible
- MINOR: conn-stream: Add ISBACK conn-stream flag
- MINOR: conn-stream: Add header file with util functions related to conn-streams
- MEDIUM: tree-wide: Use CS util functions instead of SI ones
- MINOR: stream-int/txn: Move buffer for L7 retries in the HTTP transaction
- CLEANUP: http-ana: Remove http_alloc_txn() function
- MINOR: stream-int/stream: Move conn_retries counter in the stream
- MINOR: stream: Simplify retries counter calculation
- MEDIUM: stream-int/conn-stream: Move src/dst addresses in the conn-stream
- MINOR: stream-int/conn-stream: Move half-close timeout in the conn-stream
- MEDIUM: stream-int/stream: Use connect expiration instead of SI expiration
- MINOR: stream-int/conn-stream: Report error to the CS instead of the SI
- MEDIUM: conn-stream: Use endpoint error instead of conn-stream error
- MINOR: channel: Use conn-streams as channel producer and consumer
- MINOR: stream-int: Remove SI_FL_KILL_CON to rely on conn-stream endpoint only
- MINOR: mux-h2/mux-fcgi: Fully rely on CS_EP_KILL_CONN
- MINOR: stream-int: Remove SI_FL_NOLINGER/NOHALF to rely on CS flags instead
- MINOR: stream-int: Remove SI_FL_DONT_WAKE to rely on CS flags instead
- MINOR: stream-int: Remove SI_FL_INDEP_STR to rely on CS flags instead
- MINOR: stream-int: Remove SI_FL_SRC_ADDR to rely on stream flags instead
- CLEANUP: stream-int: Remove unused SI_FL_CLEAN_ABRT flag
- MINOR: stream: Only save previous connection state for the server side
- MEDIUM: stream-int: Move SI err_type in the stream
- MEDIUM: stream-int/conn-stream: Move stream-interface state in the conn-stream
- MINOR: stream-int/stream: Move si_retnclose() in the stream scope
- MINOR: stream-int/backend: Move si_connect() in the backend scope
- MINOR: stream-int/conn-stream: Move si_conn_ready() in the conn-stream scope
- MINOR: conn-stream/connection: Move SHR/SHW modes in the connection scope
- MEDIUM: conn-stream: Be prepared to fail to attach a cs to a mux
- MEDIUM: stream-int/conn-stream: Handle I/O subscriptions in the conn-stream
- MINOR: conn-stream: Rename CS functions dedicated to connections
- MINOR: stream-int/conn-stream: Move si_shut* and si_chk* in conn-stream scope
- MEDIUM: stream-int/conn-stream: Move si_ops in the conn-stream scope
- MINOR: applet: Use the CS to register and release applets instead of SI
- MINOR: connection: unconst mux's get_fist_cs() callback function
- MINOR: stream-int/connection: Move conn_si_send_proxy() in the connection scope
- REORG: stream-int: Export si_cs_recv(), si_cs_send() and si_cs_process()
- REORG: stream-int: Move si_is_conn_error() in the header file
- REORG: conn-stream: Move cs_shut* and cs_chk* in cs_utils
- REORG: conn-stream: Move cs_app_ops in conn_stream.c
- MINOR: stream-int-conn-stream: Move si_update_* in conn-stream scope
- MINOR: stream-int/stream: Move si_update_both in stream scope
- MEDIUM: conn-stream/applet: Add a data callback for applets
- MINOR: stream-int/conn-stream: Move stream_int_read0() in the conn-stream scope
- MINOR: stream-int/conn-stream: Move stream_int_notify() in the conn-stream scope
- MINOR: stream-int/conn-stream: Move si_cs_io_cb() in the conn-stream scope
- MINOR: stream-int/conn-stream: Move si_sync_recv/send() in conn-stream scope
- MINOR: conn-stream: Move si_conn_cb in the conn-stream scope
- MINOR: stream-int/conn-stream Move si_is_conn_error() in the conn-stream scope
- MINOR: stream-int/conn-stream: Move si_alloc_ibuf() in the conn-stream scope
- CLEANUP: stream-int: Remove unused SI functions
- MEDIUM: stream-int/conn-stream: Move blocking flags from SI to CS
- MEDIUM: stream-int/conn-stream: Move I/O functions to conn-stream
- REORG: stream-int/conn-stream: Move remaining functions to conn-stream
- MINOR: stream: Use conn-stream to report server error
- MINOR: http-ana: Use CS to perform L7 retries
- MEDIUM: stream: Don't use the stream-int anymore in process_stream()
- MINOR: conn-stream: Remove the stream-interface from the conn-stream
- DEV: flags: No longer dump SI flags
- CLEANUP: tree-wide: Remove any ref to stream-interfaces
- CLEANUP: conn-stream: Don't export internal functions
- DOC: conn-stream: Add comments on functions of the new CS api
- MEDIUM: check: Use a new conn-stream for each health-check run
- CLEANUP: muxes: Remove MX_FL_CLEAN_ABRT flag
- MINOR: conn-stream: Use a dedicated function to conditionally remove a CS
- CLEANUP: conn-stream: rename cs_register_applet() to cs_applet_create()
- MINOR: muxes: Improve show_fd callbacks to dump endpoint flags
- MINOR: mux-h1: Rely on the endpoint instead of the conn-stream when possible
- BUG/MINOR: quic: Avoid starting the mux if no ALPN sent by the client
- BUILD: debug: mark the __start_mem_stats/__stop_mem_stats symbols as weak
- BUILD: initcall: mark the __start_i_* symbols as weak, not global
- BUG/MINOR: mux-h2: do not use timeout http-keep-alive on backend side
- BUG/MINOR: mux-h2: use timeout http-request as a fallback for http-keep-alive
- MINOR: muxes: Don't expect to have a mux without connection in destroy callback
- MINOR: muxes: Don't handle proto upgrade for muxes not supporting it
- MINOR: muxes: Don't expect to call release function with no mux defined
- MINOR: conn-stream: Use unsafe functions to get conn/appctx in cs_detach_endp
- BUG/MEDIUM: mux-h1: Don't request more room on partial trailers
- BUILD: http-client: Avoid dead code when compiled without SSL support
- BUG/MINOR: mux-quic: prevent a crash in session_free on mux.destroy
- BUG/MINOR: quic-sock: do not double free session on conn init failure
- BUG/MINOR: quic: fix return value for error in start
- MINOR: quic: emit CONNECTION_CLOSE on app init error
- BUILD: sched: workaround crazy and dangerous warning in Clang 14
- BUILD: compiler: use a more portable set of asm(".weak") statements
- BUG/MEDIUM: stream: do not abort connection setup too early
- CLEANUP: extcheck: do not needlessly preset the server's address/port
- MINOR: extcheck: fill in the server's UNIX socket address when known
- BUG/MEDIUM: connection: Don't crush context pointer location if it is a CS
- BUG/MEDIUM: quic: properly clean frames on stream free
- BUG/MEDIUM: fcgi-app: Use http_msg flags to know if C-L header can be added
- BUG/MEDIUM: compression: Don't forget to update htx_sl and http_msg flags
- MINOR: tcp_sample: clarifying samples support per os, for further expansion.
- MINOR: tcp_sample: extend support for get_tcp_info to macOs.
- SCRIPTS: announce-release: update the doc's URL
- DOC: lua: update a few doc URLs
- SCRIPTS: announce-release: add shortened links to pending issues
The HAProxy doc was updated to point to docs.haproxy.org.
The HAProxy API doc was returning a 404, let's point to version 2.6.
This should be backported with 1.9dev modified to match the respective
versions.
Released version 2.6-dev5 with the following main changes :
- DOC: reflect H2 timeout changes
- BUG/MEDIUM: mux-fcgi: Properly handle return value of headers/trailers parsing
- BUG/MEDIUM: mux-h1: Properly detect full buffer cases during message parsing
- BUG/MINOR: log: Initialize the list element when allocating a new log server
- BUG/MINOR: samples: add missing context names for sample fetch functions
- MINOR: management: add some basic keyword dump infrastructure
- MINOR: config: add a function to dump all known config keywords
- MINOR: filters: extend flt_dump_kws() to dump to stdout
- MINOR: services: extend list_services() to dump to stdout
- MINOR: cli: add a new keyword dump function
- MINOR: acl: add a function to dump the list of known ACL keywords
- MINOR: samples: add a function to list register sample fetch keywords
- MINOR: sample: list registered sample converter functions
- MINOR: tools: add strordered() to check whether strings are ordered
- MINOR: action: add a function to dump the list of actions for a ruleset
- MINOR: config: alphanumerically sort config keywords output
- MINOR: sample: alphanumerically sort sample & conv keyword dumps
- MINOR: acl: alphanumerically sort the ACL dump
- MINOR: cli: alphanumerically sort the dump of supported commands
- MINOR: filters: alphabetically sort the list of filter names
- MINOR: services: alphabetically sort service names
- MEDIUM: httpclient/lua: be stricter with httpclient parameters
- MINOR: ssl: split the cert commit io handler
- MINOR: ssl: move the cert_exts and the CERT_TYPE enum
- MINOR: ssl: simplify the certificate extensions array
- MINOR: ssl: export ckch_inst_rebuild()
- MINOR: ssl: add "crt" in the cert_exts array
- MINOR: ssl/lua: CertCache.set() allows to update an SSL certificate file
- BUILD: ssl/lua: CacheCert needs OpenSSL
- DOC: lua: CertCache class documentation
- BUG/MEDIUM: quic: do not use qcs from quic_stream on ACK parsing
- MINOR: mux-quic: return qcs instance from qcc_get_qcs
- MINOR: mux-quic: reorganize qcs free
- MINOR: mux-quic: define release app-ops
- BUG/MINOR: h3: release resources on close
- BUG/MINOR: mux-quic: ensure to free all qcs on MUX release
- CLEANUP: quic: complete comment on qcs_try_to_consume
- MINOR: quic: implement stream descriptor for transport layer
- MEDIUM: quic: move transport fields from qcs to qc_conn_stream
- MEDIUM: mux-quic: remove qcs tree node
- BUG/MINOR: cli/stream: fix "shutdown session" to iterate over all threads
- DOC: management: add missing dot in 9.4.1
- BUG/MAJOR: mux_pt: always report the connection error to the conn_stream
- DOC: remove double blanks in configuration.txt
- CI: github actions: update OpenSSL to 3.0.2
- BUG/MEDIUM: quic: Possible crash in ha_quic_set_encryption_secrets()
- CLEANUP: quic: Remove all atomic operations on quic_conn struct
- CLEANUP: quic: Remove all atomic operations on packet number spaces
- MEDIUM: quic: Send ACK frames asap
- BUG/MINOR: quic: Missing probing packets when coalescing
- BUG/MINOR: quic: Discard Initial packet number space only one time
- MINOR: quic: Do not display any timer value from process_timer()
- BUG/MINOR: quic: Do not probe from an already probing packet number space
- BUG/MINOR: quic: Non duplicated frames upon fast retransmission
- BUG/MINOR: quic: Too much prepared retransmissions due to anti-amplification
- MINOR: quic: Useless call to SSL_CTX_set_default_verify_paths()
- MINOR: quic: Add traces about list of frames
- BUG/MINOR: h3: Missing wait event struct field initialization
- BUG/MINOR: quic: QUIC TLS secrets memory leak
- BUG/MINOR: quic: Missing ACK range deallocations
- BUG/MINOR: quic: Missing TX packet deallocations
- CLEANUP: hpack: be careful about integer promotion from uint8_t
- OPTIM: hpack: read 32 bits at once when possible.
- MEDIUM: ssl: allow loading of a directory with the ca-file directive
- BUG/MINOR: ssl: continue upon error when opening a directory w/ ca-file
- MINOR: ssl: ca-file @system-ca loads the system trusted CA
- DOC: configuration: add the ca-file changes
- MINOR: sample: converter: Add add_item convertor
- BUG/MINOR: ssl: handle X509_get_default_cert_dir() returning NULL
- BUG/MINOR: ssl/cli: Remove empty lines from CLI output
- MINOR: httpclient: enable request buffering
- MEDIUM: httpclient: enable l7-retry
- BUG/MINOR: httpclient: end callback in applet release
- MINOR: quic: Add draining connection state.
- MINOR: quic: Add closing connection state
- BUG/MEDIUM: quic: ensure quic-conn survives to the MUX
- CLEANUP: quic: use static qualifer on quic_close
- CLEANUP: mux-quic: remove unused QC_CF_CC_RECV
- BUG/MINOR: fix memleak on quic-conn streams cleaning
- MINOR: mux-quic: factorize conn-stream attach
- MINOR: mux-quic: adjust timeout to accelerate closing
- MINOR: mux-quic: define is_active app-ops
- MINOR: mux-quic: centralize send operations in qc_send
- MEDIUM: mux-quic: report CO_FL_ERROR on send
- MEDIUM: mux-quic: report errors on conn-streams
- MEDIUM: quic: report closing state for the MUX
- BUG/MINOR: fcgi-app: Don't add C-L header on response to HEAD requests
- BUG/MEDIUM: stats: Be sure to never set EOM flag on an empty HTX message
- BUG/MEDIUM: hlua: Don't set EOM flag on an empty HTX message in HTTP applet
- BUG/MEDIUM: promex: Be sure to never set EOM flag on an empty HTX message
- BUG/MEDIUM: mux-h1: Set outgoing message to DONE when payload length is reached
- BUG/MINOR: http_client: Don't add input data on an empty request buffer
- BUG/MEDIUM: http-conv: Fix url_enc() to not crush const samples
- BUG/MEDIUM: http-act: Don't replace URI if path is not found or invalid
- CLEANUP: mux-quic: remove uneeded TODO in qc_detach
- BUG/MEDIUM: mux-quic: properly release conn-stream on detach
- BUG/MINOR: quic: set the source not the destination address on accept()
- BUG/MEDIUM: quic: Possible crash from quic_free_arngs()
- MINOR: quic_tls: Add reusable cipher contexts to QUIC TLS contexts
- MINOR: quic_tls: Stop hardcoding cipher IV lengths
- CLEANUP: quic: Do not set any cipher/group from ssl_quic_initial_ctx()
- MINOR: quic: Add short packet key phase bit values to traces
- MINOR: quic_tls: Make key update use of reusable cipher contexts
- BUG/MINOR: opentracing: setting the return value in function flt_ot_var_set()
- BUG/BUILD: opentracing: fixed OT_DEFINE variable setting
- EXAMPLES: opentracing: refined shell scripts for testing filter performance
- DOC: opentracing: corrected comments in function descriptions
- CLEANUP: opentracing: removed unused function flt_ot_var_unset()
- CLEANUP: opentracing: removed unused function flt_ot_var_get()
- Revert "MINOR: opentracing: change the scope of the variable 'ot.uuid' from 'sess' to 'txn'"
- MINOR: opentracing: only takes the variables lock on shared entries
- CLEANUP: opentracing: added flt_ot_smp_init() function
- CLEANUP: opentracing: added variable to store variable length
- MINOR: opentracing: improved normalization of context variable names
- DEBUG: opentracing: show return values of all functions in the debug output
- CLEANUP: opentracing: added FLT_OT_PARSE_INVALID_enum enum
- DEBUG: opentracing: display the contents of the err variable after setting
- MAJOR: opentracing: reenable usage of vars to transmit opentracing context
- Revert "BUILD: opentracing: display warning in case of using OT_USE_VARS at compile time"
- MEDIUM: global: Add a "close-spread-time" option to spread soft-stop on time window
The new 'close-spread-time' global option can be used to spread idle and
active HTTP connction closing after a SIGUSR1 signal is received. This
allows to limit bursts of reconnections when too many idle connections
are closed at once. Indeed, without this new mechanism, in case of
soft-stop, all the idle connections would be closed at once (after the
grace period is over), and all active HTTP connections would be closed
by appending a "Connection: close" header to the next response that goes
over it (or via a GOAWAY frame in case of HTTP2).
This patch adds the support of this new option for HTTP as well as HTTP2
connections. It works differently on active and idle connections.
On active connections, instead of sending systematically the GOAWAY
frame or adding the 'Connection: close' header like before once the
soft-stop has started, a random based on the remainder of the close
window is calculated, and depending on its result we could decide to
keep the connection alive. The random will be recalculated for any
subsequent request/response on this connection so the GOAWAY will still
end up being sent, but we might wait a few more round trips. This will
ensure that goaways are distributed along a longer time window than
before.
On idle connections, a random factor is used when determining the expire
field of the connection's task, which should naturally spread connection
closings on the time window (see h2c_update_timeout).
This feature request was described in GitHub issue #1614.
This patch should be backported to 2.5. It depends on "BUG/MEDIUM:
mux-h2: make use of http-request and keep-alive timeouts" which
refactorized the timeout management of HTTP2 connections.
This new converter is similar to the concat converter and can be used to
build new variables made of a succession of other variables but the main
difference is that it does the checks if adding a delimiter makes sense as
wouldn't be the case if e.g the current input sample is empty. That
situation would require 2 separate rules using concat converter where the
first rule would have to check if the current sample string is empty before
adding a delimiter. This resolves GitHub Issue #1621.
It's difficult from outside haproxy to detect the supported keywords
and syntax. Interestingly, many of our modern keywords are enumerated
since they're registered from constructors, so it's not very hard to
enumerate most of them.
This patch creates some basic infrastructure to support dumping existing
keywords from different classes on stdout. The format will differ depending
on the classes, but the idea is that the output could easily be passed to
a script that generates some simple syntax highlighting rules, completion
rules for editors, syntax checkers or config parsers.
The principle chosen here is that if "-dK" is passed on the command-line,
at the end of the parsing the registered keywords will be dumped for the
requested classes passed after "-dK". Special name "help" will show known
classes, while "all" will execute all of them. The reason for doing that
after the end of the config processor is that it will also enumerate
internally-generated keywords, Lua or even those loaded from external
code (e.g. if an add-on is loaded using LD_PRELOAD). A typical way to
call this with a valid config would be:
./haproxy -dKall -q -c -f /path/to/config
If there's no config available, feeding /dev/null will also do the job,
though it will not be able to detect dynamically created keywords, of
course.
This patch also updates the management doc.
For now nothing but the help is listed, various subsystems will follow
in subsequent patches.
Reverts 75df9d7a7 ("DOC: explain HTTP2 timeout behavior") since H2
connections now respect "timeout http-keep-alive".
If commit 15a4733d5d ("BUG/MEDIUM: mux-h2: make use of http-request
and keep-alive timeouts") is backported, this DOC change needs to
be backported along with it.
Released version 2.6-dev4 with the following main changes :
- BUG/MEDIUM: httpclient: don't consume data before it was analyzed
- CLEANUP: htx: remove unused co_htx_remove_blk()
- BUG/MINOR: httpclient: consume partly the blocks when necessary
- BUG/MINOR: httpclient: remove the UNUSED block when parsing headers
- BUG/MEDIUM: httpclient: must manipulate head, not first
- REGTESTS: fix the race conditions in be2hex.vtc
- BUG/MEDIUM: quic: Blocked STREAM when retransmitted
- BUG/MAJOR: quic: Possible crash with full congestion control window
- BUG/MINOR: httpclient/lua: stuck when closing without data
- BUG/MEDIUM: applet: Don't call .release callback function twice
- BUG/MEDIUM: cli/debug: Properly get the stream-int in all debug I/O handlers
- BUG/MEDIUM: sink: Properly get the stream-int in appctx callback functions
- DEV: udp: switch parser to getopt() instead of positional arguments
- DEV: udp: add support for random packet corruption
- MINOR: server: export server_parse_sni_expr() function
- BUG/MINOR: httpclient: send the SNI using the host header
- BUILD: httpclient: fix build without SSL
- BUG/MINOR: server/ssl: free the SNI sample expression
- BUG/MINOR: logs: fix logsrv leaks on clean exit
- MINOR: actions: add new function free_act_rule() to free a single rule
- BUG/MINOR: tcp-rules: completely free incorrect TCP rules on error
- BUG/MINOR: http-rules: completely free incorrect TCP rules on error
- BUG/MINOR: httpclient: only check co_data() instead of HTTP_MSG_DATA
- BUG/MINOR: httpclient: process the response when received before the end of the request
- BUG/MINOR: httpclient: CF_SHUTW_NOW should be tested with channel_is_empty()
- CI: github actions: switch to LibreSSL-3.5.1
- BUG/MEDIUM: mux-h1: only turn CO_FL_ERROR to CS_FL_ERROR with empty ibuf
- BUG/MEDIUM: stream-int: do not rely on the connection error once established
- BUG/MEDIUM: trace: avoid race condition when retrieving session from conn->owner
- MEDIUM: mux-h2: slightly relax timeout management rules
- BUG/MEDIUM: mux-h2: make use of http-request and keep-alive timeouts
- BUG/MINOR: rules: Initialize the list element when allocating a new rule
- BUG/MINOR: http-rules: Don't free new rule on allocation failure
- DEV: coccinelle: Fix incorrect replacement in ist.cocci
- CLEANUP: Reapply ist.cocci with `--include-headers-for-types --recursive-includes`
- DEV: coccinelle: Add a new pattern to ist.cocci
- CLEANUP: Reapply ist.cocci
- REGTESTS: Do not use REQUIRE_VERSION for HAProxy 2.5+
- MINOR: quic: Code factorization (TX buffer reuse)
- CLEANUP: quic: "largest_acked_pn" pktns struc member moving
- MEDIUM: quic: Limit the number of ACK ranges
- MEDIUM: quic: Rework of the TX packets memory handling
- BUG/MINOR: quic: Possible crash in parse_retry_token()
- BUG/MINOR: quic: Possible leak in quic_build_post_handshake_frames()
- BUG/MINOR: quic: Unsent frame because of qc_build_frms()
- BUG/MINOR: mux-quic: Access to empty frame list from qc_send_frames()
- BUG/MINOR: mux-quic: Missing I/O handler events initialization
- BUG/MINOR: quic: Missing TX packet initializations
- BUG/MINOR: quic: 1RTT packets ignored after mux was released
- BUG/MINOR: quic: Incorrect peer address validation
- BUG/MINOR: quic: Non initialized variable in quic_build_post_handshake_frames()
- BUG/MINOR: quic: Wrong TX packet related counters handling
- MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value converters for MQTTv3.1
- DOC: config: Explictly add supported MQTT versions
- MINOR: quic: Add traces about stream TX buffer consumption
- MINOR: quic: Add traces in qc_set_timer() (scheduling)
- CLEANUP: mux-quic: change comment style to not mess with git conflict
- CLEANUP: mux-quic: adjust comment for coding-style
- MINOR: mux-quic: complete trace when stream is not found
- MINOR: mux-quic: add comments for send functions
- MINOR: mux-quic: use shorter name for flow-control fields
- MEDIUM: mux-quic: respect peer bidirectional stream data limit
- MEDIUM: mux-quic: respect peer connection data limit
- MINOR: mux-quic: support MAX_STREAM_DATA frame parsing
- MINOR: mux-quic: support MAX_DATA frame parsing
- BUILD: stream-int: avoid a build warning when DEBUG is empty
- BUG/MINOR: quic: Wrong buffer length passed to generate_retry_token()
- BUG/MINOR: tools: fix url2sa return value with IPv4
- MINOR: mux-quic: convert fin on push-frame as boolean
- BUILD: quic: add missing includes
- REORG: quic: use a dedicated quic_loss.c
- MINOR: mux-quic: declare the qmux trace module
- MINOR: mux-quic: replace printfs by traces
- MINOR: mux-quic: add trace event for frame sending
- MINOR: mux-quic: add trace event for qcs_push_frame
- MINOR: mux-quic: activate qmux traces on stdout via macro
- BUILD: qpack: fix unused value when not using DEBUG_HPACK
- CLEANUP: qpack: suppress by default stdout traces
- CLEANUP: h3: suppress by default stdout traces
- BUG/MINOR: tools: url2sa reads too far when no port nor path
This avoids any ambiguities on supported versions. This patch depends on
129579813 ("MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value
converters for MQTTv3.1").
It must be backported with the above commit.
Released version 2.6-dev3 with the following main changes :
- DEBUG: rename WARN_ON_ONCE() to CHECK_IF()
- DEBUG: improve BUG_ON output message accuracy
- DEBUG: implement 4 levels of choices between warn and crash.
- DEBUG: add two new macros to enable debugging in hot paths
- DEBUG: buf: replace some sensitive BUG_ON() with BUG_ON_HOT()
- DEBUG: buf: add BUG_ON_HOT() to most buffer management functions
- MINOR: channel: don't use co_set_data() to decrement output
- DEBUG: channel: add consistency checks using BUG_ON_HOT() in some key functions
- MINOR: conn-stream: Improve API to have safe/unsafe accessors
- MEDIUM: tree-wide: Use unsafe conn-stream API when it is relevant
- CLEANUP: stream-int: Make si_cs_send() function static
- REORG: stream-int: Uninline si_sync_recv() and make si_cs_recv() private
- BUG/MEDIUM: mux-fcgi: Don't rely on SI src/dst addresses for FCGI health-checks
- BUG/MEDIUM: htx: Fix a possible null derefs in htx_xfer_blks()
- REGTESTS: fix the race conditions in normalize_uri.vtc
- DEBUG: stream-int: Fix BUG_ON used to test appctx in si_applet_ops callbacks
- BUILD: debug: fix build warning on older compilers around DEBUG_STRICT_ACTION
- CLEANUP: connection: Indicate unreachability to the compiler in conn_recv_proxy
- MINOR: connection: Transform safety check in PROXYv2 parsing into BUG_ON()
- DOC: install: it's DEBUG_CFLAGS, not DEBUG, which is set to -g
- DOC: install: describe the DEP variable
- DOC: install: describe how to choose options used in the DEBUG variable
- MINOR: queue: Replace if() + abort() with BUG_ON()
- CLEANUP: adjust indentation in bidir STREAM handling function
- MINOR: quic: simplify copy of STREAM frames to RX buffer
- MINOR: quic: handle partially received buffered stream frame
- MINOR: mux-quic: define flag for last received frame
- BUG/MINOR: quic: support FIN on Rx-buffered STREAM frames
- MEDIUM: quic: rearchitecture Rx path for bidirectional STREAM frames
- REGTESTS: fix the race conditions in secure_memcmp.vtc
- CLEANUP: stream: Remove useless tests on conn-stream in stream_dump()
- BUILD: ssl: another build warning on LIBRESSL_VERSION_NUMBER
- MINOR: quic: Ensure PTO timer is not set in the past
- MINOR: quic: Post handshake I/O callback switching
- MINOR: quic: Drop the packets of discarded packet number spaces
- CLEANUP: quic: Useless tests in qc_try_rm_hp()
- CLEANUP: quic: Indentation fix in qc_prep_pkts()
- MINOR: quic: Assemble QUIC TLS flags at the same level
- BUILD: conn_stream: avoid null-deref warnings on gcc 6
- BUILD: connection: do not declare register_mux_proto() inline
- BUILD: http_rules: do not declare http_*_keywords_registre() inline
- BUILD: trace: do not declare trace_registre_source() inline
- BUILD: tcpcheck: do not declare tcp_check_keywords_register() inline
- DEBUG: reduce the footprint of BUG_ON() calls
- BUG/MEDIUM: httpclient/lua: infinite appctx loop with POST
- BUG/MINOR: pool: always align pool_heads to 64 bytes
- DEV: udp: add a tiny UDP proxy for testing
- DEV: udp: implement pseudo-random reordering/loss
- DEV: udp: add an optional argument to set the prng seed
- BUG/MINOR: quic: fix segfault on CC if mux uninitialized
- BUG/MEDIUM: pools: fix ha_free() on area in the process of being freed
- CLEANUP: tree-wide: remove a few rare non-ASCII chars
- CI: coverity: simplify debugging options
- CLEANUP: quic: complete ABORT_NOW with a TODO comment
- MINOR: quic: qc_prep_app_pkts() implementation
- MINOR: quic: Send short packet from a frame list
- MINOR: quic: Make qc_build_frms() build ack-eliciting frames from a list
- MINOR: quic: Export qc_send_app_pkts()
- MINOR: mux-quic: refactor transport parameters init
- MINOR: mux-quic: complete functions to detect stream type
- MINOR: mux-quic: define new unions for flow-control fields
- MEDIUM: mux-quic: use direct send transport API for STREAMs
- MINOR: mux-quic: retry send opportunistically for remaining frames
- MEDIUM: mux-quic: implement MAX_STREAMS emission for bidir streams
- BUILD: fix kFreeBSD build.
- MINOR: quic: Retry on qc_build_pkt() failures
- BUG/MINOR: quic: Missing recovery start timer reset
- CLEANUP: quic: Remove QUIC path manipulations out of the congestion controller
- MINOR: quic: Add a "slow start" callback to congestion controller
- MINOR: quic: Persistent congestion detection outside of controllers
- CLEANUP: quic: Remove useless definitions from quic_cc_event struct
- BUG/MINOR: quic: Confusion betwen "in_flight" and "prep_in_flight" in quic_path_prep_data()
- MINOR: quic: More precise window update calculation
- CLEANUP: quic: Remove window redundant variable from NewReno algorithm state struct
- MINOR: quic: Add quic_max_int_by_size() function
- BUG/MAJOR: quic: Wrong quic_max_available_room() returned value
- MINOR: pools: add a new global option "no-memory-trimming"
- BUG/MINOR: add missing modes in proxy_mode_str()
- BUG/MINOR: cli: shows correct mode in "show sess"
- BUG/MEDIUM: quic: do not drop packet on duplicate stream/decoding error
- MINOR: stats: Add dark mode support for socket rows
- BUILD: fix recent build breakage of freebsd caused by kFreeBSD build fix
- BUG/MINOR: httpclient: Set conn-stream/channel EOI flags at the end of request
- BUG/MINOR: hlua: Set conn-stream/channel EOI flags at the end of request
- BUG/MINOR: stats: Set conn-stream/channel EOI flags at the end of request
- BUG/MINOR: cache: Set conn-stream/channel EOI flags at the end of request
- BUG/MINOR: promex: Set conn-stream/channel EOI flags at the end of request
- BUG/MEDIUM: stream: Use the front analyzers for new listener-less streams
- DEBUG: cache: Update underlying buffer when loading HTX message in cache applet
- BUG/MEDIUM: mcli: Properly handle errors and timeouts during reponse processing
- DEBUG: stream: Add the missing descriptions for stream trace events
- DEBUG: stream: Fix stream trace message to print response buffer state
- MINOR: proxy: Store monitor_uri as a `struct ist`
- MINOR: proxy: Store fwdfor_hdr_name as a `struct ist`
- MINOR: proxy: Store orgto_hdr_name as a `struct ist`
- MEDIUM: proxy: Store server_id_hdr_name as a `struct ist`
- CLEANUP: fcgi: Replace memcpy() on ist by istcat()
- CLEANUP: fcgi: Use `istadv()` in `fcgi_strm_send_params`
- BUG/MAJOR: mux-pt: Always destroy the backend connection on detach
- DOC: sample fetch methods: move distcc_* to the right locations
- MINOR: rules: record the last http/tcp rule that gave a final verdict
- MINOR: stream: add "last_rule_file" and "last_rule_line" samples
- BUG/MINOR: session: fix theoretical risk of memleak in session_accept_fd()
- MINOR: quic: Add max_idle_timeout advertisement handling
- MEDIUM: quic: Remove the QUIC connection reference counter
- BUG/MINOR: quic: ACK_REQUIRED and ACK_RECEIVED flag collision
- BUG/MINOR: quic: Missing check when setting the anti-amplification limit as reached
- MINOR: quic: Add a function to compute the current PTO
- MEDIUM: quic: Implement the idle timeout feature
- BUG/MEDIUM: quic: qc_prep_app_pkts() retries on qc_build_pkt() failures
- CLEANUP: quic: Comments fix for qc_prep_(app)pkts() functions
- MINOR: mux-quic: prevent push frame for unidir streams
- MINOR: mux-quic: improve opportunistic retry sending for STREAM frames
- MINOR: quic: implement sending confirmation
- MEDIUM: mux-quic: improve bidir STREAM frames sending
- MEDIUM: check: do not auto configure SSL/PROXY for dynamic servers
- REGTESTS: server: test SSL/PROXY with checks for dynamic servers
- MEDIUM: server: remove experimental-mode for dynamic servers
- BUG/MINOR: buffer: fix debugging condition in b_peek_varint()
Dynamic servers feature is now judged to be stable enough. Remove the
experimental-mode requirement for "add/del server" commands. This should
facilitate dynamic servers adoption.
For server checks, SSL and PROXY is automatically inherited from the
server settings if no specific check port is specified. Change this
behavior for dynamic servers : explicit "check-ssl"/"check-send-proxy"
are required for them.
Without this change, it is impossible to add a dynamic server with
SSL/PROXY settings and checks without, if the check port is not
explicit. This is because "no-check-ssl"/"no-check-send-proxy" keywords
are not available for dynamic servers.
This change respects the principle that dynamic servers on the CLI
should not reuse the same shortcuts used during the config file parsing.
Mostly because we expect this feature to be manipulated by automated
tools, contrary to the config file which should aim to be the shortest
possible for human readability.
Update the documentation of the "check" keyword to reflect this change.
These two sample fetch methods report respectively the file name and the
line number where was located the last rule that was final. This is aimed
at being used on log-format lines to help admins figure what rule in the
configuration gave a final verdict, and help understand the condition
that led to the action.
For example, it's now possible to log the last matched rule by adding
this to the log-format:
... lr=%[last_rule_file]:%[last_rule_line]
A regtest is provided to test various combinations of final rules, some
even on top of each other from different rulesets.
The distcc* sample fetch methods were surprisingly located within the
"internal state" section, while they in fact depend on L6 contents.
This can be backported to all versions where they appear.
Some users with very large numbers of connections have been facing
extremely long malloc_trim() calls on reload that managed to trigger
the watchdog! That's a bit counter-productive. It's even possible
that some implementations are not perfectly reliable or that their
trimming time grows quadratically with the memory used. Instead of
constantly trying to work around these issues, let's offer an option
to disable this mechanism, since nobody had been complaining in the
past, and this was only meant to be an improvement.
This should be backported to 2.4 where trimming on reload started to
appear.
Released version 2.6-dev2 with the following main changes :
- DOC: management: rework the Master CLI section
- DOC: management: add expert and experimental mode in 9.4.1
- CLEANUP: cleanup a commentary in pcli_parse_request()
- BUG/MINOR: mworker/cli: don't display help on master applet
- MINOR: mworker/cli: mcli-debug-mode enables every command
- MINOR: mworker/cli: add flags in the prompt
- BUG/MINOR: httpclient: Revisit HC request and response buffers allocation
- BUG/MEDIUM: httpclient: Xfer the request when the stream is created
- MINOR: httpclient: Don't limit data transfer to 1024 bytes
- BUILD: ssl: adjust guard for X509_get_X509_PUBKEY(x)
- REGTESTS: ssl: skip show_ssl_ocspresponse.vtc when BoringSSL is used
- MINOR: quic: Do not modify a marked as consumed datagram
- MINOR: quic: Wrong datagram buffer passed to quic_lstnr_dgram_dispatch()
- MINOR: quic: Remove a useless test in quic_get_dgram_dcid()
- BUG/MINOR: ssl: Remove empty lines from "show ssl ocsp-response <id>" output
- CLEANUP: ssl: Remove unused ssl_sock_create_cert function
- MINOR: ssl: Use high level OpenSSL APIs in sha2 converter
- MINOR: ssl: Remove EC_KEY related calls when preparing SSL context
- REGTESTS: ssl: Add test for "curves" and "ecdhe" SSL options
- MINOR: ssl: Remove EC_KEY related calls when creating a certificate
- REGTESTS: ssl: Add test for "generate-certificates" SSL option
- MINOR: ssl: Remove call to SSL_CTX_set_tlsext_ticket_key_cb with OpenSSLv3
- MINOR: ssl: Remove call to HMAC_Init_ex with OpenSSLv3
- MINOR: h3: hardcode the stream id of control stream
- MINOR: mux-quic: remove quic_transport_params_update
- MINOR: quic: rename local tid variable
- MINOR: quic: remove unused xprt rcv_buf operation
- MINOR: quic: take out xprt snd_buf operation
- CI: enable QUIC for Coverity scan
- BUG/MINOR: mworker: does not erase the pidfile upon reload
- MINOR: ssl: Remove call to ERR_func_error_string with OpenSSLv3
- MINOR: ssl: Remove call to ERR_load_SSL_strings with OpenSSLv3
- REGTESTS: ssl: Add tests for DH related options
- MINOR: ssl: Create HASSL_DH wrapper structure
- MINOR: ssl: Add ssl_sock_get_dh_from_bio helper function
- MINOR: ssl: Factorize ssl_get_tmp_dh and append a cbk to its name
- MINOR: ssl: Add ssl_sock_set_tmp_dh helper function
- MINOR: ssl: Add ssl_sock_set_tmp_dh_from_pkey helper function
- MINOR: ssl: Add ssl_new_dh_fromdata helper function
- MINOR: ssl: Build local DH of right size when needed
- MINOR: ssl: Set default dh size to 2048
- MEDIUM: ssl: Replace all DH objects by EVP_PKEY on OpenSSLv3 (via HASSL_DH type)
- MINOR: ssl: Remove calls to SSL_CTX_set_tmp_dh_callback on OpenSSLv3
- MINOR: quic: Remove an RX buffer useless lock
- MINOR: quic: Variable used before being checked in ha_quic_add_handshake_data()
- MINOR: quic: EINTR error ignored
- MINOR: quic: Potential overflow expression in qc_parse_frm()
- MINOR: quic: Possible overflow in qpack_get_varint()
- CLEANUP: h3: Unreachable target in h3_uqs_init()
- MINOR: quic: Possible memleak in qc_new_conn()
- MINOR: quic: Useless statement in quic_crypto_data_cpy()
- BUG/MEDIUM: pools: ensure items are always large enough for the pool_cache_item
- BUG/MINOR: pools: always flush pools about to be destroyed
- CLEANUP: pools: don't needlessly set a call mark during refilling of caches
- DEBUG: pools: add extra sanity checks when picking objects from a local cache
- DEBUG: pools: let's add reverse mapping from cache heads to thread and pool
- DEBUG: pools: replace the link pointer with the caller's address on pool_free()
- BUG/MAJOR: sched: prevent rare concurrent wakeup of multi-threaded tasks
- MINOR: quic: use a global dghlrs for each thread
- BUG/MEDIUM: quic: fix crash on CC if mux not present
- MINOR: qpack: fix typo in trace
- BUG/MINOR: quic: fix FIN stream signaling
- BUG/MINOR: h3: fix the header length for QPACK decoding
- MINOR: h3: remove transfer-encoding header
- MINOR: h3: add documentation on h3_decode_qcs
- MINOR: h3: set properly HTX EOM/BODYLESS on HEADERS parsing
- MINOR: mux-quic: implement rcv_buf
- MINOR: mux-quic: set EOS on rcv_buf
- MINOR: h3: set CS_FL_NOT_FIRST
- MINOR: h3: report frames bigger than rx buffer
- MINOR: h3: extract HEADERS parsing in a dedicated function
- MINOR: h3: implement DATA parsing
- MINOR: quic: Wrong smoothed rtt initialization
- MINOR: quic: Wrong loss delay computation
- MINOR: quic: Code never reached in qc_ssl_sess_init()
- MINOR: quic: ha_quic_set_encryption_secrets without server specific code
- MINOR: quic: Avoid warning about NULL pointer dereferences
- MINOR: quic: Useless test in quic_lstnr_dghdlr()
- MINOR: quic: Non checked returned value for cs_new() in hq_interop_decode_qcs()
- MINOR: h3: Dead code in h3_uqs_init()
- MINOR: quic: Non checked returned value for cs_new() in h3_decode_qcs()
- MINOR: quic: Possible frame parsers array overrun
- MINOR: quic: Do not retransmit too much packets.
- MINOR: quic: Move quic_rxbuf_pool pool out of xprt part
- MINOR: h3: report error on HEADERS/DATA parsing
- BUG/MINOR: jwt: Double free in deinit function
- BUG/MINOR: jwt: Missing pkey free during cleanup
- BUG/MINOR: jwt: Memory leak if same key is used in multiple jwt_verify calls
- BUG/MINOR: httpclient/cli: display junk characters in vsn
- MINOR: h3: remove unused return value on decode_qcs
- BUG/MAJOR: http/htx: prevent unbounded loop in http_manage_server_side_cookies
- BUG/MAJOR: spoe: properly detach all agents when releasing the applet
- REGTESTS: server: close an occasional race on dynamic_server_ssl.vtc
- REGTESTS: peers: leave a bit more time to peers to synchronize
- BUG/MEDIUM: h2/hpack: fix emission of HPACK DTSU after settings change
- BUG/MINOR: mux-h2: update the session's idle delay before creating the stream
- BUG/MINOR: httpclient: reinit flags in httpclient_start()
- BUG/MINOR: mailers: negotiate SMTP, not ESMTP
- MINOR: httpclient: sets an alternative destination
- MINOR: httpclient/lua: add 'dst' optionnal field
- BUG/MINOR: ssl: Add missing return value check in ssl_ocsp_response_print
- BUG/MINOR: ssl: Fix leak in "show ssl ocsp-response" CLI command
- BUG/MINOR: ssl: Missing return value check in ssl_ocsp_response_print
- CLEANUP: httpclient/cli: fix indentation alignment of the help message
- BUG/MINOR: tools: url2sa reads ipv4 too far
- BUG/MEDIUM: httpclient: limit transfers to the maximum available room
- DEBUG: buffer: check in __b_put_blk() whether the buffer room is respected
- MINOR: mux-quic: fix a possible null dereference in qc_timeout_task
- BUG/MEDIUM: htx: Be sure to have a buffer to perform a raw copy of a message
- BUG/MEDIUM: mux-h1: Don't wake h1s if mux is blocked on lack of output buffer
- BUG/MAJOR: mux-h2: Be sure to always report HTX parsing error to the app layer
- DEBUG: stream-int: Check CS_FL_WANT_ROOM is not set with an empty input buffer
- MINOR: quic: do not modify offset node if quic_rx_strm_frm in tree
- MINOR: h3: fix compiler warning variable set but not used
- MINOR: mux-quic: fix uninitialized return on qc_send
- MINOR: quic: fix handling of out-of-order received STREAM frames
- MINOR: pools: mark most static pool configuration variables as read-mostly
- CLEANUP: pools: remove the now unused pool_is_crowded()
- REGTESTS: fix the race conditions in 40be_2srv_odd_health_checks
- BUG/MEDIUM: stream: Abort processing if response buffer allocation fails
- MINOR: httpclient/lua: ability to set a server timeout
- BUG/MINOR: httpclient/lua: missing pop for new timeout parameter
- DOC: httpclient/lua: fix the type of the dst parameter
- CLEANUP: httpclient: initialize the client in stage INIT not REGISTER
- CLEANUP: muxes: do not use a dynamic trash in list_mux_protos()
- CLEANUP: vars: move the per-process variables initialization to vars.c
- CLEANUP: init: remove the ifdef on HAPROXY_MEMMAX
- MINOR: pools: disable redundant poisonning on pool_free()
- MINOR: pools: introduce a new pool_debugging global variable
- MINOR: pools: switch the fail-alloc test to runtime only
- MINOR: pools: switch DEBUG_DONT_SHARE_POOLS to runtime
- MINOR: pools: add a new debugging flag POOL_DBG_COLD_FIRST
- MINOR: pools: add a new debugging flag POOL_DBG_INTEGRITY
- MINOR: pools: make the global pools a runtime option.
- MEDIUM: pools: replace CONFIG_HAP_POOLS with a runtime "NO_CACHE" flag.
- MINOR: pools: store the allocated size for each pool
- MINOR: pools: get rid of POOL_EXTRA
- MINOR: pools: replace DEBUG_POOL_TRACING with runtime POOL_DBG_CALLER
- MINOR: pools: replace DEBUG_MEMORY_POOLS with runtime POOL_DBG_TAG
- MINOR: pools: add a debugging flag for memory poisonning option
- MEDIUM: initcall: move STG_REGISTER earlier
- MEDIUM: init: split the early initialization in its own function
- MINOR: init: extract args parsing to their own function
- MEDIUM: init: handle arguments earlier
- MINOR: pools: delegate parsing of command line option -dM to a new function
- MINOR: pools: support setting debugging options using -dM
- BUILD: makefile: enable both DEBUG_STRICT and DEBUG_MEMORY_POOLS by default
- CI: github: enable pool debugging by default
- DOC: Fix usage/examples of deprecated ACLs
- DOC: internal: update the pools API to mention boot-time settings
- DOC: design: add design thoughts for later simplification of the pools
- DOC: design: commit the temporary design notes on thread groups
- MINOR: stream-int: Handle appctx case first when releasing the endpoint
- MINOR: connection: Be prepared to handle conn-stream with no connection
- MINOR: stream: Handle appctx case first when creating a new stream
- MINOR: connection: Add a function to detach a conn-stream from the connection
- MINOR: stream-int: Add function to reset a SI endpoint
- MINOR: stream-int: Add function to attach a connection to a SI
- MINOR: stream-int: Be able to allocate a CS without connection
- MEDIUM: stream: No longer release backend conn-stream on connection retry
- MEDIUM: stream: Allocate backend CS when the stream is created
- REORG: conn_stream: move conn-stream stuff in dedicated files
- MEDIUM: conn-stream: No longer access connection field directly
- MEDIUM: conn-stream: Be prepared to use an appctx as conn-stream endpoint
- MAJOR: conn_stream/stream-int: move the appctx to the conn-stream
- MEDIUM: applet: Set the conn-stream as appctx owner instead of the stream-int
- MEDIUM: conn_stream: Add a pointer to the app object into the conn-stream
- MINOR: stream: Add pointer to front/back conn-streams into stream struct
- MINOR: stream: Slightly rework stream_new to separate CS/SI initialization
- MINOR: stream-int: Always access the stream-int via the conn-stream
- MINOR: backend: Always access the stream-int via the conn-stream
- MINOR: stream: Always access the stream-int via the conn-stream
- MINOR: http-ana: Always access the stream-int via the conn-stream
- MINOR: cli: Always access the stream-int via the conn-stream
- MINOR: log: Always access the stream-int via the conn-stream
- MINOR: frontend: Always access the stream-int via the conn-stream
- MINOR: proxy: Always access the stream-int via the conn-stream
- MINOR: peers: Always access the stream-int via the conn-stream
- MINOR: debug: Always access the stream-int via the conn-stream
- MINOR: hlua: Always access the stream-int via the conn-stream
- MINOR: cache: Always access the stream-int via the conn-stream
- MINOR: dns: Always access the stream-int via the conn-stream
- MINOR: http-act: Always access the stream-int via the conn-stream
- MINOR: httpclient: Always access the stream-int via the conn-stream
- MINOR: tcp-act: Always access the stream-int via the conn-stream
- MINOR: sink: Always access the stream-int via the conn-stream
- MINOR: conn-stream: Rename cs_detach() to cs_detach_endp()
- CLEANUP: conn-stream: Don't export conn-stream pool
- MAJOR: stream/conn_stream: Move the stream-interface into the conn-stream
- CLEANUP: stream-int: rename si_reset() to si_init()
- MINOR: conn-stream: Release a CS when both app and endp are detached
- MINOR: stream: Don't destroy conn-streams but detach app and endp
- MAJOR: check: Use a persistent conn-stream for health-checks
- CLEANUP: conn-stream: Remove cs_destroy()
- CLEANUP: backend: Don't export connect_server anymore
- BUG/MINOR: h3/hq_interop: Fix CS and stream creation
- BUILD: tree-wide: Avoid warnings about undefined entities retrieved from a CS
- BUG/MINOR: proxy: preset the error message pointer to NULL in parse_new_proxy()
- BUG/MEDIUM: quic: fix received ACK stream calculation
- BUILD: stream: fix build warning with older compilers
- BUG/MINOR: debug: fix get_tainted() to properly read an atomic value
- DEBUG: move the tainted stuff to bug.h for easier inclusion
- DEBUG: cleanup back trace generation
- DEBUG: cleanup BUG_ON() configuration
- DEBUG: mark ABORT_NOW() as unreachable
- DBEUG: add a new WARN_ON() macro
- DEBUG: make the _BUG_ON() macro return the condition
- DEBUG: add a new WARN_ON_ONCE() macro
- DEBUG: report BUG_ON() and WARN_ON() in the tainted flags
- MINOR: quic: adjust buffer handling for STREAM transmission
- MINOR: quic: liberate the TX stream buffer after ACK processing
- MINOR: quic: add a TODO for a memleak frame on ACK consume
The pools currently have plenty of options (and some usefull ones were
even lost with the modern design), but most of them could be categorized
along a few use cases, namely, performance, reliability, debuggability.
This document explores various ways to try to combine them and their
effect in a less complex way for the long term.
The 9 currently available debugging options may now be checked, set, or
cleared using -dM. The directive now takes a comma-delimited list of
options after the optional poisonning byte. With "help", the list of
available options is displayed with a short help and their current
status.
The management doc was updated.
The STG_REGISTER init level is used to register known keywords and
protocol stacks. It must be called earlier because some of the init
code already relies on it to be known. For example, "haproxy -vv"
for now is constrained to start very late only because of this.
This patch moves it between STG_LOCK and STG_ALLOC, which is fine as
it's used for static registration.
The poisonning performed on pool_free() used to help a little bit with
use-after-free detection, but usually did more harm than good in that
it was never possible to perform post-mortem analysis on released
objects once poisonning was enabled on allocation. Now that there is
a dedicated DEBUG_POOL_INTEGRITY, let's get rid of this annoyance
which is not even documented in the management manual.
Add the ability to set a "server timeout" on the httpclient with either
the httpclient_set_timeout() API or the timeout argument in a request.
Issue #1470.
The 'dst' optionnal field on a httpclient request can be used to set an
alternative server address in the haproxy address format. Which means it
could be use with unix@, ipv6@ etc.
Should fix issue #1471.
"mcli-debug-mode on" enables every command that were meant for a worker,
on the CLI of the master. Which mean you can issue, "show fd", show
stat" in order to debug the MASTER proxy.
You can also combine it with "expert-mode on" or "experimental-mode on"
to access to more commands.
Released version 2.6-dev1 with the following main changes :
- BUG/MINOR: cache: Fix loop on cache entries in "show cache"
- BUG/MINOR: httpclient: allow to replace the host header
- BUG/MINOR: lua: don't expose internal proxies
- MEDIUM: mworker: seamless reload use the internal sockpairs
- BUG/MINOR: lua: remove loop initial declarations
- BUG/MINOR: mworker: does not add the -sf in wait mode
- BUG/MEDIUM: mworker: FD leak of the eventpoll in wait mode
- MINOR: quic: do not reject PADDING followed by other frames
- REORG: quic: add comment on rare thread concurrence during CID alloc
- CLEANUP: quic: add comments on CID code
- MEDIUM: quic: handle CIDs to rattach received packets to connection
- MINOR: qpack: support litteral field line with non-huff name
- MINOR: quic: activate QUIC traces at compilation
- MINOR: quic: use more verbose QUIC traces set at compile-time
- MEDIUM: pool: refactor malloc_trim/glibc and jemalloc api addition detections.
- MEDIUM: pool: support purging jemalloc arenas in trim_all_pools()
- BUG/MINOR: mworker: deinit of thread poller was called when not initialized
- BUILD: pools: only detect link-time jemalloc on ELF platforms
- CI: github actions: add the output of $CC -dM -E-
- BUG/MEDIUM: cli: Properly set stream analyzers to process one command at a time
- BUILD: evports: remove a leftover from the dead_fd cleanup
- MINOR: quic: Set "no_application_protocol" alert
- MINOR: quic: More accurate immediately close.
- MINOR: quic: Immediately close if no transport parameters extension found
- MINOR: quic: Rename qc_prep_hdshk_pkts() to qc_prep_pkts()
- MINOR: quic: Possible crash when inspecting the xprt context
- MINOR: quic: Dynamically allocate the secrete keys
- MINOR: quic: Add a function to derive the key update secrets
- MINOR: quic: Add structures to maintain key phase information
- MINOR: quic: Optional header protection key for quic_tls_derive_keys()
- MINOR: quic: Add quic_tls_key_update() function for Key Update
- MINOR: quic: Enable the Key Update process
- MINOR: quic: Delete the ODCIDs asap
- BUG/MINOR: vars: Fix the set-var and unset-var converters
- MEDIUM: pool: Following up on previous pool trimming update.
- BUG/MEDIUM: mux-h1: Fix splicing by properly detecting end of message
- BUG/MINOR: mux-h1: Fix splicing for messages with unknown length
- MINOR: mux-h1: Improve H1 traces by adding info about http parsers
- MINOR: mux-h1: register a stats module
- MINOR: mux-h1: add counters instance to h1c
- MINOR: mux-h1: count open connections/streams on stats
- MINOR: mux-h1: add stat for total count of connections/streams
- MINOR: mux-h1: add stat for total amount of bytes received and sent
- REGTESTS: h1: Add a script to validate H1 splicing support
- BUG/MINOR: server: Don't rely on last default-server to init server SSL context
- BUG/MEDIUM: resolvers: Detach query item on response error
- MEDIUM: resolvers: No longer store query items in a list into the response
- BUG/MAJOR: segfault using multiple log forward sections.
- BUG/MEDIUM: h1: Properly reset h1m flags when headers parsing is restarted
- BUG/MINOR: resolvers: Don't overwrite the error for invalid query domain name
- BUILD: bug: Fix error when compiling with -DDEBUG_STRICT_NOCRASH
- BUG/MEDIUM: sample: Fix memory leak in sample_conv_jwt_member_query
- DOC: spoe: Clarify use of the event directive in spoe-message section
- DOC: config: Specify %Ta is only available in HTTP mode
- BUILD: tree-wide: avoid warnings caused by redundant checks of obj_types
- IMPORT: slz: use the correct CRC32 instruction when running in 32-bit mode
- MINOR: quic: fix segfault on CONNECTION_CLOSE parsing
- MINOR: h3: add BUG_ON on control receive function
- MEDIUM: xprt-quic: finalize app layer initialization after ALPN nego
- MINOR: h3: remove duplicated FIN flag position
- MAJOR: mux-quic: implement a simplified mux version
- MEDIUM: mux-quic: implement release mux operation
- MEDIUM: quic: detect the stream FIN
- MINOR: mux-quic: implement subscribe on stream
- MEDIUM: mux-quic: subscribe on xprt if remaining data after send
- MEDIUM: mux-quic: wake up xprt on data transferred
- MEDIUM: mux-quic: handle when sending buffer is full
- MINOR: quic: RX buffer full due to wrong CRYPTO data handling
- MINOR: quic: Race issue when consuming RX packets buffer
- MINOR: quic: QUIC encryption level RX packets race issue
- MINOR: quic: Delete remaining RX handshake packets
- MINOR: quic: Remove QUIC TX packet length evaluation function
- MINOR: hq-interop: fix tx buffering
- MINOR: mux-quic: remove uneeded code to check fin on TX
- MINOR: quic: add HTX EOM on request end
- BUILD: mux-quic: fix compilation with DEBUG_MEM_STATS
- MINOR: http-rules: Add capture action to http-after-response ruleset
- BUG/MINOR: cli/server: Don't crash when a server is added with a custom id
- MINOR: mux-quic: do not release qcs if there is remaining data to send
- MINOR: quic: notify the mux on CONNECTION_CLOSE
- BUG/MINOR: mux-quic: properly initialize flow control
- MINOR: quic: Compilation fix for quic_rx_packet_refinc()
- MINOR: h3: fix possible invalid dereference on htx parsing
- DOC: config: retry-on list is space-delimited
- DOC: config: fix error-log-format example
- BUG/MEDIUM: mworker/cli: crash when trying to access an old PID in prompt mode
- MINOR: hq-interop: refix tx buffering
- REGTESTS: ssl: use X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY for cert check
- MINOR: cli: "show version" displays the current process version
- CLEANUP: cfgparse: modify preprocessor guards around numa detection code
- MEDIUM: cfgparse: numa detect topology on FreeBSD.
- BUILD: ssl: unbreak the build with newer libressl
- MINOR: vars: Move UPDATEONLY flag test to vars_set_ifexist
- MINOR: vars: Set variable type to ANY upon creation
- MINOR: vars: Delay variable content freeing in var_set function
- MINOR: vars: Parse optional conditions passed to the set-var converter
- MINOR: vars: Parse optional conditions passed to the set-var actions
- MEDIUM: vars: Enable optional conditions to set-var converter and actions
- DOC: vars: Add documentation about the set-var conditions
- REGTESTS: vars: Add new test for conditional set-var
- MINOR: quic: Attach timer task to thread for the connection.
- CLEANUP: quic_frame: Remove a useless suffix to STOP_SENDING
- MINOR: quic: Add traces for STOP_SENDING frame and modify others
- CLEANUP: quic: Remove cdata_len from quic_tx_packet struct
- MINOR: quic: Enable TLS 0-RTT if needed
- MINOR: quic: No TX secret at EARLY_DATA encryption level
- MINOR: quic: Add quic_set_app_ops() function
- MINOR: ssl_sock: Set the QUIC application from ssl_sock_advertise_alpn_protos.
- MINOR: quic: Make xprt support 0-RTT.
- MINOR: qpack: Missing check for truncated QPACK fields
- CLEANUP: quic: Comment fix for qc_strm_cpy()
- MINOR: hq_interop: Stop BUG_ON() truncated streams
- MINOR: quic: Do not mix packet number space and connection flags
- CLEANUP: quic: Shorten a litte bit the traces in lstnr_rcv_pkt()
- MINOR: mux-quic: fix trace on stream creation
- CLEANUP: quic: fix spelling mistake in a trace
- CLEANUP: quic: rename quic_conn conn to qc in quic_conn_free
- MINOR: quic: add missing lock on cid tree
- MINOR: quic: rename constant for haproxy CIDs length
- MINOR: quic: refactor concat DCID with address for Initial packets
- MINOR: quic: compare coalesced packets by DCID
- MINOR: quic: refactor DCID lookup
- MINOR: quic: simplify the removal from ODCID tree
- REGTESTS: vars: Remove useless ssl tunes from conditional set-var test
- MINOR: ssl: Remove empty lines from "show ssl ocsp-response" output
- MINOR: quic: Increase the RX buffer for each connection
- MINOR: quic: Add a function to list remaining RX packets by encryption level
- MINOR: quic: Stop emptying the RX buffer asap.
- MINOR: quic: Do not expect to receive only one O-RTT packet
- MINOR: quic: Do not forget STREAM frames received in disorder
- MINOR: quic: Wrong packet refcount handling in qc_pkt_insert()
- DOC: fix misspelled keyword "resolve_retries" in resolvers
- CLEANUP: quic: rename quic_conn instances to qc
- REORG: quic: move mux function outside of xprt
- MINOR: quic: add reference to quic_conn in ssl context
- MINOR: quic: add const qualifier for traces function
- MINOR: trace: add quic_conn argument definition
- MINOR: quic: use quic_conn as argument to traces
- MINOR: quic: add quic_conn instance in traces for qc_new_conn
- MINOR: quic: Add stream IDs to qcs_push_frame() traces
- MINOR: quic: unchecked qc_retrieve_conn_from_cid() returned value
- MINOR: quic: Wrong dropped packet skipping
- MINOR: quic: Handle the cases of overlapping STREAM frames
- MINOR: quic: xprt traces fixes
- MINOR: quic: Drop asap Retry or Version Negotiation packets
- MINOR: pools: work around possibly slow malloc_trim() during gc
- DEBUG: ssl: make sure we never change a servername on established connections
- MINOR: quic: Add traces for RX frames (flow control related)
- MINOR: quic: Add CONNECTION_CLOSE phrase to trace
- REORG: quic: remove qc_ prefix on functions which not used it directly
- BUG/MINOR: quic: upgrade rdlock to wrlock for ODCID removal
- MINOR: quic: remove unnecessary call to free_quic_conn_cids()
- MINOR: quic: store ssl_sock_ctx reference into quic_conn
- MINOR: quic: remove unnecessary if in qc_pkt_may_rm_hp()
- MINOR: quic: replace usage of ssl_sock_ctx by quic_conn
- MINOR: quic: delete timer task on quic_close()
- MEDIUM: quic: implement refcount for quic_conn
- BUG/MINOR: quic: fix potential null dereference
- BUG/MINOR: quic: fix potential use of uninit pointer
- BUG/MEDIUM: backend: fix possible sockaddr leak on redispatch
- BUG/MEDIUM: peers: properly skip conn_cur from incoming messages
- CI: Github Actions: do not show VTest failures if build failed
- BUILD: opentracing: display warning in case of using OT_USE_VARS at compile time
- MINOR: compat: detect support for dl_iterate_phdr()
- MINOR: debug: add ability to dump loaded shared libraries
- MINOR: debug: add support for -dL to dump library names at boot
- BUG/MEDIUM: ssl: initialize correctly ssl w/ default-server
- REGTESTS: ssl: fix ssl_default_server.vtc
- BUG/MINOR: ssl: free the fields in srv->ssl_ctx
- BUG/MEDIUM: ssl: free the ckch instance linked to a server
- REGTESTS: ssl: update of a crt with server deletion
- BUILD/MINOR: cpuset FreeBSD 14 build fix.
- MINOR: pools: always evict oldest objects first in pool_evict_from_local_cache()
- DOC: pool: document the purpose of various structures in the code
- CLEANUP: pools: do not use the extra pointer to link shared elements
- CLEANUP: pools: get rid of the POOL_LINK macro
- MINOR: pool: allocate from the shared cache through the local caches
- CLEANUP: pools: group list updates in pool_get_from_cache()
- MINOR: pool: rely on pool_free_nocache() in pool_put_to_shared_cache()
- MINOR: pool: make pool_is_crowded() always true when no shared pools are used
- MINOR: pool: check for pool's fullness outside of pool_put_to_shared_cache()
- MINOR: pool: introduce pool_item to represent shared pool items
- MINOR: pool: add a function to estimate how many may be released at once
- MEDIUM: pool: compute the number of evictable entries once per pool
- MINOR: pools: prepare pool_item to support chained clusters
- MINOR: pools: pass the objects count to pool_put_to_shared_cache()
- MEDIUM: pools: centralize cache eviction in a common function
- MEDIUM: pools: start to batch eviction from local caches
- MEDIUM: pools: release cached objects in batches
- OPTIM: pools: reduce local pool cache size to 512kB
- CLEANUP: assorted typo fixes in the code and comments This is 29th iteration of typo fixes
- CI: github actions: update OpenSSL to 3.0.1
- BUILD/MINOR: tools: solaris build fix on dladdr.
- BUG/MINOR: cli: fix _getsocks with musl libc
- BUG/MEDIUM: http-ana: Preserve response's FLT_END analyser on L7 retry
- MINOR: quic: Wrong traces after rework
- MINOR: quic: Add trace about in flight bytes by packet number space
- MINOR: quic: Wrong first packet number space computation
- MINOR: quic: Wrong packet number space computation for PTO
- MINOR: quic: Wrong loss time computation in qc_packet_loss_lookup()
- MINOR: quic: Wrong ack_delay compution before calling quic_loss_srtt_update()
- MINOR: quic: Remove nb_pto_dgrams quic_conn struct member
- MINOR: quic: Wrong packet number space trace in qc_prep_pkts()
- MINOR: quic: Useless test in qc_prep_pkts()
- MINOR: quic: qc_prep_pkts() code moving
- MINOR: quic: Speeding up Handshake Completion
- MINOR: quic: Probe Initial packet number space more often
- MINOR: quic: Probe several packet number space upon timer expiration
- MINOR: quic: Comment fix.
- MINOR: quic: Improve qc_prep_pkts() flexibility
- MINOR: quic: Do not drop secret key but drop the CRYPTO data
- MINOR: quic: Prepare Handshake packets asap after completed handshake
- MINOR: quic: Flag asap the connection having reached the anti-amplification limit
- MINOR: quic: PTO timer too often reset
- MINOR: quic: Re-arm the PTO timer upon datagram receipt
- MINOR: proxy: add option idle-close-on-response
- MINOR: cpuset: switch to sched_setaffinity for FreeBSD 14 and above.
- CI: refactor spelling check
- CLEANUP: assorted typo fixes in the code and comments
- BUILD: makefile: add -Wno-atomic-alignment to work around clang abusive warning
- MINOR: quic: Only one CRYPTO frame by encryption level
- MINOR: quic: Missing retransmission from qc_prep_fast_retrans()
- MINOR: quic: Non-optimal use of a TX buffer
- BUG/MEDIUM: mworker: don't use _getsocks in wait mode
- BUG/MINOR: ssl: Store client SNI in SSL context in case of ClientHello error
- BUG/MAJOR: mux-h1: Don't decrement .curr_len for unsent data
- DOC: internals: document the pools architecture and API
- CI: github actions: clean default step conditions
- BUILD: cpuset: fix build issue on macos introduced by previous change
- MINOR: quic: Remaining TRACEs with connection as firt arg
- MINOR: quic: Reset ->conn quic_conn struct member when calling qc_release()
- MINOR: quic: Flag the connection as being attached to a listener
- MINOR: quic: Wrong CRYPTO frame concatenation
- MINOR: quid: Add traces quic_close() and quic_conn_io_cb()
- REGTESTS: ssl: Fix ssl_errors regtest with OpenSSL 1.0.2
- MINOR: quic: Do not dereference ->conn quic_conn struct member
- MINOR: quic: fix return of quic_dgram_read
- MINOR: quic: add config parse source file
- MINOR: quic: implement Retry TLS AEAD tag generation
- MEDIUM: quic: implement Initial token parsing
- MINOR: quic: define retry_source_connection_id TP
- MEDIUM: quic: implement Retry emission
- MINOR: quic: free xprt tasklet on its thread
- BUG/MEDIUM: connection: properly leave stopping list on error
- MINOR: pools: enable pools with DEBUG_FAIL_ALLOC as well
- MINOR: quic: As server, skip 0-RTT packet number space
- MINOR: quic: Do not wakeup the I/O handler before the mux is started
- BUG/MEDIUM: htx: Adjust length to add DATA block in an empty HTX buffer
- CI: github actions: use cache for OpenTracing
- BUG/MINOR: httpclient: don't send an empty body
- BUG/MINOR: httpclient: set default Accept and User-Agent headers
- BUG/MINOR: httpclient/lua: don't pop the lua stack when getting headers
- BUILD/MINOR: fix solaris build with clang.
- BUG/MEDIUM: server: avoid changing healthcheck ctx with set server ssl
- CI: refactor OpenTracing build script
- DOC: management: mark "set server ssl" as deprecated
- MEDIUM: cli: yield between each pipelined command
- MINOR: channel: add new function co_getdelim() to support multiple delimiters
- BUG/MINOR: cli: avoid O(bufsize) parsing cost on pipelined commands
- MEDIUM: h2/hpack: emit a Dynamic Table Size Update after settings change
- MINOR: quic: Retransmit the TX frames in the same order
- MINOR: quic: Remove the packet number space TX MT_LIST
- MINOR: quic: Splice the frames which could not be added to packets
- MINOR: quic: Add the number of TX bytes to traces
- CLEANUP: quic: Replace <nb_pto_dgrams> by <probe>
- MINOR: quic: Send two ack-eliciting packets when probing packet number spaces
- MINOR: quic: Probe regardless of the congestion control
- MINOR: quic: Speeding up handshake completion
- MINOR: quic: Release RX Initial packets asap
- MINOR: quic: Release asap TX frames to be transmitted
- MINOR: quic: Probe even if coalescing
- BUG/MEDIUM: cli: Never wait for more data on client shutdown
- BUG/MEDIUM: mcli: do not try to parse empty buffers
- BUG/MEDIUM: mcli: always realign wrapping buffers before parsing them
- BUG/MINOR: stream: make the call_rate only count the no-progress calls
- MINOR: quic: do not use quic_conn after dropping it
- MINOR: quic: adjust quic_conn refcount decrement
- MINOR: quic: fix race-condition on xprt tasklet free
- MINOR: quic: free SSL context on quic_conn free
- MINOR: quic: Add QUIC_FT_RETIRE_CONNECTION_ID parsing case
- MINOR: quic: Wrong packet number space selection
- DEBUG: pools: add new build option DEBUG_POOL_INTEGRITY
- MINOR: quic: add missing include in quic_sock
- MINOR: quic: fix indentation in qc_send_ppkts
- MINOR: quic: remove dereferencement of connection when possible
- MINOR: quic: set listener accept cb on parsing
- MEDIUM: quic/ssl: add new ex data for quic_conn
- MINOR: quic: initialize ssl_sock_ctx alongside the quic_conn
- MINOR: ssl: fix build in release mode
- MINOR: pools: partially uninline pool_free()
- MINOR: pools: partially uninline pool_alloc()
- MINOR: pools: prepare POOL_EXTRA to be split into multiple extra fields
- MINOR: pools: extend pool_cache API to pass a pointer to a caller
- DEBUG: pools: add new build option DEBUG_POOL_TRACING
- DEBUG: cli: add a new "debug dev fd" expert command
- MINOR: fd: register the write side of the poller pipe as well
- CI: github actions: use cache for SSL libs
- BUILD: debug/cli: condition test of O_ASYNC to its existence
- BUILD: pools: fix build error on DEBUG_POOL_TRACING
- MINOR: quic: refactor header protection removal
- MINOR: quic: handle app data according to mux/connection layer status
- MINOR: quic: refactor app-ops initialization
- MINOR: receiver: define a flag for local accept
- MEDIUM: quic: flag listener for local accept
- MINOR: quic: do not manage connection in xprt snd_buf
- MINOR: quic: remove wait handshake/L6 flags on init connection
- MINOR: listener: add flags field
- MINOR: quic: define QUIC flag on listener
- MINOR: quic: create accept queue for QUIC connections
- MINOR: listener: define per-thr struct
- MAJOR: quic: implement accept queue
- CLEANUP: mworker: simplify mworker_free_child()
- BUILD/DEBUG: lru: update the standalone code to support the revision
- DEBUG: lru: use a xorshift generator in the testing code
- BUG/MAJOR: compiler: relax alignment constraints on certain structures
- BUG/MEDIUM: fd: always align fdtab[] to 64 bytes
- MINOR: quic: No DCID length for datagram context
- MINOR: quic: Comment fix about the token found in Initial packets
- MINOR: quic: Get rid of a struct buffer in quic_lstnr_dgram_read()
- MINOR: quic: Remove the QUIC haproxy server packet parser
- MINOR: quic: Add new defintion about DCIDs offsets
- MINOR: quic: Add a list to QUIC sock I/O handler RX buffer
- MINOR: quic: Allocate QUIC datagrams from sock I/O handler
- MINOR: proto_quic: Allocate datagram handlers
- MINOR: quic: Pass CID as a buffer to quic_get_cid_tid()
- MINOR: quic: Convert quic_dgram_read() into a task
- CLEANUP: quic: Remove useless definition
- MINOR: proto_quic: Wrong allocations for TX rings and RX bufs
- MINOR: quic: Do not consume the RX buffer on QUIC sock i/o handler side
- MINOR: quic: Do not reset a full RX buffer
- MINOR: quic: Attach all the CIDs to the same connection
- MINOR: quic: Make usage of by datagram handler trees
- MEDIUM: da: new optional data file download scheduler service.
- MEDIUM: da: update doc and build for new scheduler mode service.
- MEDIUM: da: update module to handle schedule mode.
- MINOR: quic: Drop Initial packets with wrong ODCID
- MINOR: quic: Wrong RX buffer tail handling when no more contiguous data
- MINOR: quic: Iterate over all received datagrams
- MINOR: quic: refactor quic CID association with threads
- BUG/MEDIUM: resolvers: Really ignore trailing dot in domain names
- DEV: flags: Add missing flags
- BUG/MINOR: sink: Use the right field in appctx context in release callback
- MINOR: sock: move the unused socket cleaning code into its own function
- BUG/MEDIUM: mworker: close unused transferred FDs on load failure
- BUILD: atomic: make the old HA_ATOMIC_LOAD() support const pointers
- BUILD: cpuset: do not use const on the source of CPU_AND/CPU_ASSIGN
- BUILD: checks: fix inlining issue on set_srv_agent_[addr,port}
- BUILD: vars: avoid overlapping field initialization
- BUILD: server-state: avoid using not-so-portable isblank()
- BUILD: mux_fcgi: avoid aliasing of a const struct in traces
- BUILD: tree-wide: mark a few numeric constants as explicitly long long
- BUILD: tools: fix warning about incorrect cast with dladdr1()
- BUILD: task: use list_to_mt_list() instead of casting list to mt_list
- BUILD: mworker: include tools.h for platforms without unsetenv()
- BUG/MINOR: mworker: fix a FD leak of a sockpair upon a failed reload
- MINOR: mworker: set the master side of ipc_fd in the worker to -1
- MINOR: mworker: allocate and initialize a mworker_proc
- CI: Consistently use actions/checkout@v2
- REGTESTS: Remove REQUIRE_VERSION=1.8 from all tests
- MINOR: mworker: sets used or closed worker FDs to -1
- MINOR: quic: Try to accept 0-RTT connections
- MINOR: quic: Do not try to treat 0-RTT packets without started mux
- MINOR: quic: Do not try to accept a connection more than one time
- MINOR: quic: Initialize the connection timer asap
- MINOR: quic: Do not use connection struct xprt_ctx too soon
- Revert "MINOR: mworker: sets used or closed worker FDs to -1"
- BUILD: makefile: avoid testing all -Wno-* options when not needed
- BUILD: makefile: validate support for extra warnings by batches
- BUILD: makefile: only compute alternative options if required
- DEBUG: fd: make sure we never try to insert/delete an impossible FD number
- MINOR: mux-quic: add comment
- MINOR: mux-quic: properly initialize qcc flags
- MINOR: mux-quic: do not consider CONNECTION_CLOSE for the moment
- MINOR: mux-quic: create a timeout task
- MEDIUM: mux-quic: delay the closing with the timeout
- MINOR: mux-quic: release idle conns on process stopping
- MINOR: listener: replace the listener's spinlock with an rwlock
- BUG/MEDIUM: listener: read-lock the listener during accept()
- MINOR: mworker/cli: set expert/experimental mode from the CLI
Allow to set the master CLI in expert or experimental mode. No command
within the master are unlocked yet, but it gives the ability to send
expert or experimental commands to the workers.
echo "@1; experimental-mode on; del server be1/s2" | socat /var/run/haproxy.master -
echo "experimental-mode on; @1 del server be1/s2" | socat /var/run/haproxy.master -
Mentions of the new database update runtime mode and update of
the legit module and the dummy part too.
Note the DeviceAtlas C API version 2.4.0 minimum required
alongside with libCURL, libzip and libgz.
This new option, when set, will cause the callers of pool_alloc() and
pool_free() to be recorded into an extra area in the pool that is expected
to be helpful for later inspection (e.g. in core dumps). For example it
may help figure that an object was released to a pool with some sub-fields
not yet released or that a use-after-free happened after releasing it,
with an immediate indication about the exact line of code that released
it (possibly an error path).
This only works with the per-thread cache, and even objects refilled from
the shared pool directly into the thread-local cache will have a NULL
there. That's not an issue since these objects have not yet been freed.
It's worth noting that pool_alloc_nocache() continues not to set any
caller pointer (e.g. when the cache is empty) because that would require
a possibly undesirable API change.
The extra cost is minimal (one pointer per object) and this completes
well with DEBUG_POOL_INTEGRITY.
When enabled, objects picked from the cache are checked for corruption
by comparing their contents against a pattern that was placed when they
were inserted into the cache. Objects are also allocated in the reverse
order, from the oldest one to the most recent, so as to maximize the
ability to detect such a corruption. The goal is to detect writes after
free (or possibly hardware memory corruptions). Contrary to DEBUG_UAF
this cannot detect reads after free, but may possibly detect later
corruptions and will not consume extra memory. The CPU usage will
increase a bit due to the cost of filling/checking the area and for the
preference for cold cache instead of hot cache, though not as much as
with DEBUG_UAF. This option is meant to be usable in production.
This command was integrated in 2.4 when it was not possible to handle
SSL with dynamic servers, this is now possible so we should prefer this
way.
Must be backported in 2.5.
While giving a fresh try to `set server ssl` (which I wrote), I realised
the behavior is a bit inconsistent. Indeed when using this command over
a server with ssl enabled for the data path but also for the health
check path we have:
- data and health check done using tls
- emit `set server be_foo/srv0 ssl off`
- data path and health check path becomes plain text
- emit `set server be_foo/srv0 ssl on`
- data path becomes tls and health check path remains plain text
while I thought the end result would be:
- data path and health check path comes back in tls
In the current code we indeed erase all connections while deactivating,
but restore only the data path while activating. I made this mistake in
the past because I was testing with a case where the health check plain
text by default.
There are several ways to solve this issue. The cleanest one would
probably be to avoid changing the health check connection when we use
`set server ssl` command, and create a new command `set server
ssl-check` to change this. For now I assumed this would be ok to simply
avoid changing the health check path and be more consistent.
This patch tries to address that and also update the documentation. It
should not break the existing usage with health check on plain text, as
in this case they should have `no-check-ssl` in defaults. Without this
patch, it makes the command unusable in an env where you have a list of
server to add along the way with initial `server-template`, and all
using tls for data and healthcheck path.
For 2.6 we should probably reconsider and add `set server ssl-check`
command for better granularity of cases.
If this solution is accepted, this patch should be backported up to >=
2.4.
The alternative solution was to restore the previous state, but I
believe this will create even more confusion in the future.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
The purpose here is to explain how memory pools work, what their
architecture is depending on the build options (4 possible combinations),
and how the various build options affect their behavior.
Two pool-specific macros that were previously documented in initcalls
were moved to pools.txt.
Avoid closing idle connections if a soft stop is in progress.
By default, idle connections will be closed during a soft stop. In some
environments, a client talking to the proxy may have prepared some idle
connections in order to send requests later. If there is no proper retry
on write errors, this can result in errors while haproxy is reloading.
Even though a proper implementation should retry on connection/write
errors, this option was introduced to support back compat with haproxy <
v2.4. Indeed before v2.4, we were waiting for a last request to be able
to add a "connection: close" header and advice the client to close the
connection.
In a real life example, this behavior was seen in AWS using the ALB in
front of a haproxy. The end result was ALB sending 502 during haproxy
reloads.
This patch was tested on haproxy v2.4, with a regular reload on the
process, and a constant trend of requests coming in. Before the patch,
we see regular 502 returned to the client; when activating the option,
the 502 disappear.
This patch should help fixing github issue #1506.
In order to unblock some v2.3 to v2.4 migraton, this patch should be
backported up to v2.4 branch.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
[wt: minor edits to the doc to mention other options to care about]
Signed-off-by: Willy Tarreau <w@1wt.eu>
This is a second help to dump loaded library names late at boot, once
external code has already been initialized. The purpose is to provide
a format that makes it easy to pass to "tar" to produce an archive
containing the executable and the list of dependencies. For example
if haproxy is started as "haproxy -f foo.cfg", a config check only
will suffice to quit before starting, "-q" will be used to disable
undesired output messages, and -dL will be use to dump libraries.
This will result in such a command to trivially produce a tarball
of loaded libraries:
./haproxy -q -c -dL -f foo.cfg | tar -T - -hzcf archive.tgz
Many times core dumps reported by users who experience trouble are
difficult to exploit due to missing system libraries. Sometimes,
having just a list of loaded libraries and their respective addresses
can already provide some hints about some problems.
This patch makes a step in that direction by adding a new "show libs"
command that will try to enumerate the list of object files that are
loaded in memory, relying on the dynamic linker for this. It may also
be used to detect that some foreign code embarks other undesired libs
(e.g. some external Lua modules).
At the moment it's only supported on glibc when USE_DL is set, but it's
implemented in a way that ought to make it reasonably easy to be extended
to other platforms.
There were empty lines in the output of the CLI's "show ssl
ocsp-response" command (after the certificate ID and between two
certificates). This patch removes them since an empty line should mark
the end of the output.
Must be backported in 2.5.
The set-var converter as well as the http and tcp set-var actions can
now be given multiple conditions that need to all be true for the
variable's contents to actually be changed. Those conditions can concern
the variable as well as the input contents and can also work by
comparing the variable and the input values.
numa_detect_topology() is always define now if USE_CPU_AFFINITY is
activated. For the moment, only on Linux an actual implementation is
provided. For other platforms, it always return 0.
This change has been made to easily add implementation of NUMA detection
for other platforms. The phrasing of the documentation has also been
edited to removed the mention of Linux-only on numa-cpu-mapping
configuration option.
This patch implements a simple "show version" command which returns
the version of the current process.
It's available from the master and the worker processes, so it is easy
to check if the master and the workers have the same version.
This is a minor patch that really improve compatibility checks
for scripts.
Could be backported in haproxy version as far as 2.0.
In commit 6f7497616 ("MEDIUM: connection: rename fc_conn_err and
bc_conn_err to fc_err and bc_err"), fc_conn_err became fc_err, so
update this example.
It is now possible to perform captures on the response when
http-after-response rules are evaluated. It may be handy to capture headers
from responses generated by HAProxy.
This patch is trivial, it may be backported if necessary.
Only one event is possible for a spoe-message section. If defined several
time, only the last one is considered. The documentation is now explicit on
this point.
This patch is related to the the issue #1351.
With the master worker, the seamless reload was still requiring an
external stats socket to the previous process, which is a pain to
configure.
This patch implements a way to use the internal socketpair between the
master and the workers to transfer the sockets during the reload.
This way, the master will always try to transfer the socket, even
without any configuration.
The master will still reload with the -x argument, followed by the
sockpair@ syntax. ( ex -x sockpair@4 ). Which use the FD of internal CLI
to the worker.
Released version 2.5.0 with the following main changes :
- BUILD: SSL: add quictls build to scripts/build-ssl.sh
- BUILD: SSL: add QUICTLS to build matrix
- CLEANUP: sock: Wrap `accept4_broken = 1` into additional parenthesis
- BUILD: cli: clear a maybe-unused warning on some older compilers
- BUG/MEDIUM: cli: make sure we can report a warning from a bind keyword
- BUG/MINOR: ssl: make SSL counters atomic
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: ssl: free correctly the sni in the backend SSL cache
- MINOR: version: mention that it's stable now
Released version 2.5-dev15 with the following main changes :
- BUG/MINOR: stick-table/cli: Check for invalid ipv6 key
- CLEANUP: peers: Remove useless test on peer variable in peer_trace()
- DOC: log: Add comments to specify when session's listener is defined or not
- BUG/MEDIUM: mux-h1: Handle delayed silent shut in h1_process() to release H1C
- REGTESTS: ssl_crt-list_filters: feature cmd incorrectly set
- DOC: internals: document the list API
- BUG/MINOR: h3: ignore unknown frame types
- MINOR: quic: redirect app_ops snd_buf through mux
- MEDIUM: quic: inspect ALPN to install app_ops
- MINOR: quic: support hq-interop
- MEDIUM: quic: send version negotiation packet on unknown version
- BUG/MEDIUM: mworker: cleanup the listeners when reexecuting
- DOC: internals: document the scheduler API
- BUG/MINOR: quic: fix version negotiation packet generation
- CLEANUP: ssl: fix wrong #else commentary
- MINOR: config: support default values for environment variables
- SCRIPTS: run-regtests: reduce the number of processes needed to check options
- SCRIPT: run-regtests: avoid several calls to grep to test for features
- SCRIPT: run-regtests: avoid calling awk to compute the version
- REGTEST: set retries count to zero for all tests that expect at 503
- REGTESTS: make tcp-check_min-recv fail fast
- REGTESTS: extend the default I/O timeouts and make them overridable
- BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3
- BUG/MEDIUM: ssl: abort with the correct SSL error when SNI not found
- REGTESTS: ssl: test the TLS resumption
- BUILD: makefile: stop opening sub-shells for each and every command
- BUILD: makefile: reorder objects by build time
- BUG/MEDIUM: mux-h2: always process a pending shut read
- MINOR: quic_sock: missing CO_FL_ADDR_TO_SET flag
- MINOR: quic: Possible wrong connection identification
- MINOR: quic: Correctly pad UDP datagrams
- MINOR: quic: Support transport parameters draft TLS extension
- MINOR: quic: Anti-amplification implementation
- MINOR: quic: Wrong Initial packet connection initialization
- MINOR: quic: Wrong ACK range building
- MINOR: quic: Update some QUIC protocol errors
- MINOR: quic: Send CONNECTION_CLOSE frame upon TLS alert
- MINOR: quic: Wrong largest acked packet number parsing
- MINOR: quic: Add minimalistic support for stream flow control frames
- MINOR: quic: Wrong value for version negotiation packet 'Unused' field
- MINOR: quic: Support draft-29 QUIC version
- BUG/MINOR: quic: fix segfault on trace for version negotiation
- BUG/MINOR: hq-interop: fix potential NULL dereference
- BUILD: quic: fix potential NULL dereference on xprt_quic
- DOC: lua: documentation about the httpclient API
- BUG/MEDIUM: cache/cli: make "show cache" thread-safe
- BUG/MEDIUM: shctx: leave the block allocator when enough blocks are found
- BUG/MINOR: shctx: do not look for available blocks when the first one is enough
- MINOR: shctx: add a few BUG_ON() for consistency checks
Sometimes it is really useful to be able to specify a default value for
an optional environment variable, like the ${name-value} construct in
shell. In fact we're really missing this for a number of settings in
reg tests, starting with timeouts.
This commit simply adds support for the common syntax above. Other
common forms like '+' to replace existing variables, or ':-' and ':+'
to act on empty variables, were not implemented at this stage, as they
are less commonly needed.
Another non-trivial part that is often needed. Exported functions
and flags available to applications were documented as well as some
restrictions and falltraps.
Released version 2.5-dev14 with the following main changes :
- DEV: coccinelle: Remove unused `expression e`
- DEV: coccinelle: Add rule to use `istend()` where possible
- CLEANUP: Apply ist.cocci
- CLEANUP: Re-apply xalloc_size.cocci
- CLEANUP: halog: make the default usage message fit in small screens
- MINOR: h3/qpack: fix gcc11 warnings
- MINOR: mux-quic: fix gcc11 warning
- MINOR: h3: fix potential NULL dereference
- MINOR: quic: Fix potential null pointer dereference
- CLEANUP: halog: remove unused strl2ui()
- OPTIM: halog: improve field parser speed for modern compilers
- OPTIM: halog: skip fields 64 bits at a time when supported
- DEV: coccinelle: Add rule to use `isttrim()` where possible
- CLEANUP: Apply ist.cocci
- DEV: coccinelle: Add rule to use `chunk_istcat()` instead of `chunk_memcat()`
- DEV: coccinelle: Add rule to use `chunk_istcat()` instead of `chunk_strncat()`
- CLEANUP: Apply ist.cocci
- CLEANUP: chunk: Remove duplicated chunk_Xcat implementation
- CLEANUP: chunk: remove misleading chunk_strncat() function
- BUG/MINOR: cache: properly ignore unparsable max-age in quotes
- Revert "DEV: coccinelle: Add rule to use `chunk_istcat()` instead of `chunk_strncat()`"
- DOC: stats: fix location of the text representation
- DOC: internals: document the IST API
- BUG/MINOR: httpclient/lua: rcv freeze when no request payload
- BUG/MEDIUM: httpclient: channel_add_input() must use htx->data
- MINOR: promex: backend aggregated server check status
- DOC: config: Fix typo in ssl_fc_unique_id description
- BUG/MINOR: http-ana: Apply stop to the current section for http-response rules
- Revert "BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on back"
- DOC: config: Be more explicit in "allow" actions description
- DOC: lua: Be explicit with the Reply object limits
- MINOR: mux-h1: Slightly Improve H1 traces
- BUG/MEDIUM: conn-stream: Don't reset CS flags on close
- CLEANUP: mworker: remove any relative PID reference
- MEDIUM: mworker: reexec in waitpid mode after successful loading
- MINOR: mworker: clarify starting/failure messages
- MINOR: mworker: only increment the number of reload in wait mode
- MINOR: mworker: implement a reload failure counter
- MINOR: mworker: ReloadFailed shown depending on failedreload
- MINOR: mworker: change the way we set PROC_O_LEAVING
- BUG/MINOR: mworker: doesn't launch the program postparser
- DOC: management: edit the "show proc" example to show the current output
- BUG/MEDIUM: httpclient/cli: free of unallocated hc->req.uri
- REGTESTS: httpclient/lua: add greater body values
- BUG/MINOR: mux-h2: Fix H2_CF_DEM_SHORT_READ value
- BUG/MINOR: pools: don't mark ourselves as harmless in DEBUG_UAF mode
- BUG/MEDIUM: connection: make cs_shutr/cs_shutw//cs_close() idempotent
- BUILD: makefile: simplify detection of libatomic
The "show proc" output changed and it's time to update the example.
The output does not display the relative PID anymore since the nbproc
keyword disappeared, and it displays the number of failed reloads since
the last successful one.
In HTTP, when a lua action is evaluated, a reply object can be used to send
a response to the client and interrupt the transaction. This reply object is
converted into HTX and is limited to the response channel buffer. Its size,
once converted, cannot exceed the buffer size. There is no streaming at this
stage. However, this limitation was not documented.
Note that, for now, there is no easy way to know if the reply will fit or
not int the response channel buffer. Thus the reply must be reasonably
small. Otherwise a 500-Internal-Error message is returned.
This patch is related to the issue #1447. It may be backported as far as
2.2.
TCP/HTTP allow actions stop rules evaluation of the current section
only. Only the http-response description was accurate on this
point. Thus, the documentation is now explicit on this point for all
other concerned rulesets.
This patch may be backported, to all supported versions for tcp-request
and http-request documentation, and as far as 2.2 for http-after-response
documentation.
In ssl_fc_unique_id decription, threre is a reference to the wrong sample
fetch. ssl_bc_unique_id is used instead of ssl_fc_unique_id.
This patch should fix the issue #1449. It may be backported to all
supportted versions.
This one was missing. It should be easier to use now. It is obvious that
some functions are missing, and it looks like ist2str() and istpad() are
exactly the same.
Released version 2.5-dev13 with the following main changes :
- SCRIPTS: git-show-backports: re-enable file-based filtering
- MINOR: jwt: Make invalid static JWT algorithms an error in `jwt_verify` converter
- MINOR: mux-h2: add trace on extended connect usage
- BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
- MINOR: stream/mux: implement websocket stream flag
- MINOR: connection: implement function to update ALPN
- MINOR: connection: add alternative mux_ops param for conn_install_mux_be
- MEDIUM: server/backend: implement websocket protocol selection
- MINOR: server: add ws keyword
- BUG/MINOR: resolvers: fix sent messages were counted twice
- BUG/MINOR: resolvers: throw log message if trash not large enough for query
- MINOR: resolvers/dns: split dns and resolver counters in dns_counter struct
- MEDIUM: resolvers: rename dns extra counters to resolvers extra counters
- BUG/MINOR: jwt: Fix jwt_parse_alg incorrectly returning JWS_ALG_NONE
- DOC: add QUIC instruction in INSTALL
- CLEANUP: halog: Remove dead stores
- DEV: coccinelle: Add ha_free.cocci
- CLEANUP: Apply ha_free.cocci
- DEV: coccinelle: Add rule to use `istnext()` where possible
- CLEANUP: Apply ist.cocci
- REGTESTS: Use `feature cmd` for 2.5+ tests (2)
- DOC: internals: move some API definitions to an "api" subdirectory
- MINOR: quic: Allocate listener RX buffers
- CLEANUP: quic: Remove useless code
- MINOR: quic: Enhance the listener RX buffering part
- MINOR: quic: Remove a useless lock for CRYPTO frames
- MINOR: quic: Use QUIC_LOCK QUIC specific lock label.
- MINOR: backend: Get client dst address to set the server's one only if needful
- MINOR: compression: Warn for 'compression offload' in defaults sections
- MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
- DOC: configuration: move the default log formats to their own section
- MINOR: ssl: make the ssl_fc_sni() sample-fetch function always available
- MEDIUM: log: add the client's SNI to the default HTTPS log format
- DOC: config: add an example of reasonably complete error-log-format
- DOC: config: move error-log-format before custom log format
All default formats were described before the custom one, except this
one. Better place them all together before the custom log format. This
only swaps and renumbers the sections.
During a troublehooting it came obvious that the SNI always ought to
be logged on httpslog, as it explains errors caused by selection of
the default certificate (or failure to do so in case of strict-sni).
This expectation was also confirmed on the mailing list.
Since the field may be empty it appeared important not to leave an
empty string in the current format, so it was decided to place the
field before a '/' preceding the SSL version and ciphers, so that
in the worst case a missing field leads to a field looking like
"/TLSv1.2/AES...", though usually a missing element still results
in a "-" in logs.
This will change the log format for users who already deployed the
2.5-dev versions (hence the medium level) but no released version
was using this format yet so there's no harm for stable deployments.
The reg-test was updated to check for "-" there since we don't send
SNI in reg-tests.
Link: https://www.mail-archive.com/haproxy@formilux.org/msg41410.html
Cc: William Lallemand <wlallemand@haproxy.org>
I'm always having a very hard time finding the log-format definition of
httplog, because it's not in the httplog description, and looking for
"httplog" doesn't yield the custom log formats section.
It would make more sense to write these log-formats into their respective
sections where they will be easier to find. That's what this commit does.
Commit 3d2093af9 ("MINOR: connection: Add a connection error code sample
fetch") added these convenient sample-fetch functions but it appears that
due to a misunderstanding the redundant "conn" part was kept in their
name, causing confusion, since "fc" already stands for "front connection".
Let's simply call them "fc_err" and "bc_err" to match all other related
ones before they appear in a final release. The VTC they appeared in were
also updated, and the alpha sort in the keywords table updated.
Cc: William Lallemand <wlallemand@haproxy.org>
This directive is documented as being ignored if set in a defaults
section. But it is only mentionned in a small note in the configuration
manual. Thus, now, a warning is emitted. To do so, the errors handling in
parse_compression_options() function was slightly changed.
In addition, this directive is now documented apart from the other
compression directives. This way, it is clearly visible that it must not be
used in a defaults section.
It's not always easy to figure that there are some docs on internal API
stuff, let's move them to their own directory. There's a diagram for
lists that could be placed there but instead would deserve a greppable
description for quick lookups, so it was not moved there.
Implement parsing for the server keyword 'ws'. This is used to configure
the mode of selection for websocket protocol. The configuration
documentation has been updated.
A new regtest has been created to test the proper behavior of the
keyword.
Released version 2.5-dev12 with the following main changes :
- MINOR: httpclient: support payload within a buffer
- MINOR: httpclient/lua: support more HTTP methods
- MINOR: httpclient/lua: return an error when it can't generate the request
- CLEANUP: lua: Remove any ambiguities about lua txn execution context flags
- BUG/MEDIUM: lua: fix invalid return types in hlua_http_msg_get_body
- CLEANUP: connection: No longer export make_proxy_line_v1/v2 functions
- CLEANUP: tools: Use const address for get_net_port() and get_host_port()
- CLEANUP: lua: Use a const address to retrieve info about a connection
- MINOR: connection: Add function to get src/dst without updating the connection
- MINOR: session: Add src and dst addresses to the session
- MINOR: stream-int: Add src and dst addresses to the stream-interface
- MINOR: frontend: Rely on client src and dst addresses at stream level
- MINOR: log: Rely on client addresses at the appropriate level to log messages
- MINOR: session: Rely on client source address at session level to log error
- MINOR: http-ana: Rely on addresses at stream level to set xff and xot headers
- MINOR: http-fetch: Rely on addresses at stream level in HTTP sample fetches
- MINOR: mux-fcgi: Rely on client addresses at stream level to set default params
- MEDIUM: tcp-sample: Rely on addresses at the appropriate level in tcp samples
- MEDIUM: connection: Rely on addresses at stream level to make proxy line
- MEDIUM: backend: Rely on addresses at stream level to init server connection
- MEDIUM: connection: Assign session addresses when PROXY line is received
- MEDIUM: connection: Assign session addresses when NetScaler CIP proto is parsed
- MEDIUM: tcp-act: Set addresses at the apprioriate level in set-(src/dst) actions
- MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
- DOC: config: Fix alphabetical order of fc_* samples
- MINOR: tcp-sample: Add samples to get original info about client connection
- REGTESTS: Add script to test client src/dst manipulation at different levels
- MINOR: stream: Use backend stream-interface dst address instead of target_addr
- BUILD: log: Fix compilation without SSL support
- DEBUG: protocol: yell loudly during registration of invalid sock_domain
- MINOR: protocols: add a new protocol type selector
- MINOR: protocols: make use of the protocol type to select the protocol
- MINOR: protocols: replace protocol_by_family() with protocol_lookup()
- MINOR: halog: Add -qry parameter allowing to preserve the query string in -uX
- CLEANUP: jwt: Remove the use of a trash buffer in jwt_jwsverify_hmac()
- CLEANUP: jwt: Remove the use of a trash buffer in jwt_jwsverify_rsa_ecdsa()
- DEV: coccinelle: Add realloc_leak.cocci
- CLEANUP: hlua: Remove obsolete branch in `hlua_alloc()`
- BUILD: atomic: prefer __atomic_compare_exchange_n() for __ha_cas_dw()
- BUILD: atomic: fix build on mac/arm64
- MINOR: atomic: remove the memcpy() call and dependency on string.h
- MINOR: httpclient: request streaming with a callback
- MINOR: httpclient/lua: handle the streaming into the lua applet
- REGTESTS: lua: test httpclient with body streaming
- DOC: halog: Move the `-qry` parameter into the correct section in help text
- MINOR: halog: Rename -qry to -query
- CLEANUP: halog: Use consistent indentation in help()
- BUG/MINOR: halog: Add missing newlines in die() messages
- MINOR: halog: Add support for extracting captures using -hdr
- DOC: Typo fixed "it" should be "is"
- BUG/MINOR: mux-h1: Save shutdown mode if the shutdown is delayed
- BUG/MEDIUM: mux-h1: Perform a connection shutdown when the h1c is released
- BUG/MEDIUM: resolvers: Don't recursively perform requester unlink
- BUG/MEDIUM: http-ana: Drain request data waiting the tarpit timeout expiration
- BUG/MINOR: http: Authorization value can have multiple spaces after the scheme
- BUG/MINOR: http: http_auth_bearer fetch does not work on custom header name
- BUG/MINOR: httpclient/lua: misplaced luaL_buffinit()
- BUILD/MINOR: cpuset freebsd build fix
- BUG/MINOR: httpclient: use a placeholder value for Host header
- BUG/MEDIUM: stream-int: Block reads if channel cannot receive more data
- BUG/MEDIUM: resolvers: Track api calls with a counter to free resolutions
- MINOR: stream: Improve dump of bogus streams
- DOC/peers: some grammar fixes for peers 2.1 spec
- MEDIUM: vars: make the var() sample fetch function really return type ANY
- MINOR: vars: add "set-var" for "tcp-request connection" rules.
Session struct is already allocated when "tcp-request connection" rules
are evaluated so session-scoped variables turned out easy to support.
This resolves github issue #1408.
Because source and destination address of the client connection are now
updated at the appropriated level (connection, session or stream), original
info about the client connection are preserved. src/src_port/src_is_local
and dst/dst_port/dst_is_local return current info about the client
connection. It is the info at the highest available level. Most of time, the
stream. Any tcp/http rules may alter this info.
To get original info, "fc_" prefix must be added. For instance
"fc_src". Here, only "tcp-request connection" rules may alter source and
destination address/port.
This patch was reverted because it was inconsitent to change connection
addresses at stream level. Especially in HTTP because all requests was
affected by this change and not only the current one. In HTTP/2, it was
worse. Several streams was able to change the connection addresses at the
same time.
It is no longer an issue, thanks to recent changes. With multi-level client
source and destination addresses, it is possible to limit the change to the
current request. Thus this patch can be reintroduced.
If it possible to set source IP/Port from "tcp-request connection",
"tcp-request session" and "http-request" rules but not from "tcp-request
content" rules. There is no reason for this limitation and it may be a
problem for anyone wanting to call a lua fetch to dynamically set source
IP/Port from a TCP proxy. Indeed, to call a lua fetch, we must have a
stream. And there is no stream when "tcp-request connection/session" rules
are evaluated.
Thanks to this patch, "set-src" and "set-src-port" action are now supported
by "tcp_request content" rules.
This patch is related to the issue #1303.
Released version 2.5-dev11 with the following main changes :
- DEV: coccinelle: Add strcmp.cocci
- CLEANUP: Apply strcmp.cocci
- CI: Add `permissions` to GitHub Actions
- CI: Clean up formatting in GitHub Action definitions
- MINOR: add ::1 to predefined LOCALHOST acl
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: Consistently `unsigned int` for bitfields
- MEDIUM: resolvers: lower-case labels when converting from/to DNS names
- MEDIUM: resolvers: replace bogus resolv_hostname_cmp() with memcmp()
- MINOR: jwt: Empty the certificate tree during deinit
- MINOR: jwt: jwt_verify returns negative values in case of error
- MINOR: jwt: Do not rely on enum order anymore
- BUG/MEDIUM: stream: Keep FLT_END analyzers if a stream detects a channel error
- MINOR: httpclient/cli: access should be only done from expert mode
- DOC: management: doc about the CLI httpclient
- BUG/MEDIUM: tcpcheck: Properly catch early HTTP parsing errors
- BUG/MAJOR: dns: tcp session can remain attached to a list after a free
- BUG/MAJOR: dns: attempt to lock globaly for msg waiter list instead of use barrier
- CLEANUP: dns: always detach the appctx from the dns session on release
- DEBUG: dns: add a few more BUG_ON at sensitive places
- BUG/MAJOR: resolvers: add other missing references during resolution removal
- CLEANUP: resolvers: do not export resolv_purge_resolution_answer_records()
- BUILD: resolvers: avoid a possible warning on null-deref
- BUG/MEDIUM: resolvers: always check a valid item in query_list
- CLEANUP: always initialize the answer_list
- CLEANUP: resolvers: simplify resolv_link_resolution() regarding requesters
- CLEANUP: resolvers: replace all LIST_DELETE with LIST_DEL_INIT
- MEDIUM: resolvers: use a kill list to preserve the list consistency
- MEDIUM: resolvers: remove the last occurrences of the "safe" argument
- BUG/MEDIUM: checks: fix the starting thread for external checks
- MEDIUM: resolvers: replace the answer_list with a (flat) tree
- MEDIUM: resolvers: hash the records before inserting them into the tree
- BUG/MAJOR: buf: fix varint API post- vs pre- increment
- OPTIM: resolvers: move the eb32 node before the data in the answer_item
- MINOR: list: add new macro LIST_INLIST_ATOMIC()
- OPTIM: dns: use an atomic check for the list membership
- BUG/MINOR: task: do not set TASK_F_USR1 for no reason
- BUG/MINOR: mux-h2: do not prevent from sending a final GOAWAY frame
- MINOR: connection: add a new CO_FL_WANT_DRAIN flag to force drain on close
- MINOR: mux-h2: perform a full cycle shutdown+drain on close
- CLEANUP: resolvers: get rid of single-iteration loop in resolv_get_ip_from_response()
- MINOR: quic: Increase the size of handshake RX UDP datagrams
- BUG/MEDIUM: lua: fix memory leaks with realloc() on non-glibc systems
- MINOR: memprof: report the delta between alloc and free on realloc()
- MINOR: memprof: add one pointer size to the size of allocations
- BUILD: fix compilation on NetBSD
- MINOR: backend: add traces for idle connections reuse
- BUG/MINOR: backend: fix improper insert in avail tree for always reuse
- MINOR: backend: improve perf with tcp proxies skipping idle conns
- MINOR: connection: remove unneeded memset 0 for idle conns
In order for all the error return values to be distributed on the same
side (instead of surrounding the success error code), the return values
for errors other than a simple verification failure are switched to
negative values. This way the result of the jwt_verify converter can be
compared strictly to 1 as well relative to 0 (any <= 0 return value is
an error).
The documentation was also modified to discourage conversion of the
return value into a boolean (which would definitely not work).
Released version 2.5-dev10 with the following main changes :
- MINOR: initcall: Rename __GLOBL and __GLOBL1.
- MINOR: rules: add a new function new_act_rule() to allocate act_rules
- MINOR: rules: add a file name and line number to act_rules
- MINOR: stream: report the current rule in "show sess all" when known
- MINOR: stream: report the current filter in "show sess all" when known
- CLEANUP: stream: Properly indent current_rule line in "show sess all"
- BUG/MINOR: lua: Fix lua error handling in `hlua_config_prepend_path()`
- CI: github: switch to OpenSSL 3.0.0
- REGTESTS: ssl: Fix references to removed option in test description
- MINOR: ssl: Add ssllib_name_startswith precondition
- REGTESTS: ssl: Fix ssl_errors test for OpenSSL v3
- REGTESTS: ssl: Reenable ssl_errors test for OpenSSL only
- REGTESTS: ssl: Use mostly TLSv1.2 in ssl_errors test
- MEDIUM: mux-quic: rationalize tx buffers between qcc/qcs
- MEDIUM: h3: properly manage tx buffers for large data
- MINOR: mux-quic: standardize h3 settings sending
- CLEANUP: h3: remove dead code
- MINOR: mux-quic: implement standard method to detect if qcc is dead
- MEDIUM: mux-quic: defer stream shut if remaining tx data
- MINOR: mux: remove last occurences of qcc ring buffer
- MINOR: quic: handle CONNECTION_CLOSE frame
- REGTESTS: ssl: re-enable set_ssl_cert_bundle.vtc
- MINOR: ssl: add ssl_fc_is_resumed to "option httpslog"
- MINOR: http: Add http_auth_bearer sample fetch
- MINOR: jwt: Parse JWT alg field
- MINOR: jwt: JWT tokenizing helper function
- MINOR: jwt: Insert public certificates into dedicated JWT tree
- MINOR: jwt: jwt_header_query and jwt_payload_query converters
- MEDIUM: jwt: Add jwt_verify converter to verify JWT integrity
- REGTESTS: jwt: Add tests for the jwt_verify converter
- BUILD: jwt: fix declaration of EVP_KEY in jwt-h.h
- MINOR: proto_tcp: use chunk_appendf() to ouput socket setup errors
- MINOR: proto_tcp: also report the attempted MSS values in error message
- MINOR: inet: report the faulty interface name in "bind" errors
- MINOR: protocol: report the file and line number for binding/listening errors
- MINOR: protocol: uniformize protocol errors
- MINOR: resolvers: fix the resolv_str_to_dn_label() API about trailing zero
- BUG/MEDIUM: resolver: make sure to always use the correct hostname length
- BUG/MINOR: resolvers: do not reject host names of length 255 in SRV records
- MINOR: resolvers: fix the resolv_dn_label_to_str() API about trailing zero
- MEDIUM: listeners: split the thread mask between receiver and bind_conf
- MINOR: listeners: add clone_listener() to duplicate listeners at boot time
- MEDIUM: listener: add the "shards" bind keyword
- BUG/MEDIUM: resolvers: use correct storage for the target address
- MINOR: resolvers: merge address and target into a union "data"
- BUG/MEDIUM: resolvers: fix truncated TLD consecutive to the API fix
- BUG/MEDIUM: jwt: fix base64 decoding error detection
- BUG/MINOR: jwt: use CRYPTO_memcmp() to compare HMACs
- DOC: jwt: fix a typo in the jwt_verify() keyword description
- BUG/MEDIUM: sample/jwt: fix another instance of base64 error detection
- BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on back
- BUG/MINOR: sample: Fix 'fix_tag_value' sample when waiting for more data
- DOC: config: Move 'tcp-response content' at the right place
- BUG/MINOR: proxy: Use .disabled field as a bitfield as documented
- MINOR: proxy: Introduce proxy flags to replace disabled bitfield
- MINOR: sample/arg: Be able to resolve args found in defaults sections
- MEDIUM: proxy: Warn about ambiguous use of named defaults sections
- MINOR: proxy: Be able to reference the defaults section used by a proxy
- MINOR: proxy: Add PR_FL_READY flag on fully configured and usable proxies
- MINOR: config: Finish configuration for referenced default proxies
- MINOR: config: No longer remove previous anonymous defaults section
- MINOR: tcpcheck: Support 2-steps args resolution in defaults sections
- MEDIUM: rules/acl: Parse TCP/HTTP rules and acls defined in defaults sections
- MEDIUM: tcp-rules: Eval TCP rules defined in defaults sections
- MEDIUM: http-ana: Eval HTTP rules defined in defaults sections
- BUG/MEDIUM: sample: Cumulate frontend and backend sample validity flags
- REGTESTS: Add scripts to test support of TCP/HTTP rules in defaults sections
- DOC: config: Add documentation about TCP/HTTP rules in defaults section
- DOC: config: Rework and uniformize how TCP/HTTP rules are documented
- BUG/MINOR: proxy: Release ACLs and TCP/HTTP rules of default proxies
- BUG/MEDIUM: cpuset: fix cpuset size for FreeBSD
- BUG/MINOR: sample: fix backend direction flags consecutive to last fix
- BUG/MINOR: listener: fix incorrect return on out-of-memory
- BUG/MINOR: listener: add an error check for unallocatable trash
- CLEANUP: listeners: remove unreachable code in clone_listener()
Now all these rules are documented using the same structure. First there is
a general description with the list of all supported actions. Then all
actions are described in details. Thus, it is easy to have a quick list of
all supported actions and this avoids to have a huge description with all
info about these actions. In addition, when it is possible, we make a
reference to already documented parts.
Documentation of each directive that can now be used in defaults section was
updated to explain how it works. A special mark was added to specify when a
keyword is supported by defaults sections with a name but not anonymous
ones. In this case an exclamation mark is added.
It is now possible to designate the defaults section to use by adding a name
of the corresponding defaults section and referencing it in the desired
proxy section. However, this introduces an ambiguity. This named defaults
section may still be implicitly used by other proxies if it is the last one
defined. In this case for instance:
default common
...
default frt from common
...
default bck from common
...
frontend fe from frt
...
backend be from bck
...
listen stats
...
Here, it is not really obvious the last section will use the 'bck' defaults
section. And it is probably not the expected behaviour. To help users to
properly configure their haproxy, a warning is now emitted if a defaults
section is explicitly AND implicitly used. The configuration manual was
updated accordingly.
Because this patch adds a warning, it should probably not be backported to
2.4. However, if is is backported, it depends on commit "MINOR: proxy:
Introduce proxy flags to replace disabled bitfield".
In multi-threaded mode, on operating systems supporting multiple listeners on
the same IP:port, this will automatically create this number of multiple
identical listeners for the same line, all bound to a fair share of the number
of the threads attached to this listener. This can sometimes be useful when
using very large thread counts where the in-kernel locking on a single socket
starts to cause a significant overhead. In this case the incoming traffic is
distributed over multiple sockets and the contention is reduced. Note that
doing this can easily increase the CPU usage by making more threads work a
little bit.
If the number of shards is higher than the number of available threads, it
will automatically be trimmed to the number of threads. A special value
"by-thread" will automatically assign one shard per thread.
This new converter takes a JSON Web Token, an algorithm (among the ones
specified for JWS tokens in RFC 7518) and a public key or a secret, and
it returns a verdict about the signature contained in the token. It does
not simply return a boolean because some specific error cases cas be
specified by returning an integer instead, such as unmanaged algorithms
or invalid tokens. This enables to distinguich malformed tokens from
tampered ones, that would be valid format-wise but would have a bad
signature.
This converter does not perform a full JWT validation as decribed in
section 7.2 of RFC 7519. For instance it does not ensure that the header
and payload parts of the token are completely valid JSON objects because
it would need a complete JSON parser. It only focuses on the signature
and checks that it matches the token's contents.
Those converters allow to extract a JSON value out of a JSON Web Token's
header part or payload part (the two first dot-separated base64url
encoded parts of a JWS in the Compact Serialization format).
They act as a json_query call on the corresponding decoded subpart when
given parameters, and they return the decoded JSON subpart when no
parameter is given.
This fetch can be used to retrieve the data contained in an HTTP
Authorization header when the Bearer scheme is used. This is used when
transmitting JSON Web Tokens for instance.
Released version 2.5-dev9 with the following main changes :
- head-truc
- REGTESTS: lua: test the httpclient:get() feature
- Revert "head-truc"
- BUG/MEDIUM: httpclient: replace ist0 by istptr
- MINOR: config: use a standard parser for the "nbthread" keyword
- CLEANUP: init: remove useless test against MAX_THREADS in affinity loop
- MEDIUM: init: de-uglify the per-thread affinity setting
- MINOR: init: extract the setup and end of threads to their own functions
- MINOR: log: Try to get the status code when MUX_EXIT_STATUS is retrieved
- MINOR: mux-h1: Set error code if possible when MUX_EXIT_STATUS is returned
- MINOR: mux-h1: Be able to set custom status code on parsing error
- MEDIUM: mux-h1: Reject HTTP/1.0 GET/HEAD/DELETE requests with a payload
- MEDIUM: h1: Force close mode for invalid uses of T-E header
- BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
- MINOR: http: Add 422-Unprocessable-Content error message
- MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice
- BUG/MEDIUM: mux-h1/mux-fcgi: Reject messages with unknown transfer encoding
- REGTESTS: Add script to validate T-E header parsing
- REORG: pools: move default settings to defaults.h
- DOC: peers: fix doc "enable" statement on "peers" sections
- MINOR: Makefile: add MEMORY_POOLS to the list of DEBUG_xxx options
- MINOR: ssl: Set connection error code in case of SSL read or write fatal failure
- MINOR: ssl: Rename ssl_bc_hsk_err to ssl_bc_err
- MINOR: ssl: Store the last SSL error code in case of read or write failure
- REGTESTS: ssl: enable show_ssl_ocspresponse.vtc again
- REGTESTS: ssl: enable ssl_crt-list_filters.vtc again
- BUG/MEDIUM: lua: fix wakeup condition from sleep()
- BUG/MAJOR: lua: use task_wakeup() to properly run a task once
- MINOR: arg: Be able to forbid unresolved args when building an argument list
- BUG/MINOR: tcpcheck: Don't use arg list for default proxies during parsing
- BUG/MINOR: tcp-rules: Stop content rules eval on read error and end-of-input
- MINOR: tasks: catch TICK_ETERNITY with BUG_ON() in __task_queue()
- REGTESTS: ssl: show_ssl_ocspresponse w/ freebsd won't use base64
- REGTESTS: ssl: wrong feature cmd in show_ssl_ocspresponse.vtc
- CLEANUP: tasks: remove the long-unused work_lists
- MINOR: task: provide 3 task_new_* wrappers to simplify the API
- MINOR: time: uninline report_idle() and move it to task.c
- REORG: sched: move idle time calculation from time.h to task.h
- REORG: sched: move the stolen CPU time detection to sched_entering_poll()
- BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the release
- BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
- MINOR: httpclient: destroy() must free the headers and the ists
- MINOR: httpclient: set HTTPCLIENT_F_ENDED only in release
- MINOR: httpclient: stop_and_destroy() ask the applet to autokill
- MINOR: httpclient: test if started during stop_and_destroy()
- MINOR: httpclient/lua: implement garbage collection
- BUG/MEDIUM: httpclient/lua: crash because of b_xfer and get_trash_chunk()
- MINOR: httpclient: destroy checks if a client was started but not stopped
- BUG/MINOR: httpclient/lua: does not process headers when failed
- MINOR: httpclient/lua: supports headers via named arguments
- CLEANUP: server: always include the storage for SSL settings
- CLEANUP: sample: rename sample_conv_var2smp() to *_sint
- CLEANUP: sample: uninline sample_conv_var2smp_str()
- MINOR: sample: provide a generic var-to-sample conversion function
- BUG/MEDIUM: sample: properly verify that variables cast to sample
- BUILD: action: add the relevant structures for function arguments
- BUILD: extcheck: needs to include stream-t.h
- BUILD: hlua: needs to include stream-t.h
- BUILD: stats: define several missing structures in stats.h
- BUILD: resolvers: define missing types in resolvers.h
- BUILD: httpclient: include missing ssl_sock-t
- BUILD: sample: include openssl-compat
- BUILD: http_ana: need to include proxy-t to get redirect_rule
- BUILD: http_rules: requires http_ana-t.h for REDIRECT_*
- BUILD: vars: need to include xxhash
- BUILD: peers: need to include eb{32/mb/pt}tree.h
- BUILD: ssl_ckch: include ebpttree.h in ssl_ckch.c
- BUILD: compiler: add the container_of() and container_of_safe() macros
- BUILD: idleconns: include missing ebmbtree.h at several places
- BUILD: connection: connection.h needs list.h and server.h
- BUILD: tree-wide: add missing http_ana.h from many places
- BUILD: cfgparse-ssl: add missing errors.h
- BUILD: tcp_sample: include missing errors.h and session-t.h
- BUILD: mworker: mworker-prog needs time.h for the 'now' variable
- BUILD: tree-wide: add several missing activity.h
- BUILD: compat: fix -Wundef on SO_REUSEADDR
- CLEANUP: pools: pools-t.h doesn't need to include thread-t.h
- REORG: pools: uninline the UAF allocator and force-inline the rest
- REORG: thread: uninline the lock-debugging code
- MINOR: thread/debug: replace nsec_now() with now_mono_time()
- CLEANUP: remove some unneeded includes from applet-t.h
- REORG: listener: move bind_conf_alloc() and listener_state_str() to listener.c
- CLEANUP: listeners: do not include openssl-compat
- CLEANUP: servers: do not include openssl-compat
- REORG: ssl: move ssl_sock_is_ssl() to connection.h and rename it
- CLEANUP: mux_fcgi: remove dependency on ssl_sock
- CLEANUP: ssl/server: move ssl_sock_set_srv() to srv_set_ssl() in server.c
- REORG: ssl-sock: move the sslconns/totalsslconns counters to global
- REORG: sample: move the crypto samples to ssl_sample.c
- REORG: sched: moved samp_time and idle_time to task.c as well
- REORG: time/ticks: move now_ms and global_now_ms definitions to ticks.h
- CLEANUP: tree-wide: remove unneeded include time.h in ~20 files
- REORG: activity: uninline activity_count_runtime()
- REORG: acitvity: uninline sched_activity_entry()
- CLEANUP: stream: remove many unneeded includes from stream-t.h
- CLEANUP: stick-table: no need to include socket nor in.h
- MINOR: connection: use uint64_t for the hashes
- REORG: connection: move the hash-related stuff to connection.c
- REORG: connection: uninline conn_notify_mux() and conn_delete_from_tree()
- REORG: server: uninline the idle conns management functions
- REORG: ebtree: split structures into their own file ebtree-t.h
- CLEANUP: tree-wide: only include ebtree-t from type files
- REORG: connection: move the largest inlines from connection.h to connection.c
- CLEANUP: connection: do not include http_ana!
- CLEANUP: connection: remove unneeded tcpcheck-t.h and use only session-t.h
- REORG: connection: uninline the rest of the alloc/free stuff
- REORG: task: uninline the loop time measurement code
- CLEANUP: time: move a few configurable defines to defaults.h
- CLEANUP: fd: do not include time.h
- REORG: fd: uninline compute_poll_timeout()
- CLENAUP: wdt: use ha_tkill() instead of accessing pthread directly
- REORG: thread: move the thread init/affinity/stop to thread.c
- REORG: thread: move ha_get_pthread_id() to thread.c
- MINOR: thread: use a dedicated static pthread_t array in thread.c
- CLEANUP: thread: uninline ha_tkill/ha_tkillall/ha_cpu_relax()
- DOC: configuration: add clarification on escaping in keyword arguments
- BUG/MINOR: task: fix missing include with DEBUG_TASK
- MINOR: pools: report the amount used by thread caches in "show pools"
- MINOR: quic: Distinguish packet and SSL read enc. level in traces
- MINOR: quic: Add a function to dump SSL stack errors
- MINOR: quic: BUG_ON() SSL errors.
- MINOR: quic: Fix SSL error issues (do not use ssl_bio_and_sess_init())
- BUG/MEDIUM: mux-quic: reinsert all streams in by_id tree
- BUG/MAJOR: xprt-quic: do not queue qc timer if not set
- MINOR: mux-quic: release connection if no more bidir streams
- BUG/MAJOR: quic: remove qc from receiver cids tree on free
- BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial frames
- MINOR: qpack: do not encode invalid http status code
- MINOR: qpack: support non-indexed http status code encoding
- MINOR: qpack: fix memory leak on huffman decoding
- CLEANUP: mux-quic: remove unused code
- BUG/MINOR: quic: fix includes for compilation
- BUILD: connection: avoid a build warning on FreeBSD with SO_USER_COOKIE
- BUILD: init: avoid a build warning on FreeBSD with USE_PROCCTL
- REORG: time: move time-keeping code and variables to clock.c
- REORG: clock: move the updates of cpu/mono time to clock.c
- MINOR: activity: get the run_time from the clock updates
- CLEANUP: clock: stop exporting before_poll and after_poll
- REORG: clock: move the clock_id initialization to clock.c
- REORG: clock/wdt: move wdt timer initialization to clock.c
- MINOR: clock: move the clock_ids to clock.c
- MINOR: wdt: move wd_timer to wdt.c
- CLEANUP: wdt: do not remap SI_TKILL to SI_LWP, test the values directly
- REORG: thread/sched: move the task_per_thread stuff to thread_ctx
- REORG: thread/clock: move the clock parts of thread_info to thread_ctx
- REORG: thread/sched: move the thread_info flags to the thread_ctx
- REORG: thread/sched: move the last dynamic thread_info to thread_ctx
- MINOR: thread: make "ti" a const pointer and clean up thread_info a bit
- MINOR: threads: introduce a minimalistic notion of thread-group
- MINOR: global: add a new "thread-groups" directive
- MINOR: global: add a new "thread-group" directive
- MINOR: threads: make tg point to the current thread's group
- MEDIUM: threads: automatically assign threads to groups
- MINOR: threads: set the group ID and its bit in the thread group
- MINOR: threads: set the tid, ltid and their bit in thread_cfg
- MEDIUM: threads: replace ha_set_tid() with ha_set_thread()
- MINOR: threads: add the current group ID in thread-local "tgid" variable
- MINOR: debug: report the group and thread ID in the thread dumps
- MEDIUM: listeners: support the definition of thread groups on bind lines
- MINOR: threads: add a new function to resolve config groups and masks
- MEDIUM: config: resolve relative threads on bind lines to absolute ones
- MEDIUM: stick-table: never learn the "conn_cur" value from peers
There have been a large number of issues reported with conn_cur
synchronization because the concept is wrong. In an active-passive
setup, pushing the local connections count from the active node to
the passive one will result in the passive node to have a higher
counter than the real number of connections. Due to this, after a
switchover, it will never be able to close enough connections to
go down to zero. The same commonly happens on reloads since the new
process preloads its values from the old process, and if no connection
happens for a key after the value is learned, it is impossible to reset
the previous ones. In active-active setups it's a bit different, as the
number of connections reflects the number on the peer that pushed last.
This patch solves this by marking the "conn_cur" local and preventing
it from being learned from peers. It is still pushed, however, so that
any monitoring system that collects values from the peers will still
see it.
The patch is tiny and trivially backportable. While a change of behavior
in stable branches is never welcome, it remains possible to fix issues
if reports become frequent.
This extends the "thread" statement of bind lines to support an optional
thread group number. When unspecified (0) it's an absolute thread range,
and when specified it's one relative to the thread group. Masks are still
used so no more than 64 threads may be specified at once, and a single
group is possible. The directive is not used for now.
This registers a mapping of threads to groups by enumerating for each thread
what group it belongs to, and marking the group as assigned. It takes care of
checking for redefinitions, overlaps, and holes. It supports both individual
numbers and ranges. The thread group is referenced from the thread config.
Add a more precise description on how backslash escaping is different
than the top-level parser, and give examples of how to handle single
quotes inside arguments.
The ssl_bc_hsk_err sample fetch will need to raise more errors than only
handshake related ones hence its renaming to a more generic ssl_bc_err.
This patch is required because some handshake failures that should have
been caught by this fetch (verify error on the server side for instance)
were missed. This is caused by a change in TLS1.3 in which the
'Finished' state on the client is reached before its certificate is sent
(and verified) on the server side (see the "Protocol Overview" part of
RFC 8446).
This means that the SSL_do_handshake call is finished long before the
server can verify and potentially reject the client certificate.
The ssl_bc_hsk_err will then need to be expanded to catch other types of
errors.
This change is also applied to the frontend fetches (ssl_fc_hsk_err
becomes ssl_fc_err) and to their string counterparts.
In case of a connection error happening after the SSL handshake is
completed, the error code stored in the connection structure would not
always be set, hence having some connection failures being described as
successful in the fc_conn_err or bc_conn_err sample fetches.
The most common case in which it could happen is when the SSL server
rejects the client's certificate. The SSL_do_handshake call on the
client side would be sucessful because the client effectively sent its
client hello and certificate information to the server, but the next
call to SSL_read on the client side would raise an SSL_ERROR_SSL code
(through the SSL_get_error function) which is decribed in OpenSSL
documentation as a non-recoverable and fatal SSL error.
This patch ensures that in such a case, the connection's error code is
set to a special CO_ERR_SSL_FATAL value.
Checking in code the right keyword is "enabled" and not "enable".
In addition the comment was also completed:
This could appear useless because the "defaults" sections not
yet apply on "peers" sections, but it could be the case in the future.
This statement can currently cancel a previous "disabled" keyword in
the same section.
This patch should be backported in all supported branches (keyword
is present since 1.5)
Released version 2.5-dev8 with the following main changes :
- BUILD: compiler: fixed a missing test on defined(__GNUC__)
- BUILD: halog: fix a -Wundef warning on non-glibc systems
- BUILD: threads: fix -Wundef for _POSIX_PRIORITY_SCHEDULING on libmusl
- BUG/MINOR: compat: make sure __WORDSIZE is always defined
- BUILD: sample: fix format warning on 32-bit archs in sample_conv_be2dec_check()
- CLEANUP: pools: factor all malloc_trim() calls into trim_all_pools()
- MINOR: pools: automatically disable malloc_trim() with external allocators
- MINOR: pools: report it when malloc_trim() is enabled
- DOC: Add .mailmap
- CLEANUP: tree-wide: fix prototypes for functions taking no arguments.
- CLEANUP: Remove prototype for non-existent thread_get_default_count()
- CLEANUP: acl: Remove unused variable when releasing an acl expression
- BUG/MAJOR: mux-h1: Don't eval input data if an error was reported
- DOC: update Tim's address in .mailmap
- MINOR: pools: use mallinfo2() when available instead of mallinfo()
- BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
- DOC: management: certificate files must be sanitized before injection
- BUG/MINOR: connection: prevent null deref on mux cleanup task allocation
- BUILD: ist: prevent gcc11 maybe-uninitialized warning on istalloc
- BUG/MINOR: cli/payload: do not search for args inside payload
- BUILD: sockpair: do not set unused flag
- BUILD: proto_uxst: do not set unused flag
- BUILD: fd: remove unused variable totlen in fd_write_frag_line()
- MINOR: applet: remove the thread mask from appctx_new()
- REORG: threads: move ha_get_pthread_id() to tinfo.h
- CLEANUP: Apply ist.cocci
- DEV: coccinelle: Add ist.cocci
- CLEANUP: Apply bug_on.cocci
- DEV: coccinelle: Add xalloc_size.cocci
- DEV: coccinelle: Add bug_on.cocci
- CLEANUP: Apply xalloc_size.cocci
- DEV: coccinelle: Add xalloc_cast.cocci
- BUG/MINOR: flt-trace: fix an infinite loop when random-parsing is set
- MINOR: httpclient: add the EOH when no headers where provided
- CLEANUP: Include check.h in flt_spoe.c
- CLEANUP: Remove unreachable `break` from parse_time_err()
- BUG/MINOR: server: allow 'enable health' only if check configured
- BUG/MINOR: server: alloc dynamic srv ssl ctx if proxy uses ssl chk rule
- MINOR: server: enable more keywords for ssl checks for dynamic servers
- MINOR: server: enable more check related keywords for dynamic servers
- REORG: server: move slowstart init outside of checks
- MINOR: server: enable slowstart for dynamic server
- MEDIUM: listener: deprecate "process" in favor of "thread" on bind lines
- BUG/MEDIUM: leastconn: fix rare possibility of divide by zero
- BUG/MINOR: quic: Possible NULL pointer dereferencing when dumping streams.
- MINOR: quic: Move transport parmaters to anynomous struct.
- MINOR: mux_quic: Add QUIC mux layer.
- MINOR: connection: Add callbacks definitions for QUIC.
- MINOR: quic: Attach QUIC mux connection objet to QUIC connection.
- MINOR: quic: Add a new definition to store STREAM frames.
- MINOR: h3: Add HTTP/3 definitions.
- MINOR: qpack: Add QPACK compression.
- MINOR: quic_sock: Finalize the QUIC connections.
- MINOR: quic: Disable the action of ->rcv_buf() xprt callback
- MINOR: quic: Add callbacks for (un)scribing to QUIC xprt.
- MINOR: quic: Variable-length integer encoding/decoding into/from buffer struct.
- BUG/MINOR: quic: Wrong ->accept() error handling
- MINOR: quic: Add a wrapper function to update transport parameters.
- MINOR: quic: Update the streams transport parameters.
- MINOR: quic: Avoid header collisions
- MINOR: quic: Replace max_packet_size by max_udp_payload size.
- MINOR: quic: Enable some quic, h3 and qpack modules compilation.
- MINOR: quic: Move an SSL func call from QUIC I/O handler to the xprt init.
- MINOR: quic: Initialize the session before starting the xprt.
- BUG/MINOR: quic: Do not check the acception of a new conn from I/O handler.
- MINOR: quic: QUIC conn initialization from I/O handler
- MINOR: quic: Remove header protection for conn with context
- MINOR: quic: Derive the initial secrets asap
- MINOR: quic: Remove header protection also for Initial packets
- BUG/MINOR: quic: Wrong memory free in quic_update_ack_ranges_list()
- MINOR: quic: quic_update_ack_ranges_list() code factorization
- MINOR: quic: Useless test in quic_update_ack_ranges_list()
- MINOR: quic: Remove a useless variable in quic_update_ack_ranges_list()
- BUG/MINOR: quic: Missing cases treatement when updating ACK ranges
- CLEAUNUP: quic: Usage of a useless variable in qc_treat_rx_pkts()
- BUG/MINOR: quic: Wrong RX packet reference counter usage
- MINOR: quic: Do not stop the packet parsing too early in qc_treat_rx_packets()
- MINOR: quic: Add a lock for RX packets
- MINOR: quic: Move the connection state
- MINOR: quic: Replace quic_conn_ctx struct by ssl_sock_ctx struct
- MINOR: quic: Replace the RX list of packet by a thread safety one.
- MINOR: quic: Replace the RX unprotected packet list by a thread safety one.
- MINOR: quic: Add useful traces for I/O dgram handler
- MINOR: quic: Do not wakeup the xprt task on ACK receipt
- MINOR: quic: Connection allocations rework
- MINOR: quic: Move conn_prepare() to ->accept_conn() callback
- MINOR: quic: Make qc_lstnr_pkt_rcv() be thread safe.
- MINOR: quic: Add a ring buffer implementation for QUIC
- MINOR: quic: Prefer x25519 as ECDH preferred parametes.
- MINOR: quic: Add the QUIC v1 initial salt.
- BUG/MINOR: quic: Too much reduced computed space to build handshake packets
- MINOR: net_helper: add functions for pointers
- MINOR: quic: Add ring buffer definition (struct qring) for QUIC
- MINOR: proto_quic: Allocate TX ring buffers for listeners
- MINOR: quic: Initialize pointers to TX ring buffer list
- MINOR: quic: Make use of TX ring buffers to send QUIC packets
- MINOR: quic_tls: Make use of the QUIC V1 salt.
- MINOR: quic: Remove old TX buffer implementation
- MINOR: Add function for TX packets reference counting
- MINOR: quic: Add TX packets at the very last time to their tree.
- MINOR: quic: Unitialized mux context upon Client Hello message receipt.
- MINOR: quic: Missing encryption level rx.crypto member initialization and lock.
- MINOR: quic: Rename ->rx.rwlock of quic_enc_level struct to ->rx.pkts_rwlock
- MINOR: quic: Make qc_treat_rx_pkts() be thread safe.
- MINOR: quic: Make ->tx.frms quic_pktns struct member be thread safe
- MINOR: quic: Replace quic_tx_frm struct by quic_frame struct
- MINOR: quic: Add a mask for TX frame builders and their authorized packet types
- MINOR: quic: Add a useful function to compute any frame length.
- MINOR: quic: Add the QUIC connection state to traces
- MINOR: quic: Store post handshake frame in ->pktns.tx.frms MT_LIST
- MINOR: quic: Add the packet type to quic_tx_packet struct
- MINOR: quic: Modify qc_do_build_hdshk_pkt() to accept any packet type
- MINOR: quic: Atomically handle packet number space ->largest_acked_pn variable
- MINOR: quic: Modify qc_build_cfrms() to support any frame
- MINOR: quic: quic_conn_io_cb() task rework
- MINOR: quic: Make qc_build_hdshk_pkt() atomically consume a packet number
- MINOR: quic: qc_do_build_hdshk_pkt() does not need to pass a copy of CRYPTO frame
- MINOR: quic: Remove Application level related functions
- MINOR: quic: Rename functions which do not build only Handshake packets
- MINOR: quic: Make circular buffer internal buffers be variable-sized.
- MINOR: quic: Add a pool for TX ring buffer internal buffer
- MINOR: quic: Make use of the last cbuf API when initializing TX ring buffers
- MINOR: quic: Missing acks encoded size updates.
- MINOR: quic: Evaluate the packet lengths in advance
- MINOR: quic: Update the TLS extension for QUIC transport parameters
- MINOR: quic: Fix handshake state debug strings
- MINOR: quic: Atomically get/set the connection state
- MINOR: quic: Missing QUIC encryption level for qc_build_pkt()
- MINOR: quic: Coalesce Application level packets with Handshake packets.
- MINOR: quic: Wrong flags handling for acks
- MINOR: quic: Missing case when discarding HANDSHAKE secrets
- MINOR: quic: Post handshake packet building improvements
- MINOR: quic: Prepare Application level packet asap.
- MINOR: h3: Send h3 settings asap
- MINOR: quic: Wrong STREAM frame length computing
- MINOR: quic: Wrong short packet minimum length
- MINOR: quic: Prepare STREAM frames to fill QUIC packets
- MINOR: h3: change default settings
- MINOR: quic-enc: fix varint encoding
- MINOR: qpack: fix wrong comment
- MINOR: qpack: generate headers list on decoder
- MINOR: h3: parse headers to htx
- MINOR: h3: allocate stream on headers
- MEDIUM: mux-quic: implement ring buffer on stream tx
- MINOR: mux-quic: send SETTINGS on uni stream
- MINOR: h3: define snd_buf callback and divert mux ops
- MINOR: mux-quic: define FIN stream flag
- MINOR: qpack: create qpack-enc module
- MINOR: qpack: encode headers functions
- MINOR: h3: encode htx headers to QPACK
- MINOR: h3: send htx data
- MINOR: h3/mux: detect fin on last h3 frame of the stream
- MINOR: quic: Shorten some handshakes
- MINOR: quic: Make QUIC-TLS support at least two initial salts
- MINOR: quic: Attach the QUIC connection to a thread.
- MINOR: quic: Missing active_connection_id_limit default value
- MINOR: quic_sock: Do not flag QUIC connections as being set
- MINOR: buf: Add b_force_xfer() function
- MINOR: quic: Make use of buffer structs to handle STREAM frames
- MINOR: mux_quic: move qc_process() code to qc_send()
- MINOR: quic: Add a typedef for unsigned long long
- MINOR: quic: Confusion between TX/RX for the frame builders
- MINOR: quic: Wrong packet flags settings during frame building
- MINOR: quic: Constantness fixes for frame builders/parsers.
- MINOR: quic_tls: Client/serveur state reordering
- MINOR: quic: Wrong packet loss detection due to wrong pktns order
- MINOR: quic: Wrong packet number space selection in quic_loss_pktns()
- MINOR: quic: Initial packet number spaced not discarded
- MINOR: quic: Add useful trace about pktns discarding
- MINOR: mux_quic: Export the mux related flags
- MINOR: quic: Implement quic_conn_subscribe()
- MINOR: quic: Wake up the mux upon ACK receipt
- MINOR: quic: Stream FIN bit fix in qcs_push_frame()
- MINOR: quic: Implement qc_process_mux()
- MINOR: quic: Wake up the xprt from mux
- CLEANUP: quic: Remove useless inline functions
- MINOR: quic: RX packets memory leak
- MINOR: quic: Possible endless loop in qc_treat_rx_pkts()
- MINOR: quic: Crash upon too big packets receipt
- MINOR: quic: define close handler
- MEDIUM: quic: implement mux release/conn free
- MINOR: quic: fix qcc subs initialization
- BUG/MINOR: h1-htx: Fix a typo when request parser is reset
- BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
- BUG/MEDIUM: stream-int: Notify stream that the mux wants more room to xfer data
- BUG/MEDIUM: stream: Stop waiting for more data if SI is blocked on RXBLK_ROOM
- MINOR: stream-int: Set CO_RFL transient/persistent flags apart in si_cs_rcv()
- MINOR: htx: Add an HTX flag to know when a message is fragmented
- MINOR: htx: Add a function to know if the free space wraps
- BUG/MEDIUM: stream-int: Defrag HTX message in si_cs_recv() if necessary
- MINOR: stream-int: Notify mux when the buffer is not stuck when calling rcv_buf
- BUG/MINOR: http-ana: increment internal_errors counter on response error
- MINOR: stats: Enable dark mode on stat web page
- CLEANUP: stats: Fix some alignment mistakes
- MINOR: httpclient: httpclient_data() returns the available data
- MINOR: httpclient: httpclient_ended() returns 1 if the client ended
- MINOR: httpclient/lua: httpclient:get() API in lua
- MINOR: httpclient/lua: implement the headers in the response object
- BUG/MINOR: httpclient/lua: return an error on argument check
- CLEANUP: slz: Mark `reset_refs` as static
The "process" directive on "bind" lines becomes quite confusing considering
that the only allowed value is 1 for the process, and that threads are
optional and come after the mandatory "1/".
Let's introduce a new "thread" directive to directly configure thread
numbers, and mark "process" as deprecated. Now "process" will emit a
warning and will suggest how to be replaced with "thread" instead.
The doc was updated accordingly (mostly a copy-paste of the previous
description which was already up to date).
This is marked as MEDIUM as it will impact users having "zero-warning"
and "process" specified.
Enable the 'slowstart' keyword for dynamic servers. The slowstart task
is allocated in 'add server' handler if slowstart is used.
As the server is created in disabled state, there is no need to start
the task. The slowstart task will be automatically started on the first
'enable server' invocation.
Allow to use the check related keywords defined in server.c. These
keywords can be enabled now that checks have been implemented for
dynamic servers.
Here is the list of the new keywords supported :
- error-limit
- observe
- on-error
- on-marked-down
- on-marked-up
Allow to configure ssl support for dynamic server checks independently
of the ssl server configuration. This is done via the keyword
"check-ssl". Also enable to configure the sni/alpn used for the check
via "check-sni/alpn".
A lot of people encounter problems when trying to inject a certificate
file which contains extra informations or empty lines.
This patch adds a paragraph and a sanitizing example.
Must be backported as far as 2.1.
Released version 2.5-dev7 with the following main changes :
- BUG/MINOR: config: reject configs using HTTP with bufsize >= 256 MB
- CLEANUP: htx: remove comments about "must be < 256 MB"
- BUG/MAJOR: htx: fix missing header name length check in htx_add_header/trailer
- Revert "BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive"
- MINOR: proxy: add a global "grace" directive to postpone soft-stop
- MINOR: vars: rename vars_init() to vars_init_head()
- CLEANUP: vars: rename sample_clear_stream() to var_unset()
- REORG: vars: remerge sample_store{,_stream}() into var_set()
- MEDIUM: vars: make the ifexist variant of set-var only apply to the proc scope
- MINOR: vars: add a VF_CREATEONLY flag for creation
- MINOR: vars: support storing empty sample data with a variable
- MINOR: vars: store flags into variables and add VF_PERMANENT
- MEDIUM: vars: make var_clear() only reset VF_PERMANENT variables
- MEDIUM: vars: pre-create parsed SCOPE_PROC variables as permanent ones
- MINOR: vars: preset a random seed to hash variables names
- MEDIUM: vars: replace the global name index with a hash
- CLEANUP: vars: remove the now unused var_names array
- MINOR: vars: centralize the lock/unlock into static inlines
- OPTIM: vars: only takes the variables lock on shared entries
- OPTIM: vars: remove internal bookkeeping for vars_global_size
- OPTIM: vars: do not keep variables usage stats if no limit is set
- BUILD: fix dragonfly build again on __read_mostly
- CI: Github Actions: temporarily disable Opentracing
- BUG/MEDIUM: mux-h1: Remove "Upgrade:" header for requests with payload
- MINOR: htx: Skip headers with no value when adding a header list to a message
- CLEANUP: mux-h1: Remove condition rejecting upgrade requests with payload
- BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is reached
- BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
- BUG/MINOR: filters: Set right FLT_END analyser depending on channel
- CLEANUP: Add haproxy/xxhash.h to avoid modifying import/xxhash.h
- CLEANUP: ebmbtree: Replace always-taken elseif by else
- CLEANUP: Move XXH3 macro from haproxy/compat.h to haproxy/xxhash.h
- BUILD: opentracing: exclude the use of haproxy variables for the OpenTracing context
- BUG/MINOR: opentracing: enable the use of http headers without a set value
- CLEANUP: opentracing: use the haproxy function to generate uuid
- MINOR: opentracing: change the scope of the variable 'ot.uuid' from 'sess' to 'txn'
- CI: Github Actions: re-enable Opentracing
- CLEANUP: opentracing: simplify the condition on the empty header
- BUG/MEDIUM lua: Add missing call to RESET_SAFE_LJMP in hlua_filter_new()
When setting variables, there are currently two variants, one which will
always create the variable, and another one, "ifexist", which will only
create or update a variable if a similarly named variable in any scope
already existed before.
The goal was to limit the risk of injecting random names in the proc
scope, but it was achieved by making use of the somewhat limited name
indexing model, which explains the scope-agnostic restriction.
With this change, we're moving the check downwards in the chain, at the
variable level, and only variables under the scope "proc" will be subject
to the restriction. A new set of VF_* flags was added to adjust how
variables are set, and VF_UPDATEONLY is used to mention this restriction.
In this exact state of affairs, this is not completely exact, as if a
similar name was not known in any scope, the variable will continue to
be rejected like before, but this will change soon.
In ticket #1348 some users expressed some concerns regarding the removal
of the "grace" directive from the proxies. Their use case very closely
mimmicks the original intent of the grace keyword, which is, let haproxy
accept traffic for some time when stopping, while indicating an external
LB that it's stopping.
This is implemented here by starting a task whose expiration triggers
the soft-stop for real. The global "stopping" variable is immediately
set however. For example, this below will be sufficient to instantly
notify an external check on port 9999 that the service is going down,
while other services remain active for 10s:
global
grace 10s
frontend ext-check
bind :9999
monitor-uri /ext-check
monitor fail if { stopping }
Released version 2.5-dev6 with the following main changes :
- BUG/MINOR threads: Use get_(local|gm)time instead of (local|gm)time
- BUG/MINOR: tools: Fix loop condition in dump_text()
- BUILD: ssl: next round of build warnings on LIBRESSL_VERSION_NUMBER
- BUILD: ssl: fix two remaining occurrences of #if USE_OPENSSL
- BUILD: tools: properly guard __GLIBC__ with defined()
- BUILD: globally enable -Wundef
- MINOR: log: Remove log-error-via-logformat option
- MINOR: log: Add new "error-log-format" option
- BUG/MAJOR: queue: better protect a pendconn being picked from the proxy
- CLEANUP: Add missing include guard to signal.h
- MINOR: ssl: Add new ssl_bc_hsk_err sample fetch
- MINOR: connection: Add a connection error code sample fetch for backend side
- REGTESTS: ssl: Add tests for bc_conn_err and ssl_bc_hsk_err sample fetches
- MINOR: http-rules: add a new "ignore-empty" option to redirects.
- CI: Github Actions: temporarily disable BoringSSL builds
- BUG/MINOR: vars: fix set-var/unset-var exclusivity in the keyword parser
- BUG/MINOR: vars: improve accuracy of the rules used to check expression validity
- MINOR: sample: add missing ARGC_ entries
- BUG/MINOR: vars: properly set the argument parsing context in the expression
- DOC: configuration: remove wrong tcp-request examples in tcp-response
- MEDIUM: vars: add a new "set-var-fmt" action
- BUG/MEDIUM: vars: run over the correct list in release_store_rules()
- BUG/MINOR: vars: truncate the variable name in error reports about scope.
- BUG/MINOR: vars: do not talk about global section in CLI errors for set-var
- CLEANUP: vars: name the temporary proxy "CFG" instead of "CLI" for global vars
- MINOR: log: make log-format expressions completely usable outside of req/resp
- MINOR: vars: add a "set-var-fmt" directive to the global section
- MEDIUM: vars: also support format strings in CLI's "set var" command
- CLEANUP: vars: factor out common code from vars_get_by_{desc,name}
- MINOR: vars: make vars_get_by_* support an optional default value
- MINOR: vars: make the vars() sample fetch function support a default value
- BUILD: ot: add argument for default value to vars_get_by_name()
It is quite common to see in configurations constructions like the
following one:
http-request set-var(txn.bodylen) 0
http-request set-var(txn.bodylen) req.hdr(content-length)
...
http-request set-header orig-len %[var(txn.bodylen)]
The set-var() rules are almost always duplicated when manipulating
integers or any other value that is mandatory along operations. This is
a problem because it makes the configurations complicated to maintain
and slower than needed. And it becomes even more complicated when several
conditions may set the same variable because the risk of forgetting to
initialize it or to accidentally reset it is high.
This patch extends the var() sample fetch function to take an optional
argument which contains a default value to be returned if the variable
was not set. This way it becomes much simpler to use the variable, just
set it where needed, and read it with a fall back to the default value:
http-request set-var(txn.bodylen) req.hdr(content-length)
...
http-request set-header orig-len %[var(txn.bodylen,0)]
The default value is always passed as a string, thus it will experience
a cast to the output type. It doesn't seem userful to complicate the
configuration to pass an explicit type at this point.
The vars.vtc regtest was updated accordingly.
Most often "set var" on the CLI is used to set a string, and using only
expressions is not always convenient, particularly when trying to
concatenate variables sur as host names and paths.
Now the "set var" command supports an optional keyword before the value
to indicate its type. "expr" takes an expression just like before this
patch, and "fmt" a format string, making it work like the "set-var-fmt"
actions.
The VTC was updated to include a test on the format string.
Just like the set-var-fmt action for tcp/http rules, the set-var-fmt
directive in global sections allows to pre-set process-wide variables
using a format string instead of a sample expression. This is often
more convenient when it is required to concatenate multiple fields,
or when emitting just one word.
The set-var() action is convenient because it preserves the input type
but it's a pain to deal with when trying to concatenate values. The
most recurring example is when it's needed to build a variable composed
of the source address and the source port. Usually it ends up like this:
tcp-request session set-var(sess.port) src_port
tcp-request session set-var(sess.addr) src,concat(":",sess.port)
This is even worse when trying to aggregate multiple fields from stick-table
data for example. Due to this a lot of users instead abuse headers from HTTP
rules:
http-request set-header(x-addr) %[src]:%[src_port]
But this requires some careful cleanups to make sure they won't leak, and
it's significantly more expensive to deal with. And generally speaking it's
not clean. Plus it must be performed for each and every request, which is
expensive for this common case of ip+port that doesn't change for the whole
session.
This patch addresses this limitation by implementing a new "set-var-fmt"
action which performs the same work as "set-var" but takes a format string
in argument instead of an expression. This way it becomes pretty simple to
just write:
tcp-request session set-var-fmt(sess.addr) %[src]:%[src_port]
It is usable in all rulesets that already support the "set-var" action.
It is not yet implemented for the global "set-var" directive (which already
takes a string) and the CLI's "set var" command, which would definitely
benefit from it but currently uses its own parser and engine, thus it
must be reworked.
The doc and regtests were updated.
There is a massive abuse of copy-paste in the doc that is visible in
the examples and arguments declaration. Let's at least remove irrelevant
examples for now.
Sometimes it is convenient to remap large sets of URIs to new ones (e.g.
after a site migration for example). This can be achieved using
"http-request redirect" combined with maps, but one difficulty there is
that non-matching entries will return an empty response. In order to
avoid this, duplicating the operation as an ACL condition ending in
"-m found" is possible but it becomes complex and error-prone while it's
known that an empty URL is not valid in a location header.
This patch addresses this by improving the redirect rules to be able to
simply ignore the rule and skip to the next one if the result of the
evaluation of the "location" expression is empty. However in order not
to break existing setups, it requires a new "ignore-empty" keyword.
There used to be an ACT_FLAG_FINAL on redirect rules that's used during
the parsing to emit a warning if followed by another rule, so here we
only set it if the option is not there. The http_apply_redirect_rule()
function now returns a 3rd value to mention that it did nothing and
that this was not an error, so that callers can just ignore the rule.
The regular "redirect" rules were not modified however since this does
not apply there.
The map_redirect VTC was completed with such a test and updated to 2.5
and an example was added into the documentation.
The bc_conn_err and bc_conn_err_str sample fetches give the status of
the connection on the backend side. The error codes and error messages
are the same than the ones that can be raised by the fc_conn_err fetch.
This new sample fetch along the ssl_bc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the backend's perspective).
This option can be used to define a specific log format that will be
used in case of error, timeout, connection failure on a frontend... It
will be used for any log line concerned by the log-separate-errors
option. It will also replace the format of specific error messages
decribed in section 8.2.6.
If no "error-log-format" is defined, the legacy error messages are still
emitted and the other error logs keep using the regular log-format.
This option will be replaced by a "error-log-format" that enables to use
a dedicated log-format for connection error messages instead of the
regular log-format (in which most of the fields would be invalid in such
a case).
The "log-error-via-logformat" mechanism will then be replaced by a test
on the presence of such an error log format or not. If a format is
defined, it is used for connection error messages, otherwise the legacy
error log format is used.
Released version 2.5-dev5 with the following main changes :
- MINOR: httpclient: initialize the proxy
- MINOR: httpclient: implement a simple HTTP Client API
- MINOR: httpclient/cli: implement a simple client over the CLI
- MINOR: httpclient/cli: change the User-Agent to "HAProxy"
- MEDIUM: ssl: Keep a reference to the client's certificate for use in logs
- BUG/MEDIUM: h2: match absolute-path not path-absolute for :path
- BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
- MINOR: server: check if srv is NULL in free_server()
- MINOR: proxy: check if p is NULL in free_proxy()
- BUG/MEDIUM: cfgparse: do not allocate IDs to automatic internal proxies
- BUG/MINOR: http_client: make sure to preset the proxy's default settings
- REGTESTS: http_upgrade: fix incorrect expectation on TCP->H1->H2
- REGTESTS: abortonclose: after retries, 503 is expected, not close
- REGTESTS: server: fix agent-check syntax and expectation
- BUG/MINOR: httpclient: fix uninitialized sl variable
- BUG/MINOR: httpclient/cli: change the appctx test in the callbacks
- BUG/MINOR: httpclient: check if hdr_num is not 0
- MINOR: httpclient: cleanup the include files
- MINOR: hlua: take the global Lua lock inside a global function
- MINOR: tools: add FreeBSD support to get_exec_path()
- BUG/MINOR: systemd: ExecStartPre must use -Ws
- MINOR: systemd: remove the ExecStartPre line in the unit file
- MINOR: ssl: add an openssl version string parser
- MINOR: cfgcond: implements openssl_version_atleast and openssl_version_before
- CLEANUP: ssl: remove useless check on p in openssl_version_parser()
- BUG/MINOR: stick-table: fix the sc-set-gpt* parser when using expressions
- BUG/MINOR: httpclient: remove deinit of the httpclient
- BUG/MEDIUM: base64: check output boundaries within base64{dec,urldec}
- MINOR: httpclient: set verify none on the https server
- MINOR: httpclient: add the server to the proxy
- BUG/MINOR: httpclient: fix Host header
- BUILD: httpclient: fix build without OpenSSL
- CI: github-actions: remove obsolete options
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: proc: setting the process to produce a core dump on FreeBSD.
- BUILD: adopt script/build-ssl.sh for OpenSSL-3.0.0beta2
- MINOR: server: return the next srv instance on free_server
- BUG/MINOR: stats: use refcount to protect dynamic server on dump
- MEDIUM: server: extend refcount for all servers
- MINOR: server: define non purgeable server flag
- MINOR: server: mark referenced servers as non purgeable
- MINOR: server: mark servers referenced by LUA script as non purgeable
- MEDIUM: server: allow to remove servers at runtime except non purgeable
- BUG/MINOR: base64: base64urldec() ignores padding in output size check
- REGTEST: add missing lua requirements on server removal test
- REGTEST: fix haproxy required version for server removal test
- BUG/MINOR: proxy: don't dump servers of internal proxies
- REGTESTS: Use `feature cmd` for 2.5+ tests
- REGTESTS: Remove REQUIRE_VERSION=1.5 from all tests
- BUG/MINOR: resolvers: mark servers with name-resolution as non purgeable
- MINOR: compiler: implement an ONLY_ONCE() macro
- BUG/MINOR: lua: use strlcpy2() not strncpy() to copy sample keywords
- MEDIUM: ssl: Capture more info from Client Hello
- MINOR: sample: Expose SSL captures using new fetchers
- MINOR: sample: Add be2dec converter
- MINOR: sample: Add be2hex converter
- MEDIUM: config: Deprecate tune.ssl.capture-cipherlist-size
- BUG/MINOR: time: fix idle time computation for long sleeps
- MINOR: time: add report_idle() to report process-wide idle time
- BUG/MINOR: ebtree: remove dependency on incorrect macro for bits per long
- BUILD: activity: use #ifdef not #if on USE_MEMORY_PROFILING
- BUILD/MINOR: defaults: eliminate warning on MAXHOSTNAMELEN with -Wundef
- BUILD/MINOR: ssl: avoid a build warning on LIBRESSL_VERSION with -Wundef
- IMPORT: slz: silence a build warning with -Wundef
- BUILD/MINOR: regex: avoid a build warning on USE_PCRE2 with -Wundef
To be able to provide JA3 compatible TLS Fingerprints we need to expose
all Client Hello captured data using fetchers. Patch provides new
and modifies existing fetchers to add ability to filter out GREASE values:
- ssl_fc_cipherlist_*
- ssl_fc_ecformats_bin
- ssl_fc_eclist_bin
- ssl_fc_extlist_bin
- ssl_fc_protocol_hello_id
When we set tune.ssl.capture-cipherlist-size to a non-zero value
we are able to capture cipherlist supported by the client. To be able to
provide JA3 compatible TLS fingerprinting we need to capture more
information from Client Hello message:
- SSL Version
- SSL Extensions
- Elliptic Curves
- Elliptic Curve Point Formats
This patch allows HAProxy to capture such information and store it for
later use.
Relax the condition on "delete server" CLI handler to be able to remove
all servers, even non dynamic, except if they are flagged as non
purgeable.
This change is necessary to extend the use cases for dynamic servers
with reload. It's expected that each dynamic server created via the CLI
is manually commited in the haproxy configuration by the user. Dynamic
servers will be present on reload only if they are present in the
configuration file. This means that non-dynamic servers must be allowed
to be removable at runtime.
The dynamic servers removal reg-test has been updated and renamed to
reflect its purpose. A new test is present to check that non-purgeable
servers cannot be removed.
Released version 2.5-dev4 with the following main changes :
- MINOR: log: rename 'dontloglegacyconnerr' to 'log-error-via-logformat'
- MINOR: doc: rename conn_status in `option httsplog`
- MINOR: proxy: disabled takes a stopping and a disabled state
- MINOR: stats: shows proxy in a stopped state
- BUG/MINOR: server: fix race on error path of 'add server' CLI if track
- CLEANUP: thread: fix fantaisist indentation of thread_harmless_till_end()
- MINOR: threads: make thread_release() not wait for other ones to complete
- MEDIUM: threads: add a stronger thread_isolate_full() call
- MEDIUM: servers: make the server deletion code run under full thread isolation
- BUG/MINOR: server: remove srv from px list on CLI 'add server' error
- MINOR: activity/fd: remove the dead_fd counter
- MAJOR: fd: get rid of the DWCAS when setting the running_mask
- CLEANUP: fd: remove the now unused fd_set_running()
- CLEANUP: fd: remove the now unneeded fd_mig_lock
- BUG/MINOR: server: update last_change on maint->ready transitions too
- MINOR: spoe: Add a pointer on the filter config in the spoe_agent structure
- BUG/MEDIUM: spoe: Create a SPOE applet if necessary when the last one is released
- BUG/MEDIUM: spoe: Fix policy to close applets when SPOE connections are queued
- MINOR: server: unmark deprecated on enable health/agent cli
- MEDIUM: task: implement tasklet kill
- MINOR: server: initialize fields for dynamic server check
- MINOR: check: allocate default check ruleset for every backends
- MINOR: check: export check init functions
- MINOR: check: do not increment global maxsock at runtime
- MINOR: server: implement a refcount for dynamic servers
- MEDIUM: check: implement check deletion for dynamic servers
- MINOR: check: enable safe keywords for dynamic servers
- MEDIUM: server: implement check for dynamic servers
- MEDIUM: server: implement agent check for dynamic servers
- REGTESTS: server: add dynamic check server test
- MINOR: doc: specify ulimit-n usage for dynamic servers
- REGTESTS: server: fix dynamic server with checks test
- CI: travis-ci: temporarily disable arm64 builds
- BUG/MINOR: check: test if server is not null in purge
- MINOR: global: define MODE_STOPPING
- BUG/MINOR: server: do not use refcount in free_server in stopping mode
- ADMIN: dyncookie: implement a simple dynamic cookie calculator
- BUG/MINOR: check: do not reset check flags on purge
- BUG/MINOR: check: fix leak on add dynamic server with agent-check error
- BUG/MEDIUM: check: fix leak on agent-check purge
- BUG/MEDIUM: server: support both check/agent-check on a dynamic instance
- BUG/MINOR: buffer: fix buffer_dump() formatting
- MINOR: channel: remove an htx block from a channel
- BUG/MINOR: tcpcheck: Properly detect pending HTTP data in output buffer
- BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
- MINOR: lua: Add a flag on lua context to know the yield capability at run time
- BUG/MINOR: lua: Yield in channel functions only if lua context can yield
- BUG/MINOR: lua: Don't yield in channel.append() and channel.set()
- MINOR: filters/lua: Release filters before the lua context
- MINOR: lua: Add a function to get a reference on a table in the stack
- MEDIUM: lua: Process buffer data using an offset and a length
- MEDIUM: lua: Improve/revisit the lua api to manipulate channels
- DOC: Improve the lua documentation
- MEDIUM: filters/lua: Add support for dummy filters written in lua
- MINOR: lua: Add a function to get a filter attached to a channel class
- MINOR: lua: Add flags on the lua TXN to know the execution context
- MEDIUM: filters/lua: Be prepared to filter TCP payloads
- MEDIUM: filters/lua: Support declaration of some filter callback functions in lua
- MEDIUM: filters/lua: Add HTTPMessage class to help HTTP filtering
- MINOR: filters/lua: Add request and response HTTP messages in the lua TXN
- MINOR: filters/lua: Support the HTTP filtering from filters written in lua
- DOC: config: Fix 'http-response send-spoe-group' documentation
- BUG/MINOR: lua: Properly check negative offset in Channel/HttpMessage functions
- BUG/MINOR: lua: Properly catch alloc errors when parsing lua filter directives
- BUG/MEDIUM: cfgcheck: verify existing log-forward listeners during config check
- MINOR: cli: delare the CLI frontend as an internal proxy
- MINOR: proxy: disable warnings for internal proxies
- BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag is set
- BUG/MINOR: lua/filters: Return right code when txn:done() is called
- DOC: lua-api: Add documentation about lua filters
- CI: Remove obsolete USE_SLZ=1 CI job
- CLEANUP: assorted typo fixes in the code and comments
- CI: github actions: relax OpenSSL-3.0.0 version comparision
- BUILD: tools: get the absolute path of the current binary on NetBSD.
- DOC: Minor typo fix - 'question mark' -> 'exclamation mark'
- DOC/MINOR: fix typo in management document
- MINOR: http: add a new function http_validate_scheme() to validate a scheme
- BUG/MAJOR: h2: verify early that non-http/https schemes match the valid syntax
- BUG/MAJOR: h2: verify that :path starts with a '/' before concatenating it
- BUG/MAJOR: h2: enforce stricter syntax checks on the :method pseudo-header
- BUG/MEDIUM: h2: give :authority precedence over Host
- REGTESTS: add a test to prevent h2 desync attacks
Lua filter support is highly experimental. The documentation was added to
allow first lua filter implementations. The API is not stabilized and must
be improved to be fully usable. This docuementation is quite light for
now. But more will be added.
First of all, following functions are now considered deprecated:
* Channel:dup()
* Channel:get()
* Channel:getline()
* Channel:get_in_len()
* Cahnnel:get_out_len()
It is just informative, there is no warning and functions may still be
used. Howver it is recommended to use new functions. New functions are more
flexible and use a better naming pattern. In addition, the same names will
be used in the http_msg class to manipulate http messages from lua filters.
The new API is:
* Channel:data()
* Channel:line()
* Channel:append()
* Channel:prepend()
* Channel:insert()
* Channel:remove()
* Channel:set()
* Channel:input()
* Channel:output()
* Channel:send()
* Channel:forward()
* Channel:is_resp()
* Channel:is_full()
* Channel:may_recv()
The lua documentation was updated accordingly.
This commit is the counterpart for agent check of
"MEDIUM: server: implement check for dynamic servers".
The "agent-check" keyword is enabled for dynamic servers. The agent
check must manually be activated via "enable agent" CLI. This can
enable the dynamic server if the agent response is "ready" without an
explicit "enable server" CLI.
Implement check support for dynamic servers. The "check" keyword is now
enabled for dynamic servers. If used, the server check is initialized
and the check task started in the "add server" CLI handler. The check is
explicitely disabled and must be manually activated via "enable health"
CLI handler.
The dynamic server refcount is incremented if a check is configured. On
"delete server" handler, the check is purged, which decrements the
refcount.
Right now we're using a DWCAS to atomically set the running_mask while
being constrained by the thread_mask. This DWCAS is annoying because we
may seriously need it later when adding support for thread groups, for
checking that the running_mask applies to the correct group.
It turns out that the DWCAS is not strictly necessary because we never
need it to set the thread_mask based on the running_mask, only the other
way around. And in fact, the running_mask is always cleared alone, and
the thread_mask is changed alone as well. The running_mask is only
relevant to indicate a takeover when the thread_mask matches it. Any
bit set in running and not present in thread_mask indicates a transition
in progress.
As such, it is possible to re-arrange this by using a regular CAS around a
consistency check between running_mask and thread_mask in fd_update_events
and by making a CAS on running_mask then an atomic store on the thread_mask
in fd_takeover(). The only other case is fd_delete() but that one already
sets the running_mask before clearing the thread_mask, which is compatible
with the consistency check above.
This change has happily survived 10 billion takeovers on a 16-thread
machine at 800k requests/s.
The fd-migration doc was updated to reflect this change.
Rename the 'dontloglegacyconnerr' option to 'log-error-via-logformat'
which is much more self-explanatory and readable.
Note: only legacy keywords don't use hyphens, it is recommended to
separate words with them in new keywords.
Released version 2.5-dev3 with the following main changes :
- BUG/MINOR: arg: free all args on make_arg_list()'s error path
- BUG/MINOR: cfgcond: revisit the condition freeing mechanism to avoid a leak
- MEDIUM: proxy: remove long-broken 'option http_proxy'
- CLEANUP: http_ana: Remove now unused label from http_process_request()
- MINOR: deinit: always deinit the init_mutex on failed initialization
- BUG/MEDIUM: cfgcond: limit recursion level in the condition expression parser
- BUG/MEDIUM: mworker: do not register an exit handler if exit is expected
- BUG/MINOR: mworker: do not export HAPROXY_MWORKER_REEXEC across programs
- BUILD/MINOR: memprof fix macOs build.
- BUG/MEDIUM: ssl_sample: fix segfault for srv samples on invalid request
- BUG/MINOR: stats: Add missing agent stats on servers
- BUG/MINOR: check: fix the condition to validate a port-less server
- BUILD: threads: fix pthread_mutex_unlock when !USE_THREAD
- BUG/MINOR: resolvers: Use a null-terminated string to lookup in servers tree
- MINOR: ssl: use __objt_* variant when retrieving counters
- BUG/MINOR: systemd: must check the configuration using -Ws
- BUG/MINOR: mux-h1: Obey dontlognull option for empty requests
- BUG/MINOR: mux-h2: Obey dontlognull option during the preface
- BUG/MINOR: mux-h1: Be sure to swap H1C to splice mode when rcv_pipe() is called
- BUG/MEDIUM: mux-h2: Handle remaining read0 cases on partial frames
- MINOR: proxy: rename PR_CAP_LUA to PR_CAP_INT
- MINOR: mworker: the mworker CLI proxy is internal
- MINOR: stats: don't output internal proxies (PR_CAP_INT)
- CLEANUP: mworker: use the proxy helper functions in mworker_cli_proxy_create()
- CLEANUP: mworker: PR_CAP already initialized with alloc_new_proxy()
- BUG/MINOR: connection: Add missing error labels to conn_err_code_str
- MINOR: connection: Add a connection error code sample fetch
- MINOR: ssl: Enable error fetches in case of handshake error
- MINOR: ssl: Add new ssl_fc_hsk_err sample fetch
- MINOR: ssl: Define a default https log format
- MEDIUM: connection: Add option to disable legacy error log
- REGTESTS: ssl: Add tests for the connection and SSL error fetches
- REGTESTS: ssl: ssl_errors.vtc does not work with old openssl version
- BUG/MEDIUM: connection: close a rare race between idle conn close and takeover
- BUG/MEDIUM: pollers: clear the sleeping bit after waking up, not before
- BUG/MINOR: select: fix excess number of dead/skip reported
- BUG/MINOR: poll: fix abnormally high skip_fd counter
- BUG/MINOR: pollers: always program an update for migrated FDs
- BUG/MINOR: fd: protect fd state harder against a concurrent takeover
- DOC: internals: document the FD takeover process
- MINOR: fd: update flags only once in fd_update_events()
- MINOR: poll/epoll: move detection of RDHUP support earlier
- REORG: fd: uninline fd_update_events()
- MEDIUM: fd: rely more on fd_update_events() to detect changes
- BUG/MINOR: freq_ctr: use stricter barriers between updates and readings
- MEDIUM: atomic: simplify the atomic load/store/exchange operations
- MEDIUM: atomic: relax the load/store barriers on x86_64
- BUILD: opentracing: fixed build when using pkg-config utility
This explains the traps to avoid and the sequence that leads to
consistent use of an FD known by multiple threads at once. This
was co-authored with Olivier.
In case of connection failure, a dedicated error message is output,
following the format described in section "Error log format" of the
documentation. These messages cannot be configured through a log-format
option.
This patch adds a new option, "dontloglegacyconnerr", that disables
those error logs when set, and "replaces" them by a regular log line
that follows the configured log-format (thanks to a call to sess_log in
session_kill_embryonic).
The new fc_conn_err sample fetch allows to add the legacy error log
information into a regular log format.
This new option is unset by default so the logging logic will remain the
same until this new option is used.
This new sample fetch along the ssl_fc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the frontend's perspective). The errors happening during
the client's certificate verification will still be given by the
ssl_c_err and ssl_c_ca_err fetches. This new fetch will only hold errors
retrieved by the OpenSSL ERR_get_error function.
The fc_conn_err and fc_conn_err_str sample fetches give information
about the problem that made the connection fail. This information would
previously only have been given by the error log messages meaning that
thanks to these fetches, the error log can now be included in a custom
log format. The log strings were all found in the conn_err_code_str
function.
This option had always been broken in HTX, which means that the first
breakage appeared in 1.9, that it was broken by default in 2.0 and that
no workaround existed starting with 2.1. The way this option works is
praticularly unfit to the rest of the configuration and to the internal
architecture. It had some uses when it was introduced 14 years ago but
nowadays it's possible to do much better and more reliable using a
set of "http-request set-dst" and "http-request set-uri" rules, which
additionally are compatible with DNS resolution (via do-resolve) and
are not exclusive to normal load balancing. The "option-http_proxy"
example config file was updated to reflect this.
The option is still parsed so that an error message gives hints about
what to look for.
Released version 2.5-dev2 with the following main changes :
- BUILD/MEDIUM: tcp: set-mark support for OpenBSD
- DOC: config: use CREATE USER for mysql-check
- BUG/MINOR: stick-table: fix several printf sign errors dumping tables
- BUG/MINOR: peers: fix data_type bit computation more than 32 data_types
- MINOR: stick-table: make skttable_data_cast to use only std types
- MEDIUM: stick-table: handle arrays of standard types into stick-tables
- MEDIUM: peers: handle arrays of std types in peers protocol
- DOC: stick-table: add missing documentation about gpt0 stored type
- MEDIUM: stick-table: add the new array of gpt data_type
- MEDIUM: stick-table: make the use of 'gpt' excluding the use of 'gpt0'
- MEDIUM: stick-table: add the new arrays of gpc and gpc_rate
- MEDIUM: stick-table: make the use of 'gpc' excluding the use of 'gpc0/1''
- BUG/MEDIUM: sock: make sure to never miss early connection failures
- BUG/MINOR: cli: fix server name output in "show fd"
- Revert "MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules"
- MEDIUM: stats: include disabled proxies that hold active sessions to stats
- BUILD: stick-table: shut up invalid "uninitialized" warning in gcc 8.3
- MINOR: http: implement http_get_scheme
- MEDIUM: http: implement scheme-based normalization
- MEDIUM: h1-htx: apply scheme-based normalization on h1 requests
- MEDIUM: h2: apply scheme-based normalization on h2 requests
- REGTESTS: add http scheme-based normalization test
- BUILD: http_htx: fix ci compilation error with isdigit for Windows
- MINOR: http: implement http uri parser
- MINOR: http: use http uri parser for scheme
- MINOR: http: use http uri parser for authority
- REORG: http_ana: split conditions for monitor-uri in wait for request
- MINOR: http: use http uri parser for path
- BUG/MEDIUM: http_ana: fix crash for http_proxy mode during uri rewrite
- MINOR: mux_h2: define config to disable h2 websocket support
- CLEANUP: applet: remove unused thread_mask
- BUG/MINOR: ssl: Default-server configuration ignored by server
- BUILD: add detection of missing important CFLAGS
- BUILD: lua: silence a build warning with TCC
- MINOR: srv: extract tracking server config function
- MINOR: srv: do not allow to track a dynamic server
- MEDIUM: server: support track keyword for dynamic servers
- REGTESTS: test track support for dynamic servers
- MINOR: init: verify that there is a single word on "-cc"
- MINOR: init: make -cc support environment variables expansion
- MINOR: arg: add a free_args() function to free an args array
- CLEANUP: config: use free_args() to release args array in cfg_eval_condition()
- CLEANUP: hlua: use free_args() to release args arrays
- REORG: config: move the condition preprocessing code to its own file
- MINOR: cfgcond: start to split the condition parser to introduce terms
- MEDIUM: cfgcond: report invalid trailing chars after expressions
- MINOR: cfgcond: remerge all arguments into a single line
- MINOR: cfgcond: support negating conditional expressions
- MINOR: cfgcond: make the conditional term parser automatically allocate nodes
- MINOR: cfgcond: insert an expression between the condition and the term
- MINOR: cfgcond: support terms made of parenthesis around expressions
- REGTEST: make check_condition.vtc fail as soon as possible
- REGTESTS: add more complex check conditions to check_conditions.vtc
- BUG/MEDIUM: init: restore behavior of command-line "-m" for memory limitation
Now it's possible to form a term using parenthesis around an expression.
This will soon allow to build more complex expressions. For now they're
still pretty limited but parenthesis do work.
Now evaluating a condition will rely on an expression (or an empty string),
and this expression will support ORing a sub-expression with another
optional expression. The sub-expressions ANDs a term with another optional
sub-expression. With this alone precedence between && and || is respected,
and the following expression:
A && B && C || D || E && F || G
will naturally evaluate as:
(A && B && C) || D || (E && F) || G
Allow the usage of the 'track' keyword for dynamic servers. On server
deletion, the server is properly removed from the tracking chain to
prevents NULL pointer dereferencing.
Define a new global config statement named
"h2-workaround-bogus-websocket-clients".
This statement will disable the automatic announce of h2 websocket
support as specified in the RFC8441. This can be use to overcome clients
which fail to implement the relatively fresh RFC8441. Clients will in
his case automatically downgrade to http/1.1 for the websocket tunnel
if the haproxy configuration allows it.
This feature is relatively simple and can be backported up to 2.4, which
saw the introduction of h2 websocket support.
This reverts commit 19bbbe0562.
For now, set-src/set-src-port actions are directly performed on the client
connection. Using these actions at the stream level is really a problem with
HTTP connection (See #90) because all requests are affected by this change
and not only the current request. And it is worse with the H2, because
several requests can set their source address into the same connection at
the same time.
It is already an issue when these actions are called from "http-request"
rules. It is safer to wait a bit before adding the support to "tcp-request
content" rules. The solution is to be able to set src/dst address on the
stream and not on the connection when the action if performed from the L7
level..
Reverting the above commit means the issue #1303 is no longer fixed.
This patch must be backported in all branches containing the above commit
(as far as 2.0 for now).
This patch makes the use of 'gpc' excluding the use of the legacy
types 'gpc0' and 'gpc1" on the same table.
It also makes the use of 'gpc_rate' excluding the use of the legacy
types 'gpc0_rate' and 'gpc1_rate" on the same table.
The 'gpc0' and 'gpc1' related fetches and actions will apply
to the first two elements of the 'gpc' array if stored in table.
The 'gpc0_rate' and 'gpc1_rate' related fetches and actions will apply
to the first two elements of the 'gpc_rate' array if stored in table.
This patch adds the definition of two new array data_types:
'gpc': This is an array of 32bits General Purpose Counters.
'gpc_rate': This is an array on increment rates of General Purpose Counters.
Like for all arrays, they are limited to 100 elements.
This patch also adds actions and fetches to handle
elements of those arrays.
Note: As documented, those new actions and fetches won't
apply to the legacy 'gpc0', 'gpc1', 'gpc0_rate' nor 'gpc1_rate'.
This patch makes the use of 'gpt' excluding the use of the legacy
type 'gpt0' on the same table.
It also makes the 'gpt0' related fetches and actions applying
to the first element of the 'gpt' array if stored in table.
This patch adds the definition of a new array data_type
'gpt'. This is an array of 32bits General Purpose Tags.
Like for all arrays, it is limited to 100 elements.
This patch also adds actions and fetches to handle
elements of this array.
Note: As documented, those new actions and fetches won't
apply to the legacy 'gpt0' data type.
The store type 'gpt0' was present in code but was not documented.
The patch fix this and should be backported since 'gpt0' is supported.
[wt: ~1.6-dev4 hence all stable]
CREATE USER has been the standard way of creating users since
MySQL-5.0 (2005).
The current syntax of INSERT INTO mysql.user won't actually work
on MariaDB-10.4+.
Because haproxy doesn't use any resources the MySQL executable comment
syntax provides resource contraints to make it more palatable
to risk adverse users.
/*!50701 is a syntax recognised by MySQL and MariaDB 5.7.1+ when
resource contraints where added.
/*M!100201 is a MariaDB executable comment syntax recognised for MariaDB
for the 10.2.1 where the MAX_STATEMENT_TIME was added.
This patch may be backported as far as 2.0.
Released version 2.5-dev1 with the following main changes :
- CLEANUP: ssl: Move ssl_store related code to ssl_ckch.c
- MINOR: ssl: Allow duplicated entries in the cafile_tree
- MEDIUM: ssl: Chain ckch instances in ca-file entries
- MINOR: ssl: Add reference to default ckch instance in bind_conf
- MINOR: ssl: Add helper functions to create/delete cafile entries
- MEDIUM: ssl: Add a way to load a ca-file content from memory
- MINOR: ssl: Add helper function to add cafile entries
- MINOR: ssl: Ckch instance rebuild and cleanup factorization in CLI handler
- MEDIUM: ssl: Add "set+commit ssl ca-file" CLI commands
- REGTESTS: ssl: Add new ca-file update tests
- MINOR: ssl: Add "abort ssl ca-file" CLI command
- MINOR: ssl: Add a cafile_entry type field
- MINOR: ssl: Refactorize the "show certificate details" code
- MEDIUM: ssl: Add "show ssl ca-file" CLI command
- MEDIUM: ssl: Add "new ssl ca-file" CLI command
- MINOR: ssl: Add "del ssl ca-file" CLI command
- REGTESTS: ssl: Add "new/del ssl ca-file" tests
- DOC: ssl: Add documentation about CA file hot update commands
- DOC: internals: update the SSL architecture schema
- MINOR: ssl: Chain instances in ca-file entries
- MEDIUM: ssl: Add "set+commit ssl crl-file" CLI commands
- MEDIUM: ssl: Add "new+del crl-file" CLI commands
- MINOR: ssl: Add "abort ssl crl-file" CLI command
- MEDIUM: ssl: Add "show ssl crl-file" CLI command
- REGTESTS: ssl: Add "new/del ssl crl-file" tests
- REGTESTS: ssl: Add "set/commit ssl crl-file" test
- DOC: ssl: Add documentation about CRL file hot update commands
- BUILD/MINOR: ssl: Fix compilation with SSL enabled
- BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
- CI: introduce scripts/build-vtest.sh for installing VTest
- CLEANUP: ssl: Fix coverity issues found in CA file hot update code
- CI: github actions: add OpenTracing builds
- BUG/MEDIUM: ebtree: Invalid read when looking for dup entry
- BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
- BUILD/MINOR: opentracing: fixed build when using clang
- BUG/MEDIUM: filters: Exec pre/post analysers only one time per filter
- BUG/MINOR: http-comp: Preserve HTTP_MSGF_COMPRESSIONG flag on the response
- MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
- CLEANUP: pattern: remove export of non-existent function pattern_delete()
- MINOR: h1-htx: Update h1 parsing functions to return result as a size_t
- MEDIUM: h1-htx: Adapt H1 data parsing to copy wrapping data in one call
- MINOR: mux-h1/mux-fcgi: Don't needlessly loop on data parsing
- MINOR: h1-htx: Move HTTP chunks parsing into a dedicated function
- MEDIUM: h1-htx: Split function to parse a chunk and the loop on the buffer
- MEDIUM: h1-htx: Add a function to parse contiguous small chunks
- MINOR: h1-htx: Use a correlation table to speed-up small chunks parsing
- MINOR: buf: Add function to realign a buffer with a specific head position
- MINOR: muxes/h1-htx: Realign input buffer using b_slow_realign_ofs()
- CLEANUP: mux-h1: Rename functions parsing input buf and filling output buf
- Revert "MEDIUM: http-ana: Deal with L7 retries in HTTP analysers"
- BUG/MINOR: http-ana: Send the right error if max retries is reached on L7 retry
- BUG/MINOR: http-ana: Handle L7 retries on refused early data before K/A aborts
- MINOR: http-ana: Perform L7 retries because of status codes in response analyser
- MINOR: cfgparse: Fail when encountering extra arguments in macro
- DOC: intro: Fix typo in starter guide
- BUG/MINOR: server: Missing calloc return value check in srv_parse_source
- BUG/MINOR: peers: Missing calloc return value check in peers_register_table
- BUG/MINOR: ssl: Missing calloc return value check in ssl_init_single_engine
- BUG/MINOR: http: Missing calloc return value check in parse_http_req_capture
- BUG/MINOR: proxy: Missing calloc return value check in proxy_parse_declare
- BUG/MINOR: proxy: Missing calloc return value check in proxy_defproxy_cpy
- BUG/MINOR: http: Missing calloc return value check while parsing tcp-request/tcp-response
- BUG/MINOR: http: Missing calloc return value check while parsing tcp-request rule
- BUG/MINOR: compression: Missing calloc return value check in comp_append_type/algo
- BUG/MINOR: worker: Missing calloc return value check in mworker_env_to_proc_list
- BUG/MINOR: http: Missing calloc return value check while parsing redirect rule
- BUG/MINOR: http: Missing calloc return value check in make_arg_list
- BUG/MINOR: proxy: Missing calloc return value check in chash_init_server_tree
- CLEANUP: http-ana: Remove useless if statement about L7 retries
- BUG/MAJOR: stream-int: Release SI endpoint on server side ASAP on retry
- MINOR: backend: Don't release SI endpoint anymore in connect_server()
- BUG/MINOR: vars: Be sure to have a session to get checks variables
- DOC/MINOR: move uuid in the configuration to the right alphabetical order
- CLEANUP: mux-fcgi: Don't needlessly store result of data/trailers parsing
- BUILD: fix compilation for OpenSSL-3.0.0-alpha17
- MINOR: http-ana: Use -1 status for client aborts during queuing and connect
- REGTESTS: Fix http_abortonclose.vtc to support -1 status for some client aborts
- CLEANUP: backend: fix incorrect comments on locking conditions for lb functions
- CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests
- CI: github actions: add OpenSSL-3.0.0 builds
- CI: github actions: -Wno-deprecated-declarations with OpenSSL 3.0.0
- MINOR: errors: allow empty va_args for diag variadic macro
- REORG: errors: split errors reporting function from log.c
- CLEANUP: server: fix cosmetic of error message on sni parsing
- MEDIUM: errors: implement user messages buffer
- MINOR: log: do not discard stderr when starting is over
- MEDIUM: errors: implement parsing context type
- MINOR: errors: use user messages context in print_message
- MINOR: log: display exec path on first warning
- MINOR: errors: specify prefix "config" for parsing output
- MINOR: log: define server user message format
- REORG: server: use parsing ctx for server parsing
- REORG: config: use parsing ctx for server config check
- MINOR: server: use parsing ctx for server init addr
- MINOR: server: use ha_alert in server parsing functions
- DOC: use the req.ssl_sni in examples
- CLEANUP: cfgparse: Remove duplication of `MAX_LINE_ARGS + 1`
- CLEANUP: tools: Make errptr const in `parse_line()`
- MINOR: haproxy: Add `-cc` argument
- BUG: errors: remove printf positional args for user messages context
- CI: Make matrix.py executable and add shebang
- BUILD: make tune.ssl.keylog available again
- BUG/MINOR: ssl: OCSP stapling does not work if expire too far in the future
- Revert "BUG/MINOR: opentracing: initialization after establishing daemon mode"
- BUG/MEDIUM: opentracing: initialization before establishing daemon and/or chroot mode
- SCRIPTS: opentracing: enable parallel builds in build-ot.sh
- BUG/MEDIUM: compression: Fix loop skipping unused blocks to get the next block
- BUG/MEDIUM: compression: Properly get the next block to iterate on payload
- BUG/MEDIUM: compression: Add a flag to know the filter is still processing data
- MINOR: ssl: Keep the actual key length in the certificate_ocsp structure
- MINOR: ssl: Add new "show ssl ocsp-response" CLI command
- MINOR: ssl: Add the OCSP entry key when displaying the details of a certificate
- MINOR: ssl: Add the "show ssl cert foo.pem.ocsp" CLI command
- REGTESTS: ssl: Add "show ssl ocsp-response" test
- BUG/MINOR: server: explicitly set "none" init-addr for dynamic servers
- BUG/MINOR: pools: fix a possible memory leak in the lockless pool_flush()
- BUG/MINOR: pools: make DEBUG_UAF always write to the to-be-freed location
- MINOR: pools: do not maintain the lock during pool_flush()
- MINOR: pools: call malloc_trim() under thread isolation
- MEDIUM: pools: use a single pool_gc() function for locked and lockless
- BUG/MAJOR: pools: fix possible race with free() in the lockless variant
- CLEANUP: pools: remove now unused seq and pool_free_list
- MEDIUM: pools: remove the locked pools implementation
- BUILD: ssl: Fix compilation with BoringSSL
- BUG/MEDIUM: errors: include missing obj_type file
- REGTESTS: ssl: show_ssl_ocspresponce.vtc is broken with BoringSSL
- BUG/MAJOR: htx: Fix htx_defrag() when an HTX block is expanded
- BUG/MINOR: mux-fcgi: Expose SERVER_SOFTWARE parameter by default
- BUG/MINOR: h1-htx: Fix a signess bug with char data type when parsing chunk size
- CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
- BUG/MINOR: resolvers: answser item list was randomly purged or errors
- MEDIUM: resolvers: add a ref on server to the used A/AAAA answer item
- MEDIUM: resolvers: add a ref between servers and srv request or used SRV record
- BUG/MINOR: server-state: load SRV resolution only if params match the config
- MINOR: config: remove support for deprecated option "tune.chksize"
- MINOR: config: completely remove support for "no option http-use-htx"
- MINOR: log: remove the long-deprecated early log-format tags
- MINOR: http: remove the long deprecated "set-cookie()" sample fetch function
- MINOR: config: reject long-deprecated "option forceclose"
- MINOR: config: remove deprecated option "http-tunnel"
- MEDIUM: proxy: remove the deprecated "grace" keyword
- MAJOR: config: remove parsing of the global "nbproc" directive
- BUILD: init: remove initialization of multi-process thread mappings
- BUILD: log: remove unused fmt_directive()
- REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests
- REGTESTS: Remove REQUIRE_VERSION=1.7 from all tests
- CI: github actions: enable alpine/musl builds
- BUG/MAJOR: resolvers: segfault using server template without SRV RECORDs
- DOC: lua: Add a warning about buffers modification in HTTP
- MINOR: ssl: Use OpenSSL's ASN1_TIME convertor when available
- BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
- BUG/MEDIUM: server: extend thread-isolate over much of CLI 'add server'
- BUG/MEDIUM: server: clear dynamic srv on delete from proxy id/name trees
- BUG/MEDIUM: server: do not forget to generate the dynamic servers ids
- BUG/MINOR: server: do not keep an invalid dynamic server in px ids tree
- BUG/MEDIUM: server: do not auto insert a dynamic server in px addr_node
- BUG/MEDIUM: shctx: use at least thread-based locking on USE_PRIVATE_CACHE
- BUG/MINOR: ssl: use atomic ops to update global shctx stats
- BUG/MINOR: mworker: fix typo in chroot error message
- CLEANUP: global: remove unused definition of stopping_task[]
- MEDIUM: init: remove the loop over processes during init
- MINOR: mworker: remove the initialization loop over processes
- CLEANUP: global: remove the nbproc field from the global structure
- CLEANUP: global: remove pid_bit and all_proc_mask
- MEDIUM: global: remove dead code from nbproc/bind_proc removal
- MEDIUM: config: simplify cpu-map handling
- MEDIUM: cpu-set: make the proc a single bit field and not an array
- CLEANUP: global: remove unused definition of MAX_PROCS
- MEDIUM: global: remove the relative_pid from global and mworker
- DOC: update references to process numbers in cpu-map and bind-process
- MEDIUM: config: warn about "bind-process" deprecation
- CLEANUP: shctx: remove the different inter-process locking techniques
- BUG/MAJOR: queue: set SF_ASSIGNED when setting strm->target on dequeue
- MINOR: backend: only skip LB when there are actual connections
- BUG/MINOR: mux-h1: do not skip the error response on bad requests
- MINOR: connection: add helper conn_append_debug_info()
- MINOR: mux-h2/trace: report a few connection-level info during h2_init()
- CLEANUP: mux-h2/traces: better align user messages
- BUG/MINOR: stats: make "show stat typed desc" work again
- MINOR: mux-h2: obey http-ignore-probes during the preface
- BUG/MINOR: mux-h2/traces: bring back the lost "rcvd H2 REQ" trace
- BUG/MINOR: mux-h2/traces: bring back the lost "sent H2 REQ/RES" traces
- CLEANUP: assorted typo fixes in the code and comments
- CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'
- REGTESTS: Replace REQUIRE_VERSION=2.5 with 'haproxy -cc'
- REGTESTS: Replace REQUIRE_OPTIONS with 'haproxy -cc' for 2.5+ tests
- REGTESTS: Replace REQUIRE_BINARIES with 'command -v'
- REGTESTS: Remove support for REQUIRE_BINARIES
- CI: ssl: enable parallel builds for OpenSSL on Linux
- CI: ssl: do not needlessly build the OpenSSL docs
- CI: ssl: keep the old method for ancient OpenSSL versions
- CLEANUP: server: a separate function for initializing the per_thr field
- BUG/MINOR: server: Forbid to set fqdn on the CLI if SRV resolution is enabled
- BUG/MEDIUM: server/cli: Fix ABBA deadlock when fqdn is set from the CLI
- MINOR: resolvers: Clean server in a dedicated function when removing a SRV item
- MINOR: resolvers: Remove server from named_servers tree when removing a SRV item
- BUG/MEDIUM: resolvers: Add a task on servers to check SRV resolution status
- BUG/MINOR: backend: restore the SF_SRV_REUSED flag original purpose
- BUG/MINOR: backend: do not set sni on connection reuse
- BUG/MINOR: resolvers: Use resolver's lock in resolv_srvrq_expire_task()
- BUG/MINOR: server/cli: Fix locking in function processing "set server" command
- BUG/MINOR: cache: Correctly handle existing-but-empty 'accept-encoding' header
- MINOR: ssl: fix typo in usage for 'new ssl ca-file'
- MINOR: ssl: always initialize random generator
- MINOR: ssl: check allocation in ssl_sock_init_srv
- MINOR: ssl: check allocation in parse ciphers/ciphersuites/verifyhost
- MINOR: ssl: check allocation in parse npn/sni
- MINOR: server: disable CLI 'set server ssl' for dynamic servers
- MINOR: ssl: render file-access optional on server crt loading
- MINOR: ssl: split parse functions for alpn/check-alpn
- MINOR: ssl: support ca-file arg for dynamic servers
- MINOR: ssl: support crt arg for dynamic servers
- MINOR: ssl: support crl arg for dynamic servers
- MINOR: ssl: enable a series of ssl keywords for dynamic servers
- MINOR: ssl: support ssl keyword for dynamic servers
- REGTESTS: server: test ssl support for dynamic servers
- MINOR: queue: update the stream's pend_pos before queuing it
- CLEANUP: Prevent channel-t.h from being detected as C++ by GitHub
- BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check
- REGTESTS: fix maxconn update with agent-check
- MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn
- MINOR: queue: update proxy->served once out of the loop
- MEDIUM: queue: refine the locking in process_srv_queue()
- MINOR: lb/api: remove the locked argument from take_conn/drop_conn
- MINOR: queue: create a new structure type "queue"
- MINOR: proxy: replace the pendconns-related stuff with a struct queue
- MINOR: server: replace the pendconns-related stuff with a struct queue
- MEDIUM: queue: use a dedicated lock for the queues
- MEDIUM: queue: simplify again the process_srv_queue() API
- MINOR: queue: factor out the proxy/server queuing code
- MINOR: queue: use atomic-ops to update the queue's index
- MEDIUM: queue: determine in process_srv_queue() if the proxy is usable
- MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()
- MEDIUM: queue: unlock as soon as possible
- MINOR: queue: make pendconn_first() take the lock by itself
- CLEANUP: backend: remove impossible case of round-robin + consistent hash
- MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
- DOC: config: Add missing actions in "tcp-request session" documentation
- CLEANUP: dns: Remove a forgotten debug message
- DOC: Replace issue templates by issue forms
- Revert "MINOR: queue: make pendconn_first() take the lock by itself"
- Revert "MEDIUM: queue: unlock as soon as possible"
- Revert "MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()"
- Revert "MEDIUM: queue: determine in process_srv_queue() if the proxy is usable"
- Revert "MINOR: queue: use atomic-ops to update the queue's index"
- Revert "MINOR: queue: factor out the proxy/server queuing code"
- Revert "MEDIUM: queue: simplify again the process_srv_queue() API"
- Revert "MEDIUM: queue: use a dedicated lock for the queues"
- Revert "MEDIUM: queue: refine the locking in process_srv_queue()"
- Revert "MINOR: queue: update proxy->served once out of the loop"
- Revert "MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn"
- MEDIUM: queue: update px->served and lb's take_conn once per loop
- MEDIUM: queue: use a dedicated lock for the queues (v2)
- MEDIUM: queue: simplify again the process_srv_queue() API (v2)
- MEDIUM: queue: determine in process_srv_queue() if the proxy is usable (v2)
- MINOR: queue: factor out the proxy/server queuing code (v2)
- MINOR: queue: use atomic-ops to update the queue's index (v2)
- MEDIUM: queue: take the proxy lock only during the px queue accesses
- MEDIUM: queue: use a trylock on the server's queue
- MINOR: queue: add queue_init() to initialize a queue
- MINOR: queue: add a pointer to the server and the proxy in the queue
- MINOR: queue: store a pointer to the queue into the pendconn
- MINOR: queue: remove the px/srv fields from pendconn
- MINOR: queue: simplify pendconn_unlink() regarding srv vs px
- BUG: backend: stop looking for queued connections once there's no more
- BUG/MINOR: queue/debug: use the correct lock labels on the queue lock
- BUG/MINOR: resolvers: Always attach server on matching record on resolution
- BUG/MINOR: resolvers: Reset server IP when no ip is found in the response
- MINOR: resolvers: Reset server IP on error in resolv_get_ip_from_response()
- BUG/MINOR: checks: return correct error code for srv_parse_agent_check
- BUILD: Makefile: fix linkage for Haiku.
- BUG/MINOR: tcpcheck: Fix numbering of implicit HTTP send/expect rules
- MINOR: http-act/tcp-act: Add "set-log-level" for tcp content rules
- MINOR: http-act/tcp-act: Add "set-nice" for tcp content rules
- MINOR: http-act/tcp-act: Add "set-mark" and "set-tos" for tcp content rules
- CLEANUP: tcp-act: Sort action lists
- BUILD/MEDIUM: tcp: set-mark setting support for FreeBSD.
- BUILD: tcp-act: avoid warning when set-mark / set-tos are not supported
- BUG/MINOR: mqtt: Fix parser for string with more than 127 characters
- BUG/MINOR: mqtt: Support empty client ID in CONNECT message
- BUG/MEDIUM: resolvers: Make 1st server of a template take part to SRV resolution
- CLEANUP: peers: re-write intdecode function comment.
It is now possible to set the Netfilter MARK and the TOS field value in all
packets sent to the client from any tcp-request rulesets or the "tcp-response
content" one. To do so, the parsing of "set-mark" and "set-tos" actions are
moved in tcp_act.c and the actions evaluation is handled in dedicated functions.
This patch may be backported as far as 2.2 if necessary.
It is now possible to set the "nice" factor of the current stream from a
"tcp-request content" or "tcp-response content" ruleset. To do so, the
action parsing is moved in stream.c and the action evaluation is handled in
a dedicated function.
This patch may be backported as far as 2.2 if necessary.
It is now possible to set the stream log level from a "tcp-request content"
or "tcp-response content" ruleset. To do so, the action parsing is moved in
stream.c and the action evaluation is handled in a dedicated function.
This patch should fix issue #1306. It may be backported as far as 2.2 if
necessary.
set-src/set-src-port and set-dst/set-dst-port actions were not listed in the
documentation of "tcp-request session".
This patch may be backported to all stable versions.
If it possible to set source IP/Port from "tcp-request connection",
"tcp-request session" and "http-request" rules but not from "tcp-request
content" rules. There is no reason for this limitation and it may be a
problem for anyone wanting to call a lua fetch to dynamically set source
IP/Port from a TCP proxy. Indeed, to call a lua fetch, we must have a
stream. And there is no stream when "tcp-request connection/session" rules
are evaluated.
Thanks to this patch, "set-src" and "set-src-port" action are now supported
by "tcp_request content" rules.
This patch is related to the issue #1303. It may be backported to all stable
versions.
Activate the 'ssl' keyword for dynamic servers. This is the final step
to have ssl dynamic servers feature implemented. If activated,
ssl_sock_prepare_srv_ctx will be called at the end of the 'add server'
CLI handler.
At the same time, update the management doc to list all ssl keywords
implemented for dynamic servers.
Since the 1.9, it is forbidden to alter the channel buffer from an HTTP
stream because there is no way to keep the HTTP parser synchronized if the
buffer content is altered. In addition, since the HTX is the only
reprensentation for HTTP messages, the data in HTTP buffers are structured
and cannot be read or updated in a raw fashion.
A warning is triggered when a user tries to alter an HTTP buffer. However,
it was not documented. This patch adds a warning in the lua documentation.
This patch is related to the issue #1287. It may be backported as far as
2.0.
This one was deprecated in 2.3 and marked for removal in 2.5. It suffers
too many limitations compared to threads, and prevents some improvements
from being engaged. Instead of a bypassable startup error, there is now
a hard error.
The parsing code was removed, and very few obvious cases were as well.
The code is deeply rooted at certain places (e.g. "for" loops iterating
from 0 to nbproc) so it will not be that trivial to remove everywhere.
The "bind" and "bind-process" parsers will have to be adjusted, though
maybe not completely changed if we later want to support thread groups
for large NUMA machines. Some stats socket restrictions were removed,
and the doc was updated according to what was done. A few places in the
doc still refer to nbproc and will have to be revisited. The master-worker
code also refers to the process number to distinguish between master and
workers and will have to be carefully adjusted. The MAX_PROCS macro was
reset to 1, this will at least reduce the size of some remaining arrays.
Two regtests were dependieng on this directive, one with an explicit
"nbproc 1" and another one testing the master's CLI using nbproc 4.
Both were adapted.
Commit ab0a5192a ("MEDIUM: config: mark "grace" as deprecated") marked
the "grace" keyword as deprecated in 2.3, tentative removal for 2.4
with a hard deadline in 2.5, so let's remove it and return an error now.
This old and outdated feature was incompatible with soft-stop, reload
and socket transfers, and keeping it forced ugly hacks in the lower
layers of the protocol stack.
It's been warning as being deprecated since 2.0-dev4, it's about time
to drop it now. The error message recommends to either remove it or
use "option httpclose" instead. It's still referred to in the old
internal doc about the connection header, which itself seems highly
inaccurate by now.
It was marked as deprecated for immediate removal as it was not used,
let's reject it and remove it from the doc. A specific error suggests
to check tune.bufsize instead.
As specified in the RFC3875 (section 4.1.17), this parameter must be set to
the name and version of the information server software making the CGI
request. Thus, it is now added to the default parameters defined by
HAProxy. It is set to the string "HAProxy $version".
This patch should fix the issue #1285 and must be backported as far as 2.2.
Define srv.init_addr_methods to SRV_IADDR_NONE on 'add server' CLI
handler. This explicitly states that no resolution will be made on the
server creation.
This is not a real bug as the default value (SRV_IADDR_END) has the same
effect in practice. However the intent is clearer and prevent to use the
default "libc,last" by mistake which cannot execute on runtime (blocking
call + file access via gethostbyname/getaddrinfo).
The doc is also updated to reflect this limitation.
This should be backported up to 2.4.
Add the ability to dump an OCSP response details through a call to "show
ssl cert cert.pem.ocsp". It can also be used on an ongoing transaction
by prefixing the certificate name with a '*'.
Even if the ckch structure holds an ocsp_response buffer, we still need
to look for the actual ocsp response entry in the ocsp response tree
rather than just dumping the ckch's buffer details because when updating
an ocsp response through a "set ssl ocsp-response" call, the
corresponding buffer in the ckch is not updated accordingly. So this
buffer, even if it is not empty, might hold an outdated ocsp response.
This patch adds the "show ssl ocsp-response [<id>]" CLI command. This
command can be used to display the IDs of the OCSP tree entries along
with details about the entries' certificate ID (issuer's name and key
hash + serial number), or to display the details of a single
ocsp-response if an ID is given. The details displayed in this latter
case are the ones shown by a "openssl ocsp -respin <ocsp-response>
-text" call.
This patch adds the `-cc` (check condition) argument to evaluate conditions on
startup and return the result as the exit code.
As an example this can be used to easily check HAProxy's version in scripts:
haproxy -cc 'version_atleast(2.4)'
This resolves GitHub issue #1246.
Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
The output of "show map/acl" now contains the 'entry_cnt' value that
represents the count of all the entries for each map/acl, not just the
active ones, which means that it also includes entries currently being
added.
Released version 2.4.0 with the following main changes :
- BUG/MINOR: http_fetch: fix possible uninit sockaddr in fetch_url_ip/port
- CLEANUP: cli/activity: Remove double spacing in set profiling command
- CI: Build VTest with clang
- CI: extend spellchecker whitelist, add "ists" as well
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: memprof: properly account for differences for realloc()
- MINOR: memprof: also report the method used by each call
- MINOR: memprof: also report the totals and delta alloc-free
- CLEANUP: pattern: remove the unused and dangerous pat_ref_reload()
- BUG/MINOR: http_act: Fix normalizer names in error messages
- MINOR: uri_normalizer: Add `fragment-strip` normalizer
- MINOR: uri_normalizer: Add `fragment-encode` normalizer
- IMPORT: slz: use the generic function for the last bytes of the crc32
- IMPORT: slz: do not produce the crc32_fast table when CRC is natively supported
- BUILD/MINOR: opentracing: fixed compilation with filter enabled
- BUILD: makefile: add a few popular ARMv8 CPU targets
- BUG/MEDIUM: stick_table: fix crash when using tcp smp_fetch_src
- REGTESTS: stick-table: add src_conn_rate test
- CLEANUP: stick-table: remove a leftover of an old keyword declaration
- BUG/MINOR: stats: fix lastchk metric that got accidently lost
- EXAMPLES: add a "basic-config-edge" example config
- EXAMPLES: add a trivial config for quick testing
- DOC: management: Correct example reload command in the document
- Revert "CI: Build VTest with clang"
- MINOR: activity/cli: optionally support sorting by address on "show profiling"
- DEBUG: ssl: export ssl_sock_close() to see its symbol resolved in profiling
- BUG/MINOR: lua/vars: prevent get_var() from allocating a new name
- DOC: config: Fix configuration example for mqtt
- BUG/MAJOR: config: properly initialize cpu_map.thread[] up to MAX_THREADS
- BUILD: config: avoid a build warning on numa_detect_topology() without threads
- DOC: update min requirements in INSTALL
- IMPORT: slz: use inttypes.h instead of stdint.h
- BUILD: sample: use strtoll() instead of atoll()
- MINOR: version: mention that it's LTS now.
"show profiling" by default sorts by usage/counts, which is suitable for
occasional use. But when called from scripts to monitor/search variations,
this is not very convenient. Let's add a new "byaddr" option to support
sorting the output by address. It also eases matching alloc/free calls
from within a same library, or reading grouped tasks costs by library.
Current example is:
`echo "reload" | socat /var/run/haproxy-master.sock`
it will cause socat error:
`exactly 2 addresses required (there are 1); use option "-h" for help`
Correct working command is:
`echo "reload" | socat /var/run/haproxy-master.sock stdin`
Released version 2.4-dev19 with the following main changes :
- BUG/MINOR: hlua: Don't rely on top of the stack when using Lua buffers
- BUG/MEDIUM: cli: prevent memory leak on write errors
- BUG/MINOR: ssl/cli: fix a lock leak when no memory available
- MINOR: debug: add a new "debug dev sym" command in expert mode
- MINOR: pools/debug: slightly relax DEBUG_DONT_SHARE_POOLS
- CI: Github Actions: switch to LibreSSL-3.3.3
- MINOR: srv: close all idle connections on shutdown
- MINOR: connection: move session_list member in a union
- MEDIUM: mux_h1: release idling frontend conns on soft-stop
- MEDIUM: connection: close front idling connection on soft-stop
- MINOR: tools: add functions to retrieve the address of a symbol
- CLEANUP: activity: mark the profiling and task_profiling_mask __read_mostly
- MINOR: activity: add a "memory" entry to "profiling"
- MINOR: activity: declare the storage for memory usage statistics
- MEDIUM: activity: collect memory allocator statistics with USE_MEMORY_PROFILING
- MINOR: activity: clean up the show profiling io_handler a little bit
- MINOR: activity: make "show profiling" support a few arguments
- MINOR: activity: make "show profiling" also dump the memoery usage
- MINOR: activity: add the profiling.memory global setting
- BUILD: makefile: add new option USE_MEMORY_PROFILING
- MINOR: channel: Rely on HTX version if appropriate in channel_may_recv()
- BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive
- MINOR: conn-stream: Force mux to wait for read events if abortonclose is set
- MEDIUM: mux-h1: Don't block reads when waiting for the other side
- BUG/MEDIUM: mux-h1: Properly report client close if abortonclose option is set
- REGTESTS: Add script to test abortonclose option
- MINOR: mux-h1: clean up conditions to enabled and disabled splicing
- MINOR: mux-h1: Subscribe for sends if output buffer is not empty in h1_snd_pipe
- MINOR: mux-h1: Always subscribe for reads when splicing is disabled
- MEDIUM: mux-h1: Wake H1 stream when both sides a synchronized
- CLEANUP: mux-h1: rename WAIT_INPUT/WAIT_OUTPUT flags
- MINOR: mux-h1: Manage processing blocking flags on the H1 stream
- BUG/MINOR: stream: Decrement server current session counter on L7 retry
- BUG/MINOR: config: fix uninitialized initial state in ".if" block evaluator
- BUG/MINOR: config: add a missing "ELIF_TAKE" test for ".elif" condition evaluator
- BUG/MINOR: config: .if/.elif should also accept negative integers
- MINOR: config: centralize the ".if"/".elif" condition parser and evaluator
- MINOR: config: keep up-to-date current file/line/section in the global struct
- MINOR: config: support some pseudo-variables for file/line/section
- BUILD: activity: do not include malloc.h
- MINOR: arg: improve the error message on missing closing parenthesis
- MINOR: global: export the build features string list
- MINOR: global: add version comparison functions
- MINOR: config: improve .if condition error reporting
- MINOR: config: make cfg_eval_condition() support predicates with arguments
- MINOR: config: add predicate "defined()" to conditional expression blocks
- MINOR: config: add predicates "streq()" and "strneq()" to conditional expressions
- MINOR: config: add predicate "feature" to detect certain built-in features
- MINOR: config: add predicates "version_atleast" and "version_before" to cond blocks
- BUG/MINOR: activity: use the new pointer to calculate the new size in realloc()
- BUG/MINOR: stream: properly clear the previous error mask on L7 retries
- MEDIUM: log: slightly refine the output format of alerts/warnings/etc
- MINOR: config: add a new message directive: .diag
- CLEANUP: cli/tree-wide: properly re-align the CLI commands' help messages
- BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
- BUG/MINOR: checks: Handle synchronous connect when a tcpcheck is started
- BUG/MINOR: checks: Reschedule check on observe mode only if fastinter is set
- MINOR: global: define tainted flag
- MINOR: cfgparse: add a new field flags in cfg_keyword
- MINOR: cfgparse: implement experimental config keywords
- MINOR: action: replace match_pfx by a keyword flags field
- MINOR: action: implement experimental actions
- MINOR: cli: set tainted when using CLI expert/experimental mode
- MINOR: stats: report tainted on show info
- MINOR: http_act: mark normalize-uri as experimental
- BUILD: fix usage of ha_alert without format string
- MINOR: proxy: define PR_CAP_LB
- BUG/MINOR: server: do not report diag for peer servers with null weight
- DOC: ssl: Extra files loading now works for backends too
- ADDONS: make addons/ discoverable by git via .gitignore
- DOC: ssl: Add information about crl-file option
- MINOR: sample: improve error reporting on missing arg to strcmp() converter
- DOC: management: mention that some fields may be emitted as floats
- MINOR: tools: implement trimming of floating point numbers
- MINOR: tools: add a float-to-ascii conversion function
- MINOR: freq_ctr: add new functions to report float measurements
- MINOR: stats: avoid excessive padding of float values with trailing zeroes
- MINOR: stats: add the HTML conversion for float types
- MINOR: stats: pass the appctx flags to stats_fill_info()
- MINOR: stats: support an optional "float" option to "show info"
- MINOR: stats: use tv_remain() to precisely compute the uptime
- MINOR: stats: report uptime and start time as floats with subsecond resolution
- MINOR: stats: make "show info" able to report rates as floats when asked
- MINOR: config: mark tune.fd.edge-triggered as experimental
- REORG: vars: move the "proc" scope variables out of the global struct
- REORG: threads: move all_thread_mask() to thread.h
- BUILD: wdt: include signal-t.h
- BUILD: auth: include missing list.h
- REORG: mworker: move proc_self from global to mworker
- BUILD: ssl: ssl_utils requires chunk.h
- BUILD: config: cfgparse-ssl.c needs tools.h
- BUILD: wurfl: wurfl.c needs tools.h
- BUILD: spoe: flt_spoe.c needs tools.h
- BUILD: promex: service-prometheus.c needs tools.h
- BUILD: resolvers: include tools.h
- BUILD: config: include tools.h in cfgparse-listen.c
- BUILD: htx: include tools.h in http_htx.c
- BUILD: proxy: include tools.h in proxy.c
- BUILD: session: include tools.h in session.c
- BUILD: cache: include tools.h in cache.c
- BUILD: sink: include tools.h in sink.c
- BUILD: connection: include tools.h in connection.c
- BUILD: server-state: include tools.h from server_state.c
- BUILD: dns: include tools.h in dns.c
- BUILD: payload: include tools.h in payload.c
- BUILD: vars: include tools.h in vars.c
- BUILD: compression: include tools.h in compression.c
- BUILD: mworker: include tools.h from mworker.c
- BUILD: queue: include tools.h from queue.c
- BUILD: udp: include tools.h from proto_udp.c
- BUILD: stick-table: include freq_ctr.h from stick_table.h
- BUILD: server: include tools.h from server.c
- BUILD: server: include missing proxy.h in server.c
- BUILD: sink: include proxy.h in sink.c
- BUILD: mworker: include proxy.h in mworker.c
- BUILD: filters: include proxy.h in filters.c
- BUILD: fcgi-app: include proxy.h in fcgi-app.c
- BUILD: connection: move list_mux_proto() to connection.c
- REORG: stick-table: uninline stktable_alloc_data_type()
- REORG: stick-table: move composite address functions to stick_table.h
- REORG: config: uninline warnifnotcap() and failifnotcap()
- BUILD: task: remove unused includes from task.c
- MINOR: task: stop including stream.h from task.c
- BUILD: connection: stop including listener-t.h
- BUILD: hlua: include proxy.h from hlua.c
- BUILD: mux-h1: include proxy.h from mux-h1.c
- BUILD: mux-fcgi: include proxy.h from mux-fcgi.c
- BUILD: listener: include proxy.h from listener.c
- BUILD: http-rules: include proxy.h from http_rules.c
- BUILD: thread: include log.h from thread.c
- BUILD: comp: include proxy.h from flt_http_comp.c
- BUILD: fd: include log.h from fd.c
- BUILD: config: do not include proxy.h nor errors.h anymore in cfgparse.h
- BUILD: makefile: reorder object files by build time
- DOC: Fix a few grammar/spelling issues and casing of HAProxy
- REGTESTS: run-regtests: match both "HAProxy" and "HA-Proxy" in the version
- MINOR: version: report "HAProxy" not "HA-Proxy" in the version output
- DOC: remove last occurrences of "HA-Proxy" syntax
- DOC: peers: fix the protocol tag name in the doc
- ADMIN: netsnmp: report "HAProxy" and not "Haproxy" in output descriptions
- MEDIUM: mailers: use "HAProxy" nor "HAproxy" in the subject of messages
- DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
- MINOR: tools/rnd: compute the result outside of the CAS loop
- BUILD: http_fetch: address a few aliasing warnings with older compilers
- BUILD: ssl: define HAVE_CRYPTO_memcmp() based on the library version
- BUILD: errors: include stdarg in errors.h
- REGTESTS: disable inter-thread idle connection sharing on sensitive tests
- MINOR: cli: make "help" support a command in argument
- MINOR: cli: sort the output of the "help" keywords
- CLEANUP: cli/mworker: properly align the help messages
- BUILD: memprof: make the old caller pointer a const in get_prof_bin()
- BUILD: compat: include malloc_np.h for USE_MEMORY_PROFILING on FreeBSD
- CI: Github Actions: enable USE_QUIC=1 for BoringSSL builds
- BUG/MEDIUM: quic: fix null deref on error path in qc_conn_init()
- BUILD: cli: appease a null-deref warning in cli_gen_usage_msg()
With ~100 commands on the CLI, it's particularly difficult to find a
specific one in the "help" output. The function used to display the
help already supports filtering on certain commands, so in the end it's
just needed to pass the argument of the help command to enable the
automatic filtering. That's what this patch does so that "help clear"
only lists commands starting with "clear" and that "help map" lists
commands containing "map" in them.
Some of the Lua doc and a few places still used "Haproxy" or "HAproxy".
There was even one "HA proxy". A few of them were in an example of VTest
output, indicating that VTest ought to be fixed as well. No big deal but
better address all the remaining ones so that these inconsistencies stop
spreading around.
The peers protocol has been using "HAProxyS" as a binary tag sent on the
wire since day one in 1.5-dev3 with commit 2b920a1af ("[MAJOR] Add new
files src/peer.c, include/proto/peers.h and include/types/peers.h for
sync stick table management"), regardless, the doc says the protocol
identifier is "HaproxyS". It is likely this got fixed in the code before
merging and not in the doc.
This should be backported to any release as the doc is wrong.
There were only a few more used as output examples and comments in a few
docs, it was the right moment to get rid of them. The file intro.txt
which explains how to parse the version also got a hint about the possible
presence of a hyphen in the name in older versions.
This patch fixes a few grammar and spelling issues in configuration.txt.
It was also noted that there was a wide range of case usage
(i.e. haproxy, HAproxy, HAProxy, etc... ). This patch updates them
all to be consistently "HAProxy" except where a binary is mentioned.
This will allow some fields to be produced with a higher accuracy when
the requester indicates being able to parse floats. Rates and times are
among the elements which can make sense.
When using the crl-file option with multiple Certificate Authority
levels in the CA chain, there must be one CRL per CA or the verify
function on the backend side will raise an "unagle to get certificate
CRL" error (error code 3).
This was required by GitHub issue #1201.
When implementing the server side certificate hot update, the ckch
mechanism was used on the backend side in order to mimic the frontend
certificate management and to enable server line certificate update via
the CLI (see GitHub issue #427). As an unexpected side effect, we now
also look for ssl extra files (cert.pem.key, cert.pem.ocsp ...) for the
backend side.
This patch updates the documentation accordingly.
This answers to GitHub issue #845.
normalize-uri http rule is marked as experimental, so it cannot be
activated without the global 'expose-experimental-directives'. The
associated vtc is updated to be able to use it.
Add a new flag to mark a keyword as experimental. An experimental
keyword cannot be used if the global 'expose-experimental-directives' is
not present first.
Only keywords parsed through a standard cfg_keywords lists in
global/proxies section will be automatically detected if declared
experimental. To support a keyword outside of these lists,
check_kw_experimental must be called manually during its parsing.
If an experimental keyword is present in the config, the tainted flag is
updated.
For the moment, no keyword is marked as experimental.
This one works just like .notice/.warning/.alert except that it prints
the message at level "DIAG" only when haproxy runs in diagnostic mode
(-dD). This can be convenient for example to pass a few hints to help
locate certain config parts or to leave messages about certain temporary
workarounds.
Example:
.diag "WTA/2021-05-07: $.LINE: replace 'redirect' with 'return' after final switch to 2.4"
http-request redirect location /goaway if ABUSE
For about 20 years we've been emitting cryptic messages on warnings and
alerts, that nobody knows how to parse:
[NOTICE] 126/080118 (3115) : haproxy version is 2.4-dev18-0b7c78-49
[NOTICE] 126/080118 (3115) : path to executable is ./haproxy
[WARNING] 126/080119 (3115) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 126/080119 (3115) : backend 'default' has no server available!
Hint: the first 3-digit number is the day of year, and the 6 digits
after it represent the time of day in format HHMMSS, then the pid in
parenthesis. These are not quite user-friendly and such cryptic into
are not useful at all.
This patch slightly adjusts the output by performing these minimal changes:
- removing the date/time, as they were added very early when haproxy
was meant to be used in foreground as a debugging tool, and they're
provided in more details in logs nowadays ;
- better aligning the fields by padding the severity tag to 10 chars.
The diag output was renamed to "DIAG" only.
Now the output provides this:
[NOTICE] (4563) : haproxy version is 2.4-dev18-75a428-51
[NOTICE] (4563) : path to executable is ./haproxy
[WARNING] (4563) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] (4563) : backend 'default' has no server available!
The useless space before the colon was kept so as not to confuse any
possible output parser.
The few entries in the doc referring to this format were adjusted to
reflect the new one.
The change was tagged "MEDIUM" as it may have visible consequences on
home-grown monitoring tools, though it is extremely unlikely due to the
limited extent of these changes.
These predicates respectively verify that the current version is at least
a given version or is before a specific one. The syntax is exactly the one
reported by "haproxy -v", though each component is optional, so both "1.5"
and "2.4-dev18-88910-48" are supported. Missing components equal zero, and
"dev" is below "pre" or "rc", which are both inferior to no such mention
(i.e. they are negative). Thus "2.4-dev18" is older than "2.4-rc1" which
is older than "2.4".
The "feature(name)" predicate will return true if <name> corresponds to
a name listed after a '+' in the features list, that is it was enabled at
build time with USE_<name>=1. Typical use cases will include OPENSSL, LUA
and LINUX_SPLICE. But maybe it will also be convenient to use with optional
addons such as PROMEX and the device detection modules to help keeping the
same configs across various deployments.
"streq(str1,str2)" will return true if the two strings match while
"strneq(str1,str2)" will return true only if they differ. This is
convenient to match an environment variable against a predefined value.
The new pseudo-variables ".FILE", ".LINE" and ".SECTION" will be resolved
on the fly by the config parser and will respectively retrieve the current
configuration file name, the current line number and the current section
being parsed. This may help emit logs, errors, and debugging information
(e.g. which rule matched).
The '.' in the first char was reserved for such pseudo-variables and no
other variable is permitted. This will allow to add support for new ones
in the future if they prove to be useful (e.g. randoms/uuid for secret
keying or automatic naming of configuration objects).
This adds the necessary flags to permit run-time enabling/disabling of
memory profiling. For now this is disabled.
A few words were added to the management doc about it and recalling that
this is limited to certain OSes.
Released version 2.4-dev18 with the following main changes :
- DOC: Fix indentation for `path-strip-dot` normalizer
- DOC: Fix RFC reference for the percent-to-uppercase normalizer
- DOC: Add RFC references for the path-strip-dot(dot)? normalizers
- MINOR: uri_normalizer: Add a `percent-decode-unreserved` normalizer
- BUG/MINOR: mux-fcgi: Don't send normalized uri to FCGI application
- REORG: htx: Inline htx functions to add HTX blocks in a message
- CLEANUP: assorted typo fixes in the code and comments
- DOC: general: fix white spaces for HTML converter
- BUG/MINOR: ssl: ssl_sock_prepare_ssl_ctx does not return an error code
- BUG/MINOR: cpuset: move include guard at the very beginning
- BUG/MAJOR: fix build on musl with cpu_set_t support
- BUG/MEDIUM: cpuset: fix build on MacOS
- BUG/MINOR: htx: Preserve HTX flags when draining data from an HTX message
- MEDIUM: htx: Refactor htx_xfer_blks() to not rely on hdrs_bytes field
- CLEANUP: htx: Remove unsued hdrs_bytes field from the HTX start-line
- BUG/MINOR: mux-h2: Don't encroach on the reserve when decoding headers
- MEDIUM: http-ana: handle read error on server side if waiting for response
- MINOR: htx: Limit length of headers name/value when a HTX message is dumped
- BUG/MINOR: applet: Notify the other side if data were consumed by an applet
- BUG/MINOR: hlua: Don't consume headers when starting an HTTP lua service
- BUG/MEDIUM: mux-h2: Handle EOM flag when sending a DATA frame with zero-copy
- CLEANUP: channel: No longer notify the producer in co_skip()/co_htx_skip()
- DOC: general: fix example in set-timeout
- CLEANUP: cfgparse: de-uglify early file error handling in readcfgfile()
- MINOR: config: add a new "default-path" global directive
- BUG/MEDIUM: peers: initialize resync timer to get an initial full resync
- BUG/MEDIUM: peers: register last acked value as origin receiving a resync req
- BUG/MEDIUM: peers: stop considering ack messages teaching a full resync
- BUG/MEDIUM: peers: reset starting point if peers appears longly disconnected
- BUG/MEDIUM: peers: reset commitupdate value in new conns
- BUG/MEDIUM: peers: re-work updates lookup during the sync on the fly
- BUG/MEDIUM: peers: reset tables stage flags stages on new conns
- MINOR: peers: add informative flags about resync process for debugging
- BUG/MEDIUM: time: fix updating of global_now upon clock drift
- CLEANUP: freq_ctr: make arguments of freq_ctr_total() const
- CLEANUP: hlua: rename hlua_appctx* appctx to luactx
- MINOR: server: fix doc/trace on lb algo for dynamic server creation
- REGTESTS: server: fix cli_add_server due to previous trace update
- REGTESTS: add minimal CLI "add map" tests
- DOC: management: move "set var" to the proper place
- CLEANUP: map: slightly reorder the add map function
- MINOR: map: get rid of map_add_key_value()
- MINOR: map: show the current and next pattern version in "show map"
- MINOR: map/acl: add the possibility to specify the version in "show map/acl"
- MINOR: pattern: support purging arbitrary ranges of generations
- MINOR: map/acl: add the possibility to specify the version in "clear map/acl"
- MINOR: map/acl: add the "prepare map/acl" CLI command
- MINOR: map/acl: add the "commit map/acl" CLI command
- MINOR: map/acl: make "add map/acl" support an optional version number
- CLEANUP: map/cli: properly align the map/acl help
- BUILD: compiler: do not use already defined __read_mostly on dragonfly
By passing a version number to "add map/acl", it becomes possible to
atomically replace maps and ACLs. The principle is that a new version
number is first retrieved by calling"prepare map/acl", and this version
number is used with "add map" and "add acl". Newly added entries then
remain invisible to the matching mechanism but are visible in "show
map/acl" when the version number is specified, or may be cleard with
"clear map/acl". Finally when the insertion is complete, a
"commit map/acl" command must be issued, and the version is atomically
updated so that there is no intermediate state with incomplete entries.
The command is used to atomically replace a map/acl with the pending
contents of the designated version. The new version must have been
allocated by "prepare map/acl" prior to this. At the moment it is not
possible to force the version when adding new entries, so this may only
be used to atomically clear an ACL/map.
This command allocates a new version for the map/acl, that will be usable
later to prepare the addition of new values to atomically replace existing
ones. Technically speaking the operation consists in atomically incrementing
the next version. There's no "undo" operation here, if a version is not
committed, it will automatically be trashed when committing a newer version.
This will ease maintenance of versionned maps by allowing to clear old or
failed updates instead of the current version. Nothing was done to allow
clearing everyhing, though if there was a need for this, implementing "@all"
or something equivalent wouldn't require more than 3 lines of code.
The maps and ACLs internally all have two versions, the "current" one,
which is the one being matched against, and the "next" one, the one being
filled during an atomic replacement. Till now the "show" commands only used
to show the current one but it can be convenient to be able to show other
ones as well, so let's add the ability to do this with "show map" and
"show acl". The method used here consists in passing the version number
as "@<ver>" before the map/acl name or ID. It would have been better after
it but that could create confusion with keys already using such a format.
Commit b8bd1ee89 ("MEDIUM: cli: add a new experimental "set var" command")
added "get var" and "set var" but "set var" was misplaced in the doc,
breaking the alphabetic ordering.
The text mentionned that only backends with consistent hash method were
supported for dynamic servers. In fact, it is only required that the lb
algorith is dynamic.
By default haproxy loads all files designated by a relative path from the
location the process is started in. In some circumstances it might be
desirable to force all relative paths to start from a different location
just as if the process was started from such locations. This is what this
directive is made for. Technically it will perform a temporary chdir() to
the designated location while processing each configuration file, and will
return to the original directory after processing each file. It takes an
argument indicating the policy to use when loading files whose path does
not start with a slash ('/').
A few options are offered, "current" (the default), "config" (files
relative to config file's dir), "parent" (files relative to config file's
parent dir), and "origin" with an absolute path.
This should address issue #1198.
The alternative arguments are always in curly brackets, let's fix it for
set-timeout.
The Example in set-timeout does not have the one of the required argument.
This commit makes the PR https://github.com/cbonte/haproxy-dconv/pull/34
obsolete.
Released version 2.4-dev17 with the following main changes :
- MINOIR: mux-pt/trace: Register a new trace source with its events
- BUG/MINOR: mux-pt: Fix a possible UAF because of traces in mux_pt_io_cb
- CI: travis: Drastically clean up .travis.yml
- CLEANUP: pattern: make all pattern tables read-only
- MINOR: trace: replace the trace() inline function with an equivalent macro
- MINOR: initcall: uniformize the section names between MacOS and other unixes
- CLEANUP: initcall: rename HA_SECTION to HA_INIT_SECTION
- MINOR: compiler: add macros to declare section names
- CLEANUP: initcall: rely on HA_SECTION_* instead of defining its own
- MINOR: global: declare a read_mostly section
- MINOR: fd: move a few read-mostly variables to their own section
- MINOR: epoll: move epoll_fd to read_mostly
- MINOR: kqueue: move kqueue_fd to read_mostly
- MINOR: pool: move pool declarations to read_mostly
- MINOR: threads: mark all_threads_mask as read_mostly
- MINOR: server: move idle_conn_task to read_mostly
- MINOR: protocol: move __protocol_by_family to read_mostly
- MINOR: pattern: make the pat_lru_seed read_mostly
- MINOR: trace: make trace sources read_mostly
- MINOR: freq_ctr: add a generic function to report the total value
- MEDIUM: freq_ctr: make read_freq_ctr_period() use freq_ctr_total()
- MEDIUM: freq_ctr: reimplement freq_ctr_remain_period() from freq_ctr_total()
- MINOR: freq_ctr: add the missing next_event_delay_period()
- MINOR: freq_ctr: unify freq_ctr and freq_ctr_period into freq_ctr
- MEDIUM: freq_ctr: replace the per-second counters with the generic ones
- MINOR: freq_ctr: add cpu_relax in the rotation loop of update_freq_ctr_period()
- MINOR: freq_ctr: simplify and improve the update function
- CLEANUP: time: remove the now unused ms_left_scaled
- MINOR: time: move the time initialization out of tv_update_date()
- MINOR: time: remove useless variable copies in tv_update_date()
- MINOR: time: change the global timeval and the the global tick at once
- MEDIUM: time: make the clock offset global and no per-thread
- MINOR: atomic: reimplement the relaxed version of x86 BTS/BTR
- MINOR: trace: Add the checks as a possible trace source
- MINOIR: checks/trace: Register a new trace source with its events
- MINOR: hlua: Add function to release a lua function
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a task
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a converter
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a fetch
- BUG/MINOR: hlua: Fix memory leaks on error path when parsing a lua action
- BUG/MINOR: hlua: Fix memory leaks on error path when registering an action
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a service
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a cli keyword
- BUG/MINOR: cfgparse/proxy: Fix some leaks during proxy section parsing
- BUG/MINOR: listener: Handle allocation error when allocating a new bind_conf
- BUG/MINOR: cfgparse/proxy: Hande allocation errors during proxy section parsing
- MINOR: cfgparse/proxy: Group alloc error handling during proxy section parsing
- DOC: internals: update the SSL architecture schema
- BUG/MEDIUM: sample: Fix adjusting size in field converter
- MINOR: sample: add ub64dec and ub64enc converters
- CLEANUP: sample: align samples list in sample.c
- MINOR: ist: Add `istclear(struct ist*)`
- CI: cirrus: install "pcre" package
- MINOR: opentracing: correct calculation of the number of arguments in the args[]
- MINOR: opentracing: transfer of context names without prefix
- MINOR: sample: converter: Add mjson library.
- MINOR: sample: converter: Add json_query converter
- CI: travis-ci: enable weekly graviton2 builds
- DOC: ssl: Certificate hot update only works on fronted certificates
- DOC: ssl: Certificate hot update works on server certificates
- BUG/MEDIUM: threads: Ignore current thread to end its harmless period
- MINOR: threads: Only consider running threads to end a thread harmeless period
- BUG/MINOR: checks: Set missing id to the dummy checks frontend
- MINOR: logs: Add support of checks as session origin to format lf strings
- BUG/MINOR: connection: Fix fc_http_major and bc_http_major for TCP connections
- MINOR: connection: Make bc_http_major compatible with tcp-checks
- BUG/MINOR: ssl-samples: Fix ssl_bc_* samples when called from a health-check
- BUG/MINOR: http-fetch: Make method smp safe if headers were already forwarded
- MINOR: tcp_samples: Add samples to get src/dst info of the backend connection
- MINOR: tcp_samples: Be able to call bc_src/bc_dst from the health-checks
- BUG/MINOR: http_htx: Remove BUG_ON() from http_get_stline() function
- BUG/MINOR: logs: Report the true number of retries if there was no connection
- BUILD: makefile: Redirect stderr to /dev/null when probing options
- MINOR: uri_normalizer: Add uri_normalizer module
- MINOR: uri_normalizer: Add `enum uri_normalizer_err`
- MINOR: uri_normalizer: Add `http-request normalize-uri`
- MINOR: uri_normalizer: Add a `merge-slashes` normalizer to http-request normalize-uri
- MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri
- MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer
- MINOR: uri_normalizer: Add a `sort-query` normalizer
- MINOR: uri_normalizer: Add a `percent-upper` normalizer
- MEDIUM: http_act: Rename uri-normalizers
- DOC: Add introduction to http-request normalize-uri
- DOC: Note that URI normalization is experimental
- BUG/MINOR: pools: maintain consistent ->allocated count on alloc failures
- BUG/MINOR: pools/buffers: make sure to always reserve the required buffers
- MINOR: pools: drop the unused static history of artificially failed allocs
- CLEANUP: pools: remove unused arguments to pool_evict_from_cache()
- MEDIUM: pools: move the cache into the pool header
- MINOR: pool: remove the size field from pool_cache_head
- MINOR: pools: rename CONFIG_HAP_LOCAL_POOLS to CONFIG_HAP_POOLS
- MINOR: pools: enable the fault injector in all allocation modes
- MINOR: pools: make the basic pool_refill_alloc()/pool_free() update needed_avg
- MEDIUM: pools: unify pool_refill_alloc() across all models
- CLEANUP: pools: re-merge pool_refill_alloc() and __pool_refill_alloc()
- MINOR: pools: call pool_alloc_nocache() out of the pool's lock
- CLEANUP: pools: move the lock to the only __pool_get_first() that needs it
- CLEANUP: pools: rename __pool_get_first() to pool_get_from_shared_cache()
- CLEANUP: pools: rename pool_*_{from,to}_cache() to *_local_cache()
- CLEANUP: pools: rename __pool_free() to pool_put_to_shared_cache()
- MINOR: tools: add statistical_prng_range() to get a random number over a range
- MINOR: pools: use cheaper randoms for fault injections
- MINOR: pools: move the fault injector to __pool_alloc()
- MINOR: pools: split the OS-based allocator in two
- MINOR: pools: always use atomic ops to maintain counters
- MINOR: pools: move pool_free_area() out of the lock in the locked version
- MINOR: pools: factor the release code into pool_put_to_os()
- MEDIUM: pools: make CONFIG_HAP_POOLS control both local and shared pools
- MINOR: pools: create unified pool_{get_from,put_to}_cache()
- MINOR: pools: evict excess objects using pool_evict_from_local_cache()
- MEDIUM: pools: make pool_put_to_cache() always call pool_put_to_local_cache()
- CLEANUP: pools: make the local cache allocator fall back to the shared cache
- CLEANUP: pools: merge pool_{get_from,put_to}_local_caches with generic ones
- CLEANUP: pools: uninline pool_put_to_cache()
- CLEANUP: pools: declare dummy pool functions to remove some ifdefs
- BUILD: pools: fix build with DEBUG_FAIL_ALLOC
- BUG/MINOR: server: make srv_alloc_lb() allocate lb_nodes for consistent hash
- CONTRIB: mod_defender: import the minimal number of includes
- CONTRIB: mod_defender: make the code build with the embedded includes
- CONTRIB: modsecurity: import the minimal number of includes
- CONTRIB: modsecurity: make the code build with the embedded includes
- CLEANUP: sample: Improve local variables in sample_conv_json_query
- CLEANUP: sample: Explicitly handle all possible enum values from mjson
- CLEANUP: sample: Use explicit return for successful `json_query`s
- CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
- CONTRIB: move spoa_example out of the tree
- BUG/MINOR: server: free srv.lb_nodes in free_server
- BUG/MINOR: logs: free logsrv.conf.file on exit
- BUG/MEDIUM: server: ensure thread-safety of server runtime creation
- MINOR: server: add log on dynamic server creation
- MINOR: server: implement delete server cli command
- CONTRIB: move spoa_server out of the tree
- CONTRIB: move modsecurity out of the tree
- BUG/MINOR: server: fix potential null gcc error in delete server
- BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers
- BUG/MEDIUM: mux-h2: Fix dfl calculation when merging CONTINUATION frames
- BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort
- CLEANUP: uri_normalizer: Remove trailing whitespace
- MINOR: uri_normalizer: Add a `strip-dot` normalizer
- CONTRIB: move mod_defender out of the tree
- CLEANUP: contrib: remove the last references to the now dead contrib/ directory
- BUG/MEDIUM: config: fix cpu-map notation with both process and threads
- MINOR: config: add a diag for invalid cpu-map statement
- BUG/MINOR: mworker/init: don't reset nb_oldpids in non-mworker cases
- BUG/MINOR: mworker: don't use oldpids[] anymore for reload
- BUILD: makefile: fix the "make clean" target on strict bourne shells
- IMPORT: slz: import slz into the tree
- BUILD: compression: switch SLZ from out-of-tree to in-tree
- CI: github: do not build libslz any more
- CLEANUP: compression: remove calls to SLZ init functions
- BUG/MEDIUM: mux-h2: Properly handle shutdowns when received with data
- MINOR: cpuset: define a platform-independent cpuset type
- MINOR: cfgparse: use hap_cpuset for parse_cpu_set
- MEDIUM: config: use platform independent type hap_cpuset for cpu-map
- MINOR: thread: implement the detection of forced cpu affinity
- MINOR: cfgparse: support the comma separator on parse_cpu_set
- MEDIUM: cfgparse: detect numa and set affinity if needed
- MINOR: global: add option to disable numa detection
- BUG/MINOR: haproxy: fix compilation on macOS
- BUG/MINOR: cpuset: fix compilation on platform without cpu affinity
- MINOR: time: avoid unneeded updates to now_offset
- MINOR: time: avoid overwriting the same values of global_now
- CLEANUP: time: use __tv_to_ms() in tv_update_date() instead of open-coding
- MINOR: time: avoid u64 needlessly expensive computations for the 32-bit now_ms
- BUG/MINOR: peers: remove useless table check if initial resync is finished
- BUG/MEDIUM: peers: re-work connection to new process during reload.
- BUG/MEDIUM: peers: re-work refcnt on table to protect against flush
- BUG/MEDIUM: config: fix missing initialization in numa_detect_topology()
Render numa detection optional with a global configuration statement
'no numa-cpu-mapping'. This can be used if the applied affinity of the
algorithm is not optimal. Also complete the documentation with this new
keyword.
Use the platform independent type hap_cpuset for the cpu-map statement
parsing. This allow to address CPU index greater than LONGBITS.
Update the documentation to reflect the removal of this limit except for
platforms without cpu_set_t type or equivalent.
This normalizer removes "/./" segments from the path component.
Usually the dot refers to the current directory which renders those segments redundant.
See GitHub Issue #714.
Implement a new CLI command 'del server'. It can be used to removed a
dynamically added server. Only servers in maintenance mode can be
removed, and without pending/active/idle connection on it.
Add a new reg-test for this feature. The scenario of the reg-test need
to first add a dynamic server. It is then deleted and a client is used
to ensure that the server is non joinable.
The management doc is updated with the new command 'del server'.
As previously mentioned SPOA code has nothing to do in the haproxy core
since they're not dependent on haproxy's version. This one was moved to
its own repository here with complete history:
https://github.com/haproxy/spoa-example
The current "ADD" vs "ADDQ" is confusing because when thinking in terms
of appending at the end of a list, "ADD" naturally comes to mind, but
here it does the opposite, it inserts. Several times already it's been
incorrectly used where ADDQ was expected, the latest of which was a
fortunate accident explained in 6fa922562 ("CLEANUP: stream: explain
why we queue the stream at the head of the server list").
Let's use more explicit (but slightly longer) names now:
LIST_ADD -> LIST_INSERT
LIST_ADDQ -> LIST_APPEND
LIST_ADDED -> LIST_INLIST
LIST_DEL -> LIST_DELETE
The same is true for MT_LISTs, including their "TRY" variant.
LIST_DEL_INIT keeps its short name to encourage to use it instead of the
lazier LIST_DELETE which is often less safe.
The change is large (~674 non-comment entries) but is mechanical enough
to remain safe. No permutation was performed, so any out-of-tree code
can easily map older names to new ones.
The list doc was updated.
This patch adds an introduction to the http-request normalize-uri section,
explaining what to expect from the normalizers and possible issues that might
arise when not being careful.
This patch renames all existing uri-normalizers into a more consistent naming
scheme:
1. The part of the URI that is being touched.
2. The modification being performed as an explicit verb.
This normalizer merges `../` path segments with the predecing segment, removing
both the preceding segment and the `../`.
Empty segments do not receive special treatment. The `merge-slashes` normalizer
should be executed first.
See GitHub Issue #714.
This patch adds 4 new sample fetches to get the source and the destination
info (ip address and port) of the backend connection :
* bc_dst : Returns the destination address of the backend connection
* bc_dst_port : Returns the destination port of the backend connection
* bc_src : Returns the source address of the backend connection
* bc_src_port : Returns the source port of the backend connection
The configuration manual was updated accordingly.
The CLI's "set ssl cert" command only works on frontend certificates but
the documentation did not specify this limitations yet.
This patch can be backported to all stable branches.
With the json_query can a JSON value be extacted from a header
or body of the request and saved to a variable.
This converter makes it possible to handle some JSON workload
to route requests to different backends.
ub64dec and ub64enc are the base64url equivalent of b64dec and base64
converters. base64url encoding is the "URL and Filename Safe Alphabet"
variant of base64 encoding. It is also used in in JWT (JSON Web Token)
standard.
RFC1421 mention in base64.c file is deprecated so it was replaced with
RFC4648 to which existing converters, base64/b64dec, still apply.
Example:
HAProxy:
http-request return content-type text/plain lf-string %[req.hdr(Authorization),word(2,.),ub64dec]
Client:
Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiZm9vIiwia2V5IjoiY2hhZTZBaFhhaTZlIn0.5VsVj7mdxVvo1wP5c0dVHnr-S_khnIdFkThqvwukmdg
$ curl -H "Authorization: Bearer ${TOKEN}" http://haproxy.local
{"user":"foo","key":"chae6AhXai6e"}
This commit adds the new fields added to the ckch_inst structure in
order to manage the backend certificate hot update (GitHub #427) and the
bug of the default certificate update (GitHub #1143).
Released version 2.4-dev16 with the following main changes :
- CLEANUP: dev/flags: remove useless test in the stdin number parser
- MINOR: No longer rely on deprecated sample fetches for predefined ACLs
- MINOR: acl: Add HTTP_2.0 predefined macro
- BUG/MINOR: hlua: Detect end of request when reading data for an HTTP applet
- BUG/MINOR: tools: fix parsing "us" unit for timers
- MINOR: server/bind: add support of new prefixes for addresses.
- MINOR: log: register config file and line number on log servers.
- MEDIUM: log: support tcp or stream addresses on log lines.
- BUG/MEDIUM: log: fix config parse error logging on stdout/stderr or any raw fd
- CLEANUP: fd: remove FD_POLL_DATA and FD_POLL_STICKY
- MEDIUM: fd: prepare FD_POLL_* to move to bits 8-15
- MEDIUM: fd: merge fdtab[].ev and state for FD_EV_* and FD_POLL_* into state
- MINOR: fd: move .linger_risk into fdtab[].state
- MINOR: fd: move .cloned into fdtab[].state
- MINOR: fd: move .initialized into fdtab[].state
- MINOR: fd: move .et_possible into fdtab[].state
- MINOR: fd: move .exported into fdtab[].state
- MINOR: fd: implement an exclusive syscall bit to remove the ugly "log" lock
- MINOR: cli/show-fd: slightly reorganize the FD status flags
- MINOR: atomic/arm64: detect and use builtins for the double-word CAS
- CLEANUP: atomic: add an explicit _FETCH variant for add/sub/and/or
- CLEANUP: atomic: make all standard add/or/and/sub operations return void
- CLEANUP: atomic: add a fetch-and-xxx variant for common operations
- CLEANUP: atomic: add HA_ATOMIC_INC/DEC for unit increments
- CLEANUP: atomic/tree-wide: replace single increments/decrements with inc/dec
- CLEANUP: atomic: use the __atomic variant of BTS/BTR on modern compilers
- MINOR: atomic: implement native BTS/BTR for x86
- MINOR: ist: Add `istappend(struct ist, char)`
- MINOR: ist: Add `istshift(struct ist*)`
- MINOR: ist: Add `istsplit(struct ist*, char)`
- BUG/MAJOR: fd: switch temp values to uint in fd_stop_both()
- MINOR: opentracing: register config file and line number on log servers
- MEDIUM: resolvers: add support of tcp address on nameserver line.
- MINOR: ist: Rename istappend() to __istappend()
- CLEANUP: htx: Make http_get_stline take a `const struct`
- CLEANUP: ist: Remove unused `count` argument from `ist2str*`
- CLEANUP: Remove useless malloc() casts
This patch re-works configuration parsing, it removes the "server"
lines from "resolvers" sections introduced in commit 56fc5d9eb:
MEDIUM: resolvers: add supports of TCP nameservers in resolvers.
It also extends the nameserver lines to support stream server
addresses such as:
resolvers
nameserver localhost tcp@127.0.0.1:53
Doing so, a part of nameserver's init code was factorized in
function 'parse_resolvers' and removed from 'post_parse_resolvers'.
An explicit stream address prefix such as "tcp6@" "tcp4@"
"stream+ipv6@" "stream+ipv4@" or "stream+unix@" will
allocate an implicit ring buffer with a forward server
targeting the given address.
This is usefull to simply send logs to a log server in tcp
and It doesn't need to declare a ring section in configuration.
Since the internal function str2sa_range is used to addresses
for different objects ('server', 'bind' but also 'log' or
'nameserver') we notice that some combinations are missing.
"ip@" is introduced to authorize the prefix "dgram+ip@" or
"stream+ip@" which dectects automatically IP version but
specify dgram or stream.
"tcp@" was introduced and is an alias for "stream+ip@".
"tcp6" and "tcp4" are now aliases for "stream+ipv6@" and
"stream+ipv4@".
"uxst@" and "uxdg@" are now aliases for "stream+unix@" and
"dgram+unix@".
This patch also adds a complete section in documentation to
describe adresses and their prefixes.
HTTP_2.0 predefined macro returns true for HTTP/2 requests. HTTP/2 doen't
convey a version information, so this macro may seem a bit strange. But for
compatiblity reasons, internally, the "HTTP/2.0" version is set. Thus, it is
handy to rely on it to differenciate HTTP/1 and HTTP/2 requests.
Some predefined ACLs were still based on deprecated sample fetches, like
req_proto_http or req_ver. Now, they use non-deprecated sample fetches. In
addition, the usage lines in the configuration manual have been updated to
be more explicit.
Released version 2.4-dev15 with the following main changes :
- BUG/MINOR: payload: Wait for more data if buffer is empty in payload/payload_lv
- BUG/MINOR: stats: Apply proper styles in HTML status page.
- BUG/MEDIUM: time: make sure to always initialize the global tick
- BUG/MINOR: tcp: fix silent-drop workaround for IPv6
- BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS
- CLEANUP: socket: replace SOL_IP/IPV6/TCP with IPPROTO_IP/IPV6/TCP
- BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields
- BUG/MINOR: mux-h2: Don't emit log twice if an error occurred on the preface
- MINOR: stream: Don't trigger errors on destructive HTTP upgrades
- MINOR: frontend: Create HTTP txn for HTX streams
- MINOR: stream: Be sure to set HTTP analysers when creating an HTX stream
- BUG/MINOR: stream: Properly handle TCP>H1>H2 upgrades in http_wait_for_request
- BUG/MINOR: config: Add warning for http-after-response rules in TCP mode
- MINOR: muxes: Add a flag to notify a mux does not support any upgrade
- MINOR: mux-h1: Don't perform implicit HTTP/2 upgrade if not supported by mux
- MINOR: mux-pt: Don't perform implicit HTTP upgrade if not supported by mux
- MEDIUM: mux-h1: Expose h1 in the list of supported mux protocols
- MEDIUM: mux-pt: Expose passthrough in the list of supported mux protocols
- MINOR: muxes: Show muxes flags when the mux list is displayed
- DOC: config: Improve documentation about proto/check-proto keywords
- MINOR: stream: Use stream type instead of proxy mode when appropriate
- MINOR: filters/http-ana: Decide to filter HTTP headers in HTTP analysers
- MINOR: http-ana: Simplify creation/destruction of HTTP transactions
- MINOR: stream: Handle stream HTTP upgrade in a dedicated function
- MEDIUM: Add tcp-request switch-mode action to perform HTTP upgrade
- MINOR: config/proxy: Don't warn for HTTP rules in TCP if 'switch-mode http' set
- MINOR: config/proxy: Warn if a TCP proxy without backend is upgradable to HTTP
- DOC: config: Add documentation about TCP to HTTP upgrades
- REGTESTS: Add script to tests TCP to HTTP upgrades
- BUG/MINOR: payload/htx: Ingore L6 sample fetches for HTX streams/checks
- MINOR: htx: Make internal.strm.is_htx an internal sample fetch
- MINOR: action: Use a generic function to check validity of an action rule list
- MINOR: payload/config: Warn if a L6 sample fetch is used from an HTTP proxy
- MEDIUM: http-rules: Add wait-for-body action on request and response side
- REGTESTS: Add script to tests the wait-for-body HTTP action
- BUG/MINOR: http-fetch: Fix test on message state to capture the version
- CLEANUP: vars: always pre-initialize smp in vars_parse_cli_get_var()
- MINOR: global: define diagnostic mode of execution
- MINOR: cfgparse: diag for multiple nbthread statements
- MINOR: server: diag for 0 weight server
- MINOR: diag: create cfgdiag module
- MINOR: diag: diag if servers use the same cookie value
- MINOR: config: diag if global section after non-global
- TESTS: slightly reorganize the code in the tests/ directory
- TESTS: move tests/*.cfg to tests/config
- REGTESTS: ssl: "set ssl cert" and multi-certificates bundle
- REGTESTS: ssl: mark set_ssl_cert_bundle.vtc as broken
- CONTRIB: halog: fix issue with array of type char
- CONTRIB: tcploop: add a shutr command
- CONTRIB: debug: add the show-fd-to-flags script
- CONTRIB: debug: split poll from flags
- CONTRIB: move some dev-specific tools to dev/
- BUILD: makefile: always build the flags utility
- DEV: flags: replace the unneeded makefile with a README
- BUILD: makefile: integrate the hpack tools
- CONTRIB: merge ip6range with iprange
- CONTRIB: move some admin-related sub-projects to admin/
- CONTRIB: move halog to admin/
- ADMIN: halog: automatically enable USE_MEMCHR on the right glibc version
- BUILD: makefile: build halog with the correct flags
- BUILD: makefile: add a "USE_PROMEX" variable to ease building prometheus-exporter
- CONTRIB: move prometheus-exporter to addons/promex
- DOC: add a few words about USE_* and the addons directory
- CONTRIB: move 51Degrees to addons/51degrees
- CONTRIB: move src/da.c and contrib/deviceatlas to addons/deviceatlas
- CONTRIB: move src/wurfl.c and contrib/wurfl to addons/wurfl
- CONTRIB: move contrib/opentracing to addons/ot
- BUG/MINOR: opentracing: initialization after establishing daemon mode
- DOC: clarify that compression works for HTTP/2
This patch clarifies that compression also works with HTTP/2. I have
picked the wording "HTTP/1.1 or above" because it is already used
elsewhere in the documentation.
I have tested that compression indeed works in HTTP/2.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
This one is the last optional module to build with haproxy, so let's move
it to addons/. It was renamed to "ot" as it was the only one whose USE_*
option did not match the directory name, now this is consistent.
Few changes were required, only the Makefile, and doc were adjusted, as
the directory was already self-contained and relocatable.
Both the source file and the dummy library are now at the same place.
Maybe the build howto could be moved there as well to make things even
cleaner.
The Makefile, MAINTAINERS, doc, and vtest matrix were updated.
Both the source file and the dummy library are now at the same place.
Maybe the build howto could be moved there as well to make things even
cleaner.
The Makefile, MAINTAINERS, doc, github build matrix, coverity checks
and travis CI's build were updated.
Now it's much cleaner, both 51d.c and the dummy library live together and
are easier to spot and maintain. The build howto probably ought to be moved
there as well. Makefile, docs and MAINTAINERS were updated, as well as
the github CI's build matrix, travis CI's, and coverity checks.
Define MODE_DIAG which is used to run haproxy in diagnostic mode. This
mode is used to output extra warnings about possible configuration
blunder or sub-optimal usage. It can be activated with argument '-dD'.
A new output function ha_diag_warning is implemented reserved for
diagnostic output. It serves to standardize the format of diagnostic
messages.
A macro HA_DIAG_WARN_COND is also available to automatically check if
diagnostic mode is on before executing the diagnostic check.
Historically, an option was added to wait for the request payload (option
http-buffer-request). This option has 2 drawbacks. First, it is an ON/OFF
option for the whole proxy. It cannot be enabled on demand depending on the
message. Then, as its name suggests, it only works on the request side. The
only option to wait for the response payload was to write a dedicated
filter. While it is an acceptable solution for complex applications, it is a
bit overkill to simply match strings in the body.
To make everyone happy, this patch adds a dedicated HTTP action to wait for
the message payload, for the request or the response depending it is used in
an http-request or an http-response ruleset. The time to wait is
configurable and, optionally, the minimum payload size to have before stop
to wait.
Both the http action and the old http analyzer rely on the same internal
function.
Use a L6 sample fetch on an HTX streams or a HTX health-check is meaningless
because data are not raw but structured. So now, these sample fetches fail
when called from an HTTP proxy. In addition, a warning has been added in the
configuration manual, at the begining of the L6 sample fetches section.
Note that req.len and res.len samples return the HTX data size instead of
failing. It is not accurate because it does not reflect the buffer size nor
the raw data length. But we keep it for backward compatibility purpose.
However it remains a bit strange to use it on an HTTP proxy.
This patch may be backported to all versions supporting the HTX, i.e as far
as 2.0. But the part about the health-checks is only valid for the 2.2 and
upper.
This patch adds explanation about chaining a TCP frontend to an HTTP
backend. It also explain how the HTTP upgrades work in this context. A note
has also been added in "Fetching HTTP samples" section to warning about HTTP
content processing in TCP.
It is now possible to perform HTTP upgrades on a TCP stream from the
frontend side. To do so, a tcp-request content rule must be defined with the
switch-mode action, specifying the mode (for now, only http is supported)
and optionnaly the proto (h1 or h2).
This way it could be possible to set HTTP directives on a TCP frontend which
will only be evaluated if an upgrade is performed. This new way to perform
HTTP upgrades should replace progressively the old way, consisting to route
the request to an HTTP backend. And it should be also a good start to remove
all HTTP processing from tcp-request content rules.
This action is terminal, it stops the ruleset evaluation. It is only
available on proxy with the frontend capability.
The configuration manual has been updated accordingly.
This patch adds a description about information provided by "haproxy -vv"
command regarding the available protocols. The description is adapted
depending the context (bind line, server line or health-check).
Released version 2.4-dev14 with the following main changes :
- MEDIUM: quic: Fix build.
- MEDIUM: quic: Fix build.
- CI: codespell: whitelist "Dragan Dosen"
- CLEANUP: assorted typo fixes in the code and comments
- CI: github actions: update LibreSSL to 3.2.5
- REGTESTS: revert workaround for a crash with recent libressl on http-reuse sni
- CLEANUP: mark defproxy as const on parse tune.fail-alloc
- REGTESTS: remove unneeded experimental-mode in cli add server test
- REGTESTS: wait for proper return of enable server in cli add server test
- MINOR: compression: use pool_alloc(), not pool_alloc_dirty()
- MINOR: spoe: use pool_alloc(), not pool_alloc_dirty()
- MINOR: fcgi-app: use pool_alloc(), not pool_alloc_dirty()
- MINOR: cache: use pool_alloc(), not pool_alloc_dirty()
- MINOR: ssl: use pool_alloc(), not pool_alloc_dirty()
- MINOR: opentracing: use pool_alloc(), not pool_alloc_dirty()
- MINOR: dynbuf: make b_alloc() always check if the buffer is allocated
- CLEANUP: compression: do not test for buffer before calling b_alloc()
- CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
- MINOR: channel: simplify the channel's buffer allocation
- MEDIUM: dynbuf: remove last usages of b_alloc_margin()
- CLEANUP: dynbuf: remove b_alloc_margin()
- CLEANUP: dynbuf: remove the unused b_alloc_fast() function
- CLEANUP: pools: remove the unused pool_get_first() function
- MINOR: pools: make the pool allocator support a few flags
- MINOR: pools: add pool_zalloc() to return a zeroed area
- CLEANUP: connection: use pool_zalloc() in conn_alloc_hash_node()
- CLEANUP: filters: use pool_zalloc() in flt_stream_add_filter()
- CLEANUP: spoe: use pool_zalloc() instead of pool_alloc+memset
- CLEANUP: frontend: use pool_zalloc() in frontend_accept()
- CLEANUP: mailers: use pool_zalloc() in enqueue_one_email_alert()
- CLEANUP: resolvers: use pool_zalloc() in resolv_link_resolution()
- CLEANUP: ssl: use pool_zalloc() in ssl_init_keylog()
- CLEANUP: tcpcheck: use pool_zalloc() instead of pool_alloc+memset
- CLEANUP: quic: use pool_zalloc() instead of pool_alloc+memset
- MINOR: time: also provide a global, monotonic global_now_ms timer
- BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable
- MINOR: tools: introduce new option PA_O_DEFAULT_DGRAM on str2sa_range.
- BUILD: tools: fix build error with new PA_O_DEFAULT_DGRAM
- BUG/MINOR: ssl: Prevent disk access when using "add ssl crt-list"
- CLEANUP: ssl: remove unused definitions
- BUILD: ssl: guard ecdh functions with SSL_CTX_set_tmp_ecdh macro
- MINOR: lua: Slightly improve function dumping the lua traceback
- BUG/MEDIUM: debug/lua: Use internal hlua function to dump the lua traceback
- BUG/MEDIUM: lua: Always init the lua stack before referencing the context
- MINOR: fd: make fd_clr_running() return the remaining running mask
- MINOR: fd: remove the unneeded running bit from fd_insert()
- BUG/MEDIUM: fd: do not wait on FD removal in fd_delete()
- CLEANUP: fd: remove unused fd_set_running_excl()
- CLEANUP: fd: slightly simplify up _fd_delete_orphan()
- BUG/MEDIUM: fd: Take the fd_mig_lock when closing if no DWCAS is available.
- BUG/MEDIUM: release lock on idle conn killing on reached pool high count
- BUG/MEDIUM: thread: Fix a deadlock if an isolated thread is marked as harmless
- MINOR: tools: make url2ipv4 return the exact number of bytes parsed
- BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters
- BUG/MEDIUM: mux-h1: make h1_shutw_conn() idempotent
- BUG/MINOR: ssl: Fix update of default certificate
- BUG/MINOR: ssl: Prevent removal of crt-list line if the instance is a default one
- BUILD: ssl: introduce fine guard for ssl random extraction functions
- REORG: global: move initcall register code in a dedicated file
- REORG: global: move free acl/action in their related source files
- REORG: split proxy allocation functions
- MINOR: proxy: implement a free_proxy function
- MINOR: proxy: define cap PR_CAP_LUA
- MINOR: lua: properly allocate the lua Socket proxy
- MINOR: lua: properly allocate the lua Socket servers
- MINOR: vars: make get_vars() allow the session to be null
- MINOR: vars: make the var() sample fetch keyword depend on nothing
- CLEANUP: sample: remove duplicate "stopping" sample fetch keyword
- MINOR: sample: make smp_resolve_args() return an allocate error message
- MINOR: sample: add a new SMP_SRC_CONST sample capability
- MINOR: sample: mark the truly constant sample fetch keywords as such
- MINOR: sample: add a new CFG_PARSER context for samples
- MINOR: action: add a new ACT_F_CFG_PARSER origin designation
- MEDIUM: vars: add support for a "set-var" global directive
- REGTESTS: add a basic reg-test for some "set-var" commands
- MINOR: sample: add a new CLI_PARSER context for samples
- MINOR: action: add a new ACT_F_CLI_PARSER origin designation
- MINOR: vars/cli: add a "get var" CLI command to retrieve global variables
- MEDIUM: cli: add a new experimental "set var" command
- MINOR: compat: add short aliases for a few very commonly used types
- BUILD: ssl: use EVP_CIPH_GCM_MODE macro instead of HA_OPENSSL_VERSION
- MEDIUM: backend: use a trylock to grab a connection on high FD counts as well
set var <name> <expression>
Allows to set or overwrite the process-wide variable 'name' with the result
of expression <expression>. Only process-wide variables may be used, so the
name must begin with 'proc.' otherwise no variable will be set. The
<expression> may only involve "internal" sample fetch keywords and converters
even though the most likely useful ones will be str('something') or int().
Note that the command line parser doesn't know about quotes, so any space in
the expression must be preceeded by a backslash. This command requires levels
"operator" or "admin". This command is only supported on a CLI connection
running in experimental mode (see "experimental-mode on").
Just like for "set-var" in the global section, the command uses a temporary
dummy proxy to create a temporary "set-var(name)" rule to assign the value.
The reg test was updated to verify that an updated global variable is properly
reflected in subsequent HTTP responses.
Process-wide variables can now be displayed from the CLI using "get var"
followed by the variable name. They must all start with "proc." otherwise
they will not be found. The output is very similar to the one of the
debug converter, with a type and value being reported for the embedded
sample.
This command is limited to clients with the level "operator" or higher,
since it can possibly expose traffic-related data.
While we do support process-wide variables ("proc.<name>"), there was
no way to preset them from the configuration. This was particularly
limiting their usefulness since configs involving them always had to
first check if the variable was set prior to performing an operation.
This patch adds a new "set-var" directive in the global section that
supports setting the proc.<name> variables from an expression, like
other set-var actions do. The syntax however follows what is already
being done for setenv, which consists in having one argument for the
variable name and another one for the expression.
Only "constant" expressions are allowed here, such as "int", "str"
etc, combined with arithmetic or string converters, and variable
lookups. A few extra sample fetch keywords like "date", "rand" and
"uuid" are also part of the constant expressions and may make sense
to allow to create a random key or differentiate processes.
The way it was done consists in parsing a dummy rule an executing the
expression in the CFG_PARSE context, then releasing the expression.
This is safe because the sample that variables store does not hold a
back pointer to expression that created them.
The hdr_ip() sample fetch function will try to extract IP addresses
from a header field. These IP addresses are parsed using url2ipv4()
and if it fails it will fall back to inet_pton(AF_INET6), otherwise
will fail.
There is a small problem there which is that if a field starts with
an IP address and is immediately followed by some garbage, the IP
address part is still returned. This is a problem with fields such
as x-forwarded-for because it prevents detection of accidental
corruption or bug along the chain. For example, the following string:
x-forwarded-for: 1.2.3.4; 5.6.7.8
or this one:
x-forwarded-for: 1.2.3.4O ( the last one being the letter 'O')
would still return "1.2.3.4" despite the trailing characters. This is
bad because it will silently cover broken code running on intermediary
proxies and may even in some cases allow haproxy to pass improperly
formatted headers after they were apparently validated, for example,
if someone extracts the address from this field to place it into
another one.
This issue would only affect the IPv4 parser, because the IPv6 parser
already uses inet_pton() which fails at the first invalid character and
rejects trailing port numbers.
In strict compliance with RFC7239, let's make sure that if there are any
characters left in the string, the parsing fails and makes hdr_ip()
return nothing. However, a special case has to be handled to support
IPv4 addresses followed by a colon and a valid port number, because till
now the parser used to implicitly accept them and it appears that this
practice, though rare, does exist at least in Azure:
https://docs.microsoft.com/en-us/azure/application-gateway/how-application-gateway-works
This issue has always been there so the fix may be backported to all
versions. It will need the following commit in order to work as expected:
MINOR: tools: make url2ipv4 return the exact number of bytes parsed
Many thanks to https://twitter.com/melardev and the BitMEX Security Team
for their detailed report.
It is never used anymore since 1.7 where it was used by b_alloc_margin()
then replaced by direct calls to the pools function, and it maintains a
dependency on the exposed pools functions. It's time to get rid of it,
as it's not even certain it still works.
Right now there is a discrepancy beteween b_alloc() and b_allow_margin():
the former forcefully overwrites the target pointer while the latter tests
it and returns it as-is if already allocated.
As a matter of fact, all callers of b_alloc() either preliminary test the
buffer, or assume it's already null.
Let's remove this pain and make the function test the buffer's allocation
before doing it again, and match call places' expectations.
Released version 2.4-dev13 with the following main changes :
- BUG/MEDIUM: cli: fix "help" crashing since recent spelling fixes
- BUG/MINOR: cfgparse: use the GLOBAL not LISTEN keywords list for spell checking
- MINOR: tools: improve word fingerprinting by counting presence
- MINOR: tools: do not sum squares of differences for word fingerprints
- MINOR: cli: improve fuzzy matching to work on all remaining words at once
- MINOR: cli: sort the suggestions by order of relevance
- MINOR: cli: limit spelling suggestions to 5
- MINOR: cfgparse/proxy: also support spelling fixes on options
- BUG/MINOR: resolvers: Add missing case-insensitive comparisons of DNS hostnames
- MINOR: time: export the global_now variable
- BUG/MINOR: freq_ctr/threads: make use of the last updated global time
- MINOR: freq_ctr/threads: relax when failing to update a sliding window value
- MINOR/BUG: mworker/cli: do not use the unix_bind prefix for the master CLI socket
- MINOR: mworker/cli: alert the user if we enabled a master CLI but not the master-worker mode
- MINOR: cli: implement experimental-mode
- REORG: server: add a free server function
- MINOR: cfgparse: always alloc idle conns task
- REORG: server: move keywords in srv_kws
- MINOR: server: remove fastinter from mistyped kw list
- REORG: server: split parse_server
- REORG: server: move alert traces in parse_server
- REORG: server: rename internal functions from parse_server
- REORG: server: attach servers in parse_server
- REORG: server: use flags for parse_server
- MINOR: server: prepare parsing for dynamic servers
- MINOR: stats: export function to allocate extra proxy counters
- MEDIUM: server: implement 'add server' cli command
- REGTESTS: implement test for 'add server' cli
- MINOR: server: enable standard options for dynamic servers
- MINOR: server: support keyword proto in 'add server' cli
- BUG/MINOR: protocol: add missing support of dgram unix socket.
- CLEANUP: Fix a typo in fix_is_valid description
- MINOR: raw_sock: Add a close method.
- MEDIUM: connections: Introduce a new XPRT method, start().
- MEDIUM: connections: Implement a start() method for xprt_handshake.
- MEDIUM: connections: Implement a start() method in ssl_sock.
- MINOR: muxes: garbage collect the reset() method.
- CLEANUP: tcp-rules: Fix a typo in error messages about expect-netscaler-cip
- MEDIUM: lua: Use a per-thread counter to track some non-reentrant parts of lua
- BUG/MEDIUM: debug/lua: Don't dump the lua stack if not dumpable
Allow to specify the mux proto for a dynamic server. It must be
compatible with the backend mode to be accepted. The reg-tests has been
extended for this error case.
Enable a subset of server options to be used as keywords on the CLI
command 'add server'. These options are safe and can be applied
flawlessly for a dynamic server.
Add a new cli command 'add server'. This command is used to create a new
server at runtime attached on an existing backend. The syntax is the
following one :
$ add server <be_name>/<sv_name> [<kws>...]
This command is only available through experimental mode for the moment.
Currently, no server keywords are supported. They will be activated
individually when deemed properly functional and safe.
Another limitation is put on the backend load-balancing algorithm. The
algorithm must use consistent hashing to guarantee a minimal
reallocation of existing connections on the new server insertion.
Released version 2.4-dev12 with the following main changes :
- CLEANUP: connection: Use `VAR_ARRAY` in `struct tlv` definition
- CLEANUP: connection: Remove useless test for NULL before calling `pool_free()`
- CLEANUP: connection: Use istptr / istlen for proxy_unique_id
- MINOR: connection: Use a `struct ist` to store proxy_authority
- CLEANUP: connection: Consistently use `struct ist` to process all TLV types
- BUILD: task: fix build at -O0 with threads disabled
- BUILD: bug: refine HA_LINK_ERROR() to only be used on gcc and derivatives
- CLEANUP: config: make the cfg_keyword parsers take a const for the defproxy
- BUILD: connection: do not use VAR_ARRAY in struct tlv
- BUG/MEDIUM: session: NULL dereference possible when accessing the listener
- MINOR: build: force CC to set a return code when probing options
- CLEANUP: stream: rename a few remaining occurrences of "stream *sess"
- BUG/MEDIUM: resolvers: handle huge responses over tcp servers.
- CLEANUP: config: also address the cfg_keyword API change in the compression code
- BUG/MEDIUM: ssl: properly remove the TASK_HEAVY flag at end of handshake
- BUG/MINOR: sample: Rename SenderComID/TargetComID to SenderCompID/TargetCompID
- MINOR: task: give the scheduler a bit more flexibility in the runqueue size
- OPTIM: task: automatically adjust the default runqueue-depth to the threads
- BUG/MINOR: connection: Missing QUIC initialization
- BUG/MEDIUM: stick-tables: fix ref counter in table entry using multiple http tracksc.
- BUILD: atomic/arm64: force the register pairs to use in __ha_cas_dw()
- BUG/MEDIUM: filters: Set CF_FL_ANALYZE on channels when filters are attached
- BUG/MINOR: tcpcheck: Update .health threshold of agent inside an agent-check
- BUG/MINOR: proxy/session: Be sure to have a listener to increment its counters
- BUG/MINOR: tcpcheck: Fix double free on error path when parsing tcp/http-check
- BUG/MINOR: server-state: properly handle the case where the base is not set
- BUG/MINOR: server-state: use the argument, not the global state
- CLEANUP: tcp-rules: add missing actions in the tcp-request error message
- CLEANUP: vars: make the error message clearer on missing arguments for set-var
- CLEANUP: http-rules: remove the unexpected comma before the list of action keywords
- CLEANUP: actions: the keyword must always be const from the rule
- MINOR: tools: add simple word fingerprinting to find similar-looking words
- MINOR: cfgparse: add cfg_find_best_match() to suggest an existing word
- MINOR: cfgparse: suggest correct spelling for unknown words in proxy sections
- MINOR: cfgparse: suggest correct spelling for unknown words in global section
- MINOR: cfgparse/server: try to fix spelling mistakes on server lines
- MINOR: cfgparse/bind: suggest correct spelling for unknown bind keywords
- MINOR: actions: add a function to suggest an action ressembling a given word
- MINOR: http-rules: suggest approaching action names on mismatch
- MINOR: tcp-rules: suggest approaching action names on mismatch
- BUG/MINOR: cfgparse/server: increment the extra keyword counter one at a time
- Revert "BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record"
- BUG/MINOR: resolvers: Consider server to have no IP on DNS resolution error
- BUG/MINOR: resolvers: Reset server address on DNS error only on status change
- BUG/MINOR: resolvers: Unlink DNS resolution to set RMAINT on SRV resolution
- BUG/MEDIUM: resolvers: Don't set an address-less server as UP
- BUG/MEDIUM: resolvers: Fix the loop looking for an existing ADD item
- MINOR: resolvers: new function find_srvrq_answer_record()
- BUG/MINOR; resolvers: Ignore DNS resolution for expired SRV item
- BUG/MEDIUM: resolvers: Trigger a DNS resolution if an ADD item is obsolete
- MINOR: resolvers: Use a function to remove answers attached to a resolution
- MINOR: resolvers: Purge answer items when a SRV resolution triggers an error
- MINOR: resolvers: Add function to change the srv status based on SRV resolution
- MINOR: resolvers: Directly call srvrq_update_srv_state() when possible
- BUG/MEDIUM: resolvers: Don't release resolution from a requester callbacks
- BUG/MEDIUM: resolvers: Skip DNS resolution at startup if SRV resolution is set
- MINOR: resolvers: Use milliseconds for cached items in resolver responses
- MINOR: resolvers: Don't try to match immediatly renewed ADD items
- CLEANUP: resolvers: Use ha_free() in srvrq_resolution_error_cb()
- CLEANUP: resolvers: Perform unsafe loop on requester list when possible
- BUG/MINOR: cli: make sure "help", "prompt", "quit" are enabled at master level
- CLEANUP: cli: fix misleading comment and better indent the access level flags
- MINOR: cli: set the ACCESS_MASTER* bits on the master bind_conf
- MINOR: cli: test the appctx level for master access instead of comparing pointers
- MINOR: cli: print the error message in the parser function itself
- MINOR: cli: filter the list of commands to the matching part
- MEDIUM: cli: apply spelling fixes for known commands before listing them
- MINOR: tools: add the ability to update a word fingerprint
- MINOR: cli: apply the fuzzy matching on the whole command instead of words
- CLEANUP: cli: rename MAX_STATS_ARGS to MAX_CLI_ARGS
- CLEANUP: cli: rename the last few "stats_" to "cli_"
- CLEANUP: task: make sure tasklet handlers always indicate their statuses
- CLEANUP: assorted typo fixes in the code and comments
The recent default runqueue size reduction appeared to have significantly
lowered performance on low-thread count configs. Testing various values
runqueue values on different workloads under thread counts ranging from
1 to 64, it appeared that lower values are more optimal for high thread
counts and conversely. It could even be drawn that the optimal value for
various workloads sits around 280/sqrt(nbthread), and probably has to do
with both the L3 cache usage and how to optimally interlace the threads'
activity to minimize contention. This is much easier to optimally
configure, so let's do this by default now.
The recently introduced Financial Information eXchange (FIX)
converters have some hard coded tags based on the specification that
were misspelled. Specifically, SenderComID and TargetComID should
be SenderCompID and TargetCompID according to the specification [1][2].
This patch updates all references, which includes the converters
themselves, the regression test, and the documentation.
[1] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag49.html
[2] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag56.html
Parameter "accepted_payload_size" is currently considered regardless
the used nameserver is using TCP or UDP. It remains mandatory to annouce
such capability to support e-dns, so a value have to be announced also
in TCP. Maximum DNS message size in TCP is limited by protocol to 65535
and so for UDP (65507) if system supports such UDP messages. But
the maximum value for this option was arbitrary forced to 8192.
This patch change this maximum to 65535 to allow user to set bigger value
for UDP if its system supports. It also sets accepted_payload_size
in TCP allowing to retrieve huge responses if the configuration uses
TCP nameservers.
The request announcing the accepted_payload_size capability is currently
built at resolvers level and is common to all used nameservers of the
section regardess transport protocol used. A further patch should be
made to at least specify a different payload size depending of the
transport, and perhaps could be forced to 65535 in case of TCP and
maximum would be forced back to 65507 matching UDP max.
This patch is appliable since 2.4 version
Released version 2.4-dev11 with the following main changes :
- CI: codespell: skip Makefile for spell check
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: tcp-act: Don't forget to set the original port for IPv4 set-dst rule
- BUG/MINOR: connection: Use the client's dst family for adressless servers
- BUG/MEDIUM: spoe: Kill applets if there are pending connections and nbthread > 1
- CLEANUP: Use ist2(const void*, size_t) whenever possible
- CLEANUP: Use IST_NULL whenever possible
- BUILD: proxy: Missing header inclusion for quic_transport_params_init()
- BUILD: quic: Implicit conversion between SSL related enums.
- DOC: spoe: Add a note about fragmentation support in HAProxy
- MINOR: contrib: add support for heartbeat control messages.
- MINOR: contrib: Enhance peers dissector heuristic.
- BUG/MINOR: mux-h2: Fix typo in scheme adjustment
- CLEANUP: Reapply the ist2() replacement patch
- CLEANUP: Use istadv(const struct ist, const size_t) whenever possible
- CLEANUP: Use isttest(const struct ist) whenever possible
- Revert "CI: Pin VTest to a known good commit"
- CLEANUP: backend: fix a wrong comment
- BUG/MINOR: backend: free allocated bind_addr if reuse conn
- MINOR: backend: handle reuse for conns with no server as target
- REGTESTS: test http-reuse if no server target
- BUG/MINOR: hlua: Don't strip last non-LWS char in hlua_pushstrippedstring()
- BUG/MINOR: server-state: Don't load server-state file for disabled backends
- CLEANUP: dns: Use DISGUISE() on a never-failing ring_attach() call
- CLEANUP: dns: Remove useless test on ns->dgram in dns_connect_nameserver()
- DOC: fix originalto except clause on destination address
- CLEANUP: Use the ist() macro whenever possible
- CLEANUP: Replace for loop with only a condition by while
- REORG: atomic: reimplement pl_cpu_relax() from atomic-ops.h
- BUG/MINOR: mt-list: always perform a cpu_relax call on failure
- MINOR: atomic: add armv8.1-a atomics variant for cas-dw
- MINOR: atomic: implement a more efficient arm64 __ha_cas_dw() using pairs
- BUG/MINOR: ssl: don't truncate the file descriptor to 16 bits in debug mode
- MEDIUM: pools: add CONFIG_HAP_NO_GLOBAL_POOLS and CONFIG_HAP_GLOBAL_POOLS
- MINOR: pools: double the local pool cache size to 1 MB
- MINOR: stream: use ABORT_NOW() and not abort() in stream_dump_and_crash()
- CLEANUP: stream: explain why we queue the stream at the head of the server list
- MEDIUM: backend: use a trylock when trying to grab an idle connection
- REORG: tools: promote the debug PRNG to more general use as a statistical one
- OPTIM: lb-random: use a cheaper PRNG to pick a server
- MINOR: task: stop abusing the nice field to detect a tasklet
- MINOR: task: move the nice field to the struct task only
- MEDIUM: task: extend the state field to 32 bits
- MINOR: task: add an application specific flag to the state: TASK_F_USR1
- MEDIUM: muxes: mark idle conns tasklets with TASK_F_USR1
- MINOR: xprt: add new xprt_set_idle and xprt_set_used methods
- MEDIUM: ssl: implement xprt_set_used and xprt_set_idle to relax context checks
- MINOR: server: don't read curr_used_conns multiple times
- CLEANUP: global: reorder some fields to respect cache lines
- CLEANUP: sockpair: silence a coverity check about fcntl()
- CLEANUP: lua: set a dummy file name and line number on the dummy servers
- MINOR: server: add a global list of all known servers
- MINOR: cfgparse: finish to set up servers outside of the proxy setup loop
- MINOR: server: allocate a per-thread struct for the per-thread connections stuff
- MINOR: server: move actconns to the per-thread structure
- CLEANUP: server: reorder some fields in the server struct to respect cache lines
- MINOR: backend: add a BUG_ON if conn mux NULL in connect_server
- BUG/MINOR: backend: fix condition for reuse on mode HTTP
- BUILD: Fix build when using clang without optimizing.
- CLEANUP: assorted typo fixes in the code and comments
Fix the description of the except clause of the originalto option. The
destination address and not the source is compared with the except range
address to prevent the addition of the X-Original-To header.
This can be backported in every releases.
Add a note in SPOE.txt to make it clear that HAPRoxy does not support the
fragmentation. It can send fragmented frames if an agent supports it but it
cannot receives and handles fragmented frames.
This patch should fix the issue #659. It may be backported as far as 1.8.
Released version 2.4-dev10 with the following main changes :
- BUILD: SSL: introduce fine guard for RAND_keep_random_devices_open
- MINOR: Configure the `cpp` userdiff driver for *.[ch] in .gitattributes
- BUG/MINOR: ssl/cli: potential null pointer dereference in "set ssl cert"
- BUG/MINOR: sample: secure convs that accept base64 string and var name as args
- BUG/MEDIUM: vars: make functions vars_get_by_{name,desc} thread-safe
- CLEANUP: vars: make smp_fetch_var() to reuse vars_get_by_desc()
- DOC: muxes: add a diagram of the exchanges between muxes and outer world
- BUG/MEDIUM: proxy: use thread-safe stream killing on hard-stop
- BUG/MEDIUM: cli/shutdown sessions: make it thread-safe
- BUG/MINOR: proxy: wake up all threads when sending the hard-stop signal
- MINOR: stream: add an "epoch" to figure which streams appeared when
- MINOR: cli/streams: make "show sess" dump all streams till the new epoch
- MINOR: streams: use one list per stream instead of a global one
- MEDIUM: streams: do not use the streams lock anymore
- BUILD: dns: avoid a build warning when threads are disabled (dss unused)
- MEDIUM: task: remove the tasks_run_queue counter and have one per thread
- MINOR: tasks: do not maintain the rqueue_size counter anymore
- CLEANUP: tasks: use a less confusing name for task_list_size
- CLEANUP: task: move the tree root detection from __task_wakeup() to task_wakeup()
- MINOR: task: limit the remote thread wakeup to the global runqueue only
- MINOR: task: move the allocated tasks counter to the per-thread struct
- CLEANUP: task: split the large tasklet_wakeup_on() function in two
- BUG/MINOR: fd: properly wait for !running_mask in fd_set_running_excl()
- BUG/MINOR: resolvers: Fix condition to release received ARs if not assigned
- BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record
- BUG/MINOR: resolvers: new callback to properly handle SRV record errors
- BUG/MEDIUM: resolvers: Reset server address and port for obselete SRV records
- BUG/MEDIUM: resolvers: Reset address for unresolved servers
- DOC: Update the module list in MAINTAINERS file
- MINOR: htx: Add function to reserve the max possible size for an HTX DATA block
- DOC: Update the HTX API documentation
- DOC: Update the filters guide
- BUG/MEDIUM: contrib/prometheus-exporter: fix segfault in listener name dump
- MINOR: task: split the counts of local and global tasks picked
- MINOR: task: do not use __task_unlink_rq() from process_runnable_tasks()
- MINOR: task: don't decrement then increment the local run queue
- CLEANUP: task: re-merge __task_unlink_rq() with task_unlink_rq()
- MINOR: task: make grq_total atomic to move it outside of the grq_lock
- MINOR: tasks: also compute the tasklet latency when DEBUG_TASK is set
- MINOR: task: make tasklet wakeup latency measurements more accurate
- MINOR: server: Be more strict on the server-state line parsing
- MINOR: server: Only fill one array when parsing a server-state line
- MEDIUM: server: Refactor apply_server_state() to make it more readable
- CLEANUP: server: Rename state_line node to node instead of name_name
- CLEANUP: server: Rename state_line structure into server_state_line
- CLEANUP: server: Use a local eb-tree to store lines of the global server-state file
- MINOR: server: Be more strict when reading the version of a server-state file
- MEDIUM: server: Store parsed params of a server-state line in the tree
- MINOR: server: Remove cached line from global server-state tree when found
- MINOR: server: Move loading state of servers in a dedicated function
- MEDIUM: server: Use a tree to store local server-state lines
- MINOR: server: Parse and store server-state lines in a dedicated function
- MEDIUM: server: Don't load server-state file if a line is corrupted
- REORG: server: Export and rename some functions updating server info
- REORG: server-state: Move functions to deal with server-state in its own file
- MINOR: server-state: Don't load server-state file for serverless proxies
- CLEANUP: muxes: Remove useless if condition in show_fd function
- BUG/MINOR: stats: fix compare of no-maint url suffix
- MINOR: task: limit the number of subsequent heavy tasks with flag TASK_HEAVY
- MINOR: ssl: mark the SSL handshake tasklet as heavy
- CLEANUP: server: rename srv_cleanup_{idle,toremove}_connections()
- BUG/MINOR: ssl: potential null pointer dereference in ckchs_dup()
- MINOR: task: add one extra tasklet class: TL_HEAVY
- MINOR: task: place the heavy elements in TL_HEAVY
- MINOR: task: only limit TL_HEAVY tasks but not others
- BUG/MINOR: http-ana: Only consider dst address to process originalto option
- MINOR: tools: Add net_addr structure describing a network addess
- MINOR: tools: Add function to compare an address to a network address
- MEDIUM: http-ana: Add IPv6 support for forwardfor and orignialto options
- CLEANUP: hlua: Use net_addr structure internally to parse and compare addresses
- REGTESTS: Add script to test except param for fowardedfor/originalto options
- DOC: scheduler: add a diagram showing the different queues and their usages
- CLEANUP: tree-wide: replace free(x);x=NULL with ha_free(&x)
- CLEANUP: config: replace a few free() with ha_free()
- CLEANUP: vars: always zero the pointers after a free()
- CLEANUP: ssl: remove a useless "if" before freeing an error message
- CLEANUP: ssl: make ssl_sock_free_srv_ctx() zero the pointers after free
- CLEANUP: ssl: use realloc() instead of free()+malloc()
The scheduler has become complex over time and the latest updates were a
good opportunity to document it. This diagram shows the time-based wait
queue(s), the priority-based run queue(s), and the class-based tasklet
queues, trying to emphasize what is local-only and what is shared between
threads. The diagram is provided in .fig, .svg, .png, and .pdf.
A network may be specified to avoid header addition for "forwardfor" and
"orignialto" option via the "except" parameter. However, only IPv4
networks/addresses are supported. This patch adds the support of IPv6.
To do so, the net_addr structure is used to store the parameter value in the
proxy structure. And ipcmp2net() function is used to perform the comparison.
This patch should fix the issue #1145. It depends on the following commit:
* c6ce0ab MINOR: tools: Add function to compare an address to a network address
* 5587287 MINOR: tools: Add net_addr structure describing a network addess
The filters guide was totally outdated. Callbacks to filter payload were
changed, especially the HTTP one because of the HTX. All the HTTP legacy
part is removed. This new guide now reflects the reality.
This patch may be backported as far as 2.2.
Missing functions have been added. And because the EOM block was removed,
some parts have been adapted to better explain how the end of the message
may be detected.
Since the muxes API is far from being obvious, let's show a stream being
forwarded between two sides through muxes with their buffers and the
transport layers. The diagram is provided in .fig, .svg, .png, and .pdf.
Released version 2.4-dev9 with the following main changes :
- BUG/MINOR: server: Remove RMAINT from admin state when loading server state
- CLEANUP: check: fix get_check_status_info declaration
- CLEANUP: contrib/prometheus-exporter: align for with srv status case
- MEDIUM: stats: allow to select one field in `stats_fill_li_stats`
- MINOR: stats: add helper to get status string
- MEDIUM: contrib/prometheus-exporter: add listen stats
- BUG/MINOR: dns: add test on result getting value from buffer into ring.
- BUG/MINOR: dns: dns_connect_server must return -1 unsupported nameserver's type
- BUG/MINOR: dns: missing test writing in output channel in session handler
- BUG/MINOR: dns: fix ring attach control on dns_session_new
- BUG/MEDIUM: dns: fix multiple double close on fd in dns.c
- BUG/MAJOR: connection: prevent double free if conn selected for removal
- BUG/MINOR: session: atomically increment the tracked sessions counter
- REGTESTS: fix http_reuse_conn_hash proxy test
- BUG/MINOR: backend: do not call smp_make_safe for sni conn hash
- MINOR: connection: remove pointers for prehash in conn_hash_params
- BUG/MINOR: checks: properly handle wrapping time in __health_adjust()
- BUG/MEDIUM: checks: don't needlessly take the server lock in health_adjust()
- DEBUG: thread: add 5 extra lock labels for statistics and debugging
- OPTIM: server: switch the actconn list to an mt-list
- Revert "MINOR: threads: change lock_t to an unsigned int"
- MINOR: lb/api: let callers of take_conn/drop_conn tell if they have the lock
- OPTIM: lb-first: do not take the server lock on take_conn/drop_conn
- OPTIM: lb-leastconn: do not take the server lock on take_conn/drop_conn
- OPTIM: lb-leastconn: do not unlink the server if it did not change
- MINOR: tasks: add DEBUG_TASK to report caller info in a task
- MINOR: tasks/debug: add some extra controls of use-after-free in DEBUG_TASK
- BUG/MINOR: sample: Always consider zero size string samples as unsafe
- MINOR: cli: add missing agent commands for set server
- BUILD/MEDIUM: da Adding pcre2 support.
- BUILD: ssl: introduce fine guard for OpenSSL specific SCTL functions
- REGTESTS: reorder reuse conn proxy protocol test
- DOC: explain the relation between pool-low-conn and tune.idle-pool.shared
- MINOR: tasks: refine the default run queue depth
- MINOR: listener: refine the default MAX_ACCEPT from 64 to 4
- MINOR: mux_h2: do not try to remove front conn from idle trees
- REGTESTS: workaround for a crash with recent libressl on http-reuse sni
- BUG/MEDIUM: lists: Avoid an infinite loop in MT_LIST_TRY_ADDQ().
- MINOR: connection: allocate dynamically hash node for backend conns
- DOC: DeviceAtlas documentation typo fix.
- BUG/MEDIUM: spoe: Resolve the sink if a SPOE logs in a ring buffer
- BUG/MINOR: http-rules: Always replace the response status on a return action
- BUG/MINOR: server: Init params before parsing a new server-state line
- BUG/MINOR: server: Be sure to cut the last parsed field of a server-state line
- MEDIUM: server: Don't introduce a new server-state file version
- DOC: contrib/prometheus-exporter: remove htx reference
- REGTESTS: contrib/prometheus-exporter: test NaN values
- REGTESTS: contrib/prometheus-exporter: test well known labels
- CI: github actions: switch to stable LibreSSL release
- BUG/MINOR: server: Fix test on number of fields allowed in a server-state line
- MINOR: dynbuf: make the buffer wait queue per thread
- MINOR: dynbuf: use regular lists instead of mt_lists for buffer_wait
- MINOR: dynbuf: pass offer_buffers() the number of buffers instead of a threshold
- MINOR: sched: have one runqueue ticks counter per thread
The maximum number of connections accepted at once by a thread for a single
listener used to default to 64 divided by the number of processes but the
tasklet-based model is much more scalable and benefits from smaller values.
Experimentation has shown that 4 gives the highest accept rate for all
thread values, and that 3 and 5 come very close, as shown below (HTTP/1
connections forwarded per second at multi-accept 4 and 64):
ac\thr| 1 2 4 8 16
------+------------------------------
4| 80k 106k 168k 270k 336k
64| 63k 89k 145k 230k 274k
Some tests were also conducted on SSL and absolutely no change was observed.
The value was placed into a define because it used to be spread all over the
code.
It might be useful at some point to backport this to 2.3 and 2.2 to help
those who observed some performance regressions from 1.6.
Since a lot of internal callbacks were turned to tasklets, the runqueue
depth had not been readjusted from the default 200 which was initially
used to favor batched processing. But nowadays it appears too large
already based on the following tests conducted on a 8c16t machine with
a simple config involving "balance leastconn" and one server. The setup
always involved the two threads of a same CPU core except for 1 thread,
and the client was running over 1000 concurrent H1 connections. The
number of requests per second is reported for each (runqueue-depth,
nbthread) couple:
rq\thr| 1 2 4 8 16
------+------------------------------
32| 120k 159k 276k 477k 698k
40| 122k 160k 276k 478k 722k
48| 121k 159k 274k 482k 720k
64| 121k 160k 274k 469k 710k
200| 114k 150k 247k 415k 613k <-- default
It's possible to save up to about 18% performance by lowering the
default value to 40. One possible explanation to this is that checking
I/Os more frequently allows to flush buffers faster and to smooth the
I/O wait time over multiple operations instead of alternating phases
of processing, waiting for locks and waiting for new I/Os.
The total round trip time also fell from 1.62ms to 1.40ms on average,
among which at least 0.5ms is attributed to the testing tools since
this is the minimum attainable on the loopback.
After some observation it would be nice to backport this to 2.3 and
2.2 which observe similar improvements, since some users have already
observed some perf regressions between 1.6 and 2.2.
Disabling idle-pool sharing can result in awful performance in presence
of a not so high number of threads, because the number of available idle
connections will be shared among threads, resulting in most of them
abandonning their connections after a request is done if there are already
enough total available. This is a case where pool-low-conn ought to be
used to preserve a number of connections for each thread, but this relation
isn't obvious as is. Let's add mentions about this with both keywords.
The DeviceAtlas Detection API now supports also the pcre2 library,
and some users wish to have exclusively this version in their
environment.
Also, there is no longer new development happening in the legacy
pcre(1) counterpart.
Simple check in the build process as the mutual exclusivity check between the
two are already taking care of early on. Moving the check to the part
only when we build haproxy + the API from source as the other case the API is
already built with the chosen regex library separately.
Released version 2.4-dev8 with the following main changes :
- BUILD: ssl: fix typo in HAVE_SSL_CTX_ADD_SERVER_CUSTOM_EXT macro
- BUILD: ssl: guard SSL_CTX_add_server_custom_ext with special macro
- BUG/MINOR: mux-h1: Don't emit extra CRLF for empty chunked messages
- MINOR: contrib/prometheus-exporter: use stats desc when possible followup
- MEDIUM: contrib/prometheus-exporter: export base stick table stats
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: check: fix some typo in comments
- CLEANUP: tools: typo in `strl2irc` mention
- BUILD: ssl: guard SSL_CTX_set_msg_callback with SSL_CTRL_SET_MSG_CALLBACK macro
- MEDIUM: ssl: add a rwlock for SSL server session cache
- BUG/MINOR: intops: fix mul32hi()'s off-by-one
- BUG/MINOR: freq_ctr: fix a wrong delay calculation in next_event_delay()
- MINOR: stick-tables/counters: add http_fail_cnt and http_fail_rate data types
- MINOR: ssl: add SSL_SERVER_LOCK label in threads.h
- BUG/MINOR: mux-h1: Don't increment HTTP error counter for 408/500/501 errors
- BUG/MINOR: http-ana: Don't increment HTTP error counter on internal errors
- BUG/MEDIUM: mux-h1: Always set CS_FL_EOI for response in MSG_DONE state
- BUG/MINOR: mux-h1: Fix data skipping for bodyless responses
- BUG/MINOR: mux-h1: Don't blindly skip EOT block for non-chunked messages
- BUG/MEDIUM: mux-h2: Add EOT block when EOM flag is set on an empty HTX message
- MINOR: mux-h1: Be sure EOM flag is set when processing end of outgoing message
- REGTESTS: Add a script to test payload skipping for bodyless HTTP responses
- BUG/MINOR: server: re-align state file fields number
- CLEANUP: muxes: Remove useless calls to b_realign_if_empty()
- BUG/MINOR: tools: Fix a memory leak on error path in parse_dotted_uints()
- CLEANUP: remove unused variable assigned found by Coverity
- CLEANUP: queue: Remove useless tests on p or pp in pendconn_process_next_strm()
- BUG/MINOR: backend: hold correctly lock when killing idle conn
- MEDIUM: connection: protect idle conn lists with locks
- MEDIUM: connection: replace idle conn lists by eb trees
- MINOR: backend: search conn in idle/safe trees after available
- MINOR: backend: search conn in idle tree after safe on always reuse
- MINOR: connection: prepare hash calcul for server conns
- MINOR: connection: use the srv pointer for the srv conn hash
- MINOR: backend: compare conn hash for session conn reuse
- MINOR: connection: use sni as parameter for srv conn hash
- MINOR: reg-tests: test http-reuse with sni
- MINOR: backend: rewrite alloc of stream target address
- MINOR: connection: use dst addr as parameter for srv conn hash
- MINOR: reg-test: test http-reuse with specific dst addr
- MINOR: backend: rewrite alloc of connection src address
- MINOR: connection: use src addr as parameter for srv conn hash
- MINOR: connection: use proxy protocol as parameter for srv conn hash
- MINOR: reg-tests: test http-reuse with proxy protocol
- MINOR: doc: update http reuse for new eligilible connections
- BUG/MINOR: backend: fix compilation without ssl
- REGTESTS: adjust http_reuse_conn_hash requirements
- REGTESTS: deactivate a failed test on CI in http_reuse_conn_hash
- REGTESTS: fix sni used in http_reuse_conn_hash for libressl 3.3.0
- CI: cirrus: update FreeBSD image to 12.2
- MEDIUM: cli: add check-addr command
- MEDIUM: cli: add agent-port command
- MEDIUM: server: add server-states version 2
- MEDIUM: server: support {check,agent}_addr, agent_port in server state
- MINOR: server: enhance error precision when applying server state
- BUG/MINOR: server: Fix server-state-file-name directive
- CLEANUP: deinit: release global and per-proxy server-state variables on deinit
- BUG/MEDIUM: config: don't pick unset values from last defaults section
- BUG/MINOR: stats: revert the change on ST_CONVDONE
- BUG/MINOR: cfgparse: do not mention "addr:port" as supported on proxy lines
- BUG/MINOR: http-htx: defpx must be a const in proxy_dup_default_conf_errors()
- BUG/MINOR: tcpheck: the source list must be a const in dup_tcpcheck_var()
- BUILD: proxy: add missing compression-t.h to proxy-t.h
- REORG: move init_default_instance() to proxy.c and pass it the defproxy pointer
- REORG: proxy: centralize the proxy allocation code into alloc_new_proxy()
- MEDIUM: proxy: only take defaults when a default proxy is passed.
- MINOR: proxy: move the defproxy freeing code to proxy.c
- MINOR: proxy: always properly reset the just freed default instance pointers
- BUG/MINOR: extcheck: proxy_parse_extcheck() must take a const for the defproxy
- BUG/MINOR: tcpcheck: proxy_parse_*check*() must take a const for the defproxy
- BUG/MINOR: server: parse_server() must take a const for the defproxy
- MINOR: cfgparse: move defproxy to cfgparse-listen as a static
- MINOR: proxy: add a new capability PR_CAP_DEF
- MINOR: cfgparse: check PR_CAP_DEF instead of comparing poiner against defproxy
- MINOR: cfgparse: use a pointer to the current default proxy
- MINOR: proxy: also store the name for a defaults section
- MINOR: proxy: support storing defaults sections into their own tree
- MEDIUM: proxy: store the default proxies in a tree by name
- MEDIUM: cfgparse: allow a proxy to designate the defaults section to use
- MINOR: http: add baseq sample fetch
- CLEANUP: tcpcheck: Remove a useless test on port variable
- BUG/MINOR: server: Don't call fopen() with server-state filepath set to NULL
- CLEANUP: server: Remove useless "filepath" variable in apply_server_state()
- MINOR: peers/cli: do not dump the peers dictionaries by default on "show peers"
- MINOR: cfgparse: implement a simple if/elif/else/endif macro block handler
- DOC: tune: explain the origin of block size for ssl.cachesize
- MINOR: tcp: add support for defer-accept on FreeBSD.
- MINOR: ring: adds new ring_init function.
- CLEANUP: channel: fix comment in ci_putblk.
- BUG/MINOR: dns: add missing sent counter and parent id to dns counters.
- BUG/MINOR: resolvers: fix attribute packed struct for dns
- MINOR: resolvers: renames some resolvers internal types and removes dns prefix
- MINOR: resolvers: renames type dns_resolvers to resolvers.
- MINOR: resolvers: renames some resolvers specific types to not use dns prefix
- MINOR: resolvers: renames some dns prefixed types using resolv prefix.
- MINOR: resolvers: renames resolvers DNS_RESP_* errcodes RSLV_RESP_*
- MINOR: resolvers: renames resolvers DNS_UPD_* returncodes to RSLV_UPD_*
- MINOR: resolvers: rework prototype suffixes to split resolving and dns.
- MEDIUM: resolvers: move resolvers section parsing from cfgparse.c to dns.c
- MINOR: resolvers: replace nameserver's resolver ref by generic parent pointer
- MINOR: resolvers: rework dns stats prototype because specific to resolvers
- MEDIUM: resolvers: split resolving and dns message exchange layers.
- MEDIUM: resolvers/dns: split dns.c into dns.c and resolvers.c
- MEDIUM: dns: adds code to support pipelined DNS requests over TCP.
- MEDIUM: resolvers: add supports of TCP nameservers in resolvers.
This patch introduce the new line "server" to set a TCP
nameserver in a "resolvers" section:
server <name> <address> [param*]
Used to configure a DNS TCP or stream server. This supports for all
"server" parameters found in 5.2 paragraph. Some of these parameters
are irrelevant for DNS resolving. Note: currently 4 queries are pipelined
on the same connections. A batch of idle connections are removed every
5 seconds. "maxconn" can be configured to limit the amount of those
concurrent connections and TLS should also usable if the server supports
. The current implementation limits to 4 pipelined
The name of the line in configuration is open to discussion
and could be changed before the next release.
A user could eventually ask himself where those 200 bytes block size are
coming from. This patch tries to better explain the origin in case
people are curious or want to double check the reality.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Very often, especially since reg-tests, it would be desirable to be able
to conditionally comment out a config block, such as removing an SSL
binding when SSL is disabled, or enabling HTX only for certain versions,
etc.
This patch introduces a very simple nested block management which takes
".if", ".elif", ".else" and ".endif" directives to take or ignore a block.
For now the conditions are limited to empty string or "0" for false versus
a non-nul integer for true, which already suffices to test environment
variables. Still, it needs to be a bit more advanced with defines, versions
etc.
A set of ".notice", ".warning" and ".alert" statements are provided to
emit messages, often in order to provide advice about how to fix certain
conditions.
The "show peers" output has become huge due to the dictionaries making it
less readable. Now this feature has reached a certain level of maturity
which doesn't warrant to dump it all the time, given that it was essentially
needed by developers. Let's make it optional, and disabled by default, only
when "show peers dict" is requested. The default output reminds about the
command. The output has been divided by 5 :
$ socat - /tmp/sock1 <<< "show peers dict" | wc -l
125
$ socat - /tmp/sock1 <<< "show peers" | wc -l
26
It could be useful to backport this to recent stable versions.
Now it becomes possible to specify "from foo" on a frontend/listen/backend
or even on a "defaults" line, to mention that defaults section "foo" needs
to be used to preset the proxy's settings.
When not set, the last section remains used. In case the designated name
is found at multiple places, it is rejected and an error indicates two
occurrences of the same name. Similarly, if the section name is found,
its name must only use valid characters. This allows multiple named
defaults section to continue to coexist without the risk that they will
cause trouble by accident.
When it comes to "defaults" relying on another defaults, what happens is
just that a new defaults section is created from the designated one. This
will make it possible for example to reuse some settings such as log-format
like below:
defaults tcp-clear
log stdout local0 info
log-format "%ci:%cp/%b/%si:%sp %ST %ts %U/%B %{+Q}r"
defaults tcp-ssl
log stdout local0 info
log-format "%ci:%cp/%b/%si:%sp %ST %ts %U/%B %{+Q}r ssl=%sslv"
defaults http-clear from tcp-clear
mode http
defaults http-ssl from tcp-ssl
mode http
frontend fe1 from http-clear
bind :8001
frontend fe2 from http-ssl
bind :8002
A small corner case remains in the error detection, if a second defaults
section appears with the same name after the point where it was used, and
nobody references it, the duplicate will not be detected. This could be
addressed by performing the syntactic checks in check_config_validity(),
and by postponing the freeing of the defaults, after tagging a defaults
section as explicitly looked up by another section. This doesn't seem
that important at the moment though.
Since the beginning, this directive is documented to accept an optional file
name. But it should also be possible to use it without any argument to use
the backend name as file name. However, when no argument is provided, an
error is reported during the configuration parsing requesting an argument, a
file name or "use-backend-name". And This last special argument is not
documented.
So, to respect the documentation and to avoid configuration breakages, all
modes are now supported. If this directive is called with no argument or
with "use-backend-name", the backend name is use as file name for the
server-state file. Otherwise, the provided string is used.
In addition, we take care to release any previously allocated file name in
case this directive is defines multiple times in the same backend. And an
error is reported if more than one argument are defined. Finally, the
documentation is updated accordingly. Sections supporting this directive are
also mentioned.
This patch should be backported as far as 1.6.
logical followup from cli commands addition, so that the state server
file stays compatible with the changes made at runtime; use previously
added helper to load server attributes.
also alloc a specific chunk to avoid mixing with other called functions
using it
Signed-off-by: William Dauchy <wdauchy@gmail.com>
this patch allows to set agent port at runtime. In order to align with
both `addr` and `check-addr` commands, also add the possibility to
optionnaly set port on `agent-addr` command. This led to a small
refactor in order to use the same function for both `agent-addr` and
`agent-port` commands.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
this patch allows to set server health check address at runtime. In
order to align with `addr` command, also allow to set port optionnaly.
This led to a small refactor in order to use the same function for both
`check-addr` and `check-port` commands.
for `check-port`, we however don't permit the change anymore if checks
are not enabled on the server.
This command becomes more and more useful for people having a consul
like architecture:
- the backend server is located on a container with its own IP
- the health checks are done the consul instance located on the host
with the host IP
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Historically we've been counting lots of client-triggered events in stick
tables to help detect misbehaving ones, but we've been missing the same on
the server side, and there's been repeated requests for being able to count
the server errors per URL in order to precisely monitor the quality of
service or even to avoid routing requests to certain dead services, which
is also called "circuit breaking" nowadays.
This commit introduces http_fail_cnt and http_fail_rate, which work like
http_err_cnt and http_err_rate in that they respectively count events and
their frequency, but they only consider server-side issues such as network
errors, unparsable and truncated responses, and 5xx status codes other
than 501 and 505 (since these ones are usually triggered by the client).
Note that retryable errors are purposely not accounted for, so that only
what the client really sees is considered.
With this it becomes very simple to put some protective measures in place
to perform a redirect or return an excuse page when the error rate goes
beyond a certain threshold for a given URL, and give more chances to the
server to recover from this condition. Typically it could look like this
to bypass a URL causing more than 10 requests per second:
stick-table type string len 80 size 4k expire 1m store http_fail_rate(1m)
http-request track-sc0 base # track host+path, ignore query string
http-request return status 503 content-type text/html \
lf-file excuse.html if { sc0_http_fail_rate gt 10 }
A more advanced mechanism using gpt0 could even implement high/low rates
to disable/enable the service.
Reg-test converteers_ref_cnt_never_dec.vtc was updated to test it.
Released version 2.4-dev7 with the following main changes :
- BUG/MINOR: stats: Continue to fill frontend stats on unimplemented metric
- BUILD: ssl: guard Client Hello callbacks with HAVE_SSL_CLIENT_HELLO_CB macro instead of openssl version
- BUG/MINOR: stats: Init the metric variable when frontend stats are filled
- MINOR: contrib/prometheus-exporter: better output of Not-a-Number
- CLEANUP: stats: improve field selection for frontend http fields
- CLEANUP: assorted typo fixes in the code and comments
- DOC: Improve documentation of the various hdr() fetches
- MEDIUM: stats: allow to select one field in `stats_fill_be_stats`
- MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump
- MEDIUM: stats: allow to select one field in `stats_fill_sv_stats`
- MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump
- MINOR: abort() on my_unreachable() when DEBUG_USE_ABORT is set.
- BUG/MEDIUM: filters/htx: Fix data forwarding when payload length is unknown
- BUG/MINOR: config: fix leak on proxy.conn_src.bind_hdr_name
- MINOR: reg-tests: add http-reuse test
- CLEANUP: srv: fix comment for pool-max-conn
- CLEANUP: backend: remove an obsolete comment on conn_backend_get
- REORG: backend: simplify conn_backend_get
- MINOR: ssl: Server ssl context prepare function refactoring
- MINOR: ssl: Certificate chain loading refactorization
- MEDIUM: ssl: Load client certificates in a ckch for backend servers
- MEDIUM: ssl: Enable backend certificate hot update
- MINOR: ssl: Remove client_crt member of the server's ssl context
- CLEANUP: ssl/cli: rework free in cli_io_handler_commit_cert()
- CLEANUP: ssl: remove SSL_CTX function parameter
- CLEANUP: ssl: make load_srv_{ckchs,cert} match their bind counterpart
- BUILD: Include stdlib.h in compiler.h if DEBUG_USE_ABORT is set
- CI: Fix DEBUG_STRICT definition for Coverity
- BUG/MINOR: stats: Remove a break preventing ST_F_QCUR to be set for servers
- BUG/MINOR: stats: Add a break after filling ST_F_MODE field for servers
- CLEANUP: ssl: remove dead code in ckch_inst_new_load_srv_store()
- BUG/MINOR: ssl: init tmp chunk correctly in ssl_sock_load_sctl_from_file()
- BUG/MEDIUM: session: only retrieve ready idle conn from session
- BUG/MEDIUM: backend: never reuse a connection for tcp mode
- REGTESTS: set_ssl_server_cert.vtc: remove the abort command
- REGTESTS: set_ssl_server_cert.vtc: check the Sha1 Fingerprint
- REGTESTS: set_ssl_server_cert.vtc: check the sha1 from the server
- MEDIUM: stream-int: Take care of EOS if the SI wake callback function
- MINOR: mux-h1: Try to wake up data layer first before calling its wake callback
- MINOR: mux-h1: Wake up H1C after its creation if input buffer is not empty
- MEDIUM: mux-h1: Add ST_READY state for the H1 connections
- MINOR: stream: Add a function to validate TCP to H1 upgrades
- MEDIUM: http-ana: Do nothing in wait-for-request analyzer if not htx
- BUG/MEDIUM: stream: Don't immediatly ack the TCP to H1 upgrades
- BUG/MAJOR: mux-h1: Properly handle TCP to H1 upgrades
- MINOR: htx/http-ana: Save info about Upgrade option in the Connection header
- MEDIUM: http-ana: Refuse invalid 101-switching-protocols responses
- BUG/MINOR: h2/mux-h2: Reject 101 responses with a PROTOCOL_ERROR h2s error
- MINOR: mux-h1/mux-fcgi: Don't set TUNNEL mode if payload length is unknown
- MINOR: mux-h1: Split H1C_F_WAIT_OPPOSITE flag to separate input/output sides
- MINOR: mux-h2: Add 2 flags to help to properly handle tunnel mode
- MEDIUM: mux-h2: Block client data on server side waiting tunnel establishment
- MEDIUM: mux-h2: Close streams when processing data for an aborted tunnel
- MEDIUM: mux-h1: Properly handle tunnel establishments and aborts
- BUG/MAJOR: mux-h1/mux-h2/htx: Fix HTTP tunnel management at the mux level
- MINOR: htx: Rename HTX_FL_EOI flag into HTX_FL_EOM
- REGTESTS: Don't run http_msg_full_on_eom script on the 2.4 anymore
- MINOR: htx: Add a function to know if a block is the only one in a message
- MAJOR: htx: Remove the EOM block type and use HTX_FL_EOM instead
- MINOR: mux-h1: Add a flag on H1 streams with a response known to be bodyless
- MEDIUM: mux-h1: Don't emit any payload for bodyless responses
- MINOR: mux-h1: Don't emit C-L and T-E headers for 204 and 1xx responses
- MINOR: mux-h1: Don't add Connection close/keep-alive header for 1xx messages
- MINOR: h2/mux-h2: Add flags to notify the response is known to have no body
- MEDIUM: mux-h2: Don't emit DATA frame for bodyless responses
- MEDIUM: http-ana: Deal with L7 retries in HTTP analysers
- MINOR: h1: reject websocket handshake if missing key
- MEDIUM: h1: generate WebSocket key on response if needed
- MINOR: mux_h2: define H2_SF_EXT_CONNECT_SENT stream flag
- MEDIUM: h2: parse Extended CONNECT reponse to htx
- MEDIUM: mux_h2: generate Extended CONNECT from htx upgrade
- MEDIUM: h1: add a WebSocket key on handshake if needed
- MEDIUM: mux_h2: generate Extended CONNECT response
- MEDIUM: h2: parse Extended CONNECT request to htx
- MEDIUM: h2: send connect protocol h2 settings
- MINOR: vtc: add test for h1/h2 protocol upgrade translation
- MINOR: vtc: add websocket test
- REGTESTS: Fix required versions for several scripts
- REGTEST: Don't use the websocket to validate http-check
- MINOR: mux-h1/trace: add traces at level ERROR for all kind of errors
- MINOR: mux-fcgi/trace: add traces at level ERROR for all kind of errors
- MINOR: h1: Raise the chunk size limit up to (2^52 - 1)
- BUG/MEDIUM: listener: do not accept connections faster than we can process them
- REGTESTS: set_ssl_server_cert.vtc: set as broken
- Revert "BUG/MEDIUM: listener: do not accept connections faster than we can process them"
- BUG/MINOR: backend: check available list allocation for reuse
- CI: Fix the coverity builds
- DOC: management: fix "show resolvers" alphabetical ordering
- MINOR: tools: add print_time_short() to print a condensed duration value
- MINOR: activity: make profiling more manageable
- MINOR: activity: declare a new structure to collect per-function activity
- MEDIUM: tasks/activity: collect per-task statistics when profiling is enabled
- MINOR: activity: also report collected tasks stats in "show profiling"
- MINOR: activity: flush scheduler stats on "set profiling tasks on"
- MINOR: activity: add a new "show tasks" command to list currently active tasks
- MINOR: listener: export accept_queue_process
- MINOR: session: export session_expire_embryonic()
- MINOR: muxes: export the timeout and shutr task handlers
- MINOR: checks: export a few functions that appear often in trace dumps
- MINOR: peers: export process_peer_sync() to improve traces
- MINOR: stick-tables: export process_table_expire()
- MINOR: mux-h1: Remove first useless test on count in h1_process_output()
- BUG/MINOR: stick-table: Always call smp_fetch_src() with a valid arg list
- MINOR: http-fetch: Don't check if argument list is set in sample fetches
- MINOR: http-conv: Don't check if argument list is set in sample converters
- MINOR: sample: Don't check if argument list is set in sample fetches
- MINOR: ssl-sample: Don't check if argument list is set in sample fetches
- MINOR: mux-h2: Don't tests the start-line when sending HEADERS frame
- MINOR: mux-h2: Slightly improve request HEADERS frames sending
- MINOR: contrib/prometheus-exporter: declare states for objects
- MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels
- MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones
- MINOR: listener: export manage_global_listener_queue()
- BUG/MINOR: activity: take care of late wakeups in "show tasks"
- REGTESTS: set_ssl_server_cert.vtc: remove SSL caching and set as working
- REGTESTS: set_ssl_server_cert: cleanup the SSL caching option
- MINOR: checks: Add function to get the result code corresponding to a status
- MAJOR: contrib/prometheus-exporter: move health check status to labels
- MINOR: contrib/prometheus-exporter: improve service status description field
- MINOR: stats: improve pending connections description
- MINOR: stats: improve max stats descriptions
- MINOR: contrib/prometheus-exporter: use stats desc when possible
- MINOR: contrib/prometheus-exporter: add uweight field
- MINOR: contrib/prometheus-exporter: add recv logs_logs_total field
- CLEANUP: contrib/prometheus-exporter: remove unused includes
- CLEANUP: contrib/prometheus-exporter: align and reorder fields
- CLEANUP: contrib/prometheus-exporter: remove description in README
- DOC: contrib/prometheus-exporter: Add missing metrics in README
- BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX
- BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos
- BUG/MEDIUM: ssl/cli: abort ssl cert is freeing the old store
- BUG/MEDIUM: ssl: check a connection's status before computing a handshake
- BUG/MINOR: mux_h2: fix incorrect stat titles
- MINOR: ssl/cli: flush the server session cache upon 'commit ssl cert'
- BUG/MINOR: cli: fix set server addr/port coherency with health checks
- MINOR: server: Don't set the check port during the update from a state file
- MINOR: dns: Don't set the check port during a server dns resolution
- MEDIUM: check: remove checkport checkaddr flag
- MEDIUM: server: adding support for check_port in server state
- BUG/MINOR: check: consitent way to set agentaddr
- MEDIUM: check: align agentaddr and agentport behaviour
- DOC: server: Add missing params in comment of the server state line parsing
- BUG/MINOR: xxhash: make sure armv6 uses memcpy()
- REGTESTS: mark http-check-send.vtc as 2.4-only
- REGTESTS: mark sample_fetches/hashes.vtc as 2.4-only
- BUG/MINOR: ssl: do not try to use early data if not configured
- REGTESTS: unbreak http-check-send.vtc
- MINOR: cli/show_fd: report local and report ports when known
- BUILD: Makefile: move REGTESTST_TYPE default setting
- BUG/MEDIUM: mux-h2: handle remaining read0 cases
- CLEANUP: http-htx: Set buffer area to NULL instead of malloc(0)
- BUG/MINOR: sock: Unclosed fd in case of connection allocation failure
- BUG/MEDIUM: mux-h2: do not quit the demux loop before setting END_REACHED
in the same manner of agentaddr, we now:
- permit to set agentport through `port` keyword, like it is the case
for agentaddr through `addr`
- set the priority on `agent-port` keyword when used
- add a flag to be able to test when the value is set like for agentaddr
it makes the behaviour between `addr` and `port` more consistent.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
We can currently change the check-port using the cli command `set server
check-port` but there is a consistency issue when using server state.
This patch aims to fix this problem but will be also a good preparation
work to get rid of checkport flag, so we are able to know when checkport
was set by config.
I am fully aware this is not making github #953 moving forward, I
however think this might be acceptable while waiting for a proper
solution and resolve consistency problem faced with port settings.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
This finally adds the long-awaited solution to inspect the run queues
and figure what is eating the CPU or causing latencies. We can even see
the experienced latencies when profiling is enabled. Example on a
saturated process:
> show tasks
Running tasks: 14983 (4 threads)
function places % lat_tot lat_avg
process_stream 4948 33.0 5.840m 70.82ms
h1_io_cb 2535 16.9 - -
main+0x9e670 2508 16.7 2.930m 70.10ms
ssl_sock_io_cb 2499 16.6 - -
si_cs_io_cb 2493 16.6 - -
If a user enables profiling by hand, it makes sense to reset the stats
counters to provide fresh new measurements. Therefore it's worth using
this as the standard method to reset counters.
"show profiling" will now dump the stats collected by the scheduler if
profiling was previously enabled. This will immediately make it obvious
what functions are responsible for others' high latencies or which ones
are suffering from others, and should help spot issues like undesired
wakeups.
Example:
Per-task CPU profiling : on # set profiling tasks {on|auto|off}
Tasks activity:
function calls cpu_tot cpu_avg lat_tot lat_avg
si_cs_io_cb 5569479 23.37s 4.196us - -
h1_io_cb 5558654 13.60s 2.446us - -
process_stream 250841 1.476s 5.882us 3.499s 13.95us
main+0x9e670 198 - - 5.526ms 27.91us
task_run_applet 17 1.509ms 88.77us 205.8us 12.11us
srv_cleanup_idle_connections 12 44.51us 3.708us 25.71us 2.142us
main+0x158c80 9 48.72us 5.413us - -
srv_cleanup_toremove_connections 5 165.1us 33.02us 123.6us 24.72us
The EOM block may be removed. The HTX_FL_EOM flags is enough. Most of time,
to know if the end of the message is reached, we just need to have an empty
HTX message with HTX_FL_EOM flag set. It may also be detected when the last
block of a message with HTX_FL_EOM flag is manipulated.
Removing EOM blocks simplifies the HTX message filling. Indeed, there is no
more edge problems when the message ends but there is no more space to write
the EOM block. However, some part are more tricky. Especially the
compression filter or the FCGI mux. The compression filter must finish the
compression on the last DATA block. Before it was performed on the EOM
block, an extra DATA block with the checksum was added. Now, we must detect
the last DATA block to be sure to finish the compression. The FCGI mux on
its part must be sure to reserve the space for the empty STDIN record on the
last DATA block while this record was inserted on the EOM block.
The H2 multiplexer is probably the part that benefits the most from this
change. Indeed, it is now fairly easier to known when to set the ES flag.
The HTX documentaion has been updated accordingly.
GitHub issue #796 notes that many administrators miss the fact that the `hdr()`
fetch (without the `f`) splits the header value at commas. This is only
mentioned at the end of a long paragraph.
This patch attempts to improve the documentation by:
- Explaning the "comma issue" as early as possible.
- Adding newlines to split the explanation into distinct sections.
- Reducing duplication by making the `res` siblings refer to their `req`
counterparts.
This patch may be backported as long as it applies cleanly. During the
refactoring I needed to adjust several explanations for consistency and not all
of them might be available in older branches.
Released version 2.4-dev6 with the following main changes :
- MINOR: converter: adding support for url_enc
- BUILD: SSL: guard TLS13 ciphersuites with HAVE_SSL_CTX_SET_CIPHERSUITES
- BUILD: ssl: guard EVP_PKEY_get_default_digest_nid with ASN1_PKEY_CTRL_DEFAULT_MD_NID
- BUILD: ssl: guard openssl specific with SSL_READ_EARLY_DATA_SUCCESS
- BUILD: Makefile: exclude broken tests by default
- CLEANUP: cfgparse: replace "realloc" with "my_realloc2" to fix to memory leak on error
- BUG/MINOR: hlua: Fix memory leak in hlua_alloc
- MINOR: contrib/prometheus-exporter: export build_info
- DOC: fix some spelling issues over multiple files
- CLEANUP: Fix spelling errors in comments
- SCRIPTS: announce-release: fix typo in help message
- CI: github: add a few more words to the codespell ignore list
- DOC: Add maintainers for the Prometheus exporter
- BUG/MINOR: sample: fix concat() converter's corruption with non-string variables
- BUG/MINOR: server: Memory leak of proxy.used_server_addr during deinit
- CLEANUP: sample: remove uneeded check in json validation
- MINOR: reg-tests: add a way to add service dependency
- BUG/MINOR: sample: check alloc_trash_chunk return value in concat()
- BUG/MINOR: reg-tests: fix service dependency script
- MINOR: reg-tests: add base prometheus test
- Revert "BUG/MINOR: dns: SRV records ignores duplicated AR records"
- BUG/MINOR: sample: Memory leak of sample_expr structure in case of error
- BUG/MINOR: check: Don't perform any check on servers defined in a frontend
- BUG/MINOR: init: enforce strict-limits when using master-worker
- MINOR: contrib/prometheus-exporter: avoid connection close header
- MINOR: contrib/prometheus-exporter: use fill_info for process dump
- BUG/MINOR: init: Use a dynamic buffer to set HAPROXY_CFGFILES env variable
- MINOR: config: Add failifnotcap() to emit an alert on proxy capabilities
- MINOR: server: Forbid server definitions in frontend sections
- BUG/MINOR: threads: Fixes the number of possible cpus report for Mac.
- CLEANUP: pattern: rename pat_ref_commit() to pat_ref_commit_elt()
- MINOR: pattern: add the missing generation ID manipulation functions
- MINOR: peers: Add traces for peer control messages.
- BUG/MINOR: dns: SRV records ignores duplicated AR records (v2)
- BUILD: peers: fix build warning about unused variable
- BUG/MEDIUM: stats: add missing INF_BUILD_INFO definition
- MINOR: cache: Do not store responses with an unknown encoding
- BUG/MINOR: peers: Possible appctx pointer dereference.
- MINOR: build: discard echoing in help target
- MINOR: cache: Remove the `hash` part of the accept-encoding secondary key
- CLEANUP: cache: Use proper data types in secondary_key_cmp()
- CLEANUP: Rename accept_encoding_hash_cmp to accept_encoding_bitmap_cmp
- BUG/MINOR: peers: Wrong "new_conn" value for "show peers" CLI command.
- MINOR: contrib: Make the wireshark peers dissector compile for more distribs.
- BUG/MINOR: mux_h2: missing space between "st" and ".flg" in the "show fd" helper
- CLEANUP: tools: make resolve_sym_name() take a const pointer
- CLEANUP: cli: make "show fd" use a const connection to access other fields
- MINOR: cli: make "show fd" also report the xprt and xprt_ctx
- MINOR: xprt: add a new show_fd() helper to complete some "show fd" dumps.
- MINOR: ssl: provide a "show fd" helper to report important SSL information
- MINOR: xprt/mux: export all *_io_cb functions so that "show fd" resolves them
- MINOR: mux-h2: make the "show fd" helper also decode the h2s subscriber when known
- MINOR: mux-h1: make the "show fd" helper also decode the h1s subscriber when known
- MINOR: mux-fcgi: make the "show fd" helper also decode the fstrm subscriber when known
- CI: Pin VTest to a known good commit
- MINOR: cli: give the show_fd helpers the ability to report a suspicious entry
- MINOR: cli/show_fd: report some easily detectable suspicious states
- MINOR: ssl/show_fd: report some FDs as suspicious when possible
- MINOR: mux-h2/show_fd: report as suspicious an entry with too many calls
- MINOR: mux-h1/show_fd: report as suspicious an entry with too many calls
- BUG/MINOR: mworker: define _GNU_SOURCE for strsignal()
- BUG/MEDIUM: tcpcheck: Don't destroy connection in the wake callback context
- BUG/MEDIUM: mux-h2: Xfer rxbuf to the upper layer when creating a front stream
- MINOR: http: Add HTTP 501-not-implemented error message
- MINOR: muxes: Add exit status for errors about not implemented features
- MINOR: mux-h1: Be prepared to return 501-not-implemented error during parsing
- MEDIUM: mux-h1: Return a 501-not-implemented for upgrade requests with a body
- DOC: Remove space after comma in converter signature
- DOC: Rename '<var name>' to '<var>' in converter signature
- MINOR: stats: duplicate 3 fields in bytes in info
- MINOR: stats: add new start time field
- MINOR: contrib/prometheus-exporter: merge info description from stats
- MEDIUM: stats: allow to select one field in `stats_fill_fe_stats`
- MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump
- MINOR: contrib/prometheus-exporter: Don't needlessly set empty label for metrics
- MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag
- MINOR: contrib/prometheus-exporter: Add promex_metric struct defining a metric
- MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics
- BUG/MINOR: stream: Don't update counters when TCP to H2 upgrades are performed
- BUG/MEDIUM: mux-h2: fix read0 handling on partial frames
- MINOR: debug: always export the my_backtrace function
- MINOR: debug: extract the backtrace dumping code to its own function
- MINOR: debug: create ha_backtrace_to_stderr() to dump an instant backtrace
- MEDIUM: debug: now always print a backtrace on CRASH_NOW() and friends
- MINOR: debug: let ha_dump_backtrace() dump a bit further for some callers
- BUILD: debug: fix build warning by consuming the write() result
- MINOR: lua: remove unused variable
- BUILD/MINOR: lua: define _GNU_SOURCE for LLONG_MAX
Add the support for the 501-not-implemented status code with the
corresponding default message. The documentation is updated accordingly
because it is now part of status codes HAProxy may emit via an errorfile or
a deny/return HTTP action.
Now the show_fd helpers at the transport and mux levels return an integer
which indicates whether or not the inspected entry looks suspicious. When
an entry is reported as suspicious, "show fd" will suffix it with an
exclamation mark ('!') in the dump, that is supposed to help detecting
them.
For now, helpers were adjusted to adapt to the new API but none of them
reports any suspicious entry yet.
If a server varies on the accept-encoding header and it sends a response
with an encoding we do not know (see parse_encoding_value function), we
will not store it. This will prevent unexpected errors caused by
cache collisions that could happen in accept_encoding_hash_cmp.
add base support for url encode following RFC3986, supporting `query`
type only.
- add test checking url_enc/url_dec/url_enc
- update documentation
- leave the door open for future changes
this should resolve github issue #941
Signed-off-by: William Dauchy <wdauchy@gmail.com>
