2490 Commits

Author SHA1 Message Date
Amaury Denoyelle
9d19fc4cf7 MINOR: build: define DEBUG_STRESS
Define a new build mode DEBUG_STRESS. This will be used to stress some
code parts which cannot be reproduce easily with an alternative
suboptimal code.

First, a global <mode_stress> is set either to 1 or 0 depending on
DEBUG_STRESS compilation. A new global keyword "stress-level" is also
defined. It allows to specify a level from 0 to 9, to increase the
stress incurred on the code.

Helper macro STRESS_RUN* are defined for each stress level. This allows
to easily specify an instruction in default execution and a stress
counterpart if running on the corresponding stress level.
2024-12-12 11:19:10 +01:00
Willy Tarreau
f36ac42274 [RELEASE] Released version 3.2-dev1
Released version 3.2-dev1 with the following main changes :
    - MINOR: pattern: split pat_ref_set()
    - MINOR: pattern: add pat_ref_gen_set() function
    - MINOR: pattern: add pat_ref_gen_find_elt() function
    - MINOR: pattern: add pat_ref_gen_delete() function
    - MEDIUM: pattern: consider gen_id in pat_ref_set_from_node()
    - MEDIUM: pattern: always consider gen_id for pat_ref lookup operations
    - MINOR: version: this is development again (3.2)
    - DEV: patchbot: prepare for new version 3.2-dev
    - BUG/MEDIUM: sock: Remove FD_POLL_HUP during connect() if FD_POLL_ERR is not set
    - MINOR: proxy: Add support of 421-Misdirected-Request in retry-on status
    - BUG/MINOR: log: fix lf_text() behavior with empty string
    - MINOR: log: always consider "+M" option in lf_text_len()
    - BUG/MINOR: improve BBR throughput on very fast links
    - MINOR: event_hdl: add PAT_REF events
    - MINOR: pattern: publish event_hdl events on pat_ref updates
    - MINOR: hlua: add patref class
    - MINOR: hlua: add core.get_patref method
    - MINOR: hlua_fcn: implement index and pair metamethods for patref class
    - MINOR: hlua_fcn: wrap pat_ref struct for patref class
    - MINOR: pattern: add pat_ref_may_commit() helper function
    - MINOR: hlua_fcn: add Patref:commit() method
    - MINOR: hlua_fcn: add Patref:prepare() method
    - MINOR: hlua_fcn: add Patref:purge() method
    - MINOR: hlua_fcn: add Patref:giveup()
    - MINOR: hlua_fcn: add Patref:add()
    - MINOR: hlua_fcn: add Patref:del()
    - MINOR: hlua_fcn: add Patref:set()
    - MINOR: hlua_fcn: add Patref:add_bulk()
    - MINOR: hlua_fcn: add Patref:event_sub()
    - DOC: lua: prefer Patref:{set,add}() over legacy methods for acl and maps
    - BUG/MINOR: hlua_fcn: fix Patref:set() force parameter
    - BUG/MEDIUM: event_hdl: fix uninitialized value in async mode when no data is provided
    - BUG/MEDIUM: quic: prevent stream freeze on pacing
    - BUG/MEDIUM: http-ana: Reset request flag about data sent to perform a L7 retry
    - BUG/MINOR: h1-htx: Use default reason if not set when formatting the response
    - BUILD: quic: fix a build error about an non initialized timestamp
    - CI: github: allow coredumps on aws-lc and wolfssl jobs
    - BUG/MINOR: listener: fix potential null pointer dereference in listener_release()
    - MINOR: hlua: fix ambiguous hlua usage in hlua_filter_delete()
    - BUG/MINOR: signal: register default handler for SIGINT in signal_init()
    - BUG/MINOR: startup: close pidfd and free global.pidfile in handle_pidfile()
    - BUG/MINOR: startup: fix pidfile creation
    - MINOR: tools: add a new macro DEFVAL() to provide a default argument
    - MINOR: tasklet: set TASK_WOKEN_OTHER on tasklets by default
    - BUG/MINOR: quic: fix bbr_inflight() calls with wrong gain value
    - BUG/MEDIUM: init: make sure only daemonized processes change their session
    - BUG/MINOR: init: do not call fork_poller() for non-forked processes
    - BUG/MEDIUM: mux-quic: remove pacing status when everything is sent
    - BUG/MINOR: quic: remove startup alert if conn socket-owner unsupported
    - BUG/MINOR: quic: remove startup alert if GSO unsupported
    - MINOR: stktable: implement "recv-only" table option
    - CLEANUP: stktable: replace nopurge attribute with flag
    - CLEANUP: stktable: add some stktable flags polishing
    - BUG/MEDIUM: mux-h2: make sure not to touch dummy streams when sending WU
    - MINOR: mux-quic: clean up zero-copy done_ff callback
    - BUG/MINOR: config: Fix parsing of accept-invalid-http-{request,response}
    - BUG/MINOR: mworker: don't save program PIDs in oldpids
    - BUG/MINOR: mworker: fix -D -W -sf/-st modes
    - BUG/MINOR: startup: fix error path for master, if can't open pidfile
    - CLEANUP: startup: make if condition to kill old pids more readable
    - DOC: config: fix confusing init-state examples
    - MINOR: mux-h1: use explicit __objt_server on idle conn reinsert
    - MINOR: mux-h2: use explicit __objt_server on idle conn reinsert
    - MINOR: mux-spop: use explicit __objt_server on idle conn reinsert
    - MINOR: mux-fcgi: use explicit __objt_server on idle conn reinsert
    - MINOR: quic: convert startup check in a freestanding function
    - MINOR: quic: split startup check function
    - MINOR: quic: implement build options report
    - BUG/MINOR: debug: COUNT_IF() should return true/false
    - MINOR: mux-h2/traces: add a missing trace on negative initial window size
    - CLEANUP: mux-h2/traces: reword certain ambiguous traces
    - MINOR: mux-h2/glitches: add a description to the H2 glitches
    - BUG/MINOR: mux-h2: fix expression when detecting excess of CONTINUATION frames
    - BUILD: debug: fix build issues in COUNT_IF() with -Wunused-value
    - MINOR: tools: make fddebug() automatically emit the location
    - MINOR: ssl: add notBefore and notAfter utility functions
    - MEDIUM: ssl/cli: "show ssl sni" list the loaded SNI in frontends
    - BUG/MEDIUM: startup: don't daemonize if started with -c
    - BUG/MEDIUM: startup: report status if daemonized process fails
    - BUG/MEDIUM: mworker: report status, if daemonized master fails
    - BUG/MINOR: mworker: detach from tty when received READY from worker
    - BUG/MINOR: namespace: handle a possible strdup() failure
    - BUG/MINOR: ssl_crtlist: handle a possible strdup() failure
    - BUG/MINOR: resolvers: handle a possible strdup() failure
    - CI: use "/tmp" as default value for TMPDIR when searching logs
    - DOC: management: fix typos and paragraph ordering in 'show ssl sni'
    - CLEANUP: ssl: fix comment in 'show ssl sni'
    - MINOR: ssl/cli: add negative filters to "show ssl sni"
    - BUG/MINOR: stats: decrement srv refcount on stats-file release
    - MINOR: list: define a watcher type
    - BUG/MEDIUM: stats/server: use watcher to track server during stats dump
    - MINOR: server: remove prev_deleted server list
    - BUG/MINOR: http-fetch: Ignore empty argument string for query()
    - BUG/MINOR: server-state: Fix expiration date of srvrq_check tasks
    - BUG/MINOR: hlua_fcn: restore server pairs iterator pointer consistency
2024-12-11 14:17:46 +01:00
Aurelien DARRAGON
7934eef25d DOC: config: fix confusing init-state examples
in 50322dff ("MEDIUM: server: add init-state"), some examples on how to
use init-state server keyword were added alongside with the keyword
documentation.

However, as reported by Nick Ramirez, there was an error because the
example that stated that haproxy will pass the traffic to the server after
3 successful health checks used the "init-state down" instead of the
"init-state fully-down". Thus the behavior wouldn't match what the
comment said (only 1 successful health check was required).

Here we fix the example in itself to match with the comment. Also the
following example ("# or") was also affected, but it is kind of
redundant as the main purpose of the examples are to illustrate the
feature in itself and not how to use server-template directive, so we
remove it.

This should be backported in 3.1 with 50322dff
2024-12-06 13:16:12 +01:00
Aurelien DARRAGON
1f73d3524d MINOR: stktable: implement "recv-only" table option
When "recv-only" keyword is added on a stick table declaration (in peers
or proxy section), haproxy considers that the table is only used for
data retrieval from a remote location and not used to perform local
updates. As such, it enables the retrieval of local-only values such
as conn_cur that are ignored by default. This can be useful in some
contexts where we want to know about local-values such are conn_cur
from a remote peer.

To do this, add stktable struct flags  which default to NONE and enable
the RECV_ONLY flag on the table then "recv-only" keyword is found in the
table declaration. Then, when in peer_treat_updatemsg(), when handling
table updates, don't ignore data updates for local-only values if the flag
is set.
2024-12-05 12:15:24 +01:00
Christopher Faulet
bc66d31985 MINOR: proxy: Add support of 421-Misdirected-Request in retry-on status
The "421" status can now be specified on retry-on directives. PR_RE_* flags
were updated to remains sorted.

This patch should fix the issue #2794. It is quite simple so it may safely
be backported to 3.1 if necessary.
2024-11-28 11:47:40 +01:00
Willy Tarreau
4d58f521ee [RELEASE] Released version 3.2-dev0
Released version 3.2-dev0 with the following main changes :
    - exact copy of 3.1.0
2024-11-26 15:33:57 +01:00
Willy Tarreau
f2b97918e8 [RELEASE] Released version 3.1.0
Released version 3.1.0 with the following main changes :
    - BUG/MAJOR: mux-h1: Properly handle wrapping on obuf when dumping the first-line
    - BUILD: activity/memprofile: fix a build warning in the posix_memalign handler
    - BUG/MINOR: quic: Avoid BUG_ON() on ->on_pkt_lost() BBR callback call
    - CI: update to the latest AWS-LC version
    - CI: update to the latest WolfSSL version
    - DOC: ot: mention planned deprecation of the OT filter
    - Revert "CI: update to the latest WolfSSL version"
    - CI: github: add a WolfSSL job which tries the latest version
    - BUILD: systemd: fix usage of reserved name "sun" in the address field
    - BUILD: init: use the more portable FD_CLOEXEC for /dev/null
    - CI: github: improve the Wolfssl job
    - CI: github: improve the AWS-LC job
    - BUG/MINOR: mux-quic: fix show quic report of QCS prepared bytes
    - BUG/MEDIUM: quic: fix sending performance due to qc_prep_pkts() return
    - MINOR: mux-quic: use sched call time for pacing
    - CI: github: allow to run the Illumos job manually
    - BUILD: tcp_sample: var_fc_counter defined but not used
    - CI: github: add 'workflow_dispatch' on remaining build jobs
    - DOC: config: refine a little bit the text on QUIC pacing
    - MINOR: proto_sockpair: send_fd_uxst: init iobuf, cmsghdr, cmsgbuf to zeros
    - MINOR: startup: rename on_new_child_failure to mworker_on_new_child_failure
    - REORG: startup: move on_new_child_failure in mworker.c
    - MINOR: startup: prefix prepare_master and run_master with mworker_*
    - REORG: startup: move mworker_prepare_master in mworker.c
    - MINOR: startup: keep updating verbosity modes only in haproxy.c
    - REORG: startup: move mworker_run_master and mworker_loop in mworker.c
    - REORG: startup: move mworker_reexec and mworker_reload in mworker.c
    - MINOR: startup: prefix apply_master_worker_mode with mworker_*
    - REORG: startup: move mworker_apply_master_worker_mode in mworker.c
    - MINOR: cfgparse-quic: strengthen quic-cc-algo parsing
    - BUG/MAJOR: quic: fix wrong packet building due to already acked frames
    - DEV: lags/show-sess-to-flags: Properly handle fd state on server side
    - BUG/MEDIUM: http-ana: Don't release too early the L7 buffer
    - MINOR: quic: make bbr consider the max window size setting
    - DOC: quic: Amend the pacing information about BBR.
    - BUG/MEDIUM: quic: prevent EMSGSIZE with GSO for larger bufsize
    - MINOR: cli: Add a "help" keyword to show sess
    - MINOR: cli/quic: Add a "help" keyword to show quic
    - DOC: management: mention "show sess help" and "show quic help"
    - DOC: install: update the list of supported versions
    - MINOR: version: mention that 3.1 is stable now
2024-11-26 15:24:10 +01:00
Frederic Lecaille
3cee8d7830 DOC: quic: Amend the pacing information about BBR.
BBR handles itself its own burst size (mentioned as send_quantum in BBR RFC).
2024-11-26 08:00:58 +01:00
Willy Tarreau
7fb98e833c DOC: config: refine a little bit the text on QUIC pacing
The QUIC pacing options changed a few times during their development.
For example the unit is now in datagrams not bytes. Also a few
sentences were slightly ambiguous so let's reword this.

No backport is needed.
2024-11-25 14:54:16 +01:00
Willy Tarreau
450528b9f5 DOC: ot: mention planned deprecation of the OT filter
Miroslav mentioned below that he's currently working on an OpenTelemetry
replacement for the OpenTracing filter since OpenTracing itself is no
longer maintained nor supported:

  https://github.com/haproxy/haproxy/issues/2782#issuecomment-2493576327

Given that he aims for 3.2, let's already settle on an upcoming deprecation
of the filter for 3.3 with a removal for 3.5. This will leave time to finish
the development and permit users to switch smoothly. At this point no warning
is emitted (since the users have no alternative) but better mention this plan
in the doc to make them aware of future changes.
2024-11-22 16:11:51 +01:00
Willy Tarreau
c5d0342fa2 [RELEASE] Released version 3.1-dev14
Released version 3.1-dev14 with the following main changes :
    - MINOR: acl: export find_acl_default()
    - MINOR: sample: extend the "when" converter to support an ACL
    - MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{client,server} as sizes
    - MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{frontend,backend} as sizes
    - MINOR: cfgparse: parse tune.pipesize as a size
    - MINOR: cfgparse: parse tune.recv_enough as a size
    - MINOR: cfgparse: parse tune.bufsize as a size
    - MINOR: cfgparse: parse tune.bufsize.small as a size
    - REGTESTS: silence the "log format ignored" warnings
    - REGTESTS: silence warning "previous 'http-response' action is final"
    - REGTESTS: make the unit explicit for very short timeouts
    - REGTESTS: silence warnings about content-type being ignored
    - REGTESTS: remove a duplicate "option httpslog" in the defaults section
    - REGTESTS: silence warning "L6 sample fetches ignored" in cond_set_var
    - REGTESTS: add missing timeouts to 30 tests
    - REGTESTS: only use tune.ssl.default-dh-param when not using AWS-LC
    - REGTESTS: enable -dW on almost all tests to fail on warnings
    - MEDIUM: config: warn on unitless timeouts < 100 ms
    - MINOR: tools: make parse_size_err() support 32/64 bits
    - MINOR: ring: support unit suffixes in the size
    - MINOR: cfgparse-global: parse options to allow non std keywords in discovery mode
    - BUG/MINOR: mworker-prog: don't warn about deprecated section with expose-deprecated-directives
    - MINOR: cli: make "show env" accessible via master CLI without enabling debug
    - MINOR: config: show HAPROXY_BRANCH in "show env" output
    - MINOR: http-ana: Add option to keep query-string on a localtion-based redirect
    - MINOR: http-ana: Add support for "set-cookie-fmt" option to redirect rules
    - MINOR: agent-check: Be able to set absolute weight via an agent
    - MINOR: stream: Add an option to "show sess" command to dump the captured URI
    - DOC: config: A a space before ':' for {bs,fs}.aborted and {bs,fs}.rst_code
    - DOC: config: Fix a typo in "1.3.1. The Request line"
    - MINOR: http: Add support for HTTP 414/431 status codes
    - DEV: phash: Update 414 and 431 status codes to phash
    - MINIR: mux-h1: Return 414 or 431 when appropriate
    - BUG/MINOR: http_ana: Report -1 for %Tr for invalid response only
    - DOC: config: Slightly improve the %Tr documentation
    - DOC: config: Move wait_end in section about internal samples
    - DOC: config: Move fs.* and bs.* in section about L5 samples
    - MINOR: stats-file: add the filename in the warning
    - MEDIUM: stats-file: explicitely ignore comments starting by //
    - DOC: quic: rename max-window-size as with default prefix
    - MINOR: mux-quic: add missing values for show flags
    - MINOR: quic: simplify qc_prep_pkts() exit path
    - MINOR: quic: support a max number of built packet per send iteration
    - MINOR: quic: extend qc_send_mux() return type with a dedicated enum
    - MINOR: quic: define quic_pacing module
    - MINOR: quic/pacing: implement quic_pacer engine
    - MINOR: quic/pacing: support pacing emission on quic_conn layer
    - MINOR: quic/pacing: add burst support
    - MINOR: mux-quic: define a tx STREAM frame list member
    - MINOR: mux-quic: encapsulate QCC tasklet wakeup
    - MAJOR: mux-quic: support pacing emission
    - MINOR: quic: use dynamic cc_algo on bind_conf
    - MINOR: quic: extend quic-cc-algo optional parameters
    - MEDIUM: quic: define cubic-pacing congestion algorithm
    - MINOR: mux_quic/pacing: display pacing info on show quic
    - MEDIUM: stats-file: silently ignore be/fe mistmatch
    - REGTESTS: use -dW by default on every reg-tests
    - DOC: lua: fix yield-dependent methods expected contexts
    - DOC: sched: add missing scheduler API documentation for tasklet_wakeup_after()
    - DOC: sched: document the missing TASK_F_UEVT* flags
    - CLEANUP: tinfo: move sched_*_date/*_mono_time to the thread-local area
    - MINOR: stream: don't update s->lat_time when the wakeup date is not set
    - MINOR: tinfo/clock: turn sched_call_date to 64-bits
    - MINOR: sched: add TASK_F_WANTS_TIME to make the scheduler update the call date
    - MINOR: tools: add new macro DEFZERO to provide a default zero argument
    - MINOR: tasklet: make the low-level tasklet API take a flag
    - MINOR: tasklet: support an optional set of wakeup flags to tasklet_wakeup_on()
    - DOC: configuration: explain the rules regarding spaces in arguments
    - DOC: configuration: explain quotes and spaces in conditional blocks
    - DOC: configuration: wrap long line for "strstr()" conditional expression
    - BUG/MINOR: http-ana: Adjust the server status before the L7 retries
    - MINOR: http-fetch: Add an option to 'query" to get the QS with the '?'
    - BUG/MINOR: cfgparse-quic: fix renaming of max-window-size
    - MEDIUM: mworker: remove USE_SYSTEMD requirement for -Ws
    - CI: vtest: temporarily build from the sd-notify PR
    - MINOR: systemd: replace SOCK_CLOEXEC by fcntl call to FD_CLOEXEC
    - BUILD: makefile: make ERR apply to build options as well
    - MINOR: startup: set HAPROXY_LOCALPEER only once
    - DOC: configuration: update "Environment variables" chapter
    - DOC: config: indent the list of environment variables
    - OPTION: map/hlua: make core.set_map() lookup more efficient
    - REGTESTS: switch to -Ws for master-worker reg-tests
    - REGTESTS: disable temporarly mworker test on OSX
    - MINOR: quic: Add the congestion window initial value to QUIC path
    - MINOR: window_filter: Implement windowed filter (only max)
    - MINOR: quic: implement delivery rate sampling algorithm
    - MINOR: quic: implement BBR congestion control algorithm for QUIC
    - MINOR: quic: quic_cc modifications to support BBR
    - MINOR: quic: quic_loss modifications to support BBR
    - MINOR: quic: RX part modifications to support BBR
    - MINOR: quic: TX part modifications to support BBR.
    - MINOR: quic: add "bbr" new "quic-cc-algo" option
    - BUG/MEDIUM: mux-h2: Increase max number of headers when encoding HEADERS frames
    - BUG/MEDIUM: mux-h2: Check the number of headers in HEADERS frame after decoding
    - BUG/MEDIUM: h3: Properly limit the number of headers received
    - BUG/MEDIUM: h3: Increase max number of headers when sending headers
    - DOC: config: Improve documentation of tune.http.maxhdr directive
    - DOC: management: Clearly state "show errors" only reports malformed H1 messages
    - BUILD: makefile: build flags.c before haproxy to speed up the build
    - BUILD: makefile: reorder object files by build time
    - MINOR: config: Improve warnings on misplaced rules by adding an optional arg
    - CLEANUP: cfgparse: Add direction in functions name that warn on misplaced rules
    - MINOR: cfgparse: Emit a warning for misplaced "tcp-response content" rules
    - BUG/MINOR: cfgparse-quic: fix bbr initialization
    - MINOR: cfgparse-quic: activate pacing only via burst argument
    - MINOR: quic: Useless rate sample member initialization
    - BUG/MINOR: cfgparse-quic: fix warning for cc-aglo with 0 burst
    - MINOR: quic: support pacing for newreno and nocc
    - BUG/MINOR: quic: Missing application limitations tracking for BBR
    - MINOR: cfgparse-global: add cfg_parse_global_chroot
    - MINOR: cfgparse-global: add more checks for "chroot" argument
    - BUG/MINOR: startup: fix UAF when set the default for log_tag
    - MINOR: capabilities: rename program_name argument to progname
    - MINOR: startup: use global progname variable
    - MINOR: cfgparse-global: add cfg_parse_global_localpeer
    - BUG/MINOR: config: allow to check HAPROXY_LOCALPEER in config
    - BUG/MINOR: startup: init_early: remove obsolete comment
    - BUG/MEDIUM: debug: don't set the STUCK flag from debug_handler()
    - BUG/MEDIUM: wdt: fix the stuck detection for warnings
    - BUG/MINOR: activity/memprofile: reinitialize the free calls on DSO summary
    - MINOR: activity/memprofile: offer a function to unregister stale info
    - BUG/MEDIUM: pools/memprofile: always clean stale pool info on pool_destroy()
    - MINOR: activity: better report nil than ffff in unknown callers
    - CLEANUP: activity: better use a mask to tests freeing methods
    - MINOR: activity/memprofile: also monitor strdup() activity
    - MINOR: activity/memprofile: monitor non-portable calls as well
    - MINOR: activity: interrupt the show profile dump more often
    - MINOR: tools: resolve main() only once in resolve_sym_name()
    - MINOR: tools: add a new function "resolve_dso_name" to find a symbol's DSO
    - MINOR: activity/memprofile: use resolve_dso_name() for the DSO summary
    - REGTESTS: relax strerror matching to avoid a failure on libmusl
    - REGTESTS: don't rely on the base64 utility when openssl base64 is already used
2024-11-21 23:26:41 +01:00
Valentine Krasnobaeva
aa88d6ee37 BUG/MINOR: config: allow to check HAPROXY_LOCALPEER in config
This fixes the commit d6ccd1738bae
("MINOR: startup: set HAPROXY_LOCALPEER only once"). HAPROXY_LOCALPEER could
be checked in the configuration to set some servers settings or listeners. So,
we need to set it just before we read the configuration at the second time.

Let's mark HAPROXY_LOCALPEER as "usable" in the configuration in the related
documentation chapter.

This should not be backported, as related to the latest master-worker
refactoring.
2024-11-21 19:55:21 +01:00
Amaury Denoyelle
de86fd1e6c MINOR: cfgparse-quic: activate pacing only via burst argument
Recently, pacing support was added for cubic congestion algorithm. This
was activated by using the new token "cubic-pacing" on quic-cc-algo.
Furthermore, it was possible to define a burst size with a new
parameters after congestion token between parenthesis.

This configuration is not oblivious to users. In particular, it can
cause to easily forgot to tweak burst size, which can dramatically
impact performance.

Simplify this by removing the extra "-pacing" suffix. Now, pacing will
be activated solely based on the burst parameter. If 0, burst is
considered as infinite and no pacing will be used. Pacing will be
activating for any positive burst. This better reflects the link between
pacing and burst and its importance.

Note that for the moment, if burst is specified, it will be ignored with
a warning for algorithm outside of cubic.

This is not a breaking change as pacing support was implemented in the
current dev version.
2024-11-21 10:55:55 +01:00
Christopher Faulet
e863d8d681 DOC: config: Improve documentation of tune.http.maxhdr directive
The description was inproved to clrealy mentionned it is applied on received
requests and responses. In addition, a comment was added about HTTP/2 and
HTTP/3 limitation when messages are encoded to be sent.
2024-11-20 18:02:36 +01:00
Frederic Lecaille
349954601f MINOR: quic: add "bbr" new "quic-cc-algo" option
Add this new "bbr" option to the list of the congestion control algorithms which
may be set by "quic-cc-algo" setting.

This new algorithm is considered as experimental and may be enabled only if
"expose-experimental-directive" is set.

Also update the documentation for this new setting.
2024-11-20 17:34:22 +01:00
Willy Tarreau
311dc748b0 DOC: config: indent the list of environment variables
In the doc our lists are indented but for any reason this one was not,
making it harder to visually delimit. Let's just indent it. No need to
backport this, it's totally cosmetic and would need adaptations since
it was recently touched.
2024-11-20 15:57:09 +01:00
Valentine Krasnobaeva
41d906d69b DOC: configuration: update "Environment variables" chapter
There are some variables, which are set by HAProxy process (HAPROXY_*). Some
of them are handy to check or to redefine in the configuration, in order to
create conditional blocks and make the configuration more flexible. But it
wasn't clear in the documentation, which variables are really safe and usefull
to redefine and which ones could be only read via "show env" output.

Latest changes in master-worker architecture makes the existed description even
more confusing.

So let's sort all HAPROXY_* variables to four categories and let's also mark
explicitly, which ones are set in which process, when haproxy is started in
master-worker mode.

In addition, update examples in chapter "2.4. Conditional blocks". This might
bring more ideas for users how HAPROXY_* variables could be used in the
conditional blocks.
2024-11-20 15:56:50 +01:00
Christopher Faulet
17d4e6eaf9 MINOR: http-fetch: Add an option to 'query" to get the QS with the '?'
As mentionned by Thayne McCombs in #2728, it could be handy to have a sample
fetch function to retrieve the query string with the question mark
character.

Indeed, for now, "query" sample fetch function already extract the query
string from the path, but the question mark character is not
included. Instead of adding a new sample fetch function with a too similar
name, an optional argument is added to "query". If "with_qm" is passed as
argument, the question mark will be included in the query string, but only
if it is not empty.

Thanks to this patch, the following rule:

  http-request redirect location /destination?%[query] if { -m found query }  some_condition
  http-request redirect location /destination if some_condition

can now be expressed this way:

  http-request redirect location /destination%[query(with_qm)] if some_condition
2024-11-20 10:20:05 +01:00
Willy Tarreau
5c15899410 DOC: configuration: wrap long line for "strstr()" conditional expression
This keyword had too long a description line, let's split it. This can be
backported to 2.8.
2024-11-20 09:04:53 +01:00
Willy Tarreau
da1620b317 DOC: configuration: explain quotes and spaces in conditional blocks
Conditional blocks inherit the same tokenizer and argument parser as
the rest of the configuration, but are also silently concatenated
around groups of spaces and tabs. This can lead to subtle failures
for configs containing spaces around commas and parenthesis, where
a string comparison might silently fail for example. Let's better
document this particular case.

Thanks to Valentine for analysing and reporting the problem.

This can be backported to 2.4.
2024-11-20 09:04:53 +01:00
Willy Tarreau
962d5e038f DOC: configuration: explain the rules regarding spaces in arguments
Spaces around commas or parenthesis in expressions are generally part
of the value due to the long history of supporting unquoted arguments.
But this tends to come as a surprise to new users and sometimes creates
subtly invalid configurations. Let's add some text covering this.

This can be backported to 2.4.
2024-11-20 08:42:02 +01:00
Amaury Denoyelle
24cea66e07 MEDIUM: quic: define cubic-pacing congestion algorithm
Define a new QUIC congestion algorithm token 'cubic-pacing' for
quic-cc-algo bind keyword. This is identical to default cubic
implementation, except that pacing is used for STREAM frames emission.

This algorithm supports an extra argument to specify a burst size. This
is stored into a new bind_conf member named quic_pacing_burst which can
be reuse to initialize quic path.

Pacing support is still considered experimental. As such, 'cubic-pacing'
can only be used with expose-experimental-directives set.
2024-11-19 16:20:58 +01:00
Amaury Denoyelle
6dfc8fbf1d MINOR: quic: extend quic-cc-algo optional parameters
Modify quic-cc-algo for better extensability of optional parameters
parsing. This will be useful to support a new parameter for maximum
allowed pacing burst size.

Take this opportunity to refine quic-cc-algo documentation. Optional
parameters are now presented as a list which would be soon extended.
2024-11-19 16:20:52 +01:00
Amaury Denoyelle
8540886f00 DOC: quic: rename max-window-size as with default prefix
Rename 'tune.quic.frontend.max-window-size' with the prefix 'default-'.
This highlights the fact that it is not a hard limit, as it can be
overriden if specifying an optional window size via quic-cc-algo on a
bind line.

No need to backport as this keyword was added on the current dev
version.
2024-11-19 16:16:48 +01:00
Christopher Faulet
e68c6852ad DOC: config: Move fs.* and bs.* in section about L5 samples
These sample fetch functions were added in the wrong section. Move them in
the section about sample fetch functions at L5 layer.
2024-11-19 15:29:41 +01:00
Christopher Faulet
4ccc3f4048 DOC: config: Move wait_end in section about internal samples
wait_end is an internal sample fetch functions and not a L6 one. So move it
in the corresponding section.
2024-11-19 15:29:40 +01:00
Christopher Faulet
e9021a4ca1 DOC: config: Slightly improve the %Tr documentation
Specify -1 can also be reported for %Tr delay when the response is invalid.
2024-11-19 15:29:40 +01:00
Christopher Faulet
62dc8750a9 MINOR: http: Add support for HTTP 414/431 status codes
414-Uri-Too-Long and 431-Request-Header-Fields-Too-Large are now part of
supported status codes that can be define as error files. The hash table
defined in http_get_status_idx() was updated accordingly.
2024-11-19 15:29:40 +01:00
Christopher Faulet
18de419f96 DOC: config: Fix a typo in "1.3.1. The Request line"
At the beginning of the last paragraph of this section, HTTP/3 was used
instead of HTTP/2. It is not fixed.
2024-11-19 15:29:40 +01:00
Christopher Faulet
3af2d91b3b DOC: config: A a space before ':' for {bs,fs}.aborted and {bs,fs}.rst_code
A space was missing before the ':' for the sample fetch functions above. It
was an issue for the text to HTML conversion script. So, let's fix it.
2024-11-19 15:29:40 +01:00
Christopher Faulet
e9bc5937c9 MINOR: agent-check: Be able to set absolute weight via an agent
Historically, an agent-check program is only able to set a proportial weight
to the initial server's weight. However, it could be handy to also set an
absolute value. It is the purpose of this patch.

Instead of changing the current way to set a server's weight, a new
agent-check command is introduced. The string "weight:", followed by an
positive interger or a positive interger percentage, can now be used. If the
value ends with the '%' sign, then the new weight will be proportional to
the initially weight of the server. Otherwise, the value is considered as an
absolute weight and must be between 0 and 256.

This patch should fix the issue #360.
2024-11-19 15:29:40 +01:00
Christopher Faulet
1be7140ade MINOR: http-ana: Add support for "set-cookie-fmt" option to redirect rules
It is now possible to use a log-format string to define the "Set-Cookie"
header value of a response generated by a redirect rule. There is no special
check on the result format and it is not possible during the configuration
parsing. It is proably not a big deal because already existing "set-cookie"
and "clear-cookie" options don't perform any check.

Here is an example:

  http-request redirect location https://someurl.com/ set-cookie haproxy="%[var(txn.var)]"

This patch should fix the issue #1784.
2024-11-19 15:20:02 +01:00
Christopher Faulet
b2877db47c MINOR: http-ana: Add option to keep query-string on a localtion-based redirect
On prefix-based redirect, there is an option to drop the query-string of the
location. Here it is the opposite. an option is added to preserve the
query-string of the original URI for a localtion-based redirect.

By setting "keep-query" option, for a location-based redirect only, the
query-string of the original URI is appended to the location. If there is no
query-string, nothing is added (no empty '?'). If there is already a
non-empty query-string on the localtion, the original one is appended with
'&' separator.

This patch should fix issue #2728.
2024-11-19 15:20:02 +01:00
Willy Tarreau
e72b525832 MINOR: cfgparse: parse tune.bufsize.small as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "4k". Let's make use of parse_size_err() on it so that
units are supported. This requires to turn it to uint as well, which
was verified to be OK.
2024-11-18 19:07:05 +01:00
Willy Tarreau
a344d37fad MINOR: cfgparse: parse tune.bufsize as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, preventing from starting when set
e.g. to "64k". Let's make use of parse_size_err() on it so that units are
supported. This requires to turn it to uint as well, and to explicitly
limit its range to INT_MAX - 2*sizeof(void*), which was previously
partially handled as part of the sign check.
2024-11-18 19:06:25 +01:00
Willy Tarreau
2f0c6ff3a5 MINOR: cfgparse: parse tune.recv_enough as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on it so that
units are supported. This requires to turn it to uint as well, and
since it's sometimes compared to an int, we limit its range to
0..INT_MAX.
2024-11-18 19:01:28 +01:00
Willy Tarreau
a90a7d4d60 MINOR: cfgparse: parse tune.pipesize as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on it so that
units are supported. This requires to turn it to uint as well, which
was verified to be OK.
2024-11-18 18:51:31 +01:00
Willy Tarreau
f9f28b7584 MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{frontend,backend} as sizes
Till now these values were parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on them so that
units are supported. This requires to turn them to uint as well, which
is OK.
2024-11-18 18:50:02 +01:00
Willy Tarreau
a923c72357 MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{client,server} as sizes
Till now these values were parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on them so that
units are supported. This requires to turn them to uint as well, which
is OK.
2024-11-18 18:49:01 +01:00
Willy Tarreau
45f9e95f22 MINOR: sample: extend the "when" converter to support an ACL
Sometimes conditions to decide of an anomaly are not as easy to define
as just an error or a success. One example use case would be to monitor
the transfer time and fix a threshold.

An idea suggested by Tristan would be to make permit the "when"
converter to refer to a more variable or dynamic condition.

Here we make this possible by making "when" rely on a named ACL. The
ACL then needs to be specified in either the proxy or the defaults
section. Since it is evaluated inline, it may even refer to information
available at the end (at log time) such as the data transfer time. If
the ACL evalutates to true, the converter passes the data.

Example: log "dbg={-}" when fine, or "dbg={... debug info ...}" on slow
transfers:

  acl slow_xfer res.timer.data ge 10000   # more than 10s is slow
  log-format "$HAPROXY_HTTP_LOG_FMT                                \
              fsdbg={%[fs.debug_str,when(acl,slow_xfer)]}          \
              bsdbg={%[bs.debug_str,when(acl,slow_xfer)]}"
2024-11-18 16:11:55 +01:00
Willy Tarreau
9539f2b097 [RELEASE] Released version 3.1-dev13
Released version 3.1-dev13 with the following main changes :
    - MEDIUM: mworker: depreciate the 'program' section
    - BUILD: ot: use a cebtree instead of a list for variable names
    - MINOR: startup: replace HAPROXY_LOAD_SUCCESS with global load_status
    - BUG/MINOR: startup: set HAPROXY_CFGFILES in read_cfg
    - BUG/MINOR: cli: don't show sockpairs in HAPROXY_CLI and HAPROXY_MASTER_CLI
    - BUG/MEDIUM: stconn: Don't forward shut for SC in connecting state
    - BUG/MEDIUM: resolvers: Insert a non-executed resulution in front of the wait list
    - MINOR: debug: explicitly permit the counter condition to be empty
    - MINOR: debug: add a new counter type for glitches
    - MINOR: mux-h2: count glitches when they're reported
    - BUG/MINOR: deinit: release uri_auth admin rules
    - MINOR: uri_auth: add stats_uri_auth_free helper
    - MEDIUM: uri_auth: implement clean uri_auth cleaning
    - MINOR: mux-quic/h3: count glitches when they're reported
    - BUG/MEDIUM: mux-h2: Don't send RST_STREAM frame for streams with no ID
    - BUG/MINOR: Don't report early srv aborts on request forwarding in DONE state
    - MINOR: promex: Expose the global node and description in process metrics
    - MINOR: promex: Add global and proxies description as labels to all metrics
    - OPTIM: pattern: only apply LRU cache for large enough lists
    - BUG/MEDIUM: checks: make sure to always apply offsets to now_ms in expiration
    - BUG/MINOR: debug: do not set task expiration to TICK_ETERNITY
    - BUG/MEDIUM: mailers: make sure to always apply offsets to now_ms in expiration
    - BUG/MINOR: mux_quic: make sure to always apply offsets to now_ms in expiration
    - BUG/MINOR: peers: make sure to always apply offsets to now_ms in expiration
    - BUG/MEDIUM: clock: make sure now_ms cannot be TICK_ETERNITY
    - MINOR: debug/cli: replace "debug dev counters" with "debug counters"
    - DOC: config: add tune.h2.{be,fe}.rxbuf to the global keywords index
    - MINOR: chunk: add a BUG_ON upon the next init_trash_buffer()
2024-11-15 18:42:29 +01:00
Willy Tarreau
5f37af7a8e DOC: config: add tune.h2.{be,fe}.rxbuf to the global keywords index
These two keywords were missing from the index, let's add them.
2024-11-15 16:32:37 +01:00
William Lallemand
581c8a27d9 MEDIUM: mworker: depreciate the 'program' section
The program section is unreliable and should not be used, more reliable
alternatives exist outside HAProxy. Let's depreciate the section so we
could remove it completely in 3.3.
2024-11-08 17:06:58 +01:00
Willy Tarreau
0434e87348 [RELEASE] Released version 3.1-dev12
Released version 3.1-dev12 with the following main changes :
    - MINOR: startup: tune.renice.{startup,runtime} allow to change priorities
    - BUG/MEDIUM: promex: Fix dump of extra counters
    - BUILD: import/mt_list: support building with TCC
    - BUILD: compiler: define __builtin_prefetch() for tcc
    - CLEANUP: quic: Remove the useless directive "tune.quic.backend.max-idle-timeou"
    - DOC: config: document connection error 44 (reverse connect failure)
    - CLEANUP: connection: properly name the CO_ER_SSL_FATAL enum entry
    - DEBUG: cli: support closing "hard" using close() in addition to fd_delete()
    - MINOR: connection: add more connection error codes to cover common errno
    - MINOR: rawsock: set connection error codes when returning from recv/send/splice
    - MINOR: connection: add new sample fetch functions fc_err_name and bc_err_name
    - MINOR: quic: Help diagnosing malformed probing packets
    - BUG/MINOR: quic: fix malformed probing packet building
    - MINOR: listener: Remove useless checks on the receiver protocol existence
    - MINOR: http-conv: Remove unreachable goto statement in sample_conv_q_preferred
    - MINOR: http: don't %-encode the payload when not relevant
    - MINOR: quic: simplify qc_parse_pkt_frms() return path
    - MINOR: quic: use dynamically allocated frame on parsing
    - MINOR: quic: extend return value of CRYPTO parsing
    - BUG/MINOR: quic: repeat packet parsing to deal with fragmented CRYPTO
    - BUG/MINOR: mworker: do 'program' postparser checks in read_cfg_in_discovery_mode
    - EXAMPLES: add "traces.cfg" with traces examples
    - BUG/MEDIUM: quic: do not consider ACK on released stream as error
    - CLEANUP: stats: fix misleading comment on top of stat_idx_info
    - MINOR: wdt: move the local timers to a struct
    - MINOR: debug: add a function to dump a stuck thread
    - DEBUG: wdt: better detect apparently locked up threads and warn about them
    - DEBUG: cli: make it possible for "debug dev loop" to trigger warnings
    - DEBUG: wdt: make the blocked traffic warning delay configurable
    - DEBUG: wdt: add a stats counter "BlockedTrafficWarnings" in show info
    - DEBUG: wdt: set the default blocked task delay to 100 ms
    - MINOR: debug: move the "recover now" warn message after the optional notes
    - MINOR: event_hdl: add event_hdl_sub_list_empty() helper func
    - MINOR: pattern: add _pat_ref_new() helper func
    - OPTIM: pattern: use malloc() to initialize new pat_ref struct
    - MINOR: pattern: add pat_ref_free() helper func
    - CLEANUP: guid: remove global tree export
    - BUG/MINOR: guid/server: ensure thread-safety on GUID insert/delete
    - DOC: management: explain the change of behavior of the program section
    - BUG/MEDIUM: mux-h2: try to wait for the peer to read the GOAWAY
    - BUG/MEDIUM: quic: prevent crash due to CRYPTO parsing error
2024-11-08 15:46:54 +01:00
William Lallemand
75b302d123 DOC: management: explain the change of behavior of the program section
The program section does not work exactly the same way with the
master-worker rework of HAProxy 3.1. Let's explain it in the program
documentation.
2024-11-08 12:00:26 +01:00
Willy Tarreau
5f4fe20116 DEBUG: wdt: set the default blocked task delay to 100 ms
The warn-blocked-traffic-after can be significantly lowered. In any
case, in order to be usable it must be well below the limit to have a
chance to emit exploitable traces before the watchdog finally fires.
Even configured at 1ms it looks very difficult to trigger it on a
laptop doing SSL and compression, so applying a 100-fold factor to
cover for large configs and small machines sounds sane for 3.1. In any
case, even at 100ms, the service degradation becomes quite visible.
2024-11-06 18:35:42 +01:00
Willy Tarreau
6127e5a4e9 DEBUG: wdt: make the blocked traffic warning delay configurable
The new global "warn-blocked-traffic-after" allows one to configure
after how much time a warning should be emitted when traffic is blocked.
2024-11-06 18:35:42 +01:00
Willy Tarreau
601b34fe7b MINOR: connection: add new sample fetch functions fc_err_name and bc_err_name
These functions return a symbolic error code such as ECONNRESET to keep
logs compact while making them human-readable. It's a good alternative
to the numeric code in that it's more expressive, and a good one to the
full message since it's shorter and more precise (some codes even match
errno names).

The doc was updated so that the symbolic names appear in the table. It
could be useful to backport this feature to help with troubleshooting
some issues, though backporting the doc might possibly be more annoying
in case users have local patches already, so maybe the table update does
not need to be backported in this case.
2024-11-05 18:57:43 +01:00
Willy Tarreau
00c383ff65 MINOR: connection: add more connection error codes to cover common errno
While we get reports of connection setup errors in fc_err/bc_err, we
don't have the equivalent for the recv/send/splice syscalls. Let's
add provisions for new codes that cover the common errno values that
recv/send/splice can return, i.e. ECONNREFUSED, ENOMEM, EBADF, EFAULT,
EINVAL, ENOTCONN, ENOTSOCK, ENOBUFS, EPIPE. We also add a special case
for when the poller reported the error itself. It's worth noting that
EBADF/EFAULT/EINVAL will generally indicate serious bugs in the code
and should not be reported.

The only thing is that it's quite hard to forcefully (and reliably)
trigger these errors in automated tests as the timing is critical.
Using iptables to manually reset established connections in the
middle of large transfers at least permits to see some ECONNRESET
and/or EPIPE, but the other ones are harder to trigger.
2024-11-05 18:57:43 +01:00
Willy Tarreau
abed9e0426 DOC: config: document connection error 44 (reverse connect failure)
It was missing from commit ac1164de7c ("MINOR: connection: define error
for reverse connect"), and can be backported to 3.0 and 2.9.
2024-11-05 18:57:42 +01:00