"show profiling" by default sorts by usage/counts, which is suitable for
occasional use. But when called from scripts to monitor/search variations,
this is not very convenient. Let's add a new "byaddr" option to support
sorting the output by address. It also eases matching alloc/free calls
from within a same library, or reading grouped tasks costs by library.
Current example is:
`echo "reload" | socat /var/run/haproxy-master.sock`
it will cause socat error:
`exactly 2 addresses required (there are 1); use option "-h" for help`
Correct working command is:
`echo "reload" | socat /var/run/haproxy-master.sock stdin`
Released version 2.4-dev19 with the following main changes :
- BUG/MINOR: hlua: Don't rely on top of the stack when using Lua buffers
- BUG/MEDIUM: cli: prevent memory leak on write errors
- BUG/MINOR: ssl/cli: fix a lock leak when no memory available
- MINOR: debug: add a new "debug dev sym" command in expert mode
- MINOR: pools/debug: slightly relax DEBUG_DONT_SHARE_POOLS
- CI: Github Actions: switch to LibreSSL-3.3.3
- MINOR: srv: close all idle connections on shutdown
- MINOR: connection: move session_list member in a union
- MEDIUM: mux_h1: release idling frontend conns on soft-stop
- MEDIUM: connection: close front idling connection on soft-stop
- MINOR: tools: add functions to retrieve the address of a symbol
- CLEANUP: activity: mark the profiling and task_profiling_mask __read_mostly
- MINOR: activity: add a "memory" entry to "profiling"
- MINOR: activity: declare the storage for memory usage statistics
- MEDIUM: activity: collect memory allocator statistics with USE_MEMORY_PROFILING
- MINOR: activity: clean up the show profiling io_handler a little bit
- MINOR: activity: make "show profiling" support a few arguments
- MINOR: activity: make "show profiling" also dump the memoery usage
- MINOR: activity: add the profiling.memory global setting
- BUILD: makefile: add new option USE_MEMORY_PROFILING
- MINOR: channel: Rely on HTX version if appropriate in channel_may_recv()
- BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive
- MINOR: conn-stream: Force mux to wait for read events if abortonclose is set
- MEDIUM: mux-h1: Don't block reads when waiting for the other side
- BUG/MEDIUM: mux-h1: Properly report client close if abortonclose option is set
- REGTESTS: Add script to test abortonclose option
- MINOR: mux-h1: clean up conditions to enabled and disabled splicing
- MINOR: mux-h1: Subscribe for sends if output buffer is not empty in h1_snd_pipe
- MINOR: mux-h1: Always subscribe for reads when splicing is disabled
- MEDIUM: mux-h1: Wake H1 stream when both sides a synchronized
- CLEANUP: mux-h1: rename WAIT_INPUT/WAIT_OUTPUT flags
- MINOR: mux-h1: Manage processing blocking flags on the H1 stream
- BUG/MINOR: stream: Decrement server current session counter on L7 retry
- BUG/MINOR: config: fix uninitialized initial state in ".if" block evaluator
- BUG/MINOR: config: add a missing "ELIF_TAKE" test for ".elif" condition evaluator
- BUG/MINOR: config: .if/.elif should also accept negative integers
- MINOR: config: centralize the ".if"/".elif" condition parser and evaluator
- MINOR: config: keep up-to-date current file/line/section in the global struct
- MINOR: config: support some pseudo-variables for file/line/section
- BUILD: activity: do not include malloc.h
- MINOR: arg: improve the error message on missing closing parenthesis
- MINOR: global: export the build features string list
- MINOR: global: add version comparison functions
- MINOR: config: improve .if condition error reporting
- MINOR: config: make cfg_eval_condition() support predicates with arguments
- MINOR: config: add predicate "defined()" to conditional expression blocks
- MINOR: config: add predicates "streq()" and "strneq()" to conditional expressions
- MINOR: config: add predicate "feature" to detect certain built-in features
- MINOR: config: add predicates "version_atleast" and "version_before" to cond blocks
- BUG/MINOR: activity: use the new pointer to calculate the new size in realloc()
- BUG/MINOR: stream: properly clear the previous error mask on L7 retries
- MEDIUM: log: slightly refine the output format of alerts/warnings/etc
- MINOR: config: add a new message directive: .diag
- CLEANUP: cli/tree-wide: properly re-align the CLI commands' help messages
- BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
- BUG/MINOR: checks: Handle synchronous connect when a tcpcheck is started
- BUG/MINOR: checks: Reschedule check on observe mode only if fastinter is set
- MINOR: global: define tainted flag
- MINOR: cfgparse: add a new field flags in cfg_keyword
- MINOR: cfgparse: implement experimental config keywords
- MINOR: action: replace match_pfx by a keyword flags field
- MINOR: action: implement experimental actions
- MINOR: cli: set tainted when using CLI expert/experimental mode
- MINOR: stats: report tainted on show info
- MINOR: http_act: mark normalize-uri as experimental
- BUILD: fix usage of ha_alert without format string
- MINOR: proxy: define PR_CAP_LB
- BUG/MINOR: server: do not report diag for peer servers with null weight
- DOC: ssl: Extra files loading now works for backends too
- ADDONS: make addons/ discoverable by git via .gitignore
- DOC: ssl: Add information about crl-file option
- MINOR: sample: improve error reporting on missing arg to strcmp() converter
- DOC: management: mention that some fields may be emitted as floats
- MINOR: tools: implement trimming of floating point numbers
- MINOR: tools: add a float-to-ascii conversion function
- MINOR: freq_ctr: add new functions to report float measurements
- MINOR: stats: avoid excessive padding of float values with trailing zeroes
- MINOR: stats: add the HTML conversion for float types
- MINOR: stats: pass the appctx flags to stats_fill_info()
- MINOR: stats: support an optional "float" option to "show info"
- MINOR: stats: use tv_remain() to precisely compute the uptime
- MINOR: stats: report uptime and start time as floats with subsecond resolution
- MINOR: stats: make "show info" able to report rates as floats when asked
- MINOR: config: mark tune.fd.edge-triggered as experimental
- REORG: vars: move the "proc" scope variables out of the global struct
- REORG: threads: move all_thread_mask() to thread.h
- BUILD: wdt: include signal-t.h
- BUILD: auth: include missing list.h
- REORG: mworker: move proc_self from global to mworker
- BUILD: ssl: ssl_utils requires chunk.h
- BUILD: config: cfgparse-ssl.c needs tools.h
- BUILD: wurfl: wurfl.c needs tools.h
- BUILD: spoe: flt_spoe.c needs tools.h
- BUILD: promex: service-prometheus.c needs tools.h
- BUILD: resolvers: include tools.h
- BUILD: config: include tools.h in cfgparse-listen.c
- BUILD: htx: include tools.h in http_htx.c
- BUILD: proxy: include tools.h in proxy.c
- BUILD: session: include tools.h in session.c
- BUILD: cache: include tools.h in cache.c
- BUILD: sink: include tools.h in sink.c
- BUILD: connection: include tools.h in connection.c
- BUILD: server-state: include tools.h from server_state.c
- BUILD: dns: include tools.h in dns.c
- BUILD: payload: include tools.h in payload.c
- BUILD: vars: include tools.h in vars.c
- BUILD: compression: include tools.h in compression.c
- BUILD: mworker: include tools.h from mworker.c
- BUILD: queue: include tools.h from queue.c
- BUILD: udp: include tools.h from proto_udp.c
- BUILD: stick-table: include freq_ctr.h from stick_table.h
- BUILD: server: include tools.h from server.c
- BUILD: server: include missing proxy.h in server.c
- BUILD: sink: include proxy.h in sink.c
- BUILD: mworker: include proxy.h in mworker.c
- BUILD: filters: include proxy.h in filters.c
- BUILD: fcgi-app: include proxy.h in fcgi-app.c
- BUILD: connection: move list_mux_proto() to connection.c
- REORG: stick-table: uninline stktable_alloc_data_type()
- REORG: stick-table: move composite address functions to stick_table.h
- REORG: config: uninline warnifnotcap() and failifnotcap()
- BUILD: task: remove unused includes from task.c
- MINOR: task: stop including stream.h from task.c
- BUILD: connection: stop including listener-t.h
- BUILD: hlua: include proxy.h from hlua.c
- BUILD: mux-h1: include proxy.h from mux-h1.c
- BUILD: mux-fcgi: include proxy.h from mux-fcgi.c
- BUILD: listener: include proxy.h from listener.c
- BUILD: http-rules: include proxy.h from http_rules.c
- BUILD: thread: include log.h from thread.c
- BUILD: comp: include proxy.h from flt_http_comp.c
- BUILD: fd: include log.h from fd.c
- BUILD: config: do not include proxy.h nor errors.h anymore in cfgparse.h
- BUILD: makefile: reorder object files by build time
- DOC: Fix a few grammar/spelling issues and casing of HAProxy
- REGTESTS: run-regtests: match both "HAProxy" and "HA-Proxy" in the version
- MINOR: version: report "HAProxy" not "HA-Proxy" in the version output
- DOC: remove last occurrences of "HA-Proxy" syntax
- DOC: peers: fix the protocol tag name in the doc
- ADMIN: netsnmp: report "HAProxy" and not "Haproxy" in output descriptions
- MEDIUM: mailers: use "HAProxy" nor "HAproxy" in the subject of messages
- DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
- MINOR: tools/rnd: compute the result outside of the CAS loop
- BUILD: http_fetch: address a few aliasing warnings with older compilers
- BUILD: ssl: define HAVE_CRYPTO_memcmp() based on the library version
- BUILD: errors: include stdarg in errors.h
- REGTESTS: disable inter-thread idle connection sharing on sensitive tests
- MINOR: cli: make "help" support a command in argument
- MINOR: cli: sort the output of the "help" keywords
- CLEANUP: cli/mworker: properly align the help messages
- BUILD: memprof: make the old caller pointer a const in get_prof_bin()
- BUILD: compat: include malloc_np.h for USE_MEMORY_PROFILING on FreeBSD
- CI: Github Actions: enable USE_QUIC=1 for BoringSSL builds
- BUG/MEDIUM: quic: fix null deref on error path in qc_conn_init()
- BUILD: cli: appease a null-deref warning in cli_gen_usage_msg()
With ~100 commands on the CLI, it's particularly difficult to find a
specific one in the "help" output. The function used to display the
help already supports filtering on certain commands, so in the end it's
just needed to pass the argument of the help command to enable the
automatic filtering. That's what this patch does so that "help clear"
only lists commands starting with "clear" and that "help map" lists
commands containing "map" in them.
Some of the Lua doc and a few places still used "Haproxy" or "HAproxy".
There was even one "HA proxy". A few of them were in an example of VTest
output, indicating that VTest ought to be fixed as well. No big deal but
better address all the remaining ones so that these inconsistencies stop
spreading around.
The peers protocol has been using "HAProxyS" as a binary tag sent on the
wire since day one in 1.5-dev3 with commit 2b920a1af ("[MAJOR] Add new
files src/peer.c, include/proto/peers.h and include/types/peers.h for
sync stick table management"), regardless, the doc says the protocol
identifier is "HaproxyS". It is likely this got fixed in the code before
merging and not in the doc.
This should be backported to any release as the doc is wrong.
There were only a few more used as output examples and comments in a few
docs, it was the right moment to get rid of them. The file intro.txt
which explains how to parse the version also got a hint about the possible
presence of a hyphen in the name in older versions.
This patch fixes a few grammar and spelling issues in configuration.txt.
It was also noted that there was a wide range of case usage
(i.e. haproxy, HAproxy, HAProxy, etc... ). This patch updates them
all to be consistently "HAProxy" except where a binary is mentioned.
This will allow some fields to be produced with a higher accuracy when
the requester indicates being able to parse floats. Rates and times are
among the elements which can make sense.
When using the crl-file option with multiple Certificate Authority
levels in the CA chain, there must be one CRL per CA or the verify
function on the backend side will raise an "unagle to get certificate
CRL" error (error code 3).
This was required by GitHub issue #1201.
When implementing the server side certificate hot update, the ckch
mechanism was used on the backend side in order to mimic the frontend
certificate management and to enable server line certificate update via
the CLI (see GitHub issue #427). As an unexpected side effect, we now
also look for ssl extra files (cert.pem.key, cert.pem.ocsp ...) for the
backend side.
This patch updates the documentation accordingly.
This answers to GitHub issue #845.
normalize-uri http rule is marked as experimental, so it cannot be
activated without the global 'expose-experimental-directives'. The
associated vtc is updated to be able to use it.
Add a new flag to mark a keyword as experimental. An experimental
keyword cannot be used if the global 'expose-experimental-directives' is
not present first.
Only keywords parsed through a standard cfg_keywords lists in
global/proxies section will be automatically detected if declared
experimental. To support a keyword outside of these lists,
check_kw_experimental must be called manually during its parsing.
If an experimental keyword is present in the config, the tainted flag is
updated.
For the moment, no keyword is marked as experimental.
This one works just like .notice/.warning/.alert except that it prints
the message at level "DIAG" only when haproxy runs in diagnostic mode
(-dD). This can be convenient for example to pass a few hints to help
locate certain config parts or to leave messages about certain temporary
workarounds.
Example:
.diag "WTA/2021-05-07: $.LINE: replace 'redirect' with 'return' after final switch to 2.4"
http-request redirect location /goaway if ABUSE
For about 20 years we've been emitting cryptic messages on warnings and
alerts, that nobody knows how to parse:
[NOTICE] 126/080118 (3115) : haproxy version is 2.4-dev18-0b7c78-49
[NOTICE] 126/080118 (3115) : path to executable is ./haproxy
[WARNING] 126/080119 (3115) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 126/080119 (3115) : backend 'default' has no server available!
Hint: the first 3-digit number is the day of year, and the 6 digits
after it represent the time of day in format HHMMSS, then the pid in
parenthesis. These are not quite user-friendly and such cryptic into
are not useful at all.
This patch slightly adjusts the output by performing these minimal changes:
- removing the date/time, as they were added very early when haproxy
was meant to be used in foreground as a debugging tool, and they're
provided in more details in logs nowadays ;
- better aligning the fields by padding the severity tag to 10 chars.
The diag output was renamed to "DIAG" only.
Now the output provides this:
[NOTICE] (4563) : haproxy version is 2.4-dev18-75a428-51
[NOTICE] (4563) : path to executable is ./haproxy
[WARNING] (4563) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] (4563) : backend 'default' has no server available!
The useless space before the colon was kept so as not to confuse any
possible output parser.
The few entries in the doc referring to this format were adjusted to
reflect the new one.
The change was tagged "MEDIUM" as it may have visible consequences on
home-grown monitoring tools, though it is extremely unlikely due to the
limited extent of these changes.
These predicates respectively verify that the current version is at least
a given version or is before a specific one. The syntax is exactly the one
reported by "haproxy -v", though each component is optional, so both "1.5"
and "2.4-dev18-88910-48" are supported. Missing components equal zero, and
"dev" is below "pre" or "rc", which are both inferior to no such mention
(i.e. they are negative). Thus "2.4-dev18" is older than "2.4-rc1" which
is older than "2.4".
The "feature(name)" predicate will return true if <name> corresponds to
a name listed after a '+' in the features list, that is it was enabled at
build time with USE_<name>=1. Typical use cases will include OPENSSL, LUA
and LINUX_SPLICE. But maybe it will also be convenient to use with optional
addons such as PROMEX and the device detection modules to help keeping the
same configs across various deployments.
"streq(str1,str2)" will return true if the two strings match while
"strneq(str1,str2)" will return true only if they differ. This is
convenient to match an environment variable against a predefined value.
The new pseudo-variables ".FILE", ".LINE" and ".SECTION" will be resolved
on the fly by the config parser and will respectively retrieve the current
configuration file name, the current line number and the current section
being parsed. This may help emit logs, errors, and debugging information
(e.g. which rule matched).
The '.' in the first char was reserved for such pseudo-variables and no
other variable is permitted. This will allow to add support for new ones
in the future if they prove to be useful (e.g. randoms/uuid for secret
keying or automatic naming of configuration objects).
This adds the necessary flags to permit run-time enabling/disabling of
memory profiling. For now this is disabled.
A few words were added to the management doc about it and recalling that
this is limited to certain OSes.
Released version 2.4-dev18 with the following main changes :
- DOC: Fix indentation for `path-strip-dot` normalizer
- DOC: Fix RFC reference for the percent-to-uppercase normalizer
- DOC: Add RFC references for the path-strip-dot(dot)? normalizers
- MINOR: uri_normalizer: Add a `percent-decode-unreserved` normalizer
- BUG/MINOR: mux-fcgi: Don't send normalized uri to FCGI application
- REORG: htx: Inline htx functions to add HTX blocks in a message
- CLEANUP: assorted typo fixes in the code and comments
- DOC: general: fix white spaces for HTML converter
- BUG/MINOR: ssl: ssl_sock_prepare_ssl_ctx does not return an error code
- BUG/MINOR: cpuset: move include guard at the very beginning
- BUG/MAJOR: fix build on musl with cpu_set_t support
- BUG/MEDIUM: cpuset: fix build on MacOS
- BUG/MINOR: htx: Preserve HTX flags when draining data from an HTX message
- MEDIUM: htx: Refactor htx_xfer_blks() to not rely on hdrs_bytes field
- CLEANUP: htx: Remove unsued hdrs_bytes field from the HTX start-line
- BUG/MINOR: mux-h2: Don't encroach on the reserve when decoding headers
- MEDIUM: http-ana: handle read error on server side if waiting for response
- MINOR: htx: Limit length of headers name/value when a HTX message is dumped
- BUG/MINOR: applet: Notify the other side if data were consumed by an applet
- BUG/MINOR: hlua: Don't consume headers when starting an HTTP lua service
- BUG/MEDIUM: mux-h2: Handle EOM flag when sending a DATA frame with zero-copy
- CLEANUP: channel: No longer notify the producer in co_skip()/co_htx_skip()
- DOC: general: fix example in set-timeout
- CLEANUP: cfgparse: de-uglify early file error handling in readcfgfile()
- MINOR: config: add a new "default-path" global directive
- BUG/MEDIUM: peers: initialize resync timer to get an initial full resync
- BUG/MEDIUM: peers: register last acked value as origin receiving a resync req
- BUG/MEDIUM: peers: stop considering ack messages teaching a full resync
- BUG/MEDIUM: peers: reset starting point if peers appears longly disconnected
- BUG/MEDIUM: peers: reset commitupdate value in new conns
- BUG/MEDIUM: peers: re-work updates lookup during the sync on the fly
- BUG/MEDIUM: peers: reset tables stage flags stages on new conns
- MINOR: peers: add informative flags about resync process for debugging
- BUG/MEDIUM: time: fix updating of global_now upon clock drift
- CLEANUP: freq_ctr: make arguments of freq_ctr_total() const
- CLEANUP: hlua: rename hlua_appctx* appctx to luactx
- MINOR: server: fix doc/trace on lb algo for dynamic server creation
- REGTESTS: server: fix cli_add_server due to previous trace update
- REGTESTS: add minimal CLI "add map" tests
- DOC: management: move "set var" to the proper place
- CLEANUP: map: slightly reorder the add map function
- MINOR: map: get rid of map_add_key_value()
- MINOR: map: show the current and next pattern version in "show map"
- MINOR: map/acl: add the possibility to specify the version in "show map/acl"
- MINOR: pattern: support purging arbitrary ranges of generations
- MINOR: map/acl: add the possibility to specify the version in "clear map/acl"
- MINOR: map/acl: add the "prepare map/acl" CLI command
- MINOR: map/acl: add the "commit map/acl" CLI command
- MINOR: map/acl: make "add map/acl" support an optional version number
- CLEANUP: map/cli: properly align the map/acl help
- BUILD: compiler: do not use already defined __read_mostly on dragonfly
By passing a version number to "add map/acl", it becomes possible to
atomically replace maps and ACLs. The principle is that a new version
number is first retrieved by calling"prepare map/acl", and this version
number is used with "add map" and "add acl". Newly added entries then
remain invisible to the matching mechanism but are visible in "show
map/acl" when the version number is specified, or may be cleard with
"clear map/acl". Finally when the insertion is complete, a
"commit map/acl" command must be issued, and the version is atomically
updated so that there is no intermediate state with incomplete entries.
The command is used to atomically replace a map/acl with the pending
contents of the designated version. The new version must have been
allocated by "prepare map/acl" prior to this. At the moment it is not
possible to force the version when adding new entries, so this may only
be used to atomically clear an ACL/map.
This command allocates a new version for the map/acl, that will be usable
later to prepare the addition of new values to atomically replace existing
ones. Technically speaking the operation consists in atomically incrementing
the next version. There's no "undo" operation here, if a version is not
committed, it will automatically be trashed when committing a newer version.
This will ease maintenance of versionned maps by allowing to clear old or
failed updates instead of the current version. Nothing was done to allow
clearing everyhing, though if there was a need for this, implementing "@all"
or something equivalent wouldn't require more than 3 lines of code.
The maps and ACLs internally all have two versions, the "current" one,
which is the one being matched against, and the "next" one, the one being
filled during an atomic replacement. Till now the "show" commands only used
to show the current one but it can be convenient to be able to show other
ones as well, so let's add the ability to do this with "show map" and
"show acl". The method used here consists in passing the version number
as "@<ver>" before the map/acl name or ID. It would have been better after
it but that could create confusion with keys already using such a format.
Commit b8bd1ee89 ("MEDIUM: cli: add a new experimental "set var" command")
added "get var" and "set var" but "set var" was misplaced in the doc,
breaking the alphabetic ordering.
The text mentionned that only backends with consistent hash method were
supported for dynamic servers. In fact, it is only required that the lb
algorith is dynamic.
By default haproxy loads all files designated by a relative path from the
location the process is started in. In some circumstances it might be
desirable to force all relative paths to start from a different location
just as if the process was started from such locations. This is what this
directive is made for. Technically it will perform a temporary chdir() to
the designated location while processing each configuration file, and will
return to the original directory after processing each file. It takes an
argument indicating the policy to use when loading files whose path does
not start with a slash ('/').
A few options are offered, "current" (the default), "config" (files
relative to config file's dir), "parent" (files relative to config file's
parent dir), and "origin" with an absolute path.
This should address issue #1198.
The alternative arguments are always in curly brackets, let's fix it for
set-timeout.
The Example in set-timeout does not have the one of the required argument.
This commit makes the PR https://github.com/cbonte/haproxy-dconv/pull/34
obsolete.
Released version 2.4-dev17 with the following main changes :
- MINOIR: mux-pt/trace: Register a new trace source with its events
- BUG/MINOR: mux-pt: Fix a possible UAF because of traces in mux_pt_io_cb
- CI: travis: Drastically clean up .travis.yml
- CLEANUP: pattern: make all pattern tables read-only
- MINOR: trace: replace the trace() inline function with an equivalent macro
- MINOR: initcall: uniformize the section names between MacOS and other unixes
- CLEANUP: initcall: rename HA_SECTION to HA_INIT_SECTION
- MINOR: compiler: add macros to declare section names
- CLEANUP: initcall: rely on HA_SECTION_* instead of defining its own
- MINOR: global: declare a read_mostly section
- MINOR: fd: move a few read-mostly variables to their own section
- MINOR: epoll: move epoll_fd to read_mostly
- MINOR: kqueue: move kqueue_fd to read_mostly
- MINOR: pool: move pool declarations to read_mostly
- MINOR: threads: mark all_threads_mask as read_mostly
- MINOR: server: move idle_conn_task to read_mostly
- MINOR: protocol: move __protocol_by_family to read_mostly
- MINOR: pattern: make the pat_lru_seed read_mostly
- MINOR: trace: make trace sources read_mostly
- MINOR: freq_ctr: add a generic function to report the total value
- MEDIUM: freq_ctr: make read_freq_ctr_period() use freq_ctr_total()
- MEDIUM: freq_ctr: reimplement freq_ctr_remain_period() from freq_ctr_total()
- MINOR: freq_ctr: add the missing next_event_delay_period()
- MINOR: freq_ctr: unify freq_ctr and freq_ctr_period into freq_ctr
- MEDIUM: freq_ctr: replace the per-second counters with the generic ones
- MINOR: freq_ctr: add cpu_relax in the rotation loop of update_freq_ctr_period()
- MINOR: freq_ctr: simplify and improve the update function
- CLEANUP: time: remove the now unused ms_left_scaled
- MINOR: time: move the time initialization out of tv_update_date()
- MINOR: time: remove useless variable copies in tv_update_date()
- MINOR: time: change the global timeval and the the global tick at once
- MEDIUM: time: make the clock offset global and no per-thread
- MINOR: atomic: reimplement the relaxed version of x86 BTS/BTR
- MINOR: trace: Add the checks as a possible trace source
- MINOIR: checks/trace: Register a new trace source with its events
- MINOR: hlua: Add function to release a lua function
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a task
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a converter
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a fetch
- BUG/MINOR: hlua: Fix memory leaks on error path when parsing a lua action
- BUG/MINOR: hlua: Fix memory leaks on error path when registering an action
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a service
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a cli keyword
- BUG/MINOR: cfgparse/proxy: Fix some leaks during proxy section parsing
- BUG/MINOR: listener: Handle allocation error when allocating a new bind_conf
- BUG/MINOR: cfgparse/proxy: Hande allocation errors during proxy section parsing
- MINOR: cfgparse/proxy: Group alloc error handling during proxy section parsing
- DOC: internals: update the SSL architecture schema
- BUG/MEDIUM: sample: Fix adjusting size in field converter
- MINOR: sample: add ub64dec and ub64enc converters
- CLEANUP: sample: align samples list in sample.c
- MINOR: ist: Add `istclear(struct ist*)`
- CI: cirrus: install "pcre" package
- MINOR: opentracing: correct calculation of the number of arguments in the args[]
- MINOR: opentracing: transfer of context names without prefix
- MINOR: sample: converter: Add mjson library.
- MINOR: sample: converter: Add json_query converter
- CI: travis-ci: enable weekly graviton2 builds
- DOC: ssl: Certificate hot update only works on fronted certificates
- DOC: ssl: Certificate hot update works on server certificates
- BUG/MEDIUM: threads: Ignore current thread to end its harmless period
- MINOR: threads: Only consider running threads to end a thread harmeless period
- BUG/MINOR: checks: Set missing id to the dummy checks frontend
- MINOR: logs: Add support of checks as session origin to format lf strings
- BUG/MINOR: connection: Fix fc_http_major and bc_http_major for TCP connections
- MINOR: connection: Make bc_http_major compatible with tcp-checks
- BUG/MINOR: ssl-samples: Fix ssl_bc_* samples when called from a health-check
- BUG/MINOR: http-fetch: Make method smp safe if headers were already forwarded
- MINOR: tcp_samples: Add samples to get src/dst info of the backend connection
- MINOR: tcp_samples: Be able to call bc_src/bc_dst from the health-checks
- BUG/MINOR: http_htx: Remove BUG_ON() from http_get_stline() function
- BUG/MINOR: logs: Report the true number of retries if there was no connection
- BUILD: makefile: Redirect stderr to /dev/null when probing options
- MINOR: uri_normalizer: Add uri_normalizer module
- MINOR: uri_normalizer: Add `enum uri_normalizer_err`
- MINOR: uri_normalizer: Add `http-request normalize-uri`
- MINOR: uri_normalizer: Add a `merge-slashes` normalizer to http-request normalize-uri
- MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri
- MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer
- MINOR: uri_normalizer: Add a `sort-query` normalizer
- MINOR: uri_normalizer: Add a `percent-upper` normalizer
- MEDIUM: http_act: Rename uri-normalizers
- DOC: Add introduction to http-request normalize-uri
- DOC: Note that URI normalization is experimental
- BUG/MINOR: pools: maintain consistent ->allocated count on alloc failures
- BUG/MINOR: pools/buffers: make sure to always reserve the required buffers
- MINOR: pools: drop the unused static history of artificially failed allocs
- CLEANUP: pools: remove unused arguments to pool_evict_from_cache()
- MEDIUM: pools: move the cache into the pool header
- MINOR: pool: remove the size field from pool_cache_head
- MINOR: pools: rename CONFIG_HAP_LOCAL_POOLS to CONFIG_HAP_POOLS
- MINOR: pools: enable the fault injector in all allocation modes
- MINOR: pools: make the basic pool_refill_alloc()/pool_free() update needed_avg
- MEDIUM: pools: unify pool_refill_alloc() across all models
- CLEANUP: pools: re-merge pool_refill_alloc() and __pool_refill_alloc()
- MINOR: pools: call pool_alloc_nocache() out of the pool's lock
- CLEANUP: pools: move the lock to the only __pool_get_first() that needs it
- CLEANUP: pools: rename __pool_get_first() to pool_get_from_shared_cache()
- CLEANUP: pools: rename pool_*_{from,to}_cache() to *_local_cache()
- CLEANUP: pools: rename __pool_free() to pool_put_to_shared_cache()
- MINOR: tools: add statistical_prng_range() to get a random number over a range
- MINOR: pools: use cheaper randoms for fault injections
- MINOR: pools: move the fault injector to __pool_alloc()
- MINOR: pools: split the OS-based allocator in two
- MINOR: pools: always use atomic ops to maintain counters
- MINOR: pools: move pool_free_area() out of the lock in the locked version
- MINOR: pools: factor the release code into pool_put_to_os()
- MEDIUM: pools: make CONFIG_HAP_POOLS control both local and shared pools
- MINOR: pools: create unified pool_{get_from,put_to}_cache()
- MINOR: pools: evict excess objects using pool_evict_from_local_cache()
- MEDIUM: pools: make pool_put_to_cache() always call pool_put_to_local_cache()
- CLEANUP: pools: make the local cache allocator fall back to the shared cache
- CLEANUP: pools: merge pool_{get_from,put_to}_local_caches with generic ones
- CLEANUP: pools: uninline pool_put_to_cache()
- CLEANUP: pools: declare dummy pool functions to remove some ifdefs
- BUILD: pools: fix build with DEBUG_FAIL_ALLOC
- BUG/MINOR: server: make srv_alloc_lb() allocate lb_nodes for consistent hash
- CONTRIB: mod_defender: import the minimal number of includes
- CONTRIB: mod_defender: make the code build with the embedded includes
- CONTRIB: modsecurity: import the minimal number of includes
- CONTRIB: modsecurity: make the code build with the embedded includes
- CLEANUP: sample: Improve local variables in sample_conv_json_query
- CLEANUP: sample: Explicitly handle all possible enum values from mjson
- CLEANUP: sample: Use explicit return for successful `json_query`s
- CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
- CONTRIB: move spoa_example out of the tree
- BUG/MINOR: server: free srv.lb_nodes in free_server
- BUG/MINOR: logs: free logsrv.conf.file on exit
- BUG/MEDIUM: server: ensure thread-safety of server runtime creation
- MINOR: server: add log on dynamic server creation
- MINOR: server: implement delete server cli command
- CONTRIB: move spoa_server out of the tree
- CONTRIB: move modsecurity out of the tree
- BUG/MINOR: server: fix potential null gcc error in delete server
- BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers
- BUG/MEDIUM: mux-h2: Fix dfl calculation when merging CONTINUATION frames
- BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort
- CLEANUP: uri_normalizer: Remove trailing whitespace
- MINOR: uri_normalizer: Add a `strip-dot` normalizer
- CONTRIB: move mod_defender out of the tree
- CLEANUP: contrib: remove the last references to the now dead contrib/ directory
- BUG/MEDIUM: config: fix cpu-map notation with both process and threads
- MINOR: config: add a diag for invalid cpu-map statement
- BUG/MINOR: mworker/init: don't reset nb_oldpids in non-mworker cases
- BUG/MINOR: mworker: don't use oldpids[] anymore for reload
- BUILD: makefile: fix the "make clean" target on strict bourne shells
- IMPORT: slz: import slz into the tree
- BUILD: compression: switch SLZ from out-of-tree to in-tree
- CI: github: do not build libslz any more
- CLEANUP: compression: remove calls to SLZ init functions
- BUG/MEDIUM: mux-h2: Properly handle shutdowns when received with data
- MINOR: cpuset: define a platform-independent cpuset type
- MINOR: cfgparse: use hap_cpuset for parse_cpu_set
- MEDIUM: config: use platform independent type hap_cpuset for cpu-map
- MINOR: thread: implement the detection of forced cpu affinity
- MINOR: cfgparse: support the comma separator on parse_cpu_set
- MEDIUM: cfgparse: detect numa and set affinity if needed
- MINOR: global: add option to disable numa detection
- BUG/MINOR: haproxy: fix compilation on macOS
- BUG/MINOR: cpuset: fix compilation on platform without cpu affinity
- MINOR: time: avoid unneeded updates to now_offset
- MINOR: time: avoid overwriting the same values of global_now
- CLEANUP: time: use __tv_to_ms() in tv_update_date() instead of open-coding
- MINOR: time: avoid u64 needlessly expensive computations for the 32-bit now_ms
- BUG/MINOR: peers: remove useless table check if initial resync is finished
- BUG/MEDIUM: peers: re-work connection to new process during reload.
- BUG/MEDIUM: peers: re-work refcnt on table to protect against flush
- BUG/MEDIUM: config: fix missing initialization in numa_detect_topology()
Render numa detection optional with a global configuration statement
'no numa-cpu-mapping'. This can be used if the applied affinity of the
algorithm is not optimal. Also complete the documentation with this new
keyword.
Use the platform independent type hap_cpuset for the cpu-map statement
parsing. This allow to address CPU index greater than LONGBITS.
Update the documentation to reflect the removal of this limit except for
platforms without cpu_set_t type or equivalent.
This normalizer removes "/./" segments from the path component.
Usually the dot refers to the current directory which renders those segments redundant.
See GitHub Issue #714.
Implement a new CLI command 'del server'. It can be used to removed a
dynamically added server. Only servers in maintenance mode can be
removed, and without pending/active/idle connection on it.
Add a new reg-test for this feature. The scenario of the reg-test need
to first add a dynamic server. It is then deleted and a client is used
to ensure that the server is non joinable.
The management doc is updated with the new command 'del server'.
As previously mentioned SPOA code has nothing to do in the haproxy core
since they're not dependent on haproxy's version. This one was moved to
its own repository here with complete history:
https://github.com/haproxy/spoa-example
The current "ADD" vs "ADDQ" is confusing because when thinking in terms
of appending at the end of a list, "ADD" naturally comes to mind, but
here it does the opposite, it inserts. Several times already it's been
incorrectly used where ADDQ was expected, the latest of which was a
fortunate accident explained in 6fa922562 ("CLEANUP: stream: explain
why we queue the stream at the head of the server list").
Let's use more explicit (but slightly longer) names now:
LIST_ADD -> LIST_INSERT
LIST_ADDQ -> LIST_APPEND
LIST_ADDED -> LIST_INLIST
LIST_DEL -> LIST_DELETE
The same is true for MT_LISTs, including their "TRY" variant.
LIST_DEL_INIT keeps its short name to encourage to use it instead of the
lazier LIST_DELETE which is often less safe.
The change is large (~674 non-comment entries) but is mechanical enough
to remain safe. No permutation was performed, so any out-of-tree code
can easily map older names to new ones.
The list doc was updated.
This patch adds an introduction to the http-request normalize-uri section,
explaining what to expect from the normalizers and possible issues that might
arise when not being careful.
This patch renames all existing uri-normalizers into a more consistent naming
scheme:
1. The part of the URI that is being touched.
2. The modification being performed as an explicit verb.
This normalizer merges `../` path segments with the predecing segment, removing
both the preceding segment and the `../`.
Empty segments do not receive special treatment. The `merge-slashes` normalizer
should be executed first.
See GitHub Issue #714.
This patch adds 4 new sample fetches to get the source and the destination
info (ip address and port) of the backend connection :
* bc_dst : Returns the destination address of the backend connection
* bc_dst_port : Returns the destination port of the backend connection
* bc_src : Returns the source address of the backend connection
* bc_src_port : Returns the source port of the backend connection
The configuration manual was updated accordingly.
The CLI's "set ssl cert" command only works on frontend certificates but
the documentation did not specify this limitations yet.
This patch can be backported to all stable branches.
With the json_query can a JSON value be extacted from a header
or body of the request and saved to a variable.
This converter makes it possible to handle some JSON workload
to route requests to different backends.
ub64dec and ub64enc are the base64url equivalent of b64dec and base64
converters. base64url encoding is the "URL and Filename Safe Alphabet"
variant of base64 encoding. It is also used in in JWT (JSON Web Token)
standard.
RFC1421 mention in base64.c file is deprecated so it was replaced with
RFC4648 to which existing converters, base64/b64dec, still apply.
Example:
HAProxy:
http-request return content-type text/plain lf-string %[req.hdr(Authorization),word(2,.),ub64dec]
Client:
Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiZm9vIiwia2V5IjoiY2hhZTZBaFhhaTZlIn0.5VsVj7mdxVvo1wP5c0dVHnr-S_khnIdFkThqvwukmdg
$ curl -H "Authorization: Bearer ${TOKEN}" http://haproxy.local
{"user":"foo","key":"chae6AhXai6e"}
This commit adds the new fields added to the ckch_inst structure in
order to manage the backend certificate hot update (GitHub #427) and the
bug of the default certificate update (GitHub #1143).
Released version 2.4-dev16 with the following main changes :
- CLEANUP: dev/flags: remove useless test in the stdin number parser
- MINOR: No longer rely on deprecated sample fetches for predefined ACLs
- MINOR: acl: Add HTTP_2.0 predefined macro
- BUG/MINOR: hlua: Detect end of request when reading data for an HTTP applet
- BUG/MINOR: tools: fix parsing "us" unit for timers
- MINOR: server/bind: add support of new prefixes for addresses.
- MINOR: log: register config file and line number on log servers.
- MEDIUM: log: support tcp or stream addresses on log lines.
- BUG/MEDIUM: log: fix config parse error logging on stdout/stderr or any raw fd
- CLEANUP: fd: remove FD_POLL_DATA and FD_POLL_STICKY
- MEDIUM: fd: prepare FD_POLL_* to move to bits 8-15
- MEDIUM: fd: merge fdtab[].ev and state for FD_EV_* and FD_POLL_* into state
- MINOR: fd: move .linger_risk into fdtab[].state
- MINOR: fd: move .cloned into fdtab[].state
- MINOR: fd: move .initialized into fdtab[].state
- MINOR: fd: move .et_possible into fdtab[].state
- MINOR: fd: move .exported into fdtab[].state
- MINOR: fd: implement an exclusive syscall bit to remove the ugly "log" lock
- MINOR: cli/show-fd: slightly reorganize the FD status flags
- MINOR: atomic/arm64: detect and use builtins for the double-word CAS
- CLEANUP: atomic: add an explicit _FETCH variant for add/sub/and/or
- CLEANUP: atomic: make all standard add/or/and/sub operations return void
- CLEANUP: atomic: add a fetch-and-xxx variant for common operations
- CLEANUP: atomic: add HA_ATOMIC_INC/DEC for unit increments
- CLEANUP: atomic/tree-wide: replace single increments/decrements with inc/dec
- CLEANUP: atomic: use the __atomic variant of BTS/BTR on modern compilers
- MINOR: atomic: implement native BTS/BTR for x86
- MINOR: ist: Add `istappend(struct ist, char)`
- MINOR: ist: Add `istshift(struct ist*)`
- MINOR: ist: Add `istsplit(struct ist*, char)`
- BUG/MAJOR: fd: switch temp values to uint in fd_stop_both()
- MINOR: opentracing: register config file and line number on log servers
- MEDIUM: resolvers: add support of tcp address on nameserver line.
- MINOR: ist: Rename istappend() to __istappend()
- CLEANUP: htx: Make http_get_stline take a `const struct`
- CLEANUP: ist: Remove unused `count` argument from `ist2str*`
- CLEANUP: Remove useless malloc() casts
This patch re-works configuration parsing, it removes the "server"
lines from "resolvers" sections introduced in commit 56fc5d9eb:
MEDIUM: resolvers: add supports of TCP nameservers in resolvers.
It also extends the nameserver lines to support stream server
addresses such as:
resolvers
nameserver localhost tcp@127.0.0.1:53
Doing so, a part of nameserver's init code was factorized in
function 'parse_resolvers' and removed from 'post_parse_resolvers'.
An explicit stream address prefix such as "tcp6@" "tcp4@"
"stream+ipv6@" "stream+ipv4@" or "stream+unix@" will
allocate an implicit ring buffer with a forward server
targeting the given address.
This is usefull to simply send logs to a log server in tcp
and It doesn't need to declare a ring section in configuration.
Since the internal function str2sa_range is used to addresses
for different objects ('server', 'bind' but also 'log' or
'nameserver') we notice that some combinations are missing.
"ip@" is introduced to authorize the prefix "dgram+ip@" or
"stream+ip@" which dectects automatically IP version but
specify dgram or stream.
"tcp@" was introduced and is an alias for "stream+ip@".
"tcp6" and "tcp4" are now aliases for "stream+ipv6@" and
"stream+ipv4@".
"uxst@" and "uxdg@" are now aliases for "stream+unix@" and
"dgram+unix@".
This patch also adds a complete section in documentation to
describe adresses and their prefixes.
HTTP_2.0 predefined macro returns true for HTTP/2 requests. HTTP/2 doen't
convey a version information, so this macro may seem a bit strange. But for
compatiblity reasons, internally, the "HTTP/2.0" version is set. Thus, it is
handy to rely on it to differenciate HTTP/1 and HTTP/2 requests.
Some predefined ACLs were still based on deprecated sample fetches, like
req_proto_http or req_ver. Now, they use non-deprecated sample fetches. In
addition, the usage lines in the configuration manual have been updated to
be more explicit.
Released version 2.4-dev15 with the following main changes :
- BUG/MINOR: payload: Wait for more data if buffer is empty in payload/payload_lv
- BUG/MINOR: stats: Apply proper styles in HTML status page.
- BUG/MEDIUM: time: make sure to always initialize the global tick
- BUG/MINOR: tcp: fix silent-drop workaround for IPv6
- BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS
- CLEANUP: socket: replace SOL_IP/IPV6/TCP with IPPROTO_IP/IPV6/TCP
- BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields
- BUG/MINOR: mux-h2: Don't emit log twice if an error occurred on the preface
- MINOR: stream: Don't trigger errors on destructive HTTP upgrades
- MINOR: frontend: Create HTTP txn for HTX streams
- MINOR: stream: Be sure to set HTTP analysers when creating an HTX stream
- BUG/MINOR: stream: Properly handle TCP>H1>H2 upgrades in http_wait_for_request
- BUG/MINOR: config: Add warning for http-after-response rules in TCP mode
- MINOR: muxes: Add a flag to notify a mux does not support any upgrade
- MINOR: mux-h1: Don't perform implicit HTTP/2 upgrade if not supported by mux
- MINOR: mux-pt: Don't perform implicit HTTP upgrade if not supported by mux
- MEDIUM: mux-h1: Expose h1 in the list of supported mux protocols
- MEDIUM: mux-pt: Expose passthrough in the list of supported mux protocols
- MINOR: muxes: Show muxes flags when the mux list is displayed
- DOC: config: Improve documentation about proto/check-proto keywords
- MINOR: stream: Use stream type instead of proxy mode when appropriate
- MINOR: filters/http-ana: Decide to filter HTTP headers in HTTP analysers
- MINOR: http-ana: Simplify creation/destruction of HTTP transactions
- MINOR: stream: Handle stream HTTP upgrade in a dedicated function
- MEDIUM: Add tcp-request switch-mode action to perform HTTP upgrade
- MINOR: config/proxy: Don't warn for HTTP rules in TCP if 'switch-mode http' set
- MINOR: config/proxy: Warn if a TCP proxy without backend is upgradable to HTTP
- DOC: config: Add documentation about TCP to HTTP upgrades
- REGTESTS: Add script to tests TCP to HTTP upgrades
- BUG/MINOR: payload/htx: Ingore L6 sample fetches for HTX streams/checks
- MINOR: htx: Make internal.strm.is_htx an internal sample fetch
- MINOR: action: Use a generic function to check validity of an action rule list
- MINOR: payload/config: Warn if a L6 sample fetch is used from an HTTP proxy
- MEDIUM: http-rules: Add wait-for-body action on request and response side
- REGTESTS: Add script to tests the wait-for-body HTTP action
- BUG/MINOR: http-fetch: Fix test on message state to capture the version
- CLEANUP: vars: always pre-initialize smp in vars_parse_cli_get_var()
- MINOR: global: define diagnostic mode of execution
- MINOR: cfgparse: diag for multiple nbthread statements
- MINOR: server: diag for 0 weight server
- MINOR: diag: create cfgdiag module
- MINOR: diag: diag if servers use the same cookie value
- MINOR: config: diag if global section after non-global
- TESTS: slightly reorganize the code in the tests/ directory
- TESTS: move tests/*.cfg to tests/config
- REGTESTS: ssl: "set ssl cert" and multi-certificates bundle
- REGTESTS: ssl: mark set_ssl_cert_bundle.vtc as broken
- CONTRIB: halog: fix issue with array of type char
- CONTRIB: tcploop: add a shutr command
- CONTRIB: debug: add the show-fd-to-flags script
- CONTRIB: debug: split poll from flags
- CONTRIB: move some dev-specific tools to dev/
- BUILD: makefile: always build the flags utility
- DEV: flags: replace the unneeded makefile with a README
- BUILD: makefile: integrate the hpack tools
- CONTRIB: merge ip6range with iprange
- CONTRIB: move some admin-related sub-projects to admin/
- CONTRIB: move halog to admin/
- ADMIN: halog: automatically enable USE_MEMCHR on the right glibc version
- BUILD: makefile: build halog with the correct flags
- BUILD: makefile: add a "USE_PROMEX" variable to ease building prometheus-exporter
- CONTRIB: move prometheus-exporter to addons/promex
- DOC: add a few words about USE_* and the addons directory
- CONTRIB: move 51Degrees to addons/51degrees
- CONTRIB: move src/da.c and contrib/deviceatlas to addons/deviceatlas
- CONTRIB: move src/wurfl.c and contrib/wurfl to addons/wurfl
- CONTRIB: move contrib/opentracing to addons/ot
- BUG/MINOR: opentracing: initialization after establishing daemon mode
- DOC: clarify that compression works for HTTP/2
This patch clarifies that compression also works with HTTP/2. I have
picked the wording "HTTP/1.1 or above" because it is already used
elsewhere in the documentation.
I have tested that compression indeed works in HTTP/2.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
This one is the last optional module to build with haproxy, so let's move
it to addons/. It was renamed to "ot" as it was the only one whose USE_*
option did not match the directory name, now this is consistent.
Few changes were required, only the Makefile, and doc were adjusted, as
the directory was already self-contained and relocatable.
Both the source file and the dummy library are now at the same place.
Maybe the build howto could be moved there as well to make things even
cleaner.
The Makefile, MAINTAINERS, doc, and vtest matrix were updated.
Both the source file and the dummy library are now at the same place.
Maybe the build howto could be moved there as well to make things even
cleaner.
The Makefile, MAINTAINERS, doc, github build matrix, coverity checks
and travis CI's build were updated.
Now it's much cleaner, both 51d.c and the dummy library live together and
are easier to spot and maintain. The build howto probably ought to be moved
there as well. Makefile, docs and MAINTAINERS were updated, as well as
the github CI's build matrix, travis CI's, and coverity checks.
Define MODE_DIAG which is used to run haproxy in diagnostic mode. This
mode is used to output extra warnings about possible configuration
blunder or sub-optimal usage. It can be activated with argument '-dD'.
A new output function ha_diag_warning is implemented reserved for
diagnostic output. It serves to standardize the format of diagnostic
messages.
A macro HA_DIAG_WARN_COND is also available to automatically check if
diagnostic mode is on before executing the diagnostic check.
Historically, an option was added to wait for the request payload (option
http-buffer-request). This option has 2 drawbacks. First, it is an ON/OFF
option for the whole proxy. It cannot be enabled on demand depending on the
message. Then, as its name suggests, it only works on the request side. The
only option to wait for the response payload was to write a dedicated
filter. While it is an acceptable solution for complex applications, it is a
bit overkill to simply match strings in the body.
To make everyone happy, this patch adds a dedicated HTTP action to wait for
the message payload, for the request or the response depending it is used in
an http-request or an http-response ruleset. The time to wait is
configurable and, optionally, the minimum payload size to have before stop
to wait.
Both the http action and the old http analyzer rely on the same internal
function.
Use a L6 sample fetch on an HTX streams or a HTX health-check is meaningless
because data are not raw but structured. So now, these sample fetches fail
when called from an HTTP proxy. In addition, a warning has been added in the
configuration manual, at the begining of the L6 sample fetches section.
Note that req.len and res.len samples return the HTX data size instead of
failing. It is not accurate because it does not reflect the buffer size nor
the raw data length. But we keep it for backward compatibility purpose.
However it remains a bit strange to use it on an HTTP proxy.
This patch may be backported to all versions supporting the HTX, i.e as far
as 2.0. But the part about the health-checks is only valid for the 2.2 and
upper.
This patch adds explanation about chaining a TCP frontend to an HTTP
backend. It also explain how the HTTP upgrades work in this context. A note
has also been added in "Fetching HTTP samples" section to warning about HTTP
content processing in TCP.
It is now possible to perform HTTP upgrades on a TCP stream from the
frontend side. To do so, a tcp-request content rule must be defined with the
switch-mode action, specifying the mode (for now, only http is supported)
and optionnaly the proto (h1 or h2).
This way it could be possible to set HTTP directives on a TCP frontend which
will only be evaluated if an upgrade is performed. This new way to perform
HTTP upgrades should replace progressively the old way, consisting to route
the request to an HTTP backend. And it should be also a good start to remove
all HTTP processing from tcp-request content rules.
This action is terminal, it stops the ruleset evaluation. It is only
available on proxy with the frontend capability.
The configuration manual has been updated accordingly.
This patch adds a description about information provided by "haproxy -vv"
command regarding the available protocols. The description is adapted
depending the context (bind line, server line or health-check).
Released version 2.4-dev14 with the following main changes :
- MEDIUM: quic: Fix build.
- MEDIUM: quic: Fix build.
- CI: codespell: whitelist "Dragan Dosen"
- CLEANUP: assorted typo fixes in the code and comments
- CI: github actions: update LibreSSL to 3.2.5
- REGTESTS: revert workaround for a crash with recent libressl on http-reuse sni
- CLEANUP: mark defproxy as const on parse tune.fail-alloc
- REGTESTS: remove unneeded experimental-mode in cli add server test
- REGTESTS: wait for proper return of enable server in cli add server test
- MINOR: compression: use pool_alloc(), not pool_alloc_dirty()
- MINOR: spoe: use pool_alloc(), not pool_alloc_dirty()
- MINOR: fcgi-app: use pool_alloc(), not pool_alloc_dirty()
- MINOR: cache: use pool_alloc(), not pool_alloc_dirty()
- MINOR: ssl: use pool_alloc(), not pool_alloc_dirty()
- MINOR: opentracing: use pool_alloc(), not pool_alloc_dirty()
- MINOR: dynbuf: make b_alloc() always check if the buffer is allocated
- CLEANUP: compression: do not test for buffer before calling b_alloc()
- CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
- MINOR: channel: simplify the channel's buffer allocation
- MEDIUM: dynbuf: remove last usages of b_alloc_margin()
- CLEANUP: dynbuf: remove b_alloc_margin()
- CLEANUP: dynbuf: remove the unused b_alloc_fast() function
- CLEANUP: pools: remove the unused pool_get_first() function
- MINOR: pools: make the pool allocator support a few flags
- MINOR: pools: add pool_zalloc() to return a zeroed area
- CLEANUP: connection: use pool_zalloc() in conn_alloc_hash_node()
- CLEANUP: filters: use pool_zalloc() in flt_stream_add_filter()
- CLEANUP: spoe: use pool_zalloc() instead of pool_alloc+memset
- CLEANUP: frontend: use pool_zalloc() in frontend_accept()
- CLEANUP: mailers: use pool_zalloc() in enqueue_one_email_alert()
- CLEANUP: resolvers: use pool_zalloc() in resolv_link_resolution()
- CLEANUP: ssl: use pool_zalloc() in ssl_init_keylog()
- CLEANUP: tcpcheck: use pool_zalloc() instead of pool_alloc+memset
- CLEANUP: quic: use pool_zalloc() instead of pool_alloc+memset
- MINOR: time: also provide a global, monotonic global_now_ms timer
- BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable
- MINOR: tools: introduce new option PA_O_DEFAULT_DGRAM on str2sa_range.
- BUILD: tools: fix build error with new PA_O_DEFAULT_DGRAM
- BUG/MINOR: ssl: Prevent disk access when using "add ssl crt-list"
- CLEANUP: ssl: remove unused definitions
- BUILD: ssl: guard ecdh functions with SSL_CTX_set_tmp_ecdh macro
- MINOR: lua: Slightly improve function dumping the lua traceback
- BUG/MEDIUM: debug/lua: Use internal hlua function to dump the lua traceback
- BUG/MEDIUM: lua: Always init the lua stack before referencing the context
- MINOR: fd: make fd_clr_running() return the remaining running mask
- MINOR: fd: remove the unneeded running bit from fd_insert()
- BUG/MEDIUM: fd: do not wait on FD removal in fd_delete()
- CLEANUP: fd: remove unused fd_set_running_excl()
- CLEANUP: fd: slightly simplify up _fd_delete_orphan()
- BUG/MEDIUM: fd: Take the fd_mig_lock when closing if no DWCAS is available.
- BUG/MEDIUM: release lock on idle conn killing on reached pool high count
- BUG/MEDIUM: thread: Fix a deadlock if an isolated thread is marked as harmless
- MINOR: tools: make url2ipv4 return the exact number of bytes parsed
- BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters
- BUG/MEDIUM: mux-h1: make h1_shutw_conn() idempotent
- BUG/MINOR: ssl: Fix update of default certificate
- BUG/MINOR: ssl: Prevent removal of crt-list line if the instance is a default one
- BUILD: ssl: introduce fine guard for ssl random extraction functions
- REORG: global: move initcall register code in a dedicated file
- REORG: global: move free acl/action in their related source files
- REORG: split proxy allocation functions
- MINOR: proxy: implement a free_proxy function
- MINOR: proxy: define cap PR_CAP_LUA
- MINOR: lua: properly allocate the lua Socket proxy
- MINOR: lua: properly allocate the lua Socket servers
- MINOR: vars: make get_vars() allow the session to be null
- MINOR: vars: make the var() sample fetch keyword depend on nothing
- CLEANUP: sample: remove duplicate "stopping" sample fetch keyword
- MINOR: sample: make smp_resolve_args() return an allocate error message
- MINOR: sample: add a new SMP_SRC_CONST sample capability
- MINOR: sample: mark the truly constant sample fetch keywords as such
- MINOR: sample: add a new CFG_PARSER context for samples
- MINOR: action: add a new ACT_F_CFG_PARSER origin designation
- MEDIUM: vars: add support for a "set-var" global directive
- REGTESTS: add a basic reg-test for some "set-var" commands
- MINOR: sample: add a new CLI_PARSER context for samples
- MINOR: action: add a new ACT_F_CLI_PARSER origin designation
- MINOR: vars/cli: add a "get var" CLI command to retrieve global variables
- MEDIUM: cli: add a new experimental "set var" command
- MINOR: compat: add short aliases for a few very commonly used types
- BUILD: ssl: use EVP_CIPH_GCM_MODE macro instead of HA_OPENSSL_VERSION
- MEDIUM: backend: use a trylock to grab a connection on high FD counts as well
set var <name> <expression>
Allows to set or overwrite the process-wide variable 'name' with the result
of expression <expression>. Only process-wide variables may be used, so the
name must begin with 'proc.' otherwise no variable will be set. The
<expression> may only involve "internal" sample fetch keywords and converters
even though the most likely useful ones will be str('something') or int().
Note that the command line parser doesn't know about quotes, so any space in
the expression must be preceeded by a backslash. This command requires levels
"operator" or "admin". This command is only supported on a CLI connection
running in experimental mode (see "experimental-mode on").
Just like for "set-var" in the global section, the command uses a temporary
dummy proxy to create a temporary "set-var(name)" rule to assign the value.
The reg test was updated to verify that an updated global variable is properly
reflected in subsequent HTTP responses.
Process-wide variables can now be displayed from the CLI using "get var"
followed by the variable name. They must all start with "proc." otherwise
they will not be found. The output is very similar to the one of the
debug converter, with a type and value being reported for the embedded
sample.
This command is limited to clients with the level "operator" or higher,
since it can possibly expose traffic-related data.
While we do support process-wide variables ("proc.<name>"), there was
no way to preset them from the configuration. This was particularly
limiting their usefulness since configs involving them always had to
first check if the variable was set prior to performing an operation.
This patch adds a new "set-var" directive in the global section that
supports setting the proc.<name> variables from an expression, like
other set-var actions do. The syntax however follows what is already
being done for setenv, which consists in having one argument for the
variable name and another one for the expression.
Only "constant" expressions are allowed here, such as "int", "str"
etc, combined with arithmetic or string converters, and variable
lookups. A few extra sample fetch keywords like "date", "rand" and
"uuid" are also part of the constant expressions and may make sense
to allow to create a random key or differentiate processes.
The way it was done consists in parsing a dummy rule an executing the
expression in the CFG_PARSE context, then releasing the expression.
This is safe because the sample that variables store does not hold a
back pointer to expression that created them.
The hdr_ip() sample fetch function will try to extract IP addresses
from a header field. These IP addresses are parsed using url2ipv4()
and if it fails it will fall back to inet_pton(AF_INET6), otherwise
will fail.
There is a small problem there which is that if a field starts with
an IP address and is immediately followed by some garbage, the IP
address part is still returned. This is a problem with fields such
as x-forwarded-for because it prevents detection of accidental
corruption or bug along the chain. For example, the following string:
x-forwarded-for: 1.2.3.4; 5.6.7.8
or this one:
x-forwarded-for: 1.2.3.4O ( the last one being the letter 'O')
would still return "1.2.3.4" despite the trailing characters. This is
bad because it will silently cover broken code running on intermediary
proxies and may even in some cases allow haproxy to pass improperly
formatted headers after they were apparently validated, for example,
if someone extracts the address from this field to place it into
another one.
This issue would only affect the IPv4 parser, because the IPv6 parser
already uses inet_pton() which fails at the first invalid character and
rejects trailing port numbers.
In strict compliance with RFC7239, let's make sure that if there are any
characters left in the string, the parsing fails and makes hdr_ip()
return nothing. However, a special case has to be handled to support
IPv4 addresses followed by a colon and a valid port number, because till
now the parser used to implicitly accept them and it appears that this
practice, though rare, does exist at least in Azure:
https://docs.microsoft.com/en-us/azure/application-gateway/how-application-gateway-works
This issue has always been there so the fix may be backported to all
versions. It will need the following commit in order to work as expected:
MINOR: tools: make url2ipv4 return the exact number of bytes parsed
Many thanks to https://twitter.com/melardev and the BitMEX Security Team
for their detailed report.
It is never used anymore since 1.7 where it was used by b_alloc_margin()
then replaced by direct calls to the pools function, and it maintains a
dependency on the exposed pools functions. It's time to get rid of it,
as it's not even certain it still works.
Right now there is a discrepancy beteween b_alloc() and b_allow_margin():
the former forcefully overwrites the target pointer while the latter tests
it and returns it as-is if already allocated.
As a matter of fact, all callers of b_alloc() either preliminary test the
buffer, or assume it's already null.
Let's remove this pain and make the function test the buffer's allocation
before doing it again, and match call places' expectations.
Released version 2.4-dev13 with the following main changes :
- BUG/MEDIUM: cli: fix "help" crashing since recent spelling fixes
- BUG/MINOR: cfgparse: use the GLOBAL not LISTEN keywords list for spell checking
- MINOR: tools: improve word fingerprinting by counting presence
- MINOR: tools: do not sum squares of differences for word fingerprints
- MINOR: cli: improve fuzzy matching to work on all remaining words at once
- MINOR: cli: sort the suggestions by order of relevance
- MINOR: cli: limit spelling suggestions to 5
- MINOR: cfgparse/proxy: also support spelling fixes on options
- BUG/MINOR: resolvers: Add missing case-insensitive comparisons of DNS hostnames
- MINOR: time: export the global_now variable
- BUG/MINOR: freq_ctr/threads: make use of the last updated global time
- MINOR: freq_ctr/threads: relax when failing to update a sliding window value
- MINOR/BUG: mworker/cli: do not use the unix_bind prefix for the master CLI socket
- MINOR: mworker/cli: alert the user if we enabled a master CLI but not the master-worker mode
- MINOR: cli: implement experimental-mode
- REORG: server: add a free server function
- MINOR: cfgparse: always alloc idle conns task
- REORG: server: move keywords in srv_kws
- MINOR: server: remove fastinter from mistyped kw list
- REORG: server: split parse_server
- REORG: server: move alert traces in parse_server
- REORG: server: rename internal functions from parse_server
- REORG: server: attach servers in parse_server
- REORG: server: use flags for parse_server
- MINOR: server: prepare parsing for dynamic servers
- MINOR: stats: export function to allocate extra proxy counters
- MEDIUM: server: implement 'add server' cli command
- REGTESTS: implement test for 'add server' cli
- MINOR: server: enable standard options for dynamic servers
- MINOR: server: support keyword proto in 'add server' cli
- BUG/MINOR: protocol: add missing support of dgram unix socket.
- CLEANUP: Fix a typo in fix_is_valid description
- MINOR: raw_sock: Add a close method.
- MEDIUM: connections: Introduce a new XPRT method, start().
- MEDIUM: connections: Implement a start() method for xprt_handshake.
- MEDIUM: connections: Implement a start() method in ssl_sock.
- MINOR: muxes: garbage collect the reset() method.
- CLEANUP: tcp-rules: Fix a typo in error messages about expect-netscaler-cip
- MEDIUM: lua: Use a per-thread counter to track some non-reentrant parts of lua
- BUG/MEDIUM: debug/lua: Don't dump the lua stack if not dumpable
Allow to specify the mux proto for a dynamic server. It must be
compatible with the backend mode to be accepted. The reg-tests has been
extended for this error case.
Enable a subset of server options to be used as keywords on the CLI
command 'add server'. These options are safe and can be applied
flawlessly for a dynamic server.
Add a new cli command 'add server'. This command is used to create a new
server at runtime attached on an existing backend. The syntax is the
following one :
$ add server <be_name>/<sv_name> [<kws>...]
This command is only available through experimental mode for the moment.
Currently, no server keywords are supported. They will be activated
individually when deemed properly functional and safe.
Another limitation is put on the backend load-balancing algorithm. The
algorithm must use consistent hashing to guarantee a minimal
reallocation of existing connections on the new server insertion.
Released version 2.4-dev12 with the following main changes :
- CLEANUP: connection: Use `VAR_ARRAY` in `struct tlv` definition
- CLEANUP: connection: Remove useless test for NULL before calling `pool_free()`
- CLEANUP: connection: Use istptr / istlen for proxy_unique_id
- MINOR: connection: Use a `struct ist` to store proxy_authority
- CLEANUP: connection: Consistently use `struct ist` to process all TLV types
- BUILD: task: fix build at -O0 with threads disabled
- BUILD: bug: refine HA_LINK_ERROR() to only be used on gcc and derivatives
- CLEANUP: config: make the cfg_keyword parsers take a const for the defproxy
- BUILD: connection: do not use VAR_ARRAY in struct tlv
- BUG/MEDIUM: session: NULL dereference possible when accessing the listener
- MINOR: build: force CC to set a return code when probing options
- CLEANUP: stream: rename a few remaining occurrences of "stream *sess"
- BUG/MEDIUM: resolvers: handle huge responses over tcp servers.
- CLEANUP: config: also address the cfg_keyword API change in the compression code
- BUG/MEDIUM: ssl: properly remove the TASK_HEAVY flag at end of handshake
- BUG/MINOR: sample: Rename SenderComID/TargetComID to SenderCompID/TargetCompID
- MINOR: task: give the scheduler a bit more flexibility in the runqueue size
- OPTIM: task: automatically adjust the default runqueue-depth to the threads
- BUG/MINOR: connection: Missing QUIC initialization
- BUG/MEDIUM: stick-tables: fix ref counter in table entry using multiple http tracksc.
- BUILD: atomic/arm64: force the register pairs to use in __ha_cas_dw()
- BUG/MEDIUM: filters: Set CF_FL_ANALYZE on channels when filters are attached
- BUG/MINOR: tcpcheck: Update .health threshold of agent inside an agent-check
- BUG/MINOR: proxy/session: Be sure to have a listener to increment its counters
- BUG/MINOR: tcpcheck: Fix double free on error path when parsing tcp/http-check
- BUG/MINOR: server-state: properly handle the case where the base is not set
- BUG/MINOR: server-state: use the argument, not the global state
- CLEANUP: tcp-rules: add missing actions in the tcp-request error message
- CLEANUP: vars: make the error message clearer on missing arguments for set-var
- CLEANUP: http-rules: remove the unexpected comma before the list of action keywords
- CLEANUP: actions: the keyword must always be const from the rule
- MINOR: tools: add simple word fingerprinting to find similar-looking words
- MINOR: cfgparse: add cfg_find_best_match() to suggest an existing word
- MINOR: cfgparse: suggest correct spelling for unknown words in proxy sections
- MINOR: cfgparse: suggest correct spelling for unknown words in global section
- MINOR: cfgparse/server: try to fix spelling mistakes on server lines
- MINOR: cfgparse/bind: suggest correct spelling for unknown bind keywords
- MINOR: actions: add a function to suggest an action ressembling a given word
- MINOR: http-rules: suggest approaching action names on mismatch
- MINOR: tcp-rules: suggest approaching action names on mismatch
- BUG/MINOR: cfgparse/server: increment the extra keyword counter one at a time
- Revert "BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record"
- BUG/MINOR: resolvers: Consider server to have no IP on DNS resolution error
- BUG/MINOR: resolvers: Reset server address on DNS error only on status change
- BUG/MINOR: resolvers: Unlink DNS resolution to set RMAINT on SRV resolution
- BUG/MEDIUM: resolvers: Don't set an address-less server as UP
- BUG/MEDIUM: resolvers: Fix the loop looking for an existing ADD item
- MINOR: resolvers: new function find_srvrq_answer_record()
- BUG/MINOR; resolvers: Ignore DNS resolution for expired SRV item
- BUG/MEDIUM: resolvers: Trigger a DNS resolution if an ADD item is obsolete
- MINOR: resolvers: Use a function to remove answers attached to a resolution
- MINOR: resolvers: Purge answer items when a SRV resolution triggers an error
- MINOR: resolvers: Add function to change the srv status based on SRV resolution
- MINOR: resolvers: Directly call srvrq_update_srv_state() when possible
- BUG/MEDIUM: resolvers: Don't release resolution from a requester callbacks
- BUG/MEDIUM: resolvers: Skip DNS resolution at startup if SRV resolution is set
- MINOR: resolvers: Use milliseconds for cached items in resolver responses
- MINOR: resolvers: Don't try to match immediatly renewed ADD items
- CLEANUP: resolvers: Use ha_free() in srvrq_resolution_error_cb()
- CLEANUP: resolvers: Perform unsafe loop on requester list when possible
- BUG/MINOR: cli: make sure "help", "prompt", "quit" are enabled at master level
- CLEANUP: cli: fix misleading comment and better indent the access level flags
- MINOR: cli: set the ACCESS_MASTER* bits on the master bind_conf
- MINOR: cli: test the appctx level for master access instead of comparing pointers
- MINOR: cli: print the error message in the parser function itself
- MINOR: cli: filter the list of commands to the matching part
- MEDIUM: cli: apply spelling fixes for known commands before listing them
- MINOR: tools: add the ability to update a word fingerprint
- MINOR: cli: apply the fuzzy matching on the whole command instead of words
- CLEANUP: cli: rename MAX_STATS_ARGS to MAX_CLI_ARGS
- CLEANUP: cli: rename the last few "stats_" to "cli_"
- CLEANUP: task: make sure tasklet handlers always indicate their statuses
- CLEANUP: assorted typo fixes in the code and comments
The recent default runqueue size reduction appeared to have significantly
lowered performance on low-thread count configs. Testing various values
runqueue values on different workloads under thread counts ranging from
1 to 64, it appeared that lower values are more optimal for high thread
counts and conversely. It could even be drawn that the optimal value for
various workloads sits around 280/sqrt(nbthread), and probably has to do
with both the L3 cache usage and how to optimally interlace the threads'
activity to minimize contention. This is much easier to optimally
configure, so let's do this by default now.
The recently introduced Financial Information eXchange (FIX)
converters have some hard coded tags based on the specification that
were misspelled. Specifically, SenderComID and TargetComID should
be SenderCompID and TargetCompID according to the specification [1][2].
This patch updates all references, which includes the converters
themselves, the regression test, and the documentation.
[1] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag49.html
[2] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag56.html
Parameter "accepted_payload_size" is currently considered regardless
the used nameserver is using TCP or UDP. It remains mandatory to annouce
such capability to support e-dns, so a value have to be announced also
in TCP. Maximum DNS message size in TCP is limited by protocol to 65535
and so for UDP (65507) if system supports such UDP messages. But
the maximum value for this option was arbitrary forced to 8192.
This patch change this maximum to 65535 to allow user to set bigger value
for UDP if its system supports. It also sets accepted_payload_size
in TCP allowing to retrieve huge responses if the configuration uses
TCP nameservers.
The request announcing the accepted_payload_size capability is currently
built at resolvers level and is common to all used nameservers of the
section regardess transport protocol used. A further patch should be
made to at least specify a different payload size depending of the
transport, and perhaps could be forced to 65535 in case of TCP and
maximum would be forced back to 65507 matching UDP max.
This patch is appliable since 2.4 version
Released version 2.4-dev11 with the following main changes :
- CI: codespell: skip Makefile for spell check
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: tcp-act: Don't forget to set the original port for IPv4 set-dst rule
- BUG/MINOR: connection: Use the client's dst family for adressless servers
- BUG/MEDIUM: spoe: Kill applets if there are pending connections and nbthread > 1
- CLEANUP: Use ist2(const void*, size_t) whenever possible
- CLEANUP: Use IST_NULL whenever possible
- BUILD: proxy: Missing header inclusion for quic_transport_params_init()
- BUILD: quic: Implicit conversion between SSL related enums.
- DOC: spoe: Add a note about fragmentation support in HAProxy
- MINOR: contrib: add support for heartbeat control messages.
- MINOR: contrib: Enhance peers dissector heuristic.
- BUG/MINOR: mux-h2: Fix typo in scheme adjustment
- CLEANUP: Reapply the ist2() replacement patch
- CLEANUP: Use istadv(const struct ist, const size_t) whenever possible
- CLEANUP: Use isttest(const struct ist) whenever possible
- Revert "CI: Pin VTest to a known good commit"
- CLEANUP: backend: fix a wrong comment
- BUG/MINOR: backend: free allocated bind_addr if reuse conn
- MINOR: backend: handle reuse for conns with no server as target
- REGTESTS: test http-reuse if no server target
- BUG/MINOR: hlua: Don't strip last non-LWS char in hlua_pushstrippedstring()
- BUG/MINOR: server-state: Don't load server-state file for disabled backends
- CLEANUP: dns: Use DISGUISE() on a never-failing ring_attach() call
- CLEANUP: dns: Remove useless test on ns->dgram in dns_connect_nameserver()
- DOC: fix originalto except clause on destination address
- CLEANUP: Use the ist() macro whenever possible
- CLEANUP: Replace for loop with only a condition by while
- REORG: atomic: reimplement pl_cpu_relax() from atomic-ops.h
- BUG/MINOR: mt-list: always perform a cpu_relax call on failure
- MINOR: atomic: add armv8.1-a atomics variant for cas-dw
- MINOR: atomic: implement a more efficient arm64 __ha_cas_dw() using pairs
- BUG/MINOR: ssl: don't truncate the file descriptor to 16 bits in debug mode
- MEDIUM: pools: add CONFIG_HAP_NO_GLOBAL_POOLS and CONFIG_HAP_GLOBAL_POOLS
- MINOR: pools: double the local pool cache size to 1 MB
- MINOR: stream: use ABORT_NOW() and not abort() in stream_dump_and_crash()
- CLEANUP: stream: explain why we queue the stream at the head of the server list
- MEDIUM: backend: use a trylock when trying to grab an idle connection
- REORG: tools: promote the debug PRNG to more general use as a statistical one
- OPTIM: lb-random: use a cheaper PRNG to pick a server
- MINOR: task: stop abusing the nice field to detect a tasklet
- MINOR: task: move the nice field to the struct task only
- MEDIUM: task: extend the state field to 32 bits
- MINOR: task: add an application specific flag to the state: TASK_F_USR1
- MEDIUM: muxes: mark idle conns tasklets with TASK_F_USR1
- MINOR: xprt: add new xprt_set_idle and xprt_set_used methods
- MEDIUM: ssl: implement xprt_set_used and xprt_set_idle to relax context checks
- MINOR: server: don't read curr_used_conns multiple times
- CLEANUP: global: reorder some fields to respect cache lines
- CLEANUP: sockpair: silence a coverity check about fcntl()
- CLEANUP: lua: set a dummy file name and line number on the dummy servers
- MINOR: server: add a global list of all known servers
- MINOR: cfgparse: finish to set up servers outside of the proxy setup loop
- MINOR: server: allocate a per-thread struct for the per-thread connections stuff
- MINOR: server: move actconns to the per-thread structure
- CLEANUP: server: reorder some fields in the server struct to respect cache lines
- MINOR: backend: add a BUG_ON if conn mux NULL in connect_server
- BUG/MINOR: backend: fix condition for reuse on mode HTTP
- BUILD: Fix build when using clang without optimizing.
- CLEANUP: assorted typo fixes in the code and comments
Fix the description of the except clause of the originalto option. The
destination address and not the source is compared with the except range
address to prevent the addition of the X-Original-To header.
This can be backported in every releases.
Add a note in SPOE.txt to make it clear that HAPRoxy does not support the
fragmentation. It can send fragmented frames if an agent supports it but it
cannot receives and handles fragmented frames.
This patch should fix the issue #659. It may be backported as far as 1.8.
Released version 2.4-dev10 with the following main changes :
- BUILD: SSL: introduce fine guard for RAND_keep_random_devices_open
- MINOR: Configure the `cpp` userdiff driver for *.[ch] in .gitattributes
- BUG/MINOR: ssl/cli: potential null pointer dereference in "set ssl cert"
- BUG/MINOR: sample: secure convs that accept base64 string and var name as args
- BUG/MEDIUM: vars: make functions vars_get_by_{name,desc} thread-safe
- CLEANUP: vars: make smp_fetch_var() to reuse vars_get_by_desc()
- DOC: muxes: add a diagram of the exchanges between muxes and outer world
- BUG/MEDIUM: proxy: use thread-safe stream killing on hard-stop
- BUG/MEDIUM: cli/shutdown sessions: make it thread-safe
- BUG/MINOR: proxy: wake up all threads when sending the hard-stop signal
- MINOR: stream: add an "epoch" to figure which streams appeared when
- MINOR: cli/streams: make "show sess" dump all streams till the new epoch
- MINOR: streams: use one list per stream instead of a global one
- MEDIUM: streams: do not use the streams lock anymore
- BUILD: dns: avoid a build warning when threads are disabled (dss unused)
- MEDIUM: task: remove the tasks_run_queue counter and have one per thread
- MINOR: tasks: do not maintain the rqueue_size counter anymore
- CLEANUP: tasks: use a less confusing name for task_list_size
- CLEANUP: task: move the tree root detection from __task_wakeup() to task_wakeup()
- MINOR: task: limit the remote thread wakeup to the global runqueue only
- MINOR: task: move the allocated tasks counter to the per-thread struct
- CLEANUP: task: split the large tasklet_wakeup_on() function in two
- BUG/MINOR: fd: properly wait for !running_mask in fd_set_running_excl()
- BUG/MINOR: resolvers: Fix condition to release received ARs if not assigned
- BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record
- BUG/MINOR: resolvers: new callback to properly handle SRV record errors
- BUG/MEDIUM: resolvers: Reset server address and port for obselete SRV records
- BUG/MEDIUM: resolvers: Reset address for unresolved servers
- DOC: Update the module list in MAINTAINERS file
- MINOR: htx: Add function to reserve the max possible size for an HTX DATA block
- DOC: Update the HTX API documentation
- DOC: Update the filters guide
- BUG/MEDIUM: contrib/prometheus-exporter: fix segfault in listener name dump
- MINOR: task: split the counts of local and global tasks picked
- MINOR: task: do not use __task_unlink_rq() from process_runnable_tasks()
- MINOR: task: don't decrement then increment the local run queue
- CLEANUP: task: re-merge __task_unlink_rq() with task_unlink_rq()
- MINOR: task: make grq_total atomic to move it outside of the grq_lock
- MINOR: tasks: also compute the tasklet latency when DEBUG_TASK is set
- MINOR: task: make tasklet wakeup latency measurements more accurate
- MINOR: server: Be more strict on the server-state line parsing
- MINOR: server: Only fill one array when parsing a server-state line
- MEDIUM: server: Refactor apply_server_state() to make it more readable
- CLEANUP: server: Rename state_line node to node instead of name_name
- CLEANUP: server: Rename state_line structure into server_state_line
- CLEANUP: server: Use a local eb-tree to store lines of the global server-state file
- MINOR: server: Be more strict when reading the version of a server-state file
- MEDIUM: server: Store parsed params of a server-state line in the tree
- MINOR: server: Remove cached line from global server-state tree when found
- MINOR: server: Move loading state of servers in a dedicated function
- MEDIUM: server: Use a tree to store local server-state lines
- MINOR: server: Parse and store server-state lines in a dedicated function
- MEDIUM: server: Don't load server-state file if a line is corrupted
- REORG: server: Export and rename some functions updating server info
- REORG: server-state: Move functions to deal with server-state in its own file
- MINOR: server-state: Don't load server-state file for serverless proxies
- CLEANUP: muxes: Remove useless if condition in show_fd function
- BUG/MINOR: stats: fix compare of no-maint url suffix
- MINOR: task: limit the number of subsequent heavy tasks with flag TASK_HEAVY
- MINOR: ssl: mark the SSL handshake tasklet as heavy
- CLEANUP: server: rename srv_cleanup_{idle,toremove}_connections()
- BUG/MINOR: ssl: potential null pointer dereference in ckchs_dup()
- MINOR: task: add one extra tasklet class: TL_HEAVY
- MINOR: task: place the heavy elements in TL_HEAVY
- MINOR: task: only limit TL_HEAVY tasks but not others
- BUG/MINOR: http-ana: Only consider dst address to process originalto option
- MINOR: tools: Add net_addr structure describing a network addess
- MINOR: tools: Add function to compare an address to a network address
- MEDIUM: http-ana: Add IPv6 support for forwardfor and orignialto options
- CLEANUP: hlua: Use net_addr structure internally to parse and compare addresses
- REGTESTS: Add script to test except param for fowardedfor/originalto options
- DOC: scheduler: add a diagram showing the different queues and their usages
- CLEANUP: tree-wide: replace free(x);x=NULL with ha_free(&x)
- CLEANUP: config: replace a few free() with ha_free()
- CLEANUP: vars: always zero the pointers after a free()
- CLEANUP: ssl: remove a useless "if" before freeing an error message
- CLEANUP: ssl: make ssl_sock_free_srv_ctx() zero the pointers after free
- CLEANUP: ssl: use realloc() instead of free()+malloc()
The scheduler has become complex over time and the latest updates were a
good opportunity to document it. This diagram shows the time-based wait
queue(s), the priority-based run queue(s), and the class-based tasklet
queues, trying to emphasize what is local-only and what is shared between
threads. The diagram is provided in .fig, .svg, .png, and .pdf.
A network may be specified to avoid header addition for "forwardfor" and
"orignialto" option via the "except" parameter. However, only IPv4
networks/addresses are supported. This patch adds the support of IPv6.
To do so, the net_addr structure is used to store the parameter value in the
proxy structure. And ipcmp2net() function is used to perform the comparison.
This patch should fix the issue #1145. It depends on the following commit:
* c6ce0ab MINOR: tools: Add function to compare an address to a network address
* 5587287 MINOR: tools: Add net_addr structure describing a network addess
The filters guide was totally outdated. Callbacks to filter payload were
changed, especially the HTTP one because of the HTX. All the HTTP legacy
part is removed. This new guide now reflects the reality.
This patch may be backported as far as 2.2.
Missing functions have been added. And because the EOM block was removed,
some parts have been adapted to better explain how the end of the message
may be detected.
Since the muxes API is far from being obvious, let's show a stream being
forwarded between two sides through muxes with their buffers and the
transport layers. The diagram is provided in .fig, .svg, .png, and .pdf.
Released version 2.4-dev9 with the following main changes :
- BUG/MINOR: server: Remove RMAINT from admin state when loading server state
- CLEANUP: check: fix get_check_status_info declaration
- CLEANUP: contrib/prometheus-exporter: align for with srv status case
- MEDIUM: stats: allow to select one field in `stats_fill_li_stats`
- MINOR: stats: add helper to get status string
- MEDIUM: contrib/prometheus-exporter: add listen stats
- BUG/MINOR: dns: add test on result getting value from buffer into ring.
- BUG/MINOR: dns: dns_connect_server must return -1 unsupported nameserver's type
- BUG/MINOR: dns: missing test writing in output channel in session handler
- BUG/MINOR: dns: fix ring attach control on dns_session_new
- BUG/MEDIUM: dns: fix multiple double close on fd in dns.c
- BUG/MAJOR: connection: prevent double free if conn selected for removal
- BUG/MINOR: session: atomically increment the tracked sessions counter
- REGTESTS: fix http_reuse_conn_hash proxy test
- BUG/MINOR: backend: do not call smp_make_safe for sni conn hash
- MINOR: connection: remove pointers for prehash in conn_hash_params
- BUG/MINOR: checks: properly handle wrapping time in __health_adjust()
- BUG/MEDIUM: checks: don't needlessly take the server lock in health_adjust()
- DEBUG: thread: add 5 extra lock labels for statistics and debugging
- OPTIM: server: switch the actconn list to an mt-list
- Revert "MINOR: threads: change lock_t to an unsigned int"
- MINOR: lb/api: let callers of take_conn/drop_conn tell if they have the lock
- OPTIM: lb-first: do not take the server lock on take_conn/drop_conn
- OPTIM: lb-leastconn: do not take the server lock on take_conn/drop_conn
- OPTIM: lb-leastconn: do not unlink the server if it did not change
- MINOR: tasks: add DEBUG_TASK to report caller info in a task
- MINOR: tasks/debug: add some extra controls of use-after-free in DEBUG_TASK
- BUG/MINOR: sample: Always consider zero size string samples as unsafe
- MINOR: cli: add missing agent commands for set server
- BUILD/MEDIUM: da Adding pcre2 support.
- BUILD: ssl: introduce fine guard for OpenSSL specific SCTL functions
- REGTESTS: reorder reuse conn proxy protocol test
- DOC: explain the relation between pool-low-conn and tune.idle-pool.shared
- MINOR: tasks: refine the default run queue depth
- MINOR: listener: refine the default MAX_ACCEPT from 64 to 4
- MINOR: mux_h2: do not try to remove front conn from idle trees
- REGTESTS: workaround for a crash with recent libressl on http-reuse sni
- BUG/MEDIUM: lists: Avoid an infinite loop in MT_LIST_TRY_ADDQ().
- MINOR: connection: allocate dynamically hash node for backend conns
- DOC: DeviceAtlas documentation typo fix.
- BUG/MEDIUM: spoe: Resolve the sink if a SPOE logs in a ring buffer
- BUG/MINOR: http-rules: Always replace the response status on a return action
- BUG/MINOR: server: Init params before parsing a new server-state line
- BUG/MINOR: server: Be sure to cut the last parsed field of a server-state line
- MEDIUM: server: Don't introduce a new server-state file version
- DOC: contrib/prometheus-exporter: remove htx reference
- REGTESTS: contrib/prometheus-exporter: test NaN values
- REGTESTS: contrib/prometheus-exporter: test well known labels
- CI: github actions: switch to stable LibreSSL release
- BUG/MINOR: server: Fix test on number of fields allowed in a server-state line
- MINOR: dynbuf: make the buffer wait queue per thread
- MINOR: dynbuf: use regular lists instead of mt_lists for buffer_wait
- MINOR: dynbuf: pass offer_buffers() the number of buffers instead of a threshold
- MINOR: sched: have one runqueue ticks counter per thread
The maximum number of connections accepted at once by a thread for a single
listener used to default to 64 divided by the number of processes but the
tasklet-based model is much more scalable and benefits from smaller values.
Experimentation has shown that 4 gives the highest accept rate for all
thread values, and that 3 and 5 come very close, as shown below (HTTP/1
connections forwarded per second at multi-accept 4 and 64):
ac\thr| 1 2 4 8 16
------+------------------------------
4| 80k 106k 168k 270k 336k
64| 63k 89k 145k 230k 274k
Some tests were also conducted on SSL and absolutely no change was observed.
The value was placed into a define because it used to be spread all over the
code.
It might be useful at some point to backport this to 2.3 and 2.2 to help
those who observed some performance regressions from 1.6.
Since a lot of internal callbacks were turned to tasklets, the runqueue
depth had not been readjusted from the default 200 which was initially
used to favor batched processing. But nowadays it appears too large
already based on the following tests conducted on a 8c16t machine with
a simple config involving "balance leastconn" and one server. The setup
always involved the two threads of a same CPU core except for 1 thread,
and the client was running over 1000 concurrent H1 connections. The
number of requests per second is reported for each (runqueue-depth,
nbthread) couple:
rq\thr| 1 2 4 8 16
------+------------------------------
32| 120k 159k 276k 477k 698k
40| 122k 160k 276k 478k 722k
48| 121k 159k 274k 482k 720k
64| 121k 160k 274k 469k 710k
200| 114k 150k 247k 415k 613k <-- default
It's possible to save up to about 18% performance by lowering the
default value to 40. One possible explanation to this is that checking
I/Os more frequently allows to flush buffers faster and to smooth the
I/O wait time over multiple operations instead of alternating phases
of processing, waiting for locks and waiting for new I/Os.
The total round trip time also fell from 1.62ms to 1.40ms on average,
among which at least 0.5ms is attributed to the testing tools since
this is the minimum attainable on the loopback.
After some observation it would be nice to backport this to 2.3 and
2.2 which observe similar improvements, since some users have already
observed some perf regressions between 1.6 and 2.2.
Disabling idle-pool sharing can result in awful performance in presence
of a not so high number of threads, because the number of available idle
connections will be shared among threads, resulting in most of them
abandonning their connections after a request is done if there are already
enough total available. This is a case where pool-low-conn ought to be
used to preserve a number of connections for each thread, but this relation
isn't obvious as is. Let's add mentions about this with both keywords.
The DeviceAtlas Detection API now supports also the pcre2 library,
and some users wish to have exclusively this version in their
environment.
Also, there is no longer new development happening in the legacy
pcre(1) counterpart.
Simple check in the build process as the mutual exclusivity check between the
two are already taking care of early on. Moving the check to the part
only when we build haproxy + the API from source as the other case the API is
already built with the chosen regex library separately.
Released version 2.4-dev8 with the following main changes :
- BUILD: ssl: fix typo in HAVE_SSL_CTX_ADD_SERVER_CUSTOM_EXT macro
- BUILD: ssl: guard SSL_CTX_add_server_custom_ext with special macro
- BUG/MINOR: mux-h1: Don't emit extra CRLF for empty chunked messages
- MINOR: contrib/prometheus-exporter: use stats desc when possible followup
- MEDIUM: contrib/prometheus-exporter: export base stick table stats
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: check: fix some typo in comments
- CLEANUP: tools: typo in `strl2irc` mention
- BUILD: ssl: guard SSL_CTX_set_msg_callback with SSL_CTRL_SET_MSG_CALLBACK macro
- MEDIUM: ssl: add a rwlock for SSL server session cache
- BUG/MINOR: intops: fix mul32hi()'s off-by-one
- BUG/MINOR: freq_ctr: fix a wrong delay calculation in next_event_delay()
- MINOR: stick-tables/counters: add http_fail_cnt and http_fail_rate data types
- MINOR: ssl: add SSL_SERVER_LOCK label in threads.h
- BUG/MINOR: mux-h1: Don't increment HTTP error counter for 408/500/501 errors
- BUG/MINOR: http-ana: Don't increment HTTP error counter on internal errors
- BUG/MEDIUM: mux-h1: Always set CS_FL_EOI for response in MSG_DONE state
- BUG/MINOR: mux-h1: Fix data skipping for bodyless responses
- BUG/MINOR: mux-h1: Don't blindly skip EOT block for non-chunked messages
- BUG/MEDIUM: mux-h2: Add EOT block when EOM flag is set on an empty HTX message
- MINOR: mux-h1: Be sure EOM flag is set when processing end of outgoing message
- REGTESTS: Add a script to test payload skipping for bodyless HTTP responses
- BUG/MINOR: server: re-align state file fields number
- CLEANUP: muxes: Remove useless calls to b_realign_if_empty()
- BUG/MINOR: tools: Fix a memory leak on error path in parse_dotted_uints()
- CLEANUP: remove unused variable assigned found by Coverity
- CLEANUP: queue: Remove useless tests on p or pp in pendconn_process_next_strm()
- BUG/MINOR: backend: hold correctly lock when killing idle conn
- MEDIUM: connection: protect idle conn lists with locks
- MEDIUM: connection: replace idle conn lists by eb trees
- MINOR: backend: search conn in idle/safe trees after available
- MINOR: backend: search conn in idle tree after safe on always reuse
- MINOR: connection: prepare hash calcul for server conns
- MINOR: connection: use the srv pointer for the srv conn hash
- MINOR: backend: compare conn hash for session conn reuse
- MINOR: connection: use sni as parameter for srv conn hash
- MINOR: reg-tests: test http-reuse with sni
- MINOR: backend: rewrite alloc of stream target address
- MINOR: connection: use dst addr as parameter for srv conn hash
- MINOR: reg-test: test http-reuse with specific dst addr
- MINOR: backend: rewrite alloc of connection src address
- MINOR: connection: use src addr as parameter for srv conn hash
- MINOR: connection: use proxy protocol as parameter for srv conn hash
- MINOR: reg-tests: test http-reuse with proxy protocol
- MINOR: doc: update http reuse for new eligilible connections
- BUG/MINOR: backend: fix compilation without ssl
- REGTESTS: adjust http_reuse_conn_hash requirements
- REGTESTS: deactivate a failed test on CI in http_reuse_conn_hash
- REGTESTS: fix sni used in http_reuse_conn_hash for libressl 3.3.0
- CI: cirrus: update FreeBSD image to 12.2
- MEDIUM: cli: add check-addr command
- MEDIUM: cli: add agent-port command
- MEDIUM: server: add server-states version 2
- MEDIUM: server: support {check,agent}_addr, agent_port in server state
- MINOR: server: enhance error precision when applying server state
- BUG/MINOR: server: Fix server-state-file-name directive
- CLEANUP: deinit: release global and per-proxy server-state variables on deinit
- BUG/MEDIUM: config: don't pick unset values from last defaults section
- BUG/MINOR: stats: revert the change on ST_CONVDONE
- BUG/MINOR: cfgparse: do not mention "addr:port" as supported on proxy lines
- BUG/MINOR: http-htx: defpx must be a const in proxy_dup_default_conf_errors()
- BUG/MINOR: tcpheck: the source list must be a const in dup_tcpcheck_var()
- BUILD: proxy: add missing compression-t.h to proxy-t.h
- REORG: move init_default_instance() to proxy.c and pass it the defproxy pointer
- REORG: proxy: centralize the proxy allocation code into alloc_new_proxy()
- MEDIUM: proxy: only take defaults when a default proxy is passed.
- MINOR: proxy: move the defproxy freeing code to proxy.c
- MINOR: proxy: always properly reset the just freed default instance pointers
- BUG/MINOR: extcheck: proxy_parse_extcheck() must take a const for the defproxy
- BUG/MINOR: tcpcheck: proxy_parse_*check*() must take a const for the defproxy
- BUG/MINOR: server: parse_server() must take a const for the defproxy
- MINOR: cfgparse: move defproxy to cfgparse-listen as a static
- MINOR: proxy: add a new capability PR_CAP_DEF
- MINOR: cfgparse: check PR_CAP_DEF instead of comparing poiner against defproxy
- MINOR: cfgparse: use a pointer to the current default proxy
- MINOR: proxy: also store the name for a defaults section
- MINOR: proxy: support storing defaults sections into their own tree
- MEDIUM: proxy: store the default proxies in a tree by name
- MEDIUM: cfgparse: allow a proxy to designate the defaults section to use
- MINOR: http: add baseq sample fetch
- CLEANUP: tcpcheck: Remove a useless test on port variable
- BUG/MINOR: server: Don't call fopen() with server-state filepath set to NULL
- CLEANUP: server: Remove useless "filepath" variable in apply_server_state()
- MINOR: peers/cli: do not dump the peers dictionaries by default on "show peers"
- MINOR: cfgparse: implement a simple if/elif/else/endif macro block handler
- DOC: tune: explain the origin of block size for ssl.cachesize
- MINOR: tcp: add support for defer-accept on FreeBSD.
- MINOR: ring: adds new ring_init function.
- CLEANUP: channel: fix comment in ci_putblk.
- BUG/MINOR: dns: add missing sent counter and parent id to dns counters.
- BUG/MINOR: resolvers: fix attribute packed struct for dns
- MINOR: resolvers: renames some resolvers internal types and removes dns prefix
- MINOR: resolvers: renames type dns_resolvers to resolvers.
- MINOR: resolvers: renames some resolvers specific types to not use dns prefix
- MINOR: resolvers: renames some dns prefixed types using resolv prefix.
- MINOR: resolvers: renames resolvers DNS_RESP_* errcodes RSLV_RESP_*
- MINOR: resolvers: renames resolvers DNS_UPD_* returncodes to RSLV_UPD_*
- MINOR: resolvers: rework prototype suffixes to split resolving and dns.
- MEDIUM: resolvers: move resolvers section parsing from cfgparse.c to dns.c
- MINOR: resolvers: replace nameserver's resolver ref by generic parent pointer
- MINOR: resolvers: rework dns stats prototype because specific to resolvers
- MEDIUM: resolvers: split resolving and dns message exchange layers.
- MEDIUM: resolvers/dns: split dns.c into dns.c and resolvers.c
- MEDIUM: dns: adds code to support pipelined DNS requests over TCP.
- MEDIUM: resolvers: add supports of TCP nameservers in resolvers.
This patch introduce the new line "server" to set a TCP
nameserver in a "resolvers" section:
server <name> <address> [param*]
Used to configure a DNS TCP or stream server. This supports for all
"server" parameters found in 5.2 paragraph. Some of these parameters
are irrelevant for DNS resolving. Note: currently 4 queries are pipelined
on the same connections. A batch of idle connections are removed every
5 seconds. "maxconn" can be configured to limit the amount of those
concurrent connections and TLS should also usable if the server supports
. The current implementation limits to 4 pipelined
The name of the line in configuration is open to discussion
and could be changed before the next release.
A user could eventually ask himself where those 200 bytes block size are
coming from. This patch tries to better explain the origin in case
people are curious or want to double check the reality.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Very often, especially since reg-tests, it would be desirable to be able
to conditionally comment out a config block, such as removing an SSL
binding when SSL is disabled, or enabling HTX only for certain versions,
etc.
This patch introduces a very simple nested block management which takes
".if", ".elif", ".else" and ".endif" directives to take or ignore a block.
For now the conditions are limited to empty string or "0" for false versus
a non-nul integer for true, which already suffices to test environment
variables. Still, it needs to be a bit more advanced with defines, versions
etc.
A set of ".notice", ".warning" and ".alert" statements are provided to
emit messages, often in order to provide advice about how to fix certain
conditions.
The "show peers" output has become huge due to the dictionaries making it
less readable. Now this feature has reached a certain level of maturity
which doesn't warrant to dump it all the time, given that it was essentially
needed by developers. Let's make it optional, and disabled by default, only
when "show peers dict" is requested. The default output reminds about the
command. The output has been divided by 5 :
$ socat - /tmp/sock1 <<< "show peers dict" | wc -l
125
$ socat - /tmp/sock1 <<< "show peers" | wc -l
26
It could be useful to backport this to recent stable versions.
Now it becomes possible to specify "from foo" on a frontend/listen/backend
or even on a "defaults" line, to mention that defaults section "foo" needs
to be used to preset the proxy's settings.
When not set, the last section remains used. In case the designated name
is found at multiple places, it is rejected and an error indicates two
occurrences of the same name. Similarly, if the section name is found,
its name must only use valid characters. This allows multiple named
defaults section to continue to coexist without the risk that they will
cause trouble by accident.
When it comes to "defaults" relying on another defaults, what happens is
just that a new defaults section is created from the designated one. This
will make it possible for example to reuse some settings such as log-format
like below:
defaults tcp-clear
log stdout local0 info
log-format "%ci:%cp/%b/%si:%sp %ST %ts %U/%B %{+Q}r"
defaults tcp-ssl
log stdout local0 info
log-format "%ci:%cp/%b/%si:%sp %ST %ts %U/%B %{+Q}r ssl=%sslv"
defaults http-clear from tcp-clear
mode http
defaults http-ssl from tcp-ssl
mode http
frontend fe1 from http-clear
bind :8001
frontend fe2 from http-ssl
bind :8002
A small corner case remains in the error detection, if a second defaults
section appears with the same name after the point where it was used, and
nobody references it, the duplicate will not be detected. This could be
addressed by performing the syntactic checks in check_config_validity(),
and by postponing the freeing of the defaults, after tagging a defaults
section as explicitly looked up by another section. This doesn't seem
that important at the moment though.
Since the beginning, this directive is documented to accept an optional file
name. But it should also be possible to use it without any argument to use
the backend name as file name. However, when no argument is provided, an
error is reported during the configuration parsing requesting an argument, a
file name or "use-backend-name". And This last special argument is not
documented.
So, to respect the documentation and to avoid configuration breakages, all
modes are now supported. If this directive is called with no argument or
with "use-backend-name", the backend name is use as file name for the
server-state file. Otherwise, the provided string is used.
In addition, we take care to release any previously allocated file name in
case this directive is defines multiple times in the same backend. And an
error is reported if more than one argument are defined. Finally, the
documentation is updated accordingly. Sections supporting this directive are
also mentioned.
This patch should be backported as far as 1.6.
logical followup from cli commands addition, so that the state server
file stays compatible with the changes made at runtime; use previously
added helper to load server attributes.
also alloc a specific chunk to avoid mixing with other called functions
using it
Signed-off-by: William Dauchy <wdauchy@gmail.com>
this patch allows to set agent port at runtime. In order to align with
both `addr` and `check-addr` commands, also add the possibility to
optionnaly set port on `agent-addr` command. This led to a small
refactor in order to use the same function for both `agent-addr` and
`agent-port` commands.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
this patch allows to set server health check address at runtime. In
order to align with `addr` command, also allow to set port optionnaly.
This led to a small refactor in order to use the same function for both
`check-addr` and `check-port` commands.
for `check-port`, we however don't permit the change anymore if checks
are not enabled on the server.
This command becomes more and more useful for people having a consul
like architecture:
- the backend server is located on a container with its own IP
- the health checks are done the consul instance located on the host
with the host IP
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Historically we've been counting lots of client-triggered events in stick
tables to help detect misbehaving ones, but we've been missing the same on
the server side, and there's been repeated requests for being able to count
the server errors per URL in order to precisely monitor the quality of
service or even to avoid routing requests to certain dead services, which
is also called "circuit breaking" nowadays.
This commit introduces http_fail_cnt and http_fail_rate, which work like
http_err_cnt and http_err_rate in that they respectively count events and
their frequency, but they only consider server-side issues such as network
errors, unparsable and truncated responses, and 5xx status codes other
than 501 and 505 (since these ones are usually triggered by the client).
Note that retryable errors are purposely not accounted for, so that only
what the client really sees is considered.
With this it becomes very simple to put some protective measures in place
to perform a redirect or return an excuse page when the error rate goes
beyond a certain threshold for a given URL, and give more chances to the
server to recover from this condition. Typically it could look like this
to bypass a URL causing more than 10 requests per second:
stick-table type string len 80 size 4k expire 1m store http_fail_rate(1m)
http-request track-sc0 base # track host+path, ignore query string
http-request return status 503 content-type text/html \
lf-file excuse.html if { sc0_http_fail_rate gt 10 }
A more advanced mechanism using gpt0 could even implement high/low rates
to disable/enable the service.
Reg-test converteers_ref_cnt_never_dec.vtc was updated to test it.
Released version 2.4-dev7 with the following main changes :
- BUG/MINOR: stats: Continue to fill frontend stats on unimplemented metric
- BUILD: ssl: guard Client Hello callbacks with HAVE_SSL_CLIENT_HELLO_CB macro instead of openssl version
- BUG/MINOR: stats: Init the metric variable when frontend stats are filled
- MINOR: contrib/prometheus-exporter: better output of Not-a-Number
- CLEANUP: stats: improve field selection for frontend http fields
- CLEANUP: assorted typo fixes in the code and comments
- DOC: Improve documentation of the various hdr() fetches
- MEDIUM: stats: allow to select one field in `stats_fill_be_stats`
- MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump
- MEDIUM: stats: allow to select one field in `stats_fill_sv_stats`
- MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump
- MINOR: abort() on my_unreachable() when DEBUG_USE_ABORT is set.
- BUG/MEDIUM: filters/htx: Fix data forwarding when payload length is unknown
- BUG/MINOR: config: fix leak on proxy.conn_src.bind_hdr_name
- MINOR: reg-tests: add http-reuse test
- CLEANUP: srv: fix comment for pool-max-conn
- CLEANUP: backend: remove an obsolete comment on conn_backend_get
- REORG: backend: simplify conn_backend_get
- MINOR: ssl: Server ssl context prepare function refactoring
- MINOR: ssl: Certificate chain loading refactorization
- MEDIUM: ssl: Load client certificates in a ckch for backend servers
- MEDIUM: ssl: Enable backend certificate hot update
- MINOR: ssl: Remove client_crt member of the server's ssl context
- CLEANUP: ssl/cli: rework free in cli_io_handler_commit_cert()
- CLEANUP: ssl: remove SSL_CTX function parameter
- CLEANUP: ssl: make load_srv_{ckchs,cert} match their bind counterpart
- BUILD: Include stdlib.h in compiler.h if DEBUG_USE_ABORT is set
- CI: Fix DEBUG_STRICT definition for Coverity
- BUG/MINOR: stats: Remove a break preventing ST_F_QCUR to be set for servers
- BUG/MINOR: stats: Add a break after filling ST_F_MODE field for servers
- CLEANUP: ssl: remove dead code in ckch_inst_new_load_srv_store()
- BUG/MINOR: ssl: init tmp chunk correctly in ssl_sock_load_sctl_from_file()
- BUG/MEDIUM: session: only retrieve ready idle conn from session
- BUG/MEDIUM: backend: never reuse a connection for tcp mode
- REGTESTS: set_ssl_server_cert.vtc: remove the abort command
- REGTESTS: set_ssl_server_cert.vtc: check the Sha1 Fingerprint
- REGTESTS: set_ssl_server_cert.vtc: check the sha1 from the server
- MEDIUM: stream-int: Take care of EOS if the SI wake callback function
- MINOR: mux-h1: Try to wake up data layer first before calling its wake callback
- MINOR: mux-h1: Wake up H1C after its creation if input buffer is not empty
- MEDIUM: mux-h1: Add ST_READY state for the H1 connections
- MINOR: stream: Add a function to validate TCP to H1 upgrades
- MEDIUM: http-ana: Do nothing in wait-for-request analyzer if not htx
- BUG/MEDIUM: stream: Don't immediatly ack the TCP to H1 upgrades
- BUG/MAJOR: mux-h1: Properly handle TCP to H1 upgrades
- MINOR: htx/http-ana: Save info about Upgrade option in the Connection header
- MEDIUM: http-ana: Refuse invalid 101-switching-protocols responses
- BUG/MINOR: h2/mux-h2: Reject 101 responses with a PROTOCOL_ERROR h2s error
- MINOR: mux-h1/mux-fcgi: Don't set TUNNEL mode if payload length is unknown
- MINOR: mux-h1: Split H1C_F_WAIT_OPPOSITE flag to separate input/output sides
- MINOR: mux-h2: Add 2 flags to help to properly handle tunnel mode
- MEDIUM: mux-h2: Block client data on server side waiting tunnel establishment
- MEDIUM: mux-h2: Close streams when processing data for an aborted tunnel
- MEDIUM: mux-h1: Properly handle tunnel establishments and aborts
- BUG/MAJOR: mux-h1/mux-h2/htx: Fix HTTP tunnel management at the mux level
- MINOR: htx: Rename HTX_FL_EOI flag into HTX_FL_EOM
- REGTESTS: Don't run http_msg_full_on_eom script on the 2.4 anymore
- MINOR: htx: Add a function to know if a block is the only one in a message
- MAJOR: htx: Remove the EOM block type and use HTX_FL_EOM instead
- MINOR: mux-h1: Add a flag on H1 streams with a response known to be bodyless
- MEDIUM: mux-h1: Don't emit any payload for bodyless responses
- MINOR: mux-h1: Don't emit C-L and T-E headers for 204 and 1xx responses
- MINOR: mux-h1: Don't add Connection close/keep-alive header for 1xx messages
- MINOR: h2/mux-h2: Add flags to notify the response is known to have no body
- MEDIUM: mux-h2: Don't emit DATA frame for bodyless responses
- MEDIUM: http-ana: Deal with L7 retries in HTTP analysers
- MINOR: h1: reject websocket handshake if missing key
- MEDIUM: h1: generate WebSocket key on response if needed
- MINOR: mux_h2: define H2_SF_EXT_CONNECT_SENT stream flag
- MEDIUM: h2: parse Extended CONNECT reponse to htx
- MEDIUM: mux_h2: generate Extended CONNECT from htx upgrade
- MEDIUM: h1: add a WebSocket key on handshake if needed
- MEDIUM: mux_h2: generate Extended CONNECT response
- MEDIUM: h2: parse Extended CONNECT request to htx
- MEDIUM: h2: send connect protocol h2 settings
- MINOR: vtc: add test for h1/h2 protocol upgrade translation
- MINOR: vtc: add websocket test
- REGTESTS: Fix required versions for several scripts
- REGTEST: Don't use the websocket to validate http-check
- MINOR: mux-h1/trace: add traces at level ERROR for all kind of errors
- MINOR: mux-fcgi/trace: add traces at level ERROR for all kind of errors
- MINOR: h1: Raise the chunk size limit up to (2^52 - 1)
- BUG/MEDIUM: listener: do not accept connections faster than we can process them
- REGTESTS: set_ssl_server_cert.vtc: set as broken
- Revert "BUG/MEDIUM: listener: do not accept connections faster than we can process them"
- BUG/MINOR: backend: check available list allocation for reuse
- CI: Fix the coverity builds
- DOC: management: fix "show resolvers" alphabetical ordering
- MINOR: tools: add print_time_short() to print a condensed duration value
- MINOR: activity: make profiling more manageable
- MINOR: activity: declare a new structure to collect per-function activity
- MEDIUM: tasks/activity: collect per-task statistics when profiling is enabled
- MINOR: activity: also report collected tasks stats in "show profiling"
- MINOR: activity: flush scheduler stats on "set profiling tasks on"
- MINOR: activity: add a new "show tasks" command to list currently active tasks
- MINOR: listener: export accept_queue_process
- MINOR: session: export session_expire_embryonic()
- MINOR: muxes: export the timeout and shutr task handlers
- MINOR: checks: export a few functions that appear often in trace dumps
- MINOR: peers: export process_peer_sync() to improve traces
- MINOR: stick-tables: export process_table_expire()
- MINOR: mux-h1: Remove first useless test on count in h1_process_output()
- BUG/MINOR: stick-table: Always call smp_fetch_src() with a valid arg list
- MINOR: http-fetch: Don't check if argument list is set in sample fetches
- MINOR: http-conv: Don't check if argument list is set in sample converters
- MINOR: sample: Don't check if argument list is set in sample fetches
- MINOR: ssl-sample: Don't check if argument list is set in sample fetches
- MINOR: mux-h2: Don't tests the start-line when sending HEADERS frame
- MINOR: mux-h2: Slightly improve request HEADERS frames sending
- MINOR: contrib/prometheus-exporter: declare states for objects
- MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels
- MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones
- MINOR: listener: export manage_global_listener_queue()
- BUG/MINOR: activity: take care of late wakeups in "show tasks"
- REGTESTS: set_ssl_server_cert.vtc: remove SSL caching and set as working
- REGTESTS: set_ssl_server_cert: cleanup the SSL caching option
- MINOR: checks: Add function to get the result code corresponding to a status
- MAJOR: contrib/prometheus-exporter: move health check status to labels
- MINOR: contrib/prometheus-exporter: improve service status description field
- MINOR: stats: improve pending connections description
- MINOR: stats: improve max stats descriptions
- MINOR: contrib/prometheus-exporter: use stats desc when possible
- MINOR: contrib/prometheus-exporter: add uweight field
- MINOR: contrib/prometheus-exporter: add recv logs_logs_total field
- CLEANUP: contrib/prometheus-exporter: remove unused includes
- CLEANUP: contrib/prometheus-exporter: align and reorder fields
- CLEANUP: contrib/prometheus-exporter: remove description in README
- DOC: contrib/prometheus-exporter: Add missing metrics in README
- BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX
- BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos
- BUG/MEDIUM: ssl/cli: abort ssl cert is freeing the old store
- BUG/MEDIUM: ssl: check a connection's status before computing a handshake
- BUG/MINOR: mux_h2: fix incorrect stat titles
- MINOR: ssl/cli: flush the server session cache upon 'commit ssl cert'
- BUG/MINOR: cli: fix set server addr/port coherency with health checks
- MINOR: server: Don't set the check port during the update from a state file
- MINOR: dns: Don't set the check port during a server dns resolution
- MEDIUM: check: remove checkport checkaddr flag
- MEDIUM: server: adding support for check_port in server state
- BUG/MINOR: check: consitent way to set agentaddr
- MEDIUM: check: align agentaddr and agentport behaviour
- DOC: server: Add missing params in comment of the server state line parsing
- BUG/MINOR: xxhash: make sure armv6 uses memcpy()
- REGTESTS: mark http-check-send.vtc as 2.4-only
- REGTESTS: mark sample_fetches/hashes.vtc as 2.4-only
- BUG/MINOR: ssl: do not try to use early data if not configured
- REGTESTS: unbreak http-check-send.vtc
- MINOR: cli/show_fd: report local and report ports when known
- BUILD: Makefile: move REGTESTST_TYPE default setting
- BUG/MEDIUM: mux-h2: handle remaining read0 cases
- CLEANUP: http-htx: Set buffer area to NULL instead of malloc(0)
- BUG/MINOR: sock: Unclosed fd in case of connection allocation failure
- BUG/MEDIUM: mux-h2: do not quit the demux loop before setting END_REACHED
in the same manner of agentaddr, we now:
- permit to set agentport through `port` keyword, like it is the case
for agentaddr through `addr`
- set the priority on `agent-port` keyword when used
- add a flag to be able to test when the value is set like for agentaddr
it makes the behaviour between `addr` and `port` more consistent.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
We can currently change the check-port using the cli command `set server
check-port` but there is a consistency issue when using server state.
This patch aims to fix this problem but will be also a good preparation
work to get rid of checkport flag, so we are able to know when checkport
was set by config.
I am fully aware this is not making github #953 moving forward, I
however think this might be acceptable while waiting for a proper
solution and resolve consistency problem faced with port settings.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
This finally adds the long-awaited solution to inspect the run queues
and figure what is eating the CPU or causing latencies. We can even see
the experienced latencies when profiling is enabled. Example on a
saturated process:
> show tasks
Running tasks: 14983 (4 threads)
function places % lat_tot lat_avg
process_stream 4948 33.0 5.840m 70.82ms
h1_io_cb 2535 16.9 - -
main+0x9e670 2508 16.7 2.930m 70.10ms
ssl_sock_io_cb 2499 16.6 - -
si_cs_io_cb 2493 16.6 - -
If a user enables profiling by hand, it makes sense to reset the stats
counters to provide fresh new measurements. Therefore it's worth using
this as the standard method to reset counters.
"show profiling" will now dump the stats collected by the scheduler if
profiling was previously enabled. This will immediately make it obvious
what functions are responsible for others' high latencies or which ones
are suffering from others, and should help spot issues like undesired
wakeups.
Example:
Per-task CPU profiling : on # set profiling tasks {on|auto|off}
Tasks activity:
function calls cpu_tot cpu_avg lat_tot lat_avg
si_cs_io_cb 5569479 23.37s 4.196us - -
h1_io_cb 5558654 13.60s 2.446us - -
process_stream 250841 1.476s 5.882us 3.499s 13.95us
main+0x9e670 198 - - 5.526ms 27.91us
task_run_applet 17 1.509ms 88.77us 205.8us 12.11us
srv_cleanup_idle_connections 12 44.51us 3.708us 25.71us 2.142us
main+0x158c80 9 48.72us 5.413us - -
srv_cleanup_toremove_connections 5 165.1us 33.02us 123.6us 24.72us
The EOM block may be removed. The HTX_FL_EOM flags is enough. Most of time,
to know if the end of the message is reached, we just need to have an empty
HTX message with HTX_FL_EOM flag set. It may also be detected when the last
block of a message with HTX_FL_EOM flag is manipulated.
Removing EOM blocks simplifies the HTX message filling. Indeed, there is no
more edge problems when the message ends but there is no more space to write
the EOM block. However, some part are more tricky. Especially the
compression filter or the FCGI mux. The compression filter must finish the
compression on the last DATA block. Before it was performed on the EOM
block, an extra DATA block with the checksum was added. Now, we must detect
the last DATA block to be sure to finish the compression. The FCGI mux on
its part must be sure to reserve the space for the empty STDIN record on the
last DATA block while this record was inserted on the EOM block.
The H2 multiplexer is probably the part that benefits the most from this
change. Indeed, it is now fairly easier to known when to set the ES flag.
The HTX documentaion has been updated accordingly.
GitHub issue #796 notes that many administrators miss the fact that the `hdr()`
fetch (without the `f`) splits the header value at commas. This is only
mentioned at the end of a long paragraph.
This patch attempts to improve the documentation by:
- Explaning the "comma issue" as early as possible.
- Adding newlines to split the explanation into distinct sections.
- Reducing duplication by making the `res` siblings refer to their `req`
counterparts.
This patch may be backported as long as it applies cleanly. During the
refactoring I needed to adjust several explanations for consistency and not all
of them might be available in older branches.
Released version 2.4-dev6 with the following main changes :
- MINOR: converter: adding support for url_enc
- BUILD: SSL: guard TLS13 ciphersuites with HAVE_SSL_CTX_SET_CIPHERSUITES
- BUILD: ssl: guard EVP_PKEY_get_default_digest_nid with ASN1_PKEY_CTRL_DEFAULT_MD_NID
- BUILD: ssl: guard openssl specific with SSL_READ_EARLY_DATA_SUCCESS
- BUILD: Makefile: exclude broken tests by default
- CLEANUP: cfgparse: replace "realloc" with "my_realloc2" to fix to memory leak on error
- BUG/MINOR: hlua: Fix memory leak in hlua_alloc
- MINOR: contrib/prometheus-exporter: export build_info
- DOC: fix some spelling issues over multiple files
- CLEANUP: Fix spelling errors in comments
- SCRIPTS: announce-release: fix typo in help message
- CI: github: add a few more words to the codespell ignore list
- DOC: Add maintainers for the Prometheus exporter
- BUG/MINOR: sample: fix concat() converter's corruption with non-string variables
- BUG/MINOR: server: Memory leak of proxy.used_server_addr during deinit
- CLEANUP: sample: remove uneeded check in json validation
- MINOR: reg-tests: add a way to add service dependency
- BUG/MINOR: sample: check alloc_trash_chunk return value in concat()
- BUG/MINOR: reg-tests: fix service dependency script
- MINOR: reg-tests: add base prometheus test
- Revert "BUG/MINOR: dns: SRV records ignores duplicated AR records"
- BUG/MINOR: sample: Memory leak of sample_expr structure in case of error
- BUG/MINOR: check: Don't perform any check on servers defined in a frontend
- BUG/MINOR: init: enforce strict-limits when using master-worker
- MINOR: contrib/prometheus-exporter: avoid connection close header
- MINOR: contrib/prometheus-exporter: use fill_info for process dump
- BUG/MINOR: init: Use a dynamic buffer to set HAPROXY_CFGFILES env variable
- MINOR: config: Add failifnotcap() to emit an alert on proxy capabilities
- MINOR: server: Forbid server definitions in frontend sections
- BUG/MINOR: threads: Fixes the number of possible cpus report for Mac.
- CLEANUP: pattern: rename pat_ref_commit() to pat_ref_commit_elt()
- MINOR: pattern: add the missing generation ID manipulation functions
- MINOR: peers: Add traces for peer control messages.
- BUG/MINOR: dns: SRV records ignores duplicated AR records (v2)
- BUILD: peers: fix build warning about unused variable
- BUG/MEDIUM: stats: add missing INF_BUILD_INFO definition
- MINOR: cache: Do not store responses with an unknown encoding
- BUG/MINOR: peers: Possible appctx pointer dereference.
- MINOR: build: discard echoing in help target
- MINOR: cache: Remove the `hash` part of the accept-encoding secondary key
- CLEANUP: cache: Use proper data types in secondary_key_cmp()
- CLEANUP: Rename accept_encoding_hash_cmp to accept_encoding_bitmap_cmp
- BUG/MINOR: peers: Wrong "new_conn" value for "show peers" CLI command.
- MINOR: contrib: Make the wireshark peers dissector compile for more distribs.
- BUG/MINOR: mux_h2: missing space between "st" and ".flg" in the "show fd" helper
- CLEANUP: tools: make resolve_sym_name() take a const pointer
- CLEANUP: cli: make "show fd" use a const connection to access other fields
- MINOR: cli: make "show fd" also report the xprt and xprt_ctx
- MINOR: xprt: add a new show_fd() helper to complete some "show fd" dumps.
- MINOR: ssl: provide a "show fd" helper to report important SSL information
- MINOR: xprt/mux: export all *_io_cb functions so that "show fd" resolves them
- MINOR: mux-h2: make the "show fd" helper also decode the h2s subscriber when known
- MINOR: mux-h1: make the "show fd" helper also decode the h1s subscriber when known
- MINOR: mux-fcgi: make the "show fd" helper also decode the fstrm subscriber when known
- CI: Pin VTest to a known good commit
- MINOR: cli: give the show_fd helpers the ability to report a suspicious entry
- MINOR: cli/show_fd: report some easily detectable suspicious states
- MINOR: ssl/show_fd: report some FDs as suspicious when possible
- MINOR: mux-h2/show_fd: report as suspicious an entry with too many calls
- MINOR: mux-h1/show_fd: report as suspicious an entry with too many calls
- BUG/MINOR: mworker: define _GNU_SOURCE for strsignal()
- BUG/MEDIUM: tcpcheck: Don't destroy connection in the wake callback context
- BUG/MEDIUM: mux-h2: Xfer rxbuf to the upper layer when creating a front stream
- MINOR: http: Add HTTP 501-not-implemented error message
- MINOR: muxes: Add exit status for errors about not implemented features
- MINOR: mux-h1: Be prepared to return 501-not-implemented error during parsing
- MEDIUM: mux-h1: Return a 501-not-implemented for upgrade requests with a body
- DOC: Remove space after comma in converter signature
- DOC: Rename '<var name>' to '<var>' in converter signature
- MINOR: stats: duplicate 3 fields in bytes in info
- MINOR: stats: add new start time field
- MINOR: contrib/prometheus-exporter: merge info description from stats
- MEDIUM: stats: allow to select one field in `stats_fill_fe_stats`
- MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump
- MINOR: contrib/prometheus-exporter: Don't needlessly set empty label for metrics
- MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag
- MINOR: contrib/prometheus-exporter: Add promex_metric struct defining a metric
- MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics
- BUG/MINOR: stream: Don't update counters when TCP to H2 upgrades are performed
- BUG/MEDIUM: mux-h2: fix read0 handling on partial frames
- MINOR: debug: always export the my_backtrace function
- MINOR: debug: extract the backtrace dumping code to its own function
- MINOR: debug: create ha_backtrace_to_stderr() to dump an instant backtrace
- MEDIUM: debug: now always print a backtrace on CRASH_NOW() and friends
- MINOR: debug: let ha_dump_backtrace() dump a bit further for some callers
- BUILD: debug: fix build warning by consuming the write() result
- MINOR: lua: remove unused variable
- BUILD/MINOR: lua: define _GNU_SOURCE for LLONG_MAX
Add the support for the 501-not-implemented status code with the
corresponding default message. The documentation is updated accordingly
because it is now part of status codes HAProxy may emit via an errorfile or
a deny/return HTTP action.
Now the show_fd helpers at the transport and mux levels return an integer
which indicates whether or not the inspected entry looks suspicious. When
an entry is reported as suspicious, "show fd" will suffix it with an
exclamation mark ('!') in the dump, that is supposed to help detecting
them.
For now, helpers were adjusted to adapt to the new API but none of them
reports any suspicious entry yet.
If a server varies on the accept-encoding header and it sends a response
with an encoding we do not know (see parse_encoding_value function), we
will not store it. This will prevent unexpected errors caused by
cache collisions that could happen in accept_encoding_hash_cmp.
add base support for url encode following RFC3986, supporting `query`
type only.
- add test checking url_enc/url_dec/url_enc
- update documentation
- leave the door open for future changes
this should resolve github issue #941
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Released version 2.4-dev5 with the following main changes :
- BUG/MEDIUM: mux_h2: Add missing braces in h2_snd_buf()around trace+wakeup
- BUILD: hpack: hpack-tbl-t.h uses VAR_ARRAY but does not include compiler.h
- MINOR: time: increase the minimum wakeup interval to 60s
- MINOR: check: do not ignore a connection header for http-check send
- REGTESTS: complete http-check test
- CI: travis-ci: drop coverity scan builds
- MINOR: atomic: don't use ; to separate instruction on aarch64.
- IMPORT: xxhash: update to v0.8.0 that introduces stable XXH3 variant
- MEDIUM: xxhash: use the XXH3 functions to generate 64-bit hashes
- MEDIUM: xxhash: use the XXH_INLINE_ALL macro to inline all functions
- CLEANUP: xxhash: remove the unused src/xxhash.c
- MINOR: sample: add the xxh3 converter
- REGTESTS: add tests for the xxh3 converter
- MINOR: protocol: Create proto_quic QUIC protocol layer.
- MINOR: connection: Attach a "quic_conn" struct to "connection" struct.
- MINOR: quic: Redefine control layer callbacks which are QUIC specific.
- MINOR: ssl_sock: Initialize BIO and SSL objects outside of ssl_sock_init()
- MINOR: connection: Add a new xprt to connection.
- MINOR: ssl: Export definitions required by QUIC.
- MINOR: cfgparse: Do not modify the QUIC xprt when parsing "ssl".
- MINOR: tools: Add support for QUIC addresses parsing.
- MINOR: quic: Add definitions for QUIC protocol.
- MINOR: quic: Import C source code files for QUIC protocol.
- MINOR: listener: Add QUIC info to listeners and receivers.
- MINOR: server: Add QUIC definitions to servers.
- MINOR: ssl: SSL CTX initialization modifications for QUIC.
- MINOR: ssl: QUIC transport parameters parsing.
- MINOR: quic: QUIC socket management finalization.
- MINOR: cfgparse: QUIC default server transport parameters init.
- MINOR: quic: Enable the compilation of QUIC modules.
- MAJOR: quic: Make usage of ebtrees to store QUIC ACK ranges.
- MINOR: quic: Attempt to make trace more readable
- MINOR: quic: Make usage of the congestion control window.
- MINOR: quic: Flag RX packet as ack-eliciting from the generic parser.
- MINOR: quic: Code reordering to help in reviewing/modifying.
- MINOR: quic: Add traces to congestion avoidance NewReno callback.
- MINOR: quic: Display the SSL alert in ->ssl_send_alert() callback.
- MINOR: quic: Update the initial salt to that of draft-29.
- MINOR: quic: Add traces for in flght ack-eliciting packet counter.
- MINOR: quic: make a packet build fails when qc_build_frm() fails.
- MINOR: quic: Add traces for quic_packet_encrypt().
- MINOR: cache: Refactoring of secondary_key building functions
- MINOR: cache: Avoid storing responses whose secondary key was not correctly calculated
- BUG/MINOR: cache: Manage multiple headers in accept-encoding normalization
- MINOR: cache: Add specific secondary key comparison mechanism
- MINOR: http: Add helper functions to trim spaces and tabs
- MEDIUM: cache: Manage a subset of encodings in accept-encoding normalizer
- REGTESTS: cache: Simplify vary.vtc file
- REGTESTS: cache: Add a specific test for the accept-encoding normalizer
- MINOR: cache: Remove redundant test in http_action_req_cache_use
- MINOR: cache: Replace the "process-vary" option's expected values
- CI: GitHub Actions: enable daily Coverity scan
- BUG/MEDIUM: cache: Fix hash collision in `accept-encoding` handling for `Vary`
- MEDIUM: stick-tables: Add srvkey option to stick-table
- REGTESTS: add test for stickiness using "srvkey addr"
- BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11
- BUG/MINOR: sink: Return an allocation failure in __sink_new if strdup() fails
- BUG/MINOR: lua: Fix memory leak error cases in hlua_config_prepend_path
- MINOR: lua: Use consistent error message 'memory allocation failed'
- CLEANUP: Compare the return value of `XXXcmp()` functions with zero
- CLEANUP: Apply the coccinelle patch for `XXXcmp()` on include/
- CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/
- MINOR: qpack: Add static header table definitions for QPACK.
- CLEANUP: qpack: Wrong comment about the draft for QPACK static header table.
- CLEANUP: quic: Remove useless QUIC event trace definitions.
- BUG/MINOR: quic: Possible CRYPTO frame building errors.
- MINOR: quic: Pass quic_conn struct to frame parsers.
- BUG/MINOR: quic: Wrong STREAM frames parsing.
- MINOR: quic: Drop packets with STREAM frames with wrong direction.
- CLEANUP: ssl: Remove useless loop in tlskeys_list_get_next()
- CLEANUP: ssl: Remove useless local variable in tlskeys_list_get_next()
- MINOR: ssl: make tlskeys_list_get_next() take a list element
- Revert "BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11"
- BUG/MINOR: cfgparse: Fail if the strdup() for `rule->be.name` for `use_backend` fails
- CLEANUP: mworker: remove duplicate pointer tests in cfg_parse_program()
- CLEANUP: Reduce scope of `header_name` in http_action_store_cache()
- CLEANUP: Reduce scope of `hdr_age` in http_action_store_cache()
- CLEANUP: spoe: fix typo on `var_check_arg` comment
- BUG/MINOR: tcpcheck: Report a L7OK if the last evaluated rule is a send rule
- CI: github actions: build several popular "contrib" tools
- DOC: Improve the message printed when running `make` w/o `TARGET`
- BUG/MEDIUM: server: srv_set_addr_desc() crashes when a server has no address
- REGTESTS: add unresolvable servers to srvkey-addr
- BUG/MINOR: stats: Make stat_l variable used to dump a stat line thread local
- BUG/MINOR: quic: NULL pointer dereferences when building post handshake frames.
- SCRIPTS: improve announce-release to support different tag and versions
- SCRIPTS: make announce release support preparing announces before tag exists
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: srv: do not init address if backend is disabled
- BUG/MINOR: srv: do not cleanup idle conns if pool max is null
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: few extra typo and fixes over last one ("ot" -> "to")
This allows using the address of the server rather than the name of the
server for keeping track of servers in a backend for stickiness.
The peers code was also extended to support feeding the dictionary using
this key instead of the name.
Fixes#814
Allow the user to specify a custom Connection header for http-check
send. This is useful for example to implement a websocket upgrade check.
If no connection header has been set, a 'Connection: close' header is
automatically appended to allow the server to close the connection
immediately after the request/response.
Update the documentation related to http-check send.
This fixes the github issue #1009.
Released version 2.4-dev4 with the following main changes :
- BUG/MEDIUM: lb-leastconn: Reposition a server using the right eweight
- BUG/MEDIUM: mux-h1: Fix a deadlock when a 408 error is pending for a client
- BUG/MEDIUM: ssl/crt-list: bad behavior with "commit ssl cert"
- BUG/MAJOR: cache: Crash because of disabled entry not removed from the tree
- BUILD: SSL: fine guard for SSL_CTX_add_server_custom_ext call
- MEDIUM: cache: Add a secondary entry counter and insertion limitation
- MEDIUM: cache: Avoid going over duplicates lists too often
- MINOR: cache: Add a max-secondary-entries cache option
- CI: cirrus: drop CentOS 6 builds
- BUILD: Makefile: have "make clean" destroy .o/.a/.s in contrib subdirs as well
- MINOR: vars: replace static functions with global ones
- MINOR: opentracing: add ARGC_OT enum
- CONTRIB: opentracing: add the OpenTracing filter
- DOC: opentracing: add the OpenTracing filter section
- REGTESTS: make use of HAPROXY_ARGS and pass -dM by default
- BUG/MINOR: http: Establish a tunnel for all 2xx responses to a CONNECT
- BUG/MINOR: mux-h1: Don't set CS_FL_EOI too early for protocol upgrade requests
- BUG/MEDIUM: http-ana: Never for sending data in TUNNEL mode
- CLEANUP: mux-h2: Rename h2s_frt_make_resp_data() to be generic
- CLEANUP: mux-h2: Rename h2c_frt_handle_data() to be generic
- BUG/MEDIUM: mux-h1: Handle h1_process() failures on a pipelined request
- CLEANUP: debug: mark the RNG's seed as unsigned
- CONTRIB: halog: fix build issue caused by %L printf format
- CONTRIB: halog: mark the has_zero* functions unused
- CONTRIB: halog: fix signed/unsigned build warnings on counts and timestamps
- CONTRIB: debug: address "poll" utility build on non-linux platforms
- BUILD: plock: remove dead code that causes a warning in gcc 11
- BUILD: ssl: fine guard for SSL_CTX_get0_privatekey call
- BUG/MINOR: dns: SRV records ignores duplicated AR records
- DOC: fix "smp_size" vs "sample_size" in "log" directive arguments
- CLEANUP: assorted typo fixes in the code and comments
- DOC: assorted typo fixes in the documentation
- CI: codespell: whitelist "te" and "nd" words
This new option allows to tune the maximum number of simultaneous
entries with the same primary key in the cache (secondary entries).
When we try to store a response in the cache and there are already
max-secondary-entries living entries in the cache, the storage will
fail (but the response will still be sent to the client).
It defaults to 10 and does not have a maximum number.
Released version 2.4-dev3 with the following main changes :
- MINOR: log: Logging HTTP path only with %HPO
- BUG/MINOR: mux-h2/stats: make stream/connection proto errors more accurate
- MINOR: traces: add a new level "error" below the "user" level
- MINOR: mux-h2/trace: add traces at level ERROR for protocol errors
- BUG/MINOR: mux-h2/stats: not all GOAWAY frames are errors
- BUG/MINOR: lua: missing "\n" in error message
- BUG/MINOR: lua: lua-load doesn't check its parameters
- BUG/MINOR: lua: Post init register function are not executed beyond the first one
- BUG/MINOR: lua: Some lua init operation are processed unsafe
- MINOR: actions: Export actions lookup functions
- MINOR: actions: add a function returning a service pointer from its name
- MINOR: cli: add a function to look up a CLI service description
- BUG/MINOR: lua: warn when registering action, conv, sf, cli or applet multiple times
- MINOR: cache: Improve accept_encoding_normalizer
- MINOR: cache: Add entry to the tree as soon as possible
- BUG/MINOR: trace: Wrong displayed trace level
- BUG/MAJOR: ring: tcp forward on ring can break the reader counter.
- MINOR: lua: simplify hlua_alloc() to only rely on realloc()
- MEDIUM: lua-thread: use atomics for memory accounting
- MINOR: lua-thread: remove struct hlua from function hlua_prepend_path()
- MEDIUM: lua-thread: make hlua_post_init() no longer use the runtime execution function
- MINOR: lua-thread: hlua_ctx_renew() is never called with main gL lua state
- MINOR: lua-thread: Use NULL context for main lua state
- MINOR: lua-thread: Stop usage of struct hlua for the global lua state
- MINOR: lua-thread: Replace embedded struct hlua_function by a pointer
- MINOR: lua-thread: Split hlua_init() function in two parts
- MINOR: lua-thread: make hlua_ctx_init() get L from its caller
- MINOR: lua-thread: Split hlua_load function in two parts
- MINOR: lua-thread: Split hlua_post_init() function in two parts
- MINOR: lua-thread: Add the "thread" core variable
- MEDIUM: lua-thread: No longer use locked context in initialization parts
- MEDIUM: lua-thread: Apply lock only if the parent state is the main thread
- MINOR: lua-thread: Replace global gL var with an array of states
- MINOR: lua-thread: Replace "struct hlua_function" allocation by dedicated function
- MINOR: lua-thread: Replace state_from by state_id
- MINOR: lua-thread: Store each function reference and init reference in array
- MEDIUM: lua-thread: Add the lua-load-per-thread directive
- MINOR: lua-thread: Add verbosity in errors
- REGTESTS: add a test for the threaded Lua code
- BUILD/MINOR: haproxy DragonFlyBSD affinity build update.
- DOC/MINOR: Fix formatting in Management Guide
- MINOR: cache: Do not store stale entry
- MINOR: cache: Add extra "cache-control" value checks
- MEDIUM: cache: Remove cache entry in case of POST on the same resource
- MINOR: cache: Consider invalid Age values as stale
- BUG/MEDIUM: lua-thread: some parts must be initialized once
- BUG/MINOR: lua-thread: close all states on deinit
- BUG/MINOR: listener: use sockaddr_in6 for IPv6
- BUG/MINOR: mux-h1: Handle keep-alive timeout for idle frontend connections
- MINOR: session: Add the idle duration field into the session
- MINOR: mux-h1: Update session idle duration when data are received
- MINOR: mux-h1: Reset session dates and durations info when the CS is detached
- MINOR: logs: Use session idle duration when no stream is provided
- MINOR: stream: Always get idle duration from the session
- MINOR: stream: Don't retrieve anymore timing info from the mux csinfo
- MINOR: mux-h1: Don't provide anymore timing info using cs_info structure
- MINOR: muxes: Remove get_cs_info callback function now useless
- MINOR: stream: Pass an optional input buffer when a stream is created
- MINOR: mux-h1: Add a flag to disable reads to wait opposite side
- MEDIUM: mux-h1: Use a h1c flag to block reads when splicing is in-progress
- MINOR: mux-h1: Introduce H1C_F_IS_BACK flag on the H1 connection
- MINOR: mux-h1: Separate parsing and formatting errors at H1 stream level
- MINOR: mux-h1: Split front/back h1 stream creation in 2 functions
- MINOR: mux-h1: Add a rxbuf into the H1 stream
- MINOR: mux-h1: Don't set CS flags in internal parsing functions
- MINOR: mux-h1: Add embryonic and attached states on the H1 connection
- MINOR: mux-h1: rework the h1_timeout_task() function
- MINOR: mux-h1: Reset more H1C flags when a H1 stream is destroyed
- MINOR: mux-h1: Disable reads if an error was reported on the H1 stream
- MINOR: mux-h1: Rework how shutdowns are handled
- MINOR: mux-h1: Rework h1_refresh_timeout to be easier to read
- MINOR: mux-h1: Process next request for IDLE connection only
- MINOR: mux-h1: Add a idle expiration date on the H1 connection
- MINOR: stick-tables: Add functions to update some values of a tracked counter
- MINOR: session: Add functions to increase http values of tracked counters
- MINOR: mux: Add a ctl parameter to get the exit status of the multiplexers
- MINOR: logs: Get the multiplexer exist status when no stream is provided
- MINOR: mux-h1: Add functions to send HTTP errors from the mux
- MAJOR: mux-h1: Create the client stream as later as possible
- DOC: config: Add notes about errors emitted by H1 mux
- CLEANUP: mux-h1: Rename H1C_F_CS_* flags and reorder H1C flags
- MINOR: http-ana: Remove useless update of t_idle duration of the stream
- CLEANUP: htx: Remove HTX_FL_UPGRADE unsued flag
- MEDIUM: http-ana: Don't process partial or empty request anymore
- CLEANUP: http-ana: Remove TX_WAIT_NEXT_RQ unsued flag
- CLEANUP: connection: Remove CS_FL_READ_PARTIAL flag
- REGTESTS: Fix proxy_protocol_tlv_validation
- MINOR: http-ana: Properly set message flags from the start-line flags
- MINOR: h1-htx/http-ana: Set BODYLESS flag on message in TUNNEL state
- MINOR: protocol: add a ->set_port() helper to address families
- MINOR: listener: automatically set the port when creating listeners
- MINOR: listener: now use a generic add_listener() function
- MEDIUM: ssl: fatal error with bundle + openssl < 1.1.1
- BUG/MEDIUM: stream: Xfer the input buffer to a fully created stream
- BUG/MINOR: stream: Don't use input buffer after the ownership xfer
- MINOR: protocol: remove the redundant ->sock_domain field
- MINOR: protocol: export protocol definitions
- CLEANUP: protocol: group protocol struct members by usage
- MINOR: protocol: add a set of ctrl_init/ctrl_close methods for setup/teardown
- MINOR: connection: use the control layer's init/close
- MINOR: udp: export udp_suspend_receiver() and udp_resume_receiver()
- BUG/MAJOR: spoa/python: Fixing return None
- DOC: spoa/python: Fixing typo in IP related error messages
- DOC: spoa/python: Rephrasing memory related error messages
- DOC: spoa/python: Fixing typos in comments
- BUG/MINOR: spoa/python: Cleanup references for failed Module Addobject operations
- BUG/MINOR: spoa/python: Cleanup ipaddress objects if initialization fails
- BUG/MEDIUM: spoa/python: Fixing PyObject_Call positional arguments
- BUG/MEDIUM: spoa/python: Fixing references to None
- DOC: email change of the DeviceAtlas maintainer
- MINOR: cache: Dump secondary entries in "show cache"
- CLEANUP: connection: use fd_stop_both() instead of conn_stop_polling()
- MINOR: stream-int: don't touch polling anymore on shutdown
- MINOR: connection: implement cs_drain_and_close()
- MINOR: mux-pt: take care of CS_SHR_DRAIN in shutr()
- MINOR: checks: use cs_drain_and_close() instead of draining the connection
- MINOR: checks: don't call conn_cond_update_polling() anymore
- CLEANUP: connection: open-code conn_cond_update_polling() and update the comment
- CLEANUP: connection: remove the unused conn_{stop,cond_update}_polling()
- BUG/MINOR: http-check: Use right condition to consider HTX message as full
- BUG/MINOR: tcpcheck: Don't rearm the check timeout on each read
- MINOR: tcpcheck: Only wait for more payload data on HTTP expect rules
- BUG/MINOR: tools: make parse_time_err() more strict on the timer validity
- BUG/MINOR: tools: Reject size format not starting by a digit
- MINOR: action: define enum for timeout type of the set-timeout rule
- MINOR: stream: prepare the hot refresh of timeouts
- MEDIUM: stream: support a dynamic server timeout
- MEDIUM: stream: support a dynamic tunnel timeout
- MEDIUM: http_act: define set-timeout server/tunnel action
- MINOR: frontend: add client timeout sample fetch
- MINOR: backend: add timeout sample fetches
- MINOR: stream: add sample fetches
- MINOR: stream: add timeout sample fetches
- REGTESTS: add regtest for http-request set-timeout
- CLEANUP: remove the unused fd_stop_send() in conn_xprt_shutw{,_hard}()
- CLEANUP: connection: remove the unneeded fd_stop_{recv,send} on read0/shutw
- MINOR: connection: remove sock-specific code from conn_sock_send()
- REORG: connection: move the socket iocb (conn_fd_handler) to sock.c
- MINOR: protocol: add a ->drain() function at the connection control layer
- MINOR: connection: make conn_sock_drain() use the control layer's ->drain()
- MINOR: protocol: add a pair of check_events/ignore_events functions at the ctrl layer
- MEDIUM: connection: make use of the control layer check_events/ignore_events
Add cur_server_timeout and cur_tunnel_timeout.
These sample fetches return the current timeout value for a stream. This
is useful to retrieve the value of a timeout which was changed via a
set-timeout rule.
Add a new http-request action 'set-timeout [server/tunnel]'. This action
can be used to update the server or tunnel timeout of a stream. It takes
two parameters, the timeout name to update and the new timeout value.
This rule is only valid for a proxy with backend capabilities. The
timeout value cannot be null. A sample expression can also be used
instead of a plain value.
The duplicated entries (in case of vary) were not taken into account by
the "show cache" command. They are now dumped too.
A new "vary" column is added to the output. It contains the complete
seocndary key (in hex format).
Now, some errors are handled by the H1 multiplexer. During the headers
parsing request, there is no stream attached to the H1 mux. Thus, if an
error is reported at this stage, it is handled by the mux itself. If
possible the corresponding frontend errorfile is used, but it should be a
static message. Custom error messages are not supported. Otherwise, default
error messages are used.
In addition, the http analysis has not started yet, so http-after-response
ruleset is not evaluated and cannot alter these early responses.
The goal is to allow execution of one main lua state per thread.
This patch contains the main job. The lua init is done using these
steps:
- "lua-load-per-thread" loads the lua code in the first thread
- it creates the structs
- it stores loaded files
- the 1st step load is completed (execution of hlua_post_init)
and now, we known the number of threads
- we initilize lua states for all remaining threads
- for each one, we load the lua file
- for each one, we execute post-init
Once all is loaded, we control consistency of functions references.
The rules are:
- a function reference cannot be in the shared lua state and in
a per-thread lua state at the same time.
- if a function reference is declared in a per-thread lua state, it
must be declared in all per-thread lua states
The goal is to allow execution of one main lua state per thread.
This commit introduces this variable in the core. Lua state initialized
by thread will have access to this variable, which reports the executing
thread. 0 indicates the shared thread. Programs which must be executed
only once can check for core.thread <= 1.
The goal is to no longer use "struct hlua" with global main lua_state.
This patch returns NULL value when some code tries go get the hlua struct
associated with a task through hlua_gethlua(). This functions is useful
only during runtime because the struct hlua contains only runtime states.
Some Lua functions allowed to yield are called from init environment.
I'm not sure this is a good practice. Maybe it will be clever to
disallow calling this kind of functions.
This patch adds a new logging variable '%HPO' for logging HTTP path only
(without query string) from relative or absolute URI.
For example:
log-format "hpo=%HPO hp=%HP hu=%HU hq=%HQ"
GET /r/1 HTTP/1.1
=>
hpo=/r/1 hp=/r/1 hu=/r/1 hq=
GET /r/2?q=2 HTTP/1.1
=>
hpo=/r/2 hp=/r/2 hu=/r/2?q=2 hq=?q=2
GET http://host/r/3 HTTP/1.1
=>
hpo=/r/3 hp=http://host/r/3 hu=http://host/r/3 hq=
GET http://host/r/4?q=4 HTTP/1.1
=>
hpo=/r/4 hp=http://host/r/4 hu=http://host/r/4?q=4 hq=?q=4
Released version 2.4-dev2 with the following main changes :
- BUILD: Make DEBUG part of .build_opts
- BUILD: Show the value of DEBUG= in haproxy -vv
- CI: Set DEBUG=-DDEBUG_STRICT=1 in GitHub Actions
- MINOR: stream: Add level 7 retries on http error 401, 403
- CLEANUP: remove unused function "ssl_sock_is_ckch_valid"
- BUILD: SSL: add BoringSSL guarding to "RAND_keep_random_devices_open"
- BUILD: SSL: do not "update" BoringSSL version equivalent anymore
- BUG/MEDIUM: http_act: Restore init of log-format list
- DOC: better describes how to configure a fallback crt
- BUG/MAJOR: filters: Always keep all offsets up to date during data filtering
- MINOR: cache: Prepare helper functions for Vary support
- MEDIUM: cache: Add the Vary header support
- MINOR: cache: Add a process-vary option that can enable/disable Vary processing
- BUG/CRITICAL: cache: Fix trivial crash by sending accept-encoding header
- BUG/MAJOR: peers: fix partial message decoding
- DOC: cache: Add new caching limitation information
- DOC: cache: Add information about Vary support
- DOC: better document the config file format and escaping/quoting rules
- DOC: Clarify %HP description in log-format
- CI: github actions: update LibreSSL to 3.3.0
- CI: github actions: enable 51degrees feature
- MINOR: fd/threads: silence a build warning with threads disabled
- BUG/MINOR: tcpcheck: Don't forget to reset tcp-check flags on new kind of check
- MINOR: tcpcheck: Don't handle anymore in-progress send rules in tcpcheck_main
- BUG/MAJOR: tcpcheck: Allocate input and output buffers from the buffer pool
- MINOR: tcpcheck: Don't handle anymore in-progress connect rules in tcpcheck_main
- MINOR: config: Deprecate and ignore tune.chksize global option
- MINOR: config: Add a warning if tune.chksize is used
- REORG: tcpcheck: Move check option parsing functions based on tcp-check
- MINOR: check: Always increment check health counter on CONPASS
- MINOR: tcpcheck: Add support of L7OKC on expect rules error-status argument
- DOC: config: Make disable-on-404 option clearer on transition conditions
- DOC: config: Move req.hdrs and req.hdrs_bin in L7 samples fetches section
- BUG/MINOR: http-fetch: Fix smp_fetch_body() when called from a health-check
- MINOR: plock: use an ARMv8 instruction barrier for the pause instruction
- MINOR: debug: add "debug dev sched" to stress the scheduler.
- MINOR: debug: add a trivial PRNG for scheduler stress-tests
- BUG/MEDIUM: lists: Lock the element while we check if it is in a list.
- MINOR: task: remove tasklet_insert_into_tasklet_list()
- MINOR: task: perform atomic counter increments only once per wakeup
- MINOR: task: remove __tasklet_remove_from_tasklet_list()
- BUG/MEDIUM: task: close a possible data race condition on a tasklet's list link
- BUG/MEDIUM: local log format regression.
Since 2.3 default local log format always adds hostame field.
This behavior change was due to log/sink re-work, because according
to rfc3164 the hostname field is mandatory.
This patch re-introduce a legacy "local" format which is analog
to rfc3164 but with hostname stripped. This is the new
default if logs are generated by haproxy.
To stay compliant with previous configurations, the option
"log-send-hostname" acts as if the default format is switched
to rfc3164.
This patch addresses the github issue #963
This patch should be backported in branches >= 2.3.
L7OKC may now be used as an error status for an HTTP/TCP expect rule. Thus
it is for instance possible to write:
option httpchk GET /isalive
http-check expect status 200,404
http-check expect status 200 error-status L7OKC
It is more or less the same than the disable-on-404 option except that if a
DOWN is up again but still replying a 404 will be set to NOLB state. While
it will stay in DOWN state with the disable-on-404 option.
This option is now ignored because I/O check buffers are now allocated using the
buffer pool. Thus, it is marked as deprecated in the documentation and ignored
during the configuration parsing. The field is also removed from the global
structure.
Because this option is ignored since a recent fix, backported as fare as 2.2,
this patch should be backported too. Especially because it updates the
documentation.
%HP is used to report HTTP request URI in logs, which might be relative
or absolute. Description in documentation should not suggest that it
behaves exactly the same as "path" sample fetch.
This is even more important after 30ee1efe67
because right now, when HTTP2 is a standard, %HP usually returns absolute
URI.
This might be backported as far as 2.1
It's always a pain to figure how to proceed when special characters need
to be embedded inside arguments of an expression. Let's document the
configuration file format and how unquoting/unescaping works at each
level (top level and argument level) so that everyone hopefully finds
suitable reminders or examples for complex cases.
This is related to github issue #200 and addresses issues #712 and #966.
Responses that do not have an explicit expiration time or a validator
will not be cached anymore.
Must be backported if cc9bf2e ("MEDIUM: cache: Change caching
conditions") is backported.
The cache section's process-vary option takes a 0 or 1 value to disable
or enable the vary processing.
When disabled, a response containing such a header will never be cached.
When enabled, we will calculate a preliminary hash for a subset of request
headers on all the incoming requests (which might come with a cpu cost) which
will be used to build a secondary key for a given request (see RFC 7234#4.1).
The default value is 0 (disabled).
A default certificate is always the first one declared in the bind line,
either from `crt` or from `crt-line` option. This commit updates the
description of how to configure a fallback certificate, clarifying that
it needs to be the first one of the bind line.
Should be merged as far as the first SNI filter implementation.
Level-7 retries are only possible with a restricted number of HTTP
return codes. While it is usually not safe to retry on 401 and 403, I
came up with an authentication backend which was not synchronizing
authentication of users. While not perfect, being allowed to also retry
on those return codes is really helpful and acts as a hotfix until we
can fix the backend.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
Released version 2.4-dev1 with the following main changes :
- MINOR: ist: Add istend() function to return a pointer to the end of the string
- MINOR: sample: Add converters to parse FIX messages
- REGTEST: converter: Add a regtest for fix converters
- MINOR: sample: Add converts to parses MQTT messages
- REGTEST: converter: Add a regtest for MQTT converters
- MINOR: compat: automatically include malloc.h on glibc
- MEDIUM: pools: call malloc_trim() from pool_gc()
- MEDIUM: pattern: call malloc_trim() on pat_ref_reload()
- MINOR: pattern: move the update revision to the pat_ref, not the expression
- CLEANUP: pattern: delete the back refs at once during pat_ref_reload()
- MINOR: pattern: new sflag PAT_SF_REGFREE indicates regex_free() is needed
- MINOR: pattern: make the delete and prune functions more generic
- MEDIUM: pattern: link all final elements from the reference
- MEDIUM: pattern: change the pat_del_* functions to delete from the references
- MINOR: pattern: remerge the list and tree deletion functions
- MINOR: pattern: perform a single call to pat_delete_gen() under the expression
- CLEANUP: acl: don't reference the generic pattern deletion function anymore
- CLEANUP: pattern: remove pat_delete_fcts[] and pattern_head->delete()
- MINOR: pattern: introduce pat_ref_delete_by_ptr() to delete a valid reference
- MINOR: pattern: store a generation number in the reference patterns
- MEDIUM: pattern: only match patterns that match the current generation
- MINOR: pattern: add pat_ref_commit() to commit a previously inserted element
- MINOR: pattern: implement pat_ref_load() to load a pattern at a given generation
- MINOR: pattern: add pat_ref_purge_older() to purge old entries
- MEDIUM: pattern: make pat_ref_prune() rely on pat_ref_purge_older()
- MINOR: pattern: during reload, delete elements frem the ref, not the expression
- MINOR: pattern: prepare removal of a pattern from the list head
- MEDIUM: pattern: turn the pattern chaining to single-linked list
- CLEANUP: cfgparse: remove duplicate registration for transparent build options
- BUG/MINOR: ssl: don't report 1024 bits DH param load error when it's higher
- MINOR: http-htx: Add understandable errors for the errorfiles parsing
- MINOR: ssl: instantiate stats module
- MINOR: ssl: count client hello for stats
- MINOR: ssl: add counters for ssl sessions
- DOC: config: Fix a typo on ssl_c_chain_der
- MINOR: server: remove idle lock in srv_cleanup_connections
- BUILD: ssl: silence build warning on uninitialised counters
- BUILD: http-htx: fix build warning regarding long type in printf
- REGTEST: ssl: test wildcard and multi-type + exclusions
- BUG/MEDIUM: ssl/crt-list: correctly insert crt-list line if crt already loaded
- CI: Expand use of GitHub Actions for CI
- REGTEST: ssl: mark reg-tests/ssl/ssl_crt-list_filters.vtc as broken
- BUG/MINOR: pattern: a sample marked as const could be written
- BUG/MINOR: lua: set buffer size during map lookups
- MEDIUM: cache: Change caching conditions
- BUG/MINOR: stats: free dynamically stats fields/lines on shutdown
- BUG/MEDIUM: stats: prevent crash if counters not alloc with dummy one
- MINOR: peers: Add traces to peer_treat_updatemsg().
- BUG/MINOR: peers: Do not ignore a protocol error for dictionary entries.
- BUG/MINOR: peers: Missing TX cache entries reset.
- BUG/MEDIUM: peers: fix decoding of multi-byte length in stick-table messages
- BUG/MINOR: http-fetch: Extract cookie value even when no cookie name
- BUG/MINOR: http-fetch: Fix calls w/o parentheses of the cookie sample fetches
- BUG/MEDIUM: check: reuse srv proto only if using same mode
- MINOR: check: report error on incompatible proto
- MINOR: check: report error on incompatible connect proto
- BUG/MINOR: http-htx: Handle warnings when parsing http-error and http-errors
- BUG/MAJOR: spoe: Be sure to remove all references on a released spoe applet
- MINOR: spoe: Don't close connection in sync mode on processing timeout
- BUG/MINOR: tcpcheck: Don't warn on unused rules if check option is after
- MINOR: init: Fix the prototype for per-thread free callbacks
- MINOR: config/mux-h2: Return ERR_ flags from init_h2() instead of a status
- CLEANUP: config: Return ERR_NONE from config callbacks instead of 0
- MINOR: cfgparse: tighten the scope of newnameserver variable, free it on error.
- REGTEST: make ssl_client_samples and ssl_server_samples require to 2.2
- REGTESTS: Add sample_fetches/cook.vtc
- BUG/MEDIUM: filters: Forward all filtered data at the end of http filtering
- BUG/MINOR: http-ana: Don't wait for the body of CONNECT requests
- CLEANUP: flt-trace: Remove unused random-parsing option
- MINOR: flt-trace: Add an option to inhibits trace messages
- MINOR: flt-trace: Use a bitfield for the trace options
- REGTESTS: Add a script to test the random forwarding with several filters
- REGTESTS: mark the abns test as broken again
- REGTESTS: converter: add url_dec test
- CI: Stop hijacking the hosts file
- CI: Make the h2spec workflow more consistent with the VTest workflow
- CI: travis-ci: remove amd64, osx builds
- CI: travis-ci: arm64 are not allowed to fail anymore
- DOC: add missing 3.10 in the summary
- MINOR: ssl: remove client hello counters
- MEDIUM: stats: add counters for failed handshake
- MINOR: ssl: create common ssl_ctx init
- MEDIUM: cli/ssl: configure ssl on server at runtime
- REGTEST: server/cli_set_ssl.vtc requires OpenSSL
- DOC: coding-style: update a few rules about pointers
- BUG/MINOR: ssl: segv on startup when AKID but no keyid
- BUILD: ssl: use SSL_MODE_ASYNC macro instead of OPENSSL_VERSION
- BUG/MEDIUM: http-ana: Don't eval http-after-response ruleset on empty messages
- BUG/MEDIUM: ssl/crt-list: bundle support broken in crt-list
- BUG/MEDIUM: ssl: error when no certificate are found
- BUG/MINOR: ssl/crt-list: load bundle in crt-list only if activated
- BUG/MEDIUM: ssl/crt-list: fix error when no file found
- CI: Github Actions: enable prometheus exporter
- CI: Github Actions: remove LibreSSL-3.0.2 builds
- CI: Github Actions: enable BoringSSL builds
- CI: travis-ci: remove builds migrated to GH actions
- BUILD: makefile: enable crypt(3) for OpenBSD
- CI: Github Action: run "apt-get update" before packages restore
- BUILD: SSL: guard TLS13 ciphersuites with HAVE_SSL_CTX_SET_CIPHERSUITES
- CI: Pass the github.event_name to matrix.py
- CI: Clean up Windows CI
- DOC: clarify how to create a fallback crt
- CLEANUP: connection: do not use conn->owner when the session is known
- BUG/MAJOR: connection: reset conn->owner when detaching from session list
- REGTESTS: mark proxy_protocol_random_fail as broken
- BUG/MINOR: http_htx: Fix searching headers by substring
- MINOR: http_act: Add -m flag for del-header name matching method
This patch adds -m flag which allows to specify header name
matching method when deleting headers from http request/response.
Currently beg, end, sub, str and reg are supported.
This is related to GitHub issue #909
HAProxy uses CN and SAN of the certificates to match incoming SNI, and
use the matching certificate in the TLS handshake. `crt-list` goes
further and allows to configure SNI filters to explicitly define the
FQDNs that should match a certificate.
The first declared certificate of the `crt-list` option follows the same
rules, and it's also used as a fallback - the certificate that should be
used if SNI isn't provided or the provided one cannot match any
certificate or SNI filter. If a provided SNI matches the CN or SAN of
the first certificate, the first certificate would be used even if a
matching SNI filter is declared later.
This change clarifies this scenario and documents a filter that can be
used to convert the first declared certificate as a proper fallback.
Should be merged as far as the first SNI filter implementation.
It's really annoying to see that in 2020 we're still facing bugs caused
by dangling pointers in the code that result from poorly written rules
about how these pointers are supposed to be handled, set and reset. Let's
add a few supposedly obvious (but apparently not) rules about how pointers
have to be used through out the code in hope to make such bad practices
disappear (or at least have something to point the authors to after
reviewing their code).
in the context of a progressive backend migration, we want to be able to
activate SSL on outgoing connections to the server at runtime without
reloading.
This patch adds a `set server ssl` command; in order to allow that:
- add `srv_use_ssl` to `show servers state` command for compatibility,
also update associated parsing
- when using default-server ssl setting, and `no-ssl` on server line,
init SSL ctx without activating it
- when triggering ssl API, de/activate SSL connections as requested
- clean ongoing connections as it is done for addr/port changes, without
checking prior server state
example config:
backend be_foo
default-server ssl
server srv0 127.0.0.1:6011 weight 1 no-ssl
show servers state:
5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - -1
where srv0 can switch to ssl later during the runtime:
set server be_foo/srv0 ssl on
5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - 1
Also update existing tests and create a new one.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
There is a typo on the ssl_c_chain_der sample fetch
(s/ssl_c_der_chain/ssl_c_chain_der/). This implies a move of the fetch to keep
it at the right place.
This should be backported as far as 2.2 or anywhere the commit a598b500b
("MINOR: ssl: add ssl_{c,s}_chain_der fetch methods") is.
This patch implements a couple of converters to validate and extract data from a
MQTT (Message Queuing Telemetry Transport) message. The validation consists of a
few checks as well as "packet size" validation. The extraction can get any field
from the variable header and the payload.
This is limited to CONNECT and CONNACK packet types only. All other messages are
considered as invalid. It is not a problem for now because only the first packet
on each side can be parsed (CONNECT for the client and CONNACK for the server).
MQTT 3.1.1 and 5.0 are supported.
Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
This patch implements a couple of converters to validate and extract tag value
from a FIX (Financial Information eXchange) message. The validation consists in
a few checks such as mandatory fields and checksum computation. The extraction
can get any tag value based on a tag string or tag id.
This patch requires the istend() function. Thus it depends on "MINOR: ist: Add
istend() function to return a pointer to the end of the string".
Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
Released version 2.4-dev0 with the following main changes :
- MINOR: version: it's development again.
- DOC: mention in INSTALL that it's development again
Released version 2.3.0 with the following main changes :
- CLEANUP: pattern: remove unused entry "tree" in pattern.val
- BUILD: ssl: use SSL_CTRL_GET_RAW_CIPHERLIST instead of OpenSSL versions
- BUG/MEDIUM: filters: Don't try to init filters for disabled proxies
- BUG/MINOR: proxy/server: Skip per-proxy/server post-check for disabled proxies
- BUG/MINOR: checks: Report a socket error before any connection attempt
- BUG/MINOR: server: Set server without addr but with dns in RMAINT on startup
- MINOR: server: Copy configuration file and line for server templates
- BUG/MEDIUM: mux-pt: Release the tasklet during an HTTP upgrade
- BUILD: ssl: use HAVE_OPENSSL_KEYLOG instead of OpenSSL versions
- MINOR: debug: don't count free(NULL) in memstats
- BUG/MINOR: filters: Skip disabled proxies during startup only
- MINOR: mux_h2: capitalize frame type in stats
- MINOR: mux_h2: add stat for total count of connections/streams
- MINOR: stats: do not display empty stat module title on html
- BUG/MEDIUM: stick-table: limit the time spent purging old entries
- BUG/MEDIUM: listener: only enable a listening listener if needed
- BUG/MEDIUM: listener: never suspend inherited sockets
- BUG/MEDIUM: listener: make the master also keep workers' inherited FDs
- MINOR: fd: add fd_want_recv_safe()
- MEDIUM: listeners: make use of fd_want_recv_safe() to enable early receivers
- REGTESTS: mark abns_socket as working now
- CLEANUP: mux-h2: Remove the h1 parser state from the h2 stream
- MINOR: sock: add a check against cross worker<->master socket activities
- CI: github actions: limit OpenSSL no-deprecated builds to "default,bug,devel" reg-tests
- BUG/MEDIUM: server: make it possible to kill last idle connections
- MINOR: mworker/cli: the master CLI use its own applet
- MINOR: ssl: define SSL_CTX_set1_curves_list to itself on BoringSSL
- BUILD: ssl: use feature macros for detecting ec curves manipulation support
- DOC: Add dns as an available domain to show stat
- BUILD: makefile: usual reorder of objects for faster builds
- DOC: update INSTALL to mention that TCC is supported
- DOC: mention in INSTALL that haproxy 2.3 is a stable version
- MINOR: version: mention that it's stable now
Within management.txt, proxy was listed as the only available option. "dns"
is now supported so let's add that. This change also updates the command to list
the available options <dns|proxy> for "domain" as previously it only specified
<domain>, which could be confusing as a user may think this field accepts
dynamic options when it actually requires a specific keyword.
Released version 2.3-dev9 with the following main changes :
- CLEANUP: http_ana: remove unused assignation of `att_beg`
- BUG/MEDIUM: ssl: OCSP must work with BoringSSL
- BUG/MINOR: log: fix memory leak on logsrv parse error
- BUG/MINOR: log: fix risk of null deref on error path
- BUILD: ssl: more elegant OpenSSL early data support check
- CI: github actions: update h2spec to 2.6.0
- BUG/MINOR: cache: Check the return value of http_replace_res_status
- MINOR: cache: Store the "Last-Modified" date in the cache_entry
- MINOR: cache: Process the If-Modified-Since header in conditional requests
- MINOR: cache: Create res.cache_hit and res.cache_name sample fetches
- MINOR: mux-h2: register a stats module
- MINOR: mux-h2: add counters instance to h2c
- MINOR: mux-h2: add stats for received frame types
- MINOR: mux-h2: report detected error on stats
- MINOR: mux-h2: count open connections/streams on stats
- BUG/MINOR: server: fix srv downtime calcul on starting
- BUG/MINOR: server: fix down_time report for stats
- BUG/MINOR: lua: initialize sample before using it
- MINOR: cache: Add Expires header value parsing
- MINOR: ist: Add a case insensitive istmatch function
- BUG/MINOR: cache: Manage multiple values in cache-control header value
- BUG/MINOR: cache: Inverted variables in http_calc_maxage function
- MINOR: pattern: make pat_ref_append() return the newly added element
- MINOR: pattern: make pat_ref_add() rely on pat_ref_append()
- MINOR: pattern: export pat_ref_push()
- CLEANUP: pattern: use calloc() rather than malloc for structures
- CLEANUP: pattern: fix spelling/grammatical/copy-paste in comments
Res.cache_hit sample fetch returns a boolean which is true when the HTTP
response was built out of a cache. The cache's name is returned by the
res.cache_name sample_fetch.
This resolves GitHub issue #900.
Released version 2.3-dev8 with the following main changes :
- MINOR: backend: replace the lbprm lock with an rwlock
- MINOR: lb/map: use seek lock and read locks where appropriate
- MINOR: lb/leastconn: only take a read lock in fwlc_get_next_server()
- MINOR: lb/first: use a read lock in fas_get_next_server()
- MINOR: lb/chash: use a read lock in chash_get_server_hash()
- BUG/MINOR: disable dynamic OCSP load with BoringSSL
- BUILD: ssl: make BoringSSL use its own version numbers
- CLEANUP: threads: don't register an initcall when not debugging
- MINOR: threads: change lock_t to an unsigned int
- CLEANUP: tree-wide: reorder a few structures to plug some holes around locks
- CLEANUP: task: remove the unused and mishandled global_rqueue_size
- BUG/MEDIUM: connection: Never cleanup server lists when freeing private conns
- MEDIUM: config: report that "nbproc" is deprecated
- BUG/MINOR: listener: close before free in `listener_accept`
- MINOR: ssl: 'ssl-load-extra-del-ext' removes the certificate extension
- BUG/MINOR: queue: properly report redistributed connections
- CONTRIB: tcploop: remove unused local variables in tcp_pause()
- BUILD: makefile: add entries to build common debugging tools
- BUG/MEDIUM: server: support changing the slowstart value from state-file
- MINOR: http: Add `enum etag_type http_get_etag_type(const struct ist)`
- MINOR: http: Add etag comparison function
- MEDIUM: cache: Store the ETag information in the cache_entry
- MEDIUM: cache: Add support for 'If-None-Match' request header
- REGTEST: cache: Add if-none-match test case
- CLEANUP: compression: Make use of http_get_etag_type()
- BUG/MINOR: http-ana: Don't send payload for internal responses to HEAD requests
- BUG/MAJOR: mux-h2: Don't try to send data if we know it is no longer possible
- MINOR: threads/debug: only report used lock stats
- MINOR: threads/debug: only report lock stats for used operations
- MINOR: proxy; replace the spinlock with an rwlock
- MINOR: server: read-lock the cookie during srv_set_dyncookie()
- MINOR: proxy/cli: only take a read lock in "show errors"
- OPTIM: queue: don't call pendconn_unlink() when the pendconn is not queued
- MINOR: queue: split __pendconn_unlink() in per-srv and per-prx
- MINOR: queue: reduce the locked area in pendconn_add()
- OPTIM: queue: make the nbpend counters atomic
- OPTIM: queue: decrement the nbpend and totpend counters outside of the lock
- MINOR: leastconn: take the queue length into account when queuing servers
- MEDIUM: fwlc: re-enable per-server queuing up to maxqueue
- Revert "OPTIM: queue: don't call pendconn_unlink() when the pendconn is not queued"
- MINOR: stats: support the "up" output modifier for "show stat"
- MINOR: stats: also support a "no-maint" show stat modifier
- MINOR: stats: indicate the number of servers in a backend's status
- MEDIUM: ssl: ssl-load-extra-del-ext work only with .crt
- REGTEST: ssl: test "set ssl cert" with separate key / crt
- DOC: management: apply the "show stat" modifiers to "show stat", not "show info"
- MINOR: stats: report server's user-configured weight next to effective weight
- CI: travis-ci: switch to Ubuntu 20.04
- CONTRIB: release-estimator: Add release estimating tool
- BUG/MEDIUM: queue: fix unsafe proxy pointer when counting nbpend
- BUG/MINOR: extcheck: add missing checks on extchk_setenv()
The "weight" column on the stats page is somewhat confusing when using
slowstart becaue it reports the effective weight, without being really
explicit about it. In some situations the user-configured weight is more
relevant (especially with long slowstarts where it's important to know
if the configured weight is correct).
This adds a new uweight stat which reports a server's user-configured
weight, and in a backend it receives the sum of all servers' uweights.
In addition it adds the mention of "effective" in a few descriptions
for the "weight" column (help and doc).
As a result, the list of servers in a backend is now always scanned
when dumping the stats. But this is not a problem given that these
servers are already scanned anyway and for way heavier processing.
By mistake I added the "up" then "maint" output modifiers to the "show info"
block instead of the "show stat" one in the two previous commits 65141ffc4
("MINOR: stats: support the "up" output modifier for "show stat"") and
3e3203670 ("MINOR: stats: also support a "no-maint" show stat modifier").
No backport is needed.
In order to be compatible with the "set ssl cert" command of the CLI,
this patch restrict the ssl-load-extra-del-ext to files with a ".crt"
extension in the configuration.
Related to issue #785.
Should be backported where 8e8581e ("MINOR: ssl: 'ssl-load-extra-del-ext'
removes the certificate extension") was backported.
"no-maint" is a bit similar to "up" except that it will only hide
servers that are in maintenance (or disabled in the configuration), and
not those that are enabled but failed a check. One benefit here is to
significantly reduce the output of the "show stat" command when using
large server-templates containing entries that are not yet provisioned.
Note that the prometheus exporter also has such an option which does
the exact same.
We already had it on the HTTP interface but it was not accessible on the
CLI. It can be very convenient to hide servers which are down, do not
resolve, or are in maintenance.
Leastconn has the nice propery of being able to sort servers by their
current usage. It's really a shame to force all requests into the backend
queue when the algo would be able to also consider their current queue.
In order not to change existing behavior but extend it, this patch allows
leastconn to elect servers which are already full if they have an explicitly
configured maxqueue setting above zero and their queue hasn't reached that
threshold. This will significantly reduce the pressure in the backend queue
when queuing a lot with lots of servers.
A test on 8 threads with 100 servers configured with maxconn 1 jumped
from 165krps to 330krps with maxqueue 15 with this patch.
This partially undoes commit 82cd5c13a ("OPTIM: backend: skip LB when we
know the backend is full") but allows to scale much better even by setting
a single-digit maxqueue value. Some better heuristics could be used to
maintain the behavior of the bypass in the patch above, consisting in
keeping it if it's known that there is no server with a configured
maxqueue in the farm (or in the backend).
When servers are queued into the leastconn tree, it's important to also
consider their queue length. There could be some servers with lots of
queued requests that we don't want to hammer with extra connections. In
order not to add extra stress to the LB algorithm, we don't update the
value when adding to the queue, only when updating the connection count
(i.e. picking from the queue or releasing a connection). This will be
sufficient to significantly improve the fairness in such situations.
In issue #785, users are reporting that it's not convenient to load a
".crt.key" when the configuration contains a ".crt".
This option allows to remove the extension of the certificate before
trying to load any extra SSL file (.key, .ocsp, .sctl, .issuer etc.)
The patch changes a little bit the way ssl_sock_load_files_into_ckch()
looks for the file.
As previously discussed, nbproc usage is bad, deprecated, and scheduled
for removal in 2.5.
If "nbproc" is found with more than one process while nbthread is not
set, a warning will be emitted encouraging to remove it or to migrate
to nbthread instead. This makes sure the user has an opportunity to
both see the message and silence it.
Released version 2.3-dev7 with the following main changes :
- CI: travis-ci: replace not defined SSL_LIB, SSL_INC for BotringSSL builds
- BUG/MINOR: init: only keep rlim_fd_cur if max is unlimited
- BUG/MINOR: mux-h2: do not stop outgoing connections on stopping
- MINOR: fd: report an error message when failing initial allocations
- MINOR: proto-tcp: make use of connect(AF_UNSPEC) for the pause
- MINOR: sock: add sock_accept_conn() to test a listening socket
- MINOR: protocol: make proto_tcp & proto_uxst report listening sockets
- MINOR: sockpair: implement the .rx_listening function
- CLEANUP: tcp: make use of sock_accept_conn() where relevant
- CLEANUP: unix: make use of sock_accept_conn() where relevant
- BUG/MINOR: listener: detect and handle shared sockets stopped in other processes
- CONTRIB: tcploop: implement a disconnect operation 'D'
- CLEANUP: protocol: intitialize all of the sockaddr when disconnecting
- BUG/MEDIUM: deinit: check fdtab before fdtab[fd].owner
- BUG/MINOR: connection: fix loop iter on connection takeover
- BUG/MEDIUM: connection: fix srv idle count on conn takeover
- MINOR: connection: improve list api usage
- MINOR: mux/connection: add a new mux flag for HOL risk
- MINOR: connection: don't check priv flag on free
- MEDIUM: backend: add new conn to session if mux marked as HOL blocking
- MEDIUM: backend: add reused conn to sess if mux marked as HOL blocking
- MEDIUM: h2: remove conn from session on detach
- MEDIUM: fcgi: remove conn from session on detach
- DOC: Describe reuse safe for HOL handling
- MEDIUM: proxy: remove obsolete "mode health"
- MEDIUM: proxy: remove obsolete "monitor-net"
- CLEANUP: protocol: remove the ->drain() function
- CLEANUP: fd: finally get rid of fd_done_recv()
- MINOR: connection: make sockaddr_alloc() take the address to be copied
- MEDIUM: listener: allocate the connection before queuing a new connection
- MINOR: session: simplify error path in session_accept_fd()
- MINOR: connection: add new error codes for accept_conn()
- MINOR: sock: rename sock_accept_conn() to sock_accepting_conn()
- MINOR: protocol: add a new function accept_conn()
- MINOR: sock: implement sock_accept_conn() to accept a connection
- MINOR: sockpair: implement sockpair_accept_conn() to accept a connection
- MEDIUM: listener: use protocol->accept_conn() to accept a connection
- MEDIUM: listener: remove the second pass of fd manipulation at the end
- MINOR: protocol: add a default I/O callback and put it into the receiver
- MINOR: log: set the UDP receiver's I/O handler in the receiver
- MINOR: protocol: register the receiver's I/O handler and not the protocol's
- CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind()
- DOC: improve the documentation for "option nolinger"
- BUG/MEDIUM: proxy: properly stop backends
- BUG/MEDIUM: task: bound the number of tasks picked from the wait queue at once
- MINOR: threads: augment rwlock debugging stats to report seek lock stats
- MINOR: threads: add the transitions to/from the seek state
- MEDIUM: task: use an upgradable seek lock when scanning the wait queue
- BUILD: listener: avoir a build warning when threads are disabled
- BUG/MINOR: peers: Possible unexpected peer seesion reset after collisions.
- MINOR: ssl: add volatile flags to ssl samples
- MEDIUM: backend: reuse connection if using a static sni
- BUG/MEDIUM: spoe: Unset variable instead of set it if no data provided
- BUG/MEDIUM: mux-h1: Get the session from the H1S when capturing bad messages
- BUG/MEDIUM: lb: Always lock the server when calling server_{take,drop}_conn
- DOC: fix typo in MAX_SESS_STKCTR
Detect if the sni used a constant value and if so, allow to reuse this
connection for later sessions. Use a combination of SMP_USE_INTRN +
!SMP_F_VOLATILE to consider a sample as a constant value.
This features has been requested on github issue #371.
fd_done_recv() used to be useful with the FD cache because it used to
allow to keep a file descriptor active in the poller without being
marked as ready in the cache, saving it from ringing immediately,
without incurring any system call. It was a way to make it yield
to wait for new events leaving a bit of time for others. The only
user left was the connection accepter (listen_accept()). We used
to suspect that with the FD cache removal it had become totally
useless since changing its readiness or not wouldn't change its
status regarding the poller itself, which would be the only one
deciding to report it again.
Careful tests showed that it indeed has exactly zero effect nowadays,
the syscall numbers are exactly the same with and without, including
when enabling edge-triggered polling.
Given that there's no more API available to manipulate it and that it
was directly called as an optimization from listener_accept(), it's
about time to remove it.
As discussed here during 2.1-dev, "monitor-net" is totally obsolete:
https://www.mail-archive.com/haproxy@formilux.org/msg35204.html
It's fundamentally incompatible with usage of SSL, and imposes the
presence of file descriptors with hard-coded syscalls directly in the
generic accept path.
It's very unlikely that anyone has used it in the last 10 years for
anything beyond testing. In the worst case if anyone would depend
on it, replacing it with "http-request return status 200 if ..." and
"mode http" would certainly do the trick.
The keyword is still detected as special by the config parser to help
users update their configurations appropriately.
As discussed here during 2.1-dev, "mode health" is totally obsolete:
https://www.mail-archive.com/haproxy@formilux.org/msg35204.html
It's fundamentally incompatible with usage of SSL, doesn't support
source filtering, and imposes the presence of file descriptors with
hard-coded syscalls directly in the generic accept path.
It's very unlikely that anyone has used it in the last 10 years for
anything beyond testing. In the worst case if anyone would depend
on it, replacing it with "http-request return status 200" and "mode
http" would certainly do the trick.
The keyword is still detected as special by the config parser to help
users update their configurations appropriately.
Released version 2.3-dev6 with the following main changes :
- REGTESTS: use "command" instead of "which" for better POSIX compatibility
- BUILD: makefile: Update feature flags for OpenBSD
- DOC: agent-check: fix typo in "fail" word expected reply
- DOC: crt: advise to move away from cert bundle
- BUG/MINOR: ssl/crt-list: exit on warning out of crtlist_parse_line()
- REGTEST: fix host part in balance-uri-path-only.vtc
- REGTEST: make ssl_client_samples and ssl_server_samples requiret to 2.3
- REGTEST: the iif converter test requires 2.3
- REGTEST: make agent-check.vtc require 1.8
- REGTEST: make abns_socket.vtc require 1.8
- REGTEST: make map_regm_with_backref require 1.7
- BUILD: makefile: Update feature flags for FreeBSD
- OPTIM: backend/random: never queue on the server, always on the backend
- OPTIM: backend: skip LB when we know the backend is full
- BUILD: makefile: Fix building with closefrom() support enabled
- BUILD: makefile: add an EXTRAVERSION variable to ease local naming
- MINOR: tools: support for word expansion of environment in parse_line
- BUILD: tools: fix minor build issue on isspace()
- BUILD: makefile: Enable closefrom() support on Solaris
- CLEANUP: ssl: Use structured format for error line report during crt-list parsing
- MINOR: ssl: Add error if a crt-list might be truncated
- MINOR: ssl: remove uneeded check in crtlist_parse_file
- BUG/MINOR: Fix several leaks of 'log_tag' in init().
- DOC: tcp-rules: Refresh details about L7 matching for tcp-request content rules
- MEDIUM: tcp-rules: Warn if a track-sc* content rule doesn't depend on content
- BUG/MINOR: tcpcheck: Set socks4 and send-proxy flags before the connect call
- DOC: ssl: new "cert bundle" behavior
- BUG/MEDIUM: queue: make pendconn_cond_unlink() really thread-safe
- CLEANUP: ssl: "bundle" is not an OpenSSL wording
- MINOR: counters: fix a typo in comment
- BUG/MINOR: stats: fix validity of the json schema
- REORG: stats: export some functions
- MINOR: stats: add stats size as a parameter for csv/json dump
- MINOR: stats: hide px/sv/li fields in applet struct
- REORG: stats: extract proxy json dump
- REORG: stats: extract proxies dump loop in a function
- MINOR: hlua: Display debug messages on stderr only in debug mode
- MINOR: stats: define the concept of domain for statistics
- MINOR: stats: define additional flag px cap on domain
- MEDIUM: stats: add delimiter for static proxy stats on csv
- MEDIUM: stats: define an API to register stat modules
- MEDIUM: stats: add abstract type to store counters
- MEDIUM: stats: integrate static proxies stats in new stats
- MINOR: stats: support clear counters for dynamic stats
- MINOR: stats: display extra proxy stats on the html page
- MINOR: stats: add config "stats show modules"
- MINOR: dns/stats: integrate dns counters in stats
- MINOR: stats: remove for loop declaration
- DOC: ssl: fix typo about ocsp files
- BUG/MINOR: peers: Inconsistency when dumping peer status codes.
- DOC: update INSTALL with supported OpenBSD / FreeBSD versions
- BUG/MINOR: proto_tcp: Report warning messages when listeners are bound
- CLEANUP: cache: Fix leak of cconf->c.name during config check
- CLEANUP: ssl: Release cached SSL sessions on deinit
- BUG/MINOR: mux-h1: Be sure to only set CO_RFL_READ_ONCE for the first read
- BUG/MINOR: mux-h1: Always set the session on frontend h1 stream
- MINOR: mux-h1: Don't wakeup the H1C when output buffer become available
- CLEANUP: sock-unix: Remove an unreachable goto clause
- BUG/MINOR: proxy: inc req counter on new syslog messages.
- BUG/MEDIUM: log: old processes with log foward section don't die on soft stop.
- MINOR: stats: inc req counter on listeners.
- MINOR: channel: new getword and getchar functions on channel.
- MEDIUM: log: syslog TCP support on log forward section.
- BUG/MINOR: proxy/log: frontend/backend and log forward names must differ
- DOC: re-work log forward bind statement documentation.
- DOC: fix a confusing typo on a regsub example
- BUILD: Add a DragonFlyBSD target
- BUG/MINOR: makefile: fix a tiny typo in the target list
- BUILD: makefile: Update feature flags for NetBSD
- CI: travis-ci: help Coverity to detect BUG_ON() as a real stop
- DOC: Add missing stats fields in the management doc
- BUG/MEDIUM: mux-fcgi: Don't handle pending read0 too early on streams
- BUG/MEDIUM: mux-h2: Don't handle pending read0 too early on streams
- DOC: Fix typos in configuration.txt
- BUG/MINOR: http: Fix content-length of the default 500 error
- BUG/MINOR: http-htx: Expect no body for 204/304 internal HTTP responses
- REGTESTS: mark abns_socket as broken
- MEDIUM: fd: always wake up one thread when enabling a foreing FD
- MEDIUM: listeners: don't bounce listeners management between queues
- MEDIUM: init: stop disabled proxies after initializing fdtab
- MEDIUM: listeners: make unbind_listener() converge if needed
- MEDIUM: deinit: close all receivers/listeners before scanning proxies
- MEDIUM: listeners: remove the now unused ZOMBIE state
- MINOR: listeners: do not uselessly try to close zombie listeners in soft_stop()
- CLEANUP: proxy: remove the first_to_listen hack in zombify_proxy()
- MINOR: listeners: introduce listener_set_state()
- MINOR: proxy: maintain per-state counters of listeners
- MEDIUM: proxy: remove the unused PR_STFULL state
- MEDIUM: proxy: remove the PR_STERROR state
- MEDIUM: proxy: remove state PR_STPAUSED
- MINOR: startup: don't rely on PR_STNEW to check for listeners
- CLEANUP: peers: don't use the PR_ST* states to mark enabled/disabled
- MEDIUM: proxy: replace proxy->state with proxy->disabled
- MEDIUM: proxy: remove start_proxies()
- MEDIUM: proxy: merge zombify_proxy() with stop_proxy()
- MINOR: listeners: check the current listener state in pause_listener()
- MINOR: listeners: check the current listener earlier state in resume_listener()
- MEDIUM: listener/proxy: make the listeners notify about proxy pause/resume
- MINOR: protocol: introduce protocol_{pause,resume}_all()
- MAJOR: signals: use protocol_pause_all() and protocol_resume_all()
- CLEANUP: proxy: remove the now unused pause_proxies() and resume_proxies()
- MEDIUM: proto_tcp: make the pause() more robust in multi-process
- BUG/MEDIUM: listeners: correctly report pause() errors
- MINOR: listeners: move fd_stop_recv() to the receiver's socket code
- CLEANUP: protocol: remove the ->disable_all method
- CLEANUP: listeners: remove unused disable_listener and disable_all_listeners
- MINOR: listeners: export enable_listener()
- MINOR: protocol: directly call enable_listener() from protocol_enable_all()
- CLEANUP: protocol: remove the ->enable_all method
- CLEANUP: listeners: remove the now unused enable_all_listeners()
- MINOR: protocol: rename the ->listeners field to ->receivers
- MINOR: protocol: replace ->pause(listener) with ->rx_suspend(receiver)
- MINOR: protocol: implement an ->rx_resume() method
- MINOR: listener: use the protocol's ->rx_resume() method when available
- MINOR: sock: provide a set of generic enable/disable functions
- MINOR: protocol: add a new pair of rx_enable/rx_disable methods
- MINOR: protocol: add a new pair of enable/disable methods for listeners
- MEDIUM: listeners: now use the listener's ->enable/disable
- MINOR: listeners: split delete_listener() in two versions
- MINOR: listeners: count unstoppable jobs on creation, not deletion
- MINOR: listeners: add a new stop_listener() function
- MEDIUM: proxy: make stop_proxy() now use stop_listener()
- MEDIUM: proxy: add mode PR_MODE_PEERS to flag peers frontends
- MEDIUM: proxy: centralize proxy status update and reporting
- MINOR: protocol: add protocol_stop_now() to instant-stop listeners
- MEDIUM: proxy: make soft_stop() stop most listeners using protocol_stop_now()
- MEDIUM: udp: implement udp_suspend() and udp_resume()
- MINOR: listener: add a few BUG_ON() statements to detect inconsistencies
- MEDIUM: listeners: always close master vs worker listeners
- BROKEN/MEDIUM: listeners: rework the unbind logic to make it idempotent
- MEDIUM: listener: let do_unbind_listener() decide whether to close or not
- CLEANUP: listeners: remove the do_close argument to unbind_listener()
- MINOR: listeners: move the LI_O_MWORKER flag to the receiver
- MEDIUM: receivers: add an rx_unbind() method in the protocols
- MINOR: listeners: split do_unbind_listener() in two
- MEDIUM: listeners: implement protocol level ->suspend/resume() calls
- MEDIUM: config: mark "grace" as deprecated
- MEDIUM: config: remove the deprecated and dangerous global "debug" directive
- BUG/MINOR: proxy: respect the proper format string in sig_pause/sig_listen
- MINOR: peers: heartbeat, collisions and handshake information for "show peers" command.
- BUILD: makefile: Enable getaddrinfo() on OS/X
This one was scheduled for removal in 2.3 since 2.2-dev3 by commit
1b85785bc ("MINOR: config: mark global.debug as deprecated"). Let's
remove it now. It remains totally possible to use -d on the command
line though.
This was introduced 15 years ago or so to delay the stopping of some
services so that a monitoring device could detect its port being down
before services were stopped. Since then, clean reloads were implemented
and this doesn't cope well with reload at all, preventing the new process
from seamlessly binding, and forcing processes to coexist with half-baked
configurations.
Now it has become a real problem because there's a significant code
portion in the proxies that is solely dedicated to this obsolete feature,
and dealing with its special cases eases the introduction of bugs in
other places so it's about time that it goes.
We could tentatively schedule its removal for 2.4 with a hard deadline
for 2.5 in any case.
The zombie state is not used anymore by the listeners, because in the
last two cases where it was tested it couldn't match as it was covered
by the test on the process mask. Instead now the FD is either in the
LISTEN state or the INIT state. This also avoids forcing the listener
to be single-dimensional because actually belonging to another process
isn't totally exclusive with the other states, which explains some of
the difficulties requiring to check the proc_mask and the fd sometimes.
So let's get rid of it now not to be tempted to reuse it.
The doc on the listeners state was updated.
Sbastien reported a confusing example in the doc about regsub when used
with quotes. Nested quotes are already not trivial to grasp, but when
typos are there and result in something valid, it's even worse. The closing
quote ought to have been inside the brackets. However haproxy will not make
any difference because the single quotes delimit a word and the delimited
word remains the same. Let's just not add yet another level of confusion.
This patch re-introduce the "bind" statement on log forward
sections to handle syslog TCP listeners as defined in
rfc-6587.
As complement it introduce "maxconn", "backlog" and "timeout
client" statements to parameter those listeners.
one of the last I saw in this section while working on github issue #872
might be backported in all still supported versions
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
Use the new stats module API to integrate the dns counters in the
standard stats. This is done in order to avoid code duplication, keep
the code related to cli out of dns and use the full possibility of the
stats function, allowing to print dns stats in csv or json format.
Use the character '-' to mark the end of static statistics on proxy
domain. After this marker, the order of the fields is not guaranteed and
should be parsed with care.
The domain option will be used to have statistics attached to other
objects than proxies/listeners/servers. At the moment, only the PROXY
domain is available.
Add an argument 'domain' on the 'show stats' cli command to specify the
domain. Only 'domain proxy' is available now. If not specified, proxy
will be considered the default domain.
For HTML output, only proxy statistics will be displayed.
Update the documentation with the new bundle behavior which does not use
the same OpenSSL certificate store anymore but loads the PEM separately
as multiple "crt" were specified.
It should fix issue #872.
The warning is only emitted for HTTP frontend. Idea is to encourage the usage of
"tcp-request session" rules to track counters that does not depend on the
request content. The documentation has been updated accordingly.
The warning is important because since the multiplexers were added in the
processing chain, the HTTP parsing is performed at a lower level. Thus parsing
errors are detected in the multiplexers, before the stream creation. In HTTP/2,
the error is reported by the multiplexer itself and the stream is never
created. This difference has a certain number of consequences, one of which is
that HTTP request counting in stick tables only works for valid H2 request, and
HTTP error tracking in stick tables never considers invalid H2 requests but only
invalid H1 ones. And the aim is to do the same with the mux-h1. This change will
not be done for the 2.3, but the 2.4. At the end, H1 and H2 parsing errors will
be caught by the multiplexers, at the session level. Thus, tracking counters at
the content level should be reserved for rules using a key based on the request
content or those using ACLs based on the request content.
To be clear, a warning will be emitted for the following rules :
tcp-request content track-sc0 src
tcp-request content track-sc0 src if ! { src 10.0.0.0/24 }
tcp-request content track-sc0 src if { ssl_fc }
But not for the following ones :
tcp-request content track-sc0 req.hdr(host)
tcp-request content track-sc0 src if { req.hdr(host) -m found }
Because the parsing of HTTP message is now performed in the HTTP multiplexers,
the content is immediatly available when "tcp-request content" rules are
evaluated for an HTTP frontend. So, it is a good idea to make the documentation
explicit on this point. In addition, because in all cases, the parsing is
already performed, there is no reason to still use "tcp-request content" rules
based on L7 matching, although it is still valid. The recommended way is to use
"http-request" rules instead. Again, it is a good idea to update the
documentation on this point.
Allow the syntax "${...[*]}" to expand an environment variable
containing several values separated by spaces as individual arguments. A
new flag PARSE_OPT_WORD_EXPAND has been added to toggle this feature on
parse_line invocation. In case of an invalid syntax, a new error
PARSE_ERR_WRONG_EXPAND will be triggered.
This feature has been asked on the github issue #165.
especially when starting to use `new ssl cert` runtime API, it might
become a bit confusing for users to mix bundle and single cert,
especially when it comes to use the commit command:
e.g.:
- start the process with `crt` loading a bundle
- use `set ssl cert my_cert.pem.ecdsa`: API detects it as a replacement
of a bundle.
- `commit` has to be done on the bundle: `commit ssl cert my_cert.pem`
however:
- add a new cert: `new ssl cert my_cert.pem.rsa`: added as a single
certificate
- `commit` has to be done on the certificate: `commit ssl cert
my_cert.pem.rsa`
this should resolve github issue #872
this should probably be backported in >= v2.2 in order to encourage
people to move away from bundle certificates loading.
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
`tcpcheck_agent_expect_reply` expects "fail" not "failed"
This should fix github issue #876
This can be backported to all maintained versions (i.e >= 1.6) as of
today.
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
Released version 2.3-dev5 with the following main changes :
- DOC: Fix typo in iif() example
- CLEANUP: Update .gitignore
- BUILD: introduce possibility to define ABORT_NOW() conditionally
- CI: travis-ci: help Coverity to recognize abort()
- BUG/MINOR: Fix type passed of sizeof() for calloc()
- CLEANUP: Do not use a fixed type for 'sizeof' in 'calloc'
- CLEANUP: tree-wide: use VAR_ARRAY instead of [0] in various definitions
- BUILD: connection: fix build on clang after the VAR_ARRAY cleanup
- BUG/MINOR: ssl: verifyhost is case sensitive
- BUILD: makefile: change default value of CC from gcc to cc
- CI: travis-ci: split asan step out of running tests
- BUG/MINOR: server: report correct error message for invalid port on "socks4"
- BUG/MEDIUM: ssl: Don't call ssl_sock_io_cb() directly.
- BUG/MINOR: ssl/crt-list: crt-list could end without a \n
- BUG/MINOR: log-forward: fail on unknown keywords
- MEDIUM: log-forward: use "dgram-bind" instead of "bind" for the listener
- BUG/MEDIUM: log-forward: always quit on parsing errors
- MEDIUM: ssl: remove bundle support in crt-list and directories
- MEDIUM: ssl/cli: remove support for multi certificates bundle
- MINOR: ssl: crtlist_dup_ssl_conf() duplicates a ssl_bind_conf
- MINOR: ssl: crtlist_entry_dup() duplicates a crtlist_entry
- MEDIUM: ssl: emulates the multi-cert bundles in the crtlist
- MEDIUM: ssl: emulate multi-cert bundles loading in standard loading
- CLEANUP: ssl: remove test on "multi" variable in ckch functions
- CLEANUP: ssl/cli: remove test on 'multi' variable in CLI functions
- CLEANUP: ssl: remove utility functions for bundle
- DOC: explain bundle emulation in configuration.txt
- BUILD: fix build with openssl < 1.0.2 since bundle removal
- BUG/MINOR: log: gracefully handle the "udp@" address format for log servers
- BUG/MINOR: dns: gracefully handle the "udp@" address format for nameservers
- MINOR: listener: create a new struct "settings" in bind_conf
- MINOR: listener: move bind_proc and bind_thread to struct settings
- MINOR: listener: move the interface to the struct settings
- MINOR: listener: move the network namespace to the struct settings
- REORG: listener: create a new struct receiver
- REORG: listener: move the listening address to a struct receiver
- REORG: listener: move the receiving FD to struct receiver
- REORG: listener: move the listener's proto to the receiver
- MINOR: listener: make sock_find_compatible_fd() check the socket type
- REORG: listener: move the receiver part to a new file
- MINOR: receiver: link the receiver to its settings
- MINOR: receiver: link the receiver to its owner
- MINOR: listener: prefer to retrieve the socket's settings via the receiver
- MINOR: receiver: add a receiver-specific flag to indicate the socket is bound
- MINOR: listener: move the INHERITED flag down to the receiver
- MINOR: receiver: move the FOREIGN and V6ONLY options from listener to settings
- MINOR: sock: make sock_find_compatible_fd() only take a receiver
- MINOR: protocol: rename the ->bind field to ->listen
- MINOR: protocol: add a new ->bind() entry to bind the receiver
- MEDIUM: sock_inet: implement sock_inet_bind_receiver()
- MEDIUM: tcp: make use of sock_inet_bind_receiver()
- MEDIUM: udp: make use of sock_inet_bind_receiver()
- MEDIUM: sock_unix: implement sock_unix_bind_receiver()
- MEDIUM: uxst: make use of sock_unix_bind_receiver()
- MEDIUM: sockpair: implement sockpair_bind_receiver()
- MEDIUM: proto_sockpair: make use of sockpair_bind_receiver()
- MEDIUM: protocol: explicitly start the receiver before the listener
- MEDIUM: protocol: do not call proto->bind() anymore from bind_listener()
- MINOR: protocol: add a new proto_fam structure for protocol families
- MINOR: protocol: retrieve the family-specific fields from the family
- CLEANUP: protocol: remove family-specific fields from struct protocol
- MINOR: protocol: add a real family for existing FDs
- CLEANUP: tools: make str2sa_range() less awful for fd@ and sockpair@
- MINOR: tools: make str2sa_range() take more options than just resolve
- MINOR: tools: add several PA_O_PORT_* flags in str2sa_range() callers
- MEDIUM: tools: make str2sa_range() validate callers' port specifications
- MEDIUM: config: remove all checks for missing/invalid ports/ranges
- MINOR: tools: add several PA_O_* flags in str2sa_range() callers
- MINOR: listener: remove the inherited arg to create_listener()
- MINOR: tools: make str2sa_range() optionally return the fd
- MINOR: log: detect LOG_TARGET_FD from the fd and not from the syntax
- MEDIUM: tools: make str2sa_range() resolve pre-bound listeners
- MINOR: config: do not test an inherited socket again
- MEDIUM: tools: make str2sa_range() check for the sockpair's FD usability
- MINOR: tools: start to distinguish stream and dgram in str2sa_range()
- MEDIUM: tools: make str2sa_range() only report AF_CUST_UDP on listeners
- MINOR: tools: remove the central test for "udp" in str2sa_range()
- MINOR: cfgparse: add str2receiver() to parse dgram receivers
- MINOR: log-forward: use str2receiver() to parse the dgram-bind address
- MEDIUM: config: make str2listener() not accept datagram sockets anymore
- MINOR: listener: pass the chosen protocol to create_listeners()
- MINOR: tools: make str2sa_range() directly return the protocol
- MEDIUM: tools: make str2sa_range() check that the protocol has ->connect()
- MINOR: protocol: add the control layer type in the protocol struct
- MEDIUM: protocol: store the socket and control type in the protocol array
- MEDIUM: tools: make str2sa_range() use protocol_lookup()
- MEDIUM: proto_udp: replace last AF_CUST_UDP* with AF_INET*
- MINOR: tools: drop listener detection hack from str2sa_range()
- BUILD: sock_unix: add missing errno.h
- MINOR: sock_inet: report the errno string in binding errors
- MINOR: sock_unix: report the errno string in binding errors
- BUILD: sock_inet: include errno.h
- MINOR: h2/trace: also display the remaining frame length in traces
- BUG/MINOR: h2/trace: do not display "stream error" after a frame ACK
- BUG/MEDIUM: h2: report frame bits only for handled types
- BUG/MINOR: http-fetch: Don't set the sample type during the htx prefetch
- BUG/MINOR: Fix memory leaks cfg_parse_peers
- BUG/MINOR: config: Fix memory leak on config parse listen
- MINOR: backend: make the "whole" option of balance uri take only one bit
- MINOR: backend: add a new "path-only" option to "balance uri"
- REGTESTS: add a few load balancing tests
- BUG/MEDIUM: listeners: do not pause foreign listeners
- BUG/MINOR: listeners: properly close listener FDs
- BUILD: trace: include tools.h
Since we've fixed the way URIs are handled in 2.1, some users have started
to experience inconsistencies in "balance uri" between requests received
over H1 and the same ones received over H2. This is caused by the fact
that H1 rarely uses absolute URIs while H2 always uses them. Similar
issues were reported already around replace-uri etc, leading to "pathq"
recently being introduced, so this isn't new.
Here what this patch does is add a new option to "balance uri" to indicate
that the hashing should only start at the path and not cover the authority.
This makes H1 relative URIs and H2 absolute URI hashes equally again.
Some extra options could be added to normalize URIs by always hashing the
authority (or host) in front of them, which would make sure that both
absolute and relative requests provide the same hash. This is left for
later if needed.
Thanks to this we don't need to specify "udp@" as it's implicitly a
datagram type listener that is expected, so any AF_INET/AF_INET4 address
will work.
The use of "bind" wasn't that wise but was temporary. The problem is that
it will not allow to coexist with tcp. Let's explicitly call it "dgram-bind"
so that datagram listeners are expected here, leaving some room for stream
listeners later. This is the only change.
Released version 2.3-dev4 with the following main changes :
- MINOR: hlua: Add error message relative to the Channel manipulation and HTTP mode
- BUG/MEDIUM: ssl: crt-list negative filters don't work
- DOC: overhauling github issue templates
- MEDIUM: cfgparse: Emit hard error on truncated lines
- DOC: cache: Use '<name>' instead of '<id>' in error message
- MINOR: cache: Reject duplicate cache names
- REGTEST: remove stray leading spaces in converteers_ref_cnt_never_dec.vtc
- MINOR: stats: prevent favicon.ico requests for stats page
- BUILD: tools: include auxv a bit later
- BUILD: task: work around a bogus warning in gcc 4.7/4.8 at -O1
- MEDIUM: ssl: Support certificate chaining for certificate generation
- MINOR: ssl: Support SAN extension for certificate generation
- MINOR: tcp: don't try to set/clear v6only on inherited sockets
- BUG/MINOR: reload: detect the OS's v6only status before choosing an old socket
- MINOR: reload: determine the foreing binding status from the socket
- MEDIUM: reload: stop passing listener options along with FDs
- BUG/MEDIUM: ssl: fix ssl_bind_conf double free w/ wildcards
- MEDIUM: fd: replace usages of fd_remove() with fd_stop_both()
- CLEANUP: fd: remove fd_remove() and rename fd_dodelete() to fd_delete()
- MINOR: fd: add a new "exported" flag and use it for all regular listeners
- MEDIUM: reload: pass all exportable FDs, not just listeners
- DOC: add description of pidfile in master-worker mode
- BUG/MINOR: reload: do not fail when no socket is sent
- REORG: tcp: move TCP actions from proto_tcp.c to tcp_act.c
- CLEANUP: tcp: stop exporting smp_fetch_src()
- REORG: tcp: move TCP sample fetches from proto_tcp.c to tcp_sample.c
- REORG: tcp: move TCP bind/server keywords from proto_tcp.c to cfgparse-tcp.c
- REORG: unix: move UNIX bind/server keywords from proto_uxst.c to cfgparse-unix.c
- REORG: sock: start to move some generic socket code to sock.c
- MINOR: sock: introduce sock_inet and sock_unix
- MINOR: tcp/udp/unix: make use of proto->addrcmp() to compare addresses
- MINOR: sock_inet: implement sock_inet_get_dst()
- REORG: inet: replace tcp_is_foreign() with sock_inet_is_foreign()
- REORG: sock_inet: move v6only_default from proto_tcp.c to sock_inet.c
- REORG: sock_inet: move default_tcp_maxseg from proto_tcp.c
- REORG: listener: move xfer_sock_list to sock.{c,h}.
- MINOR: sock: add interface and namespace length to xfer_sock_list
- MINOR: sock: implement sock_find_compatible_fd()
- MINOR: sock_inet: move the IPv4/v6 transparent mode code to sock_inet
- REORG: sock: move get_old_sockets() from haproxy.c
- MINOR: sock: do not use LI_O_* in xfer_sock_list anymore
- MINOR: sock: distinguish dgram from stream types when retrieving old sockets
- BUILD: sock_unix: fix build issue with isdigit()
- BUG/MEDIUM: http-ana: Don't wait to send 1xx responses received from servers
- MINOR: http-htx: Add an option to eval query-string when the path is replaced
- BUG/MINOR: http-rules: Replace path and query-string in "replace-path" action
- MINOR: http-htx: Handle an optional reason when replacing the response status
- MINOR: contrib/spoa-server: allow MAX_FRAME_SIZE override
- BUG/MAJOR: contrib/spoa-server: Fix unhandled python call leading to memory leak
- BUG/MINOR: contrib/spoa-server: Ensure ip address references are freed
- BUG/MINOR: contrib/spoa-server: Do not free reference to NULL
- BUG/MINOR: contrib/spoa-server: Updating references to free in case of failure
- BUG/MEDIUM: contrib/spoa-server: Fix ipv4_address used instead of ipv6_address
- CLEANUP: http: silence a cppcheck warning in get_http_auth()
- REGTEST: increase some short timeouts to make tests more reliable
- BUG/MINOR: threads: work around a libgcc_s issue with chrooting
- BUILD: thread: limit the libgcc_s workaround to glibc only
- MINOR: protocol: do not call proto->bind_all() anymore
- MINOR: protocol: do not call proto->unbind_all() anymore
- CLEANUP: protocol: remove all ->bind_all() and ->unbind_all() functions
- MAJOR: init: start all listeners via protocols and not via proxies anymore
- BUG/MINOR: startup: haproxy -s cause 100% cpu
- Revert "BUG/MINOR: http-rules: Replace path and query-string in "replace-path" action"
- BUG/MEDIUM: doc: Fix replace-path action description
- MINOR: http-rules: Add set-pathq and replace-pathq actions
- MINOR: http-fetch: Add pathq sample fetch
- REGTEST: Add a test for request path manipulations, with and without the QS
- MINOR: Commit .gitattributes
- CLEANUP: Update .gitignore
- BUG/MEDIUM: dns: Don't store additional records in a linked-list
- BUG/MEDIUM: dns: Be sure to renew IP address for already known servers
- MINOR: server: Improve log message sent when server address is updated
- DOC: ssl-load-extra-files only applies to certificates on bind lines
- BUG/MINOR: auth: report valid crypto(3) support depending on build options
- BUG/MEDIUM: mux-h1: always apply the timeout on half-closed connections
- BUILD: threads: better workaround for late loading of libgcc_s
- BUILD: compiler: reserve the gcc version checks to the gcc compiler
- BUILD: compiler: workaround a glibc madness around __attribute__()
- BUILD: intops: on x86_64, the bswap instruction is called bswapq
- BUILD: trace: always have an argument before variadic args in macros
- BUILD: traces: don't pass an empty argument for missing ones
- BUG/MINOR: haproxy: Free uri_auth->scope during deinit
- CLEANUP: Free old_argv on deinit
- CLEANUP: haproxy: Free post_proxy_check_list in deinit()
- CLEANUP: haproxy: Free per_thread_*_list in deinit()
- CLEANUP: haproxy: Free post_check_list in deinit()
- BUG/MEDIUM: pattern: Renew the pattern expression revision when it is pruned
- REORG: tools: move PARSE_OPT_* from tools.h to tools-t.h
- MINOR: sample: Add iif(<true>,<false>) converter
iif() takes a boolean as input and returns one of the two argument
strings depending on whether the boolean is true.
This converter most likely is most useful to return the proper scheme
depending on the value returned by the `ssl_fc` fetch, e.g. for use within
the `x-forwarded-proto` request header.
However it can also be useful for use within a template that is sent to
the client using `http-request return` with a `lf-file`. It allows the
administrator to implement a simple condition, without needing to prefill
variables within the regular configuration using `http-request
set-var(req.foo)`.
The pathq sample fetch extract the relative URI of a request, i.e the path with
the query-string, excluding the scheme and the authority, if any. It is pretty
handy to always get a relative URI independently on the HTTP version. Indeed,
while relative URIs are common in HTTP/1.1, in HTTP/2, most of time clients use
absolute URIs.
This patch may be backported to 2.2.
These actions do the same as corresponding "-path" versions except the
query-string is included to the manipulated request path. This means set-pathq
action replaces the path and the query-string and replace-pathq action matches
and replace the path including the query-string.
This patch may be backported to 2.2.
The description of the replace-path action does not reflect what the code
do. When the request path is replaced, the query-string is preserved. But the
documentation stated the query-string is part of the replacement, if any is
present. Most of time, when the doc and the code differ, the code is fixed. But
here, the replace-path action is pretty confusing because the set-path action is
only applied on the path. The query-string is left intact. And the path sample
fetch also ignores the query-string. In addition, the replace-path action is
quite recent. It was added in the 2.2. Thus, exceptionally, the documentation is
fixed instead.
Note that set-pathq and replace-pathq actions and pathq sample fetch will be
added to manipulate the path with the query-string.
This patch must be backported as far as 2.0.
Previously, pidfile was only described for daemon mode. In the case of
master-worker mode, the handling of pidfile is different from daemon mode,
so the description has been added.
Released version 2.3-dev3 with the following main changes :
- SCRIPTS: git-show-backports: make -m most only show the left branch
- SCRIPTS: git-show-backports: emit the shell command to backport a commit
- BUILD: Makefile: require SSL_LIB, SSL_INC to be explicitly set
- CI: travis-ci: specify SLZ_LIB, SLZ_INC for travis builds
- BUG/MEDIUM: mux-h1: Refresh H1 connection timeout after a synchronous send
- CLEANUP: dns: typo in reported error message
- BUG/MAJOR: dns: disabled servers through SRV records never recover
- BUG/MINOR: spoa-server: fix size_t format printing
- DOC: spoa-server: fix false friends `actually`
- BUG/MINOR: ssl: fix memory leak at OCSP loading
- BUG/MEDIUM: ssl: memory leak of ocsp data at SSL_CTX_free()
- BUG/MEDIUM: map/lua: Return an error if a map is loaded during runtime
- MINOR: arg: Add an argument type to keep a reference on opaque data
- BUG/MINOR: converters: Store the sink in an arg pointer for debug() converter
- BUG/MINOR: lua: Duplicate map name to load it when a new Map object is created
- BUG/MINOR: arg: Fix leaks during arguments validation for fetches/converters
- BUG/MINOR: lua: Check argument type to convert it to IPv4/IPv6 arg validation
- BUG/MINOR: lua: Check argument type to convert it to IP mask in arg validation
- MINOR: hlua: Don't needlessly copy lua strings in trash during args validation
- BUG/MINOR: lua: Duplicate lua strings in sample fetches/converters arg array
- MEDIUM: lua: Don't filter exported fetches and converters
- MINOR: lua: Add support for userlist as fetches and converters arguments
- MINOR: lua: Add support for regex as fetches and converters arguments
- MINOR: arg: Use chunk_destroy() to release string arguments
- BUG/MINOR: snapshots: leak of snapshots on deinit()
- CLEANUP: ssl: ssl_sock_crt2der semicolon and spaces
- MINOR: ssl: add ssl_{c,s}_chain_der fetch methods
- CLEANUP: fix all duplicated semicolons
- BUG/MEDIUM: ssl: fix the ssl-skip-self-issued-ca option
- BUG/MINOR: ssl: ssl-skip-self-issued-ca requires >= 1.0.2
- BUG/MINOR: stats: use strncmp() instead of memcmp() on health states
- BUILD: makefile: don't disable -Wstringop-overflow anymore
- BUG/MINOR: ssl: double free w/ smp_fetch_ssl_x_chain_der()
- BUG/MEDIUM: htx: smp_prefetch_htx() must always validate the direction
- BUG/MEDIUM: ssl: never generates the chain from the verify store
- OPTIM: regex: PCRE2 use JIT match when JIT optimisation occured.
- BUG/MEDIUM: ssl: does not look for all SNIs before chosing a certificate
- CLEANUP: ssl: remove poorly readable nested ternary
Following work from Arjen and Mathilde, it adds ssl_{c,s}_chain_der
methods; it returns DER encoded certs from SSL_get_peer_cert_chain
Also update existing vtc tests to add random intermediate certificates
When getting the result through this header:
http-response add-header x-ssl-chain-der %[ssl_c_chain_der,hex]
One can parse it with any lib accepting ASN.1 DER data, such as in go:
bin, err := encoding/hex.DecodeString(cert)
certs_parsed, err := x509.ParseCertificates(bin)
Cc: Arjen Nienhuis <arjen@zorgdoc.nl>
Signed-off-by: Mathilde Gilles <m.gilles@criteo.com>
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
Released version 2.3-dev2 with the following main changes :
- DOC: ssl: req_ssl_sni needs implicit TLS
- BUG/MEDIUM: arg: empty args list must be dropped
- BUG/MEDIUM: resolve: fix init resolving for ring and peers section.
- BUG/MAJOR: tasks: don't requeue global tasks into the local queue
- MINOR: tasks/debug: make the thread affinity BUG_ON check a bit stricter
- MINOR: tasks/debug: add a few BUG_ON() to detect use of wrong timer queue
- MINOR: tasks/debug: add a BUG_ON() check to detect requeued task on free
- BUG/MAJOR: dns: Make the do-resolve action thread-safe
- BUG/MEDIUM: dns: Release answer items when a DNS resolution is freed
- MEDIUM: htx: Add a flag on a HTX message when no more data are expected
- BUG/MEDIUM: stream-int: Don't set MSG_MORE flag if no more data are expected
- BUG/MEDIUM: http-ana: Only set CF_EXPECT_MORE flag on data filtering
- CLEANUP: dns: remove 45 "return" statements from dns_validate_dns_response()
- BUG/MINOR: htx: add two missing HTX_FL_EOI and remove an unexpected one
- BUG/MINOR: mux-fcgi: Don't url-decode the QUERY_STRING parameter anymore
- BUILD: tools: fix build with static only toolchains
- DOC: Use gender neutral language
- BUG/MINOR: debug: Don't dump the lua stack if it is not initialized
- BUG/MAJOR: dns: fix null pointer dereference in snr_update_srv_status
- BUG/MAJOR: dns: don't treat Authority records as an error
- CI : travis-ci : prepare for using stock OpenSSL
- CI: travis-ci : switch to stock openssl when openssl-1.1.1 is used
- MEDIUM: lua: Add support for the Lua 5.4
- BUG/MEDIUM: dns: Don't yield in do-resolve action on a final evaluation
- BUG/MINOR: lua: Abort execution of actions that yield on a final evaluation
- MINOR: tcp-rules: Return an internal error if an action yields on a final eval
- BUG/MINOR: tcp-rules: Preserve the right filter analyser on content eval abort
- BUG/MINOR: tcp-rules: Set the inspect-delay when a tcp-response action yields
- MEDIUM: tcp-rules: Use a dedicated expiration date for tcp ruleset
- MEDIUM: lua: Set the analyse expiration date with smaller wake_time only
- BUG/MEDIUM: connection: Be sure to always install a mux for sync connect
- MINOR: connection: Preinstall the mux for non-ssl connect
- MINOR: stream-int: Be sure to have a mux to do sends and receives
- BUG/MINOR: lua: Fix a possible null pointer deref on lua ctx
- SCRIPTS: announce-release: add the link to the wiki in the announce messages
- CI: travis-ci: use better name for Coverity scan job
- CI: travis-ci: use proper linking flags for SLZ build
- BUG/MEDIUM: backend: always attach the transport before installing the mux
- BUG/MEDIUM: tcp-checks: always attach the transport before installing the mux
- MINOR: connection: avoid a useless recvfrom() on outgoing connections
- MINOR: mux-h1: do not even try to receive if the connection is not fully set up
- MINOR: mux-h1: do not try to receive on backend before sending a request
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MEDIUM: ssl: check OCSP calloc in ssl_sock_load_ocsp()
req_ssl_sni is not compatible with protocols negotiating TLS
explicitly, like SMTP on port 25 or 587 and IMAP on port 143.
Fix an example referring to 587 (SMTPS port with implicit TLS
is 465) and amend the req_ssl_sni documentation.
This doc fix should be backported to supported versions.
Released version 2.3-dev1 with the following main changes :
- MINOR: config: make strict limits enabled by default
- BUG/MINOR: acl: Fix freeing of expr->smp in prune_acl_expr
- BUG/MINOR: sample: Fix freeing of conv_exprs in release_sample_expr
- BUG/MINOR: haproxy: Free proxy->format_unique_id during deinit
- BUG/MINOR: haproxy: Add missing free of server->(hostname|resolvers_id)
- BUG/MINOR: haproxy: Free proxy->unique_id_header during deinit
- BUG/MINOR: haproxy: Free srule->file during deinit
- BUG/MINOR: haproxy: Free srule->expr during deinit
- BUG/MINOR: sample: Free str.area in smp_check_const_bool
- BUG/MINOR: sample: Free str.area in smp_check_const_meth
- CLEANUP: haproxy: Free proxy_deinit_list in deinit()
- CLEANUP: haproxy: Free post_deinit_list in deinit()
- CLEANUP: haproxy: Free server_deinit_list in deinit()
- CLEANUP: haproxy: Free post_server_check_list in deinit()
- CLEANUP: Add static void vars_deinit()
- CLEANUP: Add static void hlua_deinit()
- CLEANUP: contrib/prometheus-exporter: typo fixes for ssl reuse metric
- BUG/MEDIUM: lists: add missing store barrier on MT_LIST_BEHEAD()
- BUG/MEDIUM: lists: add missing store barrier in MT_LIST_ADD/MT_LIST_ADDQ
- MINOR: tcp: Support TCP keepalive parameters customization
- BUILD: tcp: condition TCP keepalive settings to platforms providing them
- MINOR: lists: rename some MT_LIST operations to clarify them
- MINOR: buffer: use MT_LIST_ADDQ() for buffer_wait lists additions
- MINOR: connection: use MT_LIST_ADDQ() to add connections to idle lists
- MINOR: tasks: use MT_LIST_ADDQ() when killing tasks.
- CONTRIB: da: fix memory leak in dummy function da_atlas_open()
- CI: travis-ci: speed up osx build by running brew scripted, switch to latest osx image
- BUG/MEDIUM: mux-h2: Don't add private connections in available connection list
- BUG/MEDIUM: mux-fcgi: Don't add private connections in available connection list
- MINOR: connection: Set the SNI on server connections before installing the mux
- MINOR: connection: Set new connection as private on reuse never
- MINOR: connection: Add a wrapper to mark a connection as private
- MEDIUM: connection: Add private connections synchronously in session server list
- MINOR: connection: Use a dedicated function to look for a session's connection
- MINOR: connection: Set the conncetion target during its initialisation
- MINOR: session: Take care to decrement idle_conns counter in session_unown_conn
- MINOR: server: Factorize code to deal with reuse of server idle connections
- MINOR: server: Factorize code to deal with connections removed from an idle list
- CLEANUP: connection: remove unused field idle_time from the connection struct
- BUG/MEDIUM: mux-h1: Continue to process request when switching in tunnel mode
- MINOR: raw_sock: Report the number of bytes emitted using the splicing
- MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics
- MINOR: backend: Add sample fetches to get the server's weight
- BUG/MINOR: mux-fcgi: Handle empty STDERR record
- BUG/MINOR: mux-fcgi: Set conn state to RECORD_P when skipping the record padding
- BUG/MINOR: mux-fcgi: Set flags on the right stream field for empty FCGI_STDOUT
- BUG/MINOR: backend: fix potential null deref on srv_conn
- BUG/MEDIUM: log: issue mixing sampled to not sampled log servers.
- MEDIUM: udp: adds minimal proto udp support for message listeners.
- MEDIUM: log/sink: re-work and merge of build message API.
- MINOR: log: adds syslog udp message handler and parsing.
- MEDIUM: log: adds log forwarding section.
- MINOR: log: adds counters on received syslog messages.
- BUG/MEDIUM: fcgi-app: fix memory leak in fcgi_flt_http_headers
- BUG/MEDIUM: server: resolve state file handle leak on reload
- BUG/MEDIUM: server: fix possibly uninitialized state file on close
- BUG/MEDIUM: channel: Be aware of SHUTW_NOW flag when output data are peeked
- BUILD: config: address build warning on raspbian+rpi4
- BUG/MAJOR: tasks: make sure to always lock the shared wait queue if needed
- BUILD: config: fix again bugs gcc warnings on calloc
Log forwarding:
It is possible to declare one or multiple log forwarding section,
haproxy will forward all received log messages to a log servers list.
log-forward <name>
Creates a new log forwarder proxy identified as <name>.
bind <addr> [param*]
Used to configure a log udp listener to receive messages to forward.
Only udp listeners are allowed, address must be prefixed using
'udp@', 'udp4@' or 'udp6@'. This supports for all "bind" parameters
found in 5.1 paragraph but most of them are irrelevant for udp/syslog case.
log global
log <address> [len <length>] [format <format>] [sample <ranges>:<smp_size>]
<facility> [<level> [<minlevel>]]
Used to configure target log servers. See more details on proxies
documentation.
If no format specified, haproxy tries to keep the incoming log format.
Configured facility is ignored, except if incoming message does not
present a facility but one is mandatory on the outgoing format.
If there is no timestamp available in the input format, but the field
exists in output format, haproxy will use the local date.
Example:
global
log stderr format iso local7
ring myring
description "My local buffer"
format rfc5424
maxlen 1200
size 32764
timeout connect 5s
timeout server 10s
# syslog tcp server
server mysyslogsrv 127.0.0.1:514 log-proto octet-count
log-forward sylog-loadb
bind udp4@127.0.0.1:1514
# all messages on stderr
log global
# all messages on local tcp syslog server
log ring@myring local0
# load balance messages on 4 udp syslog servers
log 127.0.0.1:10001 sample 1:4 local0
log 127.0.0.1:10002 sample 2:4 local0
log 127.0.0.1:10003 sample 3:4 local0
log 127.0.0.1:10004 sample 4:4 local0
This patch merges build message code between sink and log
and introduce a new API based on struct ist array to
prepare message header with zero copy, targeting the
log forwarding feature.
Log format 'iso' and 'timed' are now avalaible on logs line.
A new log format 'priority' is also added.
This patch introduce proto_udp.c targeting a further support of
log forwarding feature.
This code was originally produced by Frederic Lecaille working on
QUIC support and only minimal requirements for syslog support
have been merged.
The following sample fetches have been added :
* srv_iweight : returns the initial server's weight
* srv_uweight : returns the user-visible server's weight
* srv_weight : returns the current (or effetctive) server's weight
The requested server must be passed as argument, evnetually preceded by the
backend name. For instance :
srv_weight(back-http/www1)
Previous commit b24bc0d ("MINOR: tcp: Support TCP keepalive parameters
customization") broke non-Linux builds as TCP_KEEP{CNT,IDLE,INTVL} are
not necessarily defined elsewhere.
This patch adds the required #ifdefs to condition the visibility of the
keywords, and adds a mention in the doc about their dependency on Linux.
It is now possible to customize TCP keepalive parameters.
These correspond to the socket options TCP_KEEPCNT, TCP_KEEPIDLE, TCP_KEEPINTVL
and are valid for the defaults, listen, frontend and backend sections.
This patch fixes GitHub issue #670.
Released version 2.2.0 with the following main changes :
- BUILD: mux-h2: fix typo breaking build when using DEBUG_LOCK
- CLEANUP: makefile: update the outdated list of DEBUG_xxx options
- BUILD: tools: make resolve_sym_name() return a const
- CLEANUP: auth: fix useless self-include of auth-t.h
- BUILD: tree-wide: cast arguments to tolower/toupper to unsigned char
- CLEANUP: assorted typo fixes in the code and comments
- WIP/MINOR: ssl: add sample fetches for keylog in frontend
- DOC: fix tune.ssl.keylog sample fetches array
- BUG/MINOR: ssl: check conn in keylog sample fetch
- DOC: configuration: various typo fixes
- MINOR: log: Remove unused case statement during the log-format string parsing
- BUG/MINOR: mux-h1: Fix the splicing in TUNNEL mode
- BUG/MINOR: mux-h1: Don't read data from a pipe if the mux is unable to receive
- BUG/MINOR: mux-h1: Disable splicing only if input data was processed
- BUG/MEDIUM: mux-h1: Disable splicing for the conn-stream if read0 is received
- MINOR: mux-h1: Improve traces about the splicing
- BUG/MINOR: backend: Remove CO_FL_SESS_IDLE if a client remains on the last server
- BUG/MEDIUM: connection: Don't consider new private connections as available
- BUG/MINOR: connection: See new connection as available only on reuse always
- DOC: configuration: remove obsolete mentions of H2 being converted to HTTP/1.x
- CLEANUP: ssl: remove unrelevant comment in smp_fetch_ssl_x_keylog()
- DOC: update INSTALL with new compiler versions
- DOC: minor update to coding style file
- MINOR: version: mention that it's an LTS release now
The first H2 implementation in version 1.8 used to turn HTTP/2 requests
to HTTP/1.1, causing many limitations. This is not true anymore and we
don't suffer from the lack of server-side H2 nor are we forced to close
mode anymore, so let's remove such obsolete mentions.
This could be backported to 2.0.
OpenSSL 1.1.1 provides a callback registering function
SSL_CTX_set_keylog_callback, which allows one to receive a string
containing the keys to deciphers TLSv1.3.
Unfortunately it is not possible to store this data in binary form and
we can only get this information using the callback. Which means that we
need to store it until the connection is closed.
This patches add 2 pools, the first one, pool_head_ssl_keylog is used to
store a struct ssl_keylog which will be inserted as a ex_data in a SSL *.
The second one is pool_head_ssl_keylog_str which will be used to store
the hexadecimal strings.
To enable the capture of the keys, you need to set "tune.ssl.keylog on"
in your configuration.
The following fetches were implemented:
ssl_fc_client_early_traffic_secret,
ssl_fc_client_handshake_traffic_secret,
ssl_fc_server_handshake_traffic_secret,
ssl_fc_client_traffic_secret_0,
ssl_fc_server_traffic_secret_0,
ssl_fc_exporter_secret,
ssl_fc_early_exporter_secret
Released version 2.2-dev12 with the following main changes :
- BUG/MINOR: mux_h2: don't lose the leaving trace in h2_io_cb()
- MINOR: cli: make "show sess" stop at the last known session
- CLEANUP: buffers: remove unused buffer_wq_lock lock
- BUG/MEDIUM: buffers: always allocate from the local cache first
- MINOR: connection: align toremove_{lock,connections} and cleanup into idle_conns
- CONTRIB: debug: add missing flags SI_FL_L7_RETRY & SI_FL_D_L7_RETRY
- BUG/MEDIUM: connections: Don't increase curr_used_conns for shared connections.
- BUG/MEDIUM: checks: Increment the server's curr_used_conns
- REORG: buffer: rename buffer.c to dynbuf.c
- REORG: includes: create tinfo.h for the thread_info struct
- CLEANUP: pool: only include the type files from types
- MINOR: pools: move the LRU cache heads to thread_info
- BUG/MINOR: debug: fix "show fd" null-deref when built with DEBUG_FD
- MINOR: stats: add 3 new output values for the per-server idle conn state
- MINOR: activity: add per-thread statistics on FD takeover
- BUG/MINOR: server: start cleaning idle connections from various points
- MEDIUM: server: improve estimate of the need for idle connections
- MINOR: stats: add the estimated need of concurrent connections per server
- BUG/MINOR: threads: Don't forget to init each thread toremove_lock.
- BUG/MEDIUM: lists: Lock the element while we check if it is in a list.
- Revert "BUG/MEDIUM: lists: Lock the element while we check if it is in a list."
- BUG/MINOR: haproxy: don't wake already stopping threads on exit
- BUG/MINOR: server: always count one idle slot for current thread
- MEDIUM: server: use the two thresholds for the connection release algorithm
- BUG/MINOR: http-rules: Fix ACLs parsing for http deny rules
- BUG/MINOR: sched: properly cover for a rare MT_LIST_ADDQ() race
- MINOR: mux-h1: avoid taking the toremove_lock in on dying tasks
- MINOR: mux-h2: avoid taking the toremove_lock in on dying tasks
- MINOR: mux-fcgi: avoid taking the toremove_lock in on dying tasks
- MINOR: pools: increase MAX_BASE_POOLS to 64
- DOC: ssl: add "allow-0rtt" and "ciphersuites" in crt-list
- BUG/MEDIUM: pattern: Add a trailing \0 to match strings only if possible
- BUG/MEDIUM: log-format: fix possible endless loop in parse_logformat_string()
- BUG/MINOR: proxy: fix dump_server_state()'s misuse of the trash
- BUG/MINOR: proxy: always initialize the trash in show servers state
- MINOR: cli/proxy: add a new "show servers conn" command
- MINOR: server: skip servers with no idle conns earlier
- BUG/MINOR: server: fix the connection release logic regarding nearly full conditions
- MEDIUM: server: add a new pool-low-conn server setting
- BUG/MEDIUM: backend: always search in the safe list after failing on the idle one
- MINOR: backend: don't always takeover from the same threads
- MINOR: sched: make sched->task_list_size atomic
- MEDIUM: sched: create a new TASK_KILLED task flag
- MEDIUM: sched: implement task_kill() to kill a task
- MEDIUM: mux-h1: use task_kill() during h1_takeover() instead of task_wakeup()
- MEDIUM: mux-h2: use task_kill() during h2_takeover() instead of task_wakeup()
- MEDIUM: mux-fcgi: use task_kill() during fcgi_takeover() instead of task_wakeup()
- MINOR: list: Add MT_LIST_DEL_SAFE_NOINIT() and MT_LIST_ADDQ_NOCHECK()
- CLEANUP: connections: rename the toremove_lock to takeover_lock
- MEDIUM: connections: Don't use a lock when moving connections to remove.
- DOC: configuration: add missing index entries for tune.pool-{low,high}-fd-ratio
- DOC: configuration: fix alphabetical ordering for tune.pool-{high,low}-fd-ratio
- MINOR: config: add a new tune.idle-pool.shared global setting.
- MINOR: 51d: silence a warning about null pointer dereference
- MINOR: debug: add a new "debug dev memstats" command
- MINOR: log-format: allow to preserve spacing in log format strings
- BUILD: debug: avoid build warnings with DEBUG_MEM_STATS
- BUG/MAJOR: sched: make sure task_kill() always queues the task
- BUG/MEDIUM: muxes: Make sure nobody stole the connection before using it.
- BUG/MEDIUM: cli/proxy: don't try to dump idle connection state if there's none
- BUILD: haproxy: fix build error when RLIMIT_AS is not set
- BUG/MAJOR: sched: make it work also when not building with DEBUG_STRICT
- MINOR: log: add time second fraction field to rfc5424 log timestamp.
- BUG/MINOR: log: missing timezone on iso dates.
- BUG/MEDIUM: server: don't kill all idle conns when there are not enough
- MINOR: sched: split tasklet_wakeup() into tasklet_wakeup_on()
- BUG/MEDIUM: connections: Set the tid for the old tasklet on takeover.
- BUG/MEDIUM: connections: Let the xprt layer know a takeover happened.
- BUG/MINOR: http_act: don't check capture id in backend (2)
- BUILD: makefile: disable threads by default on OpenBSD
- BUILD: peers: fix build warning with gcc 4.2.1
- CI: cirrus-ci: exclude slow reg-tests
Now it's possible to preserve spacing everywhere except in "log-format",
"log-format-sd" and "unique-id-format" directives, where spaces are
delimiters and are merged. That may be useful when the response payload
is specified as a log format string by "lf-file" or "lf-string", or even
for headers or anything else.
In order to merge spaces, a new option LOG_OPT_MERGE_SPACES is applied
exclusively on options passed to function parse_logformat_string().
This patch fixes an issue #701 ("http-request return log-format file
evaluation altering spacing of ASCII output/art").
Enables ('on') or disables ('off') sharing of idle connection pools between
threads for a same server. The default is to share them between threads in
order to minimize the number of persistent connections to a server, and to
optimize the connection reuse rate. But to help with debugging or when
suspecting a bug in HAProxy around connection reuse, it can be convenient to
forcefully disable this idle pool sharing between multiple threads, and force
this option to "off". The default is on.
This could have been nice to have during the idle connections debugging,
but it's not too late to add it!
In addition they were in the wrong alphabetical order in the doc. They
were added in 2.0 by commit 88698d966 ("MEDIUM: connections: Add a way
to control the number of idling connections.") so this must be backported
to 2.0.
These two keywords didn't have an entry in the index. They were added in
2.0 by commit 88698d966 ("MEDIUM: connections: Add a way to control the
number of idling connections.") so this must be backported to 2.0.
The problem with the way idle connections currently work is that it's
easy for a thread to steal all of its siblings' connections, then release
them, then it's done by another one, etc. This happens even more easily
due to scheduling latencies, or merged events inside the same pool loop,
which, when dealing with a fast server responding in sub-millisecond
delays, can really result in one thread being fully at work at a time.
In such a case, we perform a huge amount of takeover() which consumes
CPU and requires quite some locking, sometimes resulting in lower
performance than expected.
In order to fight against this problem, this patch introduces a new server
setting "pool-low-conn", whose purpose is to dictate when it is allowed to
steal connections from a sibling. As long as the number of idle connections
remains at least as high as this value, it is permitted to take over another
connection. When the idle connection count becomes lower, a thread may only
use its own connections or create a new one. By proceeding like this even
with a low number (typically 2*nbthreads), we quickly end up in a situation
where all active threads have a few connections. It then becomes possible
to connect to a server without bothering other threads the vast majority
of the time, while still being able to use these connections when the
number of available FDs becomes low.
We also use this threshold instead of global.nbthread in the connection
release logic, allowing to keep more extra connections if needed.
A test performed with 10000 concurrent HTTP/1 connections, 16 threads
and 210 servers with 1 millisecond of server response time showed the
following numbers:
haproxy 2.1.7: 185000 requests per second
haproxy 2.2: 314000 requests per second
haproxy 2.2 lowconn 32: 352000 requests per second
The takeover rate goes down from 300k/s to 13k/s. The difference is
further amplified as the response time shrinks.
This command reuses the existing "show servers state" to also dump the
state of active and idle connections. The main use is to serve as a
debugging tool to troubleshot connection reuse issues.
"show sess" and particularly "show sess all" can be very slow when dumping
lots of information, and while dumping, new sessions might appear, making
the output really endless. When threads are used, this causes a double
problem:
- all threads are paused during the dump, so an overly long dump degrades
the quality of service ;
- since all threads are paused, more events get postponed, possibly
resulting in more streams to be dumped on next invocation of the dump
function.
This patch addresses this long-lasting issue by doing something simple:
the CLI's stream is moved at the end of the steams list, serving as an
identifiable marker to end the dump, because all entries past it were
added after the command was entered. As a result, the CLI's stream always
appears as the last one.
It may make sense to backport this to stable branches where dumping live
streams is difficult as well.
Released version 2.2-dev11 with the following main changes :
- REGTEST: Add a simple script to tests errorfile directives in proxy sections
- BUG/MEDIUM: fcgi-app: Resolve the sink if a fcgi-app logs in a ring buffer
- BUG/MINOR: spoe: correction of setting bits for analyzer
- BUG/MINOR: cfgparse: Support configurations without newline at EOF
- MINOR: cfgparse: Warn on truncated lines / files
- BUG/MINOR: http_ana: clarify connection pointer check on L7 retry
- MINOR: debug: add a new DEBUG_FD build option
- BUG/MINOR: tasks: make sure never to exceed max_processed
- MINOR: task: add a new pointer to current tasklet queue
- BUG/MEDIUM: task: be careful not to run too many tasks at TL_URGENT
- BUG/MINOR: cfgparse: Fix argument reference in PARSE_ERR_TOOMANY message
- BUG/MINOR: cfgparse: Fix calculation of position for PARSE_ERR_TOOMANY message
- BUG/MEDIUM: ssl: fix ssl_bind_conf double free
- MINOR: ssl: free bind_conf_node in crtlist_free()
- MINOR: ssl: free the crtlist and the ckch during the deinit()
- BUG/MINOR: ssl: fix build with ckch_deinit() and crtlist_deinit()
- BUG/MINOR: ssl/cli: certs added from the CLI can't be deleted
- MINOR: ssl: move the ckch/crtlist deinit to ssl_sock.c
- MEDIUM: tasks: apply a fair CPU distribution between tasklet classes
- MINOR: tasks: make current_queue an index instead of a pointer
- MINOR: tasks: add a mask of the queues with active tasklets
- MINOR: tasks: pass the queue index to run_task_from_list()
- MINOR: tasks: make run_tasks_from_lists() scan the queues itself
- MEDIUM: tasks: add a tune.sched.low-latency option
- BUG/MEDIUM: ssl/cli: 'commit ssl cert' crashes when no private key
- BUG/MINOR: cfgparse: don't increment linenum on incomplete lines
- MINOR: tools: make parse_line() always terminate the args list
- BUG/MINOR: cfgparse: report extraneous args *after* the string is allocated
- MINOR: cfgparse: sanitize the output a little bit
- MINOR: cli/ssl: handle trailing slashes in crt-list commands
- MINOR: ssl: add the ssl_s_* sample fetches for server side certificate
- BUG/MEDIUM: http-ana: Don't loop trying to generate a malformed 500 response
- BUG/MINOR: stream-int: Don't wait to send truncated HTTP messages
- BUG/MINOR: http-ana: Set CF_EOI on response channel for generated responses
- BUG/MINOR: http-ana: Don't wait to send 1xx responses generated by HAProxy
- MINOR: spoe: Don't systematically create new applets if processing rate is low
- DOC: fix some typos in the ssl_s_{s|i}_dn documentation
- BUILD: fix ssl_sample.c when building against BoringSSL
- CI: travis-ci: switch BoringSSL builds to ninja
- CI: extend spellchecker whitelist
- DOC: assorted typo fixes in the documentation
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: http: Add support for http 413 status
- REGTEST: ssl: tests the ssl_f_* sample fetches
- REGTEST: ssl: add some ssl_c_* sample fetches test
- DOC: ssl: update the documentation of "commit ssl cert"
- BUG/MINOR: cfgparse: correctly deal with empty lines
- BUG/MEDIUM: fetch: Fix hdr_ip misparsing IPv4 addresses due to missing NUL
This commit adds some sample fetches that were lacking on the server
side:
ssl_s_key_alg, ssl_s_notafter, ssl_s_notbefore, ssl_s_sig_alg,
ssl_s_i_dn, ssl_s_s_dn, ssl_s_serial, ssl_s_sha1, ssl_s_der,
ssl_s_version
Now that all tasklet queues are scanned at once by run_tasks_from_lists(),
it becomes possible to always check for lower priority classes and jump
back to them when they exist.
This patch adds tune.sched.low-latency global setting to enable this
behavior. What it does is stick to the lowest ranked priority list in
which tasks are still present with an available budget, and leave the
loop to refill the tasklet lists if the trees got new tasks or if new
work arrived into the shared urgent queue.
Doing so allows to cut the latency in half when running with extremely
deep run queues (10k-100k), thus allowing forwarding of small and large
objects to coexist better. It remains off by default since it does have
a small impact on large traffic by default (shorter batches).
Released version 2.2-dev10 with the following main changes :
- BUILD: include: add sys/types before netinet/tcp.h
- BUG/MEDIUM: log: don't hold the log lock during writev() on a file descriptor
- BUILD: Remove nowarn for warnings that do not trigger
- BUG/MEDIUM: pattern: fix thread safety of pattern matching
- BUILD: Re-enable -Wimplicit-fallthrough
- BUG/MINOR: ssl: fix ssl-{min,max}-ver with openssl < 1.1.0
- BUILD: thread: add parenthesis around values of locking macros
- BUILD: proto_uxst: shut up yet another gcc's absurd warning
- BUG/MEDIUM: checks: Fix off-by-one in allocation of SMTP greeting cmd
- CI: travis-ci: use "-O1" for clang builds
- MINOR: haproxy: Add void deinit_and_exit(int)
- MINOR: haproxy: Make use of deinit_and_exit() for clean exits
- BUG/MINOR: haproxy: Free rule->arg.vars.expr during deinit_act_rules
- BUILD: compression: make gcc 10 happy with free_zlib()
- BUILD: atomic: add string.h for memcpy() on ARM64
- BUG/MINOR: http: make smp_fetch_body() report that the contents may change
- BUG/MINOR: tcp-rules: tcp-response must check the buffer's fullness
- BUILD: haproxy: mark deinit_and_exit() as noreturn
- BUG/MAJOR: vars: Fix bogus free() during deinit() for http-request rules
- BUG/MEDIUM: ebtree: use a byte-per-byte memcmp() to compare memory blocks
- MINOR: tools: add a new configurable line parse, parse_line()
- BUG/MEDIUM: cfgparse: use parse_line() to expand/unquote/unescape config lines
- BUG/MEDIUM: cfgparse: stop after a reasonable amount of fatal error
- MINOR: http: do not close connections anymore after internal responses
- BUG/MINOR: cfgparse: Add missing fatal++ in PARSE_ERR_HEX case
- BUG/MINOR: spoe: add missing key length check before checking key names
- MINOR: version: put the compiler version output into version.c not haproxy.c
- MINOR: compiler: always define __has_feature()
- MINOR: version: report the presence of the compiler's address sanitizer
- BUILD: Fix build by including haproxy/global.h
- BUG/MAJOR: connection: always disable ready events once reported
- CLEANUP: activity: remove unused counter fd_lock
- DOC: fd: make it clear that some fields ordering must absolutely be respected
- MINOR: activity: report the number of times poll() reports I/O
- MINOR: activity: rename confusing poll_* fields in the output
- MINOR: fd: Fix a typo in a coment.
- BUG/MEDIUM: fd: Don't fd_stop_recv() a fd we don't own.
- BUG/MEDIUM: fd: Call fd_stop_recv() when we just got a fd.
- MINOR: activity: group the per-loop counters at the top
- MINOR: activity: rename the "stream" field to "stream_calls"
- MEDIUM: fd: refine the fd_takeover() migration lock
- MINOR: fd: slightly optimize the fd_takeover double-CAS loop
- MINOR: fd: factorize the fd_takeover() exit path to make it safer
- MINOR: peers: do not use localpeer as an array anymore
- MEDIUM: peers: add the "localpeer" global option
- MEDIUM: fd: add experimental support for edge-triggered polling
- CONTRIB: debug: add the missing flags CO_FL_SAFE_LIST and CO_FL_IDLE_LIST
- MINOR: haproxy: process signals before runnable tasks
- MEDIUM: tasks: clean up the front side of the wait queue in wake_expired_tasks()
- MEDIUM: tasks: also process late wakeups in process_runnable_tasks()
- BUG/MINOR: cli: allow space escaping on the CLI
- BUG/MINOR: mworker/cli: fix the escaping in the master CLI
- BUG/MINOR: mworker/cli: fix semicolon escaping in master CLI
- REGTEST: http-rules: test spaces in ACLs
- REGTEST: http-rules: test spaces in ACLs with master CLI
- BUG/MAJOR: init: properly compute the default global.maxpipes value
- MEDIUM: map: make the "clear map" operation yield
- BUG/MEDIUM: stream-int: fix loss of CO_SFL_MSG_MORE flag in forwarding
- MINOR: mux_h1: Set H1_F_CO_MSG_MORE if we know we have more to send.
- BUG/MINOR: systemd: Wait for network to be online
- DOC: configuration: Unindent non-code sentences in the protobuf example
- DOC: configuration: http-check send was missing from matrix
The new directive and its doc were added by commit 8acb1284b ("MINOR:
checks: Add a way to send custom headers and payload during http chekcs")
but the index was not updated.
Unindent to make the explanation go back to text from code formatted
example in tyhe HTMLized version. Still it's not perfect since these
are not haproxy examples but protobuf config, but... way better.
Some of the recent optimizations around the polling to save a few
epoll_ctl() calls have shown that they could also cause some trouble.
However, over time our code base has become totally asynchronous with
I/Os always attempted from the upper layers and only retried at the
bottom, making it look like we're getting closer to EPOLLET support.
There are showstoppers there such as the listeners which cannot support
this. But given that most of the epoll_ctl() dance comes from the
connections, we can try to enable edge-triggered polling on connections.
What this patch does is to add a new global tunable "tune.fd.edge-triggered",
that makes fd_insert() automatically set an et_possible bit on the fd if
the I/O callback is conn_fd_handler. When the epoll code sees an update
for such an FD, it immediately registers it in both directions the first
time and doesn't update it anymore.
On a few tests it proved quite useful with a 14% request rate increase in
a H2->H1 scenario, reducing the epoll_ctl() calls from 2 per request to
2 per connection.
The option is obviously disabled by default as bugs are still expected,
particularly around the subscribe() code where it is possible that some
layers do not always re-attempt reading data after being woken up.
localpeer <name>
Sets the local instance's peer name. It will be ignored if the "-L"
command line argument is specified or if used after "peers" section
definitions. In such cases, a warning message will be emitted during
the configuration parsing.
This option will also set the HAPROXY_LOCALPEER environment variable.
See also "-L" in the management guide and "peers" section in the
configuration manual.
Released version 2.2-dev9 with the following main changes :
- BUG/MINOR: http-htx: Don't forget to release the http reply in release function
- BUG/MINOR: http-htx: Fix a leak on error path during http reply parsing
- MINOR: checks: Remove dead code from process_chk_conn()
- REGTESTS: checks: Fix tls_health_checks when IPv6 addresses are used
- REGTESTS: Add missing OPENSSL to REQUIRE_OPTIONS for lua/txn_get_priv
- MINOR: lua: Use vars_unset_by_name_ifexist()
- CLEANUP: vars: Remove void vars_unset_by_name(const char*, size_t, struct sample*)
- MINOR: vars: Make vars_(un|)set_by_name(_ifexist|) return a success value
- MINOR: lua: Make `set_var()` and `unset_var()` return success
- MEDIUM: lua: Add `ifexist` parameter to `set_var`
- MEDIUM: ring: new section ring to declare custom ring buffers.
- REGTESTS: Add missing OPENSSL to REQUIRE_OPTIONS for compression/lua_validation
- REGTESTS: Require the version 2.2 to execute lua/set_var
- BUG/MEDIUM: checks: Refresh the conn-stream and the connection after a connect
- MINOR: checks: Remove useless tests on the connection and conn-stream
- BUG/MEDIUM: contrib/spoa: do not register python3.8 if --embed fail
- BUG/MEDIUM: connection: Ignore PP2 unique ID for stream-less connections
- BUG/MINOR: connection: Always get the stream when available to send PP2 line
- BUG/MEDIUM: backend: set the connection owner to the session when using alpn.
- MINOR: pools: compute an estimate of each pool's average needed objects
- MEDIUM: pools: directly free objects when pools are too much crowded
- REGTEST: Add connection/proxy_protocol_send_unique_id_alpn
- MINOR: http-ana: Make the function http_reply_to_htx() public
- MINOR: http-ana: Use proxy's error replies to emit 401/407 responses
- MINOR: http-rules: Use an action function to eval http-request auth rules
- CLEANUP: http: Remove unused HTTP message templates
- BUG/MEDIUM: checks: Don't blindly subscribe for receive if waiting for connect
- MINOR: checks: I/O callback function only rely on the data layer wake callback
- BUG/MINOR: lua: Add missing string length for lua sticktable lookup
- BUG/MEDIUM: logs: fix trailing zeros on log message.
- CI: cirrus-ci: skip reg-tests/connection/proxy_protocol_send_unique_id_alpn.vtc on CentOS 6
- BUG/MINOR: nameservers: fix error handling in parsing of resolv.conf
- BUG/MEDIUM: checks: Don't add a tcpcheck ruleset twice in the shared tree
- MEDIUM: ssl: use TLSv1.2 as the minimum default on bind lines
- CLEANUP: pools: use the regular lock for the flush operation on lockless pools
- SCRIPTS: publish-release: pass -n to gzip to remove timestamp
- MINOR: ring: re-work ring attach generic API.
- BUG/MINOR: error on unknown statement in ring section.
- MEDIUM: ring: add server statement to forward messages from a ring
- MEDIUM: ring: add new srv statement to support octet counting forward
- MINOR: ssl: set ssl-min-ver in ambiguous configurations
- CLEANUP: ssl: remove comment from dump_crtlist_sslconf()
- BUILD: sink: address build warning on 32-bit architectures
- BUG/MINOR: peers: fix internal/network key type mapping.
- CLEANUP: regex: remove outdated support for regex actions
- Revert "MINOR: ssl: rework add cert chain to CTX to be libssl independent"
- MINOR: mux-h1/proxy: Add a proxy option to disable clear h2 upgrade
- BUG/MEDIUM: lua: Reset analyse expiration timeout before executing a lua action
- DOC: add a line about comments in crt-list
- BUG/MEDIUM: hlua: Lock pattern references to perform set/add/del operations
- BUG/MINOR: checks: Fix test on http-check rulesets during config validity check
- BUG/MEDIUM: contrib/prometheus-exporter: Properly set flags to dump metrics
- BUG/MEDIUM: mworker: fix the copy of options in copy_argv()
- BUG/MINOR: init: -x can have a parameter starting with a dash
- BUG/MINOR: init: -S can have a parameter starting with a dash
- BUG/MEDIUM: mworker: fix the reload with an -- option
- BUG/MINOR: ssl: fix a trash buffer leak in some error cases
- BUG/MINOR: mworker: fix a memleak when execvp() failed
- MINOR: sample: Add secure_memcmp converter
- REORG: ebtree: move the C files from ebtree/ to src/
- REORG: ebtree: move the include files from ebtree to include/import/
- REORG: ebtree: clean up remains of the ebtree/ directory
- REORG: include: create new file haproxy/api-t.h
- REORG: include: create new file haproxy/api.h
- REORG: include: update all files to use haproxy/api.h or api-t.h if needed
- CLEANUP: include: remove common/config.h
- CLEANUP: include: remove unused template.h
- REORG: include: move MIN/MAX from tools.h to compat.h
- REORG: include: move SWAP/MID_RANGE/MAX_RANGE from tools.h to standard.h
- CLEANUP: include: remove unused common/tools.h
- REORG: include: move the base files from common/ to haproxy/
- REORG: include: move version.h to haproxy/
- REORG: include: move base64.h, errors.h and hash.h from common to to haproxy/
- REORG: include: move openssl-compat.h from common/ to haproxy/
- REORG: include: move ist.h from common/ to import/
- REORG: include: move the BUG_ON() code to haproxy/bug.h
- REORG: include: move debug.h from common/ to haproxy/
- CLEANUP: debug: drop unused function p_malloc()
- REORG: include: split buf.h into haproxy/buf-t.h and haproxy/buf.h
- REORG: include: move istbuf.h to haproxy/
- REORG: include: split mini-clist into haproxy/list and list-t.h
- REORG: threads: extract atomic ops from hathreads.h
- CLEANUP: threads: remove a few needless includes of hathreads.h
- REORG: include: split hathreads into haproxy/thread.h and haproxy/thread-t.h
- CLEANUP: thread: rename __decl_hathreads() to __decl_thread()
- REORG: include: move time.h from common/ to haproxy/
- REORG: include: move integer manipulation functions from standard.h to intops.h
- CLEANUP: include: remove excessive includes of common/standard.h
- REORG: include: move freq_ctr to haproxy/
- CLEANUP: pool: include freq_ctr.h and remove locally duplicated functions
- REORG: memory: move the pool type definitions to haproxy/pool-t.h
- REORG: memory: move the OS-level allocator to haproxy/pool-os.h
- MINOR: memory: don't let __pool_get_first() pick from the cache
- MEDIUM: memory: don't let pool_put_to_cache() free the objects itself
- MINOR: memory: move pool-specific path of the locked pool_free() to __pool_free()
- MEDIUM: memory: make local pools independent on lockless pools
- REORG: include: move common/memory.h to haproxy/pool.h
- REORG: include: move common/chunk.h to haproxy/chunk.h
- REORG: include: move activity to haproxy/
- REORG: include: move common/buffer.h to haproxy/dynbuf{,-t}.h
- REORG: include: move common/net_helper.h to haproxy/net_helper.h
- REORG: include: move common/namespace.h to haproxy/namespace{,-t}.h
- REORG: include: split common/regex.h into haproxy/regex{,-t}.h
- REORG: include: split common/xref.h into haproxy/xref{,-t}.h
- REORG: include: move common/ticks.h to haproxy/ticks.h
- REORG: include: split common/http.h into haproxy/http{,-t}.h
- REORG: include: split common/http-hdr.h into haproxy/http-hdr{,-t}.h
- REORG: include: move common/h1.h to haproxy/h1.h
- REORG: include: split common/htx.h into haproxy/htx{,-t}.h
- REORG: include: move hpack*.h to haproxy/ and split hpack-tbl
- REORG: include: move common/h2.h to haproxy/h2.h
- REORG: include: move common/fcgi.h to haproxy/
- REORG: include: move protocol.h to haproxy/protocol{,-t}.h
- REORG: tools: split common/standard.h into haproxy/tools{,-t}.h
- REORG: include: move dict.h to hparoxy/dict{,-t}.h
- REORG: include: move shctx to haproxy/shctx{,-t}.h
- REORG: include: move port_range.h to haproxy/port_range{,-t}.h
- REORG: include: move fd.h to haproxy/fd{,-t}.h
- REORG: include: move ring to haproxy/ring{,-t}.h
- REORG: include: move sink.h to haproxy/sink{,-t}.h
- REORG: include: move pipe.h to haproxy/pipe{,-t}.h
- CLEANUP: include: remove empty raw_sock.h
- REORG: include: move proto_udp.h to haproxy/proto_udp{,-t}.h
- REORG: include: move proto/proto_sockpair.h to haproxy/proto_sockpair.h
- REORG: include: move compression.h to haproxy/compression{,-t}.h
- REORG: include: move h1_htx.h to haproxy/h1_htx.h
- REORG: include: move http_htx.h to haproxy/http_htx{,-t}.h
- REORG: include: move hlua.h to haproxy/hlua{,-t}.h
- REORG: include: move hlua_fcn.h to haproxy/hlua_fcn.h
- REORG: include: move action.h to haproxy/action{,-t}.h
- REORG: include: move arg.h to haproxy/arg{,-t}.h
- REORG: include: move auth.h to haproxy/auth{,-t}.h
- REORG: include: move dns.h to haproxy/dns{,-t}.h
- REORG: include: move flt_http_comp.h to haproxy/
- REORG: include: move counters.h to haproxy/counters-t.h
- REORG: include: split mailers.h into haproxy/mailers{,-t}.h
- REORG: include: move capture.h to haproxy/capture{,-t}.h
- REORG: include: move frontend.h to haproxy/frontend.h
- REORG: include: move obj_type.h to haproxy/obj_type{,-t}.h
- REORG: include: move http_rules.h to haproxy/http_rules.h
- CLEANUP: include: remove unused mux_pt.h
- REORG: include: move mworker.h to haproxy/mworker{,-t}.h
- REORG: include: move ssl_utils.h to haproxy/ssl_utils.h
- REORG: include: move ssl_ckch.h to haproxy/ssl_ckch{,-t}.h
- REORG: move ssl_crtlist.h to haproxy/ssl_crtlist{,-t}.h
- REORG: include: move lb_chash.h to haproxy/lb_chash{,-t}.h
- REORG: include: move lb_fas.h to haproxy/lb_fas{,-t}.h
- REORG: include: move lb_fwlc.h to haproxy/lb_fwlc{,-t}.h
- REORG: include: move lb_fwrr.h to haproxy/lb_fwrr{,-t}.h
- REORG: include: move listener.h to haproxy/listener{,-t}.h
- REORG: include: move pattern.h to haproxy/pattern{,-t}.h
- REORG: include: move map to haproxy/map{,-t}.h
- REORG: include: move payload.h to haproxy/payload.h
- REORG: include: move sample.h to haproxy/sample{,-t}.h
- REORG: include: move protocol_buffers.h to haproxy/protobuf{,-t}.h
- REORG: include: move vars.h to haproxy/vars{,-t}.h
- REORG: include: split global.h into haproxy/global{,-t}.h
- REORG: include: move task.h to haproxy/task{,-t}.h
- REORG: include: move proto_tcp.h to haproxy/proto_tcp.h
- REORG: include: move signal.h to haproxy/signal{,-t}.h
- REORG: include: move tcp_rules.h to haproxy/tcp_rules.h
- REORG: include: move connection.h to haproxy/connection{,-t}.h
- REORG: include: move checks.h to haproxy/check{,-t}.h
- REORG: include: move http_fetch.h to haproxy/http_fetch.h
- REORG: include: move peers.h to haproxy/peers{,-t}.h
- REORG: include: move stick_table.h to haproxy/stick_table{,-t}.h
- REORG: include: move session.h to haproxy/session{,-t}.h
- REORG: include: move trace.h to haproxy/trace{,-t}.h
- REORG: include: move acl.h to haproxy/acl.h{,-t}.h
- REORG: include: split common/uri_auth.h into haproxy/uri_auth{,-t}.h
- REORG: move applet.h to haproxy/applet{,-t}.h
- REORG: include: move stats.h to haproxy/stats{,-t}.h
- REORG: include: move cli.h to haproxy/cli{,-t}.h
- REORG: include: move lb_map.h to haproxy/lb_map{,-t}.h
- REORG: include: move ssl_sock.h to haproxy/ssl_sock{,-t}.h
- REORG: include: move stream_interface.h to haproxy/stream_interface{,-t}.h
- REORG: include: move channel.h to haproxy/channel{,-t}.h
- REORG: include: move http_ana.h to haproxy/http_ana{,-t}.h
- REORG: include: move filters.h to haproxy/filters{,-t}.h
- REORG: include: move fcgi-app.h to haproxy/fcgi-app{,-t}.h
- REORG: include: move log.h to haproxy/log{,-t}.h
- REORG: include: move proxy.h to haproxy/proxy{,-t}.h
- REORG: include: move spoe.h to haproxy/spoe{,-t}.h
- REORG: include: move backend.h to haproxy/backend{,-t}.h
- REORG: include: move queue.h to haproxy/queue{,-t}.h
- REORG: include: move server.h to haproxy/server{,-t}.h
- REORG: include: move stream.h to haproxy/stream{,-t}.h
- REORG: include: move cfgparse.h to haproxy/cfgparse.h
- CLEANUP: hpack: export debug functions and move inlines to .h
- REORG: check: move the e-mail alerting code to mailers.c
- REORG: check: move tcpchecks away from check.c
- REORG: check: move email_alert* from proxy-t.h to mailers-t.h
- REORG: check: extract the external checks from check.{c,h}
- CLEANUP: include: don't include stddef.h directly
- CLEANUP: include: don't include proxy-t.h in global-t.h
- CLEANUP: include: move sample_data out of sample-t.h
- REORG: include: move the error reporting functions to from log.h to errors.h
- BUILD: reorder objects in the Makefile for faster builds
- CLEANUP: compiler: add a THREAD_ALIGNED macro and use it where appropriate
- CLEANUP: include: make atomic.h part of the base API
- REORG: include: move MAX_THREADS to defaults.h
- REORG: include: move THREAD_LOCAL and __decl_thread() to compiler.h
- CLEANUP: include: tree-wide alphabetical sort of include files
- REORG: include: make list-t.h part of the base API
- REORG: dgram: rename proto_udp to dgram
By default, HAProxy is able to implicitly upgrade an H1 client connection to an
H2 connection if the first request it receives from a given HTTP connection
matches the HTTP/2 connection preface. This way, it is possible to support H1
and H2 clients on a non-SSL connections. It could be a problem if for any
reason, the H2 upgrade is not acceptable. "option disable-h2-upgrade" may now be
used to disable it, per proxy. The main puprose of this option is to let an
admin to totally disable the H2 support for security reasons. Recently, a
critical issue in the HPACK decoder was fixed, forcing everyone to upgrade their
HAProxy version to fix the bug. It is possible to disable H2 for SSL
connections, but not on clear ones. This option would have been a viable
workaround.
Network types were directly and mistakenly mapped on sample types:
This patch fix the doc with values effectively used to keep backward
compatiblitiy on existing implementations.
In addition it adds an internal/network mapping for key types to avoid
further mistakes adding or modifying internals types.
This patch should be backported on all maintained branches,
particularly until v1.8 included for documentation part.
Using ssl-max-ver without ssl-min-ver is ambiguous.
When the ssl-min-ver is not configured, and ssl-max-ver is set to a
value lower than the default ssl-min-ver (which is TLSv1.2 currently),
set the ssl-min-ver to the value of ssl-max-ver, and emit a warning.
log-proto <logproto>
The "log-proto" specifies the protocol used to forward event messages to
a server configured in a ring section. Possible values are "legacy"
and "octet-count" corresponding respectively to "Non-transparent-framing"
and "Octet counting" in rfc6587. "legacy" is the default.
Notes: a separated io_handler was created to avoid per messages test
and to prepare code to set different log protocols such as
request- response based ones.
This patch adds new statement "server" into ring section, and the
related "timeout connect" and "timeout server".
server <name> <address> [param*]
Used to configure a syslog tcp server to forward messages from ring buffer.
This supports for all "server" parameters found in 5.2 paragraph.
Some of these parameters are irrelevant for "ring" sections.
timeout connect <timeout>
Set the maximum time to wait for a connection attempt to a server to succeed.
Arguments :
<timeout> is the timeout value specified in milliseconds by default, but
can be in any other unit if the number is suffixed by the unit,
as explained at the top of this document.
timeout server <timeout>
Set the maximum time for pending data staying into output buffer.
Arguments :
<timeout> is the timeout value specified in milliseconds by default, but
can be in any other unit if the number is suffixed by the unit,
as explained at the top of this document.
Example:
global
log ring@myring local7
ring myring
description "My local buffer"
format rfc3164
maxlen 1200
size 32764
timeout connect 5s
timeout server 10s
server mysyslogsrv 127.0.0.1:6514
There is no reason to not use proxy's error replies to emit 401/407
responses. The function http_reply_40x_unauthorized(), responsible to emit those
responses, is not really complex. It only adds a
WWW-Authenticate/Proxy-Authenticate header to a generic message.
So now, error replies can be defined for 401 and 407 status codes, using
errorfile or http-error directives. When an http-request auth rule is evaluated,
the corresponding error reply is used. For 401 responses, all occurrences of the
WWW-Authenticate header are removed and replaced by a new one with a basic
authentication challenge for the configured realm. For 407 responses, the same
is done on the Proxy-Authenticate header. If the error reply must not be
altered, "http-request return" rule must be used instead.
It is possible to globally declare ring-buffers, to be used as target for log
servers or traces.
ring <ringname>
Creates a new ring-buffer with name <ringname>.
description <text>
The descritpition is an optional description string of the ring. It will
appear on CLI. By default, <name> is reused to fill this field.
format <format>
Format used to store events into the ring buffer.
Arguments:
<format> is the log format used when generating syslog messages. It may be
one of the following :
iso A message containing only the ISO date, followed by the text.
The PID, process name and system name are omitted. This is
designed to be used with a local log server.
raw A message containing only the text. The level, PID, date, time,
process name and system name are omitted. This is designed to be
used in containers or during development, where the severity
only depends on the file descriptor used (stdout/stderr). This
is the default.
rfc3164 The RFC3164 syslog message format. This is the default.
(https://tools.ietf.org/html/rfc3164)
rfc5424 The RFC5424 syslog message format.
(https://tools.ietf.org/html/rfc5424)
short A message containing only a level between angle brackets such as
'<3>', followed by the text. The PID, date, time, process name
and system name are omitted. This is designed to be used with a
local log server. This format is compatible with what the systemd
logger consumes.
timed A message containing only a level between angle brackets such as
'<3>', followed by ISO date and by the text. The PID, process
name and system name are omitted. This is designed to be
used with a local log server.
maxlen <length>
The maximum length of an event message stored into the ring,
including formatted header. If an event message is longer than
<length>, it will be truncated to this length.
size <size>
This is the optional size in bytes for the ring-buffer. Default value is
set to BUFSIZE.
Example:
global
log ring@myring local7
ring myring
description "My local buffer"
format rfc3164
maxlen 1200
Note: ring names are resolved during post configuration processing.
As discussed in GitHub issue #624 Lua scripts should not use
variables that are never going to be read, because the memory
for variable names is never going to be freed.
Add an optional `ifexist` parameter to the `set_var` function
that allows a Lua developer to set variables that are going to
be ignored if the variable name was not used elsewhere before.
Usually this mean that there is no `var()` sample fetch for the
variable in question within the configuration.
Released version 2.2-dev8 with the following main changes :
- MINOR: checks: Improve report of unexpected errors for expect rules
- MEDIUM: checks: Add matching on log-format string for expect rules
- DOC: Fix req.body and co documentation to be accurate
- MEDIUM: checks: Remove dedicated sample fetches and use response ones instead
- CLEANUP: checks: sort and rename tcpcheck_expect_type types
- MINOR: checks: Use dedicated actions to send log-format strings in send rules
- MINOR: checks: Simplify matching on HTTP headers in HTTP expect rules
- MINOR: checks/sample: Remove unnecessary tests on the sample session
- REGTEST: checks: Adapt SSL error message reported when connection is rejected
- MINOR: mworker: replace ha_alert by ha_warning when exiting successfuly
- MINOR: checks: Support log-format string to set the URI for HTTP send rules
- MINOR: checks: Support log-format string to set the body for HTTP send rules
- DOC: Be more explicit about configurable check ok/error/timeout status
- MINOR: checks: Make matching on HTTP headers for expect rules less obscure
- BUG/MEDIUM: lua: Fix dumping of stick table entries for STD_T_DICT
- BUG/MINOR: config: Make use_backend and use-server post-parsing less obscur
- REGTESTS: make the http-check-send test require version 2.2
- BUG/MINOR: http-ana: fix NTLM response parsing again
- BUG/MEDIUM: http_ana: make the detection of NTLM variants safer
- BUG/MINOR: cfgparse: Abort parsing the current line if an invalid \x sequence is encountered
- MINOR: cfgparse: Improve error message for invalid \x sequences
- CI: travis-ci: enable arm64 builds again
- MEDIUM: ssl: increase default-dh-param to 2048
- CI: travis-ci: skip pcre2 on arm64 build
- CI: travis-ci: extend the build time for SSL to 60 minutes
- CLEANUP: config: drop unused setting CONFIG_HAP_MEM_OPTIM
- CLEANUP: config: drop unused setting CONFIG_HAP_INLINE_FD_SET
- CLENAUP: config: move CONFIG_HAP_LOCKLESS_POOLS out of config.h
- CLEANUP: remove THREAD_LOCAL from config.h
- CI: travis-ci: upgrade LibreSSL versions
- DOC: assorted typo fixes in the documentation
- CI: extend spellchecker whitelist
- CLEANUP: assorted typo fixes in the code and comments
- MAJOR: contrib: porting spoa_server to support python3
- BUG/MEDIUM: checks: Subscribe to I/O events on an unfinished connect
- BUG/MINOR: checks: Don't subscribe to I/O events if it is already done
- BUG/MINOR: checks: Rely on next I/O oriented rule when waiting for a connection
- MINOR: checks: Don't try to send outgoing data if waiting to be able to send
- MINOR: sample: Move aes_gcm_dec implementation into sample.c
- MINOR: sample: Add digest and hmac converters
- BUG/MEDIUM: checks: Subscribe to I/O events only if a mux was installed
- BUG/MINOR: sample/ssl: Fix digest converter for openssl < 1.1.0
- BUG/MINOR: pools: use %u not %d to report pool stats in "show pools"
- BUG/MINOR: pollers: remove uneeded free in global init
- CLEANUP: select: enhance readability in init
- BUG/MINOR: soft-stop: always wake up waiting threads on stopping
- MINOR: soft-stop: let the first stopper only signal other threads
- BUILD: select: only declare existing local labels to appease clang
- BUG/MEDIUM: streams: Remove SF_ADDR_SET if we're retrying due to L7 retry.
- BUG/MEDIUM: stream: Only allow L7 retries when using HTTP.
- DOC: retry-on can only be used with mode http
- MEDIUM: ssl: allow to register callbacks for SSL/TLS protocol messages
- MEDIUM: ssl: split ssl_sock_msgcbk() and use a new callback mechanism
- MINOR: ssl: add a new function ssl_sock_get_ssl_object()
- MEDIUM: ssl: use ssl_sock_get_ssl_object() in fetchers where appropriate
- REORG: ssl: move macros and structure definitions to ssl_sock.h
- CLEANUP: ssl: remove the shsess_* macros
- REORG: move the crt-list structures in their own .h
- REORG: ssl: move the ckch structures to types/ssl_ckch.h
- CLEANUP: ssl: add ckch prototypes in proto/ssl_ckch.h
- REORG: ssl: move crtlist functions to src/ssl_crtlist.c
- CLEANUP: ssl: avoid circular dependencies in ssl_crtlist.h
- REORG: ssl: move the ckch_store related functions to src/ssl_ckch.c
- REORG: ssl: move ckch_inst functions to src/ssl_ckch.c
- REORG: ssl: move the crt-list CLI functions in src/ssl_crtlist.c
- REORG: ssl: move the CLI 'cert' functions to src/ssl_ckch.c
- REORG: ssl: move ssl configuration to cfgparse-ssl.c
- MINOR: ssl: remove static keyword in some SSL utility functions
- REORG: ssl: move ssl_sock_ctx and fix cross-dependencies issues
- REORG: ssl: move sample fetches to src/ssl_sample.c
- REORG: ssl: move utility functions to src/ssl_utils.c
- DOC: ssl: update MAINTAINERS file
- CI: travis-ci: switch arm64 builds to use openssl from distro
- MINOR: stats: Prepare for more accurate moving averages
- MINOR: stats: Expose native cum_req metric for a server
- MEDIUM: stats: Enable more accurate moving average calculation for stats
- BUILD: ssl: include buffer common headers for ssl_sock_ctx
- BUILD: ssl: include errno.h in ssl_crtlist.c
- CLEANUP: acl: remove unused assignment
- DOC/MINOR: halog: Add long help info for ic flag
- BUILD: ssl: fix build without OPENSSL_NO_ENGINE
- DOC: SPOE is no longer experimental
- BUG/MINOR: cache: Don't needlessly test "cache" keyword in parse_cache_flt()
- MINOR: config: Don't dump keywords if argument is NULL
- MEDIUM: checks: Make post-41 the default mode for mysql checks
- BUG/MINOR: logs: prevent double line returns in some events.
- MEDIUM: sink: build header in sink_write for log formats
- MEDIUM: logs: buffer targets now rely on new sink_write
- MEDIUM: sink: add global statement to create a new ring (sink buffer)
- MEDIUM: hpack: use a pool for the hpack table
- BUG/MAJOR: mux-fcgi: Stop sending loop if FCGI stream is blocked for any reason
- BUG/MEDIUM: ring: write-lock the ring while attaching/detaching
- MINOR: applet: adopt the wait list entry from the CLI
- MINOR: ring: make the applet code not depend on the CLI
- Revert "MEDIUM: sink: add global statement to create a new ring (sink buffer)"
- CI: travis-ci: fix libslz download URL
- MINOR: ssl: split config and runtime variable for ssl-{min,max}-ver
- CLEANUP: http_ana: Remove unused TXN flags
- BUG/MINOR: http-rules: Mark http return rules as final
- MINOR: http-htx: Add http_reply type based on what is used for http return rules
- CLEANUP: http-htx: Rename http_error structure into http_error_msg
- MINOR: http-rules: Use http_reply structure for http return rules
- MINOR: http-htx: Use a dedicated function to release http_reply objects
- MINOR: http-htx: Use a dedicated function to parse http reply arguments
- MINOR: http-htx: Use a dedicated function to check http reply validity
- MINOR: http-ana: Use a dedicated function to send a response from an http reply
- MEDIUM: http-rules: Rely on http reply for http deny/tarpit rules
- MINOR: http-htx: Store default error messages in a global http reply array
- MINOR: http-htx: Store messages of an http-errors section in a http reply array
- MINOR: http-htx: Store errorloc/errorfile messages in http replies
- MINOR: proxy: Add references on http replies for proxy error messages
- MINOR: http-htx: Use http reply from the http-errors section
- MINOR: http-ana: Use a TXN flag to prevent after-response ruleset evaluation
- MEDIUM: http-ana: Use http replies for HTTP error messages
- CLEANUP: http-htx: Remove unused storage of error messages in buffers
- MINOR: htx: Add a function to copy a buffer in an HTX message
- CLEANUP: channel: Remove channel_htx_copy_msg() function
- MINOR: http-ana: Add a function to write an http reply in an HTX message
- MINOR: http-htx/proxy: Add http-error directive using http return syntax
- DOC: Fix "errorfile" description in the configuration manual
- BUG/MINOR: checks: Respect check-ssl param when a port or an addr is specified
- BUILD: hpack: make sure the hpack table can still be built standalone
- CONTRIB: hpack: make use of the simplified standalone HPACK API
- MINOR: connection: add pp2-never-send-local to support old PP2 behavior
A bug in the PROXY protocol v2 implementation was present in HAProxy up to
version 2.1, causing it to emit a PROXY command instead of a LOCAL command
for health checks. This is particularly minor but confuses some servers'
logs. Sadly, the bug was discovered very late and revealed that some servers
which possibly only tested their PROXY protocol implementation against
HAProxy fail to properly handle the LOCAL command, and permanently remain in
the "down" state when HAProxy checks them. When this happens, it is possible
to enable this global option to revert to the older (bogus) behavior for the
time it takes to contact the affected components' vendors and get them fixed.
This option is disabled by default and acts on all servers having the
"send-proxy-v2" statement.
Older versions were reverted to the old behavior and should not attempt to
be fixed by default again. However a variant of this patch could possibly
be implemented to ask to explicitly send LOCAL if needed by some servers.
More context here:
https://www.mail-archive.com/haproxy@formilux.org/msg36890.htmlhttps://www.mail-archive.com/haproxy@formilux.org/msg37218.html
"errorfile" description is outdated. Now, error messages are parsed at startup
and converted to HTX messages. So they must be valid according to the HTTP
standards. In addition, there is now a comment about the reserved buffer space
that should remain available to allow header rewrites.
The http-error directive can now be used instead of errorfile to define an error
message in a proxy section (including default sections). This directive uses the
same syntax that http return rules. The only real difference is the limitation
on status code that may be specified. Only status codes supported by errorfile
directives are supported for this new directive. Parsing of errorfile directive
remains independent from http-error parsing. But functionally, it may be
expressed in terms of http-errors :
errorfile <status> <file> ==> http-errror status <status> errorfile <file>
"http-request deny", "http-request tarpit" and "http-response deny" rules now
use the same syntax than http return rules and internally rely on the http
replies. The behaviour is not the same when no argument is specified (or only
the status code). For http replies, a dummy response is produced, with no
payload. For old deny/tarpit rules, the proxy's error messages are used. Thus,
to be compatible with existing configuration, the "default-errorfiles" parameter
is implied. For instance :
http-request deny deny_status 404
is now an alias of
http-request deny status 404 default-errorfiles
This reverts commit 957ec59571.
As discussed with Emeric, the current syntax is not extensible enough,
this will be turned to a section instead in a forthcoming patch.
This patch adds the new global statement:
ring <name> [desc <desc>] [format <format>] [size <size>] [maxlen <length>]
Creates a named ring buffer which could be used on log line for instance.
<desc> is an optionnal description string of the ring. It will appear on
CLI. By default, <name> is reused to fill this field.
<format> is the log format used when generating syslog messages. It may be
one of the following :
iso A message containing only the ISO date, followed by the text.
The PID, process name and system name are omitted. This is
designed to be used with a local log server.
raw A message containing only the text. The level, PID, date, time,
process name and system name are omitted. This is designed to be
used in containers or during development, where the severity only
depends on the file descriptor used (stdout/stderr). This is
the default.
rfc3164 The RFC3164 syslog message format. This is the default.
(https://tools.ietf.org/html/rfc3164)
rfc5424 The RFC5424 syslog message format.
(https://tools.ietf.org/html/rfc5424)
short A message containing only a level between angle brackets such as
'<3>', followed by the text. The PID, date, time, process name
and system name are omitted. This is designed to be used with a
local log server. This format is compatible with what the systemd
logger consumes.
timed A message containing only a level between angle brackets such as
'<3>', followed by ISO date and by the text. The PID, process
name and system name are omitted. This is designed to be
used with a local log server.
<length> is the maximum length of event message stored into the ring,
including formatted header. If the event message is longer
than <length>, it would be truncated to this length.
<name> is the ring identifier, which follows the same naming convention as
proxies and servers.
<size> is the optionnal size in bytes. Default value is set to BUFSIZE.
Note: Historically sink's name and desc were refs on const strings. But with new
configurable rings a dynamic allocation is needed.
MySQL 4.1 is old enough to be the default mode for mysql checks. So now, once a
username is defined, post-41 mode is automatically used. To do mysql checks on
previous MySQL version, the argument "pre-41" must be used.
Note, it is a compatibility breakage for everyone using an antique and
unsupported MySQL version.
The documentation for retry-on hints at it being meant to be used
in conjuction with mode http, but since we've a had bug report
involving mode tcp and retry-on, lets make it explicit in the
documentation that it only works with mode http and will be
ignored otherwise.
Make the digest and HMAC function of OpenSSL accessible to the user via
converters. They can be used to sign and validate content.
Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
For 6 years now we've been seeing a warning suggesting to set dh-param
beyond 1024 if possible when it was not set. It's about time to do it
and get rid of this warning since most users seem to already use 2048.
It will remain possible to set a lower value of course, so only those
who were experiencing the warning and were relying on the default value
may notice a change (higher CPU usage). For more context, please refer
to this thread :
https://www.mail-archive.com/haproxy@formilux.org/msg37226.html
This commit removes a big chunk of code which happened to be needed
exclusively to figure if it was required to emit a warning or not :-)
It is possible to configure the check status on success, on error and on
timeout, for http-check and tcp-check expect rules. But the documentation relies
on internal names. These names are reported on the stats and are describe in the
management guide. But it is probably a good idea to be more explicit too in the
doc describing these options.
For http-check send rules, it is now possible to use a log-format string to set
the request's body. the keyword "body-lf" should be used instead of "body". If the
string eval fails, no body is added.
For http-check send rules, it is now possible to use a log-format string to set
the request URI. the keyword "uri-lf" should be used instead of "uri". If the
string eval fails, we fall back on the default uri "/".
Extra parameters on http-check expect rules, for the header matching method, to
use log-format string or to match full header line have been removed. There is
now separate matching methods to match a full header line or to match each
comma-separated values. "http-check expect fhdr" must be used in the first case,
and "http-check expect hdr" in the second one. In addition, to match log-format
header name or value, "-lf" suffix must be added to "name" or "value"
keyword. For intance:
http-check expect hdr name "set-cookie" value-lf -m beg "sessid=%[var(check.cookie)]"
Thanks to this changes, each parameter may only be interpreted in one way.
Following actions have been added to send log-format strings from a tcp-check
ruleset instead the log-format parameter:
* tcp-check send-lf <fmt>
* tcp-check send-binary-lf <fmt>
It is easier for tools generating configurations. Each action may only be
interpreted in one way.
All sample fetches in the scope "check." have been removed. Response sample
fetches must be used instead. It avoids keyword duplication. So, for instance,
res.hdr() must be now used instead of check.hdr().
To do so, following sample fetches have been added on the response :
* res.body, res.body_len and res.body_size
* res.hdrs and res.hdrs_bin
Sample feches dealing with the response's body are only useful in the health
checks context. When called from a stream context, there is no warranty on the
body presence. There is no option to wait the response's body.
Because the HTX is the only mode to represent HTTP data, "option
http-request-buffer" is no longer mandatory to have body data. Without this
option, there is no warranty on the body presence. So it is recommanded to use
it. But it is not a requirement. In addition, the note about chunked body is
removed because outdated.
It is now possible to use log-format string (or hexadecimal string for the
binary version) to match a content in tcp-check based expect rules. For
hexadecimal log-format string, the conversion in binary is performed after the
string evaluation, during health check execution. The pattern keywords to use
are "string-lf" for the log-format string and "binary-lf" for the hexadecimal
log-format string.
Released version 2.2-dev7 with the following main changes :
- MINOR: version: Show uname output in display_version()
- CI: run weekly OpenSSL "no-deprecated" builds
- CLEANUP: log: fix comment of parse_logformat_string()
- DOC: Improve documentation on http-request set-src
- MINOR: ssl/cli: disallow SSL options for directory in 'add ssl crt-list'
- MINOR: ssl/cli: restrain certificate path when inserting into a directory
- MINOR: ssl: add ssl-skip-self-issued-ca global option
- BUG/MINOR: ssl: default settings for ssl server options are not used
- MINOR: config: add a global directive to set default SSL curves
- BUG/MEDIUM: http-ana: Handle NTLM messages correctly.
- DOC: internals: update the SSL architecture schema
- BUG/MINOR: tools: fix the i386 version of the div64_32 function
- BUG/MINOR: mux-fcgi/trace: fix wrong set of trace flags in fcgi_strm_add_eom()
- BUG/MINOR: http: make url_decode() optionally convert '+' to SP
- DOC: option logasap does not depend on mode
- MEDIUM: memory: make pool_gc() run under thread isolation
- MINOR: contrib: make the peers wireshark dissector a plugin
- BUG/MINOR: http-ana: Throw a 500 error if after-response ruleset fails on errors
- BUG/MINOR: check: Update server address and port to execute an external check
- MINOR: mini-clist: Add functions to iterate backward on a list
- MINOR: checks: Add a way to send custom headers and payload during http chekcs
- MINOR: server: respect warning and alert semantic
- BUG/MINOR: checks: Respect the no-check-ssl option
- BUG/MEDIUM: server/checks: Init server check during config validity check
- CLEANUP: checks: Don't export anymore init_check and srv_check_healthcheck_port
- BUG/MINOR: checks: chained expect will not properly wait for enough data
- BUG/MINOR: checks: Forbid tcp-check lines in default section as documented
- MINOR: checks: Use an enum to describe the tcp-check rule type
- MINOR: checks: Simplify connection flag parsing in tcp-check connect
- MEDIUM: checks: rewind to the first inverse expect rule of a chain on new data
- MINOR: checks: simplify tcp expect config parser
- MINOR: checks: add min-recv tcp-check expect option
- MINOR: checks: add linger option to tcp connect
- MINOR: checks: define a tcp expect type
- MEDIUM: checks: rewrite tcp-check expect block
- MINOR: checks: Stop xform buffers to null-terminated string for tcp-check rules
- MINOR: checks: add rbinary expect match type
- MINOR: checks: Simplify functions to get step id and comment
- MEDIUM: checks: capture groups in expect regexes
- MINOR: checks: Don't use a static tcp rule list head
- MEDIUM: checks: Use a non-comment rule iterator to get next rule
- MEDIUM: proxy/checks: Register a keyword to parse tcp-check rules
- MINOR: checks: Set the tcp-check rule index during parsing
- MINOR: checks: define tcp-check send type
- MINOR: checks: define a tcp-check connect type
- MEDIUM: checks: Add implicit tcp-check connect rule
- MAJOR: checks: Refactor and simplify the tcp-check loop
- MEDIUM: checks: Associate a session to each tcp-check healthcheck
- MINOR: checks/vars: Add a check scope for variables
- MEDIUM: checks: Parse custom action rules in tcp-checks
- MINOR: checks: Add support to set-var and unset-var rules in tcp-checks
- MINOR: checks: Add the sni option for tcp-check connect rules
- MINOR: checks: Add the via-socks4 option for tcp-check connect rules
- MINOR: checks: Add the alpn option for tcp-check connect rules
- MINOR: ssl: Export a generic function to parse an alpn string
- MINOR: checks: Add the default option for tcp-check connect rules
- MINOR: checks: Add the addr option for tcp-check connect rule
- MEDIUM: checks: Support expression to set the port
- MEDIUM: checks: Support log-format strings for tcp-check send rules
- MINOR: log: Don't depends on a stream to process samples in log-format string
- MINOR: log: Don't systematically set LW_REQ when a sample expr is added
- MEDIUM: checks: Add a shared list of tcp-check rules
- MINOR: sample: add htonl converter
- MINOR: sample: add cut_crlf converter
- MINOR: sample: add ltrim converter
- MINOR: sample: add rtrim converter
- MINOR: checks: Use a name for the healthcheck status enum
- MINOR: checks: Add option to tcp-check expect rules to customize error status
- MINOR: checks: Merge tcp-check comment rules with the others at config parsing
- MINOR: checks: Add a sample fetch to extract a block from the input check buffer
- MEDIUM: checks: Add on-error/on-success option on tcp-check expect rules
- MEDIUM: checks: Add status-code sample expression on tcp-check expect rules
- MINOR: checks: Relax the default option for tcp-check connect rules
- MEDIUM: checks: Add a list of vars to set before executing a tpc-check ruleset
- MINOR: checks: Export the tcpcheck_eval_ret enum
- MINOR: checks: Use dedicated function to handle onsuccess/onerror messages
- MINOR: checks: Support custom functions to eval a tcp-check expect rules
- MEDIUM: checks: Implement redis check using tcp-check rules
- MEDIUM: checks: Implement ssl-hello check using tcp-check rules
- MEDIUM: checks: Implement smtp check using tcp-check rules
- MEDIUM: checks: Implement postgres check using tcp-check rules
- MEDIUM: checks: Implement MySQL check using tcp-check rules
- MEDIUM: checks: Implement LDAP check using tcp-check rules
- MEDIUM: checks: Implement SPOP check using tcp-check rules
- MINOR: server/checks: Move parsing of agent keywords in checks.c
- MINOR: server/checks: Move parsing of server check keywords in checks.c
- MEDIUM: checks: Implement agent check using tcp-check rules
- REGTEST: Adapt regtests about checks to recent changes
- MINOR: Produce tcp-check info message for pure tcp-check rules only
- MINOR: checks: Add an option to set success status of tcp-check expect rules
- MINOR: checks: Improve log message of tcp-checks on success
- MINOR: proxy/checks: Move parsing of httpchk option in checks.c
- MINOR: proxy/checks: Move parsing of tcp-check option in checks.c
- MINOR: proxy/checks: Register a keyword to parse http-check rules
- MINOR: proxy/checks: Move parsing of external-check option in checks.c
- MINOR: proxy/checks: Register a keyword to parse external-check rules
- MEDIUM: checks: Use a shared ruleset to store tcp-check rules
- MINOR: checks: Use an indirect string to represent the expect matching string
- MINOR: checks: Introduce flags to configure in tcp-check expect rules
- MINOR: standard: Add my_memspn and my_memcspn
- MINOR: checks: Add a reverse non-comment rule iterator to get last rule
- MAJOR: checks: Implement HTTP check using tcp-check rules
- MINOR: checks: Make resume conditions more explicit in tcpcheck_main()
- MINOR: connection: Add macros to know if a conn or a cs uses an HTX mux
- MEDIUM: checks: Refactor how data are received in tcpcheck_main()
- MINOR: checks/obj_type: Add a new object type for checks
- BUG/MINOR: obj_type: Handle stream object in obj_base_ptr() function
- MINOR: checks: Use the check as origin when a session is created
- MINOR: checks: Add a mux proto to health-check and tcp-check connect rule
- MINOR: connection: Add a function to install a mux for a health-check
- MAJOR: checks: Use the best mux depending on the protocol for health checks
- MEDIUM: checks: Implement default TCP check using tcp-check rules
- MINOR: checks: Remove unused code about pure TCP checks
- CLEANUP: checks: Reorg checks.c file to be more readable
- REGTEST: Fix reg-tests about health-checks to adapt them to recent changes
- MINOR: ist: Add a function to retrieve the ist pointer
- MINOR: checks: Use ist API as far as possible
- BUG/MEDIUM: checks: Be sure to subscribe for sends if outgoing data remains
- MINOR: checks: Use a tree instead of a list to store tcp-check rulesets
- BUG/MINOR: checks: Send the right amount of outgoing data for HTTP checks
- REGTEST: Add scripts to test based tcp-check health-checks
- Revert "MEDIUM: checks: capture groups in expect regexes"
- DOC: Add documentation about comments for tcp-check and http-check directives
- DOC: Fix the tcp-check and http-check directives layout
- BUG/MEDIUM: checks: Use the mux protocol specified on the server line
- MINOR: checks: Support mux protocol definition for tcp and http health checks
- BUG/MINOR: mux-fcgi: Be sure to have a connection as session's origin to use it
- MINOR: checks: Support list of status codes on http-check expect rules
- BUG/MEDIUM: checks: Unsubscribe to mux events when a conn-stream is destroyed
- REGTEST: Add a script to validate agent checks
- BUG/MINOR: server: Fix server_finalize_init() to avoid unused variable
- BUG/MEDIUM: checks: unsubscribe for events on the old conn-stream on connect
- BUG/MINOR: checks: Only use ssl_sock_is_ssl() if compiled with SSL support
- BUG/MINOR: checks/server: use_ssl member must be signed
- BUG/MEDIUM: sessions: Always pass the mux context as argument to destroy a mux
- BUG/MEDIUM: checks: Destroy the conn-stream before the session
- BUG/MINOR: checks: Fix PostgreSQL regex on the authentication packet
- CI: cirrus-ci: remove reg-tests/checks/tcp-check-ssl.vtc on CentOS 6
- MINOR: checks: Support HTTP/2 version (without '.0') for http-check send rules
- MINOR: checks: Use ver keyword to specify the HTTP version for http checks
- BUG/MINOR: checks: Remove wrong variable redeclaration
- BUG/MINOR: checks: Properly handle truncated mysql server messages
- CLEANUP: checks: Remove unused code when ldap server message is parsed
- MINOR: checks: Make the use of the check's server more explicit on connect
- BUG/MINOR: checks: Avoid incompatible cast when a binary string is parsed
- BUG/MINOR: checks: Remove bad call to free() when an expect rule is parsed
- BUG/MINOR: checks: Don't lose warning on proxy capability
- MINOR: log: Add "Tu" timer
- BUG/MINOR: checks: Set the output buffer length before calling parse_binary()
- BUG/MEDIUM: mux-h1: make sure we always have a timeout on front connections
- REGTEST: ssl: test the client certificate authentication
- DOC: give a more accurate description of what check does
- BUG/MEDIUM: capture: capture-req/capture-res converters crash without a stream
- BUG/MEDIUM: capture: capture.{req,res}.* crash without a stream
- BUG/MEDIUM: http: the "http_first_req" sample fetch could crash without a steeam
- BUG/MEDIUM: http: the "unique-id" sample fetch could crash without a steeam
- CLEANUP: http: add a few comments on certain functions' assumptions about streams
- BUG/MEDIUM: sample: make the CPU and latency sample fetches check for a stream
- MINOR: http-htx: Export functions to update message authority and host
- MINOR: checks: Don't support multiple host header for http-check send rule
- MINOR: checks: Skip some headers for http-check send rules
- MINOR: checks: Keep the Host header and the request uri synchronized
- CLEANUP: checks: Fix checks includes
- DOC: Fix send rules in the http-check connect example
- DOC: Add more info about request formatting in http-check send description
- REGTEST: http-rules: Require PCRE or PCRE2 option to run map_redirect script
- REGTEST: ssl: remove curl from the "add ssl crt-list" test
- REGTEST: ssl: improve the "set ssl cert" test
- CLEANUP: ssl: silence a build warning when threads are disabled
- BUG/MEDIUM: listener: mark the thread as not stuck inside the loop
- MINOR: threads: export the POSIX thread ID in panic dumps
- BUG/MINOR: debug: properly use long long instead of long for the thread ID
- BUG/MEDIUM: shctx: really check the lock's value while waiting
- BUG/MEDIUM: shctx: bound the number of loops that can happen around the lock
- MINOR: stream: report the list of active filters on stream crashes
- BUG/MEDIUM: mux-fcgi: Return from detach if server don't keep the connection
- BUG/MEDIUM: mux_fcgi: Free the FCGI connection at the end of fcgi_release()
- BUG/MEDIUM: mux-fcgi: Fix wrong test on FCGI_CF_KEEP_CONN in fcgi_detach()
- BUG/MEDIUM: connections: force connections cleanup on server changes
- BUG/MEDIUM: h1: Don't compare host and authority if only h1 headers are parsed
- BUG/MEDIUM: ssl: fix the id length check within smp_fetch_ssl_fc_session_id()
- CLEANUP: connections: align function declaration
- BUG/MINOR: sample: Set the correct type when a binary is converted to a string
- MEDIUM: checks/http-fetch: Support htx prefetch from a check for HTTP samples
- DOC: Document the log-format parameter for tcp-check send/send-binary rules
- MINOR: checks: Add support of payload-based sample fetches
- MINOR: checks: Add support of be_id, be_name, srv_id and srv_name sample fetches
- MINOR: checks: Add support of server side ssl sample fetches
- MINOR: checks: Add support of HTTP response sample fetches
- MINOR: http-htx: Support different methods to look for header names
- MINOR: checks: Set by default expect rule status to UNKNOWN during parsing
- BUG/MINOR: checks: Support multiple HTTP expect rules
- REGTEST: checks: Fix sync condition for agent-check
- MEDIUM: checks: Support matching on headers for http-check expect rules
- MINOR: lua: allow changing port with set_addr
- BUG/MINOR: da: Fix HTX message prefetch
- BUG/MINOR: wurfl: Fix HTX message prefetch
- BUG/MINOR: 51d: Fix HTX message prefetch
- MINOR: ist: add istadv() function
- MINOR: ist: add istissame() function
- MINOR: istbuf: add ist2buf() function
- BUG/MINOR: threads: fix multiple use of argument inside HA_ATOMIC_CAS()
- BUG/MINOR: threads: fix multiple use of argument inside HA_ATOMIC_UPDATE_{MIN,MAX}()
- DOC: update intro.txt for 2.2
- DOC: intro: add a contacts section
It is now possible to add http-check expect rules matching HTTP header names and
values. Here is the format of these rules:
http-check expect header name [ -m <meth> ] <name> [log-format] \
[ value [ -m <meth> ] <value> [log-format] [full] ]
the name pattern (name ...) is mandatory but the value pattern (value ...) is
optionnal. If not specified, only the header presence is verified. <meth> is the
matching method, applied on the header name or the header value. Supported
matching methods are:
* "str" (exact match)
* "beg" (prefix match)
* "end" (suffix match)
* "sub" (substring match)
* "reg" (regex match)
If not specified, exact matching method is used. If the "log-format" option is
used, the pattern (<name> or <value>) is evaluated as a log-format string. This
option cannot be used with the regex matching method. Finally, by default, the
header value is considered as comma-separated list. Each part may be tested. The
"full" option may be used to test the full header line. Note that matchings are
case insensitive on the header names.
HTPP sample fetches acting on the response can now be called from any sample
expression or log-format string in a tcp-check based ruleset. To avoid any
ambiguities, all these sample fetches are in the check scope, for instance
check.hdr() or check.cook().
SSL sample fetches acting on the server connection can now be called from any
sample expression or log-format string in a tcp-check based ruleset. ssl_bc and
ssl_bc_* sample fetches are concerned.
It is now possible to call be_id, be_name, srv_id and srv_name sample fetches
from any sample expression or log-format string in a tcp-check based ruleset.
It is now possible to call check.payload(), check.payload_lv() and check.len()
sample fetches from any sample expression or log-format string in a tcp-check
based ruleset. In fact, check.payload() was already added. But instead of having
a specific function to handle this sample fetch, we use the same than
req.payload().
These sample fetches act on the check input buffer, containing data received for
the server. So it should be part of or after an expect rule, but before any send
rule. Because the input buffer is cleared at this stage.
Only one Host header can be defined and some headers are automatically skipped
(Connection, Content-Length and Transfer-Encoding). In addition, a note about
the synchronisation of the Host header value and the request uri has been added.
The documentation for check implies that without an application
level check configured, it only enables simple tcp checks. What it
actually does is verify that the configured transport layer is available,
and that optional application level checks succeed.
It can be sometimes useful to measure total time of a request as seen
from an end user, including TCP/TLS negotiation, server response time
and transfer time. "Tt" currently provides something close to that, but
it also takes client idle time into account, which is problematic for
keep-alive requests as idle time can be very long. "Ta" is also not
sufficient as it hides TCP/TLS negotiationtime. To improve that, introduce
a "Tu" timer, without idle time and everything else. It roughly estimates
time spent time spent from user point of view (without DNS resolution
time), assuming network latency is the same in both directions.
It is now possible to match on a comma-separated list of status codes or range
of codes. In addtion, instead of a string comparison to match the response's
status code, a integer comparison is performed. Here is an example:
http-check expect status 200,201,300-310
It is now possible to force the mux protocol for a tcp-check based health check
using the server keyword "check-proto". If set, this parameter overwrites the
server one.
In the same way, a "proto" parameter has been added for tcp-check and http-check
connect rules. If set, this mux protocol overwrites all others for the current
connection.
The documentation about the comment argument for some tcp-check and http-check
directives was missing. As well as the description of "tcp-check comment" and
"http-check comment" directives.
When a tcp-check connect rule is evaluated, the mux protocol corresponding to
the health-check is chosen. So for TCP based health-checks, the mux-pt is
used. For HTTP based health-checks, the mux-h1 is used. The connection is marked
as private to be sure to not ruse regular HTTP connection for
health-checks. Connections reuse will be evaluated later.
The functions evaluating HTTP send rules and expect rules have been updated to
be HTX compliant. The main change for users is that HTTP health-checks are now
stricter on the HTTP message format. While before, the HTTP formatting and
parsing were minimalist, now messages should be well formatted.
HTTP health-checks are now internally based on tcp-checks. Of course all the
configuration parsing of the "http-check" keyword and the httpchk option has
been rewritten. But the main changes is that now, as for tcp-check ruleset, it
is possible to perform several send/expect sequences into the same
health-checks. Thus the connect rule is now also available from HTTP checks, jst
like set-var, unset-var and comment rules.
Because the request defined by the "option httpchk" line is used for the first
request only, it is now possible to set the method, the uri and the version on a
"http-check send" line.
All tcp-check rules are now stored in the globla shared list. The ones created
to parse a specific protocol, for instance redis, are already stored in this
list. Now pure tcp-check rules are also stored in it. The ruleset name is
created using the proxy name and its config file and line. tcp-check rules
declared in a defaults section are also stored this way using "defaults" as
proxy name.
For now, all tcp-check ruleset are stored in a list. But it could be a bit slow
to looks for a specific ruleset with a huge number of backends. So, it could be
a good idea to use a tree instead.
It is now possible to specified the healthcheck status to use on success of a
tcp-check rule, if it is the last evaluated rule. The option "ok-status"
supports "L4OK", "L6OK", "L7OK" and "L7OKC" status.
This option defines a sample expression, evaluated as an integer, to set the
status code (check->code) if a tcp-check healthcheck ends on the corresponding
expect rule.
These options define log-format strings used to produce the info message if a
tcp-check expect rule fails (on-error option) or succeeds (on-success
option). For this last option, it must be the ending rule, otherwise the
parameter is ignored.
It is now possible to extract information from the check input buffer using the
check.payload sample fetch. As req.payload or res.payload, an offset and a
length must be specified.
A new section has been added in the configuration manual. Now check sample
fetches will have to be documented under the section 7.3.7 (Fetching
health-check samples).
It is now possible to specified the healthcheck status to use on error or on
timeout for tcp-check expect rules. First, to define the error status, the
option "error-status" must be used followed by "L4CON", "L6RSP", "L7RSP" or
"L7STS". Then, to define the timeout status, the option "tout-status" must be
used followed by "L4TOUT", "L6TOUT" or "L7TOUT".
These options will be used to convert specific protocol healthchecks (redis,
pgsql...) to tcp-check ones.
x
This converter tranform a integer to its binary representation in the network
byte order. Integer are already automatically converted to binary during sample
expression evaluation. But because samples own 8-bytes integers, the conversion
produces 8 bytes. the htonl converter do the same but for 4-bytes integer.
Since we have a session attached to tcp-check healthchecks, It is possible use
sample expression and variables. In addition, it is possible to add tcp-check
set-var rules to define custom variables. So, now, a sample expression can be
used to define the port to use to establish a connection for a tcp-check connect
rule. For instance:
tcp-check set-var(check.port) int(8888)
tcp-check connect port var(check.port)
With this option, it is now possible to use a specific address to open the
connection for a tcp-check connect rule. If the port option is also specified,
it is used in priority.
With this option, it is possible to open a connection from a tcp-check connect
rule using all parameter of the server line, like any other healthcheck. For
now, this parameter is exclusive with all other option for a tcp-check connect
rule.
With this option, it is possible to establish the connection opened by a
tcp-check connect rule using upstream socks4 proxy. Info from the socks4
parameter on the server are used.
Evaluate the registered action_ptr associated with each CHK_ACTION_KW rules from
a ruleset. Currently only the 'set-var' and 'unset-var' are parsed by the
tcp-check parser. Thus it is now possible to set or unset variables. It is
possible to use such rules before the first connect of the ruleset.
The rbinary match works similarly to the rstring match type, however the
received data is rewritten as hex-string before the match operation is
done.
This allows using regexes on binary content even with the POSIX regex
engine.
[Cf: I slightly updated the patch. mem2hex function was removed and dump_binary
is used instead.]
Allow declaring tcpcheck connect commands with a new parameter,
"linger". This option will configure the connection to avoid using an
RST segment to close, instead following the four-way termination
handshake. Some servers would otherwise log each healthcheck as
an error.
Some expect rules cannot be satisfied due to inherent ambiguity towards
the received data: in the absence of match, the current behavior is to
be forced to wait either the end of the connection or a buffer full,
whichever comes first. Only then does the matching diagnostic is
considered conclusive. For instance :
tcp-check connect
tcp-check expect !rstring "^error"
tcp-check expect string "valid"
This check will only succeed if the connection is closed by the server before
the check timeout. Otherwise the first expect rule will wait for more data until
"^error" regex matches or the check expires.
Allow the user to explicitly define an amount of data that will be
considered enough to determine the value of the check.
This allows succeeding on negative rstring rules, as previously
in valid condition no match happened, and the matching was repeated
until the end of the connection. This could timeout the check
while no error was happening.
[Cf: I slighly updated the patch. The parameter was renamed and the value is a
signed integer to support -1 as default value to ignore the parameter.]
The 'http-check send' directive have been added to add headers and optionnaly a
payload to the request sent during HTTP healthchecks. The request line may be
customized by the "option httpchk" directive but there was not official way to
add extra headers. An old trick consisted to hide these headers at the end of
the version string, on the "option httpchk" line. And it was impossible to add
an extra payload with an "http-check expect" directive because of the
"Connection: close" header appended to the request (See issue #16 for details).
So to make things official and fully support payload additions, the "http-check
send" directive have been added :
option httpchk POST /status HTTP/1.1
http-check send hdr Content-Type "application/json;charset=UTF-8" \
hdr X-test-1 value1 hdr X-test-2 value2 \
body "{id: 1, field: \"value\"}"
When a payload is defined, the Content-Length header is automatically added. So
chunk-encoded requests are not supported yet. For now, there is no special
validity checks on the extra headers.
This patch is inspired by Kiran Gavali's work. It should fix the issue #16 and
as far as possible, it may be backported, at least as far as 1.8.
The documentation for option logasap misleads into thinking it is
only valid for mode http. It is actually valid for mode tcp too,
so this patch tries to disambiguate the current wording.
The url_decode() function used by the url_dec converter and a few other
call points is ambiguous on its processing of the '+' character which
itself isn't stable in the spec. This one belongs to the reserved
characters for the query string but not for the path nor the scheme,
in which it must be left as-is. It's only in argument strings that
follow the application/x-www-form-urlencoded encoding that it must be
turned into a space, that is, in query strings and POST arguments.
The problem is that the function is used to process full URLs and
paths in various configs, and to process query strings from the stats
page for example.
This patch updates the function to differentiate the situation where
it's parsing a path and a query string. A new argument indicates if a
query string should be assumed, otherwise it's only assumed after seeing
a question mark.
The various locations in the code making use of this function were
updated to take care of this (most call places were using it to decode
POST arguments).
The url_dec converter is usually called on path or url samples, so it
needs to remain compatible with this and will default to parsing a path
and turning the '+' to a space only after a question mark. However in
situations where it would explicitly be extracted from a POST or a
query string, it now becomes possible to enforce the decoding by passing
a non-null value in argument.
It seems to be what was reported in issue #585. This fix may be
backported to older stable releases.
This option activate the feature introduce in commit 16739778:
"MINOR: ssl: skip self issued CA in cert chain for ssl_ctx".
The patch disable the feature per default.
This patch adds more explanation on how to use "http-request set-src"
and a link to "option forwardfor".
This patch can be applied to all previous version starting at 1.6
Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
Released version 2.2-dev6 with the following main changes :
- BUG/MINOR: ssl: memory leak when find_chain is NULL
- CLEANUP: ssl: rename ssl_get_issuer_chain to ssl_get0_issuer_chain
- MINOR: ssl: rework add cert chain to CTX to be libssl independent
- BUG/MINOR: peers: init bind_proc to 1 if it wasn't initialized
- BUG/MINOR: peers: avoid an infinite loop with peers_fe is NULL
- BUG/MINOR: peers: Use after free of "peers" section.
- CI: github actions: add weekly h2spec test
- BUG/MEDIUM: mux_h1: Process a new request if we already received it.
- MINOR: build: Fix build in mux_h1
- CLEANUP: remove obsolete comments
- BUG/MEDIUM: dns: improper parsing of aditional records
- MINOR: ssl: skip self issued CA in cert chain for ssl_ctx
- MINOR: listener: add so_name sample fetch
- MEDIUM: stream: support use-server rules with dynamic names
- MINOR: servers: Add a counter for the number of currently used connections.
- MEDIUM: connections: Revamp the way idle connections are killed
- MINOR: cli: add a general purpose pointer in the CLI struct
- MINOR: ssl: add a list of bind_conf in struct crtlist
- REORG: ssl: move SETCERT enum to ssl_sock.h
- BUG/MINOR: ssl: ckch_inst wrongly inserted in crtlist_entry
- REORG: ssl: move some functions above crtlist_load_cert_dir()
- MINOR: ssl: use crtlist_free() upon error in directory loading
- MINOR: ssl: add a list of crtlist_entry in ckch_store
- MINOR: ssl: store a ptr to crtlist in crtlist_entry
- MINOR: ssl/cli: update pointer to store in 'commit ssl cert'
- MEDIUM: ssl/cli: 'add ssl crt-list' command
- REGTEST: ssl/cli: test the 'add ssl crt-list' command
- BUG/MINOR: ssl: entry->ckch_inst not initialized
- REGTEST: ssl/cli: change test type to devel
- REGTEST: make the PROXY TLV validation depend on version 2.2
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: stats: Fix color of draining servers on stats page
- DOC: internals: Fix spelling errors in filters.txt
- MINOR: connections: Don't mark conn flags 0x00000001 and 0x00000002 as unused.
- REGTEST: make the unique-id test depend on version 2.0
- BUG/MEDIUM: dns: Consider the fact that dns answers are case-insensitive
- MINOR: ssl: split the line parsing of the crt-list
- MINOR: ssl/cli: support filters and options in add ssl crt-list
- MINOR: ssl: add a comment above the ssl_bind_conf keywords
- REGTEST: ssl/cli: tests options and filters w/ add ssl crt-list
- REGTEST: ssl: pollute the crt-list file
- BUG/CRITICAL: hpack: never index a header into the headroom after wrapping
- BUG/MINOR: protocol_buffer: Wrong maximum shifting.
- CLEANUP: src/fd.c: mask setsockopt with DISGUISE
- BUG/MINOR: ssl/cli: initialize fcount int crtlist_entry
- REGTEST: ssl/cli: add other cases of 'add ssl crt-list'
- CLEANUP: assorted typo fixes in the code and comments
- DOC: management: add the new crt-list CLI commands
- BUG/MINOR: ssl/cli: fix spaces in 'show ssl crt-list'
- MINOR: ssl/cli: 'del ssl crt-list' delete an entry
- MINOR: ssl/cli: replace dump/show ssl crt-list by '-n' option
- CI: use better SSL library definition
- CI: travis-ci: enable DEBUG_STRICT=1 for CI builds
- CI: travis-ci: upgrade openssl to 1.1.1f
- MINOR: ssl: improve the errors when a crt can't be open
- CI: cirrus-ci: rename openssl package after it is renamed in FreeBSD
- CI: adopt openssl download script to download all versions
- BUG/MINOR: ssl/cli: lock the ckch structures during crt-list delete
- MINOR: ssl/cli: improve error for bundle in add/del ssl crt-list
- MINOR: ssl/cli: 'del ssl cert' deletes a certificate
- BUG/MINOR: ssl: trailing slashes in directory names wrongly cached
- BUG/MINOR: ssl/cli: memory leak in 'set ssl cert'
- CLEANUP: ssl: use the refcount for the SSL_CTX'
- CLEANUP: ssl/cli: use the list of filters in the crtlist_entry
- BUG/MINOR: ssl: memleak of the struct cert_key_and_chain
- CLEANUP: ssl: remove a commentary in struct ckch_inst
- MINOR: ssl: initialize all list in ckch_inst_new()
- MINOR: ssl: free instances and SNIs with ckch_inst_free()
- MINOR: ssl: replace ckchs_free() by ckch_store_free()
- BUG/MEDIUM: ssl/cli: trying to access to free'd memory
- MINOR: ssl: ckch_store_new() alloc and init a ckch_store
- MINOR: ssl: crtlist_new() alloc and initialize a struct crtlist
- REORG: ssl: move some free/new functions
- MINOR: ssl: crtlist_entry_{new, free}
- BUG/MINOR: ssl: ssl_conf always set to NULL on crt-list parsing
- MINOR: ssl: don't alloc ssl_conf if no option found
- BUG/MINOR: connection: always send address-less LOCAL PROXY connections
- BUG/MINOR: peers: Incomplete peers sections should be validated.
- MINOR: init: report in "haproxy -c" whether there were warnings or not
- MINOR: init: add -dW and "zero-warning" to reject configs with warnings
- MINOR: init: report the compiler version in haproxy -vv
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: init: report the haproxy version and executable path once on errors
- DOC: Make how "option redispatch" works more explicit
- BUILD: Makefile: add linux-musl to TARGET
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: http: Fixed small typo in parse_http_return
- DOC: hashing: update link to hashing functions
Bret Mulvey, the author of the article cited in this pulication
has migrated his work to papa.bretmulvey.com. I was able to
view an archival version of Bret M.'s original post
(http://home.comcast.net/~bretm/hash/3.html) and have validated
that this is the same paper that is originally cited.
People are often misled and think that this option can redirect
connections to backup servers.
This patch makes the documentation more specific about how the option
handles backup servers.
Since some systems switched to service managers which hide all warnings
by default, some users are not aware of some possibly important warnings
and get caught too late with errors that could have been detected earlier.
This patch adds a new global keyword, "zero-warning" and an equivalent
command-line option "-dW" to refuse to start in case any warning is
detected. It is recommended to use these with configurations that are
managed by humans in order to catch mistakes very early.
This helps quickly checking if the config produces any warning. For
this we reuse the "warned" bit field to add a new WARN_ANY bit that is
set by ha_warning(). The rest of the bit field was also cleaned from
unused bits.
Delete a certificate store from HAProxy and free its memory. The
certificate must be unused and removed from any crt-list or directory.
The deletion doesn't work with a certificate referenced directly with
the "crt" directive in the configuration.
The dump and show ssl crt-list commands does the same thing, they dump
the content of a crt-list, but the 'show' displays an ID in the first
column. Delete the 'dump' command so it is replaced by the 'show' one.
The old 'show' command is replaced by an '-n' option to dump the ID.
And the ID which was a pointer is replaced by a line number and placed
after colons in the filename.
Example:
$ echo "show ssl crt-list -n kikyo.crt-list" | socat /tmp/sock1 -
# kikyo.crt-list
kikyo.pem.rsa:1 secure.domain.tld
kikyo.pem.ecdsa:2 secure.domain.tld
Delete an entry in a crt-list, this is done by iterating over the
ckch_inst in the crtlist_entry. For each ckch_inst the bind_conf lock is
held during the deletion of the sni_ctx in the SNI trees. Everything
is free'd.
If there is several entries with the same certificate, a line number
must be provided to chose with entry delete.
With server-template was introduced the possibility to scale the
number of servers in a backend without needing a configuration change
and associated reload. On the other hand it became impractical to
write use-server rules for these servers as they would only accept
existing server labels as argument. This patch allows the use of
log-format notation to describe targets of a use-server rules, such
as in the example below:
listen test
bind *:1234
use-server %[hdr(srv)] if { hdr(srv) -m found }
use-server s1 if { path / }
server s1 127.0.0.1:18080
server s2 127.0.0.1:18081
If a use-server rule is applied because it was conditionned by an
ACL returning true, but the target of the use-server rule cannot be
resolved, no other use-server rule is evaluated and we fall back to
load balancing.
This feature was requested on the ML, and bumped with issue #563.
Add a sample fetch for the name of a bind. This can be useful to
take decisions when PROXY protocol is used and we can't rely on dst,
such as the sample config below.
defaults
mode http
listen bar
bind 127.0.0.1:1111
server s1 127.0.1.1:1234 send-proxy
listen foo
bind 127.0.1.1:1234 name foo accept-proxy
http-request return status 200 hdr dst %[dst] if { dst 127.0.1.1 }
Released version 2.2-dev5 with the following main changes :
- CLEANUP: ssl: is_default is a bit in ckch_inst
- BUG/MINOR: ssl/cli: sni_ctx' mustn't always be used as filters
- DOC: ssl: clarify security implications of TLS tickets
- CLEANUP: remove support for Linux i686 vsyscalls
- CLEANUP: drop support for USE_MY_ACCEPT4
- CLEANUP: remove support for USE_MY_EPOLL
- CLEANUP: remove support for USE_MY_SPLICE
- CLEANUP: remove the now unused common/syscall.h
- BUILD: make dladdr1 depend on glibc version and not __USE_GNU
- BUILD: wdt: only test for SI_TKILL when compiled with thread support
- BUILD: Makefile: the compiler-specific flags should all be in SPEC_CFLAGS
- CLEANUP: ssl: separate the directory loading in a new function
- BUG/MINOR: buffers: MT_LIST_DEL_SAFE() expects the temporary pointer.
- BUG/MEDIUM: mt_lists: Make sure we set the deleted element to NULL;
- MINOR: init: move the maxsock calculation code to compute_ideal_maxsock()
- MEDIUM: init: always try to push the FD limit when maxconn is set from -m
- BUG/MAJOR: list: fix invalid element address calculation
- BUILD: stream-int: fix a few includes dependencies
- MINOR: mt_lists: Appease gcc.
- MINOR: lists: Implement function to convert list => mt_list and mt_list => list
- MINOR: servers: Kill priv_conns.
- MINOR: lists: fix indentation.
- BUG/MEDIUM: random: align the state on 2*64 bits for ARM64
- BUG/MEDIUM: connections: Don't assume the connection has a valid session.
- BUG/MEDIUM: pools: Always update free_list in pool_gc().
- BUG/MINOR: haproxy: always initialize sleeping_thread_mask
- BUG/MINOR: listener/mq: do not dispatch connections to remote threads when stopping
- BUG/MINOR: haproxy/threads: try to make all threads leave together
- Revert "BUILD: travis-ci: enable s390x builds"
- BUILD: travis-ci: enable regular s390x builds
- DOC: proxy_protocol: Reserve TLV type 0x05 as PP2_TYPE_UNIQUE_ID
- MINOR: proxy_protocol: Ingest PP2_TYPE_UNIQUE_ID on incoming connections
- MEDIUM: proxy_protocol: Support sending unique IDs using PPv2
- CLEANUP: connection: Add blank line after declarations in PP handling
- CLEANUP: assorted typo fixes in the code and comments
- CI: add spellcheck github action
- DOC: correct typo in alert message about rspirep
- CI: travis: switch linux builds to clang-9
- MINOR: debug: add a new DISGUISE() macro to pass a value as identity
- MINOR: debug: consume the write() result in BUG_ON() to silence a warning
- MINOR: use DISGUISE() everywhere we deliberately want to ignore a result
- BUILD: pools: silence build warnings with DEBUG_MEMORY_POOLS and DEBUG_UAF
- CLEANUP: connection: Stop directly setting an ist's .ptr
- CI: travis: revert to clang-7 for BoringSSL tests
- BUILD: on ARM, must be linked to libatomic.
- BUILD: makefile: fix regex syntax in ARM platform detection
- BUG/MEDIUM: peers: resync ended with RESYNC_PARTIAL in wrong cases.
- REORG: ssl: move ssl_sock_load_cert()
- MINOR: ssl: pass ckch_inst to ssl_sock_load_ckchs()
- MEDIUM: ssl: allow crt-list caching
- MINOR: ssl: directories are loaded like crt-list
- BUG/MINOR: ssl: can't open directories anymore
- BUG/MEDIUM: spoe: dup agent's engine_id string from trash.area
- MINOR: fd: Use a separate lock for logs instead of abusing the fd lock.
- MINOR: mux_pt: Don't try to remove the connection from the idle list.
- MINOR: ssl/cli: show/dump ssl crt-list
- BUG/MINOR: ssl/cli: free the trash chunk in dump_crtlist
- MEDIUM: fd: Introduce a running mask, and use it instead of the spinlock.
- BUG/MINOR: ssl: memory leak in crtlist_parse_file()
- MINOR: tasks: Provide the tasklet to the callback.
- BUG/MINOR: ssl: memleak of struct crtlist_entry
- BUG/MINOR: pattern: Do not pass len = 0 to calloc()
- BUILD: makefile: fix expression again to detect ARM platform
- CI: travis: re-enable ASAN on clang
- CI: travis: proper group output redirection together with travis_wait
- DOC: assorted typo fixes in the documentation
- MINOR: wdt: Move the definitions of WDTSIG and DEBUGSIG into types/signal.h.
- BUG/MEDIUM: wdt: Don't ignore WDTSIG and DEBUGSIG in __signal_process_queue().
- MINOR: memory: Change the flush_lock to a spinlock, and don't get it in alloc.
- MINOR: ssl/cli: 'new ssl cert' command
- MINOR: ssl/cli: show certificate status in 'show ssl cert'
- MEDIUM: sessions: Don't be responsible for connections anymore.
- MEDIUM: servers: Split the connections into idle, safe, and available.
- MINOR: fd: Implement fd_takeover().
- MINOR: connections: Add a new mux method, "takeover".
- MINOR: connections: Make the "list" element a struct mt_list instead of list.
- MINOR: connections: Add a flag to know if we're in the safe or idle list.
- MEDIUM: connections: Attempt to get idle connections from other threads.
- MEDIUM: mux_h1: Implement the takeover() method.
- MEDIUM: mux_h2: Implement the takeover() method.
- MEDIUM: mux_fcgi: Implement the takeover() method.
- MEDIUM: connections: Kill connections even if we are reusing one.
- BUG/MEDIUM: connections: Don't forget to decrement idle connection counters.
- BUG/MINOR: ssl: Do not free garbage pointers on memory allocation failure
- BUG/MINOR: ssl: Correctly add the 1 for the sentinel to the number of elements
- BUG/MINOR: ssl: crtlist_dup_filters() must return NULL with fcount == 0
- BUG/MEDIUM: build: Fix compilation by spelling decl correctly.
- BUILD/MEDIUM: fd: Declare fd_mig_lock as extern.
- CI: run travis-ci builds on push only, skip pull requests
- CI: temporarily disable unstable travis arm64 builds
- BUG/MINOR: ssl/cli: free BIO upon error in 'show ssl cert'
- BUG/MINOR: connections: Make sure we free the connection on failure.
- BUG/MINOR: ssl/cli: fix a potential NULL dereference
- BUG/MEDIUM: h1: Make sure we subscribe before going into idle list.
- BUG/MINOR: connections: Set idle_time before adding to idle list.
- MINOR: muxes: Note that we can't usee a connection when added to the srv idle.
- REGTEST: increase timeouts on the seamless-reload test
- BUG/MINOR: haproxy/threads: close a possible race in soft-stop detection
- CLEANUP: haproxy/threads: don't check global_tasks_mask twice
This patch adds the `unique-id` option to `proxy-v2-options`. If this
option is set a unique ID will be generated based on the `unique-id-format`
while sending the proxy protocol v2 header and stored as the unique id for
the first stream of the connection.
This feature is meant to be used in `tcp` mode. It works on HTTP mode, but
might result in inconsistent unique IDs for the first request on a keep-alive
connection, because the unique ID for the first stream is generated earlier
than the others.
Now that we can send unique IDs in `tcp` mode the `%ID` log variable is made
available in TCP mode.
Clarifies security implications of TLS ticket usage when not
rotating TLS ticket keys, after commit 7b5e136458 ("DOC:
improve description of no-tls-tickets").
Released version 2.2-dev4 with the following main changes :
- MEDIUM: buffer: remove the buffer_wq lock
- MINOR: ssl: move find certificate chain code to its own function
- MINOR: ssl: resolve issuers chain later
- MINOR: ssl: resolve ocsp_issuer later
- MINOR: ssl/cli: "show ssl cert" command should print the "Chain Filename:"
- BUG/MINOR: h2: reject again empty :path pseudo-headers
- MINOR: wdt: always clear sigev_value to make valgrind happy
- MINOR: epoll: always initialize all of epoll_event to please valgrind
- BUG/MINOR: sample: Make sure to return stable IDs in the unique-id fetch
- BUG/MEDIUM: ssl: chain must be initialized with sk_X509_new_null()
- BUILD: cirrus-ci: suppress OS version check when installing packages
- BUG/MINOR: http_ana: make sure redirect flags don't have overlapping bits
- CLEANUP: fd: remove the FD_EV_STATUS aggregate
- CLEANUP: fd: remove some unneeded definitions of FD_EV_* flags
- MINOR: fd: merge the read and write error bits into RW error
- BUG/MINOR: dns: ignore trailing dot
- MINOR: contrib/prometheus-exporter: Add the last heathcheck duration metric
- BUG/MINOR: http-htx: Do case-insensive comparisons on Host header name
- MINOR: mux-h1: Remove useless case-insensitive comparisons
- MINOR: rawsock: always mark the FD not ready when we're certain it happens
- MEDIUM: connection: make the subscribe() call able to wakeup if ready
- MEDIUM: connection: don't stop receiving events in the FD handler
- MEDIUM: mux-h1: do not blindly wake up the tasklet at end of request anymore
- BUG/MINOR: arg: don't reject missing optional args
- MINOR: tools: make sure to correctly check the returned 'ms' in date2std_log
- MINOR: debug: report the task handler's pointer relative to main
- BUG/MEDIUM: debug: make the debug_handler check for the thread in threads_to_dump
- MINOR: haproxy: export main to ease access from debugger
- MINOR: haproxy: export run_poll_loop
- MINOR: task: export run_tasks_from_list
- BUILD: tools: remove obsolete and conflicting trace() from standard.c
- MINOR: tools: add new function dump_addr_and_bytes()
- MINOR: tools: add resolve_sym_name() to resolve function pointers
- MINOR: debug: use resolve_sym_name() to dump task handlers
- MINOR: cli: make "show fd" rely on resolve_sym_name()
- MEDIUM: debug: add support for dumping backtraces of stuck threads
- MINOR: debug: call backtrace() once upon startup
- MINOR: ssl: add "ca-verify-file" directive
- BUG/MINOR: wdt: do not return an error when the watchdog couldn't be enabled
- BUILD: Makefile: include librt before libpthread
- MEDIUM: wdt: fall back to CLOCK_REALTIME if CLOCK_THREAD_CPUTIME is not available
- MINOR: wdt: do not depend on USE_THREAD
- MINOR: debug: report the number of entries in the backtrace
- MINOR: debug: improve backtrace() on aarch64 and possibly other systems
- MINOR: debug: use our own backtrace function on clang+x86_64
- MINOR: debug: dump the whole trace if we can't spot the starting point
- BUILD: tools: unbreak resolve_sym_name() on non-GNU platforms
- BUILD: tools: rely on __ELF__ not USE_DL to enable use of dladdr()
- CLEANUP: contrib/spoa_example: Fix several typos
- BUILD: makefile: do not modify the build options during make reg-tests
- BUG/MEDIUM: connection: stop polling for sending when the event is ready
- MEDIUM: stream-int: make sure to try to immediately validate the connection
- MINOR: tcp/uxst/sockpair: only ask for I/O when really waiting for a connect()
- MEDIUM: connection: only call ->wake() for connect() without I/O
- OPTIM: connection: disable receiving on disabled events when the run queue is too high
- OPTIM: mux-h1: subscribe rather than waking up at a few other places
- REGTEST: Add unique-id reg-test
- MINOR: stream: Add stream_generate_unique_id function
- MINOR: stream: Use stream_generate_unique_id
- BUG/MINOR: connection/debug: do not enforce !event_type on subscribe() anymore
- MINOR: ssl/cli: support crt-list filters
- MINOR: ssl: reach a ckch_store from a sni_ctx
- DOC: fix incorrect indentation of http_auth_*
- BUG/MINOR: ssl-sock: do not return an uninitialized pointer in ckch_inst_sni_ctx_to_sni_filters
- MINOR: debug: add CLI command "debug dev write" to write an arbitrary size
- MINOR: ist: Add `IST_NULL` macro
- MINOR: ist: Add `int isttest(const struct ist)`
- MINOR: ist: Add `struct ist istalloc(size_t)` and `void istfree(struct ist*)`
- CLEANUP: Use `isttest()` and `istfree()`
- MINOR: ist: Add `struct ist istdup(const struct ist)`
- MINOR: proxy: Make `header_unique_id` a `struct ist`
- MEDIUM: stream: Make the `unique_id` member of `struct stream` a `struct ist`
- OPTIM: startup: fast unique_id allocation for acl.
- DOC: configuration.txt: fix various typos
- DOC: assorted typo fixes in the documentation and Makefile
- BUG/MINOR: init: make the automatic maxconn consider the max of soft/hard limits
- BUG/MAJOR: proxy_protocol: Properly validate TLV lengths
- CLEANUP: proxy_protocol: Use `size_t` when parsing TLVs
- MINOR: buf: Add function to insert a string at an absolute offset in a buffer
- MINOR: htx: Add a function to return a block at a specific offset
- MINOR: htx: Use htx_find_offset() to truncate an HTX message
- MINOR: flt_trace: Use htx_find_offset() to get the available payload length
- BUG/MINOR: filters: Use filter offset to decude the amount of forwarded data
- BUG/MINOR: filters: Forward everything if no data filters are called
- BUG/MEDIUM: cache/filters: Fix loop on HTX blocks caching the response payload
- BUG/MEDIUM: compression/filters: Fix loop on HTX blocks compressing the payload
- BUG/MINOR: http-ana: Reset request analysers on a response side error
- BUG/MINOR: lua: Abort when txn:done() is called from a Lua action
- BUG/MINOR: lua: Ignore the reserve to know if a channel is full or not
- MINOR: lua: Add function to know if a channel is a response one
- MINOR: lua: Stop using the lua txn in hlua_http_get_headers()
- MINOR: lua: Stop using the lua txn in hlua_http_rep_hdr()
- MINOR: lua: Stop using lua txn in hlua_http_del_hdr() and hlua_http_add_hdr()
- MINOR: lua: Remove the flag HLUA_TXN_HTTP_RDY
- MINOR: lua: Rename hlua_action_wake_time() to hlua_set_wake_time()
- BUG/MINOR: lua: Init the lua wake_time value before calling a lua function
- BUG/MINOR: http-rules: Return ACT_RET_ABRT to abort a transaction
- BUG/MINOR: http-rules: Preserve FLT_END analyzers on reject action
- BUG/MINOR: http-rules: Fix a typo in the reject action function
- MINOR: cache/filters: Initialize the cache filter when stream is created
- MINOR: compression/filters: Initialize the comp filter when stream is created
- BUG/MINOR: rules: Preserve FLT_END analyzers on silent-drop action
- BUG/MINOR: rules: Return ACT_RET_ABRT when a silent-drop action is executed
- BUG/MINOR: rules: Increment be_counters if backend is assigned for a silent-drop
- BUG/MINOR: http-rules: Abort transaction when a redirect is applied on response
- BUILD: buffer: types/{ring.h,checks.h} should include buf.h, not buffer.h
- BUILD: ssl: include mini-clist.h
- BUILD: global: must not include common/standard.h but only types/freq_ctr.h
- BUILD: freq_ctr: proto/freq_ctr needs to include common/standard.h
- BUILD: listener: types/listener.h must not include standard.h
- BUG/MEDIUM: random: initialize the random pool a bit better
- BUG/MEDIUM: random: implement per-thread and per-process random sequences
- Revert "BUG/MEDIUM: random: implement per-thread and per-process random sequences"
- BUILD: cirrus-ci: get rid of unstable freebsd images
- MINOR: tools: add 64-bit rotate operators
- BUG/MEDIUM: random: implement a thread-safe and process-safe PRNG
- MINOR: backend: use a single call to ha_random32() for the random LB algo
- BUG/MINOR: checks/threads: use ha_random() and not rand()
- MINOR: sample: make all bits random on the rand() sample fetch
- MINOR: tools: add a generic function to generate UUIDs
- DOC: fix typo about no-tls-tickets
- DOC: improve description of no-tls-tickets
- DOC: assorted typo fixes in the documentation
- CLEANUP: remove unused code in 'my_ffsl/my_flsl' functions
It was not obvious, that this setting only affects TLS versions <= 1.2 and it
we should also mention the security implication of session tickets here.
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
It's only available for bind line. "ca-verify-file" allows to separate
CA certificates from "ca-file". CA names sent in server hello message is
only compute from "ca-file". Typically, "ca-file" must be defined with
intermediate certificates and "ca-verify-file" with certificates to
ending the chain, like root CA.
Fix issue #404.
Released version 2.2-dev3 with the following main changes :
- SCRIPTS: announce-release: place the send command in the mail's header
- SCRIPTS: announce-release: allow the user to force to overwrite old files
- SCRIPTS: backport: fix the master branch detection
- BUG/MINOR: http-act: Set stream error flag before returning an error
- BUG/MINOR: http-act: Fix bugs on error path during parsing of return actions
- BUG/MEDIUM: ssl/cli: 'commit ssl cert' wrong SSL_CTX init
- BUG/MEDIUM: tcp-rules: Fix track-sc* actions for L4/L5 TCP rules
- DOC: schematic of the SSL certificates architecture
- BUG/MAJOR: mux-h2: don't wake streams after connection was destroyed
- BUG/MINOR: unix: better catch situations where the unix socket path length is close to the limit
- BUILD: cirrus-ci: switch to "snap" images to unify openssl naming
- BUILD: cirrus-ci: workaround "pkg install" bug
- BUILD: cirrus-ci: add ERR=1 to freebsd builds
- BUG/MINOR: connection: correctly retry I/O on signals
- CLEANUP: mini-clist: simplify nested do { while(1) {} } while (0)
- BUILD: http_act: cast file sizes when reporting file size error
- BUG/MEDIUM: listener: only consider running threads when resuming listeners
- BUG/MINOR: listener: enforce all_threads_mask on bind_thread on init
- BUG/MINOR: tcp: avoid closing fd when socket failed in tcp_bind_listener
- MINOR: build: add aix72-gcc build TARGET and power{8,9} CPUs
- BUILD: travis-ci: no more allowed failures for openssl-1.0.2
- BUILD: travis-ci: harden builds, add ERR=1 (warning ought to be errors)
- BUILD: scripts/build-ssl.sh: use "uname" instead of ${TRAVIS_OS_NAME}
- BUG/MINOR: tcp: don't try to set defaultmss when value is negative
- SCRIPTS: make announce-release executable again
- BUG/MINOR: namespace: avoid closing fd when socket failed in my_socketat
- BUG/MEDIUM: muxes: Use the right argument when calling the destroy method.
- BUG/MINOR: mux-fcgi: Forbid special characters when matching PATH_INFO param
- CLEANUP: ssl: remove unused functions in openssl-compat.h
- MINOR: mux-fcgi: Make the capture of the path-info optional in pathinfo regex
- MINOR: tools: add is_idchar() to tell if a char may belong to an identifier
- MINOR: chunk: implement chunk_strncpy() to copy partial strings
- MINOR: sample/acl: use is_idchar() to locate the fetch/conv name
- MEDIUM: arg: make make_arg_list() stop after its own arguments
- MEDIUM: arg: copy parsed arguments into the trash instead of allocating them
- MEDIUM: arg: make make_arg_list() support quotes in arguments
- MINOR: sample: make sample_parse_expr() able to return an end pointer
- MEDIUM: log-format: make the LF parser aware of sample expressions' end
- BUG/MINOR: arg: report an error if an argument is larger than bufsize
- SCRIPTS: announce-release: use mutt -H instead of -i to include the draft
- BUILD: enable ERR=1 in github cygwin builds
- BUG/MINOR: arg: fix again incorrect argument length check
- MINOR: sample: regsub now supports backreferences
- BUG/MINOR: tools: also accept '+' as a valid character in an identifier
- MINOR: http-htx: Add a function to retrieve the headers size of an HTX message
- MINOR: filters: Forward data only if the last filter forwards something
- BUG/MINOR: filters: Count HTTP headers as filtered data but don't forward them
- BUG/MINOR: http-htx: Don't return error if authority is updated without changes
- BUG/MINOR: stream: Don't incr frontend cum_req counter when stream is closed
- BUG/MINOR: sample: exit regsub() in case of trash allocation error
- MINOR: ssl: add "issuers-chain-path" directive.
- REGTESTS: use "command -v" instead of "which"
- BUG/MINOR: http-ana: Matching on monitor-uri should be case-sensitive
- MINOR: http-ana: Match on the path if the monitor-uri starts by a /
- BUG/MINOR: ssl: Stop passing dynamic strings as format arguments
- BUG/MAJOR: http-ana: Always abort the request when a tarpit is triggered
- BUG/MINOR: mux: do not call conn_xprt_stop_recv() on buffer shortage
- MINOR: checks: do not call conn_xprt_stop_send() anymore
- CLEANUP: epoll: place the struct epoll_event in the stack
- MEDIUM: connection: remove the intermediary polling state from the connection
- MINOR: raw_sock: directly call fd_stop_send() and not conn_xprt_stop_send()
- MINOR: tcp/uxst/sockpair: use fd_want_send() instead of conn_xprt_want_send()
- MINOR: connection: remove the last calls to conn_xprt_{want,stop}_*
- CLEANUP: connection: remove the definitions of conn_xprt_{stop,want}_{send,recv}
- MINOR: connection: introduce a new receive flag: CO_RFL_READ_ONCE
- MINOR: mux-h1: pass CO_RFL_READ_ONCE to the lower layers when relevant
- MINOR: ist: add an iststop() function
- BUG/MINOR: http: http-request replace-path duplicates the query string
- CLEANUP: sample: use iststop instead of a for loop
- BUG/MEDIUM: shctx: make sure to keep all blocks aligned
- MINOR: compiler: move CPU capabilities definition from config.h and complete them
- BUG/MEDIUM: ebtree: don't set attribute packed without unaligned access support
- CLEANUP: http/h1: rely on HA_UNALIGNED_LE instead of checking for CPU families
- BUILD: fix recent build failure on unaligned archs
- MINOR: ssl: load the key from a dedicated file
- BUG/MINOR: ssl: load .key in a directory only after PEM
- MINOR: compiler: drop special cases of likely/unlikely for older compilers
- CLEANUP: conn: Do not pass a pointer to likely
- CLEANUP: net_helper: Do not negate the result of unlikely
- BUILD: remove obsolete support for -mregparm / USE_REGPARM
- CLEANUP: cfgparse: Fix type of second calloc() parameter
- BUILD: ssl: only pass unsigned chars to isspace()
- BUILD: general: always pass unsigned chars to is* functions
- BUG/MINOR: sample: fix the json converter's endian-sensitivity
- BUG/MEDIUM: ssl: fix several bad pointer aliases in a few sample fetch functions
- CLEANUP: fd: use a union in fd_rm_from_fd_list() to shut aliasing warnings
- CLEANUP: cache: use read_u32/write_u32 to access the cache entry's hash
- CLEANUP: stick-tables: use read_u32() to display a node's key
- CLEANUP: sample: use read_u64() in ipmask() to apply an IPv6 mask
- MINOR: pattern: fix all remaining strict aliasing issues
- CLEANUP: lua: fix aliasing issues in the address matching code
- CLEANUP: connection: use read_u32() instead of a cast in the netscaler parser
- BUILD: makefile: re-enable strict aliasing
- BUG/MINOR: connection: make sure to correctly tag local PROXY connections
- MINOR: compiler: add new alignment macros
- BUILD: ebtree: improve architecture-specific alignment
- MINOR: config: mark global.debug as deprecated
- BUILD: travis-ci: enable s390x builds
- MINOR: ssl/cli: 'show ssl cert' displays the chain
- MINOR: ssl/cli: 'show ssl cert'displays the issuer in the chain
- MINOR: ssl/cli: reorder 'show ssl cert' output
- CLEANUP: ssl: move issuer_chain tree and definition
- DOC: proxy-protocol: clarify IPv6 address representation in the spec
Daniel Barclay reported that the wording around "IPv6 addresses must be
indicated as series of 4 hex digits" is confusing and can be interpreted
two ways (only 4 digits or series of sets of 4 digits), so let's adjust
the wording to resolve this ambiguity.
This directive has never made any sense and has already caused trouble
by forcing the process to stay in foreground during the boot process.
Let's emit a warning mentioning it's deprecated and will be removed in
2.3.
Don't try to load a .key in a directory without loading its associated
certificate file.
This patch ignores the .key files when iterating over the files in a
directory.
Introduced by 4c5adbf ("MINOR: ssl: load the key from a dedicated
file").
For a certificate on a bind line, if the private key was not found in
the PEM file, look for a .key and load it.
This default behavior can be changed by using the ssl-load-extra-files
directive in the global section
This feature was mentionned in the issue #221.
if the monitor-uri starts by a slash ('/'), the matching is performed against
the request's path instead of the request's uri. It is a workaround to let the
HTTP/2 requests match the monitor-uri. Indeed, in HTTP/2, clients are encouraged
to send absolute URIs only.
This patch is not tagged as a bug, because the previous behavior matched exactly
what the doc describes. But it may surprise that HTTP/2 requests don't match the
monitor-uri.
This patch may be backported to 2.1 because URIs of HTTP/2 are stored using the
absolute-form starting this version. For previous versions, this patch will only
helps explicitely absolute HTTP/1 requests (and only the HTX part because on the
legacy HTTP, all the URI is matched).
It should fix the issue #509.
Certificates loaded with "crt" and "crt-list" commonly share the same
intermediate certificate in PEM file. "issuers-chain-path" is a global
directive to share intermediate chain certificates in a directory. If
certificates chain is not included in certificate PEM file, haproxy
will complete chain if issuer match the first certificate of the chain
stored via "issuers-chain-path" directive. Such chains will be shared
in memory.
Now that the configuration parser is more flexible with samples,
converters and their arguments, we can leverage this to enable
support for backreferences in regsub.
For a very long time it used to be impossible to pass a closing square
bracket as a valid character in argument to a sample fetch function or
to a converter because the LF parser used to stop on the first such
character found and to pass what was between the first '[' and the first
']' to sample_parse_expr().
This patch addresses this by passing the whole string to sample_parse_expr()
which is the only one authoritative to indicate the first character that
does not belong to the expression. The LF parser then verifies it matches
a ']' or fails. As a result it is finally possible to write rules such as
the following, which is totally valid an unambigous :
http-request redirect location %[url,regsub([.:/?-],!,g)]
|-----| | |
arg1 | `---> arg3
`-----> arg2
|-----------------|
converter
|---------------------|
sample expression
|------------------------|
log-format tag
Now it becomes possible to reuse the quotes within arguments, allowing
the parser to distinguish a ',' or ')' that is part of the value from
one which delimits the argument. In addition, ',' and ')' may be escaped
using a backslash. However, it is also important to keep in mind that
just like in shell, quotes are first resolved by the word tokenizer, so
in order to pass quotes that are visible to the argument parser, a second
level is needed, either using backslash escaping, or by using an alternate
type.
For example, it's possible to write this to append a comma:
http-request add-header paren-comma-paren "%[str('(--,--)')]"
or this:
http-request add-header paren-comma-paren '%[str("(--,--)")]'
or this:
http-request add-header paren-comma-paren %[str(\'(--,--)\')]
or this:
http-request add-header paren-comma-paren %[str(\"(--,--)\")]
or this:
http-request add-header paren-comma-paren %[str(\"(\"--\',\'--\")\")]
Note that due to the wide use of '\' in front of parenthesis in regex,
the backslash character will purposely *not* escape parenthesis, so that
'\)' placed in quotes is passed verbatim to a regex engine.
Now, only one capture is mandatory in the path-info regex, the one matching the
script-name. The path-info capture is optional. Of couse, it must be defined to
fill the PATH_INFO parameter. But it is not mandatory. This way, it is possible
to get the script-name part from the path, excluding the path-info.
This patch is small enough to be backported to 2.1.
If a regex to match the PATH_INFO parameter is configured, it systematically
fails if a newline or a null character is present in the URL-decoded path. So,
from the moment there is at least a "%0a" or a "%00" in the request path, we
always fail to get the PATH_INFO parameter and all the decoded path is used for
the SCRIPT_NAME parameter.
It is probably not the expected behavior. Because, most of time, these
characters are not expected at all in a path, an error is now triggered when one
of these characters is found in the URL-decoded path before trying to execute
the path_info regex. However, this test is not performed if there is no regex
configured.
Note that in reality, the newline character is only a problem when HAProxy is
complied with pcre or pcre2 library and conversely, the null character is only a
problem for the libc's regex library. But both are always excluded to avoid any
inconsistency depending on compile options.
An alternative, not implemented yet, is to replace these characters by another
one. If someone complains about this behavior, it will be re-evaluated.
This patch must be backported to all versions supporting the FastCGI
applications, so to 2.1 for now.
This patch provides a schematic of the new architecture based on the
struct cert_key_and_chain which appeared with haproxy 2.1.
Could be backported in 2.1
Released version 2.2-dev2 with the following main changes :
- BUILD: CI: temporarily mark openssl-1.0.2 as allowed failure
- MEDIUM: cli: Allow multiple filter entries for "show table"
- BUG/MEDIUM: netscaler: Don't forget to allocate storage for conn->src/dst.
- BUG/MINOR: ssl: ssl_sock_load_pem_into_ckch is not consistent
- BUILD: stick-table: fix build errors introduced by last stick-table change
- BUG/MINOR: cli: Missing arg offset for filter data values.
- MEDIUM: streams: Always create a conn_stream in connect_server().
- MEDIUM: connections: Get ride of the xprt_done callback.
- CLEANUP: changelog: remove the duplicate entry for 2.2-dev1
- BUILD: CI: move cygwin builds to Github Actions
- MINOR: cli: Report location of errors or any extra data for "show table"
- BUG/MINOR: ssl/cli: free the previous ckch content once a PEM is loaded
- CLEANUP: backend: remove useless test for inexistent connection
- CLEANUP: backend: shut another false null-deref in back_handle_st_con()
- CLEANUP: stats: shut up a wrong null-deref warning from gcc 9.2
- BUG/MINOR: ssl: increment issuer refcount if in chain
- BUG/MINOR: ssl: memory leak w/ the ocsp_issuer
- BUG/MINOR: ssl: typo in previous patch
- BUG/MEDIUM: connections: Set CO_FL_CONNECTED in conn_complete_session().
- BUG/MINOR: ssl/cli: ocsp_issuer must be set w/ "set ssl cert"
- MEDIUM: connection: remove CO_FL_CONNECTED and only rely on CO_FL_WAIT_*
- BUG/MEDIUM: 0rtt: Only consider the SSL handshake.
- MINOR: stream-int: always report received shutdowns
- MINOR: connection: remove CO_FL_SSL_WAIT_HS from CO_FL_HANDSHAKE
- MEDIUM: connection: use CO_FL_WAIT_XPRT more consistently than L4/L6/HANDSHAKE
- MINOR: connection: remove checks for CO_FL_HANDSHAKE before I/O
- MINOR: connection: do not check for CO_FL_SOCK_RD_SH too early
- MINOR: connection: don't check for CO_FL_SOCK_WR_SH too early in handshakes
- MINOR: raw-sock: always check for CO_FL_SOCK_WR_SH before sending
- MINOR: connection: remove some unneeded checks for CO_FL_SOCK_WR_SH
- BUG/MINOR: stktable: report the current proxy name in error messages
- BUG/MEDIUM: mux-h2: make sure we don't emit TE headers with anything but "trailers"
- MINOR: lua: Add hlua_prepend_path function
- MINOR: lua: Add lua-prepend-path configuration option
- MINOR: lua: Add HLUA_PREPEND_C?PATH build option
- BUILD: cfgparse: silence a bogus gcc warning on 32-bit machines
- BUG/MINOR: http-ana: Increment the backend counters on the backend
- BUG/MINOR: stream: Be sure to have a listener to increment its counters
- BUG/MEDIUM: streams: Move the conn_stream allocation outside #IF USE_OPENSSL.
- REGTESTS: make the set_ssl_cert test require version 2.2
- BUG/MINOR: ssl: Possible memleak when allowing the 0RTT data buffer.
- MINOR: ssl: Remove dead code.
- BUG/MEDIUM: ssl: Don't forget to free ctx->ssl on failure.
- BUG/MEDIUM: stream: Don't install the mux in back_handle_st_con().
- MEDIUM: streams: Don't close the connection in back_handle_st_con().
- MEDIUM: streams: Don't close the connection in back_handle_st_rdy().
- BUILD: CI: disable slow regtests on Travis
- BUG/MINOR: tcpchecks: fix the connect() flags regarding delayed ack
- BUG/MINOR: http-rules: Always init log-format expr for common HTTP actions
- BUG/MINOR: connection: fix ip6 dst_port copy in make_proxy_line_v2
- BUG/MINOR: dns: allow 63 char in hostname
- MINOR: proxy: clarify number of connections log when stopping
- DOC: word converter ignores delimiters at the start or end of input string
- MEDIUM: raw-sock: remove obsolete calls to fd_{cant,cond,done}_{send,recv}
- BUG/MINOR: ssl/cli: fix unused variable with openssl < 1.0.2
- MEDIUM: pipe/thread: reduce the locking overhead
- MEDIUM: pipe/thread: maintain a per-thread local cache of recently used pipes
- BUG/MEDIUM: pipe/thread: fix atomicity of pipe counters
- MINOR: tasks: move the list walking code to its own function
- MEDIUM: tasks: implement 3 different tasklet classes with their own queues
- MEDIUM: tasks: automatically requeue into the bulk queue an already running tasklet
- OPTIM: task: refine task classes default CPU bandwidth ratios
- BUG/MEDIUM: connections: Don't forget to unlock when killing a connection.
- MINOR: task: permanently flag tasklets waking themselves up
- MINOR: task: make sched->current also reflect tasklets
- MINOR: task: detect self-wakeups on tl==sched->current instead of TASK_RUNNING
- OPTIM: task: readjust CPU bandwidth distribution since last update
- MINOR: task: don't set TASK_RUNNING on tasklets
- BUG/MEDIUM: memory_pool: Update the seq number in pool_flush().
- MINOR: memory: Only init the pool spinlock once.
- BUG/MEDIUM: memory: Add a rwlock before freeing memory.
- BUG/MAJOR: memory: Don't forget to unlock the rwlock if the pool is empty.
- MINOR: ssl: ssl-load-extra-files configure loading of files
- SCRIPTS: add a new "backport" script to simplify long series of backports
- BUG/MINOR: ssl: we may only ignore the first 64 errors
- SCRIPTS: use /usr/bin/env bash instead of /bin/bash for scripts
- BUG/MINOR: ssl: clear the SSL errors on DH loading failure
- CLEANUP: hpack: remove a redundant test in the decoder
- CLEANUP: peers: Remove unused static function `free_dcache`
- CLEANUP: peers: Remove unused static function `free_dcache_tx`
- CONTRIB: debug: add missing flags SF_HTX and SF_MUX
- CONTRIB: debug: add the possibility to decode the value as certain types only
- CONTRIB: debug: support reporting multiple values at once
- BUG/MINOR: http-act: Use the good message to test strict rewritting mode
- MINOR: global: Set default tune.maxrewrite value during global structure init
- MINOR: http-rules: Set SF_ERR_PRXCOND termination flag when a header rewrite fails
- MINOR: http-htx: Emit a warning if an error file runs over the buffer's reserve
- MINOR: htx: Add a function to append an HTX message to another one
- MINOR: htx/channel: Add a function to copy an HTX message in a channel's buffer
- BUG/MINOR: http-ana: Don't overwrite outgoing data when an error is reported
- MINOR: dns: Dynamically allocate dns options to reduce the act_rule size
- MINOR: dns: Add function to release memory allocated for a do-resolve rule
- BUG/MINOR: http-ana: Reset HTX first index when HAPRoxy sends a response
- BUG/MINOR: http-ana: Set HTX_FL_PROXY_RESP flag if a server perform a redirect
- MINOR: http-rules: Add a flag on redirect rules to know the rule direction
- MINOR: http-rules: Handle the rule direction when a redirect is evaluated
- MINOR: http-ana: Rely on http_reply_and_close() to handle server error
- MINOR: http-ana: Add a function for forward internal responses
- MINOR: http-ana/http-rules: Use dedicated function to forward internal responses
- MEDIUM: http: Add a ruleset evaluated on all responses just before forwarding
- MEDIUM: http-rules: Add the return action to HTTP rules
- MEDIUM: http-rules: Support extra headers for HTTP return actions
- CLEANUP: lua: Remove consistency check for sample fetches and actions
- BUG/MINOR: http-ana: Increment failed_resp counters on invalid response
- MINOR: lua: Get the action return code on the stack when an action finishes
- MINOR: lua: Create the global 'act' object to register all action return codes
- MINOR: lua: Add act:wake_time() function to set a timeout when an action yields
- MEDIUM: lua: Add ability for actions to intercept HTTP messages
- REGTESTS: Add reg tests for the HTTP return action
- REGTESTS: Add a reg test for http-after-response rulesets
- BUILD: lua: silence a warning on systems where longjmp is not marked as noreturn
- MINOR: acl: Warn when an ACL is named 'or'
- CONTRIB: debug: also support reading values from stdin
- SCRIPTS: backport: use short revs and resolve the initial commit
- BUG/MINOR: acl: Fix type of log message when an acl is named 'or'
It is now possible to intercept HTTP messages from a lua action and reply to
clients. To do so, a reply object must be provided to the function
txn:done(). It may contain a status code with a reason, a header list and a
body. By default, if an empty reply object is used, an empty 200 response is
returned. If no reply is passed when txn:done() is called, the previous
behaviour is respected, the transaction is terminated and nothing is returned to
the client. The same is done for TCP streams. When txn:done() is called, the
action is terminated with the code ACT_RET_DONE on success and ACT_RET_ERR on
error, interrupting the message analysis.
The reply object may be created for the lua, by hand. Or txn:reply() may be
called. If so, this object provides some methods to fill it:
* Reply:set_status(<status> [ <reason>]) : Set the status and optionally the
reason. If no reason is provided, the default one corresponding to the status
code is used.
* Reply:add_header(<name>, <value>) : Add a header. For a given name, the
values are stored in an ordered list.
* Reply:del_header(<name>) : Removes all occurrences of a header name.
* Reply:set_body(<body>) : Set the reply body.
Here are some examples, all doing the same:
-- ex. 1
txn:done{
status = 400,
reason = "Bad request",
headers = {
["content-type"] = { "text/html" },
["cache-control"] = { "no-cache", "no-store" },
},
body = "<html><body><h1>invalid request<h1></body></html>"
}
-- ex. 2
local reply = txn:reply{
status = 400,
reason = "Bad request",
headers = {
["content-type"] = { "text/html" },
["cache-control"] = { "no-cache", "no-store" }
},
body = "<html><body><h1>invalid request<h1></body></html>"
}
txn:done(reply)
-- ex. 3
local reply = txn:reply()
reply:set_status(400, "Bad request")
reply:add_header("content-length", "text/html")
reply:add_header("cache-control", "no-cache")
reply:add_header("cache-control", "no-store")
reply:set_body("<html><body><h1>invalid request<h1></body></html>")
txn:done(reply)
This function may be used to defined a timeout when a lua action returns
act:YIELD. It is a way to force to reexecute the script after a short time
(defined in milliseconds).
Unlike core:sleep() or core:yield(), the script is fully reexecuted if it
returns act:YIELD. With core functions to yield, the script is interrupted and
restarts from the yield point. When a script returns act:YIELD, it is finished
but the message analysis is blocked on the action waiting its end.
ACT_RET_* code are now available from lua scripts. The gloabl object "act" is
used to register these codes as constant. Now, lua actions can return any of
following codes :
* act.CONTINUE for ACT_RET_CONT
* act.STOP for ACT_RET_STOP
* act.YIELD for ACT_RET_YIELD
* act.ERROR for ACT_RET_ERR
* act.DONE for ACT_RET_DONE
* act.DENY for ACT_RET_DENY
* act.ABORT for ACT_RET_ABRT
* act.INVALID for ACT_RET_INV
For instance, following script denied all requests :
core.register_action("deny", { "http-req" }, function (txn)
return act.DENY
end)
Thus "http-request lua.deny" do exactly the same than "http-request deny".
It is now possible to append extra headers to the generated responses by HTTP
return actions, while it is not based on an errorfile. For return actions based
on errorfiles, these extra headers are ignored. To define an extra header, a
"hdr" argument must be used with a name and a value. The value is a log-format
string. For instance:
http-request status 200 hdr "x-src" "%[src]" hdr "x-dst" "%[dst]"
Thanks to this new action, it is now possible to return any responses from
HAProxy, with any status code, based on an errorfile, a file or a string. Unlike
the other internal messages generated by HAProxy, these ones are not interpreted
as errors. And it is not necessary to use a file containing a full HTTP
response, although it is still possible. In addition, using a log-format string
or a log-format file, it is possible to have responses with a dynamic
content. This action can be used on the request path or the response path. The
only constraint is to have a responses smaller than a buffer. And to avoid any
warning the buffer space reserved to the headers rewritting should also be free.
When a response is returned with a file or a string as payload, it only contains
the content-length header and the content-type header, if applicable. Here are
examples:
http-request return content-type image/x-icon file /var/www/favicon.ico \
if { path /favicon.ico }
http-request return status 403 content-type text/plain \
lf-string "Access denied. IP %[src] is blacklisted." \
if { src -f /etc/haproxy/blacklist.lst }
This patch introduces the 'http-after-response' rules. These rules are evaluated
at the end of the response analysis, just before the data forwarding, on ALL
HTTP responses, the server ones but also all responses generated by
HAProxy. Thanks to this ruleset, it is now possible for instance to add some
headers to the responses generated by the stats applet. Following actions are
supported :
* allow
* add-header
* del-header
* replace-header
* replace-value
* set-header
* set-status
* set-var
* strict-mode
* unset-var
This new setting in the global section alters the way HAProxy will look
for unspecified files (.ocsp, .sctl, .issuer, bundles) during the
loading of the SSL certificates.
By default, HAProxy discovers automatically a lot of files not specified
in the configuration, and you may want to disable this behavior if you
want to optimize the startup time.
This patch sets flags in global_ssl.extra_files and then check them
before trying to load an extra file.
The comments for match_word() in pattern.c mention that delimiters
at the start or end of the input string will be ignored, but this
is not mentionned in the documentation.
Backport to all supported versions.
lua-prepend-path allows the administrator to specify a custom Lua library
path to load custom Lua modules that are useful within the context of HAProxy
without polluting the global Lua library folder.
For complex stick tables with many entries/columns, it can be beneficial
to filter using multiple criteria. The maximum number of filter entries
can be controlled by defining STKTABLE_FILTER_LEN during build time.
This patch can be backported to older releases.
Released version 2.2-dev1 with the following main changes :
- DOC: this is development again
- MINOR: version: this is development again, update the status
- SCRIPTS: update create-release to fix the changelog on new branches
- CLEANUP: ssl: Clean up error handling
- BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only
- BUG/MINOR: h1: Don't test the host header during response parsing
- BUILD/MINOR: trace: fix use of long type in a few printf format strings
- DOC: Clarify behavior of server maxconn in HTTP mode
- MINOR: ssl: deduplicate ca-file
- MINOR: ssl: compute ca-list from deduplicate ca-file
- MINOR: ssl: deduplicate crl-file
- CLEANUP: dns: resolution can never be null
- BUG/MINOR: http-htx: Don't make http_find_header() fail if the value is empty
- DOC: ssl/cli: set/commit/abort ssl cert
- BUG/MINOR: ssl: fix SSL_CTX_set1_chain compatibility for openssl < 1.0.2
- BUG/MINOR: fcgi-app: Make the directive pass-header case insensitive
- BUG/MINOR: stats: Fix HTML output for the frontends heading
- BUG/MINOR: ssl: fix X509 compatibility for openssl < 1.1.0
- DOC: clarify matching strings on binary fetches
- DOC: Fix ordered list in summary
- DOC: move the "group" keyword at the right place
- MEDIUM: init: prevent process and thread creation at runtime
- BUG/MINOR: ssl/cli: 'ssl cert' cmd only usable w/ admin rights
- BUG/MEDIUM: stream-int: don't subscribed for recv when we're trying to flush data
- BUG/MINOR: stream-int: avoid calling rcv_buf() when splicing is still possible
- BUG/MINOR: ssl/cli: don't overwrite the filters variable
- BUG/MEDIUM: listener/thread: fix a race when pausing a listener
- BUG/MINOR: ssl: certificate choice can be unexpected with openssl >= 1.1.1
- BUG/MEDIUM: mux-h1: Never reuse H1 connection if a shutw is pending
- BUG/MINOR: mux-h1: Don't rely on CO_FL_SOCK_RD_SH to set H1C_F_CS_SHUTDOWN
- BUG/MINOR: mux-h1: Fix conditions to know whether or not we may receive data
- BUG/MEDIUM: tasks: Make sure we switch wait queues in task_set_affinity().
- BUG/MEDIUM: checks: Make sure we set the task affinity just before connecting.
- MINOR: debug: replace popen() with pipe+fork() in "debug dev exec"
- MEDIUM: init: set NO_NEW_PRIVS by default when supported
- BUG/MINOR: mux-h1: Be sure to set CS_FL_WANT_ROOM when EOM can't be added
- BUG/MEDIUM: mux-fcgi: Handle cases where the HTX EOM block cannot be inserted
- BUG/MINOR: proxy: make soft_stop() also close FDs in LI_PAUSED state
- BUG/MINOR: listener/threads: always use atomic ops to clear the FD events
- BUG/MINOR: listener: also clear the error flag on a paused listener
- BUG/MEDIUM: listener/threads: fix a remaining race in the listener's accept()
- MINOR: listener: make the wait paths cleaner and more reliable
- MINOR: listener: split dequeue_all_listener() in two
- REORG: listener: move the global listener queue code to listener.c
- DOC: document the listener state transitions
- BUG/MEDIUM: kqueue: Make sure we report read events even when no data.
- BUG/MAJOR: dns: add minimalist error processing on the Rx path
- BUG/MEDIUM: proto_udp/threads: recv() and send() must not be exclusive.
- DOC: listeners: add a few missing transitions
- BUG/MINOR: tasks: only requeue a task if it was already in the queue
- MINOR: tasks: split wake_expired_tasks() in two parts to avoid useless wakeups
- DOC: proxies: HAProxy only supports 3 connection modes
- DOC: remove references to the outdated architecture.txt
- BUG/MINOR: log: fix minor resource leaks on logformat error path
- BUG/MINOR: mworker: properly pass SIGTTOU/SIGTTIN to workers
- BUG/MINOR: listener: do not immediately resume on transient error
- BUG/MINOR: server: make "agent-addr" work on default-server line
- BUG/MINOR: listener: fix off-by-one in state name check
- BUILD/MINOR: unix sockets: silence an absurd gcc warning about strncpy()
- MEDIUM: h1-htx: Add HTX EOM block when the message is in H1_MSG_DONE state
- MINOR: http-htx: Add some htx sample fetches for debugging purpose
- REGTEST: Add an HTX reg-test to check an edge case
- DOC: clarify the fact that replace-uri works on a full URI
- BUG/MINOR: sample: fix the closing bracket and LF in the debug converter
- BUG/MINOR: sample: always check converters' arguments
- MINOR: sample: Validate the number of bits for the sha2 converter
- BUG/MEDIUM: ssl: Don't set the max early data we can receive too early.
- MINOR: ssl/cli: 'show ssl cert' give information on the certificates
- BUG/MINOR: ssl/cli: fix build for openssl < 1.0.2
- MINOR: debug: support logging to various sinks
- MINOR: http: add a new "replace-path" action
- REGTEST: ssl: test the "set ssl cert" CLI command
- REGTEST: run-regtests: implement #REQUIRE_BINARIES
- MINOR: task: only check TASK_WOKEN_ANY to decide to requeue a task
- BUG/MAJOR: task: add a new TASK_SHARED_WQ flag to fix foreing requeuing
- BUG/MEDIUM: ssl: Revamp the way early data are handled.
- MINOR: fd/threads: make _GET_NEXT()/_GET_PREV() use the volatile attribute
- BUG/MEDIUM: fd/threads: fix a concurrency issue between add and rm on the same fd
- REGTEST: make the "set ssl cert" require version 2.1
- BUG/MINOR: ssl: openssl-compat: Fix getm_ defines
- BUG/MEDIUM: state-file: do not allocate a full buffer for each server entry
- BUG/MINOR: state-file: do not store duplicates in the global tree
- BUG/MINOR: state-file: do not leak memory on parse errors
- BUG/MAJOR: mux-h1: Don't pretend the input channel's buffer is full if empty
- BUG/MEDIUM: stream: Be sure to never assign a TCP backend to an HTX stream
- BUILD: ssl: improve SSL_CTX_set_ecdh_auto compatibility
- BUILD: travis-ci: link with ssl libraries using rpath instead of LD_LIBRARY_PATH/DYLD_LIBRARY_PATH
- BUILD: travis-ci: reenable address sanitizer for clang builds
- BUG/MINOR: checks: refine which errno values are really errors.
- BUG/MINOR: connection: only wake send/recv callbacks if the FD is active
- CLEANUP: connection: conn->xprt is never NULL
- MINOR: pollers: add a new flag to indicate pollers reporting ERR & HUP
- MEDIUM: tcp: make tcp_connect_probe() consider ERR/HUP
- REORG: connection: move tcp_connect_probe() to conn_fd_check()
- MINOR: connection: check for connection validation earlier
- MINOR: connection: remove the double test on xprt_done_cb()
- CLEANUP: connection: merge CO_FL_NOTIFY_DATA and CO_FL_NOTIFY_DONE
- MINOR: poller: do not call the IO handler if the FD is not active
- OPTIM: epoll: always poll for recv if neither active nor ready
- OPTIM: polling: do not create update entries for FD removal
- BUG/MEDIUM: checks: Only attempt to do handshakes if the connection is ready.
- BUG/MEDIUM: connections: Hold the lock when wanting to kill a connection.
- BUILD: CI: modernize cirrus-ci
- MINOR: config: disable busy polling on old processes
- MINOR: ssl: Remove unused variable "need_out".
- BUG/MINOR: h1: Report the right error position when a header value is invalid
- BUG/MINOR: proxy: Fix input data copy when an error is captured
- BUG/MEDIUM: http-ana: Truncate the response when a redirect rule is applied
- BUG/MINOR: channel: inject output data at the end of output
- BUG/MEDIUM: session: do not report a failure when rejecting a session
- MEDIUM: dns: implement synchronous send
- MINOR: raw_sock: make sure to disable polling once everything is sent
- MINOR: http: Add 410 to http-request deny
- MINOR: http: Add 404 to http-request deny
- CLEANUP: mux-h2: remove unused goto "out_free_h2s"
- BUILD: cirrus-ci: choose proper openssl package name
- BUG/MAJOR: listener: do not schedule a task-less proxy
- CLEANUP: server: remove unused err section in server_finalize_init
- REGTEST: set_ssl_cert.vtc: replace "echo" with "printf"
- BUG/MINOR: stream-int: Don't trigger L7 retry if max retries is already reached
- BUG/MEDIUM: tasks: Use the MT macros in tasklet_free().
- BUG/MINOR: mux-h2: use a safe list_for_each_entry in h2_send()
- BUG/MEDIUM: mux-h2: fix missing test on sending_list in previous patch
- CLEANUP: ssl: remove opendir call in ssl_sock_load_cert
- MEDIUM: lua: don't call the GC as often when dealing with outgoing connections
- BUG/MEDIUM: mux-h2: don't stop sending when crossing a buffer boundary
- BUG/MINOR: cli/mworker: can't start haproxy with 2 programs
- REGTEST: mcli/mcli_start_progs: start 2 programs
- BUG/MEDIUM: mworker: remain in mworker mode during reload
- DOC: clarify crt-base usage
- CLEANUP: compression: remove unused deinit_comp_ctx section
- BUG/MEDIUM: mux_h1: Don't call h1_send if we subscribed().
- BUG/MEDIUM: raw_sock: Make sur the fd and conn are sync.
- CLEANUP: proxy: simplify proxy_parse_rate_limit proxy checks
- BUG/MAJOR: hashes: fix the signedness of the hash inputs
- REGTEST: add sample_fetches/hashes.vtc to validate hashes
- BUG/MEDIUM: cli: _getsocks must send the peers sockets
- CLEANUP: cli: deduplicate the code in _getsocks
- BUG/MINOR: stream: don't mistake match rules for store-request rules
- BUG/MEDIUM: connection: add a mux flag to indicate splice usability
- BUG/MINOR: pattern: handle errors from fgets when trying to load patterns
- MINOR: connection: move the CO_FL_WAIT_ROOM cleanup to the reader only
- MINOR: stream-int: remove dependency on CO_FL_WAIT_ROOM for rcv_buf()
- MEDIUM: connection: get rid of CO_FL_CURR_* flags
- BUILD: pattern: include errno.h
- MEDIUM: mux-h2: do not try to stop sending streams on blocked mux
- MEDIUM: mux-fcgi: do not try to stop sending streams on blocked mux
- MEDIUM: mux-h2: do not make an h2s subscribe to itself on deferred shut
- MEDIUM: mux-fcgi: do not make an fstrm subscribe to itself on deferred shut
- REORG: stream/backend: move backend-specific stuff to backend.c
- MEDIUM: backend: move the connection finalization step to back_handle_st_con()
- MEDIUM: connection: merge the send_wait and recv_wait entries
- MEDIUM: xprt: merge recv_wait and send_wait in xprt_handshake
- MEDIUM: ssl: merge recv_wait and send_wait in ssl_sock
- MEDIUM: mux-h1: merge recv_wait and send_wait
- MEDIUM: mux-h2: merge recv_wait and send_wait event notifications
- MEDIUM: mux-fcgi: merge recv_wait and send_wait event notifications
- MINOR: connection: make the last arg of subscribe() a struct wait_event*
- MINOR: ssl: Add support for returning the dn samples from ssl_(c|f)_(i|s)_dn in LDAP v3 (RFC2253) format.
- DOC: Fix copy and paste mistake in http-response replace-value doc
- BUG/MINOR: cache: Fix leak of cache name in error path
- BUG/MINOR: dns: Make dns_query_id_seed unsigned
- BUG/MINOR: 51d: Fix bug when HTX is enabled
- MINOR: http-htx: Move htx sample fetches in the scope "internal"
- MINOR: http-htx: Rename 'internal.htx_blk.val' to 'internal.htx_blk.data'
- MINOR: http-htx: Make 'internal.htx_blk_data' return a binary string
- DOC: Add a section to document the internal sample fetches
- MINOR: mux-h1: Inherit send flags from the upper layer
- MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics
- BUG/MINOR: http-ana/filters: Wait end of the http_end callback for all filters
- BUG/MINOR: http-rules: Remove buggy deinit functions for HTTP rules
- BUG/MINOR: stick-table: Use MAX_SESS_STKCTR as the max track ID during parsing
- MEDIUM: http-rules: Register an action keyword for all http rules
- MINOR: tcp-rules: Always set from which ruleset a rule comes from
- MINOR: actions: Use ACT_RET_CONT code to ignore an error from a custom action
- MINOR: tcp-rules: Kill connections when custom actions return ACT_RET_ERR
- MINOR: http-rules: Return an error when custom actions return ACT_RET_ERR
- MINOR: counters: Add a counter to report internal processing errors
- MEDIUM: http-ana: Properly handle internal processing errors
- MINOR: http-rules: Add a rule result to report internal error
- MINOR: http-rules: Handle internal errors during HTTP rules evaluation
- MINOR: http-rules: Add more return codes to let custom actions act as normal ones
- MINOR: tcp-rules: Handle denied/aborted/invalid connections from TCP rules
- MINOR: http-rules: Handle denied/aborted/invalid connections from HTTP rules
- MINOR: stats: Report internal errors in the proxies/listeners/servers stats
- MINOR: contrib/prometheus-exporter: Export internal errors per proxy/server
- MINOR: counters: Remove failed_secu counter and use denied_resp instead
- MINOR: counters: Review conditions to increment counters from analysers
- MINOR: http-ana: Add a txn flag to support soft/strict message rewrites
- MINOR: http-rules: Handle all message rewrites the same way
- MINOR: http-rules: Add a rule to enable or disable the strict rewriting mode
- MEDIUM: http-rules: Enable the strict rewriting mode by default
- REGTEST: Fix format of set-uri HTTP request rule in h1or2_to_h1c.vtc
- MINOR: actions: Add a function pointer to release args used by actions
- MINOR: actions: Regroup some info about HTTP rules in the same struct
- MINOR: http-rules/tcp-rules: Call the defined action function first if defined
- MINOR: actions: Rename the act_flag enum into act_opt
- MINOR: actions: Add flags to configure the action behaviour
- MINOR: actions: Use an integer to set the action type
- MINOR: http-rules: Use a specific action type for some custom HTTP actions
- MINOR: http-rules: Make replace-header and replace-value custom actions
- MINOR: http-rules: Make set-header and add-header custom actions
- MINOR: http-rules: Make set/del-map and add/del-acl custom actions
- MINOR: http-rules: Group all processing of early-hint rule in its case clause
- MEDIUM: http-rules: Make early-hint custom actions
- MINOR: http-rule/tcp-rules: Make track-sc* custom actions
- MINOR: tcp-rules: Make tcp-request capture a custom action
- MINOR: http-rules: Add release functions for existing HTTP actions
- BUG/MINOR: http-rules: Fix memory releases on error path during action parsing
- MINOR: tcp-rules: Add release functions for existing TCP actions
- BUG/MINOR: tcp-rules: Fix memory releases on error path during action parsing
- MINOR: http-htx: Add functions to read a raw error file and convert it in HTX
- MINOR: http-htx: Add functions to create HTX redirect message
- MINOR: config: Use dedicated function to parse proxy's errorfiles
- MINOR: config: Use dedicated function to parse proxy's errorloc
- MEDIUM: http-htx/proxy: Use a global and centralized storage for HTTP error messages
- MINOR: proxy: Register keywords to parse errorfile and errorloc directives
- MINOR: http-htx: Add a new section to create groups of custom HTTP errors
- MEDIUM: proxy: Add a directive to reference an http-errors section in a proxy
- MINOR: http-rules: Update txn flags and status when a deny rule is executed
- MINOR: http-rules: Support an optional status on deny rules for http reponses
- MINOR: http-rules: Use same function to parse request and response deny actions
- MINOR: http-ana: Add an error message in the txn and send it when defined
- MEDIUM: http-rules: Support an optional error message in http deny rules
- REGTEST: Add a strict rewriting mode reg test
- REGEST: Add reg tests about error files
- MINOR: ssl: accept 'verify' bind option with 'set ssl cert'
- BUG/MINOR: ssl: ssl_sock_load_ocsp_response_from_file memory leak
- BUG/MINOR: ssl: ssl_sock_load_issuer_file_into_ckch memory leak
- BUG/MINOR: ssl: ssl_sock_load_sctl_from_file memory leak
- BUG/MINOR: http_htx: Fix some leaks on error path when error files are loaded
- CLEANUP: http-ana: Remove useless test on txn when the error message is retrieved
- BUILD: CI: introduce ARM64 builds
- BUILD: ssl: more elegant anti-replay feature presence check
- MINOR: proxy/http-ana: Add support of extra attributes for the cookie directive
- MEDIUM: dns: use Additional records from SRV responses
- CLEANUP: Consistently `unsigned int` for bitfields
- CLEANUP: pattern: remove the pat_time definition
- BUG/MINOR: http_act: don't check capture id in backend
- BUG/MINOR: ssl: fix build on development versions of openssl-1.1.x
A wrong behavior was introduced by
e9544935e8, leading to preventing loading
any configuration where a capture slot id is used in a backend.
IE, the configuration below does not parse:
frontend f
bind *:80
declare capture request len 32
default_backend webserver
backend webserver
http-request capture req.hdr(Host) id 1
The point is that such type of configuration is valid and should run.
This patch enforces the check of capture slot id only if the action rule
is configured in a frontend.
The point is that at configuration parsing time, it is impossible to
check which frontend could point to this backend (furthermore if we use
dynamic backend name resolution at runtime).
The documentation has been updated to warn the user to ensure that
relevant frontends have required declaration when such rule has to be
used in a backend.
If no capture slot can be found, then the action will just not be
executed and HAProxy will process the next one in the list, as expected.
This should be backported to all supported branches (bug created as part
of a bug fix introduced into 1.7 and backported to 1.6).
It is now possible to insert any attribute when a cookie is inserted by
HAProxy. Any value may be set, no check is performed except the syntax validity
(CTRL chars and ';' are forbidden). For instance, it may be used to add the
SameSite attribute:
cookie SRV insert attr "SameSite=Strict"
The attr option may be repeated to add several attributes.
This patch should fix the issue #361.
It is now possible to set the error message to use when a deny rule is
executed. It may be a specific error file, adding "errorfile <file>" :
http-request deny deny_status 400 errorfile /etc/haproxy/errorfiles/400badreq.http
It may also be an error file from an http-errors section, adding "errorfiles
<name>" :
http-request deny errorfiles my-errors # use 403 error from "my-errors" section
When defined, this error message is set in the HTTP transaction. The tarpit rule
is also concerned by this change.
It is now possible to import in a proxy, fully or partially, error files
declared in an http-errors section. It may be done using the "errorfiles"
directive, followed by a name and optionally a list of status code. If there is
no status code specified, all error files of the http-errors section are
imported. Otherwise, only error files associated to the listed status code are
imported. For instance :
http-errors my-errors
errorfile 400 ...
errorfile 403 ...
errorfile 404 ...
frontend frt
errorfiles my-errors 403 404 # ==> error 400 not imported
Now, by default, when a rule performing a rewrite on an HTTP message fails, an
internal error is triggered. Before, the failure was ignored. But most of users
are not aware of this behavior. And it does not happen very often because the
buffer reserve space in large enough. So it may be surprising. Returning an
internal error makes the rewrite failure explicit. If it is acceptable to
silently ignore it, the strict rewriting mode can be disabled.
It is now possible to explicitly instruct rewriting rules to be strict or not
towards errors. It means that in this mode, an internal error is trigger if a
rewrite rule fails. The HTTP action "strict-mode" can be used to enable or
disable the strict rewriting mode. It can be used in an http-request and an
http-response ruleset.
For now, by default the strict rewriting mode is disabled. Because it is the
current behavior. But it will be changed in another patch.
The stats field ST_F_EINT has been added to report internal errors encountered
per proxy, per listener and per server. It appears in the CLI export and on the
HTML stats page.
The section 7.3.7. is now dedicated to internal sample fetches. For now, only
HTX sample fetches are referenced in this section. But it should contain the
documentation of all sample fetches reserved to an internal use, for debugging
or testing purposes.
Modifies the existing sample extraction methods (smp_fetch_ssl_x_i_dn,
smp_fetch_ssl_x_s_dn) to accommodate a third argument that indicates the
DN should be returned in LDAP v3 format. When the third argument is
present, the new function (ssl_sock_get_dn_formatted) is called with
three parameters including the X509_NAME, a buffer containing the format
argument, and a buffer for the output. If the supplied format matches
the supported format string (currently only "rfc2253" is supported), the
formatted value is extracted into the supplied output buffer using
OpenSSL's X509_NAME_print_ex and BIO_s_mem. 1 is returned when a dn
value is retrieved. 0 is returned when a value is not retrieved.
Argument validation is added to each of the related sample
configurations to ensure the third argument passed is either blank or
"rfc2253" using strcmp. An error is returned if the third argument is
present with any other value.
Documentation was updated in configuration.txt and it was noted during
preliminary reviews that a CLEANUP patch should follow that adjusts the
documentation. Currently, this patch and the existing documentation are
copied with some minor revisions for each sample configuration. It
might be better to have one entry for all of the samples or entries for
each that reference back to a primary entry that explains the sample in
detail.
Special thanks to Chris, Willy, Tim and Aleks for the feedback.
Author: Elliot Otchet <degroens@yahoo.com>
Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
in the context of seamless reload and busy polling, older processes will
create unecessary cpu conflicts; we can assume there is no need for busy
polling for old processes which are waiting to be terminated.
This patch is not a bug fix itself but might be a good stability
improvment when you are un the context of frequent seamless reloads with
a high "hard-stop-after" value; for that reasons I think this patch
should be backported in all 2.x versions.
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
This action is very similar to "replace-uri" except that it only acts on the
path component. This is assumed to better match users' expectations when they
used to rely on "replace-uri" in HTTP/1 because mostly origin forms were used
in H1 while mostly absolute URI form is used in H2, and their rules very often
start with a '/', and as such do not match.
It could help users to get this backported to 2.0 and 2.1.
As discussed in the thread below [1], the debug converter is currently
not of much use given that it's only built when DEBUG_EXPR is set, and
it is limited to stderr only.
This patch changes this to make it take an optional prefix and an optional
target sink so that it can log to stdout, stderr or a ring buffer. The
default output is the "buf0" ring buffer, that can be consulted from the
CLI.
[1] https://www.mail-archive.com/haproxy@formilux.org/msg35671.html
Note: if this patch is backported, it also requires the following commit to
work: 46dfd78cbf ("BUG/MINOR: sample: always check converters' arguments").
With H2 deployments becoming more common, replace-uri starts to hit
users by not always matching absolute URIs due to rules expecting the
URI to start with a '/'.
As mentionned in bug #405 we continue to reference architecture.txt from
places in the doc despite this file not being packaged for many years.
Better drop the reference if it's confusing.
HAProxy doesn't need to call executables at run time (except when using
external checks which are strongly recommended against), and is even expected
to isolate itself into an empty chroot. As such, there basically is no valid
reason to allow a setuid executable to be called without the user being fully
aware of the risks. In a situation where haproxy would need to call external
checks and/or disable chroot, exploiting a vulnerability in a library or in
haproxy itself could lead to the execution of an external program. On Linux
it is possible to lock the process so that any setuid bit present on such an
executable is ignored. This significantly reduces the risk of privilege
escalation in such a situation. This is what haproxy does by default. In case
this causes a problem to an external check (for example one which would need
the "ping" command), then it is possible to disable this protection by
explicitly adding this directive in the global section. If enabled, it is
possible to turn it back off by prefixing it with the "no" keyword.
Before the option:
$ socat - /tmp/sock1 <<< "expert-mode on; debug dev exec sudo /bin/id"
uid=0(root) gid=0(root) groups=0(root
After the option:
$ socat - /tmp/sock1 <<< "expert-mode on; debug dev exec sudo /bin/id"
sudo: effective uid is not 0, is /usr/bin/sudo on a file system with the
'nosuid' option set or an NFS file system without root privileges?
