Released version 2.5-dev10 with the following main changes :
- MINOR: initcall: Rename __GLOBL and __GLOBL1.
- MINOR: rules: add a new function new_act_rule() to allocate act_rules
- MINOR: rules: add a file name and line number to act_rules
- MINOR: stream: report the current rule in "show sess all" when known
- MINOR: stream: report the current filter in "show sess all" when known
- CLEANUP: stream: Properly indent current_rule line in "show sess all"
- BUG/MINOR: lua: Fix lua error handling in `hlua_config_prepend_path()`
- CI: github: switch to OpenSSL 3.0.0
- REGTESTS: ssl: Fix references to removed option in test description
- MINOR: ssl: Add ssllib_name_startswith precondition
- REGTESTS: ssl: Fix ssl_errors test for OpenSSL v3
- REGTESTS: ssl: Reenable ssl_errors test for OpenSSL only
- REGTESTS: ssl: Use mostly TLSv1.2 in ssl_errors test
- MEDIUM: mux-quic: rationalize tx buffers between qcc/qcs
- MEDIUM: h3: properly manage tx buffers for large data
- MINOR: mux-quic: standardize h3 settings sending
- CLEANUP: h3: remove dead code
- MINOR: mux-quic: implement standard method to detect if qcc is dead
- MEDIUM: mux-quic: defer stream shut if remaining tx data
- MINOR: mux: remove last occurences of qcc ring buffer
- MINOR: quic: handle CONNECTION_CLOSE frame
- REGTESTS: ssl: re-enable set_ssl_cert_bundle.vtc
- MINOR: ssl: add ssl_fc_is_resumed to "option httpslog"
- MINOR: http: Add http_auth_bearer sample fetch
- MINOR: jwt: Parse JWT alg field
- MINOR: jwt: JWT tokenizing helper function
- MINOR: jwt: Insert public certificates into dedicated JWT tree
- MINOR: jwt: jwt_header_query and jwt_payload_query converters
- MEDIUM: jwt: Add jwt_verify converter to verify JWT integrity
- REGTESTS: jwt: Add tests for the jwt_verify converter
- BUILD: jwt: fix declaration of EVP_KEY in jwt-h.h
- MINOR: proto_tcp: use chunk_appendf() to ouput socket setup errors
- MINOR: proto_tcp: also report the attempted MSS values in error message
- MINOR: inet: report the faulty interface name in "bind" errors
- MINOR: protocol: report the file and line number for binding/listening errors
- MINOR: protocol: uniformize protocol errors
- MINOR: resolvers: fix the resolv_str_to_dn_label() API about trailing zero
- BUG/MEDIUM: resolver: make sure to always use the correct hostname length
- BUG/MINOR: resolvers: do not reject host names of length 255 in SRV records
- MINOR: resolvers: fix the resolv_dn_label_to_str() API about trailing zero
- MEDIUM: listeners: split the thread mask between receiver and bind_conf
- MINOR: listeners: add clone_listener() to duplicate listeners at boot time
- MEDIUM: listener: add the "shards" bind keyword
- BUG/MEDIUM: resolvers: use correct storage for the target address
- MINOR: resolvers: merge address and target into a union "data"
- BUG/MEDIUM: resolvers: fix truncated TLD consecutive to the API fix
- BUG/MEDIUM: jwt: fix base64 decoding error detection
- BUG/MINOR: jwt: use CRYPTO_memcmp() to compare HMACs
- DOC: jwt: fix a typo in the jwt_verify() keyword description
- BUG/MEDIUM: sample/jwt: fix another instance of base64 error detection
- BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on back
- BUG/MINOR: sample: Fix 'fix_tag_value' sample when waiting for more data
- DOC: config: Move 'tcp-response content' at the right place
- BUG/MINOR: proxy: Use .disabled field as a bitfield as documented
- MINOR: proxy: Introduce proxy flags to replace disabled bitfield
- MINOR: sample/arg: Be able to resolve args found in defaults sections
- MEDIUM: proxy: Warn about ambiguous use of named defaults sections
- MINOR: proxy: Be able to reference the defaults section used by a proxy
- MINOR: proxy: Add PR_FL_READY flag on fully configured and usable proxies
- MINOR: config: Finish configuration for referenced default proxies
- MINOR: config: No longer remove previous anonymous defaults section
- MINOR: tcpcheck: Support 2-steps args resolution in defaults sections
- MEDIUM: rules/acl: Parse TCP/HTTP rules and acls defined in defaults sections
- MEDIUM: tcp-rules: Eval TCP rules defined in defaults sections
- MEDIUM: http-ana: Eval HTTP rules defined in defaults sections
- BUG/MEDIUM: sample: Cumulate frontend and backend sample validity flags
- REGTESTS: Add scripts to test support of TCP/HTTP rules in defaults sections
- DOC: config: Add documentation about TCP/HTTP rules in defaults section
- DOC: config: Rework and uniformize how TCP/HTTP rules are documented
- BUG/MINOR: proxy: Release ACLs and TCP/HTTP rules of default proxies
- BUG/MEDIUM: cpuset: fix cpuset size for FreeBSD
- BUG/MINOR: sample: fix backend direction flags consecutive to last fix
- BUG/MINOR: listener: fix incorrect return on out-of-memory
- BUG/MINOR: listener: add an error check for unallocatable trash
- CLEANUP: listeners: remove unreachable code in clone_listener()
Now all these rules are documented using the same structure. First there is
a general description with the list of all supported actions. Then all
actions are described in details. Thus, it is easy to have a quick list of
all supported actions and this avoids to have a huge description with all
info about these actions. In addition, when it is possible, we make a
reference to already documented parts.
Documentation of each directive that can now be used in defaults section was
updated to explain how it works. A special mark was added to specify when a
keyword is supported by defaults sections with a name but not anonymous
ones. In this case an exclamation mark is added.
It is now possible to designate the defaults section to use by adding a name
of the corresponding defaults section and referencing it in the desired
proxy section. However, this introduces an ambiguity. This named defaults
section may still be implicitly used by other proxies if it is the last one
defined. In this case for instance:
default common
...
default frt from common
...
default bck from common
...
frontend fe from frt
...
backend be from bck
...
listen stats
...
Here, it is not really obvious the last section will use the 'bck' defaults
section. And it is probably not the expected behaviour. To help users to
properly configure their haproxy, a warning is now emitted if a defaults
section is explicitly AND implicitly used. The configuration manual was
updated accordingly.
Because this patch adds a warning, it should probably not be backported to
2.4. However, if is is backported, it depends on commit "MINOR: proxy:
Introduce proxy flags to replace disabled bitfield".
In multi-threaded mode, on operating systems supporting multiple listeners on
the same IP:port, this will automatically create this number of multiple
identical listeners for the same line, all bound to a fair share of the number
of the threads attached to this listener. This can sometimes be useful when
using very large thread counts where the in-kernel locking on a single socket
starts to cause a significant overhead. In this case the incoming traffic is
distributed over multiple sockets and the contention is reduced. Note that
doing this can easily increase the CPU usage by making more threads work a
little bit.
If the number of shards is higher than the number of available threads, it
will automatically be trimmed to the number of threads. A special value
"by-thread" will automatically assign one shard per thread.
This new converter takes a JSON Web Token, an algorithm (among the ones
specified for JWS tokens in RFC 7518) and a public key or a secret, and
it returns a verdict about the signature contained in the token. It does
not simply return a boolean because some specific error cases cas be
specified by returning an integer instead, such as unmanaged algorithms
or invalid tokens. This enables to distinguich malformed tokens from
tampered ones, that would be valid format-wise but would have a bad
signature.
This converter does not perform a full JWT validation as decribed in
section 7.2 of RFC 7519. For instance it does not ensure that the header
and payload parts of the token are completely valid JSON objects because
it would need a complete JSON parser. It only focuses on the signature
and checks that it matches the token's contents.
Those converters allow to extract a JSON value out of a JSON Web Token's
header part or payload part (the two first dot-separated base64url
encoded parts of a JWS in the Compact Serialization format).
They act as a json_query call on the corresponding decoded subpart when
given parameters, and they return the decoded JSON subpart when no
parameter is given.
This fetch can be used to retrieve the data contained in an HTTP
Authorization header when the Bearer scheme is used. This is used when
transmitting JSON Web Tokens for instance.
Released version 2.5-dev9 with the following main changes :
- head-truc
- REGTESTS: lua: test the httpclient:get() feature
- Revert "head-truc"
- BUG/MEDIUM: httpclient: replace ist0 by istptr
- MINOR: config: use a standard parser for the "nbthread" keyword
- CLEANUP: init: remove useless test against MAX_THREADS in affinity loop
- MEDIUM: init: de-uglify the per-thread affinity setting
- MINOR: init: extract the setup and end of threads to their own functions
- MINOR: log: Try to get the status code when MUX_EXIT_STATUS is retrieved
- MINOR: mux-h1: Set error code if possible when MUX_EXIT_STATUS is returned
- MINOR: mux-h1: Be able to set custom status code on parsing error
- MEDIUM: mux-h1: Reject HTTP/1.0 GET/HEAD/DELETE requests with a payload
- MEDIUM: h1: Force close mode for invalid uses of T-E header
- BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
- MINOR: http: Add 422-Unprocessable-Content error message
- MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice
- BUG/MEDIUM: mux-h1/mux-fcgi: Reject messages with unknown transfer encoding
- REGTESTS: Add script to validate T-E header parsing
- REORG: pools: move default settings to defaults.h
- DOC: peers: fix doc "enable" statement on "peers" sections
- MINOR: Makefile: add MEMORY_POOLS to the list of DEBUG_xxx options
- MINOR: ssl: Set connection error code in case of SSL read or write fatal failure
- MINOR: ssl: Rename ssl_bc_hsk_err to ssl_bc_err
- MINOR: ssl: Store the last SSL error code in case of read or write failure
- REGTESTS: ssl: enable show_ssl_ocspresponse.vtc again
- REGTESTS: ssl: enable ssl_crt-list_filters.vtc again
- BUG/MEDIUM: lua: fix wakeup condition from sleep()
- BUG/MAJOR: lua: use task_wakeup() to properly run a task once
- MINOR: arg: Be able to forbid unresolved args when building an argument list
- BUG/MINOR: tcpcheck: Don't use arg list for default proxies during parsing
- BUG/MINOR: tcp-rules: Stop content rules eval on read error and end-of-input
- MINOR: tasks: catch TICK_ETERNITY with BUG_ON() in __task_queue()
- REGTESTS: ssl: show_ssl_ocspresponse w/ freebsd won't use base64
- REGTESTS: ssl: wrong feature cmd in show_ssl_ocspresponse.vtc
- CLEANUP: tasks: remove the long-unused work_lists
- MINOR: task: provide 3 task_new_* wrappers to simplify the API
- MINOR: time: uninline report_idle() and move it to task.c
- REORG: sched: move idle time calculation from time.h to task.h
- REORG: sched: move the stolen CPU time detection to sched_entering_poll()
- BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the release
- BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
- MINOR: httpclient: destroy() must free the headers and the ists
- MINOR: httpclient: set HTTPCLIENT_F_ENDED only in release
- MINOR: httpclient: stop_and_destroy() ask the applet to autokill
- MINOR: httpclient: test if started during stop_and_destroy()
- MINOR: httpclient/lua: implement garbage collection
- BUG/MEDIUM: httpclient/lua: crash because of b_xfer and get_trash_chunk()
- MINOR: httpclient: destroy checks if a client was started but not stopped
- BUG/MINOR: httpclient/lua: does not process headers when failed
- MINOR: httpclient/lua: supports headers via named arguments
- CLEANUP: server: always include the storage for SSL settings
- CLEANUP: sample: rename sample_conv_var2smp() to *_sint
- CLEANUP: sample: uninline sample_conv_var2smp_str()
- MINOR: sample: provide a generic var-to-sample conversion function
- BUG/MEDIUM: sample: properly verify that variables cast to sample
- BUILD: action: add the relevant structures for function arguments
- BUILD: extcheck: needs to include stream-t.h
- BUILD: hlua: needs to include stream-t.h
- BUILD: stats: define several missing structures in stats.h
- BUILD: resolvers: define missing types in resolvers.h
- BUILD: httpclient: include missing ssl_sock-t
- BUILD: sample: include openssl-compat
- BUILD: http_ana: need to include proxy-t to get redirect_rule
- BUILD: http_rules: requires http_ana-t.h for REDIRECT_*
- BUILD: vars: need to include xxhash
- BUILD: peers: need to include eb{32/mb/pt}tree.h
- BUILD: ssl_ckch: include ebpttree.h in ssl_ckch.c
- BUILD: compiler: add the container_of() and container_of_safe() macros
- BUILD: idleconns: include missing ebmbtree.h at several places
- BUILD: connection: connection.h needs list.h and server.h
- BUILD: tree-wide: add missing http_ana.h from many places
- BUILD: cfgparse-ssl: add missing errors.h
- BUILD: tcp_sample: include missing errors.h and session-t.h
- BUILD: mworker: mworker-prog needs time.h for the 'now' variable
- BUILD: tree-wide: add several missing activity.h
- BUILD: compat: fix -Wundef on SO_REUSEADDR
- CLEANUP: pools: pools-t.h doesn't need to include thread-t.h
- REORG: pools: uninline the UAF allocator and force-inline the rest
- REORG: thread: uninline the lock-debugging code
- MINOR: thread/debug: replace nsec_now() with now_mono_time()
- CLEANUP: remove some unneeded includes from applet-t.h
- REORG: listener: move bind_conf_alloc() and listener_state_str() to listener.c
- CLEANUP: listeners: do not include openssl-compat
- CLEANUP: servers: do not include openssl-compat
- REORG: ssl: move ssl_sock_is_ssl() to connection.h and rename it
- CLEANUP: mux_fcgi: remove dependency on ssl_sock
- CLEANUP: ssl/server: move ssl_sock_set_srv() to srv_set_ssl() in server.c
- REORG: ssl-sock: move the sslconns/totalsslconns counters to global
- REORG: sample: move the crypto samples to ssl_sample.c
- REORG: sched: moved samp_time and idle_time to task.c as well
- REORG: time/ticks: move now_ms and global_now_ms definitions to ticks.h
- CLEANUP: tree-wide: remove unneeded include time.h in ~20 files
- REORG: activity: uninline activity_count_runtime()
- REORG: acitvity: uninline sched_activity_entry()
- CLEANUP: stream: remove many unneeded includes from stream-t.h
- CLEANUP: stick-table: no need to include socket nor in.h
- MINOR: connection: use uint64_t for the hashes
- REORG: connection: move the hash-related stuff to connection.c
- REORG: connection: uninline conn_notify_mux() and conn_delete_from_tree()
- REORG: server: uninline the idle conns management functions
- REORG: ebtree: split structures into their own file ebtree-t.h
- CLEANUP: tree-wide: only include ebtree-t from type files
- REORG: connection: move the largest inlines from connection.h to connection.c
- CLEANUP: connection: do not include http_ana!
- CLEANUP: connection: remove unneeded tcpcheck-t.h and use only session-t.h
- REORG: connection: uninline the rest of the alloc/free stuff
- REORG: task: uninline the loop time measurement code
- CLEANUP: time: move a few configurable defines to defaults.h
- CLEANUP: fd: do not include time.h
- REORG: fd: uninline compute_poll_timeout()
- CLENAUP: wdt: use ha_tkill() instead of accessing pthread directly
- REORG: thread: move the thread init/affinity/stop to thread.c
- REORG: thread: move ha_get_pthread_id() to thread.c
- MINOR: thread: use a dedicated static pthread_t array in thread.c
- CLEANUP: thread: uninline ha_tkill/ha_tkillall/ha_cpu_relax()
- DOC: configuration: add clarification on escaping in keyword arguments
- BUG/MINOR: task: fix missing include with DEBUG_TASK
- MINOR: pools: report the amount used by thread caches in "show pools"
- MINOR: quic: Distinguish packet and SSL read enc. level in traces
- MINOR: quic: Add a function to dump SSL stack errors
- MINOR: quic: BUG_ON() SSL errors.
- MINOR: quic: Fix SSL error issues (do not use ssl_bio_and_sess_init())
- BUG/MEDIUM: mux-quic: reinsert all streams in by_id tree
- BUG/MAJOR: xprt-quic: do not queue qc timer if not set
- MINOR: mux-quic: release connection if no more bidir streams
- BUG/MAJOR: quic: remove qc from receiver cids tree on free
- BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial frames
- MINOR: qpack: do not encode invalid http status code
- MINOR: qpack: support non-indexed http status code encoding
- MINOR: qpack: fix memory leak on huffman decoding
- CLEANUP: mux-quic: remove unused code
- BUG/MINOR: quic: fix includes for compilation
- BUILD: connection: avoid a build warning on FreeBSD with SO_USER_COOKIE
- BUILD: init: avoid a build warning on FreeBSD with USE_PROCCTL
- REORG: time: move time-keeping code and variables to clock.c
- REORG: clock: move the updates of cpu/mono time to clock.c
- MINOR: activity: get the run_time from the clock updates
- CLEANUP: clock: stop exporting before_poll and after_poll
- REORG: clock: move the clock_id initialization to clock.c
- REORG: clock/wdt: move wdt timer initialization to clock.c
- MINOR: clock: move the clock_ids to clock.c
- MINOR: wdt: move wd_timer to wdt.c
- CLEANUP: wdt: do not remap SI_TKILL to SI_LWP, test the values directly
- REORG: thread/sched: move the task_per_thread stuff to thread_ctx
- REORG: thread/clock: move the clock parts of thread_info to thread_ctx
- REORG: thread/sched: move the thread_info flags to the thread_ctx
- REORG: thread/sched: move the last dynamic thread_info to thread_ctx
- MINOR: thread: make "ti" a const pointer and clean up thread_info a bit
- MINOR: threads: introduce a minimalistic notion of thread-group
- MINOR: global: add a new "thread-groups" directive
- MINOR: global: add a new "thread-group" directive
- MINOR: threads: make tg point to the current thread's group
- MEDIUM: threads: automatically assign threads to groups
- MINOR: threads: set the group ID and its bit in the thread group
- MINOR: threads: set the tid, ltid and their bit in thread_cfg
- MEDIUM: threads: replace ha_set_tid() with ha_set_thread()
- MINOR: threads: add the current group ID in thread-local "tgid" variable
- MINOR: debug: report the group and thread ID in the thread dumps
- MEDIUM: listeners: support the definition of thread groups on bind lines
- MINOR: threads: add a new function to resolve config groups and masks
- MEDIUM: config: resolve relative threads on bind lines to absolute ones
- MEDIUM: stick-table: never learn the "conn_cur" value from peers
There have been a large number of issues reported with conn_cur
synchronization because the concept is wrong. In an active-passive
setup, pushing the local connections count from the active node to
the passive one will result in the passive node to have a higher
counter than the real number of connections. Due to this, after a
switchover, it will never be able to close enough connections to
go down to zero. The same commonly happens on reloads since the new
process preloads its values from the old process, and if no connection
happens for a key after the value is learned, it is impossible to reset
the previous ones. In active-active setups it's a bit different, as the
number of connections reflects the number on the peer that pushed last.
This patch solves this by marking the "conn_cur" local and preventing
it from being learned from peers. It is still pushed, however, so that
any monitoring system that collects values from the peers will still
see it.
The patch is tiny and trivially backportable. While a change of behavior
in stable branches is never welcome, it remains possible to fix issues
if reports become frequent.
This extends the "thread" statement of bind lines to support an optional
thread group number. When unspecified (0) it's an absolute thread range,
and when specified it's one relative to the thread group. Masks are still
used so no more than 64 threads may be specified at once, and a single
group is possible. The directive is not used for now.
This registers a mapping of threads to groups by enumerating for each thread
what group it belongs to, and marking the group as assigned. It takes care of
checking for redefinitions, overlaps, and holes. It supports both individual
numbers and ranges. The thread group is referenced from the thread config.
Add a more precise description on how backslash escaping is different
than the top-level parser, and give examples of how to handle single
quotes inside arguments.
The ssl_bc_hsk_err sample fetch will need to raise more errors than only
handshake related ones hence its renaming to a more generic ssl_bc_err.
This patch is required because some handshake failures that should have
been caught by this fetch (verify error on the server side for instance)
were missed. This is caused by a change in TLS1.3 in which the
'Finished' state on the client is reached before its certificate is sent
(and verified) on the server side (see the "Protocol Overview" part of
RFC 8446).
This means that the SSL_do_handshake call is finished long before the
server can verify and potentially reject the client certificate.
The ssl_bc_hsk_err will then need to be expanded to catch other types of
errors.
This change is also applied to the frontend fetches (ssl_fc_hsk_err
becomes ssl_fc_err) and to their string counterparts.
In case of a connection error happening after the SSL handshake is
completed, the error code stored in the connection structure would not
always be set, hence having some connection failures being described as
successful in the fc_conn_err or bc_conn_err sample fetches.
The most common case in which it could happen is when the SSL server
rejects the client's certificate. The SSL_do_handshake call on the
client side would be sucessful because the client effectively sent its
client hello and certificate information to the server, but the next
call to SSL_read on the client side would raise an SSL_ERROR_SSL code
(through the SSL_get_error function) which is decribed in OpenSSL
documentation as a non-recoverable and fatal SSL error.
This patch ensures that in such a case, the connection's error code is
set to a special CO_ERR_SSL_FATAL value.
Checking in code the right keyword is "enabled" and not "enable".
In addition the comment was also completed:
This could appear useless because the "defaults" sections not
yet apply on "peers" sections, but it could be the case in the future.
This statement can currently cancel a previous "disabled" keyword in
the same section.
This patch should be backported in all supported branches (keyword
is present since 1.5)
Released version 2.5-dev8 with the following main changes :
- BUILD: compiler: fixed a missing test on defined(__GNUC__)
- BUILD: halog: fix a -Wundef warning on non-glibc systems
- BUILD: threads: fix -Wundef for _POSIX_PRIORITY_SCHEDULING on libmusl
- BUG/MINOR: compat: make sure __WORDSIZE is always defined
- BUILD: sample: fix format warning on 32-bit archs in sample_conv_be2dec_check()
- CLEANUP: pools: factor all malloc_trim() calls into trim_all_pools()
- MINOR: pools: automatically disable malloc_trim() with external allocators
- MINOR: pools: report it when malloc_trim() is enabled
- DOC: Add .mailmap
- CLEANUP: tree-wide: fix prototypes for functions taking no arguments.
- CLEANUP: Remove prototype for non-existent thread_get_default_count()
- CLEANUP: acl: Remove unused variable when releasing an acl expression
- BUG/MAJOR: mux-h1: Don't eval input data if an error was reported
- DOC: update Tim's address in .mailmap
- MINOR: pools: use mallinfo2() when available instead of mallinfo()
- BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
- DOC: management: certificate files must be sanitized before injection
- BUG/MINOR: connection: prevent null deref on mux cleanup task allocation
- BUILD: ist: prevent gcc11 maybe-uninitialized warning on istalloc
- BUG/MINOR: cli/payload: do not search for args inside payload
- BUILD: sockpair: do not set unused flag
- BUILD: proto_uxst: do not set unused flag
- BUILD: fd: remove unused variable totlen in fd_write_frag_line()
- MINOR: applet: remove the thread mask from appctx_new()
- REORG: threads: move ha_get_pthread_id() to tinfo.h
- CLEANUP: Apply ist.cocci
- DEV: coccinelle: Add ist.cocci
- CLEANUP: Apply bug_on.cocci
- DEV: coccinelle: Add xalloc_size.cocci
- DEV: coccinelle: Add bug_on.cocci
- CLEANUP: Apply xalloc_size.cocci
- DEV: coccinelle: Add xalloc_cast.cocci
- BUG/MINOR: flt-trace: fix an infinite loop when random-parsing is set
- MINOR: httpclient: add the EOH when no headers where provided
- CLEANUP: Include check.h in flt_spoe.c
- CLEANUP: Remove unreachable `break` from parse_time_err()
- BUG/MINOR: server: allow 'enable health' only if check configured
- BUG/MINOR: server: alloc dynamic srv ssl ctx if proxy uses ssl chk rule
- MINOR: server: enable more keywords for ssl checks for dynamic servers
- MINOR: server: enable more check related keywords for dynamic servers
- REORG: server: move slowstart init outside of checks
- MINOR: server: enable slowstart for dynamic server
- MEDIUM: listener: deprecate "process" in favor of "thread" on bind lines
- BUG/MEDIUM: leastconn: fix rare possibility of divide by zero
- BUG/MINOR: quic: Possible NULL pointer dereferencing when dumping streams.
- MINOR: quic: Move transport parmaters to anynomous struct.
- MINOR: mux_quic: Add QUIC mux layer.
- MINOR: connection: Add callbacks definitions for QUIC.
- MINOR: quic: Attach QUIC mux connection objet to QUIC connection.
- MINOR: quic: Add a new definition to store STREAM frames.
- MINOR: h3: Add HTTP/3 definitions.
- MINOR: qpack: Add QPACK compression.
- MINOR: quic_sock: Finalize the QUIC connections.
- MINOR: quic: Disable the action of ->rcv_buf() xprt callback
- MINOR: quic: Add callbacks for (un)scribing to QUIC xprt.
- MINOR: quic: Variable-length integer encoding/decoding into/from buffer struct.
- BUG/MINOR: quic: Wrong ->accept() error handling
- MINOR: quic: Add a wrapper function to update transport parameters.
- MINOR: quic: Update the streams transport parameters.
- MINOR: quic: Avoid header collisions
- MINOR: quic: Replace max_packet_size by max_udp_payload size.
- MINOR: quic: Enable some quic, h3 and qpack modules compilation.
- MINOR: quic: Move an SSL func call from QUIC I/O handler to the xprt init.
- MINOR: quic: Initialize the session before starting the xprt.
- BUG/MINOR: quic: Do not check the acception of a new conn from I/O handler.
- MINOR: quic: QUIC conn initialization from I/O handler
- MINOR: quic: Remove header protection for conn with context
- MINOR: quic: Derive the initial secrets asap
- MINOR: quic: Remove header protection also for Initial packets
- BUG/MINOR: quic: Wrong memory free in quic_update_ack_ranges_list()
- MINOR: quic: quic_update_ack_ranges_list() code factorization
- MINOR: quic: Useless test in quic_update_ack_ranges_list()
- MINOR: quic: Remove a useless variable in quic_update_ack_ranges_list()
- BUG/MINOR: quic: Missing cases treatement when updating ACK ranges
- CLEAUNUP: quic: Usage of a useless variable in qc_treat_rx_pkts()
- BUG/MINOR: quic: Wrong RX packet reference counter usage
- MINOR: quic: Do not stop the packet parsing too early in qc_treat_rx_packets()
- MINOR: quic: Add a lock for RX packets
- MINOR: quic: Move the connection state
- MINOR: quic: Replace quic_conn_ctx struct by ssl_sock_ctx struct
- MINOR: quic: Replace the RX list of packet by a thread safety one.
- MINOR: quic: Replace the RX unprotected packet list by a thread safety one.
- MINOR: quic: Add useful traces for I/O dgram handler
- MINOR: quic: Do not wakeup the xprt task on ACK receipt
- MINOR: quic: Connection allocations rework
- MINOR: quic: Move conn_prepare() to ->accept_conn() callback
- MINOR: quic: Make qc_lstnr_pkt_rcv() be thread safe.
- MINOR: quic: Add a ring buffer implementation for QUIC
- MINOR: quic: Prefer x25519 as ECDH preferred parametes.
- MINOR: quic: Add the QUIC v1 initial salt.
- BUG/MINOR: quic: Too much reduced computed space to build handshake packets
- MINOR: net_helper: add functions for pointers
- MINOR: quic: Add ring buffer definition (struct qring) for QUIC
- MINOR: proto_quic: Allocate TX ring buffers for listeners
- MINOR: quic: Initialize pointers to TX ring buffer list
- MINOR: quic: Make use of TX ring buffers to send QUIC packets
- MINOR: quic_tls: Make use of the QUIC V1 salt.
- MINOR: quic: Remove old TX buffer implementation
- MINOR: Add function for TX packets reference counting
- MINOR: quic: Add TX packets at the very last time to their tree.
- MINOR: quic: Unitialized mux context upon Client Hello message receipt.
- MINOR: quic: Missing encryption level rx.crypto member initialization and lock.
- MINOR: quic: Rename ->rx.rwlock of quic_enc_level struct to ->rx.pkts_rwlock
- MINOR: quic: Make qc_treat_rx_pkts() be thread safe.
- MINOR: quic: Make ->tx.frms quic_pktns struct member be thread safe
- MINOR: quic: Replace quic_tx_frm struct by quic_frame struct
- MINOR: quic: Add a mask for TX frame builders and their authorized packet types
- MINOR: quic: Add a useful function to compute any frame length.
- MINOR: quic: Add the QUIC connection state to traces
- MINOR: quic: Store post handshake frame in ->pktns.tx.frms MT_LIST
- MINOR: quic: Add the packet type to quic_tx_packet struct
- MINOR: quic: Modify qc_do_build_hdshk_pkt() to accept any packet type
- MINOR: quic: Atomically handle packet number space ->largest_acked_pn variable
- MINOR: quic: Modify qc_build_cfrms() to support any frame
- MINOR: quic: quic_conn_io_cb() task rework
- MINOR: quic: Make qc_build_hdshk_pkt() atomically consume a packet number
- MINOR: quic: qc_do_build_hdshk_pkt() does not need to pass a copy of CRYPTO frame
- MINOR: quic: Remove Application level related functions
- MINOR: quic: Rename functions which do not build only Handshake packets
- MINOR: quic: Make circular buffer internal buffers be variable-sized.
- MINOR: quic: Add a pool for TX ring buffer internal buffer
- MINOR: quic: Make use of the last cbuf API when initializing TX ring buffers
- MINOR: quic: Missing acks encoded size updates.
- MINOR: quic: Evaluate the packet lengths in advance
- MINOR: quic: Update the TLS extension for QUIC transport parameters
- MINOR: quic: Fix handshake state debug strings
- MINOR: quic: Atomically get/set the connection state
- MINOR: quic: Missing QUIC encryption level for qc_build_pkt()
- MINOR: quic: Coalesce Application level packets with Handshake packets.
- MINOR: quic: Wrong flags handling for acks
- MINOR: quic: Missing case when discarding HANDSHAKE secrets
- MINOR: quic: Post handshake packet building improvements
- MINOR: quic: Prepare Application level packet asap.
- MINOR: h3: Send h3 settings asap
- MINOR: quic: Wrong STREAM frame length computing
- MINOR: quic: Wrong short packet minimum length
- MINOR: quic: Prepare STREAM frames to fill QUIC packets
- MINOR: h3: change default settings
- MINOR: quic-enc: fix varint encoding
- MINOR: qpack: fix wrong comment
- MINOR: qpack: generate headers list on decoder
- MINOR: h3: parse headers to htx
- MINOR: h3: allocate stream on headers
- MEDIUM: mux-quic: implement ring buffer on stream tx
- MINOR: mux-quic: send SETTINGS on uni stream
- MINOR: h3: define snd_buf callback and divert mux ops
- MINOR: mux-quic: define FIN stream flag
- MINOR: qpack: create qpack-enc module
- MINOR: qpack: encode headers functions
- MINOR: h3: encode htx headers to QPACK
- MINOR: h3: send htx data
- MINOR: h3/mux: detect fin on last h3 frame of the stream
- MINOR: quic: Shorten some handshakes
- MINOR: quic: Make QUIC-TLS support at least two initial salts
- MINOR: quic: Attach the QUIC connection to a thread.
- MINOR: quic: Missing active_connection_id_limit default value
- MINOR: quic_sock: Do not flag QUIC connections as being set
- MINOR: buf: Add b_force_xfer() function
- MINOR: quic: Make use of buffer structs to handle STREAM frames
- MINOR: mux_quic: move qc_process() code to qc_send()
- MINOR: quic: Add a typedef for unsigned long long
- MINOR: quic: Confusion between TX/RX for the frame builders
- MINOR: quic: Wrong packet flags settings during frame building
- MINOR: quic: Constantness fixes for frame builders/parsers.
- MINOR: quic_tls: Client/serveur state reordering
- MINOR: quic: Wrong packet loss detection due to wrong pktns order
- MINOR: quic: Wrong packet number space selection in quic_loss_pktns()
- MINOR: quic: Initial packet number spaced not discarded
- MINOR: quic: Add useful trace about pktns discarding
- MINOR: mux_quic: Export the mux related flags
- MINOR: quic: Implement quic_conn_subscribe()
- MINOR: quic: Wake up the mux upon ACK receipt
- MINOR: quic: Stream FIN bit fix in qcs_push_frame()
- MINOR: quic: Implement qc_process_mux()
- MINOR: quic: Wake up the xprt from mux
- CLEANUP: quic: Remove useless inline functions
- MINOR: quic: RX packets memory leak
- MINOR: quic: Possible endless loop in qc_treat_rx_pkts()
- MINOR: quic: Crash upon too big packets receipt
- MINOR: quic: define close handler
- MEDIUM: quic: implement mux release/conn free
- MINOR: quic: fix qcc subs initialization
- BUG/MINOR: h1-htx: Fix a typo when request parser is reset
- BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
- BUG/MEDIUM: stream-int: Notify stream that the mux wants more room to xfer data
- BUG/MEDIUM: stream: Stop waiting for more data if SI is blocked on RXBLK_ROOM
- MINOR: stream-int: Set CO_RFL transient/persistent flags apart in si_cs_rcv()
- MINOR: htx: Add an HTX flag to know when a message is fragmented
- MINOR: htx: Add a function to know if the free space wraps
- BUG/MEDIUM: stream-int: Defrag HTX message in si_cs_recv() if necessary
- MINOR: stream-int: Notify mux when the buffer is not stuck when calling rcv_buf
- BUG/MINOR: http-ana: increment internal_errors counter on response error
- MINOR: stats: Enable dark mode on stat web page
- CLEANUP: stats: Fix some alignment mistakes
- MINOR: httpclient: httpclient_data() returns the available data
- MINOR: httpclient: httpclient_ended() returns 1 if the client ended
- MINOR: httpclient/lua: httpclient:get() API in lua
- MINOR: httpclient/lua: implement the headers in the response object
- BUG/MINOR: httpclient/lua: return an error on argument check
- CLEANUP: slz: Mark `reset_refs` as static
The "process" directive on "bind" lines becomes quite confusing considering
that the only allowed value is 1 for the process, and that threads are
optional and come after the mandatory "1/".
Let's introduce a new "thread" directive to directly configure thread
numbers, and mark "process" as deprecated. Now "process" will emit a
warning and will suggest how to be replaced with "thread" instead.
The doc was updated accordingly (mostly a copy-paste of the previous
description which was already up to date).
This is marked as MEDIUM as it will impact users having "zero-warning"
and "process" specified.
Released version 2.5-dev7 with the following main changes :
- BUG/MINOR: config: reject configs using HTTP with bufsize >= 256 MB
- CLEANUP: htx: remove comments about "must be < 256 MB"
- BUG/MAJOR: htx: fix missing header name length check in htx_add_header/trailer
- Revert "BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive"
- MINOR: proxy: add a global "grace" directive to postpone soft-stop
- MINOR: vars: rename vars_init() to vars_init_head()
- CLEANUP: vars: rename sample_clear_stream() to var_unset()
- REORG: vars: remerge sample_store{,_stream}() into var_set()
- MEDIUM: vars: make the ifexist variant of set-var only apply to the proc scope
- MINOR: vars: add a VF_CREATEONLY flag for creation
- MINOR: vars: support storing empty sample data with a variable
- MINOR: vars: store flags into variables and add VF_PERMANENT
- MEDIUM: vars: make var_clear() only reset VF_PERMANENT variables
- MEDIUM: vars: pre-create parsed SCOPE_PROC variables as permanent ones
- MINOR: vars: preset a random seed to hash variables names
- MEDIUM: vars: replace the global name index with a hash
- CLEANUP: vars: remove the now unused var_names array
- MINOR: vars: centralize the lock/unlock into static inlines
- OPTIM: vars: only takes the variables lock on shared entries
- OPTIM: vars: remove internal bookkeeping for vars_global_size
- OPTIM: vars: do not keep variables usage stats if no limit is set
- BUILD: fix dragonfly build again on __read_mostly
- CI: Github Actions: temporarily disable Opentracing
- BUG/MEDIUM: mux-h1: Remove "Upgrade:" header for requests with payload
- MINOR: htx: Skip headers with no value when adding a header list to a message
- CLEANUP: mux-h1: Remove condition rejecting upgrade requests with payload
- BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is reached
- BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
- BUG/MINOR: filters: Set right FLT_END analyser depending on channel
- CLEANUP: Add haproxy/xxhash.h to avoid modifying import/xxhash.h
- CLEANUP: ebmbtree: Replace always-taken elseif by else
- CLEANUP: Move XXH3 macro from haproxy/compat.h to haproxy/xxhash.h
- BUILD: opentracing: exclude the use of haproxy variables for the OpenTracing context
- BUG/MINOR: opentracing: enable the use of http headers without a set value
- CLEANUP: opentracing: use the haproxy function to generate uuid
- MINOR: opentracing: change the scope of the variable 'ot.uuid' from 'sess' to 'txn'
- CI: Github Actions: re-enable Opentracing
- CLEANUP: opentracing: simplify the condition on the empty header
- BUG/MEDIUM lua: Add missing call to RESET_SAFE_LJMP in hlua_filter_new()
In ticket #1348 some users expressed some concerns regarding the removal
of the "grace" directive from the proxies. Their use case very closely
mimmicks the original intent of the grace keyword, which is, let haproxy
accept traffic for some time when stopping, while indicating an external
LB that it's stopping.
This is implemented here by starting a task whose expiration triggers
the soft-stop for real. The global "stopping" variable is immediately
set however. For example, this below will be sufficient to instantly
notify an external check on port 9999 that the service is going down,
while other services remain active for 10s:
global
grace 10s
frontend ext-check
bind :9999
monitor-uri /ext-check
monitor fail if { stopping }
Released version 2.5-dev6 with the following main changes :
- BUG/MINOR threads: Use get_(local|gm)time instead of (local|gm)time
- BUG/MINOR: tools: Fix loop condition in dump_text()
- BUILD: ssl: next round of build warnings on LIBRESSL_VERSION_NUMBER
- BUILD: ssl: fix two remaining occurrences of #if USE_OPENSSL
- BUILD: tools: properly guard __GLIBC__ with defined()
- BUILD: globally enable -Wundef
- MINOR: log: Remove log-error-via-logformat option
- MINOR: log: Add new "error-log-format" option
- BUG/MAJOR: queue: better protect a pendconn being picked from the proxy
- CLEANUP: Add missing include guard to signal.h
- MINOR: ssl: Add new ssl_bc_hsk_err sample fetch
- MINOR: connection: Add a connection error code sample fetch for backend side
- REGTESTS: ssl: Add tests for bc_conn_err and ssl_bc_hsk_err sample fetches
- MINOR: http-rules: add a new "ignore-empty" option to redirects.
- CI: Github Actions: temporarily disable BoringSSL builds
- BUG/MINOR: vars: fix set-var/unset-var exclusivity in the keyword parser
- BUG/MINOR: vars: improve accuracy of the rules used to check expression validity
- MINOR: sample: add missing ARGC_ entries
- BUG/MINOR: vars: properly set the argument parsing context in the expression
- DOC: configuration: remove wrong tcp-request examples in tcp-response
- MEDIUM: vars: add a new "set-var-fmt" action
- BUG/MEDIUM: vars: run over the correct list in release_store_rules()
- BUG/MINOR: vars: truncate the variable name in error reports about scope.
- BUG/MINOR: vars: do not talk about global section in CLI errors for set-var
- CLEANUP: vars: name the temporary proxy "CFG" instead of "CLI" for global vars
- MINOR: log: make log-format expressions completely usable outside of req/resp
- MINOR: vars: add a "set-var-fmt" directive to the global section
- MEDIUM: vars: also support format strings in CLI's "set var" command
- CLEANUP: vars: factor out common code from vars_get_by_{desc,name}
- MINOR: vars: make vars_get_by_* support an optional default value
- MINOR: vars: make the vars() sample fetch function support a default value
- BUILD: ot: add argument for default value to vars_get_by_name()
It is quite common to see in configurations constructions like the
following one:
http-request set-var(txn.bodylen) 0
http-request set-var(txn.bodylen) req.hdr(content-length)
...
http-request set-header orig-len %[var(txn.bodylen)]
The set-var() rules are almost always duplicated when manipulating
integers or any other value that is mandatory along operations. This is
a problem because it makes the configurations complicated to maintain
and slower than needed. And it becomes even more complicated when several
conditions may set the same variable because the risk of forgetting to
initialize it or to accidentally reset it is high.
This patch extends the var() sample fetch function to take an optional
argument which contains a default value to be returned if the variable
was not set. This way it becomes much simpler to use the variable, just
set it where needed, and read it with a fall back to the default value:
http-request set-var(txn.bodylen) req.hdr(content-length)
...
http-request set-header orig-len %[var(txn.bodylen,0)]
The default value is always passed as a string, thus it will experience
a cast to the output type. It doesn't seem userful to complicate the
configuration to pass an explicit type at this point.
The vars.vtc regtest was updated accordingly.
Just like the set-var-fmt action for tcp/http rules, the set-var-fmt
directive in global sections allows to pre-set process-wide variables
using a format string instead of a sample expression. This is often
more convenient when it is required to concatenate multiple fields,
or when emitting just one word.
The set-var() action is convenient because it preserves the input type
but it's a pain to deal with when trying to concatenate values. The
most recurring example is when it's needed to build a variable composed
of the source address and the source port. Usually it ends up like this:
tcp-request session set-var(sess.port) src_port
tcp-request session set-var(sess.addr) src,concat(":",sess.port)
This is even worse when trying to aggregate multiple fields from stick-table
data for example. Due to this a lot of users instead abuse headers from HTTP
rules:
http-request set-header(x-addr) %[src]:%[src_port]
But this requires some careful cleanups to make sure they won't leak, and
it's significantly more expensive to deal with. And generally speaking it's
not clean. Plus it must be performed for each and every request, which is
expensive for this common case of ip+port that doesn't change for the whole
session.
This patch addresses this limitation by implementing a new "set-var-fmt"
action which performs the same work as "set-var" but takes a format string
in argument instead of an expression. This way it becomes pretty simple to
just write:
tcp-request session set-var-fmt(sess.addr) %[src]:%[src_port]
It is usable in all rulesets that already support the "set-var" action.
It is not yet implemented for the global "set-var" directive (which already
takes a string) and the CLI's "set var" command, which would definitely
benefit from it but currently uses its own parser and engine, thus it
must be reworked.
The doc and regtests were updated.
There is a massive abuse of copy-paste in the doc that is visible in
the examples and arguments declaration. Let's at least remove irrelevant
examples for now.
Sometimes it is convenient to remap large sets of URIs to new ones (e.g.
after a site migration for example). This can be achieved using
"http-request redirect" combined with maps, but one difficulty there is
that non-matching entries will return an empty response. In order to
avoid this, duplicating the operation as an ACL condition ending in
"-m found" is possible but it becomes complex and error-prone while it's
known that an empty URL is not valid in a location header.
This patch addresses this by improving the redirect rules to be able to
simply ignore the rule and skip to the next one if the result of the
evaluation of the "location" expression is empty. However in order not
to break existing setups, it requires a new "ignore-empty" keyword.
There used to be an ACT_FLAG_FINAL on redirect rules that's used during
the parsing to emit a warning if followed by another rule, so here we
only set it if the option is not there. The http_apply_redirect_rule()
function now returns a 3rd value to mention that it did nothing and
that this was not an error, so that callers can just ignore the rule.
The regular "redirect" rules were not modified however since this does
not apply there.
The map_redirect VTC was completed with such a test and updated to 2.5
and an example was added into the documentation.
The bc_conn_err and bc_conn_err_str sample fetches give the status of
the connection on the backend side. The error codes and error messages
are the same than the ones that can be raised by the fc_conn_err fetch.
This new sample fetch along the ssl_bc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the backend's perspective).
This option can be used to define a specific log format that will be
used in case of error, timeout, connection failure on a frontend... It
will be used for any log line concerned by the log-separate-errors
option. It will also replace the format of specific error messages
decribed in section 8.2.6.
If no "error-log-format" is defined, the legacy error messages are still
emitted and the other error logs keep using the regular log-format.
This option will be replaced by a "error-log-format" that enables to use
a dedicated log-format for connection error messages instead of the
regular log-format (in which most of the fields would be invalid in such
a case).
The "log-error-via-logformat" mechanism will then be replaced by a test
on the presence of such an error log format or not. If a format is
defined, it is used for connection error messages, otherwise the legacy
error log format is used.
Released version 2.5-dev5 with the following main changes :
- MINOR: httpclient: initialize the proxy
- MINOR: httpclient: implement a simple HTTP Client API
- MINOR: httpclient/cli: implement a simple client over the CLI
- MINOR: httpclient/cli: change the User-Agent to "HAProxy"
- MEDIUM: ssl: Keep a reference to the client's certificate for use in logs
- BUG/MEDIUM: h2: match absolute-path not path-absolute for :path
- BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
- MINOR: server: check if srv is NULL in free_server()
- MINOR: proxy: check if p is NULL in free_proxy()
- BUG/MEDIUM: cfgparse: do not allocate IDs to automatic internal proxies
- BUG/MINOR: http_client: make sure to preset the proxy's default settings
- REGTESTS: http_upgrade: fix incorrect expectation on TCP->H1->H2
- REGTESTS: abortonclose: after retries, 503 is expected, not close
- REGTESTS: server: fix agent-check syntax and expectation
- BUG/MINOR: httpclient: fix uninitialized sl variable
- BUG/MINOR: httpclient/cli: change the appctx test in the callbacks
- BUG/MINOR: httpclient: check if hdr_num is not 0
- MINOR: httpclient: cleanup the include files
- MINOR: hlua: take the global Lua lock inside a global function
- MINOR: tools: add FreeBSD support to get_exec_path()
- BUG/MINOR: systemd: ExecStartPre must use -Ws
- MINOR: systemd: remove the ExecStartPre line in the unit file
- MINOR: ssl: add an openssl version string parser
- MINOR: cfgcond: implements openssl_version_atleast and openssl_version_before
- CLEANUP: ssl: remove useless check on p in openssl_version_parser()
- BUG/MINOR: stick-table: fix the sc-set-gpt* parser when using expressions
- BUG/MINOR: httpclient: remove deinit of the httpclient
- BUG/MEDIUM: base64: check output boundaries within base64{dec,urldec}
- MINOR: httpclient: set verify none on the https server
- MINOR: httpclient: add the server to the proxy
- BUG/MINOR: httpclient: fix Host header
- BUILD: httpclient: fix build without OpenSSL
- CI: github-actions: remove obsolete options
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: proc: setting the process to produce a core dump on FreeBSD.
- BUILD: adopt script/build-ssl.sh for OpenSSL-3.0.0beta2
- MINOR: server: return the next srv instance on free_server
- BUG/MINOR: stats: use refcount to protect dynamic server on dump
- MEDIUM: server: extend refcount for all servers
- MINOR: server: define non purgeable server flag
- MINOR: server: mark referenced servers as non purgeable
- MINOR: server: mark servers referenced by LUA script as non purgeable
- MEDIUM: server: allow to remove servers at runtime except non purgeable
- BUG/MINOR: base64: base64urldec() ignores padding in output size check
- REGTEST: add missing lua requirements on server removal test
- REGTEST: fix haproxy required version for server removal test
- BUG/MINOR: proxy: don't dump servers of internal proxies
- REGTESTS: Use `feature cmd` for 2.5+ tests
- REGTESTS: Remove REQUIRE_VERSION=1.5 from all tests
- BUG/MINOR: resolvers: mark servers with name-resolution as non purgeable
- MINOR: compiler: implement an ONLY_ONCE() macro
- BUG/MINOR: lua: use strlcpy2() not strncpy() to copy sample keywords
- MEDIUM: ssl: Capture more info from Client Hello
- MINOR: sample: Expose SSL captures using new fetchers
- MINOR: sample: Add be2dec converter
- MINOR: sample: Add be2hex converter
- MEDIUM: config: Deprecate tune.ssl.capture-cipherlist-size
- BUG/MINOR: time: fix idle time computation for long sleeps
- MINOR: time: add report_idle() to report process-wide idle time
- BUG/MINOR: ebtree: remove dependency on incorrect macro for bits per long
- BUILD: activity: use #ifdef not #if on USE_MEMORY_PROFILING
- BUILD/MINOR: defaults: eliminate warning on MAXHOSTNAMELEN with -Wundef
- BUILD/MINOR: ssl: avoid a build warning on LIBRESSL_VERSION with -Wundef
- IMPORT: slz: silence a build warning with -Wundef
- BUILD/MINOR: regex: avoid a build warning on USE_PCRE2 with -Wundef
To be able to provide JA3 compatible TLS Fingerprints we need to expose
all Client Hello captured data using fetchers. Patch provides new
and modifies existing fetchers to add ability to filter out GREASE values:
- ssl_fc_cipherlist_*
- ssl_fc_ecformats_bin
- ssl_fc_eclist_bin
- ssl_fc_extlist_bin
- ssl_fc_protocol_hello_id
When we set tune.ssl.capture-cipherlist-size to a non-zero value
we are able to capture cipherlist supported by the client. To be able to
provide JA3 compatible TLS fingerprinting we need to capture more
information from Client Hello message:
- SSL Version
- SSL Extensions
- Elliptic Curves
- Elliptic Curve Point Formats
This patch allows HAProxy to capture such information and store it for
later use.
Released version 2.5-dev4 with the following main changes :
- MINOR: log: rename 'dontloglegacyconnerr' to 'log-error-via-logformat'
- MINOR: doc: rename conn_status in `option httsplog`
- MINOR: proxy: disabled takes a stopping and a disabled state
- MINOR: stats: shows proxy in a stopped state
- BUG/MINOR: server: fix race on error path of 'add server' CLI if track
- CLEANUP: thread: fix fantaisist indentation of thread_harmless_till_end()
- MINOR: threads: make thread_release() not wait for other ones to complete
- MEDIUM: threads: add a stronger thread_isolate_full() call
- MEDIUM: servers: make the server deletion code run under full thread isolation
- BUG/MINOR: server: remove srv from px list on CLI 'add server' error
- MINOR: activity/fd: remove the dead_fd counter
- MAJOR: fd: get rid of the DWCAS when setting the running_mask
- CLEANUP: fd: remove the now unused fd_set_running()
- CLEANUP: fd: remove the now unneeded fd_mig_lock
- BUG/MINOR: server: update last_change on maint->ready transitions too
- MINOR: spoe: Add a pointer on the filter config in the spoe_agent structure
- BUG/MEDIUM: spoe: Create a SPOE applet if necessary when the last one is released
- BUG/MEDIUM: spoe: Fix policy to close applets when SPOE connections are queued
- MINOR: server: unmark deprecated on enable health/agent cli
- MEDIUM: task: implement tasklet kill
- MINOR: server: initialize fields for dynamic server check
- MINOR: check: allocate default check ruleset for every backends
- MINOR: check: export check init functions
- MINOR: check: do not increment global maxsock at runtime
- MINOR: server: implement a refcount for dynamic servers
- MEDIUM: check: implement check deletion for dynamic servers
- MINOR: check: enable safe keywords for dynamic servers
- MEDIUM: server: implement check for dynamic servers
- MEDIUM: server: implement agent check for dynamic servers
- REGTESTS: server: add dynamic check server test
- MINOR: doc: specify ulimit-n usage for dynamic servers
- REGTESTS: server: fix dynamic server with checks test
- CI: travis-ci: temporarily disable arm64 builds
- BUG/MINOR: check: test if server is not null in purge
- MINOR: global: define MODE_STOPPING
- BUG/MINOR: server: do not use refcount in free_server in stopping mode
- ADMIN: dyncookie: implement a simple dynamic cookie calculator
- BUG/MINOR: check: do not reset check flags on purge
- BUG/MINOR: check: fix leak on add dynamic server with agent-check error
- BUG/MEDIUM: check: fix leak on agent-check purge
- BUG/MEDIUM: server: support both check/agent-check on a dynamic instance
- BUG/MINOR: buffer: fix buffer_dump() formatting
- MINOR: channel: remove an htx block from a channel
- BUG/MINOR: tcpcheck: Properly detect pending HTTP data in output buffer
- BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
- MINOR: lua: Add a flag on lua context to know the yield capability at run time
- BUG/MINOR: lua: Yield in channel functions only if lua context can yield
- BUG/MINOR: lua: Don't yield in channel.append() and channel.set()
- MINOR: filters/lua: Release filters before the lua context
- MINOR: lua: Add a function to get a reference on a table in the stack
- MEDIUM: lua: Process buffer data using an offset and a length
- MEDIUM: lua: Improve/revisit the lua api to manipulate channels
- DOC: Improve the lua documentation
- MEDIUM: filters/lua: Add support for dummy filters written in lua
- MINOR: lua: Add a function to get a filter attached to a channel class
- MINOR: lua: Add flags on the lua TXN to know the execution context
- MEDIUM: filters/lua: Be prepared to filter TCP payloads
- MEDIUM: filters/lua: Support declaration of some filter callback functions in lua
- MEDIUM: filters/lua: Add HTTPMessage class to help HTTP filtering
- MINOR: filters/lua: Add request and response HTTP messages in the lua TXN
- MINOR: filters/lua: Support the HTTP filtering from filters written in lua
- DOC: config: Fix 'http-response send-spoe-group' documentation
- BUG/MINOR: lua: Properly check negative offset in Channel/HttpMessage functions
- BUG/MINOR: lua: Properly catch alloc errors when parsing lua filter directives
- BUG/MEDIUM: cfgcheck: verify existing log-forward listeners during config check
- MINOR: cli: delare the CLI frontend as an internal proxy
- MINOR: proxy: disable warnings for internal proxies
- BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag is set
- BUG/MINOR: lua/filters: Return right code when txn:done() is called
- DOC: lua-api: Add documentation about lua filters
- CI: Remove obsolete USE_SLZ=1 CI job
- CLEANUP: assorted typo fixes in the code and comments
- CI: github actions: relax OpenSSL-3.0.0 version comparision
- BUILD: tools: get the absolute path of the current binary on NetBSD.
- DOC: Minor typo fix - 'question mark' -> 'exclamation mark'
- DOC/MINOR: fix typo in management document
- MINOR: http: add a new function http_validate_scheme() to validate a scheme
- BUG/MAJOR: h2: verify early that non-http/https schemes match the valid syntax
- BUG/MAJOR: h2: verify that :path starts with a '/' before concatenating it
- BUG/MAJOR: h2: enforce stricter syntax checks on the :method pseudo-header
- BUG/MEDIUM: h2: give :authority precedence over Host
- REGTESTS: add a test to prevent h2 desync attacks
Rename the 'dontloglegacyconnerr' option to 'log-error-via-logformat'
which is much more self-explanatory and readable.
Note: only legacy keywords don't use hyphens, it is recommended to
separate words with them in new keywords.
Released version 2.5-dev3 with the following main changes :
- BUG/MINOR: arg: free all args on make_arg_list()'s error path
- BUG/MINOR: cfgcond: revisit the condition freeing mechanism to avoid a leak
- MEDIUM: proxy: remove long-broken 'option http_proxy'
- CLEANUP: http_ana: Remove now unused label from http_process_request()
- MINOR: deinit: always deinit the init_mutex on failed initialization
- BUG/MEDIUM: cfgcond: limit recursion level in the condition expression parser
- BUG/MEDIUM: mworker: do not register an exit handler if exit is expected
- BUG/MINOR: mworker: do not export HAPROXY_MWORKER_REEXEC across programs
- BUILD/MINOR: memprof fix macOs build.
- BUG/MEDIUM: ssl_sample: fix segfault for srv samples on invalid request
- BUG/MINOR: stats: Add missing agent stats on servers
- BUG/MINOR: check: fix the condition to validate a port-less server
- BUILD: threads: fix pthread_mutex_unlock when !USE_THREAD
- BUG/MINOR: resolvers: Use a null-terminated string to lookup in servers tree
- MINOR: ssl: use __objt_* variant when retrieving counters
- BUG/MINOR: systemd: must check the configuration using -Ws
- BUG/MINOR: mux-h1: Obey dontlognull option for empty requests
- BUG/MINOR: mux-h2: Obey dontlognull option during the preface
- BUG/MINOR: mux-h1: Be sure to swap H1C to splice mode when rcv_pipe() is called
- BUG/MEDIUM: mux-h2: Handle remaining read0 cases on partial frames
- MINOR: proxy: rename PR_CAP_LUA to PR_CAP_INT
- MINOR: mworker: the mworker CLI proxy is internal
- MINOR: stats: don't output internal proxies (PR_CAP_INT)
- CLEANUP: mworker: use the proxy helper functions in mworker_cli_proxy_create()
- CLEANUP: mworker: PR_CAP already initialized with alloc_new_proxy()
- BUG/MINOR: connection: Add missing error labels to conn_err_code_str
- MINOR: connection: Add a connection error code sample fetch
- MINOR: ssl: Enable error fetches in case of handshake error
- MINOR: ssl: Add new ssl_fc_hsk_err sample fetch
- MINOR: ssl: Define a default https log format
- MEDIUM: connection: Add option to disable legacy error log
- REGTESTS: ssl: Add tests for the connection and SSL error fetches
- REGTESTS: ssl: ssl_errors.vtc does not work with old openssl version
- BUG/MEDIUM: connection: close a rare race between idle conn close and takeover
- BUG/MEDIUM: pollers: clear the sleeping bit after waking up, not before
- BUG/MINOR: select: fix excess number of dead/skip reported
- BUG/MINOR: poll: fix abnormally high skip_fd counter
- BUG/MINOR: pollers: always program an update for migrated FDs
- BUG/MINOR: fd: protect fd state harder against a concurrent takeover
- DOC: internals: document the FD takeover process
- MINOR: fd: update flags only once in fd_update_events()
- MINOR: poll/epoll: move detection of RDHUP support earlier
- REORG: fd: uninline fd_update_events()
- MEDIUM: fd: rely more on fd_update_events() to detect changes
- BUG/MINOR: freq_ctr: use stricter barriers between updates and readings
- MEDIUM: atomic: simplify the atomic load/store/exchange operations
- MEDIUM: atomic: relax the load/store barriers on x86_64
- BUILD: opentracing: fixed build when using pkg-config utility
In case of connection failure, a dedicated error message is output,
following the format described in section "Error log format" of the
documentation. These messages cannot be configured through a log-format
option.
This patch adds a new option, "dontloglegacyconnerr", that disables
those error logs when set, and "replaces" them by a regular log line
that follows the configured log-format (thanks to a call to sess_log in
session_kill_embryonic).
The new fc_conn_err sample fetch allows to add the legacy error log
information into a regular log format.
This new option is unset by default so the logging logic will remain the
same until this new option is used.
This new sample fetch along the ssl_fc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the frontend's perspective). The errors happening during
the client's certificate verification will still be given by the
ssl_c_err and ssl_c_ca_err fetches. This new fetch will only hold errors
retrieved by the OpenSSL ERR_get_error function.
The fc_conn_err and fc_conn_err_str sample fetches give information
about the problem that made the connection fail. This information would
previously only have been given by the error log messages meaning that
thanks to these fetches, the error log can now be included in a custom
log format. The log strings were all found in the conn_err_code_str
function.
This option had always been broken in HTX, which means that the first
breakage appeared in 1.9, that it was broken by default in 2.0 and that
no workaround existed starting with 2.1. The way this option works is
praticularly unfit to the rest of the configuration and to the internal
architecture. It had some uses when it was introduced 14 years ago but
nowadays it's possible to do much better and more reliable using a
set of "http-request set-dst" and "http-request set-uri" rules, which
additionally are compatible with DNS resolution (via do-resolve) and
are not exclusive to normal load balancing. The "option-http_proxy"
example config file was updated to reflect this.
The option is still parsed so that an error message gives hints about
what to look for.
Released version 2.5-dev2 with the following main changes :
- BUILD/MEDIUM: tcp: set-mark support for OpenBSD
- DOC: config: use CREATE USER for mysql-check
- BUG/MINOR: stick-table: fix several printf sign errors dumping tables
- BUG/MINOR: peers: fix data_type bit computation more than 32 data_types
- MINOR: stick-table: make skttable_data_cast to use only std types
- MEDIUM: stick-table: handle arrays of standard types into stick-tables
- MEDIUM: peers: handle arrays of std types in peers protocol
- DOC: stick-table: add missing documentation about gpt0 stored type
- MEDIUM: stick-table: add the new array of gpt data_type
- MEDIUM: stick-table: make the use of 'gpt' excluding the use of 'gpt0'
- MEDIUM: stick-table: add the new arrays of gpc and gpc_rate
- MEDIUM: stick-table: make the use of 'gpc' excluding the use of 'gpc0/1''
- BUG/MEDIUM: sock: make sure to never miss early connection failures
- BUG/MINOR: cli: fix server name output in "show fd"
- Revert "MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules"
- MEDIUM: stats: include disabled proxies that hold active sessions to stats
- BUILD: stick-table: shut up invalid "uninitialized" warning in gcc 8.3
- MINOR: http: implement http_get_scheme
- MEDIUM: http: implement scheme-based normalization
- MEDIUM: h1-htx: apply scheme-based normalization on h1 requests
- MEDIUM: h2: apply scheme-based normalization on h2 requests
- REGTESTS: add http scheme-based normalization test
- BUILD: http_htx: fix ci compilation error with isdigit for Windows
- MINOR: http: implement http uri parser
- MINOR: http: use http uri parser for scheme
- MINOR: http: use http uri parser for authority
- REORG: http_ana: split conditions for monitor-uri in wait for request
- MINOR: http: use http uri parser for path
- BUG/MEDIUM: http_ana: fix crash for http_proxy mode during uri rewrite
- MINOR: mux_h2: define config to disable h2 websocket support
- CLEANUP: applet: remove unused thread_mask
- BUG/MINOR: ssl: Default-server configuration ignored by server
- BUILD: add detection of missing important CFLAGS
- BUILD: lua: silence a build warning with TCC
- MINOR: srv: extract tracking server config function
- MINOR: srv: do not allow to track a dynamic server
- MEDIUM: server: support track keyword for dynamic servers
- REGTESTS: test track support for dynamic servers
- MINOR: init: verify that there is a single word on "-cc"
- MINOR: init: make -cc support environment variables expansion
- MINOR: arg: add a free_args() function to free an args array
- CLEANUP: config: use free_args() to release args array in cfg_eval_condition()
- CLEANUP: hlua: use free_args() to release args arrays
- REORG: config: move the condition preprocessing code to its own file
- MINOR: cfgcond: start to split the condition parser to introduce terms
- MEDIUM: cfgcond: report invalid trailing chars after expressions
- MINOR: cfgcond: remerge all arguments into a single line
- MINOR: cfgcond: support negating conditional expressions
- MINOR: cfgcond: make the conditional term parser automatically allocate nodes
- MINOR: cfgcond: insert an expression between the condition and the term
- MINOR: cfgcond: support terms made of parenthesis around expressions
- REGTEST: make check_condition.vtc fail as soon as possible
- REGTESTS: add more complex check conditions to check_conditions.vtc
- BUG/MEDIUM: init: restore behavior of command-line "-m" for memory limitation
Now it's possible to form a term using parenthesis around an expression.
This will soon allow to build more complex expressions. For now they're
still pretty limited but parenthesis do work.
Now evaluating a condition will rely on an expression (or an empty string),
and this expression will support ORing a sub-expression with another
optional expression. The sub-expressions ANDs a term with another optional
sub-expression. With this alone precedence between && and || is respected,
and the following expression:
A && B && C || D || E && F || G
will naturally evaluate as:
(A && B && C) || D || (E && F) || G
Define a new global config statement named
"h2-workaround-bogus-websocket-clients".
This statement will disable the automatic announce of h2 websocket
support as specified in the RFC8441. This can be use to overcome clients
which fail to implement the relatively fresh RFC8441. Clients will in
his case automatically downgrade to http/1.1 for the websocket tunnel
if the haproxy configuration allows it.
This feature is relatively simple and can be backported up to 2.4, which
saw the introduction of h2 websocket support.
This reverts commit 19bbbe0562.
For now, set-src/set-src-port actions are directly performed on the client
connection. Using these actions at the stream level is really a problem with
HTTP connection (See #90) because all requests are affected by this change
and not only the current request. And it is worse with the H2, because
several requests can set their source address into the same connection at
the same time.
It is already an issue when these actions are called from "http-request"
rules. It is safer to wait a bit before adding the support to "tcp-request
content" rules. The solution is to be able to set src/dst address on the
stream and not on the connection when the action if performed from the L7
level..
Reverting the above commit means the issue #1303 is no longer fixed.
This patch must be backported in all branches containing the above commit
(as far as 2.0 for now).
This patch makes the use of 'gpc' excluding the use of the legacy
types 'gpc0' and 'gpc1" on the same table.
It also makes the use of 'gpc_rate' excluding the use of the legacy
types 'gpc0_rate' and 'gpc1_rate" on the same table.
The 'gpc0' and 'gpc1' related fetches and actions will apply
to the first two elements of the 'gpc' array if stored in table.
The 'gpc0_rate' and 'gpc1_rate' related fetches and actions will apply
to the first two elements of the 'gpc_rate' array if stored in table.
This patch adds the definition of two new array data_types:
'gpc': This is an array of 32bits General Purpose Counters.
'gpc_rate': This is an array on increment rates of General Purpose Counters.
Like for all arrays, they are limited to 100 elements.
This patch also adds actions and fetches to handle
elements of those arrays.
Note: As documented, those new actions and fetches won't
apply to the legacy 'gpc0', 'gpc1', 'gpc0_rate' nor 'gpc1_rate'.
This patch makes the use of 'gpt' excluding the use of the legacy
type 'gpt0' on the same table.
It also makes the 'gpt0' related fetches and actions applying
to the first element of the 'gpt' array if stored in table.
This patch adds the definition of a new array data_type
'gpt'. This is an array of 32bits General Purpose Tags.
Like for all arrays, it is limited to 100 elements.
This patch also adds actions and fetches to handle
elements of this array.
Note: As documented, those new actions and fetches won't
apply to the legacy 'gpt0' data type.
The store type 'gpt0' was present in code but was not documented.
The patch fix this and should be backported since 'gpt0' is supported.
[wt: ~1.6-dev4 hence all stable]
CREATE USER has been the standard way of creating users since
MySQL-5.0 (2005).
The current syntax of INSERT INTO mysql.user won't actually work
on MariaDB-10.4+.
Because haproxy doesn't use any resources the MySQL executable comment
syntax provides resource contraints to make it more palatable
to risk adverse users.
/*!50701 is a syntax recognised by MySQL and MariaDB 5.7.1+ when
resource contraints where added.
/*M!100201 is a MariaDB executable comment syntax recognised for MariaDB
for the 10.2.1 where the MAX_STATEMENT_TIME was added.
This patch may be backported as far as 2.0.
Released version 2.5-dev1 with the following main changes :
- CLEANUP: ssl: Move ssl_store related code to ssl_ckch.c
- MINOR: ssl: Allow duplicated entries in the cafile_tree
- MEDIUM: ssl: Chain ckch instances in ca-file entries
- MINOR: ssl: Add reference to default ckch instance in bind_conf
- MINOR: ssl: Add helper functions to create/delete cafile entries
- MEDIUM: ssl: Add a way to load a ca-file content from memory
- MINOR: ssl: Add helper function to add cafile entries
- MINOR: ssl: Ckch instance rebuild and cleanup factorization in CLI handler
- MEDIUM: ssl: Add "set+commit ssl ca-file" CLI commands
- REGTESTS: ssl: Add new ca-file update tests
- MINOR: ssl: Add "abort ssl ca-file" CLI command
- MINOR: ssl: Add a cafile_entry type field
- MINOR: ssl: Refactorize the "show certificate details" code
- MEDIUM: ssl: Add "show ssl ca-file" CLI command
- MEDIUM: ssl: Add "new ssl ca-file" CLI command
- MINOR: ssl: Add "del ssl ca-file" CLI command
- REGTESTS: ssl: Add "new/del ssl ca-file" tests
- DOC: ssl: Add documentation about CA file hot update commands
- DOC: internals: update the SSL architecture schema
- MINOR: ssl: Chain instances in ca-file entries
- MEDIUM: ssl: Add "set+commit ssl crl-file" CLI commands
- MEDIUM: ssl: Add "new+del crl-file" CLI commands
- MINOR: ssl: Add "abort ssl crl-file" CLI command
- MEDIUM: ssl: Add "show ssl crl-file" CLI command
- REGTESTS: ssl: Add "new/del ssl crl-file" tests
- REGTESTS: ssl: Add "set/commit ssl crl-file" test
- DOC: ssl: Add documentation about CRL file hot update commands
- BUILD/MINOR: ssl: Fix compilation with SSL enabled
- BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
- CI: introduce scripts/build-vtest.sh for installing VTest
- CLEANUP: ssl: Fix coverity issues found in CA file hot update code
- CI: github actions: add OpenTracing builds
- BUG/MEDIUM: ebtree: Invalid read when looking for dup entry
- BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
- BUILD/MINOR: opentracing: fixed build when using clang
- BUG/MEDIUM: filters: Exec pre/post analysers only one time per filter
- BUG/MINOR: http-comp: Preserve HTTP_MSGF_COMPRESSIONG flag on the response
- MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
- CLEANUP: pattern: remove export of non-existent function pattern_delete()
- MINOR: h1-htx: Update h1 parsing functions to return result as a size_t
- MEDIUM: h1-htx: Adapt H1 data parsing to copy wrapping data in one call
- MINOR: mux-h1/mux-fcgi: Don't needlessly loop on data parsing
- MINOR: h1-htx: Move HTTP chunks parsing into a dedicated function
- MEDIUM: h1-htx: Split function to parse a chunk and the loop on the buffer
- MEDIUM: h1-htx: Add a function to parse contiguous small chunks
- MINOR: h1-htx: Use a correlation table to speed-up small chunks parsing
- MINOR: buf: Add function to realign a buffer with a specific head position
- MINOR: muxes/h1-htx: Realign input buffer using b_slow_realign_ofs()
- CLEANUP: mux-h1: Rename functions parsing input buf and filling output buf
- Revert "MEDIUM: http-ana: Deal with L7 retries in HTTP analysers"
- BUG/MINOR: http-ana: Send the right error if max retries is reached on L7 retry
- BUG/MINOR: http-ana: Handle L7 retries on refused early data before K/A aborts
- MINOR: http-ana: Perform L7 retries because of status codes in response analyser
- MINOR: cfgparse: Fail when encountering extra arguments in macro
- DOC: intro: Fix typo in starter guide
- BUG/MINOR: server: Missing calloc return value check in srv_parse_source
- BUG/MINOR: peers: Missing calloc return value check in peers_register_table
- BUG/MINOR: ssl: Missing calloc return value check in ssl_init_single_engine
- BUG/MINOR: http: Missing calloc return value check in parse_http_req_capture
- BUG/MINOR: proxy: Missing calloc return value check in proxy_parse_declare
- BUG/MINOR: proxy: Missing calloc return value check in proxy_defproxy_cpy
- BUG/MINOR: http: Missing calloc return value check while parsing tcp-request/tcp-response
- BUG/MINOR: http: Missing calloc return value check while parsing tcp-request rule
- BUG/MINOR: compression: Missing calloc return value check in comp_append_type/algo
- BUG/MINOR: worker: Missing calloc return value check in mworker_env_to_proc_list
- BUG/MINOR: http: Missing calloc return value check while parsing redirect rule
- BUG/MINOR: http: Missing calloc return value check in make_arg_list
- BUG/MINOR: proxy: Missing calloc return value check in chash_init_server_tree
- CLEANUP: http-ana: Remove useless if statement about L7 retries
- BUG/MAJOR: stream-int: Release SI endpoint on server side ASAP on retry
- MINOR: backend: Don't release SI endpoint anymore in connect_server()
- BUG/MINOR: vars: Be sure to have a session to get checks variables
- DOC/MINOR: move uuid in the configuration to the right alphabetical order
- CLEANUP: mux-fcgi: Don't needlessly store result of data/trailers parsing
- BUILD: fix compilation for OpenSSL-3.0.0-alpha17
- MINOR: http-ana: Use -1 status for client aborts during queuing and connect
- REGTESTS: Fix http_abortonclose.vtc to support -1 status for some client aborts
- CLEANUP: backend: fix incorrect comments on locking conditions for lb functions
- CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests
- CI: github actions: add OpenSSL-3.0.0 builds
- CI: github actions: -Wno-deprecated-declarations with OpenSSL 3.0.0
- MINOR: errors: allow empty va_args for diag variadic macro
- REORG: errors: split errors reporting function from log.c
- CLEANUP: server: fix cosmetic of error message on sni parsing
- MEDIUM: errors: implement user messages buffer
- MINOR: log: do not discard stderr when starting is over
- MEDIUM: errors: implement parsing context type
- MINOR: errors: use user messages context in print_message
- MINOR: log: display exec path on first warning
- MINOR: errors: specify prefix "config" for parsing output
- MINOR: log: define server user message format
- REORG: server: use parsing ctx for server parsing
- REORG: config: use parsing ctx for server config check
- MINOR: server: use parsing ctx for server init addr
- MINOR: server: use ha_alert in server parsing functions
- DOC: use the req.ssl_sni in examples
- CLEANUP: cfgparse: Remove duplication of `MAX_LINE_ARGS + 1`
- CLEANUP: tools: Make errptr const in `parse_line()`
- MINOR: haproxy: Add `-cc` argument
- BUG: errors: remove printf positional args for user messages context
- CI: Make matrix.py executable and add shebang
- BUILD: make tune.ssl.keylog available again
- BUG/MINOR: ssl: OCSP stapling does not work if expire too far in the future
- Revert "BUG/MINOR: opentracing: initialization after establishing daemon mode"
- BUG/MEDIUM: opentracing: initialization before establishing daemon and/or chroot mode
- SCRIPTS: opentracing: enable parallel builds in build-ot.sh
- BUG/MEDIUM: compression: Fix loop skipping unused blocks to get the next block
- BUG/MEDIUM: compression: Properly get the next block to iterate on payload
- BUG/MEDIUM: compression: Add a flag to know the filter is still processing data
- MINOR: ssl: Keep the actual key length in the certificate_ocsp structure
- MINOR: ssl: Add new "show ssl ocsp-response" CLI command
- MINOR: ssl: Add the OCSP entry key when displaying the details of a certificate
- MINOR: ssl: Add the "show ssl cert foo.pem.ocsp" CLI command
- REGTESTS: ssl: Add "show ssl ocsp-response" test
- BUG/MINOR: server: explicitly set "none" init-addr for dynamic servers
- BUG/MINOR: pools: fix a possible memory leak in the lockless pool_flush()
- BUG/MINOR: pools: make DEBUG_UAF always write to the to-be-freed location
- MINOR: pools: do not maintain the lock during pool_flush()
- MINOR: pools: call malloc_trim() under thread isolation
- MEDIUM: pools: use a single pool_gc() function for locked and lockless
- BUG/MAJOR: pools: fix possible race with free() in the lockless variant
- CLEANUP: pools: remove now unused seq and pool_free_list
- MEDIUM: pools: remove the locked pools implementation
- BUILD: ssl: Fix compilation with BoringSSL
- BUG/MEDIUM: errors: include missing obj_type file
- REGTESTS: ssl: show_ssl_ocspresponce.vtc is broken with BoringSSL
- BUG/MAJOR: htx: Fix htx_defrag() when an HTX block is expanded
- BUG/MINOR: mux-fcgi: Expose SERVER_SOFTWARE parameter by default
- BUG/MINOR: h1-htx: Fix a signess bug with char data type when parsing chunk size
- CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
- BUG/MINOR: resolvers: answser item list was randomly purged or errors
- MEDIUM: resolvers: add a ref on server to the used A/AAAA answer item
- MEDIUM: resolvers: add a ref between servers and srv request or used SRV record
- BUG/MINOR: server-state: load SRV resolution only if params match the config
- MINOR: config: remove support for deprecated option "tune.chksize"
- MINOR: config: completely remove support for "no option http-use-htx"
- MINOR: log: remove the long-deprecated early log-format tags
- MINOR: http: remove the long deprecated "set-cookie()" sample fetch function
- MINOR: config: reject long-deprecated "option forceclose"
- MINOR: config: remove deprecated option "http-tunnel"
- MEDIUM: proxy: remove the deprecated "grace" keyword
- MAJOR: config: remove parsing of the global "nbproc" directive
- BUILD: init: remove initialization of multi-process thread mappings
- BUILD: log: remove unused fmt_directive()
- REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests
- REGTESTS: Remove REQUIRE_VERSION=1.7 from all tests
- CI: github actions: enable alpine/musl builds
- BUG/MAJOR: resolvers: segfault using server template without SRV RECORDs
- DOC: lua: Add a warning about buffers modification in HTTP
- MINOR: ssl: Use OpenSSL's ASN1_TIME convertor when available
- BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
- BUG/MEDIUM: server: extend thread-isolate over much of CLI 'add server'
- BUG/MEDIUM: server: clear dynamic srv on delete from proxy id/name trees
- BUG/MEDIUM: server: do not forget to generate the dynamic servers ids
- BUG/MINOR: server: do not keep an invalid dynamic server in px ids tree
- BUG/MEDIUM: server: do not auto insert a dynamic server in px addr_node
- BUG/MEDIUM: shctx: use at least thread-based locking on USE_PRIVATE_CACHE
- BUG/MINOR: ssl: use atomic ops to update global shctx stats
- BUG/MINOR: mworker: fix typo in chroot error message
- CLEANUP: global: remove unused definition of stopping_task[]
- MEDIUM: init: remove the loop over processes during init
- MINOR: mworker: remove the initialization loop over processes
- CLEANUP: global: remove the nbproc field from the global structure
- CLEANUP: global: remove pid_bit and all_proc_mask
- MEDIUM: global: remove dead code from nbproc/bind_proc removal
- MEDIUM: config: simplify cpu-map handling
- MEDIUM: cpu-set: make the proc a single bit field and not an array
- CLEANUP: global: remove unused definition of MAX_PROCS
- MEDIUM: global: remove the relative_pid from global and mworker
- DOC: update references to process numbers in cpu-map and bind-process
- MEDIUM: config: warn about "bind-process" deprecation
- CLEANUP: shctx: remove the different inter-process locking techniques
- BUG/MAJOR: queue: set SF_ASSIGNED when setting strm->target on dequeue
- MINOR: backend: only skip LB when there are actual connections
- BUG/MINOR: mux-h1: do not skip the error response on bad requests
- MINOR: connection: add helper conn_append_debug_info()
- MINOR: mux-h2/trace: report a few connection-level info during h2_init()
- CLEANUP: mux-h2/traces: better align user messages
- BUG/MINOR: stats: make "show stat typed desc" work again
- MINOR: mux-h2: obey http-ignore-probes during the preface
- BUG/MINOR: mux-h2/traces: bring back the lost "rcvd H2 REQ" trace
- BUG/MINOR: mux-h2/traces: bring back the lost "sent H2 REQ/RES" traces
- CLEANUP: assorted typo fixes in the code and comments
- CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'
- REGTESTS: Replace REQUIRE_VERSION=2.5 with 'haproxy -cc'
- REGTESTS: Replace REQUIRE_OPTIONS with 'haproxy -cc' for 2.5+ tests
- REGTESTS: Replace REQUIRE_BINARIES with 'command -v'
- REGTESTS: Remove support for REQUIRE_BINARIES
- CI: ssl: enable parallel builds for OpenSSL on Linux
- CI: ssl: do not needlessly build the OpenSSL docs
- CI: ssl: keep the old method for ancient OpenSSL versions
- CLEANUP: server: a separate function for initializing the per_thr field
- BUG/MINOR: server: Forbid to set fqdn on the CLI if SRV resolution is enabled
- BUG/MEDIUM: server/cli: Fix ABBA deadlock when fqdn is set from the CLI
- MINOR: resolvers: Clean server in a dedicated function when removing a SRV item
- MINOR: resolvers: Remove server from named_servers tree when removing a SRV item
- BUG/MEDIUM: resolvers: Add a task on servers to check SRV resolution status
- BUG/MINOR: backend: restore the SF_SRV_REUSED flag original purpose
- BUG/MINOR: backend: do not set sni on connection reuse
- BUG/MINOR: resolvers: Use resolver's lock in resolv_srvrq_expire_task()
- BUG/MINOR: server/cli: Fix locking in function processing "set server" command
- BUG/MINOR: cache: Correctly handle existing-but-empty 'accept-encoding' header
- MINOR: ssl: fix typo in usage for 'new ssl ca-file'
- MINOR: ssl: always initialize random generator
- MINOR: ssl: check allocation in ssl_sock_init_srv
- MINOR: ssl: check allocation in parse ciphers/ciphersuites/verifyhost
- MINOR: ssl: check allocation in parse npn/sni
- MINOR: server: disable CLI 'set server ssl' for dynamic servers
- MINOR: ssl: render file-access optional on server crt loading
- MINOR: ssl: split parse functions for alpn/check-alpn
- MINOR: ssl: support ca-file arg for dynamic servers
- MINOR: ssl: support crt arg for dynamic servers
- MINOR: ssl: support crl arg for dynamic servers
- MINOR: ssl: enable a series of ssl keywords for dynamic servers
- MINOR: ssl: support ssl keyword for dynamic servers
- REGTESTS: server: test ssl support for dynamic servers
- MINOR: queue: update the stream's pend_pos before queuing it
- CLEANUP: Prevent channel-t.h from being detected as C++ by GitHub
- BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check
- REGTESTS: fix maxconn update with agent-check
- MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn
- MINOR: queue: update proxy->served once out of the loop
- MEDIUM: queue: refine the locking in process_srv_queue()
- MINOR: lb/api: remove the locked argument from take_conn/drop_conn
- MINOR: queue: create a new structure type "queue"
- MINOR: proxy: replace the pendconns-related stuff with a struct queue
- MINOR: server: replace the pendconns-related stuff with a struct queue
- MEDIUM: queue: use a dedicated lock for the queues
- MEDIUM: queue: simplify again the process_srv_queue() API
- MINOR: queue: factor out the proxy/server queuing code
- MINOR: queue: use atomic-ops to update the queue's index
- MEDIUM: queue: determine in process_srv_queue() if the proxy is usable
- MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()
- MEDIUM: queue: unlock as soon as possible
- MINOR: queue: make pendconn_first() take the lock by itself
- CLEANUP: backend: remove impossible case of round-robin + consistent hash
- MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
- DOC: config: Add missing actions in "tcp-request session" documentation
- CLEANUP: dns: Remove a forgotten debug message
- DOC: Replace issue templates by issue forms
- Revert "MINOR: queue: make pendconn_first() take the lock by itself"
- Revert "MEDIUM: queue: unlock as soon as possible"
- Revert "MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()"
- Revert "MEDIUM: queue: determine in process_srv_queue() if the proxy is usable"
- Revert "MINOR: queue: use atomic-ops to update the queue's index"
- Revert "MINOR: queue: factor out the proxy/server queuing code"
- Revert "MEDIUM: queue: simplify again the process_srv_queue() API"
- Revert "MEDIUM: queue: use a dedicated lock for the queues"
- Revert "MEDIUM: queue: refine the locking in process_srv_queue()"
- Revert "MINOR: queue: update proxy->served once out of the loop"
- Revert "MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn"
- MEDIUM: queue: update px->served and lb's take_conn once per loop
- MEDIUM: queue: use a dedicated lock for the queues (v2)
- MEDIUM: queue: simplify again the process_srv_queue() API (v2)
- MEDIUM: queue: determine in process_srv_queue() if the proxy is usable (v2)
- MINOR: queue: factor out the proxy/server queuing code (v2)
- MINOR: queue: use atomic-ops to update the queue's index (v2)
- MEDIUM: queue: take the proxy lock only during the px queue accesses
- MEDIUM: queue: use a trylock on the server's queue
- MINOR: queue: add queue_init() to initialize a queue
- MINOR: queue: add a pointer to the server and the proxy in the queue
- MINOR: queue: store a pointer to the queue into the pendconn
- MINOR: queue: remove the px/srv fields from pendconn
- MINOR: queue: simplify pendconn_unlink() regarding srv vs px
- BUG: backend: stop looking for queued connections once there's no more
- BUG/MINOR: queue/debug: use the correct lock labels on the queue lock
- BUG/MINOR: resolvers: Always attach server on matching record on resolution
- BUG/MINOR: resolvers: Reset server IP when no ip is found in the response
- MINOR: resolvers: Reset server IP on error in resolv_get_ip_from_response()
- BUG/MINOR: checks: return correct error code for srv_parse_agent_check
- BUILD: Makefile: fix linkage for Haiku.
- BUG/MINOR: tcpcheck: Fix numbering of implicit HTTP send/expect rules
- MINOR: http-act/tcp-act: Add "set-log-level" for tcp content rules
- MINOR: http-act/tcp-act: Add "set-nice" for tcp content rules
- MINOR: http-act/tcp-act: Add "set-mark" and "set-tos" for tcp content rules
- CLEANUP: tcp-act: Sort action lists
- BUILD/MEDIUM: tcp: set-mark setting support for FreeBSD.
- BUILD: tcp-act: avoid warning when set-mark / set-tos are not supported
- BUG/MINOR: mqtt: Fix parser for string with more than 127 characters
- BUG/MINOR: mqtt: Support empty client ID in CONNECT message
- BUG/MEDIUM: resolvers: Make 1st server of a template take part to SRV resolution
- CLEANUP: peers: re-write intdecode function comment.
It is now possible to set the Netfilter MARK and the TOS field value in all
packets sent to the client from any tcp-request rulesets or the "tcp-response
content" one. To do so, the parsing of "set-mark" and "set-tos" actions are
moved in tcp_act.c and the actions evaluation is handled in dedicated functions.
This patch may be backported as far as 2.2 if necessary.
It is now possible to set the "nice" factor of the current stream from a
"tcp-request content" or "tcp-response content" ruleset. To do so, the
action parsing is moved in stream.c and the action evaluation is handled in
a dedicated function.
This patch may be backported as far as 2.2 if necessary.
It is now possible to set the stream log level from a "tcp-request content"
or "tcp-response content" ruleset. To do so, the action parsing is moved in
stream.c and the action evaluation is handled in a dedicated function.
This patch should fix issue #1306. It may be backported as far as 2.2 if
necessary.
set-src/set-src-port and set-dst/set-dst-port actions were not listed in the
documentation of "tcp-request session".
This patch may be backported to all stable versions.
If it possible to set source IP/Port from "tcp-request connection",
"tcp-request session" and "http-request" rules but not from "tcp-request
content" rules. There is no reason for this limitation and it may be a
problem for anyone wanting to call a lua fetch to dynamically set source
IP/Port from a TCP proxy. Indeed, to call a lua fetch, we must have a
stream. And there is no stream when "tcp-request connection/session" rules
are evaluated.
Thanks to this patch, "set-src" and "set-src-port" action are now supported
by "tcp_request content" rules.
This patch is related to the issue #1303. It may be backported to all stable
versions.
This one was deprecated in 2.3 and marked for removal in 2.5. It suffers
too many limitations compared to threads, and prevents some improvements
from being engaged. Instead of a bypassable startup error, there is now
a hard error.
The parsing code was removed, and very few obvious cases were as well.
The code is deeply rooted at certain places (e.g. "for" loops iterating
from 0 to nbproc) so it will not be that trivial to remove everywhere.
The "bind" and "bind-process" parsers will have to be adjusted, though
maybe not completely changed if we later want to support thread groups
for large NUMA machines. Some stats socket restrictions were removed,
and the doc was updated according to what was done. A few places in the
doc still refer to nbproc and will have to be revisited. The master-worker
code also refers to the process number to distinguish between master and
workers and will have to be carefully adjusted. The MAX_PROCS macro was
reset to 1, this will at least reduce the size of some remaining arrays.
Two regtests were dependieng on this directive, one with an explicit
"nbproc 1" and another one testing the master's CLI using nbproc 4.
Both were adapted.
Commit ab0a5192a ("MEDIUM: config: mark "grace" as deprecated") marked
the "grace" keyword as deprecated in 2.3, tentative removal for 2.4
with a hard deadline in 2.5, so let's remove it and return an error now.
This old and outdated feature was incompatible with soft-stop, reload
and socket transfers, and keeping it forced ugly hacks in the lower
layers of the protocol stack.
It was marked as deprecated for immediate removal as it was not used,
let's reject it and remove it from the doc. A specific error suggests
to check tune.bufsize instead.
As specified in the RFC3875 (section 4.1.17), this parameter must be set to
the name and version of the information server software making the CGI
request. Thus, it is now added to the default parameters defined by
HAProxy. It is set to the string "HAProxy $version".
This patch should fix the issue #1285 and must be backported as far as 2.2.
This patch adds the `-cc` (check condition) argument to evaluate conditions on
startup and return the result as the exit code.
As an example this can be used to easily check HAProxy's version in scripts:
haproxy -cc 'version_atleast(2.4)'
This resolves GitHub issue #1246.
Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
Released version 2.4.0 with the following main changes :
- BUG/MINOR: http_fetch: fix possible uninit sockaddr in fetch_url_ip/port
- CLEANUP: cli/activity: Remove double spacing in set profiling command
- CI: Build VTest with clang
- CI: extend spellchecker whitelist, add "ists" as well
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: memprof: properly account for differences for realloc()
- MINOR: memprof: also report the method used by each call
- MINOR: memprof: also report the totals and delta alloc-free
- CLEANUP: pattern: remove the unused and dangerous pat_ref_reload()
- BUG/MINOR: http_act: Fix normalizer names in error messages
- MINOR: uri_normalizer: Add `fragment-strip` normalizer
- MINOR: uri_normalizer: Add `fragment-encode` normalizer
- IMPORT: slz: use the generic function for the last bytes of the crc32
- IMPORT: slz: do not produce the crc32_fast table when CRC is natively supported
- BUILD/MINOR: opentracing: fixed compilation with filter enabled
- BUILD: makefile: add a few popular ARMv8 CPU targets
- BUG/MEDIUM: stick_table: fix crash when using tcp smp_fetch_src
- REGTESTS: stick-table: add src_conn_rate test
- CLEANUP: stick-table: remove a leftover of an old keyword declaration
- BUG/MINOR: stats: fix lastchk metric that got accidently lost
- EXAMPLES: add a "basic-config-edge" example config
- EXAMPLES: add a trivial config for quick testing
- DOC: management: Correct example reload command in the document
- Revert "CI: Build VTest with clang"
- MINOR: activity/cli: optionally support sorting by address on "show profiling"
- DEBUG: ssl: export ssl_sock_close() to see its symbol resolved in profiling
- BUG/MINOR: lua/vars: prevent get_var() from allocating a new name
- DOC: config: Fix configuration example for mqtt
- BUG/MAJOR: config: properly initialize cpu_map.thread[] up to MAX_THREADS
- BUILD: config: avoid a build warning on numa_detect_topology() without threads
- DOC: update min requirements in INSTALL
- IMPORT: slz: use inttypes.h instead of stdint.h
- BUILD: sample: use strtoll() instead of atoll()
- MINOR: version: mention that it's LTS now.
Released version 2.4-dev19 with the following main changes :
- BUG/MINOR: hlua: Don't rely on top of the stack when using Lua buffers
- BUG/MEDIUM: cli: prevent memory leak on write errors
- BUG/MINOR: ssl/cli: fix a lock leak when no memory available
- MINOR: debug: add a new "debug dev sym" command in expert mode
- MINOR: pools/debug: slightly relax DEBUG_DONT_SHARE_POOLS
- CI: Github Actions: switch to LibreSSL-3.3.3
- MINOR: srv: close all idle connections on shutdown
- MINOR: connection: move session_list member in a union
- MEDIUM: mux_h1: release idling frontend conns on soft-stop
- MEDIUM: connection: close front idling connection on soft-stop
- MINOR: tools: add functions to retrieve the address of a symbol
- CLEANUP: activity: mark the profiling and task_profiling_mask __read_mostly
- MINOR: activity: add a "memory" entry to "profiling"
- MINOR: activity: declare the storage for memory usage statistics
- MEDIUM: activity: collect memory allocator statistics with USE_MEMORY_PROFILING
- MINOR: activity: clean up the show profiling io_handler a little bit
- MINOR: activity: make "show profiling" support a few arguments
- MINOR: activity: make "show profiling" also dump the memoery usage
- MINOR: activity: add the profiling.memory global setting
- BUILD: makefile: add new option USE_MEMORY_PROFILING
- MINOR: channel: Rely on HTX version if appropriate in channel_may_recv()
- BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive
- MINOR: conn-stream: Force mux to wait for read events if abortonclose is set
- MEDIUM: mux-h1: Don't block reads when waiting for the other side
- BUG/MEDIUM: mux-h1: Properly report client close if abortonclose option is set
- REGTESTS: Add script to test abortonclose option
- MINOR: mux-h1: clean up conditions to enabled and disabled splicing
- MINOR: mux-h1: Subscribe for sends if output buffer is not empty in h1_snd_pipe
- MINOR: mux-h1: Always subscribe for reads when splicing is disabled
- MEDIUM: mux-h1: Wake H1 stream when both sides a synchronized
- CLEANUP: mux-h1: rename WAIT_INPUT/WAIT_OUTPUT flags
- MINOR: mux-h1: Manage processing blocking flags on the H1 stream
- BUG/MINOR: stream: Decrement server current session counter on L7 retry
- BUG/MINOR: config: fix uninitialized initial state in ".if" block evaluator
- BUG/MINOR: config: add a missing "ELIF_TAKE" test for ".elif" condition evaluator
- BUG/MINOR: config: .if/.elif should also accept negative integers
- MINOR: config: centralize the ".if"/".elif" condition parser and evaluator
- MINOR: config: keep up-to-date current file/line/section in the global struct
- MINOR: config: support some pseudo-variables for file/line/section
- BUILD: activity: do not include malloc.h
- MINOR: arg: improve the error message on missing closing parenthesis
- MINOR: global: export the build features string list
- MINOR: global: add version comparison functions
- MINOR: config: improve .if condition error reporting
- MINOR: config: make cfg_eval_condition() support predicates with arguments
- MINOR: config: add predicate "defined()" to conditional expression blocks
- MINOR: config: add predicates "streq()" and "strneq()" to conditional expressions
- MINOR: config: add predicate "feature" to detect certain built-in features
- MINOR: config: add predicates "version_atleast" and "version_before" to cond blocks
- BUG/MINOR: activity: use the new pointer to calculate the new size in realloc()
- BUG/MINOR: stream: properly clear the previous error mask on L7 retries
- MEDIUM: log: slightly refine the output format of alerts/warnings/etc
- MINOR: config: add a new message directive: .diag
- CLEANUP: cli/tree-wide: properly re-align the CLI commands' help messages
- BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
- BUG/MINOR: checks: Handle synchronous connect when a tcpcheck is started
- BUG/MINOR: checks: Reschedule check on observe mode only if fastinter is set
- MINOR: global: define tainted flag
- MINOR: cfgparse: add a new field flags in cfg_keyword
- MINOR: cfgparse: implement experimental config keywords
- MINOR: action: replace match_pfx by a keyword flags field
- MINOR: action: implement experimental actions
- MINOR: cli: set tainted when using CLI expert/experimental mode
- MINOR: stats: report tainted on show info
- MINOR: http_act: mark normalize-uri as experimental
- BUILD: fix usage of ha_alert without format string
- MINOR: proxy: define PR_CAP_LB
- BUG/MINOR: server: do not report diag for peer servers with null weight
- DOC: ssl: Extra files loading now works for backends too
- ADDONS: make addons/ discoverable by git via .gitignore
- DOC: ssl: Add information about crl-file option
- MINOR: sample: improve error reporting on missing arg to strcmp() converter
- DOC: management: mention that some fields may be emitted as floats
- MINOR: tools: implement trimming of floating point numbers
- MINOR: tools: add a float-to-ascii conversion function
- MINOR: freq_ctr: add new functions to report float measurements
- MINOR: stats: avoid excessive padding of float values with trailing zeroes
- MINOR: stats: add the HTML conversion for float types
- MINOR: stats: pass the appctx flags to stats_fill_info()
- MINOR: stats: support an optional "float" option to "show info"
- MINOR: stats: use tv_remain() to precisely compute the uptime
- MINOR: stats: report uptime and start time as floats with subsecond resolution
- MINOR: stats: make "show info" able to report rates as floats when asked
- MINOR: config: mark tune.fd.edge-triggered as experimental
- REORG: vars: move the "proc" scope variables out of the global struct
- REORG: threads: move all_thread_mask() to thread.h
- BUILD: wdt: include signal-t.h
- BUILD: auth: include missing list.h
- REORG: mworker: move proc_self from global to mworker
- BUILD: ssl: ssl_utils requires chunk.h
- BUILD: config: cfgparse-ssl.c needs tools.h
- BUILD: wurfl: wurfl.c needs tools.h
- BUILD: spoe: flt_spoe.c needs tools.h
- BUILD: promex: service-prometheus.c needs tools.h
- BUILD: resolvers: include tools.h
- BUILD: config: include tools.h in cfgparse-listen.c
- BUILD: htx: include tools.h in http_htx.c
- BUILD: proxy: include tools.h in proxy.c
- BUILD: session: include tools.h in session.c
- BUILD: cache: include tools.h in cache.c
- BUILD: sink: include tools.h in sink.c
- BUILD: connection: include tools.h in connection.c
- BUILD: server-state: include tools.h from server_state.c
- BUILD: dns: include tools.h in dns.c
- BUILD: payload: include tools.h in payload.c
- BUILD: vars: include tools.h in vars.c
- BUILD: compression: include tools.h in compression.c
- BUILD: mworker: include tools.h from mworker.c
- BUILD: queue: include tools.h from queue.c
- BUILD: udp: include tools.h from proto_udp.c
- BUILD: stick-table: include freq_ctr.h from stick_table.h
- BUILD: server: include tools.h from server.c
- BUILD: server: include missing proxy.h in server.c
- BUILD: sink: include proxy.h in sink.c
- BUILD: mworker: include proxy.h in mworker.c
- BUILD: filters: include proxy.h in filters.c
- BUILD: fcgi-app: include proxy.h in fcgi-app.c
- BUILD: connection: move list_mux_proto() to connection.c
- REORG: stick-table: uninline stktable_alloc_data_type()
- REORG: stick-table: move composite address functions to stick_table.h
- REORG: config: uninline warnifnotcap() and failifnotcap()
- BUILD: task: remove unused includes from task.c
- MINOR: task: stop including stream.h from task.c
- BUILD: connection: stop including listener-t.h
- BUILD: hlua: include proxy.h from hlua.c
- BUILD: mux-h1: include proxy.h from mux-h1.c
- BUILD: mux-fcgi: include proxy.h from mux-fcgi.c
- BUILD: listener: include proxy.h from listener.c
- BUILD: http-rules: include proxy.h from http_rules.c
- BUILD: thread: include log.h from thread.c
- BUILD: comp: include proxy.h from flt_http_comp.c
- BUILD: fd: include log.h from fd.c
- BUILD: config: do not include proxy.h nor errors.h anymore in cfgparse.h
- BUILD: makefile: reorder object files by build time
- DOC: Fix a few grammar/spelling issues and casing of HAProxy
- REGTESTS: run-regtests: match both "HAProxy" and "HA-Proxy" in the version
- MINOR: version: report "HAProxy" not "HA-Proxy" in the version output
- DOC: remove last occurrences of "HA-Proxy" syntax
- DOC: peers: fix the protocol tag name in the doc
- ADMIN: netsnmp: report "HAProxy" and not "Haproxy" in output descriptions
- MEDIUM: mailers: use "HAProxy" nor "HAproxy" in the subject of messages
- DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
- MINOR: tools/rnd: compute the result outside of the CAS loop
- BUILD: http_fetch: address a few aliasing warnings with older compilers
- BUILD: ssl: define HAVE_CRYPTO_memcmp() based on the library version
- BUILD: errors: include stdarg in errors.h
- REGTESTS: disable inter-thread idle connection sharing on sensitive tests
- MINOR: cli: make "help" support a command in argument
- MINOR: cli: sort the output of the "help" keywords
- CLEANUP: cli/mworker: properly align the help messages
- BUILD: memprof: make the old caller pointer a const in get_prof_bin()
- BUILD: compat: include malloc_np.h for USE_MEMORY_PROFILING on FreeBSD
- CI: Github Actions: enable USE_QUIC=1 for BoringSSL builds
- BUG/MEDIUM: quic: fix null deref on error path in qc_conn_init()
- BUILD: cli: appease a null-deref warning in cli_gen_usage_msg()
This patch fixes a few grammar and spelling issues in configuration.txt.
It was also noted that there was a wide range of case usage
(i.e. haproxy, HAproxy, HAProxy, etc... ). This patch updates them
all to be consistently "HAProxy" except where a binary is mentioned.
When using the crl-file option with multiple Certificate Authority
levels in the CA chain, there must be one CRL per CA or the verify
function on the backend side will raise an "unagle to get certificate
CRL" error (error code 3).
This was required by GitHub issue #1201.
When implementing the server side certificate hot update, the ckch
mechanism was used on the backend side in order to mimic the frontend
certificate management and to enable server line certificate update via
the CLI (see GitHub issue #427). As an unexpected side effect, we now
also look for ssl extra files (cert.pem.key, cert.pem.ocsp ...) for the
backend side.
This patch updates the documentation accordingly.
This answers to GitHub issue #845.
normalize-uri http rule is marked as experimental, so it cannot be
activated without the global 'expose-experimental-directives'. The
associated vtc is updated to be able to use it.
Add a new flag to mark a keyword as experimental. An experimental
keyword cannot be used if the global 'expose-experimental-directives' is
not present first.
Only keywords parsed through a standard cfg_keywords lists in
global/proxies section will be automatically detected if declared
experimental. To support a keyword outside of these lists,
check_kw_experimental must be called manually during its parsing.
If an experimental keyword is present in the config, the tainted flag is
updated.
For the moment, no keyword is marked as experimental.
This one works just like .notice/.warning/.alert except that it prints
the message at level "DIAG" only when haproxy runs in diagnostic mode
(-dD). This can be convenient for example to pass a few hints to help
locate certain config parts or to leave messages about certain temporary
workarounds.
Example:
.diag "WTA/2021-05-07: $.LINE: replace 'redirect' with 'return' after final switch to 2.4"
http-request redirect location /goaway if ABUSE
These predicates respectively verify that the current version is at least
a given version or is before a specific one. The syntax is exactly the one
reported by "haproxy -v", though each component is optional, so both "1.5"
and "2.4-dev18-88910-48" are supported. Missing components equal zero, and
"dev" is below "pre" or "rc", which are both inferior to no such mention
(i.e. they are negative). Thus "2.4-dev18" is older than "2.4-rc1" which
is older than "2.4".
The "feature(name)" predicate will return true if <name> corresponds to
a name listed after a '+' in the features list, that is it was enabled at
build time with USE_<name>=1. Typical use cases will include OPENSSL, LUA
and LINUX_SPLICE. But maybe it will also be convenient to use with optional
addons such as PROMEX and the device detection modules to help keeping the
same configs across various deployments.
"streq(str1,str2)" will return true if the two strings match while
"strneq(str1,str2)" will return true only if they differ. This is
convenient to match an environment variable against a predefined value.
The new pseudo-variables ".FILE", ".LINE" and ".SECTION" will be resolved
on the fly by the config parser and will respectively retrieve the current
configuration file name, the current line number and the current section
being parsed. This may help emit logs, errors, and debugging information
(e.g. which rule matched).
The '.' in the first char was reserved for such pseudo-variables and no
other variable is permitted. This will allow to add support for new ones
in the future if they prove to be useful (e.g. randoms/uuid for secret
keying or automatic naming of configuration objects).
Released version 2.4-dev18 with the following main changes :
- DOC: Fix indentation for `path-strip-dot` normalizer
- DOC: Fix RFC reference for the percent-to-uppercase normalizer
- DOC: Add RFC references for the path-strip-dot(dot)? normalizers
- MINOR: uri_normalizer: Add a `percent-decode-unreserved` normalizer
- BUG/MINOR: mux-fcgi: Don't send normalized uri to FCGI application
- REORG: htx: Inline htx functions to add HTX blocks in a message
- CLEANUP: assorted typo fixes in the code and comments
- DOC: general: fix white spaces for HTML converter
- BUG/MINOR: ssl: ssl_sock_prepare_ssl_ctx does not return an error code
- BUG/MINOR: cpuset: move include guard at the very beginning
- BUG/MAJOR: fix build on musl with cpu_set_t support
- BUG/MEDIUM: cpuset: fix build on MacOS
- BUG/MINOR: htx: Preserve HTX flags when draining data from an HTX message
- MEDIUM: htx: Refactor htx_xfer_blks() to not rely on hdrs_bytes field
- CLEANUP: htx: Remove unsued hdrs_bytes field from the HTX start-line
- BUG/MINOR: mux-h2: Don't encroach on the reserve when decoding headers
- MEDIUM: http-ana: handle read error on server side if waiting for response
- MINOR: htx: Limit length of headers name/value when a HTX message is dumped
- BUG/MINOR: applet: Notify the other side if data were consumed by an applet
- BUG/MINOR: hlua: Don't consume headers when starting an HTTP lua service
- BUG/MEDIUM: mux-h2: Handle EOM flag when sending a DATA frame with zero-copy
- CLEANUP: channel: No longer notify the producer in co_skip()/co_htx_skip()
- DOC: general: fix example in set-timeout
- CLEANUP: cfgparse: de-uglify early file error handling in readcfgfile()
- MINOR: config: add a new "default-path" global directive
- BUG/MEDIUM: peers: initialize resync timer to get an initial full resync
- BUG/MEDIUM: peers: register last acked value as origin receiving a resync req
- BUG/MEDIUM: peers: stop considering ack messages teaching a full resync
- BUG/MEDIUM: peers: reset starting point if peers appears longly disconnected
- BUG/MEDIUM: peers: reset commitupdate value in new conns
- BUG/MEDIUM: peers: re-work updates lookup during the sync on the fly
- BUG/MEDIUM: peers: reset tables stage flags stages on new conns
- MINOR: peers: add informative flags about resync process for debugging
- BUG/MEDIUM: time: fix updating of global_now upon clock drift
- CLEANUP: freq_ctr: make arguments of freq_ctr_total() const
- CLEANUP: hlua: rename hlua_appctx* appctx to luactx
- MINOR: server: fix doc/trace on lb algo for dynamic server creation
- REGTESTS: server: fix cli_add_server due to previous trace update
- REGTESTS: add minimal CLI "add map" tests
- DOC: management: move "set var" to the proper place
- CLEANUP: map: slightly reorder the add map function
- MINOR: map: get rid of map_add_key_value()
- MINOR: map: show the current and next pattern version in "show map"
- MINOR: map/acl: add the possibility to specify the version in "show map/acl"
- MINOR: pattern: support purging arbitrary ranges of generations
- MINOR: map/acl: add the possibility to specify the version in "clear map/acl"
- MINOR: map/acl: add the "prepare map/acl" CLI command
- MINOR: map/acl: add the "commit map/acl" CLI command
- MINOR: map/acl: make "add map/acl" support an optional version number
- CLEANUP: map/cli: properly align the map/acl help
- BUILD: compiler: do not use already defined __read_mostly on dragonfly
By default haproxy loads all files designated by a relative path from the
location the process is started in. In some circumstances it might be
desirable to force all relative paths to start from a different location
just as if the process was started from such locations. This is what this
directive is made for. Technically it will perform a temporary chdir() to
the designated location while processing each configuration file, and will
return to the original directory after processing each file. It takes an
argument indicating the policy to use when loading files whose path does
not start with a slash ('/').
A few options are offered, "current" (the default), "config" (files
relative to config file's dir), "parent" (files relative to config file's
parent dir), and "origin" with an absolute path.
This should address issue #1198.
The alternative arguments are always in curly brackets, let's fix it for
set-timeout.
The Example in set-timeout does not have the one of the required argument.
This commit makes the PR https://github.com/cbonte/haproxy-dconv/pull/34
obsolete.
Released version 2.4-dev17 with the following main changes :
- MINOIR: mux-pt/trace: Register a new trace source with its events
- BUG/MINOR: mux-pt: Fix a possible UAF because of traces in mux_pt_io_cb
- CI: travis: Drastically clean up .travis.yml
- CLEANUP: pattern: make all pattern tables read-only
- MINOR: trace: replace the trace() inline function with an equivalent macro
- MINOR: initcall: uniformize the section names between MacOS and other unixes
- CLEANUP: initcall: rename HA_SECTION to HA_INIT_SECTION
- MINOR: compiler: add macros to declare section names
- CLEANUP: initcall: rely on HA_SECTION_* instead of defining its own
- MINOR: global: declare a read_mostly section
- MINOR: fd: move a few read-mostly variables to their own section
- MINOR: epoll: move epoll_fd to read_mostly
- MINOR: kqueue: move kqueue_fd to read_mostly
- MINOR: pool: move pool declarations to read_mostly
- MINOR: threads: mark all_threads_mask as read_mostly
- MINOR: server: move idle_conn_task to read_mostly
- MINOR: protocol: move __protocol_by_family to read_mostly
- MINOR: pattern: make the pat_lru_seed read_mostly
- MINOR: trace: make trace sources read_mostly
- MINOR: freq_ctr: add a generic function to report the total value
- MEDIUM: freq_ctr: make read_freq_ctr_period() use freq_ctr_total()
- MEDIUM: freq_ctr: reimplement freq_ctr_remain_period() from freq_ctr_total()
- MINOR: freq_ctr: add the missing next_event_delay_period()
- MINOR: freq_ctr: unify freq_ctr and freq_ctr_period into freq_ctr
- MEDIUM: freq_ctr: replace the per-second counters with the generic ones
- MINOR: freq_ctr: add cpu_relax in the rotation loop of update_freq_ctr_period()
- MINOR: freq_ctr: simplify and improve the update function
- CLEANUP: time: remove the now unused ms_left_scaled
- MINOR: time: move the time initialization out of tv_update_date()
- MINOR: time: remove useless variable copies in tv_update_date()
- MINOR: time: change the global timeval and the the global tick at once
- MEDIUM: time: make the clock offset global and no per-thread
- MINOR: atomic: reimplement the relaxed version of x86 BTS/BTR
- MINOR: trace: Add the checks as a possible trace source
- MINOIR: checks/trace: Register a new trace source with its events
- MINOR: hlua: Add function to release a lua function
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a task
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a converter
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a fetch
- BUG/MINOR: hlua: Fix memory leaks on error path when parsing a lua action
- BUG/MINOR: hlua: Fix memory leaks on error path when registering an action
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a service
- BUG/MINOR: hlua: Fix memory leaks on error path when registering a cli keyword
- BUG/MINOR: cfgparse/proxy: Fix some leaks during proxy section parsing
- BUG/MINOR: listener: Handle allocation error when allocating a new bind_conf
- BUG/MINOR: cfgparse/proxy: Hande allocation errors during proxy section parsing
- MINOR: cfgparse/proxy: Group alloc error handling during proxy section parsing
- DOC: internals: update the SSL architecture schema
- BUG/MEDIUM: sample: Fix adjusting size in field converter
- MINOR: sample: add ub64dec and ub64enc converters
- CLEANUP: sample: align samples list in sample.c
- MINOR: ist: Add `istclear(struct ist*)`
- CI: cirrus: install "pcre" package
- MINOR: opentracing: correct calculation of the number of arguments in the args[]
- MINOR: opentracing: transfer of context names without prefix
- MINOR: sample: converter: Add mjson library.
- MINOR: sample: converter: Add json_query converter
- CI: travis-ci: enable weekly graviton2 builds
- DOC: ssl: Certificate hot update only works on fronted certificates
- DOC: ssl: Certificate hot update works on server certificates
- BUG/MEDIUM: threads: Ignore current thread to end its harmless period
- MINOR: threads: Only consider running threads to end a thread harmeless period
- BUG/MINOR: checks: Set missing id to the dummy checks frontend
- MINOR: logs: Add support of checks as session origin to format lf strings
- BUG/MINOR: connection: Fix fc_http_major and bc_http_major for TCP connections
- MINOR: connection: Make bc_http_major compatible with tcp-checks
- BUG/MINOR: ssl-samples: Fix ssl_bc_* samples when called from a health-check
- BUG/MINOR: http-fetch: Make method smp safe if headers were already forwarded
- MINOR: tcp_samples: Add samples to get src/dst info of the backend connection
- MINOR: tcp_samples: Be able to call bc_src/bc_dst from the health-checks
- BUG/MINOR: http_htx: Remove BUG_ON() from http_get_stline() function
- BUG/MINOR: logs: Report the true number of retries if there was no connection
- BUILD: makefile: Redirect stderr to /dev/null when probing options
- MINOR: uri_normalizer: Add uri_normalizer module
- MINOR: uri_normalizer: Add `enum uri_normalizer_err`
- MINOR: uri_normalizer: Add `http-request normalize-uri`
- MINOR: uri_normalizer: Add a `merge-slashes` normalizer to http-request normalize-uri
- MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri
- MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer
- MINOR: uri_normalizer: Add a `sort-query` normalizer
- MINOR: uri_normalizer: Add a `percent-upper` normalizer
- MEDIUM: http_act: Rename uri-normalizers
- DOC: Add introduction to http-request normalize-uri
- DOC: Note that URI normalization is experimental
- BUG/MINOR: pools: maintain consistent ->allocated count on alloc failures
- BUG/MINOR: pools/buffers: make sure to always reserve the required buffers
- MINOR: pools: drop the unused static history of artificially failed allocs
- CLEANUP: pools: remove unused arguments to pool_evict_from_cache()
- MEDIUM: pools: move the cache into the pool header
- MINOR: pool: remove the size field from pool_cache_head
- MINOR: pools: rename CONFIG_HAP_LOCAL_POOLS to CONFIG_HAP_POOLS
- MINOR: pools: enable the fault injector in all allocation modes
- MINOR: pools: make the basic pool_refill_alloc()/pool_free() update needed_avg
- MEDIUM: pools: unify pool_refill_alloc() across all models
- CLEANUP: pools: re-merge pool_refill_alloc() and __pool_refill_alloc()
- MINOR: pools: call pool_alloc_nocache() out of the pool's lock
- CLEANUP: pools: move the lock to the only __pool_get_first() that needs it
- CLEANUP: pools: rename __pool_get_first() to pool_get_from_shared_cache()
- CLEANUP: pools: rename pool_*_{from,to}_cache() to *_local_cache()
- CLEANUP: pools: rename __pool_free() to pool_put_to_shared_cache()
- MINOR: tools: add statistical_prng_range() to get a random number over a range
- MINOR: pools: use cheaper randoms for fault injections
- MINOR: pools: move the fault injector to __pool_alloc()
- MINOR: pools: split the OS-based allocator in two
- MINOR: pools: always use atomic ops to maintain counters
- MINOR: pools: move pool_free_area() out of the lock in the locked version
- MINOR: pools: factor the release code into pool_put_to_os()
- MEDIUM: pools: make CONFIG_HAP_POOLS control both local and shared pools
- MINOR: pools: create unified pool_{get_from,put_to}_cache()
- MINOR: pools: evict excess objects using pool_evict_from_local_cache()
- MEDIUM: pools: make pool_put_to_cache() always call pool_put_to_local_cache()
- CLEANUP: pools: make the local cache allocator fall back to the shared cache
- CLEANUP: pools: merge pool_{get_from,put_to}_local_caches with generic ones
- CLEANUP: pools: uninline pool_put_to_cache()
- CLEANUP: pools: declare dummy pool functions to remove some ifdefs
- BUILD: pools: fix build with DEBUG_FAIL_ALLOC
- BUG/MINOR: server: make srv_alloc_lb() allocate lb_nodes for consistent hash
- CONTRIB: mod_defender: import the minimal number of includes
- CONTRIB: mod_defender: make the code build with the embedded includes
- CONTRIB: modsecurity: import the minimal number of includes
- CONTRIB: modsecurity: make the code build with the embedded includes
- CLEANUP: sample: Improve local variables in sample_conv_json_query
- CLEANUP: sample: Explicitly handle all possible enum values from mjson
- CLEANUP: sample: Use explicit return for successful `json_query`s
- CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
- CONTRIB: move spoa_example out of the tree
- BUG/MINOR: server: free srv.lb_nodes in free_server
- BUG/MINOR: logs: free logsrv.conf.file on exit
- BUG/MEDIUM: server: ensure thread-safety of server runtime creation
- MINOR: server: add log on dynamic server creation
- MINOR: server: implement delete server cli command
- CONTRIB: move spoa_server out of the tree
- CONTRIB: move modsecurity out of the tree
- BUG/MINOR: server: fix potential null gcc error in delete server
- BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers
- BUG/MEDIUM: mux-h2: Fix dfl calculation when merging CONTINUATION frames
- BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort
- CLEANUP: uri_normalizer: Remove trailing whitespace
- MINOR: uri_normalizer: Add a `strip-dot` normalizer
- CONTRIB: move mod_defender out of the tree
- CLEANUP: contrib: remove the last references to the now dead contrib/ directory
- BUG/MEDIUM: config: fix cpu-map notation with both process and threads
- MINOR: config: add a diag for invalid cpu-map statement
- BUG/MINOR: mworker/init: don't reset nb_oldpids in non-mworker cases
- BUG/MINOR: mworker: don't use oldpids[] anymore for reload
- BUILD: makefile: fix the "make clean" target on strict bourne shells
- IMPORT: slz: import slz into the tree
- BUILD: compression: switch SLZ from out-of-tree to in-tree
- CI: github: do not build libslz any more
- CLEANUP: compression: remove calls to SLZ init functions
- BUG/MEDIUM: mux-h2: Properly handle shutdowns when received with data
- MINOR: cpuset: define a platform-independent cpuset type
- MINOR: cfgparse: use hap_cpuset for parse_cpu_set
- MEDIUM: config: use platform independent type hap_cpuset for cpu-map
- MINOR: thread: implement the detection of forced cpu affinity
- MINOR: cfgparse: support the comma separator on parse_cpu_set
- MEDIUM: cfgparse: detect numa and set affinity if needed
- MINOR: global: add option to disable numa detection
- BUG/MINOR: haproxy: fix compilation on macOS
- BUG/MINOR: cpuset: fix compilation on platform without cpu affinity
- MINOR: time: avoid unneeded updates to now_offset
- MINOR: time: avoid overwriting the same values of global_now
- CLEANUP: time: use __tv_to_ms() in tv_update_date() instead of open-coding
- MINOR: time: avoid u64 needlessly expensive computations for the 32-bit now_ms
- BUG/MINOR: peers: remove useless table check if initial resync is finished
- BUG/MEDIUM: peers: re-work connection to new process during reload.
- BUG/MEDIUM: peers: re-work refcnt on table to protect against flush
- BUG/MEDIUM: config: fix missing initialization in numa_detect_topology()
Render numa detection optional with a global configuration statement
'no numa-cpu-mapping'. This can be used if the applied affinity of the
algorithm is not optimal. Also complete the documentation with this new
keyword.
Use the platform independent type hap_cpuset for the cpu-map statement
parsing. This allow to address CPU index greater than LONGBITS.
Update the documentation to reflect the removal of this limit except for
platforms without cpu_set_t type or equivalent.
This normalizer removes "/./" segments from the path component.
Usually the dot refers to the current directory which renders those segments redundant.
See GitHub Issue #714.
This patch adds an introduction to the http-request normalize-uri section,
explaining what to expect from the normalizers and possible issues that might
arise when not being careful.
This patch renames all existing uri-normalizers into a more consistent naming
scheme:
1. The part of the URI that is being touched.
2. The modification being performed as an explicit verb.
This normalizer merges `../` path segments with the predecing segment, removing
both the preceding segment and the `../`.
Empty segments do not receive special treatment. The `merge-slashes` normalizer
should be executed first.
See GitHub Issue #714.
This patch adds 4 new sample fetches to get the source and the destination
info (ip address and port) of the backend connection :
* bc_dst : Returns the destination address of the backend connection
* bc_dst_port : Returns the destination port of the backend connection
* bc_src : Returns the source address of the backend connection
* bc_src_port : Returns the source port of the backend connection
The configuration manual was updated accordingly.
With the json_query can a JSON value be extacted from a header
or body of the request and saved to a variable.
This converter makes it possible to handle some JSON workload
to route requests to different backends.
ub64dec and ub64enc are the base64url equivalent of b64dec and base64
converters. base64url encoding is the "URL and Filename Safe Alphabet"
variant of base64 encoding. It is also used in in JWT (JSON Web Token)
standard.
RFC1421 mention in base64.c file is deprecated so it was replaced with
RFC4648 to which existing converters, base64/b64dec, still apply.
Example:
HAProxy:
http-request return content-type text/plain lf-string %[req.hdr(Authorization),word(2,.),ub64dec]
Client:
Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiZm9vIiwia2V5IjoiY2hhZTZBaFhhaTZlIn0.5VsVj7mdxVvo1wP5c0dVHnr-S_khnIdFkThqvwukmdg
$ curl -H "Authorization: Bearer ${TOKEN}" http://haproxy.local
{"user":"foo","key":"chae6AhXai6e"}
Released version 2.4-dev16 with the following main changes :
- CLEANUP: dev/flags: remove useless test in the stdin number parser
- MINOR: No longer rely on deprecated sample fetches for predefined ACLs
- MINOR: acl: Add HTTP_2.0 predefined macro
- BUG/MINOR: hlua: Detect end of request when reading data for an HTTP applet
- BUG/MINOR: tools: fix parsing "us" unit for timers
- MINOR: server/bind: add support of new prefixes for addresses.
- MINOR: log: register config file and line number on log servers.
- MEDIUM: log: support tcp or stream addresses on log lines.
- BUG/MEDIUM: log: fix config parse error logging on stdout/stderr or any raw fd
- CLEANUP: fd: remove FD_POLL_DATA and FD_POLL_STICKY
- MEDIUM: fd: prepare FD_POLL_* to move to bits 8-15
- MEDIUM: fd: merge fdtab[].ev and state for FD_EV_* and FD_POLL_* into state
- MINOR: fd: move .linger_risk into fdtab[].state
- MINOR: fd: move .cloned into fdtab[].state
- MINOR: fd: move .initialized into fdtab[].state
- MINOR: fd: move .et_possible into fdtab[].state
- MINOR: fd: move .exported into fdtab[].state
- MINOR: fd: implement an exclusive syscall bit to remove the ugly "log" lock
- MINOR: cli/show-fd: slightly reorganize the FD status flags
- MINOR: atomic/arm64: detect and use builtins for the double-word CAS
- CLEANUP: atomic: add an explicit _FETCH variant for add/sub/and/or
- CLEANUP: atomic: make all standard add/or/and/sub operations return void
- CLEANUP: atomic: add a fetch-and-xxx variant for common operations
- CLEANUP: atomic: add HA_ATOMIC_INC/DEC for unit increments
- CLEANUP: atomic/tree-wide: replace single increments/decrements with inc/dec
- CLEANUP: atomic: use the __atomic variant of BTS/BTR on modern compilers
- MINOR: atomic: implement native BTS/BTR for x86
- MINOR: ist: Add `istappend(struct ist, char)`
- MINOR: ist: Add `istshift(struct ist*)`
- MINOR: ist: Add `istsplit(struct ist*, char)`
- BUG/MAJOR: fd: switch temp values to uint in fd_stop_both()
- MINOR: opentracing: register config file and line number on log servers
- MEDIUM: resolvers: add support of tcp address on nameserver line.
- MINOR: ist: Rename istappend() to __istappend()
- CLEANUP: htx: Make http_get_stline take a `const struct`
- CLEANUP: ist: Remove unused `count` argument from `ist2str*`
- CLEANUP: Remove useless malloc() casts
This patch re-works configuration parsing, it removes the "server"
lines from "resolvers" sections introduced in commit 56fc5d9eb:
MEDIUM: resolvers: add supports of TCP nameservers in resolvers.
It also extends the nameserver lines to support stream server
addresses such as:
resolvers
nameserver localhost tcp@127.0.0.1:53
Doing so, a part of nameserver's init code was factorized in
function 'parse_resolvers' and removed from 'post_parse_resolvers'.
An explicit stream address prefix such as "tcp6@" "tcp4@"
"stream+ipv6@" "stream+ipv4@" or "stream+unix@" will
allocate an implicit ring buffer with a forward server
targeting the given address.
This is usefull to simply send logs to a log server in tcp
and It doesn't need to declare a ring section in configuration.
Since the internal function str2sa_range is used to addresses
for different objects ('server', 'bind' but also 'log' or
'nameserver') we notice that some combinations are missing.
"ip@" is introduced to authorize the prefix "dgram+ip@" or
"stream+ip@" which dectects automatically IP version but
specify dgram or stream.
"tcp@" was introduced and is an alias for "stream+ip@".
"tcp6" and "tcp4" are now aliases for "stream+ipv6@" and
"stream+ipv4@".
"uxst@" and "uxdg@" are now aliases for "stream+unix@" and
"dgram+unix@".
This patch also adds a complete section in documentation to
describe adresses and their prefixes.
HTTP_2.0 predefined macro returns true for HTTP/2 requests. HTTP/2 doen't
convey a version information, so this macro may seem a bit strange. But for
compatiblity reasons, internally, the "HTTP/2.0" version is set. Thus, it is
handy to rely on it to differenciate HTTP/1 and HTTP/2 requests.
Some predefined ACLs were still based on deprecated sample fetches, like
req_proto_http or req_ver. Now, they use non-deprecated sample fetches. In
addition, the usage lines in the configuration manual have been updated to
be more explicit.
Released version 2.4-dev15 with the following main changes :
- BUG/MINOR: payload: Wait for more data if buffer is empty in payload/payload_lv
- BUG/MINOR: stats: Apply proper styles in HTML status page.
- BUG/MEDIUM: time: make sure to always initialize the global tick
- BUG/MINOR: tcp: fix silent-drop workaround for IPv6
- BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS
- CLEANUP: socket: replace SOL_IP/IPV6/TCP with IPPROTO_IP/IPV6/TCP
- BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields
- BUG/MINOR: mux-h2: Don't emit log twice if an error occurred on the preface
- MINOR: stream: Don't trigger errors on destructive HTTP upgrades
- MINOR: frontend: Create HTTP txn for HTX streams
- MINOR: stream: Be sure to set HTTP analysers when creating an HTX stream
- BUG/MINOR: stream: Properly handle TCP>H1>H2 upgrades in http_wait_for_request
- BUG/MINOR: config: Add warning for http-after-response rules in TCP mode
- MINOR: muxes: Add a flag to notify a mux does not support any upgrade
- MINOR: mux-h1: Don't perform implicit HTTP/2 upgrade if not supported by mux
- MINOR: mux-pt: Don't perform implicit HTTP upgrade if not supported by mux
- MEDIUM: mux-h1: Expose h1 in the list of supported mux protocols
- MEDIUM: mux-pt: Expose passthrough in the list of supported mux protocols
- MINOR: muxes: Show muxes flags when the mux list is displayed
- DOC: config: Improve documentation about proto/check-proto keywords
- MINOR: stream: Use stream type instead of proxy mode when appropriate
- MINOR: filters/http-ana: Decide to filter HTTP headers in HTTP analysers
- MINOR: http-ana: Simplify creation/destruction of HTTP transactions
- MINOR: stream: Handle stream HTTP upgrade in a dedicated function
- MEDIUM: Add tcp-request switch-mode action to perform HTTP upgrade
- MINOR: config/proxy: Don't warn for HTTP rules in TCP if 'switch-mode http' set
- MINOR: config/proxy: Warn if a TCP proxy without backend is upgradable to HTTP
- DOC: config: Add documentation about TCP to HTTP upgrades
- REGTESTS: Add script to tests TCP to HTTP upgrades
- BUG/MINOR: payload/htx: Ingore L6 sample fetches for HTX streams/checks
- MINOR: htx: Make internal.strm.is_htx an internal sample fetch
- MINOR: action: Use a generic function to check validity of an action rule list
- MINOR: payload/config: Warn if a L6 sample fetch is used from an HTTP proxy
- MEDIUM: http-rules: Add wait-for-body action on request and response side
- REGTESTS: Add script to tests the wait-for-body HTTP action
- BUG/MINOR: http-fetch: Fix test on message state to capture the version
- CLEANUP: vars: always pre-initialize smp in vars_parse_cli_get_var()
- MINOR: global: define diagnostic mode of execution
- MINOR: cfgparse: diag for multiple nbthread statements
- MINOR: server: diag for 0 weight server
- MINOR: diag: create cfgdiag module
- MINOR: diag: diag if servers use the same cookie value
- MINOR: config: diag if global section after non-global
- TESTS: slightly reorganize the code in the tests/ directory
- TESTS: move tests/*.cfg to tests/config
- REGTESTS: ssl: "set ssl cert" and multi-certificates bundle
- REGTESTS: ssl: mark set_ssl_cert_bundle.vtc as broken
- CONTRIB: halog: fix issue with array of type char
- CONTRIB: tcploop: add a shutr command
- CONTRIB: debug: add the show-fd-to-flags script
- CONTRIB: debug: split poll from flags
- CONTRIB: move some dev-specific tools to dev/
- BUILD: makefile: always build the flags utility
- DEV: flags: replace the unneeded makefile with a README
- BUILD: makefile: integrate the hpack tools
- CONTRIB: merge ip6range with iprange
- CONTRIB: move some admin-related sub-projects to admin/
- CONTRIB: move halog to admin/
- ADMIN: halog: automatically enable USE_MEMCHR on the right glibc version
- BUILD: makefile: build halog with the correct flags
- BUILD: makefile: add a "USE_PROMEX" variable to ease building prometheus-exporter
- CONTRIB: move prometheus-exporter to addons/promex
- DOC: add a few words about USE_* and the addons directory
- CONTRIB: move 51Degrees to addons/51degrees
- CONTRIB: move src/da.c and contrib/deviceatlas to addons/deviceatlas
- CONTRIB: move src/wurfl.c and contrib/wurfl to addons/wurfl
- CONTRIB: move contrib/opentracing to addons/ot
- BUG/MINOR: opentracing: initialization after establishing daemon mode
- DOC: clarify that compression works for HTTP/2
This patch clarifies that compression also works with HTTP/2. I have
picked the wording "HTTP/1.1 or above" because it is already used
elsewhere in the documentation.
I have tested that compression indeed works in HTTP/2.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
This one is the last optional module to build with haproxy, so let's move
it to addons/. It was renamed to "ot" as it was the only one whose USE_*
option did not match the directory name, now this is consistent.
Few changes were required, only the Makefile, and doc were adjusted, as
the directory was already self-contained and relocatable.
Historically, an option was added to wait for the request payload (option
http-buffer-request). This option has 2 drawbacks. First, it is an ON/OFF
option for the whole proxy. It cannot be enabled on demand depending on the
message. Then, as its name suggests, it only works on the request side. The
only option to wait for the response payload was to write a dedicated
filter. While it is an acceptable solution for complex applications, it is a
bit overkill to simply match strings in the body.
To make everyone happy, this patch adds a dedicated HTTP action to wait for
the message payload, for the request or the response depending it is used in
an http-request or an http-response ruleset. The time to wait is
configurable and, optionally, the minimum payload size to have before stop
to wait.
Both the http action and the old http analyzer rely on the same internal
function.
Use a L6 sample fetch on an HTX streams or a HTX health-check is meaningless
because data are not raw but structured. So now, these sample fetches fail
when called from an HTTP proxy. In addition, a warning has been added in the
configuration manual, at the begining of the L6 sample fetches section.
Note that req.len and res.len samples return the HTX data size instead of
failing. It is not accurate because it does not reflect the buffer size nor
the raw data length. But we keep it for backward compatibility purpose.
However it remains a bit strange to use it on an HTTP proxy.
This patch may be backported to all versions supporting the HTX, i.e as far
as 2.0. But the part about the health-checks is only valid for the 2.2 and
upper.
This patch adds explanation about chaining a TCP frontend to an HTTP
backend. It also explain how the HTTP upgrades work in this context. A note
has also been added in "Fetching HTTP samples" section to warning about HTTP
content processing in TCP.
It is now possible to perform HTTP upgrades on a TCP stream from the
frontend side. To do so, a tcp-request content rule must be defined with the
switch-mode action, specifying the mode (for now, only http is supported)
and optionnaly the proto (h1 or h2).
This way it could be possible to set HTTP directives on a TCP frontend which
will only be evaluated if an upgrade is performed. This new way to perform
HTTP upgrades should replace progressively the old way, consisting to route
the request to an HTTP backend. And it should be also a good start to remove
all HTTP processing from tcp-request content rules.
This action is terminal, it stops the ruleset evaluation. It is only
available on proxy with the frontend capability.
The configuration manual has been updated accordingly.
This patch adds a description about information provided by "haproxy -vv"
command regarding the available protocols. The description is adapted
depending the context (bind line, server line or health-check).
Released version 2.4-dev14 with the following main changes :
- MEDIUM: quic: Fix build.
- MEDIUM: quic: Fix build.
- CI: codespell: whitelist "Dragan Dosen"
- CLEANUP: assorted typo fixes in the code and comments
- CI: github actions: update LibreSSL to 3.2.5
- REGTESTS: revert workaround for a crash with recent libressl on http-reuse sni
- CLEANUP: mark defproxy as const on parse tune.fail-alloc
- REGTESTS: remove unneeded experimental-mode in cli add server test
- REGTESTS: wait for proper return of enable server in cli add server test
- MINOR: compression: use pool_alloc(), not pool_alloc_dirty()
- MINOR: spoe: use pool_alloc(), not pool_alloc_dirty()
- MINOR: fcgi-app: use pool_alloc(), not pool_alloc_dirty()
- MINOR: cache: use pool_alloc(), not pool_alloc_dirty()
- MINOR: ssl: use pool_alloc(), not pool_alloc_dirty()
- MINOR: opentracing: use pool_alloc(), not pool_alloc_dirty()
- MINOR: dynbuf: make b_alloc() always check if the buffer is allocated
- CLEANUP: compression: do not test for buffer before calling b_alloc()
- CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
- MINOR: channel: simplify the channel's buffer allocation
- MEDIUM: dynbuf: remove last usages of b_alloc_margin()
- CLEANUP: dynbuf: remove b_alloc_margin()
- CLEANUP: dynbuf: remove the unused b_alloc_fast() function
- CLEANUP: pools: remove the unused pool_get_first() function
- MINOR: pools: make the pool allocator support a few flags
- MINOR: pools: add pool_zalloc() to return a zeroed area
- CLEANUP: connection: use pool_zalloc() in conn_alloc_hash_node()
- CLEANUP: filters: use pool_zalloc() in flt_stream_add_filter()
- CLEANUP: spoe: use pool_zalloc() instead of pool_alloc+memset
- CLEANUP: frontend: use pool_zalloc() in frontend_accept()
- CLEANUP: mailers: use pool_zalloc() in enqueue_one_email_alert()
- CLEANUP: resolvers: use pool_zalloc() in resolv_link_resolution()
- CLEANUP: ssl: use pool_zalloc() in ssl_init_keylog()
- CLEANUP: tcpcheck: use pool_zalloc() instead of pool_alloc+memset
- CLEANUP: quic: use pool_zalloc() instead of pool_alloc+memset
- MINOR: time: also provide a global, monotonic global_now_ms timer
- BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable
- MINOR: tools: introduce new option PA_O_DEFAULT_DGRAM on str2sa_range.
- BUILD: tools: fix build error with new PA_O_DEFAULT_DGRAM
- BUG/MINOR: ssl: Prevent disk access when using "add ssl crt-list"
- CLEANUP: ssl: remove unused definitions
- BUILD: ssl: guard ecdh functions with SSL_CTX_set_tmp_ecdh macro
- MINOR: lua: Slightly improve function dumping the lua traceback
- BUG/MEDIUM: debug/lua: Use internal hlua function to dump the lua traceback
- BUG/MEDIUM: lua: Always init the lua stack before referencing the context
- MINOR: fd: make fd_clr_running() return the remaining running mask
- MINOR: fd: remove the unneeded running bit from fd_insert()
- BUG/MEDIUM: fd: do not wait on FD removal in fd_delete()
- CLEANUP: fd: remove unused fd_set_running_excl()
- CLEANUP: fd: slightly simplify up _fd_delete_orphan()
- BUG/MEDIUM: fd: Take the fd_mig_lock when closing if no DWCAS is available.
- BUG/MEDIUM: release lock on idle conn killing on reached pool high count
- BUG/MEDIUM: thread: Fix a deadlock if an isolated thread is marked as harmless
- MINOR: tools: make url2ipv4 return the exact number of bytes parsed
- BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters
- BUG/MEDIUM: mux-h1: make h1_shutw_conn() idempotent
- BUG/MINOR: ssl: Fix update of default certificate
- BUG/MINOR: ssl: Prevent removal of crt-list line if the instance is a default one
- BUILD: ssl: introduce fine guard for ssl random extraction functions
- REORG: global: move initcall register code in a dedicated file
- REORG: global: move free acl/action in their related source files
- REORG: split proxy allocation functions
- MINOR: proxy: implement a free_proxy function
- MINOR: proxy: define cap PR_CAP_LUA
- MINOR: lua: properly allocate the lua Socket proxy
- MINOR: lua: properly allocate the lua Socket servers
- MINOR: vars: make get_vars() allow the session to be null
- MINOR: vars: make the var() sample fetch keyword depend on nothing
- CLEANUP: sample: remove duplicate "stopping" sample fetch keyword
- MINOR: sample: make smp_resolve_args() return an allocate error message
- MINOR: sample: add a new SMP_SRC_CONST sample capability
- MINOR: sample: mark the truly constant sample fetch keywords as such
- MINOR: sample: add a new CFG_PARSER context for samples
- MINOR: action: add a new ACT_F_CFG_PARSER origin designation
- MEDIUM: vars: add support for a "set-var" global directive
- REGTESTS: add a basic reg-test for some "set-var" commands
- MINOR: sample: add a new CLI_PARSER context for samples
- MINOR: action: add a new ACT_F_CLI_PARSER origin designation
- MINOR: vars/cli: add a "get var" CLI command to retrieve global variables
- MEDIUM: cli: add a new experimental "set var" command
- MINOR: compat: add short aliases for a few very commonly used types
- BUILD: ssl: use EVP_CIPH_GCM_MODE macro instead of HA_OPENSSL_VERSION
- MEDIUM: backend: use a trylock to grab a connection on high FD counts as well
While we do support process-wide variables ("proc.<name>"), there was
no way to preset them from the configuration. This was particularly
limiting their usefulness since configs involving them always had to
first check if the variable was set prior to performing an operation.
This patch adds a new "set-var" directive in the global section that
supports setting the proc.<name> variables from an expression, like
other set-var actions do. The syntax however follows what is already
being done for setenv, which consists in having one argument for the
variable name and another one for the expression.
Only "constant" expressions are allowed here, such as "int", "str"
etc, combined with arithmetic or string converters, and variable
lookups. A few extra sample fetch keywords like "date", "rand" and
"uuid" are also part of the constant expressions and may make sense
to allow to create a random key or differentiate processes.
The way it was done consists in parsing a dummy rule an executing the
expression in the CFG_PARSE context, then releasing the expression.
This is safe because the sample that variables store does not hold a
back pointer to expression that created them.
The hdr_ip() sample fetch function will try to extract IP addresses
from a header field. These IP addresses are parsed using url2ipv4()
and if it fails it will fall back to inet_pton(AF_INET6), otherwise
will fail.
There is a small problem there which is that if a field starts with
an IP address and is immediately followed by some garbage, the IP
address part is still returned. This is a problem with fields such
as x-forwarded-for because it prevents detection of accidental
corruption or bug along the chain. For example, the following string:
x-forwarded-for: 1.2.3.4; 5.6.7.8
or this one:
x-forwarded-for: 1.2.3.4O ( the last one being the letter 'O')
would still return "1.2.3.4" despite the trailing characters. This is
bad because it will silently cover broken code running on intermediary
proxies and may even in some cases allow haproxy to pass improperly
formatted headers after they were apparently validated, for example,
if someone extracts the address from this field to place it into
another one.
This issue would only affect the IPv4 parser, because the IPv6 parser
already uses inet_pton() which fails at the first invalid character and
rejects trailing port numbers.
In strict compliance with RFC7239, let's make sure that if there are any
characters left in the string, the parsing fails and makes hdr_ip()
return nothing. However, a special case has to be handled to support
IPv4 addresses followed by a colon and a valid port number, because till
now the parser used to implicitly accept them and it appears that this
practice, though rare, does exist at least in Azure:
https://docs.microsoft.com/en-us/azure/application-gateway/how-application-gateway-works
This issue has always been there so the fix may be backported to all
versions. It will need the following commit in order to work as expected:
MINOR: tools: make url2ipv4 return the exact number of bytes parsed
Many thanks to https://twitter.com/melardev and the BitMEX Security Team
for their detailed report.
Released version 2.4-dev13 with the following main changes :
- BUG/MEDIUM: cli: fix "help" crashing since recent spelling fixes
- BUG/MINOR: cfgparse: use the GLOBAL not LISTEN keywords list for spell checking
- MINOR: tools: improve word fingerprinting by counting presence
- MINOR: tools: do not sum squares of differences for word fingerprints
- MINOR: cli: improve fuzzy matching to work on all remaining words at once
- MINOR: cli: sort the suggestions by order of relevance
- MINOR: cli: limit spelling suggestions to 5
- MINOR: cfgparse/proxy: also support spelling fixes on options
- BUG/MINOR: resolvers: Add missing case-insensitive comparisons of DNS hostnames
- MINOR: time: export the global_now variable
- BUG/MINOR: freq_ctr/threads: make use of the last updated global time
- MINOR: freq_ctr/threads: relax when failing to update a sliding window value
- MINOR/BUG: mworker/cli: do not use the unix_bind prefix for the master CLI socket
- MINOR: mworker/cli: alert the user if we enabled a master CLI but not the master-worker mode
- MINOR: cli: implement experimental-mode
- REORG: server: add a free server function
- MINOR: cfgparse: always alloc idle conns task
- REORG: server: move keywords in srv_kws
- MINOR: server: remove fastinter from mistyped kw list
- REORG: server: split parse_server
- REORG: server: move alert traces in parse_server
- REORG: server: rename internal functions from parse_server
- REORG: server: attach servers in parse_server
- REORG: server: use flags for parse_server
- MINOR: server: prepare parsing for dynamic servers
- MINOR: stats: export function to allocate extra proxy counters
- MEDIUM: server: implement 'add server' cli command
- REGTESTS: implement test for 'add server' cli
- MINOR: server: enable standard options for dynamic servers
- MINOR: server: support keyword proto in 'add server' cli
- BUG/MINOR: protocol: add missing support of dgram unix socket.
- CLEANUP: Fix a typo in fix_is_valid description
- MINOR: raw_sock: Add a close method.
- MEDIUM: connections: Introduce a new XPRT method, start().
- MEDIUM: connections: Implement a start() method for xprt_handshake.
- MEDIUM: connections: Implement a start() method in ssl_sock.
- MINOR: muxes: garbage collect the reset() method.
- CLEANUP: tcp-rules: Fix a typo in error messages about expect-netscaler-cip
- MEDIUM: lua: Use a per-thread counter to track some non-reentrant parts of lua
- BUG/MEDIUM: debug/lua: Don't dump the lua stack if not dumpable
Released version 2.4-dev12 with the following main changes :
- CLEANUP: connection: Use `VAR_ARRAY` in `struct tlv` definition
- CLEANUP: connection: Remove useless test for NULL before calling `pool_free()`
- CLEANUP: connection: Use istptr / istlen for proxy_unique_id
- MINOR: connection: Use a `struct ist` to store proxy_authority
- CLEANUP: connection: Consistently use `struct ist` to process all TLV types
- BUILD: task: fix build at -O0 with threads disabled
- BUILD: bug: refine HA_LINK_ERROR() to only be used on gcc and derivatives
- CLEANUP: config: make the cfg_keyword parsers take a const for the defproxy
- BUILD: connection: do not use VAR_ARRAY in struct tlv
- BUG/MEDIUM: session: NULL dereference possible when accessing the listener
- MINOR: build: force CC to set a return code when probing options
- CLEANUP: stream: rename a few remaining occurrences of "stream *sess"
- BUG/MEDIUM: resolvers: handle huge responses over tcp servers.
- CLEANUP: config: also address the cfg_keyword API change in the compression code
- BUG/MEDIUM: ssl: properly remove the TASK_HEAVY flag at end of handshake
- BUG/MINOR: sample: Rename SenderComID/TargetComID to SenderCompID/TargetCompID
- MINOR: task: give the scheduler a bit more flexibility in the runqueue size
- OPTIM: task: automatically adjust the default runqueue-depth to the threads
- BUG/MINOR: connection: Missing QUIC initialization
- BUG/MEDIUM: stick-tables: fix ref counter in table entry using multiple http tracksc.
- BUILD: atomic/arm64: force the register pairs to use in __ha_cas_dw()
- BUG/MEDIUM: filters: Set CF_FL_ANALYZE on channels when filters are attached
- BUG/MINOR: tcpcheck: Update .health threshold of agent inside an agent-check
- BUG/MINOR: proxy/session: Be sure to have a listener to increment its counters
- BUG/MINOR: tcpcheck: Fix double free on error path when parsing tcp/http-check
- BUG/MINOR: server-state: properly handle the case where the base is not set
- BUG/MINOR: server-state: use the argument, not the global state
- CLEANUP: tcp-rules: add missing actions in the tcp-request error message
- CLEANUP: vars: make the error message clearer on missing arguments for set-var
- CLEANUP: http-rules: remove the unexpected comma before the list of action keywords
- CLEANUP: actions: the keyword must always be const from the rule
- MINOR: tools: add simple word fingerprinting to find similar-looking words
- MINOR: cfgparse: add cfg_find_best_match() to suggest an existing word
- MINOR: cfgparse: suggest correct spelling for unknown words in proxy sections
- MINOR: cfgparse: suggest correct spelling for unknown words in global section
- MINOR: cfgparse/server: try to fix spelling mistakes on server lines
- MINOR: cfgparse/bind: suggest correct spelling for unknown bind keywords
- MINOR: actions: add a function to suggest an action ressembling a given word
- MINOR: http-rules: suggest approaching action names on mismatch
- MINOR: tcp-rules: suggest approaching action names on mismatch
- BUG/MINOR: cfgparse/server: increment the extra keyword counter one at a time
- Revert "BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record"
- BUG/MINOR: resolvers: Consider server to have no IP on DNS resolution error
- BUG/MINOR: resolvers: Reset server address on DNS error only on status change
- BUG/MINOR: resolvers: Unlink DNS resolution to set RMAINT on SRV resolution
- BUG/MEDIUM: resolvers: Don't set an address-less server as UP
- BUG/MEDIUM: resolvers: Fix the loop looking for an existing ADD item
- MINOR: resolvers: new function find_srvrq_answer_record()
- BUG/MINOR; resolvers: Ignore DNS resolution for expired SRV item
- BUG/MEDIUM: resolvers: Trigger a DNS resolution if an ADD item is obsolete
- MINOR: resolvers: Use a function to remove answers attached to a resolution
- MINOR: resolvers: Purge answer items when a SRV resolution triggers an error
- MINOR: resolvers: Add function to change the srv status based on SRV resolution
- MINOR: resolvers: Directly call srvrq_update_srv_state() when possible
- BUG/MEDIUM: resolvers: Don't release resolution from a requester callbacks
- BUG/MEDIUM: resolvers: Skip DNS resolution at startup if SRV resolution is set
- MINOR: resolvers: Use milliseconds for cached items in resolver responses
- MINOR: resolvers: Don't try to match immediatly renewed ADD items
- CLEANUP: resolvers: Use ha_free() in srvrq_resolution_error_cb()
- CLEANUP: resolvers: Perform unsafe loop on requester list when possible
- BUG/MINOR: cli: make sure "help", "prompt", "quit" are enabled at master level
- CLEANUP: cli: fix misleading comment and better indent the access level flags
- MINOR: cli: set the ACCESS_MASTER* bits on the master bind_conf
- MINOR: cli: test the appctx level for master access instead of comparing pointers
- MINOR: cli: print the error message in the parser function itself
- MINOR: cli: filter the list of commands to the matching part
- MEDIUM: cli: apply spelling fixes for known commands before listing them
- MINOR: tools: add the ability to update a word fingerprint
- MINOR: cli: apply the fuzzy matching on the whole command instead of words
- CLEANUP: cli: rename MAX_STATS_ARGS to MAX_CLI_ARGS
- CLEANUP: cli: rename the last few "stats_" to "cli_"
- CLEANUP: task: make sure tasklet handlers always indicate their statuses
- CLEANUP: assorted typo fixes in the code and comments
The recent default runqueue size reduction appeared to have significantly
lowered performance on low-thread count configs. Testing various values
runqueue values on different workloads under thread counts ranging from
1 to 64, it appeared that lower values are more optimal for high thread
counts and conversely. It could even be drawn that the optimal value for
various workloads sits around 280/sqrt(nbthread), and probably has to do
with both the L3 cache usage and how to optimally interlace the threads'
activity to minimize contention. This is much easier to optimally
configure, so let's do this by default now.
The recently introduced Financial Information eXchange (FIX)
converters have some hard coded tags based on the specification that
were misspelled. Specifically, SenderComID and TargetComID should
be SenderCompID and TargetCompID according to the specification [1][2].
This patch updates all references, which includes the converters
themselves, the regression test, and the documentation.
[1] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag49.html
[2] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag56.html
Parameter "accepted_payload_size" is currently considered regardless
the used nameserver is using TCP or UDP. It remains mandatory to annouce
such capability to support e-dns, so a value have to be announced also
in TCP. Maximum DNS message size in TCP is limited by protocol to 65535
and so for UDP (65507) if system supports such UDP messages. But
the maximum value for this option was arbitrary forced to 8192.
This patch change this maximum to 65535 to allow user to set bigger value
for UDP if its system supports. It also sets accepted_payload_size
in TCP allowing to retrieve huge responses if the configuration uses
TCP nameservers.
The request announcing the accepted_payload_size capability is currently
built at resolvers level and is common to all used nameservers of the
section regardess transport protocol used. A further patch should be
made to at least specify a different payload size depending of the
transport, and perhaps could be forced to 65535 in case of TCP and
maximum would be forced back to 65507 matching UDP max.
This patch is appliable since 2.4 version
Released version 2.4-dev11 with the following main changes :
- CI: codespell: skip Makefile for spell check
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: tcp-act: Don't forget to set the original port for IPv4 set-dst rule
- BUG/MINOR: connection: Use the client's dst family for adressless servers
- BUG/MEDIUM: spoe: Kill applets if there are pending connections and nbthread > 1
- CLEANUP: Use ist2(const void*, size_t) whenever possible
- CLEANUP: Use IST_NULL whenever possible
- BUILD: proxy: Missing header inclusion for quic_transport_params_init()
- BUILD: quic: Implicit conversion between SSL related enums.
- DOC: spoe: Add a note about fragmentation support in HAProxy
- MINOR: contrib: add support for heartbeat control messages.
- MINOR: contrib: Enhance peers dissector heuristic.
- BUG/MINOR: mux-h2: Fix typo in scheme adjustment
- CLEANUP: Reapply the ist2() replacement patch
- CLEANUP: Use istadv(const struct ist, const size_t) whenever possible
- CLEANUP: Use isttest(const struct ist) whenever possible
- Revert "CI: Pin VTest to a known good commit"
- CLEANUP: backend: fix a wrong comment
- BUG/MINOR: backend: free allocated bind_addr if reuse conn
- MINOR: backend: handle reuse for conns with no server as target
- REGTESTS: test http-reuse if no server target
- BUG/MINOR: hlua: Don't strip last non-LWS char in hlua_pushstrippedstring()
- BUG/MINOR: server-state: Don't load server-state file for disabled backends
- CLEANUP: dns: Use DISGUISE() on a never-failing ring_attach() call
- CLEANUP: dns: Remove useless test on ns->dgram in dns_connect_nameserver()
- DOC: fix originalto except clause on destination address
- CLEANUP: Use the ist() macro whenever possible
- CLEANUP: Replace for loop with only a condition by while
- REORG: atomic: reimplement pl_cpu_relax() from atomic-ops.h
- BUG/MINOR: mt-list: always perform a cpu_relax call on failure
- MINOR: atomic: add armv8.1-a atomics variant for cas-dw
- MINOR: atomic: implement a more efficient arm64 __ha_cas_dw() using pairs
- BUG/MINOR: ssl: don't truncate the file descriptor to 16 bits in debug mode
- MEDIUM: pools: add CONFIG_HAP_NO_GLOBAL_POOLS and CONFIG_HAP_GLOBAL_POOLS
- MINOR: pools: double the local pool cache size to 1 MB
- MINOR: stream: use ABORT_NOW() and not abort() in stream_dump_and_crash()
- CLEANUP: stream: explain why we queue the stream at the head of the server list
- MEDIUM: backend: use a trylock when trying to grab an idle connection
- REORG: tools: promote the debug PRNG to more general use as a statistical one
- OPTIM: lb-random: use a cheaper PRNG to pick a server
- MINOR: task: stop abusing the nice field to detect a tasklet
- MINOR: task: move the nice field to the struct task only
- MEDIUM: task: extend the state field to 32 bits
- MINOR: task: add an application specific flag to the state: TASK_F_USR1
- MEDIUM: muxes: mark idle conns tasklets with TASK_F_USR1
- MINOR: xprt: add new xprt_set_idle and xprt_set_used methods
- MEDIUM: ssl: implement xprt_set_used and xprt_set_idle to relax context checks
- MINOR: server: don't read curr_used_conns multiple times
- CLEANUP: global: reorder some fields to respect cache lines
- CLEANUP: sockpair: silence a coverity check about fcntl()
- CLEANUP: lua: set a dummy file name and line number on the dummy servers
- MINOR: server: add a global list of all known servers
- MINOR: cfgparse: finish to set up servers outside of the proxy setup loop
- MINOR: server: allocate a per-thread struct for the per-thread connections stuff
- MINOR: server: move actconns to the per-thread structure
- CLEANUP: server: reorder some fields in the server struct to respect cache lines
- MINOR: backend: add a BUG_ON if conn mux NULL in connect_server
- BUG/MINOR: backend: fix condition for reuse on mode HTTP
- BUILD: Fix build when using clang without optimizing.
- CLEANUP: assorted typo fixes in the code and comments
Fix the description of the except clause of the originalto option. The
destination address and not the source is compared with the except range
address to prevent the addition of the X-Original-To header.
This can be backported in every releases.
Released version 2.4-dev10 with the following main changes :
- BUILD: SSL: introduce fine guard for RAND_keep_random_devices_open
- MINOR: Configure the `cpp` userdiff driver for *.[ch] in .gitattributes
- BUG/MINOR: ssl/cli: potential null pointer dereference in "set ssl cert"
- BUG/MINOR: sample: secure convs that accept base64 string and var name as args
- BUG/MEDIUM: vars: make functions vars_get_by_{name,desc} thread-safe
- CLEANUP: vars: make smp_fetch_var() to reuse vars_get_by_desc()
- DOC: muxes: add a diagram of the exchanges between muxes and outer world
- BUG/MEDIUM: proxy: use thread-safe stream killing on hard-stop
- BUG/MEDIUM: cli/shutdown sessions: make it thread-safe
- BUG/MINOR: proxy: wake up all threads when sending the hard-stop signal
- MINOR: stream: add an "epoch" to figure which streams appeared when
- MINOR: cli/streams: make "show sess" dump all streams till the new epoch
- MINOR: streams: use one list per stream instead of a global one
- MEDIUM: streams: do not use the streams lock anymore
- BUILD: dns: avoid a build warning when threads are disabled (dss unused)
- MEDIUM: task: remove the tasks_run_queue counter and have one per thread
- MINOR: tasks: do not maintain the rqueue_size counter anymore
- CLEANUP: tasks: use a less confusing name for task_list_size
- CLEANUP: task: move the tree root detection from __task_wakeup() to task_wakeup()
- MINOR: task: limit the remote thread wakeup to the global runqueue only
- MINOR: task: move the allocated tasks counter to the per-thread struct
- CLEANUP: task: split the large tasklet_wakeup_on() function in two
- BUG/MINOR: fd: properly wait for !running_mask in fd_set_running_excl()
- BUG/MINOR: resolvers: Fix condition to release received ARs if not assigned
- BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record
- BUG/MINOR: resolvers: new callback to properly handle SRV record errors
- BUG/MEDIUM: resolvers: Reset server address and port for obselete SRV records
- BUG/MEDIUM: resolvers: Reset address for unresolved servers
- DOC: Update the module list in MAINTAINERS file
- MINOR: htx: Add function to reserve the max possible size for an HTX DATA block
- DOC: Update the HTX API documentation
- DOC: Update the filters guide
- BUG/MEDIUM: contrib/prometheus-exporter: fix segfault in listener name dump
- MINOR: task: split the counts of local and global tasks picked
- MINOR: task: do not use __task_unlink_rq() from process_runnable_tasks()
- MINOR: task: don't decrement then increment the local run queue
- CLEANUP: task: re-merge __task_unlink_rq() with task_unlink_rq()
- MINOR: task: make grq_total atomic to move it outside of the grq_lock
- MINOR: tasks: also compute the tasklet latency when DEBUG_TASK is set
- MINOR: task: make tasklet wakeup latency measurements more accurate
- MINOR: server: Be more strict on the server-state line parsing
- MINOR: server: Only fill one array when parsing a server-state line
- MEDIUM: server: Refactor apply_server_state() to make it more readable
- CLEANUP: server: Rename state_line node to node instead of name_name
- CLEANUP: server: Rename state_line structure into server_state_line
- CLEANUP: server: Use a local eb-tree to store lines of the global server-state file
- MINOR: server: Be more strict when reading the version of a server-state file
- MEDIUM: server: Store parsed params of a server-state line in the tree
- MINOR: server: Remove cached line from global server-state tree when found
- MINOR: server: Move loading state of servers in a dedicated function
- MEDIUM: server: Use a tree to store local server-state lines
- MINOR: server: Parse and store server-state lines in a dedicated function
- MEDIUM: server: Don't load server-state file if a line is corrupted
- REORG: server: Export and rename some functions updating server info
- REORG: server-state: Move functions to deal with server-state in its own file
- MINOR: server-state: Don't load server-state file for serverless proxies
- CLEANUP: muxes: Remove useless if condition in show_fd function
- BUG/MINOR: stats: fix compare of no-maint url suffix
- MINOR: task: limit the number of subsequent heavy tasks with flag TASK_HEAVY
- MINOR: ssl: mark the SSL handshake tasklet as heavy
- CLEANUP: server: rename srv_cleanup_{idle,toremove}_connections()
- BUG/MINOR: ssl: potential null pointer dereference in ckchs_dup()
- MINOR: task: add one extra tasklet class: TL_HEAVY
- MINOR: task: place the heavy elements in TL_HEAVY
- MINOR: task: only limit TL_HEAVY tasks but not others
- BUG/MINOR: http-ana: Only consider dst address to process originalto option
- MINOR: tools: Add net_addr structure describing a network addess
- MINOR: tools: Add function to compare an address to a network address
- MEDIUM: http-ana: Add IPv6 support for forwardfor and orignialto options
- CLEANUP: hlua: Use net_addr structure internally to parse and compare addresses
- REGTESTS: Add script to test except param for fowardedfor/originalto options
- DOC: scheduler: add a diagram showing the different queues and their usages
- CLEANUP: tree-wide: replace free(x);x=NULL with ha_free(&x)
- CLEANUP: config: replace a few free() with ha_free()
- CLEANUP: vars: always zero the pointers after a free()
- CLEANUP: ssl: remove a useless "if" before freeing an error message
- CLEANUP: ssl: make ssl_sock_free_srv_ctx() zero the pointers after free
- CLEANUP: ssl: use realloc() instead of free()+malloc()
A network may be specified to avoid header addition for "forwardfor" and
"orignialto" option via the "except" parameter. However, only IPv4
networks/addresses are supported. This patch adds the support of IPv6.
To do so, the net_addr structure is used to store the parameter value in the
proxy structure. And ipcmp2net() function is used to perform the comparison.
This patch should fix the issue #1145. It depends on the following commit:
* c6ce0ab MINOR: tools: Add function to compare an address to a network address
* 5587287 MINOR: tools: Add net_addr structure describing a network addess
Released version 2.4-dev9 with the following main changes :
- BUG/MINOR: server: Remove RMAINT from admin state when loading server state
- CLEANUP: check: fix get_check_status_info declaration
- CLEANUP: contrib/prometheus-exporter: align for with srv status case
- MEDIUM: stats: allow to select one field in `stats_fill_li_stats`
- MINOR: stats: add helper to get status string
- MEDIUM: contrib/prometheus-exporter: add listen stats
- BUG/MINOR: dns: add test on result getting value from buffer into ring.
- BUG/MINOR: dns: dns_connect_server must return -1 unsupported nameserver's type
- BUG/MINOR: dns: missing test writing in output channel in session handler
- BUG/MINOR: dns: fix ring attach control on dns_session_new
- BUG/MEDIUM: dns: fix multiple double close on fd in dns.c
- BUG/MAJOR: connection: prevent double free if conn selected for removal
- BUG/MINOR: session: atomically increment the tracked sessions counter
- REGTESTS: fix http_reuse_conn_hash proxy test
- BUG/MINOR: backend: do not call smp_make_safe for sni conn hash
- MINOR: connection: remove pointers for prehash in conn_hash_params
- BUG/MINOR: checks: properly handle wrapping time in __health_adjust()
- BUG/MEDIUM: checks: don't needlessly take the server lock in health_adjust()
- DEBUG: thread: add 5 extra lock labels for statistics and debugging
- OPTIM: server: switch the actconn list to an mt-list
- Revert "MINOR: threads: change lock_t to an unsigned int"
- MINOR: lb/api: let callers of take_conn/drop_conn tell if they have the lock
- OPTIM: lb-first: do not take the server lock on take_conn/drop_conn
- OPTIM: lb-leastconn: do not take the server lock on take_conn/drop_conn
- OPTIM: lb-leastconn: do not unlink the server if it did not change
- MINOR: tasks: add DEBUG_TASK to report caller info in a task
- MINOR: tasks/debug: add some extra controls of use-after-free in DEBUG_TASK
- BUG/MINOR: sample: Always consider zero size string samples as unsafe
- MINOR: cli: add missing agent commands for set server
- BUILD/MEDIUM: da Adding pcre2 support.
- BUILD: ssl: introduce fine guard for OpenSSL specific SCTL functions
- REGTESTS: reorder reuse conn proxy protocol test
- DOC: explain the relation between pool-low-conn and tune.idle-pool.shared
- MINOR: tasks: refine the default run queue depth
- MINOR: listener: refine the default MAX_ACCEPT from 64 to 4
- MINOR: mux_h2: do not try to remove front conn from idle trees
- REGTESTS: workaround for a crash with recent libressl on http-reuse sni
- BUG/MEDIUM: lists: Avoid an infinite loop in MT_LIST_TRY_ADDQ().
- MINOR: connection: allocate dynamically hash node for backend conns
- DOC: DeviceAtlas documentation typo fix.
- BUG/MEDIUM: spoe: Resolve the sink if a SPOE logs in a ring buffer
- BUG/MINOR: http-rules: Always replace the response status on a return action
- BUG/MINOR: server: Init params before parsing a new server-state line
- BUG/MINOR: server: Be sure to cut the last parsed field of a server-state line
- MEDIUM: server: Don't introduce a new server-state file version
- DOC: contrib/prometheus-exporter: remove htx reference
- REGTESTS: contrib/prometheus-exporter: test NaN values
- REGTESTS: contrib/prometheus-exporter: test well known labels
- CI: github actions: switch to stable LibreSSL release
- BUG/MINOR: server: Fix test on number of fields allowed in a server-state line
- MINOR: dynbuf: make the buffer wait queue per thread
- MINOR: dynbuf: use regular lists instead of mt_lists for buffer_wait
- MINOR: dynbuf: pass offer_buffers() the number of buffers instead of a threshold
- MINOR: sched: have one runqueue ticks counter per thread
The maximum number of connections accepted at once by a thread for a single
listener used to default to 64 divided by the number of processes but the
tasklet-based model is much more scalable and benefits from smaller values.
Experimentation has shown that 4 gives the highest accept rate for all
thread values, and that 3 and 5 come very close, as shown below (HTTP/1
connections forwarded per second at multi-accept 4 and 64):
ac\thr| 1 2 4 8 16
------+------------------------------
4| 80k 106k 168k 270k 336k
64| 63k 89k 145k 230k 274k
Some tests were also conducted on SSL and absolutely no change was observed.
The value was placed into a define because it used to be spread all over the
code.
It might be useful at some point to backport this to 2.3 and 2.2 to help
those who observed some performance regressions from 1.6.
Since a lot of internal callbacks were turned to tasklets, the runqueue
depth had not been readjusted from the default 200 which was initially
used to favor batched processing. But nowadays it appears too large
already based on the following tests conducted on a 8c16t machine with
a simple config involving "balance leastconn" and one server. The setup
always involved the two threads of a same CPU core except for 1 thread,
and the client was running over 1000 concurrent H1 connections. The
number of requests per second is reported for each (runqueue-depth,
nbthread) couple:
rq\thr| 1 2 4 8 16
------+------------------------------
32| 120k 159k 276k 477k 698k
40| 122k 160k 276k 478k 722k
48| 121k 159k 274k 482k 720k
64| 121k 160k 274k 469k 710k
200| 114k 150k 247k 415k 613k <-- default
It's possible to save up to about 18% performance by lowering the
default value to 40. One possible explanation to this is that checking
I/Os more frequently allows to flush buffers faster and to smooth the
I/O wait time over multiple operations instead of alternating phases
of processing, waiting for locks and waiting for new I/Os.
The total round trip time also fell from 1.62ms to 1.40ms on average,
among which at least 0.5ms is attributed to the testing tools since
this is the minimum attainable on the loopback.
After some observation it would be nice to backport this to 2.3 and
2.2 which observe similar improvements, since some users have already
observed some perf regressions between 1.6 and 2.2.
Disabling idle-pool sharing can result in awful performance in presence
of a not so high number of threads, because the number of available idle
connections will be shared among threads, resulting in most of them
abandonning their connections after a request is done if there are already
enough total available. This is a case where pool-low-conn ought to be
used to preserve a number of connections for each thread, but this relation
isn't obvious as is. Let's add mentions about this with both keywords.
Released version 2.4-dev8 with the following main changes :
- BUILD: ssl: fix typo in HAVE_SSL_CTX_ADD_SERVER_CUSTOM_EXT macro
- BUILD: ssl: guard SSL_CTX_add_server_custom_ext with special macro
- BUG/MINOR: mux-h1: Don't emit extra CRLF for empty chunked messages
- MINOR: contrib/prometheus-exporter: use stats desc when possible followup
- MEDIUM: contrib/prometheus-exporter: export base stick table stats
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: check: fix some typo in comments
- CLEANUP: tools: typo in `strl2irc` mention
- BUILD: ssl: guard SSL_CTX_set_msg_callback with SSL_CTRL_SET_MSG_CALLBACK macro
- MEDIUM: ssl: add a rwlock for SSL server session cache
- BUG/MINOR: intops: fix mul32hi()'s off-by-one
- BUG/MINOR: freq_ctr: fix a wrong delay calculation in next_event_delay()
- MINOR: stick-tables/counters: add http_fail_cnt and http_fail_rate data types
- MINOR: ssl: add SSL_SERVER_LOCK label in threads.h
- BUG/MINOR: mux-h1: Don't increment HTTP error counter for 408/500/501 errors
- BUG/MINOR: http-ana: Don't increment HTTP error counter on internal errors
- BUG/MEDIUM: mux-h1: Always set CS_FL_EOI for response in MSG_DONE state
- BUG/MINOR: mux-h1: Fix data skipping for bodyless responses
- BUG/MINOR: mux-h1: Don't blindly skip EOT block for non-chunked messages
- BUG/MEDIUM: mux-h2: Add EOT block when EOM flag is set on an empty HTX message
- MINOR: mux-h1: Be sure EOM flag is set when processing end of outgoing message
- REGTESTS: Add a script to test payload skipping for bodyless HTTP responses
- BUG/MINOR: server: re-align state file fields number
- CLEANUP: muxes: Remove useless calls to b_realign_if_empty()
- BUG/MINOR: tools: Fix a memory leak on error path in parse_dotted_uints()
- CLEANUP: remove unused variable assigned found by Coverity
- CLEANUP: queue: Remove useless tests on p or pp in pendconn_process_next_strm()
- BUG/MINOR: backend: hold correctly lock when killing idle conn
- MEDIUM: connection: protect idle conn lists with locks
- MEDIUM: connection: replace idle conn lists by eb trees
- MINOR: backend: search conn in idle/safe trees after available
- MINOR: backend: search conn in idle tree after safe on always reuse
- MINOR: connection: prepare hash calcul for server conns
- MINOR: connection: use the srv pointer for the srv conn hash
- MINOR: backend: compare conn hash for session conn reuse
- MINOR: connection: use sni as parameter for srv conn hash
- MINOR: reg-tests: test http-reuse with sni
- MINOR: backend: rewrite alloc of stream target address
- MINOR: connection: use dst addr as parameter for srv conn hash
- MINOR: reg-test: test http-reuse with specific dst addr
- MINOR: backend: rewrite alloc of connection src address
- MINOR: connection: use src addr as parameter for srv conn hash
- MINOR: connection: use proxy protocol as parameter for srv conn hash
- MINOR: reg-tests: test http-reuse with proxy protocol
- MINOR: doc: update http reuse for new eligilible connections
- BUG/MINOR: backend: fix compilation without ssl
- REGTESTS: adjust http_reuse_conn_hash requirements
- REGTESTS: deactivate a failed test on CI in http_reuse_conn_hash
- REGTESTS: fix sni used in http_reuse_conn_hash for libressl 3.3.0
- CI: cirrus: update FreeBSD image to 12.2
- MEDIUM: cli: add check-addr command
- MEDIUM: cli: add agent-port command
- MEDIUM: server: add server-states version 2
- MEDIUM: server: support {check,agent}_addr, agent_port in server state
- MINOR: server: enhance error precision when applying server state
- BUG/MINOR: server: Fix server-state-file-name directive
- CLEANUP: deinit: release global and per-proxy server-state variables on deinit
- BUG/MEDIUM: config: don't pick unset values from last defaults section
- BUG/MINOR: stats: revert the change on ST_CONVDONE
- BUG/MINOR: cfgparse: do not mention "addr:port" as supported on proxy lines
- BUG/MINOR: http-htx: defpx must be a const in proxy_dup_default_conf_errors()
- BUG/MINOR: tcpheck: the source list must be a const in dup_tcpcheck_var()
- BUILD: proxy: add missing compression-t.h to proxy-t.h
- REORG: move init_default_instance() to proxy.c and pass it the defproxy pointer
- REORG: proxy: centralize the proxy allocation code into alloc_new_proxy()
- MEDIUM: proxy: only take defaults when a default proxy is passed.
- MINOR: proxy: move the defproxy freeing code to proxy.c
- MINOR: proxy: always properly reset the just freed default instance pointers
- BUG/MINOR: extcheck: proxy_parse_extcheck() must take a const for the defproxy
- BUG/MINOR: tcpcheck: proxy_parse_*check*() must take a const for the defproxy
- BUG/MINOR: server: parse_server() must take a const for the defproxy
- MINOR: cfgparse: move defproxy to cfgparse-listen as a static
- MINOR: proxy: add a new capability PR_CAP_DEF
- MINOR: cfgparse: check PR_CAP_DEF instead of comparing poiner against defproxy
- MINOR: cfgparse: use a pointer to the current default proxy
- MINOR: proxy: also store the name for a defaults section
- MINOR: proxy: support storing defaults sections into their own tree
- MEDIUM: proxy: store the default proxies in a tree by name
- MEDIUM: cfgparse: allow a proxy to designate the defaults section to use
- MINOR: http: add baseq sample fetch
- CLEANUP: tcpcheck: Remove a useless test on port variable
- BUG/MINOR: server: Don't call fopen() with server-state filepath set to NULL
- CLEANUP: server: Remove useless "filepath" variable in apply_server_state()
- MINOR: peers/cli: do not dump the peers dictionaries by default on "show peers"
- MINOR: cfgparse: implement a simple if/elif/else/endif macro block handler
- DOC: tune: explain the origin of block size for ssl.cachesize
- MINOR: tcp: add support for defer-accept on FreeBSD.
- MINOR: ring: adds new ring_init function.
- CLEANUP: channel: fix comment in ci_putblk.
- BUG/MINOR: dns: add missing sent counter and parent id to dns counters.
- BUG/MINOR: resolvers: fix attribute packed struct for dns
- MINOR: resolvers: renames some resolvers internal types and removes dns prefix
- MINOR: resolvers: renames type dns_resolvers to resolvers.
- MINOR: resolvers: renames some resolvers specific types to not use dns prefix
- MINOR: resolvers: renames some dns prefixed types using resolv prefix.
- MINOR: resolvers: renames resolvers DNS_RESP_* errcodes RSLV_RESP_*
- MINOR: resolvers: renames resolvers DNS_UPD_* returncodes to RSLV_UPD_*
- MINOR: resolvers: rework prototype suffixes to split resolving and dns.
- MEDIUM: resolvers: move resolvers section parsing from cfgparse.c to dns.c
- MINOR: resolvers: replace nameserver's resolver ref by generic parent pointer
- MINOR: resolvers: rework dns stats prototype because specific to resolvers
- MEDIUM: resolvers: split resolving and dns message exchange layers.
- MEDIUM: resolvers/dns: split dns.c into dns.c and resolvers.c
- MEDIUM: dns: adds code to support pipelined DNS requests over TCP.
- MEDIUM: resolvers: add supports of TCP nameservers in resolvers.
This patch introduce the new line "server" to set a TCP
nameserver in a "resolvers" section:
server <name> <address> [param*]
Used to configure a DNS TCP or stream server. This supports for all
"server" parameters found in 5.2 paragraph. Some of these parameters
are irrelevant for DNS resolving. Note: currently 4 queries are pipelined
on the same connections. A batch of idle connections are removed every
5 seconds. "maxconn" can be configured to limit the amount of those
concurrent connections and TLS should also usable if the server supports
. The current implementation limits to 4 pipelined
The name of the line in configuration is open to discussion
and could be changed before the next release.
A user could eventually ask himself where those 200 bytes block size are
coming from. This patch tries to better explain the origin in case
people are curious or want to double check the reality.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Very often, especially since reg-tests, it would be desirable to be able
to conditionally comment out a config block, such as removing an SSL
binding when SSL is disabled, or enabling HTX only for certain versions,
etc.
This patch introduces a very simple nested block management which takes
".if", ".elif", ".else" and ".endif" directives to take or ignore a block.
For now the conditions are limited to empty string or "0" for false versus
a non-nul integer for true, which already suffices to test environment
variables. Still, it needs to be a bit more advanced with defines, versions
etc.
A set of ".notice", ".warning" and ".alert" statements are provided to
emit messages, often in order to provide advice about how to fix certain
conditions.
Now it becomes possible to specify "from foo" on a frontend/listen/backend
or even on a "defaults" line, to mention that defaults section "foo" needs
to be used to preset the proxy's settings.
When not set, the last section remains used. In case the designated name
is found at multiple places, it is rejected and an error indicates two
occurrences of the same name. Similarly, if the section name is found,
its name must only use valid characters. This allows multiple named
defaults section to continue to coexist without the risk that they will
cause trouble by accident.
When it comes to "defaults" relying on another defaults, what happens is
just that a new defaults section is created from the designated one. This
will make it possible for example to reuse some settings such as log-format
like below:
defaults tcp-clear
log stdout local0 info
log-format "%ci:%cp/%b/%si:%sp %ST %ts %U/%B %{+Q}r"
defaults tcp-ssl
log stdout local0 info
log-format "%ci:%cp/%b/%si:%sp %ST %ts %U/%B %{+Q}r ssl=%sslv"
defaults http-clear from tcp-clear
mode http
defaults http-ssl from tcp-ssl
mode http
frontend fe1 from http-clear
bind :8001
frontend fe2 from http-ssl
bind :8002
A small corner case remains in the error detection, if a second defaults
section appears with the same name after the point where it was used, and
nobody references it, the duplicate will not be detected. This could be
addressed by performing the syntactic checks in check_config_validity(),
and by postponing the freeing of the defaults, after tagging a defaults
section as explicitly looked up by another section. This doesn't seem
that important at the moment though.
Since the beginning, this directive is documented to accept an optional file
name. But it should also be possible to use it without any argument to use
the backend name as file name. However, when no argument is provided, an
error is reported during the configuration parsing requesting an argument, a
file name or "use-backend-name". And This last special argument is not
documented.
So, to respect the documentation and to avoid configuration breakages, all
modes are now supported. If this directive is called with no argument or
with "use-backend-name", the backend name is use as file name for the
server-state file. Otherwise, the provided string is used.
In addition, we take care to release any previously allocated file name in
case this directive is defines multiple times in the same backend. And an
error is reported if more than one argument are defined. Finally, the
documentation is updated accordingly. Sections supporting this directive are
also mentioned.
This patch should be backported as far as 1.6.
Historically we've been counting lots of client-triggered events in stick
tables to help detect misbehaving ones, but we've been missing the same on
the server side, and there's been repeated requests for being able to count
the server errors per URL in order to precisely monitor the quality of
service or even to avoid routing requests to certain dead services, which
is also called "circuit breaking" nowadays.
This commit introduces http_fail_cnt and http_fail_rate, which work like
http_err_cnt and http_err_rate in that they respectively count events and
their frequency, but they only consider server-side issues such as network
errors, unparsable and truncated responses, and 5xx status codes other
than 501 and 505 (since these ones are usually triggered by the client).
Note that retryable errors are purposely not accounted for, so that only
what the client really sees is considered.
With this it becomes very simple to put some protective measures in place
to perform a redirect or return an excuse page when the error rate goes
beyond a certain threshold for a given URL, and give more chances to the
server to recover from this condition. Typically it could look like this
to bypass a URL causing more than 10 requests per second:
stick-table type string len 80 size 4k expire 1m store http_fail_rate(1m)
http-request track-sc0 base # track host+path, ignore query string
http-request return status 503 content-type text/html \
lf-file excuse.html if { sc0_http_fail_rate gt 10 }
A more advanced mechanism using gpt0 could even implement high/low rates
to disable/enable the service.
Reg-test converteers_ref_cnt_never_dec.vtc was updated to test it.
Released version 2.4-dev7 with the following main changes :
- BUG/MINOR: stats: Continue to fill frontend stats on unimplemented metric
- BUILD: ssl: guard Client Hello callbacks with HAVE_SSL_CLIENT_HELLO_CB macro instead of openssl version
- BUG/MINOR: stats: Init the metric variable when frontend stats are filled
- MINOR: contrib/prometheus-exporter: better output of Not-a-Number
- CLEANUP: stats: improve field selection for frontend http fields
- CLEANUP: assorted typo fixes in the code and comments
- DOC: Improve documentation of the various hdr() fetches
- MEDIUM: stats: allow to select one field in `stats_fill_be_stats`
- MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump
- MEDIUM: stats: allow to select one field in `stats_fill_sv_stats`
- MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump
- MINOR: abort() on my_unreachable() when DEBUG_USE_ABORT is set.
- BUG/MEDIUM: filters/htx: Fix data forwarding when payload length is unknown
- BUG/MINOR: config: fix leak on proxy.conn_src.bind_hdr_name
- MINOR: reg-tests: add http-reuse test
- CLEANUP: srv: fix comment for pool-max-conn
- CLEANUP: backend: remove an obsolete comment on conn_backend_get
- REORG: backend: simplify conn_backend_get
- MINOR: ssl: Server ssl context prepare function refactoring
- MINOR: ssl: Certificate chain loading refactorization
- MEDIUM: ssl: Load client certificates in a ckch for backend servers
- MEDIUM: ssl: Enable backend certificate hot update
- MINOR: ssl: Remove client_crt member of the server's ssl context
- CLEANUP: ssl/cli: rework free in cli_io_handler_commit_cert()
- CLEANUP: ssl: remove SSL_CTX function parameter
- CLEANUP: ssl: make load_srv_{ckchs,cert} match their bind counterpart
- BUILD: Include stdlib.h in compiler.h if DEBUG_USE_ABORT is set
- CI: Fix DEBUG_STRICT definition for Coverity
- BUG/MINOR: stats: Remove a break preventing ST_F_QCUR to be set for servers
- BUG/MINOR: stats: Add a break after filling ST_F_MODE field for servers
- CLEANUP: ssl: remove dead code in ckch_inst_new_load_srv_store()
- BUG/MINOR: ssl: init tmp chunk correctly in ssl_sock_load_sctl_from_file()
- BUG/MEDIUM: session: only retrieve ready idle conn from session
- BUG/MEDIUM: backend: never reuse a connection for tcp mode
- REGTESTS: set_ssl_server_cert.vtc: remove the abort command
- REGTESTS: set_ssl_server_cert.vtc: check the Sha1 Fingerprint
- REGTESTS: set_ssl_server_cert.vtc: check the sha1 from the server
- MEDIUM: stream-int: Take care of EOS if the SI wake callback function
- MINOR: mux-h1: Try to wake up data layer first before calling its wake callback
- MINOR: mux-h1: Wake up H1C after its creation if input buffer is not empty
- MEDIUM: mux-h1: Add ST_READY state for the H1 connections
- MINOR: stream: Add a function to validate TCP to H1 upgrades
- MEDIUM: http-ana: Do nothing in wait-for-request analyzer if not htx
- BUG/MEDIUM: stream: Don't immediatly ack the TCP to H1 upgrades
- BUG/MAJOR: mux-h1: Properly handle TCP to H1 upgrades
- MINOR: htx/http-ana: Save info about Upgrade option in the Connection header
- MEDIUM: http-ana: Refuse invalid 101-switching-protocols responses
- BUG/MINOR: h2/mux-h2: Reject 101 responses with a PROTOCOL_ERROR h2s error
- MINOR: mux-h1/mux-fcgi: Don't set TUNNEL mode if payload length is unknown
- MINOR: mux-h1: Split H1C_F_WAIT_OPPOSITE flag to separate input/output sides
- MINOR: mux-h2: Add 2 flags to help to properly handle tunnel mode
- MEDIUM: mux-h2: Block client data on server side waiting tunnel establishment
- MEDIUM: mux-h2: Close streams when processing data for an aborted tunnel
- MEDIUM: mux-h1: Properly handle tunnel establishments and aborts
- BUG/MAJOR: mux-h1/mux-h2/htx: Fix HTTP tunnel management at the mux level
- MINOR: htx: Rename HTX_FL_EOI flag into HTX_FL_EOM
- REGTESTS: Don't run http_msg_full_on_eom script on the 2.4 anymore
- MINOR: htx: Add a function to know if a block is the only one in a message
- MAJOR: htx: Remove the EOM block type and use HTX_FL_EOM instead
- MINOR: mux-h1: Add a flag on H1 streams with a response known to be bodyless
- MEDIUM: mux-h1: Don't emit any payload for bodyless responses
- MINOR: mux-h1: Don't emit C-L and T-E headers for 204 and 1xx responses
- MINOR: mux-h1: Don't add Connection close/keep-alive header for 1xx messages
- MINOR: h2/mux-h2: Add flags to notify the response is known to have no body
- MEDIUM: mux-h2: Don't emit DATA frame for bodyless responses
- MEDIUM: http-ana: Deal with L7 retries in HTTP analysers
- MINOR: h1: reject websocket handshake if missing key
- MEDIUM: h1: generate WebSocket key on response if needed
- MINOR: mux_h2: define H2_SF_EXT_CONNECT_SENT stream flag
- MEDIUM: h2: parse Extended CONNECT reponse to htx
- MEDIUM: mux_h2: generate Extended CONNECT from htx upgrade
- MEDIUM: h1: add a WebSocket key on handshake if needed
- MEDIUM: mux_h2: generate Extended CONNECT response
- MEDIUM: h2: parse Extended CONNECT request to htx
- MEDIUM: h2: send connect protocol h2 settings
- MINOR: vtc: add test for h1/h2 protocol upgrade translation
- MINOR: vtc: add websocket test
- REGTESTS: Fix required versions for several scripts
- REGTEST: Don't use the websocket to validate http-check
- MINOR: mux-h1/trace: add traces at level ERROR for all kind of errors
- MINOR: mux-fcgi/trace: add traces at level ERROR for all kind of errors
- MINOR: h1: Raise the chunk size limit up to (2^52 - 1)
- BUG/MEDIUM: listener: do not accept connections faster than we can process them
- REGTESTS: set_ssl_server_cert.vtc: set as broken
- Revert "BUG/MEDIUM: listener: do not accept connections faster than we can process them"
- BUG/MINOR: backend: check available list allocation for reuse
- CI: Fix the coverity builds
- DOC: management: fix "show resolvers" alphabetical ordering
- MINOR: tools: add print_time_short() to print a condensed duration value
- MINOR: activity: make profiling more manageable
- MINOR: activity: declare a new structure to collect per-function activity
- MEDIUM: tasks/activity: collect per-task statistics when profiling is enabled
- MINOR: activity: also report collected tasks stats in "show profiling"
- MINOR: activity: flush scheduler stats on "set profiling tasks on"
- MINOR: activity: add a new "show tasks" command to list currently active tasks
- MINOR: listener: export accept_queue_process
- MINOR: session: export session_expire_embryonic()
- MINOR: muxes: export the timeout and shutr task handlers
- MINOR: checks: export a few functions that appear often in trace dumps
- MINOR: peers: export process_peer_sync() to improve traces
- MINOR: stick-tables: export process_table_expire()
- MINOR: mux-h1: Remove first useless test on count in h1_process_output()
- BUG/MINOR: stick-table: Always call smp_fetch_src() with a valid arg list
- MINOR: http-fetch: Don't check if argument list is set in sample fetches
- MINOR: http-conv: Don't check if argument list is set in sample converters
- MINOR: sample: Don't check if argument list is set in sample fetches
- MINOR: ssl-sample: Don't check if argument list is set in sample fetches
- MINOR: mux-h2: Don't tests the start-line when sending HEADERS frame
- MINOR: mux-h2: Slightly improve request HEADERS frames sending
- MINOR: contrib/prometheus-exporter: declare states for objects
- MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels
- MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones
- MINOR: listener: export manage_global_listener_queue()
- BUG/MINOR: activity: take care of late wakeups in "show tasks"
- REGTESTS: set_ssl_server_cert.vtc: remove SSL caching and set as working
- REGTESTS: set_ssl_server_cert: cleanup the SSL caching option
- MINOR: checks: Add function to get the result code corresponding to a status
- MAJOR: contrib/prometheus-exporter: move health check status to labels
- MINOR: contrib/prometheus-exporter: improve service status description field
- MINOR: stats: improve pending connections description
- MINOR: stats: improve max stats descriptions
- MINOR: contrib/prometheus-exporter: use stats desc when possible
- MINOR: contrib/prometheus-exporter: add uweight field
- MINOR: contrib/prometheus-exporter: add recv logs_logs_total field
- CLEANUP: contrib/prometheus-exporter: remove unused includes
- CLEANUP: contrib/prometheus-exporter: align and reorder fields
- CLEANUP: contrib/prometheus-exporter: remove description in README
- DOC: contrib/prometheus-exporter: Add missing metrics in README
- BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX
- BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos
- BUG/MEDIUM: ssl/cli: abort ssl cert is freeing the old store
- BUG/MEDIUM: ssl: check a connection's status before computing a handshake
- BUG/MINOR: mux_h2: fix incorrect stat titles
- MINOR: ssl/cli: flush the server session cache upon 'commit ssl cert'
- BUG/MINOR: cli: fix set server addr/port coherency with health checks
- MINOR: server: Don't set the check port during the update from a state file
- MINOR: dns: Don't set the check port during a server dns resolution
- MEDIUM: check: remove checkport checkaddr flag
- MEDIUM: server: adding support for check_port in server state
- BUG/MINOR: check: consitent way to set agentaddr
- MEDIUM: check: align agentaddr and agentport behaviour
- DOC: server: Add missing params in comment of the server state line parsing
- BUG/MINOR: xxhash: make sure armv6 uses memcpy()
- REGTESTS: mark http-check-send.vtc as 2.4-only
- REGTESTS: mark sample_fetches/hashes.vtc as 2.4-only
- BUG/MINOR: ssl: do not try to use early data if not configured
- REGTESTS: unbreak http-check-send.vtc
- MINOR: cli/show_fd: report local and report ports when known
- BUILD: Makefile: move REGTESTST_TYPE default setting
- BUG/MEDIUM: mux-h2: handle remaining read0 cases
- CLEANUP: http-htx: Set buffer area to NULL instead of malloc(0)
- BUG/MINOR: sock: Unclosed fd in case of connection allocation failure
- BUG/MEDIUM: mux-h2: do not quit the demux loop before setting END_REACHED
in the same manner of agentaddr, we now:
- permit to set agentport through `port` keyword, like it is the case
for agentaddr through `addr`
- set the priority on `agent-port` keyword when used
- add a flag to be able to test when the value is set like for agentaddr
it makes the behaviour between `addr` and `port` more consistent.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
The EOM block may be removed. The HTX_FL_EOM flags is enough. Most of time,
to know if the end of the message is reached, we just need to have an empty
HTX message with HTX_FL_EOM flag set. It may also be detected when the last
block of a message with HTX_FL_EOM flag is manipulated.
Removing EOM blocks simplifies the HTX message filling. Indeed, there is no
more edge problems when the message ends but there is no more space to write
the EOM block. However, some part are more tricky. Especially the
compression filter or the FCGI mux. The compression filter must finish the
compression on the last DATA block. Before it was performed on the EOM
block, an extra DATA block with the checksum was added. Now, we must detect
the last DATA block to be sure to finish the compression. The FCGI mux on
its part must be sure to reserve the space for the empty STDIN record on the
last DATA block while this record was inserted on the EOM block.
The H2 multiplexer is probably the part that benefits the most from this
change. Indeed, it is now fairly easier to known when to set the ES flag.
The HTX documentaion has been updated accordingly.
GitHub issue #796 notes that many administrators miss the fact that the `hdr()`
fetch (without the `f`) splits the header value at commas. This is only
mentioned at the end of a long paragraph.
This patch attempts to improve the documentation by:
- Explaning the "comma issue" as early as possible.
- Adding newlines to split the explanation into distinct sections.
- Reducing duplication by making the `res` siblings refer to their `req`
counterparts.
This patch may be backported as long as it applies cleanly. During the
refactoring I needed to adjust several explanations for consistency and not all
of them might be available in older branches.
Released version 2.4-dev6 with the following main changes :
- MINOR: converter: adding support for url_enc
- BUILD: SSL: guard TLS13 ciphersuites with HAVE_SSL_CTX_SET_CIPHERSUITES
- BUILD: ssl: guard EVP_PKEY_get_default_digest_nid with ASN1_PKEY_CTRL_DEFAULT_MD_NID
- BUILD: ssl: guard openssl specific with SSL_READ_EARLY_DATA_SUCCESS
- BUILD: Makefile: exclude broken tests by default
- CLEANUP: cfgparse: replace "realloc" with "my_realloc2" to fix to memory leak on error
- BUG/MINOR: hlua: Fix memory leak in hlua_alloc
- MINOR: contrib/prometheus-exporter: export build_info
- DOC: fix some spelling issues over multiple files
- CLEANUP: Fix spelling errors in comments
- SCRIPTS: announce-release: fix typo in help message
- CI: github: add a few more words to the codespell ignore list
- DOC: Add maintainers for the Prometheus exporter
- BUG/MINOR: sample: fix concat() converter's corruption with non-string variables
- BUG/MINOR: server: Memory leak of proxy.used_server_addr during deinit
- CLEANUP: sample: remove uneeded check in json validation
- MINOR: reg-tests: add a way to add service dependency
- BUG/MINOR: sample: check alloc_trash_chunk return value in concat()
- BUG/MINOR: reg-tests: fix service dependency script
- MINOR: reg-tests: add base prometheus test
- Revert "BUG/MINOR: dns: SRV records ignores duplicated AR records"
- BUG/MINOR: sample: Memory leak of sample_expr structure in case of error
- BUG/MINOR: check: Don't perform any check on servers defined in a frontend
- BUG/MINOR: init: enforce strict-limits when using master-worker
- MINOR: contrib/prometheus-exporter: avoid connection close header
- MINOR: contrib/prometheus-exporter: use fill_info for process dump
- BUG/MINOR: init: Use a dynamic buffer to set HAPROXY_CFGFILES env variable
- MINOR: config: Add failifnotcap() to emit an alert on proxy capabilities
- MINOR: server: Forbid server definitions in frontend sections
- BUG/MINOR: threads: Fixes the number of possible cpus report for Mac.
- CLEANUP: pattern: rename pat_ref_commit() to pat_ref_commit_elt()
- MINOR: pattern: add the missing generation ID manipulation functions
- MINOR: peers: Add traces for peer control messages.
- BUG/MINOR: dns: SRV records ignores duplicated AR records (v2)
- BUILD: peers: fix build warning about unused variable
- BUG/MEDIUM: stats: add missing INF_BUILD_INFO definition
- MINOR: cache: Do not store responses with an unknown encoding
- BUG/MINOR: peers: Possible appctx pointer dereference.
- MINOR: build: discard echoing in help target
- MINOR: cache: Remove the `hash` part of the accept-encoding secondary key
- CLEANUP: cache: Use proper data types in secondary_key_cmp()
- CLEANUP: Rename accept_encoding_hash_cmp to accept_encoding_bitmap_cmp
- BUG/MINOR: peers: Wrong "new_conn" value for "show peers" CLI command.
- MINOR: contrib: Make the wireshark peers dissector compile for more distribs.
- BUG/MINOR: mux_h2: missing space between "st" and ".flg" in the "show fd" helper
- CLEANUP: tools: make resolve_sym_name() take a const pointer
- CLEANUP: cli: make "show fd" use a const connection to access other fields
- MINOR: cli: make "show fd" also report the xprt and xprt_ctx
- MINOR: xprt: add a new show_fd() helper to complete some "show fd" dumps.
- MINOR: ssl: provide a "show fd" helper to report important SSL information
- MINOR: xprt/mux: export all *_io_cb functions so that "show fd" resolves them
- MINOR: mux-h2: make the "show fd" helper also decode the h2s subscriber when known
- MINOR: mux-h1: make the "show fd" helper also decode the h1s subscriber when known
- MINOR: mux-fcgi: make the "show fd" helper also decode the fstrm subscriber when known
- CI: Pin VTest to a known good commit
- MINOR: cli: give the show_fd helpers the ability to report a suspicious entry
- MINOR: cli/show_fd: report some easily detectable suspicious states
- MINOR: ssl/show_fd: report some FDs as suspicious when possible
- MINOR: mux-h2/show_fd: report as suspicious an entry with too many calls
- MINOR: mux-h1/show_fd: report as suspicious an entry with too many calls
- BUG/MINOR: mworker: define _GNU_SOURCE for strsignal()
- BUG/MEDIUM: tcpcheck: Don't destroy connection in the wake callback context
- BUG/MEDIUM: mux-h2: Xfer rxbuf to the upper layer when creating a front stream
- MINOR: http: Add HTTP 501-not-implemented error message
- MINOR: muxes: Add exit status for errors about not implemented features
- MINOR: mux-h1: Be prepared to return 501-not-implemented error during parsing
- MEDIUM: mux-h1: Return a 501-not-implemented for upgrade requests with a body
- DOC: Remove space after comma in converter signature
- DOC: Rename '<var name>' to '<var>' in converter signature
- MINOR: stats: duplicate 3 fields in bytes in info
- MINOR: stats: add new start time field
- MINOR: contrib/prometheus-exporter: merge info description from stats
- MEDIUM: stats: allow to select one field in `stats_fill_fe_stats`
- MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump
- MINOR: contrib/prometheus-exporter: Don't needlessly set empty label for metrics
- MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag
- MINOR: contrib/prometheus-exporter: Add promex_metric struct defining a metric
- MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics
- BUG/MINOR: stream: Don't update counters when TCP to H2 upgrades are performed
- BUG/MEDIUM: mux-h2: fix read0 handling on partial frames
- MINOR: debug: always export the my_backtrace function
- MINOR: debug: extract the backtrace dumping code to its own function
- MINOR: debug: create ha_backtrace_to_stderr() to dump an instant backtrace
- MEDIUM: debug: now always print a backtrace on CRASH_NOW() and friends
- MINOR: debug: let ha_dump_backtrace() dump a bit further for some callers
- BUILD: debug: fix build warning by consuming the write() result
- MINOR: lua: remove unused variable
- BUILD/MINOR: lua: define _GNU_SOURCE for LLONG_MAX
Add the support for the 501-not-implemented status code with the
corresponding default message. The documentation is updated accordingly
because it is now part of status codes HAProxy may emit via an errorfile or
a deny/return HTTP action.
If a server varies on the accept-encoding header and it sends a response
with an encoding we do not know (see parse_encoding_value function), we
will not store it. This will prevent unexpected errors caused by
cache collisions that could happen in accept_encoding_hash_cmp.
add base support for url encode following RFC3986, supporting `query`
type only.
- add test checking url_enc/url_dec/url_enc
- update documentation
- leave the door open for future changes
this should resolve github issue #941
Signed-off-by: William Dauchy <wdauchy@gmail.com>
Released version 2.4-dev5 with the following main changes :
- BUG/MEDIUM: mux_h2: Add missing braces in h2_snd_buf()around trace+wakeup
- BUILD: hpack: hpack-tbl-t.h uses VAR_ARRAY but does not include compiler.h
- MINOR: time: increase the minimum wakeup interval to 60s
- MINOR: check: do not ignore a connection header for http-check send
- REGTESTS: complete http-check test
- CI: travis-ci: drop coverity scan builds
- MINOR: atomic: don't use ; to separate instruction on aarch64.
- IMPORT: xxhash: update to v0.8.0 that introduces stable XXH3 variant
- MEDIUM: xxhash: use the XXH3 functions to generate 64-bit hashes
- MEDIUM: xxhash: use the XXH_INLINE_ALL macro to inline all functions
- CLEANUP: xxhash: remove the unused src/xxhash.c
- MINOR: sample: add the xxh3 converter
- REGTESTS: add tests for the xxh3 converter
- MINOR: protocol: Create proto_quic QUIC protocol layer.
- MINOR: connection: Attach a "quic_conn" struct to "connection" struct.
- MINOR: quic: Redefine control layer callbacks which are QUIC specific.
- MINOR: ssl_sock: Initialize BIO and SSL objects outside of ssl_sock_init()
- MINOR: connection: Add a new xprt to connection.
- MINOR: ssl: Export definitions required by QUIC.
- MINOR: cfgparse: Do not modify the QUIC xprt when parsing "ssl".
- MINOR: tools: Add support for QUIC addresses parsing.
- MINOR: quic: Add definitions for QUIC protocol.
- MINOR: quic: Import C source code files for QUIC protocol.
- MINOR: listener: Add QUIC info to listeners and receivers.
- MINOR: server: Add QUIC definitions to servers.
- MINOR: ssl: SSL CTX initialization modifications for QUIC.
- MINOR: ssl: QUIC transport parameters parsing.
- MINOR: quic: QUIC socket management finalization.
- MINOR: cfgparse: QUIC default server transport parameters init.
- MINOR: quic: Enable the compilation of QUIC modules.
- MAJOR: quic: Make usage of ebtrees to store QUIC ACK ranges.
- MINOR: quic: Attempt to make trace more readable
- MINOR: quic: Make usage of the congestion control window.
- MINOR: quic: Flag RX packet as ack-eliciting from the generic parser.
- MINOR: quic: Code reordering to help in reviewing/modifying.
- MINOR: quic: Add traces to congestion avoidance NewReno callback.
- MINOR: quic: Display the SSL alert in ->ssl_send_alert() callback.
- MINOR: quic: Update the initial salt to that of draft-29.
- MINOR: quic: Add traces for in flght ack-eliciting packet counter.
- MINOR: quic: make a packet build fails when qc_build_frm() fails.
- MINOR: quic: Add traces for quic_packet_encrypt().
- MINOR: cache: Refactoring of secondary_key building functions
- MINOR: cache: Avoid storing responses whose secondary key was not correctly calculated
- BUG/MINOR: cache: Manage multiple headers in accept-encoding normalization
- MINOR: cache: Add specific secondary key comparison mechanism
- MINOR: http: Add helper functions to trim spaces and tabs
- MEDIUM: cache: Manage a subset of encodings in accept-encoding normalizer
- REGTESTS: cache: Simplify vary.vtc file
- REGTESTS: cache: Add a specific test for the accept-encoding normalizer
- MINOR: cache: Remove redundant test in http_action_req_cache_use
- MINOR: cache: Replace the "process-vary" option's expected values
- CI: GitHub Actions: enable daily Coverity scan
- BUG/MEDIUM: cache: Fix hash collision in `accept-encoding` handling for `Vary`
- MEDIUM: stick-tables: Add srvkey option to stick-table
- REGTESTS: add test for stickiness using "srvkey addr"
- BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11
- BUG/MINOR: sink: Return an allocation failure in __sink_new if strdup() fails
- BUG/MINOR: lua: Fix memory leak error cases in hlua_config_prepend_path
- MINOR: lua: Use consistent error message 'memory allocation failed'
- CLEANUP: Compare the return value of `XXXcmp()` functions with zero
- CLEANUP: Apply the coccinelle patch for `XXXcmp()` on include/
- CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/
- MINOR: qpack: Add static header table definitions for QPACK.
- CLEANUP: qpack: Wrong comment about the draft for QPACK static header table.
- CLEANUP: quic: Remove useless QUIC event trace definitions.
- BUG/MINOR: quic: Possible CRYPTO frame building errors.
- MINOR: quic: Pass quic_conn struct to frame parsers.
- BUG/MINOR: quic: Wrong STREAM frames parsing.
- MINOR: quic: Drop packets with STREAM frames with wrong direction.
- CLEANUP: ssl: Remove useless loop in tlskeys_list_get_next()
- CLEANUP: ssl: Remove useless local variable in tlskeys_list_get_next()
- MINOR: ssl: make tlskeys_list_get_next() take a list element
- Revert "BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11"
- BUG/MINOR: cfgparse: Fail if the strdup() for `rule->be.name` for `use_backend` fails
- CLEANUP: mworker: remove duplicate pointer tests in cfg_parse_program()
- CLEANUP: Reduce scope of `header_name` in http_action_store_cache()
- CLEANUP: Reduce scope of `hdr_age` in http_action_store_cache()
- CLEANUP: spoe: fix typo on `var_check_arg` comment
- BUG/MINOR: tcpcheck: Report a L7OK if the last evaluated rule is a send rule
- CI: github actions: build several popular "contrib" tools
- DOC: Improve the message printed when running `make` w/o `TARGET`
- BUG/MEDIUM: server: srv_set_addr_desc() crashes when a server has no address
- REGTESTS: add unresolvable servers to srvkey-addr
- BUG/MINOR: stats: Make stat_l variable used to dump a stat line thread local
- BUG/MINOR: quic: NULL pointer dereferences when building post handshake frames.
- SCRIPTS: improve announce-release to support different tag and versions
- SCRIPTS: make announce release support preparing announces before tag exists
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: srv: do not init address if backend is disabled
- BUG/MINOR: srv: do not cleanup idle conns if pool max is null
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: few extra typo and fixes over last one ("ot" -> "to")
This allows using the address of the server rather than the name of the
server for keeping track of servers in a backend for stickiness.
The peers code was also extended to support feeding the dictionary using
this key instead of the name.
Fixes#814
Allow the user to specify a custom Connection header for http-check
send. This is useful for example to implement a websocket upgrade check.
If no connection header has been set, a 'Connection: close' header is
automatically appended to allow the server to close the connection
immediately after the request/response.
Update the documentation related to http-check send.
This fixes the github issue #1009.
Released version 2.4-dev4 with the following main changes :
- BUG/MEDIUM: lb-leastconn: Reposition a server using the right eweight
- BUG/MEDIUM: mux-h1: Fix a deadlock when a 408 error is pending for a client
- BUG/MEDIUM: ssl/crt-list: bad behavior with "commit ssl cert"
- BUG/MAJOR: cache: Crash because of disabled entry not removed from the tree
- BUILD: SSL: fine guard for SSL_CTX_add_server_custom_ext call
- MEDIUM: cache: Add a secondary entry counter and insertion limitation
- MEDIUM: cache: Avoid going over duplicates lists too often
- MINOR: cache: Add a max-secondary-entries cache option
- CI: cirrus: drop CentOS 6 builds
- BUILD: Makefile: have "make clean" destroy .o/.a/.s in contrib subdirs as well
- MINOR: vars: replace static functions with global ones
- MINOR: opentracing: add ARGC_OT enum
- CONTRIB: opentracing: add the OpenTracing filter
- DOC: opentracing: add the OpenTracing filter section
- REGTESTS: make use of HAPROXY_ARGS and pass -dM by default
- BUG/MINOR: http: Establish a tunnel for all 2xx responses to a CONNECT
- BUG/MINOR: mux-h1: Don't set CS_FL_EOI too early for protocol upgrade requests
- BUG/MEDIUM: http-ana: Never for sending data in TUNNEL mode
- CLEANUP: mux-h2: Rename h2s_frt_make_resp_data() to be generic
- CLEANUP: mux-h2: Rename h2c_frt_handle_data() to be generic
- BUG/MEDIUM: mux-h1: Handle h1_process() failures on a pipelined request
- CLEANUP: debug: mark the RNG's seed as unsigned
- CONTRIB: halog: fix build issue caused by %L printf format
- CONTRIB: halog: mark the has_zero* functions unused
- CONTRIB: halog: fix signed/unsigned build warnings on counts and timestamps
- CONTRIB: debug: address "poll" utility build on non-linux platforms
- BUILD: plock: remove dead code that causes a warning in gcc 11
- BUILD: ssl: fine guard for SSL_CTX_get0_privatekey call
- BUG/MINOR: dns: SRV records ignores duplicated AR records
- DOC: fix "smp_size" vs "sample_size" in "log" directive arguments
- CLEANUP: assorted typo fixes in the code and comments
- DOC: assorted typo fixes in the documentation
- CI: codespell: whitelist "te" and "nd" words
This new option allows to tune the maximum number of simultaneous
entries with the same primary key in the cache (secondary entries).
When we try to store a response in the cache and there are already
max-secondary-entries living entries in the cache, the storage will
fail (but the response will still be sent to the client).
It defaults to 10 and does not have a maximum number.
Released version 2.4-dev3 with the following main changes :
- MINOR: log: Logging HTTP path only with %HPO
- BUG/MINOR: mux-h2/stats: make stream/connection proto errors more accurate
- MINOR: traces: add a new level "error" below the "user" level
- MINOR: mux-h2/trace: add traces at level ERROR for protocol errors
- BUG/MINOR: mux-h2/stats: not all GOAWAY frames are errors
- BUG/MINOR: lua: missing "\n" in error message
- BUG/MINOR: lua: lua-load doesn't check its parameters
- BUG/MINOR: lua: Post init register function are not executed beyond the first one
- BUG/MINOR: lua: Some lua init operation are processed unsafe
- MINOR: actions: Export actions lookup functions
- MINOR: actions: add a function returning a service pointer from its name
- MINOR: cli: add a function to look up a CLI service description
- BUG/MINOR: lua: warn when registering action, conv, sf, cli or applet multiple times
- MINOR: cache: Improve accept_encoding_normalizer
- MINOR: cache: Add entry to the tree as soon as possible
- BUG/MINOR: trace: Wrong displayed trace level
- BUG/MAJOR: ring: tcp forward on ring can break the reader counter.
- MINOR: lua: simplify hlua_alloc() to only rely on realloc()
- MEDIUM: lua-thread: use atomics for memory accounting
- MINOR: lua-thread: remove struct hlua from function hlua_prepend_path()
- MEDIUM: lua-thread: make hlua_post_init() no longer use the runtime execution function
- MINOR: lua-thread: hlua_ctx_renew() is never called with main gL lua state
- MINOR: lua-thread: Use NULL context for main lua state
- MINOR: lua-thread: Stop usage of struct hlua for the global lua state
- MINOR: lua-thread: Replace embedded struct hlua_function by a pointer
- MINOR: lua-thread: Split hlua_init() function in two parts
- MINOR: lua-thread: make hlua_ctx_init() get L from its caller
- MINOR: lua-thread: Split hlua_load function in two parts
- MINOR: lua-thread: Split hlua_post_init() function in two parts
- MINOR: lua-thread: Add the "thread" core variable
- MEDIUM: lua-thread: No longer use locked context in initialization parts
- MEDIUM: lua-thread: Apply lock only if the parent state is the main thread
- MINOR: lua-thread: Replace global gL var with an array of states
- MINOR: lua-thread: Replace "struct hlua_function" allocation by dedicated function
- MINOR: lua-thread: Replace state_from by state_id
- MINOR: lua-thread: Store each function reference and init reference in array
- MEDIUM: lua-thread: Add the lua-load-per-thread directive
- MINOR: lua-thread: Add verbosity in errors
- REGTESTS: add a test for the threaded Lua code
- BUILD/MINOR: haproxy DragonFlyBSD affinity build update.
- DOC/MINOR: Fix formatting in Management Guide
- MINOR: cache: Do not store stale entry
- MINOR: cache: Add extra "cache-control" value checks
- MEDIUM: cache: Remove cache entry in case of POST on the same resource
- MINOR: cache: Consider invalid Age values as stale
- BUG/MEDIUM: lua-thread: some parts must be initialized once
- BUG/MINOR: lua-thread: close all states on deinit
- BUG/MINOR: listener: use sockaddr_in6 for IPv6
- BUG/MINOR: mux-h1: Handle keep-alive timeout for idle frontend connections
- MINOR: session: Add the idle duration field into the session
- MINOR: mux-h1: Update session idle duration when data are received
- MINOR: mux-h1: Reset session dates and durations info when the CS is detached
- MINOR: logs: Use session idle duration when no stream is provided
- MINOR: stream: Always get idle duration from the session
- MINOR: stream: Don't retrieve anymore timing info from the mux csinfo
- MINOR: mux-h1: Don't provide anymore timing info using cs_info structure
- MINOR: muxes: Remove get_cs_info callback function now useless
- MINOR: stream: Pass an optional input buffer when a stream is created
- MINOR: mux-h1: Add a flag to disable reads to wait opposite side
- MEDIUM: mux-h1: Use a h1c flag to block reads when splicing is in-progress
- MINOR: mux-h1: Introduce H1C_F_IS_BACK flag on the H1 connection
- MINOR: mux-h1: Separate parsing and formatting errors at H1 stream level
- MINOR: mux-h1: Split front/back h1 stream creation in 2 functions
- MINOR: mux-h1: Add a rxbuf into the H1 stream
- MINOR: mux-h1: Don't set CS flags in internal parsing functions
- MINOR: mux-h1: Add embryonic and attached states on the H1 connection
- MINOR: mux-h1: rework the h1_timeout_task() function
- MINOR: mux-h1: Reset more H1C flags when a H1 stream is destroyed
- MINOR: mux-h1: Disable reads if an error was reported on the H1 stream
- MINOR: mux-h1: Rework how shutdowns are handled
- MINOR: mux-h1: Rework h1_refresh_timeout to be easier to read
- MINOR: mux-h1: Process next request for IDLE connection only
- MINOR: mux-h1: Add a idle expiration date on the H1 connection
- MINOR: stick-tables: Add functions to update some values of a tracked counter
- MINOR: session: Add functions to increase http values of tracked counters
- MINOR: mux: Add a ctl parameter to get the exit status of the multiplexers
- MINOR: logs: Get the multiplexer exist status when no stream is provided
- MINOR: mux-h1: Add functions to send HTTP errors from the mux
- MAJOR: mux-h1: Create the client stream as later as possible
- DOC: config: Add notes about errors emitted by H1 mux
- CLEANUP: mux-h1: Rename H1C_F_CS_* flags and reorder H1C flags
- MINOR: http-ana: Remove useless update of t_idle duration of the stream
- CLEANUP: htx: Remove HTX_FL_UPGRADE unsued flag
- MEDIUM: http-ana: Don't process partial or empty request anymore
- CLEANUP: http-ana: Remove TX_WAIT_NEXT_RQ unsued flag
- CLEANUP: connection: Remove CS_FL_READ_PARTIAL flag
- REGTESTS: Fix proxy_protocol_tlv_validation
- MINOR: http-ana: Properly set message flags from the start-line flags
- MINOR: h1-htx/http-ana: Set BODYLESS flag on message in TUNNEL state
- MINOR: protocol: add a ->set_port() helper to address families
- MINOR: listener: automatically set the port when creating listeners
- MINOR: listener: now use a generic add_listener() function
- MEDIUM: ssl: fatal error with bundle + openssl < 1.1.1
- BUG/MEDIUM: stream: Xfer the input buffer to a fully created stream
- BUG/MINOR: stream: Don't use input buffer after the ownership xfer
- MINOR: protocol: remove the redundant ->sock_domain field
- MINOR: protocol: export protocol definitions
- CLEANUP: protocol: group protocol struct members by usage
- MINOR: protocol: add a set of ctrl_init/ctrl_close methods for setup/teardown
- MINOR: connection: use the control layer's init/close
- MINOR: udp: export udp_suspend_receiver() and udp_resume_receiver()
- BUG/MAJOR: spoa/python: Fixing return None
- DOC: spoa/python: Fixing typo in IP related error messages
- DOC: spoa/python: Rephrasing memory related error messages
- DOC: spoa/python: Fixing typos in comments
- BUG/MINOR: spoa/python: Cleanup references for failed Module Addobject operations
- BUG/MINOR: spoa/python: Cleanup ipaddress objects if initialization fails
- BUG/MEDIUM: spoa/python: Fixing PyObject_Call positional arguments
- BUG/MEDIUM: spoa/python: Fixing references to None
- DOC: email change of the DeviceAtlas maintainer
- MINOR: cache: Dump secondary entries in "show cache"
- CLEANUP: connection: use fd_stop_both() instead of conn_stop_polling()
- MINOR: stream-int: don't touch polling anymore on shutdown
- MINOR: connection: implement cs_drain_and_close()
- MINOR: mux-pt: take care of CS_SHR_DRAIN in shutr()
- MINOR: checks: use cs_drain_and_close() instead of draining the connection
- MINOR: checks: don't call conn_cond_update_polling() anymore
- CLEANUP: connection: open-code conn_cond_update_polling() and update the comment
- CLEANUP: connection: remove the unused conn_{stop,cond_update}_polling()
- BUG/MINOR: http-check: Use right condition to consider HTX message as full
- BUG/MINOR: tcpcheck: Don't rearm the check timeout on each read
- MINOR: tcpcheck: Only wait for more payload data on HTTP expect rules
- BUG/MINOR: tools: make parse_time_err() more strict on the timer validity
- BUG/MINOR: tools: Reject size format not starting by a digit
- MINOR: action: define enum for timeout type of the set-timeout rule
- MINOR: stream: prepare the hot refresh of timeouts
- MEDIUM: stream: support a dynamic server timeout
- MEDIUM: stream: support a dynamic tunnel timeout
- MEDIUM: http_act: define set-timeout server/tunnel action
- MINOR: frontend: add client timeout sample fetch
- MINOR: backend: add timeout sample fetches
- MINOR: stream: add sample fetches
- MINOR: stream: add timeout sample fetches
- REGTESTS: add regtest for http-request set-timeout
- CLEANUP: remove the unused fd_stop_send() in conn_xprt_shutw{,_hard}()
- CLEANUP: connection: remove the unneeded fd_stop_{recv,send} on read0/shutw
- MINOR: connection: remove sock-specific code from conn_sock_send()
- REORG: connection: move the socket iocb (conn_fd_handler) to sock.c
- MINOR: protocol: add a ->drain() function at the connection control layer
- MINOR: connection: make conn_sock_drain() use the control layer's ->drain()
- MINOR: protocol: add a pair of check_events/ignore_events functions at the ctrl layer
- MEDIUM: connection: make use of the control layer check_events/ignore_events
Add cur_server_timeout and cur_tunnel_timeout.
These sample fetches return the current timeout value for a stream. This
is useful to retrieve the value of a timeout which was changed via a
set-timeout rule.
Add a new http-request action 'set-timeout [server/tunnel]'. This action
can be used to update the server or tunnel timeout of a stream. It takes
two parameters, the timeout name to update and the new timeout value.
This rule is only valid for a proxy with backend capabilities. The
timeout value cannot be null. A sample expression can also be used
instead of a plain value.
Now, some errors are handled by the H1 multiplexer. During the headers
parsing request, there is no stream attached to the H1 mux. Thus, if an
error is reported at this stage, it is handled by the mux itself. If
possible the corresponding frontend errorfile is used, but it should be a
static message. Custom error messages are not supported. Otherwise, default
error messages are used.
In addition, the http analysis has not started yet, so http-after-response
ruleset is not evaluated and cannot alter these early responses.
The goal is to allow execution of one main lua state per thread.
This patch contains the main job. The lua init is done using these
steps:
- "lua-load-per-thread" loads the lua code in the first thread
- it creates the structs
- it stores loaded files
- the 1st step load is completed (execution of hlua_post_init)
and now, we known the number of threads
- we initilize lua states for all remaining threads
- for each one, we load the lua file
- for each one, we execute post-init
Once all is loaded, we control consistency of functions references.
The rules are:
- a function reference cannot be in the shared lua state and in
a per-thread lua state at the same time.
- if a function reference is declared in a per-thread lua state, it
must be declared in all per-thread lua states
This patch adds a new logging variable '%HPO' for logging HTTP path only
(without query string) from relative or absolute URI.
For example:
log-format "hpo=%HPO hp=%HP hu=%HU hq=%HQ"
GET /r/1 HTTP/1.1
=>
hpo=/r/1 hp=/r/1 hu=/r/1 hq=
GET /r/2?q=2 HTTP/1.1
=>
hpo=/r/2 hp=/r/2 hu=/r/2?q=2 hq=?q=2
GET http://host/r/3 HTTP/1.1
=>
hpo=/r/3 hp=http://host/r/3 hu=http://host/r/3 hq=
GET http://host/r/4?q=4 HTTP/1.1
=>
hpo=/r/4 hp=http://host/r/4 hu=http://host/r/4?q=4 hq=?q=4
Released version 2.4-dev2 with the following main changes :
- BUILD: Make DEBUG part of .build_opts
- BUILD: Show the value of DEBUG= in haproxy -vv
- CI: Set DEBUG=-DDEBUG_STRICT=1 in GitHub Actions
- MINOR: stream: Add level 7 retries on http error 401, 403
- CLEANUP: remove unused function "ssl_sock_is_ckch_valid"
- BUILD: SSL: add BoringSSL guarding to "RAND_keep_random_devices_open"
- BUILD: SSL: do not "update" BoringSSL version equivalent anymore
- BUG/MEDIUM: http_act: Restore init of log-format list
- DOC: better describes how to configure a fallback crt
- BUG/MAJOR: filters: Always keep all offsets up to date during data filtering
- MINOR: cache: Prepare helper functions for Vary support
- MEDIUM: cache: Add the Vary header support
- MINOR: cache: Add a process-vary option that can enable/disable Vary processing
- BUG/CRITICAL: cache: Fix trivial crash by sending accept-encoding header
- BUG/MAJOR: peers: fix partial message decoding
- DOC: cache: Add new caching limitation information
- DOC: cache: Add information about Vary support
- DOC: better document the config file format and escaping/quoting rules
- DOC: Clarify %HP description in log-format
- CI: github actions: update LibreSSL to 3.3.0
- CI: github actions: enable 51degrees feature
- MINOR: fd/threads: silence a build warning with threads disabled
- BUG/MINOR: tcpcheck: Don't forget to reset tcp-check flags on new kind of check
- MINOR: tcpcheck: Don't handle anymore in-progress send rules in tcpcheck_main
- BUG/MAJOR: tcpcheck: Allocate input and output buffers from the buffer pool
- MINOR: tcpcheck: Don't handle anymore in-progress connect rules in tcpcheck_main
- MINOR: config: Deprecate and ignore tune.chksize global option
- MINOR: config: Add a warning if tune.chksize is used
- REORG: tcpcheck: Move check option parsing functions based on tcp-check
- MINOR: check: Always increment check health counter on CONPASS
- MINOR: tcpcheck: Add support of L7OKC on expect rules error-status argument
- DOC: config: Make disable-on-404 option clearer on transition conditions
- DOC: config: Move req.hdrs and req.hdrs_bin in L7 samples fetches section
- BUG/MINOR: http-fetch: Fix smp_fetch_body() when called from a health-check
- MINOR: plock: use an ARMv8 instruction barrier for the pause instruction
- MINOR: debug: add "debug dev sched" to stress the scheduler.
- MINOR: debug: add a trivial PRNG for scheduler stress-tests
- BUG/MEDIUM: lists: Lock the element while we check if it is in a list.
- MINOR: task: remove tasklet_insert_into_tasklet_list()
- MINOR: task: perform atomic counter increments only once per wakeup
- MINOR: task: remove __tasklet_remove_from_tasklet_list()
- BUG/MEDIUM: task: close a possible data race condition on a tasklet's list link
- BUG/MEDIUM: local log format regression.
Since 2.3 default local log format always adds hostame field.
This behavior change was due to log/sink re-work, because according
to rfc3164 the hostname field is mandatory.
This patch re-introduce a legacy "local" format which is analog
to rfc3164 but with hostname stripped. This is the new
default if logs are generated by haproxy.
To stay compliant with previous configurations, the option
"log-send-hostname" acts as if the default format is switched
to rfc3164.
This patch addresses the github issue #963
This patch should be backported in branches >= 2.3.
L7OKC may now be used as an error status for an HTTP/TCP expect rule. Thus
it is for instance possible to write:
option httpchk GET /isalive
http-check expect status 200,404
http-check expect status 200 error-status L7OKC
It is more or less the same than the disable-on-404 option except that if a
DOWN is up again but still replying a 404 will be set to NOLB state. While
it will stay in DOWN state with the disable-on-404 option.
This option is now ignored because I/O check buffers are now allocated using the
buffer pool. Thus, it is marked as deprecated in the documentation and ignored
during the configuration parsing. The field is also removed from the global
structure.
Because this option is ignored since a recent fix, backported as fare as 2.2,
this patch should be backported too. Especially because it updates the
documentation.
%HP is used to report HTTP request URI in logs, which might be relative
or absolute. Description in documentation should not suggest that it
behaves exactly the same as "path" sample fetch.
This is even more important after 30ee1efe67
because right now, when HTTP2 is a standard, %HP usually returns absolute
URI.
This might be backported as far as 2.1
It's always a pain to figure how to proceed when special characters need
to be embedded inside arguments of an expression. Let's document the
configuration file format and how unquoting/unescaping works at each
level (top level and argument level) so that everyone hopefully finds
suitable reminders or examples for complex cases.
This is related to github issue #200 and addresses issues #712 and #966.
Responses that do not have an explicit expiration time or a validator
will not be cached anymore.
Must be backported if cc9bf2e ("MEDIUM: cache: Change caching
conditions") is backported.
The cache section's process-vary option takes a 0 or 1 value to disable
or enable the vary processing.
When disabled, a response containing such a header will never be cached.
When enabled, we will calculate a preliminary hash for a subset of request
headers on all the incoming requests (which might come with a cpu cost) which
will be used to build a secondary key for a given request (see RFC 7234#4.1).
The default value is 0 (disabled).
A default certificate is always the first one declared in the bind line,
either from `crt` or from `crt-line` option. This commit updates the
description of how to configure a fallback certificate, clarifying that
it needs to be the first one of the bind line.
Should be merged as far as the first SNI filter implementation.
Level-7 retries are only possible with a restricted number of HTTP
return codes. While it is usually not safe to retry on 401 and 403, I
came up with an authentication backend which was not synchronizing
authentication of users. While not perfect, being allowed to also retry
on those return codes is really helpful and acts as a hotfix until we
can fix the backend.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
Released version 2.4-dev1 with the following main changes :
- MINOR: ist: Add istend() function to return a pointer to the end of the string
- MINOR: sample: Add converters to parse FIX messages
- REGTEST: converter: Add a regtest for fix converters
- MINOR: sample: Add converts to parses MQTT messages
- REGTEST: converter: Add a regtest for MQTT converters
- MINOR: compat: automatically include malloc.h on glibc
- MEDIUM: pools: call malloc_trim() from pool_gc()
- MEDIUM: pattern: call malloc_trim() on pat_ref_reload()
- MINOR: pattern: move the update revision to the pat_ref, not the expression
- CLEANUP: pattern: delete the back refs at once during pat_ref_reload()
- MINOR: pattern: new sflag PAT_SF_REGFREE indicates regex_free() is needed
- MINOR: pattern: make the delete and prune functions more generic
- MEDIUM: pattern: link all final elements from the reference
- MEDIUM: pattern: change the pat_del_* functions to delete from the references
- MINOR: pattern: remerge the list and tree deletion functions
- MINOR: pattern: perform a single call to pat_delete_gen() under the expression
- CLEANUP: acl: don't reference the generic pattern deletion function anymore
- CLEANUP: pattern: remove pat_delete_fcts[] and pattern_head->delete()
- MINOR: pattern: introduce pat_ref_delete_by_ptr() to delete a valid reference
- MINOR: pattern: store a generation number in the reference patterns
- MEDIUM: pattern: only match patterns that match the current generation
- MINOR: pattern: add pat_ref_commit() to commit a previously inserted element
- MINOR: pattern: implement pat_ref_load() to load a pattern at a given generation
- MINOR: pattern: add pat_ref_purge_older() to purge old entries
- MEDIUM: pattern: make pat_ref_prune() rely on pat_ref_purge_older()
- MINOR: pattern: during reload, delete elements frem the ref, not the expression
- MINOR: pattern: prepare removal of a pattern from the list head
- MEDIUM: pattern: turn the pattern chaining to single-linked list
- CLEANUP: cfgparse: remove duplicate registration for transparent build options
- BUG/MINOR: ssl: don't report 1024 bits DH param load error when it's higher
- MINOR: http-htx: Add understandable errors for the errorfiles parsing
- MINOR: ssl: instantiate stats module
- MINOR: ssl: count client hello for stats
- MINOR: ssl: add counters for ssl sessions
- DOC: config: Fix a typo on ssl_c_chain_der
- MINOR: server: remove idle lock in srv_cleanup_connections
- BUILD: ssl: silence build warning on uninitialised counters
- BUILD: http-htx: fix build warning regarding long type in printf
- REGTEST: ssl: test wildcard and multi-type + exclusions
- BUG/MEDIUM: ssl/crt-list: correctly insert crt-list line if crt already loaded
- CI: Expand use of GitHub Actions for CI
- REGTEST: ssl: mark reg-tests/ssl/ssl_crt-list_filters.vtc as broken
- BUG/MINOR: pattern: a sample marked as const could be written
- BUG/MINOR: lua: set buffer size during map lookups
- MEDIUM: cache: Change caching conditions
- BUG/MINOR: stats: free dynamically stats fields/lines on shutdown
- BUG/MEDIUM: stats: prevent crash if counters not alloc with dummy one
- MINOR: peers: Add traces to peer_treat_updatemsg().
- BUG/MINOR: peers: Do not ignore a protocol error for dictionary entries.
- BUG/MINOR: peers: Missing TX cache entries reset.
- BUG/MEDIUM: peers: fix decoding of multi-byte length in stick-table messages
- BUG/MINOR: http-fetch: Extract cookie value even when no cookie name
- BUG/MINOR: http-fetch: Fix calls w/o parentheses of the cookie sample fetches
- BUG/MEDIUM: check: reuse srv proto only if using same mode
- MINOR: check: report error on incompatible proto
- MINOR: check: report error on incompatible connect proto
- BUG/MINOR: http-htx: Handle warnings when parsing http-error and http-errors
- BUG/MAJOR: spoe: Be sure to remove all references on a released spoe applet
- MINOR: spoe: Don't close connection in sync mode on processing timeout
- BUG/MINOR: tcpcheck: Don't warn on unused rules if check option is after
- MINOR: init: Fix the prototype for per-thread free callbacks
- MINOR: config/mux-h2: Return ERR_ flags from init_h2() instead of a status
- CLEANUP: config: Return ERR_NONE from config callbacks instead of 0
- MINOR: cfgparse: tighten the scope of newnameserver variable, free it on error.
- REGTEST: make ssl_client_samples and ssl_server_samples require to 2.2
- REGTESTS: Add sample_fetches/cook.vtc
- BUG/MEDIUM: filters: Forward all filtered data at the end of http filtering
- BUG/MINOR: http-ana: Don't wait for the body of CONNECT requests
- CLEANUP: flt-trace: Remove unused random-parsing option
- MINOR: flt-trace: Add an option to inhibits trace messages
- MINOR: flt-trace: Use a bitfield for the trace options
- REGTESTS: Add a script to test the random forwarding with several filters
- REGTESTS: mark the abns test as broken again
- REGTESTS: converter: add url_dec test
- CI: Stop hijacking the hosts file
- CI: Make the h2spec workflow more consistent with the VTest workflow
- CI: travis-ci: remove amd64, osx builds
- CI: travis-ci: arm64 are not allowed to fail anymore
- DOC: add missing 3.10 in the summary
- MINOR: ssl: remove client hello counters
- MEDIUM: stats: add counters for failed handshake
- MINOR: ssl: create common ssl_ctx init
- MEDIUM: cli/ssl: configure ssl on server at runtime
- REGTEST: server/cli_set_ssl.vtc requires OpenSSL
- DOC: coding-style: update a few rules about pointers
- BUG/MINOR: ssl: segv on startup when AKID but no keyid
- BUILD: ssl: use SSL_MODE_ASYNC macro instead of OPENSSL_VERSION
- BUG/MEDIUM: http-ana: Don't eval http-after-response ruleset on empty messages
- BUG/MEDIUM: ssl/crt-list: bundle support broken in crt-list
- BUG/MEDIUM: ssl: error when no certificate are found
- BUG/MINOR: ssl/crt-list: load bundle in crt-list only if activated
- BUG/MEDIUM: ssl/crt-list: fix error when no file found
- CI: Github Actions: enable prometheus exporter
- CI: Github Actions: remove LibreSSL-3.0.2 builds
- CI: Github Actions: enable BoringSSL builds
- CI: travis-ci: remove builds migrated to GH actions
- BUILD: makefile: enable crypt(3) for OpenBSD
- CI: Github Action: run "apt-get update" before packages restore
- BUILD: SSL: guard TLS13 ciphersuites with HAVE_SSL_CTX_SET_CIPHERSUITES
- CI: Pass the github.event_name to matrix.py
- CI: Clean up Windows CI
- DOC: clarify how to create a fallback crt
- CLEANUP: connection: do not use conn->owner when the session is known
- BUG/MAJOR: connection: reset conn->owner when detaching from session list
- REGTESTS: mark proxy_protocol_random_fail as broken
- BUG/MINOR: http_htx: Fix searching headers by substring
- MINOR: http_act: Add -m flag for del-header name matching method
This patch adds -m flag which allows to specify header name
matching method when deleting headers from http request/response.
Currently beg, end, sub, str and reg are supported.
This is related to GitHub issue #909
HAProxy uses CN and SAN of the certificates to match incoming SNI, and
use the matching certificate in the TLS handshake. `crt-list` goes
further and allows to configure SNI filters to explicitly define the
FQDNs that should match a certificate.
The first declared certificate of the `crt-list` option follows the same
rules, and it's also used as a fallback - the certificate that should be
used if SNI isn't provided or the provided one cannot match any
certificate or SNI filter. If a provided SNI matches the CN or SAN of
the first certificate, the first certificate would be used even if a
matching SNI filter is declared later.
This change clarifies this scenario and documents a filter that can be
used to convert the first declared certificate as a proper fallback.
Should be merged as far as the first SNI filter implementation.
in the context of a progressive backend migration, we want to be able to
activate SSL on outgoing connections to the server at runtime without
reloading.
This patch adds a `set server ssl` command; in order to allow that:
- add `srv_use_ssl` to `show servers state` command for compatibility,
also update associated parsing
- when using default-server ssl setting, and `no-ssl` on server line,
init SSL ctx without activating it
- when triggering ssl API, de/activate SSL connections as requested
- clean ongoing connections as it is done for addr/port changes, without
checking prior server state
example config:
backend be_foo
default-server ssl
server srv0 127.0.0.1:6011 weight 1 no-ssl
show servers state:
5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - -1
where srv0 can switch to ssl later during the runtime:
set server be_foo/srv0 ssl on
5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - 1
Also update existing tests and create a new one.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
There is a typo on the ssl_c_chain_der sample fetch
(s/ssl_c_der_chain/ssl_c_chain_der/). This implies a move of the fetch to keep
it at the right place.
This should be backported as far as 2.2 or anywhere the commit a598b500b
("MINOR: ssl: add ssl_{c,s}_chain_der fetch methods") is.
This patch implements a couple of converters to validate and extract data from a
MQTT (Message Queuing Telemetry Transport) message. The validation consists of a
few checks as well as "packet size" validation. The extraction can get any field
from the variable header and the payload.
This is limited to CONNECT and CONNACK packet types only. All other messages are
considered as invalid. It is not a problem for now because only the first packet
on each side can be parsed (CONNECT for the client and CONNACK for the server).
MQTT 3.1.1 and 5.0 are supported.
Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
This patch implements a couple of converters to validate and extract tag value
from a FIX (Financial Information eXchange) message. The validation consists in
a few checks such as mandatory fields and checksum computation. The extraction
can get any tag value based on a tag string or tag id.
This patch requires the istend() function. Thus it depends on "MINOR: ist: Add
istend() function to return a pointer to the end of the string".
Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
Released version 2.4-dev0 with the following main changes :
- MINOR: version: it's development again.
- DOC: mention in INSTALL that it's development again
Released version 2.3.0 with the following main changes :
- CLEANUP: pattern: remove unused entry "tree" in pattern.val
- BUILD: ssl: use SSL_CTRL_GET_RAW_CIPHERLIST instead of OpenSSL versions
- BUG/MEDIUM: filters: Don't try to init filters for disabled proxies
- BUG/MINOR: proxy/server: Skip per-proxy/server post-check for disabled proxies
- BUG/MINOR: checks: Report a socket error before any connection attempt
- BUG/MINOR: server: Set server without addr but with dns in RMAINT on startup
- MINOR: server: Copy configuration file and line for server templates
- BUG/MEDIUM: mux-pt: Release the tasklet during an HTTP upgrade
- BUILD: ssl: use HAVE_OPENSSL_KEYLOG instead of OpenSSL versions
- MINOR: debug: don't count free(NULL) in memstats
- BUG/MINOR: filters: Skip disabled proxies during startup only
- MINOR: mux_h2: capitalize frame type in stats
- MINOR: mux_h2: add stat for total count of connections/streams
- MINOR: stats: do not display empty stat module title on html
- BUG/MEDIUM: stick-table: limit the time spent purging old entries
- BUG/MEDIUM: listener: only enable a listening listener if needed
- BUG/MEDIUM: listener: never suspend inherited sockets
- BUG/MEDIUM: listener: make the master also keep workers' inherited FDs
- MINOR: fd: add fd_want_recv_safe()
- MEDIUM: listeners: make use of fd_want_recv_safe() to enable early receivers
- REGTESTS: mark abns_socket as working now
- CLEANUP: mux-h2: Remove the h1 parser state from the h2 stream
- MINOR: sock: add a check against cross worker<->master socket activities
- CI: github actions: limit OpenSSL no-deprecated builds to "default,bug,devel" reg-tests
- BUG/MEDIUM: server: make it possible to kill last idle connections
- MINOR: mworker/cli: the master CLI use its own applet
- MINOR: ssl: define SSL_CTX_set1_curves_list to itself on BoringSSL
- BUILD: ssl: use feature macros for detecting ec curves manipulation support
- DOC: Add dns as an available domain to show stat
- BUILD: makefile: usual reorder of objects for faster builds
- DOC: update INSTALL to mention that TCC is supported
- DOC: mention in INSTALL that haproxy 2.3 is a stable version
- MINOR: version: mention that it's stable now
Released version 2.3-dev9 with the following main changes :
- CLEANUP: http_ana: remove unused assignation of `att_beg`
- BUG/MEDIUM: ssl: OCSP must work with BoringSSL
- BUG/MINOR: log: fix memory leak on logsrv parse error
- BUG/MINOR: log: fix risk of null deref on error path
- BUILD: ssl: more elegant OpenSSL early data support check
- CI: github actions: update h2spec to 2.6.0
- BUG/MINOR: cache: Check the return value of http_replace_res_status
- MINOR: cache: Store the "Last-Modified" date in the cache_entry
- MINOR: cache: Process the If-Modified-Since header in conditional requests
- MINOR: cache: Create res.cache_hit and res.cache_name sample fetches
- MINOR: mux-h2: register a stats module
- MINOR: mux-h2: add counters instance to h2c
- MINOR: mux-h2: add stats for received frame types
- MINOR: mux-h2: report detected error on stats
- MINOR: mux-h2: count open connections/streams on stats
- BUG/MINOR: server: fix srv downtime calcul on starting
- BUG/MINOR: server: fix down_time report for stats
- BUG/MINOR: lua: initialize sample before using it
- MINOR: cache: Add Expires header value parsing
- MINOR: ist: Add a case insensitive istmatch function
- BUG/MINOR: cache: Manage multiple values in cache-control header value
- BUG/MINOR: cache: Inverted variables in http_calc_maxage function
- MINOR: pattern: make pat_ref_append() return the newly added element
- MINOR: pattern: make pat_ref_add() rely on pat_ref_append()
- MINOR: pattern: export pat_ref_push()
- CLEANUP: pattern: use calloc() rather than malloc for structures
- CLEANUP: pattern: fix spelling/grammatical/copy-paste in comments
Res.cache_hit sample fetch returns a boolean which is true when the HTTP
response was built out of a cache. The cache's name is returned by the
res.cache_name sample_fetch.
This resolves GitHub issue #900.
Released version 2.3-dev8 with the following main changes :
- MINOR: backend: replace the lbprm lock with an rwlock
- MINOR: lb/map: use seek lock and read locks where appropriate
- MINOR: lb/leastconn: only take a read lock in fwlc_get_next_server()
- MINOR: lb/first: use a read lock in fas_get_next_server()
- MINOR: lb/chash: use a read lock in chash_get_server_hash()
- BUG/MINOR: disable dynamic OCSP load with BoringSSL
- BUILD: ssl: make BoringSSL use its own version numbers
- CLEANUP: threads: don't register an initcall when not debugging
- MINOR: threads: change lock_t to an unsigned int
- CLEANUP: tree-wide: reorder a few structures to plug some holes around locks
- CLEANUP: task: remove the unused and mishandled global_rqueue_size
- BUG/MEDIUM: connection: Never cleanup server lists when freeing private conns
- MEDIUM: config: report that "nbproc" is deprecated
- BUG/MINOR: listener: close before free in `listener_accept`
- MINOR: ssl: 'ssl-load-extra-del-ext' removes the certificate extension
- BUG/MINOR: queue: properly report redistributed connections
- CONTRIB: tcploop: remove unused local variables in tcp_pause()
- BUILD: makefile: add entries to build common debugging tools
- BUG/MEDIUM: server: support changing the slowstart value from state-file
- MINOR: http: Add `enum etag_type http_get_etag_type(const struct ist)`
- MINOR: http: Add etag comparison function
- MEDIUM: cache: Store the ETag information in the cache_entry
- MEDIUM: cache: Add support for 'If-None-Match' request header
- REGTEST: cache: Add if-none-match test case
- CLEANUP: compression: Make use of http_get_etag_type()
- BUG/MINOR: http-ana: Don't send payload for internal responses to HEAD requests
- BUG/MAJOR: mux-h2: Don't try to send data if we know it is no longer possible
- MINOR: threads/debug: only report used lock stats
- MINOR: threads/debug: only report lock stats for used operations
- MINOR: proxy; replace the spinlock with an rwlock
- MINOR: server: read-lock the cookie during srv_set_dyncookie()
- MINOR: proxy/cli: only take a read lock in "show errors"
- OPTIM: queue: don't call pendconn_unlink() when the pendconn is not queued
- MINOR: queue: split __pendconn_unlink() in per-srv and per-prx
- MINOR: queue: reduce the locked area in pendconn_add()
- OPTIM: queue: make the nbpend counters atomic
- OPTIM: queue: decrement the nbpend and totpend counters outside of the lock
- MINOR: leastconn: take the queue length into account when queuing servers
- MEDIUM: fwlc: re-enable per-server queuing up to maxqueue
- Revert "OPTIM: queue: don't call pendconn_unlink() when the pendconn is not queued"
- MINOR: stats: support the "up" output modifier for "show stat"
- MINOR: stats: also support a "no-maint" show stat modifier
- MINOR: stats: indicate the number of servers in a backend's status
- MEDIUM: ssl: ssl-load-extra-del-ext work only with .crt
- REGTEST: ssl: test "set ssl cert" with separate key / crt
- DOC: management: apply the "show stat" modifiers to "show stat", not "show info"
- MINOR: stats: report server's user-configured weight next to effective weight
- CI: travis-ci: switch to Ubuntu 20.04
- CONTRIB: release-estimator: Add release estimating tool
- BUG/MEDIUM: queue: fix unsafe proxy pointer when counting nbpend
- BUG/MINOR: extcheck: add missing checks on extchk_setenv()
In order to be compatible with the "set ssl cert" command of the CLI,
this patch restrict the ssl-load-extra-del-ext to files with a ".crt"
extension in the configuration.
Related to issue #785.
Should be backported where 8e8581e ("MINOR: ssl: 'ssl-load-extra-del-ext'
removes the certificate extension") was backported.
Leastconn has the nice propery of being able to sort servers by their
current usage. It's really a shame to force all requests into the backend
queue when the algo would be able to also consider their current queue.
In order not to change existing behavior but extend it, this patch allows
leastconn to elect servers which are already full if they have an explicitly
configured maxqueue setting above zero and their queue hasn't reached that
threshold. This will significantly reduce the pressure in the backend queue
when queuing a lot with lots of servers.
A test on 8 threads with 100 servers configured with maxconn 1 jumped
from 165krps to 330krps with maxqueue 15 with this patch.
This partially undoes commit 82cd5c13a ("OPTIM: backend: skip LB when we
know the backend is full") but allows to scale much better even by setting
a single-digit maxqueue value. Some better heuristics could be used to
maintain the behavior of the bypass in the patch above, consisting in
keeping it if it's known that there is no server with a configured
maxqueue in the farm (or in the backend).
When servers are queued into the leastconn tree, it's important to also
consider their queue length. There could be some servers with lots of
queued requests that we don't want to hammer with extra connections. In
order not to add extra stress to the LB algorithm, we don't update the
value when adding to the queue, only when updating the connection count
(i.e. picking from the queue or releasing a connection). This will be
sufficient to significantly improve the fairness in such situations.
In issue #785, users are reporting that it's not convenient to load a
".crt.key" when the configuration contains a ".crt".
This option allows to remove the extension of the certificate before
trying to load any extra SSL file (.key, .ocsp, .sctl, .issuer etc.)
The patch changes a little bit the way ssl_sock_load_files_into_ckch()
looks for the file.
As previously discussed, nbproc usage is bad, deprecated, and scheduled
for removal in 2.5.
If "nbproc" is found with more than one process while nbthread is not
set, a warning will be emitted encouraging to remove it or to migrate
to nbthread instead. This makes sure the user has an opportunity to
both see the message and silence it.
Released version 2.3-dev7 with the following main changes :
- CI: travis-ci: replace not defined SSL_LIB, SSL_INC for BotringSSL builds
- BUG/MINOR: init: only keep rlim_fd_cur if max is unlimited
- BUG/MINOR: mux-h2: do not stop outgoing connections on stopping
- MINOR: fd: report an error message when failing initial allocations
- MINOR: proto-tcp: make use of connect(AF_UNSPEC) for the pause
- MINOR: sock: add sock_accept_conn() to test a listening socket
- MINOR: protocol: make proto_tcp & proto_uxst report listening sockets
- MINOR: sockpair: implement the .rx_listening function
- CLEANUP: tcp: make use of sock_accept_conn() where relevant
- CLEANUP: unix: make use of sock_accept_conn() where relevant
- BUG/MINOR: listener: detect and handle shared sockets stopped in other processes
- CONTRIB: tcploop: implement a disconnect operation 'D'
- CLEANUP: protocol: intitialize all of the sockaddr when disconnecting
- BUG/MEDIUM: deinit: check fdtab before fdtab[fd].owner
- BUG/MINOR: connection: fix loop iter on connection takeover
- BUG/MEDIUM: connection: fix srv idle count on conn takeover
- MINOR: connection: improve list api usage
- MINOR: mux/connection: add a new mux flag for HOL risk
- MINOR: connection: don't check priv flag on free
- MEDIUM: backend: add new conn to session if mux marked as HOL blocking
- MEDIUM: backend: add reused conn to sess if mux marked as HOL blocking
- MEDIUM: h2: remove conn from session on detach
- MEDIUM: fcgi: remove conn from session on detach
- DOC: Describe reuse safe for HOL handling
- MEDIUM: proxy: remove obsolete "mode health"
- MEDIUM: proxy: remove obsolete "monitor-net"
- CLEANUP: protocol: remove the ->drain() function
- CLEANUP: fd: finally get rid of fd_done_recv()
- MINOR: connection: make sockaddr_alloc() take the address to be copied
- MEDIUM: listener: allocate the connection before queuing a new connection
- MINOR: session: simplify error path in session_accept_fd()
- MINOR: connection: add new error codes for accept_conn()
- MINOR: sock: rename sock_accept_conn() to sock_accepting_conn()
- MINOR: protocol: add a new function accept_conn()
- MINOR: sock: implement sock_accept_conn() to accept a connection
- MINOR: sockpair: implement sockpair_accept_conn() to accept a connection
- MEDIUM: listener: use protocol->accept_conn() to accept a connection
- MEDIUM: listener: remove the second pass of fd manipulation at the end
- MINOR: protocol: add a default I/O callback and put it into the receiver
- MINOR: log: set the UDP receiver's I/O handler in the receiver
- MINOR: protocol: register the receiver's I/O handler and not the protocol's
- CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind()
- DOC: improve the documentation for "option nolinger"
- BUG/MEDIUM: proxy: properly stop backends
- BUG/MEDIUM: task: bound the number of tasks picked from the wait queue at once
- MINOR: threads: augment rwlock debugging stats to report seek lock stats
- MINOR: threads: add the transitions to/from the seek state
- MEDIUM: task: use an upgradable seek lock when scanning the wait queue
- BUILD: listener: avoir a build warning when threads are disabled
- BUG/MINOR: peers: Possible unexpected peer seesion reset after collisions.
- MINOR: ssl: add volatile flags to ssl samples
- MEDIUM: backend: reuse connection if using a static sni
- BUG/MEDIUM: spoe: Unset variable instead of set it if no data provided
- BUG/MEDIUM: mux-h1: Get the session from the H1S when capturing bad messages
- BUG/MEDIUM: lb: Always lock the server when calling server_{take,drop}_conn
- DOC: fix typo in MAX_SESS_STKCTR
Detect if the sni used a constant value and if so, allow to reuse this
connection for later sessions. Use a combination of SMP_USE_INTRN +
!SMP_F_VOLATILE to consider a sample as a constant value.
This features has been requested on github issue #371.
As discussed here during 2.1-dev, "monitor-net" is totally obsolete:
https://www.mail-archive.com/haproxy@formilux.org/msg35204.html
It's fundamentally incompatible with usage of SSL, and imposes the
presence of file descriptors with hard-coded syscalls directly in the
generic accept path.
It's very unlikely that anyone has used it in the last 10 years for
anything beyond testing. In the worst case if anyone would depend
on it, replacing it with "http-request return status 200 if ..." and
"mode http" would certainly do the trick.
The keyword is still detected as special by the config parser to help
users update their configurations appropriately.
As discussed here during 2.1-dev, "mode health" is totally obsolete:
https://www.mail-archive.com/haproxy@formilux.org/msg35204.html
It's fundamentally incompatible with usage of SSL, doesn't support
source filtering, and imposes the presence of file descriptors with
hard-coded syscalls directly in the generic accept path.
It's very unlikely that anyone has used it in the last 10 years for
anything beyond testing. In the worst case if anyone would depend
on it, replacing it with "http-request return status 200" and "mode
http" would certainly do the trick.
The keyword is still detected as special by the config parser to help
users update their configurations appropriately.
Released version 2.3-dev6 with the following main changes :
- REGTESTS: use "command" instead of "which" for better POSIX compatibility
- BUILD: makefile: Update feature flags for OpenBSD
- DOC: agent-check: fix typo in "fail" word expected reply
- DOC: crt: advise to move away from cert bundle
- BUG/MINOR: ssl/crt-list: exit on warning out of crtlist_parse_line()
- REGTEST: fix host part in balance-uri-path-only.vtc
- REGTEST: make ssl_client_samples and ssl_server_samples requiret to 2.3
- REGTEST: the iif converter test requires 2.3
- REGTEST: make agent-check.vtc require 1.8
- REGTEST: make abns_socket.vtc require 1.8
- REGTEST: make map_regm_with_backref require 1.7
- BUILD: makefile: Update feature flags for FreeBSD
- OPTIM: backend/random: never queue on the server, always on the backend
- OPTIM: backend: skip LB when we know the backend is full
- BUILD: makefile: Fix building with closefrom() support enabled
- BUILD: makefile: add an EXTRAVERSION variable to ease local naming
- MINOR: tools: support for word expansion of environment in parse_line
- BUILD: tools: fix minor build issue on isspace()
- BUILD: makefile: Enable closefrom() support on Solaris
- CLEANUP: ssl: Use structured format for error line report during crt-list parsing
- MINOR: ssl: Add error if a crt-list might be truncated
- MINOR: ssl: remove uneeded check in crtlist_parse_file
- BUG/MINOR: Fix several leaks of 'log_tag' in init().
- DOC: tcp-rules: Refresh details about L7 matching for tcp-request content rules
- MEDIUM: tcp-rules: Warn if a track-sc* content rule doesn't depend on content
- BUG/MINOR: tcpcheck: Set socks4 and send-proxy flags before the connect call
- DOC: ssl: new "cert bundle" behavior
- BUG/MEDIUM: queue: make pendconn_cond_unlink() really thread-safe
- CLEANUP: ssl: "bundle" is not an OpenSSL wording
- MINOR: counters: fix a typo in comment
- BUG/MINOR: stats: fix validity of the json schema
- REORG: stats: export some functions
- MINOR: stats: add stats size as a parameter for csv/json dump
- MINOR: stats: hide px/sv/li fields in applet struct
- REORG: stats: extract proxy json dump
- REORG: stats: extract proxies dump loop in a function
- MINOR: hlua: Display debug messages on stderr only in debug mode
- MINOR: stats: define the concept of domain for statistics
- MINOR: stats: define additional flag px cap on domain
- MEDIUM: stats: add delimiter for static proxy stats on csv
- MEDIUM: stats: define an API to register stat modules
- MEDIUM: stats: add abstract type to store counters
- MEDIUM: stats: integrate static proxies stats in new stats
- MINOR: stats: support clear counters for dynamic stats
- MINOR: stats: display extra proxy stats on the html page
- MINOR: stats: add config "stats show modules"
- MINOR: dns/stats: integrate dns counters in stats
- MINOR: stats: remove for loop declaration
- DOC: ssl: fix typo about ocsp files
- BUG/MINOR: peers: Inconsistency when dumping peer status codes.
- DOC: update INSTALL with supported OpenBSD / FreeBSD versions
- BUG/MINOR: proto_tcp: Report warning messages when listeners are bound
- CLEANUP: cache: Fix leak of cconf->c.name during config check
- CLEANUP: ssl: Release cached SSL sessions on deinit
- BUG/MINOR: mux-h1: Be sure to only set CO_RFL_READ_ONCE for the first read
- BUG/MINOR: mux-h1: Always set the session on frontend h1 stream
- MINOR: mux-h1: Don't wakeup the H1C when output buffer become available
- CLEANUP: sock-unix: Remove an unreachable goto clause
- BUG/MINOR: proxy: inc req counter on new syslog messages.
- BUG/MEDIUM: log: old processes with log foward section don't die on soft stop.
- MINOR: stats: inc req counter on listeners.
- MINOR: channel: new getword and getchar functions on channel.
- MEDIUM: log: syslog TCP support on log forward section.
- BUG/MINOR: proxy/log: frontend/backend and log forward names must differ
- DOC: re-work log forward bind statement documentation.
- DOC: fix a confusing typo on a regsub example
- BUILD: Add a DragonFlyBSD target
- BUG/MINOR: makefile: fix a tiny typo in the target list
- BUILD: makefile: Update feature flags for NetBSD
- CI: travis-ci: help Coverity to detect BUG_ON() as a real stop
- DOC: Add missing stats fields in the management doc
- BUG/MEDIUM: mux-fcgi: Don't handle pending read0 too early on streams
- BUG/MEDIUM: mux-h2: Don't handle pending read0 too early on streams
- DOC: Fix typos in configuration.txt
- BUG/MINOR: http: Fix content-length of the default 500 error
- BUG/MINOR: http-htx: Expect no body for 204/304 internal HTTP responses
- REGTESTS: mark abns_socket as broken
- MEDIUM: fd: always wake up one thread when enabling a foreing FD
- MEDIUM: listeners: don't bounce listeners management between queues
- MEDIUM: init: stop disabled proxies after initializing fdtab
- MEDIUM: listeners: make unbind_listener() converge if needed
- MEDIUM: deinit: close all receivers/listeners before scanning proxies
- MEDIUM: listeners: remove the now unused ZOMBIE state
- MINOR: listeners: do not uselessly try to close zombie listeners in soft_stop()
- CLEANUP: proxy: remove the first_to_listen hack in zombify_proxy()
- MINOR: listeners: introduce listener_set_state()
- MINOR: proxy: maintain per-state counters of listeners
- MEDIUM: proxy: remove the unused PR_STFULL state
- MEDIUM: proxy: remove the PR_STERROR state
- MEDIUM: proxy: remove state PR_STPAUSED
- MINOR: startup: don't rely on PR_STNEW to check for listeners
- CLEANUP: peers: don't use the PR_ST* states to mark enabled/disabled
- MEDIUM: proxy: replace proxy->state with proxy->disabled
- MEDIUM: proxy: remove start_proxies()
- MEDIUM: proxy: merge zombify_proxy() with stop_proxy()
- MINOR: listeners: check the current listener state in pause_listener()
- MINOR: listeners: check the current listener earlier state in resume_listener()
- MEDIUM: listener/proxy: make the listeners notify about proxy pause/resume
- MINOR: protocol: introduce protocol_{pause,resume}_all()
- MAJOR: signals: use protocol_pause_all() and protocol_resume_all()
- CLEANUP: proxy: remove the now unused pause_proxies() and resume_proxies()
- MEDIUM: proto_tcp: make the pause() more robust in multi-process
- BUG/MEDIUM: listeners: correctly report pause() errors
- MINOR: listeners: move fd_stop_recv() to the receiver's socket code
- CLEANUP: protocol: remove the ->disable_all method
- CLEANUP: listeners: remove unused disable_listener and disable_all_listeners
- MINOR: listeners: export enable_listener()
- MINOR: protocol: directly call enable_listener() from protocol_enable_all()
- CLEANUP: protocol: remove the ->enable_all method
- CLEANUP: listeners: remove the now unused enable_all_listeners()
- MINOR: protocol: rename the ->listeners field to ->receivers
- MINOR: protocol: replace ->pause(listener) with ->rx_suspend(receiver)
- MINOR: protocol: implement an ->rx_resume() method
- MINOR: listener: use the protocol's ->rx_resume() method when available
- MINOR: sock: provide a set of generic enable/disable functions
- MINOR: protocol: add a new pair of rx_enable/rx_disable methods
- MINOR: protocol: add a new pair of enable/disable methods for listeners
- MEDIUM: listeners: now use the listener's ->enable/disable
- MINOR: listeners: split delete_listener() in two versions
- MINOR: listeners: count unstoppable jobs on creation, not deletion
- MINOR: listeners: add a new stop_listener() function
- MEDIUM: proxy: make stop_proxy() now use stop_listener()
- MEDIUM: proxy: add mode PR_MODE_PEERS to flag peers frontends
- MEDIUM: proxy: centralize proxy status update and reporting
- MINOR: protocol: add protocol_stop_now() to instant-stop listeners
- MEDIUM: proxy: make soft_stop() stop most listeners using protocol_stop_now()
- MEDIUM: udp: implement udp_suspend() and udp_resume()
- MINOR: listener: add a few BUG_ON() statements to detect inconsistencies
- MEDIUM: listeners: always close master vs worker listeners
- BROKEN/MEDIUM: listeners: rework the unbind logic to make it idempotent
- MEDIUM: listener: let do_unbind_listener() decide whether to close or not
- CLEANUP: listeners: remove the do_close argument to unbind_listener()
- MINOR: listeners: move the LI_O_MWORKER flag to the receiver
- MEDIUM: receivers: add an rx_unbind() method in the protocols
- MINOR: listeners: split do_unbind_listener() in two
- MEDIUM: listeners: implement protocol level ->suspend/resume() calls
- MEDIUM: config: mark "grace" as deprecated
- MEDIUM: config: remove the deprecated and dangerous global "debug" directive
- BUG/MINOR: proxy: respect the proper format string in sig_pause/sig_listen
- MINOR: peers: heartbeat, collisions and handshake information for "show peers" command.
- BUILD: makefile: Enable getaddrinfo() on OS/X
This one was scheduled for removal in 2.3 since 2.2-dev3 by commit
1b85785bc ("MINOR: config: mark global.debug as deprecated"). Let's
remove it now. It remains totally possible to use -d on the command
line though.
This was introduced 15 years ago or so to delay the stopping of some
services so that a monitoring device could detect its port being down
before services were stopped. Since then, clean reloads were implemented
and this doesn't cope well with reload at all, preventing the new process
from seamlessly binding, and forcing processes to coexist with half-baked
configurations.
Now it has become a real problem because there's a significant code
portion in the proxies that is solely dedicated to this obsolete feature,
and dealing with its special cases eases the introduction of bugs in
other places so it's about time that it goes.
We could tentatively schedule its removal for 2.4 with a hard deadline
for 2.5 in any case.
Sébastien reported a confusing example in the doc about regsub when used
with quotes. Nested quotes are already not trivial to grasp, but when
typos are there and result in something valid, it's even worse. The closing
quote ought to have been inside the brackets. However haproxy will not make
any difference because the single quotes delimit a word and the delimited
word remains the same. Let's just not add yet another level of confusion.
This patch re-introduce the "bind" statement on log forward
sections to handle syslog TCP listeners as defined in
rfc-6587.
As complement it introduce "maxconn", "backlog" and "timeout
client" statements to parameter those listeners.
one of the last I saw in this section while working on github issue #872
might be backported in all still supported versions
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
Update the documentation with the new bundle behavior which does not use
the same OpenSSL certificate store anymore but loads the PEM separately
as multiple "crt" were specified.
It should fix issue #872.
The warning is only emitted for HTTP frontend. Idea is to encourage the usage of
"tcp-request session" rules to track counters that does not depend on the
request content. The documentation has been updated accordingly.
The warning is important because since the multiplexers were added in the
processing chain, the HTTP parsing is performed at a lower level. Thus parsing
errors are detected in the multiplexers, before the stream creation. In HTTP/2,
the error is reported by the multiplexer itself and the stream is never
created. This difference has a certain number of consequences, one of which is
that HTTP request counting in stick tables only works for valid H2 request, and
HTTP error tracking in stick tables never considers invalid H2 requests but only
invalid H1 ones. And the aim is to do the same with the mux-h1. This change will
not be done for the 2.3, but the 2.4. At the end, H1 and H2 parsing errors will
be caught by the multiplexers, at the session level. Thus, tracking counters at
the content level should be reserved for rules using a key based on the request
content or those using ACLs based on the request content.
To be clear, a warning will be emitted for the following rules :
tcp-request content track-sc0 src
tcp-request content track-sc0 src if ! { src 10.0.0.0/24 }
tcp-request content track-sc0 src if { ssl_fc }
But not for the following ones :
tcp-request content track-sc0 req.hdr(host)
tcp-request content track-sc0 src if { req.hdr(host) -m found }
Because the parsing of HTTP message is now performed in the HTTP multiplexers,
the content is immediatly available when "tcp-request content" rules are
evaluated for an HTTP frontend. So, it is a good idea to make the documentation
explicit on this point. In addition, because in all cases, the parsing is
already performed, there is no reason to still use "tcp-request content" rules
based on L7 matching, although it is still valid. The recommended way is to use
"http-request" rules instead. Again, it is a good idea to update the
documentation on this point.
Allow the syntax "${...[*]}" to expand an environment variable
containing several values separated by spaces as individual arguments. A
new flag PARSE_OPT_WORD_EXPAND has been added to toggle this feature on
parse_line invocation. In case of an invalid syntax, a new error
PARSE_ERR_WRONG_EXPAND will be triggered.
This feature has been asked on the github issue #165.
