The detail level initially based on syslog levels is not used, while
something related is missing, trace verbosity, to indicate whether or
not we want to call the decoding callback and what level of decoding
we want (raw captures etc). Let's change the field to "verbosity" for
this. A verbosity of zero means that the decoding callback is not
called, and all other levels are handled by this callback and are
source-specific. The source is now prompted to list the levels that
are proposed to the user. When the source doesn't define anything,
"quiet" and "default" are available.
Working on adding traces to mux-h2 revealed that the function names are
manually copied a lot in developer traces. The reason is that they are
not preprocessor macros and as such cannot be concatenated. Let's
slightly adjust the trace() function call to take a function name just
after the file:line argument. This argument is only added for the
TRACE_DEVEL and 3 new TRACE_ENTER, TRACE_LEAVE, and TRACE_POINT macros
and left NULL for others. This way the function name is only reported
for traces aimed at the developers. The pretty-print callback was also
extended to benefit from this. This will also significantly shrink the
data segment as the "entering" and "leaving" strings will now be merged.
One technical point worth mentioning is that the function name is *not*
passed as an ist to the inline function because it's not considered as
a builtin constant by the compiler, and would lead to strlen() being
run on it from all call places before calling the inline function. Thus
instead we pass the const char * (that the compiler knows where to find)
and it's the __trace() function that converts it to an ist for internal
consumption and for the pretty-print callback. Doing this avoids losing
5-10% peak performance.
The "payload" trace level was ambigous because its initial purpose was
to be able to dump received data. But it doesn't make sense to force to
report data transfers just to be able to report state changes. For
example, all snd_buf()/rcv_buf() operations coming from the application
layer should be tagged at this level. So here we move this payload level
above the state transitions and rename it to avoid the ambiguity making
one think it's only about request/response payload. Now it clearly is
about any data transfer and is thus just below the developer level. The
help messages on the CLI and the doc were slightly reworded to help
remove this ambiguity.
Save the authority TLV in a PROXYv2 header from the client connection,
if present, and make it available as fc_pp_authority.
The fetch can be used, for example, to set the SNI for a backend TLS
connection.
The new "show events" CLI keyword lists supported event sinks. When
passed a buffer-type sink it completely dumps it.
no drops at all during attachment even at 8 millon evts/s.
still missing the attachment limit though.
The converter can be useful to look up a server queue from a dynamic value.
It takes an input value of type string, either a server name or
<backend>/<server> format and returns the number of queued sessions
on that server. Can be used in places where we want to look up
queued sessions from a dynamic name, like a cookie value (e.g.
req.cook(SRVID),srv_queue) and then make a decision to break
persistence or direct a request elsewhere.
Signed-off-by: Nenad Merdanovic <nmerdan@haproxy.com>
At the moment the subsystem is still not complete and the various modules
do not yet produce traces (some dirty experimental code for H2 exists) but
this aims at easing a broad adoption.
Among the missing elements, we can enumerate the lack of configuration
of the sinks (e.g. it's still not possible to change their output format
nor enable/disable timestamps) and since timestamps are not availalbe in
the sinks, they are not collected nor passed by the traces.
Since last commit there's no point anymore in having two variants of the
same function, let's switch to b_free() only. __b_drop() was renamed to
__b_free() for obvious consistency reasons.
A small race exists in buffers with "show sess all". This one wants to show
some information grabbed from the buffer (especially in HTX mode). But the
thread owning this buffer might just be releasing its area, right after a
free() or munmap() call, resulting in a head that is not seen as empty yet
though the area was released. It may then be dereferenced by "show sess all"
causing a crash. Note that in practice it only happens in debug mode with
UAF enabled, but it's tricky enough to fix it right now.
This should be backported to stable versions which support threads and a
store barrier. It's worth noting that by performing the clearing first,
b_free() and b_drop() now become two exact equivalent.
There is no standard case for HTTP header names because, as stated in the
RFC7230, they are case-insensitive. So applications must handle them in a
case-insensitive manner. But some bogus applications erroneously rely on the
case used by most browsers. This problem becomes critical with HTTP/2
because all header names must be exchanged in lowercase. And HAProxy uses the
same convention. All header names are sent in lowercase to clients and servers,
regardless of the HTTP version.
This design choice is linked to the HTX implementation. So, for previous
versions (2.0 and 1.9), a workaround is to disable the HTX mode to fall
back to the legacy HTTP mode.
Since the legacy HTTP mode was removed, some users reported interoperability
issues because their application was not able anymore to handle HTTP/1 message
received from HAProxy. So, we've decided to add a way to change the case of some
headers before sending them. It is now possible to define a "mapping" between a
lowercase header name and a version supported by the bogus application. To do
so, you must use the global directives "h1-case-adjust" and
"h1-case-adjust-file". Then options "h1-case-adjust-bogus-client" and
"h1-case-adjust-bogus-server" may be used in proxy sections to enable the
conversion. See the configuration manual for more info.
Of course, our advice is to urgently upgrade these applications for
interoperability concerns and because they may be vulnerable to various types of
content smuggling attacks. But, if your are really forced to use an unmaintained
bogus application, you may use these directive, at your own risks.
If it is relevant, this feature may be backported to 2.0.
Counters for cache_hits and cache_lookups were added with commit
a1214a50 ("MINOR: cache: report the number of cache lookups and cache
hits") but not documented in management.txt.
Counters for connect and reuse were added in the stats with commit
f1573848 ("MINOR: backend: count the number of connect and reuse
per server and per backend") but not documented the CSV format in
management.txt
Released version 2.1-dev1 with the following main changes :
- BUG/MEDIUM: h2/htx: Update data length of the HTX when the cookie list is built
- DOC: this is a development branch again.
- MEDIUM: Make 'block' directive fatal
- MEDIUM: Make 'redispatch' directive fatal
- MEDIUM: Make '(cli|con|srv)timeout' directive fatal
- MEDIUM: Remove 'option independant-streams'
- MINOR: sample: Add sha2([<bits>]) converter
- MEDIUM: server: server-state global file stored in a tree
- BUG/MINOR: lua/htx: Make txn.req_req_* and txn.res_rep_* HTX aware
- BUG/MINOR: mux-h1: Add the header connection in lower case in outgoing messages
- BUG/MEDIUM: compression: Set Vary: Accept-Encoding for compressed responses
- MINOR: htx: Add the function htx_change_blk_value_len()
- BUG/MEDIUM: htx: Fully update HTX message when the block value is changed
- BUG/MEDIUM: mux-h2: Reset padlen when several frames are demux
- BUG/MEDIUM: mux-h2: Remove the padding length when a DATA frame size is checked
- BUG/MEDIUM: lb_fwlc: Don't test the server's lb_tree from outside the lock
- BUG/MAJOR: sample: Wrong stick-table name parsing in "if/unless" ACL condition.
- BUILD: mworker: silence two printf format warnings around getpid()
- BUILD: makefile: use :space: instead of digits to count commits
- BUILD: makefile: adjust the sed expression of "make help" for solaris
- BUILD: makefile: do not rely on shell substitutions to determine git version
- BUG/MINOR: mworker-prog: Fix segmentation fault during cfgparse
- BUG/MINOR: spoe: Fix memory leak if failing to allocate memory
- BUG/MEDIUM: mworker: don't call the thread and fdtab deinit
- BUG/MEDIUM: stream_interface: Don't add SI_FL_ERR the state is < SI_ST_CON.
- BUG/MEDIUM: connections: Always add the xprt handshake if needed.
- BUG/MEDIUM: ssl: Don't do anything in ssl_subscribe if we have no ctx.
- BUG/MEDIUM: mworker/cli: command pipelining doesn't work anymore
- BUG/MINOR: htx: Save hdrs_bytes when the HTX start-line is replaced
- BUG/MAJOR: mux-h1: Don't crush trash chunk area when outgoing message is formatted
- BUG/MINOR: memory: Set objects size for pools in the per-thread cache
- BUG/MINOR: log: Detect missing sampling ranges in config
- BUG/MEDIUM: proto_htx: Don't add EOM on 1xx informational messages
- BUG/MEDIUM: mux-h1: Use buf_room_for_htx_data() to detect too large messages
- BUG/MINOR: mux-h1: Make format errors during output formatting fatal
- BUG/MEDIUM: ssl: Don't attempt to set alpn if we're not using SSL.
- BUG/MEDIUM: mux-h1: Always release H1C if a shutdown for writes was reported
- BUG/MINOR: mworker/cli: don't output a \n before the response
- BUG/MEDIUM: checks: unblock signals in external checks
- BUG/MINOR: mux-h1: Skip trailers for non-chunked outgoing messages
- BUG/MINOR: mux-h1: Don't return the empty chunk on HEAD responses
- BUG/MEDIUM: connections: Always call shutdown, with no linger.
- BUG/MEDIUM: checks: Make sure the tasklet won't run if the connection is closed.
- BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()
- BUG/MINOR: hlua: Don't use channel_htx_recv_max()
- BUG/MEDIUM: channel/htx: Use the total HTX size in channel_htx_recv_limit()
- BUG/MINOR: hlua/htx: Respect the reserve when HTX data are sent
- BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent
- BUG/MEDIUM: connections: Make sure we're unsubscribe before upgrading the mux.
- BUG/MEDIUM: servers: Authorize tfo in default-server.
- BUG/MEDIUM: sessions: Don't keep an extra idle connection in sessions.
- MINOR: server: Add "no-tfo" option.
- BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks
- MINOR: action: Add the return code ACT_RET_DONE for actions
- BUG/MEDIUM: http/applet: Finish request processing when a service is registered
- BUG/MEDIUM: lb_fas: Don't test the server's lb_tree from outside the lock
- BUG/MEDIUM: mux-h1: Handle TUNNEL state when outgoing messages are formatted
- BUG/MINOR: mux-h1: Don't process input or ouput if an error occurred
- MINOR: stream-int: Factorize processing done after sending data in si_cs_send()
- BUG/MEDIUM: stream-int: Don't rely on CF_WRITE_PARTIAL to unblock opposite si
- DOC: contrib: spoa_server Add some hints for building spoa_server
- DOC: Fix typo in intro.txt
- BUG/MEDIUM: servers: Don't forget to set srv_cs to NULL if we can't reuse it.
- BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2
- MINOR: pools: release the pool's lock during the malloc/free calls
- MINOR: pools: always pre-initialize allocated memory outside of the lock
- MINOR: pools: make the thread harmless during the mmap/munmap syscalls
- BUG/MEDIUM: fd/threads: fix excessive CPU usage on multi-thread accept
- BUG/MINOR: server: Be really able to keep "pool-max-conn" idle connections
- BUG/MEDIUM: checks: Don't attempt to read if we destroyed the connection.
- BUG/MEDIUM: da: cast the chunk to string.
- DOC: Fix typos and grammer in configuration.txt
- CLEANUP: proto_tcp: Remove useless header inclusions.
- BUG/MEDIUM: servers: Fix a race condition with idle connections.
- MINOR: task: introduce work lists
- BUG/MAJOR: listener: fix thread safety in resume_listener()
- BUG/MEDIUM: mux-h1: Don't release h1 connection if there is still data to send
- BUG/MINOR: mux-h1: Correctly report Ti timer when HTX and keepalives are used
- BUG/MEDIUM: streams: Don't give up if we couldn't send the request.
- BUG/MEDIUM: streams: Don't redispatch with L7 retries if redispatch isn't set.
- BUG/MINOR: mux-pt: do not pretend there's more data after a read0
- BUG/MEDIUM: tcp-check: unbreak multiple connect rules again
- MEDIUM: mworker-prog: Add user/group options to program section
- REGTESTS: checks: tcp-check connect to multiple ports
- BUG/MEDIUM: threads: cpu-map designating a single thread/process are ignored
This adds a converter for the SHA-2 family, supporting SHA-224, SHA-256
SHA-384 and SHA-512.
The converter relies on the OpenSSL implementation, thus only being available
when HAProxy is compiled with USE_OPENSSL.
See GitHub issue #123. The hypothetical `ssl_?_sha256` fetch can then be
simulated using `ssl_?_der,sha2(256)`:
http-response set-header Server-Cert-FP %[ssl_f_der,sha2(256),hex]
Released version 2.0.0 with the following main changes :
- MINOR: fd: Don't use atomic operations when it's not needed.
- DOC: mworker-prog: documentation for the program section
- MINOR: http: add a new "http-request replace-uri" action
- BUG/MINOR: 51d/htx: The _51d_fetch method, and the methods it calls are now HTX aware.
- MINOR: 51d: Added dummy libraries for the 51Degrees module for testing.
- MINOR: mworker: change formatting in uptime field of "show proc"
- MINOR: mworker: add the HAProxy version in "show proc"
- MINOR: doc: Remove -Ds option in man page
- MINOR: doc: add master-worker in the man page
- MINOR: doc: mention HAPROXY_LOCALPEER in the man
- BUILD: Silence gcc warning about unused return value
- CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real lib
- BUILD: travis-ci: add 51Degree device detection, update openssl to 1.1.1c
- MINOR: doc: update the manpage and usage message about -S
- BUILD/MINOR: 51d: Updated build registration output to indicate thatif the library is a dummy one or not.
- BUG/MEDIUM: h1: Don't wait for handshake if we had an error.
- BUG/MEDIUM: h1: Wait for the connection if the handshake didn't complete.
- BUG/MINOR: task: prevent schedulable tasks from starving under high I/O activity
- BUG/MINOR: fl_trace/htx: Be sure to always forward trailers and EOM
- BUG/MINOR: channel/htx: Call channel_htx_full() from channel_full()
- BUG/MINOR: http: Use the global value to limit the number of parsed headers
- BUG/MINOR: htx: Detect when tail_addr meet end_addr to maximize free rooms
- BUG/MEDIUM: htx: Don't change position of the first block during HTX analysis
- CLEANUP: channel: Remove channel_htx_fwd_payload() and channel_htx_fwd_all()
- BUG/MEDIUM: proto_htx: Introduce the state ENDING during forwarding
- MINOR: htx: Add 3 flags on the start-line to deal with the request schemes
- MINOR: h2: Set flags about the request's scheme on the start-line
- MINOR: mux-h1: Set flags about the request's scheme on the start-line
- MINOR: mux-h2: Forward clients scheme to servers checking start-line flags
- MEDIUM: server: server-state only rely on server name
- CLEANUP: connection: rename the wait_event.task field to .tasklet
- CLEANUP: tasks: rename task_remove_from_tasklet_list() to tasklet_remove_*
- BUG/MEDIUM: connections: Don't call shutdown() if we want to disable linger.
- DOC: add some environment variables in section 2.3
- BUILD: makefile: clarify the "help" output and list options
- BUG/MINOR: mux-h1: Wake busy mux for I/O when message is fully sent
- BUG: tasks: fix bug introduced by latest scheduler cleanup
- BUG/MEDIUM: mux-h2: fix early close with option abortonclose
- BUG/MEDIUM: connections: Don't use ALPN to pick mux when in mode TCP.
- BUG/MEDIUM: connections: Don't try to send early data if we have no mux.
- BUG/MEDIUM: mux-h2: properly account for the appended data in HTX
- BUILD: makefile: further clarify the "help" output and list targets
- BUILD: makefile: rename "linux2628" to "linux-glibc" and remove older targets
- BUILD: travis-ci: switch to linux-glibc instead of linux2628
- DOC: update few references to the linux* targets and change them to linux-glibc
- BUILD: makefile: detect and reject recently removed linux targets
- BUILD: makefile: enable linux namespaces by default on linux
- BUILD: makefile: enable TFO on linux platforms
- BUILD: makefile: enable getaddrinfo on the linux-glibc target
- DOC: small updates to the CONTRIBUTING file
- BUG/MEDIUM: ssl: Make sure we initiate the handshake after using early data.
- CLEANUP: removed obsolete examples an move a few to better places
- DOC: Fix typos in CONTRIBUTING
- DOC: update the outdated ROADMAP file
- DOC: create a BRANCHES file to explain the life cycle
- DOC: mention in INSTALL haproxy 2.0 is a long-term supported stable version
- BUILD: travis-ci: TFO and GETADDRINFO are now enabled by default
- BUILD: makefile: make the obsolete target detection compatible with make-3.80
- BUILD: tools: work around an internal compiler bug in gcc-3.4
- BUILD: pattern: work around an internal compiler bug in gcc-3.4
- BUILD: makefile: enable USE_RT on Solaris
- BUILD: makefile: do not use echo -n
- DOC: mention a few common build errors in the INSTALL file
The following example files awere removed as irrelevant by this
time :
auth.cfg check.conf ssl.cfg haproxy.spec
The following scripts were removed as having been unused for more
than a decade :
debug2ansi debug2html debugfind check init.haproxy stats_haproxy.sh
seemless_reload.txt was moved to doc/ where it's more suitable.
haproxy.vim was moved to contrib/syntax-highlight/
scripts/create-release was updated not to try to update haproxy.spec
anymore.
The INSTALL guide, the Lua doc and the Prometheus exporter's README all
used to reference "linux2628", "linux26" or even "linux". These were all
updated to consistently reflect "linux-glibc" instead. The default options
were updated there as well so that it should build cleanly on most distros.
This way the directory structure remains the same as with the real lib and
one can apply the same build options regardless of where the lib is stored,
removing any possible confusion.
These are intended for use by HAProxy developers to ensure any changes
did not affect the 51Degrees implementation. The 51Degrees module can be
enabled and used by using the source in contrib/51d. This will run
without breaking, but will not return any meaningful information.
This is ideal for testing HAProxy core code, and other modules alongside
51Degrees, but should never be used as an actual module as it does
nothing.
This action is particularly convenient to replace some deprecated usees
of "reqrep". It takes a match and a format string including back-
references. The reqrep warning was updated to suggest it as well.
Released version 2.0-dev7 with the following main changes :
- BUG/MEDIUM: mux-h2: make sure the connection timeout is always set
- MINOR: tools: add new bitmap manipulation functions
- MINOR: logs: use the new bitmap functions instead of fd_sets for encoding maps
- MINOR: chunks: Make sure trash_size is only set once.
- Revert "MINOR: chunks: Make sure trash_size is only set once."
- MINOR: threads: serialize threads initialization
- MINOR peers: data structure simplifications for server names dictionary cache.
- DOC: peers: Update for dictionary cache entries for peers protocol.
- MINOR: dict: Store the length of the dictionary entries.
- MINOR: peers: A bit of optimization when encoding cached server names.
- MINOR: peers: Optimization for dictionary cache lookup.
- MEDIUM: tools: improve time format error detection
- BUG/MEDIUM: H1: When upgrading, make sure we don't free the buffer too early.
- BUG/MEDIUM: stream_interface: Make sure we call si_cs_process() if CS_FL_EOI.
- MINOR: threads: avoid clearing harmless twice in thread_release()
- MEDIUM: threads: add thread_sync_release() to synchronize steps
- BUG/MEDIUM: init/threads: prevent initialized threads from starting before others
- OPTIM/MINOR: init/threads: only call protocol_enable_all() on first thread
- BUG/MINOR: dict: race condition fix when inserting dictionary entries.
- MEDIUM: init/threads: don't use spinlocks during the init phase
- BUG/MINOR: cache/htx: Fix the counting of data already sent by the cache applet
- BUG/MEDIUM: compression/htx: Fix the adding of the last data block
- MINOR: flt_trace: Don't scrash the original offset during the random forwarding
- MAJOR: htx: Rework how free rooms are tracked in an HTX message
- MINOR: htx: Add the function htx_move_blk_before()
- Revert "BUG/MEDIUM: H1: When upgrading, make sure we don't free the buffer too early."
- BUG/MINOR: http-rules: mention "deny_status" for "deny" in the error message
- MINOR: http: turn default error files to HTTP/1.1
- BUG/MEDIUM: h1: Don't try to subscribe if we had a connection error.
- BUG/MEDIUM: h1: Don't consider we're connected if the handshake isn't done.
- MINOR: contrib/spoa_server: Upgrade SPOP to 2.0
- BUG/MEDIUM: contrib/spoa_server: Set FIN flag on agent frames
- MINOR: contrib/spoa_server: Add random IP score
- DOC/MINOR: contrib/spoa_server: Fix typo in README
Add information about how the peers protocol send/receive entries of
LRU caches for literal dictionaries (e.g. server names in replacement
for server IDs).
Released version 2.0-dev6 with the following main changes :
- BUG/MEDIUM: connection: fix multiple handshake polling issues
- MINOR: connection: also stop receiving after a SOCKS4 response
- MINOR: mux-h1: don't try to recv() before the connection is ready
- BUG/MEDIUM: mux-h1: only check input data for the current stream, not next one
- MEDIUM: mux-h1: don't use CS_FL_REOS anymore
- CLEANUP: connection: remove the now unused CS_FL_REOS flag
- CONTRIB: debug: add 4 missing connection/conn_stream flags
- MEDIUM: stream: make a full process_stream() loop when completing I/O on exit
- MINOR: server: increase the default pool-purge-delay to 5 seconds
- BUILD: tools: do not use the weak attribute for trace() on obsolete linkers
- BUG/MEDIUM: vars: make sure the scope is always valid when accessing vars
- BUG/MEDIUM: vars: make the tcp/http unset-var() action support conditions
- BUILD: task: fix a build warning when threads are disabled
- CLEANUP: peers: Remove tabs characters.
- CLEANUP: peers: Replace hard-coded values by macros.
- BUG/MINOR: peers: Wrong stick-table update message building.
- MINOR: dict: Add dictionary new data structure.
- MINOR: peers: Add a LRU cache implementation for dictionaries.
- MINOR: stick-table: Add "server_name" new data type.
- MINOR: cfgparse: Space allocation for "server_name" stick-table data type.
- MINOR: proxy: Add a "server by name" tree to proxy.
- MINOR: server: Add a dictionary for server names.
- MINOR: stream: Stickiness server lookup by name.
- MINOR: peers: Make peers protocol support new "server_name" data type.
- MINOR: stick-table: Make the CLI stick-table handler support dictionary entry data type.
- REGTEST: Add a basic server by name stickiness reg test.
- MINOR: peers: Add dictionary cache information to "show peers" CLI command.
- MINOR: peers: Replace hard-coded for peer protocol 64-bits value encoding by macros.
- MINOR: peers: Replace hard-coded values for peer protocol messaging by macros.
- CLEANUP: ssl: remove unneeded defined(OPENSSL_IS_BORINGSSL)
- BUILD: travis-ci improvements
- MINOR: SSL: add client/server random sample fetches
- BUG/MINOR: channel/htx: Don't alter channel during forward for empty HTX message
- BUG/MINOR: contrib/prometheus-exporter: Add HTX data block in one time
- BUG/MINOR: mux-h1: errflag must be set on H1S and not H1M during output processing
- MEDIUM: mux-h1: refactor output processing
- MINOR: mux-h1: Add the flag HAVE_O_CONN on h1s
- MINOR: mux-h1: Add h1_eval_htx_hdrs_size() to estimate size of the HTX headers
- MINOR: mux-h1: Don't count the EOM in the estimated size of headers
- MEDIUM: cache/htx: Always store info about HTX blocks in the cache
- MEDIUM: htx: Add the parsing of trailers of chunked messages
- MINOR: htx: Don't use end-of-data blocks anymore
- BUG/MINOR: mux-h1: Don't send more data than expected
- BUG/MINOR: flt_trace/htx: Only apply the random forwarding on the message body.
- BUG/MINOR: peers: Wrong "server_name" decoding.
- BUG/MEDIUM: servers: Don't attempt to destroy idle connections if disabled.
- MEDIUM: checks: Make sure we unsubscribe before calling cs_destroy().
- MEDIUM: connections: Wake the upper layer even if sending/receiving is disabled.
- MEDIUM: ssl: Handle subscribe by itself.
- MINOR: ssl: Make ssl_sock_handshake() static.
- MINOR: connections: Add a new xprt method, remove_xprt.
- MINOR: connections: Add a new xprt method, add_xprt().
- MEDIUM: connections: Introduce a handshake pseudo-XPRT.
- MEDIUM: connections: Remove CONN_FL_SOCK*
- BUG/MEDIUM: ssl: Don't forget to initialize ctx->send_recv and ctx->recv_wait.
- BUG/MINOR: peers: Wrong server name parsing.
- MINOR: server: really increase the pool-purge-delay default to 5 seconds
- BUG/MINOR: stream: don't emit a send-name-header in conn error or disconnect states
- MINOR: stream-int: use bit fields to match multiple stream-int states at once
- MEDIUM: stream-int: remove dangerous interval checks for stream-int states
- MEDIUM: stream-int: introduce a new state SI_ST_RDY
- MAJOR: stream-int: switch from SI_ST_CON to SI_ST_RDY on I/O
- MEDIUM: stream-int: make idle-conns switch to ST_RDY
- MEDIUM: stream: re-arrange the connection setup status reporting
- MINOR: stream-int: split si_update() into si_update_rx() and si_update_tx()
- MINOR: stream-int: make si_sync_send() from the send code of si_update_both()
- MEDIUM: stream: rearrange the events to remove the loop
- MEDIUM: stream: only loop on flags relevant to the analysers
- MEDIUM: stream: don't abusively loop back on changes on CF_SHUT*_NOW
- BUILD: stream-int: avoid a build warning in dev mode in si_state_bit()
- BUILD: peers: fix a build warning about an incorrect intiialization
- BUG/MINOR: time: make sure only one thread sets global_now at boot
- BUG/MEDIUM: tcp: Make sure we keep the polling consistent in tcp_probe_connect.
The main reason for all the trouble we're facing with stream interface
error or timeout reports during the connection phase is that we currently
can't make the difference between a connection attempt and a validated
connection attempt. It is problematic because we tend to switch early
to SI_ST_EST but can't always do what we want in this state since it's
supposed to be set when we don't need to visit sess_establish() again.
This patch introduces a new state betwen SI_ST_CON and SI_ST_EST, which
is SI_ST_RDY. It indicates that we've verified that the connection is
ready. It's a transient state, like SI_ST_DIS, that cannot persist when
leaving process_stream(). For now it is not set, only verified in various
tests where SI_ST_CON was used or SI_ST_EST depending on the cases.
The stream-int state diagram was minimally updated to reflect the new
state, though it is largely obsolete and would need to be seriously
updated.
This adds 4 sample fetches:
- ssl_fc_client_random
- ssl_fc_server_random
- ssl_bc_client_random
- ssl_bc_server_random
These fetches retrieve the client or server random value sent during the
handshake.
Their use is to be able to decrypt traffic sent using ephemeral ciphers. Tools
like wireshark expect a TLS log file with lines in a few known formats
(https://code.wireshark.org/review/gitweb?p=wireshark.git;a=blob;f=epan/dissectors/packet-tls-utils.c;h=28a51fb1fb029eae5cea52d37ff5b67d9b11950f;hb=HEAD#l5209).
Previously the only format supported using data retrievable from HAProxy state
was the one utilizing the Session-ID. However an SSL/TLS session ID is
optional, and thus cannot be relied upon for this purpose.
This change introduces the ability to extract the client random instead which
can be used for one of the other formats. The change also adds the ability to
extract the server random, just in case it might have some other use, as the
code change to support this was trivial.
The default used to be a very aggressive delay of 1 second before starting
to purge idle connections, but tests show that with bursty traffic it's a
bit short. Let's increase this to 5 seconds.
Released version 2.0-dev5 with the following main changes :
- BUILD: watchdog: use si_value.sival_int, not si_int for the timer's value
- BUILD: signals: FreeBSD has SI_LWP instead of SI_TKILL
- BUILD: watchdog: condition it to USE_RT
- MINOR: raw_sock: report global traffic statistics
- MINOR: stats: report the global output bit rate in human readable form
- BUG/MINOR: proto-htx: Try to keep connections alive on redirect
- BUG/MEDIUM: spoe: Don't use the SPOE applet after releasing it
- BUG/MINOR: lua: Set right direction and flags on new HTTP objects
- BUG/MINOR: mux-h2: Count EOM in bytes sent when a HEADERS frame is formatted
- BUG/MINOR: mux-h1: Report EOI instead EOS on parsing error or H2 upgrade
- BUG/MEDIUM: proto-htx: Not forward too much data when 1xx reponses are handled
- BUG/MINOR: htx: Remove a forgotten while loop in htx_defrag()
- DOC: fix typos
- BUG/MINOR: ssl_sock: Fix memory leak when disabling compression
- OPTIM: freq-ctr: don't take the date lock for most updates
- MEDIUM: mux-h2: avoid doing expensive buffer realigns when not absolutely needed
- CLEANUP: debug: remove the TRACE() macro
- MINOR: buffer: introduce b_make() to make a buffer from its parameters
- MINOR: buffer: add a new buffer ring API to manipulate rings of buffers
- MEDIUM: mux-h2: replace all occurrences of mbuf with a buffer ring
- MEDIUM: mux-h2: make the conditions to send based on mbuf, not just its tail
- MINOR: mux-h2: introduce h2_release_mbuf() to release all buffers in the mbuf ring
- MEDIUM: mux-h2: make the send() function iterate over all mux buffers
- CLEANUP: mux-h2: consistently use a local variable for the mbuf
- MINOR: mux-h2: report the mbuf's head and tail in "show fd"
- MAJOR: mux-h2: switch to next mux buffer on buffer full condition.
- BUILD: connections: shut up gcc about impossible out-of-bounds warning
- BUILD: ssl: fix latest LibreSSL reg-test error
- MINOR: cli/activity: remove "fd_del" and "fd_skip" from show activity
- MINOR: cli/activity: add 3 general purpose counters in development mode
- BUG/MAJOR: lb/threads: make sure the avoided server is not full on second pass
- BUG/MEDIUM: queue: fix the tree walk in pendconn_redistribute.
- BUG/MEDIUM: threads: fix double-word CAS on non-optimized 32-bit platforms
- MEDIUM: config: now alert when two servers have the same name
- MINOR: htx: Remove the macro IS_HTX_SMP() and always use IS_HTX_STRM() instead
- MINOR: htx: Move the macro IS_HTX_STRM() in proto/stream.h
- MINOR: htx: Store the head position instead of the wrap one
- MINOR: htx: Store start-line block's position instead of address of its payload
- MINOR: htx: Add functions to get the first block of an HTX message
- MINOR: mux-h2/htx: Get the start-line from the head when HEADERS frame is built
- MINOR: htx: Replace the function http_find_stline() by http_get_stline()
- CLEANUP: htx: Remove unused function htx_get_stline()
- MINOR: http/htx: Use sl_pos directly to replace the start-line
- MEDIUM: http/htx: Perform analysis relatively to the first block
- MINOR: channel/htx: Call channel_htx_recv_max() from channel_recv_max()
- MINOR: htx: Add function htx_get_max_blksz()
- BUG/MINOR: htx: Change htx_xfer_blk() to also count metadata
- MEDIUM: mux-h1: Use the count value received from the SI in h1_rcv_buf()
- MINOR: mux-h2: Use the count value received from the SI in h2_rcv_buf()
- MINOR: stream-int: Don't use the flag CO_RFL_KEEP_RSV anymore in si_cs_recv()
- MINOR: connection: Remove the unused flag CO_RFL_KEEP_RSV
- MINOR: mux-h2/htx: Support zero-copy when possible in h2_rcv_buf()
- MINOR: htx: Add a field to set the memory used by headers in the HTX start-line
- MINOR: h2/htx: Set hdrs_bytes on the SL when an HTX message is produced
- MINOR: mux-h1: Set hdrs_bytes on the SL when an HTX message is produced
- MINOR: htx: Be sure to xfer all headers in one time in htx_xfer_blks()
- MEDIUM: htx: 1xx messages are now part of the final reponses
- MINOR: channel/htx: Add function to forward headers of an HTX message
- MINOR: filters/htx: Use channel_htx_fwd_headers() after headers filtering
- MINOR: proto-htx: Use channel_htx_fwd_headers() to forward 1xx responses
- MEDIUM: htx: Store the first block position instead of the start-line one
- MINOR: stats/htx: don't use the first block position but the head one
- MINOR: channel/htx: Add functions to forward a part or all HTX payload
- MINOR: proto-htx: Use channel_htx_fwd_all() when unfiltered body are forwarded
- MEDIUM: filters/htx: Filter body relatively to the first block
- MINOR: htx: Optimize htx_drain() when all data are drained
- MINOR: htx: don't rely on htx_find_blk() anymore in the function htx_truncate()
- MINOR: htx: remove the unused function htx_find_blk()
- MINOR: htx: Remove support of pseudo headers because it is unused
- BUG/MEDIUM: http: fix "http-request reject" when not final
- MINOR: ssl: Make sure the underlying xprt's init method doesn't fail.
- MINOR: ssl: Don't forget to call the close method of the underlying xprt.
- MINOR: htx: rename htx_append_blk_value() to htx_add_data_atonce()
- MINOR: htx: make htx_add_data() return the transmitted byte count
- MEDIUM: htx: make htx_add_data() never defragment the buffer
- MINOR: activity: write totals on the "show activity" output
- MINOR: activity: report totals and average separately
- MEDIUM: poller: separate the wait time from the wake events
- MINOR: activity: report the number of failed pool/buffer allocations
- MEDIUM: buffers: relax the buffer lock a little bit
- MINOR: task: turn the WQ lock to an RW_LOCK
- MEDIUM: task: don't grab the WR lock just to check the WQ
- BUG/MEDIUM: mux-h1: Don't skip the TCP splicing when there is no more data to read
- MEDIUM: sessions: Introduce session flags.
- BUG/MEDIUM: h2: Don't forget to set h2s->cs to NULL after having free'd cs.
- BUG/MEDIUM: mux-h2: fix the conditions to end the h2_send() loop
- BUG/MEDIUM: mux-h2: don't refrain from offering oneself a used buffer
- BUG/MEDIUM: connection: Use the session to get the origin address if needed.
- MEDIUM: tasks: Get rid of active_tasks_mask.
- MEDIUM: connection: Upstream SOCKS4 proxy support
- BUILD: contrib/prometheus: fix build breakage caused by move of idle_pct
- BUG/MINOR: deinit/threads: make hard-stop-after perform a clean exit
Have "socks4" and "check-via-socks4" server keyword added.
Implement handshake with SOCKS4 proxy server for tcp stream connection.
See issue #82.
I have the "SOCKS: A protocol for TCP proxy across firewalls" doc found
at "https://www.openssh.com/txt/socks4.protocol". Please reference to it.
[wt: for now connecting to the SOCKS4 proxy over unix sockets is not
supported, and mixing IPv4/IPv6 is discouraged; indeed, the control
layer is unique for a connection and will be used both for connecting
and for target address manipulation. As such it may for example report
incorrect destination addresses in logs if the proxy is reached over
IPv6]
s/accidently/accidentally/
s/any ot these messages/any of theses messages/
s/catched/caught/
s/completly/completely/
s/convertor/converter/
s/desribing/describing/
s/developper/developer/
s/eventhough/even though/
s/exectution/execution/
s/functionnality/functionality/
s/If it receive a/If it receives a/
s/In can even/It can even/
s/informations/information/
s/it will be remove /it will be removed /
s/langage/language/
s/mentionned/mentioned/
s/negociated/negotiated/
s/Optionnaly/Optionally/
s/ouputs/outputs/
s/outweights/outweighs/
s/ressources/resources/
Released version 2.0-dev4 with the following main changes :
- BUILD: enable freebsd builds on cirrus-ci
- BUG/MINOR: http_fetch: Rely on the smp direction for "cookie()" and "hdr()"
- MEDIUM: Make 'option forceclose' actually warn
- MEDIUM: Make 'resolution_pool_size' directive fatal
- DOC: management: place "show activity" at the right place
- MINOR: cli/activity: show the dumping thread ID starting at 1
- MINOR: task: export global_task_mask
- MINOR: cli/debug: add a thread dump function
- BUG/MEDIUM: streams: Don't use CF_EOI to decide if the request is complete.
- BUG/MEDIUM: streams: Try to L7 retry before aborting the connection.
- BUG/MINOR: debug: make ha_task_dump() always check the task before dumping it
- BUG/MINOR: debug: make ha_task_dump() actually dump the requested task
- MINOR: debug: make ha_thread_dump() and ha_task_dump() take a buffer
- BUG/MINOR: debug: don't check the call date on tasklets
- MINOR: thread: implement ha_thread_relax()
- MINOR: task: put barriers after each write to curr_task
- MINOR: task: always reset curr_task when freeing a task or tasklet
- MINOR: stream: detach the stream from its own task on stream_free()
- MEDIUM: debug/threads: implement an advanced thread dump system
- REGTEST: extend the check duration on tls_health_checks and mark it slow
- DOC: fix "successful" typo
- MINOR: init: setenv HAPROXY_CFGFILES
- MINOR: threads/init: synchronize the threads startup
- MEDIUM: init/mworker: make the pipe register function a regular initcall
- CLEANUP: memory: make the fault injection code use the OTHER_LOCK label
- CLEANUP: threads: remove the now unused START_LOCK label
- MINOR: init/threads: make the global threads an array of structs
- MINOR: threads: add each thread's clockid into the global thread_info
- CLEANUP: stream: remove an obsolete debugging test
- MINOR: tools: add dump_hex()
- MINOR: debug: implement ha_panic()
- MINOR: debug/cli: add some debugging commands for developers
- MINOR: tools: provide a may_access() function and make dump_hex() use it
- MINOR: debug: make ha_panic() report threads starting at 1
- REORG: compat: move some integer limit definitions from standard.h to compat.h
- REORG: threads: move the struct thread_info from global.h to hathreads.h
- MINOR: compat: make sure to always define clockid_t
- MINOR: threads: always place the clockid in the struct thread_info
- MINOR: threads: add a thread-local thread_info pointer "ti"
- MINOR: time: move the cpu, mono, and idle time to thread_info
- MINOR: time: add a function to retrieve another thread's cputime
- MINOR: debug: report each thread's cpu usage in "show thread"
- BUILD: threads: only assign the clock_id when supported
- BUILD: makefile: use USE_OBSOLETE_LINKER for solaris
- BUILD: makefile: remove -fomit-frame-pointer optimisation (solaris)
- MAJOR: polling: add event ports support (Solaris)
- BUG/MEDIUM: streams: Don't switch from SI_ST_CON to SI_ST_DIS on read0.
- CLEANUP: time: refine the test on _POSIX_TIMERS
- MINOR: compat: define a new empty type empty_t for non-implemented fields
- CLEANUP: time: switch clockid_t to empty_t when not available
- BUG/MINOR: mworker: Fix memory leak of mworker_proc members
- CLEANUP: objtype: make obj_type() and obj_type_name() take consts
- MINOR: debug: switch to SIGURG for thread dumps
- CLEANUP: threads: really move thread_info to hathreads.c
- MINOR: threads: make threads_{harmless|want_rdv}_mask constant 0 without threads
- CLEANUP: debug: always report harmless/want_rdv even without threads
- MINOR: threads: implement ha_tkill() and ha_tkillall()
- CLEANUP: debug: make use of ha_tkill() and remove ifdefs
- MINOR: stream: introduce a stream_dump() function and use it in stream_dump_and_crash()
- MINOR: debug: dump streams when an applet, iocb or stream is known
- MINOR: threads: add a "stuck" flag to the thread_info struct
- MINOR: threads: add a timer_t per thread in thread_info
- MAJOR: watchdog: implement a thread lockup detection mechanism
- MINOR: stream: remove the cpu time detection from process_stream()
- MINOR: connection: report the mux names in "haproxy -vv"
- CLEANUP: mux-h1: use "H1" and not "h1" as the mux's name
- BUG/MEDIUM: WURFL: segfault in wurfl-get() with missing info.
- MINOR: WURFL: call header_retireve_callback() in dummy library
- MINOR: WURFL: fixed Engine load failed error when wurfl-information-list contains wurfl_root_id
- MINOR: WURFL: shows log messages during module initialization
- MINOR: WURFL: removes heading wurfl-information-separator from wurfl-get-all() and wurfl-get() results
- MINOR: WURFL: wurfl_get() and wurfl_get_all() now return an empty string if device detection fails
- MEDIUM: WURFL: HTX awareness.
- MINOR: WURFL: module version bump to 2.0
- MINOR: WURFL: do not emit warnings when not configured
- CONTRIB: wurfl: address 3 build issues in the wurfl dummy library
- BUG/MEDIUM: init/threads: provide per-thread alloc/free function callbacks
- BUILD: travis: add sanitizers to travis-ci builds
- BUILD: time: remove the test on _POSIX_C_SOURCE
- CLEANUP: build: rename some build macros to use the USE_* ones
- CLEANUP: raw_sock: remove support for very old linux splice bug workaround
- BUG/MEDIUM: dns: make the port numbers unsigned
- MEDIUM: config: deprecate the antique req* and rsp* commands
These commands don't follow the same flow as the rest of the commands,
each of them iterates over all header lines before switching to the
next directive. In addition they make no distinction between start
line and headers and can lead to unparsable rewrites which are very
difficult to deal with internally.
Most of them are still occasionally found in configurations, mainly
because of the usual "we've always done this way". By marking them
deprecated and emitting a warning and recommendation on first use of
each of them, we will raise users' awareness of users regarding the
cleaner, faster and more reliable alternatives.
Some use cases of "reqrep" still appear from time to time for URL
rewriting that is not so convenient with other rules. But at least
users facing this requirement will explain their use case so that we
can best serve them. Some discussion started on this subject in a
thread linked to from github issue #100.
The goal is to remove them in 2.1 since they require to reparse the
result before indexing it and we don't want this hack to live long.
The following directives were marked deprecated :
-reqadd
-reqallow
-reqdel
-reqdeny
-reqiallow
-reqidel
-reqideny
-reqipass
-reqirep
-reqitarpit
-reqpass
-reqrep
-reqtarpit
-rspadd
-rspdel
-rspdeny
-rspidel
-rspideny
-rspirep
-rsprep
This flag is constantly cleared by the scheduler and will be set by the
watchdog timer to detect stuck threads. It is also set by the "show
threads" command so that it is easy to spot if the situation has evolved
between two subsequent calls : if the first "show threads" shows no stuck
thread and the second one shows such a stuck thread, it indicates that
this thread didn't manage to make any forward progress since the previous
call, which is extremely suspicious.
Event ports are kqueue/epoll polling class for Solaris. Code is based
on https://github.com/joyent/haproxy-1.8/tree/joyent/dev-v1.8.8.
Event ports are available only on SunOS systems derived from
Solaris 10 and later (including illumos systems).
When haproxy is built with DEBUG_DEV, the following commands are added
to the CLI :
debug dev close <fd> : close this file descriptor
debug dev delay [ms] : sleep this long
debug dev exec [cmd] ... : show this command's output
debug dev exit [code] : immediately exit the process
debug dev hex <addr> [len]: dump a memory area
debug dev log [msg] ... : send this msg to global logs
debug dev loop [ms] : loop this long
debug dev panic : immediately trigger a panic
debug dev tkill [thr] [sig] : send signal to thread
These are essentially aimed at helping developers trigger certain
conditions and are expected to be complemented over time.
The current "show threads" command was too limited as it was not possible
to dump other threads' detailed states (e.g. their tasks). This patch
goes further by using thread signals so that each thread can dump its
own state in turn into a shared buffer provided by the caller. Threads
are synchronized using a mechanism very similar to the rendez-vous point
and using this method, each thread can safely dump any of its contents
and the caller can finally report the aggregated ones from the buffer.
It is important to keep in mind that the list of signal-safe functions
is limited, so we take care of only using chunk_printf() to write to a
pre-allocated buffer.
This mechanism is enabled by USE_THREAD_DUMP and is enabled by default
on Linux 2.6.28+. On other platforms it falls back to the previous
solution using the loop and the less precise dump.
The new function ha_thread_dump() will dump debugging info about all known
threads. The current thread will contain a bit more info. The long-term goal
is to make it possible to use it in signal handlers to improve the accuracy
of some dumps.
The function dumps its output into the trash so as it was trivial to add,
a new "show threads" command appeared on the CLI.
This directive never appeared in a stable release and instead was
introduced and deprecated within 1.8-dev. While it technically could
be outright removed we detect it and error out for good measure.
Released version 2.0-dev3 with the following main changes :
- BUG/MINOR: peers: Really close the sessions with no heartbeat.
- CLEANUP: peers: remove useless annoying tabulations.
- CLEANUP: peers: replace timeout constants by macros.
- REGTEST: Enable again reg tests with HEAD HTTP method usage.
- DOC: The option httplog is no longer valid in a backend.
- DOC: peers: Peers protocol documentation update.
- REGTEST: remove unexpected "nbthread" statement from Lua test cases
- BUILD: Makefile: remove 11-years old workarounds for deprecated options
- BUILD: remove 10-years old error message for obsolete option USE_TCPSPLICE
- BUILD: Makefile: remove outdated support for dlmalloc
- BUILD: Makefile: consider a variable's origin and not its value for the options list
- BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
- BUILD: Makefile: shorten default settings declaration
- BUILD: Makefile: clean up the target declarations
- BUILD: report the whole feature set with their status in haproxy -vv
- BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
- REGTEST: script: make the script use the new features list
- REGTEST: script: remove platform-specific assigments of OPTIONS
- BUG/MINOR: peers: Missing initializations after peer session shutdown.
- BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes
- BUILD/MINOR: listener: Silent a few signedness warnings.
- BUG/MINOR: mux-h1: Only skip invalid C-L headers on output
- BUG/MEDIUM: mworker: don't free the wrong child when not found
- BUG/MEDIUM: checks: Don't bother subscribing if we have a connection error.
- BUG/MAJOR: checks: segfault during tcpcheck_main
- BUILD: makefile: work around an old bug in GNU make-3.80
- BUILD: makefile: work around another bug in make 3.80
- BUILD: http: properly mark some struct as extern
- BUILD: chunk: properly declare pool_head_trash as extern
- BUILD: cache: avoid a build warning with some compilers/linkers
- MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
- MINOR: tools: add an unsetenv() implementation
- BUILD: re-implement an initcall variant without using executable sections
- BUILD: use inttypes.h instead of stdint.h
- BUILD: connection: fix naming of ip_v field
- BUILD: makefile: fix build of IPv6 header on aix51
- BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
- BUILD: define unsetenv on AIX 5.1
- BUILD: Makefile: disable shared cache on AIX 5.1
- MINOR: ssl: Add aes_gcm_dec converter
- REORG: mworker: move serializing functions to mworker.c
- REORG: mworker: move signals functions to mworker.c
- REORG: mworker: move IPC functions to mworker.c
- REORG: mworker: move signal handlers and related functions
- REORG: mworker: move mworker_cleanlisteners to mworker.c
- MINOR: mworker: calloc mworker_proc structures
- MINOR: mworker: don't use children variable anymore
- MINOR: cli: export cli_parse_default() definition in cli.h
- REORG: mworker/cli: move CLI functions to mworker.c
- MEDIUM: mworker-prog: implement program for master-worker
- MINOR: mworker/cli: show programs in 'show proc'
- BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
- MINOR: cli: start addresses by a prefix in 'show cli sockets'
- MINOR: cli: export HAPROXY_CLI environment variable
- BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing error
- BUG/MINOR: proto_htx: Reset to_forward value when a message is set to DONE
- REGTEST: http-capture/h00000: Relax a regex matching the log message
- REGTEST: http-messaging/h00000: Fix the test when the HTX is enabled
- REGTEST: http-rules/h00003: Use a different client for requests expecting a 301
- REGTEST: log/b00000: Be sure the client always hits its timeout
- REGTEST: lua/b00003: Relax the regex matching the log message
- REGTEST: lua/b00003: Specify the HAProxy pid when the command ss is executed
- BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on release.
- BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we subscribed.
- BUG/MEDIUM: htx: fix random premature abort of data transfers
- BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
- BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
- BUG/MEDIUM: stream: Don't clear the stream_interface flags in si_update_both.
- MINOR: initcall: Don't forget to define the __start/stop_init_##stg symbols.
- MINOR: threads: Implement thread_cpus_enabled() for FreeBSD.
- BUG/MEDIUM: pattern: assign pattern IDs after checking the config validity
- MINOR: skip get_gmtime where tm is unused
- MINOR: ssl: Activate aes_gcm_dec converter for BoringSSL
- BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected state.
- BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not connected.
- BUG/MEDIUM: task/threads: address a fairness issue between local and global tasks
- BUG/MINOR: tasks: make sure the first task to be queued keeps its nice value
- BUG/MINOR: listener: renice the accept ring processing task
- MINOR: cli/listener: report the number of accepts on "show activity"
- MINOR: cli/activity: report the accept queue sizes in "show activity"
- BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to the stream
- BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented messages
- BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is encoded
- BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads wrap
- BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a data block
- MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
- MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
- BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
- MINOR: mux-h1: Simplify handling of 1xx responses
- MINOR: stats/htx: Don't add "Connection: close" header anymore in stats responses
- MEDIUM: h1: Add an option to sanitize connection headers during parsing
- MEDIUM: mux-h1: Simplify the connection mode management by sanitizing headers
- MINOR: mux-h1: Don't release the conn_stream anymore when h1s is destroyed
- BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown read/write
- MINOR: mux-h2: Add a mux_ops dedicated to the HTX mode
- MINOR: muxes: Add a flag to specify a multiplexer uses the HTX
- MINOR: stream: Set a flag when the stream uses the HTX
- MINOR: http: update the macro IS_HTX_STRM() to check the stream flag SF_HTX
- MINOR: http_fetch/htx: Use stream flags instead of px mode in smp_prefetch_htx
- MINOR: filters/htx: Use stream flags instead of px mode to instanciate a filter
- MINOR: muxes: Rely on conn_is_back() during init to handle front/back conn
- MEDIUM: muxes: Add an optional input buffer during mux initialization
- MINOR: muxes: Pass the context of the mux to destroy() instead of the connection
- MEDIUM: muxes: Be prepared to don't own connection during the release
- MEDIUM: connection: Add conn_upgrade_mux_fe() to handle mux upgrades
- MEDIUM: htx: Allow the option http-use-htx to be used on TCP proxies too
- MAJOR: proxy/htx: Handle mux upgrades from TCP to HTTP in HTX mode
- MAJOR: muxes/htx: Handle inplicit upgrades from h1 to h2
- MAJOR: htx: Enable the HTX mode by default for all proxies
- REGTEST: Use HTX by default and add '--no-htx' option to disable it
- BUG/MEDIUM: muxes: Don't dereference mux context if null in release functions
- CLEANUP: task: do not export rq_next anymore
- MEDIUM: tasks: improve fairness between the local and global queues
- MEDIUM: tasks: only base the nice offset on the run queue depth
- MINOR: tasks: restore the lower latency scheduling when niced tasks are present
- BUG/MEDIUM: map: Fix memory leak in the map converter
- BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation
- BUILD: task/thread: fix single-threaded build of task.c
- BUILD: cli/threads: fix build in single-threaded mode
- BUG/MEDIUM: muxes: Make sure we unsubcribed when destroying mux ctx.
- BUG/MEDIUM: h2: Make sure we're not already in the send_list in h2_subscribe().
- BUG/MEDIUM: h2: Revamp the way send subscriptions works.
- MINOR: connections: Remove the SUB_CALL_UNSUBSCRIBE flag.
- BUG/MEDIUM: Threads: Only use the gcc >= 4.7 builtins when using gcc >= 4.7.
- BUILD: address a few cases of "static <type> inline foo()"
- BUILD: do not specify "const" on functions returning structs or scalars
- BUILD: htx: fix a used uninitialized warning on is_cookie2
- MINOR: peers: Add a new command to the CLI for peers.
- DOC: update for "show peers" CLI command.
- BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
- MEDIUM: mworker: store the leaving state of a process
- MEDIUM: mworker-prog: implements 'option start-on-reload'
- CLEANUP: mworker: remove the type field in mworker_proc
- MEDIUM: mworker/cli: export the HAPROXY_MASTER_CLI variable
- MINOR: cli: don't add a semicolon at the end of HAPROXY_CLI
- MINOR: mworker: export HAPROXY_MWORKER=1 when running in mworker mode
- MINOR: init: add a "set-dumpable" global directive to enable core dumps
- BUG/MINOR: listener/mq: correctly scan all bound threads under low load
- BUG/MINOR: mworker: mworker_kill should apply on every children
- BUG/MINOR: mworker: don't exit with an ambiguous value
- BUG/MINOR: mworker: ensure that we still quits with SIGINT
- REGTESTS: exclude tests that require ssl, pcre if no such feature is enabled
- BUG/MINOR: mux-h1: Process input even if the input buffer is empty
- BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side has done
- BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf is empty
- BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
- MINOR: mux-h1: Handle read0 during TCP splicing
- BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
- BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
- BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
- BUG/MINOR: http_fetch/htx: Use HTX versions if the proxy enables the HTX mode
- BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to the rq.
- BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the rq_lock.
- MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
- MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
- MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet list.
- BUG/MEDIUM: applets: Don't use task_in_rq().
- BUG/MAJOR: task: make sure never to delete a queued task
- MINOR: task/thread: factor out a wake-up condition
- CLEANUP: task: remain consistent when using the task's handler
- MEDIUM: tasks: Merge task_delete() and task_free() into task_destroy().
- MEDIUM: tasks: Don't account a destroyed task as a runned task.
- BUG/MINOR: contrib/prometheus-exporter: Fix a typo in the run-queue metric type
- MINOR: contrib/prometheus-exporter: Remove usless rate metrics
- MINOR: contrib/prometheus-exporter: Rename some metrics to be more usable
- MINOR: contrib/prometheus-exporter: Follow best practices about metrics type
- BUG/MINOR: mworker: disable busy polling in the master process
- MEDIUM: tasks: Use __ha_barrier_store after modifying global_tasks_mask.
- MEDIUM: ssl: Give ssl_sock its own context.
- MEDIUM: connections: Move some fields from struct connection to ssl_sock_ctx.
- MEDIUM: ssl: provide its own subscribe/unsubscribe function.
- MEDIUM: connections: Provide a xprt_ctx for each xprt method.
- MEDIUM: ssl: provide our own BIO.
- BUILD/medium: ssl: Fix build with OpenSSL < 1.1.0
- MINOR: peers: adds counters on show peers about tasks calls.
- MEDIUM: enable travis-ci builds
- MINOR: fd: Add a counter of used fds.
- MEDIUM: connections: Add a way to control the number of idling connections.
- BUG/MEDIUM: maps: only try to parse the default value when it's present
- BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
- REGTEST: Missing REQUIRE_VERSION declarations.
- MINOR: proto_tcp: tcp-request content: enable set-dst and set-dst-var
- BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
- BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP rules
- BUG/MEDIUM: stream: Don't request a server connection if a shutw was scheduled
- BUG/MINOR: 51d: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
- BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
- MINOR: gcc: Fix a silly gcc warning in connect_server()
- MINOR: ssl/cli: async fd io-handlers printable on show fd
- Revert "CLEANUP: wurfl: remove dead, broken and unmaintained code"
- BUILD: add USE_WURFL to the list of known build options
- MINOR: wurfl: indicate in haproxy -vv the wurfl version in use
- BUILD: wurfl: build fix for 1.9/2.0 code base
- CLEANUP: wurfl: removed deprecated methods
- DOC: wurfl: added point of contact in MAINTAINERS file
- MINOR: wurfl: enabled multithreading mode
- MINOR: contrib: dummy wurfl library
- MINOR: dns: dns_requester structures are now in a memory pool
- MINOR: dns: move callback affection in dns_link_resolution()
- MINOR: obj_type: new object type for struct stream
- MINOR: action: new '(http-request|tcp-request content) do-resolve' action
- MINOR: log: Extract some code to send syslog messages.
- REGTEST: replace LEVEL option by a more human readable one.
- REGTEST: rename the reg test files.
- REGTEST: adapt some reg tests after renaming.
- REGTEST: make the "run-regtests" script search for tests in reg-tests by default
- BUG/MAJOR: stream: Missing DNS context initializations.
- BUG/MEDIUM: stream: Fix the way early aborts on the client side are handled
- BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet handler
- BUG/MEDIUM: ssl: Return -1 on recv/send if we got EAGAIN.
- BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB
- BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies only
- BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during startup
- CLEANUP: task: report calls as unsigned in show sess
- MINOR: tasks/activity: report the context switch and task wakeup rates
- MINOR: stream: measure and report a stream's call rate in "show sess"
- MINOR: applet: measure and report an appctx's call rate in "show sess"
- BUILD: extend Travis CI config to support more platforms
- REGTEST: exclude osx and generic targets for 40be_2srv_odd_health_checks
- REGTEST: relax the IPv6 address format checks in converters_ipmask_concat_strcmp_field_word
- REGTEST: exclude OSX and generic targets from abns_socket.vtc
- BUILD: travis: remove the "allow_failures" entry
- BUG/MINOR: activity: always initialize the profiling variable
- MINOR: activity: make the profiling status per thread and not global
- MINOR: activity: enable automatic profiling turn on/off
- CLEANUP: standard: use proper const to addr_to_str() and port_to_str()
- BUG/MINOR: proto_http: properly reset the stream's call rate on keep-alive
- MINOR: connection: make the debugging helper functions safer
- MINOR: stream/debug: make a stream dump and crash function
- MEDIUM: appctx/debug: force a crash if an appctx spins over itself forever
- MEDIUM: stream/debug: force a crash if a stream spins over itself forever
- MEDIUM: streams: measure processing time and abort when detecting bugs
- BUILD/MEDIUM: contrib: Dummy DeviceAtlas API.
- MEDIUM: da: HTX mode support.
- BUG/MEDIUM: mux-h2: properly deal with too large headers frames
- BUG/MINOR: http: Call stream_inc_be_http_req_ctr() only one time per request
- BUG/MEDIUM: spoe: arg len encoded in previous frag frame but len changed
- MINOR: spoe: Use the sample context to pass frag_ctx info during encoding
- DOC: contrib/modsecurity: Typos and fix the reject example
- BUG/MEDIUM: contrib/modsecurity: If host header is NULL, don't try to strdup it
- MINOR: log: Add "sample" new keyword to "log" lines.
- MINOR: log: Enable the log sampling and load-balancing feature.
- DOC: log: Document the sampling and load-balancing logging feature.
- REGTEST: Add a new reg test for log load-balancing feature.
- BUG/MAJOR: map/acl: real fix segfault during show map/acl on CLI
- REGTEST: Make this reg test be Linux specific.
- CLEANUP: task: move the task_per_thread definition to task.h
- MINOR: activity: report context switch counts instead of rates
- MINOR: threads: Implement HA_ATOMIC_LOAD().
- BUG/MEDIUM: port_range: Make the ring buffer lock-free.
- BUG/MEDIUM: listener: Fix how unlimited number of consecutive accepts is handled
- MINOR: config: Test validity of tune.maxaccept during the config parsing
- CLEANUP: config: Don't alter listener->maxaccept when nbproc is set to 1
- BUG/MEDIUM: servers: fix typo "src" instead of "srv"
- BUG/MEDIUM: ssl: Don't pretend we can retry a recv/send if we got a shutr/w.
- BUG/MINOR: haproxy: fix rule->file memory leak
- BUG/MINOR: log: properly free memory on logformat parse error and deinit()
- BUG/MINOR: checks: free memory allocated for tasklets
- BUG/MEDIUM: pattern: fix memory leak in regex pattern functions
- BUG/MEDIUM: channels: Don't forget to reset output in channel_erase().
- BUG/MEDIUM: connections: Make sure we remove CO_FL_SESS_IDLE on disown.
- MINOR: threads: flatten the per-thread cpu-map
- MINOR: init/threads: remove the useless tids[] array
- MINOR: init/threads: make the threads array global
- BUG/MEDIUM: ssl: Use the early_data API the right way.
- BUG/MEDIUM: streams: Don't add CF_WRITE_ERROR if early data were rejected.
- MEDIUM: streams: Add the ability to retry a request on L7 failure.
- MEDIUM: streams: Add a way to replay failed 0rtt requests.
- MEDIUM: streams: Add a new keyword for retry-on, "junk-response"
- BUG/MINOR: stream: also increment the retry stats counter on L7 retries
- BUG/MEDIUM: checks: make sure the warmup task takes the server lock
- BUG/MINOR: logs/threads: properly split the log area upon startup
- BUILD: extend travis-ci matrix
- CLEANUP: Remove appsession documentation
- DOC: Fix typo in keyword matrix
- BUILD: remove "build_libressl" duplicate declaration
- BUILD: travis-ci: get back to osx without openssl support
- BUILD: enable several LibreSSL hacks, including
- BUILD: temporarily mark LibreSSL builds as allowed to fail
- BUILD: travis: TMPDIR replacement.
- BUG/MEDIUM: ssl: Don't attempt to use early data with libressl.
- MINOR: doc: Document allow-0rtt on the server line.
- MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
- MEDIUM: proto: Change the prototype of the connect() method.
- MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
- MINOR: config: Extract the code of "stick-table" line parsing.
- BUILD/MINOR: stick-table: Compilation fix.
- MEDIUM: stick-table: Stop handling stick-tables as proxies.
- MINOR: stick-tables: Add peers process binding computing.
- MINOR: stick-table: Add prefixes to stick-table names.
- MINOR: peers: Do not emit global stick-table names.
- DOC: Update for "table" lines in "peers" section.
- REGTEST: Add reg tests for "table" lines in "peers" sections.
- MEDIUM: regex: modify regex_comp() to atomically allocate/free the my_regex struct
- REGTEST: make the tls_health_checks test much faster
- REGTEST: make the "table in peers" test require v2.0
- BUG/MINOR: mux-h2: rely on trailers output not input to turn them to empty data
- BUG/MEDIUM: h2/htx: always fail on too large trailers
- MEDIUM: mux-h2: discard contents that are to be sent after a shutdown
- BUG/MEDIUM: mux-h2/htx: never wait for EOM when processing trailers
- BUG/MEDIUM: h2/htx: never leave a trailers block alone with no EOM block
- REGTEST: Flag some slow reg tests.
- REGTEST: Reg tests file renaming.
- REGTEST: Wrong renaming for one reg test.
- REGTEST: Wrong assumption in IP:port logging test.
- BUG/MINOR: mworker/ssl: close OpenSSL FDs on reload
- MINOR: systemd: Use the variables from /etc/default/haproxy
- MINOR: systemd: Make use of master socket in systemd unit
- MINOR: systemd: support /etc/sysconfig/ for redhat based distrib
- BUG/MEDIUM: stick-table: fix regression caused by a change in proxy struct
- BUG/MEDIUM: tasks: fix possible segfault on task_destroy()
- CLEANUP: task: remove unneeded tests before task_destroy()
- MINOR: mworker: support a configurable maximum number of reloads
- BUG/MINOR: mux-h2: fix the condition to close a cs-less h2s on the backend
- BUG/MEDIUM: spoe: Be sure the sample is found before setting its context
- BUG/MINOR: mux-h1: Fix the parsing of trailers
- BUG/MINOR: htx: Never transfer more than expected in htx_xfer_blks()
- MINOR: htx: Split on DATA blocks only when blocks are moved to an HTX message
- MINOR: htx: Don't try to append a trailer block with the previous one
- MINOR: htx: Remove support for unused OOB HTX blocks
- BUILD: travis-ci bugfixes and improvements
- BUG/MEDIUM: servers: Don't use the same srv flag for cookie-set and TFO.
- BUG/MEDIUM: h2: Make sure we set send_list to NULL in h2_detach().
- BUILD: ssl: fix again a libressl build failure after the openssl FD leak fix
- CLEANUP: ssl-sock: use HA_OPENSSL_VERSION_NUMBER instead of OPENSSL_VERSION_NUMBER
- BUILD: ssl: make libressl use its own version numbers
- CLEANUP: ssl: remove 57 occurrences of useless tests on LIBRESSL_VERSION_NUMBER
- MINOR: ssl: enable aes_gcm_dec on LibreSSL
- BUILD: ssl: fix libressl build again after aes-gcm-enc
- REORG: ssl: move openssl-compat from proto to common
- REORG: ssl: move some OpenSSL defines from ssl_sock to openssl-compat
- CLEANUP: ssl: never include openssl/*.h outside of openssl-compat.h anymore
- CLEANUP: ssl: make inclusion of openssl headers safe
- BUILD: add BoringSSL to travis-ci build matrix
- BUILD: threads: Add __ha_cas_dw fallback for single threaded builds
- BUG/MINOR: stream: Attach the read side on the response as soon as possible
- BUG/MEDIUM: http: Use pointer to the begining of input to parse message headers
- BUG/MEDIUM: h2: Don't check send_wait to know if we're in the send_list.
- BUG/MEDIUM: streams: Make sur SI_FL_L7_RETRY is set before attempting a retry.
- MEDIUM: streams: Add a new http action, disable-l7-retry.
- MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
- BUG/MINOR: vars: Fix memory leak in vars_check_arg
- BUILD: travis-ci: make TMPDIR global variable in travis-ci
- CLEANUP: ssl: move the SSL_OP_* and SSL_MODE_* definitions to openssl-compat
- CLEANUP: ssl: remove ifdef around SSL_CTX_get_extra_chain_certs()
- CLEANUP: ssl: move all BIO_* definitions to openssl-compat
- BUILD: threads: fix again the __ha_cas_dw() definition
- BUG/MAJOR: mux-h2: do not add a stream twice to the send list
- Revert "BUG/MINOR: vars: Fix memory leak in vars_check_arg"
- BUG/MINOR: peers: Fix memory leak in cfg_parse_peers
- BUG/MINOR: htx: make sure to always initialize the HTTP method when parsing a buffer
- REGTEST: fix tls_health_checks random failures on MacOS in Travis-CI
- MINOR: spoe: Set the argument chunk size to 0 when SPOE variables are checked
- BUG/MINOR: vars: Fix memory leak in vars_check_arg
- BUG/MAJOR: ssl: segfault upon an heartbeat request
- MINOR: spoa-server: Clone the v1.7 spoa-example project
- MINOR: spoa-server: move some definition from spoa_server.c to spoa_server.h
- MINOR: spoa-server: Externalise debug functions
- MINOR: spoe-server: rename "worker" functions
- MINOR: spoa-server: Replace the thread init system by processes
- MINOR: spoa-server: With debug mode, start only one process
- MINOR: spoa-server: Allow registering external processes
- MINOR: spoa-server: Allow registering message processors
- MINOR: spoa-server: Load files
- MINOR: spoa-server: Prepare responses
- MINOR: spoa-server: Execute registered callbacks
- MINOR: spoa-server: Add Lua processing
- MINOR: spoa-server: Add python
- MINOR/DOC: spoe-server: Add documentation
- BUG/MEDIUM: connections: Don't forget to set xprt_ctx to NULL on close.
- MINOR: lists: add LIST_ADDED() to check if an element belongs to a list
- CLEANUP: mux-h2: use LIST_ADDED() instead of LIST_ISEMPTY() where relevant
- MINOR: mux-h2: add two H2S flags to report the need for shutr/shutw
- CLEANUP: mux-h2: simply use h2s->flags instead of ret in h2_deferred_shut()
- CLEANUP: connection: remove the handle field from the wait_event struct
- BUG/MINOR: log: Wrong log format initialization.
- BUG/MINOR: mux-h2: make the do_shut{r,w} functions more robust against retries
- BUG/MINOR: mworker: use after free when the PID not assigned
- MINOR: mux-h2: remove useless test on stream ID vs last in wake function
- MINOR: mux-h2: make h2_wake_some_streams() not depend on the CS flags
- MINOR: mux-h2: make h2s_wake_one_stream() the only function to deal with CS
- MINOR: mux-h2: make h2s_wake_one_stream() not depend on temporary CS flags
- BUG/MINOR: mux-h2: make sure to honor KILL_CONN in do_shut{r,w}
- CLEANUP: mux-h2: don't test for impossible CS_FL_REOS conditions
- MINOR: mux-h2: add macros to check multiple stream states at once
- MINOR: mux-h2: stop relying on CS_FL_REOS
- BUG/MEDIUM: mux-h2: Set EOI on the conn_stream during h2_rcv_buf()
- BUILD: debug: make gcc not complain on the ABORT_NOW() macro
- MINOR: debug: add a new BUG_ON macro
- MINOR: h2: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
- MINOR: h1: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
- MINOR: connections: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
- BUILD: ist: turn the lower/upper case tables to literal on obsolete linkers
Add a new retry-on keyword, "all-retryable-errors", that activates retry
for all errors that are considered retryable.
This currently activates retry for "conn-failure", "empty-response",
"junk-respones", "response-timeout", "0rtt-rejected", "500", "502", "503" and
"504".
Add a new action for http-request, disable-l7-retry, that can be used to
disable any attempt at retry requests (see retry-on) if it fails for any
reason other than a connection failure.
This is useful for example to make sure POST requests aren't retried.
This patch implements a new global parameter for the master-worker mode.
When setting the mworker-max-reloads value, a worker receive a SIGTERM
if its number of reloads is greater than this value.
This implements support for the new API which relies on a call to
setsockopt().
On systems that support it (currently, only Linux >= 4.11), this enables
using TCP fast open when connecting to server.
Please note that you should use the retry-on "conn-failure", "empty-response"
and "response-timeout" keywords, or the request won't be able to be retried
on failure.
Co-authored-by: Olivier Houchard <ohouchard@haproxy.com>
I was about to partly revert 294d0f08b3,
because there were no 'X' for 'appsession' in the keyword matrix until
I checked the blame, realizing that the feature does not exist any more.
Clearly the documentation is confusing here, the removal note is only
listed *below* the old documentation and the supported sections still
show 'backend' and 'listen'.
It's been 3.5 years and 4 releases (1.6, 1.7, 1.8 and 1.9), I guess
this can be removed from the documentation of future versions.
Add a way to retry requests if we got a junk response from the server, ie
an incomplete response, or something that is not valid HTTP.
To do so, one can use the new "junk-response" keyword for retry-on.
Add a new keyword for retry-on, 0rtt-rejected. If set, we will try to
replay requests for which we sent early data that got rejected by the
server.
If that option is set, we will attempt to use 0rtt if "allow-0rtt" is set
on the server line even if the client didn't send early data.
When running in HTX mode, if we sent the request, but failed to get the
answer, either because the server just closed its socket, we hit a server
timeout, or we get a 404, 408, 425, 500, 501, 502, 503 or 504 error,
attempt to retry the request, exactly as if we just failed to connect to
the server.
To do so, add a new backend keyword, "retry-on".
It accepts a list of keywords, which can be "none" (never retry),
"conn-failure" (we failed to connect, or to do the SSL handshake),
"empty-response" (the server closed the connection without answering),
"response-timeout" (we timed out while waiting for the server response),
or "404", "408", "425", "500", "501", "502", "503" and "504".
The default is "conn-failure".
This document should come with these commits:
'MINOR: log: Enable the log sampling and load-balancing feature'
'MINOR: log: Add "sample" new keyword to "log" lines.'
Instead of having to manually turn task profiling on/off in the
configuration, by default it will work in "auto" mode, which
automatically turns on on any thread experiencing sustained loop
latencies over one millisecond averaged over the last 1024 samples.
This may happen with configs using lots of regex (thing map_reg for
example, which is the lazy way to convert Apache's rewrite rules but
must not be abused), and such high latencies affect all the process
and the problem is most often intermittent (e.g. hitting a map which
is only used for certain host names).
Thus now by default, with profiling set to "auto", it remains off all
the time until something bad happens. This also helps better focus on
the issues when looking at the logs as well as in "show sess" output.
It automatically turns off when the average loop latency over the last
1024 calls goes below 990 microseconds (which typically takes a while
when in idle).
This patch could be backported to stable versions after a bit more
exposure, as it definitely improves observability and the ability to
quickly spot the culprit. In this case, previous patch ("MINOR:
activity: make the profiling status per thread and not global") must
also be taken.
The 'do-resolve' action is an http-request or tcp-request content action
which allows to run DNS resolution at run time in HAProxy.
The name to be resolved can be picked up in the request sent by the
client and the result of the resolution is stored in a variable.
The time the resolution is being performed, the request is on pause.
If the resolution can't provide a suitable result, then the variable
will be empty. It's up to the admin to take decisions based on this
statement (return 503 to prevent loops).
Read carefully the documentation concerning this feature, to ensure your
setup is secure and safe to be used in production.
This patch creates a global counter to track various errors reported by
the action 'do-resolve'.
This is dummy version of the Scientiamobile WURFL C API that can be used
to successfully build/run haproxy compiled with USE_WURFL=1.
It is marked as version 1.11.2.100 to distinguish it from any real version
of the lib. It has no external dependencies so it should work out of the
box by building it like this :
$ make -C contrib/wurfl
In order to use it, simply reference this directory as the WURFL include
and library paths :
$ make TARGET=<target> USE_WURFL=1 WURFL_INC=$PWD/contrib/wurfl WURFL_LIB=$PWD/contrib/wurfl
last 2 major releases of libwurfl included a complete review of engine options with
the result of deprecating many features. The patch removes unecessary code and fixes
the documentation.
Can be backported on any version of haproxy.
[wt: must not be backported since it removes config keywords and would
thus break existing configurations]
Signed-off-by: Willy Tarreau <w@1wt.eu>
The set-dst and set dst-var are available at both 'tcp-request
connection' and 'http-request' but not at the layer in the middle.
This patch fixes this miss and enables both set-dst and set-dst-var at
'tcp-request content' layer.
As by default we add all keepalive connections to the idle pool, if we run
into a pathological case, where all client don't do keepalive, but the server
does, and haproxy is configured to only reuse "safe" connections, we will
soon find ourself having lots of idling, unusable for new sessions, connections,
while we won't have any file descriptors available to create new connections.
To fix this, add 2 new global settings, "pool_low_ratio" and "pool_high_ratio".
pool-low-fd-ratio is the % of fds we're allowed to use (against the maximum
number of fds available to haproxy) before we stop adding connections to the
idle pool, and destroy them instead. The default is 20. pool-high-fd-ratio is
the % of fds we're allowed to use (against the maximum number of fds available
to haproxy) before we start killing idling connection in the event we have to
create a new outgoing connection, and no reuse is possible. The default is 25.
This patch adds a counter of calls on the orchestator peers task
and a counter on the tasks linked to applet i/o handler for
each peer.
Those two counters are useful to detect if a peer sync is active
or frozen.
This patch is related to the commit:
"MINOR: peers: Add a new command to the CLI for peers."
and should be backported with it.
It's always a pain to get a core dump when enabling user/group setting
(which disables the dumpable flag on Linux), when using a chroot and/or
when haproxy is started by a service management tool which requires
complex operations to just raise the core dump limit.
This patch introduces a new "set-dumpable" global directive to work
around these troubles by doing the following :
- remove file size limits (equivalent of ulimit -f unlimited)
- remove core size limits (equivalent of ulimit -c unlimited)
- mark the process dumpable again (equivalent of suid_dumpable=1)
Some of these will depend on the operating system. This way it becomes
much easier to retrieve a core file. Temporarily moving the chroot to
a user-writable place generally enough.
The legacy HTTP mode is no more the default one. So now, by default, without any
option in your configuration, all proxies will use the HTX mode. The line
"option http-use-htx" in proxy sections are now useless, except to cancel the
legacy HTTP mode. To fallback on legacy HTTP mode, you should use the line "no
option http-use-htx" explicitly.
Note that the reg-tests still work by default on legacy HTTP mode. The HTX will
be enabled by default in a futur commit.
The option http-tunnel disables any HTTP processing past the first
transaction. In HTX, it works for full h1 transactions. As for the legacy HTTP,
it is a workaround, but it works. But it is impossible to make it works with an
h2 connection. In such case, it has no effect, the stream is closed at the end
of the transaction. So to avoid any inconsistancies between h1 and h2
connections, this option is now always ignored when the HTX is enabled. It is
also a good opportinity to deprecate an old and ugly option. A warning is
emitted during HAProxy startup to encourage users to remove this option.
Note that in legacy HTTP, this option only works with full h1 transactions
too. If an h2 connection is established on a frontend with this option enabled,
it will have no effect at all. But we keep it for the legacy HTTP for
compatibility purpose. It will be removed with the legacy HTTP.
So to be short, if you have to really (REALLY) use it, it will only work for
legacy HTTP frontends with H1 clients.
The documentation has been updated accordingly.
This patch must be backported to 1.9. It is not strictly speaking required but
it will ease futur backports.
The converter can be used to decrypt the raw byte input using the
AES-GCM algorithm, using provided nonce, key and AEAD tag. This can
be useful to decrypt encrypted cookies for example and make decisions
based on the content.
Released version 2.0-dev2 with the following main changes :
- CLEANUP: http: Remove unreachable code in parse_http_req_capture
- CLEANUP: stream: Remove bogus loop in conn_si_send_proxy
- MINOR: lists: Implement locked variations.
- MEDIUM: servers: Used a locked list for idle_orphan_conns.
- MEDIUM: servers: Reorganize the way idle connections are cleaned.
- BUG/MEDIUM: lists: Properly handle the case we're removing the first elt.
- MINOR: cfgparse: Add a cast to make gcc happier.
- BUG/MEDIUM: standard: Wrong reallocation size.
- BUG/MINOR: listener: keep accept rate counters accurate under saturation
- DOC: fix alphabetic ordering for "tune.fail-alloc" setting
- MAJOR: config: disable support for nbproc and nbthread in parallel
- MEDIUM: listener: keep a single thread-mask and warn on "process" misuse
- MAJOR: listener: do not hold the listener lock in listener_accept()
- MINOR: listener: maintain a per-thread count of the number of connections on a listener
- MINOR: tools: implement functions to look up the nth bit set in a mask
- MINOR: listener: pre-compute some thread counts per bind_conf
- MINOR: listener: implement multi-queue accept for threads
- MAJOR: listener: use the multi-queue for multi-thread listeners
- MINOR: activity: add accept queue counters for pushed and overflows
- MINOR: config: add global tune.listener.multi-queue setting
- MAJOR: threads: enable one thread per CPU by default
- DOC: update management.txt to reflect that threads are used by default
- BUG/MINOR: config: don't over-count the global maxsock value
- BUG/MEDIUM: list: fix the rollback on addq in the locked liss
- BUG/MEDIUM: list: fix LIST_POP_LOCKED's removal of the last pointer
- BUG/MEDIUM: list: add missing store barriers when updating elements and head
- MINOR: list: make the delete and pop operations idempotent
- MINOR: server: remove a few unneeded LIST_INIT calls after LIST_DEL_LOCKED
- BUG/MEDIUM: listener: use a self-locked list for the dequeue lists
- BUG/MEDIUM: listener: make sure the listener never accepts too many conns
- BUG/MEDIUM: list: correct fix for LIST_POP_LOCKED's removal of last element
- MINOR: listener: introduce listener_backlog() to report the backlog value
- MINOR: listener: do not needlessly set l->maxconn
- MINOR: proxy: do not change the listeners' maxconn when updating the frontend's
- MEDIUM: config: don't enforce a low frontend maxconn value anymore
- MINOR: peers: Add a message for heartbeat.
- MINOR: global: keep a copy of the initial rlim_fd_cur and rlim_fd_max values
- BUG/MINOR: init: never lower rlim_fd_max
- BUG/MINOR: checks: make external-checks restore the original rlim_fd_cur/max
- BUG/MINOR: mworker: be careful to restore the original rlim_fd_cur/max on reload
- MINOR: init: make the maxpipe computation more accurate
- MINOR: init: move some maxsock updates earlier
- MEDIUM: init: make the global maxconn default to what rlim_fd_cur permits
- REGTEST: fix a spurious "nbthread 4" in the connection test
- DOC: update the text related to the global maxconn value
- BUG/MAJOR: mux-h2: fix race condition between close on both ends
- MINOR: sample: Replace "req.ungrpc" smp fetch by a "ungrpc" converter.
- BUG/MEDIUM: list: fix again LIST_ADDQ_LOCKED
- MINOR: htx: unconditionally handle parsing errors in requests or responses
- MINOR: mux-h2: always pass HTX_FL_PARSING_ERROR between h2s and buf on RX
- BUG/MEDIUM: h2/htx: verify that :path doesn't contain invalid chars
- MINOR: sample: Code factorization "ungrpc" converter.
- MINOR: sample: Rework gRPC converter code.
- CLEANUP: wurfl: remove dead, broken and unmaintained code
- MINOR: config: relax the range checks on cpu-map
- BUG/MINOR: ssl: fix warning about ssl-min/max-ver support
- MINOR: sample: Extract some protocol buffers specific code.
- DOC: Remove tabs and fixed punctuation.
- MINOR: sample: Add a protocol buffers specific converter.
- REGTEST: Peers reg tests.
- REGTEST: Enable reg tests with HEAD HTTP method usage.
- MINOR: lists: add a LIST_DEL_INIT() macro
- MINOR: task: use LIST_DEL_INIT() to remove a task from the queue
- MINOR: listener: improve incoming traffic distribution
- MINOR: tools: implement my_flsl()
- MEDIUM: listener: change the LB algorithm again to use two round robins instead
- CLEANUP: listener: remove old thread bit mapping
- MINOR: listener: move thr_idx from the bind_conf to the listener
- BUG/MEDIUM: logs: Only attempt to free startup_logs once.
- BUG/MAJOR: config: Wrong maxconn adjustment.
- BUG/MEDIUM: 51d: fix possible segfault on deinit_51degrees()
- OPTIM: task: limit the impact of memory barriers in taks_remove_from_task_list()
- MINOR: fd: Remove debugging code.
- BUG/MEDIUM: listeners: Don't call fd_stop_recv() if fd_updt is NULL.
- MINOR: threads: Implement __ha_barrier_atomic*.
- MEDIUM: threads: Use __ATOMIC_SEQ_CST when using the newer atomic API.
- MINOR: threads: Add macros to do atomic operation with no memory barrier.
- MEDIUM: various: Use __ha_barrier_atomic* when relevant.
- MEDIUM: applets: Use the new _HA_ATOMIC_* macros.
- MEDIUM: xref: Use the new _HA_ATOMIC_* macros.
- MEDIUM: fd: Use the new _HA_ATOMIC_* macros.
- MEDIUM: freq_ctr: Use the new _HA_ATOMIC_* macros.
- MEDIUM: proxy: Use the new _HA_ATOMIC_* macros.
- MEDIUM: server: Use the new _HA_ATOMIC_* macros.
- MEDIUM: task: Use the new _HA_ATOMIC_* macros.
- MEDIUM: activity: Use the new _HA_ATOMIC_* macros.
- MEDIUM: backend: Use the new _HA_ATOMIC_* macros.
- MEDIUM: cache: Use the new _HA_ATOMIC_* macros.
- MEDIUM: checks: Use the new _HA_ATOMIC_* macros.
- MEDIUM: pollers: Use the new _HA_ATOMIC_* macros.
- MEDIUM: compression: Use the new _HA_ATOMIC_* macros.
- MEDIUM: spoe: Use the new _HA_ATOMIC_* macros.
- MEDIUM: threads: Use the new _HA_ATOMIC_* macros.
- MEDIUM: http: Use the new _HA_ATOMIC_* macros.
- MEDIUM: lb/threads: Use the new _HA_ATOMIC_* macros.
- MEDIUM: listeners: Use the new _HA_ATOMIC_* macros.
- MEDIUM: logs: Use the new _HA_ATOMIC_* macros.
- MEDIUM: memory: Use the new _HA_ATOMIC_* macros.
- MEDIUM: peers: Use the new _HA_ATOMIC_* macros.
- MEDIUM: proto_tcp: Use the new _HA_ATOMIC_* macros.
- MEDIUM: queues: Use the new _HA_ATOMIC_* macros.
- MEDIUM: sessions: Use the new _HA_ATOMIC_* macros.
- MEDIUM: ssl: Use the new _HA_ATOMIC_* macros.
- MEDIUM: stream: Use the new _HA_ATOMIC_* macros.
- MEDIUM: tcp_rules: Use the new _HA_ATOMIC_* macros.
- MEDIUM: time: Use the new _HA_ATOMIC_* macros.
- MEDIUM: vars: Use the new _HA_ATOMIC_* macros.
- MINOR: config: remove obsolete use of DEFAULT_MAXCONN at various places
- MINOR: config: continue to rely on DEFAULT_MAXCONN to set the minimum maxconn
- BUG/MEDIUM: list: fix incorrect pointer unlocking in LIST_DEL_LOCKED()
- BUG/MEDIUM: listener: make sure we don't pick stopped threads
- MEDIUM: list: Remove useless barriers.
- MEDIUM: list: Use _HA_ATOMIC_*
- MEDIUM: connections: Use _HA_ATOMIC_*
- BUG/MAJOR: tasks: Use the TASK_GLOBAL flag to know if we're in the global rq.
- BUG/MEDIUM: threads/fd: do not forget to take into account epoll_fd/pipes
- BUG/MEDIUM: init/threads: consider epoll_fd/pipes for automatic maxconn calculation
- BUG/MEDIUM: tasks: Make sure we wake sleeping threads if needed.
- BUG/MINOR: mux-h1: Don't report an error on EOS if no message was received
- BUG/MINOR: stats/htx: Call channel_add_input() when response headers are sent
- BUG/MINOR: lua/htx: Use channel_add_input() when response data are added
- BUG/MINOR: lua/htx: Don't forget to call htx_to_buf() when appropriate
- MINOR: stats: Add the status code STAT_STATUS_IVAL to handle invalid requests
- MINOR: stats: Move stuff about the stats status codes in stats files
- BUG/MINOR: stats: Be more strict on what is a valid request to the stats applet
- Revert "REGTEST: Enable reg tests with HEAD HTTP method usage."
- BUILD: listener: shut up a build warning when threads are disabled
- BUILD: Makefile: allow the reg-tests target to be verbose
- BUILD: Makefile: resolve LEVEL before calling run-regtests
- BUG/MAJOR: spoe: Fix initialization of thread-dependent fields
- BUG/MAJOR: stats: Fix how huge POST data are read from the channel
- BUG/MINOR: http/counters: fix missing increment of fe->srv_aborts
- BUG/MEDIUM: mux-h2: Always wakeup streams with no id to avoid frozen streams
- MINOR: mux-h2: Set REFUSED_STREAM error to reset a stream if no data was never sent
- MINOR: muxes: Report the Last read with a dedicated flag
- MINOR: proto-http/proto-htx: Make error handling clearer during data forwarding
- BUILD: tools: fix a build warning on some 32-bit archs
- MINOR: init: report the list of optionally available services
- MEDIUM: proto_htx: Switch to infinite forwarding if there is no data filter
- BUG/MINOR: cache: Fully consume large requests in the cache applet
- BUG/MINOR: stats: Fully consume large requests in the stats applet
- BUG/MEDIUM: lua: Fully consume large requests when an HTTP applet ends
- MINOR: proto_http: Add function to handle the header "Expect: 100-continue"
- MINOR: proto_htx: Add function to handle the header "Expect: 100-continue"
- MINOR: stats/cache: Handle the header Expect when applets are registered
- MINOR: http/applets: Handle all applets intercepting HTTP requests the same way
- CLEANUP: cache: don't export http_cache_applet anymore
- MINOR: lua: Don't handle the header Expect in lua HTTP applets anymore
- BUG/MINOR: doc: Be accurate on the behavior on pool-purge-delay.
- Revert "MEDIUM: proto_htx: Switch to infinite forwarding if there is no data filter"
- BUG/MEDIUM: mux-h2: Make sure we destroyed the h2s once shutr/shutw is done.
- BUG/MEDIUM: mux-h2: Don't bother keeping the h2s if detaching and nothing to send.
- BUG/MEDIUM: mux-h2: Use the right list in h2_stop_senders().
- MINOR: mux-h2: copy small data blocks more often and reduce the number of pauses
- CLEANUP: mux-h2: add some comments to help understand the code
- BUG/MEDIUM: ssl: ability to set TLS 1.3 ciphers using ssl-default-server-ciphersuites
- BUG/MINOR: log: properly format IPv6 address when LOG_OPT_HEXA modifier is used.
- BUG/MEDIUM: h2: Try to be fair when sending data.
- BUG/MINOR: proto-http: Don't forward request body anymore on error
- MINOR: mux-h2: Remove useless test on ES flag in h2_frt_transfer_data()
- MINOR: connection: and new flag to mark end of input (EOI)
- MINOR: channel: Report EOI on the input channel if it was reached in the mux
- MEDIUM: mux-h2: Don't mix the end of the message with the end of stream
- MINOR: mux-h1: Set CS_FL_EOI the end of the message is reached
- BUG/MEDIUM: http/htx: Fix handling of the option abortonclose
- CLEANUP: muxes/stream-int: Remove flags CS_FL_READ_NULL and SI_FL_READ_NULL
- MEDIUM: proto_htx: Reintroduce the infinite forwarding on data
- BUG/MEDIUM: h2: only destroy the h2s if h2s->cs is NULL.
- BUG/MEDIUM: h2: Use the new sending_list in h2s_notify_send().
- BUG/MEDIUM: h2: Follow the same logic in h2_deferred_shut than in h2_snd_buf.
- BUG/MEDIUM: h2: Remove the tasklet from the task list if unsubscribing.
- BUG/MEDIUM: task/h2: add an idempotent task removal fucntion
- CLEANUP: task: only perform a LIST_DEL() when the list is not empty
- BUG/MEDIUM: mux-h2: make sure to always notify streams of EOS condition
- CONTRIB: debug: report the CS and CF's EOI flags
- MINOR: channel: don't unset CF_SHUTR_NOW after shutting down.
This patch adds "protobuf" protocol buffers specific converter wich
may used in combination with "ungrpc" as first converter to extract
a protocol buffers field value. It is simply implemented reusing
protobuf_field_lookup() which is the protocol buffers specific parser already
used by "ungrpc" converter which only parse a gRPC header in addition of
parsing protocol buffers message.
Update the documentation for this new "protobuf" converter.
Since the "wurfl" device detection engine was merged slightly more than
two years ago (2016-11-04), it never received a single fix nor update.
For almost two years it didn't receive even the minimal review or changes
needed to be compatible with threads, and it's remained build-broken for
about the last 9 months, consecutive to the last buffer API changes,
without anyone ever noticing! When asked on the list, nobody confirmed
using it :
https://www.mail-archive.com/haproxy@formilux.org/msg32516.html
And obviously nobody even cared to verify that it did still build. So we
are left with this broken code with no user and no maintainer. It might
even suffer from remotely exploitable vulnerabilities without anyone
being able to check if it presents any risk. It's a pain to update each
time there is an API change because it doesn't build as it depends on
external libraries that are not publicly accessible, leading to careful
blind changes. It slows down the whole project. This situation is not
acceptable at all.
It's time to cure the problem where it is. This patch removes all this
dead, non-buildable, non-working code. If anyone ever decides to use it,
which I seriously doubt based on history, it could be reintegrated, but
this time the following guarantees will be required :
- someone has to step up as a maintainer and have his name listed in
the MAINTAINERS file (I should have been more careful last time).
This person will take the sole blame for all issues and will be
responsible for fixing the bugs and incompatibilities affecting
this code, and for making it evolve to follow regular internal API
updates.
- support building on a standard distro with automated tools (i.e. no
more "click on this site, register your e-mail and download an
archive then figure how to place this into your build system").
Dummy libs are OK though as long as they allow the mainline code to
build and start.
- multi-threaded support must be fixed. I mean seriously, not worked
around with a check saying "please disable threads, we've been busy
fishing for the last two years".
This may be backported to 1.9 given that the code has never worked there
either, thus at least we're certain nobody will miss it.
For now on, "ungrpc" may take a second optional argument to provide
the protocol buffers types used to encode the field value to be extracted.
When absent the field value is extracted as a binary sample which may then
followed by others converters like "hex" which takes binary as input sample.
When this second argument is a type which does not match the one found by "ungrpc",
this field is considered as not found even if present.
With this patch we also remove the useless "varint" and "svarint" converters.
Update the documentation about "ungrpc" converters.
This patch simply extracts the code of smp_fetch_req_ungrpc() for "req.ungrpc"
from http_fetch.c to move it to sample.c with very few modifications.
Furthermore smp_fetch_body_buf() used to fetch the body contents is no more needed.
Update the documentation for gRPC.
Historically the default frontend's maxconn used to be quite low (2000),
which was sufficient two decades ago but often proved to be a problem
when users had purposely set the global maxconn value but forgot to set
the frontend's.
There is no point in keeping this arbitrary limit for frontends : when
the global maxconn is lower, it's already too high and when the global
maxconn is much higher, it becomes a limiting factor which causes trouble
in production.
This commit allows the value to be set to zero, which becomes the new
default value, to mean it's not directly limited, or in fact it's set
to the global maxconn. Since this operation used to be performed before
computing a possibly automatic global maxconn based on memory limits,
the calculation of the maxconn value and its propagation to the backends'
fullconn has now moved to a dedicated function, proxy_adjust_all_maxconn(),
which is called once the global maxconn is stabilized.
This comes with two benefits :
1) a configuration missing "maxconn" in the defaults section will not
limit itself to a magically hardcoded value but will scale up to the
global maxconn ;
2) when the global maxconn is not set and memory limits are used instead,
the frontends' maxconn automatically adapts, and the backends' fullconn
as well.
In an attempt to try to provide automatic maxconn settings, we need to
decorrelate a listner's backlog and maxconn so that these values can be
independent. This introduces a listener_backlog() function which retrieves
the backlog value from the listener's backlog, the frontend's, the
listener's maxconn, the frontend's or falls back to 1024. This
corresponds to what was done in cfgparse.c to force a value there except
the last fallback which was not set since the frontend's maxconn is always
known.
