In H2 the conditions to create a new stream differ for a client and a server when a GOAWAY was exchanged. While on the server, any stream whose ID is lower than or equal to the one advertised in GOAWAY is valid, for a client it's forbidden to create any stream after receipt of a GOAWAY, even if its ID is lower than or equal to the last one, despite the server not being able to tell the difference from the number of streams in flight. Unfortunately, the logic in the code did not always reflect this specificity of the client (the backend code in our case), and most often considered that it was still permitted to create a new stream until the max_id was greater than or equal to the advertised last_id. This is for example what h2c_is_dead() and h2c_streams_left() do. In other places, such as h2_avail_streams(), the rule is properly taken into account. Very often the advertised last_id is the same, and this is also what haproxy does (which explains why it's impossible to reproduce the issue by chaining two haproxy layers), but a server may wish to advertise any ID including 2^31-1 as mentioned in the spec, and in this case the functions would behave differently. This discrepancy results in a corner case where a GOAWAY received on an idle connection will cause the next stream creation to be initially accepted but then rejected via h2_avail_streams(), and the connection left in a bad state, still attached to the session due to http-reuse safe, but not reinserted into idle list, since the backend code currently is not able to properly recover from this situation. Worse, the idle flags are no longer on it but TASK_F_USR1 still is, and this makes the recently added BUG_ON() rightfully trigger since this case is not supposed to happen. Admittedly more of the backend recovery code needs to be reworked, however the mux must consistently decide whether or not a connection may be reused or needs to be released. This commit fixes the affected logic by introducing a new function "h2c_reached_last_stream()" which says if a connection has reached its last stream, regardless of the side, and using this one everywhere max_id was compared to last_id. This is sufficient to address the corner case that be_reuse_connection() currently cannot recover from. This is in relation to GH issue #3215 and it should be sufficient to fix the issue there. Thanks to Chris Staite for reporting the issue and kudos to Amaury for spotting the events sequence that can lead to this situation. This patch must be backported to 3.3 first, then to older versions later. It's worth noting that it's much more difficult to observe the issue before 3.3 because the BUG_ON() is not there, and the possibly non-released connection might end up being killed for other reasons (timeouts etc). But one possible visible effect might be the impossibility to delete a server (which Chris observed in 3.3).
HAProxy
HAProxy is a free, very fast and reliable reverse-proxy offering high availability, load balancing, and proxying for TCP and HTTP-based applications.
Installation
The INSTALL file describes how to build HAProxy. A list of packages is also available on the wiki.
Getting help
The discourse and the mailing-list are available for questions or configuration assistance. You can also use the slack or IRC channel. Please don't use the issue tracker for these.
The issue tracker is only for bug reports or feature requests.
Documentation
The HAProxy documentation has been split into a number of different files for ease of use. It is available in text format as well as HTML. The wiki is also meant to replace the old architecture guide.
Please refer to the following files depending on what you're looking for:
- INSTALL for instructions on how to build and install HAProxy
- BRANCHES to understand the project's life cycle and what version to use
- LICENSE for the project's license
- CONTRIBUTING for the process to follow to submit contributions
The more detailed documentation is located into the doc/ directory:
- doc/intro.txt for a quick introduction on HAProxy
- doc/configuration.txt for the configuration's reference manual
- doc/lua.txt for the Lua's reference manual
- doc/SPOE.txt for how to use the SPOE engine
- doc/network-namespaces.txt for how to use network namespaces under Linux
- doc/management.txt for the management guide
- doc/regression-testing.txt for how to use the regression testing suite
- doc/peers.txt for the peers protocol reference
- doc/coding-style.txt for how to adopt HAProxy's coding style
- doc/internals for developer-specific documentation (not all up to date)
License
HAProxy is licensed under GPL 2 or any later version, the headers under LGPL 2.1. See the LICENSE file for a more detailed explanation.
