mirrors/haproxy
1
0
Fork 0
mirror of https://git.haproxy.org/git/haproxy.git/ synced 2025-08-18 13:06:58 +02:00
Code Issues Projects Releases Wiki Activity
f210191dcd
haproxy/src
History
Willy Tarreau f210191dcd BUG/MEDIUM: h2: don't accept new streams if conn_streams are still in excess 
The streams bookkeeping made in H2 is used for protocol compliance only
but it doesn't consider the number of conn_streams still attached to the
mux. It causes an issue when http-request set-nice rules are applied on
H2 requests processed on a saturated machine. Indeed, in this case, the
requests are accepted and assigned a default nice value of zero. When
they are processed, their nice value changes to a higher one (say 1024).
The response is sent through the H2 mux, which detects the end of stream
and decrements the protocol-level stream count (h2c->nb_streams). The
client may then send a new request. But the conn_stream is still attached
and will require a new call to process_stream() to finish, which is made
through the scheduler. Given that the machine is saturated, it is assumed
that many tasks are present in the scheduler. Thus the closing tasks holding
a higher nice value will pass after the new stream creations. If the client
is fast enough with a low latency link, it may add a lot of new stream
creations before the stream terminations have a chance to disappear due
to their high nice value, resulting in a huge amount of memory being used.

The solution consists in letting a mux always monitor its conn_streams and
refrain from creating new ones when it is full. Here the H2 mux checks the
nb_cs counter and sets a new blocked flag (H2_CF_DEM_TOOMANY) if the limit
was reached, so that the frame parser requests a pause in the new stream
creation, leaving some time for the pending conn_streams to vanish.

Several experiments were made using varying thresholds to see if
overbooking would provide any benefit here but it turned out not to be
the case, so the conn_stream limit remains set to the exact streams
limit. Interestingly various performance measurements showed that the
code tends to be slightly faster now than without the limit, probably
due to the smoother memory usage.

This commit requires previous patch ("MINOR: h2: keep a count of the number
of conn_streams attached to the mux"). It needs to be backported to 1.8.
2018-07-19 10:23:15 +02:00
..
51d.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
acl.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
action.c MINOR: action: Add function to check rules using an action ACT_ACTION_TRK_* 2017-10-31 11:36:12 +01:00
applet.c MAJOR: applets: Use tasks, instead of rolling our own scheduler. 2018-05-26 20:03:30 +02:00
arg.c MINOR: config: Add support for ARGT_MSK6 2018-01-25 22:25:40 +01:00
auth.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
backend.c MINOR: backend: implement random-based load balancing 2018-05-03 07:20:40 +02:00
base64.c [MINOR] add encode/decode function for 30-bit integers from/to base64 2010-10-30 19:04:33 +02:00
buffer.c DOC: buffers: clarify the purpose of the <from> pointer in offer_buffers() 2018-03-08 18:33:48 +01:00
cache.c MEDIUM: cli: Add payload support 2018-04-26 14:19:33 +02:00
cfgparse.c MINOR: dns: Implement parse-resolv-conf directive 2018-05-30 05:17:16 +02:00
channel.c BUG/MAJOR: channel: Fix crash when trying to read from a closed socket 2018-04-25 05:39:49 +02:00
checks.c BUG/MINOR: signals: ha_sigmask macro for multithreading 2018-06-08 18:24:53 +02:00
chunk.c CLEANUP: pools: rename all pool functions and pointers to remove this "2" 2017-11-24 17:49:53 +01:00
cli.c MAJOR: applets: Use tasks, instead of rolling our own scheduler. 2018-05-26 20:03:30 +02:00
compression.c CLEANUP: pools: rename all pool functions and pointers to remove this "2" 2017-11-24 17:49:53 +01:00
connection.c MINOR: accept-proxy: support proxy protocol v2 CRC32c checksum 2018-03-21 05:04:01 +01:00
da.c BUILD/MINOR: deviceatlas: enable thread support 2017-11-27 14:22:21 +01:00
dns.c MINOR: dns: new DNS options to allow/prevent IP address duplication 2018-07-12 17:56:44 +02:00
ev_epoll.c MINOR: pollers: move polled_mask outside of struct fdtab. 2018-05-06 06:27:34 +02:00
ev_kqueue.c BUG/MEDIUM: pollers/kqueue: use incremented position in event list 2018-05-11 14:08:56 +02:00
ev_poll.c MINOR: pollers: move polled_mask outside of struct fdtab. 2018-05-06 06:27:34 +02:00
ev_select.c MINOR: pollers: move polled_mask outside of struct fdtab. 2018-05-06 06:27:34 +02:00
fd.c BUG/MEDIUM: fd: Don't modify the update_mask in fd_dodelete(). 2018-06-20 10:21:44 +02:00
filters.c CLEANUP: pools: rename all pool functions and pointers to remove this "2" 2017-11-24 17:49:53 +01:00
flt_http_comp.c CLEANUP: pools: rename all pool functions and pointers to remove this "2" 2017-11-24 17:49:53 +01:00
flt_spoe.c MAJOR: spoe: upgrade the SPOP version to 2.0 and remove the support for 1.0 2018-06-04 17:33:48 +02:00
flt_trace.c MINOR: threads/filters: Update trace filter to add _per_thread callbacks 2017-10-31 13:58:32 +01:00
freq_ctr.c BUG/MAJOR: threads/freq_ctr: use a memory barrier to detect changes 2017-10-31 18:01:18 +01:00
frontend.c MINOR: proxy: Add fe_defbe fetcher 2018-04-16 15:51:57 +02:00
h1.c BUG/MINOR: h1: the HTTP/1 make status code parser check for digits 2017-11-09 11:15:45 +01:00
h2.c BUG/MEDIUM: h2: implement missing support for chunked encoded uploads 2018-04-26 10:20:44 +02:00
haproxy.c MINOR: startup: change session/process group settings 2018-07-04 19:29:56 +02:00
hash.c MINOR: hash: add new function hash_crc32c 2018-03-21 05:04:01 +01:00
hathreads.c BUG/MEDIUM: pollers: Use a global list for fd shared between threads. 2018-05-06 06:27:09 +02:00
hdr_idx.c CLEANUP: pools: rename all pool functions and pointers to remove this "2" 2017-11-24 17:49:53 +01:00
hlua_fcn.c BUG/MINOR: lua: ensure large proxy IDs can be represented 2018-05-06 14:50:09 +02:00
hlua.c BUG/MEDIUM: lua: possible CLOSE-WAIT state with '\n' headers 2018-07-01 06:08:43 +02:00
hpack-dec.c DEBUG: hpack: add more traces to the hpack decoder 2017-12-30 17:37:08 +01:00
hpack-enc.c MEDIUM: hpack: implement basic hpack encoding 2017-10-31 18:03:24 +01:00
hpack-huff.c BUG/MINOR: hpack: must reject huffman literals padded with more than 7 bits 2017-12-03 21:08:39 +01:00
hpack-tbl.c BUG/MINOR: hpack: fix harmless use of uninitialized value in hpack_dht_insert 2018-03-27 20:05:13 +02:00
i386-linux-vsys.c MEDIUM: listener: add support for linux's accept4() syscall 2012-10-08 20:11:03 +02:00
lb_chash.c BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix 2017-11-07 11:10:24 +01:00
lb_fas.c BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix 2017-11-07 11:10:24 +01:00
lb_fwlc.c BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix 2017-11-07 11:10:24 +01:00
lb_fwrr.c BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix 2017-11-07 11:10:24 +01:00
lb_map.c BUG: MAJOR: lb_map: server map calculation broken 2017-12-14 17:36:39 +01:00
listener.c BUG/MINOR: listener: Don't decrease actconn twice when a new session is rejected 2018-03-23 16:21:50 +01:00
log.c MINOR: log: Keep the ref when a log server is copied to avoid duplicate entries 2018-04-05 15:13:54 +02:00
lru.c MINOR: lru: new function to delete <nb> least recently used keys 2016-01-11 07:31:35 +01:00
mailers.c MEDIUM: Add parsing of mailers section 2015-02-03 00:24:16 +01:00
map.c BUG/MINOR: map: correctly track reference to the last ref_elt being dumped 2018-05-04 17:14:39 +02:00
memory.c BUG/MINOR: pools/threads: don't ignore DEBUG_UAF on double-word CAS capable archs 2018-02-22 14:18:45 +01:00
mux_h2.c BUG/MEDIUM: h2: don't accept new streams if conn_streams are still in excess 2018-07-19 10:23:15 +02:00
mux_pt.c BUG/MAJOR: connection: refine the situations where we don't send shutw() 2017-12-22 18:54:05 +01:00
namespace.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
pattern.c BUG/MAJOR: map: fix a segfault when using http-request set-map 2018-06-11 11:02:06 +02:00
payload.c MINOR: payload: add new sample fetch functions to process distcc protocol 2017-10-13 11:47:19 +02:00
peers.c MINOR: tasks: Change the task API so that the callback takes 3 arguments. 2018-05-26 19:23:57 +02:00
pipe.c CLEANUP: pools: rename all pool functions and pointers to remove this "2" 2017-11-24 17:49:53 +01:00
proto_http.c MINOR: http: Log warning if (add|set)-header fails 2018-05-28 14:53:59 +02:00
proto_tcp.c MINOR: fd: pass the iocb and owner to fd_insert() 2018-01-29 16:07:25 +01:00
proto_udp.c CLEANUP: fix inconsistency between fd->iocb, proto->accept and accept() 2016-04-14 11:18:22 +02:00
proto_uxst.c BUG/MINOR: unix: Make sure we can transfer abns sockets on seamless reload. 2018-06-07 14:33:44 +02:00
protocol.c BUILD: protocol: fix some build errors on OpenBSD 2016-08-10 19:31:58 +02:00
proxy.c MINOR: tasks: Change the task API so that the callback takes 3 arguments. 2018-05-26 19:23:57 +02:00
queue.c BUILD/MINOR: fix build when USE_THREAD is not defined 2018-03-26 17:17:59 +02:00
raw_sock.c BUG/MINOR: threads: Add missing THREAD_LOCAL on static here and there 2017-10-31 13:58:33 +01:00
regex.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
sample.c MINOR: sample: Add strcmp sample converter 2018-04-28 07:03:39 +02:00
server.c MINOR: dns: new DNS options to allow/prevent IP address duplication 2018-07-12 17:56:44 +02:00
session.c MINOR: Some spelling cleanup in the comments. 2018-06-21 20:43:52 +02:00
sha1.c IMPORT: sha1: import SHA1 functions 2017-10-25 04:45:48 +02:00
shctx.c BUILD: shctx: do not depend on openssl anymore 2017-11-08 14:33:36 +01:00
signal.c BUG/MINOR: signals: ha_sigmask macro for multithreading 2018-06-08 18:24:53 +02:00
ssl_sock.c BUG/MINOR: ssl: properly ref-count the tls_keys entries 2018-07-18 08:59:50 +02:00
standard.c MINOR: servers: Support alphanumeric characters for the server templates names 2018-04-06 19:16:18 +02:00
stats.c MINOR: stats: also report the failed header rewrites warnings on the stats page 2018-05-28 15:16:23 +02:00
stick_table.c MINOR: stick-tables: make stktable_release() do nothing on NULL 2018-06-27 06:33:20 +02:00
stream_interface.c MINOR: early data: Don't rely on CO_FL_EARLY_DATA to wake up streams. 2018-02-05 14:24:50 +01:00
stream.c MINOR: stats: also report the nice and number of calls for applets 2018-06-05 11:18:21 +02:00
task.c BUG/MINOR: tasklets: Just make sure we don't pass a tasklet to the handler. 2018-06-14 18:57:26 +02:00
tcp_rules.c MINOR: config: Enable tracking of up to MAX_SESS_STKCTR stick counters. 2018-01-29 13:53:56 +01:00
time.c BUG/MINOR: time/threads: ensure the adjusted time is always correct 2018-02-05 20:11:38 +01:00
trace.c CONTRIB: trace: try to display the function's return value on exit 2017-10-24 19:54:25 +02:00
uri_auth.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
vars.c BUG/MEDIUM: threads/vars: Fix deadlock in register_name 2017-12-08 10:37:24 +01:00
wurfl.c CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
xxhash.c CLEANUP: remove unneeded casts 2016-04-03 14:17:42 +02:00