Go to file
Willy Tarreau ddab05b98a BUG/MEDIUM: queue/threads: limit the number of entries dequeued at once
When testing strong queue contention on a 48-thread machine, some crashes
would frequently happen due to process_srv_queue() never leaving and
processing pending requests forever. A dump showed more than 500000
loops at once. The problem is that other threads find it working so
they don't do anything and are free to process their pending requests.
Because of this, the dequeuing thread can be kept busy forever and does
not process its own requests anymore (fortunately the watchdog stops it).

This patch adds a limit to the number of rounds, it limits it to
maxpollevents, which is reasonable because it's also an indicator of
latency and batches size. However there's a catch. If all requests
are finished when the thread ends the loop, there might not be enough
anymore to restart processing the queue. Thus we tolerate to re-enter
the loop to process one request at a time when it doesn't have any
anymore. This way we're leaving more room for another thread to take
on this task, and we're sure to eventually end this loop.

Doing this has also improved the overall dequeuing performance by ~20%
in highly contended situations with 48 threads.

It should be backported at least to 2.4, maybe even 2.2 since issues
were faced in the past on machines having many cores.
2022-07-30 10:00:59 +02:00
.github CI: re-enable gcc asan builds 2022-07-04 17:28:58 +02:00
addons CLEANUP: applet: rename appctx_cs() to appctx_sc() 2022-05-27 19:33:35 +02:00
admin BUILD: halog: fix some incorrect signs in printf formats for integers 2022-04-12 08:40:38 +02:00
dev CLEANUP: task: remove the now unused TASK_GLOBAL flag 2022-07-01 19:15:14 +02:00
doc MINOR: quic: New "quic-cc-algo" bind keyword 2022-07-29 17:32:05 +02:00
examples EXAMPLES: remove completely outdated acl-content-sw.cfg 2022-05-30 18:14:24 +02:00
include MINOR: quic: New "quic-cc-algo" bind keyword 2022-07-29 17:32:05 +02:00
reg-tests REGTEESTS: filters: Fix CONNECT request in random-forwarding script 2022-07-07 09:52:56 +02:00
scripts SCRIPTS: make publish-release try to launch make-releases-json 2022-05-30 15:44:10 +02:00
src BUG/MEDIUM: queue/threads: limit the number of entries dequeued at once 2022-07-30 10:00:59 +02:00
tests TESTS: add a unit test for one_among_mask() 2022-06-21 20:29:57 +02:00
.cirrus.yml CI: cirrus: switch to FreeBSD-13.0 2022-04-12 07:59:06 +02:00
.gitattributes MINOR: Configure the cpp userdiff driver for *.[ch] in .gitattributes 2021-02-22 18:17:57 +01:00
.gitignore MINOR: qpack: implement standalone decoder tool 2022-06-15 15:42:10 +02:00
.mailmap DOC: update Tim's address in .mailmap 2021-09-16 09:14:14 +02:00
.travis.yml CI: travis-ci: temporarily disable arm64 builds 2021-08-07 07:28:15 +02:00
BRANCHES DOC: fix some spelling issues over multiple files 2021-01-08 14:53:47 +01:00
CHANGELOG [RELEASE] Released version 2.7-dev2 2022-07-16 17:17:22 +02:00
CONTRIBUTING CLEANUP: assorted typo fixes in the code and comments 2021-08-16 12:37:59 +02:00
INSTALL BUILD: Makefile: Add Lua 5.4 autodetect 2022-07-04 17:28:48 +02:00
LICENSE LICENSE: add licence exception for OpenSSL 2012-09-07 13:52:26 +02:00
MAINTAINERS DOC: add maintainers for QUIC and HTTP/3 2022-05-30 17:34:51 +02:00
Makefile MEDIUM: quic: Cubic congestion control algorithm implementation 2022-07-29 17:32:05 +02:00
README DOC: create a BRANCHES file to explain the life cycle 2019-06-15 22:00:14 +02:00
SUBVERS BUILD: use format tags in VERDATE and SUBVERS files 2013-12-10 11:22:49 +01:00
VERDATE [RELEASE] Released version 2.7-dev2 2022-07-16 17:17:22 +02:00
VERSION [RELEASE] Released version 2.7-dev2 2022-07-16 17:17:22 +02:00

The HAProxy documentation has been split into a number of different files for
ease of use.

Please refer to the following files depending on what you're looking for :

  - INSTALL for instructions on how to build and install HAProxy
  - BRANCHES to understand the project's life cycle and what version to use
  - LICENSE for the project's license
  - CONTRIBUTING for the process to follow to submit contributions

The more detailed documentation is located into the doc/ directory :

  - doc/intro.txt for a quick introduction on HAProxy
  - doc/configuration.txt for the configuration's reference manual
  - doc/lua.txt for the Lua's reference manual
  - doc/SPOE.txt for how to use the SPOE engine
  - doc/network-namespaces.txt for how to use network namespaces under Linux
  - doc/management.txt for the management guide
  - doc/regression-testing.txt for how to use the regression testing suite
  - doc/peers.txt for the peers protocol reference
  - doc/coding-style.txt for how to adopt HAProxy's coding style
  - doc/internals for developer-specific documentation (not all up to date)