Willy Tarreau ddab05b98a BUG/MEDIUM: queue/threads: limit the number of entries dequeued at once
When testing strong queue contention on a 48-thread machine, some crashes
would frequently happen due to process_srv_queue() never leaving and
processing pending requests forever. A dump showed more than 500000
loops at once. The problem is that other threads find it working so
they don't do anything and are free to process their pending requests.
Because of this, the dequeuing thread can be kept busy forever and does
not process its own requests anymore (fortunately the watchdog stops it).

This patch adds a limit to the number of rounds, it limits it to
maxpollevents, which is reasonable because it's also an indicator of
latency and batches size. However there's a catch. If all requests
are finished when the thread ends the loop, there might not be enough
anymore to restart processing the queue. Thus we tolerate to re-enter
the loop to process one request at a time when it doesn't have any
anymore. This way we're leaving more room for another thread to take
on this task, and we're sure to eventually end this loop.

Doing this has also improved the overall dequeuing performance by ~20%
in highly contended situations with 48 threads.

It should be backported at least to 2.4, maybe even 2.2 since issues
were faced in the past on machines having many cores.
2022-07-30 10:00:59 +02:00
2022-07-04 17:28:58 +02:00
2022-04-12 07:59:06 +02:00
2021-09-16 09:14:14 +02:00
2022-07-16 17:17:22 +02:00
2022-07-16 17:17:22 +02:00
2022-07-16 17:17:22 +02:00

The HAProxy documentation has been split into a number of different files for
ease of use.

Please refer to the following files depending on what you're looking for :

  - INSTALL for instructions on how to build and install HAProxy
  - BRANCHES to understand the project's life cycle and what version to use
  - LICENSE for the project's license
  - CONTRIBUTING for the process to follow to submit contributions

The more detailed documentation is located into the doc/ directory :

  - doc/intro.txt for a quick introduction on HAProxy
  - doc/configuration.txt for the configuration's reference manual
  - doc/lua.txt for the Lua's reference manual
  - doc/SPOE.txt for how to use the SPOE engine
  - doc/network-namespaces.txt for how to use network namespaces under Linux
  - doc/management.txt for the management guide
  - doc/regression-testing.txt for how to use the regression testing suite
  - doc/peers.txt for the peers protocol reference
  - doc/coding-style.txt for how to adopt HAProxy's coding style
  - doc/internals for developer-specific documentation (not all up to date)
Description
No description provided
Readme 51 MiB
Languages
C 98.1%
Shell 0.8%
Makefile 0.5%
Lua 0.2%
Python 0.2%