mirrors/haproxy
1
0
Fork 0
mirror of https://git.haproxy.org/git/haproxy.git/ synced 2025-08-07 07:37:02 +02:00
Code Issues Projects Releases Wiki Activity
18,920 Commits 16 Branches 394 Tags 180 MiB
C 98.1%
Shell 0.8%
Makefile 0.5%
Lua 0.2%
Python 0.2%
a56798ea4d
Go to file
Willy Tarreau a56798ea4d BUG/MEDIUM: checks: do not reschedule a possibly running task on state change 
Aurélien found an issue introduced in 2.7-dev8 with commit d114f4a68
("MEDIUM: checks: spread the checks load over random threads"), but which
in fact has deeper roots.

When a server's state is changed via __health_adjust(), if a fastinter
setting is set, the task gets rescheduled to run at the new date. The
way it's done is not thread safe, as nothing prevents another thread
where the task is already running from also updating the expire field
in parallel. But since such events are quite rare, this statistically
never happens. However, with the commit above, the tasks are no longer
required to go to the shared wait queue and are no longer marked as
shared between multiple threads. It's just that *any* thread may run
them at a time without implying that all of them are allowed to modify
them. And this change is sufficient to trigger the BUG_ON() condition
in the scheduler that detects the inconsistency between a task queued
in one thread and being manipulated in parallel by another one:

  FATAL: bug condition "task->tid != tid" matched at
  include/haproxy/task.h:670
    call trace(13):
    | 0x55f61cf520c9 [c6 04 25 01 00 00 00 00]: main-0x2ee7
    | 0x55f61d0646e8 [8b 45 08 a8 40 0f 85 65]: back_handle_st_cer+0x78/0x4d7
    | 0x55f61cff3e72 [41 0f b6 4f 01 e9 c8 df]: process_stream+0x2252/0x364f
    | 0x55f61d0d2fab [48 89 c3 48 85 db 74 75]: run_tasks_from_lists+0x34b/0x8c4
    | 0x55f61d0d38ad [29 44 24 18 8b 54 24 18]: process_runnable_tasks+0x37d/0x6c6
    | 0x55f61d0a22fa [83 3d 0b 63 1e 00 01 0f]: run_poll_loop+0x13a/0x536
    | 0x55f61d0a28c9 [48 8b 1d f0 46 19 00 48]: main+0x14d919
    | 0x55f61cf56dfe [31 c0 e8 eb 93 1b 00 31]: main+0x1e4e/0x2d5d

At first glance it looked like it could be addressed in the scheduler
only, but in fact the problem clearly is at the application level, since
some shared fields are manipulated without protection. At minima, the
task's expiry ought to be touched only under the server's lock. While
it's arguable that the scheduler could make such updates easier, changing
it alone will not be sufficient here.

Looking at the sequencing closer, it becomes obvious that we do not need
this task_schedule() at all: a simple task_wakeup() is sufficient for the
callee to update its timers. Indeed, the process_chk_con() function already
deals with spurious wakeups, and already uses srv_getinter() to calculate
the next wakeup date based on the current state. So here, instead of
having to queue the task from __health_adjust() to anticipate a new check,
we can simply wake the task up and let it decide when it needs to run
next. This is much cleaner as the expiry calculation remains performed at
a single place, from the task itself, as it should be, and it fixes the
problem above.

This should be backported to 2.7, but not to older versions where the
risks of breakage are higher than the chance to fix something that
ever happened.
2022-12-06 14:14:41 +01:00
.github CI: emit the compiler's version in the build reports 2022-11-14 11:14:02 +01:00
addons LICENSE: wurfl: clarify the dummy library license. 2022-12-02 05:32:46 +01:00
admin BUILD: halog: fix missing double-quote at end of help line 2022-11-25 11:11:41 +01:00
dev DEV: tcploop: add optional support for epoll 2022-11-25 17:08:06 +01:00
doc MINOR: quic: activate socket per conn by default 2022-12-02 14:45:43 +01:00
examples EXAMPLES: remove completely outdated acl-content-sw.cfg 2022-05-30 18:14:24 +02:00
include MINOR: server/event_hdl: add support for SERVER_UP and SERVER_DOWN events 2022-12-06 10:22:07 +01:00
reg-tests CLEANUP: assorted typo fixes in the code and comments 2022-11-30 14:02:36 +01:00
scripts SCRIPTS: run-regtests: add a version check 2022-11-30 18:44:33 +01:00
src BUG/MEDIUM: checks: do not reschedule a possibly running task on state change 2022-12-06 14:14:41 +01:00
tests TESTS: add a unit test for one_among_mask() 2022-06-21 20:29:57 +02:00
.cirrus.yml CI: cirrus-ci: bump FreeBSD image to 13-1 2022-09-09 13:30:17 +02:00
.gitattributes MINOR: Configure the cpp userdiff driver for *.[ch] in .gitattributes 2021-02-22 18:17:57 +01:00
.gitignore CLEANUP: exclude udp-perturb with .gitignore 2022-09-16 15:47:04 +02:00
.mailmap DOC: update Tim's address in .mailmap 2021-09-16 09:14:14 +02:00
.travis.yml CI: travis-ci: temporarily disable arm64 builds 2021-08-07 07:28:15 +02:00
BRANCHES DOC: fix some spelling issues over multiple files 2021-01-08 14:53:47 +01:00
CHANGELOG [RELEASE] Released version 2.8-dev0 2022-12-01 15:25:34 +01:00
CONTRIBUTING CLEANUP: assorted typo fixes in the code and comments 2021-08-16 12:37:59 +02:00
INSTALL MINOR: version: mention that it's development again 2022-12-01 15:24:10 +01:00
LICENSE LICENSE: add licence exception for OpenSSL 2012-09-07 13:52:26 +02:00
MAINTAINERS CLEANUP: assorted typo fixes in the code and comments 2022-11-30 14:02:36 +01:00
Makefile MINOR: event_hdl: add event handler base api 2022-12-02 09:40:52 +01:00
README DOC: create a BRANCHES file to explain the life cycle 2019-06-15 22:00:14 +02:00
SUBVERS BUILD: use format tags in VERDATE and SUBVERS files 2013-12-10 11:22:49 +01:00
VERDATE [RELEASE] Released version 2.7.0 2022-12-01 15:16:46 +01:00
VERSION [RELEASE] Released version 2.8-dev0 2022-12-01 15:25:34 +01:00

README 

The HAProxy documentation has been split into a number of different files for
ease of use.

Please refer to the following files depending on what you're looking for :

  - INSTALL for instructions on how to build and install HAProxy
  - BRANCHES to understand the project's life cycle and what version to use
  - LICENSE for the project's license
  - CONTRIBUTING for the process to follow to submit contributions

The more detailed documentation is located into the doc/ directory :

  - doc/intro.txt for a quick introduction on HAProxy
  - doc/configuration.txt for the configuration's reference manual
  - doc/lua.txt for the Lua's reference manual
  - doc/SPOE.txt for how to use the SPOE engine
  - doc/network-namespaces.txt for how to use network namespaces under Linux
  - doc/management.txt for the management guide
  - doc/regression-testing.txt for how to use the regression testing suite
  - doc/peers.txt for the peers protocol reference
  - doc/coding-style.txt for how to adopt HAProxy's coding style
  - doc/internals for developer-specific documentation (not all up to date)