Go to file
Willy Tarreau 01abd02508 BUG/MEDIUM: listener: use a self-locked list for the dequeue lists
There is a very difficult to reproduce race in the listener's accept
code, which is much easier to reproduce once connection limits are
properly enforced. It's an ABBA lock issue :

  - the following functions take l->lock then lq_lock :
      disable_listener, pause_listener, listener_full, limit_listener,
      do_unbind_listener

  - the following ones take lq_lock then l->lock :
      resume_listener, dequeue_all_listener

This is because __resume_listener() only takes the listener's lock
and expects to be called with lq_lock held. The problem can easily
happen when listener_full() and limit_listener() are called a lot
while in parallel another thread releases sessions for the same
listener using listener_release() which in turn calls resume_listener().

This scenario is more prevalent in 2.0-dev since the removal of the
accept lock in listener_accept(). However in 1.9 and before, a different
but extremely unlikely scenario can happen :

      thread1                                  thread2
         ............................  enter listener_accept()
  limit_listener()
         ............................  long pause before taking the lock
  session_free()
    dequeue_all_listeners()
      lock(lq_lock) [1]
         ............................  try_lock(l->lock) [2]
      __resume_listener()
        spin_lock(l->lock) =>WAIT[2]
         ............................  accept()
                                       l->accept()
                                       nbconn==maxconn =>
                                         listener_full()
                                           state==LI_LIMITED =>
                                             lock(lq_lock) =>DEADLOCK[1]!

In practice it is almost impossible to trigger it because it requires
to limit both on the listener's maxconn and the frontend's rate limit,
at the same time, and to release the listener when the connection rate
goes below the limit between poll() returns the FD and the lock is
taken (a few nanoseconds). But maybe with threads competing on the
same core it has more chances to appear.

This patch removes the lq_lock and replaces it with a lockless queue
for the listener's wait queue (well, technically speaking a self-locked
queue) brought by commit a8434ec14 ("MINOR: lists: Implement locked
variations.") and its few subsequent fixes. This relieves us from the
need of the lq_lock and removes the deadlock. It also gets rid of the
distinction between __resume_listener() and resume_listener() since the
only difference was the lq_lock. All listener removals from the list
are now unconditional to avoid races on the state. It's worth noting
that the list used to never be initialized and that it used to work
only thanks to the state tests, so the initialization has now been
added.

This patch must carefully be backported to 1.9 and very likely 1.8.
It is mandatory to be careful about replacing all manipulations of
l->wait_queue, global.listener_queue and p->listener_queue.
2019-02-28 16:08:54 +01:00
.github/ISSUE_TEMPLATE DOC: add github issue templates 2019-01-17 22:53:55 +01:00
contrib CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy 2019-02-08 13:55:21 +01:00
doc DOC: update management.txt to reflect that threads are used by default 2019-02-27 15:01:46 +01:00
ebtree CLEANUP: fix typos in comments in ebtree 2018-11-18 22:23:15 +01:00
examples [RELEASE] Released version 2.0-dev1 2019-02-26 16:43:49 +01:00
include BUG/MEDIUM: listener: use a self-locked list for the dequeue lists 2019-02-28 16:08:54 +01:00
reg-tests BUG/MEDIUM: compression: Rewrite strong ETags 2019-01-29 20:26:06 +01:00
scripts SCRIPTS: add the issue tracker URL to the announce script 2019-01-29 06:51:16 +01:00
src BUG/MEDIUM: listener: use a self-locked list for the dequeue lists 2019-02-28 16:08:54 +01:00
tests CLEANUP: fix a misspell in tests/filltab25.c 2018-11-18 22:23:15 +01:00
.gitignore DOC: split the README into README + INSTALL 2018-12-16 22:30:57 +01:00
CHANGELOG [RELEASE] Released version 2.0-dev1 2019-02-26 16:43:49 +01:00
CONTRIBUTING DOC: Fix typos in README and CONTRIBUTING 2018-11-12 08:54:12 +01:00
INSTALL [RELEASE] Released version 2.0-dev0 2018-12-22 11:20:35 +01:00
LICENSE LICENSE: add licence exception for OpenSSL 2012-09-07 13:52:26 +02:00
MAINTAINERS DOC: fix reference to map files in MAINTAINERS 2018-10-17 04:21:04 +02:00
Makefile MINOR: fd: Use closefrom() as my_closefrom() if supported. 2019-02-25 16:51:03 +01:00
README DOC: split the README into README + INSTALL 2018-12-16 22:30:57 +01:00
ROADMAP DOC: update the roadmap about priority queues 2018-08-10 17:12:04 +02:00
SUBVERS BUILD: use format tags in VERDATE and SUBVERS files 2013-12-10 11:22:49 +01:00
VERDATE [RELEASE] Released version 2.0-dev1 2019-02-26 16:43:49 +01:00
VERSION [RELEASE] Released version 2.0-dev1 2019-02-26 16:43:49 +01:00

The HAProxy documentation has been split into a number of different files for
ease of use.

Please refer to the following files depending on what you're looking for :

  - INSTALL for instructions on how to build and install HAProxy
  - LICENSE for the project's license
  - CONTRIBUTING for the process to follow to submit contributions

The more detailed documentation is located into the doc/ directory :

  - doc/intro.txt for a quick introduction on HAProxy
  - doc/configuration.txt for the configuration's reference manual
  - doc/lua.txt for the Lua's reference manual
  - doc/SPOE.txt for how to use the SPOE engine
  - doc/network-namespaces.txt for how to use network namespaces under Linux
  - doc/management.txt for the management guide
  - doc/regression-testing.txt for how to use the regression testing suite
  - doc/peers.txt for the peers protocol reference
  - doc/coding-style.txt for how to adopt HAProxy's coding style
  - doc/internals for developer-specific documentation (not all up to date)