From f8bce3125ef44fe599ad8793c85ed3a65ee08642 Mon Sep 17 00:00:00 2001 From: Willy Tarreau Date: Fri, 12 Apr 2019 15:11:02 +0200 Subject: [PATCH] BUG/MEDIUM: task/threads: address a fairness issue between local and global tasks It is possible to hit a fairness issue in the scheduler when a local task runs for a long time (i.e. process_stream() returns running), and a global task wants to run on the same thread and remains in the global queue. What happens in this case is that the condition to extract tasks from the global queue will rarely be satisfied for very low task counts since whatever non-null queue size multiplied by a thread count >1 is always greater than the small remaining number of tasks in the queue. In theory another thread should pick the task but we do have some mono threaded tasks in the global queue as well during inter-thread wakeups. Note that this can only happen with task counts lower than the thread counts, typically one task in each queue for more than two threads. This patch works around the problem by allowing a very small unfairness, making sure that we can always pick at least one task from the global queue even if there is already one in the local queue. A better approach will consist in scanning the two trees in parallel and always pick the best task. This will be more complex and will constitute a separate patch. This fix must be backported to 1.9. --- src/task.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/task.c b/src/task.c index 7f34bc5a3..f8f9d995f 100644 --- a/src/task.c +++ b/src/task.c @@ -339,7 +339,7 @@ void process_runnable_tasks() * than the average. */ rq_next = eb32sc_lookup_ge(&rqueue, rqueue_ticks - TIMER_LOOK_BACK, tid_bit); - while ((task_per_thread[tid].task_list_size + task_per_thread[tid].rqueue_size) * global.nbthread <= tasks_run_queue) { + while ((task_per_thread[tid].task_list_size + task_per_thread[tid].rqueue_size) * global.nbthread <= tasks_run_queue + global.nbthread - 1) { if (unlikely(!rq_next)) { /* either we just started or we reached the end * of the tree, typically because