mirrors/haproxy
1
0
Fork 0
mirror of https://git.haproxy.org/git/haproxy.git/ synced 2025-08-05 22:56:57 +02:00
Code Issues Projects Releases Wiki Activity

MEDIUM: config: change default limits to 1024 threads and 32 groups

Browse Source 
A test run on a dual-socket EPYC 9845 (2x160 cores) showed that we'll
be facing new limits during the lifetime of 3.2 with our current 16
groups and 256 threads max:

  $ cat test.cfg
  global
      cpu-policy perforamnce

  $ ./haproxy -dc -c -f test.cfg
  ...
  Thread CPU Bindings:
    Tgrp/Thr  Tid        CPU set
    1/1-32    1-32       32: 0-15,320-335
    2/1-32    33-64      32: 16-31,336-351
    3/1-32    65-96      32: 32-47,352-367
    4/1-32    97-128     32: 48-63,368-383
    5/1-32    129-160    32: 64-79,384-399
    6/1-32    161-192    32: 80-95,400-415
    7/1-32    193-224    32: 96-111,416-431
    8/1-32    225-256    32: 112-127,432-447

Raising the default limit to 1024 threads and 32 groups is sufficient
to buy us enough margin for a long time (hopefully, please don't laugh,
you, reader from the future):

  $ ./haproxy -dc -c -f test.cfg
  ...
  Thread CPU Bindings:
    Tgrp/Thr  Tid        CPU set
    1/1-32    1-32       32: 0-15,320-335
    2/1-32    33-64      32: 16-31,336-351
    3/1-32    65-96      32: 32-47,352-367
    4/1-32    97-128     32: 48-63,368-383
    5/1-32    129-160    32: 64-79,384-399
    6/1-32    161-192    32: 80-95,400-415
    7/1-32    193-224    32: 96-111,416-431
    8/1-32    225-256    32: 112-127,432-447
    9/1-32    257-288    32: 128-143,448-463
    10/1-32   289-320    32: 144-159,464-479
    11/1-32   321-352    32: 160-175,480-495
    12/1-32   353-384    32: 176-191,496-511
    13/1-32   385-416    32: 192-207,512-527
    14/1-32   417-448    32: 208-223,528-543
    15/1-32   449-480    32: 224-239,544-559
    16/1-32   481-512    32: 240-255,560-575
    17/1-32   513-544    32: 256-271,576-591
    18/1-32   545-576    32: 272-287,592-607
    19/1-32   577-608    32: 288-303,608-623
    20/1-32   609-640    32: 304-319,624-639

We can change this default now because it has no functional effect
without any configured cpu-policy, so this will only be an opt-in
and it's better to do it now than to have an effect during the
maintenance phase. A tiny effect is a doubling of the number of
pool buckets and stick-table shards internally, which means that
aside slightly reducing contention in these areas, a dump of tables
can enumerate keys in a different order (hence the adjustment in the
vtc).

The only really visible effect is a slightly higher static memory
consumption (29->35 MB on a small config), but that difference
remains even with 50k servers so that's pretty much acceptable.

Thanks to Erwan Velu for the quick tests and the insights!
This commit is contained in:
Willy Tarreau 2025-05-13 17:58:28 +02:00
parent 158da59c34
commit e049bd00ab
2 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
include/haproxy/defaults.h
View File

@ -44,7 +44,7 @@
* doesn't engage us too far.
*/
#ifndef MAX_TGROUPS
#define MAX_TGROUPS 16
#define MAX_TGROUPS 32
#endif
#define MAX_THREADS_PER_GROUP __WORDSIZE
@ -53,7 +53,7 @@
* long bits if more tgroups are enabled.
*/
#ifndef MAX_THREADS
#define MAX_THREADS ((((MAX_TGROUPS) > 1) ? 4 : 1) * (MAX_THREADS_PER_GROUP))
#define MAX_THREADS ((((MAX_TGROUPS) > 1) ? 16 : 1) * (MAX_THREADS_PER_GROUP))
#endif
#endif // USE_THREAD

2
reg-tests/http-rules/converters_ipmask_concat_strcmp_field_word.vtc
View File

@ -211,7 +211,7 @@ client c1 -connect ${h1_fe1_sock} -proxy2 "192.168.1.101:1234 127.0.0.1:2345" {
# cli show be1 stick table
haproxy h1 -cli {
send "show table be1"
expect ~ "^# table: be1, type: ipv6, size:20, used:3\\n0x[a-f0-9]+: key=::ffff:192\\.168\\.1\\.0 use=0 exp=[[:digit:]]+ shard=0 gpc0=0 conn_cnt=1\\n0x[a-f0-9]+: key=::ffff:192\\.168\\.1\\.101 use=0 exp=[[:digit:]]+ shard=0 gpc0=0 conn_cnt=1\\n0x[a-f0-9]+: key=2001:db8:c001:c01a:[0:]+ use=0 exp=[[:digit:]]+ shard=0 gpc0=0 conn_cnt=1\\n"
expect ~ "^# table: be1, type: ipv6, size:20, used:3\\n0x[a-f0-9]+: key=::ffff:192\\.168\\.1\\.101 use=0 exp=[[:digit:]]+ shard=0 gpc0=0 conn_cnt=1\\n0x[a-f0-9]+: key=::ffff:192\\.168\\.1\\.0 use=0 exp=[[:digit:]]+ shard=0 gpc0=0 conn_cnt=1\\n0x[a-f0-9]+: key=2001:db8:c001:c01a:[0:]+ use=0 exp=[[:digit:]]+ shard=0 gpc0=0 conn_cnt=1\\n"
}
# concat,strcmp,word,field tests