1217 Commits

Author SHA1 Message Date
Valentine Krasnobaeva
02af1fe067 MEDIUM: startup: call chroot() if needed in one place
There are two 'chroot' code blocks, both under quite same:

	'if ((global.mode & (MODE_MWORKER|MODE_DAEMON...)...'

The first block serves to perform chroot only for process in the foreground
mode. The second comes after master-worker fork and allows to do chroot
in daemon and in worker modes.

Due to moving the master-worker fork in init() in some previous commit, the
second 'chroot' code block now is no longer under the 'if'. So, it is executed
for all modes, except MODE_MWORKER. Now in MODE_MWORKER process enters in its
wait polling loop just after forking a worker and it terminates almost
immediately, if it exits this loop.

Worker, daemon and process in a foreground mode will perform the chroot as
before, but now it will be done in a one place at main().
2024-10-16 22:02:39 +02:00
Valentine Krasnobaeva
7a2ee10d71 REORG: mworker: move mworker_create_master_cli in master 'case'
Let's move mworker_create_master_cli() call in 'master' case just above and get
rid of redundant global.mode tests.
2024-10-16 22:02:39 +02:00
Valentine Krasnobaeva
e4c10a704d MINOR: init: check MODE_MWORKER before creating master CLI
mworker_create_master_cli() creates MASTER proxy and allocates listeners,
which are attached to this proxy. It also creates a reload sockpair.

So, it's more appropriate to do the check, that we are in a MODE_MWORKER, if
master CLI settings were provided via command line, just after the config
parsing. And only then, if runtime mode and command line settings are
coherent, try to perform master-worker fork and try to create master CLI.
2024-10-16 22:02:39 +02:00
Valentine Krasnobaeva
26e53e2e8c REORG: mworker: set nbthread=1 for master after fork
After moving master-worker fork into init() and reintroducing it into a
switch-case (see the previous commit), it is more appropriate to set
nbthread=1 and nbtgroups=1 immediately in the 'case' for the parent process.
2024-10-16 22:02:39 +02:00
Valentine Krasnobaeva
ae84f06025 BUG/MINOR: mworker: fix memory leak due to master-worker fork
Before this fix, startup logs ring was duplicated before the fork(), so master
and worker had both the original startup_logs ring and the duplicated one. In
the worker context we freed the original ring and used a duplicated one. In
the master context we did nothing, but we still create a duplicated copy again
and again during the reload.

So, let's duplicate startup logs ring only in the worker context. Master
continues to use the original ring initialized in init() before its fork().
2024-10-16 22:02:39 +02:00
Valentine Krasnobaeva
8dd4efe42f MAJOR: mworker: move master-worker fork in init()
This refactoring allows to simplify 'master-worker' logic. The master process
with this change will fork a worker very early at the initialization stage,
which allows to perform a configuration parsing only for the worker. In reality
only the worker process needs to parse and to apply the whole configuration.

Master process just polls master CLI sockets, watches worker status, catches
its termination state and handles the signals. With this refactoring there is
no longer need for master to perform re-execution after reading the whole
configuration file to free additional memory. And there is no longer need for
worker to register atexit callbacks, in order to free the memory, when it
fails to apply the new configuration. In contrast, we now need to set
proc_self pointer to the new worker entry in processes list just after
the fork in the worker process context. proc_self is dereferenced in
mworker_sockpair_register_per_thread(), which is called when worker enters in
its polling loop.

Following patches will try to gather more 'worker' and 'master' specific' code
in the dedicated cases of this new fork() switch, or in a separate functions.
2024-10-16 22:00:58 +02:00
Valentine Krasnobaeva
4cbfcc60f4 MEDIUM: startup: move PID handling in init()
Let's move PID handling in init() from the main() code. It is more appropriate
to open and to write the PID of the process just after daemonization fork. In
case of daemon monoprocess mode, we will simply write a PID of the process,
which is already in the background. In case of 'master-worker' mode, we keep
the previous behaviour and we write only a PID of the master process.

This allows to remove redundant tests of the process execution mode, tests of
the pidfd value and consequent writes to this pidfd. This patch prepares the
refactoring of master-worker fork by moving it in init() function as well.
2024-10-16 22:00:58 +02:00
Valentine Krasnobaeva
95c19be2ab MINOR: startup: refactor "daemonization" fork
Let's put "daemonization" fork into a switch-case. This is more readable and we
don't need to allocate memory for the fork() return value here.
2024-10-16 22:00:58 +02:00
Valentine Krasnobaeva
90b8181c0a MEDIUM: startup: move daemonization fork in init
Let's move daemonization fork in init(). We need to perform this fork always
before forking a worker process, in order to be able to launch master and then
its worker in daemon, i.e. background mode, if haproxy was started with '-D'
option.

This refactoring is a preparation step, needed for replacing then master-worker
fork in init() as well. This allows the master process not to read the whole
configuration file and not to do re-execution in order to free additional
memory, when worker was forked. In the new refactored design only the worker
process will read and apply a new configuration, while the master will arrive
very fast in its polling loop to wait worker's termination and to handle
signals. See more details in the following commits.
2024-10-16 22:00:58 +02:00
Valentine Krasnobaeva
df12791da3 MINOR: startup: add O_CLOEXEC flag to open /dev/null
As master process performs execvp() syscall to handle USR2 and HUP signals in
mworker_reexec(), let's add O_CLOEXEC flag, when we open '/dev/null' in order
to avoid fd leak.

This a preparation step to refactor master-worker logic. See more details in
the next commits.
2024-10-16 22:00:58 +02:00
Willy Tarreau
d6c060c5ae MINOR: tools: add minimal file name management
In proxies, stick-tables, servers, etc... at plenty of places we store
a file name and a line number. Some file names are the result of strdup()
(e.g. in proxies), others not (e.g. stick-tables) and leave dangling
pointers at the end of parsing. The risk of double-free is not null
either.

In order to stop this, let's first add a simple tool that allows to
register short strings inside a global list, these strings happening
to be server names. The strings are either duplicated and stored upon
failure to find them, or just added to this storage. Since file names
are not expected to disappear before the end of the process, for now
we don't even implement refcounting, and we free them all at the end.
There's already a drop_file_name() function to reset the pointer like
ha_free() used to do, and even if not strictly needed it's a good
habit to get used to doing it.

The strings are returned as const so that they're stored as-is in
structs, and that nasty free() calls are easily caught. The pointer
points to the char[] storage inside the node itself. This way later
if we want to implement refcounting, it will be trivial to just look
up a string and change its associated node's refcount. If needed,
comparisons can also be made on pointers.

For now they're not used yet and are released on deinit().
2024-09-19 15:36:58 +02:00
Nicolas CARPi
534e7e4598 CLEANUP: haproxy: fix typos in code comment
Use "from" instead of "form" in ha_random_boot function code comments.
2024-08-30 14:58:59 +02:00
Valentine Krasnobaeva
28ca7fc594 BUG/MINOR: haproxy: free init_env in deinit only if allocated
This fixes 7b78e1571 (" MINOR: mworker: restore initial env before wait
mode").

In cases, when haproxy starts without any configuration, for example:
'haproxy -vv', init_env array to backup env variables is never allocated. So,
we need to check in deinit(), when we free its memory, that init_env is not a
NULL ptr.
2024-08-23 19:08:53 +02:00
Valentine Krasnobaeva
7b78e1571b MINOR: mworker: restore initial env before wait mode
This patch is the follow-up of 1811d2a6ba (MINOR: tools: add helpers to
backup/clean/restore env).

In order to avoid unexpected behaviour in master-worker mode during the process
reload with a new configuration, when the old one has contained '*env' keywords,
let's backup its initial environment before calling parse_cfg() and let's clean
and restore it in the context of master process, just before it enters in a wait
polling loop.

This will garantee that new workers will have a new updated environment and not
the previous one inherited from the master, which does not read the configuration,
when it's in a wait-mode.
2024-08-23 17:06:59 +02:00
Nathan Wehrman
5c07d58e08 MINOR: config: Created env variables for http and tcp clf formats
Since we already have variables for the other formats and the
change is trivial I thought it would be a nice addition for
completeness
2024-08-22 09:15:58 +02:00
Amaury Denoyelle
d0d8e57d47 MINOR: quic: define sbuf pool
Define a new buffer pool reserved to allocate smaller memory area. For
the moment, its usage will be restricted to QUIC, as such it is declared
in quic_stream module.

Add a new config option "tune.bufsize.small" to specify the size of the
allocated objects. A special check ensures that it is not greater than
the default bufsize to avoid unexpected effects.
2024-08-20 18:12:27 +02:00
Amaury Denoyelle
1de5f718cf MINOR: quic/config: adapt settings to new conn buffer limit
QUIC MUX buffer allocation limit is now directly based on the underlying
congestion window size. previous static limit based on conn-tx-buffers
is now unused. As such, this commit adds a warning to users to prevent
that it is now obsolete.

Secondly, update max-window-size setting. It is now the main entrypoint
to limit both the maximum congestion window size and the number of QUIC
MUX allocated buffer on emission. Remove its special value '0' which was
used to automatically adjust it on now unused conn-tx-buffers.
2024-08-20 17:59:35 +02:00
Amaury Denoyelle
c24c8667b2 MINOR: quic: define max-window-size config setting
Define a new global keyword tune.quic.frontend.max-window-size. This
allows to set globally the maximum congestion window size for each QUIC
frontend connections.

The default value is 0. It is a special value which automatically derive
the size from the configured QUIC connection buffer limit. This is
similar to the previous "quic-cc-algo" behavior, which can be used to
override the maximum window size per bind line.
2024-08-20 17:02:29 +02:00
Valentine Krasnobaeva
aae2ff7691 MINOR: startup: fix unused value reported by coverity
Unused 0 is assigned to ret, as it's rewritten by error code of read_cfg().
This issue was reported by coverity.
2024-08-08 19:54:12 +02:00
Valentine Krasnobaeva
c6cfa7cb4a MINOR: startup: rename readcfgfile in parse_cfg
As readcfgfile no longer opens configuration files and reads them with fgets,
but performs only the parsing of provided data, let's rename it to parse_cfg by
analogy with read_cfg in haproxy.c.
2024-08-07 18:41:41 +02:00
Valentine Krasnobaeva
5b52df4c4d MEDIUM: startup: load and parse configs from memory
Let's call load_cfg_in_ram() helper for each configuration file to load it's
content in some area in memory. Adapt readcfgfile() parser function
respectively. In order to limit changes in its scope we give as an argument a
cfgfile structure, already filled in init_args() and in load_cfg_in_ram() with
file metadata and content.

Parser function (readcfgfile()) uses now fgets_from_mem() instead of standard
fgets from libc implementations.

SPOE filter parses its own configuration file, pointed by 'config' keyword in
the configuration already loaded in memory. So, let's allocate and fill for
this a supplementary cfgfile structure, which is not referenced in cfg_cfgfiles
list. This structure and the memory with content of SPOE filter configuration
are freed immediately in parse_spoe_flt(), when readcfgfile() returns.

HAProxy OpenTracing filter also uses its own configuration file. So, let's
follow the same logic as we do for SPOE filter.
2024-08-07 18:41:41 +02:00
Valentine Krasnobaeva
2bb34edb0b MEDIUM: startup: make read_cfg() return immediately on ENOMEM
This commit prepares read_cfg() to call load_cfg_in_mem() helper in order to
load configuration files in memory. Before, read_cfg() calls the parser for all
files from cfg_cfgfiles list and cumulates parser's errors and memprintf's
errors in for_each loop. memprintf's errors did not stop this loop and were
accounted just after.

Now, as we plan to load configuration files in memory, we stop the loop, if
memprintf() fails, and we show appropraite error message with ha_alert. Then
process terminates. So not all cumulated syntax-related errors will be shown
before exit in this case and we has to stop, because we run out of memory.

If we can't open the current file or we fail to allocate a memory to store
some configuration line, the previous behaviour is kept, process emits
appropriate alert message and exits.

If parser returns some syntax-related error on the current file, the previous
behaviour is kept as well. We cumulate such errors for all parsed files and we
check them just after the loop. All syntax-related errors for all files is
shown then  as before in ha_alert messages line by line during the startup.
Then process will exit with 1.

As now cfg_cfgfiles list contains many pointers to some memory areas with
configuration files content and this content could be big, it's better to
free the list explicitly, when parsing was finished. So, let's change
read_cfg() to return some integer value to its caller init(), and let's perform
the free  routine at a caller level, as cfg_cfgfiles list was initialized and
initially filled at this level.
2024-08-07 18:41:41 +02:00
Valentine Krasnobaeva
bafb0ce272 MINOR: startup: adapt list_append_word to use cfgfile
list_append_word() helper was used before only to chain configuration file names
in a list. As now we start to use cfgfile structure which represents entire file
in memory and its metadata, let's adapt this helper to use this structure and
let's rename it to list_append_cfgfile().

Adapt functions, which process configuration files and directories to use
cfgfile structure and list_append_cfgfile() instead of wordlist.
2024-08-07 18:41:41 +02:00
William Lallemand
f76e8e50f4 BUILD: ssl: replace USE_OPENSSL_AWSLC by OPENSSL_IS_AWSLC
Replace USE_OPENSSL_AWSLC by OPENSSL_IS_AWSLC in the code source, so we
won't need to set USE_OPENSSL_AWSLC in the Makefile on the long term.
2024-07-30 18:53:08 +02:00
Valentine Krasnobaeva
fcd4bf54c8 BUG/MEDIUM: startup: fix zero-warning mode
Let's check the second time a global counter of "ha_warning" messages, if
zero-warning is set. And let's do this just before forking. At this moment we
are sure, that we've already done all init operations, where we could emit
"ha_warning", and we still have stderr fd opened.

Even with the second check, we could lost some late and rare warnings
about failing to drop supplementary groups and about re-enabling core dumps.
Notes about this are added into 'zero-warning' keyword description.
2024-07-18 05:24:56 +02:00
Valentine Krasnobaeva
1f8addfdc2 REORG: haproxy: move limits handlers to limits
This patch moves handlers to compute process related limits in 'limits'
compilation unit.
2024-07-10 18:05:48 +02:00
Valentine Krasnobaeva
22db643648 MINOR: haproxy: prepare to move limits-related code
This patch is done in order to prepare the move of handlers to compute and to
check process related limits as maxconn, maxsock, maxpipes.

So, these handlers become no longer static due to the future move.

We add the handlers declarations in limits.h in this patch as well, in order to
keep the next patch, dedicated to code replacement, without any additional
modifications.

Such split also assures that this patch can be compiled separately from the
next one, where we moving the handlers. This  is important in case of
git-bisect.
2024-07-10 18:05:48 +02:00
Valentine Krasnobaeva
b8dc783eb9 REORG: global: move rlim_fd_*_at_boot in limits
Let's move in 'limits' compilation unit global variables to keep the initial
process fd limits.
2024-07-10 18:05:48 +02:00
Valentine Krasnobaeva
1517bcb5e3 MINOR: limits: prepare to keep limits in one place
The code which gets, sets and checks initial and current fd limits and process
related limits (maxconn, maxsock, ulimit-n, fd-hard-limit) is spread around
different functions in haproxy.c and in fd.c. Let's group it together in
dedicated limits.c and limits.h.

This patch is done in order to prepare the moving of limits-related functions
from different places to the new 'limits' compilation unit. It helps to keep
clean the next patch, which will do only the move without any additional
modifications.

Such detailed split is needed in order to be sure not to break accidentally
limits logic and in order to be able to compile each commit separately in case
of git-bisect.
2024-07-10 18:05:48 +02:00
Valentine Krasnobaeva
16a5fac4bb BUG/MEDIUM: init: fix fd_hard_limit default in compute_ideal_maxconn
This commit fixes 41275a691 ("MEDIUM: init: set default for fd_hard_limit via
DEFAULT_MAXFD").

fd_hard_limit is taken in account implicitly via 'ideal_maxconn' value in
all maxconn adjustements, when global.rlimit_memmax is set:

	MIN(global.maxconn, capped by global.rlimit_memmax, ideal_maxconn);

It also caps provided global.rlimit_nofile, if it couldn't be set as a current
process fd limit (see more details in the main() code).

So, lets set the default value for fd_hard_limit only, when there is no any
other haproxy-specific limit provided, i.e. rlimit_memmax, maxconn,
rlimit_nofile. Otherwise we may break users configs.

Please, note, that in master-worker mode, master does not need the
DEFAULT_MAXFD (1048576) as well, as we explicitly limit its maxconn to 100.

Must be backported in all stable versions until v2.6.0, including v2.6.0,
like the commit above.
2024-07-08 11:26:16 +02:00
Valentine Krasnobaeva
41275a6918 MEDIUM: init: set default for fd_hard_limit via DEFAULT_MAXFD
Let's provide a default value for fd_hard_limit, if it's not set in the
configuration. With this patch we could set some specific default via
compile-time variable DEFAULT_MAXFD as well. Hope, this will be helpfull for
haproxy package maintainers.

    make -j 8 TARGET=linux-glibc DEBUG=-DDEFAULT_MAXFD=50000

If haproxy is comipled without DEFAULT_MAXFD defined, the default will be set
to 1048576.

This is done to avoid killing the process by its watchdog, while it started
without any limitations in its configuration or in the command line and the
hard RLIMIT_NOFILE is extremely huge (~1000000000). We use in this case
compute_ideal_maxconn() to calculate maxconn and maxsock, maxsock defines the
size of internal fdtab, which becames very-very large as well. When
the process starts to simply loop over this fdtab (0(n)), this takes a lot of
time, so watchdog does it job.

To avoid this, maxconn now is always reduced to some reasonable value either
by explicit global.fd-hard-limit from configuration, or by its default. The
default may be changed at build-time and overwritten then by
global.fd-hard-limit at runtime. Explicit global.fd-hard-limit from the
configuration has always precedence over DEFAULT_MAXFD, if set.

Must be backported in all stable versions until v2.6.0, including v2.6.0.
2024-07-04 07:52:42 +02:00
Valentine Krasnobaeva
ed90ad895c REORG: init: encapsulate code that reads cfg files
Haproxy master process should not read its configuration the second time
after performing reexec and passing to MODE_MWORKER_WAIT. So, to make
this part of init() function more readable and to distinguish better the
point, where configs have been read, let's encapsulate it in a separate
function.
2024-06-27 16:09:38 +02:00
Valentine Krasnobaeva
5e06d45df7 REORG: init: encapsulate 'reload' sockpair and master CLI listeners creation
Let's encapsulate the logic of 'reload' sockpair and master CLI listeners
creation, used by master CLI into a separate function, as we needed this
only in master-worker runtime  mode. This makes the code of init() more
readable.
2024-06-27 16:08:42 +02:00
Valentine Krasnobaeva
6f613faa71 REORG: init: encapsulate CHECK_CONDITION logic in a func
As MODE_CHECK_CONDITION logic terminates the process anyway, no matter if
the test for the provided condition was successfull or not, let's
encapsulate it in a separate function. This makes the code of init() more
readable.
2024-06-27 16:01:01 +02:00
Valentine Krasnobaeva
10de58fbfb REORG: init: do MODE_CHECK_CONDITION logic first
In MODE_CHECK_CONDITION we only parse check_condition string, provided by
'-cc', and then we evaluate it. Haproxy process terminates at the
end of {if..else} block anyway, if the test has failed or passed. So, it
will be more appropriate to perform MODE_CHECK_CONDITION test first and
then do all other process runtime mode verifications.
2024-06-27 15:59:43 +02:00
Valentine Krasnobaeva
0d79c9bedf MINOR: cli/debug: show dev: add cmdline and version
'show dev' command is very convenient to obtain haproxy debugging information,
while process is run in container. Let's extend its output with version and
cmdline. cmdline is useful in a way, as it shows absolute binary path and its
arguments, because sometimes the person, who is debugging failing container is
not the same, who has created and deployed it.

argc and argv are stored in the exported global structure, because
feed_post_mortem() is added as a post check function callback in the
post_check_list. So we can't simply change the signature of
feed_post_mortem(), without breaking other post check callbacks APIs.

Parsers are not supposed to modify argv, so we can safely bypass its pointer
to debug_parse_cli_show_dev(), without copying all argument stings somewhere
in the heap or on stack.
2024-06-26 07:38:21 +02:00
Amaury Denoyelle
f55748a422 MAJOR: config: prevent QUIC with clients privileged port by default
Previous commit introduce new protection mechanism to forbid
communications with clients which use a privileged source port. By
default, this mechanism is disabled for every protocols.

This patch changes the default value and activate the protection
mechanism for QUIC protocol. This is justified as it is a probable sign
of DNS/NTP amplification attack.

This is labelled as major as it can be a breaking change with some
network environments.
2024-05-24 14:36:31 +02:00
Amaury Denoyelle
45f40bac4c MEDIUM: config: prevent communication with privileged ports
This commit introduces a new global setting named
harden.reject_privileged_ports.{tcp|quic}. When active, communications
with clients which use privileged source ports are forbidden. Such
behavior is considered suspicious as it can be used as spoofing or
DNS/NTP amplication attack.

Value is configured per transport protocol. For each TCP and QUIC
distinct code locations are impacted by this setting. The first one is
in sock_accept_conn() which acts as a filter for all TCP based
communications just after accept() returns a new connection. The second
one is dedicated for QUIC communication in quic_recv(). In both cases,
if a privileged source port is used and setting is disabled, received
message is silently dropped.

By default, protection are disabled for both protocols. This is to be
able to backport it without breaking changes on stable release.

This should be backported as it is an interesting security feature yet
relatively simple to implement.
2024-05-24 14:36:31 +02:00
Valentine Krasnobaeva
4a9e3e102e BUG/MINOR: haproxy: only tid 0 must not sleep if got signal
This patch fixes the commit eea152ee68
("BUG/MINOR: signals/poller: ensure wakeup from signals").

There is some probability that run_poll_loop() becomes inifinite, if
TH_FL_SLEEPING is withdrawn from all threads in the second signal_queue_len
check, when a signal has received just after the first one.

In such particular case, the 'wake' variable, which is used to terminate
thread's poll loop is never reset to 0. So, we never enter to the "stopping"
part of the run_poll_loop() and threads, except the one with id 0 (tid 0
handles signals), will continue to call _do_poll() eternally and will never
sleep, as its TH_FL_SLEEPING flag was unset.

This flag needs to be removed only for the tid 0, as it was done in the first
signal_queue_len check.

This fixes an issue #2537 "infinite loop when shutting down".

This fix must be backported in every stable version.
2024-05-06 18:39:08 +02:00
Ilia Shipitsin
a65c6d3574 CLEANUP: assorted typo fixes in the code and comments
This is 42nd iteration of typo fixes
2024-05-03 09:01:36 +02:00
Valentine Krasnobaeva
5cbb278fae MINOR: capabilities: add cap_sys_admin support
If 'namespace' keyword is used in the backend server settings or/and in the
bind string, it means that haproxy process will call setns() to change its
default namespace to the configured one and then, it will create a
socket in this new namespace. setns() syscall requires CAP_SYS_ADMIN
capability in the process Effective set (see man 2 setns). Otherwise, the
process must be run as root.

To avoid to run haproxy as root, let's add cap_sys_admin capability in the
same way as we already added the support for some other network capabilities.

As CAP_SYS_ADMIN belongs to CAP_SYS_* capabilities type, let's add a separate
flag LSTCHK_SYSADM for it. This flag is set, if the 'namespace' keyword was
found during configuration parsing. The flag may be unset only in
prepare_caps_for_setuid() or in prepare_caps_from_permitted_set(), which
inspect process EUID/RUID and Effective and Permitted capabilities sets.

If system doesn't support Linux capabilities or 'cap_sys_admin' was not set
in 'setcap', but 'namespace' keyword is presented in the configuration, we
keep the previous strict behaviour. Process, that has changed uid to the
non-priviledged user, will terminate with alert. This alert invites the user
to recheck its configuration.

In the case, when haproxy will start and run under a non-root user and
'cap_sys_admin' is not set, but 'namespace' keyword is presented, this patch
does not change previous behaviour as well. We'll still let the user to try
its configuration, but we inform via warning, that unexpected things, like
socket creation errors, may occur.
2024-04-30 21:40:17 +02:00
William Lallemand
2ab42dddc4 BUG/MINOR: mworker: reintroduce way to disable seamless reload with -x /dev/null
Since the introduction of the automatic seamless reload using the
internal socketpair, there is no way of disabling the seamless reload.

Previously we just needed to remove -x from the startup command line,
and remove any "expose-fd" keyword on stats socket lines.

This was introduced in 2be557f7c ("MEDIUM: mworker: seamless reload use
the internal sockpairs").

The patch copy /dev/null again and pass it to the next exec so we never
try to get socket from the -x.

Must be backported as far as 2.6.
2024-04-26 15:25:49 +02:00
Amaury Denoyelle
34ae7755b3 MINOR: stats: apply stats-file on process startup
This commit is the first one of a serie to implement preloading of
haproxy counters via stats-file parsing.

This patch defines a basic apply_stats_file() function. It implements
reading line by line of a stats-file without any parsing for the moment.
It is called automatically on process startup via init().
2024-04-26 11:29:25 +02:00
Amaury Denoyelle
c02ec9a9db BUG/MINOR: backend: use cum_sess counters instead of cum_conn
This commit is part of a serie to align counters usage between
frontends/listeners on one side and backends/servers on the other.

"stot" metric refers to the total number of sessions. On backend side,
it is interpreted as a number of streams. Previously, this was accounted
using <cum_sess> be_counters field for servers, but <cum_conn> instead
for backend proxies.

Adjust this by using <cum_sess> for both proxies and servers. As such,
<cum_conn> field can be removed from be_counters.

Note that several diagnostic messages which reports total frontend and
backend connections were adjusted to use <cum_sess>. However, this is an
outdated and misleading information as it does reports streams count on
backend side. These messages should be fixed in a separate commit.

This should be backported to all stable releases.
2024-04-22 10:35:18 +02:00
Valentine Krasnobaeva
865db6307f MINOR: init: use RLIMIT_DATA instead of RLIMIT_AS
Limiting total allocatable process memory (VSZ) via setting RLIMIT_AS limit is
no longer effective, in order to restrict memory consumption at run time.
We can see from process memory map below, that there are many holes within
the process VA space, which bumps its VSZ to 1.5G. These holes are here by
many reasons and could be explaned at first by the full randomization of
system VA space. Now it is usually enabled in Linux kernels by default. There
are always gaps around the process stack area to trap overflows. Holes before
and after shared libraries could be explained by the fact, that on many
architectures libraries have a 'preferred' address to be loaded at; putting
them elsewhere requires relocation work, and probably some unshared pages.
Repetitive holes of 65380K are most probably correspond to the header that
malloc has to allocate before asked a claimed memory block. This header is
used by malloc to link allocated chunks together and for its internal book
keeping.

	$ sudo pmap -x -p `pidof haproxy`
	127136:   ./haproxy -f /home/haproxy/haproxy/haproxy_h2.cfg
	Address           Kbytes     RSS   Dirty Mode  Mapping
	0000555555554000     388      64       0 r---- /home/haproxy/haproxy/haproxy
	00005555555b5000    2608    1216       0 r-x-- /home/haproxy/haproxy/haproxy
	0000555555841000     916      64       0 r---- /home/haproxy/haproxy/haproxy
	0000555555926000      60      60      60 r---- /home/haproxy/haproxy/haproxy
	0000555555935000     116     116     116 rw--- /home/haproxy/haproxy/haproxy
	0000555555952000    7872    5236    5236 rw---   [ anon ]
	00007fff98000000     156      36      36 rw---   [ anon ]
	00007fff98027000   65380       0       0 -----   [ anon ]
	00007fffa0000000     156      36      36 rw---   [ anon ]
	00007fffa0027000   65380       0       0 -----   [ anon ]
	00007fffa4000000     156      36      36 rw---   [ anon ]
	00007fffa4027000   65380       0       0 -----   [ anon ]
	00007fffa8000000     156      36      36 rw---   [ anon ]
	00007fffa8027000   65380       0       0 -----   [ anon ]
	00007fffac000000     156      36      36 rw---   [ anon ]
	00007fffac027000   65380       0       0 -----   [ anon ]
	00007fffb0000000     156      36      36 rw---   [ anon ]
	00007fffb0027000   65380       0       0 -----   [ anon ]
	...
	00007ffff7fce000       4       4       0 r-x--   [ anon ]
	00007ffff7fcf000       4       4       0 r---- /usr/lib/x86_64-linux-gnu/ld-2.31.so
	00007ffff7fd0000     140     140       0 r-x-- /usr/lib/x86_64-linux-gnu/ld-2.31.so
	...
	00007ffff7ffe000       4       4       4 rw---   [ anon ]
	00007ffffffde000     132      20      20 rw---   [ stack ]
	ffffffffff600000       4       0       0 --x--   [ anon ]
	---------------- ------- ------- -------
	total kB         1499288   75504   72760

This exceeded VSZ makes impossible to start an haproxy process with 200M
memory limit, set at its initialization stage as RLIMIT_AS. We usually
have in this case such cryptic output at stderr:

	$ haproxy -m 200 -f haproxy_quic.cfg
        (null)(null)(null)(null)(null)(null)

At the same time the process RSS (a memory really used) is only 75,5M.
So to make process memory accounting more realistic let's base the memory
limit, set by -m option, on RSS measurement and let's use RLIMIT_DATA instead
of RLIMIT_AS.

RLIMIT_AS was used before, because earlier versions of haproxy always allocate
memory buffers for new connections, but data were not written there
immediately. So these buffers were not instantly counted in RSS, but were
always counted in VSZ. Now we allocate new buffers only in the case, when we
will write there some data immediately, so using RLIMIT_DATA becomes more
appropriate.
2024-04-19 17:36:40 +02:00
Ilya Shipitsin
ab7f05daba CLEANUP: assorted typo fixes in the code and comments
This is 41st iteration of typo fixes
2024-04-17 11:14:44 +02:00
Willy Tarreau
018443b8a1 BUILD: makefile: get rid of the CPU variable
The CPU variable, when used, is almost always exclusively used with
"generic" to disable any CPU-specific optimizations, or "native" to
enable "-march=native". Other options are not used and are just making
CPU_CFLAGS more confusing.

This commit just drops all pre-configured variants and replaces them
with documentation about examples of supported options. CPU_CFLAGS is
preserved as it appears that it's mostly used as a proxy to inject the
distro's CFLAGS, and it's just empty by default.

The CPU variable is checked, and if set to anything but "generic", it
emits a warning about its deprecation and invites the user to read
INSTALL.

Users who would just set CPU_CFLAGS will be able to continue to do so,
those who were using CPU=native will have to pass CPU_CFLAGS=-march=native
and those who were passing one of the other options will find it in the
doc as well.

Note that this also removes the "CPU=" line from haproxy -vv, that most
users got used to seeing set to "generic" or occasionally "native"
anyway, thus that didn't provide any useful information.
2024-04-11 17:33:28 +02:00
Valentine Krasnobaeva
f0b6436f57 MEDIUM: capabilities: check process capabilities sets
Since the Linux capabilities support add-on (see the commit bd84387beb26
("MEDIUM: capabilities: enable support for Linux capabilities")), we can also
check haproxy process effective and permitted capabilities sets, when it
starts and runs as non-root.

Like this, if needed network capabilities are presented only in the process
permitted set, we can get this information with capget and put them in the
process effective set via capset. To do this properly, let's introduce
prepare_caps_from_permitted_set().

First, it checks if binary effective set has CAP_NET_ADMIN or CAP_NET_RAW. If
there is a match, LSTCHK_NETADM is removed from global.last_checks list to
avoid warning, because in the initialization sequence some last configuration
checks are based on LSTCHK_NETADM flag and haproxy process euid may stay
unpriviledged.

If there are no CAP_NET_ADMIN and CAP_NET_RAW in the effective set, permitted
set will be checked and only capabilities given in 'setcap' keyword will be
promoted in the process effective set. LSTCHK_NETADM will be also removed in
this case by the same reason. In order to be transparent, we promote from
permitted set only capabilities given by user in 'setcap' keyword. So, if
caplist doesn't include CAP_NET_ADMIN or CAP_NET_RAW, LSTCHK_NETADM would not
be unset and warning about missing priviledges will be emitted at
initialization.

Need to call it before protocol_bind_all() to allow binding to priviledged
ports under non-root and 'setcap cap_net_bind_service' must be set in the
global section in this case.
2024-04-05 18:01:54 +02:00
Valentine Krasnobaeva
e4306fb822 BUG/MINOR: init: relax LSTCHK_NETADM checks for non root
Linux capabilities support and ability to preserve it for running process
after switching to a global.uid was added recently by the commit bd84387beb26
("MEDIUM: capabilities: enable support for Linux capabilities")).
This new feature hasn't yet been taken into account by last config checks,
which are performed at initialization stage.

So, to update it, let's perform it after set_identity() call. Like this,
current EUID is already changed to a global.uid and prepare_caps_for_setuid()
would unset LSTCHK_NETADM flag, only if capabilities given in the 'setcap'
keyword in the configuration file were preserved.

Otherwise, if system doesn't support Linux capabilities or they were not set
via 'setcap', we keep the previous strict behaviour: process will terminate
with an alert, in order to insist that user: either needs to change
run UID (worst case: start and run as root), or he needs to set/recheck
capabilities listed as 'setcap' arguments.

In the case, when haproxy will start and run under a non-root user this patch
doesn't change the previous behaviour: we'll still let him try the
configuration, but we inform via warning that unexpected things may occur.

Need to be backported until v2.9, including v2.9.
2024-04-05 18:01:54 +02:00
Aurelien DARRAGON
56d8074798 MINOR: proxy: add PR_FL_CHECKED flag
PR_FL_CHECKED is set on proxy once the proxy configuration was fully
checked (including postparsing checks).

This information may be useful to functions that need to know if some
config-related proxy properties are likely to change or not due to parsing
or postparsing/check logics. Also, during runtime, except for some rare cases
config-related proxy properties are not supposed to be changed.
2024-04-04 19:10:01 +02:00