Commit Graph

23650 Commits

Author SHA1 Message Date
Willy Tarreau
033db091fc BUILD: import/mt_list: support building with TCC
TCC is often convenient to quickly test builds, run CI tests etc. It has
limited thread support (e.g. no thread-local stuff) but that is often
sufficient for testing. TCC lacks __atomic_exchange_n() but has the
exactly equivalent __atomic_exchange(), and doesn't have any barrier.
For this reason we force the atomic_exchange to use the stricter SEQ_CST
mem ordering that allows to ignore the barrier.

[wt: that's upstream commit ca8b865 ("BUILD: support building with TCC")]
2024-11-05 15:43:17 +01:00
Christopher Faulet
d1adfd9fe4 BUG/MEDIUM: promex: Fix dump of extra counters
When extra counters are dumped for an entity (frontend, backend, server or
listener), there is a filter on capabilities. Some extra counters are not
available for all entities and must be ignored. However, when this was
performed, the field number, used as an index to dump the metric value, was
still incremented while it should not and leads to an overflow or a stats
mix-up.

This patch must be backported to 3.0.
2024-11-05 15:36:41 +01:00
William Lallemand
e75a019fba MINOR: startup: tune.renice.{startup,runtime} allow to change priorities
This commit introduces the tune.renice.startup and tune.renice.runtime
global keywords that allows to change the priority with setpriority().

tune.renice.startup is parsed and applied in the worker or the standalone
process for configuration parsing. If this keyword is used alone, the
nice value is changed to the previous one after configuration parsing.

tune.renice.runtime is applied after configuration parsing, so in the
worker or a standalone process. Combined with tune.renice.startup it
allows to have a different nice value during configuration parsing and
during runtime.

The feature was discussed in github issue #1919.

Example:

   global
        tune.renice.startup 15
        tune.renice.runtime 0
2024-11-04 17:48:58 +01:00
Willy Tarreau
2092199353 [RELEASE] Released version 3.1-dev11
Released version 3.1-dev11 with the following main changes :
    - BUG/MINOR: httpclient: return NULL when no proxy available during httpclient_new()
    - BUG/MEDIUM: mworker/httpclient: initialization skipped by accident in mworker mode
    - BUG/MINOR: resolvers/mworker: missing default resolvers in mworker mode
    - MINOR: mworker/ocsp: skip ocsp-update proxy init in master
    - BUG/MEDIUM: stconn: Wait iobuf is empty to shut SE down during a check send
    - MINOR: mux-h1: Show the SD iobuf in trace messages on stream send events
    - MINOR: mux-h1: Add a trace on shutdown when keep-alive is not possible
    - BUG/MINOR: http-ana: Don't report a server abort if response payload is invalid
    - BUG/MEDIUM: stconn: Check FF data of SC to perform a shutdown in sc_notify()
    - BUG/MAJOR: filters/htx: Add a flag to state the payload is altered by a filter
    - REGTESTS: Never reuse server connection in http-messaging/truncated.vtc
    - BUG/MINOR: quic: avoid leaking post handshake frames
    - MINOR: quic: send new tokens (NEW_TOKEN) even for 1RTT sessions
    - BUG/MEDIUM: quic: avoid freezing 0RTT connections
    - DOC: config: fix rfc7239 forwarded typo in desc
    - MINOR: http_ext: implement rfc7239_{nn,np} converters
    - CLEANUP: http_ext: remove useless BUG_ON() in http_handle_xot_header()
    - BUG/MINOR: sample: free err2 in smp_resolve_args for type ARGT_REG
    - MINOR: arg: add an argument type for identifier
    - BUILD: buffers: keep b_getblk_nc() and b_peek_varint() in buf.h
    - CLEANUP: buffers: simplify b_get_varint()
    - OPTIM: buffers: avoid a useless wrapping check for ofs == 0
    - MINOR: debug: make mark_tainted() return the previous value
    - MINOR: chunk: drop the global thread_dump_buffer
    - MINOR: debug: split ha_thread_dump() in two parts
    - MINOR: debug: slightly change the thread_dump_pointer signification
    - MINOR: debug: make ha_thread_dump_done() take the pointer to be used
    - MINOR: debug: replace ha_thread_dump() with its two components
    - MEDIUM: debug: on panic, make the target thread automatically allocate its buf
    - BUILD: mux-h2/traces: fix build on 32-bit due to size of the DATA frame
    - CI: prepare Coverity build for Ubuntu 24
    - CI: bump development builds explicitely to Ubuntu 24.04
    - CI: modernize macos builds to macos-15
    - BUG/MINOR: mworker: fix mworker-max-reloads parser
    - MINOR: mux-quic: simplify sending of empty STREAM FIN
    - BUG/MINOR: mux-quic: do not close STREAM with empty FIN if no data sent
    - CLEANUP: debug: make the BUG_ON() macros check the condition in the outer one
    - MEDIUM: debug: add match counters for BUG_ON/WARN_ON/CHECK_IF
    - MINOR: debug: add a new debug macro COUNT_IF()
    - MINOR: debug: add "debug dev counters" to list code counters
    - BUG/MEDIUM: stats-html: Never dump more data than expected during 0-copy FF
    - BUG/MEDIUM: mux-h2: Remove H2S from send list if data are sent via 0-copy FF
    - BUG/MINOR: stconn: Pretend the SE have more data to deliver on abortonclose
    - CLEANUP: stream: remove outdated comments
    - DEBUG: stream: Add debug counters to track some client/server aborts
    - DEBUG: mux-h1: Add debug counters to track some errors
    - MINOR: mux-h1: Add support of the debug string for logs
    - MINOR: stream: maintain per-stream counters of the number of passes on code
    - MINOR: filters: add per-filter call counters
    - MINOR: sample: add the "when" converter to condition some expressions
    - BUG/MEDIUM: connection/http-reuse: fix address collision on unhandled address families
    - BUILD: spoe: fix build warning on older gcc around sub-struct initialization
    - Revert "OPTIM: mux-h2: make h2_send() report more accurate wake up conditions"
    - DEBUG: mux-h1: Add debug counters to track errors with in/out pending data
    - BUG/MINOR: mux-h1: Fix conditions on pipe in some COUNT_IF()
    - MINOR: activity/memprofile: show per-DSO stats
    - BUG/MINOR: mworker/cli: show master startup logs in recovery mode
    - MINOR: mworker: stop MASTER proxy listener on worker mcli sockpair
    - MINOR: error: simplify startup_logs_init_shm
    - BUG/MINOR: mworker: show worker warnings in startup logs
    - CLEANUP: mworker: clean mworker_reexec
    - MINOR: mworker/cli: split mworker_cli_proxy_create
    - BUG/MINOR: server: fix dynamic server leak with check on failed init
    - BUG/MEDIUM: server: fix race on servers_list during server deletion
    - BUG/MEDIUM: stconn: Report blocked send if sends are blocked by an error
    - BUG/MINOR: http-ana: Fix wrong client abort reports during responses forwarding
    - BUG/MINOR: stconn: Don't disable 0-copy FF if EOS was reported on consumer side
    - MINOR: mworker/cli: add 'debug' to 'show proc'
    - MINOR: mworker/cli: remove comment line for program when useless
    - MINOR: mworker/cli: 'show proc debug' for old workers
    - BUILD: debug: silence a build warning with threads disabled
    - CLEANUP: mux-h2: remove the unused "full" variable in h2_frt_transfer_data()
    - MINOR: pools: export the pools variable
    - MINOR: debug: place a magic pattern at the beginning of post_mortem
    - MINOR: debug: place the post_mortem struct in its own section.
    - MINOR: debug: store important pointers in post_mortem
    - MINOR: debug: do not limit backtraces to stuck threads
    - MINOR: cli: remove non-printable characters from 'debug dev fd'
    - MINOR: cli: add an 'echo' command
    - MINOR: debug: also add a pointer to struct global to post_mortem
    - CLEANUP: mworker: make mworker_create_master_cli more readable
    - BUG/MEIDUM: mworker: fix fd leak from master to worker
    - BUG/MINOR: mworker/cli: fix mworker_cli_global_proxy_new_listener
    - MINOR: tools: add strnlen2() helper
    - CLEANUP: log: use strnlen2() in _lf_text_len() to compute string length
    - DOC: design: add notes about more detailed error reporting for logs
    - MINOR: debug: also add fdtab and acitvity to struct post_mortem
    - MINOR: debug: remove the redundant process.thread_info array from post_mortem
    - DEV: gdb: add a number of gdb scripts to navigate in core dumps
    - BUG/MINOR: trace: stop rewriting argv with -dt
    - MEDIUM: protocol: make abns a custom unix socket address family
    - MEDIUM: protocol: rely on AF_CUST_ABNS family to recognize ABNS sockets
    - CLEANUP: tools: rely on address family to detect ABNS sockets
    - MINOR: protocol: create abnsz socket address family
    - MINOR: sock: restore effective UNIX family in sock_get_old_sockets()
    - MEDIUM: sock: also restore effective unix family in get_{src,dst}()
    - MEDIUM: sock_unix: use per-family addrcmp function
    - MEDIUM: socket: add zero-terminated ABNS alternative
    - BUG/MINOR: ssl/cli: 'set ssl cert' does not check the transaction name correctly
    - BUG/MINOR: mworker: mworker_reexec: unset MODE_STARTING before free startup logs ring
    - BUG/MINOR: errors: startup_logs_free: set global startup_logs ptr to NULL
    - BUG/MINOR: errors: print_message: don't allocate startup logs ring
    - BUG/MINOR: startup: don't fork worker if started with -c -W
    - BUG/MINOR: startup: dump libs only in worker if started with -W -dL
    - BUG/MINOR: startup: dump keywords only in worker if started with -W -dKAll
    - BUG/MINOR: startup: don't dump polling info for master in verbose mode
    - CI: switch QUIC Interop on AWS-LC to common docker image
    - CI: switch QUIC Interop on LibreSSL to common docker image
    - CI: enable chacha20 test on LibreSSL QUIC Interop
    - DOC: config: add missing glitch_{cnt,rate} data types
    - DOC: config: add missing glitch_{cnt,rate} sample definitions
    - CI: LibreSSL QUIC Interop: fix docker context
    - DEBUG: mux-h1: Add H1C expiration dates in trace messages
    - BUG/MEDIUM: mux-h1: Fix how timeouts are applied on H1 connections
    - BUG/MINOR: http-ana: Report internal error if an action yields on a final eval
    - MINOR: stream: Save last evaluated rule on invalid yield
    - MINOR: quic: complete trace in qc_may_build_pkt()
    - MINOR: quic: move qc_send_mux() prototype into quic_tx.h
    - MINOR: stream: Replace last_rule_file/line fields by a more generic field
    - MINOR: stream: Save the last filter evaluated interrupting the processing
    - MINOR: stream: Save the entity waiting to continue its processing
    - MINOR: stream: Use an enum to identify last and waiting entities for streams
    - MINOR: stream: Add http-buffer-request option in the waiting entities
    - DOC: config: Add documentation about last_entity sample fetch
    - DOC: config: Add documentation about waiting_entity sample fetch
2024-11-01 10:17:02 +01:00
Christopher Faulet
1cd8173687 DOC: config: Add documentation about waiting_entity sample fetch
The commit adds the documentation for the waiting_entity sample fetch.
2024-10-31 20:47:59 +01:00
Christopher Faulet
6034080c49 DOC: config: Add documentation about last_entity sample fetch
The commit adds the documentation for the last_entity sample fetch.
2024-10-31 20:25:07 +01:00
Christopher Faulet
64554a55f4 MINOR: stream: Add http-buffer-request option in the waiting entities
When http-buffer-request option is set on a proxy, the processing will be
paused to wait the full request payload or a full buffer. So it is an entity
that block the processing, just like a rule or a filter that yields. So now,
it is reported as a waiting entity if an error or a timeout occurred.

To do so, an stream entity type is added for this option. There is no
pointer. And "waiting_entity" sample fetch returns the option name.
2024-10-31 20:24:50 +01:00
Christopher Faulet
c64712b085 MINOR: stream: Use an enum to identify last and waiting entities for streams
Instead of using 1 for last/waiting rule and 2 for last/waiting filter, an
enum is used. It is less ambiguous this way.
2024-10-31 20:24:37 +01:00
Christopher Faulet
537f20eb3e MINOR: stream: Save the entity waiting to continue its processing
When a rule or a filter yields because it waits for something to be able to
continue its processing, this entity is saved in the stream. If an error or
a timeout occurred, info on this entity may be retrieved via the
"waiting_entity" sample fetch, for instance to dump it in the logs. This
info may be useful to found root cause of some bugs because it is a way to
know the processing was temporarily stopped. This may explain timeouts for
instance.

The sample fetch is not documented yet.
2024-10-31 16:40:09 +01:00
Christopher Faulet
53de6da1c0 MINOR: stream: Save the last filter evaluated interrupting the processing
It is very similar to the last evaluated rule. When a filter returns an
error that interrupts the processing, it is saved in the stream, in the
last_entity field, with the type 2. The pointer on filter config is
saved. This pointer never changes during runtime and is part of the proxy's
structure. It is an element of the filter_configs list in the proxy
structure.

"last_entity" sample fetch was update accordingly. The filter identifier is
returned, if defined. Otherwise the save pointer.
2024-10-31 16:39:04 +01:00
Christopher Faulet
c9fa78e747 MINOR: stream: Replace last_rule_file/line fields by a more generic field
The last evaluated rule is now saved in a generic structure, named
last_entity, with a type to identify it. The idea is to be able to store
other kind of entity that may interrupt a specific processing.

The type of the last evaluated rule is set to 1. It will be replace later by
an enum to be more explicit. In addition, the pointer to the rule itself is
saved instead of its location.

The sample fetch "last_entity" was added to retrieve the information about
it. In this case, it is the rule localtion, the config file containing the
rule followed by the line where the rule is defined, separated by a
colon. This sample fetch is not documented yet.
2024-10-31 16:36:39 +01:00
Amaury Denoyelle
dcf334168c MINOR: quic: move qc_send_mux() prototype into quic_tx.h
qc_send_mux() is defined in quic_tx.c. As such, its prototype is moved
from quic_conn.h to quic_tx.h.
2024-10-31 15:35:31 +01:00
Amaury Denoyelle
a8738f4156 MINOR: quic: complete trace in qc_may_build_pkt()
Log the encryption level in qc_may_build_pkt(). This is necessary to
fully understand the sending conditions of the QUIC stack.
2024-10-31 15:35:31 +01:00
Christopher Faulet
0b7605491e MINOR: stream: Save last evaluated rule on invalid yield
When an action yields while it is not allowed, an internal error is
reported. This interrupts the processing. So info about the last evaluated
rule must be filled.

This patch may be bakcported if needed. If so, the commit ("MINOR: stream:
Save last evaluated rule on invalid yield") must be backported first.
2024-10-31 09:30:52 +01:00
Christopher Faulet
65ea29dcf8 BUG/MINOR: http-ana: Report internal error if an action yields on a final eval
This was already performed for tcp actions at content level, but not for
HTTP actions. It is always a bug, so it must be reported accordingly.

This patch may be backported to all stable versions.
2024-10-31 09:30:52 +01:00
Christopher Faulet
3c09b34325 BUG/MEDIUM: mux-h1: Fix how timeouts are applied on H1 connections
There were several flaws in the way the different timeouts were applied on
H1 connections. First, the H1C task handling timeouts was not created if no
client/server timeout was specified. But there are other timeouts to
consider. First, the client-fin/server-fin timeouts. But for frontend
connections, http-keey-alive and http-request timeouts may also be used. And
finally, on soft-stop, the close-spread-time value must be considered too.

So at the end, it is probably easier to always create a task to manage H1C
timeouts. Especially since the client/server timeouts are most often set.

Then, when the expiration date of the H1C's task must only be updated if the
considered timeout is set. So tick_add_ifset() must be used instead of
tick_add(). Otherwise, if a timeout is undefined, the taks may expire
immediately while it should in fact never expire.

Finally, the idle expiration date must only be considered for idle
connections.

This patch should be backported in all stable versions, at least as far as
2.6. On the 2.4, it will have to be slightly adapted for the idle_exp
part. On 2.2 and 2.0, the patch will have to be rewrite because
h1_refresh_timeout() is quite different.
2024-10-31 09:30:52 +01:00
Christopher Faulet
9fa5b379fa DEBUG: mux-h1: Add H1C expiration dates in trace messages
The expiration date of the H1C task and the H1C idle expiration date are now
dumped in the trace messages.
2024-10-31 09:30:52 +01:00
Ilia Shipitsin
976af317a4 CI: LibreSSL QUIC Interop: fix docker context
in the commit 98099287ee
building docker was switched to URL, but I forgotten to change context.

this is a followup fix.
2024-10-30 19:42:31 +01:00
Aurelien DARRAGON
0686fd8cfc DOC: config: add missing glitch_{cnt,rate} sample definitions
Following previous commit, when glitch_cnt and glitch_rate data types were
implemented in c9c6b683f ("MEDIUM: stick-tables: add a new stored type for
glitch_cnt and glitch_rate"), newly exposed samples such as
table_glitch_cnt(), table_glitch_rate, src_glitch_cnt() and
src_glitch_rate() were documented but their definitions was missing in
supported keywords list.

It should be backported in 3.0 with c9c6b683f
2024-10-30 17:47:30 +01:00
Aurelien DARRAGON
9a6fc2d474 DOC: config: add missing glitch_{cnt,rate} data types
When glitch_cnt and glitch_rate data types were implemented in
c9c6b683f ("MEDIUM: stick-tables: add a new stored type for glitch_cnt and
glitch_rate"), the data types list for "stick-table" keyword documentation
was overlooked.

This was reported by Nick Ramirez.

It should be backported in 3.0 with c9c6b683f.
2024-10-30 17:47:24 +01:00
Ilia Shipitsin
3ecca216b4 CI: enable chacha20 test on LibreSSL QUIC Interop
it was commented on purpose "until LibreSSL-4.0 is released".
lets enable it
2024-10-30 16:46:22 +01:00
Ilia Shipitsin
98099287ee CI: switch QUIC Interop on LibreSSL to common docker image
previously we used different docker images for different SSL libs,
now all of them are merged into one, lets switch to it
2024-10-30 16:46:06 +01:00
Ilia Shipitsin
4d40e9384c CI: switch QUIC Interop on AWS-LC to common docker image
previously we used different docker images for different SSL libs,
now all of them are merged into one, lets switch to it
2024-10-30 16:45:36 +01:00
Valentine Krasnobaeva
d3eb00e61d BUG/MINOR: startup: don't dump polling info for master in verbose mode
As master-worker fork happens now before step_init_2(), when pollers are
initialized and polling settings and dumped then in verbose and in debug modes
to stdout, it turns out that master and worker dump its same polling
settings separately. This creates long and messy output in these modes.

Polling settings are the same for master and for worker process for the moment.
Even if they would diverge in future we are interested here in worker's
settings. So, when started in the master-worker mode let's dump it only in the
worker context.

This doesn't need to be backported as related to the latest master-worker
refactoring.
2024-10-30 10:50:09 +01:00
Valentine Krasnobaeva
bbe7828d49 BUG/MINOR: startup: dump keywords only in worker if started with -W -dKAll
If haproxy was started with -W -dK*, after master-worker refactoring, we dump
registered keywords to stdout twice in master and in worker processes. This
information is redundant and output has no longer the right format. So, as the
keyword registration happens very early before the fork, let's dump keywords
only in the worker context, if haproxy was launched with -W.

This does not need to be backported, as related to the latest master-worker
refactoring.
2024-10-30 10:01:28 +01:00
Valentine Krasnobaeva
ea824aebc1 BUG/MINOR: startup: dump libs only in worker if started with -W -dL
If haproxy was started with -W -dL, after master-worker refactoring we dump
libs to stdout twice in master and in worker processes. This is information is
redundant. So let's show linked libraries only in the worker context, if
haproxy was started also with -W.

This does not need to be backported, as related to the latest master-worker
rework.
2024-10-30 10:00:40 +01:00
Valentine Krasnobaeva
d1c6d44976 BUG/MINOR: startup: don't fork worker if started with -c -W
Don't do master-worker fork if MODE_CHECK is detected from the command line along
with the master-worker mode. We should exit in MODE_CHECK, after the
configuration parsing and validation. So, with the new master-worker architecture
it's better to align this mode with the standalone.

This patch does not need to be backported, as related to the latest
master-worker rework.
2024-10-30 09:59:59 +01:00
Valentine Krasnobaeva
f0f03b98f7 BUG/MINOR: errors: print_message: don't allocate startup logs ring
Don't call startup_logs_init() in order to allocate the startup logs ring
again, if startup_logs pointer is NULL. Startup logs ring is allocated
explicitly in step_init_1 routine, when the process starts, and it's freed
explicitly for master process at the end of mworker_reexec scope. So, when
we no longer have this pointer, let's just save the log message in the
message buffer.

Otherwise, in case of master process, we will allocate the startup logs ring
again here and we will lost its address after execvp.

No need to backport this fix as it's related to the latest master-worker
refactoring.
2024-10-29 18:17:49 +01:00
Valentine Krasnobaeva
bf8c871e26 BUG/MINOR: errors: startup_logs_free: set global startup_logs ptr to NULL
ring_free() calls free() on the ring struct pointer, but startup_logs continues
to keep this address. So let's reset at the end startup_logs to NULL.
startup_logs is checked in print_message().

No need to backport this fix, as it's related to the latest master-worker
refactoring.
2024-10-29 18:17:49 +01:00
Valentine Krasnobaeva
cd57ee7ffa BUG/MINOR: mworker: mworker_reexec: unset MODE_STARTING before free startup logs ring
Flag MODE_STARTING should be unset for master just before freeing the startup
logs ring, as it triggers the copy of process logs to this ring, see the code
of print_message().

Moreover with this flag set, if startup logs ring pointer is NULL, any
print_message() triggered just before the execvp in mworker_reexec() will call
startup_logs_init(). So ring will be allocated again "discretely" and after
execvp we will lost its address, as in step_init_1() we will call again
startup_logs_init().

No need to backport this fix as it's related to the latest master-worker
refactoring.
2024-10-29 18:17:49 +01:00
William Lallemand
984d2cfb61 BUG/MINOR: ssl/cli: 'set ssl cert' does not check the transaction name correctly
Since commit  089c13850f ("MEDIUM: ssl: ssl-load-extra-del-ext work
only with .crt"), the 'set ssl cert' CLI command does not check
correctly if the transaction you are trying to update is the right one.

The consequence is that you could commit accidentaly a transaction on
the wrong certificate.

The fix introduces the check again in case you are not using
ssl-load-extra-del-ext.

This must be backported in all stable versions.
2024-10-29 16:01:07 +01:00
Tristan
18582ede05 MEDIUM: socket: add zero-terminated ABNS alternative
When an abstract unix socket is bound by HAProxy (using "abns@" prefix),
NUL bytes are appended at the end of its path until sun_path is filled
(for a total of 108 characters).

Here we add an alternative to pass only the non-NUL length of that path
to connect/bind calls, such that the effective path of the socket's name
is as humanly written. This may be useful to interconnect with existing
softwares that implement abstract sockets with this logic instead of the
default haproxy one.

This is achieved by implementing the "abnsz" socket prefix (instead of
"abns"), which stands for "zero-terminated ABNS". "abnsz" prefix may be
used anywhere "abns" is. Internally, haproxy uses the custom socket
family (AF_CUST_ABNS vs AF_CUST_ABNSZ) to differentiate default abns
sockets from zero-terminated ones.

Documentation was updated and regtest was added.

Fixes GH issues #977 and #2479

Co-authored-by: Aurelien DARRAGON <adarragon@haproxy.com>
2024-10-29 12:15:24 +01:00
Aurelien DARRAGON
43861e3234 MEDIUM: sock_unix: use per-family addrcmp function
Thanks to previous commit, we may now use dedicated addrcmp functions for
each UNIX address family. This allows to simplify sock_unix_addrcmp()
function and avoid useless checks in order to try to guess the socket
type.

In this patch we implement sock_abns_addrcmp() and sock_abnsz_addrcmp()
functions, which are respectively used for ABNS and ABNSZ custom families

sock_unix_addrcmp() now only holds regular UNIX socket comparing logic.
2024-10-29 12:15:09 +01:00
Aurelien DARRAGON
d879bf6600 MEDIUM: sock: also restore effective unix family in get_{src,dst}()
As in previous commit, let's push the logic a bit further in order to
properly restore the effective UNIX socket type when leveraging
get_src() and get_dst() sock functions, since they rely on getpeername()
and getsockname() under the hood, both of which will actually loose the
effective family and return AF_UNIX for all our custom UNIX sockets.

To do this, add sock_restore_unix_family() helper function from the logic
implemented in the previous commit, and call this function from get_src()
and get_dst() in case of unix socket prior to returning.
2024-10-29 12:15:03 +01:00
Aurelien DARRAGON
ae64444303 MINOR: sock: restore effective UNIX family in sock_get_old_sockets()
When getting sockets from older process in sock_get_old_sockets(), we
leverage getsockname() to fill sockaddr struct from known fd.

However, the kernel doesn't know about our custom UNIX families such
as CUST_ABNS and CUST_ABNSZ which are both based on AF_UNIX real family.

Since haproxy socket API relies on effective family (and not real family)
to recognize the socket type instead of having to guess it by analyzing
the path content, let's restore it right after getsockname() since we
have all the infos needed to deduce the right family.

If the path starts with a NULL byte, we know that it is an abstract sock.
Then we simply check <addrlen> value from getsockname() to know if the
addr makes uses of the whole path space (normal ABNS) or partial path
space (zero ABNS / aka ABNZ) terminated by 0.
2024-10-29 12:14:57 +01:00
Willy Tarreau
d24768ab44 MINOR: protocol: create abnsz socket address family
For now it's the same as abns. We'll need to modify sock_unix_addrcmp(),
and a few other ones to support effective path length when dealing with
the \0. Let's check with Tristan's patch for this (upcoming patch).

Co-authored-by: Aurelien DARRAGON <adarragon@haproxy.com>
2024-10-29 12:14:50 +01:00
Aurelien DARRAGON
9fea4a3ca5 CLEANUP: tools: rely on address family to detect ABNS sockets
Following previous commit, in str2sa_range(), make use of address' family
which was just set to check if the socket is ABNS or not instead of
relying on an extra boolean to save this info.
2024-10-29 12:14:44 +01:00
Aurelien DARRAGON
5d766260f0 MEDIUM: protocol: rely on AF_CUST_ABNS family to recognize ABNS sockets
Now that we can easily distinguish regular UNIX socket from ABNS sockets
by simply looking at the address family, stop looking at the first byte
from addr->sun_path to guess if the socket is an ABNS one or not. Looking
at the family is straightforward and will allow to differentiate between
upcoming ABNSZ and ABNS (where looking at the first byte from path won't
help anymore).
2024-10-29 12:14:37 +01:00
Willy Tarreau
78ac312bbd MEDIUM: protocol: make abns a custom unix socket address family
This is a pre-requisite to adding the abnsz socket address family:

in this patch we make use of protocol API rework started by 732913f
("MINOR: protocol: properly assign the sock_domain and sock_family") in
order to implement a dedicated address family for ABNS sockets (based on
UNIX parent family).

Thanks to this, it will become trivial to implement a new ABNSZ (for abns
zero) family which is essentially the same as ABNS but with a slight
difference when it comes to path handling (ABNS uses the whole sun_path
length, while ABNSZ's path is zero terminated and evaluation stops at 0)

It was verified that this patch doesn't break reg-tests and behaves
properly (tests performed on the CLI with show sess and show fd).

Anywhere relevant, AF_CUST_ABNS is handled alongside AF_UNIX. If no
distinction needs to be made, real_family() is used to fetch the proper
real family type to handle it properly.

Both stream and dgram were converted, so no functional change should be
expected for this "internal" rework, except that proto will be displayed
as "abns_{stream,dgram}" instead of "unix_{stream,dgram}".

Before ("show sess" output):
  0x64c35528aab0: proto=unix_stream src=unix:1 fe=GLOBAL be=<NONE> srv=<none> ts=00 epoch=0 age=0s calls=1 rate=0 cpu=0 lat=0 rq[f=848000h,i=0,an=00h,ax=] rp[f=80008000h,i=0,an=00h,ax=] scf=[8,0h,fd=21,rex=10s,wex=] scb=[8,1h,fd=-1,rex=,wex=] exp=10s rc=0 c_exp=

After:
  0x619da7ad74c0: proto=abns_stream src=unix:1 fe=GLOBAL be=<NONE> srv=<none> ts=00 epoch=0 age=0s calls=1 rate=0 cpu=0 lat=0 rq[f=848000h,i=0,an=00h,ax=] rp[f=80008000h,i=0,an=00h,ax=] scf=[8,0h,fd=22,rex=10s,wex=] scb=[8,1h,fd=-1,rex=,wex=] exp=10s rc=0 c_exp=

Co-authored-by: Aurelien DARRAGON <adarragon@haproxy.com>
2024-10-29 12:14:25 +01:00
William Lallemand
596db3ef86 BUG/MINOR: trace: stop rewriting argv with -dt
When using trace with -dt, the trace_parse_cmd() function is doing a
strtok which write \0 into the argv string.

When using the mworker mode, and reloading, argv was modified and the
trace won't work anymore because the first : is replaced by a '\0'.

This patch fixes the issue by allocating a temporary string so we don't
modify the source string directly. It also replace strtok by its
reentrant version strtok_r.

Must be backported as far as 2.9.
2024-10-29 11:01:47 +01:00
Willy Tarreau
e240be5495 DEV: gdb: add a number of gdb scripts to navigate in core dumps
These is a collection of functions I'm occasionally using to navigate
in core dumps. Only working ones were extracted.

Those requiring knowledge of global variables (e.g. pools, proxy list)
use the one extracted from the post_mortem struct. That one is defined
in post-mortem.gdb and needs to be initialized using "pm_init post_mortem"
or "pm_init <pointer>". From this point a number of global variables are
accessible even if symbols are missing; those ones are then used by other
functions to dump streams, threads, pools, proxies etc.

The files can be sourced or copy-pasted into a gdb session. It's worth
trying to keep them up-to-date, as the old ones used to navigate through
tasks are no longer usable due to massive changes.
2024-10-28 17:55:08 +01:00
Willy Tarreau
52240680f1 MINOR: debug: remove the redundant process.thread_info array from post_mortem
That one is huge and unneeded since we now have the pointer to the
whole thread_info[] array, which does contain the freshest version
of these info and many more. Let's just get rid of it entirely.
2024-10-28 17:14:48 +01:00
Willy Tarreau
da5cf52173 MINOR: debug: also add fdtab and acitvity to struct post_mortem
These ones are often used as well when trying to analyse sequences of
events, let's add them.
2024-10-28 17:14:48 +01:00
Willy Tarreau
20ffa35f66 DOC: design: add notes about more detailed error reporting for logs
These are the notes of a day long code analysis session (CFA+WTA)
aimed at figuring what's missing during most code troubleshooting
sessions.  The goal is to provide good indications about what rules/
filters were still active when the processing ended (timeout, error
etc), what subscribers are still active (indicating waiting for an
event), and what shut/abort events were met at the various levels
of each side's stack, in each direction.
2024-10-28 17:14:48 +01:00
Aurelien DARRAGON
6d5b32daad CLEANUP: log: use strnlen2() in _lf_text_len() to compute string length
Thanks to previous commit, we can now use strnlen2() function to perform
strnlen() portable equivalent instead of re-implementing the logic under
_lf_text_len() function.
2024-10-28 14:59:42 +01:00
Aurelien DARRAGON
24131dee30 MINOR: tools: add strnlen2() helper
strnlen2() is functionally equivalent to strnlen(). Goal is to provide
an alternative to strnlen() which is not portable since it requires
_POSIX_C_SOURCE >= 200809L
2024-10-28 14:59:35 +01:00
Valentine Krasnobaeva
7855069655 BUG/MINOR: mworker/cli: fix mworker_cli_global_proxy_new_listener
There is no need to close proc->ipc_fd[0] on the error path in
mworker_cli_global_proxy_new_listener(), as it's already closed before by the
caller.
2024-10-26 22:53:24 +02:00
Valentine Krasnobaeva
4931d1ca5f BUG/MEIDUM: mworker: fix fd leak from master to worker
During re-execution master keeps always opened "reload" sockpair FDs and
shared sockpair ipc_fd[0], the latter is using to transfert listeners sockets
from the previously forked worker to the new one. So, these master's FDs are
inherited in the newly forked worker and must be closed in its context.

"reload" sockpair inherited FDs and shared sockpair FD (ipc_fd[0]) are closed
separately, becase master doesn't recreate "reload" sockpair each time after
its re-exec. It always keeps the same FDs for this "reload" sockpair. So in
worker context it can be closed immediately after the fork.

At contrast, shared sockpair is created each time after reload, when the new
worker will be forked. So, if N previous workers are still exist at this moment,
the new worker will inherit N ipc_fd[0] from master. So, it's more save to
close all these FDs after get_listeners_fd() and bind_listeners() calls.
Otherwise, early closed FDs in the worker context will be immediately bound to
listeners and we could potentially have some bugs.
2024-10-26 22:53:24 +02:00
Valentine Krasnobaeva
745a4c5e93 CLEANUP: mworker: make mworker_create_master_cli more readable
Using nested 'if' operator, while checking if we will need to allocate again the
"reload" sockpair, does not degrade performance, as mworker_create_master_cli is
a startup routine.

This nested 'if' (we check one condition in each operator) makes more visible the
fact, that the "reload" sockpair is allocated only once, when the master process
starts and it does not re-allocated again (hence, its FDs are not closed) during
reloads. This way of checking multiple conditions here makes more easy to spot
this fact, while analysing the code in order to investigate FD leaks between
master and worker.
2024-10-26 22:26:49 +02:00
Willy Tarreau
2f04ebe14a MINOR: debug: also add a pointer to struct global to post_mortem
The pointer to struct global is also an important element to have in
post_mortem given that it's used a lot to take decisions in the code.
Let's just add it. It's worth noting that we could get rid of argc/argv
at this point since they're also present in the global struct, but they
don't cost much there anyway.
2024-10-26 11:33:09 +02:00