2081 Commits

Author SHA1 Message Date
Remi Tricot-Le Breton
98b930d043 MINOR: ssl: Define a default https log format
This patch adds a new httpslog option and a new HTTP over SSL log-format
that expands the default HTTP format and adds SSL specific information.
2021-07-29 15:40:45 +02:00
Remi Tricot-Le Breton
7c6898ee49 MINOR: ssl: Add new ssl_fc_hsk_err sample fetch
This new sample fetch along the ssl_fc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the frontend's perspective). The errors happening during
the client's certificate verification will still be given by the
ssl_c_err and ssl_c_ca_err fetches. This new fetch will only hold errors
retrieved by the OpenSSL ERR_get_error function.
2021-07-29 15:40:45 +02:00
Remi Tricot-Le Breton
3d2093af9b MINOR: connection: Add a connection error code sample fetch
The fc_conn_err and fc_conn_err_str sample fetches give information
about the problem that made the connection fail. This information would
previously only have been given by the error log messages meaning that
thanks to these fetches, the error log can now be included in a custom
log format. The log strings were all found in the conn_err_code_str
function.
2021-07-29 15:40:45 +02:00
Willy Tarreau
252412316e MEDIUM: proxy: remove long-broken 'option http_proxy'
This option had always been broken in HTX, which means that the first
breakage appeared in 1.9, that it was broken by default in 2.0 and that
no workaround existed starting with 2.1. The way this option works is
praticularly unfit to the rest of the configuration and to the internal
architecture. It had some uses when it was introduced 14 years ago but
nowadays it's possible to do much better and more reliable using a
set of "http-request set-dst" and "http-request set-uri" rules, which
additionally are compatible with DNS resolution (via do-resolve) and
are not exclusive to normal load balancing. The "option-http_proxy"
example config file was updated to reflect this.

The option is still parsed so that an error message gives hints about
what to look for.
2021-07-18 19:35:32 +02:00
Willy Tarreau
bccc91d33e [RELEASE] Released version 2.5-dev2
Released version 2.5-dev2 with the following main changes :
    - BUILD/MEDIUM: tcp: set-mark support for OpenBSD
    - DOC: config: use CREATE USER for mysql-check
    - BUG/MINOR: stick-table: fix several printf sign errors dumping tables
    - BUG/MINOR: peers: fix data_type bit computation more than 32 data_types
    - MINOR: stick-table: make skttable_data_cast to use only std types
    - MEDIUM: stick-table: handle arrays of standard types into stick-tables
    - MEDIUM: peers: handle arrays of std types in peers protocol
    - DOC: stick-table: add missing documentation about gpt0 stored type
    - MEDIUM: stick-table: add the new array of gpt data_type
    - MEDIUM: stick-table: make the use of 'gpt' excluding the use of 'gpt0'
    - MEDIUM: stick-table: add the new arrays of gpc and gpc_rate
    - MEDIUM: stick-table: make the use of 'gpc' excluding the use of 'gpc0/1''
    - BUG/MEDIUM: sock: make sure to never miss early connection failures
    - BUG/MINOR: cli: fix server name output in "show fd"
    - Revert "MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules"
    - MEDIUM: stats: include disabled proxies that hold active sessions to stats
    - BUILD: stick-table: shut up invalid "uninitialized" warning in gcc 8.3
    - MINOR: http: implement http_get_scheme
    - MEDIUM: http: implement scheme-based normalization
    - MEDIUM: h1-htx: apply scheme-based normalization on h1 requests
    - MEDIUM: h2: apply scheme-based normalization on h2 requests
    - REGTESTS: add http scheme-based normalization test
    - BUILD: http_htx: fix ci compilation error with isdigit for Windows
    - MINOR: http: implement http uri parser
    - MINOR: http: use http uri parser for scheme
    - MINOR: http: use http uri parser for authority
    - REORG: http_ana: split conditions for monitor-uri in wait for request
    - MINOR: http: use http uri parser for path
    - BUG/MEDIUM: http_ana: fix crash for http_proxy mode during uri rewrite
    - MINOR: mux_h2: define config to disable h2 websocket support
    - CLEANUP: applet: remove unused thread_mask
    - BUG/MINOR: ssl: Default-server configuration ignored by server
    - BUILD: add detection of missing important CFLAGS
    - BUILD: lua: silence a build warning with TCC
    - MINOR: srv: extract tracking server config function
    - MINOR: srv: do not allow to track a dynamic server
    - MEDIUM: server: support track keyword for dynamic servers
    - REGTESTS: test track support for dynamic servers
    - MINOR: init: verify that there is a single word on "-cc"
    - MINOR: init: make -cc support environment variables expansion
    - MINOR: arg: add a free_args() function to free an args array
    - CLEANUP: config: use free_args() to release args array in cfg_eval_condition()
    - CLEANUP: hlua: use free_args() to release args arrays
    - REORG: config: move the condition preprocessing code to its own file
    - MINOR: cfgcond: start to split the condition parser to introduce terms
    - MEDIUM: cfgcond: report invalid trailing chars after expressions
    - MINOR: cfgcond: remerge all arguments into a single line
    - MINOR: cfgcond: support negating conditional expressions
    - MINOR: cfgcond: make the conditional term parser automatically allocate nodes
    - MINOR: cfgcond: insert an expression between the condition and the term
    - MINOR: cfgcond: support terms made of parenthesis around expressions
    - REGTEST: make check_condition.vtc fail as soon as possible
    - REGTESTS: add more complex check conditions to check_conditions.vtc
    - BUG/MEDIUM: init: restore behavior of command-line "-m" for memory limitation
2021-07-17 12:35:11 +02:00
Willy Tarreau
316ea7ede5 MINOR: cfgcond: support terms made of parenthesis around expressions
Now it's possible to form a term using parenthesis around an expression.
This will soon allow to build more complex expressions. For now they're
still pretty limited but parenthesis do work.
2021-07-16 19:18:41 +02:00
Willy Tarreau
ca81887599 MINOR: cfgcond: insert an expression between the condition and the term
Now evaluating a condition will rely on an expression (or an empty string),
and this expression will support ORing a sub-expression with another
optional expression. The sub-expressions ANDs a term with another optional
sub-expression. With this alone precedence between && and || is respected,
and the following expression:

     A && B && C || D || E && F || G

will naturally evaluate as:

     (A && B && C) || D || (E && F) || G
2021-07-16 19:18:41 +02:00
Willy Tarreau
ca56d3d28b MINOR: cfgcond: support negating conditional expressions
Now preceeding a config condition term with "!" will simply negate it.
Example:

   .if !feature(OPENSSL)
       .alert "SSL support is mandatory"
   .endif
2021-07-16 19:18:41 +02:00
Amaury Denoyelle
56eb8ed37d MEDIUM: server: support track keyword for dynamic servers
Allow the usage of the 'track' keyword for dynamic servers. On server
deletion, the server is properly removed from the tracking chain to
prevents NULL pointer dereferencing.
2021-07-16 10:22:58 +02:00
Amaury Denoyelle
befeae88e8 MINOR: mux_h2: define config to disable h2 websocket support
Define a new global config statement named
"h2-workaround-bogus-websocket-clients".

This statement will disable the automatic announce of h2 websocket
support as specified in the RFC8441. This can be use to overcome clients
which fail to implement the relatively fresh RFC8441. Clients will in
his case automatically downgrade to http/1.1 for the websocket tunnel
if the haproxy configuration allows it.

This feature is relatively simple and can be backported up to 2.4, which
saw the introduction of h2 websocket support.
2021-07-12 10:41:45 +02:00
Christopher Faulet
23048875a4 Revert "MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules"
This reverts commit 19bbbe05629ea947dd60d5b96d96f0066b047b97.

For now, set-src/set-src-port actions are directly performed on the client
connection. Using these actions at the stream level is really a problem with
HTTP connection (See #90) because all requests are affected by this change
and not only the current request. And it is worse with the H2, because
several requests can set their source address into the same connection at
the same time.

It is already an issue when these actions are called from "http-request"
rules. It is safer to wait a bit before adding the support to "tcp-request
content" rules. The solution is to be able to set src/dst address on the
stream and not on the connection when the action if performed from the L7
level..

Reverting the above commit means the issue #1303 is no longer fixed.

This patch must be backported in all branches containing the above commit
(as far as 2.0 for now).
2021-07-06 11:44:04 +02:00
Emeric Brun
726783db18 MEDIUM: stick-table: make the use of 'gpc' excluding the use of 'gpc0/1''
This patch makes the use of 'gpc' excluding the use of the legacy
types 'gpc0' and 'gpc1" on the same table.

It also makes the use of 'gpc_rate' excluding the use of the legacy
types 'gpc0_rate' and 'gpc1_rate" on the same table.

The 'gpc0' and 'gpc1' related fetches and actions will apply
to the first two elements of the 'gpc' array if stored in table.

The 'gpc0_rate' and 'gpc1_rate' related fetches and actions will apply
to the first two elements of the 'gpc_rate' array if stored in table.
2021-07-06 07:24:42 +02:00
Emeric Brun
4d7ada8f9e MEDIUM: stick-table: add the new arrays of gpc and gpc_rate
This patch adds the definition of two new array data_types:
'gpc': This is an array of 32bits General Purpose Counters.
'gpc_rate': This is an array on increment rates of General Purpose Counters.

Like for all arrays, they are limited to 100 elements.

This patch also adds actions and fetches to handle
elements of those arrays.

Note: As documented, those new actions and fetches won't
apply to the legacy 'gpc0', 'gpc1', 'gpc0_rate' nor 'gpc1_rate'.
2021-07-06 07:24:42 +02:00
Emeric Brun
f7ab0bfb62 MEDIUM: stick-table: make the use of 'gpt' excluding the use of 'gpt0'
This patch makes the use of 'gpt' excluding the use of the legacy
type 'gpt0' on the same table.

It also makes the 'gpt0' related fetches and actions applying
to the first element of the 'gpt' array if stored in table.
2021-07-06 07:24:42 +02:00
Emeric Brun
877b0b5a7b MEDIUM: stick-table: add the new array of gpt data_type
This patch adds the definition of a new array data_type
'gpt'. This is an array of 32bits General Purpose Tags.

Like for all arrays, it is limited to 100 elements.

This patch also adds actions and fetches to handle
elements of this array.

Note: As documented, those new actions and fetches won't
apply to the legacy 'gpt0' data type.
2021-07-06 07:24:42 +02:00
Emeric Brun
1a6b7254de DOC: stick-table: add missing documentation about gpt0 stored type
The store type 'gpt0' was present in code but was not documented.

The patch fix this and should be backported since 'gpt0' is supported.
[wt: ~1.6-dev4 hence all stable]
2021-07-06 07:24:42 +02:00
Daniel Black
d3e7dc498b DOC: config: use CREATE USER for mysql-check
CREATE USER has been the standard way of creating users since
MySQL-5.0 (2005).

The current syntax of INSERT INTO mysql.user won't actually work
on MariaDB-10.4+.

Because haproxy doesn't use any resources the MySQL executable comment
syntax provides resource contraints to make it more palatable
to risk adverse users.

/*!50701 is a syntax recognised by MySQL and MariaDB 5.7.1+ when
resource contraints where added.

/*M!100201 is a MariaDB executable comment syntax recognised for MariaDB
for the 10.2.1 where the MAX_STATEMENT_TIME was added.

This patch may be backported as far as 2.0.
2021-07-05 10:53:18 +02:00
David Carlier
bae4cb2790 BUILD/MEDIUM: tcp: set-mark support for OpenBSD
set-mark support for this platform, for routing table purpose.
Follow-up from f7f53afcf9d367d19, this time for OpenBSD.
2021-07-05 10:53:18 +02:00
Willy Tarreau
96a2f50b51 [RELEASE] Released version 2.5-dev1
Released version 2.5-dev1 with the following main changes :
    - CLEANUP: ssl: Move ssl_store related code to ssl_ckch.c
    - MINOR: ssl: Allow duplicated entries in the cafile_tree
    - MEDIUM: ssl: Chain ckch instances in ca-file entries
    - MINOR: ssl: Add reference to default ckch instance in bind_conf
    - MINOR: ssl: Add helper functions to create/delete cafile entries
    - MEDIUM: ssl: Add a way to load a ca-file content from memory
    - MINOR: ssl: Add helper function to add cafile entries
    - MINOR: ssl: Ckch instance rebuild and cleanup factorization in CLI handler
    - MEDIUM: ssl: Add "set+commit ssl ca-file" CLI commands
    - REGTESTS: ssl: Add new ca-file update tests
    - MINOR: ssl: Add "abort ssl ca-file" CLI command
    - MINOR: ssl: Add a cafile_entry type field
    - MINOR: ssl: Refactorize the "show certificate details" code
    - MEDIUM: ssl: Add "show ssl ca-file" CLI command
    - MEDIUM: ssl: Add "new ssl ca-file" CLI command
    - MINOR: ssl: Add "del ssl ca-file" CLI command
    - REGTESTS: ssl: Add "new/del ssl ca-file" tests
    - DOC: ssl: Add documentation about CA file hot update commands
    - DOC: internals: update the SSL architecture schema
    - MINOR: ssl: Chain instances in ca-file entries
    - MEDIUM: ssl: Add "set+commit ssl crl-file" CLI commands
    - MEDIUM: ssl: Add "new+del crl-file" CLI commands
    - MINOR: ssl: Add "abort ssl crl-file" CLI command
    - MEDIUM: ssl: Add "show ssl crl-file" CLI command
    - REGTESTS: ssl: Add "new/del ssl crl-file" tests
    - REGTESTS: ssl: Add "set/commit ssl crl-file" test
    - DOC: ssl: Add documentation about CRL file hot update commands
    - BUILD/MINOR: ssl: Fix compilation with SSL enabled
    - BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
    - CI: introduce scripts/build-vtest.sh for installing VTest
    - CLEANUP: ssl: Fix coverity issues found in CA file hot update code
    - CI: github actions: add OpenTracing builds
    - BUG/MEDIUM: ebtree: Invalid read when looking for dup entry
    - BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
    - BUILD/MINOR: opentracing: fixed build when using clang
    - BUG/MEDIUM: filters: Exec pre/post analysers only one time per filter
    - BUG/MINOR: http-comp: Preserve HTTP_MSGF_COMPRESSIONG flag on the response
    - MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
    - CLEANUP: pattern: remove export of non-existent function pattern_delete()
    - MINOR: h1-htx: Update h1 parsing functions to return result as a size_t
    - MEDIUM: h1-htx: Adapt H1 data parsing to copy wrapping data in one call
    - MINOR: mux-h1/mux-fcgi: Don't needlessly loop on data parsing
    - MINOR: h1-htx: Move HTTP chunks parsing into a dedicated function
    - MEDIUM: h1-htx: Split function to parse a chunk and the loop on the buffer
    - MEDIUM: h1-htx: Add a function to parse contiguous small chunks
    - MINOR: h1-htx: Use a correlation table to speed-up small chunks parsing
    - MINOR: buf: Add function to realign a buffer with a specific head position
    - MINOR: muxes/h1-htx: Realign input buffer using b_slow_realign_ofs()
    - CLEANUP: mux-h1: Rename functions parsing input buf and filling output buf
    - Revert "MEDIUM: http-ana: Deal with L7 retries in HTTP analysers"
    - BUG/MINOR: http-ana: Send the right error if max retries is reached on L7 retry
    - BUG/MINOR: http-ana: Handle L7 retries on refused early data before K/A aborts
    - MINOR: http-ana: Perform L7 retries because of status codes in response analyser
    - MINOR: cfgparse: Fail when encountering extra arguments in macro
    - DOC: intro: Fix typo in starter guide
    - BUG/MINOR: server: Missing calloc return value check in srv_parse_source
    - BUG/MINOR: peers: Missing calloc return value check in peers_register_table
    - BUG/MINOR: ssl: Missing calloc return value check in ssl_init_single_engine
    - BUG/MINOR: http: Missing calloc return value check in parse_http_req_capture
    - BUG/MINOR: proxy: Missing calloc return value check in proxy_parse_declare
    - BUG/MINOR: proxy: Missing calloc return value check in proxy_defproxy_cpy
    - BUG/MINOR: http: Missing calloc return value check while parsing tcp-request/tcp-response
    - BUG/MINOR: http: Missing calloc return value check while parsing tcp-request rule
    - BUG/MINOR: compression: Missing calloc return value check in comp_append_type/algo
    - BUG/MINOR: worker: Missing calloc return value check in mworker_env_to_proc_list
    - BUG/MINOR: http: Missing calloc return value check while parsing redirect rule
    - BUG/MINOR: http: Missing calloc return value check in make_arg_list
    - BUG/MINOR: proxy: Missing calloc return value check in chash_init_server_tree
    - CLEANUP: http-ana: Remove useless if statement about L7 retries
    - BUG/MAJOR: stream-int: Release SI endpoint on server side ASAP on retry
    - MINOR: backend: Don't release SI endpoint anymore in connect_server()
    - BUG/MINOR: vars: Be sure to have a session to get checks variables
    - DOC/MINOR: move uuid in the configuration to the right alphabetical order
    - CLEANUP: mux-fcgi: Don't needlessly store result of data/trailers parsing
    - BUILD: fix compilation for OpenSSL-3.0.0-alpha17
    - MINOR: http-ana: Use -1 status for client aborts during queuing and connect
    - REGTESTS: Fix http_abortonclose.vtc to support -1 status for some client aborts
    - CLEANUP: backend: fix incorrect comments on locking conditions for lb functions
    - CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests
    - CI: github actions: add OpenSSL-3.0.0 builds
    - CI: github actions: -Wno-deprecated-declarations with OpenSSL 3.0.0
    - MINOR: errors: allow empty va_args for diag variadic macro
    - REORG: errors: split errors reporting function from log.c
    - CLEANUP: server: fix cosmetic of error message on sni parsing
    - MEDIUM: errors: implement user messages buffer
    - MINOR: log: do not discard stderr when starting is over
    - MEDIUM: errors: implement parsing context type
    - MINOR: errors: use user messages context in print_message
    - MINOR: log: display exec path on first warning
    - MINOR: errors: specify prefix "config" for parsing output
    - MINOR: log: define server user message format
    - REORG: server: use parsing ctx for server parsing
    - REORG: config: use parsing ctx for server config check
    - MINOR: server: use parsing ctx for server init addr
    - MINOR: server: use ha_alert in server parsing functions
    - DOC: use the req.ssl_sni in examples
    - CLEANUP: cfgparse: Remove duplication of `MAX_LINE_ARGS + 1`
    - CLEANUP: tools: Make errptr const in `parse_line()`
    - MINOR: haproxy: Add `-cc` argument
    - BUG: errors: remove printf positional args for user messages context
    - CI: Make matrix.py executable and add shebang
    - BUILD: make tune.ssl.keylog available again
    - BUG/MINOR: ssl: OCSP stapling does not work if expire too far in the future
    - Revert "BUG/MINOR: opentracing: initialization after establishing daemon mode"
    - BUG/MEDIUM: opentracing: initialization before establishing daemon and/or chroot mode
    - SCRIPTS: opentracing: enable parallel builds in build-ot.sh
    - BUG/MEDIUM: compression: Fix loop skipping unused blocks to get the next block
    - BUG/MEDIUM: compression: Properly get the next block to iterate on payload
    - BUG/MEDIUM: compression: Add a flag to know the filter is still processing data
    - MINOR: ssl: Keep the actual key length in the certificate_ocsp structure
    - MINOR: ssl: Add new "show ssl ocsp-response" CLI command
    - MINOR: ssl: Add the OCSP entry key when displaying the details of a certificate
    - MINOR: ssl: Add the "show ssl cert foo.pem.ocsp" CLI command
    - REGTESTS: ssl: Add "show ssl ocsp-response" test
    - BUG/MINOR: server: explicitly set "none" init-addr for dynamic servers
    - BUG/MINOR: pools: fix a possible memory leak in the lockless pool_flush()
    - BUG/MINOR: pools: make DEBUG_UAF always write to the to-be-freed location
    - MINOR: pools: do not maintain the lock during pool_flush()
    - MINOR: pools: call malloc_trim() under thread isolation
    - MEDIUM: pools: use a single pool_gc() function for locked and lockless
    - BUG/MAJOR: pools: fix possible race with free() in the lockless variant
    - CLEANUP: pools: remove now unused seq and pool_free_list
    - MEDIUM: pools: remove the locked pools implementation
    - BUILD: ssl: Fix compilation with BoringSSL
    - BUG/MEDIUM: errors: include missing obj_type file
    - REGTESTS: ssl: show_ssl_ocspresponce.vtc is broken with BoringSSL
    - BUG/MAJOR: htx: Fix htx_defrag() when an HTX block is expanded
    - BUG/MINOR: mux-fcgi: Expose SERVER_SOFTWARE parameter by default
    - BUG/MINOR: h1-htx: Fix a signess bug with char data type when parsing chunk size
    - CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
    - BUG/MINOR: resolvers: answser item list was randomly purged or errors
    - MEDIUM: resolvers: add a ref on server to the used A/AAAA answer item
    - MEDIUM: resolvers: add a ref between servers and srv request or used SRV record
    - BUG/MINOR: server-state: load SRV resolution only if params match the config
    - MINOR: config: remove support for deprecated option "tune.chksize"
    - MINOR: config: completely remove support for "no option http-use-htx"
    - MINOR: log: remove the long-deprecated early log-format tags
    - MINOR: http: remove the long deprecated "set-cookie()" sample fetch function
    - MINOR: config: reject long-deprecated "option forceclose"
    - MINOR: config: remove deprecated option "http-tunnel"
    - MEDIUM: proxy: remove the deprecated "grace" keyword
    - MAJOR: config: remove parsing of the global "nbproc" directive
    - BUILD: init: remove initialization of multi-process thread mappings
    - BUILD: log: remove unused fmt_directive()
    - REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests
    - REGTESTS: Remove REQUIRE_VERSION=1.7 from all tests
    - CI: github actions: enable alpine/musl builds
    - BUG/MAJOR: resolvers: segfault using server template without SRV RECORDs
    - DOC: lua: Add a warning about buffers modification in HTTP
    - MINOR: ssl: Use OpenSSL's ASN1_TIME convertor when available
    - BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
    - BUG/MEDIUM: server: extend thread-isolate over much of CLI 'add server'
    - BUG/MEDIUM: server: clear dynamic srv on delete from proxy id/name trees
    - BUG/MEDIUM: server: do not forget to generate the dynamic servers ids
    - BUG/MINOR: server: do not keep an invalid dynamic server in px ids tree
    - BUG/MEDIUM: server: do not auto insert a dynamic server in px addr_node
    - BUG/MEDIUM: shctx: use at least thread-based locking on USE_PRIVATE_CACHE
    - BUG/MINOR: ssl: use atomic ops to update global shctx stats
    - BUG/MINOR: mworker: fix typo in chroot error message
    - CLEANUP: global: remove unused definition of stopping_task[]
    - MEDIUM: init: remove the loop over processes during init
    - MINOR: mworker: remove the initialization loop over processes
    - CLEANUP: global: remove the nbproc field from the global structure
    - CLEANUP: global: remove pid_bit and all_proc_mask
    - MEDIUM: global: remove dead code from nbproc/bind_proc removal
    - MEDIUM: config: simplify cpu-map handling
    - MEDIUM: cpu-set: make the proc a single bit field and not an array
    - CLEANUP: global: remove unused definition of MAX_PROCS
    - MEDIUM: global: remove the relative_pid from global and mworker
    - DOC: update references to process numbers in cpu-map and bind-process
    - MEDIUM: config: warn about "bind-process" deprecation
    - CLEANUP: shctx: remove the different inter-process locking techniques
    - BUG/MAJOR: queue: set SF_ASSIGNED when setting strm->target on dequeue
    - MINOR: backend: only skip LB when there are actual connections
    - BUG/MINOR: mux-h1: do not skip the error response on bad requests
    - MINOR: connection: add helper conn_append_debug_info()
    - MINOR: mux-h2/trace: report a few connection-level info during h2_init()
    - CLEANUP: mux-h2/traces: better align user messages
    - BUG/MINOR: stats: make "show stat typed desc" work again
    - MINOR: mux-h2: obey http-ignore-probes during the preface
    - BUG/MINOR: mux-h2/traces: bring back the lost "rcvd H2 REQ" trace
    - BUG/MINOR: mux-h2/traces: bring back the lost "sent H2 REQ/RES" traces
    - CLEANUP: assorted typo fixes in the code and comments
    - CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'
    - REGTESTS: Replace REQUIRE_VERSION=2.5 with 'haproxy -cc'
    - REGTESTS: Replace REQUIRE_OPTIONS with 'haproxy -cc' for 2.5+ tests
    - REGTESTS: Replace REQUIRE_BINARIES with 'command -v'
    - REGTESTS: Remove support for REQUIRE_BINARIES
    - CI: ssl: enable parallel builds for OpenSSL on Linux
    - CI: ssl: do not needlessly build the OpenSSL docs
    - CI: ssl: keep the old method for ancient OpenSSL versions
    - CLEANUP: server: a separate function for initializing the per_thr field
    - BUG/MINOR: server: Forbid to set fqdn on the CLI if SRV resolution is enabled
    - BUG/MEDIUM: server/cli: Fix ABBA deadlock when fqdn is set from the CLI
    - MINOR: resolvers: Clean server in a dedicated function when removing a SRV item
    - MINOR: resolvers: Remove server from named_servers tree when removing a SRV item
    - BUG/MEDIUM: resolvers: Add a task on servers to check SRV resolution status
    - BUG/MINOR: backend: restore the SF_SRV_REUSED flag original purpose
    - BUG/MINOR: backend: do not set sni on connection reuse
    - BUG/MINOR: resolvers: Use resolver's lock in resolv_srvrq_expire_task()
    - BUG/MINOR: server/cli: Fix locking in function processing "set server" command
    - BUG/MINOR: cache: Correctly handle existing-but-empty 'accept-encoding' header
    - MINOR: ssl: fix typo in usage for 'new ssl ca-file'
    - MINOR: ssl: always initialize random generator
    - MINOR: ssl: check allocation in ssl_sock_init_srv
    - MINOR: ssl: check allocation in parse ciphers/ciphersuites/verifyhost
    - MINOR: ssl: check allocation in parse npn/sni
    - MINOR: server: disable CLI 'set server ssl' for dynamic servers
    - MINOR: ssl: render file-access optional on server crt loading
    - MINOR: ssl: split parse functions for alpn/check-alpn
    - MINOR: ssl: support ca-file arg for dynamic servers
    - MINOR: ssl: support crt arg for dynamic servers
    - MINOR: ssl: support crl arg for dynamic servers
    - MINOR: ssl: enable a series of ssl keywords for dynamic servers
    - MINOR: ssl: support ssl keyword for dynamic servers
    - REGTESTS: server: test ssl support for dynamic servers
    - MINOR: queue: update the stream's pend_pos before queuing it
    - CLEANUP: Prevent channel-t.h from being detected as C++ by GitHub
    - BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check
    - REGTESTS: fix maxconn update with agent-check
    - MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn
    - MINOR: queue: update proxy->served once out of the loop
    - MEDIUM: queue: refine the locking in process_srv_queue()
    - MINOR: lb/api: remove the locked argument from take_conn/drop_conn
    - MINOR: queue: create a new structure type "queue"
    - MINOR: proxy: replace the pendconns-related stuff with a struct queue
    - MINOR: server: replace the pendconns-related stuff with a struct queue
    - MEDIUM: queue: use a dedicated lock for the queues
    - MEDIUM: queue: simplify again the process_srv_queue() API
    - MINOR: queue: factor out the proxy/server queuing code
    - MINOR: queue: use atomic-ops to update the queue's index
    - MEDIUM: queue: determine in process_srv_queue() if the proxy is usable
    - MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()
    - MEDIUM: queue: unlock as soon as possible
    - MINOR: queue: make pendconn_first() take the lock by itself
    - CLEANUP: backend: remove impossible case of round-robin + consistent hash
    - MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
    - DOC: config: Add missing actions in "tcp-request session" documentation
    - CLEANUP: dns: Remove a forgotten debug message
    - DOC: Replace issue templates by issue forms
    - Revert "MINOR: queue: make pendconn_first() take the lock by itself"
    - Revert "MEDIUM: queue: unlock as soon as possible"
    - Revert "MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()"
    - Revert "MEDIUM: queue: determine in process_srv_queue() if the proxy is usable"
    - Revert "MINOR: queue: use atomic-ops to update the queue's index"
    - Revert "MINOR: queue: factor out the proxy/server queuing code"
    - Revert "MEDIUM: queue: simplify again the process_srv_queue() API"
    - Revert "MEDIUM: queue: use a dedicated lock for the queues"
    - Revert "MEDIUM: queue: refine the locking in process_srv_queue()"
    - Revert "MINOR: queue: update proxy->served once out of the loop"
    - Revert "MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn"
    - MEDIUM: queue: update px->served and lb's take_conn once per loop
    - MEDIUM: queue: use a dedicated lock for the queues (v2)
    - MEDIUM: queue: simplify again the process_srv_queue() API (v2)
    - MEDIUM: queue: determine in process_srv_queue() if the proxy is usable (v2)
    - MINOR: queue: factor out the proxy/server queuing code (v2)
    - MINOR: queue: use atomic-ops to update the queue's index (v2)
    - MEDIUM: queue: take the proxy lock only during the px queue accesses
    - MEDIUM: queue: use a trylock on the server's queue
    - MINOR: queue: add queue_init() to initialize a queue
    - MINOR: queue: add a pointer to the server and the proxy in the queue
    - MINOR: queue: store a pointer to the queue into the pendconn
    - MINOR: queue: remove the px/srv fields from pendconn
    - MINOR: queue: simplify pendconn_unlink() regarding srv vs px
    - BUG: backend: stop looking for queued connections once there's no more
    - BUG/MINOR: queue/debug: use the correct lock labels on the queue lock
    - BUG/MINOR: resolvers: Always attach server on matching record on resolution
    - BUG/MINOR: resolvers: Reset server IP when no ip is found in the response
    - MINOR: resolvers: Reset server IP on error in resolv_get_ip_from_response()
    - BUG/MINOR: checks: return correct error code for srv_parse_agent_check
    - BUILD: Makefile: fix linkage for Haiku.
    - BUG/MINOR: tcpcheck: Fix numbering of implicit HTTP send/expect rules
    - MINOR: http-act/tcp-act: Add "set-log-level" for tcp content rules
    - MINOR: http-act/tcp-act: Add "set-nice" for tcp content rules
    - MINOR: http-act/tcp-act: Add "set-mark" and "set-tos" for tcp content rules
    - CLEANUP: tcp-act: Sort action lists
    - BUILD/MEDIUM: tcp: set-mark setting support for FreeBSD.
    - BUILD: tcp-act: avoid warning when set-mark / set-tos are not supported
    - BUG/MINOR: mqtt: Fix parser for string with more than 127 characters
    - BUG/MINOR: mqtt: Support empty client ID in CONNECT message
    - BUG/MEDIUM: resolvers: Make 1st server of a template take part to SRV resolution
    - CLEANUP: peers: re-write intdecode function comment.
2021-06-30 16:16:14 +02:00
David Carlier
f7f53afcf9 BUILD/MEDIUM: tcp: set-mark setting support for FreeBSD.
This platform has a similar socket option from Linux's SO_MARK,
marking a socket with an id for packet filter purpose, DTrace
monitoring and so on.
2021-06-28 07:03:35 +02:00
Christopher Faulet
469c06c30e MINOR: http-act/tcp-act: Add "set-mark" and "set-tos" for tcp content rules
It is now possible to set the Netfilter MARK and the TOS field value in all
packets sent to the client from any tcp-request rulesets or the "tcp-response
content" one. To do so, the parsing of "set-mark" and "set-tos" actions are
moved in tcp_act.c and the actions evaluation is handled in dedicated functions.

This patch may be backported as far as 2.2 if necessary.
2021-06-25 16:11:58 +02:00
Christopher Faulet
1da374af2f MINOR: http-act/tcp-act: Add "set-nice" for tcp content rules
It is now possible to set the "nice" factor of the current stream from a
"tcp-request content" or "tcp-response content" ruleset. To do so, the
action parsing is moved in stream.c and the action evaluation is handled in
a dedicated function.

This patch may be backported as far as 2.2 if necessary.
2021-06-25 16:11:53 +02:00
Christopher Faulet
551a641cff MINOR: http-act/tcp-act: Add "set-log-level" for tcp content rules
It is now possible to set the stream log level from a "tcp-request content"
or "tcp-response content" ruleset. To do so, the action parsing is moved in
stream.c and the action evaluation is handled in a dedicated function.

This patch should fix issue #1306. It may be backported as far as 2.2 if
necessary.
2021-06-25 16:11:46 +02:00
Christopher Faulet
14aec6e8ae DOC: config: Add missing actions in "tcp-request session" documentation
set-src/set-src-port and set-dst/set-dst-port actions were not listed in the
documentation of "tcp-request session".

This patch may be backported to all stable versions.
2021-06-23 12:19:26 +02:00
Christopher Faulet
19bbbe0562 MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
If it possible to set source IP/Port from "tcp-request connection",
"tcp-request session" and "http-request" rules but not from "tcp-request
content" rules. There is no reason for this limitation and it may be a
problem for anyone wanting to call a lua fetch to dynamically set source
IP/Port from a TCP proxy. Indeed, to call a lua fetch, we must have a
stream. And there is no stream when "tcp-request connection/session" rules
are evaluated.

Thanks to this patch, "set-src" and "set-src-port" action are now supported
by "tcp_request content" rules.

This patch is related to the issue #1303. It may be backported to all stable
versions.
2021-06-23 12:07:24 +02:00
Amaury Denoyelle
34897d2eff MINOR: ssl: support ssl keyword for dynamic servers
Activate the 'ssl' keyword for dynamic servers. This is the final step
to have ssl dynamic servers feature implemented. If activated,
ssl_sock_prepare_srv_ctx will be called at the end of the 'add server'
CLI handler.

At the same time, update the management doc to list all ssl keywords
implemented for dynamic servers.
2021-06-18 16:42:26 +02:00
Willy Tarreau
317804d288 DOC: update references to process numbers in cpu-map and bind-process
Let's mention that cpu-map is limited to process 1 and that bind-process
is deprecated. Other minor adjustments were made to "process" on bind
lines.
2021-06-15 16:52:42 +02:00
Christopher Faulet
0953039567 DOC: lua: Add a warning about buffers modification in HTTP
Since the 1.9, it is forbidden to alter the channel buffer from an HTTP
stream because there is no way to keep the HTTP parser synchronized if the
buffer content is altered. In addition, since the HTX is the only
reprensentation for HTTP messages, the data in HTTP buffers are structured
and cannot be read or updated in a raw fashion.

A warning is triggered when a user tries to alter an HTTP buffer. However,
it was not documented. This patch adds a warning in the lua documentation.

This patch is related to the issue #1287. It may be backported as far as
2.0.
2021-06-14 12:03:32 +02:00
Willy Tarreau
b63dbb7b2e MAJOR: config: remove parsing of the global "nbproc" directive
This one was deprecated in 2.3 and marked for removal in 2.5. It suffers
too many limitations compared to threads, and prevents some improvements
from being engaged. Instead of a bypassable startup error, there is now
a hard error.

The parsing code was removed, and very few obvious cases were as well.
The code is deeply rooted at certain places (e.g. "for" loops iterating
from 0 to nbproc) so it will not be that trivial to remove everywhere.
The "bind" and "bind-process" parsers will have to be adjusted, though
maybe not completely changed if we later want to support thread groups
for large NUMA machines. Some stats socket restrictions were removed,
and the doc was updated according to what was done. A few places in the
doc still refer to nbproc and will have to be revisited. The master-worker
code also refers to the process number to distinguish between master and
workers and will have to be carefully adjusted. The MAX_PROCS macro was
reset to 1, this will at least reduce the size of some remaining arrays.

Two regtests were dependieng on this directive, one with an explicit
"nbproc 1" and another one testing the master's CLI using nbproc 4.
Both were adapted.
2021-06-11 17:02:13 +02:00
Willy Tarreau
eb778248d9 MEDIUM: proxy: remove the deprecated "grace" keyword
Commit ab0a5192a ("MEDIUM: config: mark "grace" as deprecated") marked
the "grace" keyword as deprecated in 2.3, tentative removal for 2.4
with a hard deadline in 2.5, so let's remove it and return an error now.
This old and outdated feature was incompatible with soft-stop, reload
and socket transfers, and keeping it forced ugly hacks in the lower
layers of the protocol stack.
2021-06-11 16:57:34 +02:00
Willy Tarreau
6ba69841f8 MINOR: config: reject long-deprecated "option forceclose"
It's been warning as being deprecated since 2.0-dev4, it's about time
to drop it now. The error message recommends to either remove it or
use "option httpclose" instead. It's still referred to in the old
internal doc about the connection header, which itself seems highly
inaccurate by now.
2021-06-11 16:57:34 +02:00
Willy Tarreau
eb9d90a5a2 MINOR: config: remove support for deprecated option "tune.chksize"
It was marked as deprecated for immediate removal as it was not used,
let's reject it and remove it from the doc. A specific error suggests
to check tune.bufsize instead.
2021-06-11 16:57:34 +02:00
Christopher Faulet
5cd0e528cf BUG/MINOR: mux-fcgi: Expose SERVER_SOFTWARE parameter by default
As specified in the RFC3875 (section 4.1.17), this parameter must be set to
the name and version of the information server software making the CGI
request. Thus, it is now added to the default parameters defined by
HAProxy. It is set to the string "HAProxy $version".

This patch should fix the issue #1285 and must be backported as far as 2.2.
2021-06-11 14:15:48 +02:00
Amaury Denoyelle
efbf35caf9 BUG/MINOR: server: explicitly set "none" init-addr for dynamic servers
Define srv.init_addr_methods to SRV_IADDR_NONE on 'add server' CLI
handler. This explicitly states that no resolution will be made on the
server creation.

This is not a real bug as the default value (SRV_IADDR_END) has the same
effect in practice. However the intent is clearer and prevent to use the
default "libc,last" by mistake which cannot execute on runtime (blocking
call + file access via gethostbyname/getaddrinfo).

The doc is also updated to reflect this limitation.

This should be backported up to 2.4.
2021-06-10 17:44:05 +02:00
Remi Tricot-Le Breton
6056e61ae2 MINOR: ssl: Add the "show ssl cert foo.pem.ocsp" CLI command
Add the ability to dump an OCSP response details through a call to "show
ssl cert cert.pem.ocsp". It can also be used on an ongoing transaction
by prefixing the certificate name with a '*'.
Even if the ckch structure holds an ocsp_response buffer, we still need
to look for the actual ocsp response entry in the ocsp response tree
rather than just dumping the ckch's buffer details because when updating
an ocsp response through a "set ssl ocsp-response" call, the
corresponding buffer in the ckch is not updated accordingly. So this
buffer, even if it is not empty, might hold an outdated ocsp response.
2021-06-10 16:44:11 +02:00
Remi Tricot-Le Breton
d92fd11c77 MINOR: ssl: Add new "show ssl ocsp-response" CLI command
This patch adds the "show ssl ocsp-response [<id>]" CLI command. This
command can be used to display the IDs of the OCSP tree entries along
with details about the entries' certificate ID (issuer's name and key
hash + serial number), or to display the details of a single
ocsp-response if an ID is given. The details displayed in this latter
case are the ones shown by a "openssl ocsp -respin <ocsp-response>
-text" call.
2021-06-10 16:44:11 +02:00
Maximilian Mader
fc0cceb08a MINOR: haproxy: Add -cc argument
This patch adds the `-cc` (check condition) argument to evaluate conditions on
startup and return the result as the exit code.

As an example this can be used to easily check HAProxy's version in scripts:

    haproxy -cc 'version_atleast(2.4)'

This resolves GitHub issue #1246.

Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
2021-06-08 11:17:19 +02:00
Alex
5c866200d4 DOC: use the req.ssl_sni in examples
This patch should be backported to at least 2.0
2021-06-08 10:51:39 +02:00
Alexandar Lazic
528adc3b18 DOC/MINOR: move uuid in the configuration to the right alphabetical order
This patch can be backported up to 2.1 where the uuid fetch was
introduced
2021-06-02 11:57:18 +02:00
Mark Mullan
e1ecfdda81 DOC: intro: Fix typo in starter guide
s/sned/send.
2021-05-27 08:26:43 +02:00
Dragan Dosen
a75eea78e2 MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
The output of "show map/acl" now contains the 'entry_cnt' value that
represents the count of all the entries for each map/acl, not just the
active ones, which means that it also includes entries currently being
added.
2021-05-25 08:44:45 +02:00
Remi Tricot-Le Breton
3c222bd99c DOC: ssl: Add documentation about CRL file hot update commands
Add description for new "abort/commit/del/new/set/show ssl crl-file" CLI
commands.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
653a16769a DOC: internals: update the SSL architecture schema
Add the new relation between the CA file tree entries and the ckch
instances introduced during CA file hot update implementation (see
GitHub #1057).
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
e88a2caf9c DOC: ssl: Add documentation about CA file hot update commands
Add documentations for "abort/commit/del/new/set/show ssl ca-file" CLI
commands.

These commands were added to answer to GitHub #1057.
2021-05-17 10:50:24 +02:00
Willy Tarreau
1f97306ecc [RELEASE] Released version 2.5-dev0
Released version 2.5-dev0 with the following main changes :
    - MINOR: version: it's development again
2021-05-14 09:36:37 +02:00
Willy Tarreau
6cbbecf097 [RELEASE] Released version 2.4.0
Released version 2.4.0 with the following main changes :
    - BUG/MINOR: http_fetch: fix possible uninit sockaddr in fetch_url_ip/port
    - CLEANUP: cli/activity: Remove double spacing in set profiling command
    - CI: Build VTest with clang
    - CI: extend spellchecker whitelist, add "ists" as well
    - CLEANUP: assorted typo fixes in the code and comments
    - BUG/MINOR: memprof: properly account for differences for realloc()
    - MINOR: memprof: also report the method used by each call
    - MINOR: memprof: also report the totals and delta alloc-free
    - CLEANUP: pattern: remove the unused and dangerous pat_ref_reload()
    - BUG/MINOR: http_act: Fix normalizer names in error messages
    - MINOR: uri_normalizer: Add `fragment-strip` normalizer
    - MINOR: uri_normalizer: Add `fragment-encode` normalizer
    - IMPORT: slz: use the generic function for the last bytes of the crc32
    - IMPORT: slz: do not produce the crc32_fast table when CRC is natively supported
    - BUILD/MINOR: opentracing: fixed compilation with filter enabled
    - BUILD: makefile: add a few popular ARMv8 CPU targets
    - BUG/MEDIUM: stick_table: fix crash when using tcp smp_fetch_src
    - REGTESTS: stick-table: add src_conn_rate test
    - CLEANUP: stick-table: remove a leftover of an old keyword declaration
    - BUG/MINOR: stats: fix lastchk metric that got accidently lost
    - EXAMPLES: add a "basic-config-edge" example config
    - EXAMPLES: add a trivial config for quick testing
    - DOC: management: Correct example reload command in the document
    - Revert "CI: Build VTest with clang"
    - MINOR: activity/cli: optionally support sorting by address on "show profiling"
    - DEBUG: ssl: export ssl_sock_close() to see its symbol resolved in profiling
    - BUG/MINOR: lua/vars: prevent get_var() from allocating a new name
    - DOC: config: Fix configuration example for mqtt
    - BUG/MAJOR: config: properly initialize cpu_map.thread[] up to MAX_THREADS
    - BUILD: config: avoid a build warning on numa_detect_topology() without threads
    - DOC: update min requirements in INSTALL
    - IMPORT: slz: use inttypes.h instead of stdint.h
    - BUILD: sample: use strtoll() instead of atoll()
    - MINOR: version: mention that it's LTS now.
2021-05-14 09:03:30 +02:00
Daniel Corbett
cc9d9b0906 DOC: config: Fix configuration example for mqtt
This patch fixes the example for mqtt_is_valid(), it was missing
curly braces within the ACL.
2021-05-13 18:10:35 +02:00
Willy Tarreau
f1c8a3846c MINOR: activity/cli: optionally support sorting by address on "show profiling"
"show profiling" by default sorts by usage/counts, which is suitable for
occasional use. But when called from scripts to monitor/search variations,
this is not very convenient. Let's add a new "byaddr" option to support
sorting the output by address. It also eases matching alloc/free calls
from within a same library, or reading grouped tasks costs by library.
2021-05-13 10:00:17 +02:00
varnav
5a3fe9fcbf DOC: management: Correct example reload command in the document
Current example is:

  `echo "reload" | socat /var/run/haproxy-master.sock`

it will cause socat error:

  `exactly 2 addresses required (there are 1); use option "-h" for help`

Correct working command is:

  `echo "reload" | socat /var/run/haproxy-master.sock stdin`
2021-05-12 20:35:41 +02:00
Tim Duesterhus
dec1c36b3a MINOR: uri_normalizer: Add fragment-encode normalizer
This normalizer encodes '#' as '%23'.

See GitHub Issue #714.
2021-05-11 17:24:32 +02:00