The "acme status" command, shows the status of every certificates
configured with ACME, not only the running task like "acme ps".
The IO handler loops on the ckch_store tree and outputs a line for each
ckch_store which has an acme section set. This is still done under the
ckch_store lock and doesn't support resuming when the buffer is full,
but we need to change that in the future.
Released version 3.2-dev14 with the following main changes :
- MINOR: acme: retry label always do a request
- MINOR: acme: does not leave task for next request
- BUG/MINOR: acme: reinit the retries only at next request
- MINOR: acme: change the default max retries to 5
- MINOR: acme: allow a delay after a valid response
- MINOR: acme: wait 5s before checking the challenges results
- MINOR: acme: emit a log when starting
- MINOR: acme: delay of 5s after the finalize
- BUG/MEDIUM: quic: Let it be known if the tasklet has been released.
- BUG/MAJOR: tasks: fix task accounting when killed
- CLEANUP: tasks: use the local state, not t->state, to check for tasklets
- DOC: acme: external account binding is not supported
- MINOR: hlua: ignore "tune.lua.bool-sample-conversion" if set after "lua-load"
- MEDIUM: peers: Give up if we fail to take locks in hot path
- MEDIUM: stick-tables: defer adding updates to a tasklet
- MEDIUM: stick-tables: Limit the number of old entries we remove
- MEDIUM: stick-tables: Limit the number of entries we expire
- MINOR: cfgparse-global: add explicit error messages in cfg_parse_global_env_opts
- MINOR: ssl: add function to extract X509 notBefore date in time_t
- BUILD: acme: need HAVE_ASN1_TIME_TO_TM
- MINOR: acme: move the acme task init in a dedicated function
- MEDIUM: acme: add a basic scheduler
- MINOR: acme: emit a log when the scheduler can't start the task
This patch implements a very basic scheduler for the ACME tasks.
The scheduler is a task which is started from the postparser function
when at least one acme section was configured.
The scheduler will loop over the certificates in the ckchs_tree, and for
each certificate will start an ACME task if the notAfter date is past
curtime + (notAfter - notBefore) / 12, or 7 days if notBefore is not
available.
Once the lookup over all certificates is terminated, the task will sleep
and will wakeup after 12 hours.
tune.lua.bool-sample-conversion must be set before any lua-load or
lua-load-per-thread is used for it to be considered. Indeed, lua-load
directives are parsed on the fly and will cause some parts of the scripts
to be executed during init already (script body/init contexts).
As such, we cannot afford to have "tune.lua.bool-sample-conversion" set
after some Lua code was loaded, because it would mean that the setting
would be handled differently for Lua's code executed during or after
config parsing.
To avoid ambiguities, the documentation now states that the setting must
be set before any lua-load(-per-thread) directive, and if the setting
is met after some Lua was already loaded, the directive is ignored and
a warning informs about that.
It should fix GH #2957
It may be backported with 29b6d8af16 ("MINOR: hlua: rename
"tune.lua.preserve-smp-bool" to "tune.lua.bool-sample-conversion"")
Released version 3.2-dev13 with the following main changes :
- MEDIUM: checks: Make sure we return the tasklet from srv_chk_io_cb
- MEDIUM: listener: Make sure w ereturn the tasklet from accept_queue_process
- MEDIUM: mux_fcgi: Make sure we return the tasklet from fcgi_deferred_shut
- MEDIUM: quic: Make sure we return the tasklet from qcc_io_cb
- MEDIUM: quic: Make sure we return NULL in quic_conn_app_io_cb if needed
- MEDIUM: quic: Make sure we return the tasklet from quic_accept_run
- BUG/MAJOR: tasklets: Make sure he tasklet can't run twice
- BUG/MAJOR: listeners: transfer connection accounting when switching listeners
- MINOR: ssl/cli: add a '-t' option to 'show ssl sni'
- DOC: config: fix ACME paragraph rendering issue
- DOC: config: clarify log-forward "host" option
- MINOR: promex: expose ST_I_PX_RATE (current_session_rate)
- BUILD: acme: use my_strndup() instead of strndup()
- BUILD: leastconn: fix build warning when building without threads on old machines
- MINOR: threads: prepare DEBUG_THREAD to receive more values
- MINOR: threads: turn the full lock debugging to DEBUG_THREAD=2
- MEDIUM: threads: keep history of taken locks with DEBUG_THREAD > 0
- MINOR: threads/cli: display the lock history on "show threads"
- MEDIUM: thread: set DEBUG_THREAD to 1 by default
- BUG/MINOR: ssl/acme: free EVP_PKEY upon error
- MINOR: acme: separate the code generating private keys
- MINOR: acme: failure when no directory is specified
- MEDIUM: acme: generate the account file when not found
- MEDIUM: acme: use 'crt-base' to load the account key
- MINOR: compiler: add more macros to detect macro definitions
- MINOR: cli: split APPCTX_CLI_ST1_PROMPT into two distinct flags
- MEDIUM: cli: make the prompt mode configurable between n/i/p
- MEDIUM: mcli: make the prompt mode configurable between i/p
- MEDIUM: mcli: replicate the current mode when enterin the worker process
- DOC: configuration: acme account key are auto generated
- CLEANUP: acme: remove old TODO for account key
- DOC: configuration: add quic4 to the ssl-f-use example
- BUG/MINOR: acme: does not try to unlock after a failed trylock
- BUG/MINOR: mux-h2: fix the offset of the pattern for the ping frame
- MINOR: tcp: add support for setting TCP_NOTSENT_LOWAT on both sides
- BUG/MINOR: acme: creating an account should not end the task
- MINOR: quic: rename min/max fields for congestion window algo
- MINOR: quic: refactor BBR API
- BUG/MINOR: quic: ensure cwnd limits are always enforced
- MINOR: thread: define cshared type
- MINOR: quic: account for global congestion window
- MEDIUM: quic: limit global Tx memory
- MEDIUM: acme: use a map to store tokens and thumbprints
- BUG/MINOR: acme: remove references to virt@acme
- MINOR: applet: add appctx_schedule() macro
- BUG/MINOR: dns: add tempo between 2 connection attempts for dns servers
- CLEANUP: dns: remove unused dns_stream_server struct member
- BUG/MINOR: dns: prevent ds accumulation within dss
- CLEANUP: proxy: mention that px->conn_retries isn't relevant in some cases
- DOC: ring: refer to newer RFC5424
- MINOR: tools: make my_strndup() take a size_t len instead of and int
- MINOR: Add "sigalg" to "sigalg name" helper function
- MINOR: ssl: Add traces to ssl init/close functions
- MINOR: ssl: Add traces to recv/send functions
- MINOR: ssl: Add traces to ssl_sock_io_cb function
- MINOR: ssl: Add traces around SSL_do_handshake call
- MINOR: ssl: Add traces to verify callback
- MINOR: ssl: Add ocsp stapling callback traces
- MINOR: ssl: Add traces to the switchctx callback
- MINOR: ssl: Add traces about sigalg extension parsing in clientHello callback
- MINOR: Add 'conn' param to ssl_sock_chose_sni_ctx
- BUG/MEDIUM: mux-spop: Wait end of handshake to declare a spop connection ready
- BUG/MEDIUM: mux-spop: Handle CLOSING state and wait for AGENT DISCONNECT frame
- BUG/MINOR: mux-h1: Don't pretend connection was released for TCP>H1>H2 upgrade
- BUG/MINOR: mux-h1: Fix trace message in h1_detroy() to not relay on connection
- BUILD: ssl: Fix wolfssl build
- BUG/MINOR: mux-spop: Use the right bitwise operator in spop_ctl()
- MEDIUM: mux-quic: increase flow-control on each bufsize
- MINOR: mux-quic: limit emitted MSD frames count per qcs
- MINOR: add hlua_yield_asap() helper
- MINOR: hlua_fcn: enforce yield after *_get_stats() methods
- DOC: config: restore default values for resolvers hold directive
- MINOR: ssl/cli: "acme ps" shows the acme tasks
- MINOR: acme: acme_ctx_destroy() returns upon NULL
- MINOR: acme: use acme_ctx_destroy() upon error
- MEDIUM: tasks: Mutualize code between tasks and tasklets.
- MEDIUM: tasks: More code factorization
- MEDIUM: tasks: Remove TASK_IN_LIST and use TASK_QUEUED instead.
- MINOR: tasks: Remove unused tasklet_remove_from_tasklet_list
- MEDIUM: tasks: Mutualize the TASK_KILLED code between tasks and tasklets
- BUG/MEDIUM: connections: Report connection closing in conn_create_mux()
- BUILD/MEDIUM: quic: Make sure we build with recent changes
Implement a way to display the running acme tasks over the CLI.
It currently only displays a "Running" status with the certificate name
and the acme section from the configuration.
The displayed running tasks are limited to the size of a buffer for now,
it will require a backref list later to be called multiple times to
resume the list.
Default values for hold directive (resolver context) used to be documented
but this was lost when the keyword description was reworked in 24b319b
("Default value is 10s for "valid", 0s for "obsolete" and 30s for
others.")
Restoring the part that describes the default value.
It may be backported to all stable versions with 24b319b
The stateless mode which was documented previously in the ACME example
is not convenient for all use cases.
First, when HAProxy generates the account key itself, you wouldn't be
able to put the thumbprint in the configuration, so you will have to get
the thumbprint and then reload.
Second, in the case you are using multiple account key, there are
multiple thumbprint, and it's not easy to know which one you want to use
when responding to the challenger.
This patch allows to configure a map in the acme section, which will be
filled by the acme task with the token corresponding to the challenge,
as the key, and the thumbprint as the value. This way it's easy to reply
the right thumbprint.
Example:
http-request return status 200 content-type text/plain lf-string "%[path,field(-1,/)].%[path,field(-1,/),map(virt@acme)]\n" if { path_beg '/.well-known/acme-challenge/' }
Define a new settings tune.quic.frontend.max-tot-window. It contains a
size argument which can be used to set a limit on the sum of all QUIC
connections congestion window. This is applied both on
quic_cc_path_set() and quic_cc_path_inc().
Note that this limitation cannot reduce a congestion window more than
the minimal limit which is set to 2 datagrams.
TCP_NOTSENT_LOWAT is very convenient as it indicates when to report
EAGAIN on the sending side. It takes a margin on top of the estimated
window, meaning that it's no longer needed to store too many data in
socket buffers. Instead there's just enough to fill the send window
and a little bit of margin to cover the scheduling time to restart
sending. Experiments on a 100ms network have shown a 10-fold reduction
in the memory used by socket buffers by just setting this value to
tune.bufsize, without noticing any performance degradation. Theoretically
the responsiveness on multiplexed protocols such as H2 should also be
improved.
While humans can find it convenient to enter the worker process in prompt
mode, for external tools it will not be convenient to have to systematically
disable it. A better approach is to replicate the master socket's mode
there, since it has already been configured to suit the user: interactive,
prompt and timed modes are automatically passed to the worker process.
This makes the using the worker commands more natural from the master
process, without having to systematically adapt it for each new connection.
Support the same syntax in master mode as in worker mode in order to
configure the prompt. The only thing is that for now the master doesn't
have a non-interactive mode and it doesn't seem necessary to implement
it, so we only support the interactive and prompt modes. However the code
was written in a way that makes it easy to change this later if desired.
Now the prompt mode can more finely be configured between non-interactive
(default), interactive without prompt, and interactive with prompt. This
will ease the usage from automated tools which are not necessarily
interested in having to consume '> ' after each command nor displaying
"+" on payload lines. This can also be convenient when coming from the
master CLI to keep the same output format.
log-forward "host" option may be confusing because we often mention the
host field when talking about syslog RFC3164 or RFC5424 messages, but
neither rfc actually define "host" field. In fact, everywhere we used
"host field" we actually meant "hostname field" as documented in RFC5424.
This was a language abuse on our side.
In this patch we replace "host" with "hostname" where relevant in the
documentation to prevent confusion.
Thanks to Nick Ramirez for having reported the issue.
Nick Ramirez reported that the ACME paragraph (3.13) caused a rendering
issue where simple text was rendered as a directive. This was caused
by the use of unescaped <name> which confuses dconv.
Let's escape <name> by putting quotes around it to prevent the rendering
issue.
No backport needed.
Add a -t option to 'show ssl sni', allowing to add an offset to the
current date so it would allow to check which certificates are expired
after a certain period of time.
Released version 3.2-dev12 with the following main changes :
- BUG/MINOR: quic: do not crash on CRYPTO ncbuf alloc failure
- BUG/MINOR: proxy: always detach a proxy from the names tree on free()
- CLEANUP: proxy: detach the name node in proxy_free_common() instead
- CLEANUP: Slightly reorder some proxy option flags to free slots
- MINOR: proxy: Add options to drop HTTP trailers during message forwarding
- MINOR: h1-htx: Skip C-L and T-E headers for 1xx and 204 messages during parsing
- MINOR: mux-h1: Keep custom "Content-Length: 0" header in 1xx and 204 messages
- MINOR: hlua/h1: Use http_parse_cont_len_header() to parse content-length value
- CLEANUP: h1: Remove now useless h1_parse_cont_len_header() function
- BUG/MEDIUM: mux-spop: Respect the negociated max-frame-size value to send frames
- MINOR: http-act: Add 'pause' action to temporarily suspend the message analysis
- MINOR: acme/cli: add the 'acme renew' command to the help message
- MINOR: httpclient: add an "https" log-format
- MEDIUM: acme: use a customized proxy
- MEDIUM: acme: rename "uri" into "directory"
- MEDIUM: acme: rename "account" into "account-key"
- MINOR: stick-table: use a separate lock label for updates
- MINOR: h3: simplify h3_rcv_buf return path
- BUG/MINOR: mux-quic: fix possible infinite loop during decoding
- BUG/MINOR: mux-quic: do not decode if conn in error
- BUG/MINOR: cli: Issue an error when too many args are passed for a command
- MINOR: cli: Use a full prompt command for bidir connections with workers
- MAJOR: cli: Refacor parsing and execution of pipelined commands
- MINOR: cli: Rename some CLI applet states to reflect recent refactoring
- CLEANUP: applet: Update st0/st1 comment in appctx structure
- BUG/MINOR: hlua: Fix I/O handler of lua CLI commands to not rely on the SC
- BUG/MINOR: ring: Fix I/O handler of "show event" command to not rely on the SC
- MINOR: cli/applet: Move appctx fields only used by the CLI in a private context
- MINOR: cache: Add a pointer on the cache in the cache applet context
- MINOR: hlua: Use the applet name in error messages for lua services
- MINOR: applet: Save the "use-service" rule in the stream to init a service applet
- CLEANUP: applet: Remove unsued rule pointer in appctx structure
- BUG/MINOR: master/cli: properly trim the '@@' process name in error messages
- MEDIUM: resolvers: add global "dns-accept-family" directive
- MINOR: resolvers: add command-line argument -4 to force IPv4-only DNS
- MINOR: sock-inet: detect apparent IPv6 connectivity
- MINOR: resolvers: add "dns-accept-family auto" to rely on detected IPv6
- MEDIUM: acme: use Retry-After value for retries
- MEDIUM: acme: reset the remaining retries
- MEDIUM: acme: better error/retry management of the challenge checks
- BUG/MEDIUM: cli: Handle applet shutdown when waiting for a command line
- Revert "BUG/MINOR: master/cli: properly trim the '@@' process name in error messages"
- BUG/MINOR: master/cli: only parse the '@@' prefix on complete lines
- MINOR: resolvers: use the runtime IPv6 status instead of boot time one
Instead of always having to force IPv4 or IPv6, let's now also offer
"auto" which will only enable IPv6 if the system has a default gateway
for it. This means that properly configured dual-stack systems will
default to "ipv4,ipv6" while those lacking a gateway will only use
"ipv4". Note that no real connectivity test is performed, so firewalled
systems may still get it wrong and might prefer to rely on a manual
"ipv4" assignment.
In order to ease troubleshooting and testing, the new "-4" command line
argument enforces queries and processing of "A" DNS records only, i.e.
those representing IPv4 addresses. This can be useful when a host lack
end-to-end dual-stack connectivity. This overrides the global
"dns-accept-family" directive and is equivalent to value "ipv4".
By default, DNS resolvers accept both IPv4 and IPv6 addresses. This can be
influenced by the "resolve-prefer" keywords on server lines as well as the
family argument to the "do-resolve" action, but that is only a preference,
which does not block the other family from being used when it's alone. In
some environments where dual-stack is not usable, stumbling on an unreachable
IPv6-only DNS record can cause significant trouble as it will replace a
previous IPv4 one which would possibly have continued to work till next
request. The "dns-accept-family" global option permits to enforce usage of
only one (or both) address families. The argument is a comma-delimited list
of the following words:
- "ipv4": query and accept IPv4 addresses ("A" records)
- "ipv6": query and accept IPv6 addresses ("AAAA" records)
When a single family is used, no request will be sent to resolvers for the
other family, and any response for the othe family will be ignored. The
default value is "ipv4,ipv6", which effectively enables both families.
The 'pause' HTTP action can now be used to suspend for a moment the message
analysis. A timeout, expressed in milliseconds using a time-format
parameter, or an expression can be used. If an expression is used, errors
and invalid values are ignored.
Internally, the action will set the analysis expiration date on the
corresponding channel to the configured value and it will yield while it is
not expired.
The 'pause' action is available for 'http-request' and 'http-response'
rules.
In RFC9110, it is stated that trailers could be merged with the
headers. While it should be performed with a speicial care, it may be a
problem for some applications. To avoid any trouble with such applications,
two new options were added to drop trailers during the message forwarding.
On the backend, "http-drop-request-trailers" option can be enabled to drop
trailers from the requests before sending them to the server. And on the
frontend, "http-drop-response-trailers" option can be enabled to drop
trailers from the responses before sending them to the client. The options
can be defined in defaults sections and disabled with "no" keyword.
This patch should fix the issue #2930.
Released version 3.2-dev11 with the following main changes :
- CI: enable weekly QuicTLS build
- DOC: management: slightly clarify the prefix role of the '@' command
- DOC: management: add a paragraph about the limitations of the '@' prefix
- MINOR: master/cli: support bidirectional communications with workers
- MEDIUM: ssl/ckch: add filename and linenum argument to crt-store parsing
- MINOR: acme: add the acme section in the configuration parser
- MINOR: acme: add configuration for the crt-store
- MINOR: acme: add private key configuration
- MINOR: acme/cli: add the 'acme renew' command
- MINOR: acme: the acme section is experimental
- MINOR: acme: get the ACME directory
- MINOR: acme: handle the nonce
- MINOR: acme: check if the account exist
- MINOR: acme: generate new account
- MINOR: acme: newOrder request retrieve authorizations URLs
- MINOR: acme: allow empty payload in acme_jws_payload()
- MINOR: acme: get the challenges object from the Auth URL
- MINOR: acme: send the request for challenge ready
- MINOR: acme: implement a check on the challenge status
- MINOR: acme: generate the CSR in a X509_REQ
- MINOR: acme: finalize by sending the CSR
- MINOR: acme: verify the order status once finalized
- MINOR: acme: implement retrieval of the certificate
- BUG/MINOR: acme: ckch_conf_acme_init() when no filename
- MINOR: ssl/ckch: handle ckch_conf in ckchs_dup() and ckch_conf_clean()
- MINOR: acme: copy the original ckch_store
- MEDIUM: acme: replace the previous ckch instance with new ones
- MINOR: acme: schedule retries with a timer
- BUILD: acme: enable the ACME feature when JWS is present
- BUG/MINOR: cpu-topo: check the correct variable for NULL after malloc()
- BUG/MINOR: acme: key not restored upon error in acme_res_certificate()
- BUG/MINOR: thread: protect thread_cpus_enabled_at_boot with USE_THREAD
- MINOR: acme: default to 2048bits for RSA
- DOC: acme: explain how to configure and run ACME
- BUG/MINOR: debug: remove the trailing \n from BUG_ON() statements
- DOC: config: add the missing "profiling.memory" to the global kw index
- DOC: config: add the missing "force-cfg-parser-pause" to the global kw index
- DEBUG: init: report invalid characters in debug description strings
- DEBUG: rename DEBUG_GLITCHES to DEBUG_COUNTERS and enable it by default
- DEBUG: counters: make COUNT_IF() only appear at DEBUG_COUNTERS>=1
- DEBUG: counters: add the ability to enable/disable updating the COUNT_IF counters
- MINOR: tools: let dump_addr_and_bytes() support dumping before the offset
- MINOR: debug: in call traces, dump the 8 bytes before the return address, not after
- MINOR: debug: detect call instructions and show the branch target in backtraces
- BUG/MINOR: acme: fix possible NULL deref
- CLEANUP: acme: stored value is overwritten before it can be used
- BUILD: incompatible pointer type suspected with -DDEBUG_UNIT
- BUG/MINOR: http-ana: Properly detect client abort when forwarding the response
- BUG/MEDIUM: http-ana: Report 502 from req analyzer only during rsp forwarding
- CI: fedora rawhide: enable unit tests
- DOC: configuration: fix a typo in ACME documentation
- MEDIUM: sink: add a new dpapi ring buffer
- Revert "BUG/MINOR: acme: key not restored upon error in acme_res_certificate()"
- BUG/MINOR: acme: key not restored upon error in acme_res_certificate() V2
- BUG/MINOR: acme: fix the exponential backoff of retries
- DOC: configuration: specify limitations of ACME for 3.2
- MINOR: acme: emit logs instead of ha_notice
- MINOR: acme: add a success message to the logs
- BUG/MINOR: acme/cli: fix certificate name in error message
- MINOR: acme: register the task in the ckch_store
- MINOR: acme: free acme_ctx once the task is done
- BUG/MEDIUM: h3: trim whitespaces when parsing headers value
- BUG/MEDIUM: h3: trim whitespaces in header value prior to QPACK encoding
- BUG/MINOR: h3: filter upgrade connection header
- BUG/MINOR: h3: reject invalid :path in request
- BUG/MINOR: h3: reject request URI with invalid characters
- MEDIUM: h3: use absolute URI form with :authority
- BUG/MEDIUM: hlua: fix hlua_applet_{http,tcp}_fct() yield regression (lost data)
- BUG/MINOR: mux-h2: prevent past scheduling with idle connections
- BUG/MINOR: rhttp: fix reconnect if timeout connect unset
- BUG/MINOR: rhttp: ensure GOAWAY can be emitted after reversal
- BUG/MINOR: mux-h2: do not apply timer on idle backend connection
- MINOR: mux-h2: refactor idle timeout calculation
- MINOR: mux-h2: prepare to support PING emission
- MEDIUM: server/mux-h2: implement idle-ping on backend side
- MEDIUM: listener/mux-h2: implement idle-ping on frontend side
- MINOR: mux-h2: do not emit GOAWAY on idle ping expiration
- MINOR: mux-h2: handle idle-ping on conn reverse
- BUILD: makefile: enable backtrace by default on musl
- BUG/MINOR: threads: set threads_idle and threads_harmless even with no threads
- BUG/MINOR debug: fix !USE_THREAD_DUMP in ha_thread_dump_fill()
- BUG/MINOR: wdt/debug: avoid signal re-entrance between debugger and watchdog
- BUG/MINOR: debug: detect and prevent re-entrance in ha_thread_dump_fill()
- MINOR: debug: do not statify a few debugging functions often used with wdt/dbg
- MINOR: tools: also protect the library name resolution against concurrent accesses
- MINOR: tools: protect dladdr() against reentrant calls from the debug handler
- MINOR: debug: protect ha_dump_backtrace() against risks of re-entrance
- MINOR: tinfo: keep a copy of the pointer to the thread dump buffer
- MINOR: debug: always reset the dump pointer when done
- MINOR: debug: remove unused case of thr!=tid in ha_thread_dump_one()
- MINOR: pass a valid buffer pointer to ha_thread_dump_one()
- MEDIUM: wdt: always make the faulty thread report its own warnings
- MINOR: debug: make ha_stuck_warning() only work for the current thread
- MINOR: debug: make ha_stuck_warning() print the whole message at once
- CLEANUP: debug: no longer set nor use TH_FL_DUMPING_OTHERS
- MINOR: sched: add a new function is_sched_alive() to report scheduler's health
- MINOR: wdt: use is_sched_alive() instead of keeping a local ctxsw copy
- MINOR: sample: add 4 new sample fetches for clienthello parsing
- REGTEST: add new reg-test for the 4 new clienthello fetches
- MINOR: servers: Move the per-thread server initialization earlier
- MINOR: proxies: Initialize the per-thread structure earlier.
- MINOR: servers: Provide a pointer to the server in srv_per_tgroup.
- MINOR: lb_fwrr: Move the next weight out of fwrr_group.
- MINOR: proxies: Add a per-thread group lbprm struct.
- MEDIUM: lb_fwrr: Use one ebtree per thread group.
- MEDIUM: lb_fwrr: Don't start all thread groups on the same server.
- MINOR: proxies: Do stage2 initialization for sinks too
This patch contains this 4 new fetches and doc changes for the new fetches:
- req.ssl_cipherlist
- req.ssl_sigalgs
- req.ssl_keyshare_groups
- req.ssl_supported_groups
Towards:#2532
Instead of using the thread dump buffer for post-mortem analysis, we'll
keep a copy of the assigned pointer whenever it's used, even for warnings
or "show threads". This will offer more opportunities to figure from a
core what happened, and will give us more freedom regarding the value of
the thread_dump_buffer itself. For example, even at the end of the dump
when the pointer is reset, the last used buffer is now preserved.
This commit extends MUX H2 connection reversal step to properly take
into account the new idle-ping feature. It first ensures that h2c task
is properly instantiated/freed depending now on both timers and
idle-ping configuration. Also, h2c_update_timeout() is now called
instead of manually requeuing the task, which ensures the proper timer
value is selected depending on the new connection side.
This commit is the counterpart of the previous one, adapted on the
frontend side. "idle-ping" is added as keyword to bind lines, to be able
to refresh client timeout of idle frontend connections.
H2 MUX behavior remains similar as the previous patch. The only
significant change is in h2c_update_timeout(), as idle-ping is now taken
into account also for frontend connection. The calculated value is
compared with http-request/http-keep-alive timeout value. The shorter
delay is then used as expired date. As hr/ka timeout are based on
idle_start, this allows to run them in parallel with an idle-ping timer.
This commit implements support for idle-ping on the backend side. First,
a new server keyword "idle-ping" is defined in configuration parsing. It
is used to set the corresponding new server member.
The second part of this commit implements idle-ping support on H2 MUX. A
new inlined function conn_idle_ping() is defined to access connection
idle-ping value. Two new connection flags are defined H2_CF_IDL_PING and
H2_CF_IDL_PING_SENT. The first one is set for idle connections via
h2c_update_timeout().
On h2_timeout_task() handler, if first flag is set, instead of releasing
the connection as before, the second flag is set and tasklet is
scheduled. As both flags are now set, h2_process_mux() will proceed to
PING emission. The timer has also been rearmed to the idle-ping value.
If a PING ACK is received before next timeout, connection timer is
refreshed. Else, the connection is released, as with timer expiration.
Also of importance, special care is needed when a backend connection is
going to idle. In this case, idle-ping timer must be rearmed. Thus a new
invokation of h2c_update_timeout() is performed on h2_detach().
These counters can have a noticeable cost on large machines, though not
dramatic. There's no single good choice to keep them enabled or disabled.
This commit adds multiple choices:
- DEBUG_COUNTERS set to 2 will automatically enable them by default, while
1 will disable them by default
- the global "debug.counters on/off" will allow to change the setting at
boot, regardless of DEBUG_COUNTERS as long as it was at least 1.
- the CLI "debug counters on/off" will also allow to change the value at
run time, allowing to observe a phenomenon while it's happening, or to
disable counters if it's suspected that their cost is too high
Finally, the "debug counters" command will append "(stopped)" at the end
of the CNT lines when these counters are stopped.
Not that the whole mechanism would easily support being extended to all
counter types by specifying the types to apply to, but it doesn't seem
useful at all and would require the user to also type "cnt" on debug
lines. This may easily be changed in the future if it's found relevant.
Till now the per-line glitches counters were only enabled with the
confusingly named DEBUG_GLITCHES (which would not turn glitches off
when disabled). Let's instead change it to DEBUG_COUNTERS and make sure
it's enabled by default (though it can still be disabled with
-DDEBUG_GLITCHES=0 just like for DEBUG_STRICT). It will later be
expanded to cover more counters.
Some rare commands in the worker require to keep their input open and
terminate when it's closed ("show events -w", "wait"). Others maintain
a per-session context ("set anon on"). But in its default operation
mode, the master CLI passes commands one at a time to the worker, and
closes the CLI's input channel so that the command can immediately
close upon response. This effectively prevents these two specific cases
from being used.
Here the approach that we take is to introduce a bidirectional mode to
connect to the worker, where everything sent to the master is immediately
forwarded to the worker (including the raw command), allowing to queue
multiple commands at once in the same session, and to continue to watch
the input to detect when the client closes. It must be a client's choice
however, since doing so means that the client cannot batch many commands
at once to the master process, but must wait for these commands to complete
before sending new ones. For this reason we use the prefix "@@<pid>" for
this. It works exactly like "@" except that it maintains the channel
open during the whole execution. Similarly to "@<pid>" with no command,
"@@<pid>" will simply open an interactive CLI session to the worker, that
will be ended by "quit" or by closing the connection. This can be convenient
for the user, and possibly for clients willing to dedicate a connection to
the worker.
The '@' prefix permits to execute a single command at once in a worker.
It is very handy but comes with some limitations affecting rare commands,
which is better to be documented (one command per session, input closed)
since it can seldom have user-visible effects.
While the examples were clear, the text did not fully imply what was
reflected there. Better have the text explicitly mention that the
'@' command may be used as a prefix or wrapper in front of a command
as well as a standalone command.
Released version 3.2-dev10 with the following main changes :
- REORG: ssl: move curves2nid and nid2nist to ssl_utils
- BUG/MEDIUM: stream: Fix a possible freeze during a forced shut on a stream
- MEDIUM: stream: Save SC and channel flags earlier in process_steam()
- BUG/MINOR: peers: fix expire learned from a peer not converted from ms to ticks
- BUG/MEDIUM: peers: prevent learning expiration too far in futur from unsync node
- CI: spell check: allow manual trigger
- CI: codespell: add "pres" to spellcheck whitelist
- CLEANUP: assorted typo fixes in the code, commits and doc
- CLEANUP: atomics: remove support for gcc < 4.7
- CLEANUP: atomics: also replace __sync_synchronize() with __atomic_thread_fence()
- TESTS: Fix build for filltab25.c
- MEDIUM: ssl: replace "crt" lines by "ssl-f-use" lines
- DOC: configuration: replace "crt" by "ssl-f-use" in listeners
- MINOR: backend: mark srv as nonnull in alloc_dst_address()
- BUG/MINOR: server: ensure check-reuse-pool is copied from default-server
- MINOR: server: activate automatically check reuse for rhttp@ protocol
- MINOR: check/backend: support conn reuse with SNI
- MINOR: check: implement check-pool-conn-name srv keyword
- MINOR: task: add thread safe notification_new and notification_wake variants
- BUG/MINOR: hlua_fcn: fix potential UAF with Queue:pop_wait()
- MINOR: hlua_fcn: register queue class using hlua_register_metatable()
- MINOR: hlua: add core.wait()
- MINOR: hlua: core.wait() takes optional delay paramater
- MINOR: hlua: split hlua_applet_tcp_recv_yield() in two functions
- MINOR: hlua: add AppletTCP:try_receive()
- MINOR: hlua_fcn: add Queue:alarm()
- MEDIUM: task: make notification_* API thread safe by default
- CLEANUP: log: adjust _lf_cbor_encode_byte() comment
- MEDIUM: ssl/crt-list: warn on negative wildcard filters
- MEDIUM: ssl/crt-list: warn on negative filters only
- BUILD: atomics: fix build issue on non-x86/non-arm systems
- BUG/MINOR: log: fix CBOR encoding with LOG_VARTEXT_START() + lf_encode_chunk()
- BUG/MEDIUM: sample: fix risk of overflow when replacing multiple regex back-refs
- DOC: configuration: rework the crt-list section
- MINOR: ring: support arbitrary delimiters through ring_dispatch_messages()
- MINOR: ring/cli: support delimiting events with a trailing \0 on "show events"
- DEV: h2: fix h2-tracer.lua nil value index
- BUG/MINOR: backend: do not use the source port when hashing clientip
- BUG/MINOR: hlua: fix invalid errmsg use in hlua_init()
- MINOR: proxy: add setup_new_proxy() function
- MINOR: checks: mark CHECKS-FE dummy frontend as internal
- MINOR: flt_spoe: mark spoe agent frontend as internal
- MEDIUM: tree-wide: avoid manually initializing proxies
- MINOR: proxy: add deinit_proxy() helper func
- MINOR: checks: deinit checks_fe upon deinit
- MINOR: flt_spoe: deinit spoe agent proxy upon agent release
At the moment it is not supported to produce multi-line events on the
"show events" output, simply because the LF character is used as the
default end-of-event mark. However it could be convenient to produce
well-formatted multi-line events, e.g. in JSON or other formats. UNIX
utilities have already faced similar needs in the past and added
"-print0" to "find" and "-0" to "xargs" to mention that the delimiter
is the NUL character. This makes perfect sense since it's never present
in contents, so let's do exactly the same here.
Thus from now on, "show events <ring> -0" will delimit messages using
a \0 instead of a \n, permitting a better and safer encapsulation.
Queue:alarm() sets a wakeup alarm on the task when new data becomes
available on Queue. It must be re-armed for each event.
Lua documentation was updated
This is the non-blocking variant for AppletTCP:receive(). It doesn't
take any argument, instead it tries to read as much data as available
at once. If no data is available, empty string is returned.
Lua documentation was updated.
core.wait() now accepts optional delay parameter in ms. Passed this delay
the task is woken up if no event woke the task before.
Lua documentation was updated.
