1950 Commits

Author SHA1 Message Date
Christopher Faulet
6072beb214 MINOR: http-ana: Match on the path if the monitor-uri starts by a /
if the monitor-uri starts by a slash ('/'), the matching is performed against
the request's path instead of the request's uri. It is a workaround to let the
HTTP/2 requests match the monitor-uri. Indeed, in HTTP/2, clients are encouraged
to send absolute URIs only.

This patch is not tagged as a bug, because the previous behavior matched exactly
what the doc describes. But it may surprise that HTTP/2 requests don't match the
monitor-uri.

This patch may be backported to 2.1 because URIs of HTTP/2 are stored using the
absolute-form starting this version. For previous versions, this patch will only
helps explicitely absolute HTTP/1 requests (and only the HTX part because on the
legacy HTTP, all the URI is matched).

It should fix the issue #509.
2020-02-18 16:29:29 +01:00
Emmanuel Hocdet
70df7bf19c MINOR: ssl: add "issuers-chain-path" directive.
Certificates loaded with "crt" and "crt-list" commonly share the same
intermediate certificate in PEM file. "issuers-chain-path" is a global
directive to share intermediate chain certificates in a directory. If
certificates chain is not included in certificate PEM file, haproxy
will complete chain if issuer match the first certificate of the chain
stored via "issuers-chain-path" directive. Such chains will be shared
in memory.
2020-02-18 14:33:05 +01:00
Jerome Magnin
07e1e3c93e MINOR: sample: regsub now supports backreferences
Now that the configuration parser is more flexible with samples,
converters and their arguments, we can leverage this to enable
support for backreferences in regsub.
2020-02-16 19:48:54 +01:00
Willy Tarreau
cd0d2ed6ee MEDIUM: log-format: make the LF parser aware of sample expressions' end
For a very long time it used to be impossible to pass a closing square
bracket as a valid character in argument to a sample fetch function or
to a converter because the LF parser used to stop on the first such
character found and to pass what was between the first '[' and the first
']' to sample_parse_expr().

This patch addresses this by passing the whole string to sample_parse_expr()
which is the only one authoritative to indicate the first character that
does not belong to the expression. The LF parser then verifies it matches
a ']' or fails. As a result it is finally possible to write rules such as
the following, which is totally valid an unambigous :

    http-request redirect location %[url,regsub([.:/?-],!,g)]
                                                |-----| | |
                                                  arg1  | `---> arg3
                                                        `-----> arg2
                                         |-----------------|
                                              converter
                                     |---------------------|
                                        sample expression
                                   |------------------------|
                                         log-format tag
2020-02-14 19:02:06 +01:00
Willy Tarreau
ef21facd99 MEDIUM: arg: make make_arg_list() support quotes in arguments
Now it becomes possible to reuse the quotes within arguments, allowing
the parser to distinguish a ',' or ')' that is part of the value from
one which delimits the argument. In addition, ',' and ')' may be escaped
using a backslash. However, it is also important to keep in mind that
just like in shell, quotes are first resolved by the word tokenizer, so
in order to pass quotes that are visible to the argument parser, a second
level is needed, either using backslash escaping, or by using an alternate
type.

For example, it's possible to write this to append a comma:

     http-request add-header paren-comma-paren "%[str('(--,--)')]"

or this:

     http-request add-header paren-comma-paren '%[str("(--,--)")]'

or this:

     http-request add-header paren-comma-paren %[str(\'(--,--)\')]

or this:

     http-request add-header paren-comma-paren %[str(\"(--,--)\")]

or this:

     http-request add-header paren-comma-paren %[str(\"(\"--\',\'--\")\")]

Note that due to the wide use of '\' in front of parenthesis in regex,
the backslash character will purposely *not* escape parenthesis, so that
'\)' placed in quotes is passed verbatim to a regex engine.
2020-02-14 19:02:06 +01:00
Christopher Faulet
6c57f2da43 MINOR: mux-fcgi: Make the capture of the path-info optional in pathinfo regex
Now, only one capture is mandatory in the path-info regex, the one matching the
script-name. The path-info capture is optional. Of couse, it must be defined to
fill the PATH_INFO parameter. But it is not mandatory. This way, it is possible
to get the script-name part from the path, excluding the path-info.

This patch is small enough to be backported to 2.1.
2020-02-14 18:31:29 +01:00
Christopher Faulet
28cb36613b BUG/MINOR: mux-fcgi: Forbid special characters when matching PATH_INFO param
If a regex to match the PATH_INFO parameter is configured, it systematically
fails if a newline or a null character is present in the URL-decoded path. So,
from the moment there is at least a "%0a" or a "%00" in the request path, we
always fail to get the PATH_INFO parameter and all the decoded path is used for
the SCRIPT_NAME parameter.

It is probably not the expected behavior. Because, most of time, these
characters are not expected at all in a path, an error is now triggered when one
of these characters is found in the URL-decoded path before trying to execute
the path_info regex. However, this test is not performed if there is no regex
configured.

Note that in reality, the newline character is only a problem when HAProxy is
complied with pcre or pcre2 library and conversely, the null character is only a
problem for the libc's regex library. But both are always excluded to avoid any
inconsistency depending on compile options.

An alternative, not implemented yet, is to replace these characters by another
one. If someone complains about this behavior, it will be re-evaluated.

This patch must be backported to all versions supporting the FastCGI
applications, so to 2.1 for now.
2020-02-14 16:02:35 +01:00
Willy Tarreau
4c47d9106a [RELEASE] Released version 2.2-dev2
Released version 2.2-dev2 with the following main changes :
    - BUILD: CI: temporarily mark openssl-1.0.2 as allowed failure
    - MEDIUM: cli: Allow multiple filter entries for "show table"
    - BUG/MEDIUM: netscaler: Don't forget to allocate storage for conn->src/dst.
    - BUG/MINOR: ssl: ssl_sock_load_pem_into_ckch is not consistent
    - BUILD: stick-table: fix build errors introduced by last stick-table change
    - BUG/MINOR: cli: Missing arg offset for filter data values.
    - MEDIUM: streams: Always create a conn_stream in connect_server().
    - MEDIUM: connections: Get ride of the xprt_done callback.
    - CLEANUP: changelog: remove the duplicate entry for 2.2-dev1
    - BUILD: CI: move cygwin builds to Github Actions
    - MINOR: cli: Report location of errors or any extra data for "show table"
    - BUG/MINOR: ssl/cli: free the previous ckch content once a PEM is loaded
    - CLEANUP: backend: remove useless test for inexistent connection
    - CLEANUP: backend: shut another false null-deref in back_handle_st_con()
    - CLEANUP: stats: shut up a wrong null-deref warning from gcc 9.2
    - BUG/MINOR: ssl: increment issuer refcount if in chain
    - BUG/MINOR: ssl: memory leak w/ the ocsp_issuer
    - BUG/MINOR: ssl: typo in previous patch
    - BUG/MEDIUM: connections: Set CO_FL_CONNECTED in conn_complete_session().
    - BUG/MINOR: ssl/cli: ocsp_issuer must be set w/ "set ssl cert"
    - MEDIUM: connection: remove CO_FL_CONNECTED and only rely on CO_FL_WAIT_*
    - BUG/MEDIUM: 0rtt: Only consider the SSL handshake.
    - MINOR: stream-int: always report received shutdowns
    - MINOR: connection: remove CO_FL_SSL_WAIT_HS from CO_FL_HANDSHAKE
    - MEDIUM: connection: use CO_FL_WAIT_XPRT more consistently than L4/L6/HANDSHAKE
    - MINOR: connection: remove checks for CO_FL_HANDSHAKE before I/O
    - MINOR: connection: do not check for CO_FL_SOCK_RD_SH too early
    - MINOR: connection: don't check for CO_FL_SOCK_WR_SH too early in handshakes
    - MINOR: raw-sock: always check for CO_FL_SOCK_WR_SH before sending
    - MINOR: connection: remove some unneeded checks for CO_FL_SOCK_WR_SH
    - BUG/MINOR: stktable: report the current proxy name in error messages
    - BUG/MEDIUM: mux-h2: make sure we don't emit TE headers with anything but "trailers"
    - MINOR: lua: Add hlua_prepend_path function
    - MINOR: lua: Add lua-prepend-path configuration option
    - MINOR: lua: Add HLUA_PREPEND_C?PATH build option
    - BUILD: cfgparse: silence a bogus gcc warning on 32-bit machines
    - BUG/MINOR: http-ana: Increment the backend counters on the backend
    - BUG/MINOR: stream: Be sure to have a listener to increment its counters
    - BUG/MEDIUM: streams: Move the conn_stream allocation outside #IF USE_OPENSSL.
    - REGTESTS: make the set_ssl_cert test require version 2.2
    - BUG/MINOR: ssl: Possible memleak when allowing the 0RTT data buffer.
    - MINOR: ssl: Remove dead code.
    - BUG/MEDIUM: ssl: Don't forget to free ctx->ssl on failure.
    - BUG/MEDIUM: stream: Don't install the mux in back_handle_st_con().
    - MEDIUM: streams: Don't close the connection in back_handle_st_con().
    - MEDIUM: streams: Don't close the connection in back_handle_st_rdy().
    - BUILD: CI: disable slow regtests on Travis
    - BUG/MINOR: tcpchecks: fix the connect() flags regarding delayed ack
    - BUG/MINOR: http-rules: Always init log-format expr for common HTTP actions
    - BUG/MINOR: connection: fix ip6 dst_port copy in make_proxy_line_v2
    - BUG/MINOR: dns: allow 63 char in hostname
    - MINOR: proxy: clarify number of connections log when stopping
    - DOC: word converter ignores delimiters at the start or end of input string
    - MEDIUM: raw-sock: remove obsolete calls to fd_{cant,cond,done}_{send,recv}
    - BUG/MINOR: ssl/cli: fix unused variable with openssl < 1.0.2
    - MEDIUM: pipe/thread: reduce the locking overhead
    - MEDIUM: pipe/thread: maintain a per-thread local cache of recently used pipes
    - BUG/MEDIUM: pipe/thread: fix atomicity of pipe counters
    - MINOR: tasks: move the list walking code to its own function
    - MEDIUM: tasks: implement 3 different tasklet classes with their own queues
    - MEDIUM: tasks: automatically requeue into the bulk queue an already running tasklet
    - OPTIM: task: refine task classes default CPU bandwidth ratios
    - BUG/MEDIUM: connections: Don't forget to unlock when killing a connection.
    - MINOR: task: permanently flag tasklets waking themselves up
    - MINOR: task: make sched->current also reflect tasklets
    - MINOR: task: detect self-wakeups on tl==sched->current instead of TASK_RUNNING
    - OPTIM: task: readjust CPU bandwidth distribution since last update
    - MINOR: task: don't set TASK_RUNNING on tasklets
    - BUG/MEDIUM: memory_pool: Update the seq number in pool_flush().
    - MINOR: memory: Only init the pool spinlock once.
    - BUG/MEDIUM: memory: Add a rwlock before freeing memory.
    - BUG/MAJOR: memory: Don't forget to unlock the rwlock if the pool is empty.
    - MINOR: ssl: ssl-load-extra-files configure loading of files
    - SCRIPTS: add a new "backport" script to simplify long series of backports
    - BUG/MINOR: ssl: we may only ignore the first 64 errors
    - SCRIPTS: use /usr/bin/env bash instead of /bin/bash for scripts
    - BUG/MINOR: ssl: clear the SSL errors on DH loading failure
    - CLEANUP: hpack: remove a redundant test in the decoder
    - CLEANUP: peers: Remove unused static function `free_dcache`
    - CLEANUP: peers: Remove unused static function `free_dcache_tx`
    - CONTRIB: debug: add missing flags SF_HTX and SF_MUX
    - CONTRIB: debug: add the possibility to decode the value as certain types only
    - CONTRIB: debug: support reporting multiple values at once
    - BUG/MINOR: http-act: Use the good message to test strict rewritting mode
    - MINOR: global: Set default tune.maxrewrite value during global structure init
    - MINOR: http-rules: Set SF_ERR_PRXCOND termination flag when a header rewrite fails
    - MINOR: http-htx: Emit a warning if an error file runs over the buffer's reserve
    - MINOR: htx: Add a function to append an HTX message to another one
    - MINOR: htx/channel: Add a function to copy an HTX message in a channel's buffer
    - BUG/MINOR: http-ana: Don't overwrite outgoing data when an error is reported
    - MINOR: dns: Dynamically allocate dns options to reduce the act_rule size
    - MINOR: dns: Add function to release memory allocated for a do-resolve rule
    - BUG/MINOR: http-ana: Reset HTX first index when HAPRoxy sends a response
    - BUG/MINOR: http-ana: Set HTX_FL_PROXY_RESP flag if a server perform a redirect
    - MINOR: http-rules: Add a flag on redirect rules to know the rule direction
    - MINOR: http-rules: Handle the rule direction when a redirect is evaluated
    - MINOR: http-ana: Rely on http_reply_and_close() to handle server error
    - MINOR: http-ana: Add a function for forward internal responses
    - MINOR: http-ana/http-rules: Use dedicated function to forward internal responses
    - MEDIUM: http: Add a ruleset evaluated on all responses just before forwarding
    - MEDIUM: http-rules: Add the return action to HTTP rules
    - MEDIUM: http-rules: Support extra headers for HTTP return actions
    - CLEANUP: lua: Remove consistency check for sample fetches and actions
    - BUG/MINOR: http-ana: Increment failed_resp counters on invalid response
    - MINOR: lua: Get the action return code on the stack when an action finishes
    - MINOR: lua: Create the global 'act' object to register all action return codes
    - MINOR: lua: Add act:wake_time() function to set a timeout when an action yields
    - MEDIUM: lua: Add ability for actions to intercept HTTP messages
    - REGTESTS: Add reg tests for the HTTP return action
    - REGTESTS: Add a reg test for http-after-response rulesets
    - BUILD: lua: silence a warning on systems where longjmp is not marked as noreturn
    - MINOR: acl: Warn when an ACL is named 'or'
    - CONTRIB: debug: also support reading values from stdin
    - SCRIPTS: backport: use short revs and resolve the initial commit
    - BUG/MINOR: acl: Fix type of log message when an acl is named 'or'
2020-02-07 04:12:19 +01:00
Christopher Faulet
4a2c142779 MEDIUM: http-rules: Support extra headers for HTTP return actions
It is now possible to append extra headers to the generated responses by HTTP
return actions, while it is not based on an errorfile. For return actions based
on errorfiles, these extra headers are ignored. To define an extra header, a
"hdr" argument must be used with a name and a value. The value is a log-format
string. For instance:

  http-request status 200 hdr "x-src" "%[src]" hdr "x-dst" "%[dst]"
2020-02-06 15:13:03 +01:00
Christopher Faulet
24231ab61f MEDIUM: http-rules: Add the return action to HTTP rules
Thanks to this new action, it is now possible to return any responses from
HAProxy, with any status code, based on an errorfile, a file or a string. Unlike
the other internal messages generated by HAProxy, these ones are not interpreted
as errors. And it is not necessary to use a file containing a full HTTP
response, although it is still possible. In addition, using a log-format string
or a log-format file, it is possible to have responses with a dynamic
content. This action can be used on the request path or the response path. The
only constraint is to have a responses smaller than a buffer. And to avoid any
warning the buffer space reserved to the headers rewritting should also be free.

When a response is returned with a file or a string as payload, it only contains
the content-length header and the content-type header, if applicable. Here are
examples:

  http-request return content-type image/x-icon file /var/www/favicon.ico  \
      if { path /favicon.ico }

  http-request return status 403 content-type text/plain    \
      lf-string "Access denied. IP %[src] is blacklisted."  \
      if { src -f /etc/haproxy/blacklist.lst }
2020-02-06 15:12:54 +01:00
Christopher Faulet
6d0c3dfac6 MEDIUM: http: Add a ruleset evaluated on all responses just before forwarding
This patch introduces the 'http-after-response' rules. These rules are evaluated
at the end of the response analysis, just before the data forwarding, on ALL
HTTP responses, the server ones but also all responses generated by
HAProxy. Thanks to this ruleset, it is now possible for instance to add some
headers to the responses generated by the stats applet. Following actions are
supported :

   * allow
   * add-header
   * del-header
   * replace-header
   * replace-value
   * set-header
   * set-status
   * set-var
   * strict-mode
   * unset-var
2020-02-06 14:55:34 +01:00
William Lallemand
3af48e706c MINOR: ssl: ssl-load-extra-files configure loading of files
This new setting in the global section alters the way HAProxy will look
for unspecified files (.ocsp, .sctl, .issuer, bundles) during the
loading of the SSL certificates.

By default, HAProxy discovers automatically a lot of files not specified
in the configuration, and you may want to disable this behavior if you
want to optimize the startup time.

This patch sets flags in global_ssl.extra_files and then check them
before trying to load an extra file.
2020-02-03 17:50:26 +01:00
Jerome Magnin
882093249a DOC: word converter ignores delimiters at the start or end of input string
The comments for match_word() in pattern.c mention that delimiters
at the start or end of the input string will be ignored, but this
is not mentionned in the documentation.

Backport to all supported versions.
2020-01-28 13:44:48 +01:00
Tim Duesterhus
dd74b5f237 MINOR: lua: Add lua-prepend-path configuration option
lua-prepend-path allows the administrator to specify a custom Lua library
path to load custom Lua modules that are useful within the context of HAProxy
without polluting the global Lua library folder.
2020-01-24 09:22:03 +01:00
Willy Tarreau
71f95fa20e [RELEASE] Released version 2.2-dev1
Released version 2.2-dev1 with the following main changes :
    - DOC: this is development again
    - MINOR: version: this is development again, update the status
    - SCRIPTS: update create-release to fix the changelog on new branches
    - CLEANUP: ssl: Clean up error handling
    - BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only
    - BUG/MINOR: h1: Don't test the host header during response parsing
    - BUILD/MINOR: trace: fix use of long type in a few printf format strings
    - DOC: Clarify behavior of server maxconn in HTTP mode
    - MINOR: ssl: deduplicate ca-file
    - MINOR: ssl: compute ca-list from deduplicate ca-file
    - MINOR: ssl: deduplicate crl-file
    - CLEANUP: dns: resolution can never be null
    - BUG/MINOR: http-htx: Don't make http_find_header() fail if the value is empty
    - DOC: ssl/cli: set/commit/abort ssl cert
    - BUG/MINOR: ssl: fix SSL_CTX_set1_chain compatibility for openssl < 1.0.2
    - BUG/MINOR: fcgi-app: Make the directive pass-header case insensitive
    - BUG/MINOR: stats: Fix HTML output for the frontends heading
    - BUG/MINOR: ssl: fix X509 compatibility for openssl < 1.1.0
    - DOC: clarify matching strings on binary fetches
    - DOC: Fix ordered list in summary
    - DOC: move the "group" keyword at the right place
    - MEDIUM: init: prevent process and thread creation at runtime
    - BUG/MINOR: ssl/cli: 'ssl cert' cmd only usable w/ admin rights
    - BUG/MEDIUM: stream-int: don't subscribed for recv when we're trying to flush data
    - BUG/MINOR: stream-int: avoid calling rcv_buf() when splicing is still possible
    - BUG/MINOR: ssl/cli: don't overwrite the filters variable
    - BUG/MEDIUM: listener/thread: fix a race when pausing a listener
    - BUG/MINOR: ssl: certificate choice can be unexpected with openssl >= 1.1.1
    - BUG/MEDIUM: mux-h1: Never reuse H1 connection if a shutw is pending
    - BUG/MINOR: mux-h1: Don't rely on CO_FL_SOCK_RD_SH to set H1C_F_CS_SHUTDOWN
    - BUG/MINOR: mux-h1: Fix conditions to know whether or not we may receive data
    - BUG/MEDIUM: tasks: Make sure we switch wait queues in task_set_affinity().
    - BUG/MEDIUM: checks: Make sure we set the task affinity just before connecting.
    - MINOR: debug: replace popen() with pipe+fork() in "debug dev exec"
    - MEDIUM: init: set NO_NEW_PRIVS by default when supported
    - BUG/MINOR: mux-h1: Be sure to set CS_FL_WANT_ROOM when EOM can't be added
    - BUG/MEDIUM: mux-fcgi: Handle cases where the HTX EOM block cannot be inserted
    - BUG/MINOR: proxy: make soft_stop() also close FDs in LI_PAUSED state
    - BUG/MINOR: listener/threads: always use atomic ops to clear the FD events
    - BUG/MINOR: listener: also clear the error flag on a paused listener
    - BUG/MEDIUM: listener/threads: fix a remaining race in the listener's accept()
    - MINOR: listener: make the wait paths cleaner and more reliable
    - MINOR: listener: split dequeue_all_listener() in two
    - REORG: listener: move the global listener queue code to listener.c
    - DOC: document the listener state transitions
    - BUG/MEDIUM: kqueue: Make sure we report read events even when no data.
    - BUG/MAJOR: dns: add minimalist error processing on the Rx path
    - BUG/MEDIUM: proto_udp/threads: recv() and send() must not be exclusive.
    - DOC: listeners: add a few missing transitions
    - BUG/MINOR: tasks: only requeue a task if it was already in the queue
    - MINOR: tasks: split wake_expired_tasks() in two parts to avoid useless wakeups
    - DOC: proxies: HAProxy only supports 3 connection modes
    - DOC: remove references to the outdated architecture.txt
    - BUG/MINOR: log: fix minor resource leaks on logformat error path
    - BUG/MINOR: mworker: properly pass SIGTTOU/SIGTTIN to workers
    - BUG/MINOR: listener: do not immediately resume on transient error
    - BUG/MINOR: server: make "agent-addr" work on default-server line
    - BUG/MINOR: listener: fix off-by-one in state name check
    - BUILD/MINOR: unix sockets: silence an absurd gcc warning about strncpy()
    - MEDIUM: h1-htx: Add HTX EOM block when the message is in H1_MSG_DONE state
    - MINOR: http-htx: Add some htx sample fetches for debugging purpose
    - REGTEST: Add an HTX reg-test to check an edge case
    - DOC: clarify the fact that replace-uri works on a full URI
    - BUG/MINOR: sample: fix the closing bracket and LF in the debug converter
    - BUG/MINOR: sample: always check converters' arguments
    - MINOR: sample: Validate the number of bits for the sha2 converter
    - BUG/MEDIUM: ssl: Don't set the max early data we can receive too early.
    - MINOR: ssl/cli: 'show ssl cert' give information on the certificates
    - BUG/MINOR: ssl/cli: fix build for openssl < 1.0.2
    - MINOR: debug: support logging to various sinks
    - MINOR: http: add a new "replace-path" action
    - REGTEST: ssl: test the "set ssl cert" CLI command
    - REGTEST: run-regtests: implement #REQUIRE_BINARIES
    - MINOR: task: only check TASK_WOKEN_ANY to decide to requeue a task
    - BUG/MAJOR: task: add a new TASK_SHARED_WQ flag to fix foreing requeuing
    - BUG/MEDIUM: ssl: Revamp the way early data are handled.
    - MINOR: fd/threads: make _GET_NEXT()/_GET_PREV() use the volatile attribute
    - BUG/MEDIUM: fd/threads: fix a concurrency issue between add and rm on the same fd
    - REGTEST: make the "set ssl cert" require version 2.1
    - BUG/MINOR: ssl: openssl-compat: Fix getm_ defines
    - BUG/MEDIUM: state-file: do not allocate a full buffer for each server entry
    - BUG/MINOR: state-file: do not store duplicates in the global tree
    - BUG/MINOR: state-file: do not leak memory on parse errors
    - BUG/MAJOR: mux-h1: Don't pretend the input channel's buffer is full if empty
    - BUG/MEDIUM: stream: Be sure to never assign a TCP backend to an HTX stream
    - BUILD: ssl: improve SSL_CTX_set_ecdh_auto compatibility
    - BUILD: travis-ci: link with ssl libraries using rpath instead of LD_LIBRARY_PATH/DYLD_LIBRARY_PATH
    - BUILD: travis-ci: reenable address sanitizer for clang builds
    - BUG/MINOR: checks: refine which errno values are really errors.
    - BUG/MINOR: connection: only wake send/recv callbacks if the FD is active
    - CLEANUP: connection: conn->xprt is never NULL
    - MINOR: pollers: add a new flag to indicate pollers reporting ERR & HUP
    - MEDIUM: tcp: make tcp_connect_probe() consider ERR/HUP
    - REORG: connection: move tcp_connect_probe() to conn_fd_check()
    - MINOR: connection: check for connection validation earlier
    - MINOR: connection: remove the double test on xprt_done_cb()
    - CLEANUP: connection: merge CO_FL_NOTIFY_DATA and CO_FL_NOTIFY_DONE
    - MINOR: poller: do not call the IO handler if the FD is not active
    - OPTIM: epoll: always poll for recv if neither active nor ready
    - OPTIM: polling: do not create update entries for FD removal
    - BUG/MEDIUM: checks: Only attempt to do handshakes if the connection is ready.
    - BUG/MEDIUM: connections: Hold the lock when wanting to kill a connection.
    - BUILD: CI: modernize cirrus-ci
    - MINOR: config: disable busy polling on old processes
    - MINOR: ssl: Remove unused variable "need_out".
    - BUG/MINOR: h1: Report the right error position when a header value is invalid
    - BUG/MINOR: proxy: Fix input data copy when an error is captured
    - BUG/MEDIUM: http-ana: Truncate the response when a redirect rule is applied
    - BUG/MINOR: channel: inject output data at the end of output
    - BUG/MEDIUM: session: do not report a failure when rejecting a session
    - MEDIUM: dns: implement synchronous send
    - MINOR: raw_sock: make sure to disable polling once everything is sent
    - MINOR: http: Add 410 to http-request deny
    - MINOR: http: Add 404 to http-request deny
    - CLEANUP: mux-h2: remove unused goto "out_free_h2s"
    - BUILD: cirrus-ci: choose proper openssl package name
    - BUG/MAJOR: listener: do not schedule a task-less proxy
    - CLEANUP: server: remove unused err section in server_finalize_init
    - REGTEST: set_ssl_cert.vtc: replace "echo" with "printf"
    - BUG/MINOR: stream-int: Don't trigger L7 retry if max retries is already reached
    - BUG/MEDIUM: tasks: Use the MT macros in tasklet_free().
    - BUG/MINOR: mux-h2: use a safe list_for_each_entry in h2_send()
    - BUG/MEDIUM: mux-h2: fix missing test on sending_list in previous patch
    - CLEANUP: ssl: remove opendir call in ssl_sock_load_cert
    - MEDIUM: lua: don't call the GC as often when dealing with outgoing connections
    - BUG/MEDIUM: mux-h2: don't stop sending when crossing a buffer boundary
    - BUG/MINOR: cli/mworker: can't start haproxy with 2 programs
    - REGTEST: mcli/mcli_start_progs: start 2 programs
    - BUG/MEDIUM: mworker: remain in mworker mode during reload
    - DOC: clarify crt-base usage
    - CLEANUP: compression: remove unused deinit_comp_ctx section
    - BUG/MEDIUM: mux_h1: Don't call h1_send if we subscribed().
    - BUG/MEDIUM: raw_sock: Make sur the fd and conn are sync.
    - CLEANUP: proxy: simplify proxy_parse_rate_limit proxy checks
    - BUG/MAJOR: hashes: fix the signedness of the hash inputs
    - REGTEST: add sample_fetches/hashes.vtc to validate hashes
    - BUG/MEDIUM: cli: _getsocks must send the peers sockets
    - CLEANUP: cli: deduplicate the code in _getsocks
    - BUG/MINOR: stream: don't mistake match rules for store-request rules
    - BUG/MEDIUM: connection: add a mux flag to indicate splice usability
    - BUG/MINOR: pattern: handle errors from fgets when trying to load patterns
    - MINOR: connection: move the CO_FL_WAIT_ROOM cleanup to the reader only
    - MINOR: stream-int: remove dependency on CO_FL_WAIT_ROOM for rcv_buf()
    - MEDIUM: connection: get rid of CO_FL_CURR_* flags
    - BUILD: pattern: include errno.h
    - MEDIUM: mux-h2: do not try to stop sending streams on blocked mux
    - MEDIUM: mux-fcgi: do not try to stop sending streams on blocked mux
    - MEDIUM: mux-h2: do not make an h2s subscribe to itself on deferred shut
    - MEDIUM: mux-fcgi: do not make an fstrm subscribe to itself on deferred shut
    - REORG: stream/backend: move backend-specific stuff to backend.c
    - MEDIUM: backend: move the connection finalization step to back_handle_st_con()
    - MEDIUM: connection: merge the send_wait and recv_wait entries
    - MEDIUM: xprt: merge recv_wait and send_wait in xprt_handshake
    - MEDIUM: ssl: merge recv_wait and send_wait in ssl_sock
    - MEDIUM: mux-h1: merge recv_wait and send_wait
    - MEDIUM: mux-h2: merge recv_wait and send_wait event notifications
    - MEDIUM: mux-fcgi: merge recv_wait and send_wait event notifications
    - MINOR: connection: make the last arg of subscribe() a struct wait_event*
    - MINOR: ssl: Add support for returning the dn samples from ssl_(c|f)_(i|s)_dn in LDAP v3 (RFC2253) format.
    - DOC: Fix copy and paste mistake in http-response replace-value doc
    - BUG/MINOR: cache: Fix leak of cache name in error path
    - BUG/MINOR: dns: Make dns_query_id_seed unsigned
    - BUG/MINOR: 51d: Fix bug when HTX is enabled
    - MINOR: http-htx: Move htx sample fetches in the scope "internal"
    - MINOR: http-htx: Rename 'internal.htx_blk.val' to 'internal.htx_blk.data'
    - MINOR: http-htx: Make 'internal.htx_blk_data' return a binary string
    - DOC: Add a section to document the internal sample fetches
    - MINOR: mux-h1: Inherit send flags from the upper layer
    - MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics
    - BUG/MINOR: http-ana/filters: Wait end of the http_end callback for all filters
    - BUG/MINOR: http-rules: Remove buggy deinit functions for HTTP rules
    - BUG/MINOR: stick-table: Use MAX_SESS_STKCTR as the max track ID during parsing
    - MEDIUM: http-rules: Register an action keyword for all http rules
    - MINOR: tcp-rules: Always set from which ruleset a rule comes from
    - MINOR: actions: Use ACT_RET_CONT code to ignore an error from a custom action
    - MINOR: tcp-rules: Kill connections when custom actions return ACT_RET_ERR
    - MINOR: http-rules: Return an error when custom actions return ACT_RET_ERR
    - MINOR: counters: Add a counter to report internal processing errors
    - MEDIUM: http-ana: Properly handle internal processing errors
    - MINOR: http-rules: Add a rule result to report internal error
    - MINOR: http-rules: Handle internal errors during HTTP rules evaluation
    - MINOR: http-rules: Add more return codes to let custom actions act as normal ones
    - MINOR: tcp-rules: Handle denied/aborted/invalid connections from TCP rules
    - MINOR: http-rules: Handle denied/aborted/invalid connections from HTTP rules
    - MINOR: stats: Report internal errors in the proxies/listeners/servers stats
    - MINOR: contrib/prometheus-exporter: Export internal errors per proxy/server
    - MINOR: counters: Remove failed_secu counter and use denied_resp instead
    - MINOR: counters: Review conditions to increment counters from analysers
    - MINOR: http-ana: Add a txn flag to support soft/strict message rewrites
    - MINOR: http-rules: Handle all message rewrites the same way
    - MINOR: http-rules: Add a rule to enable or disable the strict rewriting mode
    - MEDIUM: http-rules: Enable the strict rewriting mode by default
    - REGTEST: Fix format of set-uri HTTP request rule in h1or2_to_h1c.vtc
    - MINOR: actions: Add a function pointer to release args used by actions
    - MINOR: actions: Regroup some info about HTTP rules in the same struct
    - MINOR: http-rules/tcp-rules: Call the defined action function first if defined
    - MINOR: actions: Rename the act_flag enum into act_opt
    - MINOR: actions: Add flags to configure the action behaviour
    - MINOR: actions: Use an integer to set the action type
    - MINOR: http-rules: Use a specific action type for some custom HTTP actions
    - MINOR: http-rules: Make replace-header and replace-value custom actions
    - MINOR: http-rules: Make set-header and add-header custom actions
    - MINOR: http-rules: Make set/del-map and add/del-acl custom actions
    - MINOR: http-rules: Group all processing of early-hint rule in its case clause
    - MEDIUM: http-rules: Make early-hint custom actions
    - MINOR: http-rule/tcp-rules: Make track-sc* custom actions
    - MINOR: tcp-rules: Make tcp-request capture a custom action
    - MINOR: http-rules: Add release functions for existing HTTP actions
    - BUG/MINOR: http-rules: Fix memory releases on error path during action parsing
    - MINOR: tcp-rules: Add release functions for existing TCP actions
    - BUG/MINOR: tcp-rules: Fix memory releases on error path during action parsing
    - MINOR: http-htx: Add functions to read a raw error file and convert it in HTX
    - MINOR: http-htx: Add functions to create HTX redirect message
    - MINOR: config: Use dedicated function to parse proxy's errorfiles
    - MINOR: config: Use dedicated function to parse proxy's errorloc
    - MEDIUM: http-htx/proxy: Use a global and centralized storage for HTTP error messages
    - MINOR: proxy: Register keywords to parse errorfile and errorloc directives
    - MINOR: http-htx: Add a new section to create groups of custom HTTP errors
    - MEDIUM: proxy: Add a directive to reference an http-errors section in a proxy
    - MINOR: http-rules: Update txn flags and status when a deny rule is executed
    - MINOR: http-rules: Support an optional status on deny rules for http reponses
    - MINOR: http-rules: Use same function to parse request and response deny actions
    - MINOR: http-ana: Add an error message in the txn and send it when defined
    - MEDIUM: http-rules: Support an optional error message in http deny rules
    - REGTEST: Add a strict rewriting mode reg test
    - REGEST: Add reg tests about error files
    - MINOR: ssl: accept 'verify' bind option with 'set ssl cert'
    - BUG/MINOR: ssl: ssl_sock_load_ocsp_response_from_file memory leak
    - BUG/MINOR: ssl: ssl_sock_load_issuer_file_into_ckch memory leak
    - BUG/MINOR: ssl: ssl_sock_load_sctl_from_file memory leak
    - BUG/MINOR: http_htx: Fix some leaks on error path when error files are loaded
    - CLEANUP: http-ana: Remove useless test on txn when the error message is retrieved
    - BUILD: CI: introduce ARM64 builds
    - BUILD: ssl: more elegant anti-replay feature presence check
    - MINOR: proxy/http-ana: Add support of extra attributes for the cookie directive
    - MEDIUM: dns: use Additional records from SRV responses
    - CLEANUP: Consistently `unsigned int` for bitfields
    - CLEANUP: pattern: remove the pat_time definition
    - BUG/MINOR: http_act: don't check capture id in backend
    - BUG/MINOR: ssl: fix build on development versions of openssl-1.1.x
2020-01-22 10:34:58 +01:00
Baptiste Assmann
19a69b3740 BUG/MINOR: http_act: don't check capture id in backend
A wrong behavior was introduced by
e9544935e86278dfa3d49fb4b97b860774730625, leading to preventing loading
any configuration where a capture slot id is used in a backend.
IE, the configuration below does not parse:

  frontend f
   bind *:80
   declare capture request len 32
   default_backend webserver

  backend webserver
   http-request capture req.hdr(Host) id 1

The point is that such type of configuration is valid and should run.

This patch enforces the check of capture slot id only if the action rule
is configured in a frontend.
The point is that at configuration parsing time, it is impossible to
check which frontend could point to this backend (furthermore if we use
dynamic backend name resolution at runtime).

The documentation has been updated to warn the user to ensure that
relevant frontends have required declaration when such rule has to be
used in a backend.
If no capture slot can be found, then the action will just not be
executed and HAProxy will process the next one in the list, as expected.

This should be backported to all supported branches (bug created as part
of a bug fix introduced into 1.7 and backported to 1.6).
2020-01-22 07:44:36 +01:00
Christopher Faulet
2f5339079b MINOR: proxy/http-ana: Add support of extra attributes for the cookie directive
It is now possible to insert any attribute when a cookie is inserted by
HAProxy. Any value may be set, no check is performed except the syntax validity
(CTRL chars and ';' are forbidden). For instance, it may be used to add the
SameSite attribute:

    cookie SRV insert attr "SameSite=Strict"

The attr option may be repeated to add several attributes.

This patch should fix the issue #361.
2020-01-22 07:18:31 +01:00
Christopher Faulet
554c0ebffd MEDIUM: http-rules: Support an optional error message in http deny rules
It is now possible to set the error message to use when a deny rule is
executed. It may be a specific error file, adding "errorfile <file>" :

  http-request deny deny_status 400 errorfile /etc/haproxy/errorfiles/400badreq.http

It may also be an error file from an http-errors section, adding "errorfiles
<name>" :

  http-request deny errorfiles my-errors  # use 403 error from "my-errors" section

When defined, this error message is set in the HTTP transaction. The tarpit rule
is also concerned by this change.
2020-01-20 15:18:46 +01:00
Christopher Faulet
040c8cdbbe MINOR: http-rules: Support an optional status on deny rules for http reponses
It is now possible to specified the status code to return an http-response deny
rules. For instance :

    http-response deny deny_status 500
2020-01-20 15:18:46 +01:00
Christopher Faulet
76edc0f29c MEDIUM: proxy: Add a directive to reference an http-errors section in a proxy
It is now possible to import in a proxy, fully or partially, error files
declared in an http-errors section. It may be done using the "errorfiles"
directive, followed by a name and optionally a list of status code. If there is
no status code specified, all error files of the http-errors section are
imported. Otherwise, only error files associated to the listed status code are
imported. For instance :

  http-errors my-errors
      errorfile 400 ...
      errorfile 403 ...
      errorfile 404 ...

  frontend frt
      errorfiles my-errors 403 404  # ==> error 400 not imported
2020-01-20 15:18:46 +01:00
Christopher Faulet
1aea50e1ff MEDIUM: http-rules: Enable the strict rewriting mode by default
Now, by default, when a rule performing a rewrite on an HTTP message fails, an
internal error is triggered. Before, the failure was ignored. But most of users
are not aware of this behavior. And it does not happen very often because the
buffer reserve space in large enough. So it may be surprising. Returning an
internal error makes the rewrite failure explicit. If it is acceptable to
silently ignore it, the strict rewriting mode can be disabled.
2020-01-20 15:18:45 +01:00
Christopher Faulet
46f95543c5 MINOR: http-rules: Add a rule to enable or disable the strict rewriting mode
It is now possible to explicitly instruct rewriting rules to be strict or not
towards errors. It means that in this mode, an internal error is trigger if a
rewrite rule fails. The HTTP action "strict-mode" can be used to enable or
disable the strict rewriting mode. It can be used in an http-request and an
http-response ruleset.

For now, by default the strict rewriting mode is disabled. Because it is the
current behavior. But it will be changed in another patch.
2020-01-20 15:18:45 +01:00
Christopher Faulet
d47941d6ac DOC: Add a section to document the internal sample fetches
The section 7.3.7. is now dedicated to internal sample fetches. For now, only
HTX sample fetches are referenced in this section. But it should contain the
documentation of all sample fetches reserved to an internal use, for debugging
or testing purposes.
2020-01-20 15:18:45 +01:00
Tim Duesterhus
6bd909b42f DOC: Fix copy and paste mistake in http-response replace-value doc
This fixes up commit 2252beb8557d73407b8f96eef91d6927fb855685.
2020-01-18 06:45:54 +01:00
Elliot Otchet
71f829767d MINOR: ssl: Add support for returning the dn samples from ssl_(c|f)_(i|s)_dn in LDAP v3 (RFC2253) format.
Modifies the existing sample extraction methods (smp_fetch_ssl_x_i_dn,
smp_fetch_ssl_x_s_dn) to accommodate a third argument that indicates the
DN should be returned in LDAP v3 format. When the third argument is
present, the new function (ssl_sock_get_dn_formatted) is called with
three parameters including the X509_NAME, a buffer containing the format
argument, and a buffer for the output.  If the supplied format matches
the supported format string (currently only "rfc2253" is supported), the
formatted value is extracted into the supplied output buffer using
OpenSSL's X509_NAME_print_ex and BIO_s_mem. 1 is returned when a dn
value is retrieved.  0 is returned when a value is not retrieved.

Argument validation is added to each of the related sample
configurations to ensure the third argument passed is either blank or
"rfc2253" using strcmp.  An error is returned if the third argument is
present with any other value.

Documentation was updated in configuration.txt and it was noted during
preliminary reviews that a CLEANUP patch should follow that adjusts the
documentation.  Currently, this patch and the existing documentation are
copied with some minor revisions for each sample configuration.  It
might be better to have one entry for all of the samples or entries for
each that reference back to a primary entry that explains the sample in
detail.

Special thanks to Chris, Willy, Tim and Aleks for the feedback.

Author: Elliot Otchet <degroens@yahoo.com>
Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
2020-01-18 06:42:30 +01:00
William Dauchy
238ea3bd75 DOC: clarify crt-base usage
crt-base is also used after "crt" directive.

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-01-15 10:55:43 +01:00
Florian Tham
9205fea13a MINOR: http: Add 404 to http-request deny
This patch adds http status code 404 Not Found to http-request deny. See
issue #80.
2020-01-08 16:15:23 +01:00
Florian Tham
272e29b5cc MINOR: http: Add 410 to http-request deny
This patch adds http status code 410 Gone to http-request deny. See
issue #80.
2020-01-08 16:15:23 +01:00
William Dauchy
3894d97fb8 MINOR: config: disable busy polling on old processes
in the context of seamless reload and busy polling, older processes will
create unecessary cpu conflicts; we can assume there is no need for busy
polling for old processes which are waiting to be terminated.

This patch is not a bug fix itself but might be a good stability
improvment when you are un the context of frequent seamless reloads with
a high "hard-stop-after" value; for that reasons I think this patch
should be backported in all 2.x versions.

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-01-02 10:29:49 +01:00
Willy Tarreau
262c3f1a00 MINOR: http: add a new "replace-path" action
This action is very similar to "replace-uri" except that it only acts on the
path component. This is assumed to better match users' expectations when they
used to rely on "replace-uri" in HTTP/1 because mostly origin forms were used
in H1 while mostly absolute URI form is used in H2, and their rules very often
start with a '/', and as such do not match.

It could help users to get this backported to 2.0 and 2.1.
2019-12-19 09:24:57 +01:00
Willy Tarreau
0851fd5eef MINOR: debug: support logging to various sinks
As discussed in the thread below [1], the debug converter is currently
not of much use given that it's only built when DEBUG_EXPR is set, and
it is limited to stderr only.

This patch changes this to make it take an optional prefix and an optional
target sink so that it can log to stdout, stderr or a ring buffer. The
default output is the "buf0" ring buffer, that can be consulted from the
CLI.

[1] https://www.mail-archive.com/haproxy@formilux.org/msg35671.html

Note: if this patch is backported, it also requires the following commit to
work: 46dfd78cbf ("BUG/MINOR: sample: always check converters' arguments").
2019-12-19 09:19:13 +01:00
Willy Tarreau
62b5913380 DOC: clarify the fact that replace-uri works on a full URI
With H2 deployments becoming more common, replace-uri starts to hit
users by not always matching absolute URIs due to rules expecting the
URI to start with a '/'.
2019-12-17 06:55:15 +01:00
Julien Pivotto
21ad315316 DOC: proxies: HAProxy only supports 3 connection modes
The 4th one (forceclose) has been deprecated and deleted from the
documentation in 10c6c16cde0b0b473a1ab16e958a7d6b61ed36fc

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-12-11 10:17:23 +01:00
Willy Tarreau
a45a8b5171 MEDIUM: init: set NO_NEW_PRIVS by default when supported
HAProxy doesn't need to call executables at run time (except when using
external checks which are strongly recommended against), and is even expected
to isolate itself into an empty chroot. As such, there basically is no valid
reason to allow a setuid executable to be called without the user being fully
aware of the risks. In a situation where haproxy would need to call external
checks and/or disable chroot, exploiting a vulnerability in a library or in
haproxy itself could lead to the execution of an external program. On Linux
it is possible to lock the process so that any setuid bit present on such an
executable is ignored. This significantly reduces the risk of privilege
escalation in such a situation. This is what haproxy does by default. In case
this causes a problem to an external check (for example one which would need
the "ping" command), then it is possible to disable this protection by
explicitly adding this directive in the global section. If enabled, it is
possible to turn it back off by prefixing it with the "no" keyword.

Before the option:

  $ socat - /tmp/sock1 <<< "expert-mode on; debug dev exec sudo /bin/id"
  uid=0(root) gid=0(root) groups=0(root

After the option:
  $ socat - /tmp/sock1 <<< "expert-mode on; debug dev exec sudo /bin/id"
  sudo: effective uid is not 0, is /usr/bin/sudo on a file system with the
        'nosuid' option set or an NFS file system without root privileges?
2019-12-06 17:20:26 +01:00
Willy Tarreau
d96f1126fe MEDIUM: init: prevent process and thread creation at runtime
Some concerns are regularly raised about the risk to inherit some Lua
files which make use of a fork (e.g. via os.execute()) as well as
whether or not some of bugs we fix might or not be exploitable to run
some code. Given that haproxy is event-driven, any foreground activity
completely stops processing and is easy to detect, but background
activity is a different story. A Lua script could very well discretely
fork a sub-process connecting to a remote location and taking commands,
and some injected code could also try to hide its activity by creating
a process or a thread without blocking the rest of the processing. While
such activities should be extremely limited when run in an empty chroot
without any permission, it would be better to get a higher assurance
they cannot happen.

This patch introduces something very simple: it limits the number of
processes and threads to zero in the workers after the last thread was
created. By doing so, it effectively instructs the system to fail on
any fork() or clone() syscall. Thus any undesired activity has to happen
in the foreground and is way easier to detect.

This will obviously break external checks (whose concept is already
totally insecure), and for this reason a new option
"insecure-fork-wanted" was added to disable this protection, and it
is suggested in the fork() error report from the checks. It is
obviously recommended not to use it and to reconsider the reasons
leading to it being enabled in the first place.

If for any reason we fail to disable forks, we still start because it
could be imaginable that some operating systems refuse to set this
limit to zero, but in this case we emit a warning, that may or may not
be reported since we're after the fork point. Ideally over the long
term it should be conditionned by strict-limits and cause a hard fail.
2019-12-03 11:49:00 +01:00
Willy Tarreau
11770ce64b DOC: move the "group" keyword at the right place
It looks like "hard-stop-after", "h1-case-adjust" and "h1-case-adjust-file"
were added before "group", breaking alphabetical ordering.
2019-12-03 08:42:13 +01:00
Julien Pivotto
6ccee41ae8 DOC: Fix ordered list in summary
Section 6 about the cache was placed between 7 and 8. This should
be backported to 2.1.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-12-03 07:37:51 +01:00
Mathias Weiersmueller
cb250fc984 DOC: clarify matching strings on binary fetches
Add clarification and example to string matching on binary samples,
as comparison stops at first null byte due to strncmp behaviour.

Backporting all the way down to 1.5 is suggested as it might save
from headaches.
2019-12-03 07:34:56 +01:00
Tim Duesterhus
cefbbd9811 DOC: Clarify behavior of server maxconn in HTTP mode
In HTTP mode the number of concurrent requests is limited, not the
number of actual connections.
2019-11-28 05:16:45 +01:00
Willy Tarreau
2e077f8d53 [RELEASE] Released version 2.2-dev0
Released version 2.2-dev0 with the following main changes :
    - exact copy of 2.1.0
2019-11-25 20:36:16 +01:00
Willy Tarreau
e54b43af1e [RELEASE] Released version 2.1.0
Released version 2.1.0 with the following main changes :
    - BUG/MINOR: init: fix set-dumpable when using uid/gid
    - MINOR: init: avoid code duplication while setting identify
    - BUG/MINOR: ssl: ssl_pkey_info_index ex_data can store a dereferenced pointer
    - BUG/MINOR: ssl: fix crt-list neg filter for openssl < 1.1.1
    - MINOR: peers: Alway show the table info for disconnected peers.
    - MINOR: peers: Add TX/RX heartbeat counters.
    - MINOR: peers: Add debugging information to "show peers".
    - BUG/MINOR: peers: Wrong null "server_name" data field handling.
    - MINOR: ssl/cli: 'abort ssl cert' deletes an on-going transaction
    - BUG/MEDIUM: mworker: don't fill the -sf argument with -1 during the reexec
    - BUG/MINOR: peers: "peer alive" flag not reset when deconnecting.
    - BUILD/MINOR: ssl: fix compiler warning about useless statement
    - BUG/MEDIUM: stream-int: Don't loose events on the CS when an EOS is reported
    - MINOR: contrib/prometheus-exporter: filter exported metrics by scope
    - MINOR: contrib/prometheus-exporter: Add a param to ignore servers in maintenance
    - BUILD: debug: Avoid warnings in dev mode with -02 because of some BUG_ON tests
    - BUG/MINOR: mux-h1: Fix tunnel mode detection on the response path
    - BUG/MINOR: http-ana: Properly catch aborts during the payload forwarding
    - DOC: Update http-buffer-request description to remove the part about chunks
    - BUG/MINOR: stream-int: Fix si_cs_recv() return value
    - DOC: internal: document the init calls
    - MEDIUM: dns: Add resolve-opts "ignore-weight"
    - MINOR: ssl: ssl_sock_prepare_ctx() return an error code
    - MEDIUM: ssl/cli: apply SSL configuration on SSL_CTX during commit
    - MINOR: ssl/cli: display warning during 'commit ssl cert'
    - MINOR: version: report the version status in "haproxy -v"
    - MINOR: version: emit the link to the known bugs in output of "haproxy -v"
    - DOC: Add documentation about the use-service action
    - MINOR: ssl: fix possible null dereference in error handling
    - BUG/MINOR: ssl: fix curve setup with LibreSSL
    - BUG/MINOR: ssl: Stop passing dynamic strings as format arguments
    - CLEANUP: ssl: check if a transaction exists once before setting it
    - BUG/MINOR: cli: fix out of bounds in -S parser
    - MINOR: ist: add ist_find_ctl()
    - BUG/MAJOR: h2: reject header values containing invalid chars
    - BUG/MAJOR: h2: make header field name filtering stronger
    - BUG/MAJOR: mux-h2: don't try to decode a response HEADERS frame in idle state
    - MINOR: h2: add a function to report H2 error codes as strings
    - MINOR: mux-h2/trace: report the connection and/or stream error code
    - SCRIPTS: create-release: show the correct origin name in suggested commands
    - SCRIPTS: git-show-backports: add "-s" to proposed cherry-pick commands
    - BUG/MEDIUM: trace: fix a typo causing an incorrect startup error
    - BUILD: reorder the objects in the makefile
    - DOC: mention in INSTALL haproxy 2.1 is a stable stable version
    - MINOR: version: indicate that this version is stable
2019-11-25 19:47:40 +01:00
Christopher Faulet
579d83b05f DOC: Add documentation about the use-service action
The use-service action may be used in tcp-request and http-request rules. It was
added to customize HAproxy reply to a client using an applet (initially a lua
applet). But the documentation was missing.

This patch may be backported as far as 1.6.
2019-11-22 15:34:57 +01:00
Daniel Corbett
f8716914c7 MEDIUM: dns: Add resolve-opts "ignore-weight"
It was noted in #48 that there are times when a configuration
may use the server-template directive with SRV records and
simultaneously want to control weights using an agent-check or
through the runtime api.  This patch adds a new option
"ignore-weight" to the "resolve-opts" directive.

When specified, any weight indicated within an SRV record will
be ignored.  This is for both initial resolution and ongoing
resolution.
2019-11-21 17:25:31 +01:00
Christopher Faulet
6db8a2e021 DOC: Update http-buffer-request description to remove the part about chunks
The limitation on the first chunk for chunked requests was true for the legacy
HTTP mode. But, it does not exist with the HTX. Becaue, the legacy HTTP mode was
removed in 2.1, this limitation does not exist anymore.
2019-11-20 14:17:36 +01:00
Willy Tarreau
846813260d [RELEASE] Released version 2.1-dev5
Released version 2.1-dev5 with the following main changes :
    - BUG/MEDIUM: ssl/cli: don't alloc path when cert not found
    - BUG/MINOR: ssl/cli: unable to update a certificate without bundle extension
    - BUG/MINOR: ssl/cli: fix an error when a file is not found
    - MINOR: ssl/cli: replace the default_ctx during 'commit ssl cert'
    - DOC: fix date and http_date keywords syntax
    - MINOR: peers: Add "log" directive to "peers" section.
    - BUG/MEDIUM: mux-h1: Disable splicing for chunked messages
    - BUG/MEDIUM: stream: Be sure to support splicing at the mux level to enable it
    - MINOR: flt_trace: Rename macros to print trace messages
    - MINOR: trace: Add a set of macros to trace events if HA is compiled with debug
    - MEDIUM: stream/trace: Register a new trace source with its events
    - MINOR: doc: http-reuse connection pool fix
    - BUG/MEDIUM: stream: Be sure to release allocated captures for TCP streams
    - MINOR: http-ana: Remove the unused function http_reset_txn()
    - BUG/MINOR: action: do-resolve now use cached response
    - BUG: dns: timeout resolve not applied for valid resolutions
    - DOC: management: fix typo on "cache_lookups" stats output
    - BUG/MINOR: stream: init variables when the list is empty
    - BUG/MEDIUM: tasks: Make tasklet_remove_from_tasklet_list() no matter the tasklet.
    - BUG/MINOR: queue/threads: make the queue unlinking atomic
    - BUG/MEDIUM: Make sure we leave the session list in session_free().
    - CLEANUP: session: slightly simplify idle connection cleanup logic
    - MINOR: memory: also poison the area on freeing
    - CLEANUP: cli: use srv_shutdown_streams() instead of open-coding it
    - CLEANUP: stats: use srv_shutdown_streams() instead of open-coding it
    - BUG/MEDIUM: listeners: always pause a listener on out-of-resource condition
    - BUILD: contrib/da: remove an "unused" warning
    - BUG/MEDIUM: filters: Don't call TCP callbacks for HTX streams
    - MEDIUM: filters: Adapt filters API to allow again TCP filtering on HTX streams
    - MINOR: freq_ctr: Make the sliding window sums thread-safe
    - MINOR: stream: Remove the lock on the proxy to update time stats
    - MINOR: counters: Add fields to store the max observed for {q,c,d,t}_time
    - MINOR: stats: Report max times in addition of the averages for sessions
    - MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions
    - BUG/MINOR: contrib/prometheus-exporter: Rename some metrics
    - MINOR: contrib/prometheus-exporter: report the number of idle conns per server
    - DOC: Add missing stats fields in the management manual
    - BUG/MINOR: mux-h1: Properly catch parsing errors on payload and trailers
    - BUG/MINOR: mux-h1: Don't set CS_FL_EOS on a read0 when receiving data to pipe
    - MINOR: mux-h1: Set EOI on the conn-stream when EOS is reported in TUNNEL state
    - MINOR: sink: Set the default max length for a message to BUFSIZE
    - MINOR: ring: make the parse function automatically set the handler/release
    - BUG/MINOR: log: make "show startup-log" use a ring buffer instead
    - MINOR: stick-table: allow sc-set-gpt0 to set value from an expression
2019-11-15 18:49:37 +01:00
Cédric Dufour
0d7712dff0 MINOR: stick-table: allow sc-set-gpt0 to set value from an expression
Allow the sc-set-gpt0 action to set GPT0 to a value dynamically evaluated from
its <expr> argument (in addition to the existing static <int> alternative).
2019-11-15 18:24:19 +01:00
Lukas Tribus
e8adfeb84b MINOR: doc: http-reuse connection pool fix
Since 1.9 we actually do use a connection pool, configurable with
pool-max-conn.

Update the documentation in this regard.

Must be backported to 1.9.
2019-11-06 11:52:07 +01:00
Frédéric Lécaille
b6f759b43d MINOR: peers: Add "log" directive to "peers" section.
This patch is easy to review: let's call parse_logsrv() function to parse
"log" directive as this is already for other sections for proxies.
This enable us to log incoming TCP connections for the listeners for "peers"
sections.

Update the documentation for "peers" section.
2019-11-06 04:49:56 +01:00
Cyril Bonté
6bcd1826c5 DOC: fix date and http_date keywords syntax
These keywords received a second argument with commit ae6f125 ("MINOR:
sample: add us/ms support to date/http_date"). Each argument is optional,
it's not either both or none.
2019-11-06 04:45:40 +01:00
Willy Tarreau
1753cb544d [RELEASE] Released version 2.1-dev4
Released version 2.1-dev4 with the following main changes :
    - BUG/MINOR: cli: don't call the kw->io_release if kw->parse failed
    - BUG/MINOR: mux-h2: Don't pretend mux buffers aren't full anymore if nothing sent
    - BUG/MAJOR: stream-int: Don't receive data from mux until SI_ST_EST is reached
    - DOC: remove obsolete section about header manipulation
    - BUG/MINOR: ssl/cli: cleanup on cli_parse_set_cert error
    - MINOR: ssl/cli: rework the 'set ssl cert' IO handler
    - BUILD: CI: comment out cygwin build, upgrade various ssl libraries
    - DOC: Improve documentation of http-re(quest|sponse) replace-(header|value|uri)
    - BUILD/MINOR: tools: shut up the format truncation warning in get_gmt_offset()
    - BUG/MINOR: spoe: fix off-by-one length in UUID format string
    - BUILD/MINOR: ssl: shut up a build warning about format truncation
    - BUILD: do not disable -Wformat-truncation anymore
    - MINOR: chunk: add chunk_istcat() to concatenate an ist after a chunk
    - Revert "MINOR: istbuf: add b_fromist() to make a buffer from an ist"
    - MINOR: mux: Add a new method to get informations about a mux.
    - BUG/MEDIUM: stream_interface: Only use SI_ST_RDY when the mux is ready.
    - BUG/MEDIUM: servers: Only set SF_SRV_REUSED if the connection if fully ready.
    - MINOR: doc: fix busy-polling performance reference
    - MINOR: config: allow no set-dumpable config option
    - MINOR: init: always fail when setrlimit fails
    - MINOR: ssl/cli: rework 'set ssl cert' as 'set/commit'
    - CLEANUP: ssl/cli: remove leftovers of bundle/certs (it < 2)
    - REGTEST: vtest can now enable mcli with its own flag
    - BUG/MINOR: config: Update cookie domain warn to RFC6265
    - MINOR: sample: add us/ms support to date/http_date
    - BUG/MINOR: ssl/cli: check trash allocation in cli_io_handler_commit_cert()
    - BUG/MEDIUM: mux-h2: report no available stream on a connection having errors
    - BUG/MEDIUM: mux-h2: immediately remove a failed connection from the idle list
    - BUG/MEDIUM: mux-h2: immediately report connection errors on streams
    - BUG/MINOR: stats: properly check the path and not the whole URI
    - BUG/MINOR: ssl: segfault in cli_parse_set_cert with old openssl/boringssl
    - BUG/MINOR: ssl: ckch->chain must be initialized
    - BUG/MINOR: ssl: double free on error for ckch->{key,cert}
    - MINOR: ssl: BoringSSL ocsp_response does not need issuer
    - BUG/MEDIUM: ssl/cli: fix dot research in cli_parse_set_cert
    - MINOR: backend: Add srv_name sample fetche
    - DOC: Add GitHub issue config.yml
2019-11-03 15:43:10 +01:00