1727 Commits

Author SHA1 Message Date
Lukas Tribus
926594f606 MINOR: ssl: set SSL_OP_PRIORITIZE_CHACHA
Sets OpenSSL 1.1.1's SSL_OP_PRIORITIZE_CHACHA unconditionally, as per [1]:

When SSL_OP_CIPHER_SERVER_PREFERENCE is set, temporarily reprioritize
ChaCha20-Poly1305 ciphers to the top of the server cipher list if a
ChaCha20-Poly1305 cipher is at the top of the client cipher list. This
helps those clients (e.g. mobile) use ChaCha20-Poly1305 if that cipher
is anywhere in the server cipher list; but still allows other clients to
use AES and other ciphers. Requires SSL_OP_CIPHER_SERVER_PREFERENCE.

[1] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_clear_options.html
2018-05-23 16:55:15 +02:00
William Lallemand
8a16fe0d05 BUG/MEDIUM: cache: don't cache when an Authorization header is present
RFC 7234 says:

A cache MUST NOT store a response to any request, unless:
[...] the Authorization header field (see Section 4.2 of [RFC7235]) does
      not appear in the request, if the cache is shared, unless the
      response explicitly allows it (see Section 3.2), [...]

In this patch we completely disable the cache upon the receipt of an
Authorization header in the request. In this case it's not possible to
either use the cache or store into the cache anymore.

Thanks to Adam Eijdenberg of Digital Transformation Agency for raising
this issue.

This patch must be backported to 1.8.
2018-05-23 10:36:44 +02:00
Willy Tarreau
760e81d356 MINOR: backend: implement random-based load balancing
For large farms where servers are regularly added or removed, picking
a random server from the pool can ensure faster load transitions than
when using round-robin and less traffic surges on the newly added
servers than when using leastconn.

This commit introduces "balance random". It internally uses a random as
the key to the consistent hashing mechanism, thus all features available
in consistent hashing such as weights and bounded load via hash-balance-
factor are usable. It is extremely convenient because one common concern
when using random is what happens when a server is hammered a bit too
much. Here that can trivially be avoided, like in the configuration below :

    backend bk0
        balance random
        hash-balance-factor 110
        server-template s 1-100 127.0.0.1:8000 check inter 1s

Note that while "balance random" internally relies on a hash algorithm,
it holds the same properties as round-robin and as such is compatible with
reusing an existing server connection with "option prefer-last-server".
2018-05-03 07:20:40 +02:00
Patrick Hemmer
e027547f8d MINOR: ssl: add fetch 'ssl_fc_session_key' and 'ssl_bc_session_key'
These fetches return the SSL master key of the front/back connection.
This is useful to decrypt traffic encrypted with ephemeral ciphers.
2018-04-30 14:56:19 +02:00
Tim Duesterhus
ca097c16a8 MINOR: sample: Add strcmp sample converter
This converter supplements the existing string matching by allowing
strings to be converted to a variable.

Example usage:

  http-request set-var(txn.host) hdr(host)
  # Check whether the client is attempting domain fronting.
  acl ssl_sni_http_host_match ssl_fc_sni,strcmp(txn.host) eq 0
2018-04-28 07:03:39 +02:00
William Lallemand
daf4cd209a MINOR: export localpeer as an environment variable
Export localpeer as the environment variable $HAPROXY_LOCALPEER,
allowing to use this variable in the configuration file.

It's useful to use this variable in the case of synchronized
configuration between peers.
2018-04-17 17:17:58 +02:00
Marcin Deranek
9631a28275 MEDIUM: sample: Extend functionality for field/word converters
Extend functionality of field/word converters, so it's possible
to extract field(s)/word(s) counting from the beginning/end and/or
extract multiple fields/words (including separators) eg.

str(f1_f2_f3__f5),field(2,_,2)  # f2_f3
str(f1_f2_f3__f5),field(2,_,0)  # f2_f3__f5
str(f1_f2_f3__f5),field(-2,_,3) # f2_f3_
str(f1_f2_f3__f5),field(-3,_,0) # f1_f2_f3

str(w1_w2_w3___w4),word(3,_,2)  # w3___w4
str(w1_w2_w3___w4),word(2,_,0)  # w2_w3___w4
str(w1_w2_w3___w4),word(-2,_,3) # w1_w2_w3
str(w1_w2_w3___w4),word(-3,_,0) # w1_w2

Change is backward compatible.
2018-04-17 11:27:48 +02:00
Marcin Deranek
9a66dfbd6c MINOR: proxy: Add fe_defbe fetcher
Patch adds ability to fetch frontend's default backend name in your
logic, so it can be used later to derive other backend names to make routing
decisions.
2018-04-16 15:51:57 +02:00
Emmanuel Hocdet
50791a7df3 MINOR: samples: add crc32c converter
This patch adds the support of CRC32c (rfc4960).
2018-03-21 16:17:00 +01:00
Emmanuel Hocdet
4399c75f6c MINOR: proxy-v2-options: add crc32c
This patch add option crc32c (PP2_TYPE_CRC32C) to proxy protocol v2.
It compute the checksum of proxy protocol v2 header as describe in
"doc/proxy-protocol.txt".
2018-03-21 05:04:01 +01:00
Cyril Bonté
3e9548777e DOC: log: more than 2 log servers are allowed
Since commit 0f99e3497, loggers are not limited to 2 instances anymore.
2018-03-21 04:56:33 +01:00
Baptiste Assmann
1fa7d2acce BUG/MINOR: dns: don't downgrade DNS accepted payload size automatically
Automatic downgrade of DNS accepted payload size may have undesired side
effect, which could make a backend with all servers DOWN.

After talking with Lukas on the ML, I realized this "feature" introduces
more issues that it fixes problem.
The "best" way to handle properly big responses will be to implement DNS
over TCP.

To be backported to 1.8.
2018-03-19 11:41:52 +01:00
Cyril Bonté
4288c5a9d8 BUG/MINOR: force-persist and ignore-persist only apply to backends
>From the very first day of force-persist and ignore-persist features,
they only applied to backends, except that the documentation stated it
could also be applied to frontends.

In order to make it clear, the documentation is updated and the parser
will raise a warning if the keywords are used in a frontend section.

This patch should be backported up to the 1.5 branch.
2018-03-12 22:52:24 +01:00
Emmanuel Hocdet
253c3b7516 MINOR: connection: add proxy-v2-options authority
This patch add option PP2_TYPE_AUTHORITY to proxy protocol v2 when a TLS
connection was negotiated. In this case, authority corresponds to the sni.
2018-03-01 11:38:32 +01:00
Emmanuel Hocdet
fa8d0f1875 MINOR: connection: add proxy-v2-options ssl-cipher,cert-sig,cert-key
This patch implement proxy protocol v2 options related to crypto information:
ssl-cipher (PP2_SUBTYPE_SSL_CIPHER), cert-sig (PP2_SUBTYPE_SSL_SIG_ALG) and
cert-key (PP2_SUBTYPE_SSL_KEY_ALG).
2018-03-01 11:38:28 +01:00
Emeric Brun
74f7ffa229 MINOR: ssl/sample: adds ssl_bc_is_resumed fetch keyword.
Returns true when the back connection was made over an SSL/TLS transport
layer and the newly created SSL session was resumed using a cached
session or a TLS ticket.
2018-02-19 16:50:20 +01:00
Willy Tarreau
280f42b99e MINOR: sample: add a new "concat" converter
It's always a pain not to be able to combine variables. This commit
introduces the "concat" converter, which appends a delimiter, a variable's
contents and another delimiter to an existing string. The result is a string.
This makes it easier to build composite variables made of other variables.
2018-02-19 15:34:12 +01:00
Tim Duesterhus
9ad9f3517e DOC: cfgparse: Warn on option (tcp|http)log in backend
The option does not seem to have any effect since at least haproxy
1.3. Also the `log-format` directive already warns when being used
in a backend.
2018-02-19 13:57:32 +01:00
Jérôme Magnin
61275198b3 DOC: Describe routing impact of using interface keyword on bind lines 2018-02-07 12:00:55 +01:00
Emmanuel Hocdet
f643b80429 MINOR: introduce proxy-v2-options for send-proxy-v2
Proxy protocol v2 can transport many optional informations. To avoid
send-proxy-v2-* explosion, this patch introduce proxy-v2-options parameter
and will allow to write: "send-proxy-v2 proxy-v2-options ssl,cert-cn".
2018-02-02 05:52:51 +01:00
Lukas Tribus
6fd698f4e6 DOC: don't suggest using http-server-close
Remove the old suggestion to use http-server-close mode, from the
beginnings of keep-alive mode in commit 16bfb021 "MINOR: config: add
option http-keep-alive").

We made http-keep-alive default in commit 70dffdaa "MAJOR: http:
switch to keep-alive mode by default".
2018-02-02 05:29:47 +01:00
Frédéric Lécaille
6778b27542 MINOR: stick-tables: Adds support for new "gpc1" and "gpc1_rate" counters.
Implement exactly the same code as this has been done for "gpc0" and "gpc0_rate"
counters.
2018-01-31 09:40:05 +01:00
Tim Duesterhus
1478aa795e MEDIUM: sample: Add IPv6 support to the ipmask converter
Add an optional second parameter to the ipmask converter that specifies
the number of bits to mask off IPv6 addresses.

If the second parameter is not given IPv6 addresses fail to mask (resulting
in an empty string), preserving backwards compatibility: Previously
a sample like `src,ipmask(24)` failed to give a result for IPv6 addresses.

This feature can be tested like this:

  defaults
  	log	global
  	mode	http
  	option	httplog
  	option	dontlognull
  	timeout connect 5000
  	timeout client  50000
  	timeout server  50000

  frontend fe
  	bind :::8080 v4v6

  	# Masked IPv4 for IPv4, empty for IPv6 (with and without this commit)
  	http-response set-header Test %[src,ipmask(24)]
  	# Correctly masked IP addresses for both IPv4 and IPv6
  	http-response set-header Test2 %[src,ipmask(24,ffff:ffff:ffff:ffff::)]
  	# Correctly masked IP addresses for both IPv4 and IPv6
  	http-response set-header Test3 %[src,ipmask(24,64)]

  	default_backend be

  backend be
  	server s example.com:80

Tested-By: Jarno Huuskonen <jarno.huuskonen@uef.fi>
2018-01-25 22:25:40 +01:00
Etienne Carriere
a792a0aa93 MINOR: sample: add date_us sample
Add date_us sample that returns the microsecond part of the timeval
structure representing the date of the structure. The "second" part of
the timeval can already be fetched by the "date" sample
2018-01-21 07:56:42 +01:00
Jérôme Magnin
4a326cba5b DOC: clarify the scope of ssl_fc_is_resumed
Clarify that it's for incoming connections.
2018-01-15 14:18:25 +01:00
Davor Ocelic
e9ed281e9f DOC/MINOR: configuration: typo, formatting fixes
- Add simple typo and formatting fixes
- Eliminate a couple > 80 column lines

Changes do not affect technical content and can be backported.
2017-12-27 19:03:32 +01:00
Willy Tarreau
c55ddce65c MINOR: http: update the list of cacheable status codes as per RFC7231
Since RFC2616, the following codes were added to the list of codes
cacheable by default : 204, 404, 405, 414, 501. For now this it only
checked by the checkcache option to detect cacheable cookies.
2017-12-22 14:43:26 +01:00
Willy Tarreau
24ea0bcb1d MINOR: http: adjust the list of supposedly cacheable methods
We used to have a rule inherited from RFC2616 saying that the POST
method was the only uncacheable one, but things have changed since
and RFC7231+7234 made it clear that in fact only GET/HEAD/OPTIONS/TRACE
are cacheable. Currently this rule is only used to detect cacheable
cookies.
2017-12-22 14:43:26 +01:00
Willy Tarreau
60a2ee7945 MINOR: sample: rename the "len" converter to "length"
This converter was recently introduced by commit ed0d24e ("MINOR:
sample: add len converter").

As found by Cyril, it causes an issue in "http-request capture"
statements. The non-obvious problem is that an old syntax for sample
expressions and converters used to support a series of words, each
representing a converter. This used to be how the "stick" directives
were created initially. By having a converter called "len", a
statement such as "http-request capture foo len 10" considers "len"
as a converter and not as the capture length.

This obsolete syntax needs to be changed in 1.9 but it's too late
for other versions. It's worth noting that the same problem can
happen if converters are registered on the fly using Lua. Other
language keywords that currently have to be avoided in converters
include "id", "table", "if", "unless".
2017-12-15 07:13:48 +01:00
Etienne Carriere
ed0d24ebed MINOR: sample: add len converter
Add len converter that returns the length of a string
2017-12-14 14:36:10 +01:00
Willy Tarreau
721d8e0286 MINOR: config: report when "monitor fail" rules are misplaced
"monitor-uri" may rely on "monitor fail" rules, which are processed
very early, immediately after the HTTP request is parsed and before
any http rulesets. It's not reported by the config parser when this
ruleset is misplaces, causing some configurations not to work like
users would expect. Let's just add the warning for a misplaced rule.
2017-12-01 18:25:08 +01:00
Cyril Bonté
7b888f1235 DOC: cache: update sections and fix some typos
Cache sections were not defined as the others, preventing them to be
correctly parsed by the HTML converter. Also, the "Cache" subsections
where not added to the summary.

This patch should be backported to the 1.8 branch.
2017-11-26 23:19:35 +01:00
Willy Tarreau
b306650c2a [RELEASE] Released version 1.9-dev0
Released version 1.9-dev0 with the following main changes :
    - BUG/MEDIUM: stream: don't automatically forward connect nor close
    - BUG/MAJOR: stream: ensure analysers are always called upon close
    - BUG/MINOR: stream-int: don't try to read again when CF_READ_DONTWAIT is set
    - MEDIUM: mworker: Add systemd `Type=notify` support
    - BUG/MEDIUM: cache: free callback to remove from tree
    - CLEANUP: cache: remove unused struct
    - MEDIUM: cache: enable the HTTP analysers
    - CLEANUP: cache: remove wrong comment
    - MINOR: threads/atomic: rename local variables in macros to avoid conflicts
    - MINOR: threads/plock: rename local variables in macros to avoid conflicts
    - MINOR: threads/atomic: implement pl_mb() in asm on x86
    - MINOR: threads/atomic: implement pl_bts() on non-x86
    - MINOR: threads/build: atomic: replace the few inlines with macros
    - BUILD: threads/plock: fix a build issue on Clang without optimization
    - BUILD: ebtree: don't redefine types u32/s32 in scope-aware trees
    - BUILD: compiler: add a new type modifier __maybe_unused
    - BUILD: h2: mark some inlined functions "unused"
    - BUILD: server: check->desc always exists
    - BUG/MEDIUM: h2: properly report connection errors in headers and data handlers
    - MEDIUM: h2: add a function to emit an HTTP/1 request from a headers list
    - MEDIUM: h2: change hpack_decode_headers() to only provide a list of headers
    - BUG/MEDIUM: h2: always reassemble the Cookie request header field
    - BUG/MINOR: systemd: ignore daemon mode
    - CONTRIB: spoa_example: allow to compile outside HAProxy.
    - CONTRIB: spoa_example: remove bref, wordlist, cond_wordlist
    - CONTRIB: spoa_example: remove last dependencies on type "sample"
    - CONTRIB: spoa_example: remove SPOE enums that are useless for clients
    - CLEANUP: cache: reorder includes
    - MEDIUM: shctx: use unsigned int for len and block_count
    - MEDIUM: cache: "show cache" on the cli
    - BUG/MEDIUM: cache: use key=0 as a condition for freeing
    - BUG/MEDIUM: cache: refcount forbids to free the objects
    - BUG/MEDIUM: cache fix cli_kws structure
    - BUG/MEDIUM: deinit: correctly deinitialize the proxy and global listener tasks
    - BUG/MINOR: ssl: Always start the handshake if we can't send early data.
    - MINOR: ssl: Don't disable early data handling if we could not write.
    - MINOR: pools: prepare functions to override malloc/free in pools
    - MINOR: pools: implement DEBUG_UAF to detect use after free
    - BUG/MEDIUM: threads/time: fix time drift correction
    - BUG/MEDIUM: threads/time: maintain a common time reference between all threads
    - MINOR: sample: Add "thread" sample fetch
    - BUG/MINOR: Use crt_base instead of ca_base when crt is parsed on a server line
    - BUG/MINOR: stream: fix tv_request calculation for applets
    - BUG/MAJOR: h2: always remove a stream from the send list before freeing it
    - BUG/MAJOR: threads/task: dequeue expired tasks under the WQ lock
    - MINOR: ssl: Handle reading early data after writing better.
    - MINOR: mux: Make sure every string is woken up after the handshake.
    - MEDIUM: cache: store sha1 for hashing the cache key
    - MINOR: http: implement the "http-request reject" rule
    - MINOR: h2: send RST_STREAM before GOAWAY on reject
    - MEDIUM: h2: don't gracefully close the connection anymore on Connection: close
    - MINOR: h2: make use of client-fin timeout after GOAWAY
    - MEDIUM: config: ensure that tune.bufsize is at least 16384 when using HTTP/2
    - MINOR: ssl: Handle early data with BoringSSL
    - BUG/MEDIUM: stream: always release the stream-interface on abort
    - BUG/MEDIUM: cache: free ressources in chn_end_analyze
    - MINOR: cache: move the refcount decrease in the applet release
    - BUG/MINOR: listener: Allow multiple "process" options on "bind" lines
    - MINOR: config: Support a range to specify processes in "cpu-map" parameter
    - MINOR: config: Slightly change how parse_process_number works
    - MINOR: config: Export parse_process_number and use it wherever it's applicable
    - MINOR: standard: Add my_ffsl function to get the position of the bit set to one
    - MINOR: config: Add auto-increment feature for cpu-map
    - MINOR: config: Support partial ranges in cpu-map directive
    - MINOR:: config: Remove thread-map directive
    - MINOR: config: Add the threads support in cpu-map directive
    - MINOR: config: Add threads support for "process" option on "bind" lines
    - MEDIUM: listener: Bind listeners on a thread subset if specified
    - CLEANUP: debug: Use DPRINTF instead of fprintf into #ifdef DEBUG_FULL/#endif
    - CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning
    - MINOR/CLEANUP: proxy: rename "proxy" to "proxies_list"
    - CLEANUP: pools: rename all pool functions and pointers to remove this "2"
    - DOC: update the roadmap file with the latest changes merged in 1.8
    - DOC: fix mangled version in peers protocol documentation
    - DOC: add initial peers protovol v2.0 documentation.
    - DOC: mention William as maintainer of the cache and master-worker
    - DOC: add Christopher and Emeric as maintainers of the threads
    - MINOR: cache: replace a fprint() by an abort()
    - MEDIUM: cache: max-age configuration keyword
    - DOC: explain HTTP2 timeout behavior
    - DOC: cache: configuration and management
    - MAJOR: mworker: exits the master on failure
    - BUG/MINOR: threads: don't drop "extern" on the lock in include files
    - MINOR: task: keep a pointer to the currently running task
    - MINOR: task: align the rq and wq locks
    - MINOR: fd: cache-align fdtab and fdcache locks
    - MINOR: buffers: cache-align buffer_wq_lock
    - CLEANUP: server: reorder some fields in struct server to save 40 bytes
    - CLEANUP: proxy: slightly reorder the struct proxy to reduce holes
    - CLEANUP: checks: remove 16 bytes of holes in struct check
    - CLEANUP: cache: more efficiently pack the struct cache
    - CLEANUP: fd: place the lock at the beginning of struct fdtab
    - CLEANUP: pools: align pools on a cache line
    - DOC: config: add a few bits about how to configure HTTP/2
    - BUG/MAJOR: threads/queue: avoid recursive locking in pendconn_get_next_strm()
    - BUILD: Makefile: reorder object files by size
2017-11-26 19:50:17 +01:00
Willy Tarreau
0b78792bbe [RELEASE] Released version 1.8.0
Released version 1.8.0 with the following main changes :
    - BUG/MEDIUM: stream: don't automatically forward connect nor close
    - BUG/MAJOR: stream: ensure analysers are always called upon close
    - BUG/MINOR: stream-int: don't try to read again when CF_READ_DONTWAIT is set
    - MEDIUM: mworker: Add systemd `Type=notify` support
    - BUG/MEDIUM: cache: free callback to remove from tree
    - CLEANUP: cache: remove unused struct
    - MEDIUM: cache: enable the HTTP analysers
    - CLEANUP: cache: remove wrong comment
    - MINOR: threads/atomic: rename local variables in macros to avoid conflicts
    - MINOR: threads/plock: rename local variables in macros to avoid conflicts
    - MINOR: threads/atomic: implement pl_mb() in asm on x86
    - MINOR: threads/atomic: implement pl_bts() on non-x86
    - MINOR: threads/build: atomic: replace the few inlines with macros
    - BUILD: threads/plock: fix a build issue on Clang without optimization
    - BUILD: ebtree: don't redefine types u32/s32 in scope-aware trees
    - BUILD: compiler: add a new type modifier __maybe_unused
    - BUILD: h2: mark some inlined functions "unused"
    - BUILD: server: check->desc always exists
    - BUG/MEDIUM: h2: properly report connection errors in headers and data handlers
    - MEDIUM: h2: add a function to emit an HTTP/1 request from a headers list
    - MEDIUM: h2: change hpack_decode_headers() to only provide a list of headers
    - BUG/MEDIUM: h2: always reassemble the Cookie request header field
    - BUG/MINOR: systemd: ignore daemon mode
    - CONTRIB: spoa_example: allow to compile outside HAProxy.
    - CONTRIB: spoa_example: remove bref, wordlist, cond_wordlist
    - CONTRIB: spoa_example: remove last dependencies on type "sample"
    - CONTRIB: spoa_example: remove SPOE enums that are useless for clients
    - CLEANUP: cache: reorder includes
    - MEDIUM: shctx: use unsigned int for len and block_count
    - MEDIUM: cache: "show cache" on the cli
    - BUG/MEDIUM: cache: use key=0 as a condition for freeing
    - BUG/MEDIUM: cache: refcount forbids to free the objects
    - BUG/MEDIUM: cache fix cli_kws structure
    - BUG/MEDIUM: deinit: correctly deinitialize the proxy and global listener tasks
    - BUG/MINOR: ssl: Always start the handshake if we can't send early data.
    - MINOR: ssl: Don't disable early data handling if we could not write.
    - MINOR: pools: prepare functions to override malloc/free in pools
    - MINOR: pools: implement DEBUG_UAF to detect use after free
    - BUG/MEDIUM: threads/time: fix time drift correction
    - BUG/MEDIUM: threads/time: maintain a common time reference between all threads
    - MINOR: sample: Add "thread" sample fetch
    - BUG/MINOR: Use crt_base instead of ca_base when crt is parsed on a server line
    - BUG/MINOR: stream: fix tv_request calculation for applets
    - BUG/MAJOR: h2: always remove a stream from the send list before freeing it
    - BUG/MAJOR: threads/task: dequeue expired tasks under the WQ lock
    - MINOR: ssl: Handle reading early data after writing better.
    - MINOR: mux: Make sure every string is woken up after the handshake.
    - MEDIUM: cache: store sha1 for hashing the cache key
    - MINOR: http: implement the "http-request reject" rule
    - MINOR: h2: send RST_STREAM before GOAWAY on reject
    - MEDIUM: h2: don't gracefully close the connection anymore on Connection: close
    - MINOR: h2: make use of client-fin timeout after GOAWAY
    - MEDIUM: config: ensure that tune.bufsize is at least 16384 when using HTTP/2
    - MINOR: ssl: Handle early data with BoringSSL
    - BUG/MEDIUM: stream: always release the stream-interface on abort
    - BUG/MEDIUM: cache: free ressources in chn_end_analyze
    - MINOR: cache: move the refcount decrease in the applet release
    - BUG/MINOR: listener: Allow multiple "process" options on "bind" lines
    - MINOR: config: Support a range to specify processes in "cpu-map" parameter
    - MINOR: config: Slightly change how parse_process_number works
    - MINOR: config: Export parse_process_number and use it wherever it's applicable
    - MINOR: standard: Add my_ffsl function to get the position of the bit set to one
    - MINOR: config: Add auto-increment feature for cpu-map
    - MINOR: config: Support partial ranges in cpu-map directive
    - MINOR:: config: Remove thread-map directive
    - MINOR: config: Add the threads support in cpu-map directive
    - MINOR: config: Add threads support for "process" option on "bind" lines
    - MEDIUM: listener: Bind listeners on a thread subset if specified
    - CLEANUP: debug: Use DPRINTF instead of fprintf into #ifdef DEBUG_FULL/#endif
    - CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning
    - MINOR/CLEANUP: proxy: rename "proxy" to "proxies_list"
    - CLEANUP: pools: rename all pool functions and pointers to remove this "2"
    - DOC: update the roadmap file with the latest changes merged in 1.8
    - DOC: fix mangled version in peers protocol documentation
    - DOC: add initial peers protovol v2.0 documentation.
    - DOC: mention William as maintainer of the cache and master-worker
    - DOC: add Christopher and Emeric as maintainers of the threads
    - MINOR: cache: replace a fprint() by an abort()
    - MEDIUM: cache: max-age configuration keyword
    - DOC: explain HTTP2 timeout behavior
    - DOC: cache: configuration and management
    - MAJOR: mworker: exits the master on failure
    - BUG/MINOR: threads: don't drop "extern" on the lock in include files
    - MINOR: task: keep a pointer to the currently running task
    - MINOR: task: align the rq and wq locks
    - MINOR: fd: cache-align fdtab and fdcache locks
    - MINOR: buffers: cache-align buffer_wq_lock
    - CLEANUP: server: reorder some fields in struct server to save 40 bytes
    - CLEANUP: proxy: slightly reorder the struct proxy to reduce holes
    - CLEANUP: checks: remove 16 bytes of holes in struct check
    - CLEANUP: cache: more efficiently pack the struct cache
    - CLEANUP: fd: place the lock at the beginning of struct fdtab
    - CLEANUP: pools: align pools on a cache line
    - DOC: config: add a few bits about how to configure HTTP/2
    - BUG/MAJOR: threads/queue: avoid recursive locking in pendconn_get_next_strm()
    - BUILD: Makefile: reorder object files by size
2017-11-26 19:25:23 +01:00
Willy Tarreau
95c4e14d0c DOC: config: add a few bits about how to configure HTTP/2
There's hardly anything to say in this file beyond the ALPN
configuration and some precisions about how the HTTP transaction model
applies to HTTP/2.
2017-11-26 12:18:55 +01:00
William Lallemand
4cfede87a3 MAJOR: mworker: exits the master on failure
This patch changes the behavior of the master during the exit of a
worker.

When a worker exits with an error code, for example in the case of a
segfault, all workers are now killed and the master leaves.

If you don't want this behavior you can use the option
"master-worker no-exit-on-failure".
2017-11-24 22:48:27 +01:00
William Lallemand
86d0df0b66 DOC: cache: configuration and management 2017-11-24 22:48:27 +01:00
Lukas Tribus
75df9d7a7a DOC: explain HTTP2 timeout behavior
Clarifies that in HTTP2 we don't consider "timeout http-keep-alive", but
"timeout client" instead.
2017-11-24 19:33:27 +01:00
Christopher Faulet
c644fa9bf5 MINOR: config: Add threads support for "process" option on "bind" lines
It is now possible on a "bind" line (or a "stats socket" line) to specify the
thread set allowed to process listener's connections. For instance:

    # HTTPS connections will be processed by all threads but the first and HTTP
    # connection will be processed on the first thread.
    bind *:80 process 1/1
    bind *:443 ssl crt mycert.pem process 1/2-
2017-11-24 15:38:50 +01:00
Christopher Faulet
cb6a94510d MINOR: config: Add the threads support in cpu-map directive
Now, it is possible to bind CPU at the thread level instead of the process level
by defining a thread set in "cpu-map" directives. Thus, its format is now:

  cpu-map [auto:]<process-set>[/<thread-set>] <cpu-set>...

where <process-set> and <thread-set> must follow the format:

  all | odd | even | number[-[number]]

Having a process range and a thread range in same time with the "auto:" prefix
is not supported. Only one range is supported, the other one must be a fixed
number. But it is allowed when there is no "auto:" prefix.

Because it is possible to define a mapping for a process and another for a
thread on this process, threads will be bound on the intersection of their
mapping and the one of the process on which they are attached. If the
intersection is null, no specific binding will be set for the threads.
2017-11-24 15:38:50 +01:00
Christopher Faulet
11da456e77 MINOR:: config: Remove thread-map directive
It was a temporary directive used for development purpose. Now, CPU mapping for
at the thread level should be done using the cpu-map directive. This feature
will be added in a next commit.
2017-11-24 15:38:50 +01:00
Christopher Faulet
ff4121f741 MINOR: config: Support partial ranges in cpu-map directive
Now, processa and CPU ranges can be partially defined. The higher bound can be
omitted. In such case, it is replaced by the corresponding maximum value, 32 or
64 depending on the machine's word size.

By extension, It is also true for the "bind-process" directive and "process"
parameter on a "bind" or a "stats socket" line.
2017-11-24 15:38:50 +01:00
Christopher Faulet
26028f6209 MINOR: config: Add auto-increment feature for cpu-map
The prefix "auto:" can be added before the process set to let HAProxy
automatically bind a process to a CPU by incrementing process and CPU sets. To
be valid, both sets must have the same size. No matter the declaration order of
the CPU sets, it will be bound from the lower to the higher bound.

  Examples:
      # all these lines bind the process 1 to the cpu 0, the process 2 to cpu 1
      #  and so on.
      cpu-map auto:1-4   0-3
      cpu-map auto:1-4   0-1 2-3
      cpu-map auto:1-4   3 2 1 0

      # bind each process to exaclty one CPU using all/odd/even keyword
      cpu-map auto:all   0-63
      cpu-map auto:even  0-31
      cpu-map auto:odd   32-63

      # invalid cpu-map because process and CPU sets have different sizes.
      cpu-map auto:1-4   0    # invalid
      cpu-map auto:1     0-3  # invalid
2017-11-24 15:38:49 +01:00
Christopher Faulet
1dcb9cb81c MINOR: config: Support a range to specify processes in "cpu-map" parameter
Now, you can define processes concerned by a cpu-map line using a range. For
instance, the following line binds the first 32 processes on CPUs 0 to 3:

  cpu-map 1-32 0-3
2017-11-24 15:38:49 +01:00
Willy Tarreau
45a66ccc55 MEDIUM: config: ensure that tune.bufsize is at least 16384 when using HTTP/2
HTTP/2 mandates the support of 16384 bytes frames by default, so we need
a large enough buffer to process them. Till now if tune.bufsize was too
small, H2 connections were simply rejected during their establishment,
making it quite hard to troubleshoot the issue.

Now we detect when HTTP/2 is enabled on an HTTP frontend and emit an
error if tune.bufsize is not large enough, with the appropriate
recommendation.
2017-11-24 11:28:00 +01:00
Willy Tarreau
599391a7c2 MINOR: h2: make use of client-fin timeout after GOAWAY
At the moment, the "client" timeout is used on an HTTP/2 connection once
it's idle with no active stream. With this patch, this timeout is replaced
by client-fin once a GOAWAY frame is sent. This closely matches what is
done on HTTP/1 since the principle is the same, as it indicates a willing
ness to quickly close a connection on which we don't expect to see anything
anymore.
2017-11-24 10:16:00 +01:00
Willy Tarreau
53275e8b02 MINOR: http: implement the "http-request reject" rule
This one acts similarly to its tcp-request counterpart. It immediately
closes the request without emitting any response. It can be suitable in
certain DoS conditions, as well as to close an HTTP/2 connection.
2017-11-24 07:52:01 +01:00
Christopher Faulet
34adb2af96 MINOR: sample: Add "thread" sample fetch
It returns id of the thread calling the function.
2017-11-23 16:33:13 +01:00
Lukas Tribus
f46bf95d2b BUG/MINOR: systemd: ignore daemon mode
Since we switched to notify mode in the systemd unit file in commit
d6942c8, haproxy won't start if the daemon keyword is present in the
configuration.

This change makes sure that haproxy remains in foreground when using
systemd mode and adds a note in the documentation.
2017-11-21 21:21:35 +01:00
Willy Tarreau
cfe14669f7 [RELEASE] Released version 1.8-rc4
Released version 1.8-rc4 with the following main changes :
    - BUG/MEDIUM: cache: does not cache if no Content-Length
    - BUILD: thread/pipe: fix build without threads
    - BUG/MINOR: spoe: check buffer size before acquiring or releasing it
    - MINOR: debug/flags: Add missing flags
    - MINOR: threads: Use __decl_hathreads to declare locks
    - BUG/MINOR: buffers: Fix b_alloc_margin to be "fonctionnaly" thread-safe
    - BUG/MAJOR: ebtree/scope: fix insertion and removal of duplicates in scope-aware trees
    - BUG/MAJOR: ebtree/scope: fix lookup of next node in scope-aware trees
    - MINOR: ebtree/scope: add a function to find next node from a parent
    - MINOR: ebtree/scope: simplify the lookup functions by using eb32sc_next_with_parent()
    - BUG/MEDIUM: mworker: Fix re-exec when haproxy is started from PATH
    - BUG/MEDIUM: cache: use msg->sov to forward header
    - MINOR: cache: forward data with headers
    - MINOR: cache: disable cache if shctx_row_data_append fail
    - BUG/MINOR: threads: tid_bit must be a unsigned long
    - CLEANUP: tasks: Remove useless double test on rq_next
    - BUG/MEDIUM: standard: itao_str/idx and quote_str/idx must be thread-local
    - MINOR: tools: add a function to dump a scope-aware tree to a file
    - MINOR: tools: improve the DOT dump of the ebtree
    - MINOR: tools: emphasize the node being worked on in the tree dump
    - BUG/MAJOR: ebtree/scope: properly tag upper nodes during insertion
    - DOC: peers: Add a first version of peers protocol v2.1.
    - CONTRIB: Wireshark dissector for HAProxy Peer Protocol.
    - MINOR: mworker: display an accurate error when the reexec fail
    - BUG/MEDIUM: mworker: wait again for signals when execvp fail
    - BUG/MEDIUM: mworker: does not deinit anymore
    - BUG/MEDIUM: mworker: does not close inherited FD
    - MINOR: tests: add a python wrapper to test inherited fd
    - BUG/MINOR: Allocate the log buffers before the proxies startup
    - MINOR: tasks: Use a bitfield to track tasks activity per-thread
    - MAJOR: polling: Use active_tasks_mask instead of tasks_run_queue
    - MINOR: applets: Use a bitfield to track applets activity per-thread
    - MAJOR: polling: Use active_appels_mask instead of applets_active_queue
    - MEDIUM: applets: Don't process more than 200 active applets at once
    - MINOR: stream: Add thread-mask of tasks/FDs/applets in "show sess all" command
    - MINOR: SSL: Store the ASN1 representation of client sessions.
    - MINOR: ssl: Make sure we don't shutw the connection before the handshake.
    - BUG/MEDIUM: deviceatlas: ignore not valuable HTTP request data
2017-11-19 09:55:29 +01:00