Commit Graph

663 Commits

Author SHA1 Message Date
Olivier Houchard
692c1d07f9 MINOR: ssl: Don't forget to call the close method of the underlying xprt.
In ssl_sock_close(), don't forget to call the underlying xprt's close method
if it exists. For now it's harmless not to do so, because the only available
layer is the raw socket, which doesn't have a close method, but that will
change when we implement QUIC.
2019-05-28 10:08:39 +02:00
Olivier Houchard
19afb274ad MINOR: ssl: Make sure the underlying xprt's init method doesn't fail.
In ssl_sock_init(), when initting the underlying xprt, check the return value,
and give up if it fails.
2019-05-28 10:08:28 +02:00
Ilya Shipitsin
0590f44254 BUILD: ssl: fix latest LibreSSL reg-test error
starting with OpenSSL 1.0.0 recommended way to disable compression is
using SSL_OP_NO_COMPRESSION when creating context.

manipulations with SSL_COMP_get_compression_methods, sk_SSL_COMP_num
are only required for OpenSSL < 1.0.0
2019-05-26 21:26:02 +02:00
Ilya Shipitsin
e242f3dfb8 BUG/MINOR: ssl_sock: Fix memory leak when disabling compression
according to manpage:

       sk_TYPE_zero() sets the number of elements in sk to zero. It
       does not free sk so after this call sk is still valid.

so we need to free all elements

[wt: seems like it has been there forever and should be backported
 to all stable branches]
2019-05-25 07:45:55 +02:00
William Lallemand
7e1770b151 BUG/MAJOR: ssl: segfault upon an heartbeat request
7b5fd1e ("MEDIUM: connections: Move some fields from struct connection
to ssl_sock_ctx.") introduced a bug in the heartbleed mitigation code.

Indeed the code used conn->ctx instead of conn->xprt_ctx for the ssl
context, resulting in a null dereference.
2019-05-13 16:03:44 +02:00
Willy Tarreau
295d614de1 CLEANUP: ssl: move all BIO_* definitions to openssl-compat
The following macros are now defined for openssl < 1.1 so that we
can remove the code performing direct access to the structures :

  BIO_get_data(), BIO_set_data(), BIO_set_init(), BIO_meth_free(),
  BIO_meth_new(), BIO_meth_set_gets(), BIO_meth_set_puts(),
  BIO_meth_set_read(), BIO_meth_set_write(), BIO_meth_set_create(),
  BIO_meth_set_ctrl(), BIO_meth_set_destroy()
2019-05-11 17:39:08 +02:00
Willy Tarreau
11b167167e CLEANUP: ssl: remove ifdef around SSL_CTX_get_extra_chain_certs()
Instead define this one in openssl-compat.h when
SSL_CTRL_GET_EXTRA_CHAIN_CERTS is not defined (which was the current
condition used in the ifdef).
2019-05-11 17:38:21 +02:00
Willy Tarreau
366a6987a7 CLEANUP: ssl: move the SSL_OP_* and SSL_MODE_* definitions to openssl-compat
These ones were defined in the middle of ssl_sock.c, better move them
to the include file to find them.
2019-05-11 17:37:44 +02:00
Willy Tarreau
8d164dc568 CLEANUP: ssl: never include openssl/*.h outside of openssl-compat.h anymore
Since we're providing a compatibility layer for multiple OpenSSL
implementations and their derivatives, it is important that no C file
directly includes openssl headers but only passes via openssl-compat
instead. As a bonus this also gets rid of redundant complex rules for
inclusion of certain files (engines etc).
2019-05-10 09:36:42 +02:00
Willy Tarreau
9356dacd22 REORG: ssl: move some OpenSSL defines from ssl_sock to openssl-compat
Some defines like OPENSSL_VERSION or X509_getm_notBefore() have nothing
to do in ssl_sock and must move to openssl-compat.h so that they are
consistently shared by the whole code. A warning in the code was added
against wild additions of macros there.
2019-05-10 09:31:06 +02:00
Willy Tarreau
5599456ee2 REORG: ssl: move openssl-compat from proto to common
This way we can include it much earlier to cover types/ as well.
2019-05-10 09:19:50 +02:00
Willy Tarreau
df17e0e1a7 BUILD: ssl: fix libressl build again after aes-gcm-enc
Enabling aes-gcm-enc in last commit (MINOR: ssl: enable aes_gcm_dec
on LibreSSL) uncovered a wrong condition on the define of the
EVP_CTRL_AEAD_SET_IVLEN macro which I forgot to add when making the
commit, resulting in breaking libressl build again. In case libressl
later defines this macro, the test will have to change for a version
range instead.
2019-05-10 09:19:07 +02:00
Willy Tarreau
86a394e44d MINOR: ssl: enable aes_gcm_dec on LibreSSL
This one requires OpenSSL 1.0.1 and above, and libressl was forked from
1.0.1g and is compatible (build-tested). No need to exclude it anymore
from using this converter.
2019-05-09 14:26:40 +02:00
Willy Tarreau
5db847ab65 CLEANUP: ssl: remove 57 occurrences of useless tests on LIBRESSL_VERSION_NUMBER
They were all check to comply with the advertised openssl version. Now
that libressl doesn't pretend to be a more recent openssl anymore, we
can simply rely on the regular openssl version tests without having to
deal with exceptions for libressl.
2019-05-09 14:26:39 +02:00
Willy Tarreau
1d158ab12d BUILD: ssl: make libressl use its own version numbers
LibreSSL causes lots of build issues by pretending to be OpenSSL 2.0.0,
and it requires lots of care for each #if added to cover any specific
OpenSSL features.

This commit addresses the problem by making LibreSSL only advertise the
version it forked from (1.0.1g) and by starting to use tests based on
its real version to enable features instead of working by exclusion.
2019-05-09 14:25:47 +02:00
Willy Tarreau
9a1ab08160 CLEANUP: ssl-sock: use HA_OPENSSL_VERSION_NUMBER instead of OPENSSL_VERSION_NUMBER
Most tests on OPENSSL_VERSION_NUMBER have become complex and break all
the time because this number is fake for some derivatives like LibreSSL.
This patch creates a new macro, HA_OPENSSL_VERSION_NUMBER, which will
carry the real openssl version defining the compatibility level, and
this version will be adjusted depending on the variants.
2019-05-09 14:25:43 +02:00
Olivier Houchard
4cd2af4e5d BUG/MEDIUM: ssl: Don't attempt to use early data with libressl.
Libressl doesn't yet provide early data, so don't put the CO_FL_EARLY_SSL_HS
on the connection if we're building with libressl, or the handshake will
never be done.
2019-05-06 15:20:42 +02:00
Ilya Shipitsin
54832b97c6 BUILD: enable several LibreSSL hacks, including
SSL_SESSION_get0_id_context is introduced in LibreSSL-2.7.0
async operations are not supported by LibreSSL
early data is not supported by LibreSSL
packet_length is removed from SSL struct in LibreSSL
2019-05-06 07:26:24 +02:00
Olivier Houchard
010941f876 BUG/MEDIUM: ssl: Use the early_data API the right way.
We can only read early data if we're a server, and write if we're a client,
so don't attempt to mix both.

This should be backported to 1.8 and 1.9.
2019-05-03 21:00:10 +02:00
Olivier Houchard
b51937ebaa BUG/MEDIUM: ssl: Don't pretend we can retry a recv/send if we got a shutr/w.
In ha_ssl_write() and ha_ssl_read(), don't pretend we can retry a read/write
if we got a shutr/shutw, or we will never properly shutdown the connection.
2019-05-01 17:37:33 +02:00
Olivier Houchard
a28454ee21 BUG/MEDIUM: ssl: Return -1 on recv/send if we got EAGAIN.
In ha_ssl_read()/ha_ssl_write(), if we couldn't send/receive data because
we got EAGAIN, return -1 and not 0, as older SSL versions expect that.
This should fix the problems with OpenSSL < 1.1.0.
2019-04-24 12:06:08 +02:00
Emeric Brun
d0e095c2aa MINOR: ssl/cli: async fd io-handlers printable on show fd
This patch exports the async fd iohandlers and make them printable
doing a 'show fd' on cli.
2019-04-19 17:27:01 +02:00
Olivier Houchard
66a7b3302a BUILD/medium: ssl: Fix build with OpenSSL < 1.1.0
Make sure it builds with OpenSSL < 1.1.0, a lot of the BIO_get/set methods
were introduced with OpenSSL 1.1.0, so fallback with the old way of doing
things if needed.
2019-04-18 15:58:58 +02:00
Olivier Houchard
a8955d57ed MEDIUM: ssl: provide our own BIO.
Instead of letting the OpenSSL code handle the file descriptor directly,
provide a custom BIO, that will use the underlying XPRT to send/recv data.
This will let us implement QUIC later, and probably clean the upper layer,
if/when the SSL code provide its own subscribe code, so that the upper layers
won't have to care if we're still waiting for the handshake to complete or not.
2019-04-18 14:56:24 +02:00
Olivier Houchard
e179d0e88f MEDIUM: connections: Provide a xprt_ctx for each xprt method.
For most of the xprt methods, provide a xprt_ctx.  This will be useful later
when we'll want to be able to stack xprts.
The init() method now has to create and provide the said xprt_ctx if needed.
2019-04-18 14:56:24 +02:00
Olivier Houchard
df35784600 MEDIUM: ssl: provide its own subscribe/unsubscribe function.
In order to prepare for the possibility of using different kinds of xprt
with ssl, make the ssl code provide its own subscribe and unsubscribe
functions, right now it just calls conn_subscribe and conn_unsubsribe.
2019-04-18 14:56:24 +02:00
Olivier Houchard
7b5fd1ec26 MEDIUM: connections: Move some fields from struct connection to ssl_sock_ctx.
Move xprt_st, tmp_early_data and sent_early_data from struct connection to
struct ssl_sock_ctx, as they are only used in the SSL code.
2019-04-18 14:56:24 +02:00
Olivier Houchard
66ab498f26 MEDIUM: ssl: Give ssl_sock its own context.
Instead of using directly a SSL * as xprt_ctx, give ssl_sock its own context.
It's useless for now, but will be useful later when we'll want to be able to
stack xprts.
2019-04-18 14:56:24 +02:00
Willy Tarreau
0e492e2ad0 BUILD: address a few cases of "static <type> inline foo()"
Older compilers don't like to see "inline" placed after the type in a
function declaration, it must be "static inline <type>" only. This
patch touches various areas. The warnings were seen with gcc-3.4.
2019-04-15 21:55:48 +02:00
Emmanuel Hocdet
2b4edfb0bd MINOR: ssl: Activate aes_gcm_dec converter for BoringSSL
BoringSSL can support it, no need to disable.
2019-04-11 15:00:13 +02:00
Nenad Merdanovic
c31499d747 MINOR: ssl: Add aes_gcm_dec converter
The converter can be used to decrypt the raw byte input using the
AES-GCM algorithm, using provided nonce, key and AEAD tag. This can
be useful to decrypt encrypted cookies for example and make decisions
based on the content.
2019-04-01 13:33:31 +02:00
Pierre Cheynier
bc34cd1de2 BUG/MEDIUM: ssl: ability to set TLS 1.3 ciphers using ssl-default-server-ciphersuites
Any attempt to put TLS 1.3 ciphers on servers failed with output 'unable
to set TLS 1.3 cipher suites'.

This was due to usage of SSL_CTX_set_cipher_list instead of
SSL_CTX_set_ciphersuites in the TLS 1.3 block (protected by
OPENSSL_VERSION_NUMBER >= 0x10101000L & so).

This should be backported to 1.9 and 1.8.

Signed-off-by: Pierre Cheynier <p.cheynier@criteo.com>
Reported-by: Damien Claisse <d.claisse@criteo.com>
Cc: Emeric Brun <ebrun@haproxy.com>
2019-03-22 17:24:14 +01:00
Olivier Houchard
2be5a4c627 MEDIUM: ssl: Use the new _HA_ATOMIC_* macros.
Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-11 17:02:38 +01:00
Lukas Tribus
1aabc93978 BUG/MINOR: ssl: fix warning about ssl-min/max-ver support
In 84e417d8 ("MINOR: ssl: support Openssl 1.1.1 early callback for
switchctx") the code was extended to also support OpenSSL 1.1.1
(code already supported BoringSSL). A configuration check warning
was updated but with the wrong logic, the #ifdef needs a && instead
of an ||.

Reported in #54.

Should be backported to 1.8.
2019-03-05 23:56:58 +01:00
Dirkjan Bussink
526894ff39 BUG/MEDIUM: ssl: Fix handling of TLS 1.3 KeyUpdate messages
In OpenSSL 1.1.1 TLS 1.3 KeyUpdate messages will trigger the callback
that is used to verify renegotiation is disabled. This means that these
KeyUpdate messages fail. In OpenSSL 1.1.1 a better mechanism is
available with the SSL_OP_NO_RENEGOTIATION flag that disables any TLS
1.2 and earlier negotiation.

So if this SSL_OP_NO_RENEGOTIATION flag is available, instead of having
a manual check, trust OpenSSL and disable the check. This means that TLS
1.3 KeyUpdate messages will work properly.

Reported-By: Adam Langley <agl@imperialviolet.org>
2019-01-23 09:51:22 +01:00
Olivier Houchard
f24502ba46 BUG/MEDIUM: connections: Add the CO_FL_CONNECTED flag if a send succeeded.
If a send succeeded, add the CO_FL_CONNECTED flag, the send may have been
called by the upper layers before we even realized we were connected, and we
may even read the response before we get the information, and si_cs_recv()
has to know if we were connected or not.

This should be backported to 1.9.
2019-01-17 19:18:20 +01:00
Emeric Brun
9e7547740c MINOR: ssl: add support of aes256 bits ticket keys on file and cli.
Openssl switched from aes128 to aes256 since may 2016  to compute
tls ticket secrets used by default. But Haproxy still handled only
128 bits keys for both tls key file and CLI.

This patch permit the user to set aes256 keys throught CLI or
the key file (80 bytes encoded in base64) in the same way that
aes128 keys were handled (48 bytes encoded in base64):
- first 16 bytes for the key name
- next 16/32 bytes for aes 128/256 key bits key
- last 16/32 bytes for hmac 128/256 bits

Both sizes are now supported (but keys from same file must be
of the same size and can but updated via CLI only using a key of
the same size).

Note: This feature need the fix "dec func ignores padding for output
size checking."
2019-01-14 19:32:58 +01:00
Emeric Brun
09852f70e0 BUG/MEDIUM: ssl: missing allocation failure checks loading tls key file
This patch fixes missing allocation checks loading tls key file
and avoid memory leak in some error cases.

This patch should be backport on branches 1.9 and 1.8
2019-01-14 19:32:45 +01:00
Olivier Houchard
51088ce68f BUG/MEDIUM: ssl: Disable anti-replay protection and set max data with 0RTT.
When using early data, disable the OpenSSL anti-replay protection, and set
the max amount of early data we're ready to accept, based on the size of
buffers, or early data won't work with the released OpenSSL 1.1.1.

This should be backported to 1.8.
2019-01-09 16:26:28 +01:00
Olivier Houchard
921501443b MEDIUM: checks: Add check-alpn.
Add a way to configure the ALPN used by check, with a new "check-alpn"
keyword. By default, the checks will use the server ALPN, but it may not
be convenient, for instance because the server may use HTTP/2, while checks
are unable to do HTTP/2 yet.
2018-12-21 19:54:16 +01:00
Olivier Houchard
ab28a320aa MINOR: ssl: Add ssl_sock_set_alpn().
Add a new function, ssl_sock_set_alpn(), to be able to change the ALPN
for a connection, instead of relying of the one defined in the SSL_CTX.
2018-12-21 19:53:30 +01:00
Rosen Penev
6818595b57 BUILD: ssl: Fix compilation without deprecated OpenSSL 1.1 APIs
Removing deprecated APIs is an optional part of OpenWrt's build system to
save some space on embedded devices.

Also added compatibility for LibreSSL.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-12-15 16:34:38 +01:00
Willy Tarreau
e0f24ee149 MINOR: connection: realign empty buffers in muxes, not transport layers
For a long time we've been realigning empty buffers in the transport
layers, where the I/Os were performed based on callbacks. Doing so is
optimal for higher data throughput but makes it trickier to optimize
unaligned data, where mux_h1/h2 have to claim some data are present
in the buffer to force unaligned accesses to skip the frame's header
or the chunk header.

We don't need to do this anymore since the I/O calls are now always
performed from top to bottom, so it's only the mux's responsibility
to realign an empty buffer if it wants to.

In practice it doesn't change anything, it's just a convention, and
it will allow the code to be simplified in a next patch.
2018-12-14 10:51:23 +01:00
Jérôme Magnin
e064a80fa3 BUG/MINOR: fix ssl_fc_alpn and actually add ssl_bc_alpn
When ssl_bc_alpn was meant to be added, a typo slipped in and as a result ssl_fc_alpn behaved as ssl_bc_alpn,
and ssl_bc_alpn was not a valid keyword. this patch aims at fixing this.
2018-12-04 05:53:45 +01:00
Baptiste Assmann
6be139f867 BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores session id
In ssl_sock_parse_clienthello(), the code considers that SSL Sessionid
size is '1', and then considers that the SSL cipher suite is availble
right after the session id size information.
This actually works in a single case, when the client does not send a
session id.

This patch fixes this issue by introducing the a propoer way to parse
the session id and move forward the cursor by the session id length when
required.

Need to be backported to 1.8.
2018-11-29 16:55:29 +01:00
Lukas Tribus
7706b85e0c MINOR: ssl: free ctx when libssl doesn't support NPN
The previous fix da95fd90 ("BUILD/MINOR: ssl: fix build with non-alpn/
non-npn libssl") does fix the build in old OpenSSL release, but I
overlooked that the ctx is only freed when NPN is supported.

Fix this by moving the #endif to the proper place (this was broken in
c7566001 ("MINOR: server: Add "alpn" and "npn" keywords")).
2018-11-27 04:32:32 +01:00
Willy Tarreau
8071338c78 MINOR: initcall: apply initcall to all register_build_opts() calls
Most register_build_opts() calls use static strings. These ones were
replaced with a trivial REGISTER_BUILD_OPTS() statement adding the string
and its call to the STG_REGISTER section. A dedicated section could be
made for this if needed, but there are very few such calls for this to
be worth it. The calls made with computed strings however, like those
which retrieve OpenSSL's version or zlib's version, were moved to a
dedicated function to guarantee they are called late in the process.
For example, the SSL call probably requires that SSL_library_init()
has been called first.
2018-11-26 19:50:32 +01:00
Willy Tarreau
86abe44e42 MEDIUM: init: use self-initializing spinlocks and rwlocks
This patch replaces a number of __decl_hathread() followed by HA_SPIN_INIT
or HA_RWLOCK_INIT by the new __decl_spinlock() or __decl_rwlock() which
automatically registers the lock for initialization in during the STG_LOCK
init stage. A few static modifiers were lost in the process, but since they
were not essential at all it was not worth extending the API to provide such
a variant.
2018-11-26 19:50:32 +01:00
Willy Tarreau
0108d90c6c MEDIUM: init: convert all trivial registration calls to initcalls
This switches explicit calls to various trivial registration methods for
keywords, muxes or protocols from constructors to INITCALL1 at stage
STG_REGISTER. All these calls have in common to consume a single pointer
and return void. Doing this removes 26 constructors. The following calls
were addressed :

- acl_register_keywords
- bind_register_keywords
- cfg_register_keywords
- cli_register_kw
- flt_register_keywords
- http_req_keywords_register
- http_res_keywords_register
- protocol_register
- register_mux_proto
- sample_register_convs
- sample_register_fetches
- srv_register_keywords
- tcp_req_conn_keywords_register
- tcp_req_cont_keywords_register
- tcp_req_sess_keywords_register
- tcp_res_cont_keywords_register
- flt_register_keywords
2018-11-26 19:50:32 +01:00
Lukas Tribus
da95fd901b BUILD/MINOR: ssl: fix build with non-alpn/non-npn libssl
In commit c7566001 ("MINOR: server: Add "alpn" and "npn" keywords") and
commit 201b9f4e ("MAJOR: connections: Defer mux creation for outgoing
connection if alpn is set"), the build was broken on older OpenSSL
releases.

Move the #ifdef's around so that we build again with older OpenSSL
releases (0.9.8 was tested).
2018-11-26 08:34:40 +01:00
Olivier Houchard
6b77f49e78 MEDIUM: ssl: Add ssl_bc_alpn and ssl_bc_npn sample fetches.
Add 2 new sample fetches, ssl_bc_alpn and ssl_bc_npn, that provides the
ALPN and the NPN for an outgoing connection.
2018-11-22 19:52:44 +01:00
Olivier Houchard
c756600103 MINOR: server: Add "alpn" and "npn" keywords.
Add new keywords to "server" lines, alpn and npn.
If set, when connecting through SSL, those alpn/npn will be negociated
during the SSL handshake.
2018-11-22 19:50:08 +01:00
Joseph Herlant
017b3da94e CLEANUP: fix typos in the ssl_sock subsystem
Fix some typos found in the code comments of the ssl_sock subsystem.
2018-11-18 22:23:15 +01:00
Willy Tarreau
db398435aa MINOR: stream-int: replace si_cant_put() with si_rx_room_{blk,rdy}()
Remaining calls to si_cant_put() were all for lack of room and were
turned to si_rx_room_blk(). A few places where SI_FL_RXBLK_ROOM was
cleared by hand were converted to si_rx_room_rdy().

The now unused si_cant_put() function was removed.
2018-11-18 21:41:50 +01:00
Willy Tarreau
0cd3bd628a MINOR: stream-int: rename si_applet_{want|stop|cant}_{get|put}
It doesn't make sense to limit this code to applets, as any stream
interface can use it. Let's rename it by simply dropping the "applet_"
part of the name. No other change was made except updating the comments.
2018-11-11 10:18:37 +01:00
Frédéric Lécaille
4c8aa117f9 BUG/MINOR: ssl: Wrong usage of shctx_init().
With this patch we check that shctx_init() does not return 0.

Must be backported to 1.8.
2018-10-26 04:54:40 +02:00
Frédéric Lécaille
b7838afe6f MINOR: shctx: Add a maximum object size parameter.
This patch adds a new parameter to shctx_init() function to be used to
limit the size of each shared object, -1 value meaning "no limit".
2018-10-24 04:39:44 +02:00
Frédéric Lécaille
0bec807e08 MINOR: shctx: Shared objects block by block allocation.
This patch makes shctx capable of storing objects in several parts,
each parts being made of several blocks. There is no more need to
walk through until reaching the end of a row to append new blocks.

A new pointer to a struct shared_block member, named last_reserved,
has been added to struct shared_block so that to memorize the last block which was
reserved by shctx_row_reserve_hot(). Same thing about "last_append" pointer which
is used to memorize the last block used by shctx_row_data_append() to store the data.
2018-10-24 04:35:53 +02:00
Willy Tarreau
a8825520b7 BUILD: ssl: fix another null-deref warning in ssl_sock_switchctx_cbk()
This null-deref cannot happen either as there necesarily is a listener
where this function is called. Let's use __objt_listener() to address
this.

This may be backported to 1.8.
2018-10-15 13:24:43 +02:00
Willy Tarreau
b729077710 BUILD: ssl: fix null-deref warning in ssl_fc_cipherlist_str sample fetch
Gcc 6.4 detects a potential null-deref warning in smp_fetch_ssl_fc_cl_str().
This one is not real since already addressed a few lines above. Let's use
__objt_conn() instead of objt_conn() to avoid the extra test that confuses
it.

This could be backported to 1.8.
2018-10-15 13:24:43 +02:00
Olivier Houchard
83a0cd8a36 MINOR: connections: Introduce an unsubscribe method.
As we don't know how subscriptions are handled, we can't just assume we can
use LIST_DEL() to unsubscribe, so introduce a new method to mux and connections
to do so.
2018-10-11 15:34:21 +02:00
Emeric Brun
7ad43e7928 BUG/MEDIUM: Cur/CumSslConns counters not threadsafe.
CurSslConns inc/dec operations are not threadsafe. The unsigned CurSslConns
counter can wrap to a negative value. So we could notice connection rejects
because of MaxSslConns limit artificially exceeded.

CumSslConns inc operation are also not threadsafe so we could miss
some connections and show inconsistenties values compared to CumConns.

This fix should be backported to v1.8.
2018-10-10 18:05:33 +02:00
Dirkjan Bussink
415150f764 MEDIUM: ssl: add support for ciphersuites option for TLSv1.3
OpenSSL released support for TLSv1.3. It also added a separate function
SSL_CTX_set_ciphersuites that is used to set the ciphers used in the
TLS 1.3 handshake. This change adds support for that new configuration
option by adding a ciphersuites configuration variable that works
essentially the same as the existing ciphers setting.

Note that it should likely be backported to 1.8 in order to ease usage
of the now released openssl-1.1.1.
2018-10-08 19:20:13 +02:00
Emmanuel Hocdet
747ca61693 MINOR: ssl: generate-certificates for BoringSSL 2018-10-08 09:42:34 +02:00
Emmanuel Hocdet
a9b84028e6 MINOR: ssl: cleanup old openssl API call
For generate-certificates, X509V3_EXT_conf is used but it's an old API
call: X509V3_EXT_nconf must be preferred. Openssl compatibility is ok
because it's inside #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME, introduce 5
years after X509V3_EXT_nconf.
2018-10-08 09:42:28 +02:00
Willy Tarreau
61c112aa5b REORG: http: move HTTP rules parsing to http_rules.c
These ones are mostly called from cfgparse.c for the parsing and do
not depend on the HTTP representation. The functions's prototypes
were moved to proto/http_rules.h, making this file work exactly like
tcp_rules. Ideally we should stop calling these functions directly
from cfgparse and register keywords, but there are a few cases where
that wouldn't work (stats http-request) so it's probably not worth
trying to go this far.
2018-10-02 18:28:05 +02:00
Willy Tarreau
07d94e48d9 BUILD: ssl_sock: remove build warnings on potential null-derefs
When building with -Wnull-dereferences, gcc sees some cases where a
pointer is dereferenced after a check may set it to null. While all of
these are already guarded by either a preliminary test or the code's
construction (eg: listeners code being called only on listeners), it
cannot be blamed for not "seeing" this, so better use the unguarded
calls everywhere this happens, particularly after checks. This is a
step towards building with -Wextra.
2018-09-20 11:42:15 +02:00
Emmanuel Hocdet
9f9b0c6a7f BUG/MEDIUM: ECC cert should work with TLS < v1.2 and openssl >= 1.1.1
With openssl >= 1.1.1 and boringssl multi-cert is natively supported.
ECDSA/RSA selection is done and work correctly with TLS >= v1.2.
TLS < v1.2 have no TLSEXT_TYPE_signature_algorithms extension: ECC
certificate can't be selected, and handshake fail if no RSA cert
is present. Safe ECC certificate selection without client announcement
can be very tricky (browser compatibilty). The safer approach is to
select ECDSA certificate if no other certificate matches, like it is
with openssl < 1.1.1: certificate selection is only done via the SNI.

Thanks to Lukas Tribus for reporting this and analysing the problem.

This patch should be backported to 1.8
2018-09-04 17:47:10 +02:00
Willy Tarreau
1c913e4232 BUG/MEDIUM: cli/ssl: don't store base64dec() result in the trash's length
By convenience or laziness we used to store base64dec()'s return code
into trash.data and to compare it against 0 to check for conversion
failure, but it's now unsigned since commit 843b7cb ("MEDIUM: chunks:
make the chunk struct's fields match the buffer struct"). Let's clean
this up and test the result itself without storing it first.

No backport is needed.
2018-08-22 05:28:32 +02:00
Bertrand Jacquin
a25282bb39 DOC: ssl: Use consistent naming for TLS protocols
In most cases, "TLSv1.x" naming is used across and documentation, lazy
people tend to grep too much and may not find what they are looking for.

Fixing people is hard.
2018-08-16 20:20:26 +02:00
Emeric Brun
e1b4ed4352 BUG/MEDIUM: ssl: loading dh param from certifile causes unpredictable error.
If the dh parameter is not found, the openssl's error global
stack was not correctly cleared causing unpredictable error
during the following parsing (chain cert parsing for instance).

This patch should be backported in 1.8 (and perhaps 1.7)
2018-08-16 19:36:08 +02:00
Emeric Brun
eb155b6ca6 BUG/MEDIUM: ssl: fix missing error loading a keytype cert from a bundle.
If there was an issue loading a keytype's part of a bundle, the bundle
was implicitly ignored without errors.

This patch should be backported in 1.8 (and perhaps 1.7)
2018-08-16 19:36:06 +02:00
Emeric Brun
77e8919fc6 BUG/MINOR: ssl: empty connections reported as errors.
Empty connection is reported as handshake error
even if dont-log-null is specified.

This bug affect is a regression du to:

BUILD: ssl: fix to build (again) with boringssl

New openssl 1.1.1 defines OPENSSL_NO_HEARTBEATS as boring ssl
so the test was replaced by OPENSSL_IS_BORINGSSL

This fix should be backported on 1.8
2018-08-16 11:59:59 +02:00
Olivier Houchard
6ff2039d13 MINOR: connections/mux: Add a new "subscribe" method.
Add a new "subscribe" method for connection, conn_stream and mux, so that
upper layer can subscribe to them, to be called when the event happens.
Right now, the only event implemented is "SUB_CAN_SEND", where the upper
layer can register to be called back when it is possible to send data.

The connection and conn_stream got a new "send_wait_list" entry, which
required to move a few struct members around to maintain an efficient
cache alignment (and actually this slightly improved performance).
2018-07-19 16:23:43 +02:00
Willy Tarreau
83061a820e MAJOR: chunks: replace struct chunk with struct buffer
Now all the code used to manipulate chunks uses a struct buffer instead.
The functions are still called "chunk*", and some of them will progressively
move to the generic buffer handling code as they are cleaned up.
2018-07-19 16:23:43 +02:00
Willy Tarreau
843b7cbe9d MEDIUM: chunks: make the chunk struct's fields match the buffer struct
Chunks are only a subset of a buffer (a non-wrapping version with no head
offset). Despite this we still carry a lot of duplicated code between
buffers and chunks. Replacing chunks with buffers would significantly
reduce the maintenance efforts. This first patch renames the chunk's
fields to match the name and types used by struct buffers, with the goal
of isolating the code changes from the declaration changes.

Most of the changes were made with spatch using this coccinelle script :

  @rule_d1@
  typedef chunk;
  struct chunk chunk;
  @@
  - chunk.str
  + chunk.area

  @rule_d2@
  typedef chunk;
  struct chunk chunk;
  @@
  - chunk.len
  + chunk.data

  @rule_i1@
  typedef chunk;
  struct chunk *chunk;
  @@
  - chunk->str
  + chunk->area

  @rule_i2@
  typedef chunk;
  struct chunk *chunk;
  @@
  - chunk->len
  + chunk->data

Some minor updates to 3 http functions had to be performed to take size_t
ints instead of ints in order to match the unsigned length here.
2018-07-19 16:23:43 +02:00
Olivier Houchard
acd1403794 MINOR: buffer: Use b_add()/bo_add() instead of accessing b->i/b->o.
Use the newly available functions instead of using the buffer fields directly.
2018-07-19 16:23:42 +02:00
Willy Tarreau
591d445049 MINOR: buffer: use b_orig() to replace most references to b->data
This patch updates most users of b->data to use b_orig().
2018-07-19 16:23:42 +02:00
Willy Tarreau
0c7ed5d264 MINOR: buffer: replace buffer_empty() with b_empty() or c_empty()
For the same consistency reasons, let's use b_empty() at the few places
where an empty buffer is expected, or c_empty() if it's done on a channel.
Some of these places were there to realign the buffer so
{b,c}_realign_if_empty() was used instead.
2018-07-19 16:23:41 +02:00
Willy Tarreau
7f3225f251 MINOR: connection: add a flags argument to rcv_buf()
The mux and transport rcv_buf() now takes a "flags" argument, just like
the snd_buf() one or like the equivalent syscall lower part. The upper
layers will use this to pass some information such as indicating whether
the buffer is free from outgoing data or if the lower layer may allocate
the buffer itself.
2018-07-19 16:23:41 +02:00
Willy Tarreau
bfc4d77ad3 MEDIUM: connection: make xprt->rcv_buf() use size_t for the count
Just like we have a size_t for xprt->snd_buf(), we adjust to use size_t
for rcv_buf()'s count argument and return value. It also removes the
ambiguity related to the possibility to see a negative value there.
2018-07-19 16:23:41 +02:00
Willy Tarreau
787db9a6a4 MEDIUM: connection: make xprt->snd_buf() take the byte count in argument
This way the senders don't need to modify the buffer's metadata anymore
nor to know about the output's split point. This way the functions can
take a const buffer and it's clearer who's in charge of updating the
buffer after a send. That's why the buffer realignment is now performed
by the caller of the transport's snd_buf() functions.

The return type was updated to return a size_t to comply with the count
argument.
2018-07-19 16:23:41 +02:00
Willy Tarreau
0e11d59af6 MINOR: buffer: remove bo_contig_data()
The two call places now make use of b_contig_data(0) and check by
themselves that the returned size is no larger than the scheduled
output data.
2018-07-19 16:23:40 +02:00
Willy Tarreau
8f9c72d301 MINOR: buffer: remove bi_end()
It was replaced by ci_tail() when the channel is known, or b_tail() in
other cases.
2018-07-19 16:23:40 +02:00
Willy Tarreau
89faf5d7c3 MINOR: buffer: remove bo_ptr()
It was replaced by co_head() when a channel was known, otherwise b_head().
2018-07-19 16:23:40 +02:00
Willy Tarreau
17b4aa1adc BUG/MINOR: ssl: properly ref-count the tls_keys entries
Commit 200b0fa ("MEDIUM: Add support for updating TLS ticket keys via
socket") introduced support for updating TLS ticket keys from the CLI,
but missed a small corner case : if multiple bind lines reference the
same tls_keys file, the same reference is used (as expected), but during
the clean shutdown, it will lead to a double free when destroying the
bind_conf contexts since none of the lines knows if others still use
it. The impact is very low however, mostly a core and/or a message in
the system's log upon old process termination.

Let's introduce some basic refcounting to prevent this from happening,
so that only the last bind_conf frees it.

Thanks to Janusz Dziemidowicz and Thierry Fournier for both reporting
the same issue with an easy reproducer.

This fix needs to be backported from 1.6 to 1.8.
2018-07-18 08:59:50 +02:00
Emmanuel Hocdet
3448c490ca BUG/MEDIUM: ssl: do not store pkinfo with SSL_set_ex_data
Bug from 96b7834e: pkinfo is stored on SSL_CTX ex_data and should
not be also stored on SSL ex_data without reservation.
Simply extract pkinfo from SSL_CTX in ssl_sock_get_pkey_algo.

No backport needed.
2018-06-18 13:34:09 +02:00
Thierry FOURNIER
28962c9941 BUG/MAJOR: ssl: OpenSSL context is stored in non-reserved memory slot
We never saw unexplicated crash with SSL, so I suppose that we are
luck, or the slot 0 is always reserved. Anyway the usage of the macro
SSL_get_app_data() and SSL_set_app_data() seem wrong. This patch change
the deprecated functions SSL_get_app_data() and SSL_set_app_data()
by the new functions SSL_get_ex_data() and SSL_set_ex_data(), and
it reserves the slot in the SSL memory space.

For information, this is the two declaration which seems wrong or
incomplete in the OpenSSL ssl.h file. We can see the usage of the
slot 0 whoch is hardcoded, but never reserved.

   #define SSL_set_app_data(s,arg)     (SSL_set_ex_data(s,0,(char *)arg))
   #define SSL_get_app_data(s)      (SSL_get_ex_data(s,0))

This patch must be backported at least in 1.8, maybe in other versions.
2018-06-18 10:32:14 +02:00
Thierry FOURNIER
16ff050478 BUG/MAJOR: ssl: Random crash with cipherlist capture
The cipher list capture struct is stored in the SSL memory space,
but the slot is reserved in the SSL_CTX memory space. This causes
ramdom crashes.

This patch should be backported to 1.8
2018-06-18 10:32:12 +02:00
Lukas Tribus
926594f606 MINOR: ssl: set SSL_OP_PRIORITIZE_CHACHA
Sets OpenSSL 1.1.1's SSL_OP_PRIORITIZE_CHACHA unconditionally, as per [1]:

When SSL_OP_CIPHER_SERVER_PREFERENCE is set, temporarily reprioritize
ChaCha20-Poly1305 ciphers to the top of the server cipher list if a
ChaCha20-Poly1305 cipher is at the top of the client cipher list. This
helps those clients (e.g. mobile) use ChaCha20-Poly1305 if that cipher
is anywhere in the server cipher list; but still allows other clients to
use AES and other ciphers. Requires SSL_OP_CIPHER_SERVER_PREFERENCE.

[1] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_clear_options.html
2018-05-23 16:55:15 +02:00
Willy Tarreau
03f4ec47d9 BUG/MEDIUM: ssl: properly protect SSL cert generation
Commit 821bb9b ("MAJOR: threads/ssl: Make SSL part thread-safe") added
insufficient locking to the cert lookup and generation code : it uses
lru64_lookup(), which will automatically remove and add a list element
to the LRU list. It cannot be simply read-locked.

A long-term improvement should consist in using a lockless mechanism
in lru64_lookup() to safely move the list element at the head. For now
let's simply use a write lock during the lookup. The effect will be
minimal since it's used only in conjunction with automatically generated
certificates, which are much more expensive and rarely used.

This fix must be backported to 1.8.
2018-05-17 10:56:47 +02:00
Patrick Hemmer
e027547f8d MINOR: ssl: add fetch 'ssl_fc_session_key' and 'ssl_bc_session_key'
These fetches return the SSL master key of the front/back connection.
This is useful to decrypt traffic encrypted with ephemeral ciphers.
2018-04-30 14:56:19 +02:00
Patrick Hemmer
419667746b MINOR: ssl: disable SSL sample fetches when unsupported
Previously these fetches would return empty results when HAProxy was
compiled
without the requisite SSL support. This results in confusion and problem
reports from people who unexpectedly encounter the behavior.
2018-04-30 14:56:19 +02:00
Aurélien Nephtali
1e0867cfbc MINOR: ssl: Add payload support to "set ssl ocsp-response"
It is now possible to use a payload with the "set ssl ocsp-response"
command.  These syntaxes will work the same way:

 # echo "set ssl ocsp-response $(base64 -w 10000 ocsp.der)" | \
     socat /tmp/sock1 -

 # echo -e "set ssl ocsp-response <<\n$(base64 ocsp.der)\n" | \
     socat /tmp/sock1 -

Signed-off-by: Aurélien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-26 14:20:09 +02:00
Aurélien Nephtali
abbf607105 MEDIUM: cli: Add payload support
In order to use arbitrary data in the CLI (multiple lines or group of words
that must be considered as a whole, for example), it is now possible to add a
payload to the commands. To do so, the first line needs to end with a special
pattern: <<\n. Everything that follows will be left untouched by the CLI parser
and will be passed to the commands parsers.

Per-command support will need to be added to take advantage of this
feature.

Signed-off-by: Aurélien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-26 14:19:33 +02:00
Aurélien Nephtali
9a4da683a6 MINOR: cli: Ensure the CLI always outputs an error when it should
When using the CLI_ST_PRINT_FREE state, always output something back
if the faulty function did not fill the 'err' variable.
The map/acl code could lead to a crash whereas the SSL code was silently
failing.

Signed-off-by: Aurélien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-16 19:23:16 +02:00
Bernard Spil
13c53f8cc2 BUILD: ssl: Fix build with OpenSSL without NPN capability
OpenSSL can be built without NEXTPROTONEG support by passing
-no-npn to the configure script. This sets the
OPENSSL_NO_NEXTPROTONEG flag in opensslconf.h

Since NEXTPROTONEG is now considered deprecated, it is superseeded
by ALPN (Application Layer Protocol Next), HAProxy should allow
building withough NPN support.
2018-03-19 12:43:15 +01:00
Aurélien Nephtali
6e8a41d8fc BUG/MINOR: cli: Ensure all command outputs end with a LF
Since 200b0fac ("MEDIUM: Add support for updating TLS ticket keys via
socket"), 4147b2ef ("MEDIUM: ssl: basic OCSP stapling support."),
4df59e9 ("MINOR: cli: add socket commands and config to prepend
informational messages with severity") and 654694e1 ("MEDIUM: stats/cli:
add support for "set table key" to enter values"), commands
'set ssl tls-key', 'set ssl ocsp-response', 'set severity-output' and
'set table' do not always send an extra LF at the end of their outputs.

This is required as mentioned in doc/management.txt:

"Since multiple commands may be issued at once, haproxy uses the empty
line as a delimiter to mark an end of output for each command"

Signed-off-by: Aurélien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-03-19 12:13:02 +01:00
Aurélien Nephtali
76de95a4c0 CLEANUP: ssl: Remove a duplicated #include
openssl/x509.h is included twice since commit fc0421fde ("MEDIUM: ssl:
add support for SNI and wildcard certificates").

Signed-off-by: Aurélien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-03-12 07:49:01 +01:00
Emmanuel Hocdet
253c3b7516 MINOR: connection: add proxy-v2-options authority
This patch add option PP2_TYPE_AUTHORITY to proxy protocol v2 when a TLS
connection was negotiated. In this case, authority corresponds to the sni.
2018-03-01 11:38:32 +01:00
Emmanuel Hocdet
283e004a85 MINOR: ssl: add ssl_sock_get_cert_sig function
ssl_sock_get_cert_sig can be used to report cert signature short name
to log and ppv2 (RSA-SHA256).
2018-03-01 11:34:08 +01:00
Emmanuel Hocdet
96b7834e98 MINOR: ssl: add ssl_sock_get_pkey_algo function
ssl_sock_get_pkey_algo can be used to report pkey algorithm to log
and ppv2 (RSA2048, EC256,...).
Extract pkey information is not free in ssl api (lock/alloc/free):
haproxy can use the pkey information computed in load_certificate.
Store and use this information in a SSL ex_data when available,
compute it if not (SSL multicert bundled and generated cert).
2018-03-01 11:34:05 +01:00
Emmanuel Hocdet
ddc090bc55 MINOR: ssl: extract full pkey info in load_certificate
Private key information is used in switchctx to implement native multicert
selection (ecdsa/rsa/anonymous). This patch extract and store full pkey
information: dsa type and pkey size in bits. This can be used for switchctx
or to report pkey informations in ppv2 and log.
2018-03-01 11:33:18 +01:00
Emeric Brun
74f7ffa229 MINOR: ssl/sample: adds ssl_bc_is_resumed fetch keyword.
Returns true when the back connection was made over an SSL/TLS transport
layer and the newly created SSL session was resumed using a cached
session or a TLS ticket.
2018-02-19 16:50:20 +01:00
Emeric Brun
eb8def9f34 BUG/MEDIUM: ssl/sample: ssl_bc_* fetch keywords are broken.
Since the split between connections and conn-stream objects, this
keywords are broken.

This patch must be backported in 1.8
2018-02-19 16:50:05 +01:00
Christopher Faulet
4ac77a98cd BUG/MEDIUM: ssl: Shutdown the connection for reading on SSL_ERROR_SYSCALL
When SSL_read returns SSL_ERROR_SYSCALL and errno is unset or set to EAGAIN, the
connection must be shut down for reading. Else, the connection loops infinitly,
consuming all the CPU.

The bug was introduced in the commit 7e2e50500 ("BUG/MEDIUM: ssl: Don't always
treat SSL_ERROR_SYSCALL as unrecovarable."). This patch must be backported in
1.8 too.
2018-02-19 15:37:47 +01:00
Christopher Faulet
16f45c87d5 BUG/MINOR: ssl/threads: Make management of the TLS ticket keys files thread-safe
A TLS ticket keys file can be updated on the CLI and used in same time. So we
need to protect it to be sure all accesses are thread-safe. Because updates are
infrequent, a R/W lock has been used.

This patch must be backported in 1.8
2018-02-19 14:15:38 +01:00
Olivier Houchard
7e2e505006 BUG/MEDIUM: ssl: Don't always treat SSL_ERROR_SYSCALL as unrecovarable.
Bart Geesink reported some random errors appearing under the form of
termination flags SD in the logs for connections involving SSL traffic
to reach the servers.

Tomek Gacek and Mateusz Malek finally narrowed down the problem to commit
c2aae74 ("MEDIUM: ssl: Handle early data with OpenSSL 1.1.1"). It happens
that the special case of SSL_ERROR_SYSCALL isn't handled anymore since
this commit.

SSL_read() might return <= 0, and SSL_get_erro() return SSL_ERROR_SYSCALL,
without meaning the connection is gone. Before flagging the connection
as in error, check the errno value.

This should be backported to 1.8.
2018-02-14 18:44:28 +01:00
Olivier Houchard
25ae45a078 MINOR: early data: Never remove the CO_FL_EARLY_DATA flag.
It may be useful to keep the CO_FL_EARLY_DATA flag, so that we know early
data were used, so instead of doing this, only add the Early-data header,
and have the sample fetch ssl_fc_has_early return 1, if CO_FL_EARLY_DATA is
set, and if the handshake isn't done yet.
2018-02-05 14:24:50 +01:00
Olivier Houchard
6fa63d9852 MINOR: early data: Don't rely on CO_FL_EARLY_DATA to wake up streams.
Instead of looking for CO_FL_EARLY_DATA to know if we have to try to wake
up a stream, because it is waiting for a SSL handshake, instead add a new
conn_stream flag, CS_FL_WAIT_FOR_HS. This way we don't have to rely on
CO_FL_EARLY_DATA, and we will only wake streams that are actually waiting.
2018-02-05 14:24:50 +01:00
Willy Tarreau
a9786b6f04 MINOR: fd: pass the iocb and owner to fd_insert()
fd_insert() is currently called just after setting the owner and iocb,
but proceeding like this prevents the operation from being atomic and
requires a lock to protect the maxfd computation in another thread from
meeting an incompletely initialized FD and computing a wrong maxfd.
Fortunately for now all fdtab[].owner are set before calling fd_insert(),
and the first lock in fd_insert() enforces a memory barrier so the code
is safe.

This patch moves the initialization of the owner and iocb to fd_insert()
so that the function will be able to properly arrange its operations and
remain safe even when modified to become lockless. There's no other change
beyond the internal API.
2018-01-29 16:07:25 +01:00
William Lallemand
99b90af621 BUG/MEDIUM: ssl: cache doesn't release shctx blocks
Since the rework of the shctx with the hot list system, the ssl cache
was putting session inside the hot list, without removing them.
Once all block were used, they were all locked in the hot list, which
was forbiding to reuse them for new sessions.

Bug introduced by 4f45bb9 ("MEDIUM: shctx: separate ssl and shctx")

Thanks to Jeffrey J. Persch for reporting this bug.

Must be backported to 1.8.
2018-01-05 11:46:54 +01:00
Willy Tarreau
5d4cafb610 BUILD: ssl: silence a warning when building without NPN nor ALPN support
When building with a library not offering any of these, ssl_conf_cur
is not used.

Can be backported to 1.8.
2018-01-04 19:04:08 +01:00
Emeric Brun
ece0c334bd BUG/MEDIUM: ssl engines: Fix async engines fds were not considered to fix fd limit automatically.
The number of async fd is computed considering the maxconn, the number
of sides using ssl and the number of engines using async mode.

This patch should be backported on haproxy 1.8
2017-12-06 14:17:41 +01:00
William Lallemand
c61c0b371b BUG/MINOR: ssl: support tune.ssl.cachesize 0 again
Since the split of the shctx and the ssl cache, we lost the ability to
disable the cache with tune.ssl.cachesize 0.

Worst than that, when using this configuration, haproxy segfaults during
the configuration parsing.

Must be backported to 1.8.
2017-12-04 18:48:26 +01:00
Emmanuel Hocdet
cebd7962e2 BUG/MINOR: ssl: CO_FL_EARLY_DATA removal is managed by stream
Manage BoringSSL early_data as it is with openssl 1.1.1.
2017-11-29 14:34:47 +01:00
William Lallemand
c3cd35f96c BUG/MEDIUM: ssl: don't allocate shctx several time
The shctx_init() function does not check anymore if the pointer is not
NULL, this check must be done is the caller.

The consequence was to allocate one shctx per ssl bind.

Bug introduced by 4f45bb9 ("MEDIUM: shctx: separate ssl and shctx")

Thanks to Maciej Zdeb for reporting this bug.

Must be backported to 1.8.
2017-11-28 12:04:16 +01:00
Willy Tarreau
bafbe01028 CLEANUP: pools: rename all pool functions and pointers to remove this "2"
During the migration to the second version of the pools, the new
functions and pool pointers were all called "pool_something2()" and
"pool2_something". Now there's no more pool v1 code and it's a real
pain to still have to deal with this. Let's clean this up now by
removing the "2" everywhere, and by renaming the pool heads
"pool_head_something".
2017-11-24 17:49:53 +01:00
Christopher Faulet
767a84bcc0 CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
Emmanuel Hocdet
ca6a957c5d MINOR: ssl: Handle early data with BoringSSL
BoringSSL early data differ from OpenSSL 1.1.1 implementation. When early
handshake is done, SSL_in_early_data report if SSL_read will be done on early
data. CO_FL_EARLY_SSL_HS and CO_FL_EARLY_DATA can be adjust accordingly.
2017-11-24 13:50:02 +01:00
Olivier Houchard
90084a133d MINOR: ssl: Handle reading early data after writing better.
It can happen that we want to read early data, write some, and then continue
reading them.
To do so, we can't reuse tmp_early_data to store the amount of data sent,
so introduce a new member.
If we read early data, then ssl_sock_to_buf() is now the only responsible
for getting back to the handshake, to make sure we don't miss any early data.
2017-11-23 19:35:28 +01:00
Christopher Faulet
ff3a41eb3f BUG/MINOR: Use crt_base instead of ca_base when crt is parsed on a server line
In srv_parse_crt, crt_base was checked but ca_base was used to build the
certifacte path.

This patch must be backported in 1.7, 1.6 and 1.5.
2017-11-23 16:34:10 +01:00
Olivier Houchard
424ecfb33c MINOR: ssl: Don't disable early data handling if we could not write.
If we can't write early data, for some reason, don't give up on reading them,
they may still be early data to be read, and if we don't do so, openssl
internal states might be inconsistent, and the handshake will fail.
2017-11-22 19:27:14 +01:00
Olivier Houchard
777e4b98a3 BUG/MINOR: ssl: Always start the handshake if we can't send early data.
The current code only tries to do the handshake in case we can't send early
data if we're acting as a client, which is wrong, it has to be done on the
server side too, or we end up in an infinite loop.
2017-11-22 19:27:09 +01:00
Olivier Houchard
e9bed53486 MINOR: ssl: Make sure we don't shutw the connection before the handshake.
Instead of trying to finish the handshake in ssl_sock_shutw, which may
fail, try not to shutdown until the handshake is finished.
2017-11-16 19:04:10 +01:00
Olivier Houchard
e6060c5d87 MINOR: SSL: Store the ASN1 representation of client sessions.
Instead of storing the SSL_SESSION pointer directly in the struct server,
store the ASN1 representation, otherwise, session resumption is broken with
TLS 1.3, when multiple outgoing connections want to use the same session.
2017-11-16 19:03:32 +01:00
Christopher Faulet
9dcf9b6f03 MINOR: threads: Use __decl_hathreads to declare locks
This macro should be used to declare variables or struct members depending on
the USE_THREAD compile option. It avoids the encapsulation of such declarations
between #ifdef/#endif. It is used to declare all lock variables.
2017-11-13 11:38:17 +01:00
Olivier Houchard
522eea7110 MINOR: ssl: Handle sending early data to server.
This adds a new keyword on the "server" line, "allow-0rtt", if set, we'll try
to send early data to the server, as long as the client sent early data, as
in case the server rejects the early data, we no longer have them, and can't
resend them, so the only option we have is to send back a 425, and we need
to be sure the client knows how to interpret it correctly.
2017-11-08 14:11:10 +01:00
Olivier Houchard
cfdef2e312 MINOR: ssl: Spell 0x10101000L correctly.
Issue added in 1.8-dev by c2aae74 ("MEDIUM: ssl: Handle early data with
OpenSSL 1.1.1"), no impact on older versions.
2017-11-08 14:10:02 +01:00
Olivier Houchard
bd84ac8737 MINOR: ssl: Handle session resumption with TLS 1.3
With TLS 1.3, session aren't established until after the main handshake
has completed. So we can't just rely on calling SSL_get1_session(). Instead,
we now register a callback for the "new session" event. This should work for
previous versions of TLS as well.
2017-11-08 14:08:07 +01:00
Olivier Houchard
35a63cc1c7 BUG/MINOR; ssl: Don't assume we have a ssl_bind_conf because a SNI is matched.
We only have a ssl_bind_conf if crt-list is used, however we can still
match a certificate SNI, so don't assume we have a ssl_bind_conf.
2017-11-08 14:08:07 +01:00
Christopher Faulet
2a944ee16b BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix
This remove any name conflicts, especially on Solaris.
2017-11-07 11:10:24 +01:00
Olivier Houchard
79a481ddde MINOR: ssl: Remove the global allow-0rtt option. 2017-10-31 15:48:42 +01:00
Emeric Brun
821bb9beaa MAJOR: threads/ssl: Make SSL part thread-safe
First, OpenSSL is now initialized to be thread-safe. This is done by setting 2
callbacks. The first one is ssl_locking_function. It handles the locks and
unlocks. The second one is ssl_id_function. It returns the current thread
id. During the init step, we create as much as R/W locks as needed, ie the
number returned by CRYPTO_num_locks function.

Next, The reusable SSL session in the server context is now thread-local.

Shctx is now also initialized if HAProxy is started with several threads.

And finally, a global lock has been added to protect the LRU cache used to store
generated certificates. The function ssl_sock_get_generated_cert is now
deprecated because the retrieved certificate can be removed by another threads
in same time. Instead, a new function has been added,
ssl_sock_assign_generated_cert. It must be used to search a certificate in the
cache and set it immediatly if found.
2017-10-31 13:58:32 +01:00
Christopher Faulet
8d8aa0d681 MEDIUM: threads/listeners: Make listeners thread-safe
First, we use atomic operations to update jobs/totalconn/actconn variables,
listener's nbconn variable and listener's counters. Then we add a lock on
listeners to protect access to their information. And finally, listener queues
(global and per proxy) are also protected by a lock. Here, because access to
these queues are unusal, we use the same lock for all queues instead of a global
one for the global queue and a lock per proxy for others.
2017-10-31 13:58:30 +01:00
Christopher Faulet
36716a7fec MEDIUM: threads/fd: Initialize the process mask during the call to fd_insert
Listeners will allow any threads to process the corresponding fd. But for other
FDs, we limit the processing to the current thread.
2017-10-31 13:58:30 +01:00
Christopher Faulet
b349e48ede MEDIUM: threads/pool: Make pool thread-safe by locking all access to a pool
A lock has been added for each memory pool. It is used to protect the pool
during allocations and releases. It is also used when pool info are dumped.
2017-10-31 13:58:30 +01:00
William Lallemand
a3c77cfdd7 MINOR: shctx: rename lock functions
Rename lock functions to shctx_lock() and shctx_unlock() to be coherent
with the new API.
2017-10-31 03:49:44 +01:00
William Lallemand
4f45bb9c46 MEDIUM: shctx: separate ssl and shctx
This patch reorganize the shctx API in a generic storage API, separating
the shared SSL session handling from its core.

The shctx API only handles the generic data part, it does not know what
kind of data you use with it.

A shared_context is a storage structure allocated in a shared memory,
allowing its usage in a multithread or a multiprocess context.

The structure use 2 linked list, one containing the available blocks,
and another for the hot locked blocks. At initialization the available
list is filled with <maxblocks> blocks of size <blocksize>. An <extra>
space is initialized outside the list in case you need some specific
storage.

+-----------------------+--------+--------+--------+--------+----
| struct shared_context | extra  | block1 | block2 | block3 | ...
+-----------------------+--------+--------+--------+--------+----
                                 <--------  maxblocks  --------->
                                            * blocksize

The API allows to store content on several linked blocks. For example,
if you allocated blocks of 16 bytes, and you want to store an object of
60 bytes, the object will be allocated in a row of 4 blocks.

The API was made for LRU usage, each time you get an object, it pushes
the object at the end of the list. When it needs more space, it discards

The functions name have been renamed in a more logical way, the part
regarding shctx have been prefixed by shctx_ and the functions for the
shared ssl session cache have been prefixed by sh_ssl_sess_.
2017-10-31 03:49:40 +01:00
William Lallemand
ed0b5ad1aa REORG: shctx: move ssl functions to ssl_sock.c
Move the ssl callback functions of the ssl shared session cache to
ssl_sock.c. The shctx functions still needs to be separated of the ssl
tree and data.
2017-10-31 03:48:39 +01:00
William Lallemand
3f85c9aec8 MEDIUM: shctx: allow the use of multiple shctx
Add an shctx argument which permits to create new independent shctx
area.
2017-10-31 03:44:11 +01:00
Emmanuel Hocdet
01da571e21 MINOR: merge ssl_sock_get calls for log and ppv2
Merge ssl_sock_get_version and ssl_sock_get_proto_version.
Change ssl_sock_get_cipher to be used in ppv2.
2017-10-27 19:32:36 +02:00
Olivier Houchard
9679ac997a MINOR: ssl: Don't abuse ssl_options.
A bind_conf does contain a ssl_bind_conf, which already has a flag to know
if early data are activated, so use that, instead of adding a new flag in
the ssl_options field.
2017-10-27 19:26:52 +02:00
Olivier Houchard
ccaa7de72e MINOR: ssl/proto_http: Add keywords to take care of early data.
Add a new sample fetch, "ssl_fc_has_early", a boolean that will be true
if early data were sent, and a new action, "wait-for-handshake", if used,
the request won't be forwarded until the SSL handshake is done.
2017-10-27 13:32:22 +02:00
Olivier Houchard
c2aae74f01 MEDIUM: ssl: Handle early data with OpenSSL 1.1.1
When compiled with Openssl >= 1.1.1, before attempting to do the handshake,
try to read any early data. If any early data is present, then we'll create
the session, read the data, and handle the request before we're doing the
handshake.

For this, we add a new connection flag, CO_FL_EARLY_SSL_HS, which is not
part of the CO_FL_HANDSHAKE set, allowing to proceed with a session even
before an SSL handshake is completed.

As early data do have security implication, we let the origin server know
the request comes from early data by adding the "Early-Data" header, as
specified in this draft from the HTTP working group :

    https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-replay
2017-10-27 10:54:05 +02:00
Emmanuel Hocdet
253c62b257 MINOR: ssl: generated certificate is missing in switchctx early callback
Openssl 1.1.1 supports switchctx early callback and generated certificate.
Generated certificate calls must be available in switchctx early callback.
2017-10-27 10:54:05 +02:00
Emmanuel Hocdet
84e417d859 MINOR: ssl: support Openssl 1.1.1 early callback for switchctx
Use Openssl-1.1.1 SSL_CTX_set_client_hello_cb to mimic BoringSSL early callback.
Native multi certificate and SSL/TLS method per certificate is now supported by
Openssl >= 1.1.1.
2017-10-27 10:54:05 +02:00
Emmanuel Hocdet
48e8755639 MEDIUM: ssl: convert CBS (BoringSSL api) usage to neutral code
switchctx early callback is only supported for BoringSSL. To prepare
the support of openssl 1.1.1 early callback, convert CBS api to neutral
code to work with any ssl libs.
2017-10-27 10:54:05 +02:00
Willy Tarreau
31d4dbe825 MINOR: ssl_sock: make use of CO_FL_WILL_UPDATE
Now when ssl_sock_{to,from}_buf are called, if the connection doesn't
feature CO_FL_WILL_UPDATE, they will first retrieve the updated flags
using conn_refresh_polling_flags() before changing any flag, then call
conn_cond_update_sock_polling() before leaving, to commit such changes.
2017-10-25 15:52:41 +02:00
Willy Tarreau
8de70bcb54 MINOR: ssl: don't abort after sending 16kB
SSL records are 16kB max. When trying to send larger data chunks at once,
SSL_read() only processes 16kB and ssl_sock_from_buf() believes it means
the system buffers are full, which is not the case, contrary to raw_sock.
This is particularly noticeable with HTTP/2 when using a 64kB buffer with
multiple streams, as the mux buffer can start to fill up pretty quickly
in this situation, slowing down the data delivery.
2017-10-25 15:52:41 +02:00
Lukas Tribus
8c6a3647dd CLEANUP: cli: remove undocumented "set ssl tls-keys" command
The documented "set ssl tls-key" command must be used instead.

This is for 1.8 only.
2017-10-25 04:11:27 +02:00
Lukas Tribus
f4bbc438fc BUG/MINOR: cli: restore "set ssl tls-key" command
in 32af203b75 ("REORG: cli: move ssl CLI functions to ssl_sock.c")
"set ssl tls-key" was accidentally replaced with "set ssl tls-keys"
(keys instead of key). This is undocumented and breaks upgrades from
1.6 to 1.7.

This patch restores "set ssl tls-key" and also registers a helptext.

This should be backported to 1.7.
2017-10-25 04:11:27 +02:00
Emmanuel Hocdet
ef60705075 BUG/MINOR: ssl: OCSP_single_get0_status can return -1
Commit 872085ce "BUG/MINOR: ssl: ocsp response with 'revoked' status is correct"
introduce a regression. OCSP_single_get0_status can return -1 and haproxy must
generate an error in this case.
Thanks to Sander Hoentjen who have spotted the regression.

This patch should be backported in 1.7, 1.6 and 1.5 if the patch above is
backported.
2017-10-24 20:02:46 +02:00
Emmanuel Hocdet
019f9b10ef MINOR: ssl: build with recent BoringSSL library
BoringSSL switch OPENSSL_VERSION_NUMBER to 1.1.0 for compatibility.
Fix BoringSSL call and openssl-compat.h/#define occordingly.
This will not break openssl/libressl compat.
2017-10-24 19:57:16 +02:00
Emmanuel Hocdet
e966e4e451 BUILD: ssl: support OPENSSL_NO_ASYNC #define
Support build without ASYNC support. This #define is set per default in
BoringSSL.
2017-10-24 19:57:16 +02:00
Emmanuel Hocdet
872085ce37 BUG/MINOR: ssl: ocsp response with 'revoked' status is correct
ocsp_status can be 'good', 'revoked', or 'unknown'. 'revoked' status
is a correct status and should not be dropped.
In case of certificate with OCSP must-stapling extension, response with
'revoked' status must be provided as well as 'good' status.

This patch can be backported in 1.7, 1.6 and 1.5.
2017-10-22 10:01:26 +02:00
Willy Tarreau
06d80a9a9c REORG: channel: finally rename the last bi_* / bo_* functions
For HTTP/2 we'll need some buffer-only equivalent functions to some of
the ones applying to channels and still squatting the bi_* / bo_*
namespace. Since these names have kept being misleading for quite some
time now and are really getting annoying, it's time to rename them. This
commit will use "ci/co" as the prefix (for "channel in", "channel out")
instead of "bi/bo". The following ones were renamed :

  bi_getblk_nc, bi_getline_nc, bi_putblk, bi_putchr,
  bo_getblk, bo_getblk_nc, bo_getline, bo_getline_nc, bo_inject,
  bi_putchk, bi_putstr, bo_getchr, bo_skip, bi_swpbuf
2017-10-19 15:01:08 +02:00
Olivier Houchard
9130a9605d MINOR: checks: Add a new keyword to specify a SNI when doing SSL checks.
Add a new keyword, "check-sni", to be able to specify the SNI to be used when
doing health checks over SSL.
2017-10-17 18:10:24 +02:00
Frederik Deweerdt
953917abc9 BUG/MEDIUM: ssl: fix OCSP expiry calculation
The hour part of the timezone offset was multiplied by 60 instead of
3600, resulting in an inaccurate expiry. This bug was introduced in
1.6-dev1 by commit 4f3c87a ("BUG/MEDIUM: ssl: Fix to not serve expired
OCSP responses."), so this fix must be backported into 1.7 and 1.6.
2017-10-16 18:14:36 +02:00
Christopher Faulet
f8bb0ce450 MINOR: ssl: Remove useless checks on bind_conf or bind_conf->is_ssl
bind_conf always exists at these steps and it is always for SSL listeners.
2017-09-15 18:42:23 +02:00
Andjelko Iharos
c3680ecdf8 MINOR: add severity information to cli feedback messages 2017-09-13 13:38:32 +02:00
Emmanuel Hocdet
ddcde195eb MINOR: ssl: rework smp_fetch_ssl_fc_cl_str without internal ssl use
smp_fetch_ssl_fc_cl_str as very limited usage (only work with openssl == 1.0.2
compiled with the option enable-ssl-trace). It use internal cipher.algorithm_ssl
attribut and SSL_CIPHER_standard_name (available with ssl-trace).
This patch implement this (debug) function in a standard way. It used common
SSL_CIPHER_get_name to display cipher name. It work with openssl >= 1.0.2
and boringssl.
2017-09-09 08:36:22 +02:00
Emmanuel Hocdet
4366476852 MINOR: ssl: remove duplicate ssl_methods in struct bind_conf
Patch "MINOR: ssl: support ssl-min-ver and ssl-max-ver with crt-list"
introduce ssl_methods in struct ssl_bind_conf. struct bind_conf have now
ssl_methods and ssl_conf.ssl_methods (unused). It's error-prone. This patch
remove the duplicate structure to avoid any confusion.
2017-09-05 09:42:30 +02:00
Willy Tarreau
585744bf2e REORG/MEDIUM: connection: introduce the notion of connection handle
Till now connections used to rely exclusively on file descriptors. It
was planned in the past that alternative solutions would be implemented,
leading to member "union t" presenting sock.fd only for now.

With QUIC, the connection will need to continue to exist but will not
rely on a file descriptor but a connection ID.

So this patch introduces a "connection handle" which is either a file
descriptor or a connection ID, to replace the existing "union t". We've
now removed the intermediate "struct sock" which was never used. There
is no functional change at all, though the struct connection was inflated
by 32 bits on 64-bit platforms due to alignment.
2017-08-24 19:30:04 +02:00
Emmanuel Hocdet
15969297af BUILD: ssl: replace SSL_CTX_get0_privatekey for openssl < 1.0.2
Commit 48a8332a introduce SSL_CTX_get0_privatekey in openssl-compat.h but
SSL_CTX_get0_privatekey access internal structure and can't be a candidate
to openssl-compat.h. The workaround with openssl < 1.0.2 is to use SSL_new
then SSL_get_privatekey.
2017-08-11 11:35:26 +02:00
Emmanuel Hocdet
aa0d637292 MINOR: ssl: allow to start without certificate if strict-sni is set
With strict-sni, ssl connection will fail if no certificate match. Have no
certificate in bind line, fail on all ssl connections. It's ok with the
behavior of strict-sni. When 'generate-certificates' is set 'strict-sni' is
never used. When 'strict-sni' is set, default_ctx is never used. Allow to start
without certificate only in this case.

Use case is to start haproxy with ssl before customer start to use certificates.
Typically with 'crt' on a empty directory and 'strict-sni' parameters.
2017-08-09 16:30:28 +02:00
Christopher Faulet
48a8332a4a BUG/MEDIUM: ssl: Fix regression about certificates generation
Since the commit f6b37c67 ["BUG/MEDIUM: ssl: in bind line, ssl-options after
'crt' are ignored."], the certificates generation is broken.

To generate a certificate, we retrieved the private key of the default
certificate using the SSL object. But since the commit f6b37c67, the SSL object
is created with a dummy certificate (initial_ctx).

So to fix the bug, we use directly the default certificate in the bind_conf
structure. We use SSL_CTX_get0_privatekey function to do so. Because this
function does not exist for OpenSSL < 1.0.2 and for LibreSSL, it has been added
in openssl-compat.h with the right #ifdef.
2017-07-28 18:25:18 +02:00
Emmanuel Hocdet
174dfe55a0 MINOR: ssl: add "no-ca-names" parameter for bind
This option prevent to send CA names in server hello message when
ca-file is used. This parameter is also available in "crt-list".
2017-07-28 15:20:48 +02:00
Willy Tarreau
71d058c288 MINOR: ssl: add a new error codes for wrong server certificates
If a server presents an unexpected certificate to haproxy, that is, a
certificate that doesn't match the expected name as configured in
verifyhost or as requested using SNI, we want to store that precious
information. Fortunately we have access to the connection in the
verification callback so it's possible to store an error code there.

For this purpose we use CO_ER_SSL_MISMATCH_SNI (for when the cert name
didn't match the one requested using SNI) and CO_ER_SSL_MISMATCH for
when it doesn't match verifyhost.
2017-07-28 11:50:16 +02:00
Willy Tarreau
ad92a9a7be BUG/MINOR: ssl: make use of the name in SNI before verifyhost
Commit 2ab8867 ("MINOR: ssl: compare server certificate names to the SNI
on outgoing connections") introduced the ability to check server cert
names against the name provided with in the SNI, but verifyhost was kept
as a way to force the name to check against. This was a mistake, because :
  - if an SNI is used, any static hostname in verifyhost will be wrong ;
    worse, if it matches and doesn't match the SNI, the server presented
    the wrong certificate ;

  - there's no way to have a default name to check against for health
    checks anymore because the point above mandates the removal of the
    verifyhost directive

This patch reverses the ordering of the check : whenever SNI is used, the
name provided always has precedence (ie the server must always present a
certificate that matches the requested name). And if no SNI is provided,
then verifyhost is used, and will be configured to match the server's
default certificate name. This will work both when SNI is not used and
for health checks.

If the commit 2ab8867 is backported in 1.7 and/or 1.6, this one must be
backported too.
2017-07-28 11:38:41 +02:00
Christopher Faulet
96c7b8dbd2 BUG/MINOR: ssl: Fix check against SNI during server certificate verification
This patch fixes the commit 2ab8867 ("MINOR: ssl: compare server certificate
names to the SNI on outgoing connections")

When we check the certificate sent by a server, in the verify callback, we get
the SNI from the session (SSL_SESSION object). In OpenSSL, tlsext_hostname value
for this session is copied from the ssl connection (SSL object). But the copy is
done only if the "server_name" extension is found in the server hello
message. This means the server has found a certificate matching the client's
SNI.

When the server returns a default certificate not matching the client's SNI, it
doesn't set any "server_name" extension in the server hello message. So no SNI
is set on the SSL session and SSL_SESSION_get0_hostname always returns NULL.

To fix the problemn, we get the SNI directly from the SSL connection. It is
always defined with the value set by the client.

If the commit 2ab8867 is backported in 1.7 and/or 1.6, this one must be
backported too.

Note: it's worth mentionning that by making the SNI check work, we
      introduce another problem by which failed SNI checks can cause
      long connection retries on the server, and in certain cases the
      SNI value used comes from the client. So this patch series must
      not be backported until this issue is resolved.
2017-07-26 19:43:33 +02:00
Emmanuel Hocdet
f80bc24dde MINOR: ssl: remove an unecessary SSL_OP_NO_* dependancy
Use methodVersions table to display "OpenSSL library supports".
2017-07-19 14:38:14 +02:00
Emmanuel Hocdet
23877ab653 BUG/MINOR: ssl: remove haproxy SSLv3 support when ssl lib have no SSLv3
The commit 5db33cbd "MEDIUM: ssl: ssl_methods implementation is reworked and
factored for min/max tlsxx" drop the case when ssl lib have removed SSLv3.
The commit 1e59fcc5 "BUG/MINOR: ssl: Be sure that SSLv3 connection methods
exist for openssl < 1.1.0" fix build but it's false because haproxy think
that ssl lib support SSLv3.

SSL_OP_NO_* are flags to set in ssl_options and is the way haproxy do the
link between ssl capabilities and haproxy configuration. (The mapping table
is done via methodVersions). SSL_OP_NO_* is set to 0 when ssl lib doesn't
support a new TLS version. Older version (like SSLv3) can be removed at
build or unsupported (like libressl). In all case OPENSSL_NO_SSL3 is define.

To keep the same logic, this patch alter SSL_OP_NO_SSLv3 to 0 when SSLv3 is
not supported by ssl lib (when OPENSSL_NO_SSL3 is define).
2017-07-19 14:38:12 +02:00
Willy Tarreau
7784f1739c OPTIM: ssl: don't consider a small ssl_read() as an indication of end of buffer
In ssl_sock_to_buf(), when we face a small read, we used to consider it
as an indication for the end of incoming data, as is the case with plain
text. The problem is that here it's quite different, SSL records are
returned at once so doing so make us wake all the upper layers for each
and every record. Given that SSL records are 16kB by default, this is
rarely observed unless the protocol employs small records or the buffers
are increased. But with 64kB buffers while trying to deal with HTTP/2
frames, the exchanges are obviously suboptimal as there are two messages
per frame (one for the frame header and another one for the frame payload),
causing the H2 parser to be woken up half of the times without being able
to proceed :

   try=65536 ret=45
   try=65536 ret=16384
   try=49152 ret=9
   try=49143 ret=16384
   try=32759 ret=9
   try=32750 ret=16384
   try=16366 ret=9
   try=32795 ret=27
   try=49161 ret=9
   try=49152 ret=16384
   try=49116 ret=9
   try=49107 ret=16384
   try=32723 ret=9
   try=32714 ret=16384
   try=16330 ret=9
   try=32831 ret=63
   try=49161 ret=9
   try=49152 ret=16384
   try=49080 ret=9
   try=49071 ret=2181

With this change, the buffer can safely be filled with all pending frames
at once when they are available.
2017-07-11 17:22:12 +02:00
Willy Tarreau
2ab88675ec MINOR: ssl: compare server certificate names to the SNI on outgoing connections
When support for passing SNI to the server was added in 1.6-dev3, there
was no way to validate that the certificate presented by the server would
really match the name requested in the SNI, which is quite a problem as
it allows other (valid) certificates to be presented instead (when hitting
the wrong server or due to a man in the middle).

This patch adds the missing check against the value passed in the SNI.
The "verifyhost" value keeps precedence if set. If no SNI is used and
no verifyhost directive is specified, then the certificate name is not
checked (this is unchanged).

In order to extract the SNI value, it was necessary to make use of
SSL_SESSION_get0_hostname(), which appeared in openssl 1.1.0. This is
a trivial function which returns the value of s->tlsext_hostname, so
it was provided in the compat layer for older versions. After some
refinements from Emmanuel, it now builds with openssl 1.0.2, openssl
1.1.0 and boringssl. A test file was provided to ease testing all cases.

After some careful observation period it may make sense to backport
this to 1.7 and 1.6 as some users rightfully consider this limitation
as a bug.

Cc: Emmanuel Hocdet <manu@gandi.net>
Signed-off-by: Willy Tarreau <w@1wt.eu>
2017-07-06 15:15:28 +02:00
Willy Tarreau
8743f7e567 MINOR: ssl: add a get_alpn() method to ssl_sock
This is used to retrieve the TLS ALPN information from a connection. We
also support a fallback to NPN if ALPN doesn't find anything or is not
available on the existing implementation. It happens that depending on
the library version, either one or the other is available. NPN was
present in openssl 1.0.1 (very common) while ALPN is in 1.0.2 and onwards
(still uncommon at the time of writing). Clients are used to send either
one or the other to ensure a smooth transition.
2017-06-27 14:38:02 +02:00
Christopher Faulet
1e59fcc588 BUG/MINOR: ssl: Be sure that SSLv3 connection methods exist for openssl < 1.1.0
For openssl 1.0.2, SSLv3_server_method and SSLv3_client_method are undefined if
OPENSSL_NO_SSL3_METHOD is set. So we must add a check on this macro before using
these functions.
2017-06-14 16:40:38 +02:00
Emeric Brun
bbc165447e BUG/MINOR: ssl: do not call directly the conn_fd_handler from async_fd_handler
This patch modifies the way to re-enable the connection from the async fd
handler calling conn_update_sock_polling instead of the conn_fd_handler.

It also ensures that the polling is really stopped on the async fd.
2017-06-08 06:47:34 +02:00
Emeric Brun
b5e42a817b BUG/MAJOR: ssl: buffer overflow using offloaded ciphering on async engine
The Openssl's ASYNC API does'nt support moving buffers on SSL_read/write
This patch disables the ASYNC mode dynamically when the handshake
is left and re-enables it on reneg.
2017-06-08 06:47:34 +02:00
Emeric Brun
ce9e01c674 BUG/MAJOR: ssl: fix segfault on connection close using async engines.
This patch ensure that the ASYNC fd handlers won't be wake up
too early, disabling the event cache for this fd on connection close
and when a WANT_ASYNC is rised by Openssl.

The calls to SSL_read/SSL_write/SSL_do_handshake before rising a real read
event from the ASYNC fd, generated an EAGAIN followed by a context switch
for some engines, or a blocked read for the others.

On connection close it resulted in a too early call to SSL_free followed
by a segmentation fault.
2017-06-08 06:47:34 +02:00
Emmanuel Hocdet
bd695fe024 MEDIUM: ssl: disable SSLv3 per default for bind
For security, disable SSLv3 on bind line must be the default configuration.
SSLv3 can be enabled with "ssl-min-ver SSLv3".
2017-06-02 16:43:16 +02:00
Emmanuel Hocdet
df701a2adb MINOR: ssl: support ssl-min-ver and ssl-max-ver with crt-list
SSL/TLS version can be changed per certificat if and only if openssl lib support
earlier callback on handshake and, of course, is implemented in haproxy. It's ok
for BoringSSL. For Openssl, version 1.1.1 have such callback and could support it.
2017-06-02 16:42:09 +02:00
Emmanuel Hocdet
4aa615ff6b MEDIUM: ssl: ctx_set_version/ssl_set_version func for methodVersions table
This patch cleanup the usage of set_version func with a more suitable name:
ctx_set_version. It introduce ssl_set_version func (unused for the moment).
2017-06-02 16:41:57 +02:00
Emmanuel Hocdet
ecb0e234b9 REORG: ssl: move defines and methodVersions table upper
It will used in ssl_sock_switchctx_cbk.
2017-06-02 16:41:36 +02:00
Emmanuel Hocdet
9ac143b607 BUILD: ssl: fix build with OPENSSL_NO_ENGINE
Build is broken with openssl library without support of engin (like boringssl).
Add OPENSSL_NO_ENGINE flag to fix that.
2017-06-02 12:07:48 +02:00
Emmanuel Hocdet
2c32d8f379 MINOR: boringssl: basic support for OCSP Stapling
Use boringssl SSL_CTX_set_ocsp_response to set OCSP response from file with
'.ocsp' extension. CLI update is not supported.
2017-05-27 07:59:34 +02:00
Emeric Brun
3854e0102b MEDIUM: ssl: handle multiple async engines
This patch adds the support of a maximum of 32 engines
in async mode.

Some tests have been done using 2 engines simultaneously.

This patch also removes specific 'async' attribute from the connection
structure. All the code relies only on Openssl functions.
2017-05-27 07:12:27 +02:00
Grant Zhang
fa6c7ee702 MAJOR: ssl: add openssl async mode support
ssl-mode-async is a global configuration parameter which enables
asynchronous processing in OPENSSL for all SSL connections haproxy
handles. With SSL_MODE_ASYNC set, TLS I/O operations may indicate a
retry with SSL_ERROR_WANT_ASYNC with this mode set if an asynchronous
capable engine is used to perform cryptographic operations. Currently
async mode only supports one async-capable engine.

This is the latest version of the patchset which includes Emeric's
updates :
  - improved async fd cleaning when openssl reports an fd to delete
  - prevent conn_fd_handler from calling SSL_{read,write,handshake} until
    the async fd is ready, as these operations are very slow and waste CPU
  - postpone of SSL_free to ensure the async operation can complete and
    does not cause a dereference a released SSL.
  - proper removal of async fd from the fdtab and removal of the unused async
    flag.
2017-05-27 07:05:54 +02:00
Grant Zhang
872f9c2139 MEDIUM: ssl: add basic support for OpenSSL crypto engine
This patch adds the global 'ssl-engine' keyword. First arg is an engine
identifier followed by a list of default_algorithms the engine will
operate.

If the openssl version is too old, an error is reported when the option
is used.
2017-05-27 07:05:00 +02:00
Emmanuel Hocdet
abd323395f MEDIUM: ssl: ssl-min-ver and ssl-max-ver compatibility.
In haproxy < 1.8, no-sslv3/no-tlsv1x are ignored when force-sslv3/force-tlsv1x
is used (without warning). With this patch, no-sslv3/no-tlsv1x are ignored when
ssl-min-ver or ssl-max-ver is used (with warning).
When all SSL/TLS versions are disable: generate an error, not a warning.
example: ssl-min-ver TLSV1.3 (or force-tlsv13) with a openssl <= 1.1.0.
2017-05-12 15:49:05 +02:00
Emmanuel Hocdet
e1c722b5e8 MEDIUM: ssl: add ssl-min-ver and ssl-max-ver parameters for bind and server
'ssl-min-ver' and 'ssl-max-ver' with argument SSLv3, TLSv1.0, TLSv1.1, TLSv1.2
or TLSv1.3 limit the SSL negotiation version to a continuous range. ssl-min-ver
and ssl-max-ver should be used in replacement of no-tls* and no-sslv3. Warning
and documentation are set accordingly.
2017-05-12 15:49:05 +02:00
Emmanuel Hocdet
50e25e1dbc MINOR: ssl: show methods supported by openssl
TLS v1.3 incoming, SSLv3 will disappears: it could be useful to list
all methods supported by haproxy/openssl (with -vvv).
2017-05-12 15:49:05 +02:00
Emmanuel Hocdet
42fb980e53 MINOR: ssl: support TLSv1.3 for bind and server
This patch add 'no-tlsv13' and 'force-tlsv13' configuration. This is
only useful with openssl-dev and boringssl.
2017-05-12 15:49:05 +02:00
Emmanuel Hocdet
b4e9ba4b36 MEDIUM: ssl: calculate the real min/max TLS version and find holes
Plan is to add min-tlsxx max-tlsxx configuration, more consistent than no-tlsxx.
Find the real min/max versions (openssl capabilities and haproxy configuration)
and generate warning with bad versions range.
'no-tlsxx' can generate 'holes':
"The list of protocols available can be further limited using the SSL_OP_NO_X
options of the SSL_CTX_set_options or SSL_set_options functions. Clients should
avoid creating 'holes' in the set of protocols they support, when disabling a
protocol, make sure that you also disable either all previous or all subsequent
protocol versions. In clients, when a protocol version is disabled without
disabling all previous protocol versions, the effect is to also disable all
subsequent protocol versions."
To not break compatibility, "holes" is authorized with warning, because openssl
1.1.0 and boringssl deal with it (keep the upper or lower range depending the
case and version).
2017-05-12 15:49:04 +02:00
Emmanuel Hocdet
5db33cbdc4 MEDIUM: ssl: ssl_methods implementation is reworked and factored for min/max tlsxx
Plan is to add min-tlsxx max-tlsxx configuration, more consistent than no-tlsxx.
This patch introduce internal min/max and replace force-tlsxx implementation.
SSL method configuration is store in 'struct tls_version_filter'.
SSL method configuration to openssl setting is abstract in 'methodVersions' table.
With openssl < 1.1.0, SSL_CTX_set_ssl_version is used for force (min == max).
With openssl >= 1.1.0, SSL_CTX_set_min/max_proto_version is used.
2017-05-12 15:49:04 +02:00
Emmanuel Hocdet
6cb2d1e963 MEDIUM: ssl: revert ssl/tls version settings relative to default-server.
Plan is to add min-tlsxx max-tlsxx configuration, more consistent than no-tlsxx.
min-tlsxx and max-tlsxx can be overwrite on local definition. This directives
should be the only ones needed in default-server.
To simplify next patches (rework of tls versions settings with min/max) all
ssl/tls version settings relative to default-server are reverted first:
remove: 'sslv3', 'tls*', 'no-force-sslv3', 'no-force-tls*'.
remove from default-server: 'no-sslv3', 'no-tls*'.
Note:
. force-tlsxx == min-tlsxx + max-tlsxx : would be ok in default-server.
. no-tlsxx is keep for compatibility: should not be propagated to default-server.
2017-05-12 15:49:04 +02:00
Lukas Tribus
53ae85c38e MINOR: ssl: add prefer-client-ciphers
Currently we unconditionally set SSL_OP_CIPHER_SERVER_PREFERENCE [1],
which may not always be a good thing.

The benefit of server side cipher prioritization may not apply to all
cases out there, and it appears that the various SSL libs are going away
from this recommendation ([2], [3]), as insecure ciphers suites are
properly blacklisted/removed and honoring the client's preference is
more likely to improve user experience  (for example using SW-friendly
ciphers on devices without HW AES support).

This is especially true for TLSv1.3, which will restrict the cipher
suites to just AES-GCM and Chacha20/Poly1305.

Apache [4], nginx [5] and others give admins full flexibility, we should
as well.

The initial proposal to change the current default and add a
"prefer-server-ciphers" option (as implemented in e566ecb) has been
declined due to the possible security impact.

This patch implements prefer-client-ciphers without changing the defaults.

[1] https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_options.html
[2] https://github.com/openssl/openssl/issues/541
[3] https://github.com/libressl-portable/portable/issues/66
[4] https://httpd.apache.org/docs/2.0/en/mod/mod_ssl.html#sslhonorcipherorder
[5] https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers
2017-05-12 15:49:04 +02:00
Emeric Brun
fa5c5c892d BUG/MINOR: ssl: fix warnings about methods for opensslv1.1.
This patch replaces the calls to TLSvX_X_client/server/_method
by the new TLS_client/server_method and it uses the new functions
SSL_set_min_proto_version and SSL_set_max_proto_version, setting them
at the wanted protocol version using 'force-' statements.
2017-04-28 18:57:15 +02:00
Frédéric Lécaille
9a146de934 MINOR: server: Make 'default-server' support 'sni' keyword.
This patch makes 'default-server' directives support 'sni' settings.
A field 'sni_expr' has been added to 'struct server' to temporary
stores SNI expressions as strings during both 'default-server' and 'server'
lines parsing. So, to duplicate SNI expressions from 'default-server' 'sni' setting
for new 'server' instances we only have to "strdup" these strings as this is
often done for most of the 'server' settings.
Then, sample expressions are computed calling sample_parse_expr() (only for 'server'
instances).
A new function has been added to produce the same error output as before in case
of any error during 'sni' settings parsing (display_parser_err()).
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
bcaf1d7397 MINOR: server: Make 'default-server' support 'ciphers' keyword.
This patch makes 'default-server' directive support 'ciphers' setting.
2017-03-27 14:37:01 +02:00