Commit Graph

431 Commits

Author SHA1 Message Date
Willy Tarreau
034c88cf03 MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
This implements support for the new API which relies on a call to
setsockopt().
On systems that support it (currently, only Linux >= 4.11), this enables
using TCP fast open when connecting to server.
Please note that you should use the retry-on "conn-failure", "empty-response"
and "response-timeout" keywords, or the request won't be able to be retried
on failure.

Co-authored-by: Olivier Houchard <ohouchard@haproxy.com>
2019-05-06 22:29:39 +02:00
Ilya Shipitsin
0c50b1ecbb BUG/MEDIUM: servers: fix typo "src" instead of "srv"
When copying the settings for all servers when using server templates,
fix a typo, or we would never copy the length of the ALPN to be used for
checks.

This should be backported to 1.9.
2019-04-30 23:04:47 +02:00
Olivier Houchard
88698d966d MEDIUM: connections: Add a way to control the number of idling connections.
As by default we add all keepalive connections to the idle pool, if we run
into a pathological case, where all client don't do keepalive, but the server
does, and haproxy is configured to only reuse "safe" connections, we will
soon find ourself having lots of idling, unusable for new sessions, connections,
while we won't have any file descriptors available to create new connections.

To fix this, add 2 new global settings, "pool_low_ratio" and "pool_high_ratio".
pool-low-fd-ratio  is the % of fds we're allowed to use (against the maximum
number of fds available to haproxy) before we stop adding connections to the
idle pool, and destroy them instead. The default is 20. pool-high-fd-ratio is
the % of fds we're allowed to use (against the maximum number of fds available
to haproxy) before we start killing idling connection in the event we have to
create a new outgoing connection, and no reuse is possible. The default is 25.
2019-04-18 19:52:03 +02:00
Willy Tarreau
0e492e2ad0 BUILD: address a few cases of "static <type> inline foo()"
Older compilers don't like to see "inline" placed after the type in a
function declaration, it must be "static inline <type>" only. This
patch touches various areas. The warnings were seen with gcc-3.4.
2019-04-15 21:55:48 +02:00
Christopher Faulet
73c1207c71 MINOR: muxes: Pass the context of the mux to destroy() instead of the connection
It is mandatory to handle mux upgrades, because during a mux upgrade, the
connection will be reassigned to another multiplexer. So when the old one is
destroyed, it does not own the connection anymore. Or in other words, conn->ctx
does not point to the old mux's context when its destroy() callback is
called. So we now rely on the multiplexer context do destroy it instead of the
connection.

In addition, h1_release() and h2_release() have also been updated in the same
way.
2019-04-12 22:06:53 +02:00
Willy Tarreau
c912f94b57 MINOR: server: remove a few unneeded LIST_INIT calls after LIST_DEL_LOCKED
Since LIST_DEL_LOCKED() and LIST_POP_LOCKED() now automatically reinitialize
the removed element, there's no need for keeping this LIST_INIT() call in the
idle connection code.
2019-02-28 16:08:54 +01:00
Olivier Houchard
9ea5d361ae MEDIUM: servers: Reorganize the way idle connections are cleaned.
Instead of having one task per thread and per server that does clean the
idling connections, have only one global task for every servers.
That tasks parses all the servers that currently have idling connections,
and remove half of them, to put them in a per-thread list of connections
to kill. For each thread that does have connections to kill, wake a task
to do so, so that the cleaning will be done in the context of said thread.
2019-02-26 18:17:32 +01:00
Olivier Houchard
f131481a0a BUG/MEDIUM: servers: Add a per-thread counter of idle connections.
Add a per-thread counter of idling connections, and use it to determine
how many connections we should kill after the timeout, instead of using
the global counter, or we're likely to just kill most of the connections.

This should be backported to 1.9.
2019-02-21 19:07:45 +01:00
Willy Tarreau
980855bd95 BUG/MEDIUM: server: initialize the orphaned conns lists and tasks at the end
This also depends on the nbthread count, so it must only be performed after
parsing the whole config file. As a side effect, this removes some code
duplication between servers and server-templates.

This must be backported to 1.9.
2019-02-07 15:08:13 +01:00
Willy Tarreau
835daa119e BUG/MEDIUM: server: initialize the idle conns list after parsing the config
The idle conns lists are sized according to the number of threads. As such
they cannot be initialized during the parsing since nbthread can be set
later, as revealed by this simple config which randomly crashes when used.
Let's do this at the end instead.

    listen proxy
        bind :4445
        mode http
        timeout client 10s
        timeout server 10s
        timeout connect 10s
        http-reuse always
        server s1 127.0.0.1:8000

    global
        nbthread 8

This fix must be backported to 1.9 and 1.8.
2019-02-07 15:08:13 +01:00
Willy Tarreau
9c538e01c2 MINOR: server: add a max-reuse parameter
Some servers may wish to limit the total number of requests they execute
over a connection because some of their components might leak resources.
In HTTP/1 it was easy, they just had to emit a "connection: close" header
field with the last response. In HTTP/2, it's less easy because the info
is not always shared with the component dealing with the H2 protocol and
it could be harder to advertise a GOAWAY with a stream limit.

This patch provides a solution to this by adding a new "max-reuse" parameter
to the server keyword. This parameter indicates how many times an idle
connection may be reused for new requests. The information is made available
and the underlying muxes will be able to use it at will.

This patch should be backported to 1.9.
2019-01-24 19:06:43 +01:00
Willy Tarreau
15c120d251 CLEANUP: server: fix indentation mess on idle connections
Apparently some code was moved around leaving the inner block incorrectly
indented and with the closing brace in the middle of nowhere.
2019-01-24 19:06:43 +01:00
Willy Tarreau
cb923d5001 MINOR: server: make sure pool-max-conn is >= -1
The keyword parser doesn't check the value range, but supported values are
-1 and positive values, thus we should check it.

This can be backported to 1.9.
2019-01-24 16:31:56 +01:00
Jrme Magnin
f57afa453a BUG/MINOR: server: don't always trust srv_check_health when loading a server state
When we load health values from a server state file, make sure what we assign
to srv->check.health actually matches the state we restore.

This should be backported as far as 1.6.
2019-01-21 11:09:03 +01:00
Willy Tarreau
1ba32032ef BUG/MEDIUM: checks: fix recent regression on agent-check making it crash
In order to address the mailers issues, we needed to store the proxy
into the checks struct, which was done by commit c98aa1f18 ("MINOR:
checks: Store the proxy in checks."). However this one did it only for
the health checks and not for the agent checks, resulting in an immediate
crash when the agent is enabled on a random config like this one :

  listen agent
      bind :8000
      server s1 255.255.255.255:1 agent-check agent-port 1

Thanks to Seri Kim for reporting it and providing a reproducer in
issue #20. This fix must be backported to 1.9.
2019-01-21 07:48:26 +01:00
Frdric Lcaille
355b2033ec MINOR: cfgparse: SSL/TLS binding in "peers" sections.
Make "bind" keywork be supported in "peers" sections.
All "bind" settings are supported on this line.
Add "default-bind" option to parse the binding options excepted the bind address.
Do not parse anymore the bind address for local peers on "server" lines.
Do not use anymore list_for_each_entry() to set the "peers" section
listener parameters because there is only one listener by "peers" section.

May be backported to 1.5 and newer.
2019-01-18 14:26:21 +01:00
Frdric Lcaille
c06b5d4f74 MINOR: cfgparse: Make "peer" lines be parsed as "server" lines.
With this patch "default-server" lines are supported in "peers" sections
to setup the default settings of peers which are from now setup
when parsing both "peer" and "server" lines.

May be backported to 1.5 and newer.
2019-01-18 14:26:21 +01:00
Olivier Houchard
c98aa1f182 MINOR: checks: Store the proxy in checks.
Instead of assuming we have a server, store the proxy directly in struct
check, and use it instead of s->server.
This should be a no-op for now, but will be useful later when we change
mail checks to avoid having a server.

This should be backported to 1.9.
2019-01-14 11:15:11 +01:00
Daniel Corbett
43bb842a08 BUG/MEDIUM: init: Initialize idle_orphan_conns for first server in server-template
When initializing server-template all of the servers after the first
have srv->idle_orphan_conns initialized within server_template_init()
The first server does not have this initialized and when http-reuse
is active this causes a segmentation fault when accessed from
srv_add_to_idle_list().  This patch removes the check for
srv->tmpl_info.prefix within server_finalize_init() and allows
the first server within a server-template to have srv->idle_orphan_conns
properly initialized.

This should be backported to 1.9.
2019-01-09 14:45:21 +01:00
Olivier Houchard
921501443b MEDIUM: checks: Add check-alpn.
Add a way to configure the ALPN used by check, with a new "check-alpn"
keyword. By default, the checks will use the server ALPN, but it may not
be convenient, for instance because the server may use HTTP/2, while checks
are unable to do HTTP/2 yet.
2018-12-21 19:54:16 +01:00
Olivier Houchard
21944019ca BUG/MEDIUM: server: Also copy "check-sni" for server templates.
When using server templates, if "check-sni" is used, make sure it shows up
in all the created servers.

This should be backported to 1.8 and 1.9.
2018-12-21 19:53:28 +01:00
Olivier Houchard
b7b3faa79c MEDIUM: servers: Replace idle-timeout with pool-purge-delay.
Instead of the old "idle-timeout" mechanism, add a new option,
"pool-purge-delay", that sets the delay before purging idle connections.
Each time the delay happens, we destroy half of the idle connections.
2018-12-15 23:50:09 +01:00
Olivier Houchard
006e3101f9 MEDIUM: servers: Add a command to limit the number of idling connections.
Add a new command, "pool-max-conn" that sets the maximum number of connections
waiting in the orphan idling connections list (as activated with idle-timeout).
Using "-1" means unlimited. Using pools is now dependant on this.
2018-12-15 23:50:08 +01:00
Olivier Houchard
0c18a6fe34 MEDIUM: servers: Add a way to keep idle connections alive.
Add a new keyword for servers, "idle-timeout". If set, unused connections are
kept alive until the timeout happens, and will be picked for reuse if no
other connection is available.
2018-12-02 18:16:53 +01:00
Willy Tarreau
76a551de2e MINOR: config: make sure to associate the proper mux to bind and servers
Currently a mux may be forced on a bind or server line by specifying the
"proto" keyword. The problem is that the mux may depend on the proxy's
mode, which is not known when parsing this keyword, so a wrong mux could
be picked.

Let's simply update the mux entry while checking its validity. We do have
the name and the side, we only need to see if a better mux fits based on
the proxy's mode. It also requires to remove the side check while parsing
the "proto" keyword since a wrong mux could be picked.

This way it becomes possible to declare multiple muxes with the same
protocol names and different sides or modes.
2018-12-02 13:29:35 +01:00
Willy Tarreau
0108d90c6c MEDIUM: init: convert all trivial registration calls to initcalls
This switches explicit calls to various trivial registration methods for
keywords, muxes or protocols from constructors to INITCALL1 at stage
STG_REGISTER. All these calls have in common to consume a single pointer
and return void. Doing this removes 26 constructors. The following calls
were addressed :

- acl_register_keywords
- bind_register_keywords
- cfg_register_keywords
- cli_register_kw
- flt_register_keywords
- http_req_keywords_register
- http_res_keywords_register
- protocol_register
- register_mux_proto
- sample_register_convs
- sample_register_fetches
- srv_register_keywords
- tcp_req_conn_keywords_register
- tcp_req_cont_keywords_register
- tcp_req_sess_keywords_register
- tcp_res_cont_keywords_register
- flt_register_keywords
2018-11-26 19:50:32 +01:00
Olivier Houchard
c756600103 MINOR: server: Add "alpn" and "npn" keywords.
Add new keywords to "server" lines, alpn and npn.
If set, when connecting through SSL, those alpn/npn will be negociated
during the SSL handshake.
2018-11-22 19:50:08 +01:00
Joseph Herlant
44466826b1 CLEANUP: fix a few typos in the comments of the server subsystem
A few misspells where detected in the server subsystem. This commit
fixes them.
2018-11-18 22:23:15 +01:00
Willy Tarreau
db398435aa MINOR: stream-int: replace si_cant_put() with si_rx_room_{blk,rdy}()
Remaining calls to si_cant_put() were all for lack of room and were
turned to si_rx_room_blk(). A few places where SI_FL_RXBLK_ROOM was
cleared by hand were converted to si_rx_room_rdy().

The now unused si_cant_put() function was removed.
2018-11-18 21:41:50 +01:00
Willy Tarreau
0cd3bd628a MINOR: stream-int: rename si_applet_{want|stop|cant}_{get|put}
It doesn't make sense to limit this code to applets, as any stream
interface can use it. Let's rename it by simply dropping the "applet_"
part of the name. No other change was made except updating the comments.
2018-11-11 10:18:37 +01:00
William Lallemand
313bfd18c1 MINOR: server: export new_server() function
The new_server() function will be useful to create a proxy for the
master-worker.
2018-10-28 13:51:38 +01:00
Willy Tarreau
5dfb6c4cc9 CLEANUP: state-file: make the path concatenation code a bit more consistent
There are as many ways to build the globalfilepathlen variable as branches
in the if/then/else, creating lots of confusion. Address the most obvious
parts, but some polishing definitely is still needed.
2018-10-16 19:26:12 +02:00
Olivier Houchard
17f8b90736 MINOR: server: Use memcpy() instead of strncpy().
Use memcpy instead of strncpy, strncpy buys us nothing, and gcc is being
annoying.
2018-10-16 19:22:20 +02:00
Dirkjan Bussink
415150f764 MEDIUM: ssl: add support for ciphersuites option for TLSv1.3
OpenSSL released support for TLSv1.3. It also added a separate function
SSL_CTX_set_ciphersuites that is used to set the ciphers used in the
TLS 1.3 handshake. This change adds support for that new configuration
option by adding a ciphersuites configuration variable that works
essentially the same as the existing ciphers setting.

Note that it should likely be backported to 1.8 in order to ease usage
of the now released openssl-1.1.1.
2018-10-08 19:20:13 +02:00
Frdric Lcaille
5afb3cfbcc BUG/MINOR: server: Crash when setting FQDN via CLI.
This patch ensures that a DNS resolution may be launched before
setting a server FQDN via the CLI. Especially, it checks that
resolvers was set.

A LEVEL 4 reg testing file is provided.

Thanks to Lukas Tribus for having reported this issue.

Must be backported to 1.8.
2018-09-12 07:41:41 +02:00
Baptiste Assmann
6d0f38f00d BUG/MEDIUM: dns/server: fix incomatibility between SRV resolution and server state file
Server state file has no indication that a server is currently managed
by a DNS SRV resolution.
And thus, both feature (DNS SRV resolution and server state), when used
together, does not provide the expected behavior: a smooth experience...

This patch introduce the "SRV record name" in the server state file and
loads and applies it if found and wherever required.

This patch applies to haproxy-dev branch only. For backport, a specific patch
is provided for 1.8.
2018-09-04 17:40:22 +02:00
Willy Tarreau
49725a0977 BUG/MEDIUM: check/threads: do not involve the rendez-vous point for status updates
thread_isolate() is currently being called with the server lock held.
This is not acceptable because it prevents other threads from reaching
the rendez-vous point. Now that the LB algos are thread-safe, let's get
rid of this call.

No backport is nedeed.
2018-08-21 19:54:09 +02:00
Willy Tarreau
3bcc2699ba BUG/MEDIUM: cli/threads: protect some server commands against concurrent operations
The server-specific CLI commands "set weight", "set maxconn",
"disable agent", "enable agent", "disable health", "enable health",
"disable server" and "enable server" were not protected against
concurrent accesses. Now they take the server lock around the
sensitive part.

This patch must be backported to 1.8.
2018-08-21 15:35:31 +02:00
Willy Tarreau
46b7f53ad9 DOC: server/threads: document which functions need to be called with/without locks
At the moment it's totally unclear while reading the server's code which
functions require to be called with the server lock held and which ones
grab it and cannot be called this way. This commit simply inventories
all of them to indicate what is detected depending on how these functions
use the struct server. Only functions used at runtime were checked, those
dedicated to config parsing were skipped. Doing so already has uncovered
a few bugs on some CLI actions.
2018-08-21 14:58:25 +02:00
Willy Tarreau
eeba36b3af BUG/MEDIUM: server: update our local state before propagating changes
Commit 3ff577e ("MAJOR: server: make server state changes synchronous again")
reintroduced synchronous server state changes. However, during the previous
change from synchronous to asynchronous, the server state propagation was
placed at the end of the function to ease the code changes, and the commit
above didn't put it back at its place. This has resulted in propagated
states to be incomplete. For example, making a server leave maintenance
would make it up but would leave its tracking servers down because they
see their tracked server is still down.

Let's just move the status update right to its place. It also adds the
benefit of reporting state changes in the order they appear and not in
reverse.

No backport is needed.
2018-08-21 08:29:25 +02:00
Patrick Hemmer
0355dabd7c MINOR: queue: replace the linked list with a tree
We'll need trees to manage the queues by priorities. This change replaces
the list with a tree based on a single key. It's effectively a list but
allows us to get rid of the list management right now.
2018-08-10 15:06:27 +02:00
Christopher Faulet
8ed0a3e32a MINOR: mux/server: Add 'proto' keyword to force the multiplexer's protocol
For now, it is parsed but not used. Tests are done on it to check if the side
and the mode are compatible with the server's definition.
2018-08-08 10:42:08 +02:00
Willy Tarreau
91c2826e1d CLEANUP: server: remove the update list and the update lock
These ones are not more used, let's get rid of them.
2018-08-08 09:57:45 +02:00
Willy Tarreau
3ff577e165 MAJOR: server: make server state changes synchronous again
Now we try to synchronously push updates as they come using the new rdv
point, so that the call to the server update function from the main poll
loop is not needed anymore.

It further reduces the apparent latency in the health checks as the response
time almost always appears as 0 ms, resulting in a slightly higher check rate
of ~1960 conn/s. Despite this, the CPU consumption has slightly dropped again
to ~32% for the same test.

The only trick is that the checks code is built with a bit of recursivity
because srv_update_status() calls server_recalc_eweight(), and the latter
needs to signal srv_update_status() in case of updates. Thus we added an
extra argument to this function to indicate whether or not it must
propagate updates (no if it comes from srv_update_status).
2018-08-08 09:57:45 +02:00
Willy Tarreau
3d3700f216 MEDIUM: checks: use the new rendez-vous point to spread check result
The current sync point causes some important stress when a high number
of threads is in use on a config with lots of checks, because it wakes
up all threads every time a server state changes.

A config like the following can easily saturate a 4-core machine reaching
only 750 checks per second out of the ~2000 configured :

    global
        nbthread 4

    defaults
        mode    http
        timeout connect 5s
        timeout client  5s
        timeout server  5s

    frontend srv
        bind :8001 process 1/1
        redirect location / if { method OPTIONS } { rand(100) ge 50 }
        stats uri /

    backend chk
        option httpchk
        server-template srv 1-100 127.0.0.1:8001 check rise 1 fall 1 inter 50

The reason is that the random on the fake server causes the responses
to randomly match an HTTP check, and results in a lot of up/down events
that are broadcasted to all threads. It's worth noting that the CPU usage
already dropped by about 60% between 1.8 and 1.9 just due to the scheduler
updates, but the sync point remains expensive.

In addition, it's visible on the stats page that a lot of requests end up
with an L7TOUT status in ~60ms. With smaller timeouts, it's even L4TOUT
around 20-25ms.

By not using THREAD_WANT_SYNC() anymore and only calling the server updates
under thread_isolate(), we can avoid all these wakeups. The CPU usage on
the same config drops to around 44% on the same machine, with all checks
being delivered at ~1900 checks per second, and the stats page shows no
more timeouts, even at 10 ms check interval. The difference is mainly
caused by the fact that there's no more need to wait for a thread to wake
up from poll() before starting to process check results.
2018-08-08 09:56:32 +02:00
Willy Tarreau
6a78e61694 BUG/MEDIUM: servers: check the queues once enabling a server
Commit 64cc49c ("MAJOR: servers: propagate server status changes
asynchronously.") heavily changed the way the server states are
updated since they became asynchronous. During this change, some
code was lost, which is used to shut down some sessions from a
backup server and to pick pending connections from a proxy once
a server is turned back from maintenance to ready state. The
effect is that when temporarily disabling a server, connections
stay in the backend's queue, and when re-enabling it, they are
not picked and they expire in the backend's queue. Now they're
properly picked again.

This fix must be backported to 1.8.
2018-08-07 10:14:53 +02:00
Olivier Houchard
306e653331 BUG/MINOR: servers: Don't make "server" in a frontend fatal.
When parsing the configuration, if "server", "default-server" or
"server-template" are found in a frontend, we first warn that it will be
ignored, only to be considered a fatal error later. Be true to our word, and
just ignore it.

This should be backported to 1.8 and 1.7.
2018-07-24 17:13:54 +02:00
Willy Tarreau
83061a820e MAJOR: chunks: replace struct chunk with struct buffer
Now all the code used to manipulate chunks uses a struct buffer instead.
The functions are still called "chunk*", and some of them will progressively
move to the generic buffer handling code as they are cleaned up.
2018-07-19 16:23:43 +02:00
Willy Tarreau
843b7cbe9d MEDIUM: chunks: make the chunk struct's fields match the buffer struct
Chunks are only a subset of a buffer (a non-wrapping version with no head
offset). Despite this we still carry a lot of duplicated code between
buffers and chunks. Replacing chunks with buffers would significantly
reduce the maintenance efforts. This first patch renames the chunk's
fields to match the name and types used by struct buffers, with the goal
of isolating the code changes from the declaration changes.

Most of the changes were made with spatch using this coccinelle script :

  @rule_d1@
  typedef chunk;
  struct chunk chunk;
  @@
  - chunk.str
  + chunk.area

  @rule_d2@
  typedef chunk;
  struct chunk chunk;
  @@
  - chunk.len
  + chunk.data

  @rule_i1@
  typedef chunk;
  struct chunk *chunk;
  @@
  - chunk->str
  + chunk->area

  @rule_i2@
  typedef chunk;
  struct chunk *chunk;
  @@
  - chunk->len
  + chunk->data

Some minor updates to 3 http functions had to be performed to take size_t
ints instead of ints in order to match the unsigned length here.
2018-07-19 16:23:43 +02:00
Baptiste Assmann
8e2d9430c0 MINOR: dns: new DNS options to allow/prevent IP address duplication
By default, HAProxy's DNS resolution at runtime ensure that there is no
IP address duplication in a backend (for servers being resolved by the
same hostname).
There are a few cases where people want, on purpose, to disable this
feature.

This patch introduces a couple of new server side options for this purpose:
"resolve-opts allow-dup-ip" or "resolve-opts prevent-dup-ip".
2018-07-12 17:56:44 +02:00
Daniel Corbett
9215ffa6b2 BUG/MEDIUM: servers: Add srv_addr default placeholder to the state file
When creating a state file using "show servers state" an empty field is
created in the srv_addr column if the server is from the socket family
AF_UNIX.  This leads to a warning on start up when using
"load-server-state-from-file". This patch defaults srv_addr to "-" if
the socket family is not covered.

This patch should be backported to 1.8.
2018-05-24 22:06:08 +02:00
Aurlien Nephtali
abbf607105 MEDIUM: cli: Add payload support
In order to use arbitrary data in the CLI (multiple lines or group of words
that must be considered as a whole, for example), it is now possible to add a
payload to the commands. To do so, the first line needs to end with a special
pattern: <<\n. Everything that follows will be left untouched by the CLI parser
and will be passed to the commands parsers.

Per-command support will need to be added to take advantage of this
feature.

Signed-off-by: Aurlien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-26 14:19:33 +02:00
Emmanuel Hocdet
4399c75f6c MINOR: proxy-v2-options: add crc32c
This patch add option crc32c (PP2_TYPE_CRC32C) to proxy protocol v2.
It compute the checksum of proxy protocol v2 header as describe in
"doc/proxy-protocol.txt".
2018-03-21 05:04:01 +01:00
Emmanuel Hocdet
253c3b7516 MINOR: connection: add proxy-v2-options authority
This patch add option PP2_TYPE_AUTHORITY to proxy protocol v2 when a TLS
connection was negotiated. In this case, authority corresponds to the sni.
2018-03-01 11:38:32 +01:00
Emmanuel Hocdet
fa8d0f1875 MINOR: connection: add proxy-v2-options ssl-cipher,cert-sig,cert-key
This patch implement proxy protocol v2 options related to crypto information:
ssl-cipher (PP2_SUBTYPE_SSL_CIPHER), cert-sig (PP2_SUBTYPE_SSL_SIG_ALG) and
cert-key (PP2_SUBTYPE_SSL_KEY_ALG).
2018-03-01 11:38:28 +01:00
Emmanuel Hocdet
f643b80429 MINOR: introduce proxy-v2-options for send-proxy-v2
Proxy protocol v2 can transport many optional informations. To avoid
send-proxy-v2-* explosion, this patch introduce proxy-v2-options parameter
and will allow to write: "send-proxy-v2 proxy-v2-options ssl,cert-cn".
2018-02-02 05:52:51 +01:00
Christopher Faulet
8d01fd6b3c BUG/MEDIUM: threads/server: Fix deadlock in srv_set_stopping/srv_set_admin_flag
Because of a typo (HA_SPIN_LOCK instead of HA_SPIN_UNLOCK), there is a deadlock
in srv_set_stopping and srv_set_admin_flag when there is at least one trackers.

This patch must be backported in 1.8.
2018-01-25 13:51:23 +01:00
Olivier Houchard
e9bad0a936 MINOR: servers: Don't report duplicate dyncookies for disabled servers.
Especially with server-templates, it can happen servers starts with a
placeholder IP, in the disabled state. In this case, we don't want to report
that the same cookie was generated for multiple servers. So defer the test
until the server is enabled.

This should be backported to 1.8.
2018-01-23 14:05:17 +01:00
Emeric Brun
e31148031f BUG/MEDIUM: checks: a server passed in maint state was not forced down.
Setting a server in maint mode, the required next_state was not set
before calling the 'lb_down' function and so the system state was never
commited.

This patch should be backported in 1.8
2017-12-21 15:23:55 +01:00
Emeric Brun
8f29829e24 BUG/MEDIUM: checks: a down server going to maint remains definitely stucked on down state.
The new admin state was not correctly commited in this case.
Checks were fully disabled but the server was not marked in MAINT state.
It results with a server definitely stucked on the DOWN state.

This patch should be backported on haproxy 1.8
2017-12-06 17:01:00 +01:00
Olivier Houchard
fbc74e8556 MINOR/CLEANUP: proxy: rename "proxy" to "proxies_list"
Rename the global variable "proxy" to "proxies_list".
There's been multiple proxies in haproxy for quite some time, and "proxy"
is a potential source of bugs, a number of functions have a "proxy" argument,
and some code used "proxy" when it really meant "px" or "curproxy". It worked
by pure luck, because it usually happened while parsing the config, and thus
"proxy" pointed to the currently parsed proxy, but we should probably not
rely on this.

[wt: some of these are definitely fixes that are worth backporting]
2017-11-24 17:21:27 +01:00
Christopher Faulet
767a84bcc0 CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning 2017-11-24 17:19:12 +01:00
Willy Tarreau
358847f026 BUILD: server: check->desc always exists
Clang reports this warning :

  src/server.c:872:14: warning: address of array 'check->desc' will
  always evaluate to 'true' [-Wpointer-bool-conversion]

Indeed, check->desc used to be a pointer to a dynamically allocated area
a long time ago and is now an array. Let's remove the useless test.
2017-11-20 21:33:21 +01:00
Christopher Faulet
9dcf9b6f03 MINOR: threads: Use __decl_hathreads to declare locks
This macro should be used to declare variables or struct members depending on
the USE_THREAD compile option. It avoids the encapsulation of such declarations
between #ifdef/#endif. It is used to declare all lock variables.
2017-11-13 11:38:17 +01:00
Christopher Faulet
2a944ee16b BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix
This remove any name conflicts, especially on Solaris.
2017-11-07 11:10:24 +01:00
Olivier Houchard
283810773a BUG/MINOR: dns: Don't lock the server lock in snr_check_ip_callback().
snr_check_ip_callback() may be called with the server lock, so don't attempt
to lock it again, instead, make sure the callers always have the lock before
calling it.
2017-11-06 18:34:42 +01:00
Olivier Houchard
55dcdf4c39 BUG/MINOR: dns: Don't try to get the server lock if it's already held.
dns_link_resolution() can be called with the server lock already held, so
don't attempt to lock it again in that case.
2017-11-06 18:34:24 +01:00
Willy Tarreau
6ce38f3eab CLEANUP: server: get rid of return statements in the CLI parser
There were two many return, some of them missing a spin_unlock call,
let's use a goto to a central place instead.
2017-11-05 10:19:23 +01:00
Willy Tarreau
a075258a2c BUG/MINOR: cli: add severity in "set server addr" parser
Commit c3680ec ("MINOR: add severity information to cli feedback messages")
introduced a severity level to CLI messages, but one of them was missed
on "set server addr". No backport is needed.
2017-11-05 10:17:49 +01:00
Willy Tarreau
1c8980f9b5 BUG/MINOR: cli: do not perform an invalid action on "set server check-port"
The "set server <srv> check-port" CLI handler forgot to return after
detecting an error on the port number, and still proceeds with the action.
This needs to be backported to 1.7.
2017-11-05 10:13:37 +01:00
Willy Tarreau
2a858a82ec BUG/MAJOR: threads/server: missing unlock in CLI fqdn parser
This one didn't properly unlock before returning an error message.
2017-11-05 10:13:37 +01:00
Emeric Brun
e9fd6b5916 BUG/MINOR: dns: fix missing lock protection on server.
To avoid inconsistencies server's attributes must be read
or updated under lock.
2017-11-03 15:17:55 +01:00
Olivier Houchard
d16bfe6c01 BUG/MINOR: dns: Fix SRV records with the new thread code.
srv_set_fqdn() may be called with the DNS lock already held, but tries to
lock it anyway. So, add a new parameter to let it know if it was already
locked or not;
2017-10-31 15:47:55 +01:00
Emeric Brun
9f0b458525 MEDIUM: threads/server: Use the server lock to protect health check and cli concurrency 2017-10-31 13:58:33 +01:00
Christopher Faulet
b2812a6240 MEDIUM: thread/dns: Make DNS thread-safe 2017-10-31 13:58:33 +01:00
Christopher Faulet
5d42e099c5 MINOR: threads/server: Add a lock to deal with insert in updates_servers list
This list is used to save changes on the servers state. So when serveral threads
are used, it must be locked. The changes are then applied in the sync-point. To
do so, servers_update_status has be moved in the sync-point. So this is useless
to lock it at this step because the sync-point is a protected area by iteself.
2017-10-31 13:58:31 +01:00
Christopher Faulet
29f77e846b MEDIUM: threads/server: Add a lock per server and atomically update server vars
The server's lock is use, among other things, to lock acces to the active
connection list of a server.
2017-10-31 13:58:31 +01:00
Christopher Faulet
40a007cf2a MEDIUM: threads/server: Make connection list (priv/idle/safe) thread-safe
For now, we have a list of each type per thread. So there is no need to lock
them. This is the easiest solution for now, but not the best one because there
is no sharing between threads. An idle connection on a thread will not be able
be used by a stream on another thread. So it could be a good idea to rework this
patch later.
2017-10-31 13:58:30 +01:00
Christopher Faulet
67957bd59e MAJOR: dns: Refactor the DNS code
This is a huge patch with many changes, all about the DNS. Initially, the idea
was to update the DNS part to ease the threads support integration. But quickly,
I started to refactor some parts. And after several iterations, it was
impossible for me to commit the different parts atomically. So, instead of
adding tens of patches, often reworking the same parts, it was easier to merge
all my changes in a uniq patch. Here are all changes made on the DNS.

First, the DNS initialization has been refactored. The DNS configuration parsing
remains untouched, in cfgparse.c. But all checks have been moved in a post-check
callback. In the function dns_finalize_config, for each resolvers, the
nameservers configuration is tested and the task used to manage DNS resolutions
is created. The links between the backend's servers and the resolvers are also
created at this step. Here no connection are kept alive. So there is no needs
anymore to reopen them after HAProxy fork. Connections used to send DNS queries
will be opened on demand.

Then, the way DNS requesters are linked to a DNS resolution has been
reworked. The resolution used by a requester is now referenced into the
dns_requester structure and the resolution pointers in server and dns_srvrq
structures have been removed. wait and curr list of requesters, for a DNS
resolution, have been replaced by a uniq list. And Finally, the way a requester
is removed from a DNS resolution has been simplified. Now everything is done in
dns_unlink_resolution.

srv_set_fqdn function has been simplified. Now, there is only 1 way to set the
server's FQDN, independently it is done by the CLI or when a SRV record is
resolved.

The static DNS resolutions pool has been replaced by a dynamoc pool. The part
has been modified by Baptiste Assmann.

The way the DNS resolutions are triggered by the task or by a health-check has
been totally refactored. Now, all timeouts are respected. Especially
hold.valid. The default frequency to wake up a resolvers is now configurable
using "timeout resolve" parameter.

Now, as documented, as long as invalid repsonses are received, we really wait
all name servers responses before retrying.

As far as possible, resources allocated during DNS configuration parsing are
releases when HAProxy is shutdown.

Beside all these changes, the code has been cleaned to ease code review and the
doc has been updated.
2017-10-31 11:36:12 +01:00
Olivier Houchard
796a2b3324 BUG/MEDIUM: server: Allocate tmptrash before using it.
Don't forget to allocate tmptrash before using it, and free it once we're
done.

[wt: introduced by commit 64cc49cf ("MAJOR: servers: propagate server
status changes asynchronously"), no backport needed]
2017-10-24 19:54:25 +02:00
Emeric Brun
5a1335110c BUG/MEDIUM: log: check result details truncated.
Fix regression introduced by commit:
'MAJOR: servers: propagate server status changes asynchronously.'

The building of the log line was re-worked to be done at the
postponed point without lack of data.

[wt: this only affects 1.8-dev, no backport needed]
2017-10-19 18:51:32 +02:00
Willy Tarreau
06d80a9a9c REORG: channel: finally rename the last bi_* / bo_* functions
For HTTP/2 we'll need some buffer-only equivalent functions to some of
the ones applying to channels and still squatting the bi_* / bo_*
namespace. Since these names have kept being misleading for quite some
time now and are really getting annoying, it's time to rename them. This
commit will use "ci/co" as the prefix (for "channel in", "channel out")
instead of "bi/bo". The following ones were renamed :

  bi_getblk_nc, bi_getline_nc, bi_putblk, bi_putchr,
  bo_getblk, bo_getblk_nc, bo_getline, bo_getline_nc, bo_inject,
  bi_putchk, bi_putstr, bo_getchr, bo_skip, bi_swpbuf
2017-10-19 15:01:08 +02:00
Emeric Brun
64cc49cf7e MAJOR: servers: propagate server status changes asynchronously.
In order to prepare multi-thread development, code was re-worked
to propagate changes asynchronoulsy.

Servers with pending status changes are registered in a list
and this one is processed and emptied only once 'run poll' loop.

Operational status changes are performed before administrative
status changes.

In a case of multiple operational status change or admin status
change in the same 'run poll' loop iteration, those changes are
merged to reach only the targeted status.
2017-10-13 12:00:27 +02:00
Willy Tarreau
6fb4ba38e0 BUG/MEDIUM: server: unwanted behavior leaving maintenance mode on tracked stopping server (take2)
Previous patch got accidently broken. This one fixes it.
2017-09-21 17:37:38 +02:00
Emeric Brun
e1e3947e7e BUG/MEDIUM: server: unwanted behavior leaving maintenance mode on tracked stopping server
Leaving the maintenance state and if the server remains in stopping mode due
to a tracked one:

- We mistakenly try to grab some pending conns and shutdown backup sessions.
- The proxy down time and last change were also mistakenly updated
2017-09-21 17:30:01 +02:00
Christopher Faulet
3bbd65b23e BUG/MINOR: dns: Fix check on nameserver in snr_resolution_cb
snr_resolution_cb can be called with <nameserver> parameter set to NULL. So we
must check it before using it. This is done most of time, except when we deal
with invalid DNS response.
2017-09-15 18:42:23 +02:00
Andjelko Iharos
c3680ecdf8 MINOR: add severity information to cli feedback messages 2017-09-13 13:38:32 +02:00
Willy Tarreau
3d609a755e Revert "BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file"
This reverts commit 19e8aa58f7.

It causes some trouble reported by Manu :
   listen tls
     [...]
     server bla 127.0.0.1:8080

   [ALERT] 248/130258 (21960) : parsing [/etc/haproxy/test.cfg:53] : 'server bla' : no method found to resolve address '(null)'
   [ALERT] 248/130258 (21960) : Failed to initialize server(s) addr.

According to Nenad :
  "It's not a good way to fix the issue we were experiencing
   before. It will need a bigger rewrite, because the logic in
   srv_iterate_initaddr needs to be changed."
2017-09-06 14:22:45 +02:00
Nenad Merdanovic
19e8aa58f7 BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file
Historically the DNS was the only way of updating the server IP dynamically
and the init-addr processing and state file load required the server to have
an FQDN defined. Given that we can now update the IP through the socket as
well and also can have different init-addr values (like IP and 'none') - this
requirement needs to be removed.

This patch should be backported to 1.7.
2017-09-05 15:52:58 +02:00
Emeric Brun
52a91d3d48 MEDIUM: check: server states and weight propagation re-work
The server state and weight was reworked to handle
"pending" values updated by checks/CLI/LUA/agent.
These values are commited to be propagated to the
LB stack.

In further dev related to multi-thread, the commit
will be handled into a sync point.

Pending values are named using the prefix 'next_'
Current values used by the LB stack are named 'cur_'
2017-09-05 15:23:16 +02:00
Baptiste Assmann
747359eeca BUG/MINOR: dns: server set by SRV records stay in "no resolution" status
This patch fixes a bug where some servers managed by SRV record query
types never ever recover from a "no resolution" status.
The problem is due to a wrong function called when breaking the
server/resolution (A/AAAA) relationship: this is performed when a server's SRV
record disappear from the SRV response.
2017-08-22 11:34:49 +02:00
Baptiste Assmann
6fb8192b28 MINOR: dns: enable caching of responses for server set by a SRV record
The function srv_set_fqdn() is used to update a server's fqdn and set
accordingly its DNS resolution.
Current implementation prevents a server whose update is triggered by a
SRV record from being linked to an existing resolution in the cache (if
applicable).
This patch aims at fixing this.
2017-08-18 11:25:41 +02:00
Olivier Houchard
8da5f98fbe MINOR: dns: Handle SRV records.
Make it so for each server, instead of specifying a hostname, one can use
a SRV label.
When doing so, haproxy will first resolve the SRV label, then use the
resulting hostnames, as well as port and weight (priority is ignored right
now), to each server using the SRV label.
It is resolved periodically, and any server disappearing from the SRV records
will be removed, and any server appearing will be added, assuming there're
free servers in haproxy.
2017-08-09 16:32:49 +02:00
Olivier Houchard
a8c6db8d2d MINOR: dns: Cache previous DNS answers.
As DNS servers may not return all IPs in one answer, we want to cache the
previous entries. Those entries are removed when considered obsolete, which
happens when the IP hasn't been returned by the DNS server for a time
defined in the "hold obsolete" parameter of the resolver section. The default
is 30s.
2017-08-09 16:32:49 +02:00
Frdric Lcaille
3169471964 MINOR: Add server port field to server state file.
This patch adds server ports to server state file at the end of each line
for backward compatibility.
2017-08-03 14:31:46 +02:00
Frdric Lcaille
0bedb8ac90 BUG/MAJOR: server: Segfault after parsing server state file.
This patch makes the server state file parser ignore servers wich are
not present in the configuration file.
2017-06-15 15:30:30 +02:00
Baptiste Assmann
201c07f681 MAJOR/REORG: dns: DNS resolution task and requester queues
This patch is a major upgrade of the internal run-time DNS resolver in
HAProxy and it brings the following 2 main changes:

1. DNS resolution task

Up to now, DNS resolution was triggered by the health check task.
From now, DNS resolution task is autonomous. It is started by HAProxy
right after the scheduler is available and it is woken either when a
network IO occurs for one of its nameserver or when a timeout is
matched.

From now, this means we can enable DNS resolution for a server without
enabling health checking.

2. Introduction of a dns_requester structure

Up to now, DNS resolution was purposely made for resolving server
hostnames.
The idea, is to ensure that any HAProxy internal object should be able
to trigger a DNS resolution. For this purpose, 2 things has to be done:
  - clean up the DNS code from the server structure (this was already
    quite clean actually) and clean up the server's callbacks from
    manipulating too much DNS resolution
  - create an agnostic structure which allows linking a DNS resolution
    and a requester of any type (using obj_type enum)

3. Manage requesters through queues

Up to now, there was an uniq relationship between a resolution and it's
owner (aka the requester now). It's a shame, because in some cases,
multiple objects may share the same hostname and may benefit from a
resolution being performed by a third party.
This patch introduces the notion of queues, which are basically lists of
either currently running resolution or waiting ones.

The resolutions are now available as a pool, which belongs to the resolvers.
The pool has has a default size of 64 resolutions per resolvers and is
allocated at configuration parsing.
2017-06-02 11:58:54 +02:00
Baptiste Assmann
fa4a663095 MINOR: dns: implement a LRU cache for DNS resolutions
Introduction of a DNS response LRU cache in HAProxy.

When a positive response is received from a DNS server, HAProxy stores
it in the struct resolution and then also populates a LRU cache with the
response.
For now, the key in the cache is a XXHASH64 of the hostname in the
domain name format concatened to the query type in string format.
2017-06-02 11:40:01 +02:00
Baptiste Assmann
729c901c3f MAJOR: dns: save a copy of the DNS response in struct resolution
Prior this patch, the DNS responses were stored in a pre-allocated
memory area (allocated at HAProxy's startup).
The problem is that this memory is erased for each new DNS responses
received and processed.

This patch removes the global memory allocation (which was not thread
safe by the way) and introduces a storage of the dns response  in the
struct
resolution.
The memory in the struct resolution is also reserved at start up and is
thread safe, since each resolution structure will have its own memory
area.

For now, we simply store the response and use it atomically per
response per server.
2017-06-02 11:30:21 +02:00
Baptiste Assmann
fb7091e213 MINOR: dns: new snr_check_ip_callback function
In the process of breaking links between dns_* functions and other
structures (mainly server and a bit of resolution), the function
dns_get_ip_from_response needs to be reworked: it now can call
"callback" functions based on resolution's owner type to allow modifying
the way the response is processed.

For now, main purpose of the callback function is to check that an IP
address is not already affected to an element of the same type.

For now, only server type has a callback.
2017-06-02 11:28:14 +02:00
Baptiste Assmann
42746373eb REORG: dns: dns_option structure, storage of hostname_dn
This patch introduces a some re-organisation around the DNS code in
HAProxy.

1. make the dns_* functions less dependent on 'struct server' and 'struct resolution'.

With this in mind, the following changes were performed:
- 'struct dns_options' has been removed from 'struct resolution' (well,
  we might need it back at some point later, we'll see)
  ==> we'll use the 'struct dns_options' from the owner of the resolution
- dns_get_ip_from_response(): takes a 'struct dns_options' instead of
  'struct resolution'
  ==> so the caller can pass its own dns options to get the most
      appropriate IP from the response
- dns_process_resolve(): struct dns_option is deduced from new
  resolution->requester_type parameter

2. add hostname_dn and hostname_dn_len into struct server

In order to avoid recomputing a server's hostname into its domain name
format (and use a trash buffer to store the result), it is safer to
compute it once at configuration parsing and to store it into the struct
server.
In the mean time, the struct resolution linked to the server doesn't
need anymore to store the hostname in domain name format. A simple
pointer to the server one will make the trick.

The function srv_alloc_dns_resolution() properly manages everything for
us: memory allocation, pointer updates, etc...

3. move resolvers pointer into struct server

This patch makes the pointer to struct dns_resolvers from struct
dns_resolution obsolete.
Purpose is to make the resolution as "neutral" as possible and since the
requester is already linked to the resolvers, then we don't need this
information anymore in the resolution itself.
2017-06-02 11:26:48 +02:00
Baptiste Assmann
4f91f7ea59 MINOR: dns: parse_server() now uses srv_alloc_dns_resolution()
In order to make DNS code more consistent, the function parse_server()
now uses srv_alloc_dns_resolution() to set up a server and its
resolution.
2017-06-02 11:20:50 +02:00
Baptiste Assmann
81ed1a0516 MINOR: dns: functions to manage memory for a DNS resolution structure
A couple of new functions to allocate and free memory for a DNS
resolution structure. Main purpose is to to make the code related to DNS
more consistent.
They allocate or free memory for the structure itself. Later, if needed,
they should also allocate / free the buffers, etc, used by this structure.
They don't set/unset any parameters, this is the role of the caller.

This patch also implement calls to these function eveywhere it is
required.
2017-06-02 11:20:29 +02:00
Baptiste Assmann
9d41fe7f98 CLEANUP: server.c: missing prototype of srv_free_dns_resolution
Prototype for the function srv_free_dns_resolution() missing at the top
of the file.
2017-06-02 11:18:28 +02:00
Frdric Lcaille
b418c1228c MINOR: server: cli: Add server FQDNs to server-state file and stats socket.
This patch adds a new stats socket command to modify server
FQDNs at run time.
Its syntax:
  set server <backend>/<server> fqdn <FQDN>
This patch also adds FQDNs to server state file at the end
of each line for backward compatibility ("-" if not present).
2017-05-03 06:58:53 +02:00
Frdric Lcaille
72ed4758d6 MINOR: server: Add server_template_init() function to initialize servers from a templates.
This patch adds server_template_init() function used to initialize servers
from server templates. It is called just after having parsed a 'server-template'
line.
2017-04-21 15:42:10 +02:00
Frdric Lcaille
b82f742b78 MINOR: server: Add 'server-template' new keyword supported in backend sections.
This patch makes backend sections support 'server-template' new keyword.
Such 'server-template' objects are parsed similarly to a 'server' object
by parse_server() function, but its first arguments are as follows:
    server-template <ID prefix> <nb | range> <ip | fqdn>:<port> ...

The remaining arguments are the same as for 'server' lines.

With such server template declarations, servers may be allocated with IDs
built from <ID prefix> and <nb | range> arguments.

For instance declaring:
    server-template foo 1-5 google.com:80 ...
or
    server-template foo 5 google.com:80 ...

would be equivalent to declare:
    server foo1 google.com:80 ...
    server foo2 google.com:80 ...
    server foo3 google.com:80 ...
    server foo4 google.com:80 ...
    server foo5 google.com:80 ...
2017-04-21 15:42:10 +02:00
Frdric Lcaille
759ea98db2 MINOR: server: Extract the code which finalizes server initializations after 'server' lines parsing.
This patch moves the code which is responsible of finalizing server initializations
after having fully parsed a 'server' line (health-check, agent check and SNI expression
initializations) from parse_server() to new functions.
2017-04-21 15:42:10 +02:00
Frdric Lcaille
58b207cdd5 MINOR: server: Extract the code responsible of copying default-server settings.
This patch moves the code responsible of copying default server settings
to a new server instance from parse_server() function to new defsrv_*_cpy()
functions which may be used both during server lines parsing and during server
templates initializations to come.

These defsrv_*_cpy() do not make any reference to anything else than default
server settings.
2017-04-21 15:42:10 +02:00
Frdric Lcaille
daa2fe6621 BUG/MINOR: server: missing default server 'resolvers' setting duplication.
'resolvers' setting was not duplicated from default server setting to
new server instances when parsing 'server' lines.
This fix is simple: strdup() default resolvers <id> string argument after
having allocated a new server when parsing 'server' lines.

This patch must be backported to 1.7 and 1.6.
2017-04-21 15:42:09 +02:00
Olivier Houchard
7d8e688953 BUG/MINOR: server: don't use "proxy" when px is really meant.
In server_parse_sni_expr(), we use the "proxy" global variable, when we
should probably be using "px" given as an argument.
It happens to work by accident right now, but may not in the future.

[wt: better backport it]
2017-04-20 19:51:10 +02:00
Frdric Lcaille
dfacd69b94 BUG/MAJOR: Broken parsing for valid keywords provided after 'source' setting.
Any valid keyword could not be parsed anymore if provided after 'source' keyword.
This was due to the fact that 'source' number of arguments is variable.
So, as its parser srv_parse_source() is the only one who may know how many arguments
was provided after 'source' keyword, it updates 'cur_arg' variable (the index
in the line of the current arg to be parsed), this is a good thing.
This variable is also incremented by one (to skip the 'source' keyword).
This patch disable this behavior.

Should have come with dba9707 commit.
2017-04-16 18:13:06 +02:00
Frdric Lcaille
8d083ed796 BUG/MINOR: server: Fix a wrong error message during 'usesrc' keyword parsing.
'usesrc' setting is not permitted on 'server' lines if not provided after
'source' setting. This is now also the case on 'default-server' lines.
Without this patch parse_server() parser displayed that 'usersrc' is
an unknown keyword.

Should have come with dba9707 commit.
2017-04-15 13:42:55 +02:00
Willy Tarreau
04bf98149b BUG/MEDIUM: servers: unbreak server weight propagation
This reverts commit 266b1a8 ("MEDIUM: server: Inherit CLI weight changes and
agent-check weight responses") from Michal Idzikowski, which is still broken.
It stops propagating weights at the first error encountered, leaving servers
in a random state depending on what LB algorithms are used on other servers
tracking the one experiencing the weight change. It's unsure what the best
way to address this is, but we cannot leave the servers in an inconsistent
state between farms. For example :

  backend site1
      mode http
      balance uri
      hash-type consistent
      server s1 127.0.0.1:8001 weight 10 track servers/s1

  backend site2
      mode http
      balance uri
      server s1 127.0.0.1:8001 weight 10 track servers/s1

  backend site3
      mode http
      balance uri
      hash-type consistent
      server s1 127.0.0.1:8001 weight 10 track servers/s1

  backend servers
      server s1 127.0.0.1:8001 weight 10 check inter 1s

The weight change is applied on "servers/s1". It tries to propagate
to the servers tracking it, which are site1/s1, site2/s1 and site3/s1.
Let's say that "weight 50%" is requested. The servers are linked in
reverse-order, so the change is applied to "servers/s1", then to
"site3/s1", then to "site2/s1" and this one fails and rejects the
change. The change is aborted and never propagated to "site1/s1",
which keeps the server in a different state from "site3/s1". At the
very least, in case of error, the changes should probably be unrolled.

Also the error reported on the CLI (when changing from the CLI) simply says :

  Backend is using a static LB algorithm and only accepts weights '0%' and '100%'.

Without more indications what the faulty backend is.

Let's revert this change for now, as initially feared it will definitely
cause more harm than good and at least needs to be revisited. It was never
backported to any stable branch so no backport is needed.
2017-04-13 15:09:26 +02:00
Michal Idzikowski
266b1a8336 MEDIUM: server: Inherit CLI weight changes and agent-check weight responses
When agent-check or CLI command executes relative weight change this patch
propagates it to tracking server allowing grouping many backends running on
same server underneath. Additionaly in case with many src IPs many backends
can have shared state checker, so there won't be unnecessary health checks.

[wt: Note: this will induce some behaviour change on some setups]
2017-04-13 11:31:38 +02:00
David Carlier
3a471935e6 BUG/MINOR: server : no transparent proxy for DragonflyBSD
IP*_BINDANY is not defined under this system thus it is
necessary to make those fields access since CONFIG_HAP_TRANSPARENT
is not defined.
[wt: problem introduced late in 1.8-dev. The same fix was also reported
  by Steven Davidovitz]
2017-04-10 15:27:46 +02:00
Olivier Houchard
b4a2d5e19a MINOR server: Restrict dynamic cookie check to the same proxy.
Each time we generate a dynamic cookie, we try to make sure the same
cookie hasn't been generated for another server, it's very unlikely, but
it may happen.
We only have to check that for the servers in the same proxy, no, need to
check in others, plus the code was buggy and would always check in the
first proxy of the proxy list.
2017-04-10 15:20:11 +02:00
David Carlier
6f1820864b CLEANUP: server: moving netinet/tcp.h inclusion
netinet/tcp.h needs sys/types.h for u_int* types usage,
issue found while building on OpenBSD.
2017-04-04 08:22:48 +02:00
Frdric Lcaille
acd4827eca BUG/MEDIUM: server: Wrong server default CRT filenames initialization.
This patch fixes a bug which came with 5e57643 commit where server
default CRT filenames were initialized to the same value as server
default CRL filenames.
2017-03-29 16:37:56 +02:00
Frdric Lcaille
6e0843c0e0 MINOR: server: Add 'no-agent-check' server keyword.
This patch adds 'no-agent-check' setting supported both by 'default-server'
and 'server' directives to disable an agent check for a specific server which would
have 'agent-check' set as default value (inherited from 'default-server'
'agent-check' setting), or, on 'default-server' lines, to disable 'agent-check' setting
as default value for any further 'server' declarations.

For instance, provided this configuration:

    default-server agent-check
    server srv1
    server srv2 no-agent-check
    server srv3
    default-server no-agent-check
    server srv4

srv1 and srv3 would have an agent check enabled contrary to srv2 and srv4.

We do not allocate anymore anything when parsing 'default-server' 'agent-check'
setting.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
2a0d061a60 MINOR: server: Make 'default-server' support 'disabled' keyword.
Before this patch, only 'server' directives could support 'disabled' setting.
This patch makes also 'default-server' directives support this setting.
It is used to disable a list of servers declared after a 'defaut-server' directive.
'enabled' new keyword has been added, both supported as 'default-server' and
'server' setting, to enable again a list of servers (so, declared after a
'default-server enabled' directive) or to explicitly enable a specific server declared
after a 'default-server disabled' directive.

For instance provided this configuration:

    default-server disabled
    server srv1...
    server srv2...
    server srv3... enabled
    server srv4... enabled

srv1 and srv2 are disabled and srv3 and srv4 enabled.

This is equivalent to this configuration:

    default-server disabled
    server srv1...
    server srv2...
    default-server enabled
    server srv3...
    server srv4...

even if it would have been preferable/shorter to declare:

    server srv3...
    server srv4...
    default-server disabled
    server srv1...
    server srv2...

as 'enabled' is the default server state.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
6e5e0d8f9e MINOR: server: Make 'default-server' support 'addr' keyword.
This patch makes 'default-server' support 'addr' setting.
The code which was responsible of parsing 'server' 'addr' setting
has moved from parse_server() to implement a new parser
callable both as 'default-server' and 'server' 'addr' setting parser.

Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
9a146de934 MINOR: server: Make 'default-server' support 'sni' keyword.
This patch makes 'default-server' directives support 'sni' settings.
A field 'sni_expr' has been added to 'struct server' to temporary
stores SNI expressions as strings during both 'default-server' and 'server'
lines parsing. So, to duplicate SNI expressions from 'default-server' 'sni' setting
for new 'server' instances we only have to "strdup" these strings as this is
often done for most of the 'server' settings.
Then, sample expressions are computed calling sample_parse_expr() (only for 'server'
instances).
A new function has been added to produce the same error output as before in case
of any error during 'sni' settings parsing (display_parser_err()).
Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
dba9707713 MINOR: server: Make 'default-server' support 'source' keyword.
Before this patch, only 'server' directives could support 'source' setting.
This patch makes also 'default-server' directives support this setting.

To do so, we had to extract the code responsible of parsing 'source' setting
arguments from parse_server() function and make it callable both
as 'default-server' and 'server' 'source' setting parser. So, the code is mostly
the same as before except that before allocating anything for 'struct conn_src'
members, we must free the memory previously allocated.

Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
22f41a2d23 MINOR: server: Make 'default-server' support 'namespace' keyword.
Before this patch, 'namespace' setting was only supported by 'server' directive.
This patch makes 'default-server' directive support this setting.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
5c3cd97550 MINOR: server: Make 'default-server' support 'tcp-ut' keyword.
This patch makes 'default-server' directive support 'tcp-ut' keyword.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
bcaf1d7397 MINOR: server: Make 'default-server' support 'ciphers' keyword.
This patch makes 'default-server' directive support 'ciphers' setting.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
9d1b95b591 MINOR: server: Make 'default-server' support 'cookie' keyword.
Before this patch, 'cookie' setting was only supported by 'server' directives.
This patch makes 'default-server' directive also support 'cookie' setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
547356e484 MINOR: server: Make 'default-server' support 'observe' keyword.
Before this path, 'observe' setting was only supported by 'server' directives.
This patch makes 'default-server' directives also support 'observe' setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
16186236dd MINOR: server: Make 'default-server' support 'redir' keyword.
Before this patch only 'server' directives could support 'redir' setting.
This patch makes also 'default-server' directives support 'redir' setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
5e57643e09 MINOR: server: Make 'default-server' support 'ca-file', 'crl-file' and 'crt' settings.
This patch makes 'default-server' directives support 'ca-file', 'crl-file' and
'crt' settings.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
67e0e61316 MINOR: server: Make 'default-server' support 'track' setting.
Before this patch only 'server' directives could support 'track' setting.
This patch makes 'default-server' directives also support this setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
65aa356c0b MINOR: server: Make 'default-server' support 'check' keyword.
Before this patch 'check' setting was only supported by 'server' directives.
This patch makes also 'default-server' directives support this setting.
A new 'no-check' keyword parser has been implemented to disable this setting both
in 'default-server' and 'server' directives.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
273f321404 MINOR: server: Make 'default-server' support 'verifyhost' setting.
This patch makes 'default-server' directive support 'verifyhost' setting.
Note: there was a little memory leak when several 'verifyhost' arguments were
supplied on the same 'server' line.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
7c8cd587c2 MINOR: server: Make 'default-server' support 'verify' keyword.
This patch makes 'default-server' directive support 'verify' keyword.
2017-03-27 14:37:01 +02:00
Frdric Lcaille
31045e4c10 MINOR: server: Make 'default-server' support 'send-proxy' and 'send-proxy-v2 keywords.
This patch makes 'default-server' directive support 'send-proxy'
(resp. 'send-proxy-v2') setting.
A new keyword 'no-send-proxy' (resp. 'no-send-proxy-v2') has been added
to disable 'send-proxy' (resp. 'send-proxy-v2') setting both in 'server' and
'default-server' directives.
2017-03-27 14:36:12 +02:00
Frdric Lcaille
f9bc1d6a13 MINOR: server: Make 'default-server' support 'non-stick' keyword.
This patch makes 'default-server' directive support 'non-stick' setting.
A new keyword 'stick' has been added so that to disable
'non-stick' setting both in 'server' and 'default-server' directives.
2017-03-27 14:36:11 +02:00
Frdric Lcaille
1502cfd1a3 CLEANUP: server: code alignement.
Code alignement.
2017-03-27 14:36:11 +02:00
Frdric Lcaille
25df89066b MINOR: server: Make 'default-server' support 'check-send-proxy' keyword.
This patch makes 'default-server' directive support 'check-send-proxy' setting.
A new keyword 'no-check-send-proxy' has been added so that to disable
'check-send-proxy' setting both in 'server' and 'default-server' directives.
2017-03-27 14:36:11 +02:00
Frdric Lcaille
f5bf903be6 MINOR: server: Make 'default-server' support 'backup' keyword.
At this time, only 'server' supported 'backup' keyword.
This patch makes also 'default-server' directive support this keyword.
A new keyword 'no-backup' has been added so that to disable 'backup' setting
both in 'server' and 'default-server' directives.

For instance, provided the following sequence of directives:

default-server backup
server srv1
server srv2 no-backup

default-server no-backup
server srv3
server srv4 backup

srv1 and srv4 are declared as backup servers,
srv2 and srv3 are declared as non-backup servers.
2017-03-27 14:36:11 +02:00
Frdric Lcaille
8065b6d4f2 MINOR: server: irrelevant error message with 'default-server' config file keyword.
There is no reason to emit such an error message:
"'default-server' expects <name> and <addr>[:<port>] as arguments."
if less than two arguments are provided on 'default-server' lines.
This is a 'server' specific error message.
2017-03-27 14:33:58 +02:00
Olivier Houchard
2cb49ebbc4 BUG/MEDIUM server: Fix crash when dynamic is defined, but not key is provided.
Wait until we're sure we have a key before trying to calculate its length.

[wt: no backport needed, was just merged]
2017-03-15 16:01:33 +01:00
Olivier Houchard
4e694049fa MINOR: server: Add dynamic session cookies.
This adds a new "dynamic" keyword for the cookie option. If set, a cookie
will be generated for each server (assuming one isn't already provided on
the "server" line), from the IP of the server, the TCP port, and a secret
key provided. To provide the secret key, a new keyword as been added,
"dynamic-cookie-key", for backends.

Example :
backend bk_web
  balance roundrobin
  dynamic-cookie-key "bla"
  cookie WEBSRV insert dynamic
  server s1 127.0.0.1:80 check
  server s2 192.168.56.1:80 check

This is a first step to be able to dynamically add and remove servers,
without modifying the configuration file, and still have all the load
balancers redirect the traffic to the right server.

Provide a way to generate session cookies, based on the IP address of the
server, the TCP port, and a secret key provided.
2017-03-15 11:37:30 +01:00
Misiek
2da082d732 MINOR: cli: Add possiblity to change agent config via CLI/socket
This change adds possibility to change agent-addr and agent-send directives
by CLI/socket. Now you can replace server's and their configuration without
reloading/restarting whole haproxy, so it's a step in no-reload/no-restart
direction.

Depends on #e9602af - agent-addr is implemented there.

Can be backported to 1.7.
2017-01-16 11:38:59 +01:00
Misiek
ea849333ca MINOR: checks: Add agent-addr config directive
This directive add possibility to set different address for agent-checks.
With this you can manage server status and weight from central place.

Can be backported to 1.7.
2017-01-16 11:38:02 +01:00
Ryabin Sergey
77ee7526de BUG/MINOR: Reset errno variable before calling strtol(3)
Sometimes errno != 0 before calling strtol(3)

[wt: this needs to be backported to 1.7]
2017-01-11 21:30:07 +01:00
Willy Tarreau
9698f4b295 MEDIUM: server: disable protocol validations when the server doesn't resolve
When a server doesn't resolve we don't know the address family so we
can't perform the basic protocol validations. However we know that we'll
ultimately resolve to AF_INET4 or AF_INET6 so the controls are OK. It is
important to proceed like this otherwise it will not be possible to start
with unresolved addresses.
2017-01-06 19:29:34 +01:00
Willy Tarreau
6ecb10aec7 MINOR: server: take the destination port from the port field, not the addr
Next patch will cause the port to disappear from the address field when servers
do not resolve so we need to take it from the separate field provided by
str2sa_range().
2017-01-06 19:29:34 +01:00
Willy Tarreau
48ef4c95b6 MINOR: tools: make str2sa_range() return the port in a separate argument
This will be needed so that we're don't have to extract it from the
returned address where it will not always be anymore (eg: for unresolved
servers).
2017-01-06 19:29:34 +01:00
Willy Tarreau
04276f3d6e MEDIUM: server: split the address and the port into two different fields
Keeping the address and the port in the same field causes a lot of problems,
specifically on the DNS part where we're forced to cheat on the family to be
able to keep the port. This causes some issues such as some families not being
resolvable anymore.

This patch first moves the service port to a new field "svc_port" so that the
port field is never used anymore in the "addr" field (struct sockaddr_storage).
All call places were adapted (there aren't that many).
2017-01-06 19:29:33 +01:00
Willy Tarreau
3acfcd1aa1 BUG/MEDIUM: server: consider AF_UNSPEC as a valid address family
The DNS code is written so as to support AF_UNSPEC to decide on the
server family based on responses, but unfortunately snr_resolution_cb()
considers it as invalid causing a DNS storm to happen when a server
arrives with this family.

This situation is not supposed to happen as long as unresolved addresses
are forced to AF_INET, but this will change with the upcoming fixes and
it's possible that it's not granted already when changing an address on
the CLI.

This fix must be backported to 1.7 and 1.6.
2017-01-06 19:21:37 +01:00
Willy Tarreau
a261e9b094 CLEANUP: connection: remove all direct references to raw_sock and ssl_sock
Now we exclusively use xprt_get(XPRT_RAW) instead of &raw_sock or
xprt_get(XPRT_SSL) for &ssl_sock. This removes a bunch of #ifdef and
include spread over a number of location including backend, cfgparse,
checks, cli, hlua, log, server and session.
2016-12-22 23:26:38 +01:00
Willy Tarreau
141ad85d10 MINOR: server: move the use_ssl field out of the ifdef USE_OPENSSL
Having it in the ifdef complicates certain operations which require
additional ifdefs just to access a member which could remain zero in
non-ssl cases. Let's move it out, it will not even increase the
struct size on 64-bit machines due to alignment.
2016-12-22 23:26:38 +01:00
Christopher Faulet
90b5abe46e BUG/MINOR: cli: be sure to always warn the cli applet when input buffer is full
[wt: may only strike if CLI commands are pipelined. Must be backported to 1.7
 and 1.6, where it's a bit different and in dumpstats.c]
2016-12-12 17:58:11 +01:00
Willy Tarreau
3b6e547be8 CLEANUP: cli: rename STAT_CLI_* to CLI_ST_*
These are in CLI states, not stats states anymore. STAT_CLI_O_CUSTOM
was more appropriately renamed CLI_ST_CALLBACK.
2016-11-24 16:59:28 +01:00
Willy Tarreau
58d9cb7d22 REORG: cli: move "{enable|disable} agent" to server.c
Also mention that "set server" is preferred now. Note that these
were the last enable/disable commands in cli.c. Also remove the
now unused expect_server_admin() function.
2016-11-24 16:59:28 +01:00
Willy Tarreau
2c04eda8b5 REORG: cli: move "{enable|disable} health" to server.c
Also mention that "set server" is preferred now.
2016-11-24 16:59:28 +01:00
Willy Tarreau
ffb4d58e1b REORG: cli: move "{enable|disable} server" to server.c
Also mention that "set server" is preferred now.
2016-11-24 16:59:28 +01:00
Willy Tarreau
b802627eb3 REORG: cli: move "set maxconn server" to server.c
It's used to manipulate the server's maxconn setting.
2016-11-24 16:59:28 +01:00
William Lallemand
6b16094355 REORG: cli: move get/set weight to server.c
Move get/set weight CLI functions to server.c and use the cli keyword API
to register it on the CLI.
2016-11-24 16:59:27 +01:00
William Lallemand
222baf20da REORG: cli: move 'set server' to server.c
Move 'set server' CLI functions to server.c and use the cli keyword API
to register it on the CLI.
2016-11-24 16:59:27 +01:00
Willy Tarreau
21b069dca8 MINOR: server: create new function cli_find_server() to find a server
Several CLI commands require a server, so let's have a function to
look this one up and prepare the appropriate error message and the
appctx's state in case of failure.
2016-11-24 16:59:27 +01:00
David Carlier
327298c215 BUILD: fix build on Solaris 10/11
uint16_t instead of u_int16_t
None ISO fields of struct tm are not present, but
by zeroyfing it, on GNU and BSD systems tm_gmtoff
field will be set.

[wt: moved the memset into each of the date functions]
2016-11-22 12:04:19 +01:00
Willy Tarreau
e5a60688a4 MEDIUM: server: do not restrict anymore usage of IP address from the state file
Now that it is possible to decide whether we prefer to use libc or the
state file to resolve the server's IP address and it is possible to change
a server's IP address at run time on the CLI, let's not restrict the reuse
of the address from the state file anymore to the DNS only.

The impact is that by default the state file will be considered first
(which matches its purpose) and only then the libc. This way any address
change performed at run time over the CLI will be preserved regardless
of DNS usage or not.
2016-11-09 15:33:52 +01:00
Willy Tarreau
3eed10e54b MINOR: init: add -dr to ignore server address resolution failures
It is very common when validating a configuration out of production not to
have access to the same resolvers and to fail on server address resolution,
making it difficult to test a configuration. This option simply appends the
"none" method to the list of address resolution methods for all servers,
ensuring that even if the libc fails to resolve an address, the startup
sequence is not interrupted.
2016-11-09 15:33:52 +01:00
Willy Tarreau
4310d36a7e MINOR: server: add support for explicit numeric address in init-addr
This will allow a server to automatically fall back to an explicit numeric
IP address when all other methods fail. The address is simply specified in
the address list.
2016-11-09 15:30:47 +01:00
Willy Tarreau
465b6e5463 MEDIUM: server: make libc resolution failure non-fatal
Now that we have "init-addr none", it becomes possible to recover on
libc resolver's failures. Thus it's preferable not to alert nor fail
at the moment the libc is called, and instead process the failure at
the end of the list. This allows "none" to be set after libc to
provide a smooth fallback in case of resolver issues.
2016-11-09 15:30:47 +01:00
Willy Tarreau
37ebe1212b MINOR: server: implement init-addr none
The server is put into the "no address" maintenance state in this case.
2016-11-09 15:30:47 +01:00
Willy Tarreau
25e515235a MEDIUM: server: make use of init-addr
It is now supported. If not set, we default to the legacy methods list
which is "last,libc".
2016-11-09 15:30:47 +01:00
Baptiste Assmann
25938278b7 MEDIUM: server: add a new init-addr server line setting
This new setting supports a comma-delimited list of methods used to
resolve the server's FQDN to an IP address. Currently supported methods
are "libc" (use the regular libc's resolver) and "last" (use the last
known valid address found in the state file).

The list is implemented in a 32-bit integer, because each init-addr
method only requires 3 bits. The last one must always be SRV_IADDR_END
(0), allowing to store up to 10 methods in a single 32 bit integer.

Note: the doc is provided at the end of this series.
2016-11-09 15:30:47 +01:00
Baptiste Assmann
3b9fe9f8f4 MAJOR: dns: runtime resolution can change server admin state
WARNING: this is a MAJOR (and disruptive) change with previous HAProxy's
behavior: before, HAProxy never ever used to change a server administrative
status when the DNS resolution failed at run time.

This patch gives HAProxy the ability to change the administrative status
of a server to MAINT (RMAINT actually) when an error is encountered for
a period longer than its own allowed by the corresponding 'hold'
parameter.

IE if the configuration sets "hold nx 10s" and a server's hostname
points to a NX for more than 10s, then the server will be set to RMAINT,
hence in MAINTENANCE mode.
2016-11-09 15:30:47 +01:00
Willy Tarreau
8b42848a44 MINOR: server: make srv_set_admin_state() capable of telling why this happens
It will be important to help debugging some DNS resolution issues to
know why a server was marked down, so let's make  the function support
a 3rd argument with an indication of the reason. Passing NULL will keep
the message as-is.
2016-11-09 15:30:47 +01:00
Willy Tarreau
e659973bfe MINOR: server: indicate in the logs when RMAINT is cleared
It's important to report in the server state change logs that RMAINT was
cleared, as it's not the regular maintenance mode, it's specific to name
resolution, and it's important to report the new state (which can be DRAIN
or READY).
2016-11-09 15:23:37 +01:00
Baptiste Assmann
83cbaa531f MAJOR: server: postpone address resolution
Server addresses are not resolved anymore upon the first pass so that we
don't fail if an address cannot be resolved by the libc. Instead they are
processed all at once after the configuration is fully loaded, by the new
function srv_init_addr(). This function only acts on the server's address
if this address uses an FQDN, which appears in server->hostname.

For now the function does two things, to followup with HAProxy's historical
default behavior:

  1. apply server IP address found in server-state file if runtime DNS
     resolution is enabled for this server

  2. use the DNS resolver provided by the libc

If none of the 2 options above can find an IP address, then an error is
returned.

All of this will be needed to support the new server parameter "init-addr".
For now, the biggest user-visible change is that all server resolution errors
are dumped at once instead of causing a startup failure one by one.
2016-11-09 14:24:20 +01:00
Willy Tarreau
757478e900 BUG/MEDIUM: servers: properly propagate the maintenance states during startup
Right now there is an issue with the way the maintenance flags are
propagated upon startup. They are not propagate, just copied from the
tracked server. This implies that depending on the server's order, some
tracking servers may not be marked down. For example this configuration
does not work as expected :

        server s1 1.1.1.1:8000 track s2
        server s2 1.1.1.1:8000 track s3
        server s3 1.1.1.1:8000 track s4
        server s4 wtap:8000 check inter 1s disabled

It results in s1/s2 being up, and s3/s4 being down, while all of them
should be down.

The only clean way to process this is to run through all "root" servers
(those not tracking any other server), and to propagate their state down
to all their trackers. This is the same algorithm used to propagate the
state changes. It has to be done both to compute the IDRAIN flag and the
IMAINT flag. However, doing so requires that tracking servers are not
marked as inherited maintenance anymore while parsing the configuration
(and given that it is wrong, better drop it).

This fix also addresses another side effect of the bug above which is
that the IDRAIN/IMAINT flags are stored in the state files, and if
restored while the tracked server doesn't have the equivalent flag,
the servers may end up in a situation where it's impossible to remove
these flags. For example in the configuration above, after removing
"disabled" on server s4, the other servers would have remained down,
and not anymore with this fix. Similarly, the combination of IMAINT
or IDRAIN with their respective forced modes was not accepted on
reload, which is wrong as well.

This bug has been present at least since 1.5, maybe even 1.4 (it came
with tracking support). The fix needs to be backported there, though
the srv-state parts are irrelevant.

This commit relies on previous patch to silence warnings on startup.
2016-11-07 14:31:52 +01:00
Willy Tarreau
6fb8dc1a5a MINOR: server: do not emit warnings/logs/alerts on server state changes at boot
We'll have to use srv_set_admin_flag() to propagate some server flags
during the startup, and we don't want the resulting actions to cause
warnings, logs nor e-mail alerts to be generated since we're just applying
the config or a state file. So let's condition these notifications to the
fact that we're starting.
2016-11-07 14:31:45 +01:00
Willy Tarreau
e1bde1492a BUG/MINOR: srv-state: allow to have both CMAINT and FDRAIN flags
CMAINT indicates that the server was *initially* disabled in the
configuration via the "disabled" keyword. FDRAIN indicates that the
server was switched to the DRAIN state from the CLI or the agent.
This it's perfectly valid to have both of them in the state file,
so the parser must not reject this combination.

This fix must be backported to 1.6.
2016-11-07 14:30:19 +01:00
Willy Tarreau
22cace2f4c BUG/MEDIUM: srv-state: properly restore the DRAIN state
There were seveal reports about the DRAIN state not being properly
restored upon reload.

It happens that the condition in the code does exactly the opposite
of what the comment says, and the comment is right so the code is
wrong.

It's worth noting that the conditions are complex here due to the 2
available methods to set the drain state (CLI/agent, and config's
weight). To paraphrase the updated comment in the code, there are
two possible reasons for FDRAIN to have been present :
  - previous config weight was zero
  - "set server b/s drain" was sent to the CLI

In the first case, we simply want to drop this drain state if the new
weight is not zero anymore, meaning the administrator has intentionally
turned the weight back to a positive value to enable the server again
after an operation. In the second case, the drain state was forced on
the CLI regardless of the config's weight so we don't want a change to
the config weight to lose this status. What this means is :
  - if previous weight was 0 and new one is >0, drop the DRAIN state.
  - if the previous weight was >0, keep it.

This fix must be backported to 1.6.
2016-11-07 14:30:19 +01:00
Willy Tarreau
dc3a9e830c CLEANUP: tools: make ipcpy() preserve the original port
ipcpy() is used to replace an IP address with another one, but it
doesn't preserve the original port so all callers have to do it
manually while it's trivial to do there. Better do it inside the
function.
2016-11-05 13:56:04 +01:00
Baptiste Assmann
65ce3f5ee4 MINOR: dns: query type change when last record is a CNAME
DNS servers don't return A or AAAA record if the query points to a CNAME
not resolving to the right type.
We know it because the last record of the response is a CNAME. We can
trigger a new query, switching to a new query type, handled by the layer
above.
2016-09-12 20:01:40 +02:00
Baptiste Assmann
c1ce5f358e MEDIUM: dns: new DNS response parser
New DNS response parser function which turn the DNS response from a
network buffer into a DNS structure, much easier for later analysis
by upper layer.

Memory is pre-allocated at start-up in a chunk dedicated to DNS
response store.

New error code to report a wrong number of queries in a DNS response.
2016-09-12 19:54:23 +02:00
Baptiste Assmann
d458adcc52 MINOR: new update_server_addr_port() function to change both server's ADDR and service PORT
This function can replace update_server_addr() where the need to change the
server's port as well as the IP address is required.
It performs some validation before performing each type of change.
2016-09-11 08:13:11 +02:00
Baptiste Assmann
6b453f166f MINOR: server: introduction of 3 new server flags
Introduction of 3 new server flags to remember if some parameters were set
during configuration parsing.

* SRV_F_CHECKADDR: this server has a check addr configured
* SRV_F_CHECKPORT: this server has a check port configured
* SRV_F_AGENTADDR: this server has a agent addr configured
2016-09-11 08:12:42 +02:00
Baptiste Assmann
95db2bcfee MAJOR: check: find out which port to use for health check at run time
HAProxy used to deduce port used for health checks when parsing configuration
at startup time.
Because of this way of working, it makes it complicated to change the port at
run time.

The current patch changes this behavior and makes HAProxy to choose the
port used for health checking when preparing the check task itself.

A new type of error is introduced and reported when no port can be found.

There won't be any impact on performance, since the process to find out the
port value is made of a few 'if' statements.

This patch also introduces a new check state CHK_ST_PORT_MISS: this flag is
used to report an error in the case when HAProxy needs to establish a TCP
connection to a server, to perform a health check but no TCP ports can be
found for it.

And last, it also introduces a new stream termination condition:
SF_ERR_CHK_PORT. Purpose of this flag is to report an error in the event when
HAProxy has to run a health check but no port can be found to perform it.
2016-09-11 08:12:13 +02:00
Baptiste Assmann
d260e1dea6 MAJOR: listen section: don't use first bind port anymore when no server ports are provided
Up to HAProxy 1.7-dev3, HAProxy used to use the first bind port from it's
local 'listen' section when no port is configured on the server.

IE, in the configuration below, the server port would be 25:

  listen smtp
   bind :25
   server s1 1.0.0.1 check

This way of working is now obsolete and can be removed, furthermore it is not
documented!

This will make the possibility to change the server's port much easier.
2016-08-14 12:18:14 +02:00
Willy Tarreau
eec1d3869d BUG/MEDIUM: dns: fix alignment issues in the DNS response parser
Alexander Lebedev reported that the DNS parser crashes in 1.6 with a bus
error on Sparc when it receives a response. This is obviously caused by
some alignment issues. The issue can also be reproduced on ARMv5 when
setting /proc/cpu/alignment to 4 (which helps debugging).

Two places cause this crash in turn, the first one is when the IP address
from the packet is compared to the current one, and the second place is
when the address is assigned because an unaligned address is passed to
update_server_addr().

This patch modifies these places to properly use memcpy() and memcmp()
to manipulate the unaligned data.

Nenad Merdanovic found another set of places specific to 1.7 in functions
in_net_ipv4() and in_net_ipv6(), which are used to compare networks. 1.6
has the functions but does not use them. There we perform a temporary copy
to a local variable to fix the problem. The type of the function's argument
is wrong since it's not necessarily aligned, so we change it for a const
void * instead.

This fix must be backported to 1.6. Note that in 1.6 the code is slightly
different, there's no rec[] array, the pointer is used directly from the
buffer.
2016-07-13 12:13:24 +02:00
Nenad Merdanovic
174dd37d88 MINOR: Add ability for agent-check to set server maxconn
This is very useful in complex architecture systems where HAproxy
is balancing DB connections for example. We want to keep the maxconn
high in order to avoid issues with queueing on the LB level when
there is slowness on another part of the system. Example is a case of
an architecture where each thread opens multiple DB connections, which
if get stuck in queue cause a snowball effect (old connections aren't
closed, new ones cannot be established). These connections are mostly
idle and the DB server has no problem handling thousands of them.

Allowing us to dynamically set maxconn depending on the backend usage
(LA, CPU, memory, etc.) enables us to have high maxconn for situations
like above, but lowering it in case there are real issues where the
backend servers become overloaded (cache issues, DB gets hit hard).
2016-04-25 17:23:50 +02:00
David Carlier
d10025c671 BUG/MINOR: server: risk of over reading the pref_net array.
dns_option struct pref_net field is an array of 5. The issue
here shows that pref_net_nb can go up to 5 as well which might lead
to read outside of this array.
2016-04-12 11:00:39 +02:00
Vincent Bernat
02779b6263 CLEANUP: uniformize last argument of malloc/calloc
Instead of repeating the type of the LHS argument (sizeof(struct ...))
in calls to malloc/calloc, we directly use the pointer
name (sizeof(*...)). The following Coccinelle patch was used:

@@
type T;
T *x;
@@

  x = malloc(
- sizeof(T)
+ sizeof(*x)
  )

@@
type T;
T *x;
@@

  x = calloc(1,
- sizeof(T)
+ sizeof(*x)
  )

When the LHS is not just a variable name, no change is made. Moreover,
the following patch was used to ensure that "1" is consistently used as
a first argument of calloc, not the last one:

@@
@@

  calloc(
+ 1,
  ...
- ,1
  )
2016-04-03 14:17:42 +02:00
Vincent Bernat
3c2f2f207f CLEANUP: remove unneeded casts
In C89, "void *" is automatically promoted to any pointer type. Casting
the result of malloc/calloc to the type of the LHS variable is therefore
unneeded.

Most of this patch was built using this Coccinelle patch:

@@
type T;
@@

- (T *)
  (\(lua_touserdata\|malloc\|calloc\|SSL_get_app_data\|hlua_checkudata\|lua_newuserdata\)(...))

@@
type T;
T *x;
void *data;
@@

  x =
- (T *)
  data

@@
type T;
T *x;
T *data;
@@

  x =
- (T *)
  data

Unfortunately, either Coccinelle or I is too limited to detect situation
where a complex RHS expression is of type "void *" and therefore casting
is not needed. Those cases were manually examined and corrected.
2016-04-03 14:17:42 +02:00
Willy Tarreau
1a786d7f33 BUG/MINOR: tcpcheck: fix incorrect list usage resulting in failure to load certain configs
Commit baf9794 ("BUG/MINOR: tcpcheck: conf parsing error when no port
configured on server and first rule(s) is (are) COMMENT") was wrong, it
incorrectly implemented a list access by dereferencing a pointer of an
incorrect type resulting in checking the next element in the list. The
consequence is that it stops before the last comment instead of at the
last one and skips the first rule. In the end, rules starting with
comments are not affected, but if a sequence of checks directly starts
with connect, it is then skipped and this is visible when no port is
configured on the server line as the config refuses to load.

There was another occurence of the same bug a few lines below, both
of them were fixed. Tests were made on different configs and confirm
the new fix is OK.

This fix must be backported to 1.6.
2016-03-08 15:20:25 +01:00
Thierry Fournier
09a9178311 MINOR: server: generalize the "updater" source
the function server_parse_addr_change_request() contain an hardcoded
updater source "stats command". this function can be called from other
sources than the "stats command", so this patch make this argument
generic.
2016-02-24 23:37:39 +01:00
Thierry Fournier
d35b7a6d93 CLEANUP: server: add "const" to some message strings
"updater" is used in "read only" mode, so I add a const qualifier
to the variable declaration.
2016-02-24 23:37:39 +01:00
Thierry Fournier
c62df8463b BUG/MINOR: server: fix the format of the warning on address change
When the server address is changed, a message with unrequired '\n' or
'.' is displayed, like this:

   [WARNING] 054/101137 (3229) : zzzz/s3 changed its IP from 127.0.0.1 to ::55 by stats command
   .

This patch remove the '\n' which is sent before the '.'.

This patch must be backported in 1.6
2016-02-24 23:37:39 +01:00
Thierry Fournier
ac88cfe452 MEDIUM: dns: add a "resolve-net" option which allow to prefer an ip in a network
This options prioritize th choice of an ip address matching a network. This is
useful with clouds to prefer a local ip. In some cases, a cloud high
avalailibility service can be announced with many ip addresses on many
differents datacenters. The latency between datacenter is not negligible, so
this patch permitsto prefers a local datacenter. If none address matchs the
configured network, another address is selected.
2016-02-19 14:37:49 +01:00
Thierry Fournier
ada348459f MEDIUM: dns: extract options
DNS selection preferences are actually declared inline in the
struct server. There are copied from the server struct to the
dns_resolution struct for each resolution.

Next patchs adds new preferences options, and it is not a good
way to copy all the configuration information before each dns
resolution.

This patch extract the configuration preference from the struct
server and declares a new dedicated struct. Only a pointer to this
new striuict will be copied before each dns resolution.
2016-02-19 14:37:46 +01:00
Willy Tarreau
f3c7a83acc BUG/MEDIUM: servers state: server port is used uninitialized
Nenad spotted that the last fix was unfortunately wrong. Needs to be
backported to 1.6 as well.
2016-01-21 13:51:56 +01:00
Baptiste Assmann
a875b1f92e BUG/MAJOR: servers state: server port is erased when dns resolution is enabled on a server
Servers state function save and apply server IP when DNS resolution is
enabled on a server.
Purpose is to prevent switching traffic from one server to an other one
when multiple IPs are returned by the DNS server for the A or AAAA
record.

That said, a bug in current code lead to erase the service port while
copying the IP found in the file into the server structure in HAProxy's
memory.
This patch fix this bug.

The bug was reported on the ML by Robert Samuel Newson and fix proposed
by Nenad Merdanovic.
Thank you both!!!

backport: can be backported to 1.6
2016-01-21 10:47:12 +01:00
Baptiste Assmann
0821bb9ec0 MINOR: server state: missing LF (\n) on error message printed when parsing server state file
There is no LF characters printed at the end of the error message
returned by the function when applying server state found in a file.
2016-01-21 07:40:51 +01:00
Baptiste Assmann
baf9794b4d BUG/MINOR: tcpcheck: conf parsing error when no port configured on server and first rule(s) is (are) COMMENT
A small configuration parsing error exists when no port is setup on the
server IP:port statement and the server's parameter 'port' is not set
and if the first tcp-check rule is a comment, like in the example below:

  backend b
   option tcp-check
   tcp-check comment blah
   tcp-check connect 8444
   server s 127.0.0.1 check

In such case, an ALERT is improperly returned, despite this
configuration is valid and works.

The new code move the pointer to the first tcp-check rule which isn't a
comment before checking the presence of the port.

backport status: 1.6 and above
2015-12-04 07:48:44 +01:00