Commit Graph

246 Commits

Author SHA1 Message Date
Christopher Faulet
3bbd65b23e BUG/MINOR: dns: Fix check on nameserver in snr_resolution_cb
snr_resolution_cb can be called with <nameserver> parameter set to NULL. So we
must check it before using it. This is done most of time, except when we deal
with invalid DNS response.
2017-09-15 18:42:23 +02:00
Andjelko Iharos
c3680ecdf8 MINOR: add severity information to cli feedback messages 2017-09-13 13:38:32 +02:00
Willy Tarreau
3d609a755e Revert "BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file"
This reverts commit 19e8aa58f7.

It causes some trouble reported by Manu :
   listen tls
     [...]
     server bla 127.0.0.1:8080

   [ALERT] 248/130258 (21960) : parsing [/etc/haproxy/test.cfg:53] : 'server bla' : no method found to resolve address '(null)'
   [ALERT] 248/130258 (21960) : Failed to initialize server(s) addr.

According to Nenad :
  "It's not a good way to fix the issue we were experiencing
   before. It will need a bigger rewrite, because the logic in
   srv_iterate_initaddr needs to be changed."
2017-09-06 14:22:45 +02:00
Nenad Merdanovic
19e8aa58f7 BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file
Historically the DNS was the only way of updating the server IP dynamically
and the init-addr processing and state file load required the server to have
an FQDN defined. Given that we can now update the IP through the socket as
well and also can have different init-addr values (like IP and 'none') - this
requirement needs to be removed.

This patch should be backported to 1.7.
2017-09-05 15:52:58 +02:00
Emeric Brun
52a91d3d48 MEDIUM: check: server states and weight propagation re-work
The server state and weight was reworked to handle
"pending" values updated by checks/CLI/LUA/agent.
These values are commited to be propagated to the
LB stack.

In further dev related to multi-thread, the commit
will be handled into a sync point.

Pending values are named using the prefix 'next_'
Current values used by the LB stack are named 'cur_'
2017-09-05 15:23:16 +02:00
Baptiste Assmann
747359eeca BUG/MINOR: dns: server set by SRV records stay in "no resolution" status
This patch fixes a bug where some servers managed by SRV record query
types never ever recover from a "no resolution" status.
The problem is due to a wrong function called when breaking the
server/resolution (A/AAAA) relationship: this is performed when a server's SRV
record disappear from the SRV response.
2017-08-22 11:34:49 +02:00
Baptiste Assmann
6fb8192b28 MINOR: dns: enable caching of responses for server set by a SRV record
The function srv_set_fqdn() is used to update a server's fqdn and set
accordingly its DNS resolution.
Current implementation prevents a server whose update is triggered by a
SRV record from being linked to an existing resolution in the cache (if
applicable).
This patch aims at fixing this.
2017-08-18 11:25:41 +02:00
Olivier Houchard
8da5f98fbe MINOR: dns: Handle SRV records.
Make it so for each server, instead of specifying a hostname, one can use
a SRV label.
When doing so, haproxy will first resolve the SRV label, then use the
resulting hostnames, as well as port and weight (priority is ignored right
now), to each server using the SRV label.
It is resolved periodically, and any server disappearing from the SRV records
will be removed, and any server appearing will be added, assuming there're
free servers in haproxy.
2017-08-09 16:32:49 +02:00
Olivier Houchard
a8c6db8d2d MINOR: dns: Cache previous DNS answers.
As DNS servers may not return all IPs in one answer, we want to cache the
previous entries. Those entries are removed when considered obsolete, which
happens when the IP hasn't been returned by the DNS server for a time
defined in the "hold obsolete" parameter of the resolver section. The default
is 30s.
2017-08-09 16:32:49 +02:00
Frédéric Lécaille
3169471964 MINOR: Add server port field to server state file.
This patch adds server ports to server state file at the end of each line
for backward compatibility.
2017-08-03 14:31:46 +02:00
Frédéric Lécaille
0bedb8ac90 BUG/MAJOR: server: Segfault after parsing server state file.
This patch makes the server state file parser ignore servers wich are
not present in the configuration file.
2017-06-15 15:30:30 +02:00
Baptiste Assmann
201c07f681 MAJOR/REORG: dns: DNS resolution task and requester queues
This patch is a major upgrade of the internal run-time DNS resolver in
HAProxy and it brings the following 2 main changes:

1. DNS resolution task

Up to now, DNS resolution was triggered by the health check task.
From now, DNS resolution task is autonomous. It is started by HAProxy
right after the scheduler is available and it is woken either when a
network IO occurs for one of its nameserver or when a timeout is
matched.

From now, this means we can enable DNS resolution for a server without
enabling health checking.

2. Introduction of a dns_requester structure

Up to now, DNS resolution was purposely made for resolving server
hostnames.
The idea, is to ensure that any HAProxy internal object should be able
to trigger a DNS resolution. For this purpose, 2 things has to be done:
  - clean up the DNS code from the server structure (this was already
    quite clean actually) and clean up the server's callbacks from
    manipulating too much DNS resolution
  - create an agnostic structure which allows linking a DNS resolution
    and a requester of any type (using obj_type enum)

3. Manage requesters through queues

Up to now, there was an uniq relationship between a resolution and it's
owner (aka the requester now). It's a shame, because in some cases,
multiple objects may share the same hostname and may benefit from a
resolution being performed by a third party.
This patch introduces the notion of queues, which are basically lists of
either currently running resolution or waiting ones.

The resolutions are now available as a pool, which belongs to the resolvers.
The pool has has a default size of 64 resolutions per resolvers and is
allocated at configuration parsing.
2017-06-02 11:58:54 +02:00
Baptiste Assmann
fa4a663095 MINOR: dns: implement a LRU cache for DNS resolutions
Introduction of a DNS response LRU cache in HAProxy.

When a positive response is received from a DNS server, HAProxy stores
it in the struct resolution and then also populates a LRU cache with the
response.
For now, the key in the cache is a XXHASH64 of the hostname in the
domain name format concatened to the query type in string format.
2017-06-02 11:40:01 +02:00
Baptiste Assmann
729c901c3f MAJOR: dns: save a copy of the DNS response in struct resolution
Prior this patch, the DNS responses were stored in a pre-allocated
memory area (allocated at HAProxy's startup).
The problem is that this memory is erased for each new DNS responses
received and processed.

This patch removes the global memory allocation (which was not thread
safe by the way) and introduces a storage of the dns response  in the
struct
resolution.
The memory in the struct resolution is also reserved at start up and is
thread safe, since each resolution structure will have its own memory
area.

For now, we simply store the response and use it atomically per
response per server.
2017-06-02 11:30:21 +02:00
Baptiste Assmann
fb7091e213 MINOR: dns: new snr_check_ip_callback function
In the process of breaking links between dns_* functions and other
structures (mainly server and a bit of resolution), the function
dns_get_ip_from_response needs to be reworked: it now can call
"callback" functions based on resolution's owner type to allow modifying
the way the response is processed.

For now, main purpose of the callback function is to check that an IP
address is not already affected to an element of the same type.

For now, only server type has a callback.
2017-06-02 11:28:14 +02:00
Baptiste Assmann
42746373eb REORG: dns: dns_option structure, storage of hostname_dn
This patch introduces a some re-organisation around the DNS code in
HAProxy.

1. make the dns_* functions less dependent on 'struct server' and 'struct resolution'.

With this in mind, the following changes were performed:
- 'struct dns_options' has been removed from 'struct resolution' (well,
  we might need it back at some point later, we'll see)
  ==> we'll use the 'struct dns_options' from the owner of the resolution
- dns_get_ip_from_response(): takes a 'struct dns_options' instead of
  'struct resolution'
  ==> so the caller can pass its own dns options to get the most
      appropriate IP from the response
- dns_process_resolve(): struct dns_option is deduced from new
  resolution->requester_type parameter

2. add hostname_dn and hostname_dn_len into struct server

In order to avoid recomputing a server's hostname into its domain name
format (and use a trash buffer to store the result), it is safer to
compute it once at configuration parsing and to store it into the struct
server.
In the mean time, the struct resolution linked to the server doesn't
need anymore to store the hostname in domain name format. A simple
pointer to the server one will make the trick.

The function srv_alloc_dns_resolution() properly manages everything for
us: memory allocation, pointer updates, etc...

3. move resolvers pointer into struct server

This patch makes the pointer to struct dns_resolvers from struct
dns_resolution obsolete.
Purpose is to make the resolution as "neutral" as possible and since the
requester is already linked to the resolvers, then we don't need this
information anymore in the resolution itself.
2017-06-02 11:26:48 +02:00
Baptiste Assmann
4f91f7ea59 MINOR: dns: parse_server() now uses srv_alloc_dns_resolution()
In order to make DNS code more consistent, the function parse_server()
now uses srv_alloc_dns_resolution() to set up a server and its
resolution.
2017-06-02 11:20:50 +02:00
Baptiste Assmann
81ed1a0516 MINOR: dns: functions to manage memory for a DNS resolution structure
A couple of new functions to allocate and free memory for a DNS
resolution structure. Main purpose is to to make the code related to DNS
more consistent.
They allocate or free memory for the structure itself. Later, if needed,
they should also allocate / free the buffers, etc, used by this structure.
They don't set/unset any parameters, this is the role of the caller.

This patch also implement calls to these function eveywhere it is
required.
2017-06-02 11:20:29 +02:00
Baptiste Assmann
9d41fe7f98 CLEANUP: server.c: missing prototype of srv_free_dns_resolution
Prototype for the function srv_free_dns_resolution() missing at the top
of the file.
2017-06-02 11:18:28 +02:00
Frédéric Lécaille
b418c1228c MINOR: server: cli: Add server FQDNs to server-state file and stats socket.
This patch adds a new stats socket command to modify server
FQDNs at run time.
Its syntax:
  set server <backend>/<server> fqdn <FQDN>
This patch also adds FQDNs to server state file at the end
of each line for backward compatibility ("-" if not present).
2017-05-03 06:58:53 +02:00
Frédéric Lécaille
72ed4758d6 MINOR: server: Add server_template_init() function to initialize servers from a templates.
This patch adds server_template_init() function used to initialize servers
from server templates. It is called just after having parsed a 'server-template'
line.
2017-04-21 15:42:10 +02:00
Frédéric Lécaille
b82f742b78 MINOR: server: Add 'server-template' new keyword supported in backend sections.
This patch makes backend sections support 'server-template' new keyword.
Such 'server-template' objects are parsed similarly to a 'server' object
by parse_server() function, but its first arguments are as follows:
    server-template <ID prefix> <nb | range> <ip | fqdn>:<port> ...

The remaining arguments are the same as for 'server' lines.

With such server template declarations, servers may be allocated with IDs
built from <ID prefix> and <nb | range> arguments.

For instance declaring:
    server-template foo 1-5 google.com:80 ...
or
    server-template foo 5 google.com:80 ...

would be equivalent to declare:
    server foo1 google.com:80 ...
    server foo2 google.com:80 ...
    server foo3 google.com:80 ...
    server foo4 google.com:80 ...
    server foo5 google.com:80 ...
2017-04-21 15:42:10 +02:00
Frédéric Lécaille
759ea98db2 MINOR: server: Extract the code which finalizes server initializations after 'server' lines parsing.
This patch moves the code which is responsible of finalizing server initializations
after having fully parsed a 'server' line (health-check, agent check and SNI expression
initializations) from parse_server() to new functions.
2017-04-21 15:42:10 +02:00
Frédéric Lécaille
58b207cdd5 MINOR: server: Extract the code responsible of copying default-server settings.
This patch moves the code responsible of copying default server settings
to a new server instance from parse_server() function to new defsrv_*_cpy()
functions which may be used both during server lines parsing and during server
templates initializations to come.

These defsrv_*_cpy() do not make any reference to anything else than default
server settings.
2017-04-21 15:42:10 +02:00
Frédéric Lécaille
daa2fe6621 BUG/MINOR: server: missing default server 'resolvers' setting duplication.
'resolvers' setting was not duplicated from default server setting to
new server instances when parsing 'server' lines.
This fix is simple: strdup() default resolvers <id> string argument after
having allocated a new server when parsing 'server' lines.

This patch must be backported to 1.7 and 1.6.
2017-04-21 15:42:09 +02:00
Olivier Houchard
7d8e688953 BUG/MINOR: server: don't use "proxy" when px is really meant.
In server_parse_sni_expr(), we use the "proxy" global variable, when we
should probably be using "px" given as an argument.
It happens to work by accident right now, but may not in the future.

[wt: better backport it]
2017-04-20 19:51:10 +02:00
Frédéric Lécaille
dfacd69b94 BUG/MAJOR: Broken parsing for valid keywords provided after 'source' setting.
Any valid keyword could not be parsed anymore if provided after 'source' keyword.
This was due to the fact that 'source' number of arguments is variable.
So, as its parser srv_parse_source() is the only one who may know how many arguments
was provided after 'source' keyword, it updates 'cur_arg' variable (the index
in the line of the current arg to be parsed), this is a good thing.
This variable is also incremented by one (to skip the 'source' keyword).
This patch disable this behavior.

Should have come with dba9707 commit.
2017-04-16 18:13:06 +02:00
Frédéric Lécaille
8d083ed796 BUG/MINOR: server: Fix a wrong error message during 'usesrc' keyword parsing.
'usesrc' setting is not permitted on 'server' lines if not provided after
'source' setting. This is now also the case on 'default-server' lines.
Without this patch parse_server() parser displayed that 'usersrc' is
an unknown keyword.

Should have come with dba9707 commit.
2017-04-15 13:42:55 +02:00
Willy Tarreau
04bf98149b BUG/MEDIUM: servers: unbreak server weight propagation
This reverts commit 266b1a8 ("MEDIUM: server: Inherit CLI weight changes and
agent-check weight responses") from Michal Idzikowski, which is still broken.
It stops propagating weights at the first error encountered, leaving servers
in a random state depending on what LB algorithms are used on other servers
tracking the one experiencing the weight change. It's unsure what the best
way to address this is, but we cannot leave the servers in an inconsistent
state between farms. For example :

  backend site1
      mode http
      balance uri
      hash-type consistent
      server s1 127.0.0.1:8001 weight 10 track servers/s1

  backend site2
      mode http
      balance uri
      server s1 127.0.0.1:8001 weight 10 track servers/s1

  backend site3
      mode http
      balance uri
      hash-type consistent
      server s1 127.0.0.1:8001 weight 10 track servers/s1

  backend servers
      server s1 127.0.0.1:8001 weight 10 check inter 1s

The weight change is applied on "servers/s1". It tries to propagate
to the servers tracking it, which are site1/s1, site2/s1 and site3/s1.
Let's say that "weight 50%" is requested. The servers are linked in
reverse-order, so the change is applied to "servers/s1", then to
"site3/s1", then to "site2/s1" and this one fails and rejects the
change. The change is aborted and never propagated to "site1/s1",
which keeps the server in a different state from "site3/s1". At the
very least, in case of error, the changes should probably be unrolled.

Also the error reported on the CLI (when changing from the CLI) simply says :

  Backend is using a static LB algorithm and only accepts weights '0%' and '100%'.

Without more indications what the faulty backend is.

Let's revert this change for now, as initially feared it will definitely
cause more harm than good and at least needs to be revisited. It was never
backported to any stable branch so no backport is needed.
2017-04-13 15:09:26 +02:00
Michal Idzikowski
266b1a8336 MEDIUM: server: Inherit CLI weight changes and agent-check weight responses
When agent-check or CLI command executes relative weight change this patch
propagates it to tracking server allowing grouping many backends running on
same server underneath. Additionaly in case with many src IPs many backends
can have shared state checker, so there won't be unnecessary health checks.

[wt: Note: this will induce some behaviour change on some setups]
2017-04-13 11:31:38 +02:00
David Carlier
3a471935e6 BUG/MINOR: server : no transparent proxy for DragonflyBSD
IP*_BINDANY is not defined under this system thus it is
necessary to make those fields access since CONFIG_HAP_TRANSPARENT
is not defined.
[wt: problem introduced late in 1.8-dev. The same fix was also reported
  by Steven Davidovitz]
2017-04-10 15:27:46 +02:00
Olivier Houchard
b4a2d5e19a MINOR server: Restrict dynamic cookie check to the same proxy.
Each time we generate a dynamic cookie, we try to make sure the same
cookie hasn't been generated for another server, it's very unlikely, but
it may happen.
We only have to check that for the servers in the same proxy, no, need to
check in others, plus the code was buggy and would always check in the
first proxy of the proxy list.
2017-04-10 15:20:11 +02:00
David Carlier
6f1820864b CLEANUP: server: moving netinet/tcp.h inclusion
netinet/tcp.h needs sys/types.h for u_int* types usage,
issue found while building on OpenBSD.
2017-04-04 08:22:48 +02:00
Frédéric Lécaille
acd4827eca BUG/MEDIUM: server: Wrong server default CRT filenames initialization.
This patch fixes a bug which came with 5e57643 commit where server
default CRT filenames were initialized to the same value as server
default CRL filenames.
2017-03-29 16:37:56 +02:00
Frédéric Lécaille
6e0843c0e0 MINOR: server: Add 'no-agent-check' server keyword.
This patch adds 'no-agent-check' setting supported both by 'default-server'
and 'server' directives to disable an agent check for a specific server which would
have 'agent-check' set as default value (inherited from 'default-server'
'agent-check' setting), or, on 'default-server' lines, to disable 'agent-check' setting
as default value for any further 'server' declarations.

For instance, provided this configuration:

    default-server agent-check
    server srv1
    server srv2 no-agent-check
    server srv3
    default-server no-agent-check
    server srv4

srv1 and srv3 would have an agent check enabled contrary to srv2 and srv4.

We do not allocate anymore anything when parsing 'default-server' 'agent-check'
setting.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
2a0d061a60 MINOR: server: Make 'default-server' support 'disabled' keyword.
Before this patch, only 'server' directives could support 'disabled' setting.
This patch makes also 'default-server' directives support this setting.
It is used to disable a list of servers declared after a 'defaut-server' directive.
'enabled' new keyword has been added, both supported as 'default-server' and
'server' setting, to enable again a list of servers (so, declared after a
'default-server enabled' directive) or to explicitly enable a specific server declared
after a 'default-server disabled' directive.

For instance provided this configuration:

    default-server disabled
    server srv1...
    server srv2...
    server srv3... enabled
    server srv4... enabled

srv1 and srv2 are disabled and srv3 and srv4 enabled.

This is equivalent to this configuration:

    default-server disabled
    server srv1...
    server srv2...
    default-server enabled
    server srv3...
    server srv4...

even if it would have been preferable/shorter to declare:

    server srv3...
    server srv4...
    default-server disabled
    server srv1...
    server srv2...

as 'enabled' is the default server state.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
6e5e0d8f9e MINOR: server: Make 'default-server' support 'addr' keyword.
This patch makes 'default-server' support 'addr' setting.
The code which was responsible of parsing 'server' 'addr' setting
has moved from parse_server() to implement a new parser
callable both as 'default-server' and 'server' 'addr' setting parser.

Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
9a146de934 MINOR: server: Make 'default-server' support 'sni' keyword.
This patch makes 'default-server' directives support 'sni' settings.
A field 'sni_expr' has been added to 'struct server' to temporary
stores SNI expressions as strings during both 'default-server' and 'server'
lines parsing. So, to duplicate SNI expressions from 'default-server' 'sni' setting
for new 'server' instances we only have to "strdup" these strings as this is
often done for most of the 'server' settings.
Then, sample expressions are computed calling sample_parse_expr() (only for 'server'
instances).
A new function has been added to produce the same error output as before in case
of any error during 'sni' settings parsing (display_parser_err()).
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
dba9707713 MINOR: server: Make 'default-server' support 'source' keyword.
Before this patch, only 'server' directives could support 'source' setting.
This patch makes also 'default-server' directives support this setting.

To do so, we had to extract the code responsible of parsing 'source' setting
arguments from parse_server() function and make it callable both
as 'default-server' and 'server' 'source' setting parser. So, the code is mostly
the same as before except that before allocating anything for 'struct conn_src'
members, we must free the memory previously allocated.

Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
22f41a2d23 MINOR: server: Make 'default-server' support 'namespace' keyword.
Before this patch, 'namespace' setting was only supported by 'server' directive.
This patch makes 'default-server' directive support this setting.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
5c3cd97550 MINOR: server: Make 'default-server' support 'tcp-ut' keyword.
This patch makes 'default-server' directive support 'tcp-ut' keyword.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
bcaf1d7397 MINOR: server: Make 'default-server' support 'ciphers' keyword.
This patch makes 'default-server' directive support 'ciphers' setting.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
9d1b95b591 MINOR: server: Make 'default-server' support 'cookie' keyword.
Before this patch, 'cookie' setting was only supported by 'server' directives.
This patch makes 'default-server' directive also support 'cookie' setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
547356e484 MINOR: server: Make 'default-server' support 'observe' keyword.
Before this path, 'observe' setting was only supported by 'server' directives.
This patch makes 'default-server' directives also support 'observe' setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
16186236dd MINOR: server: Make 'default-server' support 'redir' keyword.
Before this patch only 'server' directives could support 'redir' setting.
This patch makes also 'default-server' directives support 'redir' setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
5e57643e09 MINOR: server: Make 'default-server' support 'ca-file', 'crl-file' and 'crt' settings.
This patch makes 'default-server' directives support 'ca-file', 'crl-file' and
'crt' settings.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
67e0e61316 MINOR: server: Make 'default-server' support 'track' setting.
Before this patch only 'server' directives could support 'track' setting.
This patch makes 'default-server' directives also support this setting.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
65aa356c0b MINOR: server: Make 'default-server' support 'check' keyword.
Before this patch 'check' setting was only supported by 'server' directives.
This patch makes also 'default-server' directives support this setting.
A new 'no-check' keyword parser has been implemented to disable this setting both
in 'default-server' and 'server' directives.
Should not break anything.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
273f321404 MINOR: server: Make 'default-server' support 'verifyhost' setting.
This patch makes 'default-server' directive support 'verifyhost' setting.
Note: there was a little memory leak when several 'verifyhost' arguments were
supplied on the same 'server' line.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
7c8cd587c2 MINOR: server: Make 'default-server' support 'verify' keyword.
This patch makes 'default-server' directive support 'verify' keyword.
2017-03-27 14:37:01 +02:00
Frédéric Lécaille
31045e4c10 MINOR: server: Make 'default-server' support 'send-proxy' and 'send-proxy-v2 keywords.
This patch makes 'default-server' directive support 'send-proxy'
(resp. 'send-proxy-v2') setting.
A new keyword 'no-send-proxy' (resp. 'no-send-proxy-v2') has been added
to disable 'send-proxy' (resp. 'send-proxy-v2') setting both in 'server' and
'default-server' directives.
2017-03-27 14:36:12 +02:00
Frédéric Lécaille
f9bc1d6a13 MINOR: server: Make 'default-server' support 'non-stick' keyword.
This patch makes 'default-server' directive support 'non-stick' setting.
A new keyword 'stick' has been added so that to disable
'non-stick' setting both in 'server' and 'default-server' directives.
2017-03-27 14:36:11 +02:00
Frédéric Lécaille
1502cfd1a3 CLEANUP: server: code alignement.
Code alignement.
2017-03-27 14:36:11 +02:00
Frédéric Lécaille
25df89066b MINOR: server: Make 'default-server' support 'check-send-proxy' keyword.
This patch makes 'default-server' directive support 'check-send-proxy' setting.
A new keyword 'no-check-send-proxy' has been added so that to disable
'check-send-proxy' setting both in 'server' and 'default-server' directives.
2017-03-27 14:36:11 +02:00
Frédéric Lécaille
f5bf903be6 MINOR: server: Make 'default-server' support 'backup' keyword.
At this time, only 'server' supported 'backup' keyword.
This patch makes also 'default-server' directive support this keyword.
A new keyword 'no-backup' has been added so that to disable 'backup' setting
both in 'server' and 'default-server' directives.

For instance, provided the following sequence of directives:

default-server backup
server srv1
server srv2 no-backup

default-server no-backup
server srv3
server srv4 backup

srv1 and srv4 are declared as backup servers,
srv2 and srv3 are declared as non-backup servers.
2017-03-27 14:36:11 +02:00
Frédéric Lécaille
8065b6d4f2 MINOR: server: irrelevant error message with 'default-server' config file keyword.
There is no reason to emit such an error message:
"'default-server' expects <name> and <addr>[:<port>] as arguments."
if less than two arguments are provided on 'default-server' lines.
This is a 'server' specific error message.
2017-03-27 14:33:58 +02:00
Olivier Houchard
2cb49ebbc4 BUG/MEDIUM server: Fix crash when dynamic is defined, but not key is provided.
Wait until we're sure we have a key before trying to calculate its length.

[wt: no backport needed, was just merged]
2017-03-15 16:01:33 +01:00
Olivier Houchard
4e694049fa MINOR: server: Add dynamic session cookies.
This adds a new "dynamic" keyword for the cookie option. If set, a cookie
will be generated for each server (assuming one isn't already provided on
the "server" line), from the IP of the server, the TCP port, and a secret
key provided. To provide the secret key, a new keyword as been added,
"dynamic-cookie-key", for backends.

Example :
backend bk_web
  balance roundrobin
  dynamic-cookie-key "bla"
  cookie WEBSRV insert dynamic
  server s1 127.0.0.1:80 check
  server s2 192.168.56.1:80 check

This is a first step to be able to dynamically add and remove servers,
without modifying the configuration file, and still have all the load
balancers redirect the traffic to the right server.

Provide a way to generate session cookies, based on the IP address of the
server, the TCP port, and a secret key provided.
2017-03-15 11:37:30 +01:00
Misiek
2da082d732 MINOR: cli: Add possiblity to change agent config via CLI/socket
This change adds possibility to change agent-addr and agent-send directives
by CLI/socket. Now you can replace server's and their configuration without
reloading/restarting whole haproxy, so it's a step in no-reload/no-restart
direction.

Depends on #e9602af - agent-addr is implemented there.

Can be backported to 1.7.
2017-01-16 11:38:59 +01:00
Misiek
ea849333ca MINOR: checks: Add agent-addr config directive
This directive add possibility to set different address for agent-checks.
With this you can manage server status and weight from central place.

Can be backported to 1.7.
2017-01-16 11:38:02 +01:00
Ryabin Sergey
77ee7526de BUG/MINOR: Reset errno variable before calling strtol(3)
Sometimes errno != 0 before calling strtol(3)

[wt: this needs to be backported to 1.7]
2017-01-11 21:30:07 +01:00
Willy Tarreau
9698f4b295 MEDIUM: server: disable protocol validations when the server doesn't resolve
When a server doesn't resolve we don't know the address family so we
can't perform the basic protocol validations. However we know that we'll
ultimately resolve to AF_INET4 or AF_INET6 so the controls are OK. It is
important to proceed like this otherwise it will not be possible to start
with unresolved addresses.
2017-01-06 19:29:34 +01:00
Willy Tarreau
6ecb10aec7 MINOR: server: take the destination port from the port field, not the addr
Next patch will cause the port to disappear from the address field when servers
do not resolve so we need to take it from the separate field provided by
str2sa_range().
2017-01-06 19:29:34 +01:00
Willy Tarreau
48ef4c95b6 MINOR: tools: make str2sa_range() return the port in a separate argument
This will be needed so that we're don't have to extract it from the
returned address where it will not always be anymore (eg: for unresolved
servers).
2017-01-06 19:29:34 +01:00
Willy Tarreau
04276f3d6e MEDIUM: server: split the address and the port into two different fields
Keeping the address and the port in the same field causes a lot of problems,
specifically on the DNS part where we're forced to cheat on the family to be
able to keep the port. This causes some issues such as some families not being
resolvable anymore.

This patch first moves the service port to a new field "svc_port" so that the
port field is never used anymore in the "addr" field (struct sockaddr_storage).
All call places were adapted (there aren't that many).
2017-01-06 19:29:33 +01:00
Willy Tarreau
3acfcd1aa1 BUG/MEDIUM: server: consider AF_UNSPEC as a valid address family
The DNS code is written so as to support AF_UNSPEC to decide on the
server family based on responses, but unfortunately snr_resolution_cb()
considers it as invalid causing a DNS storm to happen when a server
arrives with this family.

This situation is not supposed to happen as long as unresolved addresses
are forced to AF_INET, but this will change with the upcoming fixes and
it's possible that it's not granted already when changing an address on
the CLI.

This fix must be backported to 1.7 and 1.6.
2017-01-06 19:21:37 +01:00
Willy Tarreau
a261e9b094 CLEANUP: connection: remove all direct references to raw_sock and ssl_sock
Now we exclusively use xprt_get(XPRT_RAW) instead of &raw_sock or
xprt_get(XPRT_SSL) for &ssl_sock. This removes a bunch of #ifdef and
include spread over a number of location including backend, cfgparse,
checks, cli, hlua, log, server and session.
2016-12-22 23:26:38 +01:00
Willy Tarreau
141ad85d10 MINOR: server: move the use_ssl field out of the ifdef USE_OPENSSL
Having it in the ifdef complicates certain operations which require
additional ifdefs just to access a member which could remain zero in
non-ssl cases. Let's move it out, it will not even increase the
struct size on 64-bit machines due to alignment.
2016-12-22 23:26:38 +01:00
Christopher Faulet
90b5abe46e BUG/MINOR: cli: be sure to always warn the cli applet when input buffer is full
[wt: may only strike if CLI commands are pipelined. Must be backported to 1.7
 and 1.6, where it's a bit different and in dumpstats.c]
2016-12-12 17:58:11 +01:00
Willy Tarreau
3b6e547be8 CLEANUP: cli: rename STAT_CLI_* to CLI_ST_*
These are in CLI states, not stats states anymore. STAT_CLI_O_CUSTOM
was more appropriately renamed CLI_ST_CALLBACK.
2016-11-24 16:59:28 +01:00
Willy Tarreau
58d9cb7d22 REORG: cli: move "{enable|disable} agent" to server.c
Also mention that "set server" is preferred now. Note that these
were the last enable/disable commands in cli.c. Also remove the
now unused expect_server_admin() function.
2016-11-24 16:59:28 +01:00
Willy Tarreau
2c04eda8b5 REORG: cli: move "{enable|disable} health" to server.c
Also mention that "set server" is preferred now.
2016-11-24 16:59:28 +01:00
Willy Tarreau
ffb4d58e1b REORG: cli: move "{enable|disable} server" to server.c
Also mention that "set server" is preferred now.
2016-11-24 16:59:28 +01:00
Willy Tarreau
b802627eb3 REORG: cli: move "set maxconn server" to server.c
It's used to manipulate the server's maxconn setting.
2016-11-24 16:59:28 +01:00
William Lallemand
6b16094355 REORG: cli: move get/set weight to server.c
Move get/set weight CLI functions to server.c and use the cli keyword API
to register it on the CLI.
2016-11-24 16:59:27 +01:00
William Lallemand
222baf20da REORG: cli: move 'set server' to server.c
Move 'set server' CLI functions to server.c and use the cli keyword API
to register it on the CLI.
2016-11-24 16:59:27 +01:00
Willy Tarreau
21b069dca8 MINOR: server: create new function cli_find_server() to find a server
Several CLI commands require a server, so let's have a function to
look this one up and prepare the appropriate error message and the
appctx's state in case of failure.
2016-11-24 16:59:27 +01:00
David Carlier
327298c215 BUILD: fix build on Solaris 10/11
uint16_t instead of u_int16_t
None ISO fields of struct tm are not present, but
by zeroyfing it, on GNU and BSD systems tm_gmtoff
field will be set.

[wt: moved the memset into each of the date functions]
2016-11-22 12:04:19 +01:00
Willy Tarreau
e5a60688a4 MEDIUM: server: do not restrict anymore usage of IP address from the state file
Now that it is possible to decide whether we prefer to use libc or the
state file to resolve the server's IP address and it is possible to change
a server's IP address at run time on the CLI, let's not restrict the reuse
of the address from the state file anymore to the DNS only.

The impact is that by default the state file will be considered first
(which matches its purpose) and only then the libc. This way any address
change performed at run time over the CLI will be preserved regardless
of DNS usage or not.
2016-11-09 15:33:52 +01:00
Willy Tarreau
3eed10e54b MINOR: init: add -dr to ignore server address resolution failures
It is very common when validating a configuration out of production not to
have access to the same resolvers and to fail on server address resolution,
making it difficult to test a configuration. This option simply appends the
"none" method to the list of address resolution methods for all servers,
ensuring that even if the libc fails to resolve an address, the startup
sequence is not interrupted.
2016-11-09 15:33:52 +01:00
Willy Tarreau
4310d36a7e MINOR: server: add support for explicit numeric address in init-addr
This will allow a server to automatically fall back to an explicit numeric
IP address when all other methods fail. The address is simply specified in
the address list.
2016-11-09 15:30:47 +01:00
Willy Tarreau
465b6e5463 MEDIUM: server: make libc resolution failure non-fatal
Now that we have "init-addr none", it becomes possible to recover on
libc resolver's failures. Thus it's preferable not to alert nor fail
at the moment the libc is called, and instead process the failure at
the end of the list. This allows "none" to be set after libc to
provide a smooth fallback in case of resolver issues.
2016-11-09 15:30:47 +01:00
Willy Tarreau
37ebe1212b MINOR: server: implement init-addr none
The server is put into the "no address" maintenance state in this case.
2016-11-09 15:30:47 +01:00
Willy Tarreau
25e515235a MEDIUM: server: make use of init-addr
It is now supported. If not set, we default to the legacy methods list
which is "last,libc".
2016-11-09 15:30:47 +01:00
Baptiste Assmann
25938278b7 MEDIUM: server: add a new init-addr server line setting
This new setting supports a comma-delimited list of methods used to
resolve the server's FQDN to an IP address. Currently supported methods
are "libc" (use the regular libc's resolver) and "last" (use the last
known valid address found in the state file).

The list is implemented in a 32-bit integer, because each init-addr
method only requires 3 bits. The last one must always be SRV_IADDR_END
(0), allowing to store up to 10 methods in a single 32 bit integer.

Note: the doc is provided at the end of this series.
2016-11-09 15:30:47 +01:00
Baptiste Assmann
3b9fe9f8f4 MAJOR: dns: runtime resolution can change server admin state
WARNING: this is a MAJOR (and disruptive) change with previous HAProxy's
behavior: before, HAProxy never ever used to change a server administrative
status when the DNS resolution failed at run time.

This patch gives HAProxy the ability to change the administrative status
of a server to MAINT (RMAINT actually) when an error is encountered for
a period longer than its own allowed by the corresponding 'hold'
parameter.

IE if the configuration sets "hold nx 10s" and a server's hostname
points to a NX for more than 10s, then the server will be set to RMAINT,
hence in MAINTENANCE mode.
2016-11-09 15:30:47 +01:00
Willy Tarreau
8b42848a44 MINOR: server: make srv_set_admin_state() capable of telling why this happens
It will be important to help debugging some DNS resolution issues to
know why a server was marked down, so let's make  the function support
a 3rd argument with an indication of the reason. Passing NULL will keep
the message as-is.
2016-11-09 15:30:47 +01:00
Willy Tarreau
e659973bfe MINOR: server: indicate in the logs when RMAINT is cleared
It's important to report in the server state change logs that RMAINT was
cleared, as it's not the regular maintenance mode, it's specific to name
resolution, and it's important to report the new state (which can be DRAIN
or READY).
2016-11-09 15:23:37 +01:00
Baptiste Assmann
83cbaa531f MAJOR: server: postpone address resolution
Server addresses are not resolved anymore upon the first pass so that we
don't fail if an address cannot be resolved by the libc. Instead they are
processed all at once after the configuration is fully loaded, by the new
function srv_init_addr(). This function only acts on the server's address
if this address uses an FQDN, which appears in server->hostname.

For now the function does two things, to followup with HAProxy's historical
default behavior:

  1. apply server IP address found in server-state file if runtime DNS
     resolution is enabled for this server

  2. use the DNS resolver provided by the libc

If none of the 2 options above can find an IP address, then an error is
returned.

All of this will be needed to support the new server parameter "init-addr".
For now, the biggest user-visible change is that all server resolution errors
are dumped at once instead of causing a startup failure one by one.
2016-11-09 14:24:20 +01:00
Willy Tarreau
757478e900 BUG/MEDIUM: servers: properly propagate the maintenance states during startup
Right now there is an issue with the way the maintenance flags are
propagated upon startup. They are not propagate, just copied from the
tracked server. This implies that depending on the server's order, some
tracking servers may not be marked down. For example this configuration
does not work as expected :

        server s1 1.1.1.1:8000 track s2
        server s2 1.1.1.1:8000 track s3
        server s3 1.1.1.1:8000 track s4
        server s4 wtap:8000 check inter 1s disabled

It results in s1/s2 being up, and s3/s4 being down, while all of them
should be down.

The only clean way to process this is to run through all "root" servers
(those not tracking any other server), and to propagate their state down
to all their trackers. This is the same algorithm used to propagate the
state changes. It has to be done both to compute the IDRAIN flag and the
IMAINT flag. However, doing so requires that tracking servers are not
marked as inherited maintenance anymore while parsing the configuration
(and given that it is wrong, better drop it).

This fix also addresses another side effect of the bug above which is
that the IDRAIN/IMAINT flags are stored in the state files, and if
restored while the tracked server doesn't have the equivalent flag,
the servers may end up in a situation where it's impossible to remove
these flags. For example in the configuration above, after removing
"disabled" on server s4, the other servers would have remained down,
and not anymore with this fix. Similarly, the combination of IMAINT
or IDRAIN with their respective forced modes was not accepted on
reload, which is wrong as well.

This bug has been present at least since 1.5, maybe even 1.4 (it came
with tracking support). The fix needs to be backported there, though
the srv-state parts are irrelevant.

This commit relies on previous patch to silence warnings on startup.
2016-11-07 14:31:52 +01:00
Willy Tarreau
6fb8dc1a5a MINOR: server: do not emit warnings/logs/alerts on server state changes at boot
We'll have to use srv_set_admin_flag() to propagate some server flags
during the startup, and we don't want the resulting actions to cause
warnings, logs nor e-mail alerts to be generated since we're just applying
the config or a state file. So let's condition these notifications to the
fact that we're starting.
2016-11-07 14:31:45 +01:00
Willy Tarreau
e1bde1492a BUG/MINOR: srv-state: allow to have both CMAINT and FDRAIN flags
CMAINT indicates that the server was *initially* disabled in the
configuration via the "disabled" keyword. FDRAIN indicates that the
server was switched to the DRAIN state from the CLI or the agent.
This it's perfectly valid to have both of them in the state file,
so the parser must not reject this combination.

This fix must be backported to 1.6.
2016-11-07 14:30:19 +01:00
Willy Tarreau
22cace2f4c BUG/MEDIUM: srv-state: properly restore the DRAIN state
There were seveal reports about the DRAIN state not being properly
restored upon reload.

It happens that the condition in the code does exactly the opposite
of what the comment says, and the comment is right so the code is
wrong.

It's worth noting that the conditions are complex here due to the 2
available methods to set the drain state (CLI/agent, and config's
weight). To paraphrase the updated comment in the code, there are
two possible reasons for FDRAIN to have been present :
  - previous config weight was zero
  - "set server b/s drain" was sent to the CLI

In the first case, we simply want to drop this drain state if the new
weight is not zero anymore, meaning the administrator has intentionally
turned the weight back to a positive value to enable the server again
after an operation. In the second case, the drain state was forced on
the CLI regardless of the config's weight so we don't want a change to
the config weight to lose this status. What this means is :
  - if previous weight was 0 and new one is >0, drop the DRAIN state.
  - if the previous weight was >0, keep it.

This fix must be backported to 1.6.
2016-11-07 14:30:19 +01:00
Willy Tarreau
dc3a9e830c CLEANUP: tools: make ipcpy() preserve the original port
ipcpy() is used to replace an IP address with another one, but it
doesn't preserve the original port so all callers have to do it
manually while it's trivial to do there. Better do it inside the
function.
2016-11-05 13:56:04 +01:00
Baptiste Assmann
65ce3f5ee4 MINOR: dns: query type change when last record is a CNAME
DNS servers don't return A or AAAA record if the query points to a CNAME
not resolving to the right type.
We know it because the last record of the response is a CNAME. We can
trigger a new query, switching to a new query type, handled by the layer
above.
2016-09-12 20:01:40 +02:00
Baptiste Assmann
c1ce5f358e MEDIUM: dns: new DNS response parser
New DNS response parser function which turn the DNS response from a
network buffer into a DNS structure, much easier for later analysis
by upper layer.

Memory is pre-allocated at start-up in a chunk dedicated to DNS
response store.

New error code to report a wrong number of queries in a DNS response.
2016-09-12 19:54:23 +02:00
Baptiste Assmann
d458adcc52 MINOR: new update_server_addr_port() function to change both server's ADDR and service PORT
This function can replace update_server_addr() where the need to change the
server's port as well as the IP address is required.
It performs some validation before performing each type of change.
2016-09-11 08:13:11 +02:00
Baptiste Assmann
6b453f166f MINOR: server: introduction of 3 new server flags
Introduction of 3 new server flags to remember if some parameters were set
during configuration parsing.

* SRV_F_CHECKADDR: this server has a check addr configured
* SRV_F_CHECKPORT: this server has a check port configured
* SRV_F_AGENTADDR: this server has a agent addr configured
2016-09-11 08:12:42 +02:00
Baptiste Assmann
95db2bcfee MAJOR: check: find out which port to use for health check at run time
HAProxy used to deduce port used for health checks when parsing configuration
at startup time.
Because of this way of working, it makes it complicated to change the port at
run time.

The current patch changes this behavior and makes HAProxy to choose the
port used for health checking when preparing the check task itself.

A new type of error is introduced and reported when no port can be found.

There won't be any impact on performance, since the process to find out the
port value is made of a few 'if' statements.

This patch also introduces a new check state CHK_ST_PORT_MISS: this flag is
used to report an error in the case when HAProxy needs to establish a TCP
connection to a server, to perform a health check but no TCP ports can be
found for it.

And last, it also introduces a new stream termination condition:
SF_ERR_CHK_PORT. Purpose of this flag is to report an error in the event when
HAProxy has to run a health check but no port can be found to perform it.
2016-09-11 08:12:13 +02:00
Baptiste Assmann
d260e1dea6 MAJOR: listen section: don't use first bind port anymore when no server ports are provided
Up to HAProxy 1.7-dev3, HAProxy used to use the first bind port from it's
local 'listen' section when no port is configured on the server.

IE, in the configuration below, the server port would be 25:

  listen smtp
   bind :25
   server s1 1.0.0.1 check

This way of working is now obsolete and can be removed, furthermore it is not
documented!

This will make the possibility to change the server's port much easier.
2016-08-14 12:18:14 +02:00