17603 Commits

Author SHA1 Message Date
Thierry FOURNIER
c365d99a07 MINOR: vars: adds get and set functions
This patch adds two functions used for variable acces using the
variable full name. If the variable doesn't exists in the variable
pool name, it is created.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
4834bc773c MEDIUM: vars: adds support of variables
This patch adds support of variables during the processing of each stream. The
variables scope can be set as 'session', 'transaction', 'request' or 'response'.
The variable type is the type returned by the assignment expression. The type
can change while the processing.

The allocated memory can be controlled for each scope and each request, and for
the global process.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
0e11863a6f MINOR: tcp/http/conf: extends the keyword registration options
This patch permits to register a new keyword with the keyword "tcp-request content"
'tcp-request connection", tcp-response content", http-request" and "http-response"
which is identified only by matching the start of the keyword.

for example, we register the keyword "set-var" with the option "match_pfx"
and the configuration keyword "set-var(var_name)" matchs this entry.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
fbdb77582d MINOR: tcp: increase the opaque data array
This patch increase the opaque data array for the tcp_rules.
It is used by the "store" action (next commited) which deal
with variables.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
9687c77c91 MINOR: debug: add a special converter which display its input sample content.
This converter displays its input sample type and content. It is useful
for debugging some complex configurations.
2015-06-13 23:01:36 +02:00
Thierry FOURNIER
9c627e84b2 MEDIUM: sample: Add type any
This type is used to accept any type of sample as input, and prevent
any automatic "cast". It runs like the type "ADDR" which accept the
type "IPV4" and "IPV6".
2015-06-13 22:59:14 +02:00
Thierry FOURNIER
0f811440d5 BUG/MINOR: sample: wrong conversion of signed values
The signed values are casted as unsigned before conversion. This patch
use the good converters according with the sample type.

Note: it depends on previous patch to parse signed ints.
2015-06-13 22:59:14 +02:00
Thierry FOURNIER
1480bd8dd2 MINOR: standard: add function that converts signed int to a string
This function is the same as "ultoa_r", but it takes a signed value
as input.
2015-06-13 22:59:14 +02:00
Baptiste Assmann
3863f97349 MINOR: dns: add DNS statistics
add a new command on the stats socket to print a DNS resolvers section
(including per server) statistics: "show stats resolvers <id>"
2015-06-13 22:07:35 +02:00
Baptiste Assmann
a68ca96375 MAJOR: server: add DNS-based server name resolution
Relies on the DNS protocol freshly implemented in HAProxy.
It performs a server IP addr resolution based on a server hostname.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
325137d603 MEDIUM: dns: implement a DNS resolver
Implementation of a DNS client in HAProxy to perform name resolution to
IP addresses.

It relies on the freshly created UDP client to perform the DNS
resolution. For now, all UDP socket calls are performed in the
DNS layer, but this might change later when the protocols are
extended to be more suited to datagram mode.

A new section called 'resolvers' is introduced thanks to this patch. It
is used to describe DNS servers IP address and also many parameters.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
5d4e4f7a57 MEDIUM: protocol: add minimalist UDP protocol client
Basic introduction of a UDP layer in HAProxy. It can be used as a
client only and manages UDP exchanges with servers.

It can't be used to load-balance UDP protocols, but only used by
internal features such as DNS resolution.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
3d8f831f13 MEDIUM: server: change server ip address from stats socket
New command available on the stats socket to change a server addr using
the command "set server <backend>/<server> addr <ip4|ip6>"
2015-06-13 22:07:35 +02:00
Baptiste Assmann
14e4014a48 MEDIUM: server: add support for changing a server's address
Ability to change a server IP address during HAProxy run time.
For now this is provided via function update_server_addr() which
currently is not called.

A log is emitted on each change. For now we do it inconditionally,
but later we'll want to do it only on certain circumstances, which
explains why the logging block is enclosed in if(1).
2015-06-13 22:07:35 +02:00
Christopher Faulet
3054880e09 MINOR: ssl: Export functions to manipulate generated certificates
Following functions are now available in the SSL public API:

  * ssl_sock_create_cert
  * ssl_sock_get_generated_cert
  * ssl_sock_set_generated_cert
  * ssl_sock_generated_cert_serial

These functions could be used to create a certificate by hand, set it in the
cache used to store generated certificates and retrieve it. Here is an example
(pseudo code):

  X509        *cacert     = ...;
  EVP_PKEY    *capkey     = ...;
  char        *servername = ...;
  unsigned int serial;

  serial = ssl_sock_generated_cert_serial(servername, strlen(servername));
  if (!ssl_sock_get_generated_cert(serial, cacert)) {
      SSL_CTX *ctx = ssl_sock_create_cert(servername, serial, cacert, capkey);
      ssl_sock_set_generated_cert(ctx, serial, cacert);
  }
2015-06-12 18:06:59 +02:00
Christopher Faulet
31af49d62b MEDIUM: ssl: Add options to forge SSL certificates
With this patch, it is possible to configure HAProxy to forge the SSL
certificate sent to a client using the SNI servername. We do it in the SNI
callback.

To enable this feature, you must pass following BIND options:

 * ca-sign-file <FILE> : This is the PEM file containing the CA certitifacte and
   the CA private key to create and sign server's certificates.

 * (optionally) ca-sign-pass <PASS>: This is the CA private key passphrase, if
   any.

 * generate-certificates: Enable the dynamic generation of certificates for a
   listener.

Because generating certificates is expensive, there is a LRU cache to store
them. Its size can be customized by setting the global parameter
'tune.ssl.ssl-ctx-cache-size'.
2015-06-12 18:06:59 +02:00
Christopher Faulet
92939d20fa MINOR: lru: Add lru64_lookup function
It lookup a key in a LRU cache for use with specified domain and revision. It
differs from lru64_get as it does not create missing keys. The function returns
NULL if an error or a cache miss occurs.
2015-06-12 18:06:59 +02:00
Christopher Faulet
f90ac55d9e MINOR: lru: Add the possibility to free data when an item is removed
Now, When a item is committed in an LRU tree, you can define a function to free
data owned by this item. This function will be called when the item is removed
from the LRU tree or when the tree is destroyed..
2015-06-12 18:06:59 +02:00
Dragan Dosen
acd6b1f767 BUG/MEDIUM: init: segfault if global._51d_property_names is not initialized
When using the "51d" converter without specifying the list of 51Degrees
properties to detect (see parameter "51degrees-property-name-list"), the
"global._51d_property_names" could be left uninitialized which will lead to
segfault during init.
2015-06-11 21:47:03 +02:00
Godbach
28b48ccbc8 CLEANUP: deinit: remove codes for cleaning p->block_rules
Since all rules listed in p->block_rules have been moved to the beginning of
the http-request rules in check_config_validity(), there is no need to clean
p->block_rules in deinit().

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2015-06-09 15:35:29 +02:00
Thierry FOURNIER
4c2479e1c4 BUG/MINOR: debug: display (null) in place of "meth"
The array which contains names of types, miss the METH entry.

[wt: should be backported to 1.5 as well]
2015-06-09 10:58:14 +02:00
Willy Tarreau
a9f4702390 BUILD: fix build warning when not using 51degrees
An ifdef was missing to avoid declaring these variables :

src/haproxy.c: In function 'deinit':
src/haproxy.c:1253:47: warning: unused variable '_51d_prop_nameb' [-Wunused-variable]
src/haproxy.c:1253:30: warning: unused variable '_51d_prop_name' [-Wunused-variable]
2015-06-02 19:30:59 +02:00
Thomas Holmes
4d441a759c MEDIUM: sample: add trie support to 51Degrees
Trie or pattern algorithm is used depending on what 51Degrees source
files are provided to MAKE.
2015-06-02 19:30:53 +02:00
Thomas Holmes
5ed33eed53 MEDIUM: deinit: add cleanup for 51Degrees to deinit 2015-06-02 14:00:25 +02:00
Thomas Holmes
951d44d24d MEDIUM: sample: add fiftyone_degrees converter.
It takes up to 5 string arguments that are to be 51Degrees property names.
It will then create a chunk with values detected based on the request header
supplied (this should be the User-Agent).
2015-06-02 14:00:25 +02:00
Thomas Holmes
6abded4f8d MINOR: init: add 51Degrees initialisation code
This creates a dataset using the file given in global._51d_data_file_path.
2015-06-02 14:00:25 +02:00
Thomas Holmes
0809f26869 MINOR: config: add 51Degrees config parsing. 2015-06-02 14:00:25 +02:00
Willy Tarreau
25eda1e182 CLEANUP: da: remove ifdef USE_DEVICEATLAS from da.c
This file is only built when the macro is defined, there's no point
having it ifdefed here.
2015-06-02 13:42:12 +02:00
Willy Tarreau
0d74f77ee7 CLEANUP: da: register the config keywords in da.c
No need to export the registration to haproxy.c, it can be done
locally.
2015-06-02 13:42:12 +02:00
Willy Tarreau
f63386ad27 CLEANUP: da: move the converter registration to da.c
There's no reason to put it into sample.c, it's better to register it
locally in da.c, it removes a number of ifdefs and exports.
2015-06-02 13:42:12 +02:00
Willy Tarreau
6bd42e7f41 BUG/MINOR: da: fix log-level comparison to emove annoying warning
When haproxy is run on the foreground with DeviceAtlas enabled, one
line of warning is seen for every test because the comparison is always
true even when loglevel is zero :

willy@wtap:haproxy$ ./haproxy  -db -f test-da.cfg
[WARNING] 151/150831 (25506) : deviceatlas : final memory image 7148029 bytes.
Deviceatlas module loaded.
[WARNING] 151/150832 (25506) : deviceatlas : .
[WARNING] 151/150833 (25506) : deviceatlas : .
[WARNING] 151/150833 (25506) : deviceatlas : .
^C

Don't emit a warning when loglevel is null.
2015-06-02 13:42:12 +02:00
David Carlier
4a02641a86 MEDIUM: init: DeviceAtlas initialization
This diff initialises few DeviceAtlas struct fields member with
their inherent default values.
Furthermore, the specific DeviceAtlas configuration keywords are
registered and the module is initialised and all necessary
resources are freed during the deinit phase.
2015-06-02 13:24:50 +02:00
David Carlier
4542b10ae1 MEDIUM: sample: add the da-csv converter
This diff declares the deviceatlas module and can accept up to 5
property names for the API lookup.

[wt: this should probably be moved to its own file using the keyword
      registration mechanism]
2015-06-02 13:24:50 +02:00
David Carlier
8167f30661 MEDIUM: config: add DeviceAtlas global keywords
This diff is for the DeviceAtlas convertor.

This patch adds the following converters :
  deviceatlas-json-file
  deviceatlas-log-level
  deviceatlas-property-separator

First, the configuration keywords handling (only the log
level configuration part does not end the haproxy process
if it is wrongly set, it fallbacks to the default level).
Furthermore, init, deinit phases and the API lookup phase,
the da_haproxy function which is fed by the input provided
and set all necessary properties chosen via the configuration
to the output, separated by the separator.
2015-06-02 13:24:44 +02:00
Remi Gacogne
d3a341a96f MEDIUM: ssl: replace standards DH groups with custom ones
It is likely that powerful adversaries have been pre-computing the
standardized DH groups, because being widely used have made them
valuable targets. While users are advised to generate their own
DH parameters, replace the ones we ship by values been randomly
generated for this product only.

[wt: replaced dh1024_p, dh2048_p, and dh4096_p with locally-generated
 ones as recommended by Rémi]
2015-05-31 22:41:28 +02:00
Remi Gacogne
47783ef05b MEDIUM: ssl: add the possibility to use a global DH parameters file
This patch adds the ssl-dh-param-file global setting. It sets the
default DH parameters that will be used during the SSL/TLS handshake when
ephemeral Diffie-Hellman (DHE) key exchange is used, for all "bind" lines
which do not explicitely define theirs.
2015-05-31 22:02:00 +02:00
Thierry FOURNIER
79318d79ba MINOR: lua: Lua tcp action are not final action
Actually, the registered lua actions with "tcp-request lua" and
"tcp-response lua" are final actions. This patch change the action
type type and permit to continue the evaluation of tcp-* processing
after the evaluation of the lua actions.
2015-05-29 17:49:50 +02:00
Thierry FOURNIER
561a0f989d MINOR: tcp: add custom actions that can continue tcp-(request|response) processing
Actually, the tcp-request and tcp-response custom ation are always final
actions. This patch create a new type of action that can permit to
continue the evaluation of tcp-request and tcp-response processing.
2015-05-29 17:49:48 +02:00
Emeric Brun
b3971ab062 MAJOR: peers: peers protocol version 2.0
This patch does'nt add any new feature: the functional behavior
is the same than version 1.0.

Technical differences:

In this version all updates on different stick tables are
multiplexed on the same tcp session. There is only one established
tcp session per peer whereas in first version there was one established
tcp session per peer and per stick table.

Messages format was reviewed to be more evolutive and to support
further types of data exchange such as SSL sessions or other sticktable's
data types (currently only the sticktable's server id is supported).
2015-05-29 15:50:33 +02:00
Willy Tarreau
b8cdf52da0 BUG/MEDIUM: http: fix body processing for the stats applet
Commit 9fbe18e ("MEDIUM: http: add a new option http-buffer-request")
introduced a regression due to a misplaced check causing the admin
mode of the HTTP stats not to work anymore.

This patch tried to ensure that when we need a request body for the
stats applet, and we have already waited for this body, we don't wait
for it again, but the condition was applied too early causing a
disabling of the entire processing the body, and based on the wrong
HTTP state (MSG_BODY) resulting in the test never matching.

Thanks to Chad Lavoie for reporting the problem.

This bug is 1.6-only, no backport is needed.
2015-05-29 01:12:38 +02:00
William Lallemand
df1425ad45 MEDIUM: cfgparse: check max arguments in the proxies sections
Add checks on the maximum number of arguments in proxies sections.
2015-05-28 18:43:03 +02:00
William Lallemand
1a748aed07 MEDIUM: cfgparse: max arguments check in the global section
Most of the keywords in the global section does not check the maximum
number of arguments. This leds sometines to unused and wrong arguments
in the configuration file. This patch add a maximum argument test in
many keywords of this section.
2015-05-28 18:43:03 +02:00
William Lallemand
6e62fb6405 MEDIUM: cfgparse: check section maximum number of arguments
This patch checks the number of arguments of the keywords:
'global', 'defaults', 'listen', 'backend', 'frontend', 'peers' and
'userlist'

The 'global' section does not take any arguments.

Proxy sections does not support bind address as argument anymore.  Those
sections supports only an <id> argument.

The 'defaults' section didn't had any check on its arguments. It takes
an optional <name> argument.

'peers' section takes a <peersect> argument.

'userlist' section takes a <listname> argument.
2015-05-28 18:43:03 +02:00
William Lallemand
5109719e77 CLEANUP: cfgparse: remove reference to 'ruleset' section
The 'ruleset' section was never implemented. This patch remove
references and tests about this keyword.
2015-05-28 18:43:03 +02:00
William Lallemand
4ac9f54612 BUG/MEDIUM: cfgparse: segfault when userlist is misused
If the 'userlist' keyword parsing returns an error and no userlist were
previously created. The parsing of 'user' and 'group' leads to NULL
derefence.

The userlist pointer is now tested to prevent this issue.
2015-05-28 18:43:03 +02:00
William Lallemand
77063bc0c6 BUG/MINOR: cfgparse: fix typo in 'option httplog' error message
The error message was displaying the wrong argument when 'option
httplog' took a wrong argument.
2015-05-28 18:43:03 +02:00
Remi Gacogne
d3a23c3eb8 MINOR: ssl: add a destructor to free allocated SSL ressources
Using valgrind or another memory leak tracking tool is easier
when the memory internally allocated by OpenSSL is cleanly released
at shutdown.
2015-05-28 18:03:29 +02:00
Remi Gacogne
4f902b8832 BUG/MEDIUM: ssl: fix tune.ssl.default-dh-param value being overwritten
Hervé Commowick reported that the logic used to avoid complaining about
ssl-default-dh-param not being set when static DH params are present
in the certificate file was clearly wrong when more than one sni_ctx
is used.
This patch stores whether static DH params are being used for each
SSL_CTX individually, and does not overwrite the value of
tune.ssl.default-dh-param.
2015-05-28 18:03:27 +02:00
Thierry FOURNIER
be1ccaea92 MEDIUM: stats: escape some strings in the CSV dump
Some strings which must be dumped in the CSV output can contain one of
the following chars : <,>, <">, or CR/LF. This patch escapes these
strings if the case is encountered.
2015-05-28 17:49:50 +02:00
Thierry FOURNIER
ddea626de4 MINOR: common: escape CSV strings
This function checks a string for using it in a CSV output format. If
the string contains one of the following four char <">, <,>, CR or LF,
the string is encapsulated between <"> and the <"> are escaped by a <"">
sequence.

The rounding by <"> is optionnal. It can be canceled, forced or the
function choose automatically the right way.
2015-05-28 17:47:19 +02:00