Commit Graph

1153 Commits

Author SHA1 Message Date
Willy Tarreau
5fc9328aa2 MINOR: tools: make str2sa_range() directly return the protocol
We'll need this so that it can return pointers to stacked protocol in
the future (for QUIC). In addition this removes a lot of tests for
protocol validity in the callers.

Some of them were checked further apart, or after a call to
str2listener() and they were simplified as well.

There's still a trick, we can fail to return a protocol in case the caller
accepts an fqdn for use later. This is what servers do and in this case it
is valid to return no protocol. A typical example is:

   server foo localhost:1111
2020-09-16 22:08:08 +02:00
Willy Tarreau
9b3178df23 MINOR: listener: pass the chosen protocol to create_listeners()
The function will need to use more than just a family, let's pass it
the selected protocol. The caller will then be able to do all the fancy
stuff required to pick the best protocol.
2020-09-16 22:08:08 +02:00
Willy Tarreau
5e1779abbf MEDIUM: config: make str2listener() not accept datagram sockets anymore
str2listener() was temporarily hacked to support datagram sockets for
the log-forward listeners. This has has an undesirable side effect that
"bind udp@1.2.3.4:5555" was silently accepted as TCP for a bind line.

We don't need this hack anymore since the only user (log-forward) now
relies on str2receiver(). Now such an address will properly be rejected.
2020-09-16 22:08:08 +02:00
Willy Tarreau
aa333123f2 MINOR: cfgparse: add str2receiver() to parse dgram receivers
This is at least temporary, as the migration at once is way too difficuly.
For now it still creates listeners but only allows DGRAM sockets. This
aims at easing the split between listeners and receivers.
2020-09-16 22:08:08 +02:00
Willy Tarreau
3baec249b1 MEDIUM: tools: make str2sa_range() only report AF_CUST_UDP on listeners
For now only listeners can make use of AF_CUST_UDP and it requires hacks
in the DNS and logsrv code to remap it to AF_INET. Make str2sa_range()
smarter by detecting that it's called for a listener and only set these
protocol families for listeners. This way we can get rid of the hacks.
2020-09-16 22:08:08 +02:00
Willy Tarreau
a215be282d MEDIUM: tools: make str2sa_range() check for the sockpair's FD usability
Just like for inherited sockets, we want to make sure that FDs that are
mentioned in "sockpair@" are actually usable. Right now this test is
performed by the callers, but not everywhere. Typically, the following
config will fail if fd #5 is not bound:

  frontend
      bind sockpair@5

But this one will pass if fd #6 is not bound:

  backend
      server s1 sockpair@6

Now both will return an error in such a case:
   - 'bind' : cannot use file descriptor '5' : Bad file descriptor.
   - 'server s1' : cannot use file descriptor '6' : Bad file descriptor.

As such the test in str2listener() is not needed anymore (and it was
wrong by the way, as it used to test for the socket by overwriting the
local address with a new address that's made of the FD encoded on 16
bits and happens to still be at the same place, but that strictly
depends on whatever the kernel wants to put there).
2020-09-16 22:08:08 +02:00
Willy Tarreau
804f11fdf8 MINOR: config: do not test an inherited socket again
Since previous patch we know that a successfully bound fd@XXX socket
is returned as its own protocol family from str2sa_range() and not as
AF_CUST_EXISTING_FD anymore o we don't need to check for that case
in str2listener().
2020-09-16 22:08:08 +02:00
Willy Tarreau
a93e5c7fae MINOR: tools: make str2sa_range() optionally return the fd
If a file descriptor was passed, we can optionally return it. This will
be useful for listening sockets which are both a pre-bound FD and a ready
socket.
2020-09-16 22:08:08 +02:00
Willy Tarreau
909c23b086 MINOR: listener: remove the inherited arg to create_listener()
This argument can now safely be determined from fd != -1, let's just
drop it.
2020-09-16 22:08:08 +02:00
Willy Tarreau
328199348b MINOR: tools: add several PA_O_* flags in str2sa_range() callers
These flags indicate whether the call is made to fill a bind or a server
line, or even just send/recv calls (like logs or dns). Some special cases
are made for outgoing FDs (e.g. pipes for logs) or socket FDs (e.g external
listeners), and there's a distinction between stream or dgram usage that's
expected to significantly help str2sa_range() proceed appropriately with
the input information. For now they are not used yet.
2020-09-16 22:08:08 +02:00
Willy Tarreau
8b0fa8f0ab MEDIUM: config: remove all checks for missing/invalid ports/ranges
Now that str2sa_range() checks for appropriate port specification, we
don't need to implement adhoc test cases in every call place, if the
result is valid, the conditions are met otherwise the error message is
appropriately filled.
2020-09-16 22:08:08 +02:00
Willy Tarreau
809587635e MINOR: tools: add several PA_O_PORT_* flags in str2sa_range() callers
These flags indicate what is expected regarding port specifications. Some
callers accept none, some need fixed ports, some have it mandatory, some
support ranges, and some take an offset. Each possibilty is reflected by
an option. For now they are not exploited, but the goal is to instrument
str2sa_range() to properly parse that.
2020-09-16 22:08:07 +02:00
Willy Tarreau
cd3a5591f6 MINOR: tools: make str2sa_range() take more options than just resolve
We currently have an argument to require that the address is resolved
but we'll soon add more, so let's turn it into a bit field. The old
"resolve" boolean is now PA_O_RESOLVE.
2020-09-16 22:08:07 +02:00
Willy Tarreau
a5b325f92c MINOR: protocol: add a real family for existing FDs
At some places (log fd@XXX, bind fd@XXX) we support using an explicit
file descriptor number, that is placed into the sockaddr for later use.
The problem is that till now it was done with an AF_UNSPEC family, which
is also used for other situations like missing info or rings (for logs).

Let's create an "official" family AF_CUST_EXISTING_FD for this case so
that we are certain the FD can be found in the address when it is set.
2020-09-16 22:08:07 +02:00
Willy Tarreau
371590661e REORG: listener: move the listening address to a struct receiver
The address will be specific to the receiver so let's move it there.
2020-09-16 22:08:01 +02:00
Willy Tarreau
e26993c098 MINOR: listener: move bind_proc and bind_thread to struct settings
As mentioned previously, these two fields come under the settings
struct since they'll be used to bind receivers as well.
2020-09-16 20:13:13 +02:00
Willy Tarreau
e42d87f3de BUG/MINOR: dns: gracefully handle the "udp@" address format for nameservers
Just like with previous commit, DNS nameservers are affected as well with
addresses starting in "udp@", but here it's different, because due to
another bug in the DNS parser, the address is rejected, indicating that
it doesn't have a ->connect() method. Similarly, the DNS code believes
it's working on top of TCP at this point and this used to work because of
this. The same fix is applied to remap the protocol and the ->connect test
was dropped.

No backport is needed, as the ->connect() test will never strike in 2.2
or below.
2020-09-16 20:11:52 +02:00
Tim Duesterhus
b53dd03dc0 BUG/MINOR: Fix type passed of sizeof() for calloc()
newsrv->curr_idle_thr is of type `unsigned int`, not `int`. Fix this issue
by simply passing the dereferenced pointer to sizeof, which is the preferred
style anyway.

This bug was introduced in commit dc2f2753e9.
It first appeared in 2.2-dev5. The patch must be backported to 2.2+.

It is notable that the `calloc` call was not introduced within the commit in
question. The allocation was already happening before that commit and it
already looked like it does after applying the patch. Apparently the
argument for the `sizeof` managed to get broken during the rearrangement
that happened in that commit:

     	for (i = 0; i < global.nbthread; i++)
    -		MT_LIST_INIT(&newsrv->idle_orphan_conns[i]);
    -	newsrv->curr_idle_thr = calloc(global.nbthread, sizeof(*newsrv->curr_idle_thr));
    +		MT_LIST_INIT(&newsrv->safe_conns[i]);
    +
    +	newsrv->curr_idle_thr = calloc(global.nbthread, sizeof(int));

Even more notable is that I previously fixed that *exact same* allocation in
commit 017484c80f.

So apparently it was managed to break this single line twice in the same
way for whatever reason there might be.
2020-09-12 20:31:25 +02:00
Tim Duesterhus
f92afb732b MEDIUM: cfgparse: Emit hard error on truncated lines
As announced within the emitted log message this is going to become a hard
error in 2.3. It's 2.3 time now, let's do this.

see 2fd5bdb439
2020-08-18 22:51:24 +02:00
Emeric Brun
d3db3846c5 BUG/MEDIUM: resolve: fix init resolving for ring and peers section.
Reported github issue #759 shows there is no name resolving
on server lines for ring and peers sections.

This patch introduce the resolving for those lines.

This patch adds  boolean a parameter to parse_server function to specify
if we want the function to perform an initial name resolving using libc.

This boolean is forced to true in case of peers or ring section.

The boolean is kept to false in case of classic servers (from
backend/listen)

This patch should be backported in branches where peers sections
support 'server' lines.
2020-07-21 17:59:20 +02:00
Willy Tarreau
9b9c174e60 BUILD: config: fix again bugs gcc warnings on calloc
Since commit ad37c7ab ("BUILD: config: address build warning on
raspbian+rpi4") gcc 7.3.0 complains again on x86_64 (while 8.2.0
does not) :

  src/cfgparse.c: In function 'check_config_validity':
  src/cfgparse.c:3593:26: warning: argument 1 range [18446744071562067968, 18446744073709551615] exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
       newsrv->idle_conns = calloc(global.nbthread, sizeof(*newsrv->idle_conns));
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This thing is completely bogus (actually the RPi one was the most wrong).
Let's try to shut them both by using an unsigned short for the cast which
is expected to satisfy everyone. It's worth noting that the exact same call
a few lines above and below do not trigger this stupid warning.

This should be backported to 2.2 since the fix above was put there already.
2020-07-17 15:04:53 +02:00
Willy Tarreau
ad37c7ab25 BUILD: config: address build warning on raspbian+rpi4
Issue #747 reports that building on raspbian for rpi4 triggers this
warning:

  src/cfgparse.c: In function 'check_config_validity':
  src/cfgparse.c:3584:26: warning: argument 1 range [2147483648, 4294967295] exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
     newsrv->idle_conns = calloc((unsigned)global.nbthread, sizeof(*newsrv->idle_conns));
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It's surprising because the declared type is size_t and the argument is
unsigned (i.e. the same type on 32-bit) precisely to avoid cast issues,
but gcc seems to be too smart at this one and to issue a warning over
the valid range, implying that passing the originally required type would
also warn. Given that these are the only casts in calloc and other ones
don't complain, let's drop them.

All 3 were added by commit dc2f2753e ("MEDIUM: servers: Split the
connections into idle, safe, and available.")  that went into 2.2, so
this should be backported.
2020-07-17 14:18:36 +02:00
Emeric Brun
54932b4408 MINOR: log: adds syslog udp message handler and parsing.
This patch introduce a new fd handler used to parse syslog
message on udp.

The parsing function returns level, facility and metadata that
can be immediatly reused to forward message to a log server.

This handler is enabled on udp listeners if proxy is internally set
to mode PR_MODE_SYSLOG
2020-07-15 17:50:12 +02:00
Emeric Brun
3835c0dcb5 MEDIUM: udp: adds minimal proto udp support for message listeners.
This patch introduce proto_udp.c targeting a further support of
log forwarding feature.

This code was originally produced by Frederic Lecaille working on
QUIC support and only minimal requirements for syslog support
have been merged.
2020-07-15 17:50:12 +02:00
Dragan Dosen
1e3b16f74f MINOR: log-format: allow to preserve spacing in log format strings
Now it's possible to preserve spacing everywhere except in "log-format",
"log-format-sd" and "unique-id-format" directives, where spaces are
delimiters and are merged. That may be useful when the response payload
is specified as a log format string by "lf-file" or "lf-string", or even
for headers or anything else.

In order to merge spaces, a new option LOG_OPT_MERGE_SPACES is applied
exclusively on options passed to function parse_logformat_string().

This patch fixes an issue #701 ("http-request return log-format file
evaluation altering spacing of ASCII output/art").
2020-07-02 10:11:44 +02:00
Olivier Houchard
f8f4c2ef60 CLEANUP: connections: rename the toremove_lock to takeover_lock
This lock was misnamed and a bit confusing. It's only used for takeover
so let's call it takeover_lock.
2020-07-01 17:09:10 +02:00
Olivier Houchard
f21695bd8b BUG/MINOR: threads: Don't forget to init each thread toremove_lock.
Don't forget to use HA_SPIN_INIT() on each toremove_lock, or DEBUG_THREAD may
not work reliably with it.

This should be backported to 2.1 and 2.0.
2020-06-29 17:50:29 +02:00
Willy Tarreau
4d82bf5c2e MINOR: connection: align toremove_{lock,connections} and cleanup into idle_conns
We used to have 3 thread-based arrays for toremove_lock, idle_cleanup,
and toremove_connections. The problem is that these items are small,
and that this creates false sharing between threads since it's possible
to pack up to 8-16 of these values into a single cache line. This can
cause real damage where there is contention on the lock.

This patch creates a new array of struct "idle_conns" that is aligned
on a cache line and which contains all three members above. This way
each thread has access to its variables without hindering the other
ones. Just doing this increased the HTTP/1 request rate by 5% on a
16-thread machine.

The definition was moved to connection.{c,h} since it appeared a more
natural evolution of the ongoing changes given that there was already
one of them declared in connection.h previously.
2020-06-28 10:52:36 +02:00
Willy Tarreau
08488f66b6 BUG/MINOR: cfgparse: correctly deal with empty lines
Issue 23653 in oss-fuzz reports a heap overflow bug which is in fact a
bug introduced by commit 9e1758efb ("BUG/MEDIUM: cfgparse: use
parse_line() to expand/unquote/unescape config lines") to address
oss-fuzz issue 22689, which was only partially fixed by commit 70f58997f
("BUG/MINOR: cfgparse: Support configurations without newline at EOF").

Actually on an empty line, end == line so we cannot dereference end-1
to check for a trailing LF without first being sure that end is greater
than line.

No backport is needed, this is 2.2 only.
2020-06-26 17:24:54 +02:00
Willy Tarreau
c54e5ad9cc MINOR: cfgparse: sanitize the output a little bit
With the rework of the config line parser, we've started to emit a dump
of the initial line underlined by a caret character indicating the error
location. But with extremely large lines it starts to take time and can
even cause trouble to slow terminals (e.g. over ssh), and this becomes
useless. In addition, control characters could be dumped as-is which is
bad, especially when the input file is accidently wrong (an executable).

This patch adds a string sanitization function which isolates an area
around the error position in order to report only that area if the string
is too large. The limit was set to 80 characters, which will result in
roughly 40 chars around the error being reported only, prefixed and suffixed
with "..." as needed. In addition, non-printable characters in the line are
now replaced with '?' so as not to corrupt the terminal. This way invalid
variable names, unmatched quotes etc will be easier to spot.

A typical output is now:

  [ALERT] 176/092336 (23852) : parsing [bad.cfg:8]: forbidden first char in environment variable name at position 811957:
    ...c$PATH$PATH$d(xlc`%?$PATH$PATH$dgc?T$%$P?AH?$PATH$PATH$d(?$PATH$PATH$dgc?%...
                                            ^
2020-06-25 09:43:27 +02:00
Willy Tarreau
07d47060e0 BUG/MINOR: cfgparse: report extraneous args *after* the string is allocated
The config parser change in commit 9e1758efb ("BUG/MEDIUM: cfgparse: use
parse_line() to expand/unquote/unescape config lines") is wrong when
displaying the last parsed word, because it doesn't verify that the output
string was properly allocated. This may fail in two cases:
  - very first line (outline is NULL, as in oss-fuzz issue 23657)
  - much longer line than previous ones, requiring a realloc(), in which
    case the final 0 is out of the allocated space.

This patch moves the reporting after the allocation check to fix this.

No backport is needed, this is 2.2 only.
2020-06-25 09:43:27 +02:00
Willy Tarreau
61dd44bbc1 MINOR: tools: make parse_line() always terminate the args list
parse_line() as added in commit c8d167bcf ("MINOR: tools: add a new
configurable line parse, parse_line()") presents an difficult usage
because it's up to the caller to determine the last written argument
based on what was passed to it. In practice the only way to safely
use it is for the caller to always pass nbarg-1 and make that last
entry point to the last arg + its strlen. This is annoying because
it makes it as painful to use as the infamous strncpy() while it has
all the information the caller needs.

This patch changes its behavior so that it guarantees that at least
one argument will point to the trailing zero at the end of the output
string, as long as there is at least one argument. The caller just
has to pass +1 to the arg count to make sure at least a last one is
empty.
2020-06-25 09:43:27 +02:00
Willy Tarreau
40cb26f6ec BUG/MINOR: cfgparse: don't increment linenum on incomplete lines
When fgets() returns an incomplete line we must not increment linenum
otherwise line numbers become incorrect. This may happen when parsing
files with extremely long lines which require a realloc().

The bug has been present since unbounded line length was supported, so
the fix should be backported to older branches.
2020-06-25 09:43:27 +02:00
Tim Duesterhus
c17a5fac57 BUG/MINOR: cfgparse: Fix calculation of position for PARSE_ERR_TOOMANY message
The arguments are relative to the outline, not relative to the input line.

This patch fixes up commit 9e1758efbd which
is 2.2 only. No backport needed.
2020-06-23 19:03:20 +02:00
Tim Duesterhus
f3d2c6d706 BUG/MINOR: cfgparse: Fix argument reference in PARSE_ERR_TOOMANY message
The returned `arg` value is the number of arguments found, but in case
of the error message it's not a valid argument index.

Because we know how many arguments we allowed (MAX_LINE_ARGS) we know
what to print in the error message, so do just that.

Consider a configuration like this:

    listen foo
      1 2 3 [...] 64 65

Then running a configuration check within valgrind reports the following:

    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E8B83: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Use of uninitialised value of size 8
    ==18265==    at 0x56E576B: _itoa_word (_itoa.c:179)
    ==18265==    by 0x56E912C: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E5775: _itoa_word (_itoa.c:179)
    ==18265==    by 0x56E912C: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E91AF: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E8C59: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E941A: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E8CAB: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56E8CE2: vfprintf (vfprintf.c:1631)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    ==18265== Conditional jump or move depends on uninitialised value(s)
    ==18265==    at 0x56EA2DB: vfprintf (vfprintf.c:1632)
    ==18265==    by 0x57B1895: __vsnprintf_chk (vsnprintf_chk.c:63)
    ==18265==    by 0x4A8642: vsnprintf (stdio2.h:77)
    ==18265==    by 0x4A8642: memvprintf (tools.c:3647)
    ==18265==    by 0x4CB8A4: print_message (log.c:1085)
    ==18265==    by 0x4CE0AC: ha_alert (log.c:1128)
    ==18265==    by 0x459E41: readcfgfile (cfgparse.c:1978)
    ==18265==    by 0x507CB5: init (haproxy.c:2029)
    ==18265==    by 0x4182A2: main (haproxy.c:3137)
    ==18265==
    [ALERT] 174/165720 (18265) : parsing [./config.cfg:2]: too many words, truncating at word 65, position -95900735: <(null)>.
    [ALERT] 174/165720 (18265) : Error(s) found in configuration file : ./config.cfg
    [ALERT] 174/165720 (18265) : Fatal errors found in configuration.

Valgrind reports conditional jumps relying on an undefined value and the
error message clearly shows incorrect stuff.

After this patch is applied the relying on undefined values is gone and
the <(null)> will actually show the argument. However the position value
still is incorrect. This will be fixed in a follow up patch.

This patch fixes up commit 9e1758efbd which
is 2.2 only. No backport needed.
2020-06-23 19:03:20 +02:00
Tim Duesterhus
2fd5bdb439 MINOR: cfgparse: Warn on truncated lines / files
As discussed on the list: https://www.mail-archive.com/haproxy@formilux.org/msg37698.html

This patch adds warnings to the configuration parser that detect the
following situations:

- A line being truncated by a null byte in the middle.
- A file not ending in a new line (and possibly being truncated).
2020-06-23 05:14:59 +02:00
Tim Duesterhus
70f58997f4 BUG/MINOR: cfgparse: Support configurations without newline at EOF
Fix parsing of configurations if the configuration file does not end with
an LF.

This patch fixes GitHub issue #704. It's a regression in
9e1758efbd which is 2.2 specific. No backport
needed.
2020-06-23 05:14:36 +02:00
Tim Duesterhus
9f658a554f BUG/MINOR: cfgparse: Add missing fatal++ in PARSE_ERR_HEX case
This fixes up commit 32234e7513. This
patch should be backported whereever that commit is backported.
2020-06-16 18:25:40 +02:00
Willy Tarreau
32234e7513 BUG/MEDIUM: cfgparse: stop after a reasonable amount of fatal error
One issue with the config parser is that while it tries to report as many
errors as possible at once, it's actually unbounded. Thus, when calling
haproxy on a wrong file, it can take ages to process, such as here on
half a gigabyte of map file instead of config file:

  $ time ./haproxy -c -f large.map 2>&1 |wc -l
  16777220

  real    0m31.324s
  user    0m22.595s
  sys     0m28.909s

This patch modifies readcfgfile() to stop reading the config file after a
reasonable amount of fatal errors. This threshold is set to 50, which seems
more than enough to spot a recurrent issue with a bit of context in a terminal
to address several issues at once, without filling logs nor taking time to
parse the file. The difference is clear now:

  $ time ./haproxy -c -f large.map 2>&1 |wc -l
  55

  real    0m0.005s
  user    0m0.004s
  sys     0m0.003s

This may be backported to older versions without causing too many
difficulties. However the patch will not apply as-is, it will require
to increment the "fatal" count for each place where ERR_FATAL is set
in the parsing loop.
2020-06-16 17:19:01 +02:00
Willy Tarreau
9e1758efbd BUG/MEDIUM: cfgparse: use parse_line() to expand/unquote/unescape config lines
Issue 22689 in oss-fuzz shows that specially crafted config files can take
a long time to process. This happens when variable expansion, backslash
escaping or unquoting causes calls to memmove() and possibly to realloc()
resulting in O(N^2) complexity with N following the line size.

By using parse_line() we now have a safe parser that remains in O(N)
regardless of the type of operation. Error reporting changed a little bit
since the errors are not reported anymore from the deepest parsing level.
As such we now report the beginning of the error. One benefit is that for
many invalid character sequences, the original line is shown and the first
bad char or sequence is designated with a caret ('^'), which tends to be
visually easier to spot, for example:

  [ALERT] 167/170507 (14633) : parsing [mini5.cfg:19]: unmatched brace in environment variable name below:
    "${VAR"}
      ^
or:

  [ALERT] 167/170645 (14640) : parsing [mini5.cfg:18]: unmatched quote below:
    timeout client 10s'
                      ^

In case the target buffer is too short for the new line, the output buffer
is grown in 1kB chunks and kept till the end, so that it should not happen
too often.

Before this patch a test like below involving a 4 MB long line would take
138s to process, 98% of which were spent in __memmove_avx_unaligned_erms(),
and now it takes only 65 milliseconds:

  $ perl -e 'print "\"\$A\""x1000000,"\n"' | ./haproxy -c -f /dev/stdin 2>/dev/null

This may be backported to stable versions after a long period of
observation to be sure nothing broke. It relies on patch "MINOR: tools:
add a new configurable line parse, parse_line()".
2020-06-16 17:07:02 +02:00
Willy Tarreau
b2551057af CLEANUP: include: tree-wide alphabetical sort of include files
This patch fixes all the leftovers from the include cleanup campaign. There
were not that many (~400 entries in ~150 files) but it was definitely worth
doing it as it revealed a few duplicates.
2020-06-11 10:18:59 +02:00
Willy Tarreau
cee013e4e0 REORG: check: move the e-mail alerting code to mailers.c
check.c is one of the largest file and contains too many things. The
e-mail alerting code is stored there while nothing is in mailers.c.
Let's move this code out. That's only 4% of the code but a good start.
In order to do so, a few tcp-check functions had to be exported.
2020-06-11 10:18:58 +02:00
Willy Tarreau
6be7849f39 REORG: include: move cfgparse.h to haproxy/cfgparse.h
There's no point splitting the file in two since only cfgparse uses the
types defined there. A few call places were updated and cleaned up. All
of them were in C files which register keywords.

There is nothing left in common/ now so this directory must not be used
anymore.
2020-06-11 10:18:58 +02:00
Willy Tarreau
dfd3de8826 REORG: include: move stream.h to haproxy/stream{,-t}.h
This one was not easy because it was embarking many includes with it,
which other files would automatically find. At least global.h, arg.h
and tools.h were identified. 93 total locations were identified, 8
additional includes had to be added.

In the rare files where it was possible to finalize the sorting of
includes by adjusting only one or two extra lines, it was done. But
all files would need to be rechecked and cleaned up now.

It was the last set of files in types/ and proto/ and these directories
must not be reused anymore.
2020-06-11 10:18:58 +02:00
Willy Tarreau
1e56f92693 REORG: include: move server.h to haproxy/server{,-t}.h
extern struct dict server_name_dict was moved from the type file to the
main file. A handful of inlined functions were moved at the bottom of
the file. Call places were updated to use server-t.h when relevant, or
to simply drop the entry when not needed.
2020-06-11 10:18:58 +02:00
Willy Tarreau
4980160ecc REORG: include: move backend.h to haproxy/backend{,-t}.h
The files remained mostly unchanged since they were OK. However, half of
the users didn't need to include them, and about as many actually needed
to have it and used to find functions like srv_currently_usable() through
a long chain that broke when moving the file.
2020-06-11 10:18:58 +02:00
Willy Tarreau
a264d960f6 REORG: include: move proxy.h to haproxy/proxy{,-t}.h
This one is particularly difficult to split because it provides all the
functions used to manipulate a proxy state and to retrieve names or IDs
for error reporting, and as such, it was included in 73 files (down to
68 after cleanup). It would deserve a small cleanup though the cut points
are not obvious at the moment given the number of structs involved in
the struct proxy itself.
2020-06-11 10:18:58 +02:00
Willy Tarreau
aeed4a85d6 REORG: include: move log.h to haproxy/log{,-t}.h
The current state of the logging is a real mess. The main problem is
that almost all files include log.h just in order to have access to
the alert/warning functions like ha_alert() etc, and don't care about
logs. But log.h also deals with real logging as well as log-format and
depends on stream.h and various other things. As such it forces a few
heavy files like stream.h to be loaded early and to hide missing
dependencies depending where it's loaded. Among the missing ones is
syslog.h which was often automatically included resulting in no less
than 3 users missing it.

Among 76 users, only 5 could be removed, and probably 70 don't need the
full set of dependencies.

A good approach would consist in splitting that file in 3 parts:
  - one for error output ("errors" ?).
  - one for log_format processing
  - and one for actual logging.
2020-06-11 10:18:58 +02:00
Willy Tarreau
c7babd8570 REORG: include: move filters.h to haproxy/filters{,-t}.h
Just a minor change, moved the macro definitions upwards. A few caller
files were updated since they didn't need to include it.
2020-06-11 10:18:58 +02:00
Willy Tarreau
c2b1ff04e5 REORG: include: move http_ana.h to haproxy/http_ana{,-t}.h
It was moved without any change, however many callers didn't need it at
all. This was a consequence of the split of proto_http.c into several
parts that resulted in many locations to still reference it.
2020-06-11 10:18:58 +02:00
Willy Tarreau
f1d32c475c REORG: include: move channel.h to haproxy/channel{,-t}.h
The files were moved with no change. The callers were cleaned up a bit
and a few of them had channel.h removed since not needed.
2020-06-11 10:18:58 +02:00
Willy Tarreau
2867159d63 REORG: include: move lb_map.h to haproxy/lb_map{,-t}.h
Nothing was changed.
2020-06-11 10:18:58 +02:00
Willy Tarreau
2eec9b5f95 REORG: include: move stats.h to haproxy/stats{,-t}.h
Just some minor reordering, and the usual cleanup of call places for
those which didn't need it. We don't include the whole tools.h into
stats-t anymore but just tools-t.h.
2020-06-11 10:18:58 +02:00
Willy Tarreau
8c42b8a147 REORG: include: split common/uri_auth.h into haproxy/uri_auth{,-t}.h
Initially it looked like this could have been placed into auth.h or
stats.h but it's not the case as it's what makes the link between them
and the HTTP layer. However the file needed to be split in two. Quite
a number of call places were dropped because these were mostly leftovers
from the early days where the stats and cli were packed together.
2020-06-11 10:18:58 +02:00
Willy Tarreau
dcc048a14a REORG: include: move acl.h to haproxy/acl.h{,-t}.h
The files were moved almost as-is, just dropping arg-t and auth-t from
acl-t but keeping arg-t in acl.h. It was useful to revisit the call places
since a handful of files used to continue to include acl.h while they did
not need it at all. Struct stream was only made a forward declaration
since not otherwise needed.
2020-06-11 10:18:58 +02:00
Willy Tarreau
48d25b3bc9 REORG: include: move session.h to haproxy/session{,-t}.h
Almost no change was needed beyond a little bit of reordering of the
types file and adjustments to use session-t instead of session at a
few places.
2020-06-11 10:18:58 +02:00
Willy Tarreau
872f2ea209 REORG: include: move stick_table.h to haproxy/stick_table{,-t}.h
The stktable_types[] array declaration was moved to the main file as
it had nothing to do in the types. A few declarations were reordered
in the types file so that defines were before the structs. Thread-t
was added since there are a few __decl_thread(). The loss of peers.h
revealed that cfgparse-listen needed it.
2020-06-11 10:18:58 +02:00
Willy Tarreau
3c2a7c2788 REORG: include: move peers.h to haproxy/peers{,-t}.h
The cfg_peers external declaration was moved to the main file instead
of the type one. A few types were still missing from the proto, causing
warnings in the functions prototypes (proxy, stick_table).
2020-06-11 10:18:58 +02:00
Willy Tarreau
4aa573da6f REORG: include: move checks.h to haproxy/check{,-t}.h
All includes that were not absolutely necessary were removed because
checks.h happens to very often be part of dependency loops. A warning
was added about this in check-t.h. The fields, enums and structs were
a bit tidied because it's particularly tedious to find anything there.
It would make sense to split this in two or more files (at least
extract tcp-checks).

The file was renamed to the singular because it was one of the rare
exceptions to have an "s" appended to its name compared to the struct
name.
2020-06-11 10:18:58 +02:00
Willy Tarreau
7ea393d95e REORG: include: move connection.h to haproxy/connection{,-t}.h
The type file is becoming a mess, half of it is for the proxy protocol,
another good part describes conn_streams and mux ops, it would deserve
being split again. At least it was reordered so that elements are easier
to find, with the PP-stuff left at the end. The MAX_SEND_FD macro was moved
to compat.h as it's said to be the value for Linux.
2020-06-11 10:18:58 +02:00
Willy Tarreau
8b550afe1e REORG: include: move tcp_rules.h to haproxy/tcp_rules.h
There's no type file on this one which is pretty simple.
2020-06-11 10:18:58 +02:00
Willy Tarreau
cea0e1bb19 REORG: include: move task.h to haproxy/task{,-t}.h
The TASK_IS_TASKLET() macro was moved to the proto file instead of the
type one. The proto part was a bit reordered to remove a number of ugly
forward declaration of static inline functions. About a tens of C and H
files had their dependency dropped since they were not using anything
from task.h.
2020-06-11 10:18:58 +02:00
Willy Tarreau
f268ee8795 REORG: include: split global.h into haproxy/global{,-t}.h
global.h was one of the messiest files, it has accumulated tons of
implicit dependencies and declares many globals that make almost all
other file include it. It managed to silence a dependency loop between
server.h and proxy.h by being well placed to pre-define the required
structs, forcing struct proxy and struct server to be forward-declared
in a significant number of files.

It was split in to, one which is the global struct definition and the
few macros and flags, and the rest containing the functions prototypes.

The UNIX_MAX_PATH definition was moved to compat.h.
2020-06-11 10:18:58 +02:00
Willy Tarreau
e6ce10be85 REORG: include: move sample.h to haproxy/sample{,-t}.h
This one is particularly tricky to move because everyone uses it
and it depends on a lot of other types. For example it cannot include
arg-t.h and must absolutely only rely on forward declarations to avoid
dependency loops between vars -> sample_data -> arg. In order to address
this one, it would be nice to split the sample_data part out of sample.h.
2020-06-11 10:18:58 +02:00
Willy Tarreau
213e99073b REORG: include: move listener.h to haproxy/listener{,-t}.h
stdlib and list were missing from listener.h, otherwise it was OK.
2020-06-11 10:18:58 +02:00
Willy Tarreau
546ba42c73 REORG: include: move lb_fwrr.h to haproxy/lb_fwrr{,-t}.h
Nothing fancy, includes were already OK. The proto didn't reference the
type, this was fixed. Still references proxy.h and server.h from types/.
2020-06-11 10:18:58 +02:00
Willy Tarreau
0254941666 REORG: include: move lb_fwlc.h to haproxy/lb_fwlc{,-t}.h
Nothing fancy, includes were already OK. The proto didn't reference the
type, this was fixed. Still references proxy.h and server.h from types/.
2020-06-11 10:18:58 +02:00
Willy Tarreau
b5fc3bf6dc REORG: include: move lb_fas.h to haproxy/lb_fas{,-t}.h
Nothing fancy, includes were already OK. The proto didn't reference the
type, this was fixed. Still references proxy.h and server.h from types/.
2020-06-11 10:18:58 +02:00
Willy Tarreau
fbe8da3320 REORG: include: move lb_chash.h to haproxy/lb_chash{,-t}.h
Nothing fancy, includes were already OK. The proto didn't reference the
type, this was fixed. Still references proxy.h and server.h from types/.
2020-06-11 10:18:58 +02:00
Willy Tarreau
c761f843da REORG: include: move http_rules.h to haproxy/http_rules.h
There was no include file. This one still includes types/proxy.h.
2020-06-11 10:18:57 +02:00
Willy Tarreau
8efbdfb77b REORG: include: move obj_type.h to haproxy/obj_type{,-t}.h
No change was necessary. It still includes lots of types/* files.
2020-06-11 10:18:57 +02:00
Willy Tarreau
762d7a5117 REORG: include: move frontend.h to haproxy/frontend.h
There was no type file for this one, it only contains frontend_accept().
2020-06-11 10:18:57 +02:00
Willy Tarreau
278161c1b8 REORG: include: move capture.h to haproxy/capture{,-t}.h
The file was split into two since it contains a variable declaration.
2020-06-11 10:18:57 +02:00
Willy Tarreau
cc9bbfb7b5 REORG: include: split mailers.h into haproxy/mailers{,-t}.h
The file mostly contained struct definitions but there was also a
variable export. Most of the stuff currently lies in checks.h and
should definitely move here!
2020-06-11 10:18:57 +02:00
Willy Tarreau
eb92deb500 REORG: include: move dns.h to haproxy/dns{,-t}.h
The files were moved as-is.
2020-06-11 10:18:57 +02:00
Willy Tarreau
ac13aeaa89 REORG: include: move auth.h to haproxy/auth{,-t}.h
The STATS_DEFAULT_REALM and STATS_DEFAULT_URI were moved to defaults.h.
It was required to include types/pattern.h and types/sample.h since they
are mentioned in function prototypes.

It would be wise to merge this with uri_auth.h later.
2020-06-11 10:18:57 +02:00
Willy Tarreau
122eba92b7 REORG: include: move action.h to haproxy/action{,-t}.h
List.h was missing for LIST_ADDQ(). A few unneeded includes of action.h
were removed from certain files.

This one still relies on applet.h and stick-table.h.
2020-06-11 10:18:57 +02:00
Willy Tarreau
48fbcae07c REORG: tools: split common/standard.h into haproxy/tools{,-t}.h
And also rename standard.c to tools.c. The original split between
tools.h and standard.h dates from version 1.3-dev and was mostly an
accident. This patch moves the files back to what they were expected
to be, and takes care of not changing anything else. However this
time tools.h was split between functions and types, because it contains
a small number of commonly used macros and structures (e.g. name_desc)
which in turn cause the massive list of includes of tools.h to conflict
with the callers.

They remain the ugliest files of the whole project and definitely need
to be cleaned and split apart. A few types are defined there only for
functions provided there, and some parts are even OS-specific and should
move somewhere else, such as the symbol resolution code.
2020-06-11 10:18:57 +02:00
Willy Tarreau
2dd7c35052 REORG: include: move protocol.h to haproxy/protocol{,-t}.h
The protocol.h files are pretty low in the dependency and (sadly) used
by some files from common/. Almost nothing was changed except lifting a
few comments.
2020-06-11 10:18:57 +02:00
Willy Tarreau
7a00efbe43 REORG: include: move common/namespace.h to haproxy/namespace{,-t}.h
The type was moved out as it's used by standard.h for netns_entry.
Instead of just being a forward declaration when not used, it's an
empty struct, which makes gdb happier (the resulting stripped executable
is the same).
2020-06-11 10:18:57 +02:00
Willy Tarreau
c13ed53b12 REORG: include: move common/chunk.h to haproxy/chunk.h
No change was necessary, it was already properly split.
2020-06-11 10:18:57 +02:00
Willy Tarreau
d0ef439699 REORG: include: move common/memory.h to haproxy/pool.h
Now the file is ready to be stored into its final destination. A few
minor reorderings were performed to keep the file properly organized,
making the various sections more visible (cache & lockless).

In addition and to stay consistent, memory.c was renamed to pool.c.
2020-06-11 10:18:57 +02:00
Willy Tarreau
92b4f1372e REORG: include: move time.h from common/ to haproxy/
This one is included almost everywhere and used to rely on a few other
.h that are not needed (unistd, stdlib, standard.h). It could possibly
make sense to split it into multiple parts to distinguish operations
performed on timers and the internal time accounting, but at this point
it does not appear much important.
2020-06-11 10:18:56 +02:00
Willy Tarreau
3f567e4949 REORG: include: split hathreads into haproxy/thread.h and haproxy/thread-t.h
This splits the hathreads.h file into types+macros and functions. Given
that most users of this file used to include it only to get the definition
of THREAD_LOCAL and MAXTHREADS, the bare minimum was placed into thread-t.h
(i.e. types and macros).

All the thread management was left to haproxy/thread.h. It's worth noting
the drop of the trailing "s" in the name, to remove the permanent confusion
that arises between this one and the system implementation (no "s") and the
makefile's option (no "s").

For consistency, src/hathreads.c was also renamed thread.c.

A number of files were updated to only include thread-t which is the one
they really needed.

Some future improvements are possible like replacing empty inlined
functions with macros for the thread-less case, as building at -O0 disables
inlining and causes these ones to be emitted. But this really is cosmetic.
2020-06-11 10:18:56 +02:00
Willy Tarreau
8d36697dee REORG: include: move base64.h, errors.h and hash.h from common to to haproxy/
These ones do not depend on any other file. One used to include
haproxy/api.h but that was solely for stddef.h.
2020-06-11 10:18:56 +02:00
Willy Tarreau
4c7e4b7738 REORG: include: update all files to use haproxy/api.h or api-t.h if needed
All files that were including one of the following include files have
been updated to only include haproxy/api.h or haproxy/api-t.h once instead:

  - common/config.h
  - common/compat.h
  - common/compiler.h
  - common/defaults.h
  - common/initcall.h
  - common/tools.h

The choice is simple: if the file only requires type definitions, it includes
api-t.h, otherwise it includes the full api.h.

In addition, in these files, explicit includes for inttypes.h and limits.h
were dropped since these are now covered by api.h and api-t.h.

No other change was performed, given that this patch is large and
affects 201 files. At least one (tools.h) was already freestanding and
didn't get the new one added.
2020-06-11 10:18:42 +02:00
Christopher Faulet
c0fcbe4150 BUG/MINOR: checks: Fix test on http-check rulesets during config validity check
When checking the config validity of the http-check rulesets, the test on the
ruleset type is inverted. So a warning about ignored directive is emitted when
the config is valid and omitted when it should be reported.

No backport needed.
2020-06-03 19:08:35 +02:00
Willy Tarreau
78675252fb BUG/MINOR: nameservers: fix error handling in parsing of resolv.conf
In issue #657, Coverity found a bug in the "nameserver" parser for the
resolv.conf when "parse-resolv-conf" is set. What happens is that if an
unparsable address appears on a "nameserver" line, it will destroy the
previously allocated pointer before reporting the warning, then the next
"nameserver" line will dereference it again and wlil cause a crash. If
the faulty nameserver is the last one, it will only be a memory leak.
Let's just make sure we preserve the pointer when handling the error.
The patch also fixes a typo in the warning.

The bug was introduced in 1.9 with commit 44e609bfa ("MINOR: dns:
Implement `parse-resolv-conf` directive") so the fix needs to be backported
up to 1.9 or 2.0.
2020-05-28 18:12:22 +02:00
Tim Duesterhus
ac4930e00c MINOR: cfgparse: Improve error message for invalid \x sequences
This patch states the invalid \x sequence within the error message.
2020-05-07 22:46:26 +02:00
Tim Duesterhus
e6291956e7 BUG/MINOR: cfgparse: Abort parsing the current line if an invalid \x sequence is encountered
This fixes OSS Fuzz issue https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21931.

OSS Fuzz detected a hang on configuration parsing for a 200kB line with a large number of
invalid escape sequences. Most likely due to the amounts of error output generated.

This issue is very minor, because usually generated configurations are to be trusted.

The bug exists since at the very least HAProxy 1.4. The patch may be backported if desired.
2020-05-07 22:46:25 +02:00
Christopher Faulet
f82ea4ae4c BUG/MINOR: config: Make use_backend and use-server post-parsing less obscur
During use_backend and use-server post-parsing, if the log-format string used to
specify the backend or the server is just a single string, the log-format string
(a list, internally) is replaced by the string itself. Because the field is an
union, the list is not emptied according to the rules of art. The element, when
released, is not removed from the list. There is no bug, but it is clearly not
obvious and error prone.

This patch should fix #544. The fix for the use_backend post-parsing may be
backported to all stable releases. use-server is static in 2.1 and prior.
2020-05-07 15:59:35 +02:00
Christopher Faulet
e5870d872b MAJOR: checks: Implement HTTP check using tcp-check rules
HTTP health-checks are now internally based on tcp-checks. Of course all the
configuration parsing of the "http-check" keyword and the httpchk option has
been rewritten. But the main changes is that now, as for tcp-check ruleset, it
is possible to perform several send/expect sequences into the same
health-checks. Thus the connect rule is now also available from HTTP checks, jst
like set-var, unset-var and comment rules.

Because the request defined by the "option httpchk" line is used for the first
request only, it is now possible to set the method, the uri and the version on a
"http-check send" line.
2020-04-27 09:39:38 +02:00
Christopher Faulet
404f919995 MEDIUM: checks: Use a shared ruleset to store tcp-check rules
All tcp-check rules are now stored in the globla shared list. The ones created
to parse a specific protocol, for instance redis, are already stored in this
list. Now pure tcp-check rules are also stored in it. The ruleset name is
created using the proxy name and its config file and line. tcp-check rules
declared in a defaults section are also stored this way using "defaults" as
proxy name.

For now, all tcp-check ruleset are stored in a list. But it could be a bit slow
to looks for a specific ruleset with a huge number of backends. So, it could be
a good idea to use a tree instead.
2020-04-27 09:39:38 +02:00
Christopher Faulet
811f78ced1 MEDIUM: checks: Implement ssl-hello check using tcp-check rules
A shared tcp-check ruleset is now created to support ssl-hello check. This way
no extra memory is used if several backends use a ssl-hello check.

The following sequence is used :

    tcp-check send-binary SSLV3_CLIENT_HELLO log-format

    tcp-check expect rbinary "^1[56]" min-recv 5 \
        error-status "L6RSP" tout-status "L6TOUT"

SSLV3_CLIENT_HELLO is a log-format hexa string representing a SSLv3 CLIENT HELLO
packet. It is the same than the one used by the old ssl-hello except the sample
expression "%[date(),htonl,hex]" is used to set the date field.
2020-04-27 09:39:38 +02:00
Christopher Faulet
5d503fcf5b MEDIUM: checks: Add a shared list of tcp-check rules
A global list to tcp-check ruleset can now be used to share common rulesets with
all backends without any duplication. It is mandatory to convert all specific
protocol checks (redis, pgsql...) to tcp-check healthchecks.

To do so, a flag is now attached to each tcp-check ruleset to know if it is a
shared ruleset or not. tcp-check rules defined in a backend are still directly
attached to the proxy and not shared. In addition a second flag is used to know
if the ruleset is inherited from the defaults section.
2020-04-27 09:39:37 +02:00
Gaetan Rivet
04578dbf37 MINOR: checks: Don't use a static tcp rule list head
To allow reusing these blocks without consuming more memory, their list
should be static and share-able accross uses. The head of the list will
be shared as well.

It is thus necessary to extract the head of the rule list from the proxy
itself. Transform it into a pointer instead, that can be easily set to
an external dynamically allocated head.
2020-04-27 09:39:37 +02:00
Christopher Faulet
8892e5d30b BUG/MEDIUM: server/checks: Init server check during config validity check
The options and directives related to the configuration of checks in a backend
may be defined after the servers declarations. So, initialization of the check
of each server must not be performed during configuration parsing, because some
info may be missing. Instead, it must be done during the configuration validity
check.

Thus, callback functions are registered to be called for each server after the
config validity check, one for the server check and another one for the server
agent-check. In addition deinit callback functions are also registered to
release these checks.

This patch should be backported as far as 1.7. But per-server post_check
callback functions are only supported since the 2.1. And the initcall mechanism
does not exist before the 1.9. Finally, in 1.7, the code is totally
different. So the backport will be harder on older versions.
2020-04-27 09:39:37 +02:00
Christopher Faulet
f61f33a1b2 BUG/MINOR: checks: Respect the no-check-ssl option
This options is used to force a non-SSL connection to check a SSL server or to
invert a check-ssl option inherited from the default section. The use_ssl field
in the check structure is used to know if a SSL connection must be used
(use_ssl=1) or not (use_ssl=0). The server configuration is used by default.

The problem is that we cannot distinguish the default case (no specific SSL
check option) and the case of an explicit non-SSL check. In both, use_ssl is set
to 0. So the server configuration is always used. For a SSL server, when
no-check-ssl option is set, the check is still performed using a SSL
configuration.

To fix the bug, instead of a boolean value (0=TCP, 1=SSL), we use a ternary value :

  * 0  = use server config
  * 1  = force SSL
  * -1 = force non-SSL

The same is done for the server parameter. It is not really necessary for
now. But it is a good way to know is the server no-ssl option is set.

In addition, the PR_O_TCPCHK_SSL proxy option is no longer used to set use_ssl
to 1 for a check. Instead the flag is directly tested to prepare or destroy the
server SSL context.

This patch should be backported as far as 1.8.
2020-04-27 09:39:37 +02:00
Frédéric Lécaille
8ba10fea69 BUG/MINOR: peers: Incomplete peers sections should be validated.
Before supporting "server" line in "peers" section, such sections without
any local peer were removed from the configuration to get it validated.

This patch fixes the issue where a "server" line without address and port which
is a remote peer without address and port makes the configuration parsing fail.
When encoutering such cases we now ignore such lines remove them from the
configuration.

Thank you to Jérôme Magnin for having reported this bug.

Must be backported to 2.1 and 2.0.
2020-04-15 10:47:39 +02:00
Ilya Shipitsin
6fb0f2148f CLEANUP: assorted typo fixes in the code and comments
This is sixth iteration of typo fixes
2020-04-02 16:25:45 +02:00