10520 Commits

Author SHA1 Message Date
Willy Tarreau
ae94d4df8f [MINOR] http: make the "HTTP 200" status code configurable.
This status code is used in response to requests matching "monitor-uri".
Some users need to adjust it to fit their needs (eg: make some strings
appear there). As it's already defined as a chunked string and used
exactly like other status codes, it makes sense to make it configurable
with the usual "errorfile", "errorloc", ...
2011-05-11 16:31:43 +02:00
Willy Tarreau
436d9ed808 [REORG] http: move HTTP error codes back to proto_http.h
This one was left isolated in its own file. It probably is a leftover
from the 1.2->1.3 split.
2011-05-11 16:31:43 +02:00
Willy Tarreau
027a85bb03 [MINOR] http: don't report the "haproxy" word on the monitoring response
Some people like to make the monitoring URL testable from unsafe locations.
Reporting haproxy's existence there can sometimes be problematic. This patch
should not be backported to 1.4 because it is possible, eventhough unlikely,
that some scripts rely on this word to appear there.
2011-05-11 16:31:43 +02:00
Cyril Bonté
7c51a732f7 [BUG] fix binary stick-tables
As reported by Lauri-Alo Adamson, version 1.5-dev6 doesn't support
stick-tables with a binary type.
This issue was introduced in the commit 4f92d32 where a line was erroneously
deleted, and is 1.5-specific.
2011-05-09 23:30:58 +02:00
Willy Tarreau
96dd079b49 [BUG] proto_tcp: fix address binding on remote source
Mark Brooks reported that commit 1b4b7c broke tproxy in 1.5-dev6. Nick
Chalk tracked the issue down to a missing address family setting in
tcp_bind_socket() which resulted in a failure to use get_addr_len().
This issue is 1.5-specific.
2011-04-19 07:20:57 +02:00
Willy Tarreau
a164fb5721 [BUG] checks: http-check expect could fail a check on multi-packet responses
Christopher Blencowe reported that the httpchk_expect() function was
lacking a test for incomplete responses : if the server sends only the
headers in the first packet and the body in a subsequent one, there is
a risk that the check fails without waiting for more data. A failure
rate of about 1% was reported.

This fix must be backported to 1.4.
2011-04-13 09:32:41 +02:00
Willy Tarreau
1fc1f45618 [CRITICAL] fix risk of crash when dealing with space in response cookies
When doing fix 24581bae022bcf97ea7818e49ef27d21c92d6aa3 to correctly handle
response cookies, an unfortunate typo was inserted in the less likely code
path, resulting in a risk of crash when cookie-based persistence is enabled
and the server emits a cookie with several spaces around the equal sign.

This bug was noticed during a code backport. Its effects were never reported
because this situation is very unlikely to appear, but it can be provoked on
purpose by the server.

This patch must be backported to 1.4 versions which contain the fix above
(anything > 1.4.8), and to similar 1.3 versions > 1.3.25. 1.5-dev versions
after 1.5-dev2 are affected too.
2011-04-08 00:50:36 +02:00
Willy Tarreau
442452034e [BUG] stick-tables did not work when converting IPv6 to IPv4
A stick-table of type IPv6 would store a wrong IPv4 address as the
result of an IPv6 to IPv4 conversion. This bug was introduced in
1.5-dev5.
2011-04-07 10:53:30 +02:00
Willy Tarreau
1b4b7ce6dd [BUG] stream_sock: use get_addr_len() instead of sizeof() on sockaddr_storage
John Helliwell reported a runtime issue on Solaris since 1.5-dev5. Traces
show that connect() returns EINVAL, which means the socket length is not
appropriate for the family. Solaris does not like being called with sizeof
and needs the address family's size on sockaddr_storage.

The fix consists in adding a get_addr_len() function which returns the
socket's address length based on its family. Tests show that this works
for both IPv4 and IPv6 addresses.
2011-04-05 16:56:50 +02:00
David du Colombier
4f92d32004 [MEDIUM] IPv6 support for stick-tables
Since IPv6 is a different type than IPv4, the pattern fetch functions
src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses
with src and dst. In this case, the IPv4 addresses are mapped to their
IPv6 counterpart, according to RFC 4291.
2011-03-29 01:09:14 +02:00
Willy Tarreau
c735a0728e [MINOR] acl: add support for table_cnt and table_avl matches
Those trivial matches respectively return the number of entries used
in a stick-table and the number of entries still available in a table.
2011-03-29 00:57:02 +02:00
Willy Tarreau
68f49da972 [BUG] stream_sock: fix handling for server side PROXY protocol
Patch 5ab04ec47c9946a2bbc535687c023215ca813da0 was incomplete,
because if the first send() fails on an empty buffer, we fail
to rearm the polling and we can't establish the connection
anymore.

The issue was reported by Ben Timby who provided large amounts
of traces of various tests helping to reliably reproduce the issue.
2011-03-28 23:17:54 +02:00
David du Colombier
11bcb6c4f5 [MEDIUM] IPv6 support for syslog 2011-03-28 18:45:15 +02:00
Willy Tarreau
0bc3493d2c [OPTIM] buffers: uninline buffer_forward()
Since the latest additions to buffer_forward(), it became too large for
inlining, so let's uninline it. The code size drops by 3kB. Should be
backported to 1.4 too.
2011-03-28 16:25:58 +02:00
Willy Tarreau
d8ee85a0a3 [BUG] http: fix content-length handling on 32-bit platforms
Despite much care around handling the content-length as a 64-bit integer,
forwarding was broken on 32-bit platforms due to the 32-bit nature of
the ->to_forward member of the "buffer" struct. The issue is that this
member is declared as a long, so while it works OK on 64-bit platforms,
32-bit truncate the content-length to the lower 32-bits.

One solution could consist in turning to_forward to a long long, but it
is used a lot in the critical path, so it's not acceptable to perform
all buffer size computations on 64-bit there.

The fix consists in changing the to_forward member to a strict 32-bit
integer and ensure in buffer_forward() that only the amount of bytes
that can fit into it is considered. Callers of buffer_forward() are
responsible for checking that their data were taken into account. We
arbitrarily ensure we never consider more than 2G at once.

That's the way it was intended to work on 32-bit platforms except that
it did not.

This issue was tracked down hard at Exosec with Bertrand Jacquin,
Thierry Fournier and Julien Thomas. It remained undetected for a long
time because files larger than 4G are almost always transferred in
chunked-encoded format, and most platforms dealing with huge contents
these days run on 64-bit.

The bug affects all 1.5 and 1.4 versions, and must be backported.
2011-03-28 16:25:16 +02:00
Willy Tarreau
26f0f17200 [BUG] http: fix possible incorrect forwarded wrapping chunk size (take 2)
Fix acd20f80 was incomplete, the computed "bytes" value was not used.

This fix must be backported to 1.4.
2011-03-27 20:00:03 +02:00
Willy Tarreau
7b7a8e9d83 [BUG] log: retrieve the target from the session, not the SI
Since we now have the copy of the target in the session, use it instead
of relying on the SI for it. The SI drops the target upon unregister()
so applets such as stats were logged as "NOSRV".
2011-03-27 19:53:06 +02:00
Willy Tarreau
0b3a411543 [BUG] session: conn_retries was not always initialized
Johannes Smith reported some wrong retries count in logs associated with bad
requests. The cause was that the conn_retries field in the stream interface
was only initialized when attempting to connect, but is used when logging,
possibly with an uninitialized value holding last connection's conn_retries.
This could have been avoided by making use of a stream interface initializer.

This bug is 1.5-specific.
2011-03-27 19:16:56 +02:00
David du Colombier
d5f4328efd [MEDIUM] use getaddrinfo to resolve names if gethostbyname fail
Function gethostbyname is deprecated since IEEE Std 1003.1-2008 and
was replaced by getaddrinfo (available since IEEE Std 1003.1-2004).
Contrary to gethostbyname, getaddrinfo is specified to support both
IPv4 and IPv4 addresses.
Since some libc doesn't handle getaddrinfo properly, constant
USE_GETADDRINFO must be defined at compile time to enable use of
getaddrinfo.
2011-03-23 22:49:55 +01:00
Willy Tarreau
2dff0c28e8 [MINOR] cfgparse: better report wrong listening addresses and make use of str2sa_range
It's always been a mess to debug wrong listening addresses because
the parsing function does not indicate the file and line number. Now
it does. Since the code was almost a duplicate of str2sa_range, it
now makes use of it and has been sensibly reduced.
2011-03-23 22:49:55 +01:00
David du Colombier
9842ff1ae6 [MINOR] update comment about IPv6 support for server 2011-03-23 22:49:55 +01:00
Willy Tarreau
fab5a43726 [MEDIUM] config: rework the IPv4/IPv6 address parser to support host-only addresses
The parser now distinguishes between pure addresses and address:port. This
is useful for some config items where only an address is required.

Raw IPv6 addresses are now parsed, but IPv6 host name resolution is still not
handled (gethostbyname does not resolve IPv6 names to addresses).
2011-03-23 19:01:18 +01:00
Willy Tarreau
6f831b446c [BUILD] proto_tcp: fix build issue with CTTPROXY
Recent sockaddr_storage changes broke the almost unused cttproxy code.
Fix is obvious.
2011-03-20 14:03:54 +01:00
Willy Tarreau
5ab04ec47c [MEDIUM] server: add support for the "send-proxy" option
This option enables use of the PROXY protocol with the server, which
allows haproxy to transport original client's address across multiple
architecture layers.
2011-03-20 11:53:50 +01:00
Willy Tarreau
b22e55bc8f [MEDIUM] stream_sock: add support for sending the proxy protocol header line
Upon connection establishment, stream_sock is now able to send a PROXY
line before sending any data. Since it's possible that the buffer is
already full, and we don't want to allocate a block for that line, we
compute it on-the-fly when we need it. We just store the offset from
which to (re-)send from the end of the line, since it's assumed that
multiple outputs of the same proxy line will be strictly equivalent. In
practice, one call is enough. We just make sure to handle the case where
the first send() would indicate an incomplete output, eventhough it's
very unlikely to ever happen.
2011-03-20 10:16:46 +01:00
Willy Tarreau
a73fcaf424 [MINOR] frontend: add a make_proxy_line function
This function will build a PROXY protocol line header from two addresses
(IPv4 or IPv6). AF_UNIX family will be reported as UNKNOWN.
2011-03-20 10:15:22 +01:00
Willy Tarreau
1b6e608c11 [BUG] session: src_conn_cur was returning src_conn_cnt instead
Issue reported by Cory Forsyth and diagnosed by Cyril Bonté.
Just a plain stupid copy-paste of the wrong fetch function call.
2011-03-16 06:56:57 +01:00
Willy Tarreau
d11ad78c26 [MINOR] checks: report it if checks fail due to socket creation error
If the check fails for a low-level socket error (eg: address family not
supportd), we currently ignore the status. We must report the error and
declare a failed health check in this case. The only real reason for this
would be when an IPv6 check is required on an IPv4-only system.
2011-03-13 22:12:54 +01:00
Willy Tarreau
6da0f6d6dd [BUG] http: stats were not incremented on http-request deny
A counter increase was missing here.
This should be backported to 1.4 with care, as the code has changed a bit.
2011-03-13 22:00:24 +01:00
Willy Tarreau
ff011f26e9 [REORG] http: move the http-request rules to proto_http
And also rename "req_acl_rule" "http_req_rule". At the beginning that
was a bit confusing to me, especially the "req_acl" list which in fact
holds what we call rules. After some digging, it appeared that some
part of the code is 100% HTTP and not just related to authentication
anymore, so let's move that part to HTTP and keep the auth-only code
in auth.c.
2011-03-13 22:00:24 +01:00
Willy Tarreau
f68a15a951 [MEDIUM] http: always evaluate http-request rules before stats http-request
Right now, http-request rules are not evaluated if the URL matches the
stats request. This is quite unexpected. For instance, in the config
below, an abuser present in the abusers list will not be prevented access
to the stats.

   listen pub
        bind :8181
	acl abuser src -f abusers.lst
        http-request deny if abuser
        stats uri /stats

It is not a big deal but it's not documented as such either. For 1.5, let's
have both lists be evaluated in turn, until one blocks. For 1.4 we'll simply
update the doc to indicate that.

Also instead of duplicating the code, the patch factors out the list walking
code. The HTTP auth has been moved slightly earlier, because it was set after
the header addition code, but we don't need to add headers to a request we're
dropping.
2011-03-13 22:00:24 +01:00
Willy Tarreau
7d0aaf39d1 [MEDIUM] stats: split frontend and backend stats
It's very annoying that frontend and backend stats are merged because we
don't know what we're observing. For instance, if a "listen" instance
makes use of a distinct backend, it's impossible to know what the bytes_out
means.

Some points take care of not updating counters twice if the backend points
to the frontend, indicating a "listen" instance. The thing becomes more
complex when we try to add support for server side keep-alive, because we
have to maintain a pointer to the backend used for last request, and to
update its stats. But we can't perform such comparisons anymore because
the counters will not match anymore.

So in order to get rid of this situation, let's have both frontend AND
backend stats in the "struct proxy". We simply update the relevant ones
during activity. Some of them are only accounted for in the backend,
while others are just for frontend. Maybe we can improve a bit on that
later, but the essential part is that those counters now reflect what
they really mean.
2011-03-13 22:00:23 +01:00
David du Colombier
6f5ccb1589 [MEDIUM] add internal support for IPv6 server addresses
This patch turns internal server addresses to sockaddr_storage to
store IPv6 addresses, and makes the connect() function use it. This
code already works but some caveats with getaddrinfo/gethostbyname
still need to be sorted out while the changes had to be merged at
this stage of internal architecture changes. So for now the config
parser will not emit an IPv6 address yet so that user experience
remains unchanged.

This change should have absolutely zero user-visible effect, otherwise
it's a bug introduced during the merge, that should be reported ASAP.
2011-03-13 22:00:12 +01:00
Willy Tarreau
827aee913f [MAJOR] session: remove the ->srv pointer from struct session
This one has been removed and is now totally superseded by ->target.
To get the server, one must use target_srv(&s->target) instead of
s->srv now.

The function ensures that non-server targets still return NULL.
2011-03-10 23:32:17 +01:00
Willy Tarreau
9e000c6ec8 [CLEANUP] stream_interface: use inline functions to manipulate targets
The connection target involves a type and a union of pointers, let's
make the code cleaner using simple wrappers.
2011-03-10 23:32:17 +01:00
Willy Tarreau
3d80d911aa [MEDIUM] session: remove s->prev_srv which is not needed anymore
s->prev_srv is used by assign_server() only, but all code paths leading
to it now take s->prev_srv from the existing s->srv. So assign_server()
can do that copy into its own stack.

If at one point a different srv is needed, we still have a copy of the
last server on which we failed a connection attempt in s->target.
2011-03-10 23:32:16 +01:00
Willy Tarreau
664beb8610 [MINOR] session: add a pointer to the new target into the session
When dealing with HTTP keep-alive, we'll have to know if we can reuse
an existing connection. For that, we'll have to check if the current
connection was made on the exact same target (referenced in the stream
interface).

Thus, we need to first assign the next target to the session, then
copy it to the stream interface upon connect(). Later we'll check for
equivalence between those two operations.
2011-03-10 23:32:16 +01:00
Willy Tarreau
d6cc532ca1 [MINOR] cfgparse: only keep one of dispatch, transparent, http_proxy
Since all of them are defined as proxy options, it's better to ensure
that at most one of them is enabled at once. The priority has been set
according to what is already performed in the backend :

   1) dispatch
   2) http_proxy
   3) transparent
2011-03-10 23:32:16 +01:00
Willy Tarreau
f5ab69aad9 [MINOR] proxy: add PR_O2_DISPATCH to detect dispatch mode
Till now we used the fact that the dispatch address was not null to use
the dispatch mode. This is very unconvenient, so let's have a dedicated
option.
2011-03-10 23:32:16 +01:00
Willy Tarreau
295a837726 [REORG] session: move the data_ctx struct to the stream interface's applet
This is in fact where those parts belong to. The old data_state was replaced
by applet.state and is now initialized when the applet is registered. It's
worth noting that the applet does not need to know the session nor the
buffer anymore since everything is brought by the stream interface.

It is possible that having a separate applet struct would simplify the
code but that's not a big deal.
2011-03-10 23:32:16 +01:00
Willy Tarreau
5ec29ffa42 [CLEANUP] stats: make all dump functions only rely on the stream interface
This will be needed to move the applet-specific data out of the session.
2011-03-10 23:32:16 +01:00
Willy Tarreau
75581aebb0 [CLEANUP] session: remove data_source from struct session
This one was only used for logging purposes, it's not needed
anymore.
2011-03-10 23:32:15 +01:00
Willy Tarreau
71904a4ee8 [MEDIUM] log: take the logged server name from the stream interface
With HTTP keep-alive, logging the right server name will be quite
complex because the assigned server will possibly change before we log.
Also, when we want to log accesses to an applet, it's not easy because
the applet becomes NULL again before logging.

The logged server's name is now taken from the target stored in the
stream interface. That way we can log an applet, a server name, or we
could even log a proxy or anything else if we wanted to. Ideally the
session should contain a desired target which is the one which should
be logged.
2011-03-10 23:32:15 +01:00
Willy Tarreau
7c0a151a2e [CLEANUP] stream_interface: remove the applet.handler pointer
Now that we have the target pointer and type in the stream interface,
we don't need the applet.handler pointer anymore. That makes the code
somewhat cleaner because we know we're dealing with an applet by checking
its type instead of checking the pointer is not null.
2011-03-10 23:32:15 +01:00
Willy Tarreau
ac82540c35 [MEDIUM] stream_interface: store the target pointer and type
When doing a connect() on a stream interface, some information is needed
from the server and from the backend. In some situations, we don't have
a server and only a backend (eg: peers). In other cases, we know we have
an applet and we don't want to connect to anything, but we'd still like
to have the info about the applet being used.

For this, we now store a pointer to the "target" into the stream interface.
The target describes what's on the other side before trying to connect. It
can be a server, a proxy or an applet for now. Later we'll probably have
descriptors for multiple-stage chains so that the final information may
still be found.

This will help removing many specific cases in the code. It already made
it possible to remove the "srv" and "be" parameters to tcpv4_connect_server().
2011-03-10 23:32:15 +01:00
Willy Tarreau
f153686a71 [REORG] tcp: make tcpv4_connect_server() take the target address from the SI
The address is now available in the stream interface, no need to pass it by
argument.
2011-03-10 23:32:15 +01:00
Willy Tarreau
957c0a5845 [REORG] session: move client and server address to the stream interface
This will be needed very soon for the keep-alive.
2011-03-10 23:32:14 +01:00
Willy Tarreau
bc4af0573c [REORG] stream_interface: move the st0, st1 and private members to the applet
Those fields are only used by the applets, so let's move them to the
struct.
2011-03-10 23:32:14 +01:00
Willy Tarreau
b24281b0ff [MINOR] stream_interface: make use of an applet descriptor for IO handlers
I/O handlers are still delicate to manipulate. They have no type, they're
just raw functions which have no knowledge of themselves. Let's have them
declared as applets once for all. That way we can have multiple applets
share the same handler functions and we can store their names there. When
we later need to add more parameters (eg: usage stats), we'll be able to
do so in the applets themselves.

The CLI functions has been prefixed with "cli" instead of "stats" as it's
clearly what is going on there.

The applet descriptor in the stream interface should get all the applet
specific data (st0, ...) but this will be done in the next patch so that
we don't pollute this one too much.
2011-03-10 23:32:14 +01:00
Willy Tarreau
dfd7fca26c [BUG] config: don't crash on empty pattern files.
Both Hank A. Paulson and Rob at pixsense reported a crash when
loading ACLs from a pattern file which contains empty lines.

From the tests, it appears that only files that contain nothing
but empty lines are causing that (in the past they would have had
their line feeds loaded as patterns).

The crash happens in the free_pattern() call which doesn't like to
be called with a NULL pattern. Let's make it accept it so that it's
more in line with the standard uses of free() which ignores NULLs.
2011-03-09 10:22:30 +01:00