As reported by Maik Broemme, if something different from "if" or
"unless" was specified after "tcp-request content accept", the
condition would silently remain void. The parser must obviously
complain since this typically corresponds to a forgotten "if".
(cherry picked from commit 606ad73e73600275aae944f00bda4af9976c0be8)
As reported by Jean-Baptiste Quenot and Robbie Aelter, sometimes a
backend server error is converted to a 502 error if the backend stops
before reading all the request. The reason is that the remote system
sends a TCP RST packet because there are still unread data pending in
the socket buffer. This RST is translated as a socket error on the
local system, and this error is reported by the poller.
However, most of the time, it's a write error, but the system is
still able to read the remaining pending data, such as in the trace
below :
send(7, "GET /aaa HTTP/1.0\r\nUser-Agent: Mo"..., 1123, MSG_DONTWAIT|MSG_NOSIGNAL) = 1123
epoll_ctl(3, EPOLL_CTL_ADD, 7, {EPOLLIN, {u32=7, u64=7}}) = 0
epoll_wait(3, {{EPOLLIN|EPOLLERR|EPOLLHUP, {u32=7, u64=7}}}, 8, 1000) = 1
gettimeofday({1247593958, 643572}, NULL) = 0
recv(7, "HTTP/1.0 400 Bad request\r\nCache-C"..., 7000, MSG_NOSIGNAL) = 187
setsockopt(6, SOL_TCP, TCP_NODELAY, [0], 4) = 0
setsockopt(6, SOL_TCP, TCP_CORK, [1], 4) = 0
send(6, "HTTP/1.0 400 Bad request\r\nCache-C"..., 187, MSG_DONTWAIT|MSG_NOSIGNAL) = 187
shutdown(6, 1 /* send */) = 0
The recv succeeded while epoll_wait() reported an error.
Note: This case is very hard to reproduce and requires that the backend
server is reached via the loopback in order to minimise latency and
reduce the risk of sent data being ACKed.
(cherry picked from commit 7154365cc60b124b543db4e98faedc75c0f3a2cb)
When we close a socket with unread data in the buffer, or when the
nolinger option is set, we regularly lose the last fragment, which
often contains the error message. This typically occurs when sending
too large a request. Only the RST is seen due to the close() (since
not all data were read) and the output message never reaches the
network.
Doing a shutdown() before the close() solves this annoying issue
because the data are really pushed before the system sends the RST.
(cherry picked from commit 720058cdcbd5285dc4e4a48216b10c9b96000141)
By default, when building from a git tree, haproxy's release date is
set to the last commit's date. But it was the wrong date which was
used, the initial patch's date, which can cause time jumps in the
past when an old patch gets merged. What we want is the commit date,
which reflects the correct code history.
(cherry picked from commit 446024e7fb5faef86cd6e2c0aba3c4524ad77705)
This bug caused TCP proxies not to report incorrect use of some
aliases of HTTP ACLs.
(cherry picked from commit a55b7dc52877ee81609db688f327e7e9586498f0)
redirect rules are documented as being processed last before
use_backend but were mistakenly processed before block rules.
Fortunately very few people use a mix of block and redirect
rules, so this bug has never been reported yet.
(cherry picked from commit 06b917c7abcd7313263d551eaecda1b31b9c03b1)
We now support up to 10 distinct configuration files. They are
all loaded in the order defined by -f <file1> -f <file2> ...
This can be useful in order to store global, private, public,
etc... configurations in distinct files.
(cherry picked from commit 5d01a63b7862235fdd3119cb29d5a0cfd04edb91)
This is a first step towards support of multiple configuration files.
Now readcfgfile() only reads a file in memory and performs very minimal
parsing. The checks are performed afterwards.
(cherry picked from commit 915e1ebe63b2137fa1634ebc9553f5b73ae2fd75)
The only way to set this previously was to set SYSTEM_MAXCONN
which serves a different purpose.
(cherry picked from commit c9fe4562c24ebacdfdf55631636e5a5a0395e43c)
Sometimes it is useful to be able to set a server's weight to zero.
It allows the server to receive only persistent traffic but never
normal traffic.
(cherry picked from commit 6704d67d656574a602ddf81a603cdb4f482f90a9)
After considering various possibilities, we compiled haproxy under cygwin.
Attached is an updated full diff that also has the TARGET=cygwin documented.
The whole thing compiles and installs with this diff only.
In cygwin 1.7 (now in beta), there is apparently support for ipv6. Cygwin
1.5 (later versions, anyway) already includes some support in the form of a
define USE_IPV6. When defined, it declares the sockaddr_in6 struct and
possibly other things. The above definition AF_INET6=23 is taken from
their /usr/include/socket.h file (where it is #if 0'd out).
We are running into a socket limit. It appears that Cygwin (running on
Windows 2003 Server) will only allow us to set ulimit -n (maximum open
files) to 3200, which means we're a little short of 1600 connections.
The limit of 3200 is an internal Cygwin limit. Perhaps they can raise it in
the future. Using the nbproc option, I was able to bring up 10 servers. It
seems to me that they were able to handle over 2000 connections (even though
each had maxconn 1500 set, and the hard Cygwin fd limit).
(cherry picked from commit 32087312e3a4ad483440d371b0b1769db23946d3)
Some users are already hitting the 64k source port limit when
connecting to servers. The system usually maintains a list of
unused source ports, regardless of the source IP they're bound
to. So in order to go beyond the 64k concurrent connections, we
have to manage the source ip:port lists ourselves.
The solution consists in assigning a source port range to each
server and use a free port in that range when connecting to that
server, either for a proxied connection or for a health check.
The port must then be put back into the server's range when the
connection is closed.
This mechanism is used only when a port range is specified on
a server. It makes it possible to reach 64k connections per
server, possibly all from the same IP address. Right now it
should be more than enough even for huge deployments.
(cherry picked from commit c6f4ce8fc4a9da9f4c31e8d088fab1ed4f631ed0)
When a new process fails to grab some ports, it sends a signal to
the old process in order to release them. Then it tries to bind
again. If it still fails (eg: one of the ports is bound to a
completely different process), it must send the continue signal
to the old process so that this one re-binds to the ports. This
is correctly done, but the newly bound ports are not released
first, which sometimes causes the old process to remain running
with no port bound. The fix simply consists in unbinding all
ports before sending the signal to the old process.
(cherry picked from commit f68da4603a092f35af627c459dbc714d9fa796e9)
It is recommended to have -D in init scripts, but -D also implies
quiet mode, which hides warning messages, and both options are now
completely unrelated. Remove the implication to get warnings with
-D.
Released version 1.3.18 with the following main changes :
- [MEDIUM] add support for "balance hdr(name)"
- [CLEANUP] give a little bit more information in error message
- [MINOR] add X-Original-To: header
- [BUG] x-original-to: fix missing initialization to default value
- [BUILD] spec file: fix broken pipe during rpmbuild and add man file
- [MINOR] improve reporting of misplaced acl/reqxxx rules
- [MEDIUM] http: add options to ignore invalid header names
- [MEDIUM] http: capture invalid requests/responses even if accepted
- [BUILD] add format(printf) to printf-like functions
- [MINOR] fix several printf formats and missing arguments
- [BUG] stats: total and lbtot are unsigned
- [MINOR] fix a few remaining printf-like formats on 64-bit platforms
- [CLEANUP] remove unused make option from haproxy.spec
- [BUILD] make it possible to pass alternative arch at build time
- [MINOR] switch all stat counters to 64-bit
- [MEDIUM] ensure we don't recursively call pool_gc2()
- [CRITICAL] uninitialized response field can sometimes cause crashes
- [BUG] fix wrong pointer arithmetics in HTTP message captures
- [MINOR] rhel init script : support the reload operation
- [MINOR] add basic signal handling functions
- [BUILD] add signal.o to all makefiles
- [MEDIUM] call signal_process_queue from run_poll_loop
- [MEDIUM] pollers: don't wait if a signal is pending
- [MEDIUM] convert all signals to asynchronous signals
- [BUG] O(1) pollers should check their FD before closing it
- [MINOR] don't close stdio fds twice
- [MINOR] add options dontlog-normal and log-separate-errors
- [DOC] minor fixes and rearrangements
- [BUG] fix parser crash on unconditional tcp content rules
- [DOC] rearrange the configuration manual and add a summary
- [MINOR] standard: provide a new 'my_strndup' function
- [MINOR] implement per-logger log level limitation
- [MINOR] compute the max of sessions/s on fe/be/srv
- [MINOR] stats: report max sessions/s and limit in CSV export
- [MINOR] stats: report max sessions/s and limit in HTML stats
- [MINOR] stats/html: use the arial font before helvetica
The stats HTML output were barely readable on some browsers such as
firefox on Linux, due to the selected helvetica font which is too
small. Specifying "arial" first fixes the issue without changing the
table size. Also, the default size of 0.8em choosen to get 10px out
of 12px is wrong because it gets 9px when rounded down.
Some users want to keep the max sessions/s seen on servers, frontends
and backends for capacity planning. It's easy to grab it while the
session count is updated, so let's keep it.
Some people are using haproxy in a shared environment where the
system logger by default sends alert and emerg messages to all
consoles, which happens when all servers go down on a backend for
instance. These people can not always change the system configuration
and would like to limit the outgoing messages level in order not to
disturb the local users.
The addition of an optional 4th field on the "log" line permits
exactly this. The minimal log level ensures that all outgoing logs
will have at least this level. So the logs are not filtered out,
just set to this level.
There is a patch made by me that allow for balancing on any http header
field.
[WT:
made minor changes:
- turned 'balance header name' into 'balance hdr(name)' to match more
closely the ACL syntax for easier future convergence
- renamed the proxy structure fields header_* => hh_*
- made it possible to use the domain name reduction to any header, not
only "host" since it makes sense to do it with other ones.
Otherwise patch looks good.
/WT]
Several people have asked for a summary in order to ease finding
of sections in the configuration manual. It was the opportunity to
tidy it up a bit and rearrange some sections.
Since 1.3.17, a config containing one of the following lines would
crash the parser :
tcp content reject
tcp content accept
This is because a check is performed on the condition which is not
specified. The obvious fix consists in checkinf for a condition
first.
Some big traffic sites have trouble dealing with logs and tend to
disable them. Here are two new options to help cope with massive
logs.
- dontlog-normal only disables logging for 100% successful
connections, other ones will still be logged
- log-separate-errors will cause non-100% successful connections
to be logged at level "err" instead of level "info" so that a
properly configured syslog daemon can send them to a different
file for longer conservation.
epoll, sepoll and kqueue pollers should check that their fd is not
closed before attempting to close it, otherwise we can end up with
multiple closes of fd #0 upon exit, which is harmless but dirty.
The small list of signals currently handled by haproxy were processed
as soon as they were received. This has caused trouble with calls to
pool_gc2() occuring in the middle of libc's memory management functions
seldom causing deadlocks preventing the old process from leaving.
Now these signals use the new async signal framework and are called
asynchronously, when there is no risk of recursion. This ensures more
reliable operation, especially for sensible processing such as memory
management.
If an asynchronous signal is received outside of the poller, we don't
want the poller to wait for a timeout to occur before processing it,
so we set its timeout to zero, just like we do with pending tasks in
the run queue.
These functions will be used to deliver asynchronous signals in order
to make the signal handling functions more robust. The goal is to keep
the same interface to signal handlers.
I have attached a patch which will add on every http request a new
header 'X-Original-To'. If you have HAProxy running in transparent mode
with a big number of SQUID servers behind it, it is very nice to have
the original destination ip as a common header to make decisions based
on it.
The whole thing is configurable with a new option 'originalto'. I have
updated the sourcecode as well as the documentation. The 'haproxy-en.txt'
and 'haproxy-fr.txt' files are untouched, due to lack of my french
language knowledge. ;)
Also the patch adds this header for IPv4 only. I haven't any IPv6 test
environment running here and don't know if getsockopt() with SO_ORIGINAL_DST
will work on IPv6. If someone knows it and wants to test it I can modify
the diff. Feel free to ask me questions or things which should be changed. :)
--Maik
The pointer arithmetics was wrong in http_capture_bad_message().
This has no impact right now because the error only msg->som was
affected and right now it's always 0. But this was a bug waiting
for keepalive support to strike.
The response message in the transaction structure was not properly
initialised at session initialisation. In theory it cannot cause any
trouble since the affected field os expected to always remain NULL.
However, in some circumstances, such as building on 64-bit platforms
with certain options, the struct session can be exactly 1024 bytes,
the same size of the requri field, so the pools are merged and the
uninitialised field may contain non-null data, causing crashes if
an invalid response is encountered and archived.
The fix simply consists in correctly initialising the missing fields.
This bug cannot affect architectures where the session pool is not
shared (32-bit architectures), but this is only by pure luck.
A race condition exists in the hot reconfiguration code. It is
theorically possible that the second signal is sent during a free()
in the first list, which can cause crashes or freezes (the later
have been observed). Just set up a counter to ensure we do not
recurse.
The byte counters have long been 64-bit to avoid overflows. But with
several sites nowadays, we see session counters wrap around every 10-days
or so. So it was the moment to switch counters to 64-bit, including
error and warning counters which can theorically rise as fast as session
counters even if in practice there is very low risk.
The performance impact should not be noticeable since those counters are
only updated once per session. The stats output have been carefully checked
for proper types on both 32- and 64-bit platforms.
When trying to build a 32-bit binary on a 64-bit platform, we generally
need to pass "-m32" to gcc, which is not convenient with current makefile.
Note that this option requires gcc >= 3.
In order to ease parameter passing, a new ARCH= makefile option has been
added. If it receives a target architecture, according "-m32"/"-m64" and
"-march=xxxx" will be passed to gcc. Only the generic makefile has been
changed to support this option right now as the need only appeared on Linux.
The spec file now makes use of this option so that rpmbuild can automatically
build with the proper architecture.
It's useful to be able to accept an invalid header name in a request
or response but still be able to monitor further such errors. Now,
when an invalid request/response is received and accepted due to
an "accept-invalid-http-{request|response}" option, the invalid
request will be captured for later analysis with "show errors" on
the stats socket.
