Commit Graph

248 Commits

Author SHA1 Message Date
Willy Tarreau
88922354fb [MINOR] config: add pointer to file name in block/redirect/use_backend/monitor rules
Those conditions already referenced the config line, but not the file.
2009-10-04 22:02:50 +02:00
Willy Tarreau
092549fa85 [MINOR] config: report places of duplicate names or IDs
When a name or ID conflict is detected, it is sometimes useful to know
where the other one was declared. Now that we have this information,
report it in error messages.
2009-10-04 21:16:08 +02:00
Willy Tarreau
90a570f025 [MINOR] config: reference file and line with any listener/proxy/server declaration
Those will be used later for cross-references of conflicts or errors.
2009-10-04 21:14:56 +02:00
Willy Tarreau
9d7e335127 [BUG] config: look for ID conflicts in all sockets, not only last ones.
ID conflicts between 'bind' lines were not detected due to this typo.
2009-10-04 21:13:39 +02:00
Krzysztof Piotr Oledzki
aeebf9ba65 [MEDIUM] Collect & provide separate statistics for sockets, v2
This patch allows to collect & provide separate statistics for each socket.
It can be very useful if you would like to distinguish between traffic
generate by local and remote users or between different types of remote
clients (peerings, domestic, foreign).

Currently no "Session rate" is supported, but adding it should be possible
if we found it useful.
2009-10-04 18:56:02 +02:00
Willy Tarreau
f27b5ea8dc [MEDIUM] new option "independant-streams" to stop updating read timeout on writes
By default, when data is sent over a socket, both the write timeout and the
read timeout for that socket are refreshed, because we consider that there is
activity on that socket, and we have no other means of guessing if we should
receive data or not.

While this default behaviour is desirable for almost all applications, there
exists a situation where it is desirable to disable it, and only refresh the
read timeout if there are incoming data. This happens on sessions with large
timeouts and low amounts of exchanged data such as telnet session. If the
server suddenly disappears, the output data accumulates in the system's
socket buffers, both timeouts are correctly refreshed, and there is no way
to know the server does not receive them, so we don't timeout. However, when
the underlying protocol always echoes sent data, it would be enough by itself
to detect the issue using the read timeout. Note that this problem does not
happen with more verbose protocols because data won't accumulate long in the
socket buffers.

When this option is set on the frontend, it will disable read timeout updates
on data sent to the client. There probably is little use of this case. When
the option is set on the backend, it will disable read timeout updates on
data sent to the server. Doing so will typically break large HTTP posts from
slow lines, so use it with caution.
2009-10-03 22:01:18 +02:00
Willy Tarreau
9757a38feb [MEDIUM] backend: introduce the "static-rr" LB algorithm
The "static-rr" is just the old round-robin algorithm. It is still
in use when a hash algorithm is used and the data to hash is not
present, but it was impossible to configure it explicitly. This one
is cheaper in terms of CPU and supports unlimited numbers of servers,
so it makes sense to be able to use it.
2009-10-03 18:41:19 +02:00
Willy Tarreau
f3e49f9521 [MINOR] backend: separate declarations of LB algos from their lookup method
LB algo macros were composed of the LB algo by itself without any indication
of the method to use to look up a server (the lb function itself). This
method was implied by the LB algo, which was not very convenient to add
more algorithms. Now we have several fields in the LB macros, some to
describe what to look for in the requests, some to describe how to transform
that (kind of algo) and some to describe what lookup function to use.

The next patch will make it possible to factor out some code for all algos
which rely on a map.
2009-10-03 18:41:18 +02:00
Krzysztof Piotr Oledzki
619caca740 [MINOR] Allow dots in show-node & add "white-space: nowrap" in th.pxname. 2009-10-03 18:37:06 +02:00
Krzysztof Piotr Oledzki
48cb2aed5a [MINOR] add "description", "node" and show-node"/"show-desc", remove "node-name", v2
This patch implements "description" (proxy and global) and "node" (global)
options, removes "node-name" and adds "show-node" & "show-desc" options
for "stats". It also changes the way the header lines (with proxy name) and
the statistics are displayed, so stats no longer look so clumsy with very
long names.

Instead of "node-name" it is possible to use show-node/show-desc with
an optional parameter that overrides a default node/description.

backend cust-0045
        # report specific values for this customer
        stats show-node Europe
        stats show-desc Master node for Europe, Asia, Africa
2009-10-03 07:10:14 +02:00
Willy Tarreau
f89c1873f8 [CLEANUP] backend: move LB algos to individual files
It was becoming painful to have all the LB algos in backend.c.
Let's move them to their own files. A few hashing functions still
need be broken in two parts, one for the contents and one for the
map position.
2009-10-01 11:19:37 +02:00
Krzysztof Piotr Oledzki
08ff959c3e [BUG] check if rise/fall has an argument and it is > 0
Check if rise/fall has an argument and it is > 0  or bad things may happen
in the health checks. ;)

Now it is verified and the code no longer allows for such condition:

backend bad
	(...)
        server o-f0 192.168.129.27:80 check inter 4000 source 0.0.0.0 rise 0
        server o-r0 192.168.129.27:80 check inter 4000 source 0.0.0.0 fall 0
        server o-f1 192.168.129.27:80 check inter 4000 source 0.0.0.0 rise
        server o-r1 192.168.129.27:80 check inter 4000 source 0.0.0.0 fall

[ALERT] 269/161830 (24136) : parsing [../git/haproxy.cfg:98]: 'rise' has to be > 0.
[ALERT] 269/161830 (24136) : parsing [../git/haproxy.cfg:99]: 'fall' has to be > 0.
[ALERT] 269/161830 (24136) : parsing [../git/haproxy.cfg:100]: 'rise' expects an integer argument.
[ALERT] 269/161830 (24136) : parsing [../git/haproxy.cfg:101]: 'fall' expects an integer argument.

Also add endline in the custom id checking code.
2009-10-01 10:17:38 +02:00
Krzysztof Piotr Oledzki
213014e587 [MEDIUM] Health check reporting code rework + health logging, v3
This patch adds health logging so it possible to check what
was happening before a crash. Failed healt checks are logged if
server is UP and succeeded healt checks if server is DOWN,
so the amount of additional information is limited.

I also reworked the code a little:

 - check_status_description[] and check_status_info[] is now
   joined into check_statuses[]

 - set_server_check_status updates not only s->check_status and
   s->check_duration but also s->result making the code simpler

Changes in v3:
 - for now calculate and use local versions of health/rise/fall/state,
   it is a slow path, no harm should be done. One day we may centralize
   processing of the checks and remove the duplicated code.
 - also log checks that are restoring current state
 - use "conditionally succeeded" for 404 with disable-on-404
2009-10-01 10:17:37 +02:00
Krzysztof Piotr Oledzki
78abe618a8 [MAJOR] struct chunk rework
Add size to struct chunk and simplify the code as there is
no longer required to pass sizeof in chunk_printf().
2009-10-01 10:17:37 +02:00
Krzysztof Piotr Oledzki
0960541e49 [MEDIUM] Collect & show information about last health check, v3
Collect information about last health check result,
including L7 code if possible (for example http or smtp
return code) and time took to finish last check.

Health check info is provided on both stats pages (html & csv)
and logged when a server is marked UP or DOWN. Currently active
check are marked with an asterisk, but only in html mode.

Currently there are 14 status codes:
  UNK     -> unknown

  INI     -> initializing
  SOCKERR -> socket error

  L4OK    -> check passed on layer 4, no upper layers testing enabled
  L4TOUT  -> layer 1-4 timeout
  L4CON   -> layer 1-4 connection problem, for example "Connection refused"
              (tcp rst) or "No route to host" (icmp)

  L6OK    -> check passed on layer 6
  L6TOUT  -> layer 6 (SSL) timeout
  L6RSP   -> layer 6 invalid response - protocol error

  L7OK    -> check passed on layer 7
  L7OKC   -> check conditionally passed on layer 7, for example
               404 with disable-on-404
  L7TOUT  -> layer 7 (HTTP/SMTP) timeout
  L7RSP   -> layer 7 invalid response - protocol error
  L7STS   -> layer 7 response error, for example HTTP 5xx
2009-09-23 23:15:36 +02:00
Willy Tarreau
27a674efb8 [MEDIUM] make it possible to change the buffer size in the configuration
The new tune.bufsize and tune.maxrewrite global directives allow one to
change the buffer size and the maxrewrite size. Right now, setting bufsize
too low will block stats sockets which will not be able to write at all.
An error checking must be added to buffer_write_chunk() so that if it
cannot write its message to an empty buffer, it causes the caller to abort.
2009-08-17 22:56:56 +02:00
Willy Tarreau
4e5b8287a6 [MEDIUM] set rep->analysers from fe and be analysers
sess_establish() used to resort to protocol-specific guesses
in order to set rep->analysers. This is no longer needed as it
gets set from the frontend and the backend as a copy of what
was defined in the configuration.
2009-08-16 22:57:50 +02:00
Willy Tarreau
c1a2167e9d [MINOR] cleanup set_session_backend by using pre-computed analysers
Analyser bitmaps are now stored in the frontend and backend, and
combined at configuration time. That way, set_session_backend()
does not need to perform any protocol-specific combinations.
2009-08-16 22:37:44 +02:00
Willy Tarreau
b55932ddaf [MEDIUM] remove old experimental tcpsplice option
This Linux-specific option was never really used in production and
has since been superseded by new splicing options brought by recent
Linux kernels.

It caused several particular cases in the code because the kernel
would take care of the session without haproxy being able to do
anything on it, which became hard to handle in the new architecture.

Let's simply get rid of it now that there is a replacement available.
2009-08-16 13:20:32 +02:00
Willy Tarreau
1d45b7cbae [MINOR] stats: add a new node-name setting
The new "node-name" stats setting enables reporting of a node ID on
the stats page. It is possible to return the system's host name as
well as a specific name.
2009-08-16 10:29:18 +02:00
Willy Tarreau
e7ded1f869 [BUG] log: option tcplog would log to global if no logger was defined
Romuald du Song reported a strange bug causing "option tcplog" to
unexpectedly use global log parameters if no log server was declared.

Eventhough it can be useful in some circumstances, it only hides
configuration bugs and can even cause traffic logs to be sent to
the wrong logger, since global settings are just for the process.

This has been fixed and a warning has been added for configurations
where tcplog or httplog are set without any logger. This fix must
be backported to 1.3.20, but not to 1.3.15.X in order not to risk
any regression on old configurations.
2009-08-09 10:11:45 +02:00
Willy Tarreau
bb9250104f [MINOR] config: improve error reporting when checking configuration
Do not exit early at the first error found while checking configuration
validity. This particularly helps spotting multiple wrong tracked server
names at once.
2009-07-26 17:30:43 +02:00
Willy Tarreau
3b6b1a984d [MINOR] config: the "capture" keyword is not allowed in backends
The "capture" keyword is only supported by frontends, fix the check.
2009-07-26 17:30:30 +02:00
Willy Tarreau
9389379f60 [MINOR] config: improve error reporting in listen sections
Try not to immediately exit on non-fatal errors while parsing a
listen section, so that the user has a chance to get most of the
errors at once, which is quite convenient especially during config
checks with the -c argument.
2009-07-26 17:29:55 +02:00
Willy Tarreau
058e907486 [MINOR] config: improve error reporting in global section
Try not to immediately exit on non-fatal errors while parsing the
global section, so that the user has a chance to get most of the
errors at once, which is quite convenient especially during config
checks with the -c argument. Some other errors such as unresolved
server names also don't make the parser exit too early.
2009-07-26 17:24:11 +02:00
Emeric Brun
3a058f3091 [MINOR] add a new CLF log format
Appending the "clf" word after "option httplog" turns the HTTP log
format into a CLF format, more suited for certain tools.
2009-07-14 12:50:40 +02:00
Emeric Brun
647caf1ebc [MEDIUM] add support for RDP cookie persistence
The new statement "persist rdp-cookie" enables RDP cookie
persistence. The RDP cookie is then extracted from the RDP
protocol, and compared against available servers. If a server
matches the RDP cookie, then it gets the connection.
2009-07-14 12:50:40 +02:00
Willy Tarreau
cd7afc0a13 [MINOR] http: take http request timeout from the backend
Since we can now switch from TCP to HTTP, we need to be able to apply
the HTTP request timeout after switching. That means we need to take
it from the backend and not from the frontend. Since the backend points
to the frontend before switching, that changes nothing for the normal
case.
2009-07-12 10:03:17 +02:00
Willy Tarreau
a9fb08317f [MINOR] report in the proxies the requirements for ACLs
This patch propagates the ACL conditions' "requires" bitfield
to the proxies. This makes it possible to know exactly what a
proxy might have to support for any request, which helps knowing
whether we have to allocate some space for certain types of
structures or not (eg: the hdr_idx struct).

The concept might be extended to a lot more types of information,
such as detecting whether we need to allocate some space for some
request ACLs which need a result in the response, etc...
2009-07-10 23:09:39 +02:00
Willy Tarreau
1d0dfb155d [MAJOR] http: complete splitting of the remaining stages
The HTTP processing has been splitted into 7 steps, one of which
is not anymore HTTP-specific (content-switching). That way, it
becomes possible to use "use_backend" rules in TCP mode. A new
"use_server" directive should follow soon.
2009-07-07 15:10:31 +02:00
Willy Tarreau
d787e6648c [MEDIUM] http: split request waiter from request processor
We want to split several steps in HTTP processing so that
we can call individual analysers depending on what processing
we want to perform. The first step consists in splitting the
part that waits for a request from the rest.
2009-07-07 10:14:51 +02:00
Willy Tarreau
915e1ebe63 [MEDIUM] config: split parser and checker in two functions
This is a first step towards support of multiple configuration files.
Now readcfgfile() only reads a file in memory and performs very minimal
parsing. The checks are performed afterwards.
2009-06-23 08:17:17 +02:00
Willy Tarreau
6704d67d65 [MEDIUM] support setting a server weight to zero
Sometimes it is useful to be able to set a server's weight to zero.
It allows the server to receive only persistent traffic but never
normal traffic.
2009-06-15 10:56:05 +02:00
Willy Tarreau
be1b91842a [MEDIUM] add support for TCP MSS adjustment for listeners
Sometimes it can be useful to limit the advertised TCP MSS on
incoming connections, for instance when requests come through
a VPN or when the system is running with jumbo frames enabled.

Passing the "mss <value>" arguments to a "bind" line will set
the value. This works under Linux >= 2.6.28, and maybe a few
earlier ones, though due to an old kernel bug most of earlier
versions will probably ignore it. It is also possible that some
other OSes will support this.
2009-06-14 18:48:19 +02:00
Willy Tarreau
d88edf2e52 [MEDIUM] implement tcp-smart-connect option at the backend
This new option enables combining of request buffer data with
the initial ACK of an outgoing TCP connection. Doing so saves
one packet per connection which is quite noticeable on workloads
mostly consisting in small objects. The option is not enabled by
default.
2009-06-14 15:48:17 +02:00
Willy Tarreau
9ea05a790f [MEDIUM] implement option tcp-smart-accept at the frontend
This option disables TCP quick ack upon accept. It is also
automatically enabled in HTTP mode, unless the option is
explicitly disabled with "no option tcp-smart-accept".

This saves one packet per connection which can bring reasonable
amounts of bandwidth for servers processing small requests.
2009-06-14 12:07:01 +02:00
Willy Tarreau
3842f00a19 [MINOR] config: support resetting options do default values
A new keyword prefix "default" has been introduced in order to
reset some options to their default values. This can be needed
for instance when an option is forced disabled or enabled in a
defaults section and when later sections want to use automatic
settings regardless of what was specified there. Right now it
is only supported by options, just like the "no" prefix.
2009-06-14 11:39:52 +02:00
Willy Tarreau
84b57dae4a [MINOR] config: track "no option"/"option" changes
Sometimes we would want to implement implicit default options,
but for this we need to be able to disable them, which requires
to keep track of "no option" settings. With this change, an option
explicitly disabled in a defaults section will still be seen as
explicitly disabled. There should be no regression as nothing makes
use of this yet.
2009-06-14 11:10:45 +02:00
Willy Tarreau
c6f4ce8fc4 [MEDIUM] add support for binding to source port ranges during connect
Some users are already hitting the 64k source port limit when
connecting to servers. The system usually maintains a list of
unused source ports, regardless of the source IP they're bound
to. So in order to go beyond the 64k concurrent connections, we
have to manage the source ip:port lists ourselves.

The solution consists in assigning a source port range to each
server and use a free port in that range when connecting to that
server, either for a proxied connection or for a health check.
The port must then be put back into the server's range when the
connection is closed.

This mechanism is used only when a port range is specified on
a server. It makes it possible to reach 64k connections per
server, possibly all from the same IP address. Right now it
should be more than enough even for huge deployments.
2009-06-10 12:23:32 +02:00
Willy Tarreau
13a34bd110 [MINOR] compute the max of sessions/s on fe/be/srv
Some users want to keep the max sessions/s seen on servers, frontends
and backends for capacity planning. It's easy to grab it while the
session count is updated, so let's keep it.
2009-05-10 18:52:49 +02:00
Willy Tarreau
f7edefa413 [MINOR] implement per-logger log level limitation
Some people are using haproxy in a shared environment where the
system logger by default sends alert and emerg messages to all
consoles, which happens when all servers go down on a backend for
instance. These people can not always change the system configuration
and would like to limit the outgoing messages level in order not to
disturb the local users.

The addition of an optional 4th field on the "log" line permits
exactly this. The minimal log level ensures that all outgoing logs
will have at least this level. So the logs are not filtered out,
just set to this level.
2009-05-10 17:20:05 +02:00
Benoit
affb481f1a [MEDIUM] add support for "balance hdr(name)"
There is a patch made by me that allow for balancing on any http header
field.

[WT:
  made minor changes:
  - turned 'balance header name' into 'balance hdr(name)' to match more
    closely the ACL syntax for easier future convergence
  - renamed the proxy structure fields header_* => hh_*
  - made it possible to use the domain name reduction to any header, not
    only "host" since it makes sense to do it with other ones.
  Otherwise patch looks good.
/WT]
2009-05-10 15:50:15 +02:00
Willy Tarreau
c9bd0cc224 [MINOR] add options dontlog-normal and log-separate-errors
Some big traffic sites have trouble dealing with logs and tend to
disable them. Here are two new options to help cope with massive
logs.

  - dontlog-normal only disables logging for 100% successful
    connections, other ones will still be logged

  - log-separate-errors will cause non-100% successful connections
    to be logged at level "err" instead of level "info" so that a
    properly configured syslog daemon can send them to a different
    file for longer conservation.
2009-05-10 11:57:02 +02:00
Maik Broemme
36db02e6b1 [BUG] x-original-to: fix missing initialization to default value 2009-05-08 17:02:07 +02:00
Maik Broemme
2850cb42b6 [MINOR] add X-Original-To: header
I have attached a patch which will add on every http request a new
header 'X-Original-To'. If you have HAProxy running in transparent mode
with a big number of SQUID servers behind it, it is very nice to have
the original destination ip as a common header to make decisions based
on it.

The whole thing is configurable with a new option 'originalto'. I have
updated the sourcecode as well as the documentation. The 'haproxy-en.txt'
and 'haproxy-fr.txt' files are untouched, due to lack of my french
language knowledge. ;)

Also the patch adds this header for IPv4 only. I haven't any IPv6 test
environment running here and don't know if getsockopt() with SO_ORIGINAL_DST
will work on IPv6. If someone knows it and wants to test it I can modify
the diff. Feel free to ask me questions or things which should be changed. :)

--Maik
2009-05-01 16:22:33 +02:00
Willy Tarreau
5e4a6f13f4 [MINOR] fix a few remaining printf-like formats on 64-bit platforms
Mainly two sizeof() returning size_t which is not the size of an int
on 64-bit platforms.
2009-04-11 19:42:49 +02:00
Jeffrey 'jf' Lim
af403fc59d [CLEANUP] give a little bit more information in error message
Indicate the error is about redirection.
2009-04-03 15:01:48 +02:00
Willy Tarreau
1772ece025 [MINOR] fix several printf formats and missing arguments
Last patch revealed a number of mistakes in printf-like calls, mostly int/long
mismatches, and a few missing arguments.
2009-04-03 14:49:12 +02:00
Willy Tarreau
32a4ec0ed7 [MEDIUM] http: add options to ignore invalid header names
Sometimes it is required to let invalid requests pass because
applications sometimes take time to be fixed and other servers
do not care. Thus we provide two new options :

     option accept-invalid-http-request  (for the frontend)
     option accept-invalid-http-response (for the backend)

When those options are set, invalid requests or responses do
not cause a 403/502 error to be generated.
2009-04-02 21:36:34 +02:00
Willy Tarreau
61d188920e [MINOR] improve reporting of misplaced acl/reqxxx rules
Now we can detect improper ordering of "block", "reqxxx", "reqadd",
"redirect" and "use_backend", and warn the user accordingly.
2009-03-31 10:49:21 +02:00
Willy Tarreau
e38388033f [BUG] server check intervals must not be null
If server check interval is null, we might end up looping in
process_srv_chk().

Prevent those values from being zero and add some control in
process_srv_chk() against infinite loops.
2009-03-21 18:58:32 +01:00
Willy Tarreau
5af24efee9 [CLEANUP] config: catch and report some possibly wrong rule ordering
There are some configurations in which redirect rules are declared
after use_backend rules. We can also find "block" rules after any
of these ones. The processing sequence is :
  - block
  - redirect
  - use_backend

So as of now we try to detect wrong ordering to warn the user about
a possibly undesired behaviour.
2009-03-15 15:23:16 +01:00
Willy Tarreau
55bc0f8eb7 [MEDIUM] reverse internal proxy declaration order to match configuration
People are regularly complaining that proxies are linked in reverse
order when reading the stats. This is now definitely fixed because
the proxy order is now fixed to match configuration order.
2009-03-15 14:51:53 +01:00
Willy Tarreau
d869b24119 [MINOR] tcp-inspect: permit the use of no-delay inspection
Sometimes it may make sense to be able to immediately apply a verdict
without waiting at all. It was not possible because no inspect-delay
meant no inspection at all. This is now fixed.
2009-03-15 14:43:58 +01:00
Willy Tarreau
3cd9af228f [MINOR] cfgparse: set backends to "balance roundrobin" by default
When a backend has no LB algo specified and is not in dispatch, proxy
nor transparent mode, use "balance roundrobin" by default instead of
complaining. This will be particularly useful with stats and redirects.
2009-03-15 14:11:27 +01:00
Willy Tarreau
ff01a21ebe [MINOR] cfgparse: some cleanups in the consistency checks
Check for servers in health mode, for health mode in pure-backends.
Some code have been refactored for better organization.
2009-03-15 13:46:16 +01:00
Willy Tarreau
39af0f663d [BUG] rate-limit in defaults section was ignored
Just a missing initialisation of the field when creating a proxy.
2009-03-07 11:53:44 +01:00
Willy Tarreau
368480cf45 [BUG] the "source" keyword must first clear optional settings
Problem reported by John Lauro. When "source ... usesrc ..." is
set in the defaults section, it is not possible anymore to remove
the "usesrc" part when declaring a more precise "source" in a
backend. The only workaround was to declare it by server.

We need to clear optional settings when declaring a new "source".
The problem was the same with the "interface" declaration.
2009-03-01 08:27:21 +01:00
Willy Tarreau
0b9c02c861 [MEDIUM] implement bind-process to limit service presence by process
The "bind-process" keyword lets the admin select which instances may
run on which process (in multi-process mode). It makes it easier to
more evenly distribute the load across multiple processes by avoiding
having too many listen to the same IP:ports.
2009-02-04 22:05:05 +01:00
Willy Tarreau
c76721da57 [MEDIUM] add support for source interface binding at the server level
Add support for "interface <name>" after the "source" statement on
the server line.
2009-02-04 20:20:58 +01:00
Willy Tarreau
d53f96b3f0 [MEDIUM] add support for source interface binding
Specifying "interface <name>" after the "source" statement allows
one to bind to a specific interface for proxy<->server traffic.

This makes it possible to use multiple links to reach multiple
servers, and to force traffic to pass via an interface different
from the one the system would have chosen based on the routing
table.
2009-02-04 18:46:54 +01:00
Willy Tarreau
5e6e204d1c [MINOR] add support for bind interface name
By appending "interface <name>" to a "bind" line, it is now possible
to specifically bind to a physical interface name. Note that this
currently only works on Linux and requires root privileges.
2009-02-04 17:19:29 +01:00
Krzysztof Piotr Oledzki
7b723efca3 [DOC] remove buggy comment for use_backend
"early blocking based on ACLs" is definitely wrong here
2009-01-27 21:30:31 +01:00
Krzysztof Piotr Oledzki
52d522b566 [BUG] Fix listen & more of 2 couples <ip>:<port>
Fix "listen www-mutualise 80.248.x.y1:80,80.248.x.y2:80,80.248.x.y3:80":

[ALERT] 309/161509 (15450) : Invalid server address: '80.248.x.y1:80,80.248.x.y2'
[ALERT] 309/161509 (15450) : Error reading configuration file : /etc/haproxy/haproxy.cfg

Bug reported by Laurent Dolosor.
2009-01-27 21:00:18 +01:00
Willy Tarreau
3ab68cf0ae [MEDIUM] splice: add the global "nosplice" option
Setting "nosplice" in the global section will disable the use of TCP
splicing (both tcpsplice and linux 2.6 splice). The same will be
achieved using the "-dS" parameter on the command line.
2009-01-25 16:03:28 +01:00
Willy Tarreau
43b78999ec [MEDIUM] move global tuning options to the global structure
The global tuning options right now only concern the polling mechanisms,
and they are not in the global struct itself. It's not very practical to
add other options so let's move them to the global struct and remove
types/polling.h which was not used for anything else.
2009-01-25 15:42:27 +01:00
Willy Tarreau
66aa61f76b [MEDIUM] splice: add configuration options and set global.maxpipes
Three new options have been added when CONFIG_HAP_LINUX_SPLICE is
set :
  - splice-request
  - splice-response
  - splice-auto

They are used to enable splicing per frontend/backend. They are also
supported in defaults sections. The "splice-auto" option is meant to
automatically turn splice on for buffers marked as fast streamers.
This should save quite a bunch of file descriptors.

It was required to add a new "options2" field to the proxy structure
because the original "options" is full.

When global.maxpipes is not set, it is automatically adjusted to
the max of the sums of all frontend's and backend's maxconns for
those which have at least one splice option enabled.
2009-01-18 21:44:07 +01:00
Willy Tarreau
3ec79b9c42 [MINOR] global.maxpipes: add the ability to reserve file descriptors for pipes
This will be needed to use linux's splice() syscall.
2009-01-18 20:39:42 +01:00
Willy Tarreau
4b1f85912c [BUG] "option transparent" is for backend, not frontend !
"option transparent" was set and checked on frontends only while it
is purely a backend thing as it replaces the "balance" mode. For this
reason, it did only work in "listen" sections. This change will then
not affect the rare users of this option.
2008-12-23 23:13:55 +01:00
Willy Tarreau
0140f2553c [MINOR] redirect: add support for "set-cookie" and "clear-cookie"
It is now possible to set or clear a cookie during a redirection. This
is useful for logout pages, or for protecting against some DoSes. Check
the documentation for the options supported by the "redirect" keyword.

(cherry-picked from commit 4af993822e880d8c932f4ad6920db4c9242b0981)
2008-12-07 23:46:38 +01:00
Willy Tarreau
79da4697ca [MINOR] redirect: add support for the "drop-query" option
If "drop-query" is present on a "redirect" line using the "prefix" mode,
then the returned Location header will be the request URI without the
query-string. This may be used on some login/logout pages, or when it
must be decided to redirect the user to a non-secure server.

(cherry-picked from commit f2d361ccd73aa16538ce767c766362dd8f0a88fd)
2008-12-07 23:42:01 +01:00
Jeffrey 'jf' Lim
65cb2f1c85 [MINOR] cfgparse: fix off-by 2 in error message size
was just looking through the source, and noticed this... :)
(cherry picked from commit 63b76be713)
(cherry picked from commit a801db6c5ea750f93a3795dbb2e70c03e05bbef4)
2008-12-07 23:37:15 +01:00
Willy Tarreau
b099aca91a [BUG] acl-related keywords are not allowed in defaults sections
Using an ACL-related keyword in the defaults section causes a
segfault during parsing because the list headers are not initialized.
We must initialize list headers for default instance and reject
keywords relying on ACLs.
(cherry picked from commit 1c90a6ec20)
(cherry picked from commit eb8131b4e418b838b2d62d991d91d94482ba49de)
2008-12-07 23:33:43 +01:00
Willy Tarreau
da250db376 [BUG] ensure that listeners from disabled proxies are correctly unbound.
There is a problem when an instance is marked "disabled". Its ports are
still bound but will not be unbound upon termination. This causes processes
to accumulate during soft restarts, and might even cause failures to restart
new ones due to the inability to bind to the same port.

The ideal solution would be to bind all ports at the end of the configuration
parsing. An acceptable workaround is to unbind all listeners of disabled
proxies. This is what the current patch does.
(cherry picked from commit a944218e9c)
(cherry picked from commit 8cfebbb82b87345bade831920177077e7d25840a)
2008-12-07 23:33:25 +01:00
Willy Tarreau
3bc13774e1 [MINOR] pre-set analyser flags on the listener at registration time
In order to achieve more generic accept() code, we can set the request
analysers at the listener registration time. It's better than doing it
during accept(), and allows more code reuse.
2008-12-07 11:50:35 +01:00
Willy Tarreau
79f5fe82f8 [BUG] Fix empty X-Forwarded-For header name when set in defaults section
The following patch introduced a minor bug :
   [MINOR] permit renaming of x-forwarded-for header

If "option forwardfor" is declared in a defaults section, the header name
is never set and we see an empty header name before the value. Also, the
header name was not reset between two defaults sections.
2008-08-26 13:22:19 +02:00
Willy Tarreau
a534fea478 [CLEANUP] remove 65 useless NULL checks before free
C specification clearly states that free(NULL) is a no-op.
So remove useless checks before calling free.
2008-08-03 20:48:50 +02:00
Ross West
af72a1d8ec [MINOR] permit renaming of x-forwarded-for header
Because I needed it in my situation - here's a quick patch to
allow changing of the "x-forwarded-for" header by using a suboption to
"option forwardfor".

Suboption "header XYZ" will set the header from "x-forwarded-for" to "XYZ".

Default is still "x-forwarded-for" if the header value isn't defined.
Also the suboption 'except a.b.c.d/z' still works on the same line.

So it's now: option forwardfor [except a.b.c.d[/z]] [header XYZ]
2008-08-03 10:51:45 +02:00
Willy Tarreau
dd64f8d394 [MEDIUM] acl: when possible, report the name and requirements of ACLs in warnings
When an ACL is referenced at a wrong place (eg: response during request, layer7
during layer4), try to indicate precisely the name and requirements of this ACL.

Only the first faulty ACL is returned. A small change consisting in iterating
that way may improve reports :
   cap = ACL_USE_any_unexpected
   while ((acl=cond_find_require(cond, cap))) {
     warning()
     cap &= ~acl->requires;
   }

This will report the first ACL of each unsupported type. But doing so will
mangle the error reporting a lot, so we need to rework error reports first.
2008-08-03 09:41:05 +02:00
Willy Tarreau
a9802633d8 [MEDIUM] acl: enforce ACL type checking
ACL now hold information on the availability of the data they rely
on. They can indicate which parts of the requests/responses they
require, and the rules parser may now report inconsistencies.

As an example, switching rules are now checked for response-specific
ACLs, though those are not still set. A warning is reported in case
of mismatch. ACLs keyword restrictions will now have to be specifically
set wherever a better control is expected.

The line number where an ACL condition is declared has been added to
the conditions in order to be able to report the faulty line number
during post-loading checks.
2008-07-25 19:13:19 +02:00
Willy Tarreau
ec6c5df018 [CLEANUP] remove many #include <types/xxx> from C files
It should be stated as a rule that a C file should never
include types/xxx.h when proto/xxx.h exists, as it gives
less exposure to declaration conflicts (one of which was
caught and fixed here) and it complicates the file headers
for nothing.

Only types/global.h, types/capture.h and types/polling.h
have been found to be valid includes from C files.
2008-07-16 10:30:42 +02:00
Willy Tarreau
b686644ad8 [MAJOR] implement tcp request content inspection
Some people need to inspect contents of TCP requests before
deciding to forward a connection or not. A future extension
of this demand might consist in selecting a server farm
depending on the protocol detected in the request.

For this reason, a new state CL_STINSPECT has been added on
the client side. It is immediately entered upon accept() if
the statement "tcp-request inspect-delay <xxx>" is found in
the frontend configuration. Haproxy will then wait up to
this amount of time trying to find a matching ACL, and will
either accept or reject the connection depending on the
"tcp-request content <action> {if|unless}" rules, where
<action> is either "accept" or "reject".

Note that it only waits that long if no definitive verdict
can be found earlier. That generally implies calling a fetch()
function which does not have enough information to decode
some contents, or a match() function which only finds the
beginning of what it's looking for.

It is only at the ACL level that partial data may be processed
as such, because we need to distinguish between MISS and FAIL
*before* applying the term negation.

Thus it is enough to add "| ACL_PARTIAL" to the last argument
when calling acl_exec_cond() to indicate that we expect
ACL_PAT_MISS to be returned if some data is missing (for
fetch() or match()). This is the only case we may return
this value. For this reason, the ACL check in process_cli()
has become a lot simpler.

A new ACL "req_len" of type "int" has been added. Right now
it is already possible to drop requests which talk too early
(eg: for SMTP) or which don't talk at all (eg: HTTP/SSL).

Also, the acl fetch() functions have been extended in order
to permit reporting of missing data in case of fetch failure,
using the ACL_TEST_F_MAY_CHANGE flag.

The default behaviour is unchanged, and if no rule matches,
the request is accepted.

As a side effect, all layer 7 fetching functions have been
cleaned up so that they now check for the validity of the
layer 7 pointer before dereferencing it.
2008-07-16 10:29:07 +02:00
Willy Tarreau
9de1bbd004 [MEDIUM] modularize the "timeout" keyword configuration parser
The "timeout" keyword already relied on an external parser, let's
make use of the new keyword registration mechanism.
2008-07-09 20:34:27 +02:00
Willy Tarreau
39f23b6c7e [MINOR] cfgparse: add support for warnings in external functions
Some parsers will need to report warnings in some cases. Let's
use positive values for that.
2008-07-09 20:23:15 +02:00
Willy Tarreau
10522fd113 [MEDIUM] modularize the global "stats" keyword configuration parser
The "stats" keyword already relied on an external parser, let's
make use of the new keyword registration mechanism.
2008-07-09 20:12:41 +02:00
Willy Tarreau
5b2c33683b [MEDIUM] add support for configuration keyword registration
Any module which needs configuration keywords may now dynamically
register a keyword in a given section, and associate it with a
configuration parsing function using cfg_register_keywords() from
a constructor function. This makes the configuration parser more
modular because it is not required anymore to touch cfg_parse.c.
Example :

static int parse_global_blah(char **args, int section_type, struct proxy *curpx,
                             struct proxy *defpx, char *err, int errlen)
{
	printf("parsing blah in global section\n");
	return 0;
}

static int parse_listen_blah(char **args, int section_type, struct proxy *curpx,
		      struct proxy *defpx, char *err, int errlen)
{
	printf("parsing blah in listen section\n");
	if (*args[1]) {
		snprintf(err, errlen, "missing arg for listen_blah!!!");
		return -1;
	}
	return 0;
}

static struct cfg_kw_list cfg_kws = {{ },{
	{ CFG_GLOBAL, "blah", parse_global_blah },
	{ CFG_LISTEN, "blah", parse_listen_blah },
	{ 0, NULL, NULL },
}};

__attribute__((constructor))
static void __module_init(void)
{
	cfg_register_keywords(&cfg_kws);
}
2008-07-09 19:44:58 +02:00
Willy Tarreau
0c303eec87 [MAJOR] convert all expiration timers from timeval to ticks
This is the first attempt at moving all internal parts from
using struct timeval to integer ticks. Those provides simpler
and faster code due to simplified operations, and this change
also saved about 64 bytes per session.

A new header file has been added : include/common/ticks.h.

It is possible that some functions should finally not be inlined
because they're used quite a lot (eg: tick_first, tick_add_ifset
and tick_is_expired). More measurements are required in order to
decide whether this is interesting or not.

Some function and variable names are still subject to change for
a better overall logics.
2008-07-07 00:09:58 +02:00
Willy Tarreau
b0b37bcd65 [MEDIUM] further improve monotonic clock by check forward jumps
The first implementation of the monotonic clock did not verify
forward jumps. The consequence is that a fast changing time may
expire a lot of tasks. While it does seem minor, in fact it is
problematic because most machines which boot with a wrong date
are in the past and suddenly see their time jump by several
years in the future.

The solution is to check if we spent more apparent time in
a poller than allowed (with a margin applied). The margin
is currently set to 1000 ms. It should be large enough for
any poll() to complete.

Tests with randomly jumping clock show that the result is quite
accurate (error less than 1 second at every change of more than
one second).
2008-06-23 14:00:57 +02:00
Willy Tarreau
b7f694f20e [MEDIUM] implement a monotonic internal clock
If the system date is set backwards while haproxy is running,
some scheduled events are delayed by the amount of time the
clock went backwards. This is particularly problematic on
systems where the date is set at boot, because it seldom
happens that health-checks do not get sent for a few hours.

Before switching to use clock_gettime() on systems which
provide it, we can at least ensure that the clock is not
going backwards and maintain two clocks : the "date" which
represents what the user wants to see (mostly for logs),
and an internal date stored in "now", used for scheduled
events.
2008-06-22 17:18:02 +02:00
Willy Tarreau
7c669d7e0f [BUG] fix the dequeuing logic to ensure that all requests get served
The dequeuing logic was completely wrong. First, a task was assigned
to all servers to process the queue, but this task was never scheduled
and was only woken up on session free. Second, there was no reservation
of server entries when a task was assigned a server. This means that
as long as the task was not connected to the server, its presence was
not accounted for. This was causing trouble when detecting whether or
not a server had reached maxconn. Third, during a redispatch, a session
could lose its place at the server's and get blocked because another
session at the same moment would have stolen the entry. Fourth, the
redispatch option did not work when maxqueue was reached for a server,
and it was not possible to do so without indefinitely hanging a session.

The root cause of all those problems was the lack of pre-reservation of
connections at the server's, and the lack of tracking of servers during
a redispatch. Everything relied on combinations of flags which could
appear similarly in quite distinct situations.

This patch is a major rework but there was no other solution, as the
internal logic was deeply flawed. The resulting code is cleaner, more
understandable, uses less magics and is overall more robust.

As an added bonus, "option redispatch" now works when maxqueue has
been reached on a server.
2008-06-20 15:08:06 +02:00
Willy Tarreau
b463dfb2de [MEDIUM] add support for conditional HTTP redirection
A new "redirect" keyword adds the ability to send an HTTP 301/302/303
redirection to either an absolute location or to a prefix followed by
the original URI. The redirection is conditionned by ACL rules, so it
becomes very easy to move parts of a site to another site using this.

This work was almost entirely done at Exceliance by Emeric Brun.

A test-case has been added in the tests/ directory.
2008-06-07 23:08:56 +02:00
Krzysztof Piotr Oledzki
1acf217366 [BUG/CLEANUP] cookiedomain -> cookie_domain rename + free(p->cookie_domain)
Rename cookiedomain -> cookie_domain to be consistent with current
naming scheme. Also make sure cookie_domain is deallocated at deinit()
2008-05-30 07:03:22 +02:00
Krzysztof Piotr Oledzki
efe3b6f524 [MINOR] Allow to specify a domain for a cookie
This patch allows to specify a domain used when inserting a cookie
providing a session stickiness. Usefull for example with wildcard domains.

The patch adds one new variable to the struct proxy: cookiedomain.
When set the domain is appended to a Set-Cookie header.

Domain name is validated using the new invalid_domainchar() function.
It is basically invalid_char() limited to [A-Za-z0-9_.-]. Yes, the test
is too trivial and does not cover all wrong situations, but the main
purpose is to detect most common mistakes, not intentional abuses.

The underscore ("_") character is not RFC-valid but as it is
often (mis)used so I decided to allow it.
2008-05-25 10:09:02 +02:00
Willy Tarreau
51406233bb [MAJOR] implementation of the "leastconn" load balancing algorithm
The new "leastconn" LB algorithm selects the server which has the
least established or pending connections. The weights are considered,
so that a server with a weight of 20 will get twice as many connections
as the server with a weight of 10.

The algorithm respects the minconn/maxconn settings, as well as the
slowstart since it is a dynamic algorithm. It also correctly supports
backup servers (one and all).

It is generally suited for protocols with long sessions (such as remote
terminals and databases), as it will ensure that upon restart, a server
with no connection will take all new ones until its load is balanced
with others.

A test configuration has been added in order to ease regression testing.
2008-03-10 22:04:30 +01:00
Krzysztof Piotr Oledzki
f58a962247 [MINOR] Implement persistent id for proxies and servers
This patch adds a possibility to set a persistent id for a proxy/server.
Now, even if some proxies/servers are inserted/deleted/moved, iids and
sids can be still used reliable.

Some people add servers with tricky names (BACKEND or FRONTEND for example).
So I also added one more field ('type') to distinguish between a
backend (0), frontend (1) and server (2) without complicated logic:
if name==BACKEND and sid==0 then type is BACKEND else type is SERVER,
etc for a FRONTEND. It also makes possible to have one frontend with more
than one IP (a patch coming soon) with independed stats - for example to
differs between remote and local traffic.

Finally, I added documentation about the CSV format.

This patch depends on '[MEDIUM] Implement "track [<backend>/]<server>"'
2008-02-28 17:23:59 +01:00
Krzysztof Piotr Oledzki
c8b16fc948 [MEDIUM] Implement "track [<backend>/]<server>"
This patch implements ability to set the current state of one server
by tracking another one. It:
 - adds two variables: *tracknext, *tracked to struct server
 - implements findserver(), similar to findproxy()
 - adds "track" keyword accepting both "proxy/server" and "server" (assuming current proxy)
 - verifies if both checks and tracking is not enabled at the same time
 - changes set_server_down() to notify tracking server
 - creates set_server_up(), set_server_disabled(), set_server_enabled() by
   moving the code from process_chk() and adding notifications
 - changes stats to show a name of tracked server instead of Chk/Dwn/Dwntime(html)
   or by adding new variable (csv)

Changes from the previuos version:
 - it is possibile to track independently of the declaration order
 - one extra comma bug is fixed
 - new condition to check if there is no disable-on-404 inconsistency
2008-02-27 10:39:53 +01:00
Willy Tarreau
3a70f94991 [BUG] timeout.check was not pre-set to eternity
If timeout.check was not set, check were using 0 as the timeout, causing
odd behaviours.
2008-02-15 11:15:34 +01:00
Willy Tarreau
21d2af3e9f Revert "[BUILD] backend.c and checks.c did not build without tproxy !"
This reverts commit 3c3c0122f8.
This commit was buggy as it also removed previous tproxy changes !
2008-02-14 20:25:24 +01:00
Willy Tarreau
3c3c0122f8 [BUILD] backend.c and checks.c did not build without tproxy !
missing #ifdefs.
2008-02-13 22:22:56 +01:00
Willy Tarreau
7a58a72e85 [MINOR] add configuration support for "redir" server keyword
The servers now support the "redir" keyword, making it possible to
return a 302 with the specified prefix in front of the request instead
of connecting to them. This is generally useful for multi-site load
balancing but may also serve in order to achieve very high traffic
rate.

The keyword has only been added to the config parser and to structures,
it's not used yet.
2008-02-13 00:55:49 +01:00