Commit Graph

2725 Commits

Author SHA1 Message Date
Willy Tarreau
126d40691a MINOR: tools: add a generic binary hex string parser
We currently use such an hex parser in pat_parse_bin() to parse hex
string patterns. We'll need another generic one so let's move it to
standard.c and have pat_parse_bin() make use of it.
2013-12-06 11:50:47 +01:00
Thierry FOURNIER
0ffe78cfe3 MEDIUM: map: merge identical maps
This patch permits to use the same struct pattern for two indentical maps.
This permits to preserve memory, and permits to update only one
"struct pattern" when the dynamic map update is supported.
2013-12-06 11:40:53 +01:00
Thierry FOURNIER
a82d202af2 MINOR: map: Cleanup the initialisation of map descriptors.
The association between struct map_descriptor and its struct
map_reference must be done in the function map_create_descriptor().
2013-12-06 11:38:36 +01:00
Thierry FOURNIER
275db69c07 BUG/MINOR: map: The map list was declared in the map.h file
This bug is harmless and post-dev19, it does not require any backport.
2013-12-06 11:37:28 +01:00
Thierry FOURNIER
ab92cf3a09 BUG/MINOR: acl: acl parser does not recognize empty converter list
Commit 348971e (MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()'
into the ACL keyword) introduced a regression in the ACL parser. The second
argument of an ACL keyword is now mistakenly confused with a converter.

This bug is post-dev19 and does not require any backport.
2013-12-06 11:32:19 +01:00
Willy Tarreau
2d351b68c2 OPTIM: checks: avoid setting SO_LINGER twice
We happened to preform this call twice on some checks, once in the
recv event handler, and another one in the main function. Remove
the one from the event handler which does not make any more sense
there.
2013-12-05 02:36:25 +01:00
Willy Tarreau
06559ac1fb OPTIM: checks: don't poll on recv when using plain TCP connects
When pure TCP checks are used, we see a useless call to recvfrom()
in strace resulting from an inconditional poll on recv after the
connect() succeeds. Let's remove this one and properly report
connection success in the write events.
2013-12-05 02:23:48 +01:00
Willy Tarreau
25e2ab53db MEDIUM: checks: centralize error reporting
Error reporting in health checks is unreliable as the number of recent
patch shows. The main reason is that the code required to detect the
exact situation where the error occurred is not simple, and the errors
have to be handled closer to where they occur in order to be accurate
(rely on getsockopt(SO_ERROR) and errno).

To solve this, we introduce chk_report_conn_err(). It does its best to
consider a possible errno passed in argument, a possible timeout passed
as well, then it completes this with getsockopt() if needed, and takes
into account the current status of the connection. The result is that
by simply calling this function with errno when it's known, we can emit
accurate log messages from every location. We can now see a messages
like "Connection error during SSL handshake (No route to host)" which
were not previously possible.
2013-12-05 02:23:48 +01:00
Willy Tarreau
20a1834219 BUG/MINOR: checks: do not trust errno in write event before any syscall
The only case where errno is supposed to be valid is when the connection
has just got the CO_FL_ERROR flag and errno is not zero, because it will
have been set by the same function that has set the flag. For all other
situations, we need to check the socket using getsockopt(), but only do
it once, since it clears the pending error code. For this reason, we
assign the error code to errno in order not to lose it. The same call
is made at the entry of event_srv_chk_r(), event_srv_chk_w(), and
wake_srv_chk() so that we get a chance to collect errors reported by
the poller or by failed syscalls.

Note that this fix relies on the 4 previous patches, so backporters
must be very careful.
2013-12-05 02:23:48 +01:00
Willy Tarreau
ce3eda7c6a MINOR: connection: clear errno prior to checking for errors
At some places, we report an error by just detecting FD_POLL_ERR.
The problem is that the caller never knows if it must use errno or
call getsockopt(SO_ERROR). And since this last one clears the
pending error from the queue, it cannot be used inconditionally.

An elegant solution consists in clearing errno prior to inspecting
FD_POLL_ERR. The caller then knows that if it gets CO_FL_ERROR and
errno == 0, it must call getsockopt().
2013-12-05 02:23:48 +01:00
Willy Tarreau
67ff7e0af3 BUG/MEDIUM: acl: fix regression introduced by latest converters support
Since commit 348971e (MEDIUM: acl: use the fetch syntax
'fetch(args),conv(),conv()' into the ACL keyword), ACLs wait on input
that may change. This is visible in the configuration below :

        tcp-request inspect-delay 3s
        tcp-request content accept if REQ_CONTENT

Nothing will pass before the end of the timer. This is because
historically, sample_process() was dedicated to stick tables where
it was absolutely necessary to wait for a stable sample. Now samples
are used by many other things and we can't afford this. So let's move
this check to the stick tables after the call to sample_process()
instead.

This is post-1.5-dev19 work, no backport is required.
2013-12-05 02:23:13 +01:00
Willy Tarreau
26f4a04744 MEDIUM: connection: set the socket shutdown flags on socket errors
When we get a hard error from a syscall indicating the socket is dead,
it makes sense to set the CO_FL_SOCK_WR_SH and CO_FL_SOCK_RD_SH flags
to indicate that the socket may not be used anymore. It will ease the
error processing in health checks where the state of socket is very
important. We'll also be able to avoid some setsockopt(nolinger) after
an error.

For now, the rest of the code is not impacted because CO_FL_ERROR is
always tested prior to these flags.
2013-12-04 23:50:36 +01:00
Willy Tarreau
7fe45698f5 BUG/MINOR: connection: check EINTR when sending a PROXY header
PROXY protocol header was not tolerant to signals, so it might cause a
connection to report an error if a signal comes in at the exact same
moment the send is done.

This is 1.5-specific and does not need any backport.
2013-12-04 23:50:26 +01:00
Willy Tarreau
f12a20ebce BUG/MINOR: tcp: check that no error is pending during a connect probe
The tcp_connect_probe() function may be called upon I/O activity when
no recv/send callbacks were called (eg: recv not possible, nothing to
send). It only relies on connect() to observe the connection establishment
progress but that does not work when some network errors are pending on
the socket (eg: a delayed connection refused).

For this reason we need to run a getsockopt() in the case where the
poller reports FD_POLL_ERR on the socket. We use this opportunity to
update errno so that the conn->data->wake() function has all relevant
info when it sees CO_FL_ERROR.

At the moment no code is impacted by this bug because recv polling is
always enabled during a connect, so recvfrom() always sees the error
first. But this may change with the health check cleanup.

No backport is needed.
2013-12-04 23:50:10 +01:00
Godbach
4f48990c1a OPTIM: stream_interface: return directly if the connection flag CO_FL_ERROR has been set
The connection flag CO_FL_ERROR will be tested in the functions both
si_conn_recv_cb() and si_conn_send_cb(). If CO_FL_ERROR has been set, out_error
branch will be executed. But the only job of out_error branch is to set
CO_FL_ERROR on connection flag. So it's better return directly than goto
out_error branch under such conditions. As a result, out_error branch becomes
needless and can be removed.

In addition, the return type of si_conn_send_loop() is also changed to void.
The caller should check conn->flags for errors just like stream_int_chk_snd_conn()
does as below:

static void stream_int_chk_snd_conn(struct stream_interface *si)
{
	...
        conn_refresh_polling_flags(si->conn);

-       if (si_conn_send(si->conn) < 0) {
+       si_conn_send(si->conn);
+       if (si->conn->flags & CO_FL_ERROR) {
	...
}

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2013-12-04 10:46:09 +01:00
Willy Tarreau
6b7764a983 MINOR: stats: remove some confusion between the DRAIN state and NOLB
We now have to report 2 conflicting information on the stats page :
  - NOLB  = server which returns 404 and stops load balancing ;
  - DRAIN = server with a weight forced to zero

The DRAIN state was previously detected from eweight==0 and represented in
blue so that a temporarily disabled server was noticed. This was done by
commit cc8bb92 (MINOR: stats: show soft-stopped servers in different color).
This choice suffered from a small defect however, which is that a server
with a zero weight was reported in this color whatever its state (even down
or switching).

Also, one of the motivations for the color above was because the NOLB state
is barely detectable as it's very close to the UP state.

Since commit 8c3d0be (MEDIUM: Add DRAIN state and report it on the stats page),
we have the new DRAIN state to show servers with a zero weight. The colors are
unfortunately very close to those of the MAINT state, and some users were
confused by the disappearance of the blue bars.

Additionally, the NOLB state had precedence over DRAIN, which could be an
issue since DRAIN is the only thing the admin can act on, so once NOLB was
shown, there was nothing to indicate that the weight was forced to zero.
By switching the two priorities we can report DRAIN (forced mode) before
NOLB (detected mode).

The best solution to fix all this is to reuse the previous blue color for
all cases where weight == 0, whether it's set by config / agent / cli (DRAIN)
or detected by a 404 response (NOLB). However we only use this color when the
server is 100% UP. If it's going down we switch to the usual yellow color
showing failed checks, and when it's down it keeps its usual red color.

That way, a blue bar on the display indicates a server not taking new
sessions but perfectly up. And other colors keep their usual meaning.
2013-12-04 00:59:55 +01:00
Willy Tarreau
0900bcbdbb BUG/MEDIUM: checks: also update the DRAIN state from the web interface
In commit 8c3d0be (MEDIUM: Add DRAIN state and report it on the stats page),
the drain state was updated on every weight change except those that can be
sent via the web interface. This caused inconsistent state combinations to
be reported in the stats depending on the sequence (web then cli vs cli
then web).

It would seem that a call to set_server_drain_state() from within
server_recalc_eweight() would simplify things but that's not completely
certain yet.
2013-12-04 00:54:18 +01:00
Willy Tarreau
ea292ae649 BUG/MINOR: checks: don't consider errno and use conn->err_code
The last fix on checks (02b0f58: BUG/MEDIUM: checks: fix a long-standing
issue with reporting connection errors) tried to isolate error codes
retrieved from the socket in order to report appropriate messages. The
only thing is that we must not pre-initialize err to errno since we're
not in I/O context anymore and errno will be the one of the last syscall
(whatever it was). However we can complete the message with more info
from the transport layer (eg: SSL can inform us we were in a handshake).

Also add a catch-all case for CO_FL_ERROR when the connection was
established. No check currently seem to leave this case open, but better
catch it because it's hard to find all possible cases.

Error handling in checks is complex because some stuff must be done in
the central task (mandatory at least for timeouts) and other stuff is
done closer to the data.

Since checks have their own buffers now, we could move everything to
the main task and only keep the low-level I/O for sending/retrieving
data to/from this buffer. It would also avoid sending logs from the
I/O context!
2013-12-03 22:54:39 +01:00
Willy Tarreau
02b0f58c43 BUG/MEDIUM: checks: fix a long-standing issue with reporting connection errors
In 1.5-dev14 we fixed a bug induced by the new connection system which caused
handshake failures not to be reported in health checks. It was done with
commit 6c560da (BUG/MEDIUM: checks: report handshake failures). This fix
caused another issue which is that every check getting a TCP RST after a
valid response was flagged as error. This was fixed using commit c5c61fc
(BUG/MEDIUM: checks: ignore late resets after valid responses).

But because of this, we completely miss the status report. These two fixes
only set the check result as failed and did not call set_server_check_status()
to pass the information to upper layers.

The impact is that some failed checks are reported as INI or are simply not
updated if they happen fast enough (eg: TCP RST in response to connect()
without data in a pure TCP check). So the server appears down but the check
status says "L4OK".

After commit 6c560da, the handshake failures have been correctly dealt with
and every error causes process_chk() to be called with the appropriate
information still present on the socket. So let's get the error code in
process_chk() instead and stop mangling it in wake_srv_chk().

Now both L4 and L6 checks are correctly reported.

This bug was first introduced in 1.5-dev12 so no backport is needed.
2013-12-03 15:56:17 +01:00
Willy Tarreau
2f075e98fb BUG/MEDIUM: checks: fix health check regression causing them to depend on declaration order
Since commit 4a74143 (MEDIUM: Paramatise functions over the check of a
server), the check type is inherited from the current proxy's check type
at the moment where the server is declared instead of when reviewing
server configs. This causes an issue where a health check is disabled
when the server is declared before the checks. In fact the server will
inherit the last known check type declared before the "server" line :

  backend foo
        # this server is not checked at all
        server s1 1.1.1.1:80 check
        option tcpchk
        # this server is tcp-checked :
        server s2 1.1.1.2:80 check
        option httpchk
        # this server is http-checked :
        server s3 1.1.1.3:80 check

The fix consists in assigning the check type during the config review
phase where the config is stable. No backport is nedeed.
2013-12-03 11:20:20 +01:00
Willy Tarreau
9eba36b726 BUILD: log: silent a warning about isblank() with latest patches
Recent commit 06d97f9 (MEDIUM: log-format: relax parsing of '%' followed
by unsupported characters) caused the following warning on some compilers
since isblank is not always present :

  src/log.c: In function 'parse_logformat_string':
  src/log.c:453: warning: implicit declaration of function 'isblank'

As usual, replace it with the two values (space and tab).
2013-12-03 00:51:09 +01:00
Willy Tarreau
60e0838f60 BUG/MINOR: http: usual deinit stuff in last commit
We need to initialize the rdr_fmt list inconditionally. Using only
a redirect rule without an http-redirect may cause a crash during
deinit because of the list iterating from null.
2013-12-03 00:48:45 +01:00
Thierry FOURNIER
d18cd0f110 MEDIUM: http: The redirect strings follows the log format rules.
We handle "http-request redirect" with a log-format string now, but we
leave "redirect" unaffected.

Note that the control of the special "/" case is move from the runtime
execution to the configuration parsing. If the format rule list is
empty, the build_logline() function does nothing.
2013-12-02 23:31:33 +01:00
Willy Tarreau
06d97f935c MEDIUM: log-format: relax parsing of '%' followed by unsupported characters
At the moment when a '%' character is followed by any unhandled character,
it is considered as a variable name, and if it cannot be resolved, a warning
is emitted and the configuration goes on.

When we start using log-format for redirect rules, it may happen that some
people accidently use '%' instead of '%%' without understanding the cause
of the issue. Thus we do two things here :

   - if a single '%' is followed by a blank or a digit, we fix it and emit a
     warning explaining how this should be done ; this ensures that existing
     configs continue to work ;

   - if a single '%' is followed by an unknown variable name, we report it
     and explain how to emit a verbatim '%' in case this is what the user
     desired.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
d5f624dde7 MEDIUM: sample: add the "map" converter
Add a new converter with the following prototype :

  map(<map_file>[,<default_value>])
  map_<match_type>(<map_file>[,<default_value>])
  map_<match_type>_<output_type>(<map_file>[,<default_value>])

It searches the for input value from <map_file> using the <match_type>
matching method, and return the associated value converted to the type
<output_type>. If the input value cannot be found in the <map_file>,
the converter returns the <default_value>. If the <default_value> is
not set, the converter fails and acts as if no input value could be
fetched. If the <match_type> is not set, it defaults to "str".
Likewise, if the <output_type> is not set, it defaults to "str". For
convenience, the "map" keyword is an alias for "map_str" and maps a
string to another string. The following array contains contains the
list of all the map* converters.

                 +----+----------+---------+-------------+------------+
                 |     `-_   out |         |             |            |
                 | input  `-_    |   str   |     int     |     ip     |
                 | / match   `-_ |         |             |            |
                 +---------------+---------+-------------+------------+
                 | str   / str   | map_str | map_str_int | map_str_ip |
                 | str   / sub   | map_sub | map_sub_int | map_sub_ip |
                 | str   / dir   | map_dir | map_dir_int | map_dir_ip |
                 | str   / dom   | map_dom | map_dom_int | map_dom_ip |
                 | str   / end   | map_end | map_end_int | map_end_ip |
                 | str   / reg   | map_reg | map_reg_int | map_reg_ip |
                 | int   / int   | map_int | map_int_int | map_int_ip |
                 | ip    / ip    | map_ip  | map_ip_int  | map_ip_ip  |
                 +---------------+---------+-------------+------------+

The names are intentionally chosen to reflect the same match methods
as ACLs use.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
b805f71d1b MEDIUM: sample: let the cast functions set their output type
This patch allows each sample cast function to specify the sample
output type. The goal is to be able to emit an output type IPv4 or
IPv6 depending on what is found in the input if the next converter
is able to process them both.

The patch also adds a new pseudo type called "ADDR". This type is an
alias for IPV4 and IPV6 which is only used as an input type by converters
who want to express their compatibility with both address formats. It may
not be emitted.

The goal is to unify as much as possible the processing of IPv4 and IPv6
in order not to add extra keywords for the maps which act as converters,
but will match samples like ACLs do with their patterns.
2013-12-02 23:31:33 +01:00
Willy Tarreau
803685fa12 MEDIUM: stick-tables: support automatic conversion from ipv4<->ipv6
Make the stick-table key converter automatically adapt to the address
family of the input sample. Samples such as "src" will return an address
with a sample type depending on the input family. We'll have to support
such combinations when we add support for maps because the output type
will not necessarily be fixed.
2013-12-02 23:31:33 +01:00
Willy Tarreau
6f8fe310cf MINOR: pattern: import acl_find_match_name() into pattern.h
It's only dedicated to pattern match lookups, so it was renamed
pat_find_match_name().
2013-12-02 23:31:33 +01:00
Willy Tarreau
0cba607400 MINOR: acl/pattern: use types different from int to clarify who does what.
We now have the following enums and all related functions return them and
consume them :

   enum pat_match_res {
	PAT_NOMATCH = 0,         /* sample didn't match any pattern */
	PAT_MATCH = 3,           /* sample matched at least one pattern */
   };

   enum acl_test_res {
	ACL_TEST_FAIL = 0,           /* test failed */
	ACL_TEST_MISS = 1,           /* test may pass with more info */
	ACL_TEST_PASS = 3,           /* test passed */
   };

   enum acl_cond_pol {
	ACL_COND_NONE,		/* no polarity set yet */
	ACL_COND_IF,		/* positive condition (after 'if') */
	ACL_COND_UNLESS,	/* negative condition (after 'unless') */
   };

It's just in order to avoid doubts when reading some code.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
a65b343eee MEDIUM: pattern: rename "acl" prefix to "pat"
This patch just renames functions, types and enums. No code was changed.
A significant number of files were touched, especially the ACL arrays,
so it is likely that some external patches will not apply anymore.

One important thing is that we had to split ACL_PAT_* into two groups :
  - ACL_TEST_{PASS|MISS|FAIL}
  - PAT_{MATCH|UNMATCH}

A future patch will enforce enums on all these places to avoid confusion.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
d163e1ce30 MEDIUM: pattern: create pattern expression
This new structure contains the data needed for pattern matching. It's
the first step to the complete independance of the pattern matching.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
ed66c297c2 REORG: acl/pattern: extract pattern matching from the acl file and create pattern.c
This patch just moves code without any change.

The ACL are just the association between sample and pattern. The pattern
contains the match method and the parse method. These two things are
different. This patch cleans the code by splitting it.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
dd69a04666 MEDIUM: acl: associate "struct sample_storage" to each "struct acl_pattern"
This will be used later with maps. Each map will associate an entry with
a sample_storage value.

This patch changes the "parse" prototype and all the parsing methods.
The goal is to associate "struct sample_storage" to each entry of
"struct acl_pattern". Only the "parse" function can add the sample value
into the "struct acl_pattern".
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
29d47b87c4 MINOR: acl: Extract the pattern matching function
The map feature will need to match acl patterns. This patch extracts
the matching function from the global ACL function "acl_exec_cond".

The code was only moved to its own function, no functional changes were made.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
3a103c5a6b MINOR: acl: Extract the pattern parsing and indexation from the "acl_read_patterns_from_file()" function
With this split, the pattern indexation can apply to any source. The map
feature needs this functionality because the map cannot be loaded with the
same file format as the ones supported by acl_read_patterns_from_file().

The code was only moved to its own function, no functional changes were made.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
319e495a96 MINOR: acl: export acl arrays
The map feature needs to use the acl parser and converters.
2013-12-02 23:31:32 +01:00
Thierry FOURNIER
d559dd8390 MINOR: tools: Add a function to convert buffer to an ipv6 address
The inet_pton function needs an input string with a final \0. This
function copies the input string to a temporary buffer, adds the final
\0 and converts to address.
2013-12-02 23:31:32 +01:00
Thierry FOURNIER
9c1d67ecbd MINOR: sample: provide the original sample_conv descriptor struct to the argument checker function.
Note that this argument checker is still unused but will be used by
maps.
2013-12-02 23:31:32 +01:00
Thierry FOURNIER
f73eb8f1dc MINOR: stick-table: use smp_expr_output_type() to retrieve the output type of a "struct sample_expr"
It's just a minor cleanup.
2013-12-02 23:31:32 +01:00
Thierry FOURNIER
348971ea28 MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()' into the ACL keyword
If the acl keyword is a "fetch", the dedicated parsing function
"sample_parse_expr()" is used. Otherwise, the acl parsing function
"parse_acl_expr()" is extended to understand the syntax of a series
of converters placed after the "fetch" keyword.

Before this patch, each acl uses a "struct sample_fetch" and executes
it with the "<fetch>->process()" function. Now, the dedicated function
"sample_process()" is called.

These syntax are now avalaible:

   acl bad req.hdr(host),lower -m str www
   http-request redirect prefix /go-away if bad

   acl bad hdr_beg(host),lower www
   http-request redirect prefix /go-away if bad
2013-12-02 23:31:32 +01:00
Thierry FOURNIER
8af6ff12b5 MINOR: sample: export sample_casts
just export the sample cast matrix "sample_casts" to prepare the
generic sample conversion parser.
2013-12-02 23:31:32 +01:00
Willy Tarreau
bf0addb6ce BUG/MINOR: log: fix log-format parsing errors
Some errors were still reported as log-format instead of their respective
contexts (acl, request header, stick, ...). This is harmless and does not
require any backport.
2013-12-02 23:31:32 +01:00
Willy Tarreau
34c2fb6f89 BUG/MINOR: config: report the correct track-sc number in tcp-rules
When parsing track-sc* actions in tcp-request rules, we now automatically
compute the track-sc identifier number using %d when displaying an error
message. But the ID has become wrong since we introduced sc0, we continue
to report id+1 in error messages causing some confusion.

No backport is needed.
2013-12-02 23:31:32 +01:00
Willy Tarreau
1903acdf3a BUG/MINOR: backend: fix target address retrieval in transparent mode
A very old bug resulting from some code refactoring causes
assign_server_address() to refrain from retrieving the destination
address from the client-side connection when transparent mode is
enabled and we're connecting to a server which has address 0.0.0.0.

The impact is low since such configurations are unlikely to ever
be encountered. The fix should be backported to older branches.
2013-12-01 21:46:24 +01:00
Thierry FOURNIER
1c0054fe83 BUG/MINOR: arg: fix error reporting for add-header/set-header sample fetch arguments
The 'add-header %[samples]' parsing errors associated to http-request
and http-response are displayed with the wrong keyword.

Configuration entry:

   http-request set-header mon-header %[res.hdr(user-agent)]

Original error message:

   [WARNING] 323/150920 (16559) : parsing [haproxy.conf:36] : 'log-format' : sample fetch <res.hdr ...

After commit error message:

   [WARNING] 323/150929 (16580) : parsing [haproxy.conf:36] : 'http-request' : sample fetch <res.hdr ...
2013-11-28 18:25:18 +01:00
Thierry FOURNIER
4a04dc368d BUG/MEDIUM: sample: The function v4tov6 cannot support input and output overlap
This patch permits to use v4tov6 with the same input and output buffer. It
might have impacted the format of IPv4 addresses stored into IPv6 tables.
2013-11-28 17:09:45 +01:00
Willy Tarreau
f465994198 BUG/MINOR: stats: do not report "via" on tracking servers in maintenance
When a server tracks another one, its state on the stats page always reports
"via xx/yy". That's convenient to know what server to act on to change the
state. But it is also possible to force the tracking server itself into
maintenance mode and in this case we should not report "via xx/yy" because
the tracked server can't do anything to change the server's state, which
is confusing. In practice there is nothing wrong in leaving it as-is,
except that it's highly misleading when looking at the stats page.

Note that we only change the HTML output, not the CSV one. The states are
already different : "MAINT" vs "MAINT(via)" and we expect anyone coding a
monitoring system based on the CSV output to know the differences between
all possible states.
2013-11-28 11:52:11 +01:00
Willy Tarreau
81cf08c5cd BUG/MAJOR: check: fix haproxy crash during soft-stop/soft-start
This is the continuation of previous fix bc16cd8 "BUG/MAJOR: fix haproxy
crash when using server tracking instead of checks", the soft-stop/start
states were not addressed by this fix.
2013-11-28 11:52:11 +01:00
Willy Tarreau
bc16cd81c4 BUG/MAJOR: fix haproxy crash when using server tracking instead of checks
Igor at owind reported a very recent bug (just present in latest snapshot).
Commit "4a741432 MEDIUM: Paramatise functions over the check of a server"
causes up/down to die with tracked servers due to a typo.

The following call in set_server_down causes the server to put itself
down recurseively because "check" is the current server's check, so once
fed to the function again, it will pass through the exact same path (note
we have the exact symmetry in set_server_up) :

	for (srv = s->tracknext; srv; srv = srv->tracknext)
		if (!(srv->state & SRV_MAINTAIN))
			/* Only notify tracking servers that are not already in maintenance. */
			set_server_down(check);

Instead we should stop the tracking server being visited in the loop :

	for (srv = s->tracknext; srv; srv = srv->tracknext)
		if (!(srv->state & SRV_MAINTAIN))
			/* Only notify tracking servers that are not already in maintenance. */
			set_server_down(&srv->check);

But that's not exactly enough because srv->check->server is only set when
checks are enabled, so ->server is NULL for tracking servers, still causing a
crash upon first iteration. The fix is easy and consists in always initializing
check->server when creating a new server, which is what was already done a few
patches later by 69d29f9 (MEDIUM: cfgparse: Factor out check initialisation).

With the fix above alone on top of current version or snapshot 20131122, the
problem disappears.

Thanks to Igor for testing and reporting the issue.
2013-11-27 17:10:07 +01:00
Willy Tarreau
86a446e685 MINOR: peers: accept to learn strings of different lengths
While analysing old bug (9d9179b) with Emeric, we first believed
that the fix was wrong and that there was a potential for learning
one extra character in the peers learning code for strings due to
the use of table->key_size instead of table->key_size-1. In fact it
cannot happen with a normally behaving sender because the key sizes
are compared when synchronizing the table.

But this unveiled a suboptimal handling of strings. It can be quite
common to see admins reload haproxy to increase some key sizes when
seeing that user agents or cookies get truncated, or conversely to
reduce them after seeing they take too much memory and are never full.
The problem is that this will get rid of the table's contents because
of the size mismatch. While this is understandable for properly
formatted data (eg: IP addresses, integers, SSLIDs...) it's too bad
for strings.

So instead, make an exception to accept string of incompatible lengths
and let the synchronization code truncate them to the appropriate size
just as if the keys were learned normally.

Thanks to this change, it is now possible to change the "len" parameter
of a string stick-table and restart without losing its contents.
2013-11-25 23:15:06 +01:00
Willy Tarreau
d6e999b127 OPTIM: connection: fold the error handling with handshake handling
Both of them are rare and are detected from the same flags source, so
let's detect errors in the handshake loop and remove two tests in the
fast path. This seems to improve overall performance by less than 0.5%
on connection-bound workloads.
2013-11-25 08:57:11 +01:00
Simon Horman
8c3d0be987 MEDIUM: Add DRAIN state and report it on the stats page
Add a DRAIN sub-state for a server which
will be shown on the stats page instead of UP if
its effective weight is zero.

Also, log if a server enters or leaves the DRAIN state
as the result of an agent check.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:16 +01:00
Simon Horman
671b6f02b5 MEDIUM: Add enable and disable agent unix socket commands
The syntax of this new commands are:

enable agent <backend>/<server>
disable agent <backend>/<server>

These commands allow temporarily stopping and subsequently
re-starting an auxiliary agent check. The effect of this is as follows:

New checks are only initialised when the agent is in the enabled. Thus,
disable agent will prevent any new agent checks from begin initiated until
the agent re-enabled using enable agent.

When an agent is disabled the processing of an auxiliary agent check that
was initiated while the agent was set as enabled is as follows: All
results that would alter the weight, specifically "drain" or a weight
returned by the agent, are ignored. The processing of agent check is
otherwise unchanged.

The motivation for this feature is to allow the weight changing effects
of the agent checks to be paused to allow the weight of a server to be
configured using set weight without being overridden by the agent.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:16 +01:00
Simon Horman
58c32978b2 MEDIUM: Set rise and fall of agent checks to 1
This is achieved by moving rise and fall from struct server to struct check.

After this move the behaviour of the primary check, server->check is
unchanged. However, the secondary agent check, server->agent now has
independent rise and fall values each of which are set to 1.

The result is that receiving "fail", "stopped" or "down" just once from the
agent will mark the server as down. And receiving a weight just once will
allow the server to be marked up if its primary check is in good health.

This opens up the scope to allow the rise and fall values of the agent
check to be configurable, however this has not been implemented at this
stage.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:16 +01:00
Simon Horman
2f1f955c8c MEDIUM: Do not mark a server as down if the agent is unavailable
In the case where agent-port is used and the agent
check is a secondary check to not mark a server as down
if the agent becomes unavailable.

In this configuration the agent should only cause a server to be marked
as down if the agent returns "fail", "stopped" or "down".

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:16 +01:00
Simon Horman
d60d69138b MEDIUM: checks: Add supplementary agent checks
Allow an auxiliary agent check to be run independently of the
regular a regular health check. This is enabled by the agent-check
server setting.

The agent-port, which specifies the TCP port to use for the agent's
connections, is required.

The agent-inter, which specifies the interval between agent checks and
timeout of agent checks, is optional. If not set the value for regular
checks is used.

e.g.
server	web1_1 127.0.0.1:80 check agent-port 10000

If either the health or agent check determines that a server is down
then it is marked as being down, otherwise it is marked as being up.

An agent health check performed by opening a TCP socket and reading an
ASCII string. The string should have one of the following forms:

* An ASCII representation of an positive integer percentage.
  e.g. "75%"

  Values in this format will set the weight proportional to the initial
  weight of a server as configured when haproxy starts.

* The string "drain".

  This will cause the weight of a server to be set to 0, and thus it
  will not accept any new connections other than those that are
  accepted via persistence.

* The string "down", optionally followed by a description string.

  Mark the server as down and log the description string as the reason.

* The string "stopped", optionally followed by a description string.

  This currently has the same behaviour as "down".

* The string "fail", optionally followed by a description string.

  This currently has the same behaviour as "down".

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:16 +01:00
Simon Horman
afc47ee7fc MEDIUM: Remove option lb-agent-chk
Remove option lb-agent-chk and thus the facility to configure
a stand-alone agent health check. This feature was added by
"MEDIUM: checks: Add agent health check". It will be replaced
by subsequent patches with a features to allow an agent check
to be run as either a secondary check, along with any of the existing
checks, or as part of an http check with the status returned
in an HTTP header.

This patch does not entirely revert "MEDIUM: checks: Add agent health
check". The infrastructure it provides to parse the results of an
agent health check remains and will be re-used by the planned features
that are mentioned above.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:16 +01:00
Simon Horman
80fefaeb57 MEDIUM: Log agent fail, stopped or down as info
In the case where an agent check returns fail, stopped or down,
log this as info when logging the server status along with any
trailing message returned by the agent after fail, stopped or down.

Previously only the trailing message was logged as info and
if omitted no info was logged.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:15 +01:00
Simon Horman
d858306ddb MEDIUM: Add helper function for failed checks
This consolidates some logic in preparation for enhancing it.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:15 +01:00
Simon Horman
5c9424258e MEDIUM: Add helper for task creation for checks
This helper is in preparation for adding a second struct check element
to struct server.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-25 07:31:15 +01:00
Kristoffer Grnlund
f65194a6fa LOW: systemd-wrapper: Write debug information to stdout
Write the command line used to call haproxy to stdout, as
well as the return code returned by the haproxy process.
2013-11-23 12:06:51 +01:00
Kristoffer Grnlund
66fd1d830e MEDIUM: systemd-wrapper: Kill child processes when interrupted
Send SIGINT to child processes when killed. This ensures that
the haproxy process managed by the systemd-wrapper is stopped
when "systemctl stop haproxy.service" is called.
2013-11-23 12:06:51 +01:00
Kristoffer Grnlund
1b6e75fa84 MEDIUM: haproxy-systemd-wrapper: Use haproxy in same directory
Locate the wrapper and use a haproxy executable found in the
same directory.

This patch lets the wrapper work in openSUSE.
2013-11-23 12:06:50 +01:00
Willy Tarreau
d32c399747 MINOR: stats: report correct throttling percentage for servers in slowstart
The column used to report the throttle percentage when a server is in
slowstart is based on the time only. This is wrong, because server weights
in slowstart are updated at most once a second, so the reported value is
wrong at least fo rone second during each step, which means all the time
when using short delays (< 20s).

The second point is that it's disturbing to see a weight < 100% without
any throttle at the end of the period (during the last second), because
the effective weight has not yet been updated.

Instead, we now compute the exact ratio between eweight and uweight and
report it. It's always accurate and describes the value being used instead
of using only the date.

It can be backported to 1.4 though it's not particularly important.
2013-11-21 15:30:45 +01:00
Willy Tarreau
004e045f31 BUG/MAJOR: server: weight calculation fails for map-based algorithms
A crash was reported by Igor at owind when changing a server's weight
on the CLI. Lukas Tribus could reproduce a related bug where setting
a server's weight would result in the new weight being multiplied by
the initial one. The two bugs are the same.

The incorrect weight calculation results in the total farm weight being
larger than what was initially allocated, causing the map index to be out
of bounds on some hashes. It's easy to reproduce using "balance url_param"
with a variable param, or with "balance static-rr".

It appears that the calculation is made at many places and is not always
right and not always wrong the same way. Thus, this patch introduces a
new function "server_recalc_eweight()" which is dedicated to this task
of computing ->eweight from many other elements including uweight and
current time (for slowstart), and all users now switch to use this
function.

The patch is a bit large but the code was not trivially fixable in a way
that could guarantee this situation would not occur anymore. The fix is
much more readable and has been verified to work with all algorithms,
with both consistent and map-based hashes, and even with static-rr.

Slowstart was tested as well, just like enable/disable server.

The same bug is very likely present in 1.4 as well, so the patch will
probably need to be backported eventhough it will not apply as-is.

Thanks to Lukas and Igor for the information they provided to reproduce it.
2013-11-21 15:09:02 +01:00
Willy Tarreau
e7b73485d0 BUG/MEDIUM: checks: fix slow start regression after fix attempt
Commit 2e99390 (BUG/MEDIUM: checks: fix slowstart behaviour when server
tracking is in use) moved the slowstart task initialization within the
health check code and leaves it unset when checks are disabled. The
problem is that it's possible to trigger slowstart from the CLI by
issuing "disable server XXX / enable server XXX" even when checks are
disabled. The result is a crash when trying to wake up the slowstart
task of that server.

Move the task initialization earlier so that it is done even if the
checks are disabled.

This patch should be backported to 1.4 since the commit above was
backported there.
2013-11-21 15:07:55 +01:00
Godbach
c3916a7fca MINOR: buffer: align the last output line if there are less than 8 characters left
Commit c08057c does the align job for buffer_dump(), but it has not fixed the
issue that less than 8 characters are left in the last line as below:

Dumping contents from byte 0 to byte 119
         0  1  2  3  4  5  6  7    8  9  a  b  c  d  e  f
  0000: 47 45 54 20 2f 69 6e 64 - 65 78 2e 68 74 6d 20 48   GET /index.htm H
  0010: 54 54 50 2f 31 2e 30 0d - 0a 55 73 65 72 2d 41 67   TTP/1.0..User-Ag
  ...
  0060: 6e 65 63 74 69 6f 6e 3a - 20 4b 65 65 70 2d 41 6c   nection: Keep-Al
  0070: 69 76 65 0d 0a 0d 0a                              ive....

The last line of the hex column is still overlapped by the text column. Since
there will be additional "- " for the output line which has no less than 8
characters, two additional spaces should be present when there is less than 8
characters in order to do alignment. The result after being fixed is as below:

Dumping contents from byte 0 to byte 119
         0  1  2  3  4  5  6  7    8  9  a  b  c  d  e  f
  0000: 47 45 54 20 2f 69 6e 64 - 65 78 2e 68 74 6d 20 48   GET /index.htm H
  0010: 54 54 50 2f 31 2e 30 0d - 0a 55 73 65 72 2d 41 67   TTP/1.0..User-Ag
  ...
  0060: 6e 65 63 74 69 6f 6e 3a - 20 4b 65 65 70 2d 41 6c   nection: Keep-Al
  0070: 69 76 65 0d 0a 0d 0a                                ive....

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2013-11-21 08:07:04 +01:00
Simon Horman
125d099662 MEDIUM: Move health element to struct check
This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-19 09:36:07 +01:00
Simon Horman
cd5d7b678e MEDIUM: Add state to struct check
Add state to struct check. This is currently used to store one bit,
CHK_RUNNING, which is set if a check is running and clear otherwise.
This bit was previously SRV_CHK_RUNNING of the state element of struct
server.

This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.

Signed-off-by: Simon Horman <horms+renesas@verge.net.au>
2013-11-19 09:36:04 +01:00
Simon Horman
69d29f996b MEDIUM: cfgparse: Factor out check initialisation
This is in preparation for struct server having two elements
of type struct check.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-19 09:36:01 +01:00
Simon Horman
4a741432be MEDIUM: Paramatise functions over the check of a server
Paramatise the following functions over the check of a server

* set_server_down
* set_server_up
* srv_getinter
* server_status_printf
* set_server_check_status
* set_server_disabled
* set_server_enabled

Generally the server parameter of these functions has been removed.
Where it is still needed it is obtained using check->server.

This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.
By paramatising these functions they may act on each of the checks
without further significant modification.

Explanation of the SSP_O_HCHK portion of this change:

* Prior to this patch SSP_O_HCHK serves a single purpose which
  is to tell server_status_printf() weather it should print
  the details of the check of a server or not.

  With the paramatisation that this patch adds there are two cases.
  1) Printing the details of the check in which case a
     valid check parameter is needed.
  2) Not printing the details of the check in which case
     the contents check parameter are unused.

  In case 1) we could pass SSP_O_HCHK and a valid check and;
  In case 2) we could pass !SSP_O_HCHK and any value for check
  including NULL.

  If NULL is used for case 2) then SSP_O_HCHK becomes supurfulous
  and as NULL is used for case 2) SSP_O_HCHK has been removed.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-19 09:35:54 +01:00
Simon Horman
28b5ffc76f MEDIUM: Move result element to struct check
Move result element from struct server to struct check
This allows check results to be independent of the check's server.

This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-19 09:35:52 +01:00
Simon Horman
6618300e13 MEDIUM: Split up struct server's check element
This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.

The split has been made by:
* Moving elements of struct server's check element that will
  be shared by both checks into a new check_common element
  of struct server.
* Moving the remaining elements to a new struct check and
  making struct server's check element a struct check.
* Adding a server element to struct check, a back-pointer
  to the server element it is a member of.
  - At this time the server could be obtained using
    container_of, however, this will not be so easy
    once a second struct check element is added to struct server
    to accommodate an agent health check.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-19 09:35:48 +01:00
Simon Horman
a360844735 CLEANUP: Make parameters of srv_downtime and srv_getinter const
The parameters of srv_downtime and srv_getinter are not modified
and thus may be const.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-11-19 08:04:58 +01:00
Willy Tarreau
e155ec245a BUG/MINOR: http: fix build warning introduced with url32/url32_src
commit 39c63c5 "url32+src - like base32+src but whole url including parameters"
was missing the last argument "const char *kw", resulting in the build warning
below :

src/proto_http.c:10351:2: warning: initialization from incompatible pointer type [enabled by default]
src/proto_http.c:10351:2: warning: (near initialization for 'sample_fetch_keywords.kw[50].process') [enabled by default]
src/proto_http.c:10352:2: warning: initialization from incompatible pointer type [enabled by default]
src/proto_http.c:10352:2: warning: (near initialization for 'sample_fetch_keywords.kw[51].process') [enabled by default]

It's harmless since it's not needed there anyway.
2013-11-18 18:33:32 +01:00
Willy Tarreau
6d4890cfea BUG/MEDIUM: http: fix possible parser crash when parsing erroneous "http-request redirect" rules
Baptiste Assmann reported a bug affecting the "http-request redirect"
parser. It may randomly crash when reporting an error message if the
syntax is not OK. It happens that this is caused by the output error
message pointer which was not initialized to NULL.

This bug is 1.5-specific (introduced in dev17), no backport is needed.
2013-11-18 18:07:35 +01:00
Neil - HAProxy List
39c63c56d2 url32+src - like base32+src but whole url including parameters
I have a need to limit traffic to each url from each source address. much
like base32+src but the whole url including parameters (this came from
looking at the recent 'Haproxy rate limit per matching request' thread)

attached is patch that seems to do the job, its a copy and paste job of the
base32 functions

the url32 function seems to work too and using 2 machines to request the
same url locks me out of both if I abuse from either with the url32 key
function and only the one if I use url32_src.

Neil
2013-11-18 06:50:38 +01:00
Willy Tarreau
3b44e729e5 CLEANUP: http: merge error handling for req* and http-request *
The reqdeny/reqtarpit and http-request deny/tarpit were using
a copy-paste of the error handling code because originally the
req* actions used to maintain their own stats. This is not the
case anymore so we can use the same error blocks for both.

The http-request rulesets still has precedence over req* so no
functionality was changed.
2013-11-16 10:30:14 +01:00
Willy Tarreau
687ba13e92 CLEANUP: http: homogenize processing of denied req counter
The reqdeny/reqideny and reqtarpit/reqitarpit rules used to maintain
the stats counters themselves while http-request deny/tarpit and
rspdeny/rspideny used to centralize them at the point where the
error is processed.

Thus, let's do the same for reqdeny/reqtarpit so that the functions
which iterate over the rules do not have to deal with these counters
anymore.
2013-11-16 10:13:35 +01:00
Willy Tarreau
8ac7249611 BUG/MINOR: stats: don't count tarpitted connections twice
When a connection is tarpitted, a denied req is counted once when the
action is applied, and then a failed req is counted when the tarpit
timeout expires. This is completely wrong as the tarpit is exactly
equivalent to a deny since it's a disguised deny.

So let's not increment the failed req anymore.

This fix may be backported to 1.4 which has the same issue.
2013-11-16 10:06:44 +01:00
Willy Tarreau
38d5892634 OPTIM/MINOR: mark the source address as already known on accept()
Commit 986a9d2d12 moved the source address from the stream interface
to the session, but it did not set the flag on the connection to
report that the source address is known. Thus when logs are enabled,
we had a call to getpeername() which is redundant with the result
from accept(). This patch simply sets the flag.
2013-11-16 00:17:59 +01:00
Willy Tarreau
2f877304ef OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes
When trying to speculatively send data to a server being connected to,
we see the following pattern :

    connect() = EINPROGRESS
    send() = EAGAIN
    epoll_ctl(add, W)
    epoll_wait() = EPOLLOUT
    send() = success
  > epoll_ctl(del, W)
  > recv() = EAGAIN
  > epoll_ctl(add, R)
    recv() = success
    epoll_ctl(del, R)

The reason for the failed recv() call is that the reading was marked
as speculative while we already have a polled I/O there. So we already
know when removing send write poll that the read is pending. Thus,
let's improve this by merging speculative I/O into polled I/O when
polled state changes. The result is now the following as expected :

    connect() = EINPROGRESS
    send() = EAGAIN
    epoll_ctl(add, W)
    epoll_wait() = EPOLLOUT
    send() = success
    epoll_ctl(mod, R)
    recv() = success
    epoll_ctl(del, R)

This is specific to epoll(), it doesn't make much sense at the moment
to do so for other pollers, because the cost of updating them is very
small.

The average performance gain on small requests is of 1.6% in TCP mode,
which is easily explained with the syscall stats below for 10000 forwarded
connections :

Before :
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 91.02    0.024608           0     60000         1 epoll_wait
  2.19    0.000593           0     20000           shutdown
  1.52    0.000412           0     10000     10000 connect
  1.36    0.000367           0     29998      9998 sendto
  1.09    0.000294           0     49993           epoll_ctl
  0.93    0.000252           0     50004     20002 recvfrom
  0.79    0.000214           0     20005           close
  0.62    0.000167           0     20001     10001 accept4
  0.25    0.000067           0     20002           setsockopt
  0.13    0.000035           0     10001           socket
  0.10    0.000028           0     10001           fcntl

After:
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 87.59    0.024269           0     50012         1 epoll_wait
  3.19    0.000884           0     20000           shutdown
  2.33    0.000646           0     29996      9996 sendto
  2.02    0.000560           0     10005     10003 connect
  1.40    0.000387           0     40013     10013 recvfrom
  1.35    0.000374           0     40000           epoll_ctl
  0.64    0.000178           0     20001     10001 accept4
  0.55    0.000152           0     20005           close
  0.45    0.000124           0     20002           setsockopt
  0.31    0.000086           0     10001           fcntl
  0.17    0.000047           0     10001           socket

Overall :
   -16.6% epoll_wait
   -20%   recvfrom
   -20%   epoll_ctl

On HTTP, the gain is even better :

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 80.43    0.015386           0     60006         1 epoll_wait
  4.61    0.000882           0     30000     10000 sendto
  3.74    0.000715           0     20001     10001 accept4
  3.35    0.000640           0     10000     10000 connect
  2.66    0.000508           0     20005           close
  1.34    0.000257           0     30002     10002 recvfrom
  1.27    0.000242           0     30005           epoll_ctl
  1.20    0.000230           0     10000           shutdown
  0.62    0.000119           0     20003           setsockopt
  0.40    0.000077           0     10001           socket
  0.39    0.000074           0     10001           fcntl
willy@wtap:haproxy$ head -15 apres.txt
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 83.47    0.020301           0     50008         1 epoll_wait
  4.26    0.001036           0     20005           close
  3.30    0.000803           0     30000     10000 sendto
  2.55    0.000621           0     20001     10001 accept4
  1.76    0.000428           0     10000     10000 connect
  1.20    0.000292           0     10000           shutdown
  1.14    0.000278           0     20001         1 recvfrom
  0.86    0.000210           0     20003           epoll_ctl
  0.71    0.000173           0     20003           setsockopt
  0.49    0.000120           0     10001           socket
  0.25    0.000060           0     10001           fcntl

Overall :
  -16.6% epoll_wait
  -33%   recvfrom
  -33%   epoll_ctl
2013-11-15 23:15:10 +01:00
Willy Tarreau
a0f4271497 MEDIUM: backend: add support for the wt6 hash
This function was designed for haproxy while testing other functions
in the past. Initially it was not planned to be used given the not
very interesting numbers it showed on real URL data : it is not as
smooth as the other ones. But later tests showed that the other ones
are extremely sensible to the server count and the type of input data,
especially DJB2 which must not be used on numeric input. So in fact
this function is still a generally average performer and it can make
sense to merge it in the end, as it can provide an alternative to
sdbm+avalanche or djb2+avalanche for consistent hashing or when hashing
on numeric data such as a source IP address or a visitor identifier in
a URL parameter.
2013-11-14 16:37:50 +01:00
Bhaskar Maddala
b6c0ac94a4 MEDIUM: backend: Implement avalanche as a modifier of the hashing functions.
Summary:
Avalanche is supported not as a native hashing choice, but a modifier
on the hashing function. Note that this means that possible configs
written after 1.5-dev4 using "hash-type avalanche" will get an informative
error instead. But as discussed on the mailing list it seems nobody ever
used it anyway, so let's fix it before the final 1.5 release.

The default values were selected for backward compatibility with previous
releases, as discussed on the mailing list, which means that the consistent
hashing will still apply the avalanche hash by default when no explicit
algorithm is specified.

Examples
  (default) hash-type map-based
	Map based hashing using sdbm without avalanche

  (default) hash-type consistent
	Consistent hashing using sdbm with avalanche

Additional Examples:

  (a) hash-type map-based sdbm
	Same as default for map-based above
  (b) hash-type map-based sdbm avalanche
	Map based hashing using sdbm with avalanche
  (c) hash-type map-based djb2
	Map based hashing using djb2 without avalanche
  (d) hash-type map-based djb2 avalanche
	Map based hashing using djb2 with avalanche
  (e) hash-type consistent sdbm avalanche
	Same as default for consistent above
  (f) hash-type consistent sdbm
	Consistent hashing using sdbm without avalanche
  (g) hash-type consistent djb2
	Consistent hashing using djb2 without avalanche
  (h) hash-type consistent djb2 avalanche
	Consistent hashing using djb2 with avalanche
2013-11-14 16:37:50 +01:00
Bhaskar
98634f0c7b MEDIUM: backend: Enhance hash-type directive with an algorithm options
Summary:
In testing at tumblr, we found that using djb2 hashing instead of the
default sdbm hashing resulted is better workload distribution to our backends.

This commit implements a change, that allows the user to specify the hash
function they want to use. It does not limit itself to consistent hashing
scenarios.

The supported hash functions are sdbm (default), and djb2.

For a discussion of the feature and analysis, see mailing list thread
"Consistent hashing alternative to sdbm" :

      http://marc.info/?l=haproxy&m=138213693909219

Note: This change does NOT make changes to new features, for instance,
applying an avalance hashing always being performed before applying
consistent hashing.
2013-11-14 16:37:50 +01:00
Willy Tarreau
60eccc1841 BUG/MINOR: acl: remove patterns from the tree before freeing them
A call to free_pattern_tree() upon exit() is made to free all ACL
patterns allocated in a tree (strings or IP addresses). Unfortunately
it happens that this function has been bogus from the beginning, it
walks over the whole tree, frees the nodes but forgets to remove them
from the tree prior to freeing them. So after visiting a leaf, the
next eb_next() call will require to revisit some of the upper nodes
that were just freed. This can remain unnoticed for a long time because
free() often just marks the area as free. But in cases of aggressive
memory freeing, the location will not be mapped anymore and the process
segfaults.

Note that the bug has no impact other than polluting kernel logs and
frightening sysadmins, since it happens just before exit().

Simply adding the debug code below makes it easier to reproduce the
same bug :

	while (node) {
		next = eb_next(node);
+		node->node_p = (void *)-1;
		free(node);
		node = next;
	}

Many thanks to the StackExchange team for their very detailed bug report
that permitted to quickly understand this non-obvious bug!

This fix should be backported to 1.4 which introduced the bug.
2013-11-14 16:37:50 +01:00
Godbach
c08057cc3f MINOR: buffer: align the last output line of buffer_dump()
If the dumped length of buffer is not multiple of 16, the last output line can
be seen as below:

Dumping contents from byte 0 to byte 125
         0  1  2  3  4  5  6  7    8  9  a  b  c  d  e  f
  0000: 47 45 54 20 2f 69 6e 64 - 65 78 2e 68 74 6d 20 48   GET /index.htm H
  0010: 54 54 50 2f 31 2e 30 0d - 0a 55 73 65 72 2d 41 67   TTP/1.0..User-Ag
  ...
  0060: 30 0d 0a 43 6f 6e 6e 65 - 63 74 69 6f 6e 3a 20 4b   0..Connection: K
  0070: 65 65 70 2d 41 6c 69 76 - 65 0d 0a 0d 0a   eep-Alive....

Yes, the hex column will be overlapped by the text column. Both the hex and
text column should be aligned at their own area as below:

Dumping contents from byte 0 to byte 125
         0  1  2  3  4  5  6  7    8  9  a  b  c  d  e  f
  0000: 47 45 54 20 2f 69 6e 64 - 65 78 2e 68 74 6d 20 48   GET /index.htm H
  0010: 54 54 50 2f 31 2e 30 0d - 0a 55 73 65 72 2d 41 67   TTP/1.0..User-Ag
  ...
  0060: 30 0d 0a 43 6f 6e 6e 65 - 63 74 69 6f 6e 3a 20 4b   0..Connection: K
  0070: 65 65 70 2d 41 6c 69 76 - 65 0d 0a 0d 0a            eep-Alive....

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2013-11-14 08:23:26 +01:00
Willy Tarreau
0bb166be5e MINOR: tcp: don't use tick_add_ifset() when timeout is known to be set
These two useless tests propably result from a copy-paste. The test is
performed in the condition to enter the block.
2013-11-04 18:12:20 +01:00
Willy Tarreau
3c3dfd5c61 MINOR: acl: add a warning when an ACL keyword is used without any value
It's quite common to write directives like the following :

  tcp-request reject if WAIT_END { sc0_inc_gpc0 }

This one will never reject, because sc0_inc_gpc0 is provided no value
to compare against. The proper form should have been something like this :

  tcp-request reject if WAIT_END { sc0_inc_gpc0 gt 0 }

or :

  tcp-request reject if WAIT_END { sc0_inc_gpc0 -m found }

Now we detect the absence of any argument on the command line and emit
a warning suggesting alternatives or the use of "--" to really avoid
matching anything (might be used when debugging).
2013-11-04 18:12:20 +01:00
Willy Tarreau
79c412b673 BUG/MEDIUM: acl: do not evaluate next terms after a miss
When a condition does something like :

   action if A B C || D E F

If B returns a miss (can't tell true or false), C must not
be evaluated. This is important when C has a side effect
(eg: sc*_inc_gpc0). However the second part after the ||
can still be evaluated.
2013-10-30 19:30:32 +01:00
Willy Tarreau
44778ad87d BUG/MEDIUM: tcp: do not skip tracking rules on second pass
The track-sc* tcp rules are bogus. The test to verify if the
tracked counter was already assigned is performed in the same
condition as the test for the action. The effect is that a
rule which tracks a counter that is already being tracked
is implicitly converted to an accept because the default
rule is an accept.

This bug only affects 1.5-dev releases.
2013-10-30 19:29:21 +01:00
Willy Tarreau
ae727bf9b4 BUG/MINOR: peers: set the accept date in outgoing connections
Without this, "show sess" on the CLI reports a wrong age.
2013-10-30 08:12:26 +01:00
Willy Tarreau
05bf5e1c36 BUG/MEDIUM: session: risk of crash on out of memory conditions
In session_accept(), if we face a memory allocation error, we try to
emit an HTTP 500 error message in HTTP mode. The problem is that we
must not use http_error_message() for this since it dereferences the
session which can be NULL in this case.

We don't need the session to build the error message anyway since
this function only uses it to retrieve the backend and frontend to
get the most suited error message. Let's pick it ourselves, we're
at the beginning of the session, only the frontend is relevant.

This bug is 1.5-specific.
2013-10-30 07:59:03 +01:00
Thierry FOURNIER
5068d96ac1 MINOR: http: change url_decode to return the size of the decoded string.
Currently url_decode returns 1 or 0 depending on whether it could decode
the string or not. For some future use cases, it will be needed to get the
decoded string length after a successful decoding, so let's make it return
that value, and fall back to a negative one in case of error.
2013-10-23 12:26:50 +02:00
Thierry FOURNIER
ef37a66628 CLEANUP: The function "regex_exec" needs the string length but in many case they expect null terminated char.
If haproxy is compiled with the USE_PCRE_JIT option, the length of the
string is used. If it is compiled without this option the function doesn't
use the length and expects a null terminated string.

The prototype of the function is ambiguous, and depends on the
compilation option. The developer can think that the length is always
used, and many bugs can be created.

This patch makes sure that the length is used. The regex_exec function
adds the final '\0' if it is needed.
2013-10-23 12:19:51 +02:00
Willy Tarreau
9ca6936c80 BUG/MINOR: acl: implicit arguments of ACL keywords were not properly resolved
William Lallemand reported a bug which happens when an ACL keyword using an
implicit argument (eg: a proxy name) is used : the keyword is not properly
set in the arglist field, resulting in an error about the previous keyword
being returned, or "(null)" if the faulty ACL appears first.

The bug only affects error reporting and is 1.5-specific, so no backport is
nedeed.
2013-10-22 19:31:04 +02:00
Willy Tarreau
472b1ee115 BUG/MEDIUM: http: accept full buffers on smp_prefetch_http
Bertrand Jacquin reported a but when using tcp_request content rules
on large POST HTTP requests. The issue is that smp_prefetch_http()
first tries to validate an input buffer, but only if the buffer is
not full. This test is wrong since it must only be performed after
the parsing has failed, otherwise we don't accept POST requests which
fill the buffer as valid HTTP requests.

This bug is 1.5-specific, no backport needed.
2013-10-14 22:47:00 +02:00
Thierry FOURNIER
ed5a4aefae CLEANUP: regex: Create regex_comp function that compiles regex using compilation options
The current file "regex.h" define an abstraction for the regex. It
provides the same struct name and the same "regexec" function for the
3 regex types supported: standard libc, basic pcre and jit pcre.

The regex compilation function is not provided by this file. If the
developper wants to use regex, he must write regex compilation code
containing "#define *JIT*".

This patch provides a unique regex compilation function according to
the compilation options.

In addition, the "regex.h" file checks the presence of the "#define
PCRE_CONFIG_JIT" when "USE_PCRE_JIT" is enabled. If this flag is not
present, the pcre lib doesn't support JIT and "#error" is emitted.
2013-10-14 14:42:50 +02:00
Godbach
e68e02dc1d CLEANUP: stream_interface: cleanup loop information in si_conn_send_loop()
Though si_conn_send_loop() does not loop over ->snd_buf() after commit ed7f836,
there is still some codes left which use `while` but only execute once. This
commit does the cleanup job and rename si_conn_send_loop() to si_conn_send().

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2013-10-12 07:53:33 +02:00
Emeric Brun
369da8565a BUG/MINOR: ssl: verifyhost does not match empty strings on wildcard.
RFC6125 does not specify if wildcard matches empty strings but
classical browsers implementations does.
After the fix foo*bar.exemple.om matches foobar.exemple.com.
2013-10-10 11:33:27 +02:00
Emeric Brun
a848dae3f0 MINOR: ssl: optimization of verifyhost on wildcard certificates.
Optimizes verifyhost on wildcard certificates avoiding travel several times
the same string.
2013-10-10 11:33:21 +02:00
Emeric Brun
9bf3ba28e1 BUG/MINOR: ssl: potential memory leaks using ssl_c_key_alg or ssl_c_sig_alg.
The leak occurs in an error case which practically never happens.
2013-10-10 11:33:14 +02:00
Godbach
af1a75d2c9 BUG/MINOR: deinit: free server map which is allocated in init_server_map()
Both static-rr and hash with type map-based call init_server_map() to allocate
server map, so the server map should be freed while doing cleanup if one of
the above load balance algorithms is used.

Signed-off-by: Godbach <nylzhaowei@gmail.com>
[wt: removed the unneeded "if" before the free]
2013-10-06 13:34:09 +02:00
Willy Tarreau
8cc16530c4 BUG/MINOR: acl: fix improper string size assignment in proxy argument
This minor bug was found using the coccinelle script "da.cocci". The
len was initialized twice instead of setting the size. It's harmless
since no operations are performed on this empty string but needs to
be fixed anyway.
2013-09-29 12:54:58 +02:00
Willy Tarreau
7959a55e15 MINOR: http: compute response time before processing headers
At the moment, HTTP response time is computed after response headers are
processed. This can misleadingly assign to the server some heavy local
processing (eg: regex), and also prevents response headers from passing
information related to the response time (which can sometimes be useful
for stats).

Let's retrieve the reponse time before processing the headers instead.

Note that in order to remain compatible with what was previously done,
we disable the response time when we get a 502 or any bad response. This
should probably be changed in 1.6 since it does not make sense anymore
to lose this information.
2013-09-23 16:53:11 +02:00
Sergiy Prykhodko
1d57e50810 BUG/MINOR: fix forcing fastinter in "on-error"
health_adjust() should requeue the task after changing its expire timer.

I noticed it on devel servers without load. We have long inter (10 seconds)
and short fasinter (100ms). But according to webserver logs, after a failed
request next check request was called with same 10s interval.

This patch should probably be backported to 1.4 which has the same feature.
2013-09-23 08:49:50 +02:00
Emeric Brun
a33410cf94 BUILD: ssl: compilation issue with openssl v0.9.6.
Failed to compile with openssl 0.9.6 since the 'verifyhost' feature.
2013-09-17 23:19:41 +02:00
Emeric Brun
4ad50a469d BUG/MEDIUM: ssl: potential memory leak using verifyhost
If server certificate presents dns aliases, a memory leak appears
on health checks when 'verifyhost' statement is used.
2013-09-17 23:19:27 +02:00
Willy Tarreau
47e8eba9a7 MINOR: payload: provide the "res.len" fetch method
This fetch method returns the response buffer len, similarly
to req.len for the request. Previously it was only possible
to rely on "res.payload(0,size) -m found" to find if at least
that amount of data was available, which was a bit tricky.
2013-09-11 23:28:51 +02:00
Willy Tarreau
cc1e04b1e8 MINOR: tcp: add new "close" action for tcp-response
This new action immediately closes the connection with the server
when the condition is met. The first such rule executed ends the
rules evaluation. The main purpose of this action is to force a
connection to be finished between a client and a server after an
exchange when the application protocol expects some long time outs
to elapse first. The goal is to eliminate idle connections which
take signifiant resources on servers with certain protocols.
2013-09-11 23:28:51 +02:00
Willy Tarreau
3a925c155d MEDIUM: stick-tables: flush old entries upon soft-stop
When a process with large stick tables is replaced by a new one and remains
present until the last connection finishes, it keeps these data in memory
for nothing since they will never be used anymore by incoming connections,
except during syncing with the new process. This is especially problematic
when dealing with long session protocols such as WebSocket as it becomes
possible to stack many processes and eat a lot of memory.

So the idea here is to know if a table still needs to be synced or not,
and to purge all unused entries once the sync is complete. This means that
after a few hundred milliseconds when everything has been synchronized with
the new process, only a few entries will remain allocated (only the ones
held by sessions during the restart) and all the remaining memory will be
freed.

Note that we carefully do that only after the grace period is expired so as
not to impact a possible proxy that needs to accept a few more connections
before leaving.

Doing this required to add a sync counter to the stick tables, to know how
many peer sync sessions are still in progress in order not to flush the entries
until all synchronizations are completed.
2013-09-04 17:54:01 +02:00
Willy Tarreau
95742a43aa BUG/MEDIUM: fix broken send_proxy on FreeBSD
David Berard reported that send-proxy was broken on FreeBSD and tracked the
issue to be an error returned by send(). We already had the same issue in
the past in another area which was addressed by the following commit :

   0ea0cf6 BUG: raw_sock: also consider ENOTCONN in addition to EAGAIN

In fact, on Linux send() returns EAGAIN when the connection is not yet
established while other OSes return ENOTCONN. Let's consider ENOTCONN for
send-proxy there as the same as EAGAIN.

David confirmed that this change properly fixed the issue.

Another place was affected as well (health checks with send-proxy), and
was fixed.

This fix does not need any backport since it only affects 1.5.
2013-09-03 09:08:31 +02:00
Evan Broder
be55431f9f MINOR: ssl: Add statement 'verifyhost' to "server" statements
verifyhost allows you to specify a hostname that the remote server's
SSL certificate must match. Connections that don't match will be
closed with an SSL error.
2013-09-01 07:55:49 +02:00
William Lallemand
afeb987c5c BUG/MINOR: log: junk at the end of syslog packet
With a facily of 2 or 1 digit, the send size was wrong and bytes with
unknown value were sent.
The size was calculated using the start of the buffer and not the start
of the data which varies with the number of digits of the facility.

This bug was reported by Samuel Stoller and reported by Lukas Tribus.
2013-08-31 08:02:09 +02:00
William Lallemand
5b7ea3afa1 BUG/MEDIUM: unique_id: junk in log on empty unique_id
When a request fail, the unique_id was allocated but not generated.
The string was not initialized and junk was printed in the log with %ID.

This patch changes the behavior of the unique_id. The unique_id is now
generated when a request failed.

This bug was reported by Patrick Hemmer.
2013-08-31 08:01:14 +02:00
Willy Tarreau
9f09521f2d BUG/MEDIUM: unique_id: HTTP request counter must be unique!
The HTTP request counter is incremented non atomically, which means that
many requests can log the same ID. Let's increment it when it is consumed
so that we avoid this case.

This bug was reported by Patrick Hemmer. It's 1.5-specific and does not
need to be backported.
2013-08-13 17:52:20 +02:00
Willy Tarreau
82ffa39bfd MINOR: config: warn when a server with no specific port uses rdp-cookie
Mathew Levett reported an issue which is a bit nasty and hard to track
down. RDP cookies contain both the IP and the port, and haproxy matches
them exactly. So if a server has no port specified (or a remapped port),
it will never match a port specified in a cookie. Better warn the user
when this is detected.
2013-08-13 17:19:08 +02:00
Willy Tarreau
380110368e MINOR: ssl: use MAXPATHLEN instead of PATH_MAX
Apollon Oikonomopoulos reported a build failure on Hurd where PATH_MAX
is not defined. The only place where it is referenced is ssl_sock.c,
all other places use MAXPATHLEN instead, with a fallback to 128 when
the OS does not define it. So let's switch to MAXPATHLEN as well.
2013-08-13 16:59:39 +02:00
Willy Tarreau
33fba6f78f BUG/MINOR: cli: "clear table" must not kill entries that don't match condition
Mark Brooks reported the following issue :

"My table looks like this -

  0x24a8294: key=192.168.136.10 use=0 exp=1761492 server_id=3
  0x24a8344: key=192.168.136.11 use=0 exp=1761506 server_id=2
  0x24a83f4: key=192.168.136.12 use=0 exp=1761520 server_id=3
  0x24a84a4: key=192.168.136.13 use=0 exp=1761534 server_id=2
  0x24a8554: key=192.168.136.14 use=0 exp=1761548 server_id=3
  0x24a8604: key=192.168.136.15 use=0 exp=1761563 server_id=2
  0x24a86b4: key=192.168.136.16 use=0 exp=1761580 server_id=3
  0x24a8764: key=192.168.136.17 use=0 exp=1761592 server_id=2
  0x24a8814: key=192.168.136.18 use=0 exp=1761607 server_id=3
  0x24a88c4: key=192.168.136.19 use=0 exp=1761622 server_id=2
  0x24a8974: key=192.168.136.20 use=0 exp=1761636 server_id=3
  0x24a8a24: key=192.168.136.21 use=0 exp=1761649 server_id=2

im running the command -

  socat unix-connect:/var/run/haproxy.stat stdio <<< 'clear table VIP_Name-2 data.server_id eq 2'

Id assume that the entries with server_id = 2 would be removed but its
removing everything each time."

The cause of the issue is a missing test for skip_entry when deciding
whether to clear the key or not. The test was present when only the
last node is to be removed, so removing only the first node from a
list of two always did the right thing, explaining why it remained
unnoticed in basic unit tests.

The bug was introduced by commit 8fa52f4e which attempted to fix a
previous issue with this feature where only the last node was removed.

This bug is 1.5-specific and does not require any backport.
2013-08-13 16:50:32 +02:00
Godbach
8f9fd2f0a0 BUG/MINOR: use the same check condition for server as other algorithms
Such load balance algorithms as roundrobin, leastconn and first will check the
server after being selected with the following condition:
	if (!s->maxconn || (!s->nbpend && s->served < srv_dynamic_maxconn(s)))

But static-rr uses the different one in map_get_server_rr()  as below:
	if (!srv->maxconn || srv->cur_sess < srv_dynamic_maxconn(srv))
After viewing this difference, it is a better choice for static-rr to use the
same check condition as other algorithms.

This change will only affect static-rr. Though all hash algorithms with type
map-based will use the same server map as static-rr, they call another function
map_get_server_hash() to get server.

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2013-08-13 16:15:08 +02:00
Willy Tarreau
00f0084752 MINOR: payload: allow the payload sample fetches to retrieve arbitrary lengths
When using req.payload and res.payload to look up for specific content at an
arbitrary location, we're often facing the problem of not knowing the input
buffer length. If the length argument is larger than the buffer length, the
function did not match, and if they're smaller, there is a risk of not getting
the expected content. This is especially true when looking for data in SOAP
requests.

So let's make some provisions for scanning the whole buffer by specifying a
length of 0 bytes. This greatly simplifies the processing of random-sized
input data.
2013-08-02 11:07:32 +02:00
Willy Tarreau
47060b6ae0 MINOR: cli: make it possible to enter multiple values at once with "set table"
The "set table" statement allows to create new entries with their respective
values. Till now it was limited to a single data type per line, requiring as
many "set table" statements as the desired data types to be set. Since this
is only a parser limitation, this patch gets rid of it. It also allows the
creation of a key with no data types (all reset to their default values).
2013-08-01 21:17:19 +02:00
Willy Tarreau
ce54d1b9f2 MEDIUM: cli: adjust the method for feeding frequency counters in tables
Since commit 654694e1, it has been possible to feed some data into
stick tables from the CLI. That commit considered that frequency
counters would only have their previous value set, so that they
progressively fade out. But this does not match any real world
use case in fact. The only reason for feeding a freq counter is
to pass some data learned outside. We certainly don't want to see
such data start to vanish immediately, otherwise it will force the
external scripts to loop very frequently to limit the losses.

So let's set the current value instead in order to guarantee that
the data remains stable over the full period, then starts to fade
out between 1* and 2* the period.
2013-08-01 21:17:14 +02:00
Willy Tarreau
0f791d42b6 MEDIUM: counters: support looking up a key in an alternate table
sc_* sample fetches now take an optional parameter which allows to look
the key in an alternate table. This is convenient to pass multiple
information for the same key at once (eg: have multiple gpc0 for the
same key, or support being fed complementary information from the CLI).
Example :

    listen front
        bind :8000
        tcp-request content track-sc0 src table local-ip
        http-response set-header src-id %[sc0_get_gpc0]+%[sc0_get_gpc0(global-ip)]
        server dummy 127.0.0.1:8001

    backend local-ip
        stick-table size 1k type ip store gpc0

    backend global-ip
        stick-table size 1k type ip store gpc0
2013-08-01 21:17:14 +02:00
Willy Tarreau
4d4149cf3e MEDIUM: counters: support passing the counter number as a fetch argument
One very annoying issue when trying to extend the sticky counters beyond
the current 3 counters is that it requires a massive copy-paste of fetch
functions (we don't have to copy-paste code anymore), just so that the
fetch names exist.

So let's have an alternate form like "sc_*(num)" to allow passing the
counter number as an argument without having to redefine new fetch names.
The MAX_SESS_STKCTR macro defines the number of usable sticky counters,
which defaults to 3.
2013-08-01 21:17:14 +02:00
Willy Tarreau
b4c8493a9f MINOR: session: make the number of stick counter entries more configurable
In preparation of more flexibility in the stick counters, make their
number configurable. It still defaults to 3 which is the minimum
accepted value. Changing the value alone is not sufficient to get
more counters, some bitfields still need to be updated and the TCP
actions need to be updated as well, but this update tries to be
easier, which is nice for experimentation purposes.
2013-08-01 21:17:14 +02:00
Willy Tarreau
563eef4e30 MEDIUM: counters: factor out smp_fetch_sc*_trackers
smp_fetch_sc0_trackers, smp_fetch_sc1_trackers and smp_fetch_sc2_trackers
were merged into a single function which relies on the fetch name to decide
what to return.

This is also a bug fix for this feature which has never worked till its bogus
introduction by commit "2406db4 MEDIUM: counters: add sc1_trackers/sc2_trackers"
(1.5-dev10).

Instead of returning the value in the sample, it was returned as the fetch
result!

There is no need to backport this fix anyway since it's 1.5-specific and
nobody uses the feature.
2013-08-01 21:17:14 +02:00
Willy Tarreau
a0b68eddef MEDIUM: counters: factor out smp_fetch_sc*_bytes_out_rate
smp_fetch_sc0_bytes_out_rate, smp_fetch_sc1_bytes_out_rate, smp_fetch_sc2_bytes_out_rate,
smp_fetch_src_bytes_out_rate and smp_fetch_bytes_out_rate were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
53aea10fe9 MEDIUM: counters: factor out smp_fetch_sc*_kbytes_out
smp_fetch_sc0_kbytes_out, smp_fetch_sc1_kbytes_out, smp_fetch_sc2_kbytes_out,
smp_fetch_src_kbytes_out and smp_fetch_kbytes_out were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
613fe99cda MEDIUM: counters: factor out smp_fetch_sc*_bytes_in_rate
smp_fetch_sc0_bytes_in_rate, smp_fetch_sc1_bytes_in_rate, smp_fetch_sc2_bytes_in_rate,
smp_fetch_src_bytes_in_rate and smp_fetch_bytes_in_rate were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
5077d4b261 MEDIUM: counters: factor out smp_fetch_sc*_kbytes_in
smp_fetch_sc0_kbytes_in, smp_fetch_sc1_kbytes_in, smp_fetch_sc2_kbytes_in,
smp_fetch_src_kbytes_in and smp_fetch_kbytes_in were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
9daf262c88 MEDIUM: counters: factor out smp_fetch_sc*_http_err_rate
smp_fetch_sc0_http_err_rate, smp_fetch_sc1_http_err_rate, smp_fetch_sc2_http_err_rate,
smp_fetch_src_http_err_rate and smp_fetch_http_err_rate were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
30d07c3b8e MEDIUM: counters: factor out smp_fetch_sc*_http_err_cnt
smp_fetch_sc0_http_err_cnt, smp_fetch_sc1_http_err_cnt, smp_fetch_sc2_http_err_cnt,
smp_fetch_src_http_err_cnt and smp_fetch_http_err_cnt were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
cf47763c92 MEDIUM: counters: factor out smp_fetch_sc*_http_req_rate
smp_fetch_sc0_http_req_rate, smp_fetch_sc1_http_req_rate, smp_fetch_sc2_http_req_rate,
smp_fetch_src_http_req_rate and smp_fetch_http_req_rate were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
91200da197 MEDIUM: counters: factor out smp_fetch_sc*_http_req_cnt
smp_fetch_sc0_http_req_cnt, smp_fetch_sc1_http_req_cnt, smp_fetch_sc2_http_req_cnt,
smp_fetch_src_http_req_cnt and smp_fetch_http_req_cnt were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:14 +02:00
Willy Tarreau
3a96f3f274 MEDIUM: counters: factor out smp_fetch_sc*_sess_rate
smp_fetch_sc0_sess_rate, smp_fetch_sc1_sess_rate, smp_fetch_sc2_sess_rate,
smp_fetch_src_sess_rate and smp_fetch_sess_rate were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
20843087f5 MEDIUM: counters: factor out smp_fetch_sc*_sess_cnt
smp_fetch_sc0_sess_cnt, smp_fetch_sc1_sess_cnt, smp_fetch_sc2_sess_cnt,
smp_fetch_src_sess_cnt and smp_fetch_sess_cnt were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
f44a553476 MEDIUM: counters: factor out smp_fetch_sc*_conn_cur
smp_fetch_sc0_conn_cur, smp_fetch_sc1_conn_cur, smp_fetch_sc2_conn_cur,
smp_fetch_src_conn_cur and smp_fetch_conn_cur were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
c8c65700de MEDIUM: counters: factor out smp_fetch_sc*_conn_rate
smp_fetch_sc0_conn_rate, smp_fetch_sc1_conn_rate, smp_fetch_sc2_conn_rate,
smp_fetch_src_conn_rate and smp_fetch_conn_rate were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
3b46c5c47d MEDIUM: counters: factor out smp_fetch_sc*_conn_cnt
smp_fetch_sc0_conn_cnt, smp_fetch_sc1_conn_cnt, smp_fetch_sc2_conn_cnt,
smp_fetch_src_conn_cnt and smp_fetch_conn_cnt were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
b9f441d2c0 MEDIUM: counters: factor out smp_fetch_sc*_clr_gpc0
smp_fetch_sc0_clr_gpc0, smp_fetch_sc1_clr_gpc0, smp_fetch_sc2_clr_gpc0,
smp_fetch_src_clr_gpc0 and smp_fetch_clr_gpc0 were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
710d38cea5 MEDIUM: counters: factor out smp_fetch_sc*_inc_gpc0
smp_fetch_sc0_inc_gpc0, smp_fetch_sc1_inc_gpc0, smp_fetch_sc2_inc_gpc0,
smp_fetch_src_inc_gpc0 and smp_fetch_inc_gpc0 were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
b5e0af0b6b MEDIUM: counters: factor out smp_fetch_sc*_gpc0_rate
smp_fetch_sc0_gpc0, smp_fetch_sc1_gpc0, smp_fetch_sc2_gpc0,
smp_fetch_src_gpc0 and smp_fetch_gpc0 were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
30b2046dfe MEDIUM: counters: factor out smp_fetch_sc*_get_gpc0
smp_fetch_sc0_get_gpc0, smp_fetch_sc1_get_gpc0, smp_fetch_sc2_get_gpc0,
smp_fetch_src_get_gpc0 and smp_fetch_get_gpc0 were merged into a single
function which relies on the fetch name to decide what to return.
2013-08-01 21:17:13 +02:00
Willy Tarreau
a65536ca4e MINOR: counters: provide a generic function to retrieve a stkctr for sc* and src.
This function aims at simplifying the prefetching of the table and entry
when using any of the session counters fetches. The principle is that the
src_* variant produces a stkctr that is used instead of the one from the
session. That way we can call the same function from all session counter
fetch functions and always have a single function to support sc[0-9]_/src_.
2013-08-01 21:17:13 +02:00
Willy Tarreau
88821241d4 MINOR: counters: factor out smp_fetch_sc*_tracked
The new function makes use of the sc# in the keyword to
get the counter ID.
2013-08-01 21:17:13 +02:00
Willy Tarreau
cadd8c9ec3 MINOR: payload: split smp_fetch_rdp_cookie()
This function is also called directly from backend.c, so let's stop
building fake args to call it as a sample fetch, and have a lower
layer more generic function instead.
2013-08-01 21:17:13 +02:00
Willy Tarreau
ef38c39287 MEDIUM: sample: systematically pass the keyword pointer to the keyword
We're having a lot of duplicate code just because of minor variants between
fetch functions that could be dealt with if the functions had the pointer to
the original keyword, so let's pass it as the last argument. An earlier
version used to pass a pointer to the sample_fetch element, but this is not
the best solution for two reasons :
  - fetch functions will solely rely on the keyword string
  - some other smp_fetch_* users do not have the pointer to the original
    keyword and were forced to pass NULL.

So finally we're passing a pointer to the keyword as a const char *, which
perfectly fits the original purpose.
2013-08-01 21:17:13 +02:00
Willy Tarreau
276fae9ab9 MINOR: samples: add the http_date([<offset>]) sample converter.
Converts an integer supposed to contain a date since epoch to
a string representing this date in a format suitable for use
in HTTP header fields. If an offset value is specified, then
it is a number of seconds that is added to the date before the
conversion is operated. This is particularly useful to emit
Date header fields, Expires values in responses when combined
with a positive offset, or Last-Modified values when the
offset is negative.
2013-07-25 15:00:38 +02:00
Willy Tarreau
6236d3abe4 MINOR: sample: add a new "date" fetch to return the current date
Returns the current date as the epoch (number of seconds since 01/01/1970).
If an offset value is specified, then it is a number of seconds that is added
to the current date before returning the value. This is particularly useful
to compute relative dates, as both positive and negative offsets are allowed.
2013-07-25 15:00:37 +02:00