Commit Graph

251 Commits

Author SHA1 Message Date
Willy Tarreau
19d14ef104 MEDIUM: make the trash be a chunk instead of a char *
The trash is used everywhere to store the results of temporary strings
built out of s(n)printf, or as a storage for a chunk when chunks are
needed.

Using global.tune.bufsize is not the most convenient thing either.

So let's replace trash with a chunk and directly use it as such. We can
then use trash.size as the natural way to get its size, and get rid of
many intermediary chunks that were previously used.

The patch is huge because it touches many areas but it makes the code
a lot more clear and even outlines places where trash was used without
being that obvious.
2012-10-29 16:57:30 +01:00
Willy Tarreau
c919dc66a3 CLEANUP: remove trashlen
trashlen is a copy of global.tune.bufsize, so let's stop using it as
a duplicate, fall back to the original bufsize, it's less confusing
this way.
2012-10-26 20:04:27 +02:00
Cyril Bonté
4c01beb64b BUG/MEDIUM: acls using IPv6 subnets patterns incorrectly match IPs
Some tests revealed that IPs not in the range of IPv6 subnets incorrectly
matched (for example "acl BUG src 2804::/16" applied to a src IP "127.0.0.1").

This is caused by the acl_match_ip() function applies a mask in host byte
order, whereas it should be in network byte order.
2012-10-24 01:00:53 +02:00
Emeric Brun
07ca496ea9 MINOR: acl: add parse and match primitives to use binary type on ACLs
Binary ACL match patterns can now be entered as hex digit strings.
2012-10-22 18:54:24 +02:00
Willy Tarreau
2e845be249 MEDIUM: sample: pass an empty list instead of a null for fetch args
ACL and sample fetches use args list and it is really not convenient to
check for null args everywhere. Now for empty args we pass a constant
list of end of lists. It will allow us to remove many useless checks.
2012-10-19 19:49:09 +02:00
Willy Tarreau
9b28e03b66 MAJOR: channel: replace the struct buffer with a pointer to a buffer
With this commit, we now separate the channel from the buffer. This will
allow us to replace buffers on the fly without touching the channel. Since
nobody is supposed to keep a reference to a buffer anymore, doing so is not
a problem and will also permit some copy-less data manipulation.

Interestingly, these changes have shown a 2% performance increase on some
workloads, probably due to a better cache placement of data.
2012-10-13 09:07:52 +02:00
Willy Tarreau
f332af7715 CLEANUP: acl: use 'chn' instead of 'b' to name channel pointers
As with previous patches, this naming is confusing.
2012-10-12 23:58:13 +02:00
Willy Tarreau
eb6cead1de MINOR: standard: make memprintf() support a NULL destination
Doing so removes many checks that were systematically made because
the callees don't know if the caller passed a valid pointer.
2012-09-24 10:53:16 +02:00
Willy Tarreau
c7e4238df0 REORG: buffers: split buffers into chunk,buffer,channel
Many parts of the channel definition still make use of the "buffer" word.
2012-09-03 20:47:32 +02:00
Willy Tarreau
572bf9095d REORG/MAJOR: extract "struct buffer" from "struct channel"
At the moment, the struct is still embedded into the struct channel, but
all the functions have been updated to use struct buffer only when possible,
otherwise struct channel. Some functions would likely need to be splitted
between a buffer-layer primitive and a channel-layer function.

Later the buffer should become a pointer in the struct buffer, but doing so
requires a few changes to the buffer allocation calls.
2012-09-02 21:54:56 +02:00
Willy Tarreau
7421efb85f REORG/MAJOR: use "struct channel" instead of "struct buffer"
This is a massive rename. We'll then split channel and buffer.

This change needs a lot of cleanups. At many locations, the parameter
or variable is still called "buf" which will become ambiguous. Also,
the "struct channel" is still defined in buffers.h.
2012-09-02 21:54:55 +02:00
Willy Tarreau
e3a461118c BUG/MINOR: ACL implicit arguments must be created with unresolved flag
Commit 496aa0 fixed a design issue by adding an "unresolved" flag to the
ACL arguments. Unfortunately this unresolved flag was not set when building
the fake argument some ACL need when using an implicit argument pointing to
the local proxy.

Special thanks to Michael Kearey who reported the issue with a reproducer
and the commit introducing the bug.
2012-06-15 08:02:34 +02:00
Willy Tarreau
496aa0111e BUG/MEDIUM: ensure that unresolved arguments are freed exactly once
When passing arguments to ACLs and samples, some types are stored as
strings then resolved later after config parsing is done. Upon exit,
the arguments need to be freed only if the string was not resolved
yet. At the moment we can encounter double free during deinit()
because some arguments (eg: userlists) are freed once as their own
type and once as a string.

The solution consists in adding an "unresolved" flag to the args to
say whether the value is still held in the <str> part or is final.

This could be debugged thanks to a useful bug report from Sander Klein.
2012-06-01 10:40:52 +02:00
David du Colombier
7af4605ef7 BUG/MAJOR: trash must always be the size of a buffer
Before it was possible to resize the buffers using global.tune.bufsize,
the trash has always been the size of a buffer by design. Unfortunately,
the recent buffer sizing at runtime forgot to adjust the trash, resulting
in it being too short for content rewriting if buffers were enlarged from
the default value.

The bug was encountered in 1.4 so the fix must be backported there.
2012-05-16 14:21:55 +02:00
Willy Tarreau
46b39d0dc6 BUG/MEDIUM: config: don't crash at config load time on invalid userlist names
Cyril Bonté reported that passing an invalid userlist name to
http_auth_group() caused haproxy to crash at load. This was due
to an attempt to use the unresolved userlist pointer later to
resolve auth groups since we report many errors before leaving
now.

This issue does not exist in earlier versions since they immediately
abort on the first error, so no backport is needed.
2012-05-10 23:42:22 +02:00
Willy Tarreau
ceb4ac9c34 MEDIUM: acl: support IPv6 address matching
Make use of the new IPv6 pattern type so that acl_match_ip() knows how to
compare pattern and sample.

IPv6 may be entered in their usual form, with or without a netmask appended.
Only bit counts are accepted for IPv6 netmasks. In order to avoid any risk of
trouble with randomly resolved IP addresses, host names are never allowed in
IPv6 patterns.

HAProxy is also able to match IPv4 addresses with IPv6 addresses in the
following situations :
  - tested address is IPv4, pattern address is IPv4, the match applies
    in IPv4 using the supplied mask if any.
  - tested address is IPv6, pattern address is IPv6, the match applies
    in IPv6 using the supplied mask if any.
  - tested address is IPv6, pattern address is IPv4, the match applies in IPv4
    using the pattern's mask if the IPv6 address matches with 2002:IPV4::,
    ::IPV4 or ::ffff:IPV4, otherwise it fails.
  - tested address is IPv4, pattern address is IPv6, the IPv4 address is first
    converted to IPv6 by prefixing ::ffff: in front of it, then the match is
    applied in IPv6 using the supplied IPv6 mask.
2012-05-08 21:28:14 +02:00
Willy Tarreau
c92ddbc37d MINOR: acl: add types to ACL patterns
We cannot currently match IPv6 addresses in ACL simply because we don't
support types on the patterns. Let's introduce this notion. For now, we
rely on the SMP_TYPES though it doesn't seem like it will last forever
given that some types are not present there (eg: regex, meth). Still it
should be enough to support mixed matchings for most types.

We use the special impossible value SMP_TYPES for types that don't exist
in the SMP_T_* space.
2012-05-08 20:57:21 +02:00
Willy Tarreau
7dcb6480db MEDIUM: acl: extend the pattern parsers to report meaningful errors
By passing the error pointer to all ACL parsers, we can make them report
useful errors and not simply fail.
2012-05-08 20:57:20 +02:00
Willy Tarreau
08ad0b38c4 MINOR: acl: report errors encountered when loading patterns from files
This happens in acl_read_patterns_from_file(). Errors are still incomplete,
parsing functions must be improved to report parsing errors.
2012-05-08 20:57:20 +02:00
Willy Tarreau
b7451bb660 MEDIUM: acl: report parsing errors to the caller
All parsing errors were known but impossible to return. Now by making use
of memprintf(), we're able to build meaningful error messages that the
caller can display.
2012-05-08 20:57:20 +02:00
Willy Tarreau
ae52f06da3 MINOR: acl: add a val_args field to keywords
This will make it possible to delegate argument validating to functions
shared with smp_fetch_*.
2012-05-08 20:57:19 +02:00
Willy Tarreau
7a777edbdf MINOR: acl: set SMP_OPT_ITERATE on fetch functions
This way, fetch functions will be able to tell if they're called for a single
request or as part of a loop. This is important for instance when we use
hdr(foo), because in an ACL this means that all hdr(foo) occurrences must
be checked while in a pattern it means only one of them (eg: last one).
2012-05-08 20:57:18 +02:00
Willy Tarreau
32a6f2e572 MEDIUM: acl/pattern: use the same direction scheme
Patterns were using a bitmask to indicate if request or response was desired
in fetch functions and keywords. ACLs were using a bitmask in fetch keywords
and a single bit in fetch functions. ACLs were also using an ACL_PARTIAL bit
in fetch functions indicating that a non-final fetch was performed, which was
an abuse of the existing direction flag.

The change now consists in using :
  - a capabilities field for fetch keywords => SMP_CAP_REQ/RES to indicate
    if a keyword supports requests, responses, both, etc...
  - an option field for fetch functions to indicate what the caller expects
    (request/response, final/non-final)

The ACL_PARTIAL bit was reversed to get SMP_OPT_FINAL as it's more explicit
to know we're working on a final buffer than on a non-final one.

ACL_DIR_* were removed, as well as PATTERN_FETCH_*. L4 fetches were improved
to support being called on responses too since they're still available.

The <dir> field of all fetch functions was changed to <opt> which is now
unsigned.

The patch is large but mostly made of cosmetic changes to accomodate this, as
almost no logic change happened.
2012-05-08 20:57:17 +02:00
Willy Tarreau
24e32d8c6b MEDIUM: acl: replace acl_expr with args in acl fetch_* functions
Having the args everywhere will make it easier to share fetch functions
between patterns and ACLs. The only place where we could have needed
the expr was in the http_prefetch function which can do well without.
2012-05-08 20:57:16 +02:00
Willy Tarreau
32389b7d04 MEDIUM: acl/pattern: switch rdp_cookie functions stack up-down
Previously, both pattern, backend and persist_rdp_cookie would build fake
ACL expressions to fetch an RDP cookie by calling acl_fetch_rdp_cookie().

Now we switch roles. The RDP cookie fetch function is provided as a sample
fetch function that all others rely on, including ACL. The code is exactly
the same, only the args handling moved from expr->args to args. The code
was moved to proto_tcp.c, but probably that a dedicated file would be more
suited to content handling.
2012-05-08 20:57:16 +02:00
Willy Tarreau
21e5b0e3cb MEDIUM: get rid of SMP_F_READ_ONLY and SMP_F_MUST_FREE
These ones were either unused or improperly used. Some integers were marked
read-only, which does not make much sense. Buffers are not read-only, they're
"constant" in that they must be kept intact after any possible change.
2012-05-08 20:57:15 +02:00
Willy Tarreau
197e10aaae MEDIUM: acl: get rid of the SET_RES flags
We now simply rely on a boolean result from a fetch to declare a match.
Booleans are not compared against patterns, they fix the result.
2012-05-08 20:57:15 +02:00
Willy Tarreau
f853c46bc3 MEDIUM: pattern/acl: get rid of temp_pattern in ACLs
This one is not needed anymore as we can return the data and its type in the
sample provided by the caller. ACLs now always return the proper type. BOOL
is already returned when the result is expected to be processed as a boolean.

temp_pattern has been unexported now.
2012-05-08 20:57:14 +02:00
Willy Tarreau
3740635b88 MAJOR: acl: make use of the new sample struct and get rid of acl_test
This change is invasive in lines of code but not much in terms of
functionalities as it's mainly a replacement of struct acl_test
with struct sample.
2012-05-08 20:57:14 +02:00
Willy Tarreau
422aa0792d MEDIUM: pattern: add new sample types to replace pattern types
The new sample types are necessary for the acl-pattern convergence.
These types are boolean and signed int. Some types were renamed for
less ambiguity (ip->ipv4, integer->uint).
2012-05-08 20:57:14 +02:00
Willy Tarreau
8f7406e9b4 MEDIUM: acl: remove the ACL_TEST_F_NULL_MATCH flag
This flag was used to force a boolean match even if there was no pattern
to match. It was used only by http_auth() and designed only for this one.
It's easier and cleaner to make the fetch function perform the test and
report the boolean result as a few other functions already do. It simplifies
the acl_exec_cond() logic and will help merging ACLs and patterns.
2012-05-08 20:57:13 +02:00
Willy Tarreau
fc2c1fd449 MAJOR: acl: ensure that implicit table and proxies are valid
A large number of ACLs make use of frontend, backend or table names in their
arguments, and fall back to the current proxy when no argument is passed. If
the expected capability is not available, the ACL silently fails at runtime.

Now we make all those names mandatory in the parser and we rely on
acl_find_targets() to replace the missing names with the holding proxy,
then to perform the appropriate tests, and to reject errors at parsing
time.

It is possible that some faulty configurations will get rejected from now
on, while they used to silently fail till now. This is the reason why this
change is marked as MAJOR.
2012-05-08 20:57:12 +02:00
Willy Tarreau
d28c353fc5 MAJOR: acl: make acl_find_targets also resolve proxy names at config time
Proxy names are now resolved when the config is parsed and not at runtime.
This means that errors will be caught for real instead of having an ACL
silently never match. Another benefit is that the fetch will be much faster
since the lookup will not have to be performed anymore, eg for all ACLs
based on explicitly named stick-tables.

However some buggy configurations which used to silently fail in the past
will now refuse to load, hence the MAJOR tag.
2012-05-08 20:57:11 +02:00
Willy Tarreau
63364eed75 MEDIUM: acl: acl_find_target() now resolves arguments based on their types
This function does not rely on the keyword anymore but just on its type.
It's much cleaner and much safer. It should be extended to do the same for
all PRX type arguments.
2012-05-08 20:57:11 +02:00
Willy Tarreau
61612d49a7 MAJOR: acl: store the ACL argument types in the ACL keyword declaration
The types and minimal number of ACL keyword arguments are now stored in
their declaration. This will allow many more fantasies if some ACL use
several arguments or types.

Doing so required to rework all ACL keyword declarations to add two
parameters. So this was a good opportunity for a general cleanup and
to sort all entries in alphabetical order.

We still have two pending issues :
  - parse_acl_expr() checks for errors but has no way to report them to
    the user ;
  - the types of some arguments are still not resolved and kept as strings
    (eg: ARGT_FE/BE/TAB) for compatibility reasons, which must be resolved
    in acl_find_targets()
2012-05-08 20:57:11 +02:00
Willy Tarreau
34db108423 MAJOR: acl: make use of the new argument parsing framework
The ACL parser now uses the argument parser to build a typed argument list.
Right now arguments are all strings and only one argument is supported since
this is what ACLs currently support.
2012-05-08 20:57:11 +02:00
Willy Tarreau
89fa706d39 MAJOR: buffers: replace buf->w with buf->p - buf->o
This change introduces the buffer's base pointer, which is the limit between
incoming and outgoing data. It's the point where the parsing should start
from. A number of computations have already been greatly simplified, but
more simplifications are expected to come from the removal of buf->r.

The changes appear good and have revealed occasional improper use of some
pointers. It is possible that this patch has introduced bugs or revealed
some, although preliminary testings tend to indicate that everything still
works as it should.
2012-05-08 12:28:10 +02:00
Willy Tarreau
02d6cfc1d7 MAJOR: buffer: replace buf->l with buf->{o+i}
We don't have buf->l anymore. We have buf->i for pending data and
the total length is retrieved by adding buf->o. Some computation
already become simpler.

Despite extreme care, bugs are not excluded.

It's worth noting that msg->err_pos as set by HTTP request/response
analysers becomes relative to pending data and not to the beginning
of the buffer. This has not been completed yet so differences might
occur when outgoing data are left in the buffer.
2012-05-08 12:28:10 +02:00
Willy Tarreau
d017f113c0 BUG/MINOR: acl: req_ssl_sni would randomly fail if a session ID is present
The wrong byte was checked for the session_id length in the payload. This
used to work when the session ID was absent because zero was found there,
but when a session ID is present, there is 1/256 chance that the inspected
data contains 0x20 (the actual session ID length), so it fails.

Thanks to Emmanuel Bézagu for reporting this bug.

This bug does not need backporting, it is 1.5 specific.
2012-04-09 09:24:11 +02:00
Willy Tarreau
62e7c7146e CLEANUP: silence signedness warning in acl.c
The recent SNI patch introduced a trivial warning in acl.c.
2012-03-10 09:05:30 +01:00
Willy Tarreau
664092ccc1 MEDIUM: acl: use temp_pattern to store any string-type information
Now strings and data blocks are stored in the temp_pattern's chunk
and matched against this one.

The rdp_cookie currently makes extensive use of acl_fetch_rdp_cookie()
and will be a good candidate for the initial rework so that ACLs use
the patterns framework and not the other way around.
2011-12-30 17:33:26 +01:00
Willy Tarreau
f4362b3e3b MEDIUM: acl: use temp_pattern to store any address-type information
IPv4 and IPv6 addresses are now stored into temp_pattern instead of
the dirty hack consisting into storing them into the consumer's target
address.

Some refactoring should now be possible since the methods used to fetch
source and destination addresses are similar between patterns and ACLs.
2011-12-30 17:33:26 +01:00
Willy Tarreau
a5e375646c MEDIUM: acl: use temp_pattern to store any integer-type information
All ACL fetches which return integer value now store the result into
the temporary pattern struct. All ACL matches which rely on integer
also get their value there.

Note: the pattern data types are not set right now.
2011-12-30 17:33:26 +01:00
Willy Tarreau
b6672b547a MINOR: acl: add support for TLS server name matching using SNI
Server Name Indication (SNI) is a TLS extension which makes a client
present the name of the server it is connecting to in the client hello.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.

The new ACL "req_ssl_sni" matches the name extracted from the TLS
handshake against a list of names which may be loaded from a file if
needed.
2011-12-12 17:26:23 +01:00
Willy Tarreau
0e69854ed4 MINOR: acl: add new matches for header/path/url length
This patch introduces hdr_len, path_len and url_len for matching these
respective parts lengths against integers. This can be used to detect
abuse or empty headers.
2011-09-16 08:32:32 +02:00
Finn Arne Gangstad
e8c7ecc2dd [MINOR] http: *_dom matching header functions now also split on ":"
*_dom is mostly used for matching Host headers, and host headers may
include port numbers. To avoid having to create multiple rules with
and without :<port-number> in hdr_dom rules, change the *_dom
matching functions to also handle : as a delimiter.

Typically there are rules like this in haproxy.cfg:

  acl is_foo  hdr_dom(host) www.foo.com

Most clients send "Host: www.foo.com" in their HTTP header, but some
send "Host: www.foo.com:80" (which is allowed), and the above
rule will now work for those clients as well.

[Note: patch was edited before merge, any unexpected bug is mine /willy]
2011-09-09 16:10:12 +02:00
Hervé COMMOWICK
daa824e513 [MINOR] acl: add srv_conn acl to count connections on a
specific backend server

These ACLs are used to check the number of active connections on the specified server in the specified backend.
2011-08-06 15:52:27 +02:00
Willy Tarreau
dfd7fca26c [BUG] config: don't crash on empty pattern files.
Both Hank A. Paulson and Rob at pixsense reported a crash when
loading ACLs from a pattern file which contains empty lines.

From the tests, it appears that only files that contain nothing
but empty lines are causing that (in the past they would have had
their line feeds loaded as patterns).

The crash happens in the free_pattern() call which doesn't like to
be called with a NULL pattern. Let's make it accept it so that it's
more in line with the standard uses of free() which ignores NULLs.
2011-03-09 10:22:30 +01:00
Willy Tarreau
6a8097f034 [BUG] acl: fd leak when reading patterns from file
The fd is not closed after patterns have successfully been read from
a file. Bug reported by Bertrand Jacquin. Should be backported to 1.4.
2011-02-26 15:14:15 +01:00
Willy Tarreau
5109196275 [BUG] acl: fix handling of empty lines in pattern files
Gabriel Sosa reported that haproxy unexpectedly reports an error
when a pattern file loaded by an ACL contains an empty line. The
test was present but inefficient as it did not consider the '\n'
as the end of the line. This fix relies on the line length instead.

It should be backported to 1.4.
2011-01-03 21:06:32 +01:00
Emeric Brun
38e7176961 [MINOR] new acls fetch req_ssl_hello_type and rep_ssl_hello_type 2010-11-11 09:28:55 +01:00
Willy Tarreau
1c7cc5bf95 [MEDIUM] acl: make use of get_std_op() to parse intger ranges
Using the common operator parser for the ACLs saves about 1.5 kB of code.
2010-08-10 14:03:40 +02:00
Willy Tarreau
44b90cc4d8 [CLEANUP] tcp: move some non tcp-specific layer6 processing out of proto_tcp
Some functions which act on generic buffer contents without being
tcp-specific were historically in proto_tcp.c. This concerns ACLs
and RDP cookies. Those have been moved away to more appropriate
locations. Ideally we should create some new files for each layer6
protocol parser. Let's do that later.
2010-06-14 10:53:09 +02:00
Willy Tarreau
06457871a4 [CLEANUP] acl: use 'L6' instead of 'L4' in ACL flags relying on contents
Just like we do on health checks, we should consider that ACLs that make
use of buffer data are layer 6 and not layer 4, because we'll soon have
to distinguish between pure layer 4 ACLs (without any buffer) and these
ones.
2010-06-14 10:53:09 +02:00
Willy Tarreau
0b1cd94c8b [MINOR] acl: add srv_is_up() to check that a specific server is up or not
This ACL was missing in complex setups where the status of a remote site
has to be considered in switching decisions. Until there, using a server's
status in an ACL required to have a dedicated backend, which is a bit heavy
when multiple servers have to be monitored.
2010-05-16 22:18:27 +02:00
Willy Tarreau
020534d6f7 [CLEANUP] acl: make use of eb_is_empty() instead of open coding the tree's emptiness test
Since ebtree 6.0.1, we now have eb_is_empty() which is cleaner and safer to
use than checking the left pointer of the tree, so let's use that.
2010-05-16 21:45:45 +02:00
Willy Tarreau
58215a01af [MINOR] acl: ignore empty lines and comments in pattern files
Most often, pattern files used by ACLs will be produced by tools
which emit some comments (eg: geolocation lists). It's very annoying
to have to clean the files before using them, and it does not make
much sense to be able to support patterns we already can't input in
the config file. So this patch makes the pattern file loader skip
lines beginning with a sharp and the empty ones, and strips leading
spaces and tabs.
2010-05-13 22:10:02 +02:00
Willy Tarreau
b337b532de [MEDIUM] acl: add tree-based lookups of networks
Networks patterns loaded from files for longest match ACL testing
will now be arranged into a prefix tree. This is possible thanks to
the new prefix features in ebtree v6.0. Longest match testing is
slightly slower than exact data maching. However, the measured impact
of running at 42000 requests per second and testing whether the IP
address found in a header belongs to a list of 52000 networks or
not is 3% CPU (increase from 66% to 69%). This is low enough to
permit true geolocation based on huge tables.
2010-05-13 21:37:50 +02:00
Willy Tarreau
c4262961f8 [MEDIUM] acl: add tree-based lookups of exact strings
Now if some ACL patterns are loaded from a file and the operation is
an exact string match, the data will be arranged in a tree, yielding
a significant performance boost on large data sets. Note that this
only works when case is sensitive.

A new dedicated function, acl_lookup_str(), has been created for this
matching. It is called for every possible input data to test and it
looks the tree up for the data. Since the keywords are loosely typed,
we would have had to add a new columns to all keywords to adjust the
function depending on the type. Instead, we just compare on the match
function. We call acl_lookup_str() when we could use acl_match_str().
The tree lookup is performed first, then the remaining patterns are
attempted if the tree returned nothing.

A quick test shows that when matching a header against a list of 52000
network names, haproxy uses 68% of one core on a core2-duo 3.2 GHz at
42000 requests per second, versus 66% without any rule, which means
only a 2% CPU increase for 52000 rules. Doing the same test without
the tree leads to 100% CPU at 6900 requests/s. Also it was possible
to run the same test at full speed with about 50 sets of 52000 rules
without any measurable performance drop.
2010-05-13 21:37:45 +02:00
Willy Tarreau
e56cda9a6a [MEDIUM] acl: add ability to insert patterns in trees
The code is now ready to support loading pattern from filesinto trees. For
that, it will be required that the ACL keyword has a flag ACL_MAY_LOOKUP
and that the expr is case sensitive. When that is true, the pattern will
have a flag ACL_PAT_F_TREE_OK to indicate that it is possible to feed the
tree instead of a usual pattern if the parsing function is able to do this.
The tree's root is pre-initialized in the pattern's value so that the
function can easily find it. At that point, if the parsing function decides
to use the tree, it just sets ACL_PAT_F_TREE in the return flags so that
the caller knows the tree has been used and the pattern can be recycled.

That way it will be possible to load some patterns into the tree when it
is compatible, and other ones as linear linked lists. A good example of
this might be IPv4 network entries : right now we support holes in masks,
but this very rare feature is not compatible with binary lookup in trees.
So the parser will be able to decide itself whether the pattern can go to
the tree or not.
2010-05-13 21:37:41 +02:00
Willy Tarreau
2b5285da33 [MINOR] acl: support loading values from files
The "acl XXX -f <file>" syntax was supported but nothing was read from
the file. This is now possible. All lines are merged verbatim, even if
they contain spaces (useful for user-agents). There are shortcomings
though. The worst one is that error reporting is too approximative.
2010-05-09 23:45:24 +02:00
Willy Tarreau
2a56c5e1c3 [BUG] don't merge anonymous ACLs !
The new anonymous ACL feature was buggy. If several ones are
declared, the first rule is always matched because all of them
share the same internal name (".noname"). Now we simply declare
them with an empty name and ensure that we disable any merging
when the name is empty.
2010-03-15 16:13:29 +01:00
Willy Tarreau
95fa4698f1 [MEDIUM] add support for anonymous ACLs
Anonymous ACLs allow the declaration of rules which rely directly on
ACL expressions without passing via the declaration of an ACL. Example :

   With named ACLs :

        acl site_dead nbsrv(dynamic) lt 2
        acl site_dead nbsrv(static)  lt 2
        monitor fail  if site_dead

   With anonymous ACLs :

        monitor fail if { nbsrv(dynamic) lt 2 } || { nbsrv(static) lt 2 }
2010-02-01 13:05:50 +01:00
Krzysztof Piotr Oledzki
8c8bd4593c [MAJOR] use the new auth framework for http stats
Support the new syntax (http-request allow/deny/auth) in
http stats.

Now it is possible to use the same syntax is the same like in
the frontend/backend http-request access control:
 acl src_nagios src 192.168.66.66
 acl stats_auth_ok http_auth(L1)

 stats http-request allow if src_nagios
 stats http-request allow if stats_auth_ok
 stats http-request auth realm LB

The old syntax is still supported, but now it is emulated
via private acls and an aditional userlist.
2010-01-31 19:14:09 +01:00
Krzysztof Piotr Oledzki
f9423ae43a [MINOR] acl: add http_auth and http_auth_group
Add two acls to match http auth data:
 acl <name> http_auth(userlist)
 acl <name> http_auth_hroup(userlist) group1 group2 (...)
2010-01-31 19:14:09 +01:00
Krzysztof Piotr Oledzki
d7528e5a60 [MINOR] add ACL_TEST_F_NULL_MATCH
Add ACL_TEST_F_NULL_MATCH so expr->kw->match can be called
even if expr->patterns is empty.
2010-01-31 19:14:07 +01:00
Willy Tarreau
f1e98b8628 [CLEANUP] config: use warnif_cond_requires_resp() to check for bad ACLs
Factor out some repetitive copy-pasted code to check for request ACLs
validity.
2010-01-28 17:59:39 +01:00
Willy Tarreau
2bbba415d7 [MINOR] acl: add build_acl_cond() to make it easier to add ACLs in config
This function automatically builds a rule, considering the if/unless
statements, and automatically updates the proxy's acl_requires, the
condition's file and line.
2010-01-28 16:48:33 +01:00
Willy Tarreau
1edb144b8d [BUG] restore accidentely removed line in last patch !
The test condition for the acl expression validity got dropped, and
only the error remained. Error reported by Krzysztof Oledzki.
2010-01-27 20:13:38 +01:00
Willy Tarreau
ac778f5ade [CLEANUP] acl, patterns: make use of my_strndup() instead of malloc+memcpy
This is simpler and more readable.
2010-01-26 19:02:46 +01:00
Krzysztof Piotr Oledzki
4cdd8314e9 [MINOR] acl: don't report valid acls as potential mistakes
Commit 404e8ab461 introduced
smart checking for stupid acl typos. However, now haproxy shows
the warning even for valid acls, like this one:
	acl Cookie-X-NoAccel    hdr_reg(cookie) (^|\ |;)X-NoAccel=1(;|$)
2009-10-08 07:00:07 +02:00
Willy Tarreau
404e8ab461 [MINOR] acl: detect and report potential mistakes in ACLs
I've discovered a configuration with lots of occurrences of the
following :

    acl xxx hdr_beg (host) xxx

The problem is that hdr_beg will match every header against patterns
(host) and xxx due to the space between both, which certainly is not
what the user wanted. Now we detect such ACLs and report a warning
with a suggestion to add "--" between "hdr_beg" and "(host)" if this
is definitely what is wanted.
2009-07-26 19:40:40 +02:00
Emeric Brun
bede3d0ef4 [MINOR] acl: add support for matching of RDP cookies
The RDP protocol is quite simple and documented, which permits
an easy detection and extraction of cookies. It can be useful
to match the MSTS cookie which can contain the username specified
by the client.
2009-07-14 12:50:39 +02:00
Willy Tarreau
a55b7dc528 [BUG] default ACLs did not properly set the ->requires flag
This bug caused session initialization not to always allocate
the hdr_idx struct for default ACLs involving HTTP content.
2009-07-12 09:21:30 +02:00
Willy Tarreau
2492d5b4d6 [MINOR] acl: add HTTP protocol detection (req_proto_http)
Now that we can perform TCP-based content switching, it makes sense
to be able to detect HTTP traffic and act accordingly. We already
have an HTTP decoder, we just have to call it in order to detect HTTP
protocol. Note that since the decoder will automatically fill in the
interesting fields of the HTTP transaction, it would make sense to
use this parsing to extend HTTP matching to TCP.
2009-07-12 08:06:20 +02:00
Willy Tarreau
dd64f8d394 [MEDIUM] acl: when possible, report the name and requirements of ACLs in warnings
When an ACL is referenced at a wrong place (eg: response during request, layer7
during layer4), try to indicate precisely the name and requirements of this ACL.

Only the first faulty ACL is returned. A small change consisting in iterating
that way may improve reports :
   cap = ACL_USE_any_unexpected
   while ((acl=cond_find_require(cond, cap))) {
     warning()
     cap &= ~acl->requires;
   }

This will report the first ACL of each unsupported type. But doing so will
mangle the error reporting a lot, so we need to rework error reports first.
2008-08-03 09:41:05 +02:00
Willy Tarreau
0ceba5af74 [MEDIUM] acl: set types on all currently known ACL verbs
All currently known ACL verbs have been assigned a type which makes
it possible to detect inconsistencies, such as response values used
in request rules.
2008-07-25 19:31:03 +02:00
Willy Tarreau
a9802633d8 [MEDIUM] acl: enforce ACL type checking
ACL now hold information on the availability of the data they rely
on. They can indicate which parts of the requests/responses they
require, and the rules parser may now report inconsistencies.

As an example, switching rules are now checked for response-specific
ACLs, though those are not still set. A warning is reported in case
of mismatch. ACLs keyword restrictions will now have to be specifically
set wherever a better control is expected.

The line number where an ACL condition is declared has been added to
the conditions in order to be able to report the faulty line number
during post-loading checks.
2008-07-25 19:13:19 +02:00
Willy Tarreau
b6fb420c7e [MINOR] acl: add the "wait_end" acl verb
The new "wait_end" acl delays evaluation of the rule (and the next ones)
to the end of the analysis period. This is intented to be used with TCP
content analysis. A rule referencing such an ACL will not match until
the delay is over. An equivalent default ACL "WAIT_END" has been created.
2008-07-20 11:18:28 +02:00
Willy Tarreau
58393e103f [MEDIUM] acl: get rid of dummy values in always_true/always_false
make use of last change in order to get rid of dummy values in
always_true/always_false.
2008-07-20 10:39:22 +02:00
Willy Tarreau
a79534fce1 [MEDIUM] acl: permit fetch() functions to set the result themselves
For protocol analysis, it's not always convenient to have to run through
a fetch then a match against dummy values. It's easier to let the fetch()
function set the result itself. This obviously works only for boolean
values.
2008-07-20 10:17:20 +02:00
Willy Tarreau
c6317703ce [MINOR] acl: add REQ_CONTENT to the list of default acls
With content inspection, checking the presence of data in the
request buffer is very important. It's getting boring to always
add such an ACL, so let's add it by default.
2008-07-20 09:29:50 +02:00
Willy Tarreau
ec6c5df018 [CLEANUP] remove many #include <types/xxx> from C files
It should be stated as a rule that a C file should never
include types/xxx.h when proto/xxx.h exists, as it gives
less exposure to declaration conflicts (one of which was
caught and fixed here) and it complicates the file headers
for nothing.

Only types/global.h, types/capture.h and types/polling.h
have been found to be valid includes from C files.
2008-07-16 10:30:42 +02:00
Willy Tarreau
4a26d2f2fa [MINOR] acl: add a new parsing function: parse_dotted_ver
This new function supports one major and one minor and makes an int of them.
It is very convenient to compare versions (eg: SSL) just as if they were plain
integers, as the comparison functions will still be based on integers.
2008-07-16 10:29:51 +02:00
Willy Tarreau
b686644ad8 [MAJOR] implement tcp request content inspection
Some people need to inspect contents of TCP requests before
deciding to forward a connection or not. A future extension
of this demand might consist in selecting a server farm
depending on the protocol detected in the request.

For this reason, a new state CL_STINSPECT has been added on
the client side. It is immediately entered upon accept() if
the statement "tcp-request inspect-delay <xxx>" is found in
the frontend configuration. Haproxy will then wait up to
this amount of time trying to find a matching ACL, and will
either accept or reject the connection depending on the
"tcp-request content <action> {if|unless}" rules, where
<action> is either "accept" or "reject".

Note that it only waits that long if no definitive verdict
can be found earlier. That generally implies calling a fetch()
function which does not have enough information to decode
some contents, or a match() function which only finds the
beginning of what it's looking for.

It is only at the ACL level that partial data may be processed
as such, because we need to distinguish between MISS and FAIL
*before* applying the term negation.

Thus it is enough to add "| ACL_PARTIAL" to the last argument
when calling acl_exec_cond() to indicate that we expect
ACL_PAT_MISS to be returned if some data is missing (for
fetch() or match()). This is the only case we may return
this value. For this reason, the ACL check in process_cli()
has become a lot simpler.

A new ACL "req_len" of type "int" has been added. Right now
it is already possible to drop requests which talk too early
(eg: for SMTP) or which don't talk at all (eg: HTTP/SSL).

Also, the acl fetch() functions have been extended in order
to permit reporting of missing data in case of fetch failure,
using the ACL_TEST_F_MAY_CHANGE flag.

The default behaviour is unchanged, and if no rule matches,
the request is accepted.

As a side effect, all layer 7 fetching functions have been
cleaned up so that they now check for the validity of the
layer 7 pointer before dereferencing it.
2008-07-16 10:29:07 +02:00
Willy Tarreau
11382813a1 [TESTS] added test-acl.cfg to test some ACL combinations
various rules constructions can be tested with this test case.
2008-07-09 16:18:21 +02:00
Krzysztof Piotr Oledzki
8001d6162e [MEDIUM] Fix memory freeing at exit, part 2
- free oldpids
- call free(exp->preg), not only regfree(exp->preg): req_exp, rsp_exp
- build a list of unique uri_auths and eventually free it
- prune_acl_cond/free for switching_rules
- add a callback pointer to free ptr from acl_pattern (used for regexs) and execute it

==1180== malloc/free: in use at exit: 0 bytes in 0 blocks.
==1180== malloc/free: 5,599 allocs, 5,599 frees, 4,220,556 bytes allocated.
==1180== All heap blocks were freed -- no leaks are possible.
2008-06-07 11:06:14 +02:00
Krzysztof Piotr Oledzki
a643baf091 [MEDIUM] Fix memory freeing at exit
New functions implemented:
 - deinit_pollers: called at the end of deinit())
 - prune_acl: called via list_for_each_entry_safe

Add missing pool_destroy2 calls:
 - p->hdr_idx_pool
 - pool2_tree64

Implement all task stopping:
 - health-check: needs new "struct task" in the struct server
 - queue processing: queue_mgt
 - appsess_refresh: appsession_refresh

before (idle system):
==6079== LEAK SUMMARY:
==6079==    definitely lost: 1,112 bytes in 75 blocks.
==6079==    indirectly lost: 53,356 bytes in 2,090 blocks.
==6079==      possibly lost: 52 bytes in 1 blocks.
==6079==    still reachable: 150,996 bytes in 504 blocks.
==6079==         suppressed: 0 bytes in 0 blocks.

after (idle system):
==6945== LEAK SUMMARY:
==6945==    definitely lost: 7,644 bytes in 137 blocks.
==6945==    indirectly lost: 9,913 bytes in 587 blocks.
==6945==      possibly lost: 0 bytes in 0 blocks.
==6945==    still reachable: 0 bytes in 0 blocks.
==6945==         suppressed: 0 bytes in 0 blocks.

before (running system for ~2m):
==9343== LEAK SUMMARY:
==9343==    definitely lost: 1,112 bytes in 75 blocks.
==9343==    indirectly lost: 54,199 bytes in 2,122 blocks.
==9343==      possibly lost: 52 bytes in 1 blocks.
==9343==    still reachable: 151,128 bytes in 509 blocks.
==9343==         suppressed: 0 bytes in 0 blocks.

after (running system for ~2m):
==11616== LEAK SUMMARY:
==11616==    definitely lost: 7,644 bytes in 137 blocks.
==11616==    indirectly lost: 9,981 bytes in 591 blocks.
==11616==      possibly lost: 0 bytes in 0 blocks.
==11616==    still reachable: 4 bytes in 1 blocks.
==11616==         suppressed: 0 bytes in 0 blocks.

Still not perfect but significant improvement.
2008-05-30 07:07:19 +02:00
Willy Tarreau
2e74c3f202 [MEDIUM] restrict the set of allowed characters for identifiers
In order to avoid issues in the future, we want to restrict
the set of allowed characters for identifiers. Starting from
now, only A-Z, a-z, 0-9, '-', '_', '.' and ':' will be allowed
for a proxy, a server or an ACL name.

A test file has been added to check the restriction.
2007-12-02 18:45:09 +01:00
Willy Tarreau
8f8e645066 [CLEANUP] shut warnings 'is*' macros from ctype.h on solaris
Solaris visibly uses an array for is*, which returns warnings
about the use of signed chars as indexes. Good opportunity to
put casts everywhere.
2007-06-17 21:51:38 +02:00
Willy Tarreau
a590983fe5 [MEDIUM] acl: added the TRUE and FALSE ACLs.
Those ACLs are sometimes useful for troubleshooting. Two ACL subjects
"always_true" and "always_false" have been added too. They return what
their subject says for every pattern. Also, acl_match_pst() has been
removed.
2007-06-17 20:40:25 +02:00
Willy Tarreau
16fbe82bfc [MEDIUM] provide default ACLs
The following ACLs are predefined :

  LOCALHOST      = src 127.0.0.1/8
  HTTP_1.0       = req_ver 1.0
  HTTP_1.1       = req_ver 1.1
  METH_CONNECT   = method CONNECT
  METH_GET       = method GET HEAD
  METH_HEAD      = method HEAD
  METH_OPTIONS   = method OPTIONS
  METH_POST      = method POST
  METH_TRACE     = method TRACE
  HTTP_URL_ABS   = url_reg ^[^/:]*://
  HTTP_URL_SLASH = url_beg /
  HTTP_URL_STAR  = url *
  HTTP_CONTENT   = hdr_val(content-length) gt 0
2007-06-17 11:54:31 +02:00
Willy Tarreau
c8d7c96b26 [MEDIUM] acl: support '-i' to ignore case when matching
Implemented the "-i" option on ACLs to state that the matching
will have to be performed for all patterns ignoring case. The
usage is :

   acl <aclname> <aclsubject> -i pattern1 ...

If a pattern must begin with "-", either it must not be the first one,
or the "--" option should be specified first.
2007-06-17 08:20:33 +02:00
Willy Tarreau
74b98a8c22 [BUG] negation in ACL conds was not cleared between terms
The exclamation mark (!) in front of an ACL condition was propagated
to the whole line instead of being flushed after parsing an acl name.
2007-06-16 19:35:18 +02:00
Willy Tarreau
97be145991 [MINOR] acl: provide a reference to the expr to fetch()
The fetch() functions may need to access the full expr to get
their args. Turn the void *arg into a struct acl_expr *expr.
2007-06-10 11:47:14 +02:00
Willy Tarreau
bb76891d0f [MINOR] acl: provide the argument length for fetch functions
Some fetch() functions will require an argument (eg: header).
It's wise to provide the argument size to optimize string
comparisons.
2007-06-10 11:17:01 +02:00
Willy Tarreau
d41f8d85e8 [MINOR] acl: specify the direction during fetches
Some fetches such as 'line' or 'hdr' need to know the direction of
the test (request or response). A new 'dir' parameter is now
propagated from the caller to achieve this.
2007-06-10 10:06:18 +02:00
Willy Tarreau
ae8b796722 [MEDIUM] smarter integer comparison support in ACLs
ACLs now support operators such as 'eq', 'le', 'lt', 'ge' and 'gt'
in order to give more flexibility to the language. Because of this
change, the 'dst_limit' keyword changed to 'dst_conn' and now requires
either a range or a test such as 'dst_conn lt 1000' which is more
understandable.
2007-06-09 23:10:04 +02:00
Willy Tarreau
f3d259868b [MINOR] ACL regex matching on the URI ; uri_reg
The URI can be matched on regexen now. The upcase/lowcase flag
can not be set yet and will soon have to.
2007-05-08 23:24:51 +02:00
Willy Tarreau
a67fad9d68 [MINOR] implement acl_parse_ip and acl_match_ip
The ACL can now compare IP addresses. The client's IP address
can be checked.
2007-05-08 23:24:51 +02:00
Willy Tarreau
a84d374367 [MAJOR] new framework for generic ACL support
This framework offers all other subsystems the ability to register
ACL matching criteria. Some generic matching functions are already
provided. Others will come soon and the framework shall evolve.
2007-05-08 23:24:50 +02:00