Commit Graph

171 Commits

Author SHA1 Message Date
Willy Tarreau
83061a820e MAJOR: chunks: replace struct chunk with struct buffer
Now all the code used to manipulate chunks uses a struct buffer instead.
The functions are still called "chunk*", and some of them will progressively
move to the generic buffer handling code as they are cleaned up.
2018-07-19 16:23:43 +02:00
Willy Tarreau
843b7cbe9d MEDIUM: chunks: make the chunk struct's fields match the buffer struct
Chunks are only a subset of a buffer (a non-wrapping version with no head
offset). Despite this we still carry a lot of duplicated code between
buffers and chunks. Replacing chunks with buffers would significantly
reduce the maintenance efforts. This first patch renames the chunk's
fields to match the name and types used by struct buffers, with the goal
of isolating the code changes from the declaration changes.

Most of the changes were made with spatch using this coccinelle script :

  @rule_d1@
  typedef chunk;
  struct chunk chunk;
  @@
  - chunk.str
  + chunk.area

  @rule_d2@
  typedef chunk;
  struct chunk chunk;
  @@
  - chunk.len
  + chunk.data

  @rule_i1@
  typedef chunk;
  struct chunk *chunk;
  @@
  - chunk->str
  + chunk->area

  @rule_i2@
  typedef chunk;
  struct chunk *chunk;
  @@
  - chunk->len
  + chunk->data

Some minor updates to 3 http functions had to be performed to take size_t
ints instead of ints in order to match the unsigned length here.
2018-07-19 16:23:43 +02:00
William Lallemand
579fb25b62 BUG/MAJOR: map: fix a segfault when using http-request set-map
The bug happens with an existing entry, when you try to overwrite the
value with wrong data, for example, a string when the type is INT.

The code path was not secure and tried to set *err and *merr while
err = merr = NULL when performing an http action.

Must be backported in 1.6, 1.7, 1.8.
2018-06-11 11:02:06 +02:00
Aurélien Nephtali
564d15a71e BUG/MINOR: pattern: Add a missing HA_SPIN_INIT() in pat_ref_newid()
pat_ref_newid() is lacking a spinlock init. It was probably forgotten
in b5997f740b ("MAJOR: threads/map: Make acls/maps thread safe").

Signed-off-by: Aurélien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-19 17:49:48 +02:00
Christopher Faulet
9dcf9b6f03 MINOR: threads: Use __decl_hathreads to declare locks
This macro should be used to declare variables or struct members depending on
the USE_THREAD compile option. It avoids the encapsulation of such declarations
between #ifdef/#endif. It is used to declare all lock variables.
2017-11-13 11:38:17 +01:00
Christopher Faulet
09fdf4b112 BUG/MINOR: pattern: Rely on the sample type to copy it in pattern_exec_match
To be thread safe, the function pattern_exec_match copy data (the pattern and
the inner sample) in thread-local variables. But when the sample is duplicated,
we must check its type and not the pattern one.

This is specific to threads, no backport is needed.
2017-11-09 17:19:20 +01:00
Christopher Faulet
2a944ee16b BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix
This remove any name conflicts, especially on Solaris.
2017-11-07 11:10:24 +01:00
Emeric Brun
b5997f740b MAJOR: threads/map: Make acls/maps thread safe
locks have been added in pat_ref and pattern_expr structures to protect all
accesses to an instance of on of them. Moreover, a global lock has been added to
protect the LRU cache used for pattern matching.

Patterns are now duplicated after a successfull matching, to avoid modification
by other threads when the result is used.

Finally, the function reloading a pattern list has been modified to be
thread-safe.
2017-10-31 13:58:32 +01:00
Emeric Brun
7d27f3c12d BUG/MEDIUM: map/acl: fix unwanted flags inheritance.
The bug: Maps/ACLs using the same file/id can mistakenly inherit
their flags from the last declared one.

i.e.

    $ cat haproxy.conf
    listen mylistener
	mode http
	bind 0.0.0.0:8080

	acl myacl1 url -i -f mine.acl
	acl myacl2 url -f mine.acl
	acl myacl3 url -i -f mine.acl
	redirect location / if myacl2
    $ cat mine.acl
    foobar

Shows an unexpected redirect for request 'GET /FOObAR HTTP/1.0\n\n'.

This fix should be backported on mainline branches v1.6 and v1.7.
2017-07-04 10:45:53 +02:00
Emeric Brun
8d85aa44da BUG/MAJOR: map: fix segfault during 'show map/acl' on cli.
The reference of the current map/acl element to dump could
be destroyed if map is updated from an 'http-request del-map'
configuration rule or throught a 'del map/acl' on CLI.

We use a 'back_refs' chaining element to fix this. As it
is done to dump sessions.

This patch needs also fix:
'BUG/MAJOR: cli: fix custom io_release was crushed by NULL.'

To clean the back_ref and avoid a crash on a further
del/clear map operation.

Those fixes should be backported on mainline branches 1.7 and 1.6.

This patch wont directly apply on 1.6.
2017-06-30 06:49:42 +02:00
Andreas Seltenreich
e6e22e8e90 BUG/MINOR: pattern: Avoid memory leak on out-of-memory condition
pattern_new_expr() failed to free the allocated list element when an
out-of-memory error occurs during initialization of the element.  As
this only happens when loading the configuration file or evaluating
commands via the CLI, it is unlikely for this leak to be relevant
unless the user makes automated, heavy use of the CLI.

Found in HAProxy 1.5.14.
2016-03-13 07:47:25 +01:00
Andreas Seltenreich
f0653192e3 CLEANUP: pattern: Ignore unknown samples in pat_match_ip().
Ignore samples that are neither SMP_T_IPV4 nor SMP_T_IPV6 instead of
matching with an uninitialized value in this case.

This situation should not occur in the current codebase but triggers
warnings in static code analysis tools.

Found in haproxy 1.5.
2016-03-08 12:55:06 +01:00
Thierry Fournier
8feaa661b6 MINOR: map: Add regex matching replacement
This patch declares a new map which provides a string based on
a string with back references replaced by the content matched
by the regex.
2016-02-10 23:38:34 +01:00
peter cai
aede6ddd1f BUG/MEDIUM: pattern: fixup use_after_free in the pat_ref_delete_by_id
I found there is use_after_free bug in the pat_ref_delete_by_id.

[wt: it seems this fix must be backported to 1.5 as well]
2015-10-13 18:31:49 +02:00
Thierry FOURNIER
136f9d34a9 MINOR: samples: rename union from "data" to "u"
The union name "data" is a little bit heavy while we read the source
code because we can read "data.data.sint". The rename from "data" to "u"
makes the read easiest like "data.u.sint".
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
8c542cac07 MEDIUM: samples: Use the "struct sample_data" in the "struct sample"
This patch remove the struct information stored both in the struct
sample_data and in the striuct sample. Now, only thestruct sample_data
contains data, and the struct sample use the struct sample_data for storing
his own data.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
503bb09873 MINOR: samples: rename some struct member from "smp" to "data"
This members contains data and not sample.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
12ba0c29d6 MINOR: samples: rename a struct from sample_storage to sample_data
This a first step of sample reorganization.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
07ee64ef4d MAJOR: sample: converts uint and sint in 64 bits signed integer
This patch removes the 32 bits unsigned integer and the 32 bit signed
integer. It replaces these types by a unique type 64 bit signed.

This makes easy the usage of integer and clarify signed and unsigned use.
With the previous version, signed and unsigned are used ones in place of
others, and sometimes the converter loose the sign. For example, divisions
are processed with "unsigned", if one entry is negative, the result is
wrong.

Note that the integer pattern matching and dotted version pattern matching
are already working with signed 64 bits integer values.

There is one user-visible change : the "uint()" and "sint()" sample fetch
functions which used to return a constant integer have been replaced with
a new more natural, unified "int()" function. These functions were only
introduced in the latest 1.6-dev2 so there's no impact on regular
deployments.
2015-07-22 00:48:23 +02:00
Christopher Faulet
f90ac55d9e MINOR: lru: Add the possibility to free data when an item is removed
Now, When a item is committed in an LRU tree, you can define a function to free
data owned by this item. This function will be called when the item is removed
from the LRU tree or when the tree is destroyed..
2015-06-12 18:06:59 +02:00
Willy Tarreau
aee9314e71 BUILD: pattern: fix build warnings introduced in the LRU cache
They're caused by the cast to long long from ptr in 32-bit.

src/pattern.c: In function 'pat_match_str':
src/pattern.c:479:44: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
2015-05-04 17:18:42 +02:00
Willy Tarreau
f3045d2a06 MAJOR: pattern: add LRU-based cache on pattern matching
The principle of this cache is to have a global cache for all pattern
matching operations which rely on lists (reg, sub, dir, dom, ...). The
input data, the expression and a random seed are used as a hashing key.
The cached entries contains a pointer to the expression and a revision
number for that expression so that we don't accidently used obsolete
data after a pattern update or a very unlikely hash collision.

Regarding the risk of collisions, 10k entries at 10k req/s mean 1% risk
of a collision after 60 years, that's already much less than the memory's
reliability in most machines and more durable than most admin's life
expectancy. A collision will result in a valid result to be returned
for a different entry from the same list. If this is not acceptable,
the cache can be disabled using tune.pattern.cache-size.

A test on a file containing 10k small regex showed that the regex
matching was limited to 6k/s instead of 70k with regular strings.
When enabling the LRU cache, the performance was back to 70k/s.
2015-04-29 19:15:24 +02:00
Willy Tarreau
72f073b6c7 MEDIUM: pattern: add a revision to all pattern expressions
This will be used to detect any change on the pattern list between
two operations, ultimately making it possible to implement a cache
which immediately invalidates obsolete keys after an update. The
revision is simply taken from the timestamp counter to ensure that
even upon a pointer reuse we cannot accidently come back to the
same (expr,revision) tuple.
2015-04-29 19:15:24 +02:00
Thierry FOURNIER
73bc285be1 BUG/MEDIUM: pattern: some entries are not deleted with case insensitive match
ACL or map entries are not deleted with the command "del acl" or "del map"
if the case insentive flag is set.

This is because the the case insensitive string are stored in a list and the
default delete function associated with string looks in a tree. I add a check
of the case insensitive flag and execute the delete function for lists if it
is set.

This patch must be backported in 1.5 version.
2015-02-06 18:04:36 +01:00
Thierry FOURNIER
8aa8384e22 BUG/MINOR: pattern: error message missing
This patch must be backported in 1.5 version.
2015-02-06 18:04:27 +01:00
Willy Tarreau
4deaf39243 BUG/MEDIUM: patterns: previous fix was incomplete
Dmitry Sivachenko <trtrmitya@gmail.com> reported that commit 315ec42
("BUG/MEDIUM: pattern: don't load more than once a pattern list.")
relies on an uninitialised variable in the stack. While it used to
work fine during the tests, if the uninitialized variable is non-null,
some patterns may be aggregated if loaded multiple times, resulting in
slower processing, which was the original issue it tried to address.

The fix needs to be backported to 1.5.
2014-11-26 13:17:03 +01:00
Thierry FOURNIER
315ec4217f BUG/MEDIUM: pattern: don't load more than once a pattern list.
A memory optimization can use the same pattern expression for many
equal pattern list (same parse method, index method and index_smp
method).

The pattern expression is returned by "pattern_new_expr", but this
function dont indicate if the returned pattern is already in use.

So, the caller function reload the list of patterns in addition with
the existing patterns. This behavior is not a problem with tree indexed
pattern, but it grows the lists indexed patterns.

This fix add a "reuse" flag in return of the function "pattern_new_expr".
If the flag is set, I suppose that the patterns are already loaded.

This fix must be backported into 1.5.
2014-11-24 15:40:16 +01:00
Willy Tarreau
5def8ef786 BUG/MINOR: pattern: remove useless allocation of unused trash in pat_parse_reg()
Just like previous patch, this is a remains of an early implementation. Also
fix the outdated comments above. The fix may be backported to 1.5 though the
bug cannot be triggerred, thus it's just a matter of keeping the code clean.
2014-08-29 15:19:33 +02:00
Thierry FOURNIER
b8f980cc19 MINOR: regex: Create JIT compatible function that return match strings
This patchs rename the "regex_exec" to "regex_exec2". It add a new
"regex_exec", "regex_exec_match" and "regex_exec_match2" function. This
function can match regex and return array containing matching parts.
Otherwise, this function use the compiled method (JIT or PCRE or POSIX).

JIT require a subject with length. PCREPOSIX and native POSIX regex
require a null terminted subject. The regex_exec* function are splited
in two version. The first version take a null terminated string, but it
execute strlen() on the subject if it is compiled with JIT. The second
version (terminated by "2") take the subject and the length. This
version adds a null character in the subject if it is compiled with
PCREPOSIX or native POSIX functions.

The documentation of posix regex and pcreposix says that the function
returns 0 if the string matche otherwise it returns REG_NOMATCH. The
REG_NOMATCH macro take the value 1 with posix regex and the value 17
with the pcreposix. The documentaion of the native pcre API (used with
JIT) returns a negative number if no match, otherwise, it returns 0 or a
positive number.

This patch fix also the return codes of the regex_exec* functions. Now,
these function returns true if the string match, otherwise it returns
false.
2014-06-18 15:07:50 +02:00
Dirkjan Bussink
07fcaaa4cd MINOR: fix a few memory usage errors
These are either use after free errors or small leaks where memory
is not free'd after some error state is detected.
2014-05-15 08:06:57 +02:00
Willy Tarreau
b1dd9bf308 MEDIUM: pattern: use ebtree's longest match to index/lookup string beginning
Being able to map prefixes to values is already used for IPv4/IPv6
but was not yet used with strings. It can be very convenient to map
directories to server farms but large lists may be slow.

By using ebmb_insert_prefix() and ebmb_lookup_longest(), we can
insert strings with their own length as a prefix, and lookup
candidate strings and ensure that the longest matching one will
be returned, which is the longest string matching the entry.
2014-05-10 08:53:48 +02:00
Thierry FOURNIER
e47e4e2385 BUG/MEDIUM: patterns: last fix was still not enough
Last fix did address the issue for inlined patterns, but it was not
enough because the flags are lost as well when updating patterns
dynamically over the CLI.

Also if the same file was used once with -i and another time without
-i, their references would have been merged and both would have used
the same matching method.

It's appear that the patterns have two types of flags. The first
ones are relative to the pattern matching, and the second are
relative to the pattern storage. The pattern matching flags are
the same for all the patterns of one expression. Now they are
stored in the expression. The storage flags are information
returned by the pattern mathing function. This information is
relative to each entry and is stored in the "struct pattern".

Now, the expression matching flags are forwarded to the parse
and index functions. These flags are stored during the
configuration parsing, and they are used during the parse and
index actions.

This issue was introduced in dev23 with the major pattern rework,
and is a continuation of commit a631fc8 ("BUG/MAJOR: patterns: -i
and -n are ignored for inlined patterns"). No backport is needed.
2014-04-28 14:19:17 +02:00
Willy Tarreau
a631fc8de8 BUG/MAJOR: patterns: -i and -n are ignored for inlined patterns
These flags are only passed to pattern_read_from_file() which
loads the patterns from a file. The functions used to parse the
patterns from the current line do not provide the means to pass
the pattern flags so they're lost.

This issue was introduced in dev23 with the major pattern rework,
and was reported by Graham Morley. No backport is needed.
2014-04-27 09:21:08 +02:00
Willy Tarreau
3b78696858 BUG/MEDIUM: pattern: a typo breaks automatic acl/map numbering
Dmitry Sivachenko reported that nice warning :
src/pattern.c:2243:43: warning: if statement has empty body [-Wempty-body]
                                if (&ref2->list == &pattern_reference);
                                                                      ^
src/pattern.c:2243:43: note: put the semicolon on a separate line to silence
      this warning

It was merged as is with the code from commit af5a29d ("MINOR: pattern:
Each pattern is identified by unique id").

So it looks like we can reassign an ID which is still in use because of
this.
2014-04-26 12:41:32 +02:00
Baptiste Assmann
953f74d1b3 MINOR: pattern: find element in a reference
This function can be used to look for an entry in either an ACL or a
MAP.
2014-04-25 17:31:13 +02:00
Thierry FOURNIER
46006bde3c MINOR: pattern: Add function to prune and reload pattern list.
This function it is used for dynamically update all the patterns
attached to one file. This function is atomic. All parsing or indexation
failures are reported in the haproxy logs.
2014-03-28 13:23:07 +01:00
Thierry FOURNIER
e5978bfc25 BUG/MEDIUM: acl: boolean only matches were broken by recent changes
The ACL changes made in the last patchset force the execution
of each pattern matching function. The function pat_match_nothing
was not provided to be excuted, it was just used as a flag that
was checked by the ACL execution code. Now this function is
executed and always returns false.

This patch makes it work as expected. Now, it returns the boolean
status of the received sample just as was done previously in the
ACL code.

This bug is a part of the patchset just merged. It does not need
to be backported.
2014-03-17 20:12:40 +01:00
Thierry FOURNIER
b7729c96a4 MINOR: pattern: forbid dns resolutions
This patch adds the flags "-n" on the acl parser. the flag "-n" forbif
the DNS resolutions. The maps have always the dns resolutions disabled.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
fc7ac7b89c MINOR: standard: Disable ip resolution during the runtime
The function str2net runs DNS resolution if valid ip cannot be parsed.
The DNS function used is the standard function of the libc and it
performs asynchronous request.

The asynchronous request is not compatible with the haproxy
archictecture.

str2net() is used during the runtime throught the "socket".

This patch remove the DNS resolution during the runtime.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
94580c9f52 MINOR: dumpstat/conf: display all the configuration lines that using pattern reference 2014-03-17 18:06:08 +01:00
Thierry FOURNIER
eeaa951726 MINOR: configuration: File and line propagation
This patch permits to communicate file and line of the
configuration file at the configuration parser.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
31db4ae7b4 MINOR: pattern: index duplicates
The indexation functions now accept duplicates. This way it is possible
to always have some consistency between lists and trees. The "add" command
will always add regardless of any previous existence. The new entry will
not be used because both trees and list retrieve keys in insertion order.
Thus the "add" operation will always succeed (as long as there is enough
memory).
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
0b6d15fdc8 MINOR: regex: The pointer regstr in the struc regex is no longer used.
The pointer <regstr> is only used to compare and identify the original
regex string with the patterns. Now the patterns have a reference map
containing this original string. It is useless to store this value two
times.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
149e0fe44f MINOR: pattern: The function pat_ref_set() have now atomic behavior
Before this patch, this function try to add values in best effort. If
the parsing iof the value fail, the operation continue until the end.

Now, this function stop on the first error and left the pattern in
coherant state.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
364cfdff7a MEDIUM: dumpstats: Display error message during add of values.
This patch adds new display type. This display returns allocated string,
when the string is flush into buffers, it is freed. This permit to
return the content of "memprintf(err, ...)" messages.

The pat_ref_add functions has changed to return error.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
c0bd9100eb MINOR: pattern: Check if the file reference is not used with acl and map
The format of the acl file are not the same than the format of the map
files. In some case, the same file can be used, but this is ambiguous
for the user because the patterns are not the expected.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
39bef456fb MINOR: pattern/map/acl: Centralization of the file parsers
The acl and map function do the same work with the file parsing. This
patch merge these code in only one.

Note that the function map_read_entries_from_file() in the file "map.c"
is moved to the the function pat_ref_read_from_file_smp() in the file
"pattern.c". The code of this function is not modified, only the the
name and the arguments order has changed.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
e369ca2e66 MEDIUM: pattern_find_smp: functions find_smp uses the pat_ref_elt to find the element to be removed
The find_smp search the smp using the value of the pat_ref_elt pointer.

The pat_find_smp_* are no longer used. The function pattern_find_smp()
known all pattern indexation, and can be found
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
7acca4b269 MEDIUM: pattern: delete() function uses the pat_ref_elt to find the element to be removed
All the pattern delete function can use her reference to the original
"struct pat_ref_elt" to find the element to be remove. The functions
pat_del_list_str() and pat_del_meth() were deleted because after
applying this modification, they have the same code than pat_del_list_ptr().
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
6bb53ff164 MINOR: pattern: Each pattern expression element store the reference struct.
Now, each pattern entry known the original "struct pat_ref_elt" from
that was built. This patch permit to delete each pattern entry without
confusion. After this patch, each reference can use his pointer to be
targeted.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
0d6ba513a5 MINOR: pattern: store configuration reference for each acl or map pattern.
This patch permit to add reference for each pattern reference. This is
useful to identify the acl listed.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d25c842326 MINOR: pattern: Merge function pattern_add() with pat_ref_push().
The function Pattern_add() is only used by pat_ref_push(). This patch
remove the function pattern_add() and merge his code in the function
pat_ref_push().
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
aa222aadb6 MINOR: pattern: The function "pattern_register()" is no longer used.
Remove the function "pattern_register()" and its prototype because it
is no longer used.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
af5a29d5f8 MINOR: pattern: Each pattern is identified by unique id.
The pattern reference are stored with two identifiers: the unique_id and
the reference.

The reference identify a file. Each file with the same name point to the
same reference. We can register many times one file. If the file is
modified, all his dependencies are also modified. The reference can be
used with map or acl.

The unique_id identify inline acl. The unique id is unique for each acl.
You cannot force the same id in the configuration file, because this
repport an error.

The format of the acl and map listing through the "socket" has changed
for displaying these new ids.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
5d34408785 MEDIUM: pattern: The expected type is stored in the pattern head, and conversion is executed once.
This patch extract the expect_type variable from the "struct pattern" to
"struct pattern_head". This variable is set during the declaration of
ACL and MAP. With this change, the function "pat_parse_len()" become
useless and can be replaced by "pat_parse_int()".

Implicit ACLs by default rely on the fetch's output type, so let's simply do
the same for all other ones. It has been verified that they all match.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
c5959fd5d4 MEDIUM: pattern: merge same pattern
Sometimes the same pattern file is used with the same index, parse and
parse_smp functions. If this two condition are true, these two pattern
are identical and the same struct can be used.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1e00d3853b MAJOR: pattern/map: Extends the map edition system in the patterns
This patch add the following socket command line options:

  show acl [<id>]
  clear acl <id>
  get acl <id> <pattern>
  del acl <id> <pattern>
  add acl <id> <pattern>

The system used for maps is backported in the pattern functions.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
3667e514d9 MEDIUM: pattern/dumpstats: The function pattern_lookup() is no longer used
This function are used in dumpstats. Now this function is replaced by
delete and find_smp function pointer.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
55d0b10f06 MEDIUM: pattern: add sample lookup function.
Some functions needs to change the sample associated to pattern. This
new pointer permit to return the a pointer to the sample pointer. The
caller can use or change the value.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
6f7203d673 MEDIUM: pattern: add prune function
This path add specific pointer to each expression to point on prune
function. Now, each pattern expression embed his own prune function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b113650e54 MEDIUM: pattern: add delete functions
This commit adds a delete function for patterns. It looks up all
instances of the pattern to delete and deletes them all. The fetch
keyword declarations have been extended to point to the appropriate
delete function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
33a7433ac9 MEDIUM: pattern: Index IPv6 addresses in a tree.
This commit adds second tree node in the pattern struct and use it to
index IPv6 addresses. This commit report feature used in the list. If
IPv4 not match the tree, try to convert the IPv4 address in IPv6 with
prefixing the IPv4 address by "::ffff", after this operation, the match
function try lookup in the IPv6 tree. If the IPv6 sample dont match the
IPv6 tree, try to convert the IPv6 addresses prefixed by "2002:IPv4",
"::ffff:IPv4" and "::0000:IPv4" in IPv4 address. after this operation,
the match function try lookup in the IPv4 tree.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
5338eea8eb MEDIUM: pattern: The match function browse itself the list or the tree.
The match function known the format of the pattern. The pattern can be
stored in a list or in a tree. The pattern matching function use itself
the good entry point and indexation type.

Each pattern matching function return the struct pattern that match. If
the flag "fill" is set, the struct pattern is filled, otherwise the
content of this struct must not be used.

With this feature, the general pattern matching function cannot have
exceptions for building the "struct pattern".
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1794fdf37e MEDIUM: pattern: The function pattern_exec_match() returns "struct pattern" if the patten match.
Before this commit, the pattern_exec_match() function returns the
associate sample, the associate struct pattern or the associate struct
pattern_tree. This is complex to use, because we can check the type of
information returned.

Now the function return always a "struct pattern". If <fill> is not set,
only the value of the pointer can be used as boolean (NULL or other). If
<fill> is set, you can use the <smp> pointer and the pattern
information.

If information must be duplicated, it is stored in trash buffer.
Otherwise, the pattern can point on existing strings.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
7654c9ff44 MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags
The operations applied on types SMP_T_CSTR and SMP_T_STR are the same,
but the check code and the declarations are double, because it must
declare action for SMP_T_C* and SMP_T_*. The declared actions and checks
are the same. this complexify the code. Only the "conv" functions can
change from "C*" to "*"

Now, if a function needs to modify input string, it can call the new
function smp_dup(). This one duplicate data in a trash buffer.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
e1bcac5b8f MINOR: pattern: Rename "pat_idx_elt" to "pattern_tree"
This is just for having coherent struct names.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
edc15c3a35 MEDIUM: pattern: The parse functions just return "struct pattern" without memory allocation
The pattern parse functions put the parsed result in a "struct pattern"
without memory allocation. If the pattern must reference the input data
without changes, the pattern point to the parsed string. If buffers are
needed to store translated data, it use th trash buffer. The indexation
function that allocate the memory later if it is needed.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b9b08460a2 MEDIUM: pattern: add indexation function.
Before this patch, the indexation function check the declared patttern
matching function and index the data according with this function. This
is not useful to add some indexation mode.

This commit adds dedicated indexation function. Each struct pattern is
associated with one indexation function. This function permit to index
data according with the type of pattern and with the type of match.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
3ead5b93c6 MINOR: pattern: separe list element from the data part.
This commit separes the "struct list" used for the chain the "struct
pattern" which contain the pattern data. Later, this change will permit
to manipulate lists ans trees with the same "struct pattern".
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
972028fa67 MEDIUM: pattern: Change the prototype of the function pattern_register().
Each pattern parser take only one string. This change is reported to the
function prototype of the function "pattern_register()". Now, it is
called with just one string and no need to browse the array of args.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
580c32cb3a MEDIUM: pattern: The pattern parser no more uses <opaque> and just takes one string.
After the previous patches, the "pat_parse_strcat()" function disappear,
and the "pat_parse_int()" and "pat_parse_dotted_ver()" functions dont
use anymore the "opaque" argument, and take only one string on his
input.

So, after this patch, each pattern parser no longer use the opaque
variable and take only one string as input. This patch change the
prototype of the pattern parsing functions.

Now, the "char **args" is replaced by a "char *arg", the "int *opaque"
is removed and these functions return 1 in succes case, and 0 if fail.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
511e9475f2 MEDIUM: acl/pattern: standardisation "of pat_parse_int()" and "pat_parse_dotted_ver()"
The goal of these patch is to simplify the prototype of
"pat_pattern_*()" functions. I want to replace the argument "char
**args" by a simple "char *arg" and remove the "opaque" argument.

"pat_parse_int()" and "pat_parse_dotted_ver()" are the unique pattern
parser using the "opaque" argument and using more than one string
argument of the char **args. These specificities are only used with ACL.
Other systems using this pattern parser (MAP and CLI) just use one
string for describing a range.

This two functions can read a range, but the min and the max must y
specified. This patch extends the syntax to describe a range with
implicit min and max. This is used for operators like "lt", "le", "gt",
and "ge". the syntax is the following:

   ":x" -> no min to "x"
   "x:" -> "x" to no max

This patch moves the parsing of the comparison operator from the
functions "pat_parse_int()" and "pat_parse_dotted_ver()" to the acl
parser. The acl parser read the operator and the values and build a
volatile string readable by the functions "pat_parse_int()" and
"pat_parse_dotted_ver()". The transformation is done with these rules:

If the parser is "pat_parse_int()":

   "eq x" -> "x"
   "le x" -> ":x"
   "lt x" -> ":y" (with y = x - 1)
   "ge x" -> "x:"
   "gt x" -> "y:" (with y = x + 1)

If the parser is "pat_parse_dotted_ver()":

   "eq x.y" -> "x.y"
   "le x.y" -> ":x.y"
   "lt x.y" -> ":w.z" (with w.z = x.y - 1)
   "ge x.y" -> "x.y:"
   "gt x.y" -> "w.z:" (with w.z = x.y + 1)

Note that, if "y" is not present, assume that is "0".

Now "pat_parse_int()" and "pat_parse_dotted_ver()" accept only one
pattern and the variable "opaque" is no longer used. The prototype of
the pattern parsers can be changed.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
9eec0a646b MAJOR: auth: Change the internal authentication system.
This patch remove the limit of 32 groups. It also permit to use standard
"pat_parse_str()" function in place of "pat_parse_strcat()". The
"pat_parse_strcat()" is no longer used and its removed. Before this
patch, the groups are stored in a bitfield, now they are stored in a
list of strings. The matching is slower, but the number of groups is
low and generally the list of allowed groups is short.

The fetch function "smp_fetch_http_auth_grp()" used with the name
"http_auth_group" return valid username. It can be used as string for
displaying the username or with the acl "http_auth_group" for checking
the group of the user.

Maybe the names of the ACL and fetch methods are no longer suitable, but
I keep the current names for conserving the compatibility with existing
configurations.

The function "userlist_postinit()" is created from verification code
stored in the big function "check_config_validity()". The code is
adapted to the new authentication storage system and it is moved in the
"src/auth.c" file. This function is used to check the validity of the
users declared in groups and to check the validity of groups declared
on the "user" entries.

This resolve function is executed before the check of all proxy because
many acl needs solved users and groups.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
59ad9d6593 BUG/MEDIUM: pattern: "pat_parse_dotted_ver()" set bad expect_type.
This is a regression introducted by the patches "MINOR: pattern: Each
pattern sets the expected input type" and "MEDIUM: acl: Last patch
change the output type". The expected value is SMP_T_CSTR in place of
SMP_T_UINT.

This bug impact all the acl using the parser "pat_parse_dotted_ver()".
The two acl are "req_ssl_ver()" and "req.ssl_ver()".

This is a recent bug, no backport is needed.
2014-01-27 16:23:01 +01:00
Thierry FOURNIER
e7ba23633b MINOR: pattern: move functions for grouping pat_match_* and pat_parse_* and add documentation. 2014-01-21 22:14:21 +01:00
Thierry FOURNIER
46ceb01c24 BUG/MEDIUM: pattern: Segfault in binary parser
The functions pat_parse_* must return 0 if fail and the number of
elements eated from **text if not fail. The function pat_parse_bin()
returns 0 or the length parsed. This causes a segfault. I just apply the
double operator "!" on the result of the function pat_parse_bin() and
the return value value match the expected value.
2014-01-21 22:14:21 +01:00
Thierry FOURNIER
35249cb045 BUG/MINOR: pattern: pattern comparison executed twice
If the pattern is set as case insensitive, the string comparison
is executed twice. The first time is insensitive comparison, the
second is sensitive.

This is a recent bug, no backport is needed.
2014-01-14 15:42:59 +01:00
Willy Tarreau
6762a3061a BUG/MAJOR: patterns: fix double free caused by loading strings from files
A null pointer assignment was missing after a free in commit 7148ce6 (MEDIUM:
pattern: Extract the index process from the pat_parse_*() functions), causing
a double free after loading a file of string patterns.

This bug was introduced in 1.5-dev20, no backport is needed.

Thanks to Sander Klein for reporting this bug and providing the config
needed to trigger it.
2013-12-16 10:40:28 +01:00
Willy Tarreau
668ae532b9 BUILD: pattern: silence a warning about uninitialized value
Gcc 3.4 warns that mask may be used uninitialized in pattern.c. This
is wrong since it's used in the same condition as its assignment,
although it's not necessarily obvious for the compiler. Fix this by
initializing the value.

This was introduced by recent commit 01cdcd4a so no backport is needed.
2013-12-16 02:23:51 +01:00
Thierry FOURNIER
c64de3f1bc MINOR: pattern/map: Each pattern must free the associated sample 2013-12-12 15:56:05 +01:00
Thierry FOURNIER
01cdcd4a62 MINOR: pattern: add function to lookup a specific entry in pattern list
This is used to dynamically delete or update map entry.
2013-12-12 15:50:01 +01:00
Thierry FOURNIER
7609064fc3 MINOR: pattern: make the pattern matching function return a pointer to the matched element
This feature will be used by the CLI to look up keys.
2013-12-12 15:44:05 +01:00
Thierry FOURNIER
0b2fe4a5cd MINOR: pattern: add support for compiling patterns for lookups
With this patch, patterns can be compiled for two modes :
  - match
  - lookup

The match mode is used for example in ACLs or maps. The lookup mode
is used to lookup a key for pattern maintenance. For example, looking
up a network is different from looking up one address belonging to
this network.

A special case is made for regex. In lookup mode they return the input
regex string and do not compile the regex.
2013-12-12 15:44:02 +01:00
Thierry FOURNIER
799c042daa MINOR: regex: Change the struct containing regex
This change permits to remove the typedef. The original regex structs
are set in haproxy's struct.
2013-12-12 15:42:58 +01:00
Thierry FOURNIER
7148ce6ef4 MEDIUM: pattern: Extract the index process from the pat_parse_*() functions
Now, the pat_parse_*() functions parses the incoming data. The input
"pattern" struct can be preallocated. If the parser needs to add some
buffers, it allocates memory.

The function pattern_register() runs the call to the parser, process
the key indexation and associate the "sample_storage" used by maps.
2013-12-12 15:42:11 +01:00
Willy Tarreau
f3489d2ccd MINOR: pattern: do not assign SMP_TYPES by default to patterns
This is never used since we exit on parse failure, and it's confusing.
2013-12-12 15:42:11 +01:00
Thierry FOURNIER
e3ded59706 MEDIUM: acl: Last patch change the output type
This patch remove the compatibility check from the input type and the
match method. Now, it checks if a casts from the input type to output
type exists and the pattern_exec_match() function apply casts before
each pattern matching.
2013-12-12 15:42:11 +01:00
Thierry FOURNIER
cc0e0b3dbb MINOR: pattern: Each pattern sets the expected input type
This is used later for increasing the compability with incoming
sample types. When multiple compatible types are supported, one
is arbitrarily used (eg: UINT).
2013-12-12 11:07:33 +01:00
Thierry FOURNIER
3ce88c76a0 BUG/MEDIUM: pattern: Pattern node has type of "struct pat_idx_elt" in place of "struct eb_node"
The free() function must free the "struct pat_idx_elt".

This bug was introduced by commit ed66c29 (REORG: acl/pattern: extract
pattern matching from the acl file and create pattern.c), no backport
is needed.
2013-12-09 15:39:01 +01:00
Willy Tarreau
126d40691a MINOR: tools: add a generic binary hex string parser
We currently use such an hex parser in pat_parse_bin() to parse hex
string patterns. We'll need another generic one so let's move it to
standard.c and have pat_parse_bin() make use of it.
2013-12-06 11:50:47 +01:00
Willy Tarreau
0cba607400 MINOR: acl/pattern: use types different from int to clarify who does what.
We now have the following enums and all related functions return them and
consume them :

   enum pat_match_res {
	PAT_NOMATCH = 0,         /* sample didn't match any pattern */
	PAT_MATCH = 3,           /* sample matched at least one pattern */
   };

   enum acl_test_res {
	ACL_TEST_FAIL = 0,           /* test failed */
	ACL_TEST_MISS = 1,           /* test may pass with more info */
	ACL_TEST_PASS = 3,           /* test passed */
   };

   enum acl_cond_pol {
	ACL_COND_NONE,		/* no polarity set yet */
	ACL_COND_IF,		/* positive condition (after 'if') */
	ACL_COND_UNLESS,	/* negative condition (after 'unless') */
   };

It's just in order to avoid doubts when reading some code.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
a65b343eee MEDIUM: pattern: rename "acl" prefix to "pat"
This patch just renames functions, types and enums. No code was changed.
A significant number of files were touched, especially the ACL arrays,
so it is likely that some external patches will not apply anymore.

One important thing is that we had to split ACL_PAT_* into two groups :
  - ACL_TEST_{PASS|MISS|FAIL}
  - PAT_{MATCH|UNMATCH}

A future patch will enforce enums on all these places to avoid confusion.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
d163e1ce30 MEDIUM: pattern: create pattern expression
This new structure contains the data needed for pattern matching. It's
the first step to the complete independance of the pattern matching.
2013-12-02 23:31:33 +01:00
Thierry FOURNIER
ed66c297c2 REORG: acl/pattern: extract pattern matching from the acl file and create pattern.c
This patch just moves code without any change.

The ACL are just the association between sample and pattern. The pattern
contains the match method and the parse method. These two things are
different. This patch cleans the code by splitting it.
2013-12-02 23:31:33 +01:00
Willy Tarreau
cd3b094618 REORG: rename "pattern" files
They're now called "sample" everywhere to match their description.
2012-05-08 20:57:21 +02:00
Willy Tarreau
1278578487 REORG: use the name "sample" instead of "pattern" to designate extracted data
This is mainly a massive renaming in the code to get it in line with the
calling convention. Next patch will rename a few files to complete this
operation.
2012-05-08 20:57:20 +02:00
Willy Tarreau
40aebd9239 MINOR: pattern: centralize handling of unstable data in pattern_process()
Pattern fetch functions currently check for unstable data and return 0
when SMP_F_MAY_CHANGE is set. Instead of doing this everywhere and having
to support specific fetch functions, better do that in pattern_process()
which is the one interested in having stable data.
2012-05-08 20:57:18 +02:00
Willy Tarreau
12e5011a76 MEDIUM: pattern: ensure that sample types always cast into other types.
We want to ensure that a dynamically returned type will always have a
cast before calling the cast function. This is done in pattern_process()
and in stktable_fetch_key().
2012-05-08 20:57:17 +02:00
Willy Tarreau
32a6f2e572 MEDIUM: acl/pattern: use the same direction scheme
Patterns were using a bitmask to indicate if request or response was desired
in fetch functions and keywords. ACLs were using a bitmask in fetch keywords
and a single bit in fetch functions. ACLs were also using an ACL_PARTIAL bit
in fetch functions indicating that a non-final fetch was performed, which was
an abuse of the existing direction flag.

The change now consists in using :
  - a capabilities field for fetch keywords => SMP_CAP_REQ/RES to indicate
    if a keyword supports requests, responses, both, etc...
  - an option field for fetch functions to indicate what the caller expects
    (request/response, final/non-final)

The ACL_PARTIAL bit was reversed to get SMP_OPT_FINAL as it's more explicit
to know we're working on a final buffer than on a non-final one.

ACL_DIR_* were removed, as well as PATTERN_FETCH_*. L4 fetches were improved
to support being called on responses too since they're still available.

The <dir> field of all fetch functions was changed to <opt> which is now
unsigned.

The patch is large but mostly made of cosmetic changes to accomodate this, as
almost no logic change happened.
2012-05-08 20:57:17 +02:00
Willy Tarreau
b8c8f1f611 MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description
We need the pattern fetchers and converters to correctly set the output type
so that they can be used by ACL fetchers. By using the sample type instead of
the keyword type, we also open the possibility to create some multi-type
pattern fetch methods later (eg: "src" being v4/v6). Right now the type in
the keyword is used to validate the configuration.
2012-05-08 20:57:16 +02:00