17603 Commits

Author SHA1 Message Date
Thierry FOURNIER
12ba0c29d6 MINOR: samples: rename a struct from sample_storage to sample_data
This a first step of sample reorganization.
2015-08-20 17:13:46 +02:00
Pradeep Jindal
cc79b003cb BUG/MINOR: ssl: TLS Ticket Key rotation broken via socket command
It seems haproxy was doing wrong pointer arithmetic to update the ticket
ring correctly.
2015-08-20 16:23:22 +02:00
Thierry FOURNIER
c7d7607351 BUG/MEDIUM: lua: bad error processing
When the lua actions (http|tcp)-(request|rersponse) fails a yield is
returned. this patch fix the behavior and returns a continue.
2015-08-19 10:54:06 +02:00
Willy Tarreau
0f4eadd483 BUG/MEDIUM: counters: ensure that src_{inc,clr}_gpc0 creates a missing entry
During 1.5-dev20 there was some code refactoring to make the src_* fetch
function use the same code as sc_*. Unfortunately this introduced a
regression where src_* doesn't create an entry anymore if it does not
exist in the table. The reason is that smp_fetch_sc_stkctr() only calls
stktable_lookup_key() while src_inc_*/src_clr_* used to make use of
stktable_update_key() which additionally create the entry if it does
not exist.

There's no point modifying the common function for these two exceptions,
so instead we now have a function dedicated to the creation of this entry
for src_* only. It is called when the entry didn't exist, so that requires
minimal modifications to existing code.

Thanks to Thierry Fournier for helping diagnose the issue.

This fix must be backported to 1.5.
2015-08-18 17:15:20 +02:00
Thierry FOURNIER
827752e7ec BUG/MEDIUM: stick-tables: refcount error after copying SC for the session to the stream
Each copyied SC counter will be used and released by the stream, so we
need to increment the refcount.
2015-08-18 13:39:49 +02:00
Thierry FOURNIER
6f82b6dc35 BUG/MEDIUM: vars: segfault during the configuration parsing
This bug is introduced by the patch 48a9cd104d07919fdd941b36e9f927a20d55edd3
"MINOR: vars: reduce the code size of some wrappers". A dereferencement was
removed.
2015-08-17 18:48:18 +02:00
Baptiste Assmann
93c20623db MINOR: server SRV_ADMF_CMAINT flag doesn't imply SRV_ADMF_FMAINT
The newly created server flag SRV_ADMF_CMAINT means that the server is
in 'disabled' mode because of configuration statement 'disabled'.
The flag SRV_ADMF_FMAINT should not be set anymore in such case and is
reserved only when the server is Forced in maintenance mode from the
stats socket.
2015-08-17 15:42:07 +02:00
Thierry FOURNIER
c8fdb983c5 BUG/MEDIUM: stream: The stream doen't inherit SC from the session
During the processing of tcp-request connection, the stream doesn't exists, so the
stick counters are stored in the session. When the stream is created it must
inherit from the session sc.

This patch fix this behavior.

[WT: this is specific to 1.6, no backport needed]
2015-08-16 23:55:32 +02:00
Thierry FOURNIER
03d0e45dbb MEDIUM: cli: rely on the map's output type instead of the sample type
Next patch will remove sample_storage->type, and the only user is the
"show map" feature on the CLI which can use the map's output type instead.
Let's do that first.
2015-08-11 14:14:11 +02:00
Thierry FOURNIER
a123ad886a MINOR: sample/proto_tcp: export "smp_fetch_src"
This patch exports the sample fetch "smp_fetch_src()".
2015-08-11 14:14:11 +02:00
Thierry FOURNIER
cc4d1716a2 MINOR: sample: Add ipv6 to ipv4 and sint to ipv6 casts
The RFC4291 says that when the IPv6 adress have the followin form:
0000::ffff:a.b.c.d, if can be converted to an IPv4 adress. This patch
enable this conversion in casts.

As the sint can be casted as ipv4, and ipv4 can be casted as ipv6, we
can directly cast sint as ipv6 using the RFC4291.
2015-08-11 14:14:10 +02:00
Thierry FOURNIER
48a9cd104d MINOR: vars: reduce the code size of some wrappers
Some function are just a wrappers. This patch reduce the size of
this wrapper for improving the readability. One check is moved
from the wrapper to the main function, and some middle vars are
removed.
2015-08-11 14:14:10 +02:00
Thierry FOURNIER
422a3af4ce MINOR: proto_tcp: add session in the action prototype
Some actions require the "struct session" while the "struct stream" is not
avalaible. This patch adds a pointer to the session.
2015-08-11 14:08:29 +02:00
Thierry FOURNIER
c89f4f5305 BUG/MINOR: proto_tcp: custom action continue is ignored
The custom action is ignored by 'tcp-request connection'. This patch
fix this behavior and take in account the value of the flag 'action'.
2015-08-11 13:45:46 +02:00
Thierry FOURNIER
a6b6343cff CLEANUP: http/tcp actions: remove the scope member
The scope member is not used. This patch removes this entry.
2015-08-11 13:44:53 +02:00
Thierry FOURNIER
9b49f589ed CLEANUP: proto_http: remove useless initialisation
This initialisation of the opaque array is useless.
2015-08-11 13:44:51 +02:00
Willy Tarreau
f3934b8c6d MEDIUM: config: emit a warning on a frontend without listener
Commit c6678e2 ("MEDIUM: config: authorize frontend and listen without bind")
completely removed the test for bind lines in frontends in order to make it
easier for automated tools to generate configs (eg: replacing a bind with
another one passing via a temporary config without any bind line). The
problem is that some common mistakes are totally hidden now. For example,
this apparently valid entry is silently ignored :

       listen 1.2.3.4:8000
           server s1 127.0.0.1:8000

Hint: 1.2.3.4:8000 is mistakenly the proxy name here.

Thus instead we now emit a warning to indicate that a frontend was found
with no listener. This should be backported to 1.5 to help spot abnormal
configurations.
2015-08-11 12:17:41 +02:00
Willy Tarreau
9496552e7d CLEANUP: appsession: remove appsession.c and sessionhash.c
Now there's no more code using appsessions we can remove them.
2015-08-10 19:17:47 +02:00
Willy Tarreau
6db62c58d6 CLEANUP: config: remove appsession initialization
Now it asks to check the documentation.
2015-08-10 19:16:18 +02:00
Willy Tarreau
53a09d520e MAJOR: http: remove references to appsession
appsessions started to be deprecated with the introduction of stick
tables, and the latter are much more powerful and flexible, and in
addition they are replicated between nodes and maintained across
reloads. Let's now remove appsession completely.
2015-08-10 19:16:18 +02:00
Thierry FOURNIER
61e96c68b9 BUG/MEDIUM: lua: timeout error with converters, wrapper and actions.
test conf:

   global
      tune.lua.session-timeout 0
      lua-load lol.lua
      debug
      maxconn 4096

   listen test
	   bind 0.0.0.0:10010
		mode tcp
		tcp-request content lua act_test
		balance roundrobin
		server test 127.0.0.1:3304

lua test:

   function act_test(txn)
      while true do
         core.Alert("TEST")
      end
   end

The function "act_test()" is not executed because a zero timeout is not
considered as TICK_ETERNITY, but is considered as 0.

This path fix this behavior. This is the same problem than the bugfix
685c014e99195d60db0a9cdbc6483f9c44fd0a67.
2015-08-10 06:18:30 +02:00
Willy Tarreau
f2ee0162c3 MINOR: init: indicate to check 'bind' lines when no listeners were found.
It used to still mention '<listen>'.
2015-08-09 11:01:51 +02:00
Camilo Lopez
685c014e99 BUG/MEDIUM: lua: Lua tasks fail to start.
I've been trying out 1.6 dev3 with lua support, and trying to start
lua tasks seems to not be working.

Using this configuration

global
  lua-load /lua/lol.lua
  debug
  maxconn 4096

backend shard_b
  server db01 mysql_shard_b:3306

backend shard_a
  server db01 mysql_shard_a:3306

listen mysql-cluster
  bind 0.0.0.0:8001
  mode tcp
  balance roundrobin
  use_backend shard_b

And this lua function

  core.register_task(function()
    while true do
      core.Alert("LOLOLOLOLOL")
    end
  end)

I'd always get a timeout error starting the registered function.

The problem lies as far as I can tell in the fact that is possible for
now_ms to not change (is this maybe a problem on my config/system?)
until the expiration check happens, in the resume function that
actually kickstarts the lua task, making HAProxy think that expiration
time for the task is up, if I understand correctly tasks are meant to
never really timeout.
2015-08-09 10:22:34 +02:00
Andrew Hayworth
e63ac871f8 MINOR: log: Add log-format variable %HQ, to log HTTP query strings
Since sample fetches are not always available in the response phase,
this patch implements %HQ such that:

  GET /foo?bar=baz HTTP/1.0

...would be logged as:

  ?bar=baz
2015-08-09 10:16:49 +02:00
Baptiste Assmann
9f5ada32e4 MINOR: server: add new SRV_ADMF_CMAINT flag
The purpose of SRV_ADMF_CMAINT flag is to keep in mind the server was
forced to maintenance status because of the configuration file.
2015-08-08 18:18:17 +02:00
Baptiste Assmann
2359ff1de2 BUG/MEDIUM: DNS resolution response parsing broken
In some cases, parsing of the DNS response is broken and the response is
considered as invalid, despite being valid.

The current patch fixes this issue. It's a temporary solution until I
rework the response parsing to store the response buffer into a real DNS
packet structure.
2015-08-08 18:14:20 +02:00
Baptiste Assmann
37bb372ea2 MINOR: DNS counters: increment valid counter
Valid counter was never incremented.
Now it is.
2015-08-08 18:13:59 +02:00
Dragan Dosen
9373fc5811 MINOR: 51d: unable to start haproxy without "51degrees-data-file"
This patch adds a few checks on "global._51degrees.data_file_path" and allows
haproxy to start even when the pattern or trie data file is not specified.

If the "51d" converter is used, a new function "_51d_conv_check" will check
"global._51degrees.data_file_path" and displays a warning if necessary.

In src/haproxy.c, the global 51Degrees "cache_size" has moved outside of the
FIFTYONEDEGREES_H_PATTERN_INCLUDED ifdef block.
2015-08-08 12:15:55 +02:00
Willy Tarreau
449d74a906 MEDIUM: backend: add the "http-reuse aggressive" strategy
This strategy is less extreme than "always", it only dispatches first
requests to validated reused connections, and moves a connection from
the idle list to the safe list once it has seen a second request, thus
proving that it could be reused.
2015-08-06 16:29:01 +02:00
Willy Tarreau
7017cb040c MINOR: server: add a list of safe, already reused idle connections
These ones are considered safe as they have already been reused.
They will be useful in "aggressive" and "always" http-reuse modes
in order to place the first request of a connection with the least
risk.
2015-08-06 16:29:01 +02:00
Willy Tarreau
161d45ffc7 MEDIUM: backend: implement "http-reuse safe"
The "safe" mode consists in picking existing connections only when
processing a request that's not the first one from a connection. This
ensures that in case where the server finally times out and closes, the
client can decide to replay idempotent requests.
2015-08-06 11:50:53 +02:00
Willy Tarreau
efb90f9dd3 MAJOR: backend: improve the connection reuse mechanism
Now instead of closing the existing connection attached to the
stream interface, we first check if the one we pick was attached to
another stream interface, in which case the connections are swapped
if possible (eg: if the current connection is not private). That way
the previous connection remains attached to an existing session and
significantly increases the chances of being reused.
2015-08-06 11:41:06 +02:00
Willy Tarreau
8dff998b91 MAJOR: backend: initial work towards connection reuse
In connect_server(), if we don't have a connection attached to the
stream-int, we first look into the server's idle_conns list and we
pick the first one there, we detach it from its owner if it had one.
If we used to have a connection, we close it.

This mechanism works well but doesn't scale : as servers increase,
the likeliness that the connection attached to the stream interface
doesn't match the server and gets closed increases.
2015-08-06 11:34:21 +02:00
Willy Tarreau
d8fecee291 MINOR: config: add new setting "http-reuse"
For now it only supports "never", meaning that we never want to reuse a
shared connection, and "always", meaning that we can use any connection
that was not marked private. When "never" is set, this also implies that
no idle connection may become a shared one.
2015-08-06 11:14:20 +02:00
Willy Tarreau
387ebf84dd MINOR: connection: add a new flag CO_FL_PRIVATE
This flag is set on an outgoing connection when this connection gets
some properties that must not be shared with other connections, such
as dynamic transparent source binding, SNI or a proxy protocol header,
or an authentication challenge from the server. This will be needed
later to implement connection reuse.
2015-08-06 11:14:17 +02:00
Willy Tarreau
173a1c6b43 MINOR: server: add a list of already used idle connections
There's a difference with the other idle conns in that these new
ones have already been used and may be reused by other streams.
2015-08-06 11:13:47 +02:00
Willy Tarreau
4320eaac62 MINOR: stream-int: make si_idle_conn() only accept valid connections
This function is now dedicated to idle connections only, which means
that it must not be used without any endpoint nor anything not a
connection. The connection remains attached to the stream interface.
2015-08-06 11:11:10 +02:00
Willy Tarreau
323a2d925c MEDIUM: stream-int: queue idle connections at the server
Now we get a per-server list of all idle connections. That way we'll
be able to reclaim them upon shortage later.
2015-08-06 11:06:25 +02:00
Willy Tarreau
600802aef0 MINOR: server: add a list of private idle connections
For now it's not populated but we have the list entry. It will carry
all idle connections that sessions don't want to share. They may be
used later to reclaim connections upon socket shortage for example.
2015-08-06 10:59:08 +02:00
Willy Tarreau
973a54235f MEDIUM: stream-int: simplify si_alloc_conn()
Since we now always call this function with the reuse parameter cleared,
let's simplify the function's logic as it cannot return the existing
connection anymore. The savings on this inline function are appreciable
(240 bytes) :

$ size haproxy.old haproxy.new
   text    data     bss     dec     hex filename
1020383   40816   36928 1098127  10c18f haproxy.old
1020143   40816   36928 1097887  10c09f haproxy.new
2015-08-05 21:51:09 +02:00
Willy Tarreau
c12b5e663d MEDIUM: backend: don't call si_alloc_conn() when we reuse a valid connection
connect_server() already does most of the check that is done again in
si_alloc_conn(), so let's simply reuse the existing connection instead
of calling the function again. It will also simplify the connection
reuse.

Indeed, for reuse to be set, it also requires srv_conn to be valid. In the
end, the only situation where we have to release the existing connection
and allocate a new one is when reuse == 0.
2015-08-05 21:42:12 +02:00
Willy Tarreau
7b00492ce3 CLEANUP: backend: factor out objt_server() in connect_server()
objt_server() is called multiple times at various places while some
places already make use of srv for this. Let's move the call at the
top of the function and use it all over the place.
2015-08-05 10:12:47 +02:00
Remi Gacogne
7fb9de25e2 MINOR: stream: initialize the current_rule field to NULL on stream init
Currently it is possible for the current_rule field to be evaluated before
being set, leading to valgrind complaining:

==16783== Conditional jump or move depends on uninitialised value(s)
==16783==    at 0x44E662: http_res_get_intercept_rule (proto_http.c:3730)
==16783==    by 0x44E662: http_process_res_common (proto_http.c:6528)
==16783==    by 0x4797B7: process_stream (stream.c:1851)
==16783==    by 0x414634: process_runnable_tasks (task.c:238)
==16783==    by 0x40B02F: run_poll_loop (haproxy.c:1528)
==16783==    by 0x407F25: main (haproxy.c:1887)

This was introduced by commit 152b81e7b2565862956af883820d4f79177d0651.
2015-07-26 21:50:50 +02:00
Jérémie Courrèges-Anglas
17c3f6284c BUILD: ssl: Allow building against libssl without SSLv3.
If SSLv3 is explicitely requested but not available, warn the user and
bail out.
2015-07-26 08:11:57 +02:00
Pieter Baauw
ed35c371dc BUG/MEDIUM: mailer: DATA part must be terminated with <CRLF>.<CRLF>
The dot is send in the wrong place.
As defined in https://www.ietf.org/rfc/rfc2821.txt 'the character sequence "<CRLF>.<CRLF>" ends the mail text'
2015-07-22 22:39:39 +02:00
Willy Tarreau
d69d6f3678 BUG/MAJOR: dns: fix the length of the string to be copied
Jan A. Bruder reported that some very specific hostnames on server
lines were causing haproxy to crash on startup. Given that hist
backtrace showed some heap corruption, it was obvious there was an
overflow somewhere. The bug in fact is a typo in dns_str_to_dn_label()
which mistakenly copies one extra byte from the host name into the
output value, thus effectively corrupting the structure.

The bug triggers while parsing the next server of similar length
after the corruption, which generally triggers at config time but
could theorically crash at any moment during runtime depending on
what malloc sizes are needed next. This is why it's tagged major.

No backport is needed, this bug was introduced in 1.6-dev2.
2015-07-22 16:53:22 +02:00
Willy Tarreau
2100b49122 CLEANUP/MINOR: dns: dns_str_to_dn_label() only needs a const char
The string is an input, let's constify it.
2015-07-22 16:42:43 +02:00
Thierry FOURNIER
5d86fae234 MEDIUM: vars/sample: operators can use variables as parameter
This patch allow the existing operators to take a variable as parameter.
This is useful to add the content of two variables. This patch modify
the behavior of operators.
2015-07-22 00:48:24 +02:00
Thierry FOURNIER
fd77e05f65 MINOR: vars: returns variable content
This patch copy the content of a variable in a sample. The function
returns 0 if the variable is not found.
2015-07-22 00:48:24 +02:00
Thierry FOURNIER
00c005c726 MEDIUM: sample: switch to saturated arithmetic
This patch check calculus for overflow and returns capped values.
This permits to protect against integer overflow in certain operations
involving ratios, percentages, limits or anything. That can sometimes
be critically important with some operations (eg: content-length < X).
2015-07-22 00:48:24 +02:00