This is very convenient to reduce SSL processing priority compared to
other traffic. This applies to CPU usage only, but has a direct impact
on latency under congestion.
SSL connections take a huge amount of memory, and unfortunately openssl
does not check malloc() returns and easily segfaults when too many
connections are used.
The only solution against this is to provide a global maxsslconn setting
to reject SSL connections above the limit in order to avoid reaching
unsafe limits.
With SSL, connections are much more expensive, so it is important to be
able to limit concurrent connections per listener in order to limit the
memory usage.
The correct spelling is "independent", not "independant". This patch
fixes the doc and the configuration parser to accept the correct form.
The config parser still allows the old naming for backwards compatibility.
I came across a couple of typos in configuration.txt and made this patch.
Also, there is an inconsistency between using the word analys/ze in
configuration.txt as well. However, I did not provide a patch for that.
-- Jamie Gloudon
[wt: won't fix the us/uk language mistakes, they'll always exist anyway]
This is used to enter values for stick tables. The most likely usage
is to set gpc0 for a specific IP address in order to block traffic
for abusers without having to reload. Since all data types are
supported, other usages are possible (eg: replace a users's assigned
server).
This one returns the concatenation of the first Host header entry with
the path. It can make content-switching rules easier, help with fighting
DDoS on certain URLs and improve shared caches efficiency.
Doing so allows us to support sticking on URL, URL's IP, URL's port and
path.
Both fetch functions should be improved to support an optional depth
allowing to stick to a server depending on just a few directory
components. This would help with portals, some prefetch-capable
caches and with outgoing connections using multiple internet links.
Default value for maxconn in the context of a proxy is 2000 and is
unrelated to any other value (like global ulimit-n or global
maxconn). Without an explicit a user may think that the default value
is either no limit or equal to the global maxconn value.
Released version 1.5-dev11 with the following main changes :
- BUG/MEDIUM: option forwardfor if-none doesn't work with some configurations
- BUG/MAJOR: trash must always be the size of a buffer
- DOC: fix minor regex example issue and improve doc on stats
- MINOR: stream_interface: add a pointer to the listener for TARG_TYPE_CLIENT
- MEDIUM: protocol: add a pointer to struct sock_ops to the listener struct
- MINOR: checks: add on-marked-up option
- MINOR: balance uri: added 'whole' parameter to include query string in hash calculation
- MEDIUM: stream_interface: remove the si->init
- MINOR: buffers: add a rewind function
- BUG/MAJOR: fix regression on content-based hashing and http-send-name-header
- MAJOR: http: stop using msg->sol outside the parsers
- CLEANUP: http: make it more obvious that msg->som is always null outside of chunks
- MEDIUM: http: get rid of msg->som which is not used anymore
- MEDIUM: http: msg->sov and msg->sol will never wrap
- BUG/MAJOR: checks: don't call set_server_status_* when no LB algo is set
- BUG/MINOR: stop connect timeout when connect succeeds
- REORG: move the send-proxy code to tcp_connect_write()
- REORG/MINOR: session: detect the TCP monitor checks at the protocol accept
- MINOR: stream_interface: introduce a new "struct connection" type
- REORG/MINOR: stream_interface: move si->fd to struct connection
- REORG/MEDIUM: stream_interface: move applet->state and private to connection
- MINOR: stream_interface: add a data channel close function
- MEDIUM: stream_interface: call si_data_close() before releasing the si
- MINOR: peers: use the socket layer operations from the peer instead of sock_raw
- BUG/MINOR: checks: expire on timeout.check if smaller than timeout.connect
- MINOR: add a new function call tracer for debugging purposes
- BUG/MINOR: perform_http_redirect also needs to rewind the buffer
- BUG/MAJOR: b_rew() must pass a signed offset to b_ptr()
- BUG/MEDIUM: register peer sync handler in the proper order
- BUG/MEDIUM: buffers: fix bi_putchr() to correctly advance the pointer
- BUG/MINOR: fix option httplog validation with TCP frontends
- BUG/MINOR: log: don't report logformat errors in backends
- REORG/MINOR: use dedicated proxy flags for the cookie handling
- BUG/MINOR: config: do not report twice the incompatibility between cookie and non-http
- MINOR: http: add support for "httponly" and "secure" cookie attributes
- BUG/MEDIUM: ensure that unresolved arguments are freed exactly once
- BUG/MINOR: commit 196729ef used wrong condition resulting in freeing constants
- MEDIUM: stats: add support for soft stop/soft start in the admin interface
- MEDIUM: stats: add the ability to kill sessions from the admin interface
- BUILD: add support for linux kernels >= 2.6.28
This implements the feature discussed in the earlier thread of killing
connections on backup servers when a non-backup server comes back up. For
example, you can use this to route to a mysql master & slave and ensure
clients don't stay on the slave after the master goes from down->up. I've done
some minimal testing and it seems to work.
[WT: added session flag & doc, moved the killing after logging the server UP,
and ensured that the new server is really usable]
httponly This option tells haproxy to add an "HttpOnly" cookie attribute
when a cookie is inserted. This attribute is used so that a
user agent doesn't share the cookie with non-HTTP components.
Please check RFC6265 for more information on this attribute.
secure This option tells haproxy to add a "Secure" cookie attribute when
a cookie is inserted. This attribute is used so that a user agent
never emits this cookie over non-secure channels, which means
that a cookie learned with this flag will be presented only over
SSL/TLS connections. Please check RFC6265 for more information on
this attribute.
This patch brings a new "whole" parameter to "balance uri" which makes
the hash work over the whole uri, not just the part before the query
string. Len and depth parameter are still honnored.
The reason for this new feature is explained below.
I have 3 backend servers, each accepting different form of HTTP queries:
http://backend1.server.tld/service1.php?q=...
http://backend1.server.tld/service2.php?q=...
http://backend2.server.tld/index.php?query=...&subquery=...
http://backend3.server.tld/image/49b8c0d9ff
Each backend server returns a different response based on either:
- the URI path (the left part of the URI before the question mark)
- the query string (the right part of the URI after the question mark)
- or the combination of both
I wanted to set up a common caching cluster (using 6 Squid servers, each
configured as reverse proxy for those 3 backends) and have HAProxy balance
the queries among the Squid servers based on URL. I also wanted to achieve
hight cache hit ration on each Squid server and send the same queries to
the same Squid servers. Initially I was considering using the 'balance uri'
algorithm, but that would not work as in case of backend2 all queries would
go to only one Squid server. The 'balance url_param' would not work either
as it would send the backend3 queries to only one Squid server.
So I thought the simplest solution would be to use 'balance uri', but to
calculate the hash based on the whole URI (URI path + query string),
instead of just the URI path.
Released version 1.5-dev10 with the following main changes :
- BUG/MINOR: stats admin: "Unexpected result" was displayed unconditionally
- BUG/MAJOR: acl: http_auth_group() must not accept any user from the userlist
- CLEANUP: auth: make the code build again with DEBUG_AUTH
- BUG/MEDIUM: config: don't crash at config load time on invalid userlist names
- REORG: use the name sock_raw instead of stream_sock
- MINOR: stream_interface: add a client target : TARG_TYPE_CLIENT
- BUG/MEDIUM: stream_interface: restore get_src/get_dst
- CLEANUP: sock_raw: remove last references to stream_sock
- CLEANUP: stream_interface: stop exporting socket layer functions
- MINOR: stream_interface: add an init callback to sock_ops
- MEDIUM: stream_interface: derive the socket operations from the target
- MAJOR: fd: remove the need for the socket layer to recheck the connection
- MINOR: session: call the socket layer init function when a session establishes
- MEDIUM: session: add support for tunnel timeouts
- MINOR: standard: add a new debug macro : fddebug()
- CLEANUP: fd: remove unused cb->b pointers in the struct fdtab
- OPTIM: proto_http: don't enable quick-ack on empty buffers
- OPTIM/MAJOR: ev_sepoll: process spec events after polled events
- OPTIM/MEDIUM: stream_interface: add a new SI_FL_NOHALF flag
Tunnel timeouts are used when TCP connections are forwarded, or
when forwarding upgraded HTTP connections (WebSocket) as well as
CONNECT requests to proxies.
This timeout allows long-lived sessions to be supported without
having to set large timeouts to normal requests.
Released version 1.5-dev9 with the following main changes :
- MINOR: Add release callback to si_applet
- CLEANUP: Fix some minor typos
- MINOR: Add TO/FROM_SET flags to struct stream_interface
- CLEANUP: Fix some minor whitespace issues
- MINOR: stats admin: allow unordered parameters in POST requests
- CLEANUP: fix typo in findserver() log message
- MINOR: stats admin: use the backend id instead of its name in the form
- MINOR: stats admin: reduce memcmp()/strcmp() calls on status codes
- DOC: cleanup indentation, alignment, columns and chapters
- DOC: fix some keywords arguments documentation
- MINOR: cli: display the 4 IP addresses and ports on "show sess XXX"
- BUG/MAJOR: log: possible segfault with logformat
- MEDIUM: log: split of log_format generation
- MEDIUM: log: New format-log flags: %Fi %Fp %Si %Sp %Ts %rt %H %pid
- MEDIUM: log: Unique ID
- MINOR: log: log-format: usable without httplog and tcplog
- BUG/MEDIUM: balance source did not properly hash IPv6 addresses
- MINOR: contrib/iprange: add a network IP range to mask converter
- MEDIUM: session: implement the "use-server" directive
- MEDIUM: log: add a new cookie flag 'U' to report situations where cookie is not used
- MEDIUM: http: make extract_cookie_value() iterate over cookie values
- MEDIUM: http: add cookie and scookie ACLs
- CLEANUP: lb_first: add reference to a paper describing the original idea
- MEDIUM: stream_sock: add a get_src and get_dst callback and remove SN_FRT_ADDR_SET
- BUG/MINOR: acl: req_ssl_sni would randomly fail if a session ID is present
- BUILD: http: make extract_cookie_value() return an int not size_t
- BUILD: http: stop gcc-4.1.2 from complaining about possibly uninitialized values
- CLEANUP: http: message parser must ignore HTTP_MSG_ERROR
- MINOR: standard: add a memprintf() function to build formatted error messages
- CLEANUP: remove a few warning about unchecked return values in debug code
- MEDIUM: move message-related flags from transaction to message
- DOC: add a diagram to explain how circular buffers work
- MAJOR: buffer rework: replace ->send_max with ->o
- MAJOR: buffer: replace buf->l with buf->{o+i}
- MINOR: buffers: provide simple pointer normalization functions
- MINOR: buffers: remove unused function buffer_contig_data()
- MAJOR: buffers: replace buf->w with buf->p - buf->o
- MAJOR: buffers: replace buf->r with buf->p + buf->i
- MAJOR: http: move buffer->lr to http_msg->next
- MAJOR: http: change msg->{som,col,sov,eoh} to be relative to buffer origin
- CLEANUP: http: remove unused http_msg->col
- MAJOR: http: turn http_msg->eol to a buffer-relative offset
- MEDIUM: http: add a pointer to the buffer in http_msg
- MAJOR: http: make http_msg->sol relative to buffer's origin
- MEDIUM: http: http_send_name_header: remove references to msg and buffer
- MEDIUM: http: remove buffer arg in a few header manipulation functions
- MEDIUM: http: remove buffer arg in http_capture_bad_message
- MEDIUM: http: remove buffer arg in http_msg_analyzer
- MEDIUM: http: remove buffer arg in http_upgrade_v09_to_v10
- MEDIUM: http: remove buffer arg in http_buffer_heavy_realign
- MEDIUM: http: remove buffer arg in chunk parsing functions
- MINOR: http: remove useless wrapping checks in http_msg_analyzer
- MEDIUM: buffers: fix unsafe use of buffer_ignore at some places
- MEDIUM: buffers: add new pointer wrappers and get rid of almost all buffer_wrap_add calls
- MEDIUM: buffers: implement b_adv() to advance a buffer's pointer
- MEDIUM: buffers: rename a number of buffer management functions
- MEDIUM: http: add a prefetch function for ACL pattern fetch
- MEDIUM: http: make all ACL fetch function use acl_prefetch_http()
- BUG/MINOR: http_auth: ACLs are volatile, not permanent
- MEDIUM: http/acl: merge all request and response ACL fetches of headers and cookies
- MEDIUM: http/acl: make acl_fetch_hdr_{ip,val} rely on acl_fetch_hdr()
- MEDIUM: add a new typed argument list parsing framework
- MAJOR: acl: make use of the new argument parsing framework
- MAJOR: acl: store the ACL argument types in the ACL keyword declaration
- MEDIUM: acl: acl_find_target() now resolves arguments based on their types
- MAJOR: acl: make acl_find_targets also resolve proxy names at config time
- MAJOR: acl: ensure that implicit table and proxies are valid
- MEDIUM: acl: remove unused tests for missing args when args are mandatory
- MEDIUM: pattern: replace type pattern_arg with type arg
- MEDIUM: pattern: get rid of arg_i in all functions making use of arguments
- MEDIUM: pattern: use the standard arg parser
- MEDIUM: pattern: add an argument validation callback to pattern descriptors
- MEDIUM: pattern: report the precise argument parsing error when known.
- MEDIUM: acl: remove the ACL_TEST_F_NULL_MATCH flag
- MINOR: pattern: add a new 'sample' type to store fetched data
- MEDIUM: pattern: add new sample types to replace pattern types
- MAJOR: acl: make use of the new sample struct and get rid of acl_test
- MEDIUM: pattern/acl: get rid of temp_pattern in ACLs
- MEDIUM: acl: get rid of the SET_RES flags
- MEDIUM: get rid of SMP_F_READ_ONLY and SMP_F_MUST_FREE
- MINOR: pattern: replace struct pattern with struct sample
- MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere
- MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description
- MEDIUM: acl/pattern: switch rdp_cookie functions stack up-down
- MEDIUM: acl: replace acl_expr with args in acl fetch_* functions
- MINOR: tcp: replace acl_fetch_rdp_cookie with smp_fetch_rdp_cookie
- MEDIUM: acl/pattern: use the same direction scheme
- MEDIUM: acl/pattern: start merging common sample fetch functions
- MEDIUM: pattern: ensure that sample types always cast into other types.
- MEDIUM: acl/pattern: factor out the src/dst address fetches
- MEDIUM: acl: implement payload and payload_lv
- CLEANUP: pattern: ensure that payload and payload_lv always stay in the buffer
- MINOR: stick_table: centralize the handling of empty keys
- MINOR: pattern: centralize handling of unstable data in pattern_process()
- MEDIUM: pattern: use smp_fetch_rdp_cookie instead of the pattern specific version
- MINOR: acl: set SMP_OPT_ITERATE on fetch functions
- MINOR: acl: add a val_args field to keywords
- MINOR: proto_tcp: validate arguments of payload and payload_lv ACLs
- MEDIUM: http: merge acl and pattern header fetch functions
- MEDIUM: http: merge ACL and pattern cookie fetches into a single one
- MEDIUM: acl: report parsing errors to the caller
- MINOR: arg: improve error reporting on invalid arguments
- MINOR: acl: report errors encountered when loading patterns from files
- MEDIUM: acl: extend the pattern parsers to report meaningful errors
- REORG: use the name "sample" instead of "pattern" to designate extracted data
- REORG: rename "pattern" files
- MINOR: acl: add types to ACL patterns
- MINOR: standard: add an IPv6 parsing function (str62net)
- MEDIUM: acl: support IPv6 address matching
- REORG: stream_interface: create a struct sock_ops to hold socket operations
- REORG/MEDIUM: move protocol->{read,write} to sock_ops
- REORG/MEDIUM: stream_interface: initialize socket ops from descriptors
- REORG/MEDIUM: replace stream interface protocol functions by a proto pointer
- REORG/MEDIUM: move the default accept function from sockstream to protocols.c
- MEDIUM: proto_tcp: remove src6 and dst6 pattern fetch methods
- BUG/MINOR: http: error snapshots are wrong if buffer wraps
- BUG/MINOR: http: ensure that msg->err_pos is always relative to buf->p
- MEDIUM: http: improve error capture reports
- MINOR: acl: add the cook_val() match to match a cookie against an integer
- BUG/MEDIUM: send_proxy: fix initialisation of send_proxy_ofs
- MEDIUM: memory: add the ability to poison memory at run time
- BUG/MEDIUM: log: ensure that unique_id is properly initialized
- MINOR: cfgparse: use a common errmsg pointer for all parsers
- MEDIUM: cfgparse: make backend_parse_balance() use memprintf to report errors
- MEDIUM: cfgparse: use the new error reporting framework for remaining cfg_keywords
- MINOR: http: replace http_message_realign() with buffer_slow_realign()
These methods have been superseded by src and dst which support
multiple families. There is no point keeping them since they appeared
in a development version anyway.
For configurations using "src6", please use "src" instead. For "dst6",
use "dst" instead.
Make use of the new IPv6 pattern type so that acl_match_ip() knows how to
compare pattern and sample.
IPv6 may be entered in their usual form, with or without a netmask appended.
Only bit counts are accepted for IPv6 netmasks. In order to avoid any risk of
trouble with randomly resolved IP addresses, host names are never allowed in
IPv6 patterns.
HAProxy is also able to match IPv4 addresses with IPv6 addresses in the
following situations :
- tested address is IPv4, pattern address is IPv4, the match applies
in IPv4 using the supplied mask if any.
- tested address is IPv6, pattern address is IPv6, the match applies
in IPv6 using the supplied mask if any.
- tested address is IPv6, pattern address is IPv4, the match applies in IPv4
using the pattern's mask if the IPv6 address matches with 2002:IPV4::,
::IPV4 or ::ffff:IPV4, otherwise it fails.
- tested address is IPv4, pattern address is IPv6, the IPv4 address is first
converted to IPv6 by prefixing ::ffff: in front of it, then the match is
applied in IPv6 using the supplied IPv6 mask.
It's easy to merge pattern and ACL fetches of cookies. It allows us
to remove two distinct fetch functions. The new function internally
uses an occurrence number to serve both purposes, but it didn't appear
worth exposing it outside so there is no keyword argument to set it.
However one of the benefits is that the "cookie" fetch for stick tables
now automatically adapts to requests and responses, so there is no more
need for set-cookie().
HTTP header fetch is now done using smp_fetch_hdr() for both ACLs and
patterns. This one also supports an occurrence number, making it possible
to specify explicit occurrences for ACLs and patterns.
These ones were easy to adapt to ACL usage and may really be useful,
so let's make them available right now. It's likely that some extension
such as regex, string-to-IP and raw IP matching will be implemented in
the near future.
src_port, dst_port and url_param have converged between ACLs and patterns.
This means that src_port is now available in patterns and that urlp_* has
been added to ACLs. Some code has moved to accommodate for static function
definitions, but there were little changes.
All ACLs which need to process HTTP contents first call this function which
performs all the preliminary tests and also triggers the request parsing if
needed. A macro was written to simplify the code.
As a side effect, it's not required anymore to check for the HTTP ACL before
checking for HTTP contents.
Also add some thoughts about the existing and new design.
Note: an earlier design used the names "head" and "tail" for both sides
of the buffer, but it appears awkward as these words may be understood
in two forms (feed by head, output by tail, or make the newcomers wait
at the tail of the queue). Also there were already a few functions in the
code making use of either terminology. So better avoid this terminology
and use "input" and "output" instead.
- Typo on "dispatch" keyword arguments.
- Reindent some blocks for better parsing by automated tools.
- "option mysql-check" and "option pgsql-check" arguments were not documented
as the others.
This patch is a group commit simplify the parsing of the documenation :
- remove remaining tabulations
- realign some lines
- break lines at 80 columns
- add missing chapters in the summary
- fix chapter numbering format
The Unique ID, is an ID generated with several informations. You can use
a log-format string to customize it, with the "unique-id-format" keyword,
and insert it in the request header, with the "unique-id-header" keyword.
%Fi: Frontend IP
%Fp: Frontend Port
%Si: Server IP
%Sp: Server Port
%Ts: Timestamp
%rt: HTTP request counter
%H: hostname
%pid: PID
+X: Hexadecimal represenation
The +X mode in logformat displays hexadecimal for the following flags
%Ci %Cp %Fi %Fp %Bi %Bp %Si %Sp %Ts %ct %pid
rename logformat_write_string() to lf_text()
Optimize size computation
The ACL matches rely on the extract_cookie_value() function as used for
for patterns. This permits ACLs to match cookie values based on the cookie
name instead of having to perform substring matching on the cookie header.
Sometimes it is desirable to forward a particular request to a specific
server without having to declare a dedicated backend for this server. This
can be achieved using the "use-server" rules. These rules are evaluated after
the "redirect" rules and before evaluating cookies, and they have precedence
on them. There may be as many "use-server" rules as desired. All of these
rules are evaluated in their declaration order, and the first one which
matches will assign the server.
Released version 1.5-dev8 with the following main changes :
- MINOR: patch for minor typo (ressources/resources)
- MEDIUM: http: add support for sending the server's name in the outgoing request
- DOC: mention that default checks are TCP connections
- BUG/MINOR: fix options forwardfor if-none when an alternative header name is specified
- CLEANUP: Make check_statuses, analyze_statuses and process_chk static
- CLEANUP: Fix HCHK spelling errors
- BUG/MINOR: fix typo in processing of http-send-name-header
- MEDIUM: log: Use linked lists for loggers
- BUILD: fix declaration inside a scope block
- REORG: log: split send_log function
- MINOR: config: Parse the string of the log-format config keyword
- MINOR: add ultoa, ulltoa, ltoa, lltoa implementations
- MINOR: Date and time fonctions that don't use snprintf
- MEDIUM: log: make http_sess_log use log_format
- DOC: log-format documentation
- MEDIUM: log: use log_format for mode tcplog
- MEDIUM: log-format: backend source address %Bi %Bp
- BUG/MINOR: log-format: fix %o flag
- BUG/MEDIUM: bad length in log_format and __send_log
- MINOR: logformat %st is signed
- BUILD/MINOR: fix the source URL in the spec file
- DOC: acl is http_first_req, not http_req_first
- BUG/MEDIUM: don't trim last spaces from headers consisting only of spaces
- MINOR: acl: add new matches for header/path/url length
- BUILD: halog: make halog build on solaris
- BUG/MINOR: don't use a wrong port when connecting to a server with mapped ports
- MINOR: remove the client/server side distinction in SI addresses
- MINOR: halog: add support for matching queued requests
- DOC: indicate that cookie "prefix" and "indirect" should not be mixed
- OPTIM/MINOR: move struct sockaddr_storage to the tail of structs
- OPTIM/MINOR: make it possible to change pipe size (tune.pipesize)
- BUILD/MINOR: silent a build warning in src/pipe.c (fcntl)
- OPTIM/MINOR: move the hdr_idx pools out of the proxy struct
- MEDIUM: tune.http.maxhdr makes it possible to configure the maximum number of HTTP headers
- BUG/MINOR: fix a segfault when parsing a config with undeclared peers
- CLEANUP: rename possibly confusing struct field "tracked"
- BUG/MEDIUM: checks: fix slowstart behaviour when server tracking is in use
- MINOR: config: tolerate server "cookie" setting in non-HTTP mode
- MEDIUM: buffers: add some new primitives and rework existing ones
- BUG: buffers: don't return a negative value on buffer_total_space_res()
- MINOR: buffers: make buffer_pointer() support negative pointers too
- CLEANUP: kill buffer_replace() and use an inline instead
- BUG: tcp: option nolinger does not work on backends
- CLEANUP: ebtree: remove a few annoying signedness warnings
- CLEANUP: ebtree: clarify licence and update to 6.0.6
- CLEANUP: ebtree: remove 4-year old harmless typo in duplicates insertion code
- CLEANUP: ebtree: remove another typo, a wrong initialization in insertion code
- BUG: ebtree: ebst_lookup() could return the wrong entry
- OPTIM: stream_sock: reduce the amount of in-flight spliced data
- OPTIM: stream_sock: save a failed recv syscall when splice returns EAGAIN
- MINOR: acl: add support for TLS server name matching using SNI
- BUG: http: re-enable TCP quick-ack upon incomplete HTTP requests
- BUG: proto_tcp: don't try to bind to a foreign address if sin_family is unknown
- MINOR: pattern: export the global temporary pattern
- CLEANUP: patterns: get rid of pattern_data_setstring()
- MEDIUM: acl: use temp_pattern to store fetched information in the "method" match
- MINOR: acl: include pattern.h to make pattern migration more transparent
- MEDIUM: pattern: change the pattern data integer from unsigned to signed
- MEDIUM: acl: use temp_pattern to store any integer-type information
- MEDIUM: acl: use temp_pattern to store any address-type information
- CLEANUP: acl: integer part of acl_test is not used anymore
- MEDIUM: acl: use temp_pattern to store any string-type information
- CLEANUP: acl: remove last data fields from the acl_test struct
- MEDIUM: http: replace get_ip_from_hdr2() with http_get_hdr()
- MEDIUM: patterns: the hdr() pattern is now of type string
- DOC: add minimal documentation on how ACLs work internally
- DOC: add a coding-style file
- OPTIM: halog: keep a fast path for the lines-count only
- CLEANUP: silence a warning when building on sparc
- BUG: http: tighten the list of allowed characters in a URI
- MEDIUM: http: block non-ASCII characters in URIs by default
- DOC: add some documentation from RFC3986 about URI format
- BUG/MINOR: cli: correctly remove the whole table on "clear table"
- BUG/MEDIUM: correctly disable servers tracking another disabled servers.
- BUG/MEDIUM: zero-weight servers must not dequeue requests from the backend
- MINOR: halog: add some help on the command line
- BUILD: fix build error on FreeBSD
- BUG: fix double free in peers config error path
- MEDIUM: improve config check return codes
- BUILD: make it possible to look for pcre in the default system paths
- MINOR: config: emit a warning when 'default_backend' masks servers
- MINOR: backend: rework the LC definition to support other connection-based algos
- MEDIUM: backend: add the 'first' balancing algorithm
- BUG: fix httplog trailing LF
- MEDIUM: increase chunk-size limit to 2GB-1
- BUG: queue: fix dequeueing sequence on HTTP keep-alive sessions
- BUG: http: disable TCP delayed ACKs when forwarding content-length data
- BUG: checks: fix server maintenance exit sequence
- BUG/MINOR: stream_sock: don't remove BF_EXPECT_MORE and BF_SEND_DONTWAIT on partial writes
- DOC: enumerate valid status codes for "observe layer7"
- MINOR: buffer: switch a number of buffer args to const
- CLEANUP: silence signedness warning in acl.c
- BUG: stream_sock: si->release was not called upon shutw()
- MINOR: log: use "%ts" to log term status only and "%tsc" to log with cookie
- BUG/CRITICAL: log: fix risk of crash in development snapshot
- BUG/MAJOR: possible crash when using capture headers on TCP frontends
- MINOR: config: disable header captures in TCP mode and complain
The main stats page says "ressources" (French spelling) rather than
"resources" (English spelling).
One little patch attached (against v1.4.20).
Many thanks,
Adrian
The difference could be seen when logging a request in HTTP mode with option
tcplog, as it would keep emitting 4 chars. Better use two distinct flags to
clear the confusion.
%Bi return the backend source IP
%Bp return the backend source port
Add a function pointer in logformat_type to do additional configuration
during the log-format variable parsing.
Merge http_sess_log() and tcp_sess_log() to sess_log() and move it to
log.c
A new field in logformat_type define if you can use a logformat
variable in TCP or HTTP mode.
doc: log-format in tcp mode
Note that due to the way log buffer allocation currently works, trying to
log an HTTP request without "option httplog" is still not possible. This
will change in the near future.
The principle behind this load balancing algorithm was first imagined
and modeled by Steen Larsen then iteratively refined through several
work sessions until it would totally address its original goal.
The purpose of this algorithm is to always use the smallest number of
servers so that extra servers can be powered off during non-intensive
hours. Additional tools may be used to do that work, possibly by
locally monitoring the servers' activity.
The first server with available connection slots receives the connection.
The servers are choosen from the lowest numeric identifier to the highest
(see server parameter "id"), which defaults to the server's position in
the farm. Once a server reaches its maxconn value, the next server is used.
It does not make sense to use this algorithm without setting maxconn. Note
that it can however make sense to use minconn so that servers are not used
at full load before starting new servers, and so that introduction of new
servers requires a progressively increasing load (the number of servers
would more or less follow the square root of the load until maxconn is
reached). This algorithm ignores the server weight, and is more beneficial
to long sessions such as RDP or IMAP than HTTP, though it can be useful
there too.
These ones are invalid and blocked unless "option accept-invalid-http-request"
is specified in the frontend. In any case, the faulty request is logged.
Note that some of the remaining invalid chars are still not checked against,
those are the invalid ones between 32 and 127 :
34 ('"'), 60 ('<'), 62 ('>'), 92 ('\'), 94 ('^'),
96 ('`'), 123 ('{'), 124 ('|'), 125 ('}')
Using a lookup table might be better at some point.
New option "http-send-name-header" specifies the name of a header which
will hold the server name in outgoing requests. This is the name of the
server the connection is really sent to, which means that upon redispatches,
the header's value is updated so that it always matches the server's name.
This pattern previously was limited to type IP. With the new header
extraction function, it becomes possible to extract strings, so that
the header can be returned as a string. This will not change anything
to existing configs, as string will automatically be converted to IP
when needed. However, new configs will be able to use IPv6 addresses
from headers in stick-tables, as well as stick on any non-IP header
(eg: host, user-agent, ...).
Server Name Indication (SNI) is a TLS extension which makes a client
present the name of the server it is connecting to in the client hello.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.
The new ACL "req_ssl_sni" matches the name extracted from the TLS
handshake against a list of names which may be loaded from a file if
needed.
This patch settles the 2 loggers limitation.
Loggers are now stored in linked lists.
Using "global log", the global loggers list content is added at the end
of the current proxy list. Each "log" entries are added at the end of
the proxy list.
"no log" flush a logger list.
For a long time, the max number of headers was taken as a part of the buffer
size. Since the header size can be configured at runtime, it does not make
much sense anymore.
Nothing was making it necessary to have a static value, so let's turn this into
a tunable with a default value of 101 which equals what was previously used.
By default, pipes are the default size for the system. But sometimes when
using TCP splicing, it can improve performance to increase pipe sizes,
especially if it is suspected that pipes are not filled and that many
calls to splice() are performed. This has an impact on the kernel's
memory footprint, so this must not be changed if impacts are not understood.
When prefix and indirect are used together, a client which connects to
a server with a cookie will never get any cookie update from this server,
which will be removed by the "indirect" option.
This patch introduces hdr_len, path_len and url_len for matching these
respective parts lengths against integers. This can be used to detect
abuse or empty headers.
Released version 1.5-dev7 with the following main changes :
- [BUG] fix binary stick-tables
- [MINOR] http: *_dom matching header functions now also split on ":"
- [BUG] checks: fix support of Mysqld >= 5.5 for mysql-check
- [MINOR] acl: add srv_conn acl to count connections on a specific backend server
- [MINOR] check: add redis check support
- [DOC] small fixes to clearly distinguish between keyword and variables
- [MINOR] halog: add support for termination code matching (-tcn/-TCN)
- [DOC] Minor spelling fixes and grammatical enhancements
- [CLEANUP] dumpstats: make symbols static where possible
- [MINOR] Break out dumping table
- [MINOR] Break out processing of clear table
- [MINOR] Allow listing of stick table by key
- [MINOR] Break out all stick table socat command parsing
- [MINOR] More flexible clearing of stick table
- [MINOR] Allow showing and clearing by key of ipv6 stick tables
- [MINOR] Allow showing and clearing by key of integer stick tables
- [MINOR] Allow showing and clearing by key of string stick tables
- [CLEANUP] Remove assigned but unused variables
- [CLEANUP] peers.h: fix declarations
- [CLEANUP] session.c: Make functions static where possible
- [MINOR] Add active connection list to server
- [MINOR] Allow shutdown of sessions when a server becomes unavailable
- [MINOR] Add down termination condition
- [MINOR] Make appsess{,ion}_refresh static
- [MINOR] Add rdp_cookie pattern fetch function
- [CLEANUP] Remove unnecessary casts
- [MINOR] Add non-stick server option
- [MINOR] Consistently use error in tcp_parse_tcp_req()
- [MINOR] Consistently free expr on error in cfg_parse_listen()
- [MINOR] Free rdp_cookie_name on denint()
- [MINOR] Free tcp rules on denint()
- [MINOR] Free stick table pool on denint()
- [MINOR] Free stick rules on denint()
- [MEDIUM] Fix stick-table replication on soft-restart
- [MEDIUM] Correct ipmask() logic
- [MINOR] Correct type in table dump examples
- [MINOR] Fix build error in stream_int_register_handler()
- [MINOR] Use DPRINTF in assign_server()
- [BUG] checks: http-check expect could fail a check on multi-packet responses
- [DOC] fix minor typo in the "dispatch" doc
- [BUG] proto_tcp: fix address binding on remote source
- [MINOR] http: don't report the "haproxy" word on the monitoring response
- [REORG] http: move HTTP error codes back to proto_http.h
- [MINOR] http: make the "HTTP 200" status code configurable.
- [MINOR] http: partially revert the chunking optimization for now
- [MINOR] stream_sock: always clear BF_EXPECT_MORE upon complete transfer
- [CLEANUP] stream_sock: remove unneeded FL_TCP and factor out test
- [MEDIUM] http: add support for "http-no-delay"
- [OPTIM] http: optimize chunking again in non-interactive mode
- [OPTIM] stream_sock: avoid fast-forwarding of partial data
- [OPTIM] stream_sock: don't use splice on too small payloads
- [MINOR] config: make it possible to specify a cookie even without a server
- [BUG] stats: support url-encoded forms
- [MINOR] config: automatically compute a default fullconn value
- [CLEANUP] config: remove some left-over printf debugging code from previous patch
- [DOC] add missing entry or stick store-response
- [MEDIUM] http: add support for 'cookie' and 'set-cookie' patterns
- [BUG] halog: correctly handle truncated last line
- [MINOR] halog: make SKIP_CHAR stop on field delimiters
- [MINOR] halog: add support for HTTP log matching (-H)
- [MINOR] halog: gain back performance before SKIP_CHAR fix
- [OPTIM] halog: cache some common fields positions
- [OPTIM] halog: check once for correct line format and reuse the pointer
- [OPTIM] halog: remove many 'if' by using a function pointer for the filters
- [OPTIM] halog: remove support for tab delimiters in input data
- [BUG] session: risk of crash on out of memory (1.5-dev regression)
- [MINOR] session: try to emit a 500 response on memory allocation errors
- [OPTIM] stream_sock: reduce the default number of accepted connections at once
- [BUG] stream_sock: disable listener when system resources are exhausted
- [MEDIUM] proxy: add a PAUSED state to listeners and move socket tricks out of proxy.c
- [BUG] stream_sock: ensure orphan listeners don't accept too many connections
- [MINOR] listeners: add listen_full() to mark a listener full
- [MINOR] listeners: add support for queueing resource limited listeners
- [MEDIUM] listeners: put listeners in queue upon resource shortage
- [MEDIUM] listeners: queue proxy-bound listeners at the proxy's
- [MEDIUM] listeners: don't stop proxies when global maxconn is reached
- [MEDIUM] listeners: don't change listeners states anymore in maintain_proxies
- [CLEANUP] proxy: rename a few proxy states (PR_STIDLE and PR_STRUN)
- [MINOR] stats: report a "WAITING" state for sockets waiting for resource
- [MINOR] proxy: make session rate-limit more accurate
- [MINOR] sessions: only wake waiting listeners up if rate limit is OK
- [BUG] proxy: peers must only be stopped once, not upon every call to maintain_proxies
- [CLEANUP] proxy: merge maintain_proxies() operation inside a single loop
- [MINOR] task: new function task_schedule() to schedule a wake up
- [MAJOR] proxy: finally get rid of maintain_proxies()
- [BUG] proxy: stats frontend and peers were missing many initializers
- [MEDIUM] listeners: add a global listener management task
- [MINOR] proxy: make findproxy() return proxies from numeric IDs too
- [DOC] fix typos, "#" is a sharp, not a dash
- [MEDIUM] stats: add support for changing frontend's maxconn at runtime
- [MEDIUM] checks: group health checks methods by values and save option bits
- [MINOR] session-counters: add the ability to clear the counters
- [BUG] check: http-check expect + regex would crash in defaults section
- [MEDIUM] http: make x-forwarded-for addition conditional
- [REORG] build: move syscall redefinition to specific places
- [CLEANUP] update the year in the copyright banner
- [BUG] possible crash in 'show table' on stats socket
- [BUG] checks: use the correct destination port for sending checks
- [BUG] backend: risk of picking a wrong port when mapping is used with crossed families
- [MINOR] make use of set_host_port() and get_host_port() to get rid of family mismatches
- [DOC] fixed a few "sensible" -> "sensitive" errors
- [MINOR] make use of addr_to_str() and get_host_port() to replace many inet_ntop()
- [BUG] http: trailing white spaces must also be trimmed after headers
- [MINOR] stats: display "<NONE>" instead of the frontend name when unknown
- [MINOR] http: take a capture of too large requests and responses
- [MINOR] http: take a capture of truncated responses
- [MINOR] http: take a capture of bad content-lengths.
- [DOC] add a few old and uncommitted docs
- [CLEANUP] cfgparse: fix reported options for the "bind" keyword
- [MINOR] halog: add -hs/-HS to filter by HTTP status code range
- [MINOR] halog: support backslash-escaped quotes
- [CLEANUP] remove dirty left-over of a debugging message
- [MEDIUM] stats: disable complex socket reservation for stats socket
- [CLEANUP] remove a useless test in manage_global_listener_queue()
- [MEDIUM] stats: add the "set maxconn" setting to the command line interface
- [MEDIUM] add support for global.maxconnrate to limit the per-process conn rate.
- [MINOR] stats: report the current and max global connection rates
- [MEDIUM] stats: add the ability to adjust the global maxconnrate
- [BUG] peers: don't pre-allocate 65000 connections to each peer
- [MEDIUM] don't limit peers nor stats socket to maxconn nor maxconnrate
- [BUG] peers: the peer frontend must not emit any log
- [CLEANUP] proxy: make pause_proxy() perform the required controls and emit the logs
- [BUG] peers: don't keep a peers section which has a NULL frontend
- [BUG] peers: ensure the peers are resumed if they were paused
- [MEDIUM] stats: add the ability to enable/disable/shutdown a frontend at runtime
- [MEDIUM] session: make session_shutdown() an independant function
- [MEDIUM] stats: offer the possibility to kill a session from the CLI
- [CLEANUP] stats: centralize tests for backend/server inputs on the CLI
- [MEDIUM] stats: offer the possibility to kill sessions by server
- [MINOR] halog: do not consider byte 0x8A as end of line
- [MINOR] frontend: ensure debug message length is always initialized
- [OPTIM] halog: make fgets parse more bytes by blocks
- [OPTIM] halog: add assembly version of the field lookup code
- [MEDIUM] poll: add a measurement of idle vs work time
- [CLEANUP] startup: report only the basename in the usage message
- [MINOR] startup: add an option to change to a new directory
- [OPTIM] task: don't scan the run queue if we know it's empty
- [BUILD] stats: stdint is not present on solaris
- [DOC] update the README file to reflect new naming rules for patches
- [MINOR] stats: report the number of requests intercepted by the frontend
- [DOC] update ROADMAP file
We already had the ability to kill a connection, but it was only
for the checks. Now we can do this for any session, and for this we
add a specific flag "K" to the logs.
The stats socket now allows the admin to disable, enable or shutdown a frontend.
This can be used when a bug is discovered in a configuration and it's desirable
to fix it but the rules in place don't allow to change a running config. Thus it
becomes possible to kill the frontend to release the port and start a new one in
a separate process.
This can also be used to temporarily make haproxy return TCP resets to incoming
requests to pretend the service is not bound. For instance, this may be useful
to quickly flush a very deep SYN backlog.
The frontend check and lookup code was factored with the "set maxconn" usage.
This one enforces a per-process connection rate limit, regardless of what
may be set per frontend. It can be a way to limit the CPU usage of a process
being severely attacked.
The side effect is that the global process connection rate is now measured
for each incoming connection, so it will be possible to report it.
This option permits to change the global maxconn setting within the
limit that was set by the initial value, which is now reported as the
hard maxconn value. This allows to immediately accept more concurrent
connections or to stop accepting new ones until the value passes below
the indicated setting.
The main use of this option is on systems where many haproxy instances
are loaded and admins need to re-adjust resource sharing at run time
to regain a bit of fairness between processes.
If "option forwardfor" has the "if-none" argument, then the header is
only added when the request did not already have one. This option has
security implications, and should not be set blindly.
Manoj Kumar reported a case where haproxy would crash upon start-up. The
cause was an "http-check expect" statement declared in the defaults section,
which caused a NULL regex to be used during the check. This statement is not
allowed in defaults sections precisely because this requires saving a copy
of the regex in the default proxy. But the check was not made to prevent it
from being declared there, hence the issue.
Instead of adding code to detect its abnormal use, we decided to implement
it. It was not that much complex because the expect_str part was not used
with regexes, so it could hold the string form of the regex in order to
compile it again for every backend (there's no way to clone regexes).
This patch has been tested and works. So it's both a bugfix and a minor
feature enhancement.
It should be backported to 1.4 though it's not critical since the config
was not supposed to be supported.
This patch provides a new "option redis-check" statement to enable server health checks based on redis PING request (http://www.redis.io/commands/ping).
The new "set maxconn frontend XXX" statement on the stats socket allows
the admin to change a frontend's maxconn value. If some connections are
queued, they will immediately be accepted up to the new limit. If the
limit is lowered, new connections acceptation might be delayed. This can
be used to temporarily reduce or increase the impact of a specific frontend's
traffic on the whole process.
This is used to perform cookie-based stickiness with table replication
between multiple masters and across restarts. This partially overrides
some of the appsession capabilities.
Never add connections allocated to this sever to a stick-table.
This may be used in conjunction with backup to ensure that
stick-table persistence is disabled for backup servers.
This pattern fetch function extracts the value of the rdp cookie <name> as
a string and uses this value to match. This enables implementation of
persistence based on the mstshash cookie. This is typically done if there
is no msts cookie present.
This differs from "balance rdp-cookie" in that any balancing algorithm may
be used and thus the distribution of clients to backend servers is not
linked to a hash of the RDP cookie. It is envisaged that using a balancing
algorithm such as "balance roundrobin" or "balance leastconnect" will lead
to a more even distribution of clients to backend servers than the hash
used by "balance rdp-cookie".
Example :
listen tse-farm
bind 0.0.0.0:3389
# wait up to 5s for an RDP cookie in the request
tcp-request inspect-delay 5s
tcp-request content accept if RDP_COOKIE
# apply RDP cookie persistence
persist rdp-cookie
# Persist based on the mstshash cookie
# This is only useful makes sense if
# balance rdp-cookie is not used
stick-table type string size 204800
stick on rdp_cookie(mstshash)
server srv1 1.1.1.1:3389
server srv1 1.1.1.2:3389
If a connection is closed by because the backend became unavailable
then log 'D' as the termination condition.
Signed-off-by: Simon Horman <horms@verge.net.au>
This adds the "on-marked-down shutdown-sessions" statement on "server" lines,
which causes all sessions established on a server to be killed at once when
the server goes down. The task's priority is reniced to the highest value
(1024) so that servers holding many tasks don't cause a massive slowdown due
to the wakeup storm.
mysqld >= 5.5 want the client to announce 4.1+ authentication support, even if we have no password, so we do this.
I also check on a debian potato mysqld 3.22 and it works too so i assume we are good from 3.22 to 5.5.
[WT: this must be backported to 1.4]
The fullconn value is not easy to get right when doing dynamic regulation,
as it should depend on the maxconns of the frontends that can reach a
backend. Since the parameter is mandatory, many configs are found with
an inappropriate default value.
Instead of rejecting configs without a fullconn value, we now set it to
10% of the sum of the configured maxconns of all the frontends which are
susceptible to branch to the backend. That way if new frontends are added,
the backend's fullconn automatically adjusts itself.
There are some very rare server-to-server applications that abuse the HTTP
protocol and expect the payload phase to be highly interactive, with many
interleaved data chunks in both directions within a single request. This is
absolutely not supported by the HTTP specification and will not work across
most proxies or servers. When such applications attempt to do this through
haproxy, it works but they will experience high delays due to the network
optimizations which favor performance by instructing the system to wait for
enough data to be available in order to only send full packets. Typical
delays are around 200 ms per round trip. Note that this only happens with
abnormal uses. Normal uses such as CONNECT requests nor WebSockets are not
affected.
When "option http-no-delay" is present in either the frontend or the backend
used by a connection, all such optimizations will be disabled in order to
make the exchanges as fast as possible. Of course this offers no guarantee on
the functionality, as it may break at any other place. But if it works via
HAProxy, it will work as fast as possible. This option should never be used
by default, and should never be used at all unless such a buggy application
is discovered. The impact of using this option is an increase of bandwidth
usage and CPU usage, which may significantly lower performance in high
latency environments.
This change should be backported to 1.4 since the first report of such a
misuse was in 1.4. Next patch will also be needed.
This status code is used in response to requests matching "monitor-uri".
Some users need to adjust it to fit their needs (eg: make some strings
appear there). As it's already defined as a chunked string and used
exactly like other status codes, it makes sense to make it configurable
with the usual "errorfile", "errorloc", ...
Released version 1.5-dev6 with the following main changes :
- [BUG] stream_sock: use get_addr_len() instead of sizeof() on sockaddr_storage
- [BUG] TCP source tracking was broken with IPv6 changes
- [BUG] stick-tables did not work when converting IPv6 to IPv4
- [CRITICAL] fix risk of crash when dealing with space in response cookies
Released version 1.5-dev5 with the following main changes :
- [BUG] standard: is_addr return value for IPv4 was inverted
- [MINOR] update comment about IPv6 support for server
- [MEDIUM] use getaddrinfo to resolve names if gethostbyname fail
- [DOC] update IPv6 support for bind
- [DOC] document IPv6 support for server
- [DOC] fix a minor typo
- [MEDIUM] IPv6 support for syslog
- [DOC] document IPv6 support for syslog
- [MEDIUM] IPv6 support for stick-tables
- [DOC] document IPv6 support for stick-tables
- [DOC] update ROADMAP file
- [BUG] session: src_conn_cur was returning src_conn_cnt instead
- [MINOR] frontend: add a make_proxy_line function
- [MEDIUM] stream_sock: add support for sending the proxy protocol header line
- [MEDIUM] server: add support for the "send-proxy" option
- [DOC] update the spec on the proxy protocol
- [BUILD] proto_tcp: fix build issue with CTTPROXY
- [DOC] update ROADMAP file
- [MEDIUM] config: rework the IPv4/IPv6 address parser to support host-only addresses
- [MINOR] cfgparse: better report wrong listening addresses and make use of str2sa_range
- [BUILD] add the USE_GETADDRINFO build option
- [TESTS] provide a test case for various address formats
- [BUG] session: conn_retries was not always initialized
- [BUG] log: retrieve the target from the session, not the SI
- [BUG] http: fix possible incorrect forwarded wrapping chunk size (take 2)
- [MINOR] tools: add two macros MID_RANGE and MAX_RANGE
- [BUG] http: fix content-length handling on 32-bit platforms
- [OPTIM] buffers: uninline buffer_forward()
- [BUG] stream_sock: fix handling for server side PROXY protocol
- [MINOR] acl: add support for table_cnt and table_avl matches
- [DOC] update ROADMAP file
This option enables use of the PROXY protocol with the server, which
allows haproxy to transport original client's address across multiple
architecture layers.
Released version 1.5-dev4 with the following main changes :
- [MINOR] cfgparse: Check whether the path given for the stats socket actually fits into the sockaddr_un structure to avoid truncation.
- [MINOR] unix sockets : inherits the backlog size from the listener
- [CLEANUP] unix sockets : move create_uxst_socket() in uxst_bind_listener()
- [DOC] fix a minor typo
- [DOC] fix ignore-persist documentation
- [MINOR] add warnings on features not compatible with multi-process mode
- [BUG] http: fix http-pretend-keepalive and httpclose/tunnel mode
- [MINOR] stats: add support for several packets in stats admin
- [BUG] stats: admin commands must check the proxy state
- [BUG] stats: admin web interface must check the proxy state
- [MINOR] http: add pattern extraction method to stick on query string parameter
- [MEDIUM] add internal support for IPv6 server addresses
- [MINOR] acl: add be_id/srv_id to match backend's and server's id
- [MINOR] log: add support for passing the forwarded hostname
- [MINOR] log: ability to override the syslog tag
- [MINOR] checks: add PostgreSQL health check
- [DOC] update ROADMAP file
- [BUILD] pattern: use 'int' instead of 'int32_t'
- [OPTIM] linux: add support for bypassing libc to force using vsyscalls
- [BUG] debug: report the correct poller list in verbose mode
- [BUG] capture: do not capture a cookie if there is no memory left
- [BUG] appsession: fix possible double free in case of out of memory
- [CRITICAL] cookies: mixing cookies in indirect mode and appsession can crash the process
- [BUG] http: correctly update the header list when removing two consecutive headers
- [BUILD] add the CPU=native and ARCH=32/64 build options
- [BUILD] add -fno-strict-aliasing to fix warnings with gcc >= 4.4
- [CLEANUP] hash: move the avalanche hash code globally available
- [MEDIUM] hash: add support for an 'avalanche' hash-type
- [DOC] update roadmap file
- [BUG] http: do not re-enable the PROXY analyser on keep-alive
- [OPTIM] http: don't send each chunk in a separate packet
- [DOC] fix minor typos reported recently in the peers section
- [DOC] fix another typo in the doc
- [MINOR] stats: report HTTP message state and buffer flags in error dumps
- [BUG] http chunking: don't report a parsing error on connection errors
- [BUG] stream_interface: truncate buffers when sending error messages
- [MINOR] http: support wrapping messages in error captures
- [MINOR] http: capture incorrectly chunked message bodies
- [MINOR] stats: add global event ID and count
- [BUG] http: analyser optimizations broke pipelining
- [CLEANUP] frontend: only apply TCP-specific settings to TCP/TCP6 sockets
- [BUG] http: fix incorrect error reporting during data transfers
- [CRITICAL] session: correctly leave turn-around and queue states on abort
- [BUG] session: release slot before processing pending connections
- [MINOR] tcp: add support for dynamic MSS setting
- [BUG] stick-table: correctly terminate string keys during lookups
- [BUG] acl: fix handling of empty lines in pattern files
- [BUG] stick-table: use the private buffer when padding strings
- [BUG] ebtree: fix ebmb_lookup() with len smaller than the tree's keys
- [OPTIM] ebtree: ebmb_lookup: reduce stack usage by moving the return code out of the loop
- [OPTIM] ebtree: inline ebst_lookup_len and ebis_lookup_len
- [REVERT] undo the stick-table string key lookup fixes
- [MINOR] http: improve url_param pattern extraction to ignore empty values
- [BUILD] frontend: shut a warning with TCP_MAXSEG
- [BUG] http: update the header list's tail when removing the last header
- [DOC] fix minor typo in the proxy protocol doc
- [DOC] fix typos (http-request instead of http-check)
- [BUG] http: use correct ACL pointer when evaluating authentication
- [BUG] cfgparse: correctly count one socket per port in ranges
- [BUG] startup: set the rlimits before binding ports, not after.
- [BUG] acl: srv_id must return no match when the server is NULL
- [MINOR] acl: add ability to check for internal response-only parameters
- [MINOR] acl: srv_id is only valid in responses
- [MINOR] config: warn if response-only conditions are used in "redirect" rules
- [BUG] acl: fd leak when reading patterns from file
- [DOC] fix minor typo in "usesrc"
- [BUG] http: fix possible incorrect forwarded wrapping chunk size
- [BUG] http: fix computation of message body length after forwarding has started
- [BUG] http: balance url_param did not work with first parameters on POST
- [TESTS] update the url_param regression test to test check_post too
- [DOC] update ROADMAP
- [DOC] internal: reflect the fact that SI_ST_ASS is transient
- [BUG] config: don't crash on empty pattern files.
- [MINOR] stream_interface: make use of an applet descriptor for IO handlers
- [REORG] stream_interface: move the st0, st1 and private members to the applet
- [REORG] stream_interface: split the struct members in 3 parts
- [REORG] session: move client and server address to the stream interface
- [REORG] tcp: make tcpv4_connect_server() take the target address from the SI
- [MEDIUM] stream_interface: store the target pointer and type
- [CLEANUP] stream_interface: remove the applet.handler pointer
- [MEDIUM] log: take the logged server name from the stream interface
- [CLEANUP] session: remove data_source from struct session
- [CLEANUP] stats: make all dump functions only rely on the stream interface
- [REORG] session: move the data_ctx struct to the stream interface's applet
- [MINOR] proxy: add PR_O2_DISPATCH to detect dispatch mode
- [MINOR] cfgparse: only keep one of dispatch, transparent, http_proxy
- [MINOR] session: add a pointer to the new target into the session
- [MEDIUM] session: remove s->prev_srv which is not needed anymore
- [CLEANUP] stream_interface: use inline functions to manipulate targets
- [MAJOR] session: remove the ->srv pointer from struct session
- [MEDIUM] stats: split frontend and backend stats
- [MEDIUM] http: always evaluate http-request rules before stats http-request
- [REORG] http: move the http-request rules to proto_http
- [BUG] http: stats were not incremented on http-request deny
- [MINOR] checks: report it if checks fail due to socket creation error
Bryan Talbot reported that POST requests with a query string were not
correctly processed if the hash parameter was the first one, because
the delimiter that was looked for to trigger the parsing was '&' instead
of '?'.
Also, while checking the code, it became apparent that it was enough for
a query string to be present in the request for POST parameters to be
ignored, even if the url_param was in the body and not in the URL.
The code has then been fixed like this :
1) look for URL param. If found, return it.
2) if no URL param was found and method is POST, then look it up into
the body
The code now seems to pass all request combinations.
This patch must be backported to 1.4 since 1.4 is equally broken right now.
Some browsers send POST requests in several packets, which was not supported
by the "stats admin" function.
This patch allows to wait for more data when they are not fully received
(we are still limited to a certain size defined by the buffer size minus its
reserved space).
It also adds support for the "Expect: 100-Continue" header.
I have written a small patch to enable a correct PostgreSQL health check
It works similar to mysql-check with the very same parameters.
E.g.:
listen pgsql 127.0.0.1:5432
mode tcp
option pgsql-check user pgsql
server masterdb pgsql.server.com:5432 check inter 10000
One of the requirements we have is to run multiple instances of haproxy on a
single host; this is so that we can split the responsibilities (and change
permissions) between product teams. An issue we ran up against is how we
would distinguish between the logs generated by each instance. The solution
we came up with (please let me know if there is a better way) is to override
the application tag written to syslog. We can then configure syslog to write
these to different files.
I have attached a patch adding a global option 'log-tag' to override the
default syslog tag 'haproxy' (actually defaults to argv[0]).
By passing a negative value to the "mss" argument of "bind" lines, it
becomes possible to subtract this value to the MSS advertised by the
client, which results in segments smaller than advertised. The effect
is useful with some TCP stacks which ACK less often when segments are
not full, because they only ACK every other full segment as suggested
by RFC1122.
NOTE: currently this has no effect on Linux kernel 2.6, a kernel patch
is still required to change the MSS of established connections.
Haproxy does not include the hostname rather the IP of the machine in
the syslog headers it sends. Unfortunately this means that for each log
line rsyslog does a reverse dns on the client IP and in the case of
non-routable IPs one gets the public hostname not the internal one.
While this is valid according to RFC3164 as one might imagine this is
troublsome if you have some machines with public IPs, internal IPs, no
reverse DNS entries, etc and you want a standardized hostname based log
directory structure. The rfc says the preferred value is the hostname.
This patch adds a global "log-send-hostname" statement which accepts an
optional string to force the host name. If unset, the local host name
is used.
We've had several issues related to data transfers. First, if a
client aborted an upload before the server started to respond, it
would get a 502 followed by a 400. The same was true (in the other
way around) if the server suddenly aborted while the client was
uploading the data.
The flags reported in the logs were misleading. Request errors could
be reported while the transfer was stopped during the data phase. The
status codes could also be overwritten by a 400 eventhough the start
of the response was transferred to the client.
The stats were also wrong in case of data aborts. The server or the
client could sometimes be miscredited for being the author of the
abort depending on where the abort was detected. Some client aborts
could also be accounted as request errors and some server aborts as
response errors.
Now it seems like all such issues are fixed. Since we don't have a
specific state for data flowing from the client to the server
before the server responds, we're still counting the client aborted
transfers as "CH", and they become "CD" when the server starts to
respond. Ideally a "P" state would be desired.
This patch should be backported to 1.4.
Using haproxy in multi-process mode (nbproc > 1), some features can be
not fully compatible or not work at all. haproxy will now display a warning on
startup for :
- appsession
- sticking rules
- stats / stats admin
- stats socket
- peers (fatal error in that case)
During the documentation of the "ignore-persist" keyword, I documented an
invalid "option ignore-persist" and forgot to remove it. It's time to fix it.
When the number of servers is a multiple of the size of the input set,
map-based hash can be inefficient. This typically happens with 64
servers when doing URI hashing. The "avalanche" hash-type applies an
avalanche hash before performing a map lookup in order to smooth the
distribution. The result is slightly less smooth than the map for small
numbers of servers, but still better than the consistent hashing.
Released version 1.5-dev3 with the following main changes :
- [DOC] fix http-request documentation
- [MEDIUM] enable/disable servers from the stats web interface
- [MEDIUM] stats: add an admin level
- [DOC] stats: document the "stats admin" statement
- [MINOR] startup: print the proxy socket which caused an error
- [CLEANUP] Remove unneeded chars allocation
- [MINOR] config: detect options not supported due to compilation options
- [MINOR] Add pattern's fetchs payload and payload_lv
- [MINOR] frontend: improve accept-proxy header parsing
- [MINOR] frontend: add tcpv6 support on accept-proxy bind
- [MEDIUM] Enhance message errors management on binds
- [MINOR] Manage unix socket source field on logs
- [MINOR] Manage unix socket source field on session dump on sock stats
- [MINOR] Support of unix listener sockets for debug and log event messages on frontend.c
- [MINOR] Add some tests on sockets family for port remapping and mode transparent.
- [MINOR] Manage socket type unix for some logs
- [MINOR] Enhance controls of socket's family on acls and pattern fetch
- [MINOR] Support listener's sockets unix on http logs.
- [MEDIUM] Add supports of bind on unix sockets.
- [BUG] stick table purge failure if size less than 255
- [BUG] stick table entries expire on counters updates/read or show table, even if there is no "expire" parameter
- [MEDIUM] Implement tcp inspect response rules
- [DOC] tcp-response content and inspect
- [MINOR] new acls fetch req_ssl_hello_type and rep_ssl_hello_type
- [DOC] acls rep_ssl_hello and req_ssl_hello
- [MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated.
- [DOC] new type binary in stick-table
- [DOC] stick store-response and new patterns payload and payload_lv
- [MINOR] Manage all types (ip, integer, string, binary) on cli "show table" command
- [MEDIUM] Create updates tree on stick table to manage sync.
- [MAJOR] Add new files src/peer.c, include/proto/peers.h and include/types/peers.h for sync stick table management
- [MEDIUM] Manage peers section parsing and stick table registration on peers.
- [MEDIUM] Manage soft stop on peers proxy
- [DOC] add documentation for peers section
- [MINOR] checks: add support for LDAPv3 health checks
- [MINOR] add better support to "mysql-check"
- [BUG] Restore info about available active/backup servers
- [CONTRIB] Update haproxy.pl
- [CONTRIB] Update Cacti Tempates
- [CONTRIB] add templates for Cacti.
- [BUG] http: don't consider commas as a header delimitor within quotes
- [MINOR] support a global jobs counter
- [DOC] add a summary about cookie incompatibilities between specs and browsers
- [DOC] fix description of cookie "insert" and "indirect" modes
- [MEDIUM] http: fix space handling in the request cookie parser
- [MEDIUM] http: fix space handling in the response cookie parser
- [DOC] fix typo in the queue() definition (backend, not frontend)
- [BUG] deinit: unbind listeners before freeing them
- [BUG] stream_interface: only call si->release when both dirs are closed
- [MEDIUM] buffers: rework the functions to exchange between SI and buffers
- [DOC] fix typo in the avg_queue() and be_conn() definition (backend, not frontend)
- [MINOR] halog: add '-tc' to sort by termination codes
- [MINOR] halog: skip non-traffic logs for -st and -tc
- [BUG] stream_sock: cleanly disable the listener in case of resource shortage
- [BUILD] stream_sock: previous fix lacked the #include, causing a warning.
- [DOC] bind option is "defer-accept", not "defer_accept"
- [DOC] missing index entry for http-check send-state
- [DOC] tcp-request inspect-delay is for backends too
- [BUG] ebtree: string_equal_bits() could return garbage on identical strings
- [BUG] stream_sock: try to flush any extra pending request data after a POST
- [BUILD] proto_http: eliminate some build warnings with gcc-2.95
- [MEDIUM] make it possible to combine http-pretend-keepalived with httpclose
- [MEDIUM] tcp-request : don't wait for inspect-delay to expire when the buffer is full
- [MEDIUM] checks: add support for HTTP contents lookup
- [TESTS] add test-check-expect to test various http-check methods
- [MINOR] global: add "tune.chksize" to change the default check buffer size
- [MINOR] cookie: add options "maxidle" and "maxlife"
- [MEDIUM] cookie: support client cookies with some contents appended to their value
- [MINOR] http: make some room in the transaction flags to extend cookies
- [MINOR] cookie: add the expired (E) and old (O) flags for request cookies
- [MEDIUM] cookie: reassign set-cookie status flags to store more states
- [MINOR] add encode/decode function for 30-bit integers from/to base64
- [MEDIUM] cookie: check for maxidle and maxlife for incoming dated cookies
- [MEDIUM] cookie: set the date in the cookie if needed
- [DOC] document the cookie maxidle and maxlife parameters
- [BUG] checks: don't log backend down for all zero-weight servers
- [MEDIUM] checks: set server state to one state from failure when leaving maintenance
- [BUG] config: report correct keywords for "observe"
- [MINOR] checks: ensure that we can inherit binary checks from the defaults section
- [MINOR] acl: add the http_req_first match
- [DOC] fix typos about bind-process syntax
- [BUG] cookie: correctly unset default cookie parameters
- [MINOR] cookie: add support for the "preserve" option
- [BUG] ebtree: fix duplicate strings insertion
- [CONTRIB] halog: report per-url counts, errors and times
- [CONTRIB] halog: minor speed improvement in timer parser
- [MINOR] buffers: add a new request analyser flag for PROXY mode
- [MINOR] listener: add the "accept-proxy" option to the "bind" keyword
- [MINOR] standard: add read_uint() to parse a delimited unsigned integer
- [MINOR] standard: change arg type from const char* to char*
- [MINOR] frontend: add a new analyser to parse a proxied connection
- [MEDIUM] session: call the frontend_decode_proxy analyser on proxied connections
- [DOC] add the proxy protocol's specifications
- [DOC] document the 'accept-proxy' bind option
- [MINOR] cfgparse: report support of <path> for the 'bind' statements
- [DOC] add references to unix socket handling
- [MINOR] move MAXPATHLEN definition to compat.h
- [MEDIUM] unix sockets: cleanup the error reporting path
- [BUG] session: don't stop forwarding of data upon last packet
- [CLEANUP] accept: replace some inappropriate Alert() calls with send_log()
- [BUILD] peers: shut a printf format warning (key_size is a size_t)
- [BUG] accept: don't close twice upon error
- [OPTIM] session: don't recheck analysers when buffer flags have not changed
- [OPTIM] stream_sock: don't clear FDs that are already cleared
- [BUG] proto_tcp: potential bug on pattern fetch dst and dport
This option makes haproxy preserve any persistence cookie emitted by
the server, which allows the server to change it or to unset it, for
instance, after a logout request.
(cherry picked from commit 52e6d75374c7900c1fe691c5633b4ae029cae8d5)
This match returns true when the request calling it is the first one of
a connection.
(cherry picked from commit 922ca979c50653c415852531f36fe409190ad76b)
The MySQL check has been revamped to be able to send real MySQL data,
and to avoid Aborted connects on MySQL side.
It is however backward compatible with older version, but it is highly
recommended to use the new mode, by adding "user <username>" on the
"mysql-check" line.
The new check consists in sending two MySQL packet, one Client
Authentication packet, with "haproxy" username (by default), and one
QUIT packet, to correctly close MySQL session. We then parse the Mysql
Handshake Initialisation packet and/or Error packet. It is a basic but
useful test which does not produce error nor aborted connect on the
server.
(cherry picked from commit a1e4dcfe5718311b7653d7dabfad65c005d0439b)
- fix some erroneous keywords for http-request
- stats http-request was not documented
--
Cyril Bont
(cherry picked from commit 17e07dad5074571c38cbeb92d54a20f83658449d)
HTTP content-based health checks will be involved in searching text in pages.
Some pages may not fit in the default buffer (16kB) and sometimes it might be
desired to have larger buffers in order to find patterns. Running checks on
smaller URIs is always preferred of course.
(cherry picked from commit 043f44aeb835f3d0b57626c4276581a73600b6b1)
This patch adds the "http-check expect [r]{string,status}" statements
which enable health checks based on whether the response status or body
to an HTTP request contains a string or matches a regex.
This probably is one of the oldest patches that remained unmerged. Over
the time, several people have contributed to it, among which FinalBSD
(first and second implementations), Nick Chalk (port to 1.4), Anze
Skerlavaj (tests and fixes), Cyril Bont (general fixes), and of course
myself for the final fixes and doc during integration.
Some people already use an old version of this patch which has several
issues, among which the inability to search for a plain string that is
not at the beginning of the data, and the inability to look for response
contents that are provided in a second and subsequent recv() calls. But
since some configs are already deployed, it was quite important to ensure
a 100% compatible behaviour on the working cases.
Thus, that patch fixes the issues while maintaining config compatibility
with already deployed versions.
(cherry picked from commit b507c43a3ce9a8e8e4b770e52e4edc20cba4c37f)
This patch provides a new "option ldap-check" statement to enable
server health checks based on LDAPv3 bind requests.
(cherry picked from commit b76b44c6fed8a7ba6f0f565dd72a9cb77aaeca7c)
Some configs may involve httpclose in a frontend and http-pretend-keepalive
in a backend. httpclose used to take priority over keepalive, thus voiding
its effect. This change ensures that when both are combined, keepalive is
still announced to the server while close is announced to the client.
(cherry picked from commit 2be7ec90fa9caf66294f446423bbab2d00db9004)
The doc was wrong as the insert mode by default does not insert in
direct requests, and by default transmits the cookies to the server.
This was right in the old doc and it has not changed since the
beginning.
Released version 1.5-dev2 with the following main changes :
- [MINOR] startup: release unused structs after forking
- [MINOR] startup: don't wait for nothing when no old pid remains
- [CLEANUP] reference product branch 1.5
- [MEDIUM] signals: add support for registering functions and tasks
- [MEDIUM] signals: support redistribution of signal zero when stopping
- [BUG] http: don't set auto_close if more data are expected
Released version 1.5-dev1 with the following main changes :
- [BUG] stats: session rate limit gets garbaged in the stats
- [DOC] mention 'option http-server-close' effect in Tq section
- [DOC] summarize and highlight persistent connections behaviour
- [DOC] add configuration samples
- [BUG] http: dispatch and http_proxy modes were broken for a long time
- [BUG] http: the transaction must be initialized even in TCP mode
- [BUG] tcp: dropped connections must be counted as "denied" not "failed"
- [BUG] consistent hash: balance on all servers, not only 2 !
- [CONTRIB] halog: report per-server status codes, errors and response times
- [BUG] http: the transaction must be initialized even in TCP mode (part 2)
- [BUG] client: always ensure to zero rep->analysers
- [BUG] session: clear BF_READ_ATTACHED before next I/O
- [BUG] http: automatically close response if req is aborted
- [BUG] proxy: connection rate limiting was eating lots of CPU
- [BUG] http: report correct flags in case of client aborts during body
- [TESTS] refine non-regression tests and add 4 new tests
- [BUG] debug: wrong pointer was used to report a status line
- [BUG] debug: correctly report truncated messages
- [DOC] document the "dispatch" keyword
- [BUG] stick_table: fix possible memory leak in case of connection error
- [CLEANUP] acl: use 'L6' instead of 'L4' in ACL flags relying on contents
- [MINOR] accept: count the incoming connection earlier
- [CLEANUP] tcp: move some non tcp-specific layer6 processing out of proto_tcp
- [CLEANUP] client: move some ACLs away to their respective locations
- [CLEANUP] rename client -> frontend
- [MEDIUM] separate protocol-level accept() from the frontend's
- [MINOR] proxy: add a list to hold future layer 4 rules
- [MEDIUM] config: parse tcp layer4 rules (tcp-request accept/reject)
- [MEDIUM] tcp: check for pure layer4 rules immediately after accept()
- [OPTIM] frontend: tell the compiler that errors are unlikely to occur
- [MEDIUM] frontend: check for LI_O_TCP_RULES in the listener
- [MINOR] frontend: only check for monitor-net rules if LI_O_CHK_MONNET is set
- [CLEANUP] buffer->cto is not used anymore
- [MEDIUM] session: finish session establishment sequence in with I/O handlers
- [MEDIUM] session: initialize server-side timeouts after connect()
- [MEDIUM] backend: initialize the server stream_interface upon connect()
- [MAJOR] frontend: don't initialize the server-side stream_int anymore
- [MEDIUM] session: move the conn_retries attribute to the stream interface
- [MEDIUM] session: don't assign conn_retries upon accept() anymore
- [MINOR] frontend: rely on the frontend and not the backend for INDEPSTR
- [MAJOR] frontend: reorder the session initialization upon accept
- [MINOR] proxy: add an accept() callback for the application layer
- [MAJOR] frontend: split accept() into frontend_accept() and session_accept()
- [MEDIUM] stats: rely on the standard session_accept() function
- [MINOR] buffer: refine the flags that may wake an analyser up.
- [MINOR] stream_sock: don't dereference a non-existing frontend
- [MINOR] session: differenciate between accepted connections and received connections
- [MEDIUM] frontend: count the incoming connection earlier
- [MINOR] frontend: count denied TCP requests separately
- [CLEANUP] stick_table: add/clarify some comments
- [BUILD] memory: add a few missing parenthesis to the pool management macros
- [MINOR] stick_table: add support for variable-sized data
- [CLEANUP] stick_table: rename some stksess struct members to avoid confusion
- [CLEANUP] stick_table: move pattern to key functions to stick_table.c
- [MEDIUM] stick_table: add room for extra data types
- [MINOR] stick_table: add support for "conn_cum" data type.
- [MEDIUM] stick_table: don't overwrite data when storing an entry
- [MINOR] config: initialize stick tables after all the parsing
- [MINOR] stick_table: provide functions to return stksess data from a type
- [MEDIUM] stick_table: move the server ID to a generic data type
- [MINOR] stick_table: enable it for frontends too
- [MINOR] stick_table: export the stick_table_key
- [MINOR] tcp: add per-source connection rate limiting
- [MEDIUM] stick_table: separate storage and update of session entries
- [MEDIUM] stick-tables: add a reference counter to each entry
- [MINOR] session: add a pointer to the tracked counters for the source
- [CLEANUP] proto_tcp: make the config parser a little bit more flexible
- [BUG] config: report the correct proxy type in tcp-request errors
- [MINOR] config: provide a function to quote args in a more friendly way
- [BUG] stick_table: the fix for the memory leak caused a regression
- [MEDIUM] backend: support servers on 0.0.0.0
- [BUG] stick-table: correctly refresh expiration timers
- [MEDIUM] stream-interface: add a ->release callback
- [MINOR] proxy: add a "parent" member to the structure
- [MEDIUM] session: make it possible to call an I/O handler on both SI
- [MINOR] tools: add a fast div64_32 function
- [MINOR] freq_ctr: add new types and functions for periods different from 1s
- [MINOR] errors: provide new status codes for config parsing functions
- [BUG] http: denied requests must not be counted as denied resps in listeners
- [MINOR] tools: add a get_std_op() function to parse operators
- [MEDIUM] acl: make use of get_std_op() to parse intger ranges
- [MAJOR] stream_sock: better wakeup conditions on read()
- [BUG] session: analysers must be checked when SI state changes
- [MINOR] http: reset analysers to listener's, not frontend's
- [MEDIUM] session: support "tcp-request content" rules in backends
- [BUILD] always match official tags when doing git-tar
- [MAJOR] stream_interface: fix the wakeup conditions for embedded iohandlers
- [MEDIUM] buffer: make buffer_feed* support writing non-contiguous chunks
- [MINOR] tcp: src_count acl does not have a permanent result
- [MAJOR] session: add track-counters to track counters related to the session
- [MINOR] stick-table: provide a table lookup function
- [MINOR] stick-table: use suffix "_cnt" for cumulated counts
- [MEDIUM] session: move counter ACL fetches from proto_tcp
- [MEDIUM] session: add concurrent connections counter
- [MEDIUM] session: add data in and out volume counters
- [MINOR] session: add the trk_conn_cnt ACL keyword to track connection counts
- [MEDIUM] session-counters: automatically update tracked connection count
- [MINOR] session: add the trk_conn_cur ACL keyword to track concurrent connection
- [MINOR] session: add trk_kbytes_* ACL keywords to track data size
- [MEDIUM] session: add a counter on the cumulated number of sessions
- [MINOR] config: support a comma-separated list of store data types in stick-table
- [MEDIUM] stick-tables: add support for arguments to data_types
- [MEDIUM] stick-tables: add stored data argument type checking
- [MEDIUM] session counters: add conn_rate and sess_rate counters
- [MEDIUM] session counters: add bytes_in_rate and bytes_out_rate counters
- [MINOR] stktable: add a stktable_update_key() function
- [MINOR] session-counters: add a general purpose counter (gpc0)
- [MEDIUM] session-counters: add HTTP req/err tracking
- [MEDIUM] stats: add "show table [<name>]" to dump a stick-table
- [MEDIUM] stats: add "clear table <name> key <value>" to clear table entries
- [CLEANUP] stick-table: declare stktable_data_types as extern
- [MEDIUM] stick-table: make use of generic types for stored data
- [MINOR] stats: correctly report errors on "show table" and "clear table"
- [MEDIUM] stats: add the ability to dump table entries matching criteria
- [DOC] configuration: document all the new tracked counters
- [DOC] stats: document "show table" and "clear table"
- [MAJOR] session-counters: split FE and BE track counters
- [MEDIUM] tcp: accept the "track-counters" in "tcp-request content" rules
- [MEDIUM] session counters: automatically remove expired entries.
- [MEDIUM] config: replace 'tcp-request <action>' with "tcp-request connection"
- [MEDIUM] session-counters: make it possible to count connections from frontend
- [MINOR] session-counters: use "track-sc{1,2}" instead of "track-{fe,be}-counters"
- [MEDIUM] session-counters: correctly unbind the counters tracked by the backend
- [CLEANUP] stats: use stksess_kill() to remove table entries
- [DOC] update the references to session counters and to tcp-request connection
- [DOC] cleanup: split a few long lines
- [MEDIUM] http: forward client's close when abortonclose is set
- [BUG] queue: don't dequeue proxy-global requests on disabled servers
- [BUG] stats: global stats timeout may be specified before stats socket.
- [BUG] conf: add tcp-request content rules to the correct list
It began to be problematic to have "tcp-request" followed by an
immediate action, as sometimes it was a keyword indicating a hook
or setting ("content" or "inspect-delay") and sometimes it was an
action.
Now the prefix for connection-level tcp-requests is "tcp-request connection"
and the ones processing contents remain "tcp-request contents".
This has allowed a nice simplification of the config parser and to
clean up the doc a bit. Also now it's a bit more clear why tcp-request
connection are not allowed in backends.
The following statements and ACLs have been added to the doc :
- tcp-request track-counters <key> [table <table>] [{if | unless} <condition>]
- src_*
- trk_*
The "_cnt" suffix is already used by ACLs to count various data,
so it makes sense to use the same one in "conn_cnt" instead of
"conn_cum" to count cumulated connections.
This is not a problem because no version was emitted with those
keywords.
Thus we'll try to stick to the following rules :
xxxx_cnt : cumulated event count for criterion xxxx
xxxx_cur : current number of concurrent entries for criterion xxxx
xxxx_rate: event rate for criterion xxxx
Sometimes it's necessary to be able to perform some "layer 6" analysis
in the backend. TCP request rules were not available till now, although
documented in the diagram. Enable them in backend now.
Till now when a server was configured with address 0.0.0.0, the
connection was forwarded to this address which generally is intercepted
by the system as a local address, so this was completely useless.
One sometimes useful feature for outgoing transparent proxies is to
be able to forward the connection to the same address the client
requested. This patch fixes the meaning of 0.0.0.0 precisely to
ensure that the connection will be forwarded to the initial client's
destination address.
configuration.txt is thorough and accurate but lacked sample configurations
clarifying both the syntax and the relations between global, defaults,
frontend, backend and listen sections. Besides, almost all examples to be found
in haproxy-en.txt or online tutorials make use of the 'listen' syntax while
'frontend/backend' is really the one to know about.
(cherry picked from commit 01ac10ad189b11c563eeb835733fba58e6c5271d)
This change makes use of the stick-tables to keep track of any source
address activity. Two ACLs make it possible to check the count of an
entry or update it and act accordingly. The typical usage will be to
reject a TCP request upon match of an excess value.
This one can be parsed on the "stick-table" after with the "store"
keyword. It will hold the number of connections matching the entry,
for use with ACLs or anything else.
The stick_tables will now be able to store extra data for a same key.
A limited set of extra data types will be defined and for each of them
an offset in the sticky session will be assigned at startup time. All
of this information will be stored in the stick table.
The extra data types will have to be specified after the new "store"
keyword of the "stick-table" directive, which will reserve some space
for them.
It's very disturbing to see the "denied req" counter increase without
any other session counter moving. In fact, we can't count a rejected
TCP connection as "denied req" as we have not yet instanciated any
session at all. Let's use a new counter for that.
Released version 1.4.6 with the following main changes :
- [BUILD] ebtree: update to v6.0.1 to remove references to dprintf()
- [CLEANUP] acl: make use of eb_is_empty() instead of open coding the tree's emptiness test
- [MINOR] acl: add srv_is_up() to check that a specific server is up or not
- [DOC] add a few precisions about the use of RDP cookies
RDP cookies are not necessarily easy to implement because they require
some configuration on the servers. Add a few hints so that people know
what to check on their servers.
This ACL was missing in complex setups where the status of a remote site
has to be considered in switching decisions. Until there, using a server's
status in an ACL required to have a dedicated backend, which is a bit heavy
when multiple servers have to be monitored.
Released version 1.4.5 with the following main changes :
- [DOC] report minimum kernel version for tproxy in the Makefile
- [MINOR] add the "ignore-persist" option to conditionally ignore persistence
- [DOC] add the "ignore-persist" option to conditionally ignore persistence
- [DOC] fix ignore-persist/force-persist documentation
- [BUG] cttproxy: socket fd leakage in check_cttproxy_version
- [DOC] doc/configuration.txt: fix typos
- [MINOR] option http-pretend-keepalive is both for FEs and BEs
- [MINOR] fix possible crash in debug mode with invalid responses
- [MINOR] halog: add support for statisticts on status codes
- [OPTIM] halog: use a faster zero test in fgets()
- [OPTIM] halog: minor speedup by using unlikely()
- [OPTIM] halog: speed up fgets2-64 by about 10%
- [DOC] refresh the README file and merge the CONTRIB file into it
- [MINOR] acl: support loading values from files
- [MEDIUM] ebtree: upgrade to version 6.0
- [MINOR] acl trees: add flags and union members to store values in trees
- [MEDIUM] acl: add ability to insert patterns in trees
- [MEDIUM] acl: add tree-based lookups of exact strings
- [MEDIUM] acl: add tree-based lookups of networks
- [MINOR] acl: ignore empty lines and comments in pattern files
- [MINOR] stick-tables: add support for "stick on hdr"
It is now possible to stick on an IP address found in a HTTP header. Right
now only the last occurrence of the header can be used, which is generally
enough for most uses. Also, the header extraction rule only knows how to
convert the header to IP. Later it will be usable as a plain string with
an implicit conversion, and the syntax will not change.
Most often, pattern files used by ACLs will be produced by tools
which emit some comments (eg: geolocation lists). It's very annoying
to have to clean the files before using them, and it does not make
much sense to be able to support patterns we already can't input in
the config file. So this patch makes the pattern file loader skip
lines beginning with a sharp and the empty ones, and strips leading
spaces and tabs.
The "acl XXX -f <file>" syntax was supported but nothing was read from
the file. This is now possible. All lines are merged verbatim, even if
they contain spaces (useful for user-agents). There are shortcomings
though. The worst one is that error reporting is too approximative.
Shame on me, I didn't correctly document the "ignore-persist" statement
(convinced I used it like this in my tests, which is not the case at all...)
This fixes the doc and updates the proxy keyword matrix to add "force-persist".
Released version 1.4.4 with the following main changes :
- [BUG] appsession should match the whole cookie name
- [CLEANUP] proxy: move PR_O_SSL3_CHK to options2 to release one flag
- [MEDIUM] backend: move the transparent proxy address selection to backend
- [MINOR] add very fast IP parsing functions
- [MINOR] add new tproxy flags for dynamic source address binding
- [MEDIUM] add ability to connect to a server from an IP found in a header
- [BUILD] config: last patch breaks build without CONFIG_HAP_LINUX_TPROXY
- [MINOR] http: make it possible to pretend keep-alive when doing close
- [MINOR] config: report "default-server" instead of "(null)" in error messages
Some servers do not completely conform with RFC2616 requirements for
keep-alive when they receive a request with "Connection: close". More
specifically, they don't bother using chunked encoding, so the client
never knows whether the response is complete or not. One immediately
visible effect is that haproxy cannot maintain client connections alive.
The second issue is that truncated responses may be cached on clients
in case of network error or timeout.
scar Fras Barranco reported this issue on Tomcat 6.0.20, and
Patrik Nilsson with Jetty 6.1.21.
Cyril Bont proposed this smart idea of pretending we run keep-alive
with the server and closing it at the last moment as is already done
with option forceclose. The advantage is that we only change one
emitted header but not the overall behaviour.
Since some servers such as nginx are able to close the connection
very quickly and save network packets when they're aware of the
close negociation in advance, we don't enable this behaviour by
default.
"option http-pretend-keepalive" will have to be used for that, in
conjunction with "option http-server-close".
Using get_ip_from_hdr2() we can look for occurrence #X or #-X and
extract the IP it contains. This is typically designed for use with
the X-Forwarded-For header.
Using "usesrc hdr_ip(name,occ)", it becomes possible to use the IP address
found in <name>, and possibly specify occurrence number <occ>, as the
source to connect to a server. This is possible both in a server and in
a backend's source statement. This is typically used to use the source
IP previously set by a upstream proxy.
Released version 1.4.3 with the following main changes :
- [CLEANUP] stats: remove printf format warning in stats_dump_full_sess_to_buffer()
- [MEDIUM] session: better fix for connection to servers with closed input
- [DOC] indicate in the doc how to bind to port ranges
- [BUG] backend: L7 hashing must not be performed on incomplete requests
- [TESTS] add a simple program to test connection resets
- [MINOR] cli: "show errors" should display "backend <NONE>" when backend was not used
- [MINOR] config: emit warnings when HTTP-only options are used in TCP mode
- [MINOR] config: allow "slowstart 0s"
- [BUILD] 'make tags' did not consider files ending in '.c'
- [MINOR] checks: add the ability to disable a server in the config
Released version 1.4.2 with the following main changes :
- [CLEANUP] product branch update
- [DOC] Some more documentation cleanups
- [BUG] clf logs segfault when capturing a non existant header
- [OPTIM] config: only allocate check buffer when checks are enabled
- [MEDIUM] checks: support multi-packet health check responses
- [CLEANUP] session: remove duplicate test
- [BUG] http: don't wait for response data to leave buffer is client has left
- [MINOR] proto_uxst: set accept_date upon accept() to the wall clock time
- [MINOR] stats: don't send empty lines in "show errors"
- [MINOR] stats: make the data dump function reusable for other purposes
- [MINOR] stats socket: add show sess <id> to dump details about a session
- [BUG] stats: connection reset counters must be plain ascii, not HTML
- [BUG] url_param hash may return a down server
- [MINOR] force null-termination of hostname
- [MEDIUM] connect to servers even when the input has already been closed
- [BUG] don't merge anonymous ACLs !
- [BUG] config: fix endless loop when parsing "on-error"
- [MINOR] http: don't mark a server as failed when it returns 501/505
- [OPTIM] checks: try to detect the end of response without polling again
- [BUG] checks: don't report an error when recv() returns an error after data
- [BUG] checks: don't abort when second poll returns an error
- [MINOR] checks: make shutdown() silently fail
- [BUG] http: fix truncated responses on chunk encoding when size divides buffer size
- [BUG] init: unconditionally catch SIGPIPE
- [BUG] checks: don't wait for a close to start parsing the response
Since the last documentation cleanups, I've found more typos that I kept
in a corner instead of sending you a mail just for one character :)
--
Cyril Bont
When trying to spot some complex bugs, it's often needed to access
information on stuck sessions, which is quite difficult. This new
command helps one get detailed information about a session, with
flags, timers, states, etc... The buffer data are not dumped yet.
Released version 1.4.1 with the following main changes :
- [BUG] Clear-cookie path issue
- [DOC] fix typo on stickiness rules
- [BUILD] fix BSD and OSX makefiles for missing files
- [BUILD] includes order breaks OpenBSD build
- [BUILD] fix some build warnings on Solaris with is* macros
- [BUG] logs: don't report "last data" when we have just closed after an error
- [BUG] logs: don't report "proxy request" when server closes early
- [BUILD] fix platform-dependant build issues related to crypt()
- [STATS] count transfer aborts caused by client and by server
- [STATS] frontend requests were not accounted for failed requests
- [MINOR] report total number of processed connections when stopping a proxy
- [DOC] be more clear about the limitation to one single monitor-net entry
Often we need to understand why some transfers were aborted or what
constitutes server response errors. With those two counters, it is
now possible to detect an unexpected transfer abort during a data
phase (eg: too short HTTP response), and to know what part of the
server response errors may in fact be assigned to aborted transfers.
Released version 1.4.0 with the following main changes :
- [MINOR] stats: report maint state for tracking servers too
- [DOC] fix summary to add pattern extraction
- [DOC] Documentation cleanups
- [BUG] cfgparse memory leak and missing free calls in deinit()
- [BUG] pxid/puid/luid: don't shift IDs when some of them are forced
- [EXAMPLES] add auth.cfg
- [BUG] uri_auth: ST_SHLGNDS should be 0x00000008 not 0x0000008
- [BUG] uri_auth: do not attemp to convert uri_auth -> http-request more than once
- [BUILD] auth: don't use unnamed unions
- [BUG] config: report unresolvable host names as errors
- [BUILD] fix build breakage with DEBUG_FULL
- [DOC] fix a typo about timeout check and clarify the explanation.
- [MEDIUM] http: don't use trash to realign large buffers
- [STATS] report HTTP requests (total and rate) in frontends
- [STATS] separate frontend and backend HTTP stats
- [MEDIUM] http: revert to use a swap buffer for realignment
- [MINOR] stats: report the request rate in frontends as cell titles
- [MINOR] stats: mark areas with an underline when tooltips are available
- [DOC] reorder some entries to maintain the alphabetical order
- [DOC] cleanup of the keyword matrix
Hi Willy,
This patch make some cleanups on the documentation :
- strictly respect the 80 columns constraint
- use space indentation everywhere
- add some missing "(deprecated)"
- fix the "monitor fail" keyword syntax
- fix a keyword in a "See Also" line (interval instead of inter)
- the "no" prefix of "option http-use-proxy-header" is displayed as the others
I've also noticed that 3 keywords are missing from the keywords reference (I don't know what to do for that) :
- dispatch
- reqsetbe
- reqisetbe
--
Cyril Bont
Released version 1.4-rc1 with the following main changes :
- [MEDIUM] add a maintenance mode to servers
- [MINOR] http-auth: last fix was wrong
- [CONTRIB] add base64rev-gen.c that was used to generate the base64rev table.
- [MINOR] Base64 decode
- [MINOR] generic auth support with groups and encrypted passwords
- [MINOR] add ACL_TEST_F_NULL_MATCH
- [MINOR] http-request: allow/deny/auth support for frontend/backend/listen
- [MINOR] acl: add http_auth and http_auth_group
- [MAJOR] use the new auth framework for http stats
- [DOC] add info about userlists, http-request and http_auth/http_auth_group acls
- [STATS] make it possible to change a CLI connection timeout
- [BUG] patterns: copy-paste typo in type conversion arguments
- [MINOR] pattern: make the converter more flexible by supporting void* and int args
- [MINOR] standard: str2mask: string to netmask converter
- [MINOR] pattern: add support for argument parsers for converters
- [MINOR] pattern: add the "ipmask()" converting function
- [MINOR] config: off-by-one in "stick-table" after list of converters
- [CLEANUP] acl, patterns: make use of my_strndup() instead of malloc+memcpy
- [BUG] restore accidentely removed line in last patch !
- [MINOR] checks: make the HTTP check code add the CRLF itself
- [MINOR] checks: add the server's status in the checks
- [BUILD] halog: make without arch-specific optimizations
- [BUG] halog: fix segfault in case of empty log in PCT mode (cherry picked from commit fe362fe476)
- [MINOR] http: disable keep-alive when process is going down
- [MINOR] acl: add build_acl_cond() to make it easier to add ACLs in config
- [CLEANUP] config: use build_acl_cond() instead of parse_acl_cond()
- [CLEANUP] config: use warnif_cond_requires_resp() to check for bad ACLs
- [MINOR] prepare req_*/rsp_* to receive a condition
- [CLEANUP] config: specify correct const char types to warnif_* functions
- [MEDIUM] config: factor out the parsing of 20 req*/rsp* keywords
- [MEDIUM] http: make the request filter loop check for optional conditions
- [MEDIUM] http: add support for conditional request filter execution
- [DOC] add some build info about the AIX platform (cherry picked from commit e41914c77e)
- [MEDIUM] http: add support for conditional request header addition
- [MEDIUM] http: add support for conditional response header rewriting
- [DOC] add some missing ACLs about response header matching
- [MEDIUM] http: add support for proxy authentication
- [MINOR] http-auth: make the 'unless' keyword work as expected
- [CLEANUP] config: use build_acl_cond() to simplify http-request ACL parsing
- [MEDIUM] add support for anonymous ACLs
- [MEDIUM] http: switch to tunnel mode after status 101 responses
- [MEDIUM] http: stricter processing of the CONNECT method
- [BUG] config: reset check request to avoid double free when switching to ssl/sql
- [MINOR] config: fix too large ssl-hello-check message.
- [BUG] fix error response in case of server error
A 101 response is accompanied with an Upgrade header indicating
a new protocol that is spoken on the connection after the exchange
completes. At least we should switch to tunnel mode after such a
response.
Anonymous ACLs allow the declaration of rules which rely directly on
ACL expressions without passing via the declaration of an ACL. Example :
With named ACLs :
acl site_dead nbsrv(dynamic) lt 2
acl site_dead nbsrv(static) lt 2
monitor fail if site_dead
With anonymous ACLs :
monitor fail if { nbsrv(dynamic) lt 2 } || { nbsrv(static) lt 2 }
This is a first attempt to add a maintenance mode on servers, using
the stat socket (in admin level).
It can be done with the following command :
- disable server <backend>/<server>
- enable server <backend>/<server>
In this mode, no more checks will be performed on the server and it
will be marked as a special DOWN state (MAINT).
If some servers were tracking it, they'll go DOWN until the server
leaves the maintenance mode. The stats page and the CSV export also
display this special state.
This can be used to disable the server in haproxy before doing some
operations on this server itself. This is a good complement to the
"http-check disable-on-404" keyword and works in TCP mode.
We're already able to know if a request is a proxy request or a
normal one, and we have an option "http-use-proxy-header" which states
that proxy headers must be checked. So let's switch to use the proxy
authentication headers and responses when this option is set and we're
facing a proxy request. That allows haproxy to enforce auth in front
of a proxy.
Just as for the req* rules, we can now condition rsp* rules with ACLs.
ACLs match on response, so volatile request information cannot be used.
A warning is emitted if a configuration contains such an anomaly.
All the req* rules except the reqadd rules can now be specified with
an if/unless condition. If a condition is specified and does not match,
the filter is ignored. This is particularly useful with reqidel, reqirep
and reqtarpit.
Now a server can check the contents of the header X-Haproxy-Server-State
to know how haproxy sees it. The same values as those reported in the stats
are provided :
- up/down status + check counts
- throttle
- weight vs backend weight
- active sessions vs backend sessions
- queue length
- haproxy node name
This converter can be applied on top of an IPv4-type pattern. It
applies a netmask which is suited for IP address storage and matching.
This can be used to make all hosts within a certain mask to share the
same table entries and as such use the same server.
The mask can be passed in dotted form (eg: 255.255.255.0) or in CIDR
form (eg: 24).
Hi Willy,
I've made a quick pass on the "defaults" column in the Proxy keywords matrix (chapter 4.1. in the documentation).
This patch resyncs the code and the documentation. I let you decide if some keywords that still work in the "defaults" section should be forbidden.
- default_backend : in the matrix, "defaults" was not supported but the keyword details say it is.
Tests also shows it works, then I've updated the matrix.
- capture cookie : in the keyword details, we can read `It is not possible to specify a capture in a "defaults" section.'.
Ok, even if the tests worked, I've added an alert in the configuration parser (as it is for capture request/response header).
- description : not supported in "defaults", I added an alert in the parser.
I've also noticed that this keyword doesn't appear in the documentation.
There's one "description" entry, but for the "global" section, which is for a different use (the patch doesn't update the documentation).
- grace : even if this is maybe useless, it works in "defaults". Documentation is updated.
- redirect : alert is added in the parser.
- rsprep : alert added in the parser.
--
Cyril Bont
Despite what is explicitly stated in HTTP specifications,
browsers still use the undocumented Proxy-Connection header
instead of the Connection header when they connect through
a proxy. As such, proxies generally implement support for
this stupid header name, breaking the standards and making
it harder to support keep-alive between clients and proxies.
Thus, we add a new "option http-use-proxy-header" to tell
haproxy that if it sees requests which look like proxy
requests, it should use the Proxy-Connection header instead
of the Connection header.
Released version 1.4-dev7 with the following main changes :
- [BUG] appsession: possible memory leak in case of out of memory condition
- [MINOR] config: don't accept 'appsession' in defaults section
- [MINOR] Add function to parse a size in configuration
- [MEDIUM] Add stick table (persistence) management functions and types
- [MEDIUM] Add pattern fetch management types and functions
- [MEDIUM] Add src dst and dport pattern fetches.
- [MEDIUM] Add stick table configuration and init.
- [MEDIUM] Add stick and store rules analysers.
- [MINOR] add option "mysql-check" to use MySQL health checks
- [BUG] health checks: fix requeued message
- [OPTIM] remove SSP_O_VIA and SSP_O_STATUS
- [BUG] checks: fix newline termination
- [MINOR] acl: add fe_id/so_id to match frontend's and socket's id
- [BUG] appsession's sessid must be reset at end of transaction
- [BUILD] appsession did not build anymore under gcc-2.95
- [BUG] server redirection used an uninitialized string.
- [MEDIUM] http: fix handling of message pointers
- [MINOR] http: fix double slash prefix with server redirect
- [MINOR] http redirect: add the ability to append a '/' to the URL
- [BUG] stream_interface: fix retnclose and remove cond_close
- [MINOR] http redirect: don't explicitly state keep-alive on 1.1
- [MINOR] http: move appsession 'sessid' from session to http_txn
- [OPTIM] reorder http_txn to optimize cache lines placement
- [MINOR] http: differentiate waiting for new request and waiting for a complete requst
- [MINOR] http: add a separate "http-keep-alive" timeout
- [MINOR] config: remove undocumented and buggy 'timeout appsession'
- [DOC] fix various too large lines
- [DOC] remove several trailing spaces
- [DOC] add the doc about stickiness
- [BUILD] remove a warning in standard.h on AIX
- [BUG] checks: chars are unsigned on AIX, check was always true
- [CLEANUP] stream_sock: MSG_NOSIGNAL is only for send(), not recv()
- [BUG] check: we must not check for error before reading a response
- [BUG] buffers: remove remains of wrong obsolete length check
- [OPTIM] stream_sock: don't shutdown(write) when the socket is in error
- [BUG] http: don't count req errors on client resets or t/o during keep-alive
- [MEDIUM] http: don't switch to tunnel mode upon close
- [DOC] add documentation about connection header processing
- [MINOR] http: add http_remove_header2() to remove a header value.
- [MINOR] tools: add a "word_match()" function to match words and ignore spaces
- [MAJOR] http: rework request Connection header handling
- [MAJOR] http: rework response Connection header handling
- [MINOR] add the ability to force kernel socket buffer size.
- [BUG] http_server_error() must not purge a previous pending response
- [OPTIM] http: don't delay response if next request is incomplete
- [MINOR] add the "force-persist" statement to force persistence on down servers
- [MINOR] http: logs must report persistent connections to down servers
- [BUG] buffer_replace2 must never change the ->w entry
This is used to force access to down servers for some requests. This
is useful when validating that a change on a server correctly works
before enabling the server again.
Sometimes we need to be able to change the default kernel socket
buffer size (recv and send). Four new global settings have been
added for this :
- tune.rcvbuf.client
- tune.rcvbuf.server
- tune.sndbuf.client
- tune.sndbuf.server
Those can be used to reduce kernel memory footprint with large numbers
of concurrent connections, and to reduce risks of write timeouts with
very slow clients due to excessive kernel buffering.
This one is the next step of previous patch. It correctly computes
the response mode and the Connection flag transformations depending
on the request mode and version, and the response version and headers.
We're now also able to add "Connection: keep-alive", and to convert
server's close during a keep-alive connection to a server-close
connection.
The connection header is complex to handle, especially in the response
path, depending on request and response HTTP versions, desired mode,
etc... Let's document it. Note that only a subset of this document is
currently implemented.
Sometimes it can be desired to return a location which is the same
as the request with a slash appended when there was not one in the
request. A typical use of this is for sending a 301 so that people
don't reference links without the trailing slash. The name of the
new option is "append-slash" and it can be used on "redirect"
statements in prefix mode.
Released version 1.4-dev6 with the following main changes :
- [BUILD] warning in stream_interface.h
- [BUILD] warning ultoa_r returns char *
- [MINOR] hana: only report stats if it is enabled
- [MINOR] stats: add "a link" & "a href" for sockets
- [MINOR]: stats: add show-legends to report additional informations
- [MEDIUM] default-server support
- [BUG]: add 'observer', 'on-error', 'error-limit' to supported options list
- [MINOR] stats: add href to tracked server
- [BUG] stats: show UP/DOWN status also in tracking servers
- [DOC] Restore ability to search a keyword at the beginning of a line
- [BUG] stats: cookie should be reported under backend not under proxy
- [BUG] cfgparser/stats: fix error message
- [BUG] http: disable auto-closing during chunk analysis
- [BUG] http: fix hopefully last closing issue on data forwarding
- [DEBUG] add an http_silent_debug function to debug HTTP states
- [MAJOR] http: fix again the forward analysers
- [BUG] http_process_res_common() must not skip the forward analyser
- [BUG] http: some possible missed close remain in the forward chain
- [BUG] http: redirect needed to be updated after recent changes
- [BUG] http: don't set no-linger on response in case of forced close
- [MEDIUM] http: restore the original behaviour of option httpclose
- [TESTS] add a file to test various connection modes
- [BUG] http: check options before the connection header
- [MAJOR] session: fix the order by which the analysers are run
- [MEDIUM] session: also consider request analysers added during response
- [MEDIUM] http: make safer use of the DONT_READ and AUTO_CLOSE flags
- [BUG] http: memory leak with captures when using keep-alive
- [BUG] http: fix for capture memory leak was incorrect
- [MINOR] http redirect: use proper call to return last response
- [MEDIUM] http: wait for some flush of the response buffer before a new request
- [MEDIUM] session: limit the number of analyser loops
This patch implements default-server support allowing to change
default server options. It can be used in [defaults] or [backend]/[listen]
sections. Currently the following options are supported:
- error-limit
- fall
- inter
- fastinter
- downinter
- maxconn
- maxqueue
- minconn
- on-error
- port
- rise
- slowstart
- weight
Supported informations, available via "tr/td title":
- cap: capabilities (proxy)
- mode: one of tcp, http or health (proxy)
- id: SNMP ID (proxy, socket, server)
- IP (socket, server)
- cookie (backend, server)
Historically, "option httpclose" has always worked the same way. It
only mangles the "Connection" header in the request and the response
if needed, but does not affect the connection by itself, and ignores
any further data. It is dangerous to change this behaviour without
leaving any other alternative. If an active close is desired, it's
better to make use of "option forceclose" which does exactly what
it intends to do.
So as of now, "option httpclose" will only mangle the headers as
before, and will only affect the connection by itself when combined
with another connection-related option (eg: keepalive or server-close).
Released version 1.4-dev5 with the following main changes :
- [MINOR] server tracking: don't care about the tracked server's mode
- [MEDIUM] appsession: add "len", "prefix" and "mode" options
- [MEDIUM] appsession: add the "request-learn" option
- [BUG] Configuration parser bug when escaping characters
- [MINOR] CSS & HTML fun
- [MINOR] Collect & provide http response codes received from servers
- [BUG] Fix silly typo: hspr_other -> hrsp_other
- [MINOR] Add "a name" to stats page
- [MINOR] add additional "a href"s to stats page
- [MINOR] Collect & provide http response codes for frontends, fix backends
- [DOC] some small spell fixes and unifications
- [MEDIUM] Decrease server health based on http responses / events, version 3
- [BUG] format '%d' expects type 'int', but argument 5 has type 'long int'
- [BUG] config: fix erroneous check on cookie domain names, again
- [BUG] Healthchecks: get a proper error code if connection cannot be completed immediately
- [DOC] trivial fix for man page
- [MINOR] config: report all supported options for the "bind" keyword
- [MINOR] tcp: add support for the defer_accept bind option
- [MINOR] unix socket: report the socket path in case of bind error
- [CONTRIB] halog: support searching by response time
- [DOC] add a reminder about obsolete documents
- [DOC] point to 1.4 doc, not 1.3
- [DOC] option tcp-smart-connect was missing from index
- [MINOR] http: detect connection: close earlier
- [CLEANUP] sepoll: clean up the fd_clr/fd_set functions
- [OPTIM] move some rarely used fields out of fdtab
- [MEDIUM] fd: merge fd_list into fdtab
- [MAJOR] buffer: flag BF_DONT_READ to disable reads when not required
- [MINOR] http: add new transaction flags for keep-alive and content-length
- [MEDIUM] http request: parse connection, content-length and transfer-encoding
- [MINOR] http request: update the TX_SRV_CONN_KA flag on rewrite
- [MINOR] http request: simplify the test of no-data
- [MEDIUM] http request: simplify POST length detection
- [MEDIUM] http request: make use of pre-parsed transfer-encoding header
- [MAJOR] http: create the analyser which waits for a response
- [MINOR] http: pre-set the persistent flags in the transaction
- [MEDIUM] http response: check body length and set transaction flags
- [MINOR] http response: update the TX_CLI_CONN_KA flag on rewrite
- [MINOR] http: remove the last call to stream_int_return
- [IMPORT] import ebtree v5.0 into directory ebtree/
- [MEDIUM] build: switch ebtree users to use new ebtree version
- [CLEANUP] ebtree: remove old unused files
- [BUG] definitely fix regparm issues between haproxy core and ebtree
- [CLEANUP] ebtree: cast to char * to get rid of gcc warning
- [BUILD] missing #ifndef in ebmbtree.h
- [BUILD] missing #ifndef in ebsttree.h
- [MINOR] tools: add hex2i() function to convert hex char to int
- [MINOR] http: create new MSG_BODY sub-states
- [BUG] stream_sock: BUF_INFINITE_FORWARD broke splice on 64-bit platforms
- [DOC] option is "defer-accept", not "defer_accept"
- [MINOR] http: keep pointer to beginning of data
- [BUG] x-original-to: name was not set in default instance
- [MINOR] http: detect tunnel mode and set it in the session
- [BUG] config: fix error message when config file is not found
- [BUG] config: fix wrong handling of too large argument count
- [BUG] config: disable 'option httplog' on TCP proxies
- [BUG] config: fix erroneous check on cookie domain names
- [BUG] config: cookie domain was ignored in defaults sections
- [MINOR] config: support passing multiple "domain" statements to cookies
- [MINOR] ebtree: add functions to lookup non-null terminated strings
- [MINOR] config: don't report error on all subsequent files on failure
- [BUG] second fix for the printf format warning
- [BUG] check_post: limit analysis to the buffer length
- [MEDIUM] http: process request body in a specific analyser
- [MEDIUM] backend: remove HTTP POST parsing from get_server_ph_post()
- [MAJOR] http: completely process the "connection" header
- [MINOR] http: only consider chunk encoding with HTTP/1.1
- [MAJOR] buffers: automatically compute the maximum buffer length
- [MINOR] http: move the http transaction init/cleanup code to proto_http
- [MINOR] http: move 1xx handling earlier to eliminate a lot of ifs
- [MINOR] http: introduce a new synchronisation state : HTTP_MSG_DONE
- [MEDIUM] http: rework chunk-size parser
- [MEDIUM] http: add a new transaction flags indicating if we know the transfer length
- [MINOR] buffers: add buffer_ignore() to skip some bytes
- [BUG] http: offsets are relative to the buffer, not to ->som
- [MEDIUM] http: automatically re-aling request buffer
- [BUG] http: body parsing must consider the start of message
- [MINOR] new function stream_int_cond_close()
- [MAJOR] http: implement body parser
- [BUG] http: typos on several unlikely() around header insertion
- [BUG] stream_sock: wrong max computation on recv
- [MEDIUM] http: rework the buffer alignment logic
- [BUG] buffers: wrong size calculation for displaced data
- [MINOR] stream_sock: prepare for closing when all pending data are sent
- [MEDIUM] http: add two more states for the closing period
- [MEDIUM] http: properly handle "option forceclose"
- [MINOR] stream_sock: add SI_FL_NOLINGER for faster close
- [MEDIUM] http: make forceclose use SI_FL_NOLINGER
- [MEDIUM] session: set SI_FL_NOLINGER when aborting on write timeouts
- [MEDIUM] http: add some SI_FL_NOLINGER around server errors
- [MINOR] config: option forceclose is valid in frontends too
- [BUILD] halog: insufficient include path in makefile
- [MEDIUM] http: make the analyser not rely on msg being initialized anymore
- [MEDIUM] http: make the parsers able to wait for a buffer flush
- [MAJOR] http: add support for option http-server-close
- [BUG] http: ensure we abort data transfer on write error
- [BUG] last fix was overzealous and disabled server-close
- [BUG] http: fix erroneous trailers size computation
- [MINOR] stream_sock: enable MSG_MORE when forwarding finite amount of data
- [OPTIM] http: set MSG_MORE on response when a pipelined request is pending
- [BUG] http: redirects were broken by chunk changes
- [BUG] http: the request URI pointer is relative to the buffer
- [OPTIM] http: don't immediately enable reading on request
- [MINOR] http: move redirect messages to HTTP/1.1 with a content-length
- [BUG] http: take care of errors, timeouts and aborts during the data phase
- [MINOR] http: don't wait for sending requests to the server
- [MINOR] http: make the conditional redirect support keep-alive
- [BUG] http: fix cookie parser to support spaces and commas in values
- [MINOR] config: some options were missing for "redirect"
- [MINOR] redirect: add support for unconditional rules
- [MINOR] config: centralize proxy struct initialization
- [MEDIUM] config: remove the limitation of 10 reqadd/rspadd statements
- [MEDIUM] config: remove the limitation of 10 config files
- [CLEANUP] http: remove a remaining impossible condition
- [OPTIM] http: optimize a bit the construct of the forward loops
This option enables HTTP keep-alive on the client side and close mode
on the server side. This offers the best latency on the slow client
side, and still saves as many resources as possible on the server side
by actively closing connections. Pipelining is supported on both requests
and responses, though there is currently no reason to get pipelined
responses.
This option was disabled for frontends in the configuration because
it was useless in its initial implementation, though it was still
checked in the code. Let's officially enable it now.
The "forceclose" option used to close the output channel to the
server once it started to respond. While this happened to work with
most servers, some of them considered this as a connection abort and
immediately stopped responding.
Now that we're aware of the end of a request and response, we're able
to trivially handle this option and properly close both sides when the
server's response is complete.
During this change it appeared that forwarding could be allowed when
the BF_SHUTW_NOW flag was set on a buffer, which obviously is not
acceptable and was causing some trouble. This has been fixed too and
is the reason for the MEDIUM status on this patch.
Implement decreasing health based on observing communication between
HAProxy and servers.
Changes in this version 2:
- documentation
- close race between a started check and health analysis event
- don't force fastinter if it is not set
- better names for options
- layer4 support
Changes in this version 3:
- add stats
- port to the current 1.4 tree
In some environments it is not possible to rely on any wildcard for a
domain name (eg: .com, .net, .fr...) so it is required to send multiple
domain extensions. (Un)fortunately the syntax check on the domain name
prevented that from being done the dirty way. So let's just build a
domain list when multiple domains are passed on the same line.
(cherry picked from commit 950245ca2b)
To sum up :
- len : it's now the max number of characters for the value, preventing
garbaged results.
- a new option "prefix" is added, this allows to use dynamic cookie
names (e.g. ASPSESSIONIDXXX).
Previously in the thread, I wanted to use the value found with
"capture cookie" but when i started to update the documentation, I
found this solution quite weird. I've made a small rework to not
depend on "capture cookie".
- There's the posssiblity to define the URL parser mode (path parameters
or query string).
This patch has 2 goals :
1. I wanted to test the appsession feature with a small PHP code,
using PHPSESSID. The problem is that when PHP gets an unknown session
id, it creates a new one with this ID. So, when sending an unknown
session to PHP, persistance is broken : haproxy won't see any new
cookie in the response and will never attach this session to a
specific server.
This also happens when you restart haproxy : the internal hash becomes
empty and all sessions loose their persistance (load balancing the
requests on all backend servers, creating a new session on each one).
For a user, it's like the service is unusable.
The patch modifies the code to make haproxy also learn the persistance
from the client : if no session is sent from the server, then the
session id found in the client part (using the URI or the client cookie)
is used to associated the server that gave the response.
As it's probably not a feature usable in all cases, I added an option
to enable it (by default it's disabled). The syntax of appsession becomes :
appsession <cookie> len <length> timeout <holdtime> [request-learn]
This helps haproxy repair the persistance (with the risk of losing its
session at the next request, as the user will probably not be load
balanced to the same server the first time).
2. This patch also tries to reduce the memory usage.
Here is a little example to explain the current behaviour :
- Take a Tomcat server where /session.jsp is valid.
- Send a request using a cookie with an unknown value AND a path
parameter with another unknown value :
curl -b "JSESSIONID=12345678901234567890123456789012" http://<haproxy>/session.jsp;jsessionid=00000000000000000000000000000001
(I know, it's unexpected to have a request like that on a live service)
Here, haproxy finds the URI session ID and stores it in its internal
hash (with no server associated). But it also finds the cookie session
ID and stores it again.
- As a result, session.jsp sends a new session ID also stored in the
internal hash, with a server associated.
=> For 1 request, haproxy has stored 3 entries, with only 1 which will be usable
The patch modifies the behaviour to store only 1 entry (maximum).
I'm working on helping Arnaud update haproxy in Debian, and one of the
package build warnings I received was about "hyphen where a minus sign
was intended" in the man page - details:
http://lintian.debian.org/tags/hyphen-used-as-minus-sign.html
Patch included in my 1.3.20 Debian package is attached.
This can ensure that data is readily available on a socket when
we accept it, but a bug in the kernel ignores the timeout so the
socket can remain pending as long as the client does not talk.
Use with care.
Released version 1.4-dev4 with the following main changes :
- [DOC] add missing rate_lim and rate_max
- [MAJOR] struct chunk rework
- [MEDIUM] Health check reporting code rework + health logging, v3
- [BUG] check if rise/fall has an argument and it is > 0
- [MINOR] health checks logging unification
- [MINOR] add "description", "node" and show-node"/"show-desc", remove "node-name", v2
- [MINOR] Allow dots in show-node & add "white-space: nowrap" in th.pxname.
- [DOC] Add information about http://haproxy.1wt.eu/contrib.html
- [MINOR] Introduce include/types/counters.h
- [CLEANUP] Move counters to dedicated structures
- [MINOR] Add "clear counters" to clear statistics counters
- [MEDIUM] Collect & provide separate statistics for sockets, v2
- [BUG] Fix NULL pointer dereference in stats_check_uri_auth(), v2
- [MINOR] acl: don't report valid acls as potential mistakes
- [MINOR] Add cut_crlf(), ltrim(), rtrim() and alltrim()
- [MINOR] Add chunk_htmlencode and chunk_asciiencode
- [MINOR] Capture & display more data from health checks, v2
- [BUG] task.c: don't assing last_timer to node-less entries
- [BUG] http stats: large outputs sometimes got some parts chopped off
- [MINOR] backend: export some functions to recount servers
- [MINOR] backend: uninline some LB functions
- [MINOR] include time.h from freq_ctr.h as is uses "now".
- [CLEANUP] backend: move LB algos to individual files
- [MINOR] lb_map: reorder code in order to ease integration of new hash functions
- [CLEANUP] proxy: move last lb-specific bits to their respective files
- [MINOR] backend: separate declarations of LB algos from their lookup method
- [MINOR] backend: reorganize the LB algorithm selection
- [MEDIUM] backend: introduce the "static-rr" LB algorithm
- [MINOR] report list of supported pollers with -vv
- [DOC] log-health-checks is an option, not a directive
- [MEDIUM] new option "independant-streams" to stop updating read timeout on writes
- [BUG] stats: don't call buffer_shutw(), but ->shutw() instead
- [MINOR] stats: strip CR and LF from the input command line
- [BUG] don't refresh timeouts late after detected activity
- [MINOR] stats_dump_errors_to_buffer: use buffer_feed_chunk()
- [MINOR] stats_dump_sess_to_buffer: use buffer_feed_chunk()
- [MINOR] stats: make stats_dump_raw_to_buffer() use buffer_feed_chunk
- [MEDIUM] stats: don't use s->ana_state anymore
- [MINOR] remove now obsolete ana_state from the session struct
- [MEDIUM] stats: make HTTP stats use an I/O handler
- [MEDIUM] stream_int: adjust WAIT_ROOM handling
- [BUG] config: look for ID conflicts in all sockets, not only last ones.
- [MINOR] config: reference file and line with any listener/proxy/server declaration
- [MINOR] config: report places of duplicate names or IDs
- [MINOR] config: add pointer to file name in block/redirect/use_backend/monitor rules
- [MINOR] tools: add a new get_next_id() function
- [MEDIUM] config: automatically find unused IDs for proxies, servers and listeners
- [OPTIM] counters: move some max numbers to the counters struct
- [BUG] counters: fix segfault on missing counters for a listener
- [MEDIUM] backend: implement consistent hashing variation
- [MINOR] acl: add fe_conn, be_conn, queue, avg_queue
- [MINOR] stats: use 'clear counters all' to clear all values
- [MEDIUM] add access restrictions to the stats socket
- [MINOR] buffers: add buffer_feed2() and make buffer_feed() measure string length
- [MINOR] proxy: provide function to retrieve backend/server pointers
- [MINOR] add the "initial weight" to the server struct.
- [MEDIUM] stats: add the "get weight" command to report a server's weight
- [MEDIUM] stats: add the "set weight" command
- [BUILD] add a 'make tags' target
- [MINOR] stats: add support for numeric IDs in set weight/get weight
- [MINOR] stats: use a dedicated state to output static data
- [OPTIM] stats: check free space before trying to print
Krzysztof reported that using names only for get weight/set weight
was not enough because it's still possible to have multiple servers
with the same name (and my test config is one of those). He suggested
to be able to designate them by their unique numeric IDs by prefixing
the ID with a dash.
That way we can have :
set weight #120/#2
as well as
get weight static/srv1 10
The stats socket can now run at 3 different levels :
- user
- operator (default one)
- admin
These levels are used to restrict access to some information
and commands. Only the admin can clear all stats. A user cannot
clear anything nor access sensible data such as sessions or
errors.
The most common use of "clear counters" should be to only clear
max values without affecting cumulated values, for instance,
after an incident. So we change "clear counters" to only clear
max values, and add "clear counters all" to clear all counters.
These ACLs are used to check the number of active connections on the
frontend, backend or in a backend's queue. The avg_queue returns the
average number of queued connections per server, and for this, divides
the total number of queued connections by the number of alive servers.
The dst_conn ACL has been slightly changed to more reflect its name and
original usage, which is to return the number of connections on the
destination address/port (the socket) and not the whole frontend.
Consistent hashing provides some interesting advantages over common
hashing. It avoids full redistribution in case of a server failure,
or when expanding the farm. This has a cost however, the hashing is
far from being perfect, as we associate a server to a request by
searching the server with the closest key in a tree. Since servers
appear multiple times based on their weights, it is recommended to
use weights larger than approximately 10-20 in order to smoothen
the distribution a bit.
In some cases, playing with weights will be the only solution to
make a server appear more often and increase chances of being picked,
so stats are very important with consistent hashing.
In order to indicate the type of hashing, use :
hash-type map-based (default, old one)
hash-type consistent (new one)
Consistent hashing can make sense in a cache farm, in order not
to redistribute everyone when a cache changes state. It could also
probably be used for long sessions such as terminal sessions, though
that has not be attempted yet.
More details on this method of hashing here :
http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/
Until now it was required that every custom ID was above 1000 in order to
avoid conflicts. Now we have the list of all assigned IDs and can automatically
pick the first unused one. This means that it is perfectly possible to interleave
automatic IDs with persistent IDs and the parser will automatically allocate
unused values starting with 1.
This patch allows to collect & provide separate statistics for each socket.
It can be very useful if you would like to distinguish between traffic
generate by local and remote users or between different types of remote
clients (peerings, domestic, foreign).
Currently no "Session rate" is supported, but adding it should be possible
if we found it useful.
By default, when data is sent over a socket, both the write timeout and the
read timeout for that socket are refreshed, because we consider that there is
activity on that socket, and we have no other means of guessing if we should
receive data or not.
While this default behaviour is desirable for almost all applications, there
exists a situation where it is desirable to disable it, and only refresh the
read timeout if there are incoming data. This happens on sessions with large
timeouts and low amounts of exchanged data such as telnet session. If the
server suddenly disappears, the output data accumulates in the system's
socket buffers, both timeouts are correctly refreshed, and there is no way
to know the server does not receive them, so we don't timeout. However, when
the underlying protocol always echoes sent data, it would be enough by itself
to detect the issue using the read timeout. Note that this problem does not
happen with more verbose protocols because data won't accumulate long in the
socket buffers.
When this option is set on the frontend, it will disable read timeout updates
on data sent to the client. There probably is little use of this case. When
the option is set on the backend, it will disable read timeout updates on
data sent to the server. Doing so will typically break large HTTP posts from
slow lines, so use it with caution.
The "static-rr" is just the old round-robin algorithm. It is still
in use when a hash algorithm is used and the data to hash is not
present, but it was impossible to configure it explicitly. This one
is cheaper in terms of CPU and supports unlimited numbers of servers,
so it makes sense to be able to use it.
This patch implements "description" (proxy and global) and "node" (global)
options, removes "node-name" and adds "show-node" & "show-desc" options
for "stats". It also changes the way the header lines (with proxy name) and
the statistics are displayed, so stats no longer look so clumsy with very
long names.
Instead of "node-name" it is possible to use show-node/show-desc with
an optional parameter that overrides a default node/description.
backend cust-0045
# report specific values for this customer
stats show-node Europe
stats show-desc Master node for Europe, Asia, Africa
This patch adds health logging so it possible to check what
was happening before a crash. Failed healt checks are logged if
server is UP and succeeded healt checks if server is DOWN,
so the amount of additional information is limited.
I also reworked the code a little:
- check_status_description[] and check_status_info[] is now
joined into check_statuses[]
- set_server_check_status updates not only s->check_status and
s->check_duration but also s->result making the code simpler
Changes in v3:
- for now calculate and use local versions of health/rise/fall/state,
it is a slow path, no harm should be done. One day we may centralize
processing of the checks and remove the duplicated code.
- also log checks that are restoring current state
- use "conditionally succeeded" for 404 with disable-on-404
We can get rid of the stats analyser by moving all the stats code
to a stream interface applet. Above being cleaner, it provides new
advantages such as the ability to process requests and responses
from the same function and work only with simple state machines.
There's no need for any hijack hack anymore.
The direct advantage for the user are the interactive mode and the
ability to chain several commands delimited by a semi-colon. Now if
the user types "prompt", he gets a prompt from which he can send
as many requests as he wants. All outputs are terminated by a
blank line followed by a new prompt, so this can be used from
external tools too.
The code is not very clean, it needs some rework, but some part
of the dirty parts are due to the remnants of the hijack mode used
in the old functions we call.
The old AN_REQ_STATS_SOCK analyser flag is now unused and has been
removed.
Collect information about last health check result,
including L7 code if possible (for example http or smtp
return code) and time took to finish last check.
Health check info is provided on both stats pages (html & csv)
and logged when a server is marked UP or DOWN. Currently active
check are marked with an asterisk, but only in html mode.
Currently there are 14 status codes:
UNK -> unknown
INI -> initializing
SOCKERR -> socket error
L4OK -> check passed on layer 4, no upper layers testing enabled
L4TOUT -> layer 1-4 timeout
L4CON -> layer 1-4 connection problem, for example "Connection refused"
(tcp rst) or "No route to host" (icmp)
L6OK -> check passed on layer 6
L6TOUT -> layer 6 (SSL) timeout
L6RSP -> layer 6 invalid response - protocol error
L7OK -> check passed on layer 7
L7OKC -> check conditionally passed on layer 7, for example
404 with disable-on-404
L7TOUT -> layer 7 (HTTP/SMTP) timeout
L7RSP -> layer 7 invalid response - protocol error
L7STS -> layer 7 response error, for example HTTP 5xx
An abort during a connect would go to the SI_ST_CLO state without
the buffers shut. This was causing some sessions to never end if
they would abort before the connect request was initiated. This
bug has been introduced after 1.4-dev2.
The doc has been extended to reflect that too.
HTTP supports status codes 100 and 101 to report protocol indications,
which are followed by the requests's response. Till now, haproxy would
only see those responses without parsing subsequent ones. That means
that cookie additions were only performed on 1xx messages for instance,
which does not work since headers must be ignored with 1xx messages.
Also, logs were not terribly useful with the common 100 status code
in response to "Expect: 100-continue" during POST some requests.
This change adds support for such messages. Now haproxy sees them,
forwards them and skips them until it finds a correct response, which
it logs and processes. As an exception, header removal/rewriting still
work on 1xx responses in order to be able to strip out sensible
information that may have accidentely been left by another equipment
(possibly an older haproxy itself). But headers addition are disabled
however.
This change brings the ability to loop on response without data, which
is a starting point to support keepalive. The change is marked as major
as a few fixes had to be performed in the HTTP message parser.
The new tune.bufsize and tune.maxrewrite global directives allow one to
change the buffer size and the maxrewrite size. Right now, setting bufsize
too low will block stats sockets which will not be able to write at all.
An error checking must be added to buffer_write_chunk() so that if it
cannot write its message to an empty buffer, it causes the caller to abort.
This Linux-specific option was never really used in production and
has since been superseded by new splicing options brought by recent
Linux kernels.
It caused several particular cases in the code because the kernel
would take care of the session without haproxy being able to do
anything on it, which became hard to handle in the new architecture.
Let's simply get rid of it now that there is a replacement available.
The new "node-name" stats setting enables reporting of a node ID on
the stats page. It is possible to return the system's host name as
well as a specific name.
Released version 1.4-dev1 with the following main changes :
- [MINOR] acl: add support for matching of RDP cookies
- [MEDIUM] add support for RDP cookie load-balancing
- [MEDIUM] add support for RDP cookie persistence
- [MINOR] add a new CLF log format
- [MINOR] startup: don't imply -q with -D
- [BUG] ensure that we correctly re-start old process in case of error
- [MEDIUM] add support for binding to source port ranges during connect
- [MINOR] config: track "no option"/"option" changes
- [MINOR] config: support resetting options do default values
- [MEDIUM] implement option tcp-smart-accept at the frontend
- [MEDIUM] stream_sock: implement tcp-cork for use during shutdowns on Linux
- [MEDIUM] implement tcp-smart-connect option at the backend
- [MEDIUM] add support for TCP MSS adjustment for listeners
- [MEDIUM] support setting a server weight to zero
- [MINOR] make DEFAULT_MAXCONN user-configurable at build time
- [MAJOR] session: don't clear buffer status flags anymore
- [MAJOR] session: only check for timeouts when they have just occurred.
- [MAJOR] session: simplify buffer error handling
- [MEDIUM] config: split parser and checker in two functions
- [MEDIUM] config: support loading multiple configuration files
- [MEDIUM] stream_sock: don't close prematurely when nolinger is set
- [MEDIUM] session: rework buffer analysis to permit permanent analysers
- [MEDIUM] splice: set the capability on each stream_interface
- [BUG] http: redirect rules were processed too early
- [CLEANUP] remove unused DEBUG_PARSE_NO_SPEEDUP define
- [MEDIUM] http: split request waiter from request processor
- [MEDIUM] session: tell analysers what bit they were called for
- [MAJOR] http: complete splitting of the remaining stages
- [MINOR] report in the proxies the requirements for ACLs
- [MINOR] http: rely on proxy->acl_requires to allocate hdr_idx
- [MINOR] acl: add HTTP protocol detection (req_proto_http)
- [MINOR] prepare callers of session_set_backend to handle errors
- [BUG] default ACLs did not properly set the ->requires flag
- [MEDIUM] allow a TCP frontend to switch to an HTTP backend
- [MINOR] ensure we can jump from swiching rules to http without data
- [MINOR] http: take http request timeout from the backend
- [MINOR] allow TCP inspection rules to make use of HTTP ACLs
- [BUILD] report commit date and not author's date as build date
- [MINOR] acl: don't complain anymore when using L7 acls in TCP
- [BUG] stream_sock: always shutdown(SHUT_WR) before closing
- [BUG] stream_sock: don't stop reading when the poller reports an error
- [BUG] config: tcp-request content only accepts "if" or "unless"
- [BUG] task: fix possible timer drift after update
- [MINOR] apply tcp-smart-connect option for the checks too
- [MINOR] stats: better displaying in MSIE
- [MINOR] config: improve error reporting in global section
- [MINOR] config: improve error reporting in listen sections
- [MINOR] config: the "capture" keyword is not allowed in backends
- [MINOR] config: improve error reporting when checking configuration
- [BUILD] fix a minor build warning on AIX
- [BUILD] use "git cmd" instead of "git-cmd"
- [CLEANUP] report 2009 not 2008 in the copyright banner.
- [MINOR] print usage on the stats sockets upon invalid commands
- [MINOR] acl: detect and report potential mistakes in ACLs
- [BUILD] fix incorrect printf arg count with tcp_splice
- [BUG] fix random pauses on last segment of a series
- [BUILD] add support for build under Cygwin
The new statement "persist rdp-cookie" enables RDP cookie
persistence. The RDP cookie is then extracted from the RDP
protocol, and compared against available servers. If a server
matches the RDP cookie, then it gets the connection.
This patch adds support for hashing RDP cookies in order to
use them as a load-balancing key. The new "rdp-cookie(name)"
load-balancing metric has to be used for this. It is still
mandatory to wait for an RDP cookie in the frontend, otherwise
it will randomly work.
The RDP protocol is quite simple and documented, which permits
an easy detection and extraction of cookies. It can be useful
to match the MSTS cookie which can contain the username specified
by the client.
Since we can now switch from TCP to HTTP, we need to be able to apply
the HTTP request timeout after switching. That means we need to take
it from the backend and not from the frontend. Since the backend points
to the frontend before switching, that changes nothing for the normal
case.
This patch allows a TCP frontend to switch to an HTTP backend.
During the switch, missing structures are automatically allocated.
The HTTP parser is enabled so that the backend first waits for a
full HTTP request.
Now that we can perform TCP-based content switching, it makes sense
to be able to detect HTTP traffic and act accordingly. We already
have an HTTP decoder, we just have to call it in order to detect HTTP
protocol. Note that since the decoder will automatically fill in the
interesting fields of the HTTP transaction, it would make sense to
use this parsing to extend HTTP matching to TCP.
The HTTP processing has been splitted into 7 steps, one of which
is not anymore HTTP-specific (content-switching). That way, it
becomes possible to use "use_backend" rules in TCP mode. A new
"use_server" directive should follow soon.
Sometimes it can be useful to limit the advertised TCP MSS on
incoming connections, for instance when requests come through
a VPN or when the system is running with jumbo frames enabled.
Passing the "mss <value>" arguments to a "bind" line will set
the value. This works under Linux >= 2.6.28, and maybe a few
earlier ones, though due to an old kernel bug most of earlier
versions will probably ignore it. It is also possible that some
other OSes will support this.
This new option enables combining of request buffer data with
the initial ACK of an outgoing TCP connection. Doing so saves
one packet per connection which is quite noticeable on workloads
mostly consisting in small objects. The option is not enabled by
default.
This option disables TCP quick ack upon accept. It is also
automatically enabled in HTTP mode, unless the option is
explicitly disabled with "no option tcp-smart-accept".
This saves one packet per connection which can bring reasonable
amounts of bandwidth for servers processing small requests.
A new keyword prefix "default" has been introduced in order to
reset some options to their default values. This can be needed
for instance when an option is forced disabled or enabled in a
defaults section and when later sections want to use automatic
settings regardless of what was specified there. Right now it
is only supported by options, just like the "no" prefix.
Some users are already hitting the 64k source port limit when
connecting to servers. The system usually maintains a list of
unused source ports, regardless of the source IP they're bound
to. So in order to go beyond the 64k concurrent connections, we
have to manage the source ip:port lists ourselves.
The solution consists in assigning a source port range to each
server and use a free port in that range when connecting to that
server, either for a proxied connection or for a health check.
The port must then be put back into the server's range when the
connection is closed.
This mechanism is used only when a port range is specified on
a server. It makes it possible to reach 64k connections per
server, possibly all from the same IP address. Right now it
should be more than enough even for huge deployments.
Released version 1.3.18 with the following main changes :
- [MEDIUM] add support for "balance hdr(name)"
- [CLEANUP] give a little bit more information in error message
- [MINOR] add X-Original-To: header
- [BUG] x-original-to: fix missing initialization to default value
- [BUILD] spec file: fix broken pipe during rpmbuild and add man file
- [MINOR] improve reporting of misplaced acl/reqxxx rules
- [MEDIUM] http: add options to ignore invalid header names
- [MEDIUM] http: capture invalid requests/responses even if accepted
- [BUILD] add format(printf) to printf-like functions
- [MINOR] fix several printf formats and missing arguments
- [BUG] stats: total and lbtot are unsigned
- [MINOR] fix a few remaining printf-like formats on 64-bit platforms
- [CLEANUP] remove unused make option from haproxy.spec
- [BUILD] make it possible to pass alternative arch at build time
- [MINOR] switch all stat counters to 64-bit
- [MEDIUM] ensure we don't recursively call pool_gc2()
- [CRITICAL] uninitialized response field can sometimes cause crashes
- [BUG] fix wrong pointer arithmetics in HTTP message captures
- [MINOR] rhel init script : support the reload operation
- [MINOR] add basic signal handling functions
- [BUILD] add signal.o to all makefiles
- [MEDIUM] call signal_process_queue from run_poll_loop
- [MEDIUM] pollers: don't wait if a signal is pending
- [MEDIUM] convert all signals to asynchronous signals
- [BUG] O(1) pollers should check their FD before closing it
- [MINOR] don't close stdio fds twice
- [MINOR] add options dontlog-normal and log-separate-errors
- [DOC] minor fixes and rearrangements
- [BUG] fix parser crash on unconditional tcp content rules
- [DOC] rearrange the configuration manual and add a summary
- [MINOR] standard: provide a new 'my_strndup' function
- [MINOR] implement per-logger log level limitation
- [MINOR] compute the max of sessions/s on fe/be/srv
- [MINOR] stats: report max sessions/s and limit in CSV export
- [MINOR] stats: report max sessions/s and limit in HTML stats
- [MINOR] stats/html: use the arial font before helvetica
Some people are using haproxy in a shared environment where the
system logger by default sends alert and emerg messages to all
consoles, which happens when all servers go down on a backend for
instance. These people can not always change the system configuration
and would like to limit the outgoing messages level in order not to
disturb the local users.
The addition of an optional 4th field on the "log" line permits
exactly this. The minimal log level ensures that all outgoing logs
will have at least this level. So the logs are not filtered out,
just set to this level.
There is a patch made by me that allow for balancing on any http header
field.
[WT:
made minor changes:
- turned 'balance header name' into 'balance hdr(name)' to match more
closely the ACL syntax for easier future convergence
- renamed the proxy structure fields header_* => hh_*
- made it possible to use the domain name reduction to any header, not
only "host" since it makes sense to do it with other ones.
Otherwise patch looks good.
/WT]
Several people have asked for a summary in order to ease finding
of sections in the configuration manual. It was the opportunity to
tidy it up a bit and rearrange some sections.
Some big traffic sites have trouble dealing with logs and tend to
disable them. Here are two new options to help cope with massive
logs.
- dontlog-normal only disables logging for 100% successful
connections, other ones will still be logged
- log-separate-errors will cause non-100% successful connections
to be logged at level "err" instead of level "info" so that a
properly configured syslog daemon can send them to a different
file for longer conservation.
I have attached a patch which will add on every http request a new
header 'X-Original-To'. If you have HAProxy running in transparent mode
with a big number of SQUID servers behind it, it is very nice to have
the original destination ip as a common header to make decisions based
on it.
The whole thing is configurable with a new option 'originalto'. I have
updated the sourcecode as well as the documentation. The 'haproxy-en.txt'
and 'haproxy-fr.txt' files are untouched, due to lack of my french
language knowledge. ;)
Also the patch adds this header for IPv4 only. I haven't any IPv6 test
environment running here and don't know if getsockopt() with SO_ORIGINAL_DST
will work on IPv6. If someone knows it and wants to test it I can modify
the diff. Feel free to ask me questions or things which should be changed. :)
--Maik
It's useful to be able to accept an invalid header name in a request
or response but still be able to monitor further such errors. Now,
when an invalid request/response is received and accepted due to
an "accept-invalid-http-{request|response}" option, the invalid
request will be captured for later analysis with "show errors" on
the stats socket.
Released version 1.3.17 with the following main changes :
- Update specfile to build for v2.6 kernel.
- [BUG] reset the stream_interface connect timeout upon connect or error
- [BUG] reject unix accepts when connection limit is reached
- [MINOR] show sess: report number of calls to each task
- [BUG] don't call epoll_ctl() on closed sockets
- [BUG] stream_sock: disable I/O on fds reporting an error
- [MINOR] sepoll: don't count two events on the same FD.
- [MINOR] show sess: report a lot more information about sessions
- [BUG] stream_sock: check for shut{r,w} before refreshing some timeouts
- [BUG] don't set an expiration date directly from now_ms
- [MINOR] implement ulltoh() to write HTML-formatted numbers
- [MINOR] stats/html: group digits by 3 to clarify numbers
- [BUILD] remove haproxy-small.spec
- [BUILD] makefile: remove unused references to linux24eold and EPOLL_CTL_WORKAROUND
Released version 1.3.16 with the following main changes :
- [BUILD] Fixed Makefile for linking pcre
- [CONTRIB] selinux policy for haproxy
- [MINOR] show errors: encode backslash as well as non-ascii characters
- [MINOR] cfgparse: some cleanups in the consistency checks
- [MINOR] cfgparse: set backends to "balance roundrobin" by default
- [MINOR] tcp-inspect: permit the use of no-delay inspection
- [MEDIUM] reverse internal proxy declaration order to match configuration
- [CLEANUP] config: catch and report some possibly wrong rule ordering
- [BUG] connect timeout is in the stream interface, not the buffer
- [BUG] session: errors were not reported in termination flags in TCP mode
- [MINOR] tcp_request: let the caller take care of errors and timeouts
- [CLEANUP] http: remove some commented out obsolete code in process_response
- [MINOR] update ebtree to version 4.1
- [MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1
- [BUG] sched: don't leave 3 lasts tasks unprocessed when niced tasks are present
- [BUG] scheduler: fix improper handling of duplicates __task_queue()
- [MINOR] sched: permit a task to stay up between calls
- [MINOR] task: keep a task count and clean up task creators
- [MINOR] stats: report number of tasks (active and running)
- [BUG] server check intervals must not be null
- [OPTIM] stream_sock: don't retry to read after a large read
- [OPTIM] buffer: new BF_READ_DONTWAIT flag reduces EAGAIN rates
- [MEDIUM] session: don't resync FSMs on non-interesting changes
- [BUG] check for global.maxconn before doing accept()
- [OPTIM] sepoll: do not re-check whole list upon accepts
Sometimes it may make sense to be able to immediately apply a verdict
without waiting at all. It was not possible because no inspect-delay
meant no inspection at all. This is now fixed.
When a backend has no LB algo specified and is not in dispatch, proxy
nor transparent mode, use "balance roundrobin" by default instead of
complaining. This will be particularly useful with stats and redirects.
Released version 1.3.16-rc1 with the following main changes :
- appsessions: cleanup DEBUG_HASH and initialize request_counter
- [MINOR] acl: add new keyword "connslots"
- [MINOR] cfgparse: fix off-by 2 in error message size
- [BUILD] fix build with gcc 4.3
- [BUILD] fix MANDIR default location to match documentation
- [TESTS] add a debug patch to help trigger the stats bug
- [BUG] Flush buffers also where there are exactly 0 bytes left
- [MINOR] Allow to specify a domain for a cookie
- [BUG/CLEANUP] cookiedomain -> cookie_domain rename + free(p->cookie_domain)
- [MEDIUM] Fix memory freeing at exit
- [MEDIUM] Fix memory freeing at exit, part 2
- [BUG] Fix listen & more of 2 couples <ip>:<port>
- [DOC] remove buggy comment for use_backend
- [CRITICAL] fix server state tracking: it was O(n!) instead of O(n)
- [MEDIUM] add support for URI hash depth and length limits
- [MINOR] permit renaming of x-forwarded-for header
- [BUILD] fix Makefile.bsd and Makefile.osx for stream_interface
- [BUILD] Haproxy won't compile if DEBUG_FULL is defined
- [MEDIUM] upgrade to ebtree v4.0
- [DOC] update the README file with new build options
- [MEDIUM] reduce risk of event starvation in ev_sepoll
- [MEDIUM] detect streaming buffers and tag them as such
- [MEDIUM] add support for conditional HTTP redirection
- [BUILD] make install should depend on haproxy not "all"
- [DEBUG] add a TRACE macro to facilitate runtime data extraction
- [BUG] event pollers must not wait if a task exists in the run queue
- [BUG] queue management: wake oldest request in queues
- [BUG] log: reported queue position was offed-by-one
- [BUG] fix the dequeuing logic to ensure that all requests get served
- [DOC] documentation for the "retries" parameter was missing.
- [MEDIUM] implement a monotonic internal clock
- [MEDIUM] further improve monotonic clock by check forward jumps
- [OPTIM] add branch prediction hints in list manipulations
- [MAJOR] replace ultree with ebtree in wait-queues
- [BUG] we could segfault during exit while freeing uri_auths
- [BUG] wqueue: perform proper timeout comparisons with wrapping values
- [MINOR] introduce now_ms, the current date in milliseconds
- [BUG] disable buffer read timeout when reading stats
- [MEDIUM] rework the wait queue mechanism
- [BUILD] change declaration of base64tab to fix build with Intel C++
- [OPTIM] shrink wake_expired_tasks() by using task_wakeup()
- [MAJOR] use an ebtree instead of a list for the run queue
- [MEDIUM] introduce task->nice and boot access to statistics
- [OPTIM] task_queue: assume most consecutive timers are equal
- [BUILD] silent a warning in unlikely() with gcc 4.x
- [MAJOR] convert all expiration timers from timeval to ticks
- [BUG] use_backend would not correctly consider "unless"
- [TESTS] added test-acl.cfg to test some ACL combinations
- [MEDIUM] add support for configuration keyword registration
- [MEDIUM] modularize the global "stats" keyword configuration parser
- [MINOR] cfgparse: add support for warnings in external functions
- [MEDIUM] modularize the "timeout" keyword configuration parser
- [MAJOR] implement tcp request content inspection
- [MINOR] acl: add a new parsing function: parse_dotted_ver
- [MINOR] acl: add req_ssl_ver in TCP, to match an SSL version
- [CLEANUP] remove unused include/types/client.h
- [CLEANUP] remove many #include <types/xxx> from C files
- [CLEANUP] remove dependency on obsolete INTBITS macro
- [DOC] document the new "tcp-request" keyword and associated ACLs
- [MINOR] acl: add REQ_CONTENT to the list of default acls
- [MEDIUM] acl: permit fetch() functions to set the result themselves
- [MEDIUM] acl: get rid of dummy values in always_true/always_false
- [MINOR] acl: add the "wait_end" acl verb
- [MEDIUM] acl: enforce ACL type checking
- [MEDIUM] acl: set types on all currently known ACL verbs
- [MEDIUM] acl: when possible, report the name and requirements of ACLs in warnings
- [CLEANUP] remove 65 useless NULL checks before free
- [MEDIUM] memory: update pool_free2() to support NULL pointers
- [MEDIUM] buffers: ensure buffer_shut* are properly called upon shutdowns
- [MEDIUM] process_srv: rely on buffer flags for client shutdown
- [MEDIUM] process_srv: don't rely at all on client state
- [MEDIUM] process_cli: don't rely at all on server state
- [BUG] fix segfault with url_param + check_post
- [BUG] server timeout was not considered in some circumstances
- [BUG] client timeout incorrectly rearmed while waiting for server
- [MAJOR] kill CL_STINSPECT and CL_STHEADERS (step 1)
- [MAJOR] get rid of SV_STANALYZE (step 2)
- [MEDIUM] simplify and centralize request timeout cancellation and request forwarding
- [MAJOR] completely separate HTTP and TCP states on the request path
- [BUG] fix recently introduced loop when client closes early
- [MAJOR] get rid of the SV_STHEADERS state
- [MAJOR] better separation of response processing and server state
- [MAJOR] clearly separate HTTP response processing from TCP server state
- [MEDIUM] remove unused references to {CL|SV}_STSHUT*
- [MINOR] term_trace: add better instrumentations to trace the code
- [BUG] ev_sepoll: closed file descriptors could persist in the spec list
- [BUG] process_response must not enable the read FD
- [BUG] buffers: remove BF_MAY_CONNECT and fix forwarding issue
- [BUG] process_response: do not touch srv_state
- [BUG] maintain_proxies must not disable backends
- [CLEANUP] get rid of BF_SHUT*_PENDING
- [MEDIUM] buffers: add BF_EMPTY and BF_FULL to remove dependency on req/rep->l
- [MAJOR] process_session: rely only on buffer flags
- [MEDIUM] use buffer->wex instead of buffer->cex for connect timeout
- [MEDIUM] centralize buffer timeout checks at the top of process_session
- [MINOR] ensure the termination flags are set by process_xxx
- [MEDIUM] session: move the analysis bit field to the buffer
- [OPTIM] process_cli/process_srv: reduce the number of tests
- [BUG] regparm is broken on gcc < 3
- [BUILD] fix warning in proto_tcp.c with gcc >= 4
- [MEDIUM] merge inspect_exp and txn->exp into request buffer
- [BUG] process_cli/process_srv: don't call shutdown when already done
- [BUG] process_request: HTTP body analysis must return zero if missing data
- [TESTS] test-fsm: 22 regression tests for state machines
- [BUG] Fix empty X-Forwarded-For header name when set in defaults section
- [BUG] fix harmless but wrong fd insertion sequence
- [MEDIUM] make it possible for analysers to follow the whole session
- [MAJOR] rework of the server FSM
- [OPTIM] remove useless fd_set(read) upon shutdown(write)
- [MEDIUM] massive cleanup of process_srv()
- [MEDIUM] second level of code cleanup for process_srv_data
- [MEDIUM] third cleanup and optimization of process_srv_data()
- [MEDIUM] process_srv_data: ensure that we always correctly re-arm timeouts
- [MEDIUM] stream_sock_process_data moved to stream_sock.c
- [MAJOR] make the client side use stream_sock_process_data()
- [MEDIUM] split stream_sock_process_data
- [OPTIM] stream_sock_read must check for null-reads more often
- [MINOR] only call flow analysers when their read side is connected.
- [MEDIUM] reintroduce BF_HIJACK with produce_content
- [MINOR] re-arrange buffer flags and rename some of them
- [MINOR] do not check for BF_SHUTR when computing write timeout
- [OPTIM] ev_sepoll: detect newly created FDs and check them once
- [OPTIM] reduce the number of calls to task_wakeup()
- [OPTIM] force inlining of large functions with gcc >= 3
- [MEDIUM] indicate a reason for a task wakeup
- [MINOR] change type of fdtab[]->owner to void*
- [MAJOR] make stream sockets aware of the stream interface
- [MEDIUM] stream interface: add the ->shutw method as well as in and out buffers
- [MEDIUM] buffers: add BF_READ_ATTACHED and BF_ANA_TIMEOUT
- [MEDIUM] process_session: make use of the new buffer flags
- [CLEANUP] process_session: move debug outputs out of the critical loop
- [MEDIUM] move QUEUE and TAR timers to stream interfaces
- [OPTIM] add compiler hints in tick_is_expired()
- [MINOR] add buffer_check_timeouts() to check what timeouts have fired.
- [MEDIUM] use buffer_check_timeouts instead of stream_sock_check_timeouts()
- [MINOR] add an expiration flag to the stream_sock_interface
- [MAJOR] migrate the connection logic to stream interface
- [MAJOR] add a connection error state to the stream_interface
- [MEDIUM] add the SN_CURR_SESS flag to the session to track open sessions
- [MEDIUM] continue layering cleanups.
- [MEDIUM] stream_interface: added a DISconnected state between CON/EST and CLO
- [MEDIUM] remove stream_sock_update_data()
- [MINOR] maintain a global session list in order to ease debugging
- [BUG] shutw must imply close during a connect
- [MEDIUM] process shutw during connection attempt
- [MEDIUM] make the stream interface control the SHUT{R,W} bits
- [MAJOR] complete layer4/7 separation
- [CLEANUP] move the session-related functions to session.c
- [MINOR] call session->do_log() for logging
- [MINOR] replace the ambiguous client_return function by stream_int_return
- [MINOR] replace client_retnclose() with stream_int_retnclose()
- [MINOR] replace srv_close_with_err() with http_server_error()
- [MEDIUM] make the http server error function a pointer in the session
- [CLEANUP] session.c: removed some migration left-overs in sess_establish()
- [MINOR] stream_sock_data_finish() should not expose fd
- [MEDIUM] extract TCP request processing from HTTP
- [MEDIUM] extract the HTTP tarpit code from process_request().
- [MEDIUM] move the HTTP request body analyser out of process_request().
- [MEDIUM] rename process_request to http_process_request
- [BUG] fix forgotten server session counter
- [MINOR] declare process_session in session.h, not proto_http.h
- [MEDIUM] first pass of lifting to proto_uxst.c:uxst_event_accept()
- [MINOR] add an analyser code for UNIX stats request
- [MINOR] pre-set analyser flags on the listener at registration time
- [BUG] do not forward close from cons to prod with analysers
- [MEDIUM] ensure that sock->shutw() also closes read for init states
- [MINOR] add an analyser state in struct session
- [MAJOR] make unix sockets work again with stats
- [MEDIUM] remove cli_fd, srv_fd, cli_state and srv_state from the session
- [MINOR] move the listener reference from fd to session
- [MEDIUM] reference the current hijack function in the buffer itself
- [MINOR] slightly rebalance stats_dump_{raw,http}
- [MINOR] add a new back-reference type : struct bref
- [MINOR] add back-references to sessions for later use by a dumper.
- [MEDIUM] add support for "show sess" in unix stats socket
- [BUG] do not release the connection slot during a retry
- [BUG] dynamic connection throttling could return a max of zero conns
- [BUG] do not try to pause backends during reload
- [BUG] ensure that listeners from disabled proxies are correctly unbound.
- [BUG] acl-related keywords are not allowed in defaults sections
- [BUG] cookie capture is declared in the frontend but checked on the backend
- [BUG] critical errors should be reported even in daemon mode
- [MINOR] redirect: add support for the "drop-query" option
- [MINOR] redirect: add support for "set-cookie" and "clear-cookie"
- [MINOR] redirect: in prefix mode a "/" means not to change the URI
- [BUG] do not dequeue requests on a dead server
- [BUG] do not dequeue the backend's pending connections on a dead server
- [MINOR] stats: indicate if a task is running in "show sess"
- [BUG] check timeout must not be changed if timeout.check is not set
- [BUG] "option transparent" is for backend, not frontend !
- [MINOR] transfer errors were not reported anymore in data phase
- [MEDIUM] add a send limit to a buffer
- [MEDIUM] don't report buffer timeout when there is I/O activity
- [MEDIUM] indicate when we don't care about read timeout
- [MINOR] add flags to indicate when a stream interface is waiting for space/data
- [MEDIUM] enable inter-stream_interface wakeup calls
- [MAJOR] implement autonomous inter-socket forwarding
- [MINOR] add the splice_len member to the buffer struct in preparation of splice support
- [MEDIUM] stream_sock: factor out the return path in case of no-writes
- [MEDIUM] i/o: rework ->to_forward and ->send_max
- [OPTIM] stream_sock: do not ask for polling on EAGAIN if we have read
- [OPTIM] buffer: replace rlim by max_len
- [OPTIM] stream_sock: factor out the buffer full handling out of the loop
- [CLEANUP] replace a few occurrences of (flags & X) && !(flags & Y)
- [CLEANUP] stream_sock: move the write-nothing condition out of the loop
- [MEDIUM] split stream_sock_write() into callback and core functions
- [MEDIUM] stream_sock_read: call ->chk_snd whenever there are data pending
- [MINOR] stream_sock: fix a few wrong empty calculations
- [MEDIUM] stream_sock: try to send pending data on chk_snd()
- [MINOR] global.maxpipes: add the ability to reserve file descriptors for pipes
- [MEDIUM] splice: add configuration options and set global.maxpipes
- [MINOR] introduce structures required to support Linux kernel splicing
- [MEDIUM] add definitions for Linux kernel splicing
- [MAJOR] complete support for linux 2.6 kernel splicing
- [BUG] reserve some pipes for backends with splice enabled
- [MEDIUM] splice: add hints to support older buggy kernels
- [MEDIUM] introduce pipe pools
- [MEDIUM] splice: make use of pipe pools
- [STATS] report pipe usage in the statistics
- [OPTIM] make global.maxpipes default to global.maxconn/4 when not specified
- [BUILD] fix snapshot date extraction with negative timezones
- [MEDIUM] move global tuning options to the global structure
- [MEDIUM] splice: add the global "nosplice" option
- [BUILD] add USE_LINUX_SPLICE to enable LINUX_SPLICE on linux 2.6
- [BUG] we must not exit if protocol binding only returns a warning
- [MINOR] add support for bind interface name
- [BUG] inform the user when root is expected but not set
- [MEDIUM] add support for source interface binding
- [MEDIUM] add support for source interface binding at the server level
- [MEDIUM] implement bind-process to limit service presence by process
- [DOC] document maxpipes, nosplice, option splice-{auto,request,response}
- [DOC] filled the logging section of the configuration manual
- [DOC] document HTTP status codes
- [DOC] document a few missing info about errorfile
- [BUG] fix random memory corruption using "show sess"
- [BUG] fix unix socket processing of interrupted output
- [DOC] add diagrams of queuing and future ACL design
- [BUILD] proto_http did not build on gcc-2.95
- [BUG] the "source" keyword must first clear optional settings
- [BUG] global.tune.maxaccept must be limited even in mono-process mode
- [MINOR] ensure that http_msg_analyzer updates pointer to invalid char
- [MEDIUM] store a complete dump of request and response errors in proxies
- [MEDIUM] implement error dump on unix socket with "show errors"
- [DOC] document "show errors"
- [MINOR] errors dump must use user-visible date, not internal date.
- [MINOR] time: add __usec_to_1024th to convert usecs to 1024th of second
- [MINOR] add curr_sec_ms and curr_sec_ms_scaled for current second.
- [MEDIUM] measure and report session rate on frontend, backends and servers
- [BUG] the "connslots" keyword was matched as "connlots"
- [MINOR] acl: add 2 new verbs: fe_sess_rate and be_sess_rate
- [MEDIUM] implement "rate-limit sessions" for the frontend
- [BUG] interface binding: length must include the trailing zero
- [BUG] typo in timeout error reporting : report *res and not *err
- [OPTIM] maintain_proxies: only wake up when the frontend will be ready
- [OPTIM] rate-limit: cleaner behaviour on low rates and reduce consumption
- [BUG] switch server-side stream interface to close in case of abort
- [CLEANUP] remove last references to term_trace
- [OPTIM] freq_ctr: do not rotate the counters when reading
- [BUG] disable any analysers for monitoring requests
- [BUG] rate-limit in defaults section was ignored
- [BUG] task: fix handling of duplicate keys
- [OPTIM] task: don't unlink a task from a wait queue when waking it up
- [OPTIM] displace tasks in the wait queue only if absolutely needed
- [MEDIUM] minor update to the task api: let the scheduler queue itself
- [BUG] event_accept() must always wake the task up, even in health mode
- [CLEANUP] task: distinguish between clock ticks and timers
- [OPTIM] task: reduce the number of calls to task_queue()
- [OPTIM] do not re-check req buffer when only response has changed
- [CLEANUP] don't enable kernel splicing when socket is closed
- [CLEANUP] buffer_flush() was misleading, rename it as buffer_erase
- [MINOR] buffers: implement buffer_flush()
- [MEDIUM] rearrange forwarding condition to enable splice during analysis
- [BUILD] build fixes for Solaris
- [BUILD] proto_http did not build on gcc-2.95 (again)
- [CONTRIB] halog: fast log parser for haproxy
- [CONTRIB] halog: faster fgets() and add support for percentile reporting
The new "rate-limit sessions" statement sets a limit on the number of
new connections per second on the frontend. As it is extremely accurate
(about 0.1%), it is efficient at limiting resource abuse or DoS.
These new ACLs match frontend session rate and backend session rate.
Examples are provided in the doc to explain how to use that in order
to limit abuse of service.
With this change, all frontends, backends, and servers maintain a session
counter and a timer to compute a session rate over the last second. This
value will be very useful because it varies instantly and can be used to
check thresholds. This value is also reported in the stats in a new "rate"
column.
On overloaded systems, it sometimes happens that hundreds or thousands
of incoming connections are queued in the system's backlog, and all get
dequeued at once. The problem is that when haproxy processes them and
does not apply any limit, this can take some time and the internal date
does not progress, resulting in wrong timer measures for all sessions.
The most common effect of this is that all of these sessions report a
large request time (around several hundreds of ms) which is in fact
caused by the time spent accepting other connections. This might happen
on shared systems when the machine swaps.
For this reason, we finally apply a reasonable limit even in mono-process
mode. Accepting 100 connections at once is fast enough for extreme cases
and will not cause that much of a trouble when the system is saturated.
Some parts from the previous doc about logging have been merged and
updated. Most of those parts have been reworked and completed. The
examples are now accurate and reflect recent versions.
The "bind-process" keyword lets the admin select which instances may
run on which process (in multi-process mode). It makes it easier to
more evenly distribute the load across multiple processes by avoiding
having too many listen to the same IP:ports.
Specifying "interface <name>" after the "source" statement allows
one to bind to a specific interface for proxy<->server traffic.
This makes it possible to use multiple links to reach multiple
servers, and to force traffic to pass via an interface different
from the one the system would have chosen based on the routing
table.
By appending "interface <name>" to a "bind" line, it is now possible
to specifically bind to a physical interface name. Note that this
currently only works on Linux and requires root privileges.
"option transparent" was set and checked on frontends only while it
is purely a backend thing as it replaces the "balance" mode. For this
reason, it did only work in "listen" sections. This change will then
not affect the rare users of this option.
If the prefix is set to "/", it means the user does not want to alter
the original URI, so we don't want to insert a new slash before the
original URI.
(cherry-picked from commit 02a35c74942c1bce762e996698add1270e6a5030)
It is now possible to set or clear a cookie during a redirection. This
is useful for logout pages, or for protecting against some DoSes. Check
the documentation for the options supported by the "redirect" keyword.
(cherry-picked from commit 4af993822e880d8c932f4ad6920db4c9242b0981)
If "drop-query" is present on a "redirect" line using the "prefix" mode,
then the returned Location header will be the request URI without the
query-string. This may be used on some login/logout pages, or when it
must be decided to redirect the user to a non-secure server.
(cherry-picked from commit f2d361ccd73aa16538ce767c766362dd8f0a88fd)
I'm in the process of setting up one haproxy instance now, and I find
the following acl option useful. I'm not too sure why this option has
not been available before, but I find this useful for my own usage, so
I'm submitting this patch in the hope that it will be useful as well.
The basic idea is to be able to measure the available connection slots
still available (connection, + queue) - anything beyond that can be
redirected to a different backend. 'connslots' = number of available
server connection slots, + number of available server queue slots. In
the case where we encounter srv maxconn = 0, or srv maxqueue = 0 (in
which case we dont need to care about connslots) the value you get is
-1. Note also that this code does not take care of dynamic connections
at this point in time.
The reason why I'm using this new acl (as opposed to 'nbsrv') is that
'nbsrv' only measures servers that are actually *down*. Whereas this
other acl is more fine-grained, and looks into the number of conn
slots available as well.
It is now possible to list all known sessions by issuing "show sess"
on the unix stats socket. The format is not much evolved but it is
very useful for debugging.
The doc has been updated to reflect the new keyword.
There were rare situations where it was not easy to detect that a failed
session attempt had occurred and needed some server cleanup. In particular,
client aborts sometimes lead to session leaks on the server side.
A new state "SI_ST_DIS" (disconnected) has been introduced for this. When
a session has been closed at a stream interface but the server cleanup has
not occurred, this state is entered instead of CLO. The cleanup is then
performed there and the state goes to CLO.
A new diagram has been added to show possible stream_interface state
transitions that can occur in a stream-sock. It makes debugging easier.
Because I needed it in my situation - here's a quick patch to
allow changing of the "x-forwarded-for" header by using a suboption to
"option forwardfor".
Suboption "header XYZ" will set the header from "x-forwarded-for" to "XYZ".
Default is still "x-forwarded-for" if the header value isn't defined.
Also the suboption 'except a.b.c.d/z' still works on the same line.
So it's now: option forwardfor [except a.b.c.d[/z]] [header XYZ]
The new "wait_end" acl delays evaluation of the rule (and the next ones)
to the end of the analysis period. This is intented to be used with TCP
content analysis. A rule referencing such an ACL will not match until
the delay is over. An equivalent default ACL "WAIT_END" has been created.
With content inspection, checking the presence of data in the
request buffer is very important. It's getting boring to always
add such an ACL, so let's add it by default.
A new "redirect" keyword adds the ability to send an HTTP 301/302/303
redirection to either an absolute location or to a prefix followed by
the original URI. The redirection is conditionned by ACL rules, so it
becomes very easy to move parts of a site to another site using this.
This work was almost entirely done at Exceliance by Emeric Brun.
A test-case has been added in the tests/ directory.
This patch allows to specify a domain used when inserting a cookie
providing a session stickiness. Usefull for example with wildcard domains.
The patch adds one new variable to the struct proxy: cookiedomain.
When set the domain is appended to a Set-Cookie header.
Domain name is validated using the new invalid_domainchar() function.
It is basically invalid_char() limited to [A-Za-z0-9_.-]. Yes, the test
is too trivial and does not cover all wrong situations, but the main
purpose is to detect most common mistakes, not intentional abuses.
The underscore ("_") character is not RFC-valid but as it is
often (mis)used so I decided to allow it.
This patch adds two optional arguments "len" and "depth" to
"balance uri". They are used to limit the length in characters
of the analysis, as well as the number of directory components
it applies to.
Released version 1.3.15 with the following main changes :
- [BUILD] Added support for 'make install'
- [BUILD] Added 'install-man' make target for installing the man page
- [BUILD] Added 'install-bin' make target
- [BUILD] Added 'install-doc' make target
- [BUILD] Removed "/" after '$(DESTDIR)' in install targets
- [BUILD] Changed 'install' target to install the binaries first
- [BUILD] Replace hardcoded 'LD = gcc' with 'LD = $(CC)'
- [MEDIUM]: Inversion for options
- [MEDIUM]: Count retries and redispatches also for servers, fix redistribute_pending, extend logs, %d->%u cleanup
- [BUG]: Restore clearing t->logs.bytes
- [MEDIUM]: rework checks handling
- [DOC] Update a "contrib" file with a hint about a scheme used for formathing subjects
- [MEDIUM] Implement "track [<backend>/]<server>"
- [MINOR] Implement persistent id for proxies and servers
- [BUG] Don't increment server connections too much + fix retries
- [MEDIUM]: Prevent redispatcher from selecting the same server, version #3
- [MAJOR] proto_uxst rework -> SNMP support
- [BUG] appsession lookup in URL does not work
- [BUG] transparent proxy address was ignored in backend
- [BUG] hot reconfiguration failed because of a wrong error check
- [DOC] big update to the configuration manual
- [DOC] large update to the configuration manual
- [DOC] document more options
- [BUILD] major rework of the GNU Makefile
- [STATS] add support for "show info" on the unix socket
- [DOC] document options forwardfor to logasap
- [MINOR] add support for the "backlog" parameter
- [OPTIM] introduce global parameter "tune.maxaccept"
- [MEDIUM] introduce "timeout http-request" in frontends
- [MINOR] tarpit timeout is also allowed in backends
- [BUG] increment server connections for each connect()
- [MEDIUM] add a turn-around state of one second after a connection failure
- [BUG] fix typo in redispatched connection
- [DOC] document options nolinger to ssl-hello-chk
- [DOC] added documentation for "option tcplog" to "use_backend"
- [BUG] connect_server: server might not exist when sending error report
- [MEDIUM] support fully transparent proxy on Linux (USE_LINUX_TPROXY)
- [MEDIUM] add non-local bind to connect() on Linux
- [MINOR] add transparent proxy support for balabit's Tproxy v4
- [BUG] use backend's source and not server's source with tproxy
- [BUG] fix overlapping server flags
- [MEDIUM] fix server health checks source address selection
- [BUG] build failed on CONFIG_HAP_LINUX_TPROXY without CONFIG_HAP_CTTPROXY
- [DOC] added "server", "source" and "stats" keywords
- [DOC] all server parameters have been documented
- [DOC] document all req* and rsp* keywords.
- [DOC] added documentation about HTTP header manipulations
- [BUG] log response byte count, not request
- [BUILD] code did not build in full debug mode
- [BUG] fix truncated responses with sepoll
- [MINOR] use s->frt_addr as the server's address in transparent proxy
- [MINOR] fix configuration hint about timeouts
- [DOC] minor cleanup of the doc and notice to contributors
- [MINOR] report correct section type for unknown keywords.
- [BUILD] update MacOS Makefile to build on newer versions
- [DOC] fix erroneous "useallbackups" option in the doc
- [DOC] applied small fixes from early readers
- [MINOR] add configuration support for "redir" server keyword
- [MEDIUM] completely implement the server redirection method
- [TESTS] add a test case for the server redirection mechanism
- [DOC] add a configuration entry for "server ... redir <prefix>"
- [BUILD] backend.c and checks.c did not build without tproxy !
- Revert "[BUILD] backend.c and checks.c did not build without tproxy !"
- [BUILD] backend.c and checks.c did not build without tproxy !
- [OPTIM] used unsigned ints for HTTP state and message offsets
- [OPTIM] GCC4's builtin_expect() is suboptimal
- [BUG] failed conns were sometimes incremented in the frontend!
- [BUG] timeout.check was not pre-set to eternity
- [TESTS] add test-pollers.cfg to easily report pollers in use
- [BUG] do not apply timeout.connect in checks if unset
- [BUILD] ensure that makefile understands USE_DLMALLOC=1
- [MINOR] silent gcc for a wrong warning
- [CLEANUP] update .gitignore to ignore more temporary files
- [CLEANUP] report dlmalloc's source path only if explictly specified
- [BUG] str2sun could leak a small buffer in case of error during parsing
- [BUG] option allbackups was not working anymore in roundrobin mode
- [MAJOR] implementation of the "leastconn" load balancing algorithm
- [BUILD] ensure that users don't build without setting the target anymore.
- [DOC] document the leastconn LB algo
- [MEDIUM] fix stats socket limitation to 16 kB
- [DOC] fix unescaped space in httpchk example.
- [BUG] fix double-decrement of server connections
- [TESTS] add a test case for port mapping
- [TESTS] add a benchmark for integer hashing
- [TESTS] add new methods in ip-hash test file
- [MAJOR] implement parameter hashing for POST requests
This patch extends the "url_param" load balancing method by introducing
the "check_post" option. Using this option enables analysis of the beginning
of POST requests to search for the specified URL parameter.
The patch also fixes a few minor typos in comments that were discovered
during code review.
Currently there is a ~16KB limit for a data size passed via unix socket.
It is caused by a trivial bug ttat is going to fixed soon, however
in most cases there is no need to dump a full stats.
This patch makes possible to select a scope of dumped data by extending
current "show stat" to "show stat [<iid> <type> <sid>]":
- iid is a proxy id, -1 to dump all proxies
- type selects type of dumpable objects: 1 for frontend, 2 for backend, 4 for
server, -1 for all types. Values can be ORed, for example:
1+2=3 -> frontend+backend.
1+2+4=7 -> frontend+backend+server.
- sid is a service id, -1 to dump everything from the selected proxy.
To do this I implemented a new session flag (SN_STAT_BOUND), added three
variables in data_ctx.stats (iid, type, sid), modified dumpstats.c and
completely revorked the process_uxst_stats: now it waits for a "\n"
terminated string, splits args and uses them. BTW: It should be quite easy
to add new commands, for example to enable/disable servers, the only problem
I can see is a not very lucky config name (*stats* socket). :|
During the work I also fixed two bug:
- s->flags were not initialized for proto_uxst
- missing comma if throttling not enabled (caused by a stupid change in
"Implement persistent id for proxies and servers")
Other changes:
- No more magic type valuse, use STATS_TYPE_FE/STATS_TYPE_BE/STATS_TYPE_SV
- Don't memset full s->data_ctx (it was clearing s->data_ctx.stats.{iid/type/sid},
instead initialize stats.sv & stats.sv_st (stats.px and stats.px_st were already
initialized)
With all that changes it was extremely easy to write a short perl plugin
for a perl-enabled net-snmp (also included in this patch).
29385 is my PEN (Private Enterprise Number) and I'm willing to donate
the SNMPv2-SMI::enterprises.29385.106.* OIDs for HAProxy if there
is nothing assigned already.
This patch adds a possibility to set a persistent id for a proxy/server.
Now, even if some proxies/servers are inserted/deleted/moved, iids and
sids can be still used reliable.
Some people add servers with tricky names (BACKEND or FRONTEND for example).
So I also added one more field ('type') to distinguish between a
backend (0), frontend (1) and server (2) without complicated logic:
if name==BACKEND and sid==0 then type is BACKEND else type is SERVER,
etc for a FRONTEND. It also makes possible to have one frontend with more
than one IP (a patch coming soon) with independed stats - for example to
differs between remote and local traffic.
Finally, I added documentation about the CSV format.
This patch depends on '[MEDIUM] Implement "track [<backend>/]<server>"'
This patch implements ability to set the current state of one server
by tracking another one. It:
- adds two variables: *tracknext, *tracked to struct server
- implements findserver(), similar to findproxy()
- adds "track" keyword accepting both "proxy/server" and "server" (assuming current proxy)
- verifies if both checks and tracking is not enabled at the same time
- changes set_server_down() to notify tracking server
- creates set_server_up(), set_server_disabled(), set_server_enabled() by
moving the code from process_chk() and adding notifications
- changes stats to show a name of tracked server instead of Chk/Dwn/Dwntime(html)
or by adding new variable (csv)
Changes from the previuos version:
- it is possibile to track independently of the declaration order
- one extra comma bug is fixed
- new condition to check if there is no disable-on-404 inconsistency
This patch adds two new variables: fastinter and downinter.
When server state is:
- non-transitionally UP -> inter (no change)
- transitionally UP (going down), unchecked or transitionally DOWN (going up) -> fastinter
- down -> downinter
It allows to set something like:
server sr6 127.0.51.61:80 cookie s6 check inter 10000 downinter 20000 fastinter 500 fall 3 weight 40
In the above example haproxy uses 10000ms between checks but as soon as
one check fails fastinter (500ms) is used. If server is down
downinter (20000) is used or fastinter (500ms) if one check pass.
Fastinter is also used when haproxy starts.
New "timeout.check" variable was added, if set haproxy uses it as an additional
read timeout, but only after a connection has been already established. I was
thinking about using "timeout.server" here but most people set this
with an addition reserve but still want checks to kick out laggy servers.
Please also note that in most cases check request is much simpler
and faster to handle than normal requests so this timeout should be smaller.
I also changed the timeout used for check connections establishing.
Changes from the previous version:
- use tv_isset() to check if the timeout is set,
- use min("timeout connect", "inter") but only if "timeout check" is set
as this min alone may be to short for full (connect + read) check,
- debug code (fprintf) commented/removed
- documentation
Compile tested only (sorry!) as I'm currently traveling but changes
are rather small and trivial.
The documentation now lists all keywords except the req* and rsp*. The
"server" keyword has been documented for mandatory parameters. Specific
settings are still waiting to be written in a dedicated section.
Using some Linux kernel patches, it is possible to redirect non-local
traffic to local sockets when IP forwarding is enabled. In order to
enable this option, we introduce the "transparent" option keyword on
the "bind" command line. It will make the socket reachable by remote
sources even if the destination address does not belong to the machine.
- options tcplog, tcpsplice and transparent have been documented.
- keywords "srvtimeout", "timeout queue", "timeout server" and
"timeout tarpit" have been documented
- keywords "transparent" and "use_backend" have been documented
Only "server", "source" and "stats *" remain undocumented
Options nolinger, persist, smtpchk and ssl-hello-chk have been
documented. All keywords and options up to and including option
tcpka are now documented.
This patch extends a little previously added functionality to also
count retries and redispatches for servers. Now it is possible to know
which server causes redispatches as it is not always the same that takes
most retries.
While working with the code I found that redistribute_pending() does not increment
srv->redispatches && be->redispatches. I don't know how to test it but
I think the fix is correct. If not I can withdraw it.
I also extended logs to show how many retries were done and if redispatching
was necessary ('+'). I'm using an additional session flag SN_REDISP to match
redispatched connections. I had to rearrange all defines in session.h to make
more room for it.
The documentation about logs was also fixed a little (sorry, english only),
as current version uses totally different format. BTW: examples are still
outdated, maybe next time...
Finally, I changed %d -> %u for retries/redispatches as those variables
are declared as unsigned.
In order to offer DoS protection, it may be required to lower the maximum
accepted time to receive a complete HTTP request without affecting the client
timeout. This helps protecting against established connections on which
nothing is sent. The client timeout cannot offer a good protection against
this abuse because it is an inactivity timeout, which means that if the
attacker sends one character every now and then, the timeout will not
trigger. With the HTTP request timeout, no matter what speed the client
types, the request will be aborted if it does not complete in time.
This new parameter makes it possible to override the default
number of consecutive incoming connections which can be
accepted on a socket. By default it is not limited on single
process mode, and limited to 8 in multi-process mode.
Add the "backlog" parameter to frontends, to give hints to
the system about the approximate listen backlog desired size.
In order to protect against SYN flood attacks, one solution is
to increase the system's SYN backlog size. Depending on the
system, sometimes it is just tunable via a system parameter,
sometimes it is not adjustable at all, and sometimes the system
relies on hints given by the application at the time of the
listen() syscall. By default, HAProxy passes the frontend's
maxconn value to the listen() syscall. On systems which can
make use of this value, it can sometimes be useful to be able
to specify a different value, hence this backlog parameter.
It is sometimes required to know some informations such as the
process uptime when consulting statistics. This patch adds the
"show info" command to query those informations on the UNIX
socket.
This patch adds a possibility to invert most of available options by
introducing the "no" keyword, available as an additional prefix.
If it is found arguments are shifted left and an additional flag (inv)
is set.
It allows to use all options from a current defaults section, except
the selected ones, for example:
-- cut here --
defaults
contimeout 4200
clitimeout 50000
srvtimeout 40000
option contstats
listen stats 1.2.3.4:80
no option contstats
-- cut here --
Currenly inversion works only with the "option" keyword.
The patch also moves last_checks calculation at the end of the readcfgfile()
function and changes "PR_O_FORCE_CLO | PR_O_HTTP_CLOSE" into "PR_O_FORCE_CLO"
in cfg_opts so it is possible to invert forceclose without breaking httpclose
(and vice versa) and to invert tcpsplice in one proxy but to keep a proper
last_checks value when tcpsplice is used in another proxy. Now, the code
checks for PR_O_FORCE_CLO everywhere it checks for PR_O_HTTP_CLOSE.
I also decided to depreciate "redisp" and "redispatch" keywords as it is IMHO
better to use "option redispatch" which can be inverted.
Some useful documentation were added and at the same time I sorted
(alfabetically) all valid options both in the code and the documentation.
New in this version is a small intro to HTTP, then a detailed
explanation of the following keywords :
acl, appsession, balance, bind, block, capture cookie,
capture request header, capture response header, clitimeout,
contimeout, cookie, default_backend, disabled, enabled, errorfile,
http-check disable-on-404, monitor fail, option contstats,
timeout client, timeout clitimeout, timeout connect,
timeout contimeout.
Others will be alphabetically added.
The code in haproxy-1.3.13.1 only supports syslogging to an internet
address. The attached patch:
- Adds support for syslogging to a UNIX domain socket (e.g., /dev/log).
If the address field begins with '/' (absolute file path), then
AF_UNIX is used to construct the socket. Otherwise, AF_INET is used.
- Achieves clean single-source build on both Mac OS X and Linux
(sockaddr_in.sin_len and sockaddr_un.sun_len field aren't always present).
For handling sendto() failures in send_log(), it appears that the existing
code is fine (no need to close/recreate socket) for both UDP and UNIX-domain
syslog server. So I left things alone (did not close/recreate socket).
Closing/recreating socket after each failure would also work, but would lead
to increased amount of unnecessary socket creation/destruction if syslog is
temporarily unavailable for some reason (especially for verbose loggers).
Please consider this patch for inclusion into the upstream haproxy codebase.
A new "timeout" keyword replaces old "{con|cli|srv}timeout", and
provides the ability to independantly set the following timeouts :
- client
- tarpit
- queue
- connect
- server
- appsession
Additionally, the "clitimeout", "contimeout" and "srvtimeout" values
are supported but deprecated. No warning is emitted yet when they are
used since the option is very new.
Other timeouts should follow soon now.
Hello,
You will find attached an updated release of previously submitted patch.
It polish some part and extend ACL engine to match IP and PORT parsed in
HTTP request. (and take care of comments made by Willy ! ;))
Best regards,
Alexandre
By default, counters used for statistics calculation are incremented
only when a session finishes. It works quite well when serving small
objects, but with big ones (for example large images or archives) or
with A/V streaming, a graph generated from haproxy counters looks like
a hedgehog.
This patch implements a contstats (continous statistics) option.
When set counters get incremented continuously, during a whole session.
Recounting touches a hotpath directly so it is not enabled by default,
as it has small performance impact (~0.5%).
Some applications do not have a strict persistence requirement, yet
it is still desirable for performance considerations, due to local
caches on the servers. For some reasons, there are some applications
which cannot rely on cookies, and for which the last resort is to use
a parameter passed in the URL.
The new 'url_param' balance method is there to solve this issue. It
accepts a parameter name which is looked up from the URL and which
is then hashed to select a server. If the parameter is not found,
then the round robin algorithm is used in order to provide a normal
load balancing across the servers for the first requests. It would
have been possible to use a source IP hash instead, but since such
applications are generally buried behind multiple levels of
reverse-proxies, it would not provide a good balance.
The doc has been updated, and two regression testing configurations
have been added.
This patch adds the "maxqueue" parameter to the server. This allows new
sessions to be immediately rebalanced when the server's queue is filled.
It's useful when session stickiness is just a performance boost (even a
huge one) but not a requirement.
This should only be used if session affinity isn't a hard functional
requirement but provides performance boost by keeping server-local
caches hot and compact).
Absence of 'maxqueue' option means unlimited queue. When queue gets filled
up to 'maxqueue' client session is moved from server-local queue to a global
one.
Removed old unused MODE_LOG and MODE_STATS, and replaced the "stats"
keyword in the global section. The new "stats" keyword in the global
section is used to create a UNIX socket on which the statistics will
be accessed. The client must issue a "show stat\n" command in order
to get a CSV-formated output similar to the output on the HTTP socket
in CSV mode.
For people who manage many haproxies, it is sometimes convenient
to be informed of their version. This patch adds this, with the
option to disable this report by specifying "stats hide-version".
Also, the feature may be permanently disabled by setting the
STATS_VERSION_STRING to "" (empty string), or the format can
simply be adjusted.
The following patch will give the ability to tweak socket linger mode.
You can use this option with "option nolinger" inside fronted or backend
configuration declaration.
This will help in environments where lots of FIN_WAIT sockets are
encountered.
Sometimes it may be desirable to automatically refresh the
stats page. Most browsers support the "Refresh:" header with
an interval in seconds. Specifying "stats refresh xxx" will
automatically add this header.
This new configuration manual intends to document every known keyword
of the configuration language. Right now, it enumerates them all and
describes how to use ACLs.
It is now possible to read error messages from local files,
using the 'errorfile' keyword. Those files are read during
parsing, so there's no I/O involved. They make it possible
to return custom error messages with custom status and headers.
ACLs now support operators such as 'eq', 'le', 'lt', 'ge' and 'gt'
in order to give more flexibility to the language. Because of this
change, the 'dst_limit' keyword changed to 'dst_conn' and now requires
either a range or a test such as 'dst_conn lt 1000' which is more
understandable.
By default, epoll/kqueue used to return as many events as possible.
This could sometimes cause huge latencies (latencies of up to 400 ms
have been observed with many thousands of fds at once). Limiting the
number of events returned also reduces the latency by avoiding too
many blind processing. The value is set to 200 by default and can be
changed in the global section using the tune.maxpollevents parameter.
Peter van Dijk contributed this patch which implements the "smtpchk"
option, which is to SMTP what "httpchk" is to HTTP. By default, it sends
"HELO localhost" to the servers, and waits for the 250 message, but it
can also send a specific request.
Since the introduction of speculative I/O, it was not always possible
to correctly detect a connection establishment. Particularly, in TCP
mode, there is no data to send and getsockopt() returns no error. The
solution consists in trying a connect() again to get its diagnostic.
The principle behind speculative I/O is to speculatively try to
perform I/O before registering the events in the system. This
considerably reduces the number of calls to epoll_ctl() and
sometimes even epoll_wait(), and manages to increase overall
performance by about 10%.
The new poller has been called "sepoll". It is used by default
on Linux when it works. A corresponding option "nosepoll" and
the command line argument "-ds" allow to disable it.
Patch from Fabrice Dulaunoy. Explanation below, and script
merged in examples/.
This patch allow to put a different address in the check part for each
server (and not only a specific port)
I need this feature because I've a complex settings where, when a specific
farm goes down, I need to switch a set of other farm either if these other
farm behave perfectly well.
For that purpose, I've made a small PERL daemon with some REGEX or PORT
test which allow me to test a bunch of thing.
Patch from Bryan Germann for 1.2.17.
In some circumstances, it is useful not to add the X-Forwarded-For
header, for instance when the client is another reverse-proxy or
stunnel running on the same machine and which already adds it. This
patch adds the "except" keyword to the "forwardfor" option, allowing
to specify an address or network which will not be added to this
header.
Patch from Marcus Rueckert for 1.2.17 :
"I added the attached patch to haproxy. I don't have a static uid/gid for
haproxy so i need to specify the username/groupname to run it as non
root user."
Some parts of HTTP processing were incorrectly called "request" while
they are messages or transactions. The following structure members
have changed :
http_msg.hdr_state => msg_state
http_msg.sor => som
http_req.req_state => removed
http_req => http_txn
The tcp-splicing code has been merged, and a doc has been written.
A configuration example has been derived from the previous content
switching sample.
The new parser uses an FSM to strictly follow RFC2616.
Headers are indexed and parsed only once they're all available.
That way, complex regexes make more sense.
HTTP processing is now performed in several phases by calling
multiple functions, making the code cleaner and easier to read.
Note that req[i]pass does not work anymore because it would
require that we mark a header to be ignored. What is really
needed is to have the ability to add an exception to a matching
(match xx except yy).
Several bugs have been fixed in appsession during the conversion
to the new FSM (method length and recovery on malloc errors).
The code does build and work with the debug examples, but is
not usable yet to connect to anything as it does not forward
the requests yet.
Released 1.3.1 with the following changes from 1.2.15 :
- now, haproxy warns about missing timeout during startup to try to
eliminate all those buggy configurations.
- added "Content-Type: text/html" in responses wherever appropriate, as
suggested by Cameron Simpson.
- implemented "option ssl-hello-chk" to use SSLv3 CLIENT HELLO messages to
test server's health
- implemented "monitor-uri" so that haproxy can reply to a specific URI with
an "HTTP/1.0 200 OK" response. This is useful to validate multiple proxies
at once.
This makes it possible to relay SSL connections in pure TCP instances while
ensuring the remote end really receives our data eventhough intermediate
agents (firewalls, proxies, ...) might acknowledge the connection.
The files are now stored under :
- include/haproxy for the generic includes
- include/types.h for the structures needed within prototypes
- include/proto.h for function prototypes and inline functions
- src/*.c for the C files
Most include files are now covered by LGPL. A last move still needs
to be done to put inline functions under GPL and not LGPL.
Version has been set to 1.3.0 in the code but some control still
needs to be done before releasing.
Released 1.2.14 with the following changes :
- new HTML status report with the 'stats' keyword.
- added the 'abortonclose' option to better resist traffic surges
- implemented dynamic traffic regulation with the 'minconn' option
- show request time on denied requests
- definitely fixed hot reconf on OpenBSD by the use of SO_REUSEPORT
- now a proxy instance is allowed to run without servers, which is
useful to dedicate one instance to stats
- added lots of error counters
- a missing parenthesis preventd matching of cacheable cookies
- a missing parenthesis in poll_loop() might have caused missed events.
When 'minconn' is set, the number of simultaneous sessions sent to the server
will be limited by a dynamic value depending on the global load on the
instance itself. The principle is to fix the maximal concurrency on the server
proportionnally to the instance's usage relative to its maxconn, with a minimum
fixed to <minconn>. The formula for the number of simultaneous sessions sent
to the server is then max(srv_minconn, srv_maxconn*px_conn/px_maxconn). This
helps unloading the servers when the load is very low.
to 'close', but does not actually close any connection. The problem
is, there are some servers which don't close the connection even if
the proxy tells them 'Connection: close'. A workaround was added by
the way of a new option 'forceclose' (which implies 'httpclose'),
and which makes the proxy close the outgoing channel to the server
once it has sent all its headers. Just don't use this with the
'CONNECT' method of course !
were erroneously load-balanced while the doc said the opposite.
Since load-balanced backup servers is one of the features some
people have been asking for, the problem was fixed to reflect the
documented behaviour and a new option 'allbackups' was introduced
to provide the feature to those who need it.
* second batch of socklen_t changes.
* clean-ups from Cameron Simpson.
* because tv_remain() does not know about eternity, using no timeout can
make select() spin around a null time-out. Bug reported by Cameron Simpson.
* client read timeout was not properly set to eternity initialized after an
accept() if it was not set in the config. It remained undetected so long
because eternity is 0 and newly allocated pages are zeroed by the system.
* do not call get_original_dst() when not in transparent mode.
* implemented a workaround for a bug in certain epoll() implementations on
linux-2.4 kernels (epoll-lt <= 0.21).
* implemented TCP keepalive with new options : tcpka, clitcpka, srvtcpka.
* changed the runtime argument to disable epoll() to '-de'
* changed the runtime argument to disable poll() to '-dp'
* added global options 'nopoll' and 'noepoll' to do the same at the
configuration level.
* added a 'linux24e' target to the Makefile for Linux 2.4 systems patched to
support epoll().
* changed default FD_SETSIZE to 65536 on Solaris (default=1024)
* conditionned signals redirection to #ifdef DEBUG_MEMORY
* implemented the HTTP 303 code for error redirection. This forces the
browser to fetch the given URI with a GET request. The new keyword for
this is 'errorloc303', and a new 'errorloc302' keyword has been created
to make them easily distinguishable.
* added more controls in the parser for valid use of '\x' sequence.
* few fixes from Alex & Klaus
* fixed a few errors in the documentation
* do not pre-initialize unused file-descriptors before select() anymore.
* add an architecture guide to the documentation
* released without any changes
* increased default BUFSIZE to 16 kB to accept max headers of 8 kB which is
compatible with Apache. This limit can be configured in the makefile now.
Thanks to Eric Fehr for the checks.
* added a per-server "source" option which now makes it possible to bind to
a different source for each (potentially identical) server.
* changed cookie-based server selection slightly to allow several servers to
share a same cookie, thus making it possible to associate backup servers to
live servers and ease soft-stop for maintenance periods. (Alexander Lazic)
* added the cookie 'prefix' mode which makes it possible to use persistence
with thin clients which support only one cookie. The server name is prefixed
before the application cookie, and restore back.
* fixed the order of servers within an instance to match documentation. Now
the servers are *really* used in the order of their declaration. This is
particularly important when multiple backup servers are in use.
* fixed a bug where a TCP connection would be logged twice if the 'logasap'
option was enabled without the 'tcplog' option.
* encode_string() would use hdr_encode_map instead of the map argument.
* the logged request is now encoded with '#XX' for unprintable characters
* new keywords 'capture request header' and 'capture response header' enable
logging of arbitrary HTTP headers in requests and responses
* removed "-DSOLARIS" after replacing the last inet_aton() with inet_pton()
* added the '-V' command line option to verbosely report errors even though
the -q or 'quiet' options are specified. This is useful with '-c'.
* added a Red Hat init script and a .spec from Simon Matter <simon.matter@invoca.ch>
* added 'rspdeny' and 'rspideny' to block certain responses to avoid sensible
information leak from servers.
* more examples added into the configuration
* the configurable HTTP health check introduced in 1.1.23 revealed a shameful
bug : the code still assumed that HTTP requests were the same size as the
original ones (22 bytes), and failed if they were not.
* added support for pidfiles.
* added the 'tcplog' option, which provides enhanced, HTTP-like logs for
generic TCP proxies, or lighter logs for HTTP proxies.
* fixed a time-out condition wrongly reported as client time-out in data
phase if the client timeout was lower than the connect timeout times the
number of retries.
* doc: added some precisions about the log timers
* if a client sent a full request then shut its write connection down, then
the request was aborted. This case was detected only when using haproxy
both as health-check client and as a server.
* if 'option httpchk' is used in a 'health' mode server, then responses will
change from 'OK' to 'HTTP/1.0 200 OK'.
* fixed a Linux-only bug in case of HTTP server health-checks, where a single
server response followed by a close could be ignored, and the server seen
as failed.
* renamed 'haproxy.txt' to 'haproxy-fr.txt'
* large documentation and examples cleanups
* fixed a stupid bug introduced in 1.1.22 which caused second and subsequent
'default' sections to keep previous parameters, and not initialize logs
correctly.
* fixed a second stupid bug introduced in 1.1.22 which caused configurations
relying on 'dispatch' mode to segfault at the first connection.
* 'option httpchk' now supports method, HTTP version and a few headers.
* now, 'option httpchk', 'cookie' and 'capture' can be specified in
'defaults' section
* a fresh new english documentation
* large Makefile cleanup for increased portability
* new build script 'build.cfg' for Formilux-0.1.8
* new startup script 'init.haproxy.flx0' for Formilux-0.1.8
* 'listen' now supports optionnal address:port-range lists
* 'bind' introduced to add new listen addresses
* fixed a bug which caused a session to be kept established on a server till
it timed out if the client closed during the DATA phase.
* the port part of each server address can now be empty to make the proxy
connect to the server on the same port it was connected to, be an absolute
unsigned number to reflect a single port (as in older versions), or an
explicitly signed number (+N/-N) to indicate that this offset must be
applied to the port the proxy was connected to, when connecting to the
server.
* the 'port' server option allows the user to specify a different
health-check port than the service one. It is mandatory when only relative
ports have been specified and check is required. By default, the checks are
sent to the service port.
* new 'defaults' section which is rather similar to 'listen' except that all
values are only used as default values for future 'listen' sections, until
a new 'defaults' resets them. At the moment, server options, regexes,
cookie names and captures cannot be set in the 'defaults' section.
* Makefile now optimizes for Ultrasparc by default on Solaris/Sparc
* large documentation updates and fixes
* new 'tests' directory with some debug files
* changed the debug output format so that it now includes the session unique
ID followed by the instance name at the beginning of each line.
* in debug mode, accept now shows the client's IP and port.
* added one 3 small debugging scripts to search and pretty print debug output
* changed the default health check request to "OPTIONS /" instead of
"OPTIONS *" since not all servers implement the later one.
* "option httpchk" now accepts an optional parameter allowing the user to
specify and URI other than '/' during health-checks.
* made Makefile more robust to pcre-config errors
* added 3 new pretty-print scripts : debug2ansi, debug2html and debugfind
* upgraded Formilux package to haproxy-1.1.21-flx.1.pkg
* removed the now obsolete haproxy2html.sh
* Haproxy can be compiled with PCRE regex instead of libc regex, by setting
REGEX=pcre on the make command line.
* HTTP health-checks now use "OPTIONS *" instead of "OPTIONS /".
* when explicit source address binding is required, it is now also used for
health-checks.
* added 'reqpass' and 'reqipass' to allow certain headers but not the request
itself.
* factored several strings to reduce binary size by about 2 kB.
* replaced setreuid() and setregid() with more standard setuid() and setgid().
* added 4 status flags to the log line indicating who ended the connection
first, the sessions state, the validity of the cookie, and action taken on
the set-cookie header.
* rearranged the changelog and removed it from haproxy.c
* large documentation updates
* add the notion of "backup" servers, which are used only when all other
servers are down.
* make Set-Cookie return "" instead of "(null)" when the server has no
cookie assigned (useful for backup servers).
* "log" now supports an optionnal level name (info, notice, err ...) above
which nothing is sent.
* replaced some strncmp() with memcmp() for better efficiency.
* added "capture cookie" option which logs client and/or server cookies
* cleaned up/down messages and dump servers states upon SIGHUP
* added a redirection feature for errors : "errorloc <errnum> <url>"
* now we won't insist on connecting to a dead server, even with a cookie,
unless option "persist" is specified.
* added HTTP/408 response for client request time-out and HTTP/50[234] for
server reply time-out or errors.
* updates to the examples files
* added a 'do_status' command to the Formilux init script
* fixed multi-cookie handling in client request to allow clean deletion
in insert+indirect mode. Now, only the server cookie is deleted and not
all the header. Should now be compliant to RFC2109.
* added a "nocache" option to "cookie" to specify that we explicitly want
to add a "cache-control" header when we add a cookie.
It is also possible to add an "Expires: <old-date>" to keep compatibility
with old/broken caches.
* some doc and examples cleanups
* if a cookie is used in insert+indirect mode, it's desirable that the
the servers don't see it. It was not possible to remove it correctly
with regexps, so now it's removed automatically.
* don't use snprintf()'s return value as an end of message since it may
be larger. This caused bus errors and segfaults in internal libc's
getenv() during localtime() in send_log().
* removed dead insecure send_syslog() function and all references to it.
* fixed warnings on Solaris due to buggy implementation of isXXXX().
* option "dontlognull"
* fixed "double space" bug in config parser
* fixed an uninitialized server field in case of dispatch
with no existing server which could cause a segfault during
logging.
* the pid logged was always the father's, which was wrong for daemons.
* fixed wrong level "LOG_INFO" for message "proxy started".
* http logging is now complete :
- ip:port, date, proxy, server
- req_time, conn_time, hdr_time, tot_time
- status, size, request
* source address binding
* connection logging displayed incorrect source address.
* added proxy start/stop and server up/down log events.
* replaced log message short buffers with larger trash.
* enlarged buffer to 8 kB and replace buffer to 4 kB.
* added a config.rc example for Formilux
* added a build script for Formilux
* added OpenBSD, Linux-2.2 and Linux-2.4 targets to the Makefile
* added a Formilux init script
* fixed a few timeout bugs
* rearranged the task scheduler subsystem to improve performance,
add new tasks, and make it easier to later port to librt ;
* allow multiple accept() for one select() wake up ;
* implemented internal load balancing with basic health-check ;
* cookie insertion and header add/replace/delete, with better strings
support.
* reworked buffer handling to fix a few rewrite bugs, and
improve overall performance.
* implement the "purge" option to delete server cookies in direct mode.
* fixed some error cases where the maxfd was not decreased.
* now supports transparent proxying, at least on linux 2.4.
* soft stop works again (fixed select timeout computation).
* it seems that TCP proxies sometimes cannot timeout.
* added a "quiet" mode.
* enforce file descriptor limitation on socket() and accept().
