This command supports "agent", "health", "state" and "weight" to adjust
various server attributes as well as changing server health check statuses
on the fly or setting the drain mode.
Now that it is possible to know whether a server is in forced maintenance
or inherits its maintenance status from another one, it is possible to
allow server tracking at more than one level. We still provide a loop
detection however.
Note that for the stats it's a bit trickier since we have to report the
check state which corresponds to the state of the server at the end of
the chain.
Function set_server_check_status() is very weird. It is called at the
end of a check to update the server's state before the new state is even
calculated, and possibly to log status changes, only if the proxy has
"option log-health-checks" set.
In order to do so, it employs an exhaustive list of the combinations
which can lead to a state change, while in practice almost all of
them may simply be deduced from the change of check status. Better,
some changes of check status are currently not detected while they
can be very valuable (eg: changes between L4/L6/TOUT/HTTP 500 for
example).
The doc was updated to reflect this.
Also, a minor change was made to consider s->uweight and not s->eweight
as meaning "DRAIN" since eweight can be null without the DRAIN mode (eg:
throttle, NOLB, ...).
Released version 1.5-dev25 with the following main changes :
- MEDIUM: connection: Implement and extented PROXY Protocol V2
- MINOR: ssl: clean unused ACLs declarations
- MINOR: ssl: adds fetchs and ACLs for ssl back connection.
- MINOR: ssl: merge client's and frontend's certificate functions.
- MINOR: ssl: adds ssl_f_sha1 fetch to return frontend's certificate fingerprint
- MINOR: ssl: adds sample converter base64 for binary type.
- MINOR: ssl: convert to binary ssl_fc_unique_id and ssl_bc_unique_id.
- BUG/MAJOR: ssl: Fallback to private session cache if current lock mode is not supported.
- MAJOR: ssl: Change default locks on ssl session cache.
- BUG/MINOR: chunk: Fix function chunk_strcmp and chunk_strcasecmp match a substring.
- MINOR: ssl: add global statement tune.ssl.force-private-cache.
- MINOR: ssl: remove fallback to SSL session private cache if lock init fails.
- BUG/MEDIUM: patterns: last fix was still not enough
- MINOR: http: export the smp_fetch_cookie function
- MINOR: http: generic pointer to rule argument
- BUG/MEDIUM: pattern: a typo breaks automatic acl/map numbering
- BUG/MAJOR: patterns: -i and -n are ignored for inlined patterns
- BUG/MINOR: proxy: unsafe initialization of HTTP transaction when switching from TCP frontend
- BUG/MINOR: http: log 407 in case of proxy auth
- MINOR: http: rely on the message body parser to send 100-continue
- MEDIUM: http: move reqadd after execution of http_request redirect
- MEDIUM: http: jump to dedicated labels after http-request processing
- BUG/MINOR: http: block rules forgot to increment the denied_req counter
- BUG/MINOR: http: block rules forgot to increment the session's request counter
- MEDIUM: http: move Connection header processing earlier
- MEDIUM: http: remove even more of the spaghetti in the request path
- MINOR: http: silently support the "block" action for http-request
- CLEANUP: proxy: rename "block_cond" to "block_rules"
- MEDIUM: http: emulate "block" rules using "http-request" rules
- MINOR: http: remove the now unused loop over "block" rules
- MEDIUM: http: factorize the "auth" action of http-request and stats
- MEDIUM: http: make http-request rules processing return a verdict instead of a rule
- MINOR: config: add minimum support for emitting warnings only once
- MEDIUM: config: inform the user about the deprecatedness of "block" rules
- MEDIUM: config: inform the user that "reqsetbe" is deprecated
- MEDIUM: config: inform the user only once that "redispatch" is deprecated
- MEDIUM: config: warn that '{cli,con,srv}timeout' are deprecated
- BUG/MINOR: auth: fix wrong return type in pat_match_auth()
- BUILD: config: remove a warning with clang
- BUG/MAJOR: http: connection setup may stall on balance url_param
- BUG/MEDIUM: http/session: disable client-side expiration only after body
- BUG/MEDIUM: http: correctly report request body timeouts
- BUG/MEDIUM: http: disable server-side expiration until client has sent the body
- MEDIUM: listener: make the accept function more robust against pauses
- BUILD: syscalls: remove improper inline statement in front of syscalls
- BUILD: ssl: SSL_CTX_set_msg_callback() needs openssl >= 0.9.7
- BUG/MAJOR: session: recover the correct connection pointer in half-initialized sessions
- DOC: add some explanation on the shared cache build options in the readme.
- MEDIUM: proxy: only adjust the backend's bind-process when already set
- MEDIUM: config: limit nbproc to the machine's word size
- MEDIUM: config: check the bind-process settings according to nbproc
- MEDIUM: listener: parse the new "process" bind keyword
- MEDIUM: listener: inherit the process mask from the proxy
- MAJOR: listener: only start listeners bound to the same processes
- MINOR: config: only report a warning when stats sockets are bound to more than 1 process
- CLEANUP: config: set the maxaccept value for peers listeners earlier
- BUG/MINOR: backend: only match IPv4 addresses with RDP cookies
- BUG/MINOR: checks: correctly configure the address family and protocol
- MINOR: tools: split is_addr() and is_inet_addr()
- MINOR: protocols: use is_inet_addr() when only INET addresses are desired
- MEDIUM: unix: add preliminary support for connecting to servers over UNIX sockets
- MEDIUM: checks: only complain about the missing port when the check uses TCP
- MEDIUM: unix: implement support for Linux abstract namespace sockets
- DOC: map_beg was missing from the table of map_* converters
- DOC: ebtree: indicate that prefix insertion/lookup may be used with strings
- MEDIUM: pattern: use ebtree's longest match to index/lookup string beginning
- BUILD: remove the obsolete BSD and OSX makefiles
- MEDIUM: unix: avoid a double connect probe when no data are sent
- DOC: stop referencing the slow git repository in the README
- BUILD: only build the systemd wrapper on Linux 2.6 and above
- DOC: update roadmap with completed tasks
- MEDIUM: session: implement half-closed timeouts (client-fin and server-fin)
Long-lived sessions are often subject to half-closed sessions resulting in
a lot of sessions appearing in FIN_WAIT state in the system tables, and no
way for haproxy to get rid of them. This typically happens because clients
suddenly disconnect without sending any packet (eg: FIN or RST was lost in
the path), and while the server detects this using an applicative heart
beat, haproxy does not close the connection.
This patch adds two new timeouts : "timeout client-fin" and
"timeout server-fin". The former allows one to override the client-facing
timeout when a FIN has been received or sent. The latter does the same for
server-facing connections, which is less useful.
These sockets are the same as Unix sockets except that there's no need
for any filesystem access. The address may be whatever string both sides
agree upon. This can be really convenient for inter-process communications
as well as for chaining backends to frontends.
These addresses are forced by prepending their address with "abns@" for
"abstract namespace".
Now that we know what processes a "bind" statement is attached to, we
have the ability to avoid starting some of them when they're not on the
proper process. This feature is disabled when running in foreground
however, so that debug mode continues to work with everything bound to
the first and only process.
The main purpose of this change is to finally allow the global stats
sockets to be each bound to a different process.
It can also be used to force haproxy to use different sockets in different
processes for the same IP:port. The purpose is that under Linux 3.9 and
above (and possibly other OSes), when multiple processes are bound to the
same IP:port via different sockets, the system is capable of performing
a perfect round-robin between the socket queues instead of letting any
process pick all the connections from a queue. This results in a smoother
load balancing and may achieve a higher performance with a large enough
maxaccept setting.
When a bind-process setting is present in a frontend or backend, we
now verify that the specified process range at least shares one common
process with those defined globally by nbproc. Then if the value is
set, it is reduced to the one enforced by nbproc.
A warning is emitted if process count does not match, and the fix is
done the following way :
- if a single process was specified in the range, it's remapped to
process #1
- if more than one process was specified, the binding is removed
and all processes are usable.
Note that since backends may inherit their settings from frontends,
depending on the declaration order, they may or may not be reported
as warnings.
Some consistency checks cannot be performed between frontends, backends
and peers at the moment because there is no way to check for intersection
between processes bound to some processes when the number of processes is
higher than the number of bits in a word.
So first, let's limit the number of processes to the machine's word size.
This means nbproc will be limited to 32 on 32-bit machines and 64 on 64-bit
machines. This is far more than enough considering that configs rarely go
above 16 processes due to scalability and management issues, so 32 or 64
should be fine.
This way we'll ensure we can always build a mask of all the processes a
section is bound to.
This commit modifies the PROXY protocol V2 specification to support headers
longer than 255 bytes allowing for optional extensions. It implements the
PROXY protocol V2 which is a binary representation of V1. This will make
parsing more efficient for clients who will know in advance exactly how
many bytes to read. Also, it defines and implements some optional PROXY
protocol V2 extensions to send information about downstream SSL/TLS
connections. Support for PROXY protocol V1 remains unchanged.
Previously ssl_fc_unique_id and ssl_bc_unique_id return a string encoded
in base64 of the RFC 5929 TLS unique identifier. This patch modify those fetches
to return directly the ID in the original binary format. The user can make the
choice to encode in base64 using the converter.
i.e. : ssl_fc_unique_id,base64
ssl_f_sha1 is a binary binary fetch used to returns the SHA-1 fingerprint of
the certificate presented by the frontend when the incoming connection was
made over an SSL/TLS transport layer. This can be used to know which
certificate was chosen using SNI.
Adds ssl fetchs and ACLs for outgoinf SSL/Transport layer connection with their
docs:
ssl_bc, ssl_bc_alg_keysize, ssl_bc_cipher, ssl_bc_protocol, ssl_bc_unique_id,
ssl_bc_session_id and ssl_bc_use_keysize.
Released version 1.5-dev24 with the following main changes :
- MINOR: pattern: find element in a reference
- MEDIUM: http: ACL and MAP updates through http-(request|response) rules
- MEDIUM: ssl: explicitly log failed handshakes after a heartbeat
- DOC: Full section dedicated to the converters
- MEDIUM: http: register http-request and http-response keywords
- BUG/MINOR: compression: correctly report incoming byte count
- BUG/MINOR: http: don't report server aborts as client aborts
- BUG/MEDIUM: channel: bi_putblk() must not wrap before the end of buffer
- CLEANUP: buffers: remove unused function buffer_contig_space_with_res()
- MEDIUM: stats: reimplement HTTP keep-alive on the stats page
- BUG/MAJOR: http: fix timeouts during data forwarding
- BUG/MEDIUM: http: 100-continue responses must process the next part immediately
- MEDIUM: http: move skipping of 100-continue earlier
- BUILD: stats: let gcc know that last_fwd cannot be used uninitialized...
- CLEANUP: general: get rid of all old occurrences of "session *t"
- CLEANUP: http: remove the useless "if (1)" inherited from version 1.4
- BUG/MEDIUM: stats: mismatch between behaviour and doc about front/back
- MEDIUM: http: enable analysers to have keep-alive on stats
- REORG: http: move HTTP Connection response header parsing earlier
- MINOR: stats: always emit HTTP/1.1 in responses
- MINOR: http: add capture.req.ver and capture.res.ver
- MINOR: checks: add a new global max-spread-checks directive
- BUG/MAJOR: http: fix the 'next' pointer when performing a redirect
- MINOR: http: implement the max-keep-alive-queue setting
- DOC: fix alphabetic order of tcp-check
- MINOR: connection: add a new error code for SSL with heartbeat
- MEDIUM: ssl: implement a workaround for the OpenSSL heartbleed attack
- BUG/MEDIUM: Revert "MEDIUM: ssl: Add standardized DH parameters >= 1024 bits"
- BUILD: http: remove a warning on strndup
- BUILD: ssl: avoid a warning about conn not used with OpenSSL < 1.0.1
- BUG/MINOR: ssl: really block OpenSSL's response to heartbleed attack
- MINOR: ssl: finally catch the heartbeats missing the padding
This patch allows manipulation of ACL and MAP content thanks to any
information available in a session: source IP address, HTTP request or
response header, etc...
It's an update "on the fly" of the content of the map/acls. This means
it does not resist to reload or restart of HAProxy.
Finn Arne Gangstad suggested that we should have the ability to break
keep-alive when the target server has reached its maxconn and that a
number of connections are present in the queue. After some discussion
around his proposed patch, the following solution was suggested : have
a per-proxy setting to fix a limit to the number of queued connections
on a server after which we break keep-alive. This ensures that even in
high latency networks where keep-alive is beneficial, we try to find a
different server.
This patch is partially based on his original proposal and implements
this configurable threshold.
These ones report a string as "HTTP/1.0" or "HTTP/1.1" depending on the
version of the request message or the response message, respectively.
The purpose is to be able to emit custom log lines reporting this version
in a persistent way.
In version 1.3.4, we got the ability to split configuration parts between
frontends and backends. The stats was attached to the backend and a control
was made to ensure that it was used only in a listen or backend section, but
not in a frontend.
The documentation clearly says that the statement may only be used in the
backend.
But since that same version above, the defaults stats configuration is
only filled in the frontend part of the proxy and not in the backend's.
So a backend will not get stats which are enabled in a defaults section,
despite what the doc says. However, a frontend configured after a defaults
section will get stats and will not emit the warning!
There were many technical limitations in 1.3.4 making it impossible to
have the stats working both in the frontend and backend, but now this has
become a total mess.
It's common however to see people create a frontend with a perfectly
working stats configuration which only emits a warning stating that it
might not work, adding to the confusion. Most people workaround the tricky
behaviour by declaring a "listen" section with no server, which was the
recommended solution in 1.3 where it was even suggested to add a dispatch
address to avoid a warning.
So the right solution seems to do the following :
- ensure that the defaults section's settings apply to the backends,
as documented ;
- let the frontends work in order not to break existing setups relying
on the defaults section ;
- officially allow stats to be declared in frontends and remove the
warninng
This patch should probably not be backported since it's not certain that
1.4 is fully compatible with having stats in frontends and backends (which
was really made possible thanks to applets).
Released version 1.5-dev23 with the following main changes :
- BUG/MINOR: reject malformed HTTP/0.9 requests
- MINOR: systemd wrapper: re-execute on SIGUSR2
- MINOR: systemd wrapper: improve logging
- MINOR: systemd wrapper: propagate exit status
- BUG/MINOR: tcpcheck connect wrong behavior
- MEDIUM: proxy: support use_backend with dynamic names
- MINOR: stats: Enhancement to stats page to provide information of last session time.
- BUG/MEDIUM: peers: fix key consistency for integer stick tables
- DOC: fix a typo on http-server-close and encapsulate options with double-quotes
- DOC: fix fetching samples syntax
- MINOR: ssl: add ssl_fc_unique_id to fetch TLS Unique ID
- MEDIUM: ssl: Use ALPN support as it will be available in OpenSSL 1.0.2
- DOC: fix typo
- CLEANUP: code style: use tabs to indent codes instead of spaces
- DOC: fix a few config typos.
- BUG/MINOR: raw_sock: also consider ENOTCONN in addition to EAGAIN for recv()
- DOC: lowercase format string in unique-id
- MINOR: set IP_FREEBIND on IPv6 sockets in transparent mode
- BUG/MINOR: acl: req_ssl_sni fails with SSLv3 record version
- BUG/MINOR: build: add missing objects in osx and bsd Makefiles
- BUG/MINOR: build: handle whitespaces in wc -l output
- BUG/MINOR: Fix name lookup ordering when compiled with USE_GETADDRINFO
- MEDIUM: ssl: Add standardized DH parameters >= 1024 bits
- BUG/MEDIUM: map: The map parser includes blank lines.
- BUG/MINOR: log: The log of quotted capture header has been terminated by 2 quotes.
- MINOR: standard: add function "encode_chunk"
- BUG/MINOR: http: fix encoding of samples used in http headers
- MINOR: sample: add hex converter
- MEDIUM: sample: change the behavior of the bin2str cast
- MAJOR: auth: Change the internal authentication system.
- MEDIUM: acl/pattern: standardisation "of pat_parse_int()" and "pat_parse_dotted_ver()"
- MEDIUM: pattern: The pattern parser no more uses <opaque> and just takes one string.
- MEDIUM: pattern: Change the prototype of the function pattern_register().
- CONTRIB: ip6range: add a network IPv6 range to mask converter
- MINOR: pattern: separe list element from the data part.
- MEDIUM: pattern: add indexation function.
- MEDIUM: pattern: The parse functions just return "struct pattern" without memory allocation
- MINOR: pattern: Rename "pat_idx_elt" to "pattern_tree"
- MINOR: sample: dont call the sample cast function "c_none"
- MINOR: standard: Add function for converting cidr to network mask.
- MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags
- MEDIUM: sample/http_proto: Add new type called method
- MINOR: dumpstats: Group map inline help
- MEDIUM: pattern: The function pattern_exec_match() returns "struct pattern" if the patten match.
- MINOR: dumpstats: change map inline sentences
- MINOR: dumpstats: change the "get map" display management
- MINOR: map/dumpstats: The cli cmd "get map ..." display the "int" format.
- MEDIUM: pattern: The match function browse itself the list or the tree.
- MEDIUM: pattern: Index IPv6 addresses in a tree.
- MEDIUM: pattern: add delete functions
- MEDIUM: pattern: add prune function
- MEDIUM: pattern: add sample lookup function.
- MEDIUM: pattern/dumpstats: The function pattern_lookup() is no longer used
- MINOR: map/pattern: The sample parser is stored in the pattern
- MAJOR: pattern/map: Extends the map edition system in the patterns
- MEDIUM: pattern: merge same pattern
- MEDIUM: pattern: The expected type is stored in the pattern head, and conversion is executed once.
- MINOR: pattern: Each pattern is identified by unique id.
- MINOR: pattern/acl: Each pattern of each acl can be load with specified id
- MINOR: pattern: The function "pattern_register()" is no longer used.
- MINOR: pattern: Merge function pattern_add() with pat_ref_push().
- MINOR: pattern: store configuration reference for each acl or map pattern.
- MINOR: pattern: Each pattern expression element store the reference struct.
- MINOR: dumpstats: display the reference for th key/pattern and value.
- MEDIUM: pattern: delete() function uses the pat_ref_elt to find the element to be removed
- MEDIUM: pattern_find_smp: functions find_smp uses the pat_ref_elt to find the element to be removed
- MEDIUM: dumpstats/pattern: display and use each pointer of each pattern dumped
- MINOR: pattern/map/acl: Centralization of the file parsers
- MINOR: pattern: Check if the file reference is not used with acl and map
- MINOR: acl/pattern: Acl "-M" option force to load file as map file with two columns
- MEDIUM: dumpstats: Display error message during add of values.
- MINOR: pattern: The function pat_ref_set() have now atomic behavior
- MINOR: regex: The pointer regstr in the struc regex is no longer used.
- MINOR: cli: Block the usage of the command "acl add" in many cases.
- MINOR: doc: Update the documentation about the map and acl
- MINOR: pattern: index duplicates
- MINOR: configuration: File and line propagation
- MINOR: dumpstat/conf: display all the configuration lines that using pattern reference
- MINOR: standard: Disable ip resolution during the runtime
- MINOR: pattern: Remove the flag "PAT_F_FROM_FILE".
- MINOR: pattern: forbid dns resolutions
- DOC: document "get map" / "get acl" on the CLI
- MEDIUM: acl: Change the acl register struct
- BUG/MEDIUM: acl: boolean only matches were broken by recent changes
- DOC: pattern: pattern organisation schematics
- MINOR: pattern/cli: Update used terms in documentation and cli
- MINOR: cli: remove information about acl or map owner.
- MINOR: session: don't always assume there's a listener
- MINOR: pattern: Add function to prune and reload pattern list.
- MINOR: standard: Add ipv6 support in the function url2sa().
- MEDIUM: config: Dynamic sections.
- BUG/MEDIUM: stick-table: fix IPv4-to-IPv6 conversion in src_* fetches
- MINOR: http: Add the "language" converter to for use with accept-language
- BUG/MINOR: log: Don't dump empty unique-id
- BUG/MAJOR: session: fix a possible crash with src_tracked
- DOC: Update "language" documentation
- MINOR: http: add the function "del-header" to the directives http-request and http-response
- DOC: add some information on capture.(req|res).hdr
- MINOR: http: capture.req.method and capture.req.uri
- MINOR: http: optimize capture.req.method and capture.req.uri
- MINOR: session: clean up the connection free code
- BUG/MEDIUM: checks: immediately report a connection success
- MEDIUM: connection: don't use real send() flags in snd_buf()
- OPTIM: ssl: implement dynamic record size adjustment
- MINOR: stats: report exact last session time in backend too
- BUG/MEDIUM: stats: the "lastsess" field must appear last in the CSV.
- BUG/MAJOR: check: fix memory leak in "tcp-check connect" over SSL
- BUG/MINOR: channel: initialize xfer_small/xfer_large on new buffers
- MINOR: channel: add the date of last read in the channel
- MEDIUM: stream-int: automatically disable CF_STREAMER flags after idle
- MINOR: ssl: add DEFAULT_SSL_MAX_RECORD to set the record size at build time
- MINOR: config: make the stream interface idle timer user-configurable
- MINOR: config: add global directives to set default SSL ciphers
- MINOR: sample: add a rand() sample fetch to return a sample.
- BUG/MEDIUM: config: immediately abort if peers section has no name
- BUG/MINOR: ssl: fix syntax in config error message
- BUG/MEDIUM: ssl: always send a full buffer after EAGAIN
- BUG/MINOR: config: server on-marked-* statement is ignored in default-server
- BUG/MEDIUM: backend: prefer-last-server breaks redispatch
- BUG/MEDIUM: http: continue to emit 503 on keep-alive to different server
- MEDIUM: acl: fix pattern type for payload / payload_lv
- BUG/MINOR: config: fix a crash on startup when a disabled backend references a peer
- BUG/MEDIUM: compression: fix the output type of the compressor name
- BUG/MEDIUM: http: don't start to forward request data before the connect
- MINOR: http: release compression context only in http_end_txn()
- MINOR: protect ebimtree/ebistree against multiple inclusions
- MEDIUM: proxy: create a tree to store proxies by name
- MEDIUM: proxy: make findproxy() use trees to look up proxies
- MEDIUM: proxy: make get_backend_server() use findproxy() to lookup proxies
- MEDIUM: stick-table: lookup table names using trees.
- MEDIUM: config: faster lookup for duplicated proxy name
- CLEANUP: acl: remove obsolete test in parse_acl_expr()
- MINOR: sample: move smp_to_type to sample.c
- MEDIUM: compression: consider the "q=" attribute in Accept-Encoding
- REORG: cfgparse: move server keyword parsing to server.c
- BUILD: adjust makefile for AIX 5.1
- BUG/MEDIUM: pattern: fix wrong definition of the pat_prune_fcts array
- CLEANUP: pattern: move array definitions to proto/ and not types/
- BUG/MAJOR: counters: check for null-deref when looking up an alternate table
- BUILD: ssl: previous patch failed
- BUILD/MEDIUM: standard: get rid of the last strcpy()
- BUILD/MEDIUM: standard: get rid of sprintf()
- BUILD/MEDIUM: cfgparse: get rid of sprintf()
- BUILD/MEDIUM: checks: get rid of sprintf()
- BUILD/MEDIUM: http: remove calls to sprintf()
- BUG/MEDIUM: systemd-wrapper: fix locating of haproxy binary
- BUILD/MINOR: ssl: remove one call to sprintf()
- MEDIUM: http: don't reject anymore message bodies not containing the url param
- MEDIUM: http: wait for the first chunk or message body length in http_process_body
- CLEANUP: http: rename http_process_request_body()
- CLEANUP: http: prepare dedicated processing for chunked encoded message bodies
- MINOR: http: make msg->eol carry the last CRLF length
- MAJOR: http: do not use msg->sol while processing messages or forwarding data
- MEDIUM: http: http_parse_chunk_crlf() must not advance the buffer pointer
- MAJOR: http: don't update msg->sov anymore while processing the body
- MINOR: http: add a small helper to compute the amount of body bytes present
- MEDIUM: http: add a small helper to compute how far to rewind to find headers
- MINOR: http: add a small helper to compute how far to rewind to find URI
- MEDIUM: http: small helpers to compute how far to rewind to find BODY and DATA
- MAJOR: http: reset msg->sov after headers are forwarded
- MEDIUM: http: forward headers again while waiting for connection to complete
- BUG/MINOR: http: deinitialize compression after a parsing error
- BUG/MINOR: http: deinitialize compression after a compression error
- MEDIUM: http: headers must be forwarded even if data was already inspected
- MAJOR: http: re-enable compression on chunked encoding
- MAJOR: http/compression: fix chunked-encoded response processing
- MEDIUM: http: cleanup: centralize a little bit HTTP compression end
- MEDIUM: http: start to centralize the forwarding code
- MINOR: http: further cleanups of response forwarding function
- MEDIUM: http: only allocate the temporary compression buffer when needed
- MAJOR: http: centralize data forwarding in the request path
- CLEANUP: http: document the response forwarding states
- CLEANUP: http: remove all calls to http_silent_debug()
- DOC: internal: add some reminders about HTTP parsing and pointer states
- BUG/MAJOR: http: fix bug in parse_qvalue() when selecting compression algo
- BUG/MINOR: stats: last session was not always set
- DOC: add pointer to the Cyril's HTML doc in the README
- MEDIUM: config: relax use_backend check to make the condition optional
- MEDIUM: config: report misplaced http-request rules
- MEDIUM: config: report misplaced use-server rules
- DOC: update roadmap with what was done.
Since it became possible to use log-format expressions in use_backend,
having a mandatory condition becomes annoying because configurations
are full of "if TRUE". Let's relax the check to accept no condition
like many other keywords (eg: redirect).
The syntax used to document fetching samples with optional arguments was not
always valid. This commit fixes this issue in order to allow an easier parsing
of the documentation.
This is the continuation of previous patch. Now that full buffers are
not rejected anymore, let's wait for at least the advertised chunk or
body length to be present or the buffer to be full. When either
condition is met, the message processing can go forward.
Thus we don't need to use url_param_post_limit anymore, which was passed
in the configuration as an optionnal <max_wait> parameter after the
"check_post" value. This setting was necessary when the feature was
implemented because there was no support for parsing message bodies.
The argument is now silently ignored if set in the configuration.
language(<value[;value[;value[;...]]]>[,<default>])
Returns the value with the highest q-factor from a list as
extracted from the "accept-language" header using "req.fhdr".
Values with no q-factor have a q-factor of 1. Values with a
q-factor of 0 are dropped. Only values which belong to the
list of semi-colon delimited <values> will be considered. If
no value matches the given list and a default value is
provided, it is returned. Note that language names may have
a variant after a dash ('-'). If this variant is present in
the list, it will be matched, but if it is not, only the base
language is checked. The match is case-sensitive, and the
output string is always one of those provided in arguments.
The ordering of arguments is meaningless, only the ordering
of the values in the request counts, as the first value among
multiple sharing the same q-factor is used.
Example :
# this configuration switches to the backend matching a
# given language based on the request :
acl de req.fhdr(accept-language),language(de;es;fr;en) de
acl es req.fhdr(accept-language),language(de;es;fr;en) es
acl fr req.fhdr(accept-language),language(de;es;fr;en) fr
acl en req.fhdr(accept-language),language(de;es;fr;en) en
use_backend german if de
use_backend spanish if es
use_backend french if fr
use_backend english if en
default_backend choose_your_language
The TLS unique id, or unique channel binding, is a byte string that can be
pulled from a TLS connection and it is unique to that connection. It is
defined in RFC 5929 section 3. The value is used by various upper layer
protocols as part of an extra layer of security. For example XMPP
(RFC 6120) and EST (RFC 7030).
Add the ssl_fc_unique_id keyword and corresponding sample fetch method.
Value is retrieved from OpenSSL and base64 encoded as described in RFC
5929 section 3.
We have a use case where we look up a customer ID in an HTTP header
and direct it to the corresponding server. This can easily be done
using ACLs and use_backend rules, but the configuration becomes
painful to maintain when the number of customers grows to a few
tens or even a several hundreds.
We realized it would be nice if we could make the use_backend
resolve its name at run time instead of config parsing time, and
use a similar expression as http-request add-header to decide on
the proper backend to use. This permits the use of prefixes or
even complex names in backend expressions. If no name matches,
then the default backend is used. Doing so allowed us to get rid
of all the use_backend rules.
Since there are some config checks on the use_backend rules to see
if the referenced backend exists, we want to keep them to detect
config errors in normal config. So this patch does not modify the
default behaviour and proceeds this way :
- if the backend name in the use_backend directive parses as a log
format rule, it's used as-is and is resolved at run time ;
- otherwise it's a static name which must be valid at config time.
There was the possibility of doing this with the use-server directive
instead of use_backend, but it seems like use_backend is more suited
to this task, as it can be used for other purposes. For example, it
becomes easy to serve a customer-specific proxy.pac file based on the
customer ID by abusing the errorfile primitive :
use_backend bk_cust_%[hdr(X-Cust-Id)] if { hdr(X-Cust-Id) -m found }
default_backend bk_err_404
backend bk_cust_1
errorfile 200 /etc/haproxy/static/proxy.pac.cust1
Signed-off-by: Bertrand Jacquin <bjacquin@exosec.fr>
This patch replace the word <name> by the word <file>. This word defines
the (string) returned by show "map/acl". This patch also update
documentation to explain how is composed the map or acl identifier.
This patch remove the limit of 32 groups. It also permit to use standard
"pat_parse_str()" function in place of "pat_parse_strcat()". The
"pat_parse_strcat()" is no longer used and its removed. Before this
patch, the groups are stored in a bitfield, now they are stored in a
list of strings. The matching is slower, but the number of groups is
low and generally the list of allowed groups is short.
The fetch function "smp_fetch_http_auth_grp()" used with the name
"http_auth_group" return valid username. It can be used as string for
displaying the username or with the acl "http_auth_group" for checking
the group of the user.
Maybe the names of the ACL and fetch methods are no longer suitable, but
I keep the current names for conserving the compatibility with existing
configurations.
The function "userlist_postinit()" is created from verification code
stored in the big function "check_config_validity()". The code is
adapted to the new authentication storage system and it is moved in the
"src/auth.c" file. This function is used to check the validity of the
users declared in groups and to check the validity of groups declared
on the "user" entries.
This resolve function is executed before the check of all proxy because
many acl needs solved users and groups.
This new filter converts BIN type to its hexadecimal
representation in STR type. It is used to keep the
compatibility with the original bin2str cast.
It will be useful when bin2str changes to copy the
string as-is without encoding anymore.
Sometimes it can be useful to generate a random value, at least
for debugging purposes, but also to take routing decisions or to
pass such a value to a backend server.
The ability to globally override the default client and server cipher
suites has been requested multiple times since the introduction of SSL.
This commit adds two new keywords to the global section for this :
- ssl-default-bind-ciphers
- ssl-default-server-ciphers
It is still possible to preset them at build time by setting the macros
LISTEN_DEFAULT_CIPHERS and CONNECT_DEFAULT_CIPHERS.
The new tune.idletimer value allows one to set a different value for
idle stream detection. The default value remains set to one second.
It is possible to disable it using zero, and to change the default
value at build time using DEFAULT_IDLE_TIMER.
It happens that latest change broke some monitoring tools which expect the
field to be found at the same position as indicated in the doc. Let's move
it to the last column instead.
Add 2 sample fetchs allowing to extract the method and the uri of an
HTTP request.
FIXME: the sample fetches parser can't add the LW_REQ requirement, at
the moment this flag is used automatically when you use sample fetches.
Note: also fixed the alphabetical order of other capture.req.* keywords
in the doc.
Released version 1.5-dev22 with the following main changes :
- MEDIUM: tcp-check new feature: connect
- MEDIUM: ssl: Set verify 'required' as global default for servers side.
- MINOR: ssl: handshake optim for long certificate chains.
- BUG/MINOR: pattern: pattern comparison executed twice
- BUG/MEDIUM: map: segmentation fault with the stats's socket command "set map ..."
- BUG/MEDIUM: pattern: Segfault in binary parser
- MINOR: pattern: move functions for grouping pat_match_* and pat_parse_* and add documentation.
- MINOR: standard: The parse_binary() returns the length consumed and his documentation is updated
- BUG/MINOR: payload: the patterns of the acl "req.ssl_ver" are no parsed with the good function.
- BUG/MEDIUM: pattern: "pat_parse_dotted_ver()" set bad expect_type.
- BUG/MINOR: sample: The c_str2int converter does not fail if the entry is not an integer
- BUG/MEDIUM: http/auth: Sometimes the authentication credentials can be mix between two requests
- MINOR: doc: Bad cli function name.
- MINOR: http: smp_fetch_capture_header_* fetch captured headers
- BUILD: last release inadvertently prepended a "+" in front of the date
- BUG/MEDIUM: stream-int: fix the keep-alive idle connection handler
- BUG/MEDIUM: backend: do not re-initialize the connection's context upon reuse
- BUG: Revert "OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes"
- BUG/MINOR: checks: successful check completion must not re-enable MAINT servers
- MINOR: http: try to stick to same server after status 401/407
- BUG/MINOR: http: always disable compression on HTTP/1.0
- OPTIM: poll: restore polling after a poll/stop/want sequence
- OPTIM: http: don't stop polling for read on the client side after a request
- BUG/MEDIUM: checks: unchecked servers could not be enabled anymore
- BUG/MEDIUM: stats: the web interface must check the tracked servers before enabling
- BUG/MINOR: channel: CHN_INFINITE_FORWARD must be unsigned
- BUG/MINOR: stream-int: do not clear the owner upon unregister
- MEDIUM: stats: add support for HTTP keep-alive on the stats page
- BUG/MEDIUM: stats: fix HTTP/1.0 breakage introduced in previous patch
- Revert "MEDIUM: stats: add support for HTTP keep-alive on the stats page"
- MAJOR: channel: add a new flag CF_WAKE_WRITE to notify the task of writes
- OPTIM: session: set the READ_DONTWAIT flag when connecting
- BUG/MINOR: http: don't clear the SI_FL_DONT_WAKE flag between requests
- MINOR: session: factor out the connect time measurement
- MEDIUM: session: prepare to support earlier transitions to the established state
- MEDIUM: stream-int: make si_connect() return an established state when possible
- MINOR: checks: use an inline function for health_adjust()
- OPTIM: session: put unlikely() around the freewheeling code
- MEDIUM: config: report a warning when multiple servers have the same name
- BUG: Revert "OPTIM: poll: restore polling after a poll/stop/want sequence"
- BUILD/MINOR: listener: remove a glibc warning on accept4()
- BUG/MAJOR: connection: fix mismatch between rcv_buf's API and usage
- BUILD: listener: fix recent accept4() again
- BUG/MAJOR: ssl: fix breakage caused by recent fix abf08d9
- BUG/MEDIUM: polling: ensure we update FD status when there's no more activity
- MEDIUM: listener: fix polling management in the accept loop
- MINOR: protocol: improve the proto->drain() API
- MINOR: connection: add a new conn_drain() function
- MEDIUM: tcp: report in tcp_drain() that lingering is already disabled on close
- MEDIUM: connection: update callers of ctrl->drain() to use conn_drain()
- MINOR: connection: add more error codes to report connection errors
- MEDIUM: tcp: report connection error at the connection level
- MEDIUM: checks: make use of chk_report_conn_err() for connection errors
- BUG/MEDIUM: unique_id: HTTP request counter is not stable
- DOC: fix misleading information about SIGQUIT
- BUG/MAJOR: fix freezes during compression
- BUG/MEDIUM: stream-interface: don't wake the task up before end of transfer
- BUILD: fix VERDATE exclusion regex
- CLEANUP: polling: rename "spec_e" to "state"
- DOC: add a diagram showing polling state transitions
- REORG: polling: rename "spec_e" to "state" and "spec_p" to "cache"
- REORG: polling: rename "fd_spec" to "fd_cache"
- REORG: polling: rename the cache allocation functions
- REORG: polling: rename "fd_process_spec_events()" to "fd_process_cached_events()"
- MAJOR: polling: rework the whole polling system
- MAJOR: connection: remove the CO_FL_WAIT_{RD,WR} flags
- MEDIUM: connection: remove conn_{data,sock}_poll_{recv,send}
- MEDIUM: connection: add check for readiness in I/O handlers
- MEDIUM: stream-interface: the polling flags must always be updated in chk_snd_conn
- MINOR: stream-interface: no need to call fd_stop_both() on error
- MEDIUM: connection: no need to recheck FD state
- CLEANUP: connection: use conn_ctrl_ready() instead of checking the flag
- CLEANUP: connection: use conn_xprt_ready() instead of checking the flag
- CLEANUP: connection: fix comments in connection.h to reflect new behaviour.
- OPTIM: raw-sock: don't speculate after a short read if polling is enabled
- MEDIUM: polling: centralize polled events processing
- MINOR: polling: create function fd_compute_new_polled_status()
- MINOR: cli: add more information to the "show info" output
- MEDIUM: listener: add support for limiting the session rate in addition to the connection rate
- MEDIUM: listener: apply a limit on the session rate submitted to SSL
- REORG: stats: move the stats socket states to dumpstats.c
- MINOR: cli: add the new "show pools" command
- BUG/MEDIUM: counters: flush content counters after each request
- BUG/MEDIUM: counters: fix stick-table entry leak when using track-sc2 in connection
- MINOR: tools: add very basic support for composite pointers
- MEDIUM: counters: stop relying on session flags at all
- BUG/MINOR: cli: fix missing break in command line parser
- BUG/MINOR: config: correctly report when log-format headers require HTTP mode
- MAJOR: http: update connection mode configuration
- MEDIUM: http: make keep-alive + httpclose be passive mode
- MAJOR: http: switch to keep-alive mode by default
- BUG/MEDIUM: http: fix regression caused by recent switch to keep-alive by default
- BUG/MEDIUM: listener: improve detection of non-working accept4()
- BUILD: listener: add fcntl.h and unistd.h
- BUG/MINOR: raw_sock: correctly set the MSG_MORE flag
A new tcp-check rule type: connect.
It allows HAProxy to test applications which stand on multiple ports or
multiple applications load-balanced through the same backend.
Since we support HTTP keep-alive, there is no more reason for staying
in tunnel mode by default. It is confusing for new users and creates
more issues than it solves. Option "http-tunnel" is available to force
to use it if really desired.
Switching to KA by default has implied to change the value of some
option flags and some transaction flags so that value zero (default)
matches keep-alive. That explains why more code has been changed than
expected. Tests have been run on the 25 combinations of frontend and
backend options, plus a few with option http-pretend-keepalive, and
no anomaly was found.
The relation between frontend and backends remains the same. Options
have been updated to take precedence over http-keep-alive which is now
implicit.
All references in the doc to haproxy not supporting keep-alive have
been fixed, and the doc for config options has been updated.
At the very beginning of haproxy, there was "option httpclose" to make
haproxy add a "Connection: close" header in both directions to invite
both sides to agree on closing the connection. It did not work with some
rare products, so "option forceclose" was added to do the same and actively
close the connection. Then client-side keep-alive was supported, so option
http-server-close was introduced. Now we have keep-alive with a fourth
option, not to mention the implicit tunnel mode.
The connection configuration has become a total mess because all the
options above may be combined together, despite almost everyone thinking
they cancel each other, as judging from the common problem reports on the
mailing list. Unfortunately, re-reading the doc shows that it's not clear
at all that options may be combined, and the opposite seems more obvious
since they're compared. The most common issue is options being set in the
defaults section that are not negated in other sections, but are just
combined when the user expects them to be overloaded. The migration to
keep-alive by default will only make things worse.
So let's start to address the first problem. A transaction can only work in
5 modes today :
- tunnel : haproxy doesn't bother with what follows the first req/resp
- passive close : option http-close
- forced close : option forceclose
- server close : option http-server-close with keep-alive on the client side
- keep-alive : option http-keep-alive, end to end
All 16 combination for each section fall into one of these cases. Same for
the 256 combinations resulting from frontend+backend different modes.
With this patch, we're doing something slightly different, which will not
change anything for users with valid configs, and will only change the
behaviour for users with unsafe configs. The principle is that these options
may not combined anymore, and that the latest one always overrides all the
other ones, including those inherited from the defaults section. The "no
option xxx" statement is still supported to cancel one option and fall back
to the default one. It is mainly needed to ignore defaults sections (eg:
force the tunnel mode). The frontend+backend combinations have not changed.
So for examplen the following configuration used to put the connection
into forceclose :
defaults http
mode http
option httpclose
frontend foo.
option http-server-close
=> http-server-close+httpclose = forceclose before this patch! Now
the frontend's config replaces the defaults config and results in
the more expected http-server-close.
All 25 combinations of the 5 modes in (frontend,backend) have been
successfully tested.
In order to prepare for upcoming changes, a new "option http-tunnel" was
added. It currently only voids all other options, and has the lowest
precedence when mixed with another option in another frontend/backend.
If no CA file specified on a server line, the config parser will show an error.
Adds an cmdline option '-dV' to re-set verify 'none' as global default on
servers side (previous behavior).
Also adds 'ssl-server-verify' global statement to set global default to
'none' or 'required'.
WARNING: this changes the default verify mode from "none" to "required" on
the server side, and it *will* break insecure setups.
One year ago, commit 5d5b5d8 ("MEDIUM: proto_tcp: add support for tracking
L7 information") brought support for tracking L7 information in tcp-request
content rules. Two years earlier, commit 0a4838c ("[MEDIUM] session-counters:
correctly unbind the counters tracked by the backend") used to flush the
backend counters after processing a request.
While that earliest patch was correct at the time, it became wrong after
the second patch was merged. The code does what it says, but the concept
is flawed. "TCP request content" rules are evaluated for each HTTP request
over a single connection. So if such a rule in the frontend decides to
track any L7 information or to track L4 information when an L7 condition
matches, then it is applied to all requests over the same connection even
if they don't match. This means that a rule such as :
tcp-request content track-sc0 src if { path /index.html }
will count one request for index.html, and another one for each of the
objects present on this page that are fetched over the same connection
which sent the initial matching request.
Worse, it is possible to make the code do stupid things by using multiple
counters:
tcp-request content track-sc0 src if { path /foo }
tcp-request content track-sc1 src if { path /bar }
Just sending two requests first, one with /foo, one with /bar, shows
twice the number of requests for all subsequent requests. Just because
both of them persist after the end of the request.
So the decision to flush backend-tracked counters was not the correct
one. In practice, what is important is to flush countent-based rules
since they are the ones evaluated for each request.
Doing so requires new flags in the session however, to keep track of
which stick-counter was tracked by what ruleset. A later change might
make this easier to maintain over time.
This bug is 1.5-specific, no backport to stable is needed.
show pools
Dump the status of internal memory pools. This is useful to track memory
usage when suspecting a memory leak for example. It does exactly the same
as the SIGQUIT when running in foreground except that it does not flush
the pools.
Just like the previous commit, we sometimes want to limit the rate of
incoming SSL connections. While it can be done for a frontend, it was
not possible for a whole process, which makes sense when multiple
processes are running on a system to server multiple customers.
The new global "maxsslrate" setting is usable to fix a limit on the
session rate going to the SSL frontends. The limits applies before
the SSL handshake and not after, so that it saves the SSL stack from
expensive key computations that would finally be aborted before being
accounted for.
The same setting may be changed at run time on the CLI using
"set rate-limit ssl-session global".
It's sometimes useful to be able to limit the connection rate on a machine
running many haproxy instances (eg: per customer) but it removes the ability
for that machine to defend itself against a DoS. Thus, better also provide a
limit on the session rate, which does not include the connections rejected by
"tcp-request connection" rules. This permits to have much higher limits on
the connection rate without having to raise the session rate limit to insane
values.
The limit can be changed on the CLI using "set rate-limit sessions global",
or in the global section using "maxsessrate".
Patrick Hemmer reported that using unique_id_format and logs did not
report the same unique ID counter since commit 9f09521 ("BUG/MEDIUM:
unique_id: HTTP request counter must be unique!"). This is because
the increment was done while producing the log message, so it was
performed twice.
A better solution consists in fetching a new value once per request
and saving it in the request or session context for all of this
request's life.
It happens that sessions already have a unique ID field which is used
for debugging and reporting errors, and which differs from the one
sent in logs and unique_id header.
So let's change this to reuse this field to have coherent IDs everywhere.
As of now, a session gets a new unique ID once it is instanciated. This
means that TCP sessions will also benefit from a unique ID that can be
logged. And this ID is renewed for each extra HTTP request received on
an existing session. Thus, all TCP sessions and HTTP requests will have
distinct IDs that will be stable along all their life, and coherent
between all places where they're used (logs, unique_id header,
"show sess", "show errors").
This feature is 1.5-specific, no backport to 1.4 is needed.
In HTTP keep-alive mode, if we receive a 401, we still have a chance
of being able to send the visitor again to the same server over the
same connection. This is required by some broken protocols such as
NTLM, and anyway whenever there is an opportunity for sending the
challenge to the proper place, it's better to do it (at least it
helps with debugging).
Released version 1.5-dev21 with the following main changes :
- MINOR: stats: don't use a monospace font to report numbers
- MINOR: session: remove debugging code
- BUG/MAJOR: patterns: fix double free caused by loading strings from files
- MEDIUM: http: make option http_proxy automatically rewrite the URL
- BUG/MEDIUM: http: cook_cnt() forgets to set its output type
- BUG/MINOR: stats: correctly report throttle rate of low weight servers
- BUG/MEDIUM: checks: servers must not start in slowstart mode
- BUG/MINOR: acl: parser must also stop at comma on ACL-only keywords
- MEDIUM: stream-int: implement a very simplistic idle connection manager
- DOC: update the ROADMAP file
Released version 1.5-dev20 with the following main changes :
- DOC: add missing options to the manpage
- DOC: add manpage references to all system calls
- DOC: update manpage reference to haproxy-en.txt
- DOC: remove -s and -l options from the manpage
- DOC: missing information for the "description" keyword
- DOC: missing http-send-name-header keyword in keyword table
- MINOR: tools: function my_memmem() to lookup binary contents
- MEDIUM: checks: add send/expect tcp based check
- MEDIUM: backend: Enhance hash-type directive with an algorithm options
- MEDIUM: backend: Implement avalanche as a modifier of the hashing functions.
- DOC: Documentation for hashing function, with test results.
- BUG/MEDIUM: ssl: potential memory leak using verifyhost
- BUILD: ssl: compilation issue with openssl v0.9.6.
- BUG/MINOR: ssl: potential memory leaks using ssl_c_key_alg or ssl_c_sig_alg.
- MINOR: ssl: optimization of verifyhost on wildcard certificates.
- BUG/MINOR: ssl: verifyhost does not match empty strings on wildcard.
- MINOR: ssl: Add statement 'verifyhost' to "server" statements
- CLEANUP: session: remove event_accept() which was not used anymore
- BUG/MINOR: deinit: free fdinfo while doing cleanup
- DOC: minor typo fix in documentation
- BUG/MEDIUM: server: set the macro for server's max weight SRV_UWGHT_MAX to SRV_UWGHT_RANGE
- BUG/MINOR: use the same check condition for server as other algorithms
- DOC: fix typo in comments
- BUG/MINOR: deinit: free server map which is allocated in init_server_map()
- CLEANUP: stream_interface: cleanup loop information in si_conn_send_loop()
- MINOR: buffer: align the last output line of buffer_dump()
- MINOR: buffer: align the last output line if there are less than 8 characters left
- DOC: stick-table: modify the description
- OPTIM: stream_interface: return directly if the connection flag CO_FL_ERROR has been set
- CLEANUP: code style: use tabs to indent codes
- DOC: checkcache: block responses with cacheable cookies
- BUG/MINOR: check_config_validity: check the returned value of stktable_init()
- MEDIUM: haproxy-systemd-wrapper: Use haproxy in same directory
- MEDIUM: systemd-wrapper: Kill child processes when interrupted
- LOW: systemd-wrapper: Write debug information to stdout
- BUG/MINOR: http: fix "set-tos" not working in certain configurations
- MEDIUM: http: add IPv6 support for "set-tos"
- DOC: ssl: update build instructions to use new SSL_* variables
- BUILD/MINOR: systemd: fix compiler warning about unused result
- url32+src - like base32+src but whole url including parameters
- BUG/MINOR: fix forcing fastinter in "on-error"
- CLEANUP: Make parameters of srv_downtime and srv_getinter const
- CLEANUP: Remove unused 'last_slowstart_change' field from struct peer
- MEDIUM: Split up struct server's check element
- MEDIUM: Move result element to struct check
- MEDIUM: Paramatise functions over the check of a server
- MEDIUM: cfgparse: Factor out check initialisation
- MEDIUM: Add state to struct check
- MEDIUM: Move health element to struct check
- MEDIUM: Add helper for task creation for checks
- MEDIUM: Add helper function for failed checks
- MEDIUM: Log agent fail, stopped or down as info
- MEDIUM: Remove option lb-agent-chk
- MEDIUM: checks: Add supplementary agent checks
- MEDIUM: Do not mark a server as down if the agent is unavailable
- MEDIUM: Set rise and fall of agent checks to 1
- MEDIUM: Add enable and disable agent unix socket commands
- MEDIUM: Add DRAIN state and report it on the stats page
- BUILD/MINOR: missing header file
- CLEANUP: regex: Create regex_comp function that compiles regex using compilation options
- CLEANUP: The function "regex_exec" needs the string length but in many case they expect null terminated char.
- MINOR: http: some exported functions were not in the header file
- MINOR: http: change url_decode to return the size of the decoded string.
- BUILD/MINOR: missing header file
- BUG/MEDIUM: sample: The function v4tov6 cannot support input and output overlap
- BUG/MINOR: arg: fix error reporting for add-header/set-header sample fetch arguments
- MINOR: sample: export the generic sample conversion parser
- MINOR: sample: export sample_casts
- MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()' into the ACL keyword
- MINOR: stick-table: use smp_expr_output_type() to retrieve the output type of a "struct sample_expr"
- MINOR: sample: provide the original sample_conv descriptor struct to the argument checker function.
- MINOR: tools: Add a function to convert buffer to an ipv6 address
- MINOR: acl: export acl arrays
- MINOR: acl: Extract the pattern parsing and indexation from the "acl_read_patterns_from_file()" function
- MINOR: acl: Extract the pattern matching function
- MINOR: sample: Define new struct sample_storage
- MEDIUM: acl: associate "struct sample_storage" to each "struct acl_pattern"
- REORG: acl/pattern: extract pattern matching from the acl file and create pattern.c
- MEDIUM: pattern: create pattern expression
- MEDIUM: pattern: rename "acl" prefix to "pat"
- MEDIUM: sample: let the cast functions set their output type
- MINOR: sample: add a private field to the struct sample_conv
- MINOR: map: Define map types
- MEDIUM: sample: add the "map" converter
- MEDIUM: http: The redirect strings follows the log format rules.
- BUG/MINOR: acl: acl parser does not recognize empty converter list
- BUG/MINOR: map: The map list was declared in the map.h file
- MINOR: map: Cleanup the initialisation of map descriptors.
- MEDIUM: map: merge identical maps
- BUG/MEDIUM: pattern: Pattern node has type of "struct pat_idx_elt" in place of "struct eb_node"
- BUG/MEDIUM: map: Bad map file parser
- CLEANUP/MINOR: standard: use the system define INET6_ADDRSTRLEN in place of MAX_IP6_LEN
- BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end
- MINOR: map: export map_get_reference() function
- MINOR: pattern: Each pattern sets the expected input type
- MEDIUM: acl: Last patch change the output type
- MEDIUM: pattern: Extract the index process from the pat_parse_*() functions
- MINOR: standard: The function parse_binary() can use preallocated buffer
- MINOR: regex: Change the struct containing regex
- MINOR: regex: Copy the original regex expression into string.
- MINOR: pattern: add support for compiling patterns for lookups
- MINOR: pattern: make the pattern matching function return a pointer to the matched element
- MINOR: map: export parse output sample functions
- MINOR: pattern: add function to lookup a specific entry in pattern list
- MINOR: pattern/map: Each pattern must free the associated sample
- MEDIUM: dumpstat: make the CLI parser understand the backslash as an escape char
- MEDIUM: map: dynamic manipulation of maps
- BUG/MEDIUM: unique_id: junk in log on empty unique_id
- BUG/MINOR: log: junk at the end of syslog packet
- MINOR: Makefile: provide cscope rule
- DOC: compression: chunk are not compressed anymore
- MEDIUM: session: disable lingering on the server when the client aborts
- BUG/MEDIUM: prevent gcc from moving empty keywords lists into BSS
- DOC: remove the comment saying that SSL certs are not checked on the server side
- BUG: counters: third counter was not stored if others unset
- BUG/MAJOR: http: don't emit the send-name-header when no server is available
- BUG/MEDIUM: http: "option checkcache" fails with the no-cache header
- BUG/MAJOR: http: sample prefetch code was not properly migrated
- BUG/MEDIUM: splicing: fix abnormal CPU usage with splicing
- BUG/MINOR: stream_interface: don't call chk_snd() on polled events
- OPTIM: splicing: use splice() for the last block when relevant
- MEDIUM: sample: handle comma-delimited converter list
- MINOR: sample: fix sample_process handling of unstable data
- CLEANUP: acl: move the 3 remaining sample fetches to samples.c
- MINOR: sample: add a new "date" fetch to return the current date
- MINOR: samples: add the http_date([<offset>]) sample converter.
- DOC: minor improvements to the part on the stats socket.
- MEDIUM: sample: systematically pass the keyword pointer to the keyword
- MINOR: payload: split smp_fetch_rdp_cookie()
- MINOR: counters: factor out smp_fetch_sc*_tracked
- MINOR: counters: provide a generic function to retrieve a stkctr for sc* and src.
- MEDIUM: counters: factor out smp_fetch_sc*_get_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_gpc0_rate
- MEDIUM: counters: factor out smp_fetch_sc*_inc_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_clr_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_conn_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_conn_rate
- MEDIUM: counters: factor out smp_fetch_sc*_conn_cur
- MEDIUM: counters: factor out smp_fetch_sc*_sess_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_sess_rate
- MEDIUM: counters: factor out smp_fetch_sc*_http_req_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_http_req_rate
- MEDIUM: counters: factor out smp_fetch_sc*_http_err_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_http_err_rate
- MEDIUM: counters: factor out smp_fetch_sc*_kbytes_in
- MEDIUM: counters: factor out smp_fetch_sc*_bytes_in_rate
- MEDIUM: counters: factor out smp_fetch_sc*_kbytes_out
- MEDIUM: counters: factor out smp_fetch_sc*_bytes_out_rate
- MEDIUM: counters: factor out smp_fetch_sc*_trackers
- MINOR: session: make the number of stick counter entries more configurable
- MEDIUM: counters: support passing the counter number as a fetch argument
- MEDIUM: counters: support looking up a key in an alternate table
- MEDIUM: cli: adjust the method for feeding frequency counters in tables
- MINOR: cli: make it possible to enter multiple values at once with "set table"
- MINOR: payload: allow the payload sample fetches to retrieve arbitrary lengths
- BUG/MINOR: cli: "clear table" must not kill entries that don't match condition
- MINOR: ssl: use MAXPATHLEN instead of PATH_MAX
- MINOR: config: warn when a server with no specific port uses rdp-cookie
- BUG/MEDIUM: unique_id: HTTP request counter must be unique!
- DOC: add a mention about the limited chunk size
- BUG/MEDIUM: fix broken send_proxy on FreeBSD
- MEDIUM: stick-tables: flush old entries upon soft-stop
- MINOR: tcp: add new "close" action for tcp-response
- MINOR: payload: provide the "res.len" fetch method
- BUILD: add SSL_INC/SSL_LIB variables to force the path to openssl
- MINOR: http: compute response time before processing headers
- BUG/MINOR: acl: fix improper string size assignment in proxy argument
- BUG/MEDIUM: http: accept full buffers on smp_prefetch_http
- BUG/MINOR: acl: implicit arguments of ACL keywords were not properly resolved
- BUG/MEDIUM: session: risk of crash on out of memory conditions
- BUG/MINOR: peers: set the accept date in outgoing connections
- BUG/MEDIUM: tcp: do not skip tracking rules on second pass
- BUG/MEDIUM: acl: do not evaluate next terms after a miss
- MINOR: acl: add a warning when an ACL keyword is used without any value
- MINOR: tcp: don't use tick_add_ifset() when timeout is known to be set
- BUG/MINOR: acl: remove patterns from the tree before freeing them
- MEDIUM: backend: add support for the wt6 hash
- OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes
- OPTIM/MINOR: mark the source address as already known on accept()
- BUG/MINOR: stats: don't count tarpitted connections twice
- CLEANUP: http: homogenize processing of denied req counter
- CLEANUP: http: merge error handling for req* and http-request *
- BUG/MEDIUM: http: fix possible parser crash when parsing erroneous "http-request redirect" rules
- BUG/MINOR: http: fix build warning introduced with url32/url32_src
- BUG/MEDIUM: checks: fix slow start regression after fix attempt
- BUG/MAJOR: server: weight calculation fails for map-based algorithms
- MINOR: stats: report correct throttling percentage for servers in slowstart
- OPTIM: connection: fold the error handling with handshake handling
- MINOR: peers: accept to learn strings of different lengths
- BUG/MAJOR: fix haproxy crash when using server tracking instead of checks
- BUG/MAJOR: check: fix haproxy crash during soft-stop/soft-start
- BUG/MINOR: stats: do not report "via" on tracking servers in maintenance
- BUG/MINOR: connection: fix typo in error message report
- BUG/MINOR: backend: fix target address retrieval in transparent mode
- BUG/MINOR: config: report the correct track-sc number in tcp-rules
- BUG/MINOR: log: fix log-format parsing errors
- DOC: add some information about how to apply converters to samples
- MINOR: acl/pattern: use types different from int to clarify who does what.
- MINOR: pattern: import acl_find_match_name() into pattern.h
- MEDIUM: stick-tables: support automatic conversion from ipv4<->ipv6
- MEDIUM: log-format: relax parsing of '%' followed by unsupported characters
- BUG/MINOR: http: usual deinit stuff in last commit
- BUILD: log: silent a warning about isblank() with latest patches
- BUG/MEDIUM: checks: fix health check regression causing them to depend on declaration order
- BUG/MEDIUM: checks: fix a long-standing issue with reporting connection errors
- BUG/MINOR: checks: don't consider errno and use conn->err_code
- BUG/MEDIUM: checks: also update the DRAIN state from the web interface
- MINOR: stats: remove some confusion between the DRAIN state and NOLB
- BUG/MINOR: tcp: check that no error is pending during a connect probe
- BUG/MINOR: connection: check EINTR when sending a PROXY header
- MEDIUM: connection: set the socket shutdown flags on socket errors
- BUG/MEDIUM: acl: fix regression introduced by latest converters support
- MINOR: connection: clear errno prior to checking for errors
- BUG/MINOR: checks: do not trust errno in write event before any syscall
- MEDIUM: checks: centralize error reporting
- OPTIM: checks: don't poll on recv when using plain TCP connects
- OPTIM: checks: avoid setting SO_LINGER twice
- MINOR: tools: add a generic binary hex string parser
- BUG/MEDIUM: checks: tcp-check: do not poll when there's nothing to send
- BUG/MEDIUM: check: tcp-check might miss some outgoing data when socket buffers are full
- BUG/MEDIUM: args: fix double free on error path in argument expression parser
- BUG/MINOR: acl: fix sample expression error reporting
- BUG/MINOR: checks: tcp-check actions are enums, not flags
- MEDIUM: checks: make tcp-check perform multiple send() at once
- BUG/MEDIUM: stick: completely remove the unused flag from the store entries
- OPTIM: ebtree: pack the struct eb_node to avoid holes on 64-bit
- BUG/MEDIUM: stick-tables: complete the latest fix about store-responses
- CLEANUP: stream_interface: remove unused field err_loc
- MEDIUM: stats: don't use conn->xprt_st anymore
- MINOR: session: add a simple function to retrieve a session from a task
- MEDIUM: stats: don't use conn->xprt_ctx anymore
- MEDIUM: peers: don't rely on conn->xprt_ctx anymore
- MINOR: http: prevent smp_fetch_url_{ip,port} from using si->conn
- MINOR: connection: make it easier to emit proxy protocol for unknown addresses
- MEDIUM: stats: prepare the HTTP stats I/O handler to support more states
- MAJOR: stats: move the HTTP stats handling to its applet
- MEDIUM: stats: move request argument processing to the final step
- MEDIUM: session: detect applets from the session by using s->target
- MAJOR: session: check for a connection to an applet in sess_prepare_conn_req()
- MAJOR: session: pass applet return traffic through the response analysers
- MEDIUM: stream-int: split the shutr/shutw functions between applet and conn
- MINOR: stream-int: make the shutr/shutw functions void
- MINOR: obj: provide a safe and an unsafe access to pointed objects
- MINOR: connection: add a field to store an object type
- MINOR: connection: always initialize conn->objt_type to OBJ_TYPE_CONN
- MEDIUM: stream interface: move the peers' ptr into the applet context
- MINOR: stream-interface: move the applet context to its own struct
- MINOR: obj: introduce a new type appctx
- MINOR: stream-int: rename ->applet to ->appctx
- MINOR: stream-int: split si_prepare_embedded into si_prepare_none and si_prepare_applet
- MINOR: stream-int: add a new pointer to the end point
- MEDIUM: stream-interface: set the pointer to the applet into the applet context
- MAJOR: stream interface: remove the ->release function pointer
- MEDIUM: stream-int: make ->end point to the connection or the appctx
- CLEANUP: stream-int: remove obsolete si_ctrl function
- MAJOR: stream-int: stop using si->conn and use si->end instead
- MEDIUM: stream-int: do not allocate a connection in parallel to applets
- MEDIUM: session: attach incoming connection to target on embryonic sessions
- MINOR: connection: add conn_init() to (re)initialize a connection
- MINOR: checks: call conn_init() to properly initialize the connection.
- MINOR: peers: make use of conn_init() to initialize the connection
- MINOR: session: use conn_init() to initialize the connections
- MINOR: http: use conn_init() to reinitialize the server connection
- MEDIUM: connection: replace conn_prepare with conn_assign
- MINOR: get rid of si_takeover_conn()
- MINOR: connection: add conn_new() / conn_free()
- MAJOR: connection: add two new flags to indicate readiness of control/transport
- MINOR: stream-interface: introduce si_reset() and si_set_state()
- MINOR: connection: reintroduce conn_prepare to set the protocol and transport
- MINOR: connection: replace conn_assign with conn_attach
- MEDIUM: stream-interface: introduce si_attach_conn to replace si_prepare_conn
- MAJOR: stream interface: dynamically allocate the outgoing connection
- MEDIUM: connection: move the send_proxy offset to the connection
- MINOR: connection: check for send_proxy during the connect(), not the SI
- MEDIUM: connection: merge the send_proxy and local_send_proxy calls
- MEDIUM: stream-int: replace occurrences of si->appctx with si_appctx()
- MEDIUM: stream-int: return the allocated appctx in stream_int_register_handler()
- MAJOR: stream-interface: dynamically allocate the applet context
- MEDIUM: session: automatically register the applet designated by the target
- MEDIUM: stats: delay appctx initialization
- CLEANUP: peers: use less confusing state/status code names
- MEDIUM: peers: delay appctx initialization
- MINOR: stats: provide some appctx information in "show sess all"
- DIET/MINOR: obj: pack the obj_type enum to 8 bits
- DIET/MINOR: connection: rearrange a few fields to save 8 bytes in the struct
- DIET/MINOR: listener: rearrange a few fields in struct listener to save 16 bytes
- DIET/MINOR: proxy: rearrange a few fields in struct proxy to save 16 bytes
- DIET/MINOR: session: reduce the struct session size by 8 bytes
- DIET/MINOR: stream-int: rearrange a few fields in struct stream_interface to save 8 bytes
- DIET/MINOR: http: reduce the size of struct http_txn by 8 bytes
- MINOR: http: switch the http state to an enum
- MINOR: http: use an enum for the auth method in http_auth_data
- DIET/MINOR: task: reduce struct task size by 8 bytes
- MINOR: stream_interface: add reporting of ressouce allocation errors
- MINOR: session: report lack of resources using the new stream-interface's error code
- BUILD: simplify the date and version retrieval in the makefile
- BUILD: prepare the makefile to skip format lines in SUBVERS and VERDATE
- BUILD: use format tags in VERDATE and SUBVERS files
- BUG/MEDIUM: channel: bo_getline() must wait for \n until buffer is full
- CLEANUP: check: server port is unsigned
- BUG/MEDIUM: checks: agent doesn't get the response if server does not closes
- MINOR: tools: buf2ip6 must not modify output on failure
- MINOR: pattern: do not assign SMP_TYPES by default to patterns
- MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors
- MINOR: arg: improve wording on error reporting
- BUG/MEDIUM: sample: simplify and fix the argument parsing
- MEDIUM: acl: fix the argument parser to let the lower layer report detailed errors
- MEDIUM: acl: fix the initialization order of the ACL expression
- CLEANUP: acl: remove useless blind copy-paste from sample converters
- TESTS: add regression tests for ACL and sample expression parsers
- BUILD: time: adapt the type of TV_ETERNITY to the local system
- MINOR: chunks: allocate the trash chunks before parsing the config
- BUILD: definitely silence some stupid GCC warnings
- MINOR: chunks: always initialize the output chunk in get_trash_chunk()
- MINOR: checks: improve handling of the servers tracking chain
- REORG: checks: retrieve the check-specific defines from server.h to checks.h
- MINOR: checks: use an enum instead of flags to report a check result
- MINOR: checks: rename the state flags
- MINOR: checks: replace state DISABLED with CONFIGURED and ENABLED
- MINOR: checks: use check->state instead of srv->state & SRV_CHECKED
- MINOR: checks: fix agent check interval computation
- MINOR: checks: add a PAUSED state for the checks
- MINOR: checks: create the agent tasks even when no check is configured
- MINOR: checks: add a flag to indicate what check is an agent
- MEDIUM: checks: enable agent checks even if health checks are disabled
- BUG/MEDIUM: checks: ensure we can enable a server after boot
- BUG/MEDIUM: checks: tracking servers must not inherit the MAINT flag
- BUG/MAJOR: session: repair tcp-request connection rules
- BUILD: fix SUBVERS extraction in the Makefile
- BUILD: pattern: silence a warning about uninitialized value
- BUILD: log: fix build warning on Solaris
- BUILD: dumpstats: fix build error on Solaris
- DOC: move option pgsql-check to the correct place
- DOC: move option tcp-check to the proper place
- MINOR: connection: add simple functions to report connection readiness
- MEDIUM: connection: centralize handling of nolinger in fd management
- OPTIM: http: set CF_READ_DONTWAIT on response message
- OPTIM: http: do not re-enable reading on client side while closing the server side
- MINOR: config: add option http-keep-alive
- MEDIUM: connection: inform si_alloc_conn() whether existing conn is OK or not
- MAJOR: stream-int: handle the connection reuse in si_connect()
- MAJOR: http: add the keep-alive transition on the server side
- MAJOR: backend: enable connection reuse
- MINOR: http: add option prefer-last-server
- MEDIUM: http: do not report connection errors for second and further requests
When the load balancing algorithm in use is not deterministic, and a previous
request was sent to a server to which haproxy still holds a connection, it is
sometimes desirable that subsequent requests on a same session go to the same
server as much as possible. Note that this is different from persistence, as
we only indicate a preference which haproxy tries to apply without any form
of warranty. The real use is for keep-alive connections sent to servers. When
this option is used, haproxy will try to reuse the same connection that is
attached to the server instead of rebalancing to another server, causing a
close of the connection. This can make sense for static file servers. It does
not make much sense to use this in combination with hashing algorithms.
This new option enables HTTP keep-alive processing on the connections.
It can be overwritten by http-server-close, httpclose and forceclose.
Right now full-chain keep-alive is not yet implemented, but we need
the option to work on it. The doc will come later.
This patch adds map manipulation commands to the socket interface.
add map <map> <key> <value>
Add the value <value> in the map <map>, at the entry corresponding to
the key <key>. This command does not verify if the entry already
exists.
clear map <map>
Remove entries from the map <map>
del map <map> <key>
Delete all the map entries corresponding to the <key> value in the map
<map>.
set map <map> <key> <value>
Modify the value corresponding to each key <key> in a map <map>. The
new value is <value>.
show map [<map>]
Dump info about map converters. Without argument, the list of all
available maps are returned. If a <map> is specified, is content is
dumped.
This patch remove the compatibility check from the input type and the
match method. Now, it checks if a casts from the input type to output
type exists and the pattern_exec_match() function apply casts before
each pattern matching.
The commit 37e340c (BUG/MEDIUM: stick: completely remove the unused flag
from the store entries) was incomplete. We also need to ensure that only
the first store-response for a table is applied and that it may coexist
with a possible store-request that was already done on this table.
This patch with the previous one should be backported to 1.4.
This is a generic health check which can be used to match a
banner or send a request and analyse a server response.
It works in a send/expect ways and many exchange can be done between
HAProxy and a server to decide the server status, making HAProxy able to
speak the server's protocol.
It can send arbitrary regular or binary strings and match content as a
regular or binary string or a regex.
Signed-off-by: Baptiste Assmann <bedis9@gmail.com>
The stickiness table can be declared in such sections as frontend, listen
and backend, but the original manual only mentioned backend. Modify the
description simply as below:
"current backend" -> "current section"
Signed-off-by: Godbach <nylzhaowei@gmail.com>
We handle "http-request redirect" with a log-format string now, but we
leave "redirect" unaffected.
Note that the control of the special "/" case is move from the runtime
execution to the configuration parsing. If the format rule list is
empty, the build_logline() function does nothing.
At the moment when a '%' character is followed by any unhandled character,
it is considered as a variable name, and if it cannot be resolved, a warning
is emitted and the configuration goes on.
When we start using log-format for redirect rules, it may happen that some
people accidently use '%' instead of '%%' without understanding the cause
of the issue. Thus we do two things here :
- if a single '%' is followed by a blank or a digit, we fix it and emit a
warning explaining how this should be done ; this ensures that existing
configs continue to work ;
- if a single '%' is followed by an unknown variable name, we report it
and explain how to emit a verbatim '%' in case this is what the user
desired.
Add a new converter with the following prototype :
map(<map_file>[,<default_value>])
map_<match_type>(<map_file>[,<default_value>])
map_<match_type>_<output_type>(<map_file>[,<default_value>])
It searches the for input value from <map_file> using the <match_type>
matching method, and return the associated value converted to the type
<output_type>. If the input value cannot be found in the <map_file>,
the converter returns the <default_value>. If the <default_value> is
not set, the converter fails and acts as if no input value could be
fetched. If the <match_type> is not set, it defaults to "str".
Likewise, if the <output_type> is not set, it defaults to "str". For
convenience, the "map" keyword is an alias for "map_str" and maps a
string to another string. The following array contains contains the
list of all the map* converters.
+----+----------+---------+-------------+------------+
| `-_ out | | | |
| input `-_ | str | int | ip |
| / match `-_ | | | |
+---------------+---------+-------------+------------+
| str / str | map_str | map_str_int | map_str_ip |
| str / sub | map_sub | map_sub_int | map_sub_ip |
| str / dir | map_dir | map_dir_int | map_dir_ip |
| str / dom | map_dom | map_dom_int | map_dom_ip |
| str / end | map_end | map_end_int | map_end_ip |
| str / reg | map_reg | map_reg_int | map_reg_ip |
| int / int | map_int | map_int_int | map_int_ip |
| ip / ip | map_ip | map_ip_int | map_ip_ip |
+---------------+---------+-------------+------------+
The names are intentionally chosen to reflect the same match methods
as ACLs use.
We've had the feature for log-format, unique-id-format and add-header for
a while now. It has just been implemented for ACLs but some doc was still
lacking.
The syntax of this new commands are:
enable agent <backend>/<server>
disable agent <backend>/<server>
These commands allow temporarily stopping and subsequently
re-starting an auxiliary agent check. The effect of this is as follows:
New checks are only initialised when the agent is in the enabled. Thus,
disable agent will prevent any new agent checks from begin initiated until
the agent re-enabled using enable agent.
When an agent is disabled the processing of an auxiliary agent check that
was initiated while the agent was set as enabled is as follows: All
results that would alter the weight, specifically "drain" or a weight
returned by the agent, are ignored. The processing of agent check is
otherwise unchanged.
The motivation for this feature is to allow the weight changing effects
of the agent checks to be paused to allow the weight of a server to be
configured using set weight without being overridden by the agent.
Signed-off-by: Simon Horman <horms@verge.net.au>
In the case where agent-port is used and the agent
check is a secondary check to not mark a server as down
if the agent becomes unavailable.
In this configuration the agent should only cause a server to be marked
as down if the agent returns "fail", "stopped" or "down".
Signed-off-by: Simon Horman <horms@verge.net.au>
Allow an auxiliary agent check to be run independently of the
regular a regular health check. This is enabled by the agent-check
server setting.
The agent-port, which specifies the TCP port to use for the agent's
connections, is required.
The agent-inter, which specifies the interval between agent checks and
timeout of agent checks, is optional. If not set the value for regular
checks is used.
e.g.
server web1_1 127.0.0.1:80 check agent-port 10000
If either the health or agent check determines that a server is down
then it is marked as being down, otherwise it is marked as being up.
An agent health check performed by opening a TCP socket and reading an
ASCII string. The string should have one of the following forms:
* An ASCII representation of an positive integer percentage.
e.g. "75%"
Values in this format will set the weight proportional to the initial
weight of a server as configured when haproxy starts.
* The string "drain".
This will cause the weight of a server to be set to 0, and thus it
will not accept any new connections other than those that are
accepted via persistence.
* The string "down", optionally followed by a description string.
Mark the server as down and log the description string as the reason.
* The string "stopped", optionally followed by a description string.
This currently has the same behaviour as "down".
* The string "fail", optionally followed by a description string.
This currently has the same behaviour as "down".
Signed-off-by: Simon Horman <horms@verge.net.au>
Remove option lb-agent-chk and thus the facility to configure
a stand-alone agent health check. This feature was added by
"MEDIUM: checks: Add agent health check". It will be replaced
by subsequent patches with a features to allow an agent check
to be run as either a secondary check, along with any of the existing
checks, or as part of an http check with the status returned
in an HTTP header.
This patch does not entirely revert "MEDIUM: checks: Add agent health
check". The infrastructure it provides to parse the results of an
agent health check remains and will be re-used by the planned features
that are mentioned above.
Signed-off-by: Simon Horman <horms@verge.net.au>
This function was designed for haproxy while testing other functions
in the past. Initially it was not planned to be used given the not
very interesting numbers it showed on real URL data : it is not as
smooth as the other ones. But later tests showed that the other ones
are extremely sensible to the server count and the type of input data,
especially DJB2 which must not be used on numeric input. So in fact
this function is still a generally average performer and it can make
sense to merge it in the end, as it can provide an alternative to
sdbm+avalanche or djb2+avalanche for consistent hashing or when hashing
on numeric data such as a source IP address or a visitor identifier in
a URL parameter.
Summary:
Avalanche is supported not as a native hashing choice, but a modifier
on the hashing function. Note that this means that possible configs
written after 1.5-dev4 using "hash-type avalanche" will get an informative
error instead. But as discussed on the mailing list it seems nobody ever
used it anyway, so let's fix it before the final 1.5 release.
The default values were selected for backward compatibility with previous
releases, as discussed on the mailing list, which means that the consistent
hashing will still apply the avalanche hash by default when no explicit
algorithm is specified.
Examples
(default) hash-type map-based
Map based hashing using sdbm without avalanche
(default) hash-type consistent
Consistent hashing using sdbm with avalanche
Additional Examples:
(a) hash-type map-based sdbm
Same as default for map-based above
(b) hash-type map-based sdbm avalanche
Map based hashing using sdbm with avalanche
(c) hash-type map-based djb2
Map based hashing using djb2 without avalanche
(d) hash-type map-based djb2 avalanche
Map based hashing using djb2 with avalanche
(e) hash-type consistent sdbm avalanche
Same as default for consistent above
(f) hash-type consistent sdbm
Consistent hashing using sdbm without avalanche
(g) hash-type consistent djb2
Consistent hashing using djb2 without avalanche
(h) hash-type consistent djb2 avalanche
Consistent hashing using djb2 with avalanche
Summary:
In testing at tumblr, we found that using djb2 hashing instead of the
default sdbm hashing resulted is better workload distribution to our backends.
This commit implements a change, that allows the user to specify the hash
function they want to use. It does not limit itself to consistent hashing
scenarios.
The supported hash functions are sdbm (default), and djb2.
For a discussion of the feature and analysis, see mailing list thread
"Consistent hashing alternative to sdbm" :
http://marc.info/?l=haproxy&m=138213693909219
Note: This change does NOT make changes to new features, for instance,
applying an avalance hashing always being performed before applying
consistent hashing.
This fetch method returns the response buffer len, similarly
to req.len for the request. Previously it was only possible
to rely on "res.payload(0,size) -m found" to find if at least
that amount of data was available, which was a bit tricky.
This new action immediately closes the connection with the server
when the condition is met. The first such rule executed ends the
rules evaluation. The main purpose of this action is to force a
connection to be finished between a client and a server after an
exchange when the application protocol expects some long time outs
to elapse first. The goal is to eliminate idle connections which
take signifiant resources on servers with certain protocols.
verifyhost allows you to specify a hostname that the remote server's
SSL certificate must match. Connections that don't match will be
closed with an SSL error.
When using req.payload and res.payload to look up for specific content at an
arbitrary location, we're often facing the problem of not knowing the input
buffer length. If the length argument is larger than the buffer length, the
function did not match, and if they're smaller, there is a risk of not getting
the expected content. This is especially true when looking for data in SOAP
requests.
So let's make some provisions for scanning the whole buffer by specifying a
length of 0 bytes. This greatly simplifies the processing of random-sized
input data.
The "set table" statement allows to create new entries with their respective
values. Till now it was limited to a single data type per line, requiring as
many "set table" statements as the desired data types to be set. Since this
is only a parser limitation, this patch gets rid of it. It also allows the
creation of a key with no data types (all reset to their default values).
sc_* sample fetches now take an optional parameter which allows to look
the key in an alternate table. This is convenient to pass multiple
information for the same key at once (eg: have multiple gpc0 for the
same key, or support being fed complementary information from the CLI).
Example :
listen front
bind :8000
tcp-request content track-sc0 src table local-ip
http-response set-header src-id %[sc0_get_gpc0]+%[sc0_get_gpc0(global-ip)]
server dummy 127.0.0.1:8001
backend local-ip
stick-table size 1k type ip store gpc0
backend global-ip
stick-table size 1k type ip store gpc0
One very annoying issue when trying to extend the sticky counters beyond
the current 3 counters is that it requires a massive copy-paste of fetch
functions (we don't have to copy-paste code anymore), just so that the
fetch names exist.
So let's have an alternate form like "sc_*(num)" to allow passing the
counter number as an argument without having to redefine new fetch names.
The MAX_SESS_STKCTR macro defines the number of usable sticky counters,
which defaults to 3.
Converts an integer supposed to contain a date since epoch to
a string representing this date in a format suitable for use
in HTTP header fields. If an offset value is specified, then
it is a number of seconds that is added to the date before the
conversion is operated. This is particularly useful to emit
Date header fields, Expires values in responses when combined
with a positive offset, or Last-Modified values when the
offset is negative.
Returns the current date as the epoch (number of seconds since 01/01/1970).
If an offset value is specified, then it is a number of seconds that is added
to the current date before returning the value. This is particularly useful
to compute relative dates, as both positive and negative offsets are allowed.
We now support having a comma-delimited converter list, which can start
right after the fetch keyword. The immediate benefit is that it allows
to use converters in log-format expressions, for example :
set-header source-net %[src,ipmask(24)]
The parser is also slightly improved and should be more resilient against
configuration errors. Also, optional arguments in converters were mistakenly
not allowed till now, so this was fixed.
The max weight of server is 256 now, but SRV_UWGHT_MAX is still 255. As a result,
FWRR will not work well when server's weight is 256. The description is as below:
There are some macros related to server's weight in include/types/server.h:
#define SRV_UWGHT_RANGE 256
#define SRV_UWGHT_MAX (SRV_UWGHT_RANGE - 1)
#define SRV_EWGHT_MAX (SRV_UWGHT_MAX * BE_WEIGHT_SCALE)
Since weight of server can be reach to 256 and BE_WEIGHT_SCALE equals to 16,
the max eweight of server should be 256*16 = 4096, it will exceed SRV_EWGHT_MAX
which equals to SRV_UWGHT_MAX*BE_WEIGHT_SCALE = 255*16 = 4080. When a server
with weight 256 is insterted into FWRR tree during initialization, the key value
of this server should be SRV_EWGHT_MAX - s->eweight = 4080 - 4096 = -16 which
is closed to UINT_MAX in unsigned type, so the server with highest weight will
be not elected as the first server to process request.
In addition, it is a better choice to compare with SRV_UWGHT_MAX than a magic
number 256 while doing check for the weight. The max number of servers for
round-robin algorithm is also updated.
Signed-off-by: Godbach <nylzhaowei@gmail.com>
s->req->prod->conn->addr.to.ss_family contains only useful data if
conn_get_to_addr() is called early. If thats not the case (nothing in the
configuration needs the destination address like logs, transparent, ...)
then "set-tos" doesn't work.
Fix this by checking s->req->prod->conn->addr.from.ss_family instead.
Also fix a minor doc issue about set-tos in http-response.
Released version 1.5-dev19 with the following main changes :
- MINOR: stats: remove the autofocus on the scope input field
- BUG/MEDIUM: Fix crt-list file parsing error: filtered name was ignored.
- BUG/MEDIUM: ssl: EDH ciphers are not usable if no DH parameters present in pem file.
- BUG/MEDIUM: shctx: makes the code independent on SSL runtime version.
- MEDIUM: ssl: improve crt-list format to support negation
- BUG: ssl: fix crt-list for clients not supporting SNI
- MINOR: stats: show soft-stopped servers in different color
- BUG/MINOR: config: "source" does not work in defaults section
- BUG: regex: fix pcre compile error when using JIT
- MINOR: ssl: add pattern fetch 'ssl_c_sha1'
- BUG: ssl: send payload gets corrupted if tune.ssl.maxrecord is used
- MINOR: show PCRE version and JIT status in -vv
- BUG/MINOR: jit: don't rely on USE flag to detect support
- DOC: readme: add suggestion to link against static openssl
- DOC: examples: provide simplified ssl configuration
- REORG: tproxy: prepare the transparent proxy defines for accepting other OSes
- MINOR: tproxy: add support for FreeBSD
- MINOR: tproxy: add support for OpenBSD
- DOC: examples: provide an example of transparent proxy configuration for FreeBSD 8
- CLEANUP: fix minor typo in error message.
- CLEANUP: fix missing include <string.h> in proto/listener.h
- CLEANUP: protect checks.h from multiple inclusions
- MINOR: compression: acl "res.comp" and fetch "res.comp_algo"
- BUG/MINOR: http: add-header/set-header did not accept the ACL condition
- BUILD: mention in the Makefile that USE_PCRE_JIT is for libpcre >= 8.32
- BUG/MEDIUM: splicing is broken since 1.5-dev12
- BUG/MAJOR: acl: add implicit arguments to the resolve list
- BUG/MINOR: tcp: fix error reporting for TCP rules
- CLEANUP: peers: remove a bit of spaghetti to prepare for the next bugfix
- MINOR: stick-table: allow to allocate an entry without filling it
- BUG/MAJOR: peers: fix an overflow when syncing strings larger than 16 bytes
- MINOR: session: only call http_send_name_header() when changing the server
- MINOR: tcp: report the erroneous word in tcp-request track*
- BUG/MAJOR: backend: consistent hash can loop forever in certain circumstances
- BUG/MEDIUM: log: fix regression on log-format handling
- MEDIUM: log: report file name, line number, and directive name with log-format errors
- BUG/MINOR: cli: "clear table" did not work anymore without a key
- BUG/MINOR: cli: "clear table xx data.xx" does not work anymore
- BUG/MAJOR: http: compression still has defects on chunked responses
- BUG/MINOR: stats: fix confirmation links on the stats interface
- BUG/MINOR: stats: the status bar does not appear anymore after a change
- BUG/MEDIUM: stats: allocate the stats frontend also on "stats bind-process"
- BUG/MEDIUM: stats: fix a regression when dealing with POST requests
- BUG/MINOR: fix unterminated ACL array in compression
- BUILD: last fix broke non-linux platforms
- MINOR: init: indicate the SSL runtime version on -vv.
- BUG/MEDIUM: compression: the deflate algorithm must use global settings as well
- BUILD: stdbool is not portable (again)
- DOC: readme: add a small reminder about restrictions to respect in the code
- MINOR: ebtree: add new eb_next_dup/eb_prev_dup() functions to visit duplicates
- BUG/MINOR: acl: fix a double free during exit when using PCRE_JIT
- DOC: fix wrong copy-paste in the rspdel example
- MINOR: counters: make it easier to extend the amount of tracked counters
- MEDIUM: counters: add support for tracking a third counter
- MEDIUM: counters: add a new "gpc0_rate" counter in stick-tables
- BUG/MAJOR: http: always ensure response buffer has some room for a response
- MINOR: counters: add fetch/acl sc*_tracked to indicate whether a counter is tracked
- MINOR: defaults: allow REQURI_LEN and CAPTURE_LEN to be redefined
- MINOR: log: add a new flag 'L' for locally processed requests
- MINOR: http: add full-length header fetch methods
- MEDIUM: protocol: implement a "drain" function in protocol layers
- MEDIUM: http: add a new "http-response" ruleset
- MEDIUM: http: add the "set-nice" action to http-request and http-response
- MEDIUM: log: add a log level override value in struct session
- MEDIUM: http: add support for action "set-log-level" in http-request/http-response
- MEDIUM: http: add support for "set-tos" in http-request/http-response
- MEDIUM: http: add the "set-mark" action on http-request/http-response rules
- MEDIUM: tcp: add "tcp-request connection expect-proxy layer4"
- MEDIUM: acl: automatically detect the type of certain fetches
- MEDIUM: acl: remove a lot of useless ACLs that are equivalent to their fetches
- MEDIUM: acl: remove 15 additional useless ACLs that are equivalent to their fetches
- DOC: major reorg of ACL + sample fetch
- CLEANUP: http: remove the bogus urlp_ip ACL match
- MINOR: acl: add the new "env()" fetch method to retrieve an environment variable
- BUG/MINOR: acl: correctly consider boolean fetches when doing casts
- BUG/CRITICAL: fix a possible crash when using negative header occurrences
- DOC: update ROADMAP file
- MEDIUM: counters: use sc0/sc1/sc2 instead of sc1/sc2/sc3
- MEDIUM: stats: add proxy name filtering on the statistic page
It was a bit inconsistent to have gpc start at 0 and sc start at 1,
so make sc start at zero like gpc. No previous release was issued
with sc3 anyway, so no existing setup should be affected.
This is useful in order to take different actions across restarts without
touching the configuration (eg: soft-stop), or to pass some information
such as the local host name to the next hop.
The split between ACL and sample fetch was a terrible mess in the doc,
as it caused all entries to be duplicated with most of them not easy to
find, some missing and some wrong.
The new approach consists in describing the sample fetch methods and
indicating the ACLs that are derived from these fetches. The doc is
much smaller (1500 lines added, 2200 removed, net gain = 700 lines)
and much clearer.
The description of the ACL mechanics was revamped to take account of
the latest evolutions and clearly describe the compatibility between
types of fetches and ACL patterns.
The deprecated keywords have been marked as such, though they still
appear in the examples given for various other keywords.
This configures the client-facing connection to receive a PROXY protocol
header before any byte is read from the socket. This is equivalent to
having the "accept-proxy" keyword on the "bind" line, except that using
the TCP rule allows the PROXY protocol to be accepted only for certain
IP address ranges using an ACL. This is convenient when multiple layers
of load balancers are passed through by traffic coming from public
hosts.
"set-mark" is used to set the Netfilter MARK on all packets sent to the
client to the value passed in <mark> on platforms which support it. This
value is an unsigned 32 bit value which can be matched by netfilter and
by the routing table. It can be expressed both in decimal or hexadecimal
format (prefixed by "0x"). This can be useful to force certain packets to
take a different route (for example a cheaper network path for bulk
downloads). This works on Linux kernels 2.6.32 and above and requires
admin privileges.
This manipulates the TOS field of the IP header of outgoing packets sent
to the client. This can be used to set a specific DSCP traffic class based
on some request or response information. See RFC2474, 2597, 3260 and 4594
for more information.
Some users want to disable logging for certain non-important requests such as
stats requests or health-checks coming from another equipment. Other users want
to log with a higher importance (eg: notice) some special traffic (POST requests,
authenticated requests, requests coming from suspicious IPs) or some abnormally
large responses.
This patch responds to all these needs at once by adding a "set-log-level" action
to http-request/http-response. The 8 syslog levels are supported, as well as "silent"
to disable logging.
Some actions were clearly missing to process response headers. This
patch adds a new "http-response" ruleset which provides the following
actions :
- allow : stop evaluating http-response rules
- deny : stop and reject the response with a 502
- add-header : add a header in log-format mode
- set-header : set a header in log-format mode
The req.hdr and res.hdr fetch methods do not work well on headers which
are allowed to contain commas, such as User-Agent, Date or Expires.
More specifically, full-length matching is impossible if a comma is
present.
This patch introduces 4 new fetch functions which are designed to work
with these full-length headers :
- req.fhdr, req.fhdr_cnt
- res.fhdr, res.fhdr_cnt
These ones do not stop at commas and permit to return full-length header
values.
People who use "option dontlog-normal" are bothered with redirects and
stats being logged and reported as errors in the logs ("PR" = proxy
blocked the request).
This patch introduces a new flag 'L' for when a request is locally
processed, that is not considered as an error by the log filters. That
way we know a request was intercepted and processed by haproxy without
logging the line when "option dontlog-normal" is in effect.
We're often missin a third counter to track base, src and base+src at
the same time. Here we introduce track_sc3 to have this third counter.
It would be wise not to add much more counters because that slightly
increases the session size and processing time though the real issue
is more the declaration of the keywords in the code and in the doc.
This new pattern fetch returns the client certificate's SHA-1 fingerprint
(i.e. SHA-1 hash of DER-encoded certificate) in a binary chunk.
This can be useful to pass it to a server in a header or to stick a client
to a server across multiple SSL connections.
Improve the crt-list file format to allow a rule to negate a certain SNI :
<crtfile> [[!]<snifilter> ...]
This can be useful when a domain supports a wildcard but you don't want to
deliver the wildcard cert for certain specific domains.
Implements the "res.comp" ACL which is a boolean returning 1 when a
response has been compressed by HAProxy or 0 otherwise.
Implements the "res.comp_algo" fetch which contains the name of the
algorithm HAProxy used to compress the response.
Released version 1.5-dev18 with the following main changes :
- DOCS: Add explanation of intermediate certs to crt paramater
- DOC: typo and minor fixes in compression paragraph
- MINOR: config: http-request configuration error message misses new keywords
- DOC: minor typo fix in documentation
- BUG/MEDIUM: ssl: ECDHE ciphers not usable without named curve configured.
- MEDIUM: ssl: add bind-option "strict-sni"
- MEDIUM: ssl: add mapping from SNI to cert file using "crt-list"
- MEDIUM: regex: Use PCRE JIT in acl
- DOC: simplify bind option "interface" explanation
- DOC: tfo: bump required kernel to linux-3.7
- BUILD: add explicit support for TFO with USE_TFO
- MEDIUM: New cli option -Ds for systemd compatibility
- MEDIUM: add haproxy-systemd-wrapper
- MEDIUM: add systemd service
- BUG/MEDIUM: systemd-wrapper: don't leak zombie processes
- BUG/MEDIUM: remove supplementary groups when changing gid
- BUG/MEDIUM: config: fix parser crash with bad bind or server address
- BUG/MINOR: Correct logic in cut_crlf()
- CLEANUP: checks: Make desc argument to set_server_check_status const
- CLEANUP: dumpstats: Make cli_release_handler() static
- MEDIUM: server: Break out set weight processing code
- MEDIUM: server: Allow relative weights greater than 100%
- MEDIUM: server: Tighten up parsing of weight string
- MEDIUM: checks: Add agent health check
- BUG/MEDIUM: ssl: openssl 0.9.8 doesn't open /dev/random before chroot
- BUG/MINOR: time: frequency counters are not totally accurate
- BUG/MINOR: http: don't process abortonclose when request was sent
- BUG/MEDIUM: stream_interface: don't close outgoing connections on shutw()
- BUG/MEDIUM: checks: ignore late resets after valid responses
- DOC: fix bogus recommendation on usage of gpc0 counter
- BUG/MINOR: http-compression: lookup Cache-Control in the response, not the request
- MINOR: signal: don't block SIGPROF by default
- OPTIM: epoll: make use of EPOLLRDHUP
- OPTIM: splice: detect shutdowns and avoid splice() == 0
- OPTIM: splice: assume by default that splice is working correctly
- BUG/MINOR: log: temporary fix for lost SSL info in some situations
- BUG/MEDIUM: peers: only the last peers section was used by tables
- BUG/MEDIUM: config: verbosely reject peers sections with multiple local peers
- BUG/MINOR: epoll: use a fix maxevents argument in epoll_wait()
- BUG/MINOR: config: fix improper check for failed memory alloc in ACL parser
- BUG/MINOR: config: free peer's address when exiting upon parsing error
- BUG/MINOR: config: check the proper variable when parsing log minlvl
- BUG/MEDIUM: checks: ensure the health_status is always within bounds
- BUG/MINOR: cli: show sess should always validate s->listener
- BUG/MINOR: log: improper NULL return check on utoa_pad()
- CLEANUP: http: remove a useless null check
- CLEANUP: tcp/unix: remove useless NULL check in {tcp,unix}_bind_listener()
- BUG/MEDIUM: signal: signal handler does not properly check for signal bounds
- BUG/MEDIUM: tools: off-by-one in quote_arg()
- BUG/MEDIUM: uri_auth: missing NULL check and memory leak on memory shortage
- BUG/MINOR: unix: remove the 'level' field from the ux struct
- CLEANUP: http: don't try to deinitialize http compression if it fails before init
- CLEANUP: config: slowstart is never negative
- CLEANUP: config: maxcompcpuusage is never negative
- BUG/MEDIUM: log: emit '-' for empty fields again
- BUG/MEDIUM: checks: fix a race condition between checks and observe layer7
- BUILD: fix a warning emitted by isblank() on non-c99 compilers
- BUILD: improve the makefile's support for libpcre
- MEDIUM: halog: add support for counting per source address (-ic)
- MEDIUM: tools: make str2sa_range support all address syntaxes
- MEDIUM: config: make use of str2sa_range() instead of str2sa()
- MEDIUM: config: use str2sa_range() to parse server addresses
- MEDIUM: config: use str2sa_range() to parse peers addresses
- MINOR: tests: add a config file to ease address parsing tests.
- MINOR: ssl: add a global tunable for the max SSL/TLS record size
- BUG/MINOR: syscall: fix NR_accept4 system call on sparc/linux
- BUILD/MINOR: syscall: add definition of NR_accept4 for ARM
- MINOR: config: report missing peers section name
- BUG/MEDIUM: tools: fix bad character handling in str2sa_range()
- BUG/MEDIUM: stats: never apply "unix-bind prefix" to the global stats socket
- MINOR: tools: prepare str2sa_range() to return an error message
- BUG/MEDIUM: checks: don't call connect() on unsupported address families
- MINOR: tools: prepare str2sa_range() to accept a prefix
- MEDIUM: tools: make str2sa_range() parse unix addresses too
- MEDIUM: config: make str2listener() use str2sa_range() to parse unix addresses
- MEDIUM: config: use a single str2sa_range() call to parse bind addresses
- MEDIUM: config: use str2sa_range() to parse log addresses
- CLEANUP: tools: remove str2sun() which is not used anymore.
- MEDIUM: config: add complete support for str2sa_range() in dispatch
- MEDIUM: config: add complete support for str2sa_range() in server addr
- MEDIUM: config: add complete support for str2sa_range() in 'server'
- MEDIUM: config: add complete support for str2sa_range() in 'peer'
- MEDIUM: config: add complete support for str2sa_range() in 'source' and 'usesrc'
- CLEANUP: minor cleanup in str2sa_range() and str2ip()
- CLEANUP: config: do not use multiple errmsg at once
- MEDIUM: tools: support specifying explicit address families in str2sa_range()
- MAJOR: listener: support inheriting a listening fd from the parent
- MAJOR: tools: support environment variables in addresses
- BUG/MEDIUM: http: add-header should not emit "-" for empty fields
- BUG/MEDIUM: config: ACL compatibility check on "redirect" was wrong
- BUG/MEDIUM: http: fix another issue caused by http-send-name-header
- DOC: mention the new HTTP 307 and 308 redirect statues
- MEDIUM: poll: do not use FD_* macros anymore
- BUG/MAJOR: ev_select: disable the select() poller if maxsock > FD_SETSIZE
- BUG/MINOR: acl: ssl_fc_{alg,use}_keysize must parse integers, not strings
- BUG/MINOR: acl: ssl_c_used, ssl_fc{,_has_crt,_has_sni} take no pattern
- BUILD: fix usual isdigit() warning on solaris
- BUG/MEDIUM: tools: vsnprintf() is not always reliable on Solaris
- OPTIM: buffer: remove one jump in buffer_count()
- OPTIM: http: improve branching in chunk size parser
- OPTIM: http: optimize the response forward state machine
- BUILD: enable poll() by default in the makefile
- BUILD: add explicit support for Mac OS/X
- BUG/MAJOR: http: use a static storage for sample fetch context
- BUG/MEDIUM: ssl: improve error processing and reporting in ssl_sock_load_cert_list_file()
- BUG/MAJOR: http: fix regression introduced by commit a890d072
- BUG/MAJOR: http: fix regression introduced by commit d655ffe
- BUG/CRITICAL: using HTTP information in tcp-request content may crash the process
- MEDIUM: acl: remove flag ACL_MAY_LOOKUP which is improperly used
- MEDIUM: samples: use new flags to describe compatibility between fetches and their usages
- MINOR: log: indicate it when some unreliable sample fetches are logged
- MEDIUM: samples: move payload-based fetches and ACLs to their own file
- MINOR: backend: rename sample fetch functions and declare the sample keywords
- MINOR: frontend: rename sample fetch functions and declare the sample keywords
- MINOR: listener: rename sample fetch functions and declare the sample keywords
- MEDIUM: http: unify acl and sample fetch functions
- MINOR: session: rename sample fetch functions and declare the sample keywords
- MAJOR: acl: make all ACLs reference the fetch function via a sample.
- MAJOR: acl: remove the arg_mask from the ACL definition and use the sample fetch's
- MAJOR: acl: remove fetch argument validation from the ACL struct
- MINOR: http: add new direction-explicit sample fetches for headers and cookies
- MINOR: payload: add new direction-explicit sample fetches
- CLEANUP: acl: remove ACL hooks which were never used
- MEDIUM: proxy: remove acl_requires and just keep a flag "http_needed"
- MINOR: sample: provide a function to report the name of a sample check point
- MAJOR: acl: convert all ACL requires to SMP use+val instead of ->requires
- CLEANUP: acl: remove unused references to ACL_USE_*
- MINOR: http: replace acl_parse_ver with acl_parse_str
- MEDIUM: acl: move the ->parse, ->match and ->smp fields to acl_expr
- MAJOR: acl: add option -m to change the pattern matching method
- MINOR: acl: remove the use_count in acl keywords
- MEDIUM: acl: have a pointer to the keyword name in acl_expr
- MEDIUM: acl: support using sample fetches directly in ACLs
- MEDIUM: http: remove val_usr() to validate user_lists
- MAJOR: sample: maintain a per-proxy list of the fetch args to resolve
- MINOR: ssl: add support for the "alpn" bind keyword
- MINOR: http: status code 303 is HTTP/1.1 only
- MEDIUM: http: implement redirect 307 and 308
- MINOR: http: status 301 should not be marked non-cacheable
The ALPN extension is meant to replace the now deprecated NPN extension.
This patch implements support for it. It requires a version of openssl
with support for this extension. Patches are available here right now :
http://html5labs.interopbridges.com/media/167447/alpn_patches.zip
Now it becomes possible to directly use sample fetches as the ACL fetch
methods. In this case, the matching method is mandatory. This allows to
form more ACL combinations from existing fetches and will limit the need
for new ACLs when everything is available to form them from sample fetches
and matches.
ACL expressions now support "-m" in addition to "-i" and "-f". This new
option is followed by the name of the pattern matching method to be used
on the extracted pattern. This makes it possible to reuse existing sample
fetch methods with other matching methods (eg: regex). A "found" matching
method ignores any pattern and only verifies that the required sample was
found (useful for cookies).
Similarly to previous commit fixing "hdr" and "cookie" in HTTP, we have to deal
with "payload" and "payload_lv" which are request-only for ACLs and req/resp for
sample fetches depending on the context, and to a less extent with other req_*
and rep_*/rep_* fetches. So let's add explicit "req." and "res." variants and
make the ACLs rely on that instead.
Since "hdr" and "cookie" were ambiguously referring to the request or response
depending on the context, we need a way to explicitly specify the direction.
By prefixing the fetches names with "req." and "res.", we can now restrict such
fetches to the appropriate direction. At the moment the fetches are explicitly
declared by later we might think about having an automatic match when "req." or
"res." appears. These explicit fetches are now used by the relevant ACLs.
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
cook, cook_cnt, cook_val, hdr_cnt, hdr_ip, hdr_val, http_auth,
http_auth_group, http_first_req, method, req_proto_http, req_ver,
resp_ver, scook, scook_cnt, scook_val, shdr, shdr_cnt, shdr_ip,
shdr_val, status, urlp, urlp_val,
Most of them won't bring much benefit at the moment, or are even aliases of
existing ones, however they'll be needed for ACL->SMP convergence.
A new val_usr() function was added to resolve userlist names into pointers.
The http_auth_group ACL forgot to make its first argument mandatory, so
there was a check in cfgparse to report a vague error. Now that args are
correctly parsed, let's report something more precise.
All urlp* ACLs now support an optional 3rd argument like their sample
counter-part which is the optional delimiter.
The fetch functions have been renamed "smp_fetch_*".
Some args controls on the sample keywords have been relaxed so that we
can soon use them for ACLs :
- cookie now accepts to have an optional name ; it will return the
first matching cookie if the name is not set ;
- same for set-cookie and hdr
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
dst_conn, so_id,
The fetch functions have been renamed "smp_fetch_*".
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
fe_conn, fe_id, fe_sess_rate
The fetch functions have been renamed "smp_fetch_*".
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
avg_queue, be_conn, be_id, be_sess_rate, connslots, nbsrv,
queue, srv_conn, srv_id, srv_is_up, srv_sess_rate
The fetch functions have been renamed "smp_fetch_*".
The file acl.c is a real mess, it both contains functions to parse and
process ACLs, and some sample extraction functions which act on buffers.
Some other payload analysers were arbitrarily dispatched to proto_tcp.c.
So now we're moving all payload-based fetches and ACLs to payload.c
which is capable of extracting data from buffers and rely on everything
that is protocol-independant. That way we can safely inflate this file
and only use the other ones when some fetches are really specific (eg:
HTTP, SSL, ...).
As a result of this cleanup, the following new sample fetches became
available even if they're not really useful :
always_false, always_true, rep_ssl_hello_type, rdp_cookie_cnt,
req_len, req_ssl_hello_type, req_ssl_sni, req_ssl_ver, wait_end
The function 'acl_fetch_nothing' was wrong and never used anywhere so it
was removed.
The "rdp_cookie" sample fetch used to have a mandatory argument while it
was optional in ACLs, which are supposed to iterate over RDP cookies. So
we're making it optional as a fetch too, and it will return the first one.
TCP Fast Open is supported in server mode since Linux 3.7, but current
libc's don't define TCP_FASTOPEN=23. Introduce the new USE flag USE_TFO
to define it manually in compat.h. Also note this in the TFO related
documentation.
fe61656b added the ability to load a list of certificates from a file,
but error control was incomplete and misleading, as some errors such
as missing files were not reported, and errors reported with Alert()
instead of memprintf() were inappropriate and mixed with upper errors.
Also, the code really supports a single SNI filter right now, so let's
correct it and the doc for that, leaving room for later change if needed.
It designates a list of PEM file with an optional list of SNI filter
per certificate, with the following format for each line :
<crtfile>[ <snifilter>]*
Wildcards are supported in the SNI filter. The certificates will be
presented to clients who provide a valid TLS Server Name Indication
field matching one of SNI filter. If no SNI filter is specified the
CN and alt subjects are used.
This change makes the "crt" block of the documentation easier to use
for those not clear on what needs to go in what file, specifically for
those using CAs that require intermediate certificates.
Now that all addresses are parsed using str2sa_range(), it becomes easy
to add support for environment variables and use them everywhere an address
is needed. Environment variables are used as $VAR or ${VAR} as in shell.
Any number of variables may compose an address, allowing various fantasies
such as "fd@${FD_HTTP}" or "${LAN_DC1}.1:80".
These ones are usable in logs, bind, servers, peers, stats socket, source,
dispatch, and check address.
Using the address syntax "fd@<num>", a listener may inherit a file
descriptor that the caller process has already bound and passed as
this number. The fd's socket family is detected using getsockname(),
and the usual initialization is performed through the existing code
for that family, but the socket creation is skipped.
Whether the parent has performed the listen() call or not is not
important as this is detected.
For UNIX sockets, we immediately clear the path after preparing a
socket so that we never remove it in case an abort would happen due
to a late error during startup.
This change allows one to force the address family in any address parsed
by str2sa_range() by specifying it as a prefix followed by '@' then the
address. Currently supported address prefixes are 'ipv4@', 'ipv6@', 'unix@'.
This also helps forcing resolving for host names (when getaddrinfo is used),
and force the family of the empty address (eg: 'ipv4@' = 0.0.0.0 while
'ipv6@' = ::).
The main benefits is that unix sockets can now get a local name without
being forced to begin with a slash. This is useful during development as
it is no longer necessary to have stats socket sent to /tmp.
Add new tunable "tune.ssl.maxrecord".
Over SSL/TLS, the client can decipher the data only once it has received
a full record. With large records, it means that clients might have to
download up to 16kB of data before starting to process them. Limiting the
record size can improve page load times on browsers located over high
latency or low bandwidth networks. It is suggested to find optimal values
which fit into 1 or 2 TCP segments (generally 1448 bytes over Ethernet
with TCP timestamps enabled, or 1460 when timestamps are disabled), keeping
in mind that SSL/TLS add some overhead. Typical values of 1419 and 2859
gave good results during tests. Use "strace -e trace=write" to find the
best value.
This trick was first suggested by Mike Belshe :
http://www.belshe.com/2010/12/17/performance-and-the-tls-record-size/
Then requested again by Ilya Grigorik who provides some hints here :
http://ofps.oreilly.com/titles/9781449344764/_transport_layer_security_tls.html#ch04_00000101
Support for server side TFO was actually introduced in linux-3.7,
linux-3.6 just has client support.
This patch fixes documentation and a code comment about the
kernel requirement. It also fixes a wrong tfo related code
comment in src/proto_tcp.c.
Support a agent health check performed by opening a TCP socket to a
pre-defined port and reading an ASCII string. The string should have one of
the following forms:
* An ASCII representation of an positive integer percentage.
e.g. "75%"
Values in this format will set the weight proportional to the initial
weight of a server as configured when haproxy starts.
* The string "drain".
This will cause the weight of a server to be set to 0, and thus it will
not accept any new connections other than those that are accepted via
persistence.
* The string "down", optionally followed by a description string.
Mark the server as down and log the description string as the reason.
* The string "stopped", optionally followed by a description string.
This currently has the same behaviour as down (iii).
* The string "fail", optionally followed by a description string.
This currently has the same behaviour as down (iii).
A agent health check may be configured using "option lb-agent-chk".
The use of an alternate check-port, used to obtain agent heath check
information described above as opposed to the port of the service,
may be useful in conjunction with this option.
e.g.
option lb-agent-chk
server http1_1 10.0.0.10:80 check port 10000 weight 100
Signed-off-by: Simon Horman <horms@verge.net.au>
Allow relative weights greater than 100%,
capping the absolute value to 256 which is
the largest supported absolute weight.
Signed-off-by: Simon Horman <horms@verge.net.au>
The current documentation of the bind option "interface" can be misleading
(as seen on the ML recently).
This patch tries to address misunderstandings by :
- avoiding the words listen or bind in the behavior description, using
"restrict to interface" instead
- using a different sentence construction (partially stolen from
"man 7 socket": SO_BINDTODEVICE)
- "defragmentation": moving behavior related explanations to the beginning
and restrictions, use-cases and requirements to the end.
This new option ensures that there is no possible fallback to a default
certificate if the client does not provide an SNI which is explicitly
handled by a certificate.
Without it, haproxy will retain the group membership of root, which may
give more access than intended to the process. For example, haproxy would
still be in the wheel group on Fedora 18, as seen with :
# haproxy -f /etc/haproxy/haproxy.cfg
# ps a -o pid,user,group,command | grep hapr
3545 haproxy haproxy haproxy -f /etc/haproxy/haproxy.cfg
4356 root root grep --color=auto hapr
# grep Group /proc/3545/status
Groups: 0 1 2 3 4 6 10
# getent group wheel
wheel❌10:root,misc
[WT: The issue has been investigated by independent security research team
and realized by itself not being able to allow security exploitation.
Additionally, dropping groups is not allowed to unprivileged users,
though this mode of deployment is quite common. Thus a warning is
emitted in this case to inform the user. The fix could be backported
into all supported versions as the issue has always been there. ]
The doc pretends that src_inc_gpc0 may be used alone without an integer
match, but this is false and has always been since its introduction in
1.5-dev1. If the ACL is called, the increment will be used, the value
returned, but it will be matched against no value so the resulting ACL
will never be true and the condition will not be met.
This means that the following config :
acl abuser src -f abusers.lst
acl blacklist src_inc_gpc0
tcp-request connection reject if abuser blacklist
Will never reject the connection and must be fixed this way :
acl abuser src -f abusers.lst
acl blacklist src_inc_gpc0 gt 0
tcp-request connection reject if abuser blacklist
Note that clr_gpc0 is trickier, as it returns the previous value which
might also be zero. Thus it's suggested to compare it against any positive
value including zero :
tcp-request connection accept if { src_clr_gpc0 ge 0 }
Some arguments were missing on the sc1/sc2 forms of most ACLs including
gpc0, so this has been fixed too.
Released version 1.5-dev17 with the following main changes :
- MINOR: ssl: Setting global tune.ssl.cachesize value to 0 disables SSL session cache.
- BUG/MEDIUM: stats: fix stats page regression introduced by commit 20b0de5
- BUG/MINOR: stats: last fix was still wrong
- BUG/MINOR: stats: http-request rules still don't cope with stats
- BUG/MINOR: http: http-request add-header emits a corrupted header
- BUG/MEDIUM: stats: disable request analyser when processing POST or HEAD
- BUG/MINOR: log: make log-format, unique-id-format and add-header more independant
- BUILD: log: unused variable svid
- CLEANUP: http: rename the misleading http_check_access_rule
- MINOR: http: move redirect rule processing to its own function
- REORG: config: move the http redirect rule parser to proto_http.c
- MEDIUM: http: add support for "http-request redirect" rules
- MEDIUM: http: add support for "http-request tarpit" rule
The "reqtarpit" rule is not very handy to use. Now that we have more
flexibility with "http-request", let's finally make the tarpit rules
usable there.
There are still semantical differences between apply_filters_to_request()
and http_req_get_intercept_rule() because the former updates the counters
while the latter does not. So we currently have almost similar code leafs
for similar conditions, but this should be cleaned up later.
These are exactly the same as the classic redirect rules except
that they can be interleaved with other http-request rules for
more flexibility.
The redirect parser should probably be changed to stop at the condition
so that the caller puts its own condition pointer. At the moment, the
redirect rule and condition are parsed at once by build_redirect_rule()
and the condition is assigned to the http_req_rule.
Released version 1.5-dev16 with the following main changes :
- BUG/MEDIUM: ssl: Prevent ssl error from affecting other connections.
- BUG/MINOR: ssl: error is not reported if it occurs simultaneously with peer close detection.
- MINOR: ssl: add fetch and acl "ssl_c_used" to check if current SSL session uses a client certificate.
- MINOR: contrib: make the iprange tool grep for addresses
- CLEANUP: polling: gcc doesn't always optimize constants away
- OPTIM: poll: optimize fd management functions for low register count CPUs
- CLEANUP: poll: remove a useless double-check on fdtab[fd].owner
- OPTIM: epoll: use a temp variable for intermediary flag computations
- OPTIM: epoll: current fd does not count as a new one
- BUG/MINOR: poll: the I/O handler was called twice for polled I/Os
- MINOR: http: make resp_ver and status ACLs check for the presence of a response
- BUG/MEDIUM: stream-interface: fix possible stalls during transfers
- BUG/MINOR: stream_interface: don't return when the fd is already set
- BUG/MEDIUM: connection: always update connection flags prior to computing polling
- CLEANUP: buffer: use buffer_empty() instead of buffer_len()==0
- BUG/MAJOR: stream_interface: fix occasional data transfer freezes
- BUG/MEDIUM: stream_interface: fix another case where the reader might not be woken up
- BUG/MINOR: http: don't abort client connection on premature responses
- BUILD: no need to clean up when making git-tar
- MINOR: log: add a tag for amount of bytes uploaded from client to server
- BUG/MEDIUM: log: fix possible segfault during config parsing
- MEDIUM: log: change a few log tokens to make them easier to remember
- BUG/MINOR: log: add_to_logformat_list() used the wrong constants
- MEDIUM: log-format: make the format parser more robust and more extensible
- MINOR: sample: support cast from bool to string
- MINOR: samples: add a function to fetch and convert any sample to a string
- MINOR: log: add lf_text_len
- MEDIUM: log: add the ability to include samples in logs
- REORG: stats: massive code reorg and cleanup
- REORG: stats: move the HTTP header injection to proto_http
- REORG: stats: functions are now HTTP/CLI agnostic
- BUG/MINOR: log: fix regression introduced by commit 8a3f52
- MINOR: chunks: centralize the trash chunk allocation
- MEDIUM: stats: use hover boxes instead of title to report details
- MEDIUM: stats: use multi-line tips to display detailed counters
- MINOR: tools: simplify the use of the int to ascii macros
- MINOR: stats: replace STAT_FMT_CSV with STAT_FMT_HTML
- MINOR: http: prepare to support more http-request actions
- MINOR: log: make parse_logformat_string() take a const char *
- MEDIUM: http: add http-request 'add-header' and 'set-header' to build headers
These two new statements allow to pass information extracted from the request
to the server. It's particularly useful for passing SSL information to the
server, but may be used for various other purposes such as combining headers
together to emulate internal variables.
Using %[expression] it becomes possible to make the log engine fetch
some samples from the request or the response and provide them in the
logs. Note that this feature is still limited, it does not yet allow
to apply converters, to limit the output length, nor to specify the
direction which should be fetched when a fetch function works in both
directions.
However it's quite convenient to log SSL information or to include some
information that are used in stick tables.
It is worth noting that this has been done in the generic log format
handler, which means that the same information may be used to build the
unique-id header and to pass the information to a backend server.
Some log tokens have evolved in a way that is not completely logical.
For example, frontend tokens sometimes begin with an 'f' and sometimes
with an 'F'. Same for backend and server.
So let's change a few cases without disrupting compatibility with existing
setups :
Bi => bi
Bp => bp
Ci => ci
Cp => cp
Fi => fi
Fp => fp
Si => si
Sp => sp
cc => CC
cs => CS
st => ST
The old ones are still supported but deprecated and will be unsupported by
the 1.5 release. However, a warning message is emitted when they're encounterd
and it indicates what token should be used to replace them.
Released version 1.5-dev15 with the following main changes :
- DOC: add a few precisions on compression
- BUG/MEDIUM: ssl: Fix handshake failure on session resumption with client cert.
- BUG/MINOR: ssl: One free session in cache remains unused.
- BUG/MEDIUM: ssl: first outgoing connection would fail with {ca,crt}-ignore-err
- MEDIUM: ssl: manage shared cache by blocks for huge sessions.
- MINOR: acl: add fetch for server session rate
- BUG/MINOR: compression: Content-Type is case insensitive
- MINOR: compression: disable on multipart or status != 200
- BUG/MINOR: http: don't report client aborts as server errors
- MINOR: stats: compute the ratio of compressed response based on 2xx responses
- MINOR: http: factor out the content-type checks
- BUG/MAJOR: stats: correctly check for a possible divide error when showing compression ratios
- BUILD: ssl: OpenSSL 0.9.6 has no renegociation
- BUG/MINOR: http: disable compression when message has no body
- MINOR: compression: make the stats a bit more robust
- BUG/MEDIUM: comp: DEFAULT_MAXZLIBMEM was expressed in bytes and not megabytes
- MINOR: connection: don't remove failed handshake flags
- MEDIUM: connection: add an error code in connections
- MEDIUM: connection: add minimal error reporting in logs for incomplete connections
- MEDIUM: connection: add error reporting for the PROXY protocol header
- MEDIUM: connection: add error reporting for the SSL
- DOC: document the connection error format in logs
- BUG/MINOR: http: don't log a 503 on client errors while waiting for requests
- BUILD: stdbool is not portable
- BUILD: ssl: NAME_MAX is not portable, use MAXPATHLEN instead
- BUG/MAJOR: raw_sock: must check error code on hangup
- BUG/MAJOR: polling: do not set speculative events on ERR nor HUP
- BUG/MEDIUM: session: fix FD leak when transport layer logging is enabled
- MINOR: stats: add a few more information on session dump
- BUG/MINOR: tcp: set the ADDR_TO_SET flag on outgoing connections
- CLEANUP: connection: remove unused server/proxy/task/si_applet declarations
- BUG/MEDIUM: tcp: process could theorically crash on lack of source ports
- MINOR: cfgparse: mention "interface" in the list of allowed "source" options
- MEDIUM: connection: introduce "struct conn_src" for servers and proxies
- CLEANUP: proto_tcp: use the same code to bind servers and backends
- CLEANUP: backend: use the same tproxy address selection code for servers and backends
- BUG/MEDIUM: stick-tables: conversions to strings were broken in dev13
- MEDIUM: proto_tcp: add support for tracking L7 information
- MEDIUM: counters: add sc1_trackers/sc2_trackers
- MINOR: http: add the "base32" pattern fetch function
- MINOR: http: add the "base32+src" fetch method.
- CLEANUP: session: use an array for the stick counters
- BUG/MINOR: proto_tcp: fix parsing of "table" in track-sc1/2
- BUG/MINOR: proto_tcp: bidirectional fetches not supported anymore in track-sc1/2
- BUG/MAJOR: connection: always recompute polling status upon I/O
- BUG/MINOR: connection: remove a few synchronous calls to polling updates
- MINOR: config: improve error checking on TCP stick-table tracking
- DOC: add some clarifications to the readme
This returns the concatenation of the base32 fetch and the src fetch.
The resulting type is of type binary, with a size of 8 or 20 bytes
depending on the source address family. This can be used to track
per-IP, per-URL counters.
This returns a 32-bit hash of the value returned by the "base"
fetch method above. This is useful to track per-URL activity on
high traffic sites without having to store all URLs. Instead a
shorter hash is stored, saving a lot of memory. The output type
is an unsigned integer.
Returns the current amount of concurrent connections tracking the same
tracked counters. This number is automatically incremented when tracking
begins and decremented when tracking stops. It differs from sc1_conn_cur in
that it does not rely on any stored information but on the table's reference
count (the "use" value which is returned by "show table" on the CLI). This
may sometimes be more suited for layer7 tracking.
Until now it was only possible to use track-sc1/sc2 with "src" which
is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch
as well as any transformation type. It works just like the "stick"
directive.
Samples are automatically converted to the correct types for the table.
Only "tcp-request content" rules may use L7 information, and such information
must already be present when the tracking is set up. For example it becomes
possible to track the IP address passed in the X-Forwarded-For header.
HTTP request processing now also considers tracking from backend rules
because we want to be able to update the counters even when the request
was already parsed and tracked.
Some more controls need to be performed (eg: samples do not distinguish
between L4 and L6).
Considering there is no option yet for maxconnrate for servers, I wrote
an ACL to check a backend server session rate which we use to send to an
"overflow" backend to prevent latency responses to our clients (very
sensitive latency requirements).
Sessions using client certs are huge (more than 1 kB) and do not fit
in session cache, or require a huge cache.
In this new implementation sshcachesize set a number of available blocks
instead a number of available sessions.
Each block is large enough (128 bytes) to store a simple session (without
client certs).
Huge sessions will take multiple blocks depending on client certificate size.
Note: some unused code for session sync with remote peers was temporarily
removed.
The compression is disabled when the HTTP status code is not 200, indeed
compression on some HTTP code can create issues (ex: 206, 416).
Multipart message should not be compressed eitherway.
Released version 1.5-dev14 with the following main changes :
- DOC: fix minor typos
- BUG/MEDIUM: compression: does not forward trailers
- MINOR: buffer_dump with ASCII
- BUG/MEDIUM: checks: mark the check as stopped after a connect error
- BUG/MEDIUM: checks: ensure we completely disable polling upon success
- BUG/MINOR: checks: don't mark the FD as closed before transport close
- MEDIUM: checks: avoid accumulating TIME_WAITs during checks
- MINOR: cli: report the msg state in full text in "show sess $PTR"
- CLEANUP: checks: rename some server check flags
- MAJOR: checks: rework completely bogus state machine
- BUG/MINOR: checks: slightly clean the state machine up
- MEDIUM: checks: avoid waking the application up for pure TCP checks
- MEDIUM: checks: close the socket as soon as we have a response
- BUG/MAJOR: checks: close FD on all timeouts
- MINOR: checks: fix recv polling after connect()
- MEDIUM: connection: provide a common conn_full_close() function
- BUG/MEDIUM: checks: prevent TIME_WAITs from appearing also on timeouts
- BUG/MAJOR: peers: the listener's maxaccept was not set and caused loops
- MINOR: listeners: make the accept loop more robust when maxaccept==0
- BUG/MEDIUM: acl: correctly resolve all args, not just the first one
- BUG/MEDIUM: acl: make prue_acl_expr() correctly free ACL expressions upon exit
- BUG/MINOR: stats: fix inversion of the report of a check in progress
- MEDIUM: tcp: add explicit support for delayed ACK in connect()
- BUG/MEDIUM: connection: always disable polling upon error
- MINOR: connection: abort earlier when errors are detected
- BUG/MEDIUM: checks: report handshake failures
- BUG/MEDIUM: connection: local_send_proxy must wait for connection to establish
- MINOR: tcp: add support for the "v6only" bind option
- MINOR: stats: also report the computed compression savings in html stats
- MINOR: stats: report the total number of compressed responses per front/back
- MINOR: tcp: add support for the "v4v6" bind option
- DOC: stats: document the comp_rsp stats column
- BUILD: buffer: fix another isprint() warning on solaris
- MINOR: cli: add support for the "show sess all" command
- BUG/MAJOR: cli: show sess <id> may randomly corrupt the back-ref list
- MINOR: cli: improve output format for show sess $ptr
Sometimes when debugging haproxy, it is important to take a full
snapshot of all sessions and their respective states. Till now it
was complicated to do because we had to use scripts and sessions
would vanish between two runs.
Now with this command we have the same output as "show sess $id"
but for all sessions in the table. This is a debugging command only,
it should only be used by developers as it is never guaranteed to
perfectly work !
Commit 9b6700f added "v6only". As suggested by Vincent Bernat, it is
sometimes useful to have the opposite option to force binding to the
two protocols when the system is configured to bind to v6 only by
default. This option does exactly this. v6only still has precedence.
Released version 1.5-dev13 with the following main changes :
- BUILD: fix build issue without USE_OPENSSL
- BUILD: fix compilation error with DEBUG_FULL
- DOC: ssl: remove prefer-server-ciphers documentation
- DOC: ssl: surround keywords with quotes
- DOC: fix minor typo on http-send-name-header
- BUG/MEDIUM: acls using IPv6 subnets patterns incorrectly match IPs
- BUG/MAJOR: fix a segfault on option http_proxy and url_ip acl
- MEDIUM: http: accept IPv6 values with (s)hdr_ip acl
- BUILD: report zlib support in haproxy -vv
- DOC: compression: add some details and clean up the formatting
- DOC: Change is_ssl acl to ssl_fc acl in example
- DOC: make it clear what the HTTP request size is
- MINOR: ssl: try to load Diffie-Hellman parameters from cert file
- DOC: ssl: update 'crt' statement on 'bind' about Diffie-Hellman parameters loading
- MINOR: ssl: add elliptic curve Diffie-Hellman support for ssl key generation
- DOC: ssl: add 'ecdhe' statement on 'bind'
- MEDIUM: ssl: add client certificate authentication support
- DOC: ssl: add 'verify', 'cafile' and 'crlfile' statements on 'bind'
- MINOR: ssl: add fetch and ACL 'client_crt' to test a client cert is present
- DOC: ssl: add fetch and ACL 'client_cert'
- MINOR: ssl: add ignore verify errors options
- DOC: ssl: add 'ca-ignore-err' and 'crt-ignore-err' statements on 'bind'
- MINOR: ssl: add fetch and ACL 'ssl_verify_result'
- DOC: ssl: add fetch and ACL 'ssl_verify_result'
- MINOR: ssl: add fetches and ACLs to return verify errors
- DOC: ssl: add fetches and ACLs 'ssl_verify_crterr', 'ssl_verify_caerr', and 'ssl_verify_crterr_depth'
- MINOR: ssl: disable shared memory and locks on session cache if nbproc == 1
- MINOR: ssl: add build param USE_PRIVATE_CACHE to build cache without shared memory
- MINOR: ssl : add statements 'notlsv11' and 'notlsv12' and rename 'notlsv1' to 'notlsv10'.
- DOC: ssl : add statements 'notlsv11' and 'notlsv12' and rename 'notlsv1' to 'notlsv10'.
- MEDIUM: config: authorize frontend and listen without bind.
- MINOR: ssl: add statement 'no-tls-tickets' on bind to disable stateless session resumption
- DOC: ssl: add 'no-tls-tickets' statement documentation.
- BUG/MINOR: ssl: Fix CRL check was not enabled when crlfile was specified.
- BUG/MINOR: build: Fix compilation issue on openssl 0.9.6 due to missing CRL feature.
- BUG/MINOR: conf: Fix 'maxsslconn' statement error if built without OPENSSL.
- BUG/MINOR: build: Fix failure with USE_OPENSSL=1 and USE_FUTEX=1 on archs i486 and i686.
- MINOR: ssl: remove prefer-server-ciphers statement and set it as the default on ssl listeners.
- BUG/MEDIUM: ssl: subsequent handshakes fail after server configuration changes
- MINOR: ssl: add 'crt-base' and 'ca-base' global statements.
- MEDIUM: conf: rename 'nosslv3' and 'notlsvXX' statements 'no-sslv3' and 'no-tlsvXX'.
- MEDIUM: conf: rename 'cafile' and 'crlfile' statements 'ca-file' and 'crl-file'
- MINOR: ssl: use bit fields to store ssl options instead of one int each
- MINOR: ssl: add 'force-sslv3' and 'force-tlsvXX' statements on bind.
- MINOR: ssl: add 'force-sslv3' and 'force-tlsvXX' statements on server
- MINOR: ssl: add defines LISTEN_DEFAULT_CIPHERS and CONNECT_DEFAULT_CIPHERS.
- BUG/MINOR: ssl: Fix issue on server statements 'no-tls*' and 'no-sslv3'
- MINOR: ssl: move ssl context init for servers from cfgparse.c to ssl_sock.c
- MEDIUM: ssl: reject ssl server keywords in default-server statement
- MINOR: ssl: add statement 'no-tls-tickets' on server side.
- MINOR: ssl: add statements 'verify', 'ca-file' and 'crl-file' on servers.
- DOC: Fix rename of options cafile and crlfile to ca-file and crl-file.
- MINOR: sample: manage binary to string type convertion in stick-table and samples.
- MINOR: acl: add parse and match primitives to use binary type on ACLs
- MINOR: sample: export 'sample_get_trash_chunk(void)'
- MINOR: conf: rename all ssl modules fetches using prefix 'ssl_fc' and 'ssl_c'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_fc_protocol', 'ssl_fc_cipher', 'ssl_fc_use_keysize' and 'ssl_fc_alg_keysize'
- MINOR: ssl: add pattern fetch 'ssl_fc_session_id'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_version' and 'ssl_f_version'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_s_dn', 'ssl_c_i_dn', 'ssl_f_s_dn' and 'ssl_c_i_dn'
- MINOR: ssl: add pattern and ACLs 'ssl_c_sig_alg' and 'ssl_f_sig_alg'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_key_alg' and 'ssl_f_key_alg'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_notbefore', 'ssl_c_notafter', 'ssl_f_notbefore' and 'ssl_f_notafter'
- MINOR: ssl: add 'crt' statement on server.
- MINOR: ssl: checks the consistency of a private key with the corresponding certificate
- BUG/MEDIUM: ssl: review polling on reneg.
- BUG/MEDIUM: ssl: Fix some reneg cases not correctly handled.
- BUG/MEDIUM: ssl: Fix sometimes reneg fails if requested by server.
- MINOR: build: allow packagers to specify the ssl cache size
- MINOR: conf: add warning if ssl is not enabled and a certificate is present on bind.
- MINOR: ssl: Add tune.ssl.lifetime statement in global.
- MINOR: compression: Enable compression for IE6 w/SP2, IE7 and IE8
- BUG: http: revert broken optimisation from 82fe75c1a7
- DOC: duplicate ssl_sni section
- MEDIUM: HTTP compression (zlib library support)
- CLEANUP: use struct comp_ctx instead of union
- BUILD: remove dependency to zlib.h
- MINOR: compression: memlevel and windowsize
- MEDIUM: use pool for zlib
- MINOR: compression: try init in cfgparse.c
- MINOR: compression: init before deleting headers
- MEDIUM: compression: limit RAM usage
- MINOR: compression: tune.comp.maxlevel
- MINOR: compression: maximum compression rate limit
- MINOR: log-format: check number of arguments in cfgparse.c
- BUG/MEDIUM: compression: no Content-Type header but type in configuration
- BUG/MINOR: compression: deinit zlib only when required
- MEDIUM: compression: don't compress when no data
- MEDIUM: compression: use pool for comp_ctx
- MINOR: compression: rate limit in 'show info'
- MINOR: compression: report zlib memory usage
- BUG/MINOR: compression: dynamic level increase
- DOC: compression: unsupported cases.
- MINOR: compression: CPU usage limit
- MEDIUM: http: add "redirect scheme" to ease HTTP to HTTPS redirection
- BUG/MAJOR: ssl: missing tests in ACL fetch functions
- MINOR: config: add a function to indent error messages
- REORG: split "protocols" files into protocol and listener
- MEDIUM: config: replace ssl_conf by bind_conf
- CLEANUP: listener: remove unused conf->file and conf->line
- MEDIUM: listener: add a minimal framework to register "bind" keyword options
- MEDIUM: config: move the "bind" TCP parameters to proto_tcp
- MEDIUM: move bind SSL parsing to ssl_sock
- MINOR: config: improve error reporting for "bind" lines
- MEDIUM: config: move the common "bind" settings to listener.c
- MEDIUM: config: move all unix-specific bind keywords to proto_uxst.c
- MEDIUM: config: enumerate full list of registered "bind" keywords upon error
- MINOR: listener: add a scope field in the bind keyword lists
- MINOR: config: pass the file and line to config keyword parsers
- MINOR: stats: fill the file and line numbers in the stats frontend
- MINOR: config: set the bind_conf entry on listeners created from a "listen" line.
- MAJOR: listeners: use dual-linked lists to chain listeners with frontends
- REORG: listener: move unix perms from the listener to the bind_conf
- BUG: backend: balance hdr was broken since 1.5-dev11
- MINOR: standard: make memprintf() support a NULL destination
- MINOR: config: make str2listener() use memprintf() to report errors.
- MEDIUM: stats: remove the stats_sock struct from the global struct
- MINOR: ssl: set the listeners' data layer to ssl during parsing
- MEDIUM: stats: make use of the standard "bind" parsers to parse global socket
- DOC: move bind options to their own section
- DOC: stats: refer to "bind" section for "stats socket" settings
- DOC: fix index to reference bind and server options
- BUG: http: do not print garbage on invalid requests in debug mode
- BUG/MINOR: config: check the proper pointer to report unknown protocol
- CLEANUP: connection: offer conn_prepare() to set up a connection
- CLEANUP: config: fix typo inteface => interface
- BUG: stats: fix regression introduced by commit 4348fad1
- MINOR: cli: allow to set frontend maxconn to zero
- BUG/MAJOR: http: chunk parser was broken with buffer changes
- MEDIUM: monitor: simplify handling of monitor-net and mode health
- MINOR: connection: add a pointer to the connection owner
- MEDIUM: connection: make use of the owner instead of container_of
- BUG/MINOR: ssl: report the L4 connection as established when possible
- BUG/MEDIUM: proxy: must not try to stop disabled proxies upon reload
- BUG/MINOR: config: use a copy of the file name in proxy configurations
- BUG/MEDIUM: listener: don't pause protocols that do not support it
- MEDIUM: proxy: add the global frontend to the list of normal proxies
- BUG/MINOR: epoll: correctly disable FD polling in fd_rem()
- MINOR: signal: really ignore signals configured with no handler
- MINOR: buffers: add a few functions to write chars, strings and blocks
- MINOR: raw_sock: always report asynchronous connection errors
- MEDIUM: raw_sock: improve connection error reporting
- REORG: connection: rename the data layer the "transport layer"
- REORG: connection: rename app_cb "data"
- MINOR: connection: provide a generic data layer wakeup callback
- MINOR: connection: split conn_prepare() in two functions
- MINOR: connection: add an init callback to the data_cb struct
- MEDIUM: session: use a specific data_cb for embryonic sessions
- MEDIUM: connection: use a generic data-layer init() callback
- MEDIUM: connection: reorganize connection flags
- MEDIUM: connection: only call the data->wake callback on activity
- MEDIUM: connection: make it possible for data->wake to return an error
- MEDIUM: session: register a data->wake callback to process errors
- MEDIUM: connection: don't call the data->init callback upon error
- MEDIUM: connection: it's not the data layer's role to validate the connection
- MEDIUM: connection: automatically disable polling on error
- REORG: connection: move the PROXY protocol management to connection.c
- MEDIUM: connection: add a new local send-proxy transport callback
- MAJOR: checks: make use of the connection layer to send checks
- REORG: server: move the check-specific parts into a check subsection
- MEDIUM: checks: use real buffers to store requests and responses
- MEDIUM: check: add the ctrl and transport layers in the server check structure
- MAJOR: checks: completely use the connection transport layer
- MEDIUM: checks: add the "check-ssl" server option
- MEDIUM: checks: enable the PROXY protocol with health checks
- CLEANUP: checks: remove minor warnings for assigned but not used variables
- MEDIUM: tcp: enable TCP Fast Open on systems which support it
- BUG: connection: fix regression from commit 9e272bf9
- CLEANUP: cttproxy: remove a warning on undeclared close()
- BUG/MAJOR: ensure that hdr_idx is always reserved when L7 fetches are used
- MEDIUM: listener: add support for linux's accept4() syscall
- MINOR: halog: sort output by cookie code
- BUG/MINOR: halog: -ad/-ac report the correct number of output lines
- BUG/MINOR: halog: fix help message for -ut/-uto
- MINOR: halog: add a parameter to limit output line count
- BUILD: accept4: move the socketcall declaration outside of accept4()
- MINOR: server: add minimal infrastructure to parse keywords
- MINOR: standard: make indent_msg() support empty messages
- MEDIUM: server: check for registered keywords when parsing unknown keywords
- MEDIUM: server: move parsing of keyword "id" to server.c
- BUG/MEDIUM: config: check-send-proxy was ignored if SSL was not builtin
- MEDIUM: ssl: move "server" keyword SSL options parsing to ssl_sock.c
- MEDIUM: log: suffix the frontend's name with '~' when using SSL
- MEDIUM: connection: always unset the transport layer upon close
- BUG/MINOR: session: fix some leftover from debug code
- BUG/MEDIUM: session: enable the conn_session_update() callback
- MEDIUM: connection: add a flag to hold the transport layer
- MEDIUM: log: add a new LW_XPRT flag to pin the transport layer
- MINOR: log: make lf_text use a const char *
- MEDIUM: log: report SSL ciphers and version in logs using logformat %sslc/%sslv
- REORG: http: rename msg->buf to msg->chn since it's a channel
- CLEANUP: http: use 'chn' to name channel variables, not 'buf'
- CLEANUP: channel: use 'chn' instead of 'buf' as local variable names
- CLEANUP: tcp: use 'chn' instead of 'buf' or 'b' for channel pointer names
- CLEANUP: stream_interface: use 'chn' instead of 'b' to name channel pointers
- CLEANUP: acl: use 'chn' instead of 'b' to name channel pointers
- MAJOR: channel: replace the struct buffer with a pointer to a buffer
- OPTIM: channel: reorganize struct members to improve cache efficiency
- CLEANUP: session: remove term_trace which is not used anymore
- OPTIM: session: reorder struct session fields
- OPTIM: connection: pack the struct target
- DOC: document relations between internal entities
- MINOR: ssl: add 'ssl_npn' sample/acl to extract TLS/NPN information
- BUILD: ssl: fix shctx build on older compilers
- MEDIUM: ssl: add support for the "npn" bind keyword
- BUG: ssl: fix ssl_sni ACLs to correctly process regular expressions
- MINOR: chunk: provide string compare functions
- MINOR: sample: accept fetch keywords without parenthesis
- MEDIUM: sample: pass an empty list instead of a null for fetch args
- MINOR: ssl: improve socket behaviour upon handshake abort.
- BUG/MEDIUM: http: set DONTWAIT on data when switching to tunnel mode
- MEDIUM: listener: provide a fallback for accept4() when not supported
- BUG/MAJOR: connection: risk of crash on certain tricky close scenario
- MEDIUM: cli: allow the stats socket to be bound to a specific set of processes
- OPTIM: channel: inline channel_forward's fast path
- OPTIM: http: inline http_parse_chunk_size() and http_skip_chunk_crlf()
- OPTIM: tools: inline hex2i()
- CLEANUP: http: rename HTTP_MSG_DATA_CRLF state
- MINOR: compression: automatically disable compression for older browsers
- MINOR: compression: optimize memLevel to improve byte rate
- BUG/MINOR: http: compression should consider all Accept-Encoding header values
- BUILD: fix coexistence of openssl and zlib
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_serial' and 'ssl_f_serial'
- BUG/MEDIUM: command-line option -D must have precedence over "debug"
- MINOR: tools: add a clear_addr() function to unset an address
- BUG/MEDIUM: tcp: transparent bind to the source only when address is set
- CLEANUP: remove trashlen
- MAJOR: session: detach the connections from the stream interfaces
- DOC: update document describing relations between internal entities
- BUILD: make it possible to specify ZLIB path
- MINOR: compression: add an offload option to remove the Accept-Encoding header
- BUG: compression: disable auto-close and enable MSG_MORE during transfer
- CLEANUP: completely remove trashlen
- MINOR: chunk: add a function to reset a chunk
- CLEANUP: replace chunk_printf() with chunk_appendf()
- MEDIUM: make the trash be a chunk instead of a char *
- MEDIUM: remove remains of BUFSIZE in HTTP auth and sample conversions
- MEDIUM: stick-table: allocate the table key of size buffer size
- BUG/MINOR: stream_interface: don't loop over ->snd_buf()
- BUG/MINOR: session: ensure that we don't retry connection if some data were sent
- OPTIM: session: don't process the whole session when only timers need a refresh
- BUG/MINOR: session: mark the handshake as complete earlier
- MAJOR: connection: remove the CO_FL_CURR_*_POL flag
- BUG/MAJOR: always clear the CO_FL_WAIT_* flags after updating polling flags
- MAJOR: sepoll: make the poller totally event-driven
- OPTIM: stream_interface: disable reading when CF_READ_DONTWAIT is set
- BUILD: compression: remove a build warning
- MEDIUM: fd: don't unset fdtab[].updated upon delete
- REORG: fd: move the speculative I/O management from ev_sepoll
- REORG: fd: move the fd state management from ev_sepoll
- REORG: fd: centralize the processing of speculative events
- BUG: raw_sock: also consider ENOTCONN in addition to EAGAIN
- BUILD: stream_interface: remove si_fd() and its references
- BUILD: compression: enable build in BSD and OSX Makefiles
- MAJOR: ev_select: make the poller support speculative events
- MAJOR: ev_poll: make the poller support speculative events
- MAJOR: ev_kqueue: make the poller support speculative events
- MAJOR: polling: replace epoll with sepoll and remove sepoll
- MAJOR: polling: remove unused callbacks from the poller struct
- MEDIUM: http: refrain from sending "Connection: close" when Upgrade is present
- CLEANUP: channel: remove any reference of the hijackers
- CLEANUP: stream_interface: remove the external task type target
- MAJOR: connection: replace struct target with a pointer to an enum
- BUG: connection: fix typo in previous commit
- BUG: polling: don't skip polled events in the spec list
- MINOR: splice: disable it when the system returns EBADF
- MINOR: build: allow packagers to specify the default maxzlibmem
- BUG: halog: fix broken output limitation
- BUG: proxy: fix server name lookup in get_backend_server()
- BUG: compression: do not always increment the round counter on allocation failure
- BUG/MEDIUM: compression: release the zlib pools between keep-alive requests
- MINOR: global: don't prevent nbproc from being redefined
- MINOR: config: support process ranges for "bind-process"
- MEDIUM: global: add support for CPU binding on Linux ("cpu-map")
- MINOR: ssl: rename and document the tune.ssl.cachesize option
- DOC: update the PROXY protocol spec to support v2
- MINOR: standard: add a simple popcount function
- MEDIUM: adjust the maxaccept per listener depending on the number of processes
- BUG: compression: properly disable compression when content-type does not match
- MINOR: cli: report connection status in "show sess xxx"
- BUG/MAJOR: stream_interface: certain workloads could cause get stuck
- BUILD: cli: fix build when SSL is enabled
- MINOR: cli: report the fd state in "show sess xxx"
- MINOR: cli: report an error message on missing argument to compression rate
- MINOR: http: add some debugging functions to pretty-print msg state names
- BUG/MAJOR: stream_interface: read0 not always handled since dev12
- DOC: documentation on http header capture is wrong
- MINOR: http: allow the cookie capture size to be changed
- DOC: http header capture has not been limited in size for a long time
- DOC: update readme with build methods for BSD
- BUILD: silence a warning on Solaris about usage of isdigit()
- MINOR: stats: report HTTP compression stats per frontend and per backend
- MINOR: log: add '%Tl' to log-format
- MINOR: samples: update the url_param fetch to match parameters in the path
It was a bit frustrating to have no idea about the bandwidth saved by
HTTP compression. Now we have per-frontend and per-backend stats. The
stats on the HTTP interface are shown in a hover title in the "bytes out"
column if at least something was fed to the compressor. 3 new columns
appeared in the CSV stats output.
It's been documented for a very long time that captured HTTP headers
were limited to 64 characters, but this has not the case anymore since
1.3.11 in 2007 (commit cf7f320f), as they all use their own pool and
have no such limit anymore.
This fix should be backported to 1.4 and 1.3.
Some users need more than 64 characters to log large cookies. The limit
was set to 63 characters (and not 64 as previously documented). Now it
is possible to change this using the global "tune.http.cookielen" setting
if required.
Since commit it is said that only the first value of the first occurrence
of a header is captured. This is wrong. Since the introduction of header
captures in version 1.1 in 2005 (commit e983144d), the WHOLE line of the
LAST occurrence has been captured and the behaviour has never changed.
At this time the doc was correct. The error was introduced in the new doc
in 1.3.14 in 2007 (commit 0ba27505).
So this fix should be backported to 1.4 and 1.3.
New option 'maxcompcpuusage' in global section.
Sets the maximum CPU usage HAProxy can reach before stopping the
compression for new requests or decreasing the compression level of
current requests. It works like 'maxcomprate' but with the Idle.
global.tune.maxaccept was used for all listeners. This becomes really not
convenient when some listeners are bound to a single process and other ones
are bound to many processes.
Now we change the principle : we count the number of processes a listener
is bound to, and apply the maxaccept either entirely if there is a single
process, or divided by twice the number of processes in order to maintain
fairness.
The default limit has also been increased from 32 to 64 as it appeared that
on small machines, 32 was too low to achieve high connection rates.
The new "cpu-map" directive allows one to assign the CPU sets that
a process is allowed to bind to. This is useful in combination with
the "nbproc" and "bind-process" directives.
The support is implicit on Linux 2.6.28 and above.
Now that all pollers make use of speculative I/O, there is no point
having two epoll implementations, so replace epoll with the sepoll code
and remove sepoll which has just become the standard epoll method.
commit 82fe75c1 provided useful details in its log message. We should report
part of them in the documentation to know which algorithms are available.
This patch also makes some formatting cleanups (including a line outside the
compression scope, which exceeded 80 chars).
This patch adds input and output rate calcutation on the HTTP compresion
feature.
Compression can be limited with a maximum rate value in kilobytes per
second. The rate is set with the global 'maxcomprate' option. You can
change this value dynamicaly with 'set rate-limit http-compression
global' on the UNIX socket.
With the global maxzlibmem option, you are able ton control the maximum
amount of RAM usable for HTTP compression.
A test is done before each zlib allocation, if the there isn't available
memory, the test fail and so the zlib initialization, so data won't be
compressed.
The window size and the memlevel of the zlib are now configurable using
global options tune.zlib.memlevel and tune.zlib.windowsize.
It affects the memory consumption of the zlib.
ssl_c_notbefore: start date of client cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_c_notafter: end date of client cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_f_notbefore: start date of frontend cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_f_notafter: end date of frontend cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_c_s_dn : client cert subject DN (string)
ssl_c_i_dn : client cert issuer DN (string)
ssl_f_s_dn : frontend cert subject DN (string)
ssl_f_i_dn : frontend cert issuer DN (string)
Return either the full DN without params, or just the DN entry (first param) or
its specific occurrence (second param).
This commit introduces HTTP compression using the zlib library.
http_response_forward_body has been modified to call the compression
functions.
This feature includes 3 algorithms: identity, gzip and deflate:
* identity: this is mostly for debugging, and it was useful for
developping the compression feature. With Content-Length in input, it
is making each chunk with the data available in the current buffer.
With chunks in input, it is rechunking, the output chunks will be
bigger or smaller depending of the size of the input chunk and the
size of the buffer. Identity does not apply any change on data.
* gzip: same as identity, but applying a gzip compression. The data
are deflated using the Z_NO_FLUSH flag in zlib. When there is no more
data in the input buffer, it flushes the data in the output buffer
(Z_SYNC_FLUSH). At the end of data, when it receives the last chunk in
input, or when there is no more data to read, it writes the end of
data with Z_FINISH and the ending chunk.
* deflate: same as gzip, but with deflate algorithm and zlib format.
Note that this algorithm has ambiguous support on many browsers and
no support at all from recent ones. It is strongly recommended not
to use it for anything else than experimentation.
You can't choose the compression ratio at the moment, it will be set to
Z_BEST_SPEED (1), as tests have shown very little benefit in terms of
compression ration when going above for HTML contents, at the cost of
a massive CPU impact.
Compression will be activated depending of the Accept-Encoding request
header. With identity, it does not take care of that header.
To build HAProxy with zlib support, use USE_ZLIB=1 in the make
parameters.
This work was initially started by David Du Colombier at Exceliance.
Using "stats bind-process", it becomes possible to indicate to haproxy which
process will get the incoming connections to the stats socket. It will also
shut down the warning when nbproc > 1.
Please consider the following patch for configuration.txt to clarify meaning
of bufsize, maxrewrite and the size of HTTP request which can be processed.
The ssl_npn match could not work by itself because clients do not use
the NPN extension unless the server advertises the protocols it supports.
Thanks to Simone Bordet for the explanations on how to get it right.
These two new log-format tags report the SSL protocol version (%sslv) and the
SSL ciphers (%sslc) used for the connection with the client. For instance, to
append these information just after the client's IP/port address information
on an HTTP log line, use the following configuration :
log-format %Ci:%Cp\ %sslv:%sslc\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %st\ %B\ %cc\ \ %cs\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ %{+Q}r
It will report a line such as the following one :
Oct 12 20:47:30 haproxy[9643]: 127.0.0.1:43602 TLSv1:AES-SHA [12/Oct/2012:20:47:30.303] stick2~ stick2/s1 7/0/12/0/19 200 145 - - ---- 0/0/0/0/0 0/0 "GET /?t=0 HTTP/1.0"
Until now it was not possible to know from the logs whether the incoming
connection was made over SSL or not. In order to address this in the existing
log formats, a new log format %ft was introduced, to log the frontend's name
suffixed with its transport layer. The only transport layer in use right now
is '~' for SSL, so that existing log formats for non-SSL traffic are not
affected at all, and SSL log formats have the frontend's name suffixed with
'~'.
The TCP, HTTP and CLF log format now use %ft instead of %f. This does not
affect existing log formats which still make use of %f however.
It now becomes possible to verify the server's certificate using the "verify"
directive. This one only supports "none" and "required", as it does not make
much sense to also support "optional" here.
When health checks are configured on a server which has the send-proxy
directive and no "port" nor "addr" settings, the health check connections
will automatically use the PROXY protocol. If "port" or "addr" are set,
the "check-send-proxy" directive may be used to force the protocol.
While working on the changes required to make the health checks use the
new connections, it started to become obvious that some naming was not
logical at all in the connections. Specifically, it is not logical to
call the "data layer" the layer which is in charge for all the handshake
and which does not yet provide a data layer once established until a
session has allocated all the required buffers.
In fact, it's more a transport layer, which makes much more sense. The
transport layer offers a medium on which data can transit, and it offers
the functions to move these data when the upper layer requests this. And
it is the upper layer which iterates over the transport layer's functions
to move data which should be called the data layer.
The use case where it's obvious is with embryonic sessions : an incoming
SSL connection is accepted. Only the connection is allocated, not the
buffers nor stream interface, etc... The connection handles the SSL
handshake by itself. Once this handshake is complete, we can't use the
data functions because the buffers and stream interface are not there
yet. Hence we have to first call a specific function to complete the
session initialization, after which we'll be able to use the data
functions. This clearly proves that SSL here is only a transport layer
and that the stream interface constitutes the data layer.
A similar change will be performed to rename app_cb => data, but the
two could not be in the same commit for obvious reasons.
Disables the stateless session resumption (RFC 5077 TLS Ticket
extension) and force to use stateful session resumption. Stateless
session resumption is more expensive in CPU usage.
We were having several different behaviours with monitor-net and
"mode health" :
- monitor-net on TCP connections was evaluated just after accept(),
did not count a connection on the frontend and were not subject
to tcp-request connection rules, and caused an immediate close().
- monitor-net in HTTP mode was evaluated once the session was
accepted (eg: on top of SSL), returned "HTTP/1.0 200 OK\r\n\r\n"
over the connection's data layer and instanciated a session which
was responsible for closing this connection. A connection AND a
session were counted for the frontend ;
- "mode health" with "option httpchk" would do exactly the same as
monitor-net in HTTP mode ;
- "mode health" without "option httpchk" would do the same as above
except that "OK" was returned instead of "HTTP/1.0 200 OK\r\n\r\n".
None of them took care of cleaning the input buffer, sometimes resulting
in a TCP reset to be emitted after the last packet if a request was received
over the connection.
Given the inconsistencies and the complexity in keeping all these features
handled at the right position, we now slightly changed the way they are
handled :
- all of them are handled just after the "tcp-request connection" rules,
so that all of them may be blocked using such rules, offering more
flexibility and consistency ;
- no connection handshake is performed anymore for non-TCP modes
- all of them send the response as raw data over the socket, there is no
more difference between TCP and HTTP mode for example (these rules were
never meant to be served over SSL connections and were never documented
as able to do that).
- any possible pending data on the incoming socket is drained before the
response is sent, in order to avoid the risk of a reset.
- none of them exactly did what was documented !
This results in more consistent, more flexible and more accurate handling of
monitor rules, with smaller and more robust code.
It is sometimes useful to completely disable accepting new connections
on a frontend during maintenance operations. By setting a frontend's
maxconn to zero, connections are not accepted anymore until the limit
is increased again.
There are now too many bind options to still have them in the middle
of the keyword matrix, so let's move them with the server options in
section 5. No new option was documented yet at this point.
Released version 1.5-dev12 with the following main changes :
- CONTRIB: halog: sort URLs by avg bytes_read or total bytes_read
- MEDIUM: ssl: add support for prefer-server-ciphers option
- MINOR: IPv6 support for transparent proxy
- MINOR: protocol: add SSL context to listeners if USE_OPENSSL is defined
- MINOR: server: add SSL context to servers if USE_OPENSSL is defined
- MEDIUM: connection: add a new handshake flag for SSL (CO_FL_SSL_WAIT_HS).
- MEDIUM: ssl: add new files ssl_sock.[ch] to provide the SSL data layer
- MEDIUM: config: add the 'ssl' keyword on 'bind' lines
- MEDIUM: config: add support for the 'ssl' option on 'server' lines
- MEDIUM: ssl: protect against client-initiated renegociation
- BUILD: add optional support for SSL via the USE_OPENSSL flag
- MEDIUM: ssl: add shared memory session cache implementation.
- MEDIUM: ssl: replace OpenSSL's session cache with the shared cache
- MINOR: ssl add global setting tune.sslcachesize to set SSL session cache size.
- MEDIUM: ssl: add support for SNI and wildcard certificates
- DOC: Typos cleanup
- DOC: fix name for "option independant-streams"
- DOC: specify the default value for maxconn in the context of a proxy
- BUG/MINOR: to_log erased with unique-id-format
- LICENSE: add licence exception for OpenSSL
- BUG/MAJOR: cookie prefix doesn't support cookie-less servers
- BUILD: add an AIX 5.2 (and later) target.
- MEDIUM: fd/si: move peeraddr from struct fdinfo to struct connection
- MINOR: halog: use the more recent dual-mode fgets2 implementation
- BUG/MEDIUM: ebtree: ebmb_insert() must not call cmp_bits on full-length matches
- CLEANUP: halog: make clean should also remove .o files
- OPTIM: halog: make use of memchr() on platforms which provide a fast one
- OPTIM: halog: improve cold-cache behaviour when loading a file
- BUG/MINOR: ACL implicit arguments must be created with unresolved flag
- MINOR: replace acl_fetch_{path,url}* with smp_fetch_*
- MEDIUM: pattern: add the "base" sample fetch method
- OPTIM: i386: make use of kernel-mode-linux when available
- BUG/MINOR: tarpit: fix condition to return the HTTP 500 message
- BUG/MINOR: polling: some events were not set in various pollers
- MINOR: http: add the urlp_val ACL match
- BUG: stktable: tcp_src_to_stktable_key() must return NULL on invalid families
- MINOR: stats/cli: add plans to support more stick-table actions
- MEDIUM: stats/cli: add support for "set table key" to enter values
- REORG/MEDIUM: fd: remove FD_STCLOSE from struct fdtab
- REORG/MEDIUM: fd: remove checks for FD_STERROR in ev_sepoll
- REORG/MEDIUM: fd: get rid of FD_STLISTEN
- REORG/MINOR: connection: move declaration to its own include file
- REORG/MINOR: checks: put a struct connection into the server
- MINOR: connection: add flags to the connection struct
- MAJOR: get rid of fdtab[].state and use connection->flags instead
- MINOR: fd: add a new I/O handler to fdtab
- MEDIUM: polling: prepare to call the iocb() function when defined.
- MEDIUM: checks: make use of fdtab->iocb instead of cb[]
- MEDIUM: protocols: use the generic I/O callback for accept callbacks
- MINOR: connection: add a handler for fd-based connections
- MAJOR: connection: replace direct I/O callbacks with the connection callback
- MINOR: fd: make fdtab->owner a connection and not a stream_interface anymore
- MEDIUM: connection: remove the FD_POLL_* flags only once
- MEDIUM: connection: extract the send_proxy callback from proto_tcp
- MAJOR: tcp: remove the specific I/O callbacks for TCP connection probes
- CLEANUP: remove the now unused fdtab direct I/O callbacks
- MAJOR: remove the stream interface and task management code from sock_*
- MEDIUM: stream_interface: pass connection instead of fd in sock_ops
- MEDIUM: stream_interface: centralize the SI_FL_ERR management
- MAJOR: connection: add a new CO_FL_CONNECTED flag
- MINOR: rearrange tcp_connect_probe() and fix wrong return codes
- MAJOR: connection: call data layer handshakes from the handler
- MEDIUM: fd: remove the EV_FD_COND_* primitives
- MINOR: sock_raw: move calls to si_data_close upper
- REORG: connection: replace si_data_close() with conn_data_close()
- MEDIUM: sock_raw: introduce a read0 callback that is different from shutr
- MAJOR: stream_int: use a common stream_int_shut*() functions regardless of the data layer
- MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls
- MEDIUM: fd: add fd_poll_{recv,send} for use when explicit polling is required
- MEDIUM: connection: add definitions for dual polling mechanisms
- MEDIUM: connection: make use of the new polling functions
- MAJOR: make use of conn_{data|sock}_{poll|stop|want}* in connection handlers
- MEDIUM: checks: don't use FD_WAIT_* anymore
- MINOR: fd: get rid of FD_WAIT_*
- MEDIUM: stream_interface: offer a generic function for connection updates
- MEDIUM: stream-interface: offer a generic chk_rcv function for connections
- MEDIUM: stream-interface: add a snd_buf() callback to sock_ops
- MEDIUM: stream-interface: provide a generic stream_int_chk_snd_conn() function
- MEDIUM: stream-interface: provide a generic si_conn_send_cb callback
- MEDIUM: stream-interface: provide a generic stream_sock_read0() function
- REORG/MAJOR: use "struct channel" instead of "struct buffer"
- REORG/MAJOR: extract "struct buffer" from "struct channel"
- MINOR: connection: provide conn_{data|sock}_{read0|shutw} functions
- REORG: sock_raw: rename the files raw_sock*
- MAJOR: raw_sock: extract raw_sock_to_buf() from raw_sock_read()
- MAJOR: raw_sock: temporarily disable splicing
- MINOR: stream-interface: add an rcv_buf callback to sock_ops
- REORG: stream-interface: move sock_raw_read() to si_conn_recv_cb()
- MAJOR: connection: split the send call into connection and stream interface
- MAJOR: stream-interface: restore splicing mechanism
- MAJOR: stream-interface: make conn_notify_si() more robust
- MEDIUM: proxy-proto: don't use buffer flags in conn_si_send_proxy()
- MAJOR: stream-interface: don't commit polling changes in every callback
- MAJOR: stream-interface: fix splice not to call chk_snd by itself
- MEDIUM: stream-interface: don't remove WAIT_DATA when a handshake is in progress
- CLEANUP: connection: split sock_ops into data_ops, app_cp and si_ops
- REORG: buffers: split buffers into chunk,buffer,channel
- MAJOR: channel: remove the BF_OUT_EMPTY flag
- REORG: buffer: move buffer_flush, b_adv and b_rew to buffer.h
- MINOR: channel: rename bi_full to channel_full as it checks the whole channel
- MINOR: buffer: provide a new buffer_full() function
- MAJOR: channel: stop relying on BF_FULL to take action
- MAJOR: channel: remove the BF_FULL flag
- REORG: channel: move buffer_{replace,insert_line}* to buffer.{c,h}
- CLEANUP: channel: usr CF_/CHN_ prefixes instead of BF_/BUF_
- CLEANUP: channel: use "channel" instead of "buffer" in function names
- REORG: connection: move the target pointer from si to connection
- MAJOR: connection: move the addr field from the stream_interface
- MEDIUM: stream_interface: remove CAP_SPLTCP/CAP_SPLICE flags
- MEDIUM: proto_tcp: remove any dependence on stream_interface
- MINOR: tcp: replace tcp_src_to_stktable_key with addr_to_stktable_key
- MEDIUM: connection: add an ->init function to data layer
- MAJOR: session: introduce embryonic sessions
- MAJOR: connection: make the PROXY decoder a handshake handler
- CLEANUP: frontend: remove the old proxy protocol decoder
- MAJOR: connection: rearrange the polling flags.
- MEDIUM: connection: only call tcp_connect_probe when nothing was attempted yet
- MEDIUM: connection: complete the polling cleanups
- MEDIUM: connection: avoid calling handshakes when polling is required
- MAJOR: stream_interface: continue to update data polling flags during handshakes
- CLEANUP: fd: remove fdtab->flags
- CLEANUP: fdtab: flatten the struct and merge the spec struct with the rest
- CLEANUP: includes: fix includes for a number of users of fd.h
- MINOR: ssl: disable TCP quick-ack by default on SSL listeners
- MEDIUM: config: add a "ciphers" keyword to set SSL cipher suites
- MEDIUM: config: add "nosslv3" and "notlsv1" on bind and server lines
- BUG: ssl: mark the connection as waiting for an SSL connection during the handshake
- BUILD: http: rename error_message http_error_message to fix conflicts on RHEL
- BUILD: ssl: fix shctx build on RHEL with futex
- BUILD: include sys/socket.h to fix build failure on FreeBSD
- BUILD: fix build error without SSL (ssl_cert)
- BUILD: ssl: use MAP_ANON instead of MAP_ANONYMOUS
- BUG/MEDIUM: workaround an eglibc bug which truncates the pidfiles when nbproc > 1
- MEDIUM: config: support per-listener backlog and maxconn
- MINOR: session: do not send an HTTP/500 error on SSL sockets
- MEDIUM: config: implement maxsslconn in the global section
- BUG: tcp: close socket fd upon connect error
- MEDIUM: connection: improve error handling around the data layer
- MINOR: config: make the tasks "nice" value configurable on "bind" lines.
- BUILD: shut a gcc warning introduced by commit 269ab31
- MEDIUM: config: centralize handling of SSL config per bind line
- BUILD: makefile: report USE_OPENSSL status in build options
- BUILD: report openssl build settings in haproxy -vv
- MEDIUM: ssl: add sample fetches for is_ssl, ssl_has_sni, ssl_sni_*
- DOC: add a special acknowledgement for the stud project
- DOC: add missing SSL options for servers and listeners
- BUILD: automatically add -lcrypto for SSL
- DOC: add some info about openssl build in the README
This is very convenient to reduce SSL processing priority compared to
other traffic. This applies to CPU usage only, but has a direct impact
on latency under congestion.
SSL connections take a huge amount of memory, and unfortunately openssl
does not check malloc() returns and easily segfaults when too many
connections are used.
The only solution against this is to provide a global maxsslconn setting
to reject SSL connections above the limit in order to avoid reaching
unsafe limits.
With SSL, connections are much more expensive, so it is important to be
able to limit concurrent connections per listener in order to limit the
memory usage.
The correct spelling is "independent", not "independant". This patch
fixes the doc and the configuration parser to accept the correct form.
The config parser still allows the old naming for backwards compatibility.
I came across a couple of typos in configuration.txt and made this patch.
Also, there is an inconsistency between using the word analys/ze in
configuration.txt as well. However, I did not provide a patch for that.
-- Jamie Gloudon
[wt: won't fix the us/uk language mistakes, they'll always exist anyway]
This is used to enter values for stick tables. The most likely usage
is to set gpc0 for a specific IP address in order to block traffic
for abusers without having to reload. Since all data types are
supported, other usages are possible (eg: replace a users's assigned
server).
This one returns the concatenation of the first Host header entry with
the path. It can make content-switching rules easier, help with fighting
DDoS on certain URLs and improve shared caches efficiency.
Doing so allows us to support sticking on URL, URL's IP, URL's port and
path.
Both fetch functions should be improved to support an optional depth
allowing to stick to a server depending on just a few directory
components. This would help with portals, some prefetch-capable
caches and with outgoing connections using multiple internet links.
Default value for maxconn in the context of a proxy is 2000 and is
unrelated to any other value (like global ulimit-n or global
maxconn). Without an explicit a user may think that the default value
is either no limit or equal to the global maxconn value.
Released version 1.5-dev11 with the following main changes :
- BUG/MEDIUM: option forwardfor if-none doesn't work with some configurations
- BUG/MAJOR: trash must always be the size of a buffer
- DOC: fix minor regex example issue and improve doc on stats
- MINOR: stream_interface: add a pointer to the listener for TARG_TYPE_CLIENT
- MEDIUM: protocol: add a pointer to struct sock_ops to the listener struct
- MINOR: checks: add on-marked-up option
- MINOR: balance uri: added 'whole' parameter to include query string in hash calculation
- MEDIUM: stream_interface: remove the si->init
- MINOR: buffers: add a rewind function
- BUG/MAJOR: fix regression on content-based hashing and http-send-name-header
- MAJOR: http: stop using msg->sol outside the parsers
- CLEANUP: http: make it more obvious that msg->som is always null outside of chunks
- MEDIUM: http: get rid of msg->som which is not used anymore
- MEDIUM: http: msg->sov and msg->sol will never wrap
- BUG/MAJOR: checks: don't call set_server_status_* when no LB algo is set
- BUG/MINOR: stop connect timeout when connect succeeds
- REORG: move the send-proxy code to tcp_connect_write()
- REORG/MINOR: session: detect the TCP monitor checks at the protocol accept
- MINOR: stream_interface: introduce a new "struct connection" type
- REORG/MINOR: stream_interface: move si->fd to struct connection
- REORG/MEDIUM: stream_interface: move applet->state and private to connection
- MINOR: stream_interface: add a data channel close function
- MEDIUM: stream_interface: call si_data_close() before releasing the si
- MINOR: peers: use the socket layer operations from the peer instead of sock_raw
- BUG/MINOR: checks: expire on timeout.check if smaller than timeout.connect
- MINOR: add a new function call tracer for debugging purposes
- BUG/MINOR: perform_http_redirect also needs to rewind the buffer
- BUG/MAJOR: b_rew() must pass a signed offset to b_ptr()
- BUG/MEDIUM: register peer sync handler in the proper order
- BUG/MEDIUM: buffers: fix bi_putchr() to correctly advance the pointer
- BUG/MINOR: fix option httplog validation with TCP frontends
- BUG/MINOR: log: don't report logformat errors in backends
- REORG/MINOR: use dedicated proxy flags for the cookie handling
- BUG/MINOR: config: do not report twice the incompatibility between cookie and non-http
- MINOR: http: add support for "httponly" and "secure" cookie attributes
- BUG/MEDIUM: ensure that unresolved arguments are freed exactly once
- BUG/MINOR: commit 196729ef used wrong condition resulting in freeing constants
- MEDIUM: stats: add support for soft stop/soft start in the admin interface
- MEDIUM: stats: add the ability to kill sessions from the admin interface
- BUILD: add support for linux kernels >= 2.6.28
This implements the feature discussed in the earlier thread of killing
connections on backup servers when a non-backup server comes back up. For
example, you can use this to route to a mysql master & slave and ensure
clients don't stay on the slave after the master goes from down->up. I've done
some minimal testing and it seems to work.
[WT: added session flag & doc, moved the killing after logging the server UP,
and ensured that the new server is really usable]
httponly This option tells haproxy to add an "HttpOnly" cookie attribute
when a cookie is inserted. This attribute is used so that a
user agent doesn't share the cookie with non-HTTP components.
Please check RFC6265 for more information on this attribute.
secure This option tells haproxy to add a "Secure" cookie attribute when
a cookie is inserted. This attribute is used so that a user agent
never emits this cookie over non-secure channels, which means
that a cookie learned with this flag will be presented only over
SSL/TLS connections. Please check RFC6265 for more information on
this attribute.
This patch brings a new "whole" parameter to "balance uri" which makes
the hash work over the whole uri, not just the part before the query
string. Len and depth parameter are still honnored.
The reason for this new feature is explained below.
I have 3 backend servers, each accepting different form of HTTP queries:
http://backend1.server.tld/service1.php?q=...
http://backend1.server.tld/service2.php?q=...
http://backend2.server.tld/index.php?query=...&subquery=...
http://backend3.server.tld/image/49b8c0d9ff
Each backend server returns a different response based on either:
- the URI path (the left part of the URI before the question mark)
- the query string (the right part of the URI after the question mark)
- or the combination of both
I wanted to set up a common caching cluster (using 6 Squid servers, each
configured as reverse proxy for those 3 backends) and have HAProxy balance
the queries among the Squid servers based on URL. I also wanted to achieve
hight cache hit ration on each Squid server and send the same queries to
the same Squid servers. Initially I was considering using the 'balance uri'
algorithm, but that would not work as in case of backend2 all queries would
go to only one Squid server. The 'balance url_param' would not work either
as it would send the backend3 queries to only one Squid server.
So I thought the simplest solution would be to use 'balance uri', but to
calculate the hash based on the whole URI (URI path + query string),
instead of just the URI path.
Released version 1.5-dev10 with the following main changes :
- BUG/MINOR: stats admin: "Unexpected result" was displayed unconditionally
- BUG/MAJOR: acl: http_auth_group() must not accept any user from the userlist
- CLEANUP: auth: make the code build again with DEBUG_AUTH
- BUG/MEDIUM: config: don't crash at config load time on invalid userlist names
- REORG: use the name sock_raw instead of stream_sock
- MINOR: stream_interface: add a client target : TARG_TYPE_CLIENT
- BUG/MEDIUM: stream_interface: restore get_src/get_dst
- CLEANUP: sock_raw: remove last references to stream_sock
- CLEANUP: stream_interface: stop exporting socket layer functions
- MINOR: stream_interface: add an init callback to sock_ops
- MEDIUM: stream_interface: derive the socket operations from the target
- MAJOR: fd: remove the need for the socket layer to recheck the connection
- MINOR: session: call the socket layer init function when a session establishes
- MEDIUM: session: add support for tunnel timeouts
- MINOR: standard: add a new debug macro : fddebug()
- CLEANUP: fd: remove unused cb->b pointers in the struct fdtab
- OPTIM: proto_http: don't enable quick-ack on empty buffers
- OPTIM/MAJOR: ev_sepoll: process spec events after polled events
- OPTIM/MEDIUM: stream_interface: add a new SI_FL_NOHALF flag
Tunnel timeouts are used when TCP connections are forwarded, or
when forwarding upgraded HTTP connections (WebSocket) as well as
CONNECT requests to proxies.
This timeout allows long-lived sessions to be supported without
having to set large timeouts to normal requests.
Released version 1.5-dev9 with the following main changes :
- MINOR: Add release callback to si_applet
- CLEANUP: Fix some minor typos
- MINOR: Add TO/FROM_SET flags to struct stream_interface
- CLEANUP: Fix some minor whitespace issues
- MINOR: stats admin: allow unordered parameters in POST requests
- CLEANUP: fix typo in findserver() log message
- MINOR: stats admin: use the backend id instead of its name in the form
- MINOR: stats admin: reduce memcmp()/strcmp() calls on status codes
- DOC: cleanup indentation, alignment, columns and chapters
- DOC: fix some keywords arguments documentation
- MINOR: cli: display the 4 IP addresses and ports on "show sess XXX"
- BUG/MAJOR: log: possible segfault with logformat
- MEDIUM: log: split of log_format generation
- MEDIUM: log: New format-log flags: %Fi %Fp %Si %Sp %Ts %rt %H %pid
- MEDIUM: log: Unique ID
- MINOR: log: log-format: usable without httplog and tcplog
- BUG/MEDIUM: balance source did not properly hash IPv6 addresses
- MINOR: contrib/iprange: add a network IP range to mask converter
- MEDIUM: session: implement the "use-server" directive
- MEDIUM: log: add a new cookie flag 'U' to report situations where cookie is not used
- MEDIUM: http: make extract_cookie_value() iterate over cookie values
- MEDIUM: http: add cookie and scookie ACLs
- CLEANUP: lb_first: add reference to a paper describing the original idea
- MEDIUM: stream_sock: add a get_src and get_dst callback and remove SN_FRT_ADDR_SET
- BUG/MINOR: acl: req_ssl_sni would randomly fail if a session ID is present
- BUILD: http: make extract_cookie_value() return an int not size_t
- BUILD: http: stop gcc-4.1.2 from complaining about possibly uninitialized values
- CLEANUP: http: message parser must ignore HTTP_MSG_ERROR
- MINOR: standard: add a memprintf() function to build formatted error messages
- CLEANUP: remove a few warning about unchecked return values in debug code
- MEDIUM: move message-related flags from transaction to message
- DOC: add a diagram to explain how circular buffers work
- MAJOR: buffer rework: replace ->send_max with ->o
- MAJOR: buffer: replace buf->l with buf->{o+i}
- MINOR: buffers: provide simple pointer normalization functions
- MINOR: buffers: remove unused function buffer_contig_data()
- MAJOR: buffers: replace buf->w with buf->p - buf->o
- MAJOR: buffers: replace buf->r with buf->p + buf->i
- MAJOR: http: move buffer->lr to http_msg->next
- MAJOR: http: change msg->{som,col,sov,eoh} to be relative to buffer origin
- CLEANUP: http: remove unused http_msg->col
- MAJOR: http: turn http_msg->eol to a buffer-relative offset
- MEDIUM: http: add a pointer to the buffer in http_msg
- MAJOR: http: make http_msg->sol relative to buffer's origin
- MEDIUM: http: http_send_name_header: remove references to msg and buffer
- MEDIUM: http: remove buffer arg in a few header manipulation functions
- MEDIUM: http: remove buffer arg in http_capture_bad_message
- MEDIUM: http: remove buffer arg in http_msg_analyzer
- MEDIUM: http: remove buffer arg in http_upgrade_v09_to_v10
- MEDIUM: http: remove buffer arg in http_buffer_heavy_realign
- MEDIUM: http: remove buffer arg in chunk parsing functions
- MINOR: http: remove useless wrapping checks in http_msg_analyzer
- MEDIUM: buffers: fix unsafe use of buffer_ignore at some places
- MEDIUM: buffers: add new pointer wrappers and get rid of almost all buffer_wrap_add calls
- MEDIUM: buffers: implement b_adv() to advance a buffer's pointer
- MEDIUM: buffers: rename a number of buffer management functions
- MEDIUM: http: add a prefetch function for ACL pattern fetch
- MEDIUM: http: make all ACL fetch function use acl_prefetch_http()
- BUG/MINOR: http_auth: ACLs are volatile, not permanent
- MEDIUM: http/acl: merge all request and response ACL fetches of headers and cookies
- MEDIUM: http/acl: make acl_fetch_hdr_{ip,val} rely on acl_fetch_hdr()
- MEDIUM: add a new typed argument list parsing framework
- MAJOR: acl: make use of the new argument parsing framework
- MAJOR: acl: store the ACL argument types in the ACL keyword declaration
- MEDIUM: acl: acl_find_target() now resolves arguments based on their types
- MAJOR: acl: make acl_find_targets also resolve proxy names at config time
- MAJOR: acl: ensure that implicit table and proxies are valid
- MEDIUM: acl: remove unused tests for missing args when args are mandatory
- MEDIUM: pattern: replace type pattern_arg with type arg
- MEDIUM: pattern: get rid of arg_i in all functions making use of arguments
- MEDIUM: pattern: use the standard arg parser
- MEDIUM: pattern: add an argument validation callback to pattern descriptors
- MEDIUM: pattern: report the precise argument parsing error when known.
- MEDIUM: acl: remove the ACL_TEST_F_NULL_MATCH flag
- MINOR: pattern: add a new 'sample' type to store fetched data
- MEDIUM: pattern: add new sample types to replace pattern types
- MAJOR: acl: make use of the new sample struct and get rid of acl_test
- MEDIUM: pattern/acl: get rid of temp_pattern in ACLs
- MEDIUM: acl: get rid of the SET_RES flags
- MEDIUM: get rid of SMP_F_READ_ONLY and SMP_F_MUST_FREE
- MINOR: pattern: replace struct pattern with struct sample
- MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere
- MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description
- MEDIUM: acl/pattern: switch rdp_cookie functions stack up-down
- MEDIUM: acl: replace acl_expr with args in acl fetch_* functions
- MINOR: tcp: replace acl_fetch_rdp_cookie with smp_fetch_rdp_cookie
- MEDIUM: acl/pattern: use the same direction scheme
- MEDIUM: acl/pattern: start merging common sample fetch functions
- MEDIUM: pattern: ensure that sample types always cast into other types.
- MEDIUM: acl/pattern: factor out the src/dst address fetches
- MEDIUM: acl: implement payload and payload_lv
- CLEANUP: pattern: ensure that payload and payload_lv always stay in the buffer
- MINOR: stick_table: centralize the handling of empty keys
- MINOR: pattern: centralize handling of unstable data in pattern_process()
- MEDIUM: pattern: use smp_fetch_rdp_cookie instead of the pattern specific version
- MINOR: acl: set SMP_OPT_ITERATE on fetch functions
- MINOR: acl: add a val_args field to keywords
- MINOR: proto_tcp: validate arguments of payload and payload_lv ACLs
- MEDIUM: http: merge acl and pattern header fetch functions
- MEDIUM: http: merge ACL and pattern cookie fetches into a single one
- MEDIUM: acl: report parsing errors to the caller
- MINOR: arg: improve error reporting on invalid arguments
- MINOR: acl: report errors encountered when loading patterns from files
- MEDIUM: acl: extend the pattern parsers to report meaningful errors
- REORG: use the name "sample" instead of "pattern" to designate extracted data
- REORG: rename "pattern" files
- MINOR: acl: add types to ACL patterns
- MINOR: standard: add an IPv6 parsing function (str62net)
- MEDIUM: acl: support IPv6 address matching
- REORG: stream_interface: create a struct sock_ops to hold socket operations
- REORG/MEDIUM: move protocol->{read,write} to sock_ops
- REORG/MEDIUM: stream_interface: initialize socket ops from descriptors
- REORG/MEDIUM: replace stream interface protocol functions by a proto pointer
- REORG/MEDIUM: move the default accept function from sockstream to protocols.c
- MEDIUM: proto_tcp: remove src6 and dst6 pattern fetch methods
- BUG/MINOR: http: error snapshots are wrong if buffer wraps
- BUG/MINOR: http: ensure that msg->err_pos is always relative to buf->p
- MEDIUM: http: improve error capture reports
- MINOR: acl: add the cook_val() match to match a cookie against an integer
- BUG/MEDIUM: send_proxy: fix initialisation of send_proxy_ofs
- MEDIUM: memory: add the ability to poison memory at run time
- BUG/MEDIUM: log: ensure that unique_id is properly initialized
- MINOR: cfgparse: use a common errmsg pointer for all parsers
- MEDIUM: cfgparse: make backend_parse_balance() use memprintf to report errors
- MEDIUM: cfgparse: use the new error reporting framework for remaining cfg_keywords
- MINOR: http: replace http_message_realign() with buffer_slow_realign()
These methods have been superseded by src and dst which support
multiple families. There is no point keeping them since they appeared
in a development version anyway.
For configurations using "src6", please use "src" instead. For "dst6",
use "dst" instead.
Make use of the new IPv6 pattern type so that acl_match_ip() knows how to
compare pattern and sample.
IPv6 may be entered in their usual form, with or without a netmask appended.
Only bit counts are accepted for IPv6 netmasks. In order to avoid any risk of
trouble with randomly resolved IP addresses, host names are never allowed in
IPv6 patterns.
HAProxy is also able to match IPv4 addresses with IPv6 addresses in the
following situations :
- tested address is IPv4, pattern address is IPv4, the match applies
in IPv4 using the supplied mask if any.
- tested address is IPv6, pattern address is IPv6, the match applies
in IPv6 using the supplied mask if any.
- tested address is IPv6, pattern address is IPv4, the match applies in IPv4
using the pattern's mask if the IPv6 address matches with 2002:IPV4::,
::IPV4 or ::ffff:IPV4, otherwise it fails.
- tested address is IPv4, pattern address is IPv6, the IPv4 address is first
converted to IPv6 by prefixing ::ffff: in front of it, then the match is
applied in IPv6 using the supplied IPv6 mask.
It's easy to merge pattern and ACL fetches of cookies. It allows us
to remove two distinct fetch functions. The new function internally
uses an occurrence number to serve both purposes, but it didn't appear
worth exposing it outside so there is no keyword argument to set it.
However one of the benefits is that the "cookie" fetch for stick tables
now automatically adapts to requests and responses, so there is no more
need for set-cookie().
HTTP header fetch is now done using smp_fetch_hdr() for both ACLs and
patterns. This one also supports an occurrence number, making it possible
to specify explicit occurrences for ACLs and patterns.
These ones were easy to adapt to ACL usage and may really be useful,
so let's make them available right now. It's likely that some extension
such as regex, string-to-IP and raw IP matching will be implemented in
the near future.
src_port, dst_port and url_param have converged between ACLs and patterns.
This means that src_port is now available in patterns and that urlp_* has
been added to ACLs. Some code has moved to accommodate for static function
definitions, but there were little changes.
All ACLs which need to process HTTP contents first call this function which
performs all the preliminary tests and also triggers the request parsing if
needed. A macro was written to simplify the code.
As a side effect, it's not required anymore to check for the HTTP ACL before
checking for HTTP contents.
- Typo on "dispatch" keyword arguments.
- Reindent some blocks for better parsing by automated tools.
- "option mysql-check" and "option pgsql-check" arguments were not documented
as the others.
This patch is a group commit simplify the parsing of the documenation :
- remove remaining tabulations
- realign some lines
- break lines at 80 columns
- add missing chapters in the summary
- fix chapter numbering format
The Unique ID, is an ID generated with several informations. You can use
a log-format string to customize it, with the "unique-id-format" keyword,
and insert it in the request header, with the "unique-id-header" keyword.
%Fi: Frontend IP
%Fp: Frontend Port
%Si: Server IP
%Sp: Server Port
%Ts: Timestamp
%rt: HTTP request counter
%H: hostname
%pid: PID
+X: Hexadecimal represenation
The +X mode in logformat displays hexadecimal for the following flags
%Ci %Cp %Fi %Fp %Bi %Bp %Si %Sp %Ts %ct %pid
rename logformat_write_string() to lf_text()
Optimize size computation
The ACL matches rely on the extract_cookie_value() function as used for
for patterns. This permits ACLs to match cookie values based on the cookie
name instead of having to perform substring matching on the cookie header.
Sometimes it is desirable to forward a particular request to a specific
server without having to declare a dedicated backend for this server. This
can be achieved using the "use-server" rules. These rules are evaluated after
the "redirect" rules and before evaluating cookies, and they have precedence
on them. There may be as many "use-server" rules as desired. All of these
rules are evaluated in their declaration order, and the first one which
matches will assign the server.
Released version 1.5-dev8 with the following main changes :
- MINOR: patch for minor typo (ressources/resources)
- MEDIUM: http: add support for sending the server's name in the outgoing request
- DOC: mention that default checks are TCP connections
- BUG/MINOR: fix options forwardfor if-none when an alternative header name is specified
- CLEANUP: Make check_statuses, analyze_statuses and process_chk static
- CLEANUP: Fix HCHK spelling errors
- BUG/MINOR: fix typo in processing of http-send-name-header
- MEDIUM: log: Use linked lists for loggers
- BUILD: fix declaration inside a scope block
- REORG: log: split send_log function
- MINOR: config: Parse the string of the log-format config keyword
- MINOR: add ultoa, ulltoa, ltoa, lltoa implementations
- MINOR: Date and time fonctions that don't use snprintf
- MEDIUM: log: make http_sess_log use log_format
- DOC: log-format documentation
- MEDIUM: log: use log_format for mode tcplog
- MEDIUM: log-format: backend source address %Bi %Bp
- BUG/MINOR: log-format: fix %o flag
- BUG/MEDIUM: bad length in log_format and __send_log
- MINOR: logformat %st is signed
- BUILD/MINOR: fix the source URL in the spec file
- DOC: acl is http_first_req, not http_req_first
- BUG/MEDIUM: don't trim last spaces from headers consisting only of spaces
- MINOR: acl: add new matches for header/path/url length
- BUILD: halog: make halog build on solaris
- BUG/MINOR: don't use a wrong port when connecting to a server with mapped ports
- MINOR: remove the client/server side distinction in SI addresses
- MINOR: halog: add support for matching queued requests
- DOC: indicate that cookie "prefix" and "indirect" should not be mixed
- OPTIM/MINOR: move struct sockaddr_storage to the tail of structs
- OPTIM/MINOR: make it possible to change pipe size (tune.pipesize)
- BUILD/MINOR: silent a build warning in src/pipe.c (fcntl)
- OPTIM/MINOR: move the hdr_idx pools out of the proxy struct
- MEDIUM: tune.http.maxhdr makes it possible to configure the maximum number of HTTP headers
- BUG/MINOR: fix a segfault when parsing a config with undeclared peers
- CLEANUP: rename possibly confusing struct field "tracked"
- BUG/MEDIUM: checks: fix slowstart behaviour when server tracking is in use
- MINOR: config: tolerate server "cookie" setting in non-HTTP mode
- MEDIUM: buffers: add some new primitives and rework existing ones
- BUG: buffers: don't return a negative value on buffer_total_space_res()
- MINOR: buffers: make buffer_pointer() support negative pointers too
- CLEANUP: kill buffer_replace() and use an inline instead
- BUG: tcp: option nolinger does not work on backends
- CLEANUP: ebtree: remove a few annoying signedness warnings
- CLEANUP: ebtree: clarify licence and update to 6.0.6
- CLEANUP: ebtree: remove 4-year old harmless typo in duplicates insertion code
- CLEANUP: ebtree: remove another typo, a wrong initialization in insertion code
- BUG: ebtree: ebst_lookup() could return the wrong entry
- OPTIM: stream_sock: reduce the amount of in-flight spliced data
- OPTIM: stream_sock: save a failed recv syscall when splice returns EAGAIN
- MINOR: acl: add support for TLS server name matching using SNI
- BUG: http: re-enable TCP quick-ack upon incomplete HTTP requests
- BUG: proto_tcp: don't try to bind to a foreign address if sin_family is unknown
- MINOR: pattern: export the global temporary pattern
- CLEANUP: patterns: get rid of pattern_data_setstring()
- MEDIUM: acl: use temp_pattern to store fetched information in the "method" match
- MINOR: acl: include pattern.h to make pattern migration more transparent
- MEDIUM: pattern: change the pattern data integer from unsigned to signed
- MEDIUM: acl: use temp_pattern to store any integer-type information
- MEDIUM: acl: use temp_pattern to store any address-type information
- CLEANUP: acl: integer part of acl_test is not used anymore
- MEDIUM: acl: use temp_pattern to store any string-type information
- CLEANUP: acl: remove last data fields from the acl_test struct
- MEDIUM: http: replace get_ip_from_hdr2() with http_get_hdr()
- MEDIUM: patterns: the hdr() pattern is now of type string
- DOC: add minimal documentation on how ACLs work internally
- DOC: add a coding-style file
- OPTIM: halog: keep a fast path for the lines-count only
- CLEANUP: silence a warning when building on sparc
- BUG: http: tighten the list of allowed characters in a URI
- MEDIUM: http: block non-ASCII characters in URIs by default
- DOC: add some documentation from RFC3986 about URI format
- BUG/MINOR: cli: correctly remove the whole table on "clear table"
- BUG/MEDIUM: correctly disable servers tracking another disabled servers.
- BUG/MEDIUM: zero-weight servers must not dequeue requests from the backend
- MINOR: halog: add some help on the command line
- BUILD: fix build error on FreeBSD
- BUG: fix double free in peers config error path
- MEDIUM: improve config check return codes
- BUILD: make it possible to look for pcre in the default system paths
- MINOR: config: emit a warning when 'default_backend' masks servers
- MINOR: backend: rework the LC definition to support other connection-based algos
- MEDIUM: backend: add the 'first' balancing algorithm
- BUG: fix httplog trailing LF
- MEDIUM: increase chunk-size limit to 2GB-1
- BUG: queue: fix dequeueing sequence on HTTP keep-alive sessions
- BUG: http: disable TCP delayed ACKs when forwarding content-length data
- BUG: checks: fix server maintenance exit sequence
- BUG/MINOR: stream_sock: don't remove BF_EXPECT_MORE and BF_SEND_DONTWAIT on partial writes
- DOC: enumerate valid status codes for "observe layer7"
- MINOR: buffer: switch a number of buffer args to const
- CLEANUP: silence signedness warning in acl.c
- BUG: stream_sock: si->release was not called upon shutw()
- MINOR: log: use "%ts" to log term status only and "%tsc" to log with cookie
- BUG/CRITICAL: log: fix risk of crash in development snapshot
- BUG/MAJOR: possible crash when using capture headers on TCP frontends
- MINOR: config: disable header captures in TCP mode and complain
The difference could be seen when logging a request in HTTP mode with option
tcplog, as it would keep emitting 4 chars. Better use two distinct flags to
clear the confusion.
%Bi return the backend source IP
%Bp return the backend source port
Add a function pointer in logformat_type to do additional configuration
during the log-format variable parsing.
Merge http_sess_log() and tcp_sess_log() to sess_log() and move it to
log.c
A new field in logformat_type define if you can use a logformat
variable in TCP or HTTP mode.
doc: log-format in tcp mode
Note that due to the way log buffer allocation currently works, trying to
log an HTTP request without "option httplog" is still not possible. This
will change in the near future.
The principle behind this load balancing algorithm was first imagined
and modeled by Steen Larsen then iteratively refined through several
work sessions until it would totally address its original goal.
The purpose of this algorithm is to always use the smallest number of
servers so that extra servers can be powered off during non-intensive
hours. Additional tools may be used to do that work, possibly by
locally monitoring the servers' activity.
The first server with available connection slots receives the connection.
The servers are choosen from the lowest numeric identifier to the highest
(see server parameter "id"), which defaults to the server's position in
the farm. Once a server reaches its maxconn value, the next server is used.
It does not make sense to use this algorithm without setting maxconn. Note
that it can however make sense to use minconn so that servers are not used
at full load before starting new servers, and so that introduction of new
servers requires a progressively increasing load (the number of servers
would more or less follow the square root of the load until maxconn is
reached). This algorithm ignores the server weight, and is more beneficial
to long sessions such as RDP or IMAP than HTTP, though it can be useful
there too.
These ones are invalid and blocked unless "option accept-invalid-http-request"
is specified in the frontend. In any case, the faulty request is logged.
Note that some of the remaining invalid chars are still not checked against,
those are the invalid ones between 32 and 127 :
34 ('"'), 60 ('<'), 62 ('>'), 92 ('\'), 94 ('^'),
96 ('`'), 123 ('{'), 124 ('|'), 125 ('}')
Using a lookup table might be better at some point.
New option "http-send-name-header" specifies the name of a header which
will hold the server name in outgoing requests. This is the name of the
server the connection is really sent to, which means that upon redispatches,
the header's value is updated so that it always matches the server's name.
This pattern previously was limited to type IP. With the new header
extraction function, it becomes possible to extract strings, so that
the header can be returned as a string. This will not change anything
to existing configs, as string will automatically be converted to IP
when needed. However, new configs will be able to use IPv6 addresses
from headers in stick-tables, as well as stick on any non-IP header
(eg: host, user-agent, ...).
Server Name Indication (SNI) is a TLS extension which makes a client
present the name of the server it is connecting to in the client hello.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.
The new ACL "req_ssl_sni" matches the name extracted from the TLS
handshake against a list of names which may be loaded from a file if
needed.
This patch settles the 2 loggers limitation.
Loggers are now stored in linked lists.
Using "global log", the global loggers list content is added at the end
of the current proxy list. Each "log" entries are added at the end of
the proxy list.
"no log" flush a logger list.
For a long time, the max number of headers was taken as a part of the buffer
size. Since the header size can be configured at runtime, it does not make
much sense anymore.
Nothing was making it necessary to have a static value, so let's turn this into
a tunable with a default value of 101 which equals what was previously used.
By default, pipes are the default size for the system. But sometimes when
using TCP splicing, it can improve performance to increase pipe sizes,
especially if it is suspected that pipes are not filled and that many
calls to splice() are performed. This has an impact on the kernel's
memory footprint, so this must not be changed if impacts are not understood.
When prefix and indirect are used together, a client which connects to
a server with a cookie will never get any cookie update from this server,
which will be removed by the "indirect" option.
This patch introduces hdr_len, path_len and url_len for matching these
respective parts lengths against integers. This can be used to detect
abuse or empty headers.
Released version 1.5-dev7 with the following main changes :
- [BUG] fix binary stick-tables
- [MINOR] http: *_dom matching header functions now also split on ":"
- [BUG] checks: fix support of Mysqld >= 5.5 for mysql-check
- [MINOR] acl: add srv_conn acl to count connections on a specific backend server
- [MINOR] check: add redis check support
- [DOC] small fixes to clearly distinguish between keyword and variables
- [MINOR] halog: add support for termination code matching (-tcn/-TCN)
- [DOC] Minor spelling fixes and grammatical enhancements
- [CLEANUP] dumpstats: make symbols static where possible
- [MINOR] Break out dumping table
- [MINOR] Break out processing of clear table
- [MINOR] Allow listing of stick table by key
- [MINOR] Break out all stick table socat command parsing
- [MINOR] More flexible clearing of stick table
- [MINOR] Allow showing and clearing by key of ipv6 stick tables
- [MINOR] Allow showing and clearing by key of integer stick tables
- [MINOR] Allow showing and clearing by key of string stick tables
- [CLEANUP] Remove assigned but unused variables
- [CLEANUP] peers.h: fix declarations
- [CLEANUP] session.c: Make functions static where possible
- [MINOR] Add active connection list to server
- [MINOR] Allow shutdown of sessions when a server becomes unavailable
- [MINOR] Add down termination condition
- [MINOR] Make appsess{,ion}_refresh static
- [MINOR] Add rdp_cookie pattern fetch function
- [CLEANUP] Remove unnecessary casts
- [MINOR] Add non-stick server option
- [MINOR] Consistently use error in tcp_parse_tcp_req()
- [MINOR] Consistently free expr on error in cfg_parse_listen()
- [MINOR] Free rdp_cookie_name on denint()
- [MINOR] Free tcp rules on denint()
- [MINOR] Free stick table pool on denint()
- [MINOR] Free stick rules on denint()
- [MEDIUM] Fix stick-table replication on soft-restart
- [MEDIUM] Correct ipmask() logic
- [MINOR] Correct type in table dump examples
- [MINOR] Fix build error in stream_int_register_handler()
- [MINOR] Use DPRINTF in assign_server()
- [BUG] checks: http-check expect could fail a check on multi-packet responses
- [DOC] fix minor typo in the "dispatch" doc
- [BUG] proto_tcp: fix address binding on remote source
- [MINOR] http: don't report the "haproxy" word on the monitoring response
- [REORG] http: move HTTP error codes back to proto_http.h
- [MINOR] http: make the "HTTP 200" status code configurable.
- [MINOR] http: partially revert the chunking optimization for now
- [MINOR] stream_sock: always clear BF_EXPECT_MORE upon complete transfer
- [CLEANUP] stream_sock: remove unneeded FL_TCP and factor out test
- [MEDIUM] http: add support for "http-no-delay"
- [OPTIM] http: optimize chunking again in non-interactive mode
- [OPTIM] stream_sock: avoid fast-forwarding of partial data
- [OPTIM] stream_sock: don't use splice on too small payloads
- [MINOR] config: make it possible to specify a cookie even without a server
- [BUG] stats: support url-encoded forms
- [MINOR] config: automatically compute a default fullconn value
- [CLEANUP] config: remove some left-over printf debugging code from previous patch
- [DOC] add missing entry or stick store-response
- [MEDIUM] http: add support for 'cookie' and 'set-cookie' patterns
- [BUG] halog: correctly handle truncated last line
- [MINOR] halog: make SKIP_CHAR stop on field delimiters
- [MINOR] halog: add support for HTTP log matching (-H)
- [MINOR] halog: gain back performance before SKIP_CHAR fix
- [OPTIM] halog: cache some common fields positions
- [OPTIM] halog: check once for correct line format and reuse the pointer
- [OPTIM] halog: remove many 'if' by using a function pointer for the filters
- [OPTIM] halog: remove support for tab delimiters in input data
- [BUG] session: risk of crash on out of memory (1.5-dev regression)
- [MINOR] session: try to emit a 500 response on memory allocation errors
- [OPTIM] stream_sock: reduce the default number of accepted connections at once
- [BUG] stream_sock: disable listener when system resources are exhausted
- [MEDIUM] proxy: add a PAUSED state to listeners and move socket tricks out of proxy.c
- [BUG] stream_sock: ensure orphan listeners don't accept too many connections
- [MINOR] listeners: add listen_full() to mark a listener full
- [MINOR] listeners: add support for queueing resource limited listeners
- [MEDIUM] listeners: put listeners in queue upon resource shortage
- [MEDIUM] listeners: queue proxy-bound listeners at the proxy's
- [MEDIUM] listeners: don't stop proxies when global maxconn is reached
- [MEDIUM] listeners: don't change listeners states anymore in maintain_proxies
- [CLEANUP] proxy: rename a few proxy states (PR_STIDLE and PR_STRUN)
- [MINOR] stats: report a "WAITING" state for sockets waiting for resource
- [MINOR] proxy: make session rate-limit more accurate
- [MINOR] sessions: only wake waiting listeners up if rate limit is OK
- [BUG] proxy: peers must only be stopped once, not upon every call to maintain_proxies
- [CLEANUP] proxy: merge maintain_proxies() operation inside a single loop
- [MINOR] task: new function task_schedule() to schedule a wake up
- [MAJOR] proxy: finally get rid of maintain_proxies()
- [BUG] proxy: stats frontend and peers were missing many initializers
- [MEDIUM] listeners: add a global listener management task
- [MINOR] proxy: make findproxy() return proxies from numeric IDs too
- [DOC] fix typos, "#" is a sharp, not a dash
- [MEDIUM] stats: add support for changing frontend's maxconn at runtime
- [MEDIUM] checks: group health checks methods by values and save option bits
- [MINOR] session-counters: add the ability to clear the counters
- [BUG] check: http-check expect + regex would crash in defaults section
- [MEDIUM] http: make x-forwarded-for addition conditional
- [REORG] build: move syscall redefinition to specific places
- [CLEANUP] update the year in the copyright banner
- [BUG] possible crash in 'show table' on stats socket
- [BUG] checks: use the correct destination port for sending checks
- [BUG] backend: risk of picking a wrong port when mapping is used with crossed families
- [MINOR] make use of set_host_port() and get_host_port() to get rid of family mismatches
- [DOC] fixed a few "sensible" -> "sensitive" errors
- [MINOR] make use of addr_to_str() and get_host_port() to replace many inet_ntop()
- [BUG] http: trailing white spaces must also be trimmed after headers
- [MINOR] stats: display "<NONE>" instead of the frontend name when unknown
- [MINOR] http: take a capture of too large requests and responses
- [MINOR] http: take a capture of truncated responses
- [MINOR] http: take a capture of bad content-lengths.
- [DOC] add a few old and uncommitted docs
- [CLEANUP] cfgparse: fix reported options for the "bind" keyword
- [MINOR] halog: add -hs/-HS to filter by HTTP status code range
- [MINOR] halog: support backslash-escaped quotes
- [CLEANUP] remove dirty left-over of a debugging message
- [MEDIUM] stats: disable complex socket reservation for stats socket
- [CLEANUP] remove a useless test in manage_global_listener_queue()
- [MEDIUM] stats: add the "set maxconn" setting to the command line interface
- [MEDIUM] add support for global.maxconnrate to limit the per-process conn rate.
- [MINOR] stats: report the current and max global connection rates
- [MEDIUM] stats: add the ability to adjust the global maxconnrate
- [BUG] peers: don't pre-allocate 65000 connections to each peer
- [MEDIUM] don't limit peers nor stats socket to maxconn nor maxconnrate
- [BUG] peers: the peer frontend must not emit any log
- [CLEANUP] proxy: make pause_proxy() perform the required controls and emit the logs
- [BUG] peers: don't keep a peers section which has a NULL frontend
- [BUG] peers: ensure the peers are resumed if they were paused
- [MEDIUM] stats: add the ability to enable/disable/shutdown a frontend at runtime
- [MEDIUM] session: make session_shutdown() an independant function
- [MEDIUM] stats: offer the possibility to kill a session from the CLI
- [CLEANUP] stats: centralize tests for backend/server inputs on the CLI
- [MEDIUM] stats: offer the possibility to kill sessions by server
- [MINOR] halog: do not consider byte 0x8A as end of line
- [MINOR] frontend: ensure debug message length is always initialized
- [OPTIM] halog: make fgets parse more bytes by blocks
- [OPTIM] halog: add assembly version of the field lookup code
- [MEDIUM] poll: add a measurement of idle vs work time
- [CLEANUP] startup: report only the basename in the usage message
- [MINOR] startup: add an option to change to a new directory
- [OPTIM] task: don't scan the run queue if we know it's empty
- [BUILD] stats: stdint is not present on solaris
- [DOC] update the README file to reflect new naming rules for patches
- [MINOR] stats: report the number of requests intercepted by the frontend
- [DOC] update ROADMAP file
We already had the ability to kill a connection, but it was only
for the checks. Now we can do this for any session, and for this we
add a specific flag "K" to the logs.
The stats socket now allows the admin to disable, enable or shutdown a frontend.
This can be used when a bug is discovered in a configuration and it's desirable
to fix it but the rules in place don't allow to change a running config. Thus it
becomes possible to kill the frontend to release the port and start a new one in
a separate process.
This can also be used to temporarily make haproxy return TCP resets to incoming
requests to pretend the service is not bound. For instance, this may be useful
to quickly flush a very deep SYN backlog.
The frontend check and lookup code was factored with the "set maxconn" usage.
This one enforces a per-process connection rate limit, regardless of what
may be set per frontend. It can be a way to limit the CPU usage of a process
being severely attacked.
The side effect is that the global process connection rate is now measured
for each incoming connection, so it will be possible to report it.
This option permits to change the global maxconn setting within the
limit that was set by the initial value, which is now reported as the
hard maxconn value. This allows to immediately accept more concurrent
connections or to stop accepting new ones until the value passes below
the indicated setting.
The main use of this option is on systems where many haproxy instances
are loaded and admins need to re-adjust resource sharing at run time
to regain a bit of fairness between processes.
If "option forwardfor" has the "if-none" argument, then the header is
only added when the request did not already have one. This option has
security implications, and should not be set blindly.
Manoj Kumar reported a case where haproxy would crash upon start-up. The
cause was an "http-check expect" statement declared in the defaults section,
which caused a NULL regex to be used during the check. This statement is not
allowed in defaults sections precisely because this requires saving a copy
of the regex in the default proxy. But the check was not made to prevent it
from being declared there, hence the issue.
Instead of adding code to detect its abnormal use, we decided to implement
it. It was not that much complex because the expect_str part was not used
with regexes, so it could hold the string form of the regex in order to
compile it again for every backend (there's no way to clone regexes).
This patch has been tested and works. So it's both a bugfix and a minor
feature enhancement.
It should be backported to 1.4 though it's not critical since the config
was not supposed to be supported.
This patch provides a new "option redis-check" statement to enable server health checks based on redis PING request (http://www.redis.io/commands/ping).
The new "set maxconn frontend XXX" statement on the stats socket allows
the admin to change a frontend's maxconn value. If some connections are
queued, they will immediately be accepted up to the new limit. If the
limit is lowered, new connections acceptation might be delayed. This can
be used to temporarily reduce or increase the impact of a specific frontend's
traffic on the whole process.
This is used to perform cookie-based stickiness with table replication
between multiple masters and across restarts. This partially overrides
some of the appsession capabilities.
Never add connections allocated to this sever to a stick-table.
This may be used in conjunction with backup to ensure that
stick-table persistence is disabled for backup servers.
This pattern fetch function extracts the value of the rdp cookie <name> as
a string and uses this value to match. This enables implementation of
persistence based on the mstshash cookie. This is typically done if there
is no msts cookie present.
This differs from "balance rdp-cookie" in that any balancing algorithm may
be used and thus the distribution of clients to backend servers is not
linked to a hash of the RDP cookie. It is envisaged that using a balancing
algorithm such as "balance roundrobin" or "balance leastconnect" will lead
to a more even distribution of clients to backend servers than the hash
used by "balance rdp-cookie".
Example :
listen tse-farm
bind 0.0.0.0:3389
# wait up to 5s for an RDP cookie in the request
tcp-request inspect-delay 5s
tcp-request content accept if RDP_COOKIE
# apply RDP cookie persistence
persist rdp-cookie
# Persist based on the mstshash cookie
# This is only useful makes sense if
# balance rdp-cookie is not used
stick-table type string size 204800
stick on rdp_cookie(mstshash)
server srv1 1.1.1.1:3389
server srv1 1.1.1.2:3389
If a connection is closed by because the backend became unavailable
then log 'D' as the termination condition.
Signed-off-by: Simon Horman <horms@verge.net.au>
This adds the "on-marked-down shutdown-sessions" statement on "server" lines,
which causes all sessions established on a server to be killed at once when
the server goes down. The task's priority is reniced to the highest value
(1024) so that servers holding many tasks don't cause a massive slowdown due
to the wakeup storm.
mysqld >= 5.5 want the client to announce 4.1+ authentication support, even if we have no password, so we do this.
I also check on a debian potato mysqld 3.22 and it works too so i assume we are good from 3.22 to 5.5.
[WT: this must be backported to 1.4]
The fullconn value is not easy to get right when doing dynamic regulation,
as it should depend on the maxconns of the frontends that can reach a
backend. Since the parameter is mandatory, many configs are found with
an inappropriate default value.
Instead of rejecting configs without a fullconn value, we now set it to
10% of the sum of the configured maxconns of all the frontends which are
susceptible to branch to the backend. That way if new frontends are added,
the backend's fullconn automatically adjusts itself.
There are some very rare server-to-server applications that abuse the HTTP
protocol and expect the payload phase to be highly interactive, with many
interleaved data chunks in both directions within a single request. This is
absolutely not supported by the HTTP specification and will not work across
most proxies or servers. When such applications attempt to do this through
haproxy, it works but they will experience high delays due to the network
optimizations which favor performance by instructing the system to wait for
enough data to be available in order to only send full packets. Typical
delays are around 200 ms per round trip. Note that this only happens with
abnormal uses. Normal uses such as CONNECT requests nor WebSockets are not
affected.
When "option http-no-delay" is present in either the frontend or the backend
used by a connection, all such optimizations will be disabled in order to
make the exchanges as fast as possible. Of course this offers no guarantee on
the functionality, as it may break at any other place. But if it works via
HAProxy, it will work as fast as possible. This option should never be used
by default, and should never be used at all unless such a buggy application
is discovered. The impact of using this option is an increase of bandwidth
usage and CPU usage, which may significantly lower performance in high
latency environments.
This change should be backported to 1.4 since the first report of such a
misuse was in 1.4. Next patch will also be needed.
This status code is used in response to requests matching "monitor-uri".
Some users need to adjust it to fit their needs (eg: make some strings
appear there). As it's already defined as a chunked string and used
exactly like other status codes, it makes sense to make it configurable
with the usual "errorfile", "errorloc", ...
Released version 1.5-dev6 with the following main changes :
- [BUG] stream_sock: use get_addr_len() instead of sizeof() on sockaddr_storage
- [BUG] TCP source tracking was broken with IPv6 changes
- [BUG] stick-tables did not work when converting IPv6 to IPv4
- [CRITICAL] fix risk of crash when dealing with space in response cookies
Released version 1.5-dev5 with the following main changes :
- [BUG] standard: is_addr return value for IPv4 was inverted
- [MINOR] update comment about IPv6 support for server
- [MEDIUM] use getaddrinfo to resolve names if gethostbyname fail
- [DOC] update IPv6 support for bind
- [DOC] document IPv6 support for server
- [DOC] fix a minor typo
- [MEDIUM] IPv6 support for syslog
- [DOC] document IPv6 support for syslog
- [MEDIUM] IPv6 support for stick-tables
- [DOC] document IPv6 support for stick-tables
- [DOC] update ROADMAP file
- [BUG] session: src_conn_cur was returning src_conn_cnt instead
- [MINOR] frontend: add a make_proxy_line function
- [MEDIUM] stream_sock: add support for sending the proxy protocol header line
- [MEDIUM] server: add support for the "send-proxy" option
- [DOC] update the spec on the proxy protocol
- [BUILD] proto_tcp: fix build issue with CTTPROXY
- [DOC] update ROADMAP file
- [MEDIUM] config: rework the IPv4/IPv6 address parser to support host-only addresses
- [MINOR] cfgparse: better report wrong listening addresses and make use of str2sa_range
- [BUILD] add the USE_GETADDRINFO build option
- [TESTS] provide a test case for various address formats
- [BUG] session: conn_retries was not always initialized
- [BUG] log: retrieve the target from the session, not the SI
- [BUG] http: fix possible incorrect forwarded wrapping chunk size (take 2)
- [MINOR] tools: add two macros MID_RANGE and MAX_RANGE
- [BUG] http: fix content-length handling on 32-bit platforms
- [OPTIM] buffers: uninline buffer_forward()
- [BUG] stream_sock: fix handling for server side PROXY protocol
- [MINOR] acl: add support for table_cnt and table_avl matches
- [DOC] update ROADMAP file
This option enables use of the PROXY protocol with the server, which
allows haproxy to transport original client's address across multiple
architecture layers.
Released version 1.5-dev4 with the following main changes :
- [MINOR] cfgparse: Check whether the path given for the stats socket actually fits into the sockaddr_un structure to avoid truncation.
- [MINOR] unix sockets : inherits the backlog size from the listener
- [CLEANUP] unix sockets : move create_uxst_socket() in uxst_bind_listener()
- [DOC] fix a minor typo
- [DOC] fix ignore-persist documentation
- [MINOR] add warnings on features not compatible with multi-process mode
- [BUG] http: fix http-pretend-keepalive and httpclose/tunnel mode
- [MINOR] stats: add support for several packets in stats admin
- [BUG] stats: admin commands must check the proxy state
- [BUG] stats: admin web interface must check the proxy state
- [MINOR] http: add pattern extraction method to stick on query string parameter
- [MEDIUM] add internal support for IPv6 server addresses
- [MINOR] acl: add be_id/srv_id to match backend's and server's id
- [MINOR] log: add support for passing the forwarded hostname
- [MINOR] log: ability to override the syslog tag
- [MINOR] checks: add PostgreSQL health check
- [DOC] update ROADMAP file
- [BUILD] pattern: use 'int' instead of 'int32_t'
- [OPTIM] linux: add support for bypassing libc to force using vsyscalls
- [BUG] debug: report the correct poller list in verbose mode
- [BUG] capture: do not capture a cookie if there is no memory left
- [BUG] appsession: fix possible double free in case of out of memory
- [CRITICAL] cookies: mixing cookies in indirect mode and appsession can crash the process
- [BUG] http: correctly update the header list when removing two consecutive headers
- [BUILD] add the CPU=native and ARCH=32/64 build options
- [BUILD] add -fno-strict-aliasing to fix warnings with gcc >= 4.4
- [CLEANUP] hash: move the avalanche hash code globally available
- [MEDIUM] hash: add support for an 'avalanche' hash-type
- [DOC] update roadmap file
- [BUG] http: do not re-enable the PROXY analyser on keep-alive
- [OPTIM] http: don't send each chunk in a separate packet
- [DOC] fix minor typos reported recently in the peers section
- [DOC] fix another typo in the doc
- [MINOR] stats: report HTTP message state and buffer flags in error dumps
- [BUG] http chunking: don't report a parsing error on connection errors
- [BUG] stream_interface: truncate buffers when sending error messages
- [MINOR] http: support wrapping messages in error captures
- [MINOR] http: capture incorrectly chunked message bodies
- [MINOR] stats: add global event ID and count
- [BUG] http: analyser optimizations broke pipelining
- [CLEANUP] frontend: only apply TCP-specific settings to TCP/TCP6 sockets
- [BUG] http: fix incorrect error reporting during data transfers
- [CRITICAL] session: correctly leave turn-around and queue states on abort
- [BUG] session: release slot before processing pending connections
- [MINOR] tcp: add support for dynamic MSS setting
- [BUG] stick-table: correctly terminate string keys during lookups
- [BUG] acl: fix handling of empty lines in pattern files
- [BUG] stick-table: use the private buffer when padding strings
- [BUG] ebtree: fix ebmb_lookup() with len smaller than the tree's keys
- [OPTIM] ebtree: ebmb_lookup: reduce stack usage by moving the return code out of the loop
- [OPTIM] ebtree: inline ebst_lookup_len and ebis_lookup_len
- [REVERT] undo the stick-table string key lookup fixes
- [MINOR] http: improve url_param pattern extraction to ignore empty values
- [BUILD] frontend: shut a warning with TCP_MAXSEG
- [BUG] http: update the header list's tail when removing the last header
- [DOC] fix minor typo in the proxy protocol doc
- [DOC] fix typos (http-request instead of http-check)
- [BUG] http: use correct ACL pointer when evaluating authentication
- [BUG] cfgparse: correctly count one socket per port in ranges
- [BUG] startup: set the rlimits before binding ports, not after.
- [BUG] acl: srv_id must return no match when the server is NULL
- [MINOR] acl: add ability to check for internal response-only parameters
- [MINOR] acl: srv_id is only valid in responses
- [MINOR] config: warn if response-only conditions are used in "redirect" rules
- [BUG] acl: fd leak when reading patterns from file
- [DOC] fix minor typo in "usesrc"
- [BUG] http: fix possible incorrect forwarded wrapping chunk size
- [BUG] http: fix computation of message body length after forwarding has started
- [BUG] http: balance url_param did not work with first parameters on POST
- [TESTS] update the url_param regression test to test check_post too
- [DOC] update ROADMAP
- [DOC] internal: reflect the fact that SI_ST_ASS is transient
- [BUG] config: don't crash on empty pattern files.
- [MINOR] stream_interface: make use of an applet descriptor for IO handlers
- [REORG] stream_interface: move the st0, st1 and private members to the applet
- [REORG] stream_interface: split the struct members in 3 parts
- [REORG] session: move client and server address to the stream interface
- [REORG] tcp: make tcpv4_connect_server() take the target address from the SI
- [MEDIUM] stream_interface: store the target pointer and type
- [CLEANUP] stream_interface: remove the applet.handler pointer
- [MEDIUM] log: take the logged server name from the stream interface
- [CLEANUP] session: remove data_source from struct session
- [CLEANUP] stats: make all dump functions only rely on the stream interface
- [REORG] session: move the data_ctx struct to the stream interface's applet
- [MINOR] proxy: add PR_O2_DISPATCH to detect dispatch mode
- [MINOR] cfgparse: only keep one of dispatch, transparent, http_proxy
- [MINOR] session: add a pointer to the new target into the session
- [MEDIUM] session: remove s->prev_srv which is not needed anymore
- [CLEANUP] stream_interface: use inline functions to manipulate targets
- [MAJOR] session: remove the ->srv pointer from struct session
- [MEDIUM] stats: split frontend and backend stats
- [MEDIUM] http: always evaluate http-request rules before stats http-request
- [REORG] http: move the http-request rules to proto_http
- [BUG] http: stats were not incremented on http-request deny
- [MINOR] checks: report it if checks fail due to socket creation error
Bryan Talbot reported that POST requests with a query string were not
correctly processed if the hash parameter was the first one, because
the delimiter that was looked for to trigger the parsing was '&' instead
of '?'.
Also, while checking the code, it became apparent that it was enough for
a query string to be present in the request for POST parameters to be
ignored, even if the url_param was in the body and not in the URL.
The code has then been fixed like this :
1) look for URL param. If found, return it.
2) if no URL param was found and method is POST, then look it up into
the body
The code now seems to pass all request combinations.
This patch must be backported to 1.4 since 1.4 is equally broken right now.
Some browsers send POST requests in several packets, which was not supported
by the "stats admin" function.
This patch allows to wait for more data when they are not fully received
(we are still limited to a certain size defined by the buffer size minus its
reserved space).
It also adds support for the "Expect: 100-Continue" header.
I have written a small patch to enable a correct PostgreSQL health check
It works similar to mysql-check with the very same parameters.
E.g.:
listen pgsql 127.0.0.1:5432
mode tcp
option pgsql-check user pgsql
server masterdb pgsql.server.com:5432 check inter 10000
One of the requirements we have is to run multiple instances of haproxy on a
single host; this is so that we can split the responsibilities (and change
permissions) between product teams. An issue we ran up against is how we
would distinguish between the logs generated by each instance. The solution
we came up with (please let me know if there is a better way) is to override
the application tag written to syslog. We can then configure syslog to write
these to different files.
I have attached a patch adding a global option 'log-tag' to override the
default syslog tag 'haproxy' (actually defaults to argv[0]).
By passing a negative value to the "mss" argument of "bind" lines, it
becomes possible to subtract this value to the MSS advertised by the
client, which results in segments smaller than advertised. The effect
is useful with some TCP stacks which ACK less often when segments are
not full, because they only ACK every other full segment as suggested
by RFC1122.
NOTE: currently this has no effect on Linux kernel 2.6, a kernel patch
is still required to change the MSS of established connections.
Haproxy does not include the hostname rather the IP of the machine in
the syslog headers it sends. Unfortunately this means that for each log
line rsyslog does a reverse dns on the client IP and in the case of
non-routable IPs one gets the public hostname not the internal one.
While this is valid according to RFC3164 as one might imagine this is
troublsome if you have some machines with public IPs, internal IPs, no
reverse DNS entries, etc and you want a standardized hostname based log
directory structure. The rfc says the preferred value is the hostname.
This patch adds a global "log-send-hostname" statement which accepts an
optional string to force the host name. If unset, the local host name
is used.
We've had several issues related to data transfers. First, if a
client aborted an upload before the server started to respond, it
would get a 502 followed by a 400. The same was true (in the other
way around) if the server suddenly aborted while the client was
uploading the data.
The flags reported in the logs were misleading. Request errors could
be reported while the transfer was stopped during the data phase. The
status codes could also be overwritten by a 400 eventhough the start
of the response was transferred to the client.
The stats were also wrong in case of data aborts. The server or the
client could sometimes be miscredited for being the author of the
abort depending on where the abort was detected. Some client aborts
could also be accounted as request errors and some server aborts as
response errors.
Now it seems like all such issues are fixed. Since we don't have a
specific state for data flowing from the client to the server
before the server responds, we're still counting the client aborted
transfers as "CH", and they become "CD" when the server starts to
respond. Ideally a "P" state would be desired.
This patch should be backported to 1.4.
Using haproxy in multi-process mode (nbproc > 1), some features can be
not fully compatible or not work at all. haproxy will now display a warning on
startup for :
- appsession
- sticking rules
- stats / stats admin
- stats socket
- peers (fatal error in that case)
During the documentation of the "ignore-persist" keyword, I documented an
invalid "option ignore-persist" and forgot to remove it. It's time to fix it.
When the number of servers is a multiple of the size of the input set,
map-based hash can be inefficient. This typically happens with 64
servers when doing URI hashing. The "avalanche" hash-type applies an
avalanche hash before performing a map lookup in order to smooth the
distribution. The result is slightly less smooth than the map for small
numbers of servers, but still better than the consistent hashing.
Released version 1.5-dev3 with the following main changes :
- [DOC] fix http-request documentation
- [MEDIUM] enable/disable servers from the stats web interface
- [MEDIUM] stats: add an admin level
- [DOC] stats: document the "stats admin" statement
- [MINOR] startup: print the proxy socket which caused an error
- [CLEANUP] Remove unneeded chars allocation
- [MINOR] config: detect options not supported due to compilation options
- [MINOR] Add pattern's fetchs payload and payload_lv
- [MINOR] frontend: improve accept-proxy header parsing
- [MINOR] frontend: add tcpv6 support on accept-proxy bind
- [MEDIUM] Enhance message errors management on binds
- [MINOR] Manage unix socket source field on logs
- [MINOR] Manage unix socket source field on session dump on sock stats
- [MINOR] Support of unix listener sockets for debug and log event messages on frontend.c
- [MINOR] Add some tests on sockets family for port remapping and mode transparent.
- [MINOR] Manage socket type unix for some logs
- [MINOR] Enhance controls of socket's family on acls and pattern fetch
- [MINOR] Support listener's sockets unix on http logs.
- [MEDIUM] Add supports of bind on unix sockets.
- [BUG] stick table purge failure if size less than 255
- [BUG] stick table entries expire on counters updates/read or show table, even if there is no "expire" parameter
- [MEDIUM] Implement tcp inspect response rules
- [DOC] tcp-response content and inspect
- [MINOR] new acls fetch req_ssl_hello_type and rep_ssl_hello_type
- [DOC] acls rep_ssl_hello and req_ssl_hello
- [MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated.
- [DOC] new type binary in stick-table
- [DOC] stick store-response and new patterns payload and payload_lv
- [MINOR] Manage all types (ip, integer, string, binary) on cli "show table" command
- [MEDIUM] Create updates tree on stick table to manage sync.
- [MAJOR] Add new files src/peer.c, include/proto/peers.h and include/types/peers.h for sync stick table management
- [MEDIUM] Manage peers section parsing and stick table registration on peers.
- [MEDIUM] Manage soft stop on peers proxy
- [DOC] add documentation for peers section
- [MINOR] checks: add support for LDAPv3 health checks
- [MINOR] add better support to "mysql-check"
- [BUG] Restore info about available active/backup servers
- [CONTRIB] Update haproxy.pl
- [CONTRIB] Update Cacti Tempates
- [CONTRIB] add templates for Cacti.
- [BUG] http: don't consider commas as a header delimitor within quotes
- [MINOR] support a global jobs counter
- [DOC] add a summary about cookie incompatibilities between specs and browsers
- [DOC] fix description of cookie "insert" and "indirect" modes
- [MEDIUM] http: fix space handling in the request cookie parser
- [MEDIUM] http: fix space handling in the response cookie parser
- [DOC] fix typo in the queue() definition (backend, not frontend)
- [BUG] deinit: unbind listeners before freeing them
- [BUG] stream_interface: only call si->release when both dirs are closed
- [MEDIUM] buffers: rework the functions to exchange between SI and buffers
- [DOC] fix typo in the avg_queue() and be_conn() definition (backend, not frontend)
- [MINOR] halog: add '-tc' to sort by termination codes
- [MINOR] halog: skip non-traffic logs for -st and -tc
- [BUG] stream_sock: cleanly disable the listener in case of resource shortage
- [BUILD] stream_sock: previous fix lacked the #include, causing a warning.
- [DOC] bind option is "defer-accept", not "defer_accept"
- [DOC] missing index entry for http-check send-state
- [DOC] tcp-request inspect-delay is for backends too
- [BUG] ebtree: string_equal_bits() could return garbage on identical strings
- [BUG] stream_sock: try to flush any extra pending request data after a POST
- [BUILD] proto_http: eliminate some build warnings with gcc-2.95
- [MEDIUM] make it possible to combine http-pretend-keepalived with httpclose
- [MEDIUM] tcp-request : don't wait for inspect-delay to expire when the buffer is full
- [MEDIUM] checks: add support for HTTP contents lookup
- [TESTS] add test-check-expect to test various http-check methods
- [MINOR] global: add "tune.chksize" to change the default check buffer size
- [MINOR] cookie: add options "maxidle" and "maxlife"
- [MEDIUM] cookie: support client cookies with some contents appended to their value
- [MINOR] http: make some room in the transaction flags to extend cookies
- [MINOR] cookie: add the expired (E) and old (O) flags for request cookies
- [MEDIUM] cookie: reassign set-cookie status flags to store more states
- [MINOR] add encode/decode function for 30-bit integers from/to base64
- [MEDIUM] cookie: check for maxidle and maxlife for incoming dated cookies
- [MEDIUM] cookie: set the date in the cookie if needed
- [DOC] document the cookie maxidle and maxlife parameters
- [BUG] checks: don't log backend down for all zero-weight servers
- [MEDIUM] checks: set server state to one state from failure when leaving maintenance
- [BUG] config: report correct keywords for "observe"
- [MINOR] checks: ensure that we can inherit binary checks from the defaults section
- [MINOR] acl: add the http_req_first match
- [DOC] fix typos about bind-process syntax
- [BUG] cookie: correctly unset default cookie parameters
- [MINOR] cookie: add support for the "preserve" option
- [BUG] ebtree: fix duplicate strings insertion
- [CONTRIB] halog: report per-url counts, errors and times
- [CONTRIB] halog: minor speed improvement in timer parser
- [MINOR] buffers: add a new request analyser flag for PROXY mode
- [MINOR] listener: add the "accept-proxy" option to the "bind" keyword
- [MINOR] standard: add read_uint() to parse a delimited unsigned integer
- [MINOR] standard: change arg type from const char* to char*
- [MINOR] frontend: add a new analyser to parse a proxied connection
- [MEDIUM] session: call the frontend_decode_proxy analyser on proxied connections
- [DOC] add the proxy protocol's specifications
- [DOC] document the 'accept-proxy' bind option
- [MINOR] cfgparse: report support of <path> for the 'bind' statements
- [DOC] add references to unix socket handling
- [MINOR] move MAXPATHLEN definition to compat.h
- [MEDIUM] unix sockets: cleanup the error reporting path
- [BUG] session: don't stop forwarding of data upon last packet
- [CLEANUP] accept: replace some inappropriate Alert() calls with send_log()
- [BUILD] peers: shut a printf format warning (key_size is a size_t)
- [BUG] accept: don't close twice upon error
- [OPTIM] session: don't recheck analysers when buffer flags have not changed
- [OPTIM] stream_sock: don't clear FDs that are already cleared
- [BUG] proto_tcp: potential bug on pattern fetch dst and dport
This option makes haproxy preserve any persistence cookie emitted by
the server, which allows the server to change it or to unset it, for
instance, after a logout request.
(cherry picked from commit 52e6d75374c7900c1fe691c5633b4ae029cae8d5)
This match returns true when the request calling it is the first one of
a connection.
(cherry picked from commit 922ca979c50653c415852531f36fe409190ad76b)
The MySQL check has been revamped to be able to send real MySQL data,
and to avoid Aborted connects on MySQL side.
It is however backward compatible with older version, but it is highly
recommended to use the new mode, by adding "user <username>" on the
"mysql-check" line.
The new check consists in sending two MySQL packet, one Client
Authentication packet, with "haproxy" username (by default), and one
QUIT packet, to correctly close MySQL session. We then parse the Mysql
Handshake Initialisation packet and/or Error packet. It is a basic but
useful test which does not produce error nor aborted connect on the
server.
(cherry picked from commit a1e4dcfe5718311b7653d7dabfad65c005d0439b)
- fix some erroneous keywords for http-request
- stats http-request was not documented
--
Cyril Bont
(cherry picked from commit 17e07dad5074571c38cbeb92d54a20f83658449d)
HTTP content-based health checks will be involved in searching text in pages.
Some pages may not fit in the default buffer (16kB) and sometimes it might be
desired to have larger buffers in order to find patterns. Running checks on
smaller URIs is always preferred of course.
(cherry picked from commit 043f44aeb835f3d0b57626c4276581a73600b6b1)
This patch adds the "http-check expect [r]{string,status}" statements
which enable health checks based on whether the response status or body
to an HTTP request contains a string or matches a regex.
This probably is one of the oldest patches that remained unmerged. Over
the time, several people have contributed to it, among which FinalBSD
(first and second implementations), Nick Chalk (port to 1.4), Anze
Skerlavaj (tests and fixes), Cyril Bont (general fixes), and of course
myself for the final fixes and doc during integration.
Some people already use an old version of this patch which has several
issues, among which the inability to search for a plain string that is
not at the beginning of the data, and the inability to look for response
contents that are provided in a second and subsequent recv() calls. But
since some configs are already deployed, it was quite important to ensure
a 100% compatible behaviour on the working cases.
Thus, that patch fixes the issues while maintaining config compatibility
with already deployed versions.
(cherry picked from commit b507c43a3ce9a8e8e4b770e52e4edc20cba4c37f)
This patch provides a new "option ldap-check" statement to enable
server health checks based on LDAPv3 bind requests.
(cherry picked from commit b76b44c6fed8a7ba6f0f565dd72a9cb77aaeca7c)
Some configs may involve httpclose in a frontend and http-pretend-keepalive
in a backend. httpclose used to take priority over keepalive, thus voiding
its effect. This change ensures that when both are combined, keepalive is
still announced to the server while close is announced to the client.
(cherry picked from commit 2be7ec90fa9caf66294f446423bbab2d00db9004)
The doc was wrong as the insert mode by default does not insert in
direct requests, and by default transmits the cookies to the server.
This was right in the old doc and it has not changed since the
beginning.
Released version 1.5-dev2 with the following main changes :
- [MINOR] startup: release unused structs after forking
- [MINOR] startup: don't wait for nothing when no old pid remains
- [CLEANUP] reference product branch 1.5
- [MEDIUM] signals: add support for registering functions and tasks
- [MEDIUM] signals: support redistribution of signal zero when stopping
- [BUG] http: don't set auto_close if more data are expected
Released version 1.5-dev1 with the following main changes :
- [BUG] stats: session rate limit gets garbaged in the stats
- [DOC] mention 'option http-server-close' effect in Tq section
- [DOC] summarize and highlight persistent connections behaviour
- [DOC] add configuration samples
- [BUG] http: dispatch and http_proxy modes were broken for a long time
- [BUG] http: the transaction must be initialized even in TCP mode
- [BUG] tcp: dropped connections must be counted as "denied" not "failed"
- [BUG] consistent hash: balance on all servers, not only 2 !
- [CONTRIB] halog: report per-server status codes, errors and response times
- [BUG] http: the transaction must be initialized even in TCP mode (part 2)
- [BUG] client: always ensure to zero rep->analysers
- [BUG] session: clear BF_READ_ATTACHED before next I/O
- [BUG] http: automatically close response if req is aborted
- [BUG] proxy: connection rate limiting was eating lots of CPU
- [BUG] http: report correct flags in case of client aborts during body
- [TESTS] refine non-regression tests and add 4 new tests
- [BUG] debug: wrong pointer was used to report a status line
- [BUG] debug: correctly report truncated messages
- [DOC] document the "dispatch" keyword
- [BUG] stick_table: fix possible memory leak in case of connection error
- [CLEANUP] acl: use 'L6' instead of 'L4' in ACL flags relying on contents
- [MINOR] accept: count the incoming connection earlier
- [CLEANUP] tcp: move some non tcp-specific layer6 processing out of proto_tcp
- [CLEANUP] client: move some ACLs away to their respective locations
- [CLEANUP] rename client -> frontend
- [MEDIUM] separate protocol-level accept() from the frontend's
- [MINOR] proxy: add a list to hold future layer 4 rules
- [MEDIUM] config: parse tcp layer4 rules (tcp-request accept/reject)
- [MEDIUM] tcp: check for pure layer4 rules immediately after accept()
- [OPTIM] frontend: tell the compiler that errors are unlikely to occur
- [MEDIUM] frontend: check for LI_O_TCP_RULES in the listener
- [MINOR] frontend: only check for monitor-net rules if LI_O_CHK_MONNET is set
- [CLEANUP] buffer->cto is not used anymore
- [MEDIUM] session: finish session establishment sequence in with I/O handlers
- [MEDIUM] session: initialize server-side timeouts after connect()
- [MEDIUM] backend: initialize the server stream_interface upon connect()
- [MAJOR] frontend: don't initialize the server-side stream_int anymore
- [MEDIUM] session: move the conn_retries attribute to the stream interface
- [MEDIUM] session: don't assign conn_retries upon accept() anymore
- [MINOR] frontend: rely on the frontend and not the backend for INDEPSTR
- [MAJOR] frontend: reorder the session initialization upon accept
- [MINOR] proxy: add an accept() callback for the application layer
- [MAJOR] frontend: split accept() into frontend_accept() and session_accept()
- [MEDIUM] stats: rely on the standard session_accept() function
- [MINOR] buffer: refine the flags that may wake an analyser up.
- [MINOR] stream_sock: don't dereference a non-existing frontend
- [MINOR] session: differenciate between accepted connections and received connections
- [MEDIUM] frontend: count the incoming connection earlier
- [MINOR] frontend: count denied TCP requests separately
- [CLEANUP] stick_table: add/clarify some comments
- [BUILD] memory: add a few missing parenthesis to the pool management macros
- [MINOR] stick_table: add support for variable-sized data
- [CLEANUP] stick_table: rename some stksess struct members to avoid confusion
- [CLEANUP] stick_table: move pattern to key functions to stick_table.c
- [MEDIUM] stick_table: add room for extra data types
- [MINOR] stick_table: add support for "conn_cum" data type.
- [MEDIUM] stick_table: don't overwrite data when storing an entry
- [MINOR] config: initialize stick tables after all the parsing
- [MINOR] stick_table: provide functions to return stksess data from a type
- [MEDIUM] stick_table: move the server ID to a generic data type
- [MINOR] stick_table: enable it for frontends too
- [MINOR] stick_table: export the stick_table_key
- [MINOR] tcp: add per-source connection rate limiting
- [MEDIUM] stick_table: separate storage and update of session entries
- [MEDIUM] stick-tables: add a reference counter to each entry
- [MINOR] session: add a pointer to the tracked counters for the source
- [CLEANUP] proto_tcp: make the config parser a little bit more flexible
- [BUG] config: report the correct proxy type in tcp-request errors
- [MINOR] config: provide a function to quote args in a more friendly way
- [BUG] stick_table: the fix for the memory leak caused a regression
- [MEDIUM] backend: support servers on 0.0.0.0
- [BUG] stick-table: correctly refresh expiration timers
- [MEDIUM] stream-interface: add a ->release callback
- [MINOR] proxy: add a "parent" member to the structure
- [MEDIUM] session: make it possible to call an I/O handler on both SI
- [MINOR] tools: add a fast div64_32 function
- [MINOR] freq_ctr: add new types and functions for periods different from 1s
- [MINOR] errors: provide new status codes for config parsing functions
- [BUG] http: denied requests must not be counted as denied resps in listeners
- [MINOR] tools: add a get_std_op() function to parse operators
- [MEDIUM] acl: make use of get_std_op() to parse intger ranges
- [MAJOR] stream_sock: better wakeup conditions on read()
- [BUG] session: analysers must be checked when SI state changes
- [MINOR] http: reset analysers to listener's, not frontend's
- [MEDIUM] session: support "tcp-request content" rules in backends
- [BUILD] always match official tags when doing git-tar
- [MAJOR] stream_interface: fix the wakeup conditions for embedded iohandlers
- [MEDIUM] buffer: make buffer_feed* support writing non-contiguous chunks
- [MINOR] tcp: src_count acl does not have a permanent result
- [MAJOR] session: add track-counters to track counters related to the session
- [MINOR] stick-table: provide a table lookup function
- [MINOR] stick-table: use suffix "_cnt" for cumulated counts
- [MEDIUM] session: move counter ACL fetches from proto_tcp
- [MEDIUM] session: add concurrent connections counter
- [MEDIUM] session: add data in and out volume counters
- [MINOR] session: add the trk_conn_cnt ACL keyword to track connection counts
- [MEDIUM] session-counters: automatically update tracked connection count
- [MINOR] session: add the trk_conn_cur ACL keyword to track concurrent connection
- [MINOR] session: add trk_kbytes_* ACL keywords to track data size
- [MEDIUM] session: add a counter on the cumulated number of sessions
- [MINOR] config: support a comma-separated list of store data types in stick-table
- [MEDIUM] stick-tables: add support for arguments to data_types
- [MEDIUM] stick-tables: add stored data argument type checking
- [MEDIUM] session counters: add conn_rate and sess_rate counters
- [MEDIUM] session counters: add bytes_in_rate and bytes_out_rate counters
- [MINOR] stktable: add a stktable_update_key() function
- [MINOR] session-counters: add a general purpose counter (gpc0)
- [MEDIUM] session-counters: add HTTP req/err tracking
- [MEDIUM] stats: add "show table [<name>]" to dump a stick-table
- [MEDIUM] stats: add "clear table <name> key <value>" to clear table entries
- [CLEANUP] stick-table: declare stktable_data_types as extern
- [MEDIUM] stick-table: make use of generic types for stored data
- [MINOR] stats: correctly report errors on "show table" and "clear table"
- [MEDIUM] stats: add the ability to dump table entries matching criteria
- [DOC] configuration: document all the new tracked counters
- [DOC] stats: document "show table" and "clear table"
- [MAJOR] session-counters: split FE and BE track counters
- [MEDIUM] tcp: accept the "track-counters" in "tcp-request content" rules
- [MEDIUM] session counters: automatically remove expired entries.
- [MEDIUM] config: replace 'tcp-request <action>' with "tcp-request connection"
- [MEDIUM] session-counters: make it possible to count connections from frontend
- [MINOR] session-counters: use "track-sc{1,2}" instead of "track-{fe,be}-counters"
- [MEDIUM] session-counters: correctly unbind the counters tracked by the backend
- [CLEANUP] stats: use stksess_kill() to remove table entries
- [DOC] update the references to session counters and to tcp-request connection
- [DOC] cleanup: split a few long lines
- [MEDIUM] http: forward client's close when abortonclose is set
- [BUG] queue: don't dequeue proxy-global requests on disabled servers
- [BUG] stats: global stats timeout may be specified before stats socket.
- [BUG] conf: add tcp-request content rules to the correct list
It began to be problematic to have "tcp-request" followed by an
immediate action, as sometimes it was a keyword indicating a hook
or setting ("content" or "inspect-delay") and sometimes it was an
action.
Now the prefix for connection-level tcp-requests is "tcp-request connection"
and the ones processing contents remain "tcp-request contents".
This has allowed a nice simplification of the config parser and to
clean up the doc a bit. Also now it's a bit more clear why tcp-request
connection are not allowed in backends.
The following statements and ACLs have been added to the doc :
- tcp-request track-counters <key> [table <table>] [{if | unless} <condition>]
- src_*
- trk_*
The "_cnt" suffix is already used by ACLs to count various data,
so it makes sense to use the same one in "conn_cnt" instead of
"conn_cum" to count cumulated connections.
This is not a problem because no version was emitted with those
keywords.
Thus we'll try to stick to the following rules :
xxxx_cnt : cumulated event count for criterion xxxx
xxxx_cur : current number of concurrent entries for criterion xxxx
xxxx_rate: event rate for criterion xxxx
Sometimes it's necessary to be able to perform some "layer 6" analysis
in the backend. TCP request rules were not available till now, although
documented in the diagram. Enable them in backend now.
Till now when a server was configured with address 0.0.0.0, the
connection was forwarded to this address which generally is intercepted
by the system as a local address, so this was completely useless.
One sometimes useful feature for outgoing transparent proxies is to
be able to forward the connection to the same address the client
requested. This patch fixes the meaning of 0.0.0.0 precisely to
ensure that the connection will be forwarded to the initial client's
destination address.
configuration.txt is thorough and accurate but lacked sample configurations
clarifying both the syntax and the relations between global, defaults,
frontend, backend and listen sections. Besides, almost all examples to be found
in haproxy-en.txt or online tutorials make use of the 'listen' syntax while
'frontend/backend' is really the one to know about.
(cherry picked from commit 01ac10ad189b11c563eeb835733fba58e6c5271d)
This change makes use of the stick-tables to keep track of any source
address activity. Two ACLs make it possible to check the count of an
entry or update it and act accordingly. The typical usage will be to
reject a TCP request upon match of an excess value.
This one can be parsed on the "stick-table" after with the "store"
keyword. It will hold the number of connections matching the entry,
for use with ACLs or anything else.
The stick_tables will now be able to store extra data for a same key.
A limited set of extra data types will be defined and for each of them
an offset in the sticky session will be assigned at startup time. All
of this information will be stored in the stick table.
The extra data types will have to be specified after the new "store"
keyword of the "stick-table" directive, which will reserve some space
for them.
It's very disturbing to see the "denied req" counter increase without
any other session counter moving. In fact, we can't count a rejected
TCP connection as "denied req" as we have not yet instanciated any
session at all. Let's use a new counter for that.
Released version 1.4.6 with the following main changes :
- [BUILD] ebtree: update to v6.0.1 to remove references to dprintf()
- [CLEANUP] acl: make use of eb_is_empty() instead of open coding the tree's emptiness test
- [MINOR] acl: add srv_is_up() to check that a specific server is up or not
- [DOC] add a few precisions about the use of RDP cookies
RDP cookies are not necessarily easy to implement because they require
some configuration on the servers. Add a few hints so that people know
what to check on their servers.
This ACL was missing in complex setups where the status of a remote site
has to be considered in switching decisions. Until there, using a server's
status in an ACL required to have a dedicated backend, which is a bit heavy
when multiple servers have to be monitored.
Released version 1.4.5 with the following main changes :
- [DOC] report minimum kernel version for tproxy in the Makefile
- [MINOR] add the "ignore-persist" option to conditionally ignore persistence
- [DOC] add the "ignore-persist" option to conditionally ignore persistence
- [DOC] fix ignore-persist/force-persist documentation
- [BUG] cttproxy: socket fd leakage in check_cttproxy_version
- [DOC] doc/configuration.txt: fix typos
- [MINOR] option http-pretend-keepalive is both for FEs and BEs
- [MINOR] fix possible crash in debug mode with invalid responses
- [MINOR] halog: add support for statisticts on status codes
- [OPTIM] halog: use a faster zero test in fgets()
- [OPTIM] halog: minor speedup by using unlikely()
- [OPTIM] halog: speed up fgets2-64 by about 10%
- [DOC] refresh the README file and merge the CONTRIB file into it
- [MINOR] acl: support loading values from files
- [MEDIUM] ebtree: upgrade to version 6.0
- [MINOR] acl trees: add flags and union members to store values in trees
- [MEDIUM] acl: add ability to insert patterns in trees
- [MEDIUM] acl: add tree-based lookups of exact strings
- [MEDIUM] acl: add tree-based lookups of networks
- [MINOR] acl: ignore empty lines and comments in pattern files
- [MINOR] stick-tables: add support for "stick on hdr"
It is now possible to stick on an IP address found in a HTTP header. Right
now only the last occurrence of the header can be used, which is generally
enough for most uses. Also, the header extraction rule only knows how to
convert the header to IP. Later it will be usable as a plain string with
an implicit conversion, and the syntax will not change.
Most often, pattern files used by ACLs will be produced by tools
which emit some comments (eg: geolocation lists). It's very annoying
to have to clean the files before using them, and it does not make
much sense to be able to support patterns we already can't input in
the config file. So this patch makes the pattern file loader skip
lines beginning with a sharp and the empty ones, and strips leading
spaces and tabs.
The "acl XXX -f <file>" syntax was supported but nothing was read from
the file. This is now possible. All lines are merged verbatim, even if
they contain spaces (useful for user-agents). There are shortcomings
though. The worst one is that error reporting is too approximative.
Shame on me, I didn't correctly document the "ignore-persist" statement
(convinced I used it like this in my tests, which is not the case at all...)
This fixes the doc and updates the proxy keyword matrix to add "force-persist".
Released version 1.4.4 with the following main changes :
- [BUG] appsession should match the whole cookie name
- [CLEANUP] proxy: move PR_O_SSL3_CHK to options2 to release one flag
- [MEDIUM] backend: move the transparent proxy address selection to backend
- [MINOR] add very fast IP parsing functions
- [MINOR] add new tproxy flags for dynamic source address binding
- [MEDIUM] add ability to connect to a server from an IP found in a header
- [BUILD] config: last patch breaks build without CONFIG_HAP_LINUX_TPROXY
- [MINOR] http: make it possible to pretend keep-alive when doing close
- [MINOR] config: report "default-server" instead of "(null)" in error messages
Some servers do not completely conform with RFC2616 requirements for
keep-alive when they receive a request with "Connection: close". More
specifically, they don't bother using chunked encoding, so the client
never knows whether the response is complete or not. One immediately
visible effect is that haproxy cannot maintain client connections alive.
The second issue is that truncated responses may be cached on clients
in case of network error or timeout.
scar Fras Barranco reported this issue on Tomcat 6.0.20, and
Patrik Nilsson with Jetty 6.1.21.
Cyril Bont proposed this smart idea of pretending we run keep-alive
with the server and closing it at the last moment as is already done
with option forceclose. The advantage is that we only change one
emitted header but not the overall behaviour.
Since some servers such as nginx are able to close the connection
very quickly and save network packets when they're aware of the
close negociation in advance, we don't enable this behaviour by
default.
"option http-pretend-keepalive" will have to be used for that, in
conjunction with "option http-server-close".
Using get_ip_from_hdr2() we can look for occurrence #X or #-X and
extract the IP it contains. This is typically designed for use with
the X-Forwarded-For header.
Using "usesrc hdr_ip(name,occ)", it becomes possible to use the IP address
found in <name>, and possibly specify occurrence number <occ>, as the
source to connect to a server. This is possible both in a server and in
a backend's source statement. This is typically used to use the source
IP previously set by a upstream proxy.
Released version 1.4.3 with the following main changes :
- [CLEANUP] stats: remove printf format warning in stats_dump_full_sess_to_buffer()
- [MEDIUM] session: better fix for connection to servers with closed input
- [DOC] indicate in the doc how to bind to port ranges
- [BUG] backend: L7 hashing must not be performed on incomplete requests
- [TESTS] add a simple program to test connection resets
- [MINOR] cli: "show errors" should display "backend <NONE>" when backend was not used
- [MINOR] config: emit warnings when HTTP-only options are used in TCP mode
- [MINOR] config: allow "slowstart 0s"
- [BUILD] 'make tags' did not consider files ending in '.c'
- [MINOR] checks: add the ability to disable a server in the config
Released version 1.4.2 with the following main changes :
- [CLEANUP] product branch update
- [DOC] Some more documentation cleanups
- [BUG] clf logs segfault when capturing a non existant header
- [OPTIM] config: only allocate check buffer when checks are enabled
- [MEDIUM] checks: support multi-packet health check responses
- [CLEANUP] session: remove duplicate test
- [BUG] http: don't wait for response data to leave buffer is client has left
- [MINOR] proto_uxst: set accept_date upon accept() to the wall clock time
- [MINOR] stats: don't send empty lines in "show errors"
- [MINOR] stats: make the data dump function reusable for other purposes
- [MINOR] stats socket: add show sess <id> to dump details about a session
- [BUG] stats: connection reset counters must be plain ascii, not HTML
- [BUG] url_param hash may return a down server
- [MINOR] force null-termination of hostname
- [MEDIUM] connect to servers even when the input has already been closed
- [BUG] don't merge anonymous ACLs !
- [BUG] config: fix endless loop when parsing "on-error"
- [MINOR] http: don't mark a server as failed when it returns 501/505
- [OPTIM] checks: try to detect the end of response without polling again
- [BUG] checks: don't report an error when recv() returns an error after data
- [BUG] checks: don't abort when second poll returns an error
- [MINOR] checks: make shutdown() silently fail
- [BUG] http: fix truncated responses on chunk encoding when size divides buffer size
- [BUG] init: unconditionally catch SIGPIPE
- [BUG] checks: don't wait for a close to start parsing the response
Since the last documentation cleanups, I've found more typos that I kept
in a corner instead of sending you a mail just for one character :)
--
Cyril Bont
When trying to spot some complex bugs, it's often needed to access
information on stuck sessions, which is quite difficult. This new
command helps one get detailed information about a session, with
flags, timers, states, etc... The buffer data are not dumped yet.
Released version 1.4.1 with the following main changes :
- [BUG] Clear-cookie path issue
- [DOC] fix typo on stickiness rules
- [BUILD] fix BSD and OSX makefiles for missing files
- [BUILD] includes order breaks OpenBSD build
- [BUILD] fix some build warnings on Solaris with is* macros
- [BUG] logs: don't report "last data" when we have just closed after an error
- [BUG] logs: don't report "proxy request" when server closes early
- [BUILD] fix platform-dependant build issues related to crypt()
- [STATS] count transfer aborts caused by client and by server
- [STATS] frontend requests were not accounted for failed requests
- [MINOR] report total number of processed connections when stopping a proxy
- [DOC] be more clear about the limitation to one single monitor-net entry
Often we need to understand why some transfers were aborted or what
constitutes server response errors. With those two counters, it is
now possible to detect an unexpected transfer abort during a data
phase (eg: too short HTTP response), and to know what part of the
server response errors may in fact be assigned to aborted transfers.
Released version 1.4.0 with the following main changes :
- [MINOR] stats: report maint state for tracking servers too
- [DOC] fix summary to add pattern extraction
- [DOC] Documentation cleanups
- [BUG] cfgparse memory leak and missing free calls in deinit()
- [BUG] pxid/puid/luid: don't shift IDs when some of them are forced
- [EXAMPLES] add auth.cfg
- [BUG] uri_auth: ST_SHLGNDS should be 0x00000008 not 0x0000008
- [BUG] uri_auth: do not attemp to convert uri_auth -> http-request more than once
- [BUILD] auth: don't use unnamed unions
- [BUG] config: report unresolvable host names as errors
- [BUILD] fix build breakage with DEBUG_FULL
- [DOC] fix a typo about timeout check and clarify the explanation.
- [MEDIUM] http: don't use trash to realign large buffers
- [STATS] report HTTP requests (total and rate) in frontends
- [STATS] separate frontend and backend HTTP stats
- [MEDIUM] http: revert to use a swap buffer for realignment
- [MINOR] stats: report the request rate in frontends as cell titles
- [MINOR] stats: mark areas with an underline when tooltips are available
- [DOC] reorder some entries to maintain the alphabetical order
- [DOC] cleanup of the keyword matrix
Hi Willy,
This patch make some cleanups on the documentation :
- strictly respect the 80 columns constraint
- use space indentation everywhere
- add some missing "(deprecated)"
- fix the "monitor fail" keyword syntax
- fix a keyword in a "See Also" line (interval instead of inter)
- the "no" prefix of "option http-use-proxy-header" is displayed as the others
I've also noticed that 3 keywords are missing from the keywords reference (I don't know what to do for that) :
- dispatch
- reqsetbe
- reqisetbe
--
Cyril Bont
Released version 1.4-rc1 with the following main changes :
- [MEDIUM] add a maintenance mode to servers
- [MINOR] http-auth: last fix was wrong
- [CONTRIB] add base64rev-gen.c that was used to generate the base64rev table.
- [MINOR] Base64 decode
- [MINOR] generic auth support with groups and encrypted passwords
- [MINOR] add ACL_TEST_F_NULL_MATCH
- [MINOR] http-request: allow/deny/auth support for frontend/backend/listen
- [MINOR] acl: add http_auth and http_auth_group
- [MAJOR] use the new auth framework for http stats
- [DOC] add info about userlists, http-request and http_auth/http_auth_group acls
- [STATS] make it possible to change a CLI connection timeout
- [BUG] patterns: copy-paste typo in type conversion arguments
- [MINOR] pattern: make the converter more flexible by supporting void* and int args
- [MINOR] standard: str2mask: string to netmask converter
- [MINOR] pattern: add support for argument parsers for converters
- [MINOR] pattern: add the "ipmask()" converting function
- [MINOR] config: off-by-one in "stick-table" after list of converters
- [CLEANUP] acl, patterns: make use of my_strndup() instead of malloc+memcpy
- [BUG] restore accidentely removed line in last patch !
- [MINOR] checks: make the HTTP check code add the CRLF itself
- [MINOR] checks: add the server's status in the checks
- [BUILD] halog: make without arch-specific optimizations
- [BUG] halog: fix segfault in case of empty log in PCT mode (cherry picked from commit fe362fe476)
- [MINOR] http: disable keep-alive when process is going down
- [MINOR] acl: add build_acl_cond() to make it easier to add ACLs in config
- [CLEANUP] config: use build_acl_cond() instead of parse_acl_cond()
- [CLEANUP] config: use warnif_cond_requires_resp() to check for bad ACLs
- [MINOR] prepare req_*/rsp_* to receive a condition
- [CLEANUP] config: specify correct const char types to warnif_* functions
- [MEDIUM] config: factor out the parsing of 20 req*/rsp* keywords
- [MEDIUM] http: make the request filter loop check for optional conditions
- [MEDIUM] http: add support for conditional request filter execution
- [DOC] add some build info about the AIX platform (cherry picked from commit e41914c77e)
- [MEDIUM] http: add support for conditional request header addition
- [MEDIUM] http: add support for conditional response header rewriting
- [DOC] add some missing ACLs about response header matching
- [MEDIUM] http: add support for proxy authentication
- [MINOR] http-auth: make the 'unless' keyword work as expected
- [CLEANUP] config: use build_acl_cond() to simplify http-request ACL parsing
- [MEDIUM] add support for anonymous ACLs
- [MEDIUM] http: switch to tunnel mode after status 101 responses
- [MEDIUM] http: stricter processing of the CONNECT method
- [BUG] config: reset check request to avoid double free when switching to ssl/sql
- [MINOR] config: fix too large ssl-hello-check message.
- [BUG] fix error response in case of server error
A 101 response is accompanied with an Upgrade header indicating
a new protocol that is spoken on the connection after the exchange
completes. At least we should switch to tunnel mode after such a
response.
Anonymous ACLs allow the declaration of rules which rely directly on
ACL expressions without passing via the declaration of an ACL. Example :
With named ACLs :
acl site_dead nbsrv(dynamic) lt 2
acl site_dead nbsrv(static) lt 2
monitor fail if site_dead
With anonymous ACLs :
monitor fail if { nbsrv(dynamic) lt 2 } || { nbsrv(static) lt 2 }
This is a first attempt to add a maintenance mode on servers, using
the stat socket (in admin level).
It can be done with the following command :
- disable server <backend>/<server>
- enable server <backend>/<server>
In this mode, no more checks will be performed on the server and it
will be marked as a special DOWN state (MAINT).
If some servers were tracking it, they'll go DOWN until the server
leaves the maintenance mode. The stats page and the CSV export also
display this special state.
This can be used to disable the server in haproxy before doing some
operations on this server itself. This is a good complement to the
"http-check disable-on-404" keyword and works in TCP mode.
We're already able to know if a request is a proxy request or a
normal one, and we have an option "http-use-proxy-header" which states
that proxy headers must be checked. So let's switch to use the proxy
authentication headers and responses when this option is set and we're
facing a proxy request. That allows haproxy to enforce auth in front
of a proxy.
Just as for the req* rules, we can now condition rsp* rules with ACLs.
ACLs match on response, so volatile request information cannot be used.
A warning is emitted if a configuration contains such an anomaly.
All the req* rules except the reqadd rules can now be specified with
an if/unless condition. If a condition is specified and does not match,
the filter is ignored. This is particularly useful with reqidel, reqirep
and reqtarpit.
Now a server can check the contents of the header X-Haproxy-Server-State
to know how haproxy sees it. The same values as those reported in the stats
are provided :
- up/down status + check counts
- throttle
- weight vs backend weight
- active sessions vs backend sessions
- queue length
- haproxy node name
This converter can be applied on top of an IPv4-type pattern. It
applies a netmask which is suited for IP address storage and matching.
This can be used to make all hosts within a certain mask to share the
same table entries and as such use the same server.
The mask can be passed in dotted form (eg: 255.255.255.0) or in CIDR
form (eg: 24).
Hi Willy,
I've made a quick pass on the "defaults" column in the Proxy keywords matrix (chapter 4.1. in the documentation).
This patch resyncs the code and the documentation. I let you decide if some keywords that still work in the "defaults" section should be forbidden.
- default_backend : in the matrix, "defaults" was not supported but the keyword details say it is.
Tests also shows it works, then I've updated the matrix.
- capture cookie : in the keyword details, we can read `It is not possible to specify a capture in a "defaults" section.'.
Ok, even if the tests worked, I've added an alert in the configuration parser (as it is for capture request/response header).
- description : not supported in "defaults", I added an alert in the parser.
I've also noticed that this keyword doesn't appear in the documentation.
There's one "description" entry, but for the "global" section, which is for a different use (the patch doesn't update the documentation).
- grace : even if this is maybe useless, it works in "defaults". Documentation is updated.
- redirect : alert is added in the parser.
- rsprep : alert added in the parser.
--
Cyril Bont
Despite what is explicitly stated in HTTP specifications,
browsers still use the undocumented Proxy-Connection header
instead of the Connection header when they connect through
a proxy. As such, proxies generally implement support for
this stupid header name, breaking the standards and making
it harder to support keep-alive between clients and proxies.
Thus, we add a new "option http-use-proxy-header" to tell
haproxy that if it sees requests which look like proxy
requests, it should use the Proxy-Connection header instead
of the Connection header.
Released version 1.4-dev7 with the following main changes :
- [BUG] appsession: possible memory leak in case of out of memory condition
- [MINOR] config: don't accept 'appsession' in defaults section
- [MINOR] Add function to parse a size in configuration
- [MEDIUM] Add stick table (persistence) management functions and types
- [MEDIUM] Add pattern fetch management types and functions
- [MEDIUM] Add src dst and dport pattern fetches.
- [MEDIUM] Add stick table configuration and init.
- [MEDIUM] Add stick and store rules analysers.
- [MINOR] add option "mysql-check" to use MySQL health checks
- [BUG] health checks: fix requeued message
- [OPTIM] remove SSP_O_VIA and SSP_O_STATUS
- [BUG] checks: fix newline termination
- [MINOR] acl: add fe_id/so_id to match frontend's and socket's id
- [BUG] appsession's sessid must be reset at end of transaction
- [BUILD] appsession did not build anymore under gcc-2.95
- [BUG] server redirection used an uninitialized string.
- [MEDIUM] http: fix handling of message pointers
- [MINOR] http: fix double slash prefix with server redirect
- [MINOR] http redirect: add the ability to append a '/' to the URL
- [BUG] stream_interface: fix retnclose and remove cond_close
- [MINOR] http redirect: don't explicitly state keep-alive on 1.1
- [MINOR] http: move appsession 'sessid' from session to http_txn
- [OPTIM] reorder http_txn to optimize cache lines placement
- [MINOR] http: differentiate waiting for new request and waiting for a complete requst
- [MINOR] http: add a separate "http-keep-alive" timeout
- [MINOR] config: remove undocumented and buggy 'timeout appsession'
- [DOC] fix various too large lines
- [DOC] remove several trailing spaces
- [DOC] add the doc about stickiness
- [BUILD] remove a warning in standard.h on AIX
- [BUG] checks: chars are unsigned on AIX, check was always true
- [CLEANUP] stream_sock: MSG_NOSIGNAL is only for send(), not recv()
- [BUG] check: we must not check for error before reading a response
- [BUG] buffers: remove remains of wrong obsolete length check
- [OPTIM] stream_sock: don't shutdown(write) when the socket is in error
- [BUG] http: don't count req errors on client resets or t/o during keep-alive
- [MEDIUM] http: don't switch to tunnel mode upon close
- [DOC] add documentation about connection header processing
- [MINOR] http: add http_remove_header2() to remove a header value.
- [MINOR] tools: add a "word_match()" function to match words and ignore spaces
- [MAJOR] http: rework request Connection header handling
- [MAJOR] http: rework response Connection header handling
- [MINOR] add the ability to force kernel socket buffer size.
- [BUG] http_server_error() must not purge a previous pending response
- [OPTIM] http: don't delay response if next request is incomplete
- [MINOR] add the "force-persist" statement to force persistence on down servers
- [MINOR] http: logs must report persistent connections to down servers
- [BUG] buffer_replace2 must never change the ->w entry
This is used to force access to down servers for some requests. This
is useful when validating that a change on a server correctly works
before enabling the server again.
Sometimes we need to be able to change the default kernel socket
buffer size (recv and send). Four new global settings have been
added for this :
- tune.rcvbuf.client
- tune.rcvbuf.server
- tune.sndbuf.client
- tune.sndbuf.server
Those can be used to reduce kernel memory footprint with large numbers
of concurrent connections, and to reduce risks of write timeouts with
very slow clients due to excessive kernel buffering.
Sometimes it can be desired to return a location which is the same
as the request with a slash appended when there was not one in the
request. A typical use of this is for sending a 301 so that people
don't reference links without the trailing slash. The name of the
new option is "append-slash" and it can be used on "redirect"
statements in prefix mode.
Released version 1.4-dev6 with the following main changes :
- [BUILD] warning in stream_interface.h
- [BUILD] warning ultoa_r returns char *
- [MINOR] hana: only report stats if it is enabled
- [MINOR] stats: add "a link" & "a href" for sockets
- [MINOR]: stats: add show-legends to report additional informations
- [MEDIUM] default-server support
- [BUG]: add 'observer', 'on-error', 'error-limit' to supported options list
- [MINOR] stats: add href to tracked server
- [BUG] stats: show UP/DOWN status also in tracking servers
- [DOC] Restore ability to search a keyword at the beginning of a line
- [BUG] stats: cookie should be reported under backend not under proxy
- [BUG] cfgparser/stats: fix error message
- [BUG] http: disable auto-closing during chunk analysis
- [BUG] http: fix hopefully last closing issue on data forwarding
- [DEBUG] add an http_silent_debug function to debug HTTP states
- [MAJOR] http: fix again the forward analysers
- [BUG] http_process_res_common() must not skip the forward analyser
- [BUG] http: some possible missed close remain in the forward chain
- [BUG] http: redirect needed to be updated after recent changes
- [BUG] http: don't set no-linger on response in case of forced close
- [MEDIUM] http: restore the original behaviour of option httpclose
- [TESTS] add a file to test various connection modes
- [BUG] http: check options before the connection header
- [MAJOR] session: fix the order by which the analysers are run
- [MEDIUM] session: also consider request analysers added during response
- [MEDIUM] http: make safer use of the DONT_READ and AUTO_CLOSE flags
- [BUG] http: memory leak with captures when using keep-alive
- [BUG] http: fix for capture memory leak was incorrect
- [MINOR] http redirect: use proper call to return last response
- [MEDIUM] http: wait for some flush of the response buffer before a new request
- [MEDIUM] session: limit the number of analyser loops
This patch implements default-server support allowing to change
default server options. It can be used in [defaults] or [backend]/[listen]
sections. Currently the following options are supported:
- error-limit
- fall
- inter
- fastinter
- downinter
- maxconn
- maxqueue
- minconn
- on-error
- port
- rise
- slowstart
- weight
Supported informations, available via "tr/td title":
- cap: capabilities (proxy)
- mode: one of tcp, http or health (proxy)
- id: SNMP ID (proxy, socket, server)
- IP (socket, server)
- cookie (backend, server)
Historically, "option httpclose" has always worked the same way. It
only mangles the "Connection" header in the request and the response
if needed, but does not affect the connection by itself, and ignores
any further data. It is dangerous to change this behaviour without
leaving any other alternative. If an active close is desired, it's
better to make use of "option forceclose" which does exactly what
it intends to do.
So as of now, "option httpclose" will only mangle the headers as
before, and will only affect the connection by itself when combined
with another connection-related option (eg: keepalive or server-close).
Released version 1.4-dev5 with the following main changes :
- [MINOR] server tracking: don't care about the tracked server's mode
- [MEDIUM] appsession: add "len", "prefix" and "mode" options
- [MEDIUM] appsession: add the "request-learn" option
- [BUG] Configuration parser bug when escaping characters
- [MINOR] CSS & HTML fun
- [MINOR] Collect & provide http response codes received from servers
- [BUG] Fix silly typo: hspr_other -> hrsp_other
- [MINOR] Add "a name" to stats page
- [MINOR] add additional "a href"s to stats page
- [MINOR] Collect & provide http response codes for frontends, fix backends
- [DOC] some small spell fixes and unifications
- [MEDIUM] Decrease server health based on http responses / events, version 3
- [BUG] format '%d' expects type 'int', but argument 5 has type 'long int'
- [BUG] config: fix erroneous check on cookie domain names, again
- [BUG] Healthchecks: get a proper error code if connection cannot be completed immediately
- [DOC] trivial fix for man page
- [MINOR] config: report all supported options for the "bind" keyword
- [MINOR] tcp: add support for the defer_accept bind option
- [MINOR] unix socket: report the socket path in case of bind error
- [CONTRIB] halog: support searching by response time
- [DOC] add a reminder about obsolete documents
- [DOC] point to 1.4 doc, not 1.3
- [DOC] option tcp-smart-connect was missing from index
- [MINOR] http: detect connection: close earlier
- [CLEANUP] sepoll: clean up the fd_clr/fd_set functions
- [OPTIM] move some rarely used fields out of fdtab
- [MEDIUM] fd: merge fd_list into fdtab
- [MAJOR] buffer: flag BF_DONT_READ to disable reads when not required
- [MINOR] http: add new transaction flags for keep-alive and content-length
- [MEDIUM] http request: parse connection, content-length and transfer-encoding
- [MINOR] http request: update the TX_SRV_CONN_KA flag on rewrite
- [MINOR] http request: simplify the test of no-data
- [MEDIUM] http request: simplify POST length detection
- [MEDIUM] http request: make use of pre-parsed transfer-encoding header
- [MAJOR] http: create the analyser which waits for a response
- [MINOR] http: pre-set the persistent flags in the transaction
- [MEDIUM] http response: check body length and set transaction flags
- [MINOR] http response: update the TX_CLI_CONN_KA flag on rewrite
- [MINOR] http: remove the last call to stream_int_return
- [IMPORT] import ebtree v5.0 into directory ebtree/
- [MEDIUM] build: switch ebtree users to use new ebtree version
- [CLEANUP] ebtree: remove old unused files
- [BUG] definitely fix regparm issues between haproxy core and ebtree
- [CLEANUP] ebtree: cast to char * to get rid of gcc warning
- [BUILD] missing #ifndef in ebmbtree.h
- [BUILD] missing #ifndef in ebsttree.h
- [MINOR] tools: add hex2i() function to convert hex char to int
- [MINOR] http: create new MSG_BODY sub-states
- [BUG] stream_sock: BUF_INFINITE_FORWARD broke splice on 64-bit platforms
- [DOC] option is "defer-accept", not "defer_accept"
- [MINOR] http: keep pointer to beginning of data
- [BUG] x-original-to: name was not set in default instance
- [MINOR] http: detect tunnel mode and set it in the session
- [BUG] config: fix error message when config file is not found
- [BUG] config: fix wrong handling of too large argument count
- [BUG] config: disable 'option httplog' on TCP proxies
- [BUG] config: fix erroneous check on cookie domain names
- [BUG] config: cookie domain was ignored in defaults sections
- [MINOR] config: support passing multiple "domain" statements to cookies
- [MINOR] ebtree: add functions to lookup non-null terminated strings
- [MINOR] config: don't report error on all subsequent files on failure
- [BUG] second fix for the printf format warning
- [BUG] check_post: limit analysis to the buffer length
- [MEDIUM] http: process request body in a specific analyser
- [MEDIUM] backend: remove HTTP POST parsing from get_server_ph_post()
- [MAJOR] http: completely process the "connection" header
- [MINOR] http: only consider chunk encoding with HTTP/1.1
- [MAJOR] buffers: automatically compute the maximum buffer length
- [MINOR] http: move the http transaction init/cleanup code to proto_http
- [MINOR] http: move 1xx handling earlier to eliminate a lot of ifs
- [MINOR] http: introduce a new synchronisation state : HTTP_MSG_DONE
- [MEDIUM] http: rework chunk-size parser
- [MEDIUM] http: add a new transaction flags indicating if we know the transfer length
- [MINOR] buffers: add buffer_ignore() to skip some bytes
- [BUG] http: offsets are relative to the buffer, not to ->som
- [MEDIUM] http: automatically re-aling request buffer
- [BUG] http: body parsing must consider the start of message
- [MINOR] new function stream_int_cond_close()
- [MAJOR] http: implement body parser
- [BUG] http: typos on several unlikely() around header insertion
- [BUG] stream_sock: wrong max computation on recv
- [MEDIUM] http: rework the buffer alignment logic
- [BUG] buffers: wrong size calculation for displaced data
- [MINOR] stream_sock: prepare for closing when all pending data are sent
- [MEDIUM] http: add two more states for the closing period
- [MEDIUM] http: properly handle "option forceclose"
- [MINOR] stream_sock: add SI_FL_NOLINGER for faster close
- [MEDIUM] http: make forceclose use SI_FL_NOLINGER
- [MEDIUM] session: set SI_FL_NOLINGER when aborting on write timeouts
- [MEDIUM] http: add some SI_FL_NOLINGER around server errors
- [MINOR] config: option forceclose is valid in frontends too
- [BUILD] halog: insufficient include path in makefile
- [MEDIUM] http: make the analyser not rely on msg being initialized anymore
- [MEDIUM] http: make the parsers able to wait for a buffer flush
- [MAJOR] http: add support for option http-server-close
- [BUG] http: ensure we abort data transfer on write error
- [BUG] last fix was overzealous and disabled server-close
- [BUG] http: fix erroneous trailers size computation
- [MINOR] stream_sock: enable MSG_MORE when forwarding finite amount of data
- [OPTIM] http: set MSG_MORE on response when a pipelined request is pending
- [BUG] http: redirects were broken by chunk changes
- [BUG] http: the request URI pointer is relative to the buffer
- [OPTIM] http: don't immediately enable reading on request
- [MINOR] http: move redirect messages to HTTP/1.1 with a content-length
- [BUG] http: take care of errors, timeouts and aborts during the data phase
- [MINOR] http: don't wait for sending requests to the server
- [MINOR] http: make the conditional redirect support keep-alive
- [BUG] http: fix cookie parser to support spaces and commas in values
- [MINOR] config: some options were missing for "redirect"
- [MINOR] redirect: add support for unconditional rules
- [MINOR] config: centralize proxy struct initialization
- [MEDIUM] config: remove the limitation of 10 reqadd/rspadd statements
- [MEDIUM] config: remove the limitation of 10 config files
- [CLEANUP] http: remove a remaining impossible condition
- [OPTIM] http: optimize a bit the construct of the forward loops
This option enables HTTP keep-alive on the client side and close mode
on the server side. This offers the best latency on the slow client
side, and still saves as many resources as possible on the server side
by actively closing connections. Pipelining is supported on both requests
and responses, though there is currently no reason to get pipelined
responses.
This option was disabled for frontends in the configuration because
it was useless in its initial implementation, though it was still
checked in the code. Let's officially enable it now.
The "forceclose" option used to close the output channel to the
server once it started to respond. While this happened to work with
most servers, some of them considered this as a connection abort and
immediately stopped responding.
Now that we're aware of the end of a request and response, we're able
to trivially handle this option and properly close both sides when the
server's response is complete.
During this change it appeared that forwarding could be allowed when
the BF_SHUTW_NOW flag was set on a buffer, which obviously is not
acceptable and was causing some trouble. This has been fixed too and
is the reason for the MEDIUM status on this patch.
Implement decreasing health based on observing communication between
HAProxy and servers.
Changes in this version 2:
- documentation
- close race between a started check and health analysis event
- don't force fastinter if it is not set
- better names for options
- layer4 support
Changes in this version 3:
- add stats
- port to the current 1.4 tree
In some environments it is not possible to rely on any wildcard for a
domain name (eg: .com, .net, .fr...) so it is required to send multiple
domain extensions. (Un)fortunately the syntax check on the domain name
prevented that from being done the dirty way. So let's just build a
domain list when multiple domains are passed on the same line.
(cherry picked from commit 950245ca2b)
To sum up :
- len : it's now the max number of characters for the value, preventing
garbaged results.
- a new option "prefix" is added, this allows to use dynamic cookie
names (e.g. ASPSESSIONIDXXX).
Previously in the thread, I wanted to use the value found with
"capture cookie" but when i started to update the documentation, I
found this solution quite weird. I've made a small rework to not
depend on "capture cookie".
- There's the posssiblity to define the URL parser mode (path parameters
or query string).
This patch has 2 goals :
1. I wanted to test the appsession feature with a small PHP code,
using PHPSESSID. The problem is that when PHP gets an unknown session
id, it creates a new one with this ID. So, when sending an unknown
session to PHP, persistance is broken : haproxy won't see any new
cookie in the response and will never attach this session to a
specific server.
This also happens when you restart haproxy : the internal hash becomes
empty and all sessions loose their persistance (load balancing the
requests on all backend servers, creating a new session on each one).
For a user, it's like the service is unusable.
The patch modifies the code to make haproxy also learn the persistance
from the client : if no session is sent from the server, then the
session id found in the client part (using the URI or the client cookie)
is used to associated the server that gave the response.
As it's probably not a feature usable in all cases, I added an option
to enable it (by default it's disabled). The syntax of appsession becomes :
appsession <cookie> len <length> timeout <holdtime> [request-learn]
This helps haproxy repair the persistance (with the risk of losing its
session at the next request, as the user will probably not be load
balanced to the same server the first time).
2. This patch also tries to reduce the memory usage.
Here is a little example to explain the current behaviour :
- Take a Tomcat server where /session.jsp is valid.
- Send a request using a cookie with an unknown value AND a path
parameter with another unknown value :
curl -b "JSESSIONID=12345678901234567890123456789012" http://<haproxy>/session.jsp;jsessionid=00000000000000000000000000000001
(I know, it's unexpected to have a request like that on a live service)
Here, haproxy finds the URI session ID and stores it in its internal
hash (with no server associated). But it also finds the cookie session
ID and stores it again.
- As a result, session.jsp sends a new session ID also stored in the
internal hash, with a server associated.
=> For 1 request, haproxy has stored 3 entries, with only 1 which will be usable
The patch modifies the behaviour to store only 1 entry (maximum).
This can ensure that data is readily available on a socket when
we accept it, but a bug in the kernel ignores the timeout so the
socket can remain pending as long as the client does not talk.
Use with care.
Released version 1.4-dev4 with the following main changes :
- [DOC] add missing rate_lim and rate_max
- [MAJOR] struct chunk rework
- [MEDIUM] Health check reporting code rework + health logging, v3
- [BUG] check if rise/fall has an argument and it is > 0
- [MINOR] health checks logging unification
- [MINOR] add "description", "node" and show-node"/"show-desc", remove "node-name", v2
- [MINOR] Allow dots in show-node & add "white-space: nowrap" in th.pxname.
- [DOC] Add information about http://haproxy.1wt.eu/contrib.html
- [MINOR] Introduce include/types/counters.h
- [CLEANUP] Move counters to dedicated structures
- [MINOR] Add "clear counters" to clear statistics counters
- [MEDIUM] Collect & provide separate statistics for sockets, v2
- [BUG] Fix NULL pointer dereference in stats_check_uri_auth(), v2
- [MINOR] acl: don't report valid acls as potential mistakes
- [MINOR] Add cut_crlf(), ltrim(), rtrim() and alltrim()
- [MINOR] Add chunk_htmlencode and chunk_asciiencode
- [MINOR] Capture & display more data from health checks, v2
- [BUG] task.c: don't assing last_timer to node-less entries
- [BUG] http stats: large outputs sometimes got some parts chopped off
- [MINOR] backend: export some functions to recount servers
- [MINOR] backend: uninline some LB functions
- [MINOR] include time.h from freq_ctr.h as is uses "now".
- [CLEANUP] backend: move LB algos to individual files
- [MINOR] lb_map: reorder code in order to ease integration of new hash functions
- [CLEANUP] proxy: move last lb-specific bits to their respective files
- [MINOR] backend: separate declarations of LB algos from their lookup method
- [MINOR] backend: reorganize the LB algorithm selection
- [MEDIUM] backend: introduce the "static-rr" LB algorithm
- [MINOR] report list of supported pollers with -vv
- [DOC] log-health-checks is an option, not a directive
- [MEDIUM] new option "independant-streams" to stop updating read timeout on writes
- [BUG] stats: don't call buffer_shutw(), but ->shutw() instead
- [MINOR] stats: strip CR and LF from the input command line
- [BUG] don't refresh timeouts late after detected activity
- [MINOR] stats_dump_errors_to_buffer: use buffer_feed_chunk()
- [MINOR] stats_dump_sess_to_buffer: use buffer_feed_chunk()
- [MINOR] stats: make stats_dump_raw_to_buffer() use buffer_feed_chunk
- [MEDIUM] stats: don't use s->ana_state anymore
- [MINOR] remove now obsolete ana_state from the session struct
- [MEDIUM] stats: make HTTP stats use an I/O handler
- [MEDIUM] stream_int: adjust WAIT_ROOM handling
- [BUG] config: look for ID conflicts in all sockets, not only last ones.
- [MINOR] config: reference file and line with any listener/proxy/server declaration
- [MINOR] config: report places of duplicate names or IDs
- [MINOR] config: add pointer to file name in block/redirect/use_backend/monitor rules
- [MINOR] tools: add a new get_next_id() function
- [MEDIUM] config: automatically find unused IDs for proxies, servers and listeners
- [OPTIM] counters: move some max numbers to the counters struct
- [BUG] counters: fix segfault on missing counters for a listener
- [MEDIUM] backend: implement consistent hashing variation
- [MINOR] acl: add fe_conn, be_conn, queue, avg_queue
- [MINOR] stats: use 'clear counters all' to clear all values
- [MEDIUM] add access restrictions to the stats socket
- [MINOR] buffers: add buffer_feed2() and make buffer_feed() measure string length
- [MINOR] proxy: provide function to retrieve backend/server pointers
- [MINOR] add the "initial weight" to the server struct.
- [MEDIUM] stats: add the "get weight" command to report a server's weight
- [MEDIUM] stats: add the "set weight" command
- [BUILD] add a 'make tags' target
- [MINOR] stats: add support for numeric IDs in set weight/get weight
- [MINOR] stats: use a dedicated state to output static data
- [OPTIM] stats: check free space before trying to print
Krzysztof reported that using names only for get weight/set weight
was not enough because it's still possible to have multiple servers
with the same name (and my test config is one of those). He suggested
to be able to designate them by their unique numeric IDs by prefixing
the ID with a dash.
That way we can have :
set weight #120/#2
as well as
get weight static/srv1 10
