One year ago, commit 5d5b5d8 ("MEDIUM: proto_tcp: add support for tracking
L7 information") brought support for tracking L7 information in tcp-request
content rules. Two years earlier, commit 0a4838c ("[MEDIUM] session-counters:
correctly unbind the counters tracked by the backend") used to flush the
backend counters after processing a request.
While that earliest patch was correct at the time, it became wrong after
the second patch was merged. The code does what it says, but the concept
is flawed. "TCP request content" rules are evaluated for each HTTP request
over a single connection. So if such a rule in the frontend decides to
track any L7 information or to track L4 information when an L7 condition
matches, then it is applied to all requests over the same connection even
if they don't match. This means that a rule such as :
tcp-request content track-sc0 src if { path /index.html }
will count one request for index.html, and another one for each of the
objects present on this page that are fetched over the same connection
which sent the initial matching request.
Worse, it is possible to make the code do stupid things by using multiple
counters:
tcp-request content track-sc0 src if { path /foo }
tcp-request content track-sc1 src if { path /bar }
Just sending two requests first, one with /foo, one with /bar, shows
twice the number of requests for all subsequent requests. Just because
both of them persist after the end of the request.
So the decision to flush backend-tracked counters was not the correct
one. In practice, what is important is to flush countent-based rules
since they are the ones evaluated for each request.
Doing so requires new flags in the session however, to keep track of
which stick-counter was tracked by what ruleset. A later change might
make this easier to maintain over time.
This bug is 1.5-specific, no backport to stable is needed.
show pools
Dump the status of internal memory pools. This is useful to track memory
usage when suspecting a memory leak for example. It does exactly the same
as the SIGQUIT when running in foreground except that it does not flush
the pools.
Just like the previous commit, we sometimes want to limit the rate of
incoming SSL connections. While it can be done for a frontend, it was
not possible for a whole process, which makes sense when multiple
processes are running on a system to server multiple customers.
The new global "maxsslrate" setting is usable to fix a limit on the
session rate going to the SSL frontends. The limits applies before
the SSL handshake and not after, so that it saves the SSL stack from
expensive key computations that would finally be aborted before being
accounted for.
The same setting may be changed at run time on the CLI using
"set rate-limit ssl-session global".
It's sometimes useful to be able to limit the connection rate on a machine
running many haproxy instances (eg: per customer) but it removes the ability
for that machine to defend itself against a DoS. Thus, better also provide a
limit on the session rate, which does not include the connections rejected by
"tcp-request connection" rules. This permits to have much higher limits on
the connection rate without having to raise the session rate limit to insane
values.
The limit can be changed on the CLI using "set rate-limit sessions global",
or in the global section using "maxsessrate".
Patrick Hemmer reported that using unique_id_format and logs did not
report the same unique ID counter since commit 9f09521 ("BUG/MEDIUM:
unique_id: HTTP request counter must be unique!"). This is because
the increment was done while producing the log message, so it was
performed twice.
A better solution consists in fetching a new value once per request
and saving it in the request or session context for all of this
request's life.
It happens that sessions already have a unique ID field which is used
for debugging and reporting errors, and which differs from the one
sent in logs and unique_id header.
So let's change this to reuse this field to have coherent IDs everywhere.
As of now, a session gets a new unique ID once it is instanciated. This
means that TCP sessions will also benefit from a unique ID that can be
logged. And this ID is renewed for each extra HTTP request received on
an existing session. Thus, all TCP sessions and HTTP requests will have
distinct IDs that will be stable along all their life, and coherent
between all places where they're used (logs, unique_id header,
"show sess", "show errors").
This feature is 1.5-specific, no backport to 1.4 is needed.
In HTTP keep-alive mode, if we receive a 401, we still have a chance
of being able to send the visitor again to the same server over the
same connection. This is required by some broken protocols such as
NTLM, and anyway whenever there is an opportunity for sending the
challenge to the proper place, it's better to do it (at least it
helps with debugging).
Released version 1.5-dev21 with the following main changes :
- MINOR: stats: don't use a monospace font to report numbers
- MINOR: session: remove debugging code
- BUG/MAJOR: patterns: fix double free caused by loading strings from files
- MEDIUM: http: make option http_proxy automatically rewrite the URL
- BUG/MEDIUM: http: cook_cnt() forgets to set its output type
- BUG/MINOR: stats: correctly report throttle rate of low weight servers
- BUG/MEDIUM: checks: servers must not start in slowstart mode
- BUG/MINOR: acl: parser must also stop at comma on ACL-only keywords
- MEDIUM: stream-int: implement a very simplistic idle connection manager
- DOC: update the ROADMAP file
Released version 1.5-dev20 with the following main changes :
- DOC: add missing options to the manpage
- DOC: add manpage references to all system calls
- DOC: update manpage reference to haproxy-en.txt
- DOC: remove -s and -l options from the manpage
- DOC: missing information for the "description" keyword
- DOC: missing http-send-name-header keyword in keyword table
- MINOR: tools: function my_memmem() to lookup binary contents
- MEDIUM: checks: add send/expect tcp based check
- MEDIUM: backend: Enhance hash-type directive with an algorithm options
- MEDIUM: backend: Implement avalanche as a modifier of the hashing functions.
- DOC: Documentation for hashing function, with test results.
- BUG/MEDIUM: ssl: potential memory leak using verifyhost
- BUILD: ssl: compilation issue with openssl v0.9.6.
- BUG/MINOR: ssl: potential memory leaks using ssl_c_key_alg or ssl_c_sig_alg.
- MINOR: ssl: optimization of verifyhost on wildcard certificates.
- BUG/MINOR: ssl: verifyhost does not match empty strings on wildcard.
- MINOR: ssl: Add statement 'verifyhost' to "server" statements
- CLEANUP: session: remove event_accept() which was not used anymore
- BUG/MINOR: deinit: free fdinfo while doing cleanup
- DOC: minor typo fix in documentation
- BUG/MEDIUM: server: set the macro for server's max weight SRV_UWGHT_MAX to SRV_UWGHT_RANGE
- BUG/MINOR: use the same check condition for server as other algorithms
- DOC: fix typo in comments
- BUG/MINOR: deinit: free server map which is allocated in init_server_map()
- CLEANUP: stream_interface: cleanup loop information in si_conn_send_loop()
- MINOR: buffer: align the last output line of buffer_dump()
- MINOR: buffer: align the last output line if there are less than 8 characters left
- DOC: stick-table: modify the description
- OPTIM: stream_interface: return directly if the connection flag CO_FL_ERROR has been set
- CLEANUP: code style: use tabs to indent codes
- DOC: checkcache: block responses with cacheable cookies
- BUG/MINOR: check_config_validity: check the returned value of stktable_init()
- MEDIUM: haproxy-systemd-wrapper: Use haproxy in same directory
- MEDIUM: systemd-wrapper: Kill child processes when interrupted
- LOW: systemd-wrapper: Write debug information to stdout
- BUG/MINOR: http: fix "set-tos" not working in certain configurations
- MEDIUM: http: add IPv6 support for "set-tos"
- DOC: ssl: update build instructions to use new SSL_* variables
- BUILD/MINOR: systemd: fix compiler warning about unused result
- url32+src - like base32+src but whole url including parameters
- BUG/MINOR: fix forcing fastinter in "on-error"
- CLEANUP: Make parameters of srv_downtime and srv_getinter const
- CLEANUP: Remove unused 'last_slowstart_change' field from struct peer
- MEDIUM: Split up struct server's check element
- MEDIUM: Move result element to struct check
- MEDIUM: Paramatise functions over the check of a server
- MEDIUM: cfgparse: Factor out check initialisation
- MEDIUM: Add state to struct check
- MEDIUM: Move health element to struct check
- MEDIUM: Add helper for task creation for checks
- MEDIUM: Add helper function for failed checks
- MEDIUM: Log agent fail, stopped or down as info
- MEDIUM: Remove option lb-agent-chk
- MEDIUM: checks: Add supplementary agent checks
- MEDIUM: Do not mark a server as down if the agent is unavailable
- MEDIUM: Set rise and fall of agent checks to 1
- MEDIUM: Add enable and disable agent unix socket commands
- MEDIUM: Add DRAIN state and report it on the stats page
- BUILD/MINOR: missing header file
- CLEANUP: regex: Create regex_comp function that compiles regex using compilation options
- CLEANUP: The function "regex_exec" needs the string length but in many case they expect null terminated char.
- MINOR: http: some exported functions were not in the header file
- MINOR: http: change url_decode to return the size of the decoded string.
- BUILD/MINOR: missing header file
- BUG/MEDIUM: sample: The function v4tov6 cannot support input and output overlap
- BUG/MINOR: arg: fix error reporting for add-header/set-header sample fetch arguments
- MINOR: sample: export the generic sample conversion parser
- MINOR: sample: export sample_casts
- MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()' into the ACL keyword
- MINOR: stick-table: use smp_expr_output_type() to retrieve the output type of a "struct sample_expr"
- MINOR: sample: provide the original sample_conv descriptor struct to the argument checker function.
- MINOR: tools: Add a function to convert buffer to an ipv6 address
- MINOR: acl: export acl arrays
- MINOR: acl: Extract the pattern parsing and indexation from the "acl_read_patterns_from_file()" function
- MINOR: acl: Extract the pattern matching function
- MINOR: sample: Define new struct sample_storage
- MEDIUM: acl: associate "struct sample_storage" to each "struct acl_pattern"
- REORG: acl/pattern: extract pattern matching from the acl file and create pattern.c
- MEDIUM: pattern: create pattern expression
- MEDIUM: pattern: rename "acl" prefix to "pat"
- MEDIUM: sample: let the cast functions set their output type
- MINOR: sample: add a private field to the struct sample_conv
- MINOR: map: Define map types
- MEDIUM: sample: add the "map" converter
- MEDIUM: http: The redirect strings follows the log format rules.
- BUG/MINOR: acl: acl parser does not recognize empty converter list
- BUG/MINOR: map: The map list was declared in the map.h file
- MINOR: map: Cleanup the initialisation of map descriptors.
- MEDIUM: map: merge identical maps
- BUG/MEDIUM: pattern: Pattern node has type of "struct pat_idx_elt" in place of "struct eb_node"
- BUG/MEDIUM: map: Bad map file parser
- CLEANUP/MINOR: standard: use the system define INET6_ADDRSTRLEN in place of MAX_IP6_LEN
- BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end
- MINOR: map: export map_get_reference() function
- MINOR: pattern: Each pattern sets the expected input type
- MEDIUM: acl: Last patch change the output type
- MEDIUM: pattern: Extract the index process from the pat_parse_*() functions
- MINOR: standard: The function parse_binary() can use preallocated buffer
- MINOR: regex: Change the struct containing regex
- MINOR: regex: Copy the original regex expression into string.
- MINOR: pattern: add support for compiling patterns for lookups
- MINOR: pattern: make the pattern matching function return a pointer to the matched element
- MINOR: map: export parse output sample functions
- MINOR: pattern: add function to lookup a specific entry in pattern list
- MINOR: pattern/map: Each pattern must free the associated sample
- MEDIUM: dumpstat: make the CLI parser understand the backslash as an escape char
- MEDIUM: map: dynamic manipulation of maps
- BUG/MEDIUM: unique_id: junk in log on empty unique_id
- BUG/MINOR: log: junk at the end of syslog packet
- MINOR: Makefile: provide cscope rule
- DOC: compression: chunk are not compressed anymore
- MEDIUM: session: disable lingering on the server when the client aborts
- BUG/MEDIUM: prevent gcc from moving empty keywords lists into BSS
- DOC: remove the comment saying that SSL certs are not checked on the server side
- BUG: counters: third counter was not stored if others unset
- BUG/MAJOR: http: don't emit the send-name-header when no server is available
- BUG/MEDIUM: http: "option checkcache" fails with the no-cache header
- BUG/MAJOR: http: sample prefetch code was not properly migrated
- BUG/MEDIUM: splicing: fix abnormal CPU usage with splicing
- BUG/MINOR: stream_interface: don't call chk_snd() on polled events
- OPTIM: splicing: use splice() for the last block when relevant
- MEDIUM: sample: handle comma-delimited converter list
- MINOR: sample: fix sample_process handling of unstable data
- CLEANUP: acl: move the 3 remaining sample fetches to samples.c
- MINOR: sample: add a new "date" fetch to return the current date
- MINOR: samples: add the http_date([<offset>]) sample converter.
- DOC: minor improvements to the part on the stats socket.
- MEDIUM: sample: systematically pass the keyword pointer to the keyword
- MINOR: payload: split smp_fetch_rdp_cookie()
- MINOR: counters: factor out smp_fetch_sc*_tracked
- MINOR: counters: provide a generic function to retrieve a stkctr for sc* and src.
- MEDIUM: counters: factor out smp_fetch_sc*_get_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_gpc0_rate
- MEDIUM: counters: factor out smp_fetch_sc*_inc_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_clr_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_conn_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_conn_rate
- MEDIUM: counters: factor out smp_fetch_sc*_conn_cur
- MEDIUM: counters: factor out smp_fetch_sc*_sess_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_sess_rate
- MEDIUM: counters: factor out smp_fetch_sc*_http_req_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_http_req_rate
- MEDIUM: counters: factor out smp_fetch_sc*_http_err_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_http_err_rate
- MEDIUM: counters: factor out smp_fetch_sc*_kbytes_in
- MEDIUM: counters: factor out smp_fetch_sc*_bytes_in_rate
- MEDIUM: counters: factor out smp_fetch_sc*_kbytes_out
- MEDIUM: counters: factor out smp_fetch_sc*_bytes_out_rate
- MEDIUM: counters: factor out smp_fetch_sc*_trackers
- MINOR: session: make the number of stick counter entries more configurable
- MEDIUM: counters: support passing the counter number as a fetch argument
- MEDIUM: counters: support looking up a key in an alternate table
- MEDIUM: cli: adjust the method for feeding frequency counters in tables
- MINOR: cli: make it possible to enter multiple values at once with "set table"
- MINOR: payload: allow the payload sample fetches to retrieve arbitrary lengths
- BUG/MINOR: cli: "clear table" must not kill entries that don't match condition
- MINOR: ssl: use MAXPATHLEN instead of PATH_MAX
- MINOR: config: warn when a server with no specific port uses rdp-cookie
- BUG/MEDIUM: unique_id: HTTP request counter must be unique!
- DOC: add a mention about the limited chunk size
- BUG/MEDIUM: fix broken send_proxy on FreeBSD
- MEDIUM: stick-tables: flush old entries upon soft-stop
- MINOR: tcp: add new "close" action for tcp-response
- MINOR: payload: provide the "res.len" fetch method
- BUILD: add SSL_INC/SSL_LIB variables to force the path to openssl
- MINOR: http: compute response time before processing headers
- BUG/MINOR: acl: fix improper string size assignment in proxy argument
- BUG/MEDIUM: http: accept full buffers on smp_prefetch_http
- BUG/MINOR: acl: implicit arguments of ACL keywords were not properly resolved
- BUG/MEDIUM: session: risk of crash on out of memory conditions
- BUG/MINOR: peers: set the accept date in outgoing connections
- BUG/MEDIUM: tcp: do not skip tracking rules on second pass
- BUG/MEDIUM: acl: do not evaluate next terms after a miss
- MINOR: acl: add a warning when an ACL keyword is used without any value
- MINOR: tcp: don't use tick_add_ifset() when timeout is known to be set
- BUG/MINOR: acl: remove patterns from the tree before freeing them
- MEDIUM: backend: add support for the wt6 hash
- OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes
- OPTIM/MINOR: mark the source address as already known on accept()
- BUG/MINOR: stats: don't count tarpitted connections twice
- CLEANUP: http: homogenize processing of denied req counter
- CLEANUP: http: merge error handling for req* and http-request *
- BUG/MEDIUM: http: fix possible parser crash when parsing erroneous "http-request redirect" rules
- BUG/MINOR: http: fix build warning introduced with url32/url32_src
- BUG/MEDIUM: checks: fix slow start regression after fix attempt
- BUG/MAJOR: server: weight calculation fails for map-based algorithms
- MINOR: stats: report correct throttling percentage for servers in slowstart
- OPTIM: connection: fold the error handling with handshake handling
- MINOR: peers: accept to learn strings of different lengths
- BUG/MAJOR: fix haproxy crash when using server tracking instead of checks
- BUG/MAJOR: check: fix haproxy crash during soft-stop/soft-start
- BUG/MINOR: stats: do not report "via" on tracking servers in maintenance
- BUG/MINOR: connection: fix typo in error message report
- BUG/MINOR: backend: fix target address retrieval in transparent mode
- BUG/MINOR: config: report the correct track-sc number in tcp-rules
- BUG/MINOR: log: fix log-format parsing errors
- DOC: add some information about how to apply converters to samples
- MINOR: acl/pattern: use types different from int to clarify who does what.
- MINOR: pattern: import acl_find_match_name() into pattern.h
- MEDIUM: stick-tables: support automatic conversion from ipv4<->ipv6
- MEDIUM: log-format: relax parsing of '%' followed by unsupported characters
- BUG/MINOR: http: usual deinit stuff in last commit
- BUILD: log: silent a warning about isblank() with latest patches
- BUG/MEDIUM: checks: fix health check regression causing them to depend on declaration order
- BUG/MEDIUM: checks: fix a long-standing issue with reporting connection errors
- BUG/MINOR: checks: don't consider errno and use conn->err_code
- BUG/MEDIUM: checks: also update the DRAIN state from the web interface
- MINOR: stats: remove some confusion between the DRAIN state and NOLB
- BUG/MINOR: tcp: check that no error is pending during a connect probe
- BUG/MINOR: connection: check EINTR when sending a PROXY header
- MEDIUM: connection: set the socket shutdown flags on socket errors
- BUG/MEDIUM: acl: fix regression introduced by latest converters support
- MINOR: connection: clear errno prior to checking for errors
- BUG/MINOR: checks: do not trust errno in write event before any syscall
- MEDIUM: checks: centralize error reporting
- OPTIM: checks: don't poll on recv when using plain TCP connects
- OPTIM: checks: avoid setting SO_LINGER twice
- MINOR: tools: add a generic binary hex string parser
- BUG/MEDIUM: checks: tcp-check: do not poll when there's nothing to send
- BUG/MEDIUM: check: tcp-check might miss some outgoing data when socket buffers are full
- BUG/MEDIUM: args: fix double free on error path in argument expression parser
- BUG/MINOR: acl: fix sample expression error reporting
- BUG/MINOR: checks: tcp-check actions are enums, not flags
- MEDIUM: checks: make tcp-check perform multiple send() at once
- BUG/MEDIUM: stick: completely remove the unused flag from the store entries
- OPTIM: ebtree: pack the struct eb_node to avoid holes on 64-bit
- BUG/MEDIUM: stick-tables: complete the latest fix about store-responses
- CLEANUP: stream_interface: remove unused field err_loc
- MEDIUM: stats: don't use conn->xprt_st anymore
- MINOR: session: add a simple function to retrieve a session from a task
- MEDIUM: stats: don't use conn->xprt_ctx anymore
- MEDIUM: peers: don't rely on conn->xprt_ctx anymore
- MINOR: http: prevent smp_fetch_url_{ip,port} from using si->conn
- MINOR: connection: make it easier to emit proxy protocol for unknown addresses
- MEDIUM: stats: prepare the HTTP stats I/O handler to support more states
- MAJOR: stats: move the HTTP stats handling to its applet
- MEDIUM: stats: move request argument processing to the final step
- MEDIUM: session: detect applets from the session by using s->target
- MAJOR: session: check for a connection to an applet in sess_prepare_conn_req()
- MAJOR: session: pass applet return traffic through the response analysers
- MEDIUM: stream-int: split the shutr/shutw functions between applet and conn
- MINOR: stream-int: make the shutr/shutw functions void
- MINOR: obj: provide a safe and an unsafe access to pointed objects
- MINOR: connection: add a field to store an object type
- MINOR: connection: always initialize conn->objt_type to OBJ_TYPE_CONN
- MEDIUM: stream interface: move the peers' ptr into the applet context
- MINOR: stream-interface: move the applet context to its own struct
- MINOR: obj: introduce a new type appctx
- MINOR: stream-int: rename ->applet to ->appctx
- MINOR: stream-int: split si_prepare_embedded into si_prepare_none and si_prepare_applet
- MINOR: stream-int: add a new pointer to the end point
- MEDIUM: stream-interface: set the pointer to the applet into the applet context
- MAJOR: stream interface: remove the ->release function pointer
- MEDIUM: stream-int: make ->end point to the connection or the appctx
- CLEANUP: stream-int: remove obsolete si_ctrl function
- MAJOR: stream-int: stop using si->conn and use si->end instead
- MEDIUM: stream-int: do not allocate a connection in parallel to applets
- MEDIUM: session: attach incoming connection to target on embryonic sessions
- MINOR: connection: add conn_init() to (re)initialize a connection
- MINOR: checks: call conn_init() to properly initialize the connection.
- MINOR: peers: make use of conn_init() to initialize the connection
- MINOR: session: use conn_init() to initialize the connections
- MINOR: http: use conn_init() to reinitialize the server connection
- MEDIUM: connection: replace conn_prepare with conn_assign
- MINOR: get rid of si_takeover_conn()
- MINOR: connection: add conn_new() / conn_free()
- MAJOR: connection: add two new flags to indicate readiness of control/transport
- MINOR: stream-interface: introduce si_reset() and si_set_state()
- MINOR: connection: reintroduce conn_prepare to set the protocol and transport
- MINOR: connection: replace conn_assign with conn_attach
- MEDIUM: stream-interface: introduce si_attach_conn to replace si_prepare_conn
- MAJOR: stream interface: dynamically allocate the outgoing connection
- MEDIUM: connection: move the send_proxy offset to the connection
- MINOR: connection: check for send_proxy during the connect(), not the SI
- MEDIUM: connection: merge the send_proxy and local_send_proxy calls
- MEDIUM: stream-int: replace occurrences of si->appctx with si_appctx()
- MEDIUM: stream-int: return the allocated appctx in stream_int_register_handler()
- MAJOR: stream-interface: dynamically allocate the applet context
- MEDIUM: session: automatically register the applet designated by the target
- MEDIUM: stats: delay appctx initialization
- CLEANUP: peers: use less confusing state/status code names
- MEDIUM: peers: delay appctx initialization
- MINOR: stats: provide some appctx information in "show sess all"
- DIET/MINOR: obj: pack the obj_type enum to 8 bits
- DIET/MINOR: connection: rearrange a few fields to save 8 bytes in the struct
- DIET/MINOR: listener: rearrange a few fields in struct listener to save 16 bytes
- DIET/MINOR: proxy: rearrange a few fields in struct proxy to save 16 bytes
- DIET/MINOR: session: reduce the struct session size by 8 bytes
- DIET/MINOR: stream-int: rearrange a few fields in struct stream_interface to save 8 bytes
- DIET/MINOR: http: reduce the size of struct http_txn by 8 bytes
- MINOR: http: switch the http state to an enum
- MINOR: http: use an enum for the auth method in http_auth_data
- DIET/MINOR: task: reduce struct task size by 8 bytes
- MINOR: stream_interface: add reporting of ressouce allocation errors
- MINOR: session: report lack of resources using the new stream-interface's error code
- BUILD: simplify the date and version retrieval in the makefile
- BUILD: prepare the makefile to skip format lines in SUBVERS and VERDATE
- BUILD: use format tags in VERDATE and SUBVERS files
- BUG/MEDIUM: channel: bo_getline() must wait for \n until buffer is full
- CLEANUP: check: server port is unsigned
- BUG/MEDIUM: checks: agent doesn't get the response if server does not closes
- MINOR: tools: buf2ip6 must not modify output on failure
- MINOR: pattern: do not assign SMP_TYPES by default to patterns
- MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors
- MINOR: arg: improve wording on error reporting
- BUG/MEDIUM: sample: simplify and fix the argument parsing
- MEDIUM: acl: fix the argument parser to let the lower layer report detailed errors
- MEDIUM: acl: fix the initialization order of the ACL expression
- CLEANUP: acl: remove useless blind copy-paste from sample converters
- TESTS: add regression tests for ACL and sample expression parsers
- BUILD: time: adapt the type of TV_ETERNITY to the local system
- MINOR: chunks: allocate the trash chunks before parsing the config
- BUILD: definitely silence some stupid GCC warnings
- MINOR: chunks: always initialize the output chunk in get_trash_chunk()
- MINOR: checks: improve handling of the servers tracking chain
- REORG: checks: retrieve the check-specific defines from server.h to checks.h
- MINOR: checks: use an enum instead of flags to report a check result
- MINOR: checks: rename the state flags
- MINOR: checks: replace state DISABLED with CONFIGURED and ENABLED
- MINOR: checks: use check->state instead of srv->state & SRV_CHECKED
- MINOR: checks: fix agent check interval computation
- MINOR: checks: add a PAUSED state for the checks
- MINOR: checks: create the agent tasks even when no check is configured
- MINOR: checks: add a flag to indicate what check is an agent
- MEDIUM: checks: enable agent checks even if health checks are disabled
- BUG/MEDIUM: checks: ensure we can enable a server after boot
- BUG/MEDIUM: checks: tracking servers must not inherit the MAINT flag
- BUG/MAJOR: session: repair tcp-request connection rules
- BUILD: fix SUBVERS extraction in the Makefile
- BUILD: pattern: silence a warning about uninitialized value
- BUILD: log: fix build warning on Solaris
- BUILD: dumpstats: fix build error on Solaris
- DOC: move option pgsql-check to the correct place
- DOC: move option tcp-check to the proper place
- MINOR: connection: add simple functions to report connection readiness
- MEDIUM: connection: centralize handling of nolinger in fd management
- OPTIM: http: set CF_READ_DONTWAIT on response message
- OPTIM: http: do not re-enable reading on client side while closing the server side
- MINOR: config: add option http-keep-alive
- MEDIUM: connection: inform si_alloc_conn() whether existing conn is OK or not
- MAJOR: stream-int: handle the connection reuse in si_connect()
- MAJOR: http: add the keep-alive transition on the server side
- MAJOR: backend: enable connection reuse
- MINOR: http: add option prefer-last-server
- MEDIUM: http: do not report connection errors for second and further requests
When the load balancing algorithm in use is not deterministic, and a previous
request was sent to a server to which haproxy still holds a connection, it is
sometimes desirable that subsequent requests on a same session go to the same
server as much as possible. Note that this is different from persistence, as
we only indicate a preference which haproxy tries to apply without any form
of warranty. The real use is for keep-alive connections sent to servers. When
this option is used, haproxy will try to reuse the same connection that is
attached to the server instead of rebalancing to another server, causing a
close of the connection. This can make sense for static file servers. It does
not make much sense to use this in combination with hashing algorithms.
This new option enables HTTP keep-alive processing on the connections.
It can be overwritten by http-server-close, httpclose and forceclose.
Right now full-chain keep-alive is not yet implemented, but we need
the option to work on it. The doc will come later.
This patch adds map manipulation commands to the socket interface.
add map <map> <key> <value>
Add the value <value> in the map <map>, at the entry corresponding to
the key <key>. This command does not verify if the entry already
exists.
clear map <map>
Remove entries from the map <map>
del map <map> <key>
Delete all the map entries corresponding to the <key> value in the map
<map>.
set map <map> <key> <value>
Modify the value corresponding to each key <key> in a map <map>. The
new value is <value>.
show map [<map>]
Dump info about map converters. Without argument, the list of all
available maps are returned. If a <map> is specified, is content is
dumped.
This patch remove the compatibility check from the input type and the
match method. Now, it checks if a casts from the input type to output
type exists and the pattern_exec_match() function apply casts before
each pattern matching.
The commit 37e340c (BUG/MEDIUM: stick: completely remove the unused flag
from the store entries) was incomplete. We also need to ensure that only
the first store-response for a table is applied and that it may coexist
with a possible store-request that was already done on this table.
This patch with the previous one should be backported to 1.4.
This is a generic health check which can be used to match a
banner or send a request and analyse a server response.
It works in a send/expect ways and many exchange can be done between
HAProxy and a server to decide the server status, making HAProxy able to
speak the server's protocol.
It can send arbitrary regular or binary strings and match content as a
regular or binary string or a regex.
Signed-off-by: Baptiste Assmann <bedis9@gmail.com>
The stickiness table can be declared in such sections as frontend, listen
and backend, but the original manual only mentioned backend. Modify the
description simply as below:
"current backend" -> "current section"
Signed-off-by: Godbach <nylzhaowei@gmail.com>
We handle "http-request redirect" with a log-format string now, but we
leave "redirect" unaffected.
Note that the control of the special "/" case is move from the runtime
execution to the configuration parsing. If the format rule list is
empty, the build_logline() function does nothing.
At the moment when a '%' character is followed by any unhandled character,
it is considered as a variable name, and if it cannot be resolved, a warning
is emitted and the configuration goes on.
When we start using log-format for redirect rules, it may happen that some
people accidently use '%' instead of '%%' without understanding the cause
of the issue. Thus we do two things here :
- if a single '%' is followed by a blank or a digit, we fix it and emit a
warning explaining how this should be done ; this ensures that existing
configs continue to work ;
- if a single '%' is followed by an unknown variable name, we report it
and explain how to emit a verbatim '%' in case this is what the user
desired.
Add a new converter with the following prototype :
map(<map_file>[,<default_value>])
map_<match_type>(<map_file>[,<default_value>])
map_<match_type>_<output_type>(<map_file>[,<default_value>])
It searches the for input value from <map_file> using the <match_type>
matching method, and return the associated value converted to the type
<output_type>. If the input value cannot be found in the <map_file>,
the converter returns the <default_value>. If the <default_value> is
not set, the converter fails and acts as if no input value could be
fetched. If the <match_type> is not set, it defaults to "str".
Likewise, if the <output_type> is not set, it defaults to "str". For
convenience, the "map" keyword is an alias for "map_str" and maps a
string to another string. The following array contains contains the
list of all the map* converters.
+----+----------+---------+-------------+------------+
| `-_ out | | | |
| input `-_ | str | int | ip |
| / match `-_ | | | |
+---------------+---------+-------------+------------+
| str / str | map_str | map_str_int | map_str_ip |
| str / sub | map_sub | map_sub_int | map_sub_ip |
| str / dir | map_dir | map_dir_int | map_dir_ip |
| str / dom | map_dom | map_dom_int | map_dom_ip |
| str / end | map_end | map_end_int | map_end_ip |
| str / reg | map_reg | map_reg_int | map_reg_ip |
| int / int | map_int | map_int_int | map_int_ip |
| ip / ip | map_ip | map_ip_int | map_ip_ip |
+---------------+---------+-------------+------------+
The names are intentionally chosen to reflect the same match methods
as ACLs use.
We've had the feature for log-format, unique-id-format and add-header for
a while now. It has just been implemented for ACLs but some doc was still
lacking.
The syntax of this new commands are:
enable agent <backend>/<server>
disable agent <backend>/<server>
These commands allow temporarily stopping and subsequently
re-starting an auxiliary agent check. The effect of this is as follows:
New checks are only initialised when the agent is in the enabled. Thus,
disable agent will prevent any new agent checks from begin initiated until
the agent re-enabled using enable agent.
When an agent is disabled the processing of an auxiliary agent check that
was initiated while the agent was set as enabled is as follows: All
results that would alter the weight, specifically "drain" or a weight
returned by the agent, are ignored. The processing of agent check is
otherwise unchanged.
The motivation for this feature is to allow the weight changing effects
of the agent checks to be paused to allow the weight of a server to be
configured using set weight without being overridden by the agent.
Signed-off-by: Simon Horman <horms@verge.net.au>
In the case where agent-port is used and the agent
check is a secondary check to not mark a server as down
if the agent becomes unavailable.
In this configuration the agent should only cause a server to be marked
as down if the agent returns "fail", "stopped" or "down".
Signed-off-by: Simon Horman <horms@verge.net.au>
Allow an auxiliary agent check to be run independently of the
regular a regular health check. This is enabled by the agent-check
server setting.
The agent-port, which specifies the TCP port to use for the agent's
connections, is required.
The agent-inter, which specifies the interval between agent checks and
timeout of agent checks, is optional. If not set the value for regular
checks is used.
e.g.
server web1_1 127.0.0.1:80 check agent-port 10000
If either the health or agent check determines that a server is down
then it is marked as being down, otherwise it is marked as being up.
An agent health check performed by opening a TCP socket and reading an
ASCII string. The string should have one of the following forms:
* An ASCII representation of an positive integer percentage.
e.g. "75%"
Values in this format will set the weight proportional to the initial
weight of a server as configured when haproxy starts.
* The string "drain".
This will cause the weight of a server to be set to 0, and thus it
will not accept any new connections other than those that are
accepted via persistence.
* The string "down", optionally followed by a description string.
Mark the server as down and log the description string as the reason.
* The string "stopped", optionally followed by a description string.
This currently has the same behaviour as "down".
* The string "fail", optionally followed by a description string.
This currently has the same behaviour as "down".
Signed-off-by: Simon Horman <horms@verge.net.au>
Remove option lb-agent-chk and thus the facility to configure
a stand-alone agent health check. This feature was added by
"MEDIUM: checks: Add agent health check". It will be replaced
by subsequent patches with a features to allow an agent check
to be run as either a secondary check, along with any of the existing
checks, or as part of an http check with the status returned
in an HTTP header.
This patch does not entirely revert "MEDIUM: checks: Add agent health
check". The infrastructure it provides to parse the results of an
agent health check remains and will be re-used by the planned features
that are mentioned above.
Signed-off-by: Simon Horman <horms@verge.net.au>
This function was designed for haproxy while testing other functions
in the past. Initially it was not planned to be used given the not
very interesting numbers it showed on real URL data : it is not as
smooth as the other ones. But later tests showed that the other ones
are extremely sensible to the server count and the type of input data,
especially DJB2 which must not be used on numeric input. So in fact
this function is still a generally average performer and it can make
sense to merge it in the end, as it can provide an alternative to
sdbm+avalanche or djb2+avalanche for consistent hashing or when hashing
on numeric data such as a source IP address or a visitor identifier in
a URL parameter.
Summary:
Avalanche is supported not as a native hashing choice, but a modifier
on the hashing function. Note that this means that possible configs
written after 1.5-dev4 using "hash-type avalanche" will get an informative
error instead. But as discussed on the mailing list it seems nobody ever
used it anyway, so let's fix it before the final 1.5 release.
The default values were selected for backward compatibility with previous
releases, as discussed on the mailing list, which means that the consistent
hashing will still apply the avalanche hash by default when no explicit
algorithm is specified.
Examples
(default) hash-type map-based
Map based hashing using sdbm without avalanche
(default) hash-type consistent
Consistent hashing using sdbm with avalanche
Additional Examples:
(a) hash-type map-based sdbm
Same as default for map-based above
(b) hash-type map-based sdbm avalanche
Map based hashing using sdbm with avalanche
(c) hash-type map-based djb2
Map based hashing using djb2 without avalanche
(d) hash-type map-based djb2 avalanche
Map based hashing using djb2 with avalanche
(e) hash-type consistent sdbm avalanche
Same as default for consistent above
(f) hash-type consistent sdbm
Consistent hashing using sdbm without avalanche
(g) hash-type consistent djb2
Consistent hashing using djb2 without avalanche
(h) hash-type consistent djb2 avalanche
Consistent hashing using djb2 with avalanche
Summary:
In testing at tumblr, we found that using djb2 hashing instead of the
default sdbm hashing resulted is better workload distribution to our backends.
This commit implements a change, that allows the user to specify the hash
function they want to use. It does not limit itself to consistent hashing
scenarios.
The supported hash functions are sdbm (default), and djb2.
For a discussion of the feature and analysis, see mailing list thread
"Consistent hashing alternative to sdbm" :
http://marc.info/?l=haproxy&m=138213693909219
Note: This change does NOT make changes to new features, for instance,
applying an avalance hashing always being performed before applying
consistent hashing.
This fetch method returns the response buffer len, similarly
to req.len for the request. Previously it was only possible
to rely on "res.payload(0,size) -m found" to find if at least
that amount of data was available, which was a bit tricky.
This new action immediately closes the connection with the server
when the condition is met. The first such rule executed ends the
rules evaluation. The main purpose of this action is to force a
connection to be finished between a client and a server after an
exchange when the application protocol expects some long time outs
to elapse first. The goal is to eliminate idle connections which
take signifiant resources on servers with certain protocols.
verifyhost allows you to specify a hostname that the remote server's
SSL certificate must match. Connections that don't match will be
closed with an SSL error.
When using req.payload and res.payload to look up for specific content at an
arbitrary location, we're often facing the problem of not knowing the input
buffer length. If the length argument is larger than the buffer length, the
function did not match, and if they're smaller, there is a risk of not getting
the expected content. This is especially true when looking for data in SOAP
requests.
So let's make some provisions for scanning the whole buffer by specifying a
length of 0 bytes. This greatly simplifies the processing of random-sized
input data.
The "set table" statement allows to create new entries with their respective
values. Till now it was limited to a single data type per line, requiring as
many "set table" statements as the desired data types to be set. Since this
is only a parser limitation, this patch gets rid of it. It also allows the
creation of a key with no data types (all reset to their default values).
sc_* sample fetches now take an optional parameter which allows to look
the key in an alternate table. This is convenient to pass multiple
information for the same key at once (eg: have multiple gpc0 for the
same key, or support being fed complementary information from the CLI).
Example :
listen front
bind :8000
tcp-request content track-sc0 src table local-ip
http-response set-header src-id %[sc0_get_gpc0]+%[sc0_get_gpc0(global-ip)]
server dummy 127.0.0.1:8001
backend local-ip
stick-table size 1k type ip store gpc0
backend global-ip
stick-table size 1k type ip store gpc0
One very annoying issue when trying to extend the sticky counters beyond
the current 3 counters is that it requires a massive copy-paste of fetch
functions (we don't have to copy-paste code anymore), just so that the
fetch names exist.
So let's have an alternate form like "sc_*(num)" to allow passing the
counter number as an argument without having to redefine new fetch names.
The MAX_SESS_STKCTR macro defines the number of usable sticky counters,
which defaults to 3.
Converts an integer supposed to contain a date since epoch to
a string representing this date in a format suitable for use
in HTTP header fields. If an offset value is specified, then
it is a number of seconds that is added to the date before the
conversion is operated. This is particularly useful to emit
Date header fields, Expires values in responses when combined
with a positive offset, or Last-Modified values when the
offset is negative.
Returns the current date as the epoch (number of seconds since 01/01/1970).
If an offset value is specified, then it is a number of seconds that is added
to the current date before returning the value. This is particularly useful
to compute relative dates, as both positive and negative offsets are allowed.
We now support having a comma-delimited converter list, which can start
right after the fetch keyword. The immediate benefit is that it allows
to use converters in log-format expressions, for example :
set-header source-net %[src,ipmask(24)]
The parser is also slightly improved and should be more resilient against
configuration errors. Also, optional arguments in converters were mistakenly
not allowed till now, so this was fixed.
The max weight of server is 256 now, but SRV_UWGHT_MAX is still 255. As a result,
FWRR will not work well when server's weight is 256. The description is as below:
There are some macros related to server's weight in include/types/server.h:
#define SRV_UWGHT_RANGE 256
#define SRV_UWGHT_MAX (SRV_UWGHT_RANGE - 1)
#define SRV_EWGHT_MAX (SRV_UWGHT_MAX * BE_WEIGHT_SCALE)
Since weight of server can be reach to 256 and BE_WEIGHT_SCALE equals to 16,
the max eweight of server should be 256*16 = 4096, it will exceed SRV_EWGHT_MAX
which equals to SRV_UWGHT_MAX*BE_WEIGHT_SCALE = 255*16 = 4080. When a server
with weight 256 is insterted into FWRR tree during initialization, the key value
of this server should be SRV_EWGHT_MAX - s->eweight = 4080 - 4096 = -16 which
is closed to UINT_MAX in unsigned type, so the server with highest weight will
be not elected as the first server to process request.
In addition, it is a better choice to compare with SRV_UWGHT_MAX than a magic
number 256 while doing check for the weight. The max number of servers for
round-robin algorithm is also updated.
Signed-off-by: Godbach <nylzhaowei@gmail.com>
s->req->prod->conn->addr.to.ss_family contains only useful data if
conn_get_to_addr() is called early. If thats not the case (nothing in the
configuration needs the destination address like logs, transparent, ...)
then "set-tos" doesn't work.
Fix this by checking s->req->prod->conn->addr.from.ss_family instead.
Also fix a minor doc issue about set-tos in http-response.
Released version 1.5-dev19 with the following main changes :
- MINOR: stats: remove the autofocus on the scope input field
- BUG/MEDIUM: Fix crt-list file parsing error: filtered name was ignored.
- BUG/MEDIUM: ssl: EDH ciphers are not usable if no DH parameters present in pem file.
- BUG/MEDIUM: shctx: makes the code independent on SSL runtime version.
- MEDIUM: ssl: improve crt-list format to support negation
- BUG: ssl: fix crt-list for clients not supporting SNI
- MINOR: stats: show soft-stopped servers in different color
- BUG/MINOR: config: "source" does not work in defaults section
- BUG: regex: fix pcre compile error when using JIT
- MINOR: ssl: add pattern fetch 'ssl_c_sha1'
- BUG: ssl: send payload gets corrupted if tune.ssl.maxrecord is used
- MINOR: show PCRE version and JIT status in -vv
- BUG/MINOR: jit: don't rely on USE flag to detect support
- DOC: readme: add suggestion to link against static openssl
- DOC: examples: provide simplified ssl configuration
- REORG: tproxy: prepare the transparent proxy defines for accepting other OSes
- MINOR: tproxy: add support for FreeBSD
- MINOR: tproxy: add support for OpenBSD
- DOC: examples: provide an example of transparent proxy configuration for FreeBSD 8
- CLEANUP: fix minor typo in error message.
- CLEANUP: fix missing include <string.h> in proto/listener.h
- CLEANUP: protect checks.h from multiple inclusions
- MINOR: compression: acl "res.comp" and fetch "res.comp_algo"
- BUG/MINOR: http: add-header/set-header did not accept the ACL condition
- BUILD: mention in the Makefile that USE_PCRE_JIT is for libpcre >= 8.32
- BUG/MEDIUM: splicing is broken since 1.5-dev12
- BUG/MAJOR: acl: add implicit arguments to the resolve list
- BUG/MINOR: tcp: fix error reporting for TCP rules
- CLEANUP: peers: remove a bit of spaghetti to prepare for the next bugfix
- MINOR: stick-table: allow to allocate an entry without filling it
- BUG/MAJOR: peers: fix an overflow when syncing strings larger than 16 bytes
- MINOR: session: only call http_send_name_header() when changing the server
- MINOR: tcp: report the erroneous word in tcp-request track*
- BUG/MAJOR: backend: consistent hash can loop forever in certain circumstances
- BUG/MEDIUM: log: fix regression on log-format handling
- MEDIUM: log: report file name, line number, and directive name with log-format errors
- BUG/MINOR: cli: "clear table" did not work anymore without a key
- BUG/MINOR: cli: "clear table xx data.xx" does not work anymore
- BUG/MAJOR: http: compression still has defects on chunked responses
- BUG/MINOR: stats: fix confirmation links on the stats interface
- BUG/MINOR: stats: the status bar does not appear anymore after a change
- BUG/MEDIUM: stats: allocate the stats frontend also on "stats bind-process"
- BUG/MEDIUM: stats: fix a regression when dealing with POST requests
- BUG/MINOR: fix unterminated ACL array in compression
- BUILD: last fix broke non-linux platforms
- MINOR: init: indicate the SSL runtime version on -vv.
- BUG/MEDIUM: compression: the deflate algorithm must use global settings as well
- BUILD: stdbool is not portable (again)
- DOC: readme: add a small reminder about restrictions to respect in the code
- MINOR: ebtree: add new eb_next_dup/eb_prev_dup() functions to visit duplicates
- BUG/MINOR: acl: fix a double free during exit when using PCRE_JIT
- DOC: fix wrong copy-paste in the rspdel example
- MINOR: counters: make it easier to extend the amount of tracked counters
- MEDIUM: counters: add support for tracking a third counter
- MEDIUM: counters: add a new "gpc0_rate" counter in stick-tables
- BUG/MAJOR: http: always ensure response buffer has some room for a response
- MINOR: counters: add fetch/acl sc*_tracked to indicate whether a counter is tracked
- MINOR: defaults: allow REQURI_LEN and CAPTURE_LEN to be redefined
- MINOR: log: add a new flag 'L' for locally processed requests
- MINOR: http: add full-length header fetch methods
- MEDIUM: protocol: implement a "drain" function in protocol layers
- MEDIUM: http: add a new "http-response" ruleset
- MEDIUM: http: add the "set-nice" action to http-request and http-response
- MEDIUM: log: add a log level override value in struct session
- MEDIUM: http: add support for action "set-log-level" in http-request/http-response
- MEDIUM: http: add support for "set-tos" in http-request/http-response
- MEDIUM: http: add the "set-mark" action on http-request/http-response rules
- MEDIUM: tcp: add "tcp-request connection expect-proxy layer4"
- MEDIUM: acl: automatically detect the type of certain fetches
- MEDIUM: acl: remove a lot of useless ACLs that are equivalent to their fetches
- MEDIUM: acl: remove 15 additional useless ACLs that are equivalent to their fetches
- DOC: major reorg of ACL + sample fetch
- CLEANUP: http: remove the bogus urlp_ip ACL match
- MINOR: acl: add the new "env()" fetch method to retrieve an environment variable
- BUG/MINOR: acl: correctly consider boolean fetches when doing casts
- BUG/CRITICAL: fix a possible crash when using negative header occurrences
- DOC: update ROADMAP file
- MEDIUM: counters: use sc0/sc1/sc2 instead of sc1/sc2/sc3
- MEDIUM: stats: add proxy name filtering on the statistic page
It was a bit inconsistent to have gpc start at 0 and sc start at 1,
so make sc start at zero like gpc. No previous release was issued
with sc3 anyway, so no existing setup should be affected.
This is useful in order to take different actions across restarts without
touching the configuration (eg: soft-stop), or to pass some information
such as the local host name to the next hop.
The split between ACL and sample fetch was a terrible mess in the doc,
as it caused all entries to be duplicated with most of them not easy to
find, some missing and some wrong.
The new approach consists in describing the sample fetch methods and
indicating the ACLs that are derived from these fetches. The doc is
much smaller (1500 lines added, 2200 removed, net gain = 700 lines)
and much clearer.
The description of the ACL mechanics was revamped to take account of
the latest evolutions and clearly describe the compatibility between
types of fetches and ACL patterns.
The deprecated keywords have been marked as such, though they still
appear in the examples given for various other keywords.
This configures the client-facing connection to receive a PROXY protocol
header before any byte is read from the socket. This is equivalent to
having the "accept-proxy" keyword on the "bind" line, except that using
the TCP rule allows the PROXY protocol to be accepted only for certain
IP address ranges using an ACL. This is convenient when multiple layers
of load balancers are passed through by traffic coming from public
hosts.
"set-mark" is used to set the Netfilter MARK on all packets sent to the
client to the value passed in <mark> on platforms which support it. This
value is an unsigned 32 bit value which can be matched by netfilter and
by the routing table. It can be expressed both in decimal or hexadecimal
format (prefixed by "0x"). This can be useful to force certain packets to
take a different route (for example a cheaper network path for bulk
downloads). This works on Linux kernels 2.6.32 and above and requires
admin privileges.
This manipulates the TOS field of the IP header of outgoing packets sent
to the client. This can be used to set a specific DSCP traffic class based
on some request or response information. See RFC2474, 2597, 3260 and 4594
for more information.
Some users want to disable logging for certain non-important requests such as
stats requests or health-checks coming from another equipment. Other users want
to log with a higher importance (eg: notice) some special traffic (POST requests,
authenticated requests, requests coming from suspicious IPs) or some abnormally
large responses.
This patch responds to all these needs at once by adding a "set-log-level" action
to http-request/http-response. The 8 syslog levels are supported, as well as "silent"
to disable logging.
Some actions were clearly missing to process response headers. This
patch adds a new "http-response" ruleset which provides the following
actions :
- allow : stop evaluating http-response rules
- deny : stop and reject the response with a 502
- add-header : add a header in log-format mode
- set-header : set a header in log-format mode
The req.hdr and res.hdr fetch methods do not work well on headers which
are allowed to contain commas, such as User-Agent, Date or Expires.
More specifically, full-length matching is impossible if a comma is
present.
This patch introduces 4 new fetch functions which are designed to work
with these full-length headers :
- req.fhdr, req.fhdr_cnt
- res.fhdr, res.fhdr_cnt
These ones do not stop at commas and permit to return full-length header
values.
People who use "option dontlog-normal" are bothered with redirects and
stats being logged and reported as errors in the logs ("PR" = proxy
blocked the request).
This patch introduces a new flag 'L' for when a request is locally
processed, that is not considered as an error by the log filters. That
way we know a request was intercepted and processed by haproxy without
logging the line when "option dontlog-normal" is in effect.
We're often missin a third counter to track base, src and base+src at
the same time. Here we introduce track_sc3 to have this third counter.
It would be wise not to add much more counters because that slightly
increases the session size and processing time though the real issue
is more the declaration of the keywords in the code and in the doc.
This new pattern fetch returns the client certificate's SHA-1 fingerprint
(i.e. SHA-1 hash of DER-encoded certificate) in a binary chunk.
This can be useful to pass it to a server in a header or to stick a client
to a server across multiple SSL connections.
Improve the crt-list file format to allow a rule to negate a certain SNI :
<crtfile> [[!]<snifilter> ...]
This can be useful when a domain supports a wildcard but you don't want to
deliver the wildcard cert for certain specific domains.
Implements the "res.comp" ACL which is a boolean returning 1 when a
response has been compressed by HAProxy or 0 otherwise.
Implements the "res.comp_algo" fetch which contains the name of the
algorithm HAProxy used to compress the response.
Released version 1.5-dev18 with the following main changes :
- DOCS: Add explanation of intermediate certs to crt paramater
- DOC: typo and minor fixes in compression paragraph
- MINOR: config: http-request configuration error message misses new keywords
- DOC: minor typo fix in documentation
- BUG/MEDIUM: ssl: ECDHE ciphers not usable without named curve configured.
- MEDIUM: ssl: add bind-option "strict-sni"
- MEDIUM: ssl: add mapping from SNI to cert file using "crt-list"
- MEDIUM: regex: Use PCRE JIT in acl
- DOC: simplify bind option "interface" explanation
- DOC: tfo: bump required kernel to linux-3.7
- BUILD: add explicit support for TFO with USE_TFO
- MEDIUM: New cli option -Ds for systemd compatibility
- MEDIUM: add haproxy-systemd-wrapper
- MEDIUM: add systemd service
- BUG/MEDIUM: systemd-wrapper: don't leak zombie processes
- BUG/MEDIUM: remove supplementary groups when changing gid
- BUG/MEDIUM: config: fix parser crash with bad bind or server address
- BUG/MINOR: Correct logic in cut_crlf()
- CLEANUP: checks: Make desc argument to set_server_check_status const
- CLEANUP: dumpstats: Make cli_release_handler() static
- MEDIUM: server: Break out set weight processing code
- MEDIUM: server: Allow relative weights greater than 100%
- MEDIUM: server: Tighten up parsing of weight string
- MEDIUM: checks: Add agent health check
- BUG/MEDIUM: ssl: openssl 0.9.8 doesn't open /dev/random before chroot
- BUG/MINOR: time: frequency counters are not totally accurate
- BUG/MINOR: http: don't process abortonclose when request was sent
- BUG/MEDIUM: stream_interface: don't close outgoing connections on shutw()
- BUG/MEDIUM: checks: ignore late resets after valid responses
- DOC: fix bogus recommendation on usage of gpc0 counter
- BUG/MINOR: http-compression: lookup Cache-Control in the response, not the request
- MINOR: signal: don't block SIGPROF by default
- OPTIM: epoll: make use of EPOLLRDHUP
- OPTIM: splice: detect shutdowns and avoid splice() == 0
- OPTIM: splice: assume by default that splice is working correctly
- BUG/MINOR: log: temporary fix for lost SSL info in some situations
- BUG/MEDIUM: peers: only the last peers section was used by tables
- BUG/MEDIUM: config: verbosely reject peers sections with multiple local peers
- BUG/MINOR: epoll: use a fix maxevents argument in epoll_wait()
- BUG/MINOR: config: fix improper check for failed memory alloc in ACL parser
- BUG/MINOR: config: free peer's address when exiting upon parsing error
- BUG/MINOR: config: check the proper variable when parsing log minlvl
- BUG/MEDIUM: checks: ensure the health_status is always within bounds
- BUG/MINOR: cli: show sess should always validate s->listener
- BUG/MINOR: log: improper NULL return check on utoa_pad()
- CLEANUP: http: remove a useless null check
- CLEANUP: tcp/unix: remove useless NULL check in {tcp,unix}_bind_listener()
- BUG/MEDIUM: signal: signal handler does not properly check for signal bounds
- BUG/MEDIUM: tools: off-by-one in quote_arg()
- BUG/MEDIUM: uri_auth: missing NULL check and memory leak on memory shortage
- BUG/MINOR: unix: remove the 'level' field from the ux struct
- CLEANUP: http: don't try to deinitialize http compression if it fails before init
- CLEANUP: config: slowstart is never negative
- CLEANUP: config: maxcompcpuusage is never negative
- BUG/MEDIUM: log: emit '-' for empty fields again
- BUG/MEDIUM: checks: fix a race condition between checks and observe layer7
- BUILD: fix a warning emitted by isblank() on non-c99 compilers
- BUILD: improve the makefile's support for libpcre
- MEDIUM: halog: add support for counting per source address (-ic)
- MEDIUM: tools: make str2sa_range support all address syntaxes
- MEDIUM: config: make use of str2sa_range() instead of str2sa()
- MEDIUM: config: use str2sa_range() to parse server addresses
- MEDIUM: config: use str2sa_range() to parse peers addresses
- MINOR: tests: add a config file to ease address parsing tests.
- MINOR: ssl: add a global tunable for the max SSL/TLS record size
- BUG/MINOR: syscall: fix NR_accept4 system call on sparc/linux
- BUILD/MINOR: syscall: add definition of NR_accept4 for ARM
- MINOR: config: report missing peers section name
- BUG/MEDIUM: tools: fix bad character handling in str2sa_range()
- BUG/MEDIUM: stats: never apply "unix-bind prefix" to the global stats socket
- MINOR: tools: prepare str2sa_range() to return an error message
- BUG/MEDIUM: checks: don't call connect() on unsupported address families
- MINOR: tools: prepare str2sa_range() to accept a prefix
- MEDIUM: tools: make str2sa_range() parse unix addresses too
- MEDIUM: config: make str2listener() use str2sa_range() to parse unix addresses
- MEDIUM: config: use a single str2sa_range() call to parse bind addresses
- MEDIUM: config: use str2sa_range() to parse log addresses
- CLEANUP: tools: remove str2sun() which is not used anymore.
- MEDIUM: config: add complete support for str2sa_range() in dispatch
- MEDIUM: config: add complete support for str2sa_range() in server addr
- MEDIUM: config: add complete support for str2sa_range() in 'server'
- MEDIUM: config: add complete support for str2sa_range() in 'peer'
- MEDIUM: config: add complete support for str2sa_range() in 'source' and 'usesrc'
- CLEANUP: minor cleanup in str2sa_range() and str2ip()
- CLEANUP: config: do not use multiple errmsg at once
- MEDIUM: tools: support specifying explicit address families in str2sa_range()
- MAJOR: listener: support inheriting a listening fd from the parent
- MAJOR: tools: support environment variables in addresses
- BUG/MEDIUM: http: add-header should not emit "-" for empty fields
- BUG/MEDIUM: config: ACL compatibility check on "redirect" was wrong
- BUG/MEDIUM: http: fix another issue caused by http-send-name-header
- DOC: mention the new HTTP 307 and 308 redirect statues
- MEDIUM: poll: do not use FD_* macros anymore
- BUG/MAJOR: ev_select: disable the select() poller if maxsock > FD_SETSIZE
- BUG/MINOR: acl: ssl_fc_{alg,use}_keysize must parse integers, not strings
- BUG/MINOR: acl: ssl_c_used, ssl_fc{,_has_crt,_has_sni} take no pattern
- BUILD: fix usual isdigit() warning on solaris
- BUG/MEDIUM: tools: vsnprintf() is not always reliable on Solaris
- OPTIM: buffer: remove one jump in buffer_count()
- OPTIM: http: improve branching in chunk size parser
- OPTIM: http: optimize the response forward state machine
- BUILD: enable poll() by default in the makefile
- BUILD: add explicit support for Mac OS/X
- BUG/MAJOR: http: use a static storage for sample fetch context
- BUG/MEDIUM: ssl: improve error processing and reporting in ssl_sock_load_cert_list_file()
- BUG/MAJOR: http: fix regression introduced by commit a890d072
- BUG/MAJOR: http: fix regression introduced by commit d655ffe
- BUG/CRITICAL: using HTTP information in tcp-request content may crash the process
- MEDIUM: acl: remove flag ACL_MAY_LOOKUP which is improperly used
- MEDIUM: samples: use new flags to describe compatibility between fetches and their usages
- MINOR: log: indicate it when some unreliable sample fetches are logged
- MEDIUM: samples: move payload-based fetches and ACLs to their own file
- MINOR: backend: rename sample fetch functions and declare the sample keywords
- MINOR: frontend: rename sample fetch functions and declare the sample keywords
- MINOR: listener: rename sample fetch functions and declare the sample keywords
- MEDIUM: http: unify acl and sample fetch functions
- MINOR: session: rename sample fetch functions and declare the sample keywords
- MAJOR: acl: make all ACLs reference the fetch function via a sample.
- MAJOR: acl: remove the arg_mask from the ACL definition and use the sample fetch's
- MAJOR: acl: remove fetch argument validation from the ACL struct
- MINOR: http: add new direction-explicit sample fetches for headers and cookies
- MINOR: payload: add new direction-explicit sample fetches
- CLEANUP: acl: remove ACL hooks which were never used
- MEDIUM: proxy: remove acl_requires and just keep a flag "http_needed"
- MINOR: sample: provide a function to report the name of a sample check point
- MAJOR: acl: convert all ACL requires to SMP use+val instead of ->requires
- CLEANUP: acl: remove unused references to ACL_USE_*
- MINOR: http: replace acl_parse_ver with acl_parse_str
- MEDIUM: acl: move the ->parse, ->match and ->smp fields to acl_expr
- MAJOR: acl: add option -m to change the pattern matching method
- MINOR: acl: remove the use_count in acl keywords
- MEDIUM: acl: have a pointer to the keyword name in acl_expr
- MEDIUM: acl: support using sample fetches directly in ACLs
- MEDIUM: http: remove val_usr() to validate user_lists
- MAJOR: sample: maintain a per-proxy list of the fetch args to resolve
- MINOR: ssl: add support for the "alpn" bind keyword
- MINOR: http: status code 303 is HTTP/1.1 only
- MEDIUM: http: implement redirect 307 and 308
- MINOR: http: status 301 should not be marked non-cacheable
The ALPN extension is meant to replace the now deprecated NPN extension.
This patch implements support for it. It requires a version of openssl
with support for this extension. Patches are available here right now :
http://html5labs.interopbridges.com/media/167447/alpn_patches.zip
Now it becomes possible to directly use sample fetches as the ACL fetch
methods. In this case, the matching method is mandatory. This allows to
form more ACL combinations from existing fetches and will limit the need
for new ACLs when everything is available to form them from sample fetches
and matches.
ACL expressions now support "-m" in addition to "-i" and "-f". This new
option is followed by the name of the pattern matching method to be used
on the extracted pattern. This makes it possible to reuse existing sample
fetch methods with other matching methods (eg: regex). A "found" matching
method ignores any pattern and only verifies that the required sample was
found (useful for cookies).
Similarly to previous commit fixing "hdr" and "cookie" in HTTP, we have to deal
with "payload" and "payload_lv" which are request-only for ACLs and req/resp for
sample fetches depending on the context, and to a less extent with other req_*
and rep_*/rep_* fetches. So let's add explicit "req." and "res." variants and
make the ACLs rely on that instead.
Since "hdr" and "cookie" were ambiguously referring to the request or response
depending on the context, we need a way to explicitly specify the direction.
By prefixing the fetches names with "req." and "res.", we can now restrict such
fetches to the appropriate direction. At the moment the fetches are explicitly
declared by later we might think about having an automatic match when "req." or
"res." appears. These explicit fetches are now used by the relevant ACLs.
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
cook, cook_cnt, cook_val, hdr_cnt, hdr_ip, hdr_val, http_auth,
http_auth_group, http_first_req, method, req_proto_http, req_ver,
resp_ver, scook, scook_cnt, scook_val, shdr, shdr_cnt, shdr_ip,
shdr_val, status, urlp, urlp_val,
Most of them won't bring much benefit at the moment, or are even aliases of
existing ones, however they'll be needed for ACL->SMP convergence.
A new val_usr() function was added to resolve userlist names into pointers.
The http_auth_group ACL forgot to make its first argument mandatory, so
there was a check in cfgparse to report a vague error. Now that args are
correctly parsed, let's report something more precise.
All urlp* ACLs now support an optional 3rd argument like their sample
counter-part which is the optional delimiter.
The fetch functions have been renamed "smp_fetch_*".
Some args controls on the sample keywords have been relaxed so that we
can soon use them for ACLs :
- cookie now accepts to have an optional name ; it will return the
first matching cookie if the name is not set ;
- same for set-cookie and hdr
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
dst_conn, so_id,
The fetch functions have been renamed "smp_fetch_*".
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
fe_conn, fe_id, fe_sess_rate
The fetch functions have been renamed "smp_fetch_*".
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :
avg_queue, be_conn, be_id, be_sess_rate, connslots, nbsrv,
queue, srv_conn, srv_id, srv_is_up, srv_sess_rate
The fetch functions have been renamed "smp_fetch_*".
The file acl.c is a real mess, it both contains functions to parse and
process ACLs, and some sample extraction functions which act on buffers.
Some other payload analysers were arbitrarily dispatched to proto_tcp.c.
So now we're moving all payload-based fetches and ACLs to payload.c
which is capable of extracting data from buffers and rely on everything
that is protocol-independant. That way we can safely inflate this file
and only use the other ones when some fetches are really specific (eg:
HTTP, SSL, ...).
As a result of this cleanup, the following new sample fetches became
available even if they're not really useful :
always_false, always_true, rep_ssl_hello_type, rdp_cookie_cnt,
req_len, req_ssl_hello_type, req_ssl_sni, req_ssl_ver, wait_end
The function 'acl_fetch_nothing' was wrong and never used anywhere so it
was removed.
The "rdp_cookie" sample fetch used to have a mandatory argument while it
was optional in ACLs, which are supposed to iterate over RDP cookies. So
we're making it optional as a fetch too, and it will return the first one.
TCP Fast Open is supported in server mode since Linux 3.7, but current
libc's don't define TCP_FASTOPEN=23. Introduce the new USE flag USE_TFO
to define it manually in compat.h. Also note this in the TFO related
documentation.
fe61656b added the ability to load a list of certificates from a file,
but error control was incomplete and misleading, as some errors such
as missing files were not reported, and errors reported with Alert()
instead of memprintf() were inappropriate and mixed with upper errors.
Also, the code really supports a single SNI filter right now, so let's
correct it and the doc for that, leaving room for later change if needed.
It designates a list of PEM file with an optional list of SNI filter
per certificate, with the following format for each line :
<crtfile>[ <snifilter>]*
Wildcards are supported in the SNI filter. The certificates will be
presented to clients who provide a valid TLS Server Name Indication
field matching one of SNI filter. If no SNI filter is specified the
CN and alt subjects are used.
This change makes the "crt" block of the documentation easier to use
for those not clear on what needs to go in what file, specifically for
those using CAs that require intermediate certificates.
Now that all addresses are parsed using str2sa_range(), it becomes easy
to add support for environment variables and use them everywhere an address
is needed. Environment variables are used as $VAR or ${VAR} as in shell.
Any number of variables may compose an address, allowing various fantasies
such as "fd@${FD_HTTP}" or "${LAN_DC1}.1:80".
These ones are usable in logs, bind, servers, peers, stats socket, source,
dispatch, and check address.
Using the address syntax "fd@<num>", a listener may inherit a file
descriptor that the caller process has already bound and passed as
this number. The fd's socket family is detected using getsockname(),
and the usual initialization is performed through the existing code
for that family, but the socket creation is skipped.
Whether the parent has performed the listen() call or not is not
important as this is detected.
For UNIX sockets, we immediately clear the path after preparing a
socket so that we never remove it in case an abort would happen due
to a late error during startup.
This change allows one to force the address family in any address parsed
by str2sa_range() by specifying it as a prefix followed by '@' then the
address. Currently supported address prefixes are 'ipv4@', 'ipv6@', 'unix@'.
This also helps forcing resolving for host names (when getaddrinfo is used),
and force the family of the empty address (eg: 'ipv4@' = 0.0.0.0 while
'ipv6@' = ::).
The main benefits is that unix sockets can now get a local name without
being forced to begin with a slash. This is useful during development as
it is no longer necessary to have stats socket sent to /tmp.
Add new tunable "tune.ssl.maxrecord".
Over SSL/TLS, the client can decipher the data only once it has received
a full record. With large records, it means that clients might have to
download up to 16kB of data before starting to process them. Limiting the
record size can improve page load times on browsers located over high
latency or low bandwidth networks. It is suggested to find optimal values
which fit into 1 or 2 TCP segments (generally 1448 bytes over Ethernet
with TCP timestamps enabled, or 1460 when timestamps are disabled), keeping
in mind that SSL/TLS add some overhead. Typical values of 1419 and 2859
gave good results during tests. Use "strace -e trace=write" to find the
best value.
This trick was first suggested by Mike Belshe :
http://www.belshe.com/2010/12/17/performance-and-the-tls-record-size/
Then requested again by Ilya Grigorik who provides some hints here :
http://ofps.oreilly.com/titles/9781449344764/_transport_layer_security_tls.html#ch04_00000101
Support for server side TFO was actually introduced in linux-3.7,
linux-3.6 just has client support.
This patch fixes documentation and a code comment about the
kernel requirement. It also fixes a wrong tfo related code
comment in src/proto_tcp.c.
Support a agent health check performed by opening a TCP socket to a
pre-defined port and reading an ASCII string. The string should have one of
the following forms:
* An ASCII representation of an positive integer percentage.
e.g. "75%"
Values in this format will set the weight proportional to the initial
weight of a server as configured when haproxy starts.
* The string "drain".
This will cause the weight of a server to be set to 0, and thus it will
not accept any new connections other than those that are accepted via
persistence.
* The string "down", optionally followed by a description string.
Mark the server as down and log the description string as the reason.
* The string "stopped", optionally followed by a description string.
This currently has the same behaviour as down (iii).
* The string "fail", optionally followed by a description string.
This currently has the same behaviour as down (iii).
A agent health check may be configured using "option lb-agent-chk".
The use of an alternate check-port, used to obtain agent heath check
information described above as opposed to the port of the service,
may be useful in conjunction with this option.
e.g.
option lb-agent-chk
server http1_1 10.0.0.10:80 check port 10000 weight 100
Signed-off-by: Simon Horman <horms@verge.net.au>
Allow relative weights greater than 100%,
capping the absolute value to 256 which is
the largest supported absolute weight.
Signed-off-by: Simon Horman <horms@verge.net.au>
The current documentation of the bind option "interface" can be misleading
(as seen on the ML recently).
This patch tries to address misunderstandings by :
- avoiding the words listen or bind in the behavior description, using
"restrict to interface" instead
- using a different sentence construction (partially stolen from
"man 7 socket": SO_BINDTODEVICE)
- "defragmentation": moving behavior related explanations to the beginning
and restrictions, use-cases and requirements to the end.
This new option ensures that there is no possible fallback to a default
certificate if the client does not provide an SNI which is explicitly
handled by a certificate.
Without it, haproxy will retain the group membership of root, which may
give more access than intended to the process. For example, haproxy would
still be in the wheel group on Fedora 18, as seen with :
# haproxy -f /etc/haproxy/haproxy.cfg
# ps a -o pid,user,group,command | grep hapr
3545 haproxy haproxy haproxy -f /etc/haproxy/haproxy.cfg
4356 root root grep --color=auto hapr
# grep Group /proc/3545/status
Groups: 0 1 2 3 4 6 10
# getent group wheel
wheelâŒ10:root,misc
[WT: The issue has been investigated by independent security research team
and realized by itself not being able to allow security exploitation.
Additionally, dropping groups is not allowed to unprivileged users,
though this mode of deployment is quite common. Thus a warning is
emitted in this case to inform the user. The fix could be backported
into all supported versions as the issue has always been there. ]
The doc pretends that src_inc_gpc0 may be used alone without an integer
match, but this is false and has always been since its introduction in
1.5-dev1. If the ACL is called, the increment will be used, the value
returned, but it will be matched against no value so the resulting ACL
will never be true and the condition will not be met.
This means that the following config :
acl abuser src -f abusers.lst
acl blacklist src_inc_gpc0
tcp-request connection reject if abuser blacklist
Will never reject the connection and must be fixed this way :
acl abuser src -f abusers.lst
acl blacklist src_inc_gpc0 gt 0
tcp-request connection reject if abuser blacklist
Note that clr_gpc0 is trickier, as it returns the previous value which
might also be zero. Thus it's suggested to compare it against any positive
value including zero :
tcp-request connection accept if { src_clr_gpc0 ge 0 }
Some arguments were missing on the sc1/sc2 forms of most ACLs including
gpc0, so this has been fixed too.
Released version 1.5-dev17 with the following main changes :
- MINOR: ssl: Setting global tune.ssl.cachesize value to 0 disables SSL session cache.
- BUG/MEDIUM: stats: fix stats page regression introduced by commit 20b0de5
- BUG/MINOR: stats: last fix was still wrong
- BUG/MINOR: stats: http-request rules still don't cope with stats
- BUG/MINOR: http: http-request add-header emits a corrupted header
- BUG/MEDIUM: stats: disable request analyser when processing POST or HEAD
- BUG/MINOR: log: make log-format, unique-id-format and add-header more independant
- BUILD: log: unused variable svid
- CLEANUP: http: rename the misleading http_check_access_rule
- MINOR: http: move redirect rule processing to its own function
- REORG: config: move the http redirect rule parser to proto_http.c
- MEDIUM: http: add support for "http-request redirect" rules
- MEDIUM: http: add support for "http-request tarpit" rule
The "reqtarpit" rule is not very handy to use. Now that we have more
flexibility with "http-request", let's finally make the tarpit rules
usable there.
There are still semantical differences between apply_filters_to_request()
and http_req_get_intercept_rule() because the former updates the counters
while the latter does not. So we currently have almost similar code leafs
for similar conditions, but this should be cleaned up later.
These are exactly the same as the classic redirect rules except
that they can be interleaved with other http-request rules for
more flexibility.
The redirect parser should probably be changed to stop at the condition
so that the caller puts its own condition pointer. At the moment, the
redirect rule and condition are parsed at once by build_redirect_rule()
and the condition is assigned to the http_req_rule.
Released version 1.5-dev16 with the following main changes :
- BUG/MEDIUM: ssl: Prevent ssl error from affecting other connections.
- BUG/MINOR: ssl: error is not reported if it occurs simultaneously with peer close detection.
- MINOR: ssl: add fetch and acl "ssl_c_used" to check if current SSL session uses a client certificate.
- MINOR: contrib: make the iprange tool grep for addresses
- CLEANUP: polling: gcc doesn't always optimize constants away
- OPTIM: poll: optimize fd management functions for low register count CPUs
- CLEANUP: poll: remove a useless double-check on fdtab[fd].owner
- OPTIM: epoll: use a temp variable for intermediary flag computations
- OPTIM: epoll: current fd does not count as a new one
- BUG/MINOR: poll: the I/O handler was called twice for polled I/Os
- MINOR: http: make resp_ver and status ACLs check for the presence of a response
- BUG/MEDIUM: stream-interface: fix possible stalls during transfers
- BUG/MINOR: stream_interface: don't return when the fd is already set
- BUG/MEDIUM: connection: always update connection flags prior to computing polling
- CLEANUP: buffer: use buffer_empty() instead of buffer_len()==0
- BUG/MAJOR: stream_interface: fix occasional data transfer freezes
- BUG/MEDIUM: stream_interface: fix another case where the reader might not be woken up
- BUG/MINOR: http: don't abort client connection on premature responses
- BUILD: no need to clean up when making git-tar
- MINOR: log: add a tag for amount of bytes uploaded from client to server
- BUG/MEDIUM: log: fix possible segfault during config parsing
- MEDIUM: log: change a few log tokens to make them easier to remember
- BUG/MINOR: log: add_to_logformat_list() used the wrong constants
- MEDIUM: log-format: make the format parser more robust and more extensible
- MINOR: sample: support cast from bool to string
- MINOR: samples: add a function to fetch and convert any sample to a string
- MINOR: log: add lf_text_len
- MEDIUM: log: add the ability to include samples in logs
- REORG: stats: massive code reorg and cleanup
- REORG: stats: move the HTTP header injection to proto_http
- REORG: stats: functions are now HTTP/CLI agnostic
- BUG/MINOR: log: fix regression introduced by commit 8a3f52
- MINOR: chunks: centralize the trash chunk allocation
- MEDIUM: stats: use hover boxes instead of title to report details
- MEDIUM: stats: use multi-line tips to display detailed counters
- MINOR: tools: simplify the use of the int to ascii macros
- MINOR: stats: replace STAT_FMT_CSV with STAT_FMT_HTML
- MINOR: http: prepare to support more http-request actions
- MINOR: log: make parse_logformat_string() take a const char *
- MEDIUM: http: add http-request 'add-header' and 'set-header' to build headers
These two new statements allow to pass information extracted from the request
to the server. It's particularly useful for passing SSL information to the
server, but may be used for various other purposes such as combining headers
together to emulate internal variables.
Using %[expression] it becomes possible to make the log engine fetch
some samples from the request or the response and provide them in the
logs. Note that this feature is still limited, it does not yet allow
to apply converters, to limit the output length, nor to specify the
direction which should be fetched when a fetch function works in both
directions.
However it's quite convenient to log SSL information or to include some
information that are used in stick tables.
It is worth noting that this has been done in the generic log format
handler, which means that the same information may be used to build the
unique-id header and to pass the information to a backend server.
Some log tokens have evolved in a way that is not completely logical.
For example, frontend tokens sometimes begin with an 'f' and sometimes
with an 'F'. Same for backend and server.
So let's change a few cases without disrupting compatibility with existing
setups :
Bi => bi
Bp => bp
Ci => ci
Cp => cp
Fi => fi
Fp => fp
Si => si
Sp => sp
cc => CC
cs => CS
st => ST
The old ones are still supported but deprecated and will be unsupported by
the 1.5 release. However, a warning message is emitted when they're encounterd
and it indicates what token should be used to replace them.
Released version 1.5-dev15 with the following main changes :
- DOC: add a few precisions on compression
- BUG/MEDIUM: ssl: Fix handshake failure on session resumption with client cert.
- BUG/MINOR: ssl: One free session in cache remains unused.
- BUG/MEDIUM: ssl: first outgoing connection would fail with {ca,crt}-ignore-err
- MEDIUM: ssl: manage shared cache by blocks for huge sessions.
- MINOR: acl: add fetch for server session rate
- BUG/MINOR: compression: Content-Type is case insensitive
- MINOR: compression: disable on multipart or status != 200
- BUG/MINOR: http: don't report client aborts as server errors
- MINOR: stats: compute the ratio of compressed response based on 2xx responses
- MINOR: http: factor out the content-type checks
- BUG/MAJOR: stats: correctly check for a possible divide error when showing compression ratios
- BUILD: ssl: OpenSSL 0.9.6 has no renegociation
- BUG/MINOR: http: disable compression when message has no body
- MINOR: compression: make the stats a bit more robust
- BUG/MEDIUM: comp: DEFAULT_MAXZLIBMEM was expressed in bytes and not megabytes
- MINOR: connection: don't remove failed handshake flags
- MEDIUM: connection: add an error code in connections
- MEDIUM: connection: add minimal error reporting in logs for incomplete connections
- MEDIUM: connection: add error reporting for the PROXY protocol header
- MEDIUM: connection: add error reporting for the SSL
- DOC: document the connection error format in logs
- BUG/MINOR: http: don't log a 503 on client errors while waiting for requests
- BUILD: stdbool is not portable
- BUILD: ssl: NAME_MAX is not portable, use MAXPATHLEN instead
- BUG/MAJOR: raw_sock: must check error code on hangup
- BUG/MAJOR: polling: do not set speculative events on ERR nor HUP
- BUG/MEDIUM: session: fix FD leak when transport layer logging is enabled
- MINOR: stats: add a few more information on session dump
- BUG/MINOR: tcp: set the ADDR_TO_SET flag on outgoing connections
- CLEANUP: connection: remove unused server/proxy/task/si_applet declarations
- BUG/MEDIUM: tcp: process could theorically crash on lack of source ports
- MINOR: cfgparse: mention "interface" in the list of allowed "source" options
- MEDIUM: connection: introduce "struct conn_src" for servers and proxies
- CLEANUP: proto_tcp: use the same code to bind servers and backends
- CLEANUP: backend: use the same tproxy address selection code for servers and backends
- BUG/MEDIUM: stick-tables: conversions to strings were broken in dev13
- MEDIUM: proto_tcp: add support for tracking L7 information
- MEDIUM: counters: add sc1_trackers/sc2_trackers
- MINOR: http: add the "base32" pattern fetch function
- MINOR: http: add the "base32+src" fetch method.
- CLEANUP: session: use an array for the stick counters
- BUG/MINOR: proto_tcp: fix parsing of "table" in track-sc1/2
- BUG/MINOR: proto_tcp: bidirectional fetches not supported anymore in track-sc1/2
- BUG/MAJOR: connection: always recompute polling status upon I/O
- BUG/MINOR: connection: remove a few synchronous calls to polling updates
- MINOR: config: improve error checking on TCP stick-table tracking
- DOC: add some clarifications to the readme
This returns the concatenation of the base32 fetch and the src fetch.
The resulting type is of type binary, with a size of 8 or 20 bytes
depending on the source address family. This can be used to track
per-IP, per-URL counters.
This returns a 32-bit hash of the value returned by the "base"
fetch method above. This is useful to track per-URL activity on
high traffic sites without having to store all URLs. Instead a
shorter hash is stored, saving a lot of memory. The output type
is an unsigned integer.
Returns the current amount of concurrent connections tracking the same
tracked counters. This number is automatically incremented when tracking
begins and decremented when tracking stops. It differs from sc1_conn_cur in
that it does not rely on any stored information but on the table's reference
count (the "use" value which is returned by "show table" on the CLI). This
may sometimes be more suited for layer7 tracking.
Until now it was only possible to use track-sc1/sc2 with "src" which
is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch
as well as any transformation type. It works just like the "stick"
directive.
Samples are automatically converted to the correct types for the table.
Only "tcp-request content" rules may use L7 information, and such information
must already be present when the tracking is set up. For example it becomes
possible to track the IP address passed in the X-Forwarded-For header.
HTTP request processing now also considers tracking from backend rules
because we want to be able to update the counters even when the request
was already parsed and tracked.
Some more controls need to be performed (eg: samples do not distinguish
between L4 and L6).
Considering there is no option yet for maxconnrate for servers, I wrote
an ACL to check a backend server session rate which we use to send to an
"overflow" backend to prevent latency responses to our clients (very
sensitive latency requirements).
Sessions using client certs are huge (more than 1 kB) and do not fit
in session cache, or require a huge cache.
In this new implementation sshcachesize set a number of available blocks
instead a number of available sessions.
Each block is large enough (128 bytes) to store a simple session (without
client certs).
Huge sessions will take multiple blocks depending on client certificate size.
Note: some unused code for session sync with remote peers was temporarily
removed.
The compression is disabled when the HTTP status code is not 200, indeed
compression on some HTTP code can create issues (ex: 206, 416).
Multipart message should not be compressed eitherway.
Released version 1.5-dev14 with the following main changes :
- DOC: fix minor typos
- BUG/MEDIUM: compression: does not forward trailers
- MINOR: buffer_dump with ASCII
- BUG/MEDIUM: checks: mark the check as stopped after a connect error
- BUG/MEDIUM: checks: ensure we completely disable polling upon success
- BUG/MINOR: checks: don't mark the FD as closed before transport close
- MEDIUM: checks: avoid accumulating TIME_WAITs during checks
- MINOR: cli: report the msg state in full text in "show sess $PTR"
- CLEANUP: checks: rename some server check flags
- MAJOR: checks: rework completely bogus state machine
- BUG/MINOR: checks: slightly clean the state machine up
- MEDIUM: checks: avoid waking the application up for pure TCP checks
- MEDIUM: checks: close the socket as soon as we have a response
- BUG/MAJOR: checks: close FD on all timeouts
- MINOR: checks: fix recv polling after connect()
- MEDIUM: connection: provide a common conn_full_close() function
- BUG/MEDIUM: checks: prevent TIME_WAITs from appearing also on timeouts
- BUG/MAJOR: peers: the listener's maxaccept was not set and caused loops
- MINOR: listeners: make the accept loop more robust when maxaccept==0
- BUG/MEDIUM: acl: correctly resolve all args, not just the first one
- BUG/MEDIUM: acl: make prue_acl_expr() correctly free ACL expressions upon exit
- BUG/MINOR: stats: fix inversion of the report of a check in progress
- MEDIUM: tcp: add explicit support for delayed ACK in connect()
- BUG/MEDIUM: connection: always disable polling upon error
- MINOR: connection: abort earlier when errors are detected
- BUG/MEDIUM: checks: report handshake failures
- BUG/MEDIUM: connection: local_send_proxy must wait for connection to establish
- MINOR: tcp: add support for the "v6only" bind option
- MINOR: stats: also report the computed compression savings in html stats
- MINOR: stats: report the total number of compressed responses per front/back
- MINOR: tcp: add support for the "v4v6" bind option
- DOC: stats: document the comp_rsp stats column
- BUILD: buffer: fix another isprint() warning on solaris
- MINOR: cli: add support for the "show sess all" command
- BUG/MAJOR: cli: show sess <id> may randomly corrupt the back-ref list
- MINOR: cli: improve output format for show sess $ptr
Sometimes when debugging haproxy, it is important to take a full
snapshot of all sessions and their respective states. Till now it
was complicated to do because we had to use scripts and sessions
would vanish between two runs.
Now with this command we have the same output as "show sess $id"
but for all sessions in the table. This is a debugging command only,
it should only be used by developers as it is never guaranteed to
perfectly work !
Commit 9b6700f added "v6only". As suggested by Vincent Bernat, it is
sometimes useful to have the opposite option to force binding to the
two protocols when the system is configured to bind to v6 only by
default. This option does exactly this. v6only still has precedence.
Released version 1.5-dev13 with the following main changes :
- BUILD: fix build issue without USE_OPENSSL
- BUILD: fix compilation error with DEBUG_FULL
- DOC: ssl: remove prefer-server-ciphers documentation
- DOC: ssl: surround keywords with quotes
- DOC: fix minor typo on http-send-name-header
- BUG/MEDIUM: acls using IPv6 subnets patterns incorrectly match IPs
- BUG/MAJOR: fix a segfault on option http_proxy and url_ip acl
- MEDIUM: http: accept IPv6 values with (s)hdr_ip acl
- BUILD: report zlib support in haproxy -vv
- DOC: compression: add some details and clean up the formatting
- DOC: Change is_ssl acl to ssl_fc acl in example
- DOC: make it clear what the HTTP request size is
- MINOR: ssl: try to load Diffie-Hellman parameters from cert file
- DOC: ssl: update 'crt' statement on 'bind' about Diffie-Hellman parameters loading
- MINOR: ssl: add elliptic curve Diffie-Hellman support for ssl key generation
- DOC: ssl: add 'ecdhe' statement on 'bind'
- MEDIUM: ssl: add client certificate authentication support
- DOC: ssl: add 'verify', 'cafile' and 'crlfile' statements on 'bind'
- MINOR: ssl: add fetch and ACL 'client_crt' to test a client cert is present
- DOC: ssl: add fetch and ACL 'client_cert'
- MINOR: ssl: add ignore verify errors options
- DOC: ssl: add 'ca-ignore-err' and 'crt-ignore-err' statements on 'bind'
- MINOR: ssl: add fetch and ACL 'ssl_verify_result'
- DOC: ssl: add fetch and ACL 'ssl_verify_result'
- MINOR: ssl: add fetches and ACLs to return verify errors
- DOC: ssl: add fetches and ACLs 'ssl_verify_crterr', 'ssl_verify_caerr', and 'ssl_verify_crterr_depth'
- MINOR: ssl: disable shared memory and locks on session cache if nbproc == 1
- MINOR: ssl: add build param USE_PRIVATE_CACHE to build cache without shared memory
- MINOR: ssl : add statements 'notlsv11' and 'notlsv12' and rename 'notlsv1' to 'notlsv10'.
- DOC: ssl : add statements 'notlsv11' and 'notlsv12' and rename 'notlsv1' to 'notlsv10'.
- MEDIUM: config: authorize frontend and listen without bind.
- MINOR: ssl: add statement 'no-tls-tickets' on bind to disable stateless session resumption
- DOC: ssl: add 'no-tls-tickets' statement documentation.
- BUG/MINOR: ssl: Fix CRL check was not enabled when crlfile was specified.
- BUG/MINOR: build: Fix compilation issue on openssl 0.9.6 due to missing CRL feature.
- BUG/MINOR: conf: Fix 'maxsslconn' statement error if built without OPENSSL.
- BUG/MINOR: build: Fix failure with USE_OPENSSL=1 and USE_FUTEX=1 on archs i486 and i686.
- MINOR: ssl: remove prefer-server-ciphers statement and set it as the default on ssl listeners.
- BUG/MEDIUM: ssl: subsequent handshakes fail after server configuration changes
- MINOR: ssl: add 'crt-base' and 'ca-base' global statements.
- MEDIUM: conf: rename 'nosslv3' and 'notlsvXX' statements 'no-sslv3' and 'no-tlsvXX'.
- MEDIUM: conf: rename 'cafile' and 'crlfile' statements 'ca-file' and 'crl-file'
- MINOR: ssl: use bit fields to store ssl options instead of one int each
- MINOR: ssl: add 'force-sslv3' and 'force-tlsvXX' statements on bind.
- MINOR: ssl: add 'force-sslv3' and 'force-tlsvXX' statements on server
- MINOR: ssl: add defines LISTEN_DEFAULT_CIPHERS and CONNECT_DEFAULT_CIPHERS.
- BUG/MINOR: ssl: Fix issue on server statements 'no-tls*' and 'no-sslv3'
- MINOR: ssl: move ssl context init for servers from cfgparse.c to ssl_sock.c
- MEDIUM: ssl: reject ssl server keywords in default-server statement
- MINOR: ssl: add statement 'no-tls-tickets' on server side.
- MINOR: ssl: add statements 'verify', 'ca-file' and 'crl-file' on servers.
- DOC: Fix rename of options cafile and crlfile to ca-file and crl-file.
- MINOR: sample: manage binary to string type convertion in stick-table and samples.
- MINOR: acl: add parse and match primitives to use binary type on ACLs
- MINOR: sample: export 'sample_get_trash_chunk(void)'
- MINOR: conf: rename all ssl modules fetches using prefix 'ssl_fc' and 'ssl_c'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_fc_protocol', 'ssl_fc_cipher', 'ssl_fc_use_keysize' and 'ssl_fc_alg_keysize'
- MINOR: ssl: add pattern fetch 'ssl_fc_session_id'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_version' and 'ssl_f_version'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_s_dn', 'ssl_c_i_dn', 'ssl_f_s_dn' and 'ssl_c_i_dn'
- MINOR: ssl: add pattern and ACLs 'ssl_c_sig_alg' and 'ssl_f_sig_alg'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_key_alg' and 'ssl_f_key_alg'
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_notbefore', 'ssl_c_notafter', 'ssl_f_notbefore' and 'ssl_f_notafter'
- MINOR: ssl: add 'crt' statement on server.
- MINOR: ssl: checks the consistency of a private key with the corresponding certificate
- BUG/MEDIUM: ssl: review polling on reneg.
- BUG/MEDIUM: ssl: Fix some reneg cases not correctly handled.
- BUG/MEDIUM: ssl: Fix sometimes reneg fails if requested by server.
- MINOR: build: allow packagers to specify the ssl cache size
- MINOR: conf: add warning if ssl is not enabled and a certificate is present on bind.
- MINOR: ssl: Add tune.ssl.lifetime statement in global.
- MINOR: compression: Enable compression for IE6 w/SP2, IE7 and IE8
- BUG: http: revert broken optimisation from 82fe75c1a7
- DOC: duplicate ssl_sni section
- MEDIUM: HTTP compression (zlib library support)
- CLEANUP: use struct comp_ctx instead of union
- BUILD: remove dependency to zlib.h
- MINOR: compression: memlevel and windowsize
- MEDIUM: use pool for zlib
- MINOR: compression: try init in cfgparse.c
- MINOR: compression: init before deleting headers
- MEDIUM: compression: limit RAM usage
- MINOR: compression: tune.comp.maxlevel
- MINOR: compression: maximum compression rate limit
- MINOR: log-format: check number of arguments in cfgparse.c
- BUG/MEDIUM: compression: no Content-Type header but type in configuration
- BUG/MINOR: compression: deinit zlib only when required
- MEDIUM: compression: don't compress when no data
- MEDIUM: compression: use pool for comp_ctx
- MINOR: compression: rate limit in 'show info'
- MINOR: compression: report zlib memory usage
- BUG/MINOR: compression: dynamic level increase
- DOC: compression: unsupported cases.
- MINOR: compression: CPU usage limit
- MEDIUM: http: add "redirect scheme" to ease HTTP to HTTPS redirection
- BUG/MAJOR: ssl: missing tests in ACL fetch functions
- MINOR: config: add a function to indent error messages
- REORG: split "protocols" files into protocol and listener
- MEDIUM: config: replace ssl_conf by bind_conf
- CLEANUP: listener: remove unused conf->file and conf->line
- MEDIUM: listener: add a minimal framework to register "bind" keyword options
- MEDIUM: config: move the "bind" TCP parameters to proto_tcp
- MEDIUM: move bind SSL parsing to ssl_sock
- MINOR: config: improve error reporting for "bind" lines
- MEDIUM: config: move the common "bind" settings to listener.c
- MEDIUM: config: move all unix-specific bind keywords to proto_uxst.c
- MEDIUM: config: enumerate full list of registered "bind" keywords upon error
- MINOR: listener: add a scope field in the bind keyword lists
- MINOR: config: pass the file and line to config keyword parsers
- MINOR: stats: fill the file and line numbers in the stats frontend
- MINOR: config: set the bind_conf entry on listeners created from a "listen" line.
- MAJOR: listeners: use dual-linked lists to chain listeners with frontends
- REORG: listener: move unix perms from the listener to the bind_conf
- BUG: backend: balance hdr was broken since 1.5-dev11
- MINOR: standard: make memprintf() support a NULL destination
- MINOR: config: make str2listener() use memprintf() to report errors.
- MEDIUM: stats: remove the stats_sock struct from the global struct
- MINOR: ssl: set the listeners' data layer to ssl during parsing
- MEDIUM: stats: make use of the standard "bind" parsers to parse global socket
- DOC: move bind options to their own section
- DOC: stats: refer to "bind" section for "stats socket" settings
- DOC: fix index to reference bind and server options
- BUG: http: do not print garbage on invalid requests in debug mode
- BUG/MINOR: config: check the proper pointer to report unknown protocol
- CLEANUP: connection: offer conn_prepare() to set up a connection
- CLEANUP: config: fix typo inteface => interface
- BUG: stats: fix regression introduced by commit 4348fad1
- MINOR: cli: allow to set frontend maxconn to zero
- BUG/MAJOR: http: chunk parser was broken with buffer changes
- MEDIUM: monitor: simplify handling of monitor-net and mode health
- MINOR: connection: add a pointer to the connection owner
- MEDIUM: connection: make use of the owner instead of container_of
- BUG/MINOR: ssl: report the L4 connection as established when possible
- BUG/MEDIUM: proxy: must not try to stop disabled proxies upon reload
- BUG/MINOR: config: use a copy of the file name in proxy configurations
- BUG/MEDIUM: listener: don't pause protocols that do not support it
- MEDIUM: proxy: add the global frontend to the list of normal proxies
- BUG/MINOR: epoll: correctly disable FD polling in fd_rem()
- MINOR: signal: really ignore signals configured with no handler
- MINOR: buffers: add a few functions to write chars, strings and blocks
- MINOR: raw_sock: always report asynchronous connection errors
- MEDIUM: raw_sock: improve connection error reporting
- REORG: connection: rename the data layer the "transport layer"
- REORG: connection: rename app_cb "data"
- MINOR: connection: provide a generic data layer wakeup callback
- MINOR: connection: split conn_prepare() in two functions
- MINOR: connection: add an init callback to the data_cb struct
- MEDIUM: session: use a specific data_cb for embryonic sessions
- MEDIUM: connection: use a generic data-layer init() callback
- MEDIUM: connection: reorganize connection flags
- MEDIUM: connection: only call the data->wake callback on activity
- MEDIUM: connection: make it possible for data->wake to return an error
- MEDIUM: session: register a data->wake callback to process errors
- MEDIUM: connection: don't call the data->init callback upon error
- MEDIUM: connection: it's not the data layer's role to validate the connection
- MEDIUM: connection: automatically disable polling on error
- REORG: connection: move the PROXY protocol management to connection.c
- MEDIUM: connection: add a new local send-proxy transport callback
- MAJOR: checks: make use of the connection layer to send checks
- REORG: server: move the check-specific parts into a check subsection
- MEDIUM: checks: use real buffers to store requests and responses
- MEDIUM: check: add the ctrl and transport layers in the server check structure
- MAJOR: checks: completely use the connection transport layer
- MEDIUM: checks: add the "check-ssl" server option
- MEDIUM: checks: enable the PROXY protocol with health checks
- CLEANUP: checks: remove minor warnings for assigned but not used variables
- MEDIUM: tcp: enable TCP Fast Open on systems which support it
- BUG: connection: fix regression from commit 9e272bf9
- CLEANUP: cttproxy: remove a warning on undeclared close()
- BUG/MAJOR: ensure that hdr_idx is always reserved when L7 fetches are used
- MEDIUM: listener: add support for linux's accept4() syscall
- MINOR: halog: sort output by cookie code
- BUG/MINOR: halog: -ad/-ac report the correct number of output lines
- BUG/MINOR: halog: fix help message for -ut/-uto
- MINOR: halog: add a parameter to limit output line count
- BUILD: accept4: move the socketcall declaration outside of accept4()
- MINOR: server: add minimal infrastructure to parse keywords
- MINOR: standard: make indent_msg() support empty messages
- MEDIUM: server: check for registered keywords when parsing unknown keywords
- MEDIUM: server: move parsing of keyword "id" to server.c
- BUG/MEDIUM: config: check-send-proxy was ignored if SSL was not builtin
- MEDIUM: ssl: move "server" keyword SSL options parsing to ssl_sock.c
- MEDIUM: log: suffix the frontend's name with '~' when using SSL
- MEDIUM: connection: always unset the transport layer upon close
- BUG/MINOR: session: fix some leftover from debug code
- BUG/MEDIUM: session: enable the conn_session_update() callback
- MEDIUM: connection: add a flag to hold the transport layer
- MEDIUM: log: add a new LW_XPRT flag to pin the transport layer
- MINOR: log: make lf_text use a const char *
- MEDIUM: log: report SSL ciphers and version in logs using logformat %sslc/%sslv
- REORG: http: rename msg->buf to msg->chn since it's a channel
- CLEANUP: http: use 'chn' to name channel variables, not 'buf'
- CLEANUP: channel: use 'chn' instead of 'buf' as local variable names
- CLEANUP: tcp: use 'chn' instead of 'buf' or 'b' for channel pointer names
- CLEANUP: stream_interface: use 'chn' instead of 'b' to name channel pointers
- CLEANUP: acl: use 'chn' instead of 'b' to name channel pointers
- MAJOR: channel: replace the struct buffer with a pointer to a buffer
- OPTIM: channel: reorganize struct members to improve cache efficiency
- CLEANUP: session: remove term_trace which is not used anymore
- OPTIM: session: reorder struct session fields
- OPTIM: connection: pack the struct target
- DOC: document relations between internal entities
- MINOR: ssl: add 'ssl_npn' sample/acl to extract TLS/NPN information
- BUILD: ssl: fix shctx build on older compilers
- MEDIUM: ssl: add support for the "npn" bind keyword
- BUG: ssl: fix ssl_sni ACLs to correctly process regular expressions
- MINOR: chunk: provide string compare functions
- MINOR: sample: accept fetch keywords without parenthesis
- MEDIUM: sample: pass an empty list instead of a null for fetch args
- MINOR: ssl: improve socket behaviour upon handshake abort.
- BUG/MEDIUM: http: set DONTWAIT on data when switching to tunnel mode
- MEDIUM: listener: provide a fallback for accept4() when not supported
- BUG/MAJOR: connection: risk of crash on certain tricky close scenario
- MEDIUM: cli: allow the stats socket to be bound to a specific set of processes
- OPTIM: channel: inline channel_forward's fast path
- OPTIM: http: inline http_parse_chunk_size() and http_skip_chunk_crlf()
- OPTIM: tools: inline hex2i()
- CLEANUP: http: rename HTTP_MSG_DATA_CRLF state
- MINOR: compression: automatically disable compression for older browsers
- MINOR: compression: optimize memLevel to improve byte rate
- BUG/MINOR: http: compression should consider all Accept-Encoding header values
- BUILD: fix coexistence of openssl and zlib
- MINOR: ssl: add pattern and ACLs fetches 'ssl_c_serial' and 'ssl_f_serial'
- BUG/MEDIUM: command-line option -D must have precedence over "debug"
- MINOR: tools: add a clear_addr() function to unset an address
- BUG/MEDIUM: tcp: transparent bind to the source only when address is set
- CLEANUP: remove trashlen
- MAJOR: session: detach the connections from the stream interfaces
- DOC: update document describing relations between internal entities
- BUILD: make it possible to specify ZLIB path
- MINOR: compression: add an offload option to remove the Accept-Encoding header
- BUG: compression: disable auto-close and enable MSG_MORE during transfer
- CLEANUP: completely remove trashlen
- MINOR: chunk: add a function to reset a chunk
- CLEANUP: replace chunk_printf() with chunk_appendf()
- MEDIUM: make the trash be a chunk instead of a char *
- MEDIUM: remove remains of BUFSIZE in HTTP auth and sample conversions
- MEDIUM: stick-table: allocate the table key of size buffer size
- BUG/MINOR: stream_interface: don't loop over ->snd_buf()
- BUG/MINOR: session: ensure that we don't retry connection if some data were sent
- OPTIM: session: don't process the whole session when only timers need a refresh
- BUG/MINOR: session: mark the handshake as complete earlier
- MAJOR: connection: remove the CO_FL_CURR_*_POL flag
- BUG/MAJOR: always clear the CO_FL_WAIT_* flags after updating polling flags
- MAJOR: sepoll: make the poller totally event-driven
- OPTIM: stream_interface: disable reading when CF_READ_DONTWAIT is set
- BUILD: compression: remove a build warning
- MEDIUM: fd: don't unset fdtab[].updated upon delete
- REORG: fd: move the speculative I/O management from ev_sepoll
- REORG: fd: move the fd state management from ev_sepoll
- REORG: fd: centralize the processing of speculative events
- BUG: raw_sock: also consider ENOTCONN in addition to EAGAIN
- BUILD: stream_interface: remove si_fd() and its references
- BUILD: compression: enable build in BSD and OSX Makefiles
- MAJOR: ev_select: make the poller support speculative events
- MAJOR: ev_poll: make the poller support speculative events
- MAJOR: ev_kqueue: make the poller support speculative events
- MAJOR: polling: replace epoll with sepoll and remove sepoll
- MAJOR: polling: remove unused callbacks from the poller struct
- MEDIUM: http: refrain from sending "Connection: close" when Upgrade is present
- CLEANUP: channel: remove any reference of the hijackers
- CLEANUP: stream_interface: remove the external task type target
- MAJOR: connection: replace struct target with a pointer to an enum
- BUG: connection: fix typo in previous commit
- BUG: polling: don't skip polled events in the spec list
- MINOR: splice: disable it when the system returns EBADF
- MINOR: build: allow packagers to specify the default maxzlibmem
- BUG: halog: fix broken output limitation
- BUG: proxy: fix server name lookup in get_backend_server()
- BUG: compression: do not always increment the round counter on allocation failure
- BUG/MEDIUM: compression: release the zlib pools between keep-alive requests
- MINOR: global: don't prevent nbproc from being redefined
- MINOR: config: support process ranges for "bind-process"
- MEDIUM: global: add support for CPU binding on Linux ("cpu-map")
- MINOR: ssl: rename and document the tune.ssl.cachesize option
- DOC: update the PROXY protocol spec to support v2
- MINOR: standard: add a simple popcount function
- MEDIUM: adjust the maxaccept per listener depending on the number of processes
- BUG: compression: properly disable compression when content-type does not match
- MINOR: cli: report connection status in "show sess xxx"
- BUG/MAJOR: stream_interface: certain workloads could cause get stuck
- BUILD: cli: fix build when SSL is enabled
- MINOR: cli: report the fd state in "show sess xxx"
- MINOR: cli: report an error message on missing argument to compression rate
- MINOR: http: add some debugging functions to pretty-print msg state names
- BUG/MAJOR: stream_interface: read0 not always handled since dev12
- DOC: documentation on http header capture is wrong
- MINOR: http: allow the cookie capture size to be changed
- DOC: http header capture has not been limited in size for a long time
- DOC: update readme with build methods for BSD
- BUILD: silence a warning on Solaris about usage of isdigit()
- MINOR: stats: report HTTP compression stats per frontend and per backend
- MINOR: log: add '%Tl' to log-format
- MINOR: samples: update the url_param fetch to match parameters in the path
It was a bit frustrating to have no idea about the bandwidth saved by
HTTP compression. Now we have per-frontend and per-backend stats. The
stats on the HTTP interface are shown in a hover title in the "bytes out"
column if at least something was fed to the compressor. 3 new columns
appeared in the CSV stats output.
It's been documented for a very long time that captured HTTP headers
were limited to 64 characters, but this has not the case anymore since
1.3.11 in 2007 (commit cf7f320f), as they all use their own pool and
have no such limit anymore.
This fix should be backported to 1.4 and 1.3.
Some users need more than 64 characters to log large cookies. The limit
was set to 63 characters (and not 64 as previously documented). Now it
is possible to change this using the global "tune.http.cookielen" setting
if required.
Since commit it is said that only the first value of the first occurrence
of a header is captured. This is wrong. Since the introduction of header
captures in version 1.1 in 2005 (commit e983144d), the WHOLE line of the
LAST occurrence has been captured and the behaviour has never changed.
At this time the doc was correct. The error was introduced in the new doc
in 1.3.14 in 2007 (commit 0ba27505).
So this fix should be backported to 1.4 and 1.3.
New option 'maxcompcpuusage' in global section.
Sets the maximum CPU usage HAProxy can reach before stopping the
compression for new requests or decreasing the compression level of
current requests. It works like 'maxcomprate' but with the Idle.
global.tune.maxaccept was used for all listeners. This becomes really not
convenient when some listeners are bound to a single process and other ones
are bound to many processes.
Now we change the principle : we count the number of processes a listener
is bound to, and apply the maxaccept either entirely if there is a single
process, or divided by twice the number of processes in order to maintain
fairness.
The default limit has also been increased from 32 to 64 as it appeared that
on small machines, 32 was too low to achieve high connection rates.
The new "cpu-map" directive allows one to assign the CPU sets that
a process is allowed to bind to. This is useful in combination with
the "nbproc" and "bind-process" directives.
The support is implicit on Linux 2.6.28 and above.
Now that all pollers make use of speculative I/O, there is no point
having two epoll implementations, so replace epoll with the sepoll code
and remove sepoll which has just become the standard epoll method.
commit 82fe75c1 provided useful details in its log message. We should report
part of them in the documentation to know which algorithms are available.
This patch also makes some formatting cleanups (including a line outside the
compression scope, which exceeded 80 chars).
This patch adds input and output rate calcutation on the HTTP compresion
feature.
Compression can be limited with a maximum rate value in kilobytes per
second. The rate is set with the global 'maxcomprate' option. You can
change this value dynamicaly with 'set rate-limit http-compression
global' on the UNIX socket.
With the global maxzlibmem option, you are able ton control the maximum
amount of RAM usable for HTTP compression.
A test is done before each zlib allocation, if the there isn't available
memory, the test fail and so the zlib initialization, so data won't be
compressed.
The window size and the memlevel of the zlib are now configurable using
global options tune.zlib.memlevel and tune.zlib.windowsize.
It affects the memory consumption of the zlib.
ssl_c_notbefore: start date of client cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_c_notafter: end date of client cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_f_notbefore: start date of frontend cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_f_notafter: end date of frontend cert (string, eg: "121022182230Z" for YYMMDDhhmmss[Z])
ssl_c_s_dn : client cert subject DN (string)
ssl_c_i_dn : client cert issuer DN (string)
ssl_f_s_dn : frontend cert subject DN (string)
ssl_f_i_dn : frontend cert issuer DN (string)
Return either the full DN without params, or just the DN entry (first param) or
its specific occurrence (second param).
This commit introduces HTTP compression using the zlib library.
http_response_forward_body has been modified to call the compression
functions.
This feature includes 3 algorithms: identity, gzip and deflate:
* identity: this is mostly for debugging, and it was useful for
developping the compression feature. With Content-Length in input, it
is making each chunk with the data available in the current buffer.
With chunks in input, it is rechunking, the output chunks will be
bigger or smaller depending of the size of the input chunk and the
size of the buffer. Identity does not apply any change on data.
* gzip: same as identity, but applying a gzip compression. The data
are deflated using the Z_NO_FLUSH flag in zlib. When there is no more
data in the input buffer, it flushes the data in the output buffer
(Z_SYNC_FLUSH). At the end of data, when it receives the last chunk in
input, or when there is no more data to read, it writes the end of
data with Z_FINISH and the ending chunk.
* deflate: same as gzip, but with deflate algorithm and zlib format.
Note that this algorithm has ambiguous support on many browsers and
no support at all from recent ones. It is strongly recommended not
to use it for anything else than experimentation.
You can't choose the compression ratio at the moment, it will be set to
Z_BEST_SPEED (1), as tests have shown very little benefit in terms of
compression ration when going above for HTML contents, at the cost of
a massive CPU impact.
Compression will be activated depending of the Accept-Encoding request
header. With identity, it does not take care of that header.
To build HAProxy with zlib support, use USE_ZLIB=1 in the make
parameters.
This work was initially started by David Du Colombier at Exceliance.
Using "stats bind-process", it becomes possible to indicate to haproxy which
process will get the incoming connections to the stats socket. It will also
shut down the warning when nbproc > 1.
Please consider the following patch for configuration.txt to clarify meaning
of bufsize, maxrewrite and the size of HTTP request which can be processed.
The ssl_npn match could not work by itself because clients do not use
the NPN extension unless the server advertises the protocols it supports.
Thanks to Simone Bordet for the explanations on how to get it right.
These two new log-format tags report the SSL protocol version (%sslv) and the
SSL ciphers (%sslc) used for the connection with the client. For instance, to
append these information just after the client's IP/port address information
on an HTTP log line, use the following configuration :
log-format %Ci:%Cp\ %sslv:%sslc\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %st\ %B\ %cc\ \ %cs\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ %{+Q}r
It will report a line such as the following one :
Oct 12 20:47:30 haproxy[9643]: 127.0.0.1:43602 TLSv1:AES-SHA [12/Oct/2012:20:47:30.303] stick2~ stick2/s1 7/0/12/0/19 200 145 - - ---- 0/0/0/0/0 0/0 "GET /?t=0 HTTP/1.0"
Until now it was not possible to know from the logs whether the incoming
connection was made over SSL or not. In order to address this in the existing
log formats, a new log format %ft was introduced, to log the frontend's name
suffixed with its transport layer. The only transport layer in use right now
is '~' for SSL, so that existing log formats for non-SSL traffic are not
affected at all, and SSL log formats have the frontend's name suffixed with
'~'.
The TCP, HTTP and CLF log format now use %ft instead of %f. This does not
affect existing log formats which still make use of %f however.
It now becomes possible to verify the server's certificate using the "verify"
directive. This one only supports "none" and "required", as it does not make
much sense to also support "optional" here.
When health checks are configured on a server which has the send-proxy
directive and no "port" nor "addr" settings, the health check connections
will automatically use the PROXY protocol. If "port" or "addr" are set,
the "check-send-proxy" directive may be used to force the protocol.
While working on the changes required to make the health checks use the
new connections, it started to become obvious that some naming was not
logical at all in the connections. Specifically, it is not logical to
call the "data layer" the layer which is in charge for all the handshake
and which does not yet provide a data layer once established until a
session has allocated all the required buffers.
In fact, it's more a transport layer, which makes much more sense. The
transport layer offers a medium on which data can transit, and it offers
the functions to move these data when the upper layer requests this. And
it is the upper layer which iterates over the transport layer's functions
to move data which should be called the data layer.
The use case where it's obvious is with embryonic sessions : an incoming
SSL connection is accepted. Only the connection is allocated, not the
buffers nor stream interface, etc... The connection handles the SSL
handshake by itself. Once this handshake is complete, we can't use the
data functions because the buffers and stream interface are not there
yet. Hence we have to first call a specific function to complete the
session initialization, after which we'll be able to use the data
functions. This clearly proves that SSL here is only a transport layer
and that the stream interface constitutes the data layer.
A similar change will be performed to rename app_cb => data, but the
two could not be in the same commit for obvious reasons.
Disables the stateless session resumption (RFC 5077 TLS Ticket
extension) and force to use stateful session resumption. Stateless
session resumption is more expensive in CPU usage.
We were having several different behaviours with monitor-net and
"mode health" :
- monitor-net on TCP connections was evaluated just after accept(),
did not count a connection on the frontend and were not subject
to tcp-request connection rules, and caused an immediate close().
- monitor-net in HTTP mode was evaluated once the session was
accepted (eg: on top of SSL), returned "HTTP/1.0 200 OK\r\n\r\n"
over the connection's data layer and instanciated a session which
was responsible for closing this connection. A connection AND a
session were counted for the frontend ;
- "mode health" with "option httpchk" would do exactly the same as
monitor-net in HTTP mode ;
- "mode health" without "option httpchk" would do the same as above
except that "OK" was returned instead of "HTTP/1.0 200 OK\r\n\r\n".
None of them took care of cleaning the input buffer, sometimes resulting
in a TCP reset to be emitted after the last packet if a request was received
over the connection.
Given the inconsistencies and the complexity in keeping all these features
handled at the right position, we now slightly changed the way they are
handled :
- all of them are handled just after the "tcp-request connection" rules,
so that all of them may be blocked using such rules, offering more
flexibility and consistency ;
- no connection handshake is performed anymore for non-TCP modes
- all of them send the response as raw data over the socket, there is no
more difference between TCP and HTTP mode for example (these rules were
never meant to be served over SSL connections and were never documented
as able to do that).
- any possible pending data on the incoming socket is drained before the
response is sent, in order to avoid the risk of a reset.
- none of them exactly did what was documented !
This results in more consistent, more flexible and more accurate handling of
monitor rules, with smaller and more robust code.
It is sometimes useful to completely disable accepting new connections
on a frontend during maintenance operations. By setting a frontend's
maxconn to zero, connections are not accepted anymore until the limit
is increased again.
There are now too many bind options to still have them in the middle
of the keyword matrix, so let's move them with the server options in
section 5. No new option was documented yet at this point.
Released version 1.5-dev12 with the following main changes :
- CONTRIB: halog: sort URLs by avg bytes_read or total bytes_read
- MEDIUM: ssl: add support for prefer-server-ciphers option
- MINOR: IPv6 support for transparent proxy
- MINOR: protocol: add SSL context to listeners if USE_OPENSSL is defined
- MINOR: server: add SSL context to servers if USE_OPENSSL is defined
- MEDIUM: connection: add a new handshake flag for SSL (CO_FL_SSL_WAIT_HS).
- MEDIUM: ssl: add new files ssl_sock.[ch] to provide the SSL data layer
- MEDIUM: config: add the 'ssl' keyword on 'bind' lines
- MEDIUM: config: add support for the 'ssl' option on 'server' lines
- MEDIUM: ssl: protect against client-initiated renegociation
- BUILD: add optional support for SSL via the USE_OPENSSL flag
- MEDIUM: ssl: add shared memory session cache implementation.
- MEDIUM: ssl: replace OpenSSL's session cache with the shared cache
- MINOR: ssl add global setting tune.sslcachesize to set SSL session cache size.
- MEDIUM: ssl: add support for SNI and wildcard certificates
- DOC: Typos cleanup
- DOC: fix name for "option independant-streams"
- DOC: specify the default value for maxconn in the context of a proxy
- BUG/MINOR: to_log erased with unique-id-format
- LICENSE: add licence exception for OpenSSL
- BUG/MAJOR: cookie prefix doesn't support cookie-less servers
- BUILD: add an AIX 5.2 (and later) target.
- MEDIUM: fd/si: move peeraddr from struct fdinfo to struct connection
- MINOR: halog: use the more recent dual-mode fgets2 implementation
- BUG/MEDIUM: ebtree: ebmb_insert() must not call cmp_bits on full-length matches
- CLEANUP: halog: make clean should also remove .o files
- OPTIM: halog: make use of memchr() on platforms which provide a fast one
- OPTIM: halog: improve cold-cache behaviour when loading a file
- BUG/MINOR: ACL implicit arguments must be created with unresolved flag
- MINOR: replace acl_fetch_{path,url}* with smp_fetch_*
- MEDIUM: pattern: add the "base" sample fetch method
- OPTIM: i386: make use of kernel-mode-linux when available
- BUG/MINOR: tarpit: fix condition to return the HTTP 500 message
- BUG/MINOR: polling: some events were not set in various pollers
- MINOR: http: add the urlp_val ACL match
- BUG: stktable: tcp_src_to_stktable_key() must return NULL on invalid families
- MINOR: stats/cli: add plans to support more stick-table actions
- MEDIUM: stats/cli: add support for "set table key" to enter values
- REORG/MEDIUM: fd: remove FD_STCLOSE from struct fdtab
- REORG/MEDIUM: fd: remove checks for FD_STERROR in ev_sepoll
- REORG/MEDIUM: fd: get rid of FD_STLISTEN
- REORG/MINOR: connection: move declaration to its own include file
- REORG/MINOR: checks: put a struct connection into the server
- MINOR: connection: add flags to the connection struct
- MAJOR: get rid of fdtab[].state and use connection->flags instead
- MINOR: fd: add a new I/O handler to fdtab
- MEDIUM: polling: prepare to call the iocb() function when defined.
- MEDIUM: checks: make use of fdtab->iocb instead of cb[]
- MEDIUM: protocols: use the generic I/O callback for accept callbacks
- MINOR: connection: add a handler for fd-based connections
- MAJOR: connection: replace direct I/O callbacks with the connection callback
- MINOR: fd: make fdtab->owner a connection and not a stream_interface anymore
- MEDIUM: connection: remove the FD_POLL_* flags only once
- MEDIUM: connection: extract the send_proxy callback from proto_tcp
- MAJOR: tcp: remove the specific I/O callbacks for TCP connection probes
- CLEANUP: remove the now unused fdtab direct I/O callbacks
- MAJOR: remove the stream interface and task management code from sock_*
- MEDIUM: stream_interface: pass connection instead of fd in sock_ops
- MEDIUM: stream_interface: centralize the SI_FL_ERR management
- MAJOR: connection: add a new CO_FL_CONNECTED flag
- MINOR: rearrange tcp_connect_probe() and fix wrong return codes
- MAJOR: connection: call data layer handshakes from the handler
- MEDIUM: fd: remove the EV_FD_COND_* primitives
- MINOR: sock_raw: move calls to si_data_close upper
- REORG: connection: replace si_data_close() with conn_data_close()
- MEDIUM: sock_raw: introduce a read0 callback that is different from shutr
- MAJOR: stream_int: use a common stream_int_shut*() functions regardless of the data layer
- MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls
- MEDIUM: fd: add fd_poll_{recv,send} for use when explicit polling is required
- MEDIUM: connection: add definitions for dual polling mechanisms
- MEDIUM: connection: make use of the new polling functions
- MAJOR: make use of conn_{data|sock}_{poll|stop|want}* in connection handlers
- MEDIUM: checks: don't use FD_WAIT_* anymore
- MINOR: fd: get rid of FD_WAIT_*
- MEDIUM: stream_interface: offer a generic function for connection updates
- MEDIUM: stream-interface: offer a generic chk_rcv function for connections
- MEDIUM: stream-interface: add a snd_buf() callback to sock_ops
- MEDIUM: stream-interface: provide a generic stream_int_chk_snd_conn() function
- MEDIUM: stream-interface: provide a generic si_conn_send_cb callback
- MEDIUM: stream-interface: provide a generic stream_sock_read0() function
- REORG/MAJOR: use "struct channel" instead of "struct buffer"
- REORG/MAJOR: extract "struct buffer" from "struct channel"
- MINOR: connection: provide conn_{data|sock}_{read0|shutw} functions
- REORG: sock_raw: rename the files raw_sock*
- MAJOR: raw_sock: extract raw_sock_to_buf() from raw_sock_read()
- MAJOR: raw_sock: temporarily disable splicing
- MINOR: stream-interface: add an rcv_buf callback to sock_ops
- REORG: stream-interface: move sock_raw_read() to si_conn_recv_cb()
- MAJOR: connection: split the send call into connection and stream interface
- MAJOR: stream-interface: restore splicing mechanism
- MAJOR: stream-interface: make conn_notify_si() more robust
- MEDIUM: proxy-proto: don't use buffer flags in conn_si_send_proxy()
- MAJOR: stream-interface: don't commit polling changes in every callback
- MAJOR: stream-interface: fix splice not to call chk_snd by itself
- MEDIUM: stream-interface: don't remove WAIT_DATA when a handshake is in progress
- CLEANUP: connection: split sock_ops into data_ops, app_cp and si_ops
- REORG: buffers: split buffers into chunk,buffer,channel
- MAJOR: channel: remove the BF_OUT_EMPTY flag
- REORG: buffer: move buffer_flush, b_adv and b_rew to buffer.h
- MINOR: channel: rename bi_full to channel_full as it checks the whole channel
- MINOR: buffer: provide a new buffer_full() function
- MAJOR: channel: stop relying on BF_FULL to take action
- MAJOR: channel: remove the BF_FULL flag
- REORG: channel: move buffer_{replace,insert_line}* to buffer.{c,h}
- CLEANUP: channel: usr CF_/CHN_ prefixes instead of BF_/BUF_
- CLEANUP: channel: use "channel" instead of "buffer" in function names
- REORG: connection: move the target pointer from si to connection
- MAJOR: connection: move the addr field from the stream_interface
- MEDIUM: stream_interface: remove CAP_SPLTCP/CAP_SPLICE flags
- MEDIUM: proto_tcp: remove any dependence on stream_interface
- MINOR: tcp: replace tcp_src_to_stktable_key with addr_to_stktable_key
- MEDIUM: connection: add an ->init function to data layer
- MAJOR: session: introduce embryonic sessions
- MAJOR: connection: make the PROXY decoder a handshake handler
- CLEANUP: frontend: remove the old proxy protocol decoder
- MAJOR: connection: rearrange the polling flags.
- MEDIUM: connection: only call tcp_connect_probe when nothing was attempted yet
- MEDIUM: connection: complete the polling cleanups
- MEDIUM: connection: avoid calling handshakes when polling is required
- MAJOR: stream_interface: continue to update data polling flags during handshakes
- CLEANUP: fd: remove fdtab->flags
- CLEANUP: fdtab: flatten the struct and merge the spec struct with the rest
- CLEANUP: includes: fix includes for a number of users of fd.h
- MINOR: ssl: disable TCP quick-ack by default on SSL listeners
- MEDIUM: config: add a "ciphers" keyword to set SSL cipher suites
- MEDIUM: config: add "nosslv3" and "notlsv1" on bind and server lines
- BUG: ssl: mark the connection as waiting for an SSL connection during the handshake
- BUILD: http: rename error_message http_error_message to fix conflicts on RHEL
- BUILD: ssl: fix shctx build on RHEL with futex
- BUILD: include sys/socket.h to fix build failure on FreeBSD
- BUILD: fix build error without SSL (ssl_cert)
- BUILD: ssl: use MAP_ANON instead of MAP_ANONYMOUS
- BUG/MEDIUM: workaround an eglibc bug which truncates the pidfiles when nbproc > 1
- MEDIUM: config: support per-listener backlog and maxconn
- MINOR: session: do not send an HTTP/500 error on SSL sockets
- MEDIUM: config: implement maxsslconn in the global section
- BUG: tcp: close socket fd upon connect error
- MEDIUM: connection: improve error handling around the data layer
- MINOR: config: make the tasks "nice" value configurable on "bind" lines.
- BUILD: shut a gcc warning introduced by commit 269ab31
- MEDIUM: config: centralize handling of SSL config per bind line
- BUILD: makefile: report USE_OPENSSL status in build options
- BUILD: report openssl build settings in haproxy -vv
- MEDIUM: ssl: add sample fetches for is_ssl, ssl_has_sni, ssl_sni_*
- DOC: add a special acknowledgement for the stud project
- DOC: add missing SSL options for servers and listeners
- BUILD: automatically add -lcrypto for SSL
- DOC: add some info about openssl build in the README
This is very convenient to reduce SSL processing priority compared to
other traffic. This applies to CPU usage only, but has a direct impact
on latency under congestion.
SSL connections take a huge amount of memory, and unfortunately openssl
does not check malloc() returns and easily segfaults when too many
connections are used.
The only solution against this is to provide a global maxsslconn setting
to reject SSL connections above the limit in order to avoid reaching
unsafe limits.
With SSL, connections are much more expensive, so it is important to be
able to limit concurrent connections per listener in order to limit the
memory usage.
The correct spelling is "independent", not "independant". This patch
fixes the doc and the configuration parser to accept the correct form.
The config parser still allows the old naming for backwards compatibility.
I came across a couple of typos in configuration.txt and made this patch.
Also, there is an inconsistency between using the word analys/ze in
configuration.txt as well. However, I did not provide a patch for that.
-- Jamie Gloudon
[wt: won't fix the us/uk language mistakes, they'll always exist anyway]
This is used to enter values for stick tables. The most likely usage
is to set gpc0 for a specific IP address in order to block traffic
for abusers without having to reload. Since all data types are
supported, other usages are possible (eg: replace a users's assigned
server).
This one returns the concatenation of the first Host header entry with
the path. It can make content-switching rules easier, help with fighting
DDoS on certain URLs and improve shared caches efficiency.
Doing so allows us to support sticking on URL, URL's IP, URL's port and
path.
Both fetch functions should be improved to support an optional depth
allowing to stick to a server depending on just a few directory
components. This would help with portals, some prefetch-capable
caches and with outgoing connections using multiple internet links.
Default value for maxconn in the context of a proxy is 2000 and is
unrelated to any other value (like global ulimit-n or global
maxconn). Without an explicit a user may think that the default value
is either no limit or equal to the global maxconn value.
Released version 1.5-dev11 with the following main changes :
- BUG/MEDIUM: option forwardfor if-none doesn't work with some configurations
- BUG/MAJOR: trash must always be the size of a buffer
- DOC: fix minor regex example issue and improve doc on stats
- MINOR: stream_interface: add a pointer to the listener for TARG_TYPE_CLIENT
- MEDIUM: protocol: add a pointer to struct sock_ops to the listener struct
- MINOR: checks: add on-marked-up option
- MINOR: balance uri: added 'whole' parameter to include query string in hash calculation
- MEDIUM: stream_interface: remove the si->init
- MINOR: buffers: add a rewind function
- BUG/MAJOR: fix regression on content-based hashing and http-send-name-header
- MAJOR: http: stop using msg->sol outside the parsers
- CLEANUP: http: make it more obvious that msg->som is always null outside of chunks
- MEDIUM: http: get rid of msg->som which is not used anymore
- MEDIUM: http: msg->sov and msg->sol will never wrap
- BUG/MAJOR: checks: don't call set_server_status_* when no LB algo is set
- BUG/MINOR: stop connect timeout when connect succeeds
- REORG: move the send-proxy code to tcp_connect_write()
- REORG/MINOR: session: detect the TCP monitor checks at the protocol accept
- MINOR: stream_interface: introduce a new "struct connection" type
- REORG/MINOR: stream_interface: move si->fd to struct connection
- REORG/MEDIUM: stream_interface: move applet->state and private to connection
- MINOR: stream_interface: add a data channel close function
- MEDIUM: stream_interface: call si_data_close() before releasing the si
- MINOR: peers: use the socket layer operations from the peer instead of sock_raw
- BUG/MINOR: checks: expire on timeout.check if smaller than timeout.connect
- MINOR: add a new function call tracer for debugging purposes
- BUG/MINOR: perform_http_redirect also needs to rewind the buffer
- BUG/MAJOR: b_rew() must pass a signed offset to b_ptr()
- BUG/MEDIUM: register peer sync handler in the proper order
- BUG/MEDIUM: buffers: fix bi_putchr() to correctly advance the pointer
- BUG/MINOR: fix option httplog validation with TCP frontends
- BUG/MINOR: log: don't report logformat errors in backends
- REORG/MINOR: use dedicated proxy flags for the cookie handling
- BUG/MINOR: config: do not report twice the incompatibility between cookie and non-http
- MINOR: http: add support for "httponly" and "secure" cookie attributes
- BUG/MEDIUM: ensure that unresolved arguments are freed exactly once
- BUG/MINOR: commit 196729ef used wrong condition resulting in freeing constants
- MEDIUM: stats: add support for soft stop/soft start in the admin interface
- MEDIUM: stats: add the ability to kill sessions from the admin interface
- BUILD: add support for linux kernels >= 2.6.28
This implements the feature discussed in the earlier thread of killing
connections on backup servers when a non-backup server comes back up. For
example, you can use this to route to a mysql master & slave and ensure
clients don't stay on the slave after the master goes from down->up. I've done
some minimal testing and it seems to work.
[WT: added session flag & doc, moved the killing after logging the server UP,
and ensured that the new server is really usable]
httponly This option tells haproxy to add an "HttpOnly" cookie attribute
when a cookie is inserted. This attribute is used so that a
user agent doesn't share the cookie with non-HTTP components.
Please check RFC6265 for more information on this attribute.
secure This option tells haproxy to add a "Secure" cookie attribute when
a cookie is inserted. This attribute is used so that a user agent
never emits this cookie over non-secure channels, which means
that a cookie learned with this flag will be presented only over
SSL/TLS connections. Please check RFC6265 for more information on
this attribute.
This patch brings a new "whole" parameter to "balance uri" which makes
the hash work over the whole uri, not just the part before the query
string. Len and depth parameter are still honnored.
The reason for this new feature is explained below.
I have 3 backend servers, each accepting different form of HTTP queries:
http://backend1.server.tld/service1.php?q=...
http://backend1.server.tld/service2.php?q=...
http://backend2.server.tld/index.php?query=...&subquery=...
http://backend3.server.tld/image/49b8c0d9ff
Each backend server returns a different response based on either:
- the URI path (the left part of the URI before the question mark)
- the query string (the right part of the URI after the question mark)
- or the combination of both
I wanted to set up a common caching cluster (using 6 Squid servers, each
configured as reverse proxy for those 3 backends) and have HAProxy balance
the queries among the Squid servers based on URL. I also wanted to achieve
hight cache hit ration on each Squid server and send the same queries to
the same Squid servers. Initially I was considering using the 'balance uri'
algorithm, but that would not work as in case of backend2 all queries would
go to only one Squid server. The 'balance url_param' would not work either
as it would send the backend3 queries to only one Squid server.
So I thought the simplest solution would be to use 'balance uri', but to
calculate the hash based on the whole URI (URI path + query string),
instead of just the URI path.
Released version 1.5-dev10 with the following main changes :
- BUG/MINOR: stats admin: "Unexpected result" was displayed unconditionally
- BUG/MAJOR: acl: http_auth_group() must not accept any user from the userlist
- CLEANUP: auth: make the code build again with DEBUG_AUTH
- BUG/MEDIUM: config: don't crash at config load time on invalid userlist names
- REORG: use the name sock_raw instead of stream_sock
- MINOR: stream_interface: add a client target : TARG_TYPE_CLIENT
- BUG/MEDIUM: stream_interface: restore get_src/get_dst
- CLEANUP: sock_raw: remove last references to stream_sock
- CLEANUP: stream_interface: stop exporting socket layer functions
- MINOR: stream_interface: add an init callback to sock_ops
- MEDIUM: stream_interface: derive the socket operations from the target
- MAJOR: fd: remove the need for the socket layer to recheck the connection
- MINOR: session: call the socket layer init function when a session establishes
- MEDIUM: session: add support for tunnel timeouts
- MINOR: standard: add a new debug macro : fddebug()
- CLEANUP: fd: remove unused cb->b pointers in the struct fdtab
- OPTIM: proto_http: don't enable quick-ack on empty buffers
- OPTIM/MAJOR: ev_sepoll: process spec events after polled events
- OPTIM/MEDIUM: stream_interface: add a new SI_FL_NOHALF flag
Tunnel timeouts are used when TCP connections are forwarded, or
when forwarding upgraded HTTP connections (WebSocket) as well as
CONNECT requests to proxies.
This timeout allows long-lived sessions to be supported without
having to set large timeouts to normal requests.
Released version 1.5-dev9 with the following main changes :
- MINOR: Add release callback to si_applet
- CLEANUP: Fix some minor typos
- MINOR: Add TO/FROM_SET flags to struct stream_interface
- CLEANUP: Fix some minor whitespace issues
- MINOR: stats admin: allow unordered parameters in POST requests
- CLEANUP: fix typo in findserver() log message
- MINOR: stats admin: use the backend id instead of its name in the form
- MINOR: stats admin: reduce memcmp()/strcmp() calls on status codes
- DOC: cleanup indentation, alignment, columns and chapters
- DOC: fix some keywords arguments documentation
- MINOR: cli: display the 4 IP addresses and ports on "show sess XXX"
- BUG/MAJOR: log: possible segfault with logformat
- MEDIUM: log: split of log_format generation
- MEDIUM: log: New format-log flags: %Fi %Fp %Si %Sp %Ts %rt %H %pid
- MEDIUM: log: Unique ID
- MINOR: log: log-format: usable without httplog and tcplog
- BUG/MEDIUM: balance source did not properly hash IPv6 addresses
- MINOR: contrib/iprange: add a network IP range to mask converter
- MEDIUM: session: implement the "use-server" directive
- MEDIUM: log: add a new cookie flag 'U' to report situations where cookie is not used
- MEDIUM: http: make extract_cookie_value() iterate over cookie values
- MEDIUM: http: add cookie and scookie ACLs
- CLEANUP: lb_first: add reference to a paper describing the original idea
- MEDIUM: stream_sock: add a get_src and get_dst callback and remove SN_FRT_ADDR_SET
- BUG/MINOR: acl: req_ssl_sni would randomly fail if a session ID is present
- BUILD: http: make extract_cookie_value() return an int not size_t
- BUILD: http: stop gcc-4.1.2 from complaining about possibly uninitialized values
- CLEANUP: http: message parser must ignore HTTP_MSG_ERROR
- MINOR: standard: add a memprintf() function to build formatted error messages
- CLEANUP: remove a few warning about unchecked return values in debug code
- MEDIUM: move message-related flags from transaction to message
- DOC: add a diagram to explain how circular buffers work
- MAJOR: buffer rework: replace ->send_max with ->o
- MAJOR: buffer: replace buf->l with buf->{o+i}
- MINOR: buffers: provide simple pointer normalization functions
- MINOR: buffers: remove unused function buffer_contig_data()
- MAJOR: buffers: replace buf->w with buf->p - buf->o
- MAJOR: buffers: replace buf->r with buf->p + buf->i
- MAJOR: http: move buffer->lr to http_msg->next
- MAJOR: http: change msg->{som,col,sov,eoh} to be relative to buffer origin
- CLEANUP: http: remove unused http_msg->col
- MAJOR: http: turn http_msg->eol to a buffer-relative offset
- MEDIUM: http: add a pointer to the buffer in http_msg
- MAJOR: http: make http_msg->sol relative to buffer's origin
- MEDIUM: http: http_send_name_header: remove references to msg and buffer
- MEDIUM: http: remove buffer arg in a few header manipulation functions
- MEDIUM: http: remove buffer arg in http_capture_bad_message
- MEDIUM: http: remove buffer arg in http_msg_analyzer
- MEDIUM: http: remove buffer arg in http_upgrade_v09_to_v10
- MEDIUM: http: remove buffer arg in http_buffer_heavy_realign
- MEDIUM: http: remove buffer arg in chunk parsing functions
- MINOR: http: remove useless wrapping checks in http_msg_analyzer
- MEDIUM: buffers: fix unsafe use of buffer_ignore at some places
- MEDIUM: buffers: add new pointer wrappers and get rid of almost all buffer_wrap_add calls
- MEDIUM: buffers: implement b_adv() to advance a buffer's pointer
- MEDIUM: buffers: rename a number of buffer management functions
- MEDIUM: http: add a prefetch function for ACL pattern fetch
- MEDIUM: http: make all ACL fetch function use acl_prefetch_http()
- BUG/MINOR: http_auth: ACLs are volatile, not permanent
- MEDIUM: http/acl: merge all request and response ACL fetches of headers and cookies
- MEDIUM: http/acl: make acl_fetch_hdr_{ip,val} rely on acl_fetch_hdr()
- MEDIUM: add a new typed argument list parsing framework
- MAJOR: acl: make use of the new argument parsing framework
- MAJOR: acl: store the ACL argument types in the ACL keyword declaration
- MEDIUM: acl: acl_find_target() now resolves arguments based on their types
- MAJOR: acl: make acl_find_targets also resolve proxy names at config time
- MAJOR: acl: ensure that implicit table and proxies are valid
- MEDIUM: acl: remove unused tests for missing args when args are mandatory
- MEDIUM: pattern: replace type pattern_arg with type arg
- MEDIUM: pattern: get rid of arg_i in all functions making use of arguments
- MEDIUM: pattern: use the standard arg parser
- MEDIUM: pattern: add an argument validation callback to pattern descriptors
- MEDIUM: pattern: report the precise argument parsing error when known.
- MEDIUM: acl: remove the ACL_TEST_F_NULL_MATCH flag
- MINOR: pattern: add a new 'sample' type to store fetched data
- MEDIUM: pattern: add new sample types to replace pattern types
- MAJOR: acl: make use of the new sample struct and get rid of acl_test
- MEDIUM: pattern/acl: get rid of temp_pattern in ACLs
- MEDIUM: acl: get rid of the SET_RES flags
- MEDIUM: get rid of SMP_F_READ_ONLY and SMP_F_MUST_FREE
- MINOR: pattern: replace struct pattern with struct sample
- MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere
- MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description
- MEDIUM: acl/pattern: switch rdp_cookie functions stack up-down
- MEDIUM: acl: replace acl_expr with args in acl fetch_* functions
- MINOR: tcp: replace acl_fetch_rdp_cookie with smp_fetch_rdp_cookie
- MEDIUM: acl/pattern: use the same direction scheme
- MEDIUM: acl/pattern: start merging common sample fetch functions
- MEDIUM: pattern: ensure that sample types always cast into other types.
- MEDIUM: acl/pattern: factor out the src/dst address fetches
- MEDIUM: acl: implement payload and payload_lv
- CLEANUP: pattern: ensure that payload and payload_lv always stay in the buffer
- MINOR: stick_table: centralize the handling of empty keys
- MINOR: pattern: centralize handling of unstable data in pattern_process()
- MEDIUM: pattern: use smp_fetch_rdp_cookie instead of the pattern specific version
- MINOR: acl: set SMP_OPT_ITERATE on fetch functions
- MINOR: acl: add a val_args field to keywords
- MINOR: proto_tcp: validate arguments of payload and payload_lv ACLs
- MEDIUM: http: merge acl and pattern header fetch functions
- MEDIUM: http: merge ACL and pattern cookie fetches into a single one
- MEDIUM: acl: report parsing errors to the caller
- MINOR: arg: improve error reporting on invalid arguments
- MINOR: acl: report errors encountered when loading patterns from files
- MEDIUM: acl: extend the pattern parsers to report meaningful errors
- REORG: use the name "sample" instead of "pattern" to designate extracted data
- REORG: rename "pattern" files
- MINOR: acl: add types to ACL patterns
- MINOR: standard: add an IPv6 parsing function (str62net)
- MEDIUM: acl: support IPv6 address matching
- REORG: stream_interface: create a struct sock_ops to hold socket operations
- REORG/MEDIUM: move protocol->{read,write} to sock_ops
- REORG/MEDIUM: stream_interface: initialize socket ops from descriptors
- REORG/MEDIUM: replace stream interface protocol functions by a proto pointer
- REORG/MEDIUM: move the default accept function from sockstream to protocols.c
- MEDIUM: proto_tcp: remove src6 and dst6 pattern fetch methods
- BUG/MINOR: http: error snapshots are wrong if buffer wraps
- BUG/MINOR: http: ensure that msg->err_pos is always relative to buf->p
- MEDIUM: http: improve error capture reports
- MINOR: acl: add the cook_val() match to match a cookie against an integer
- BUG/MEDIUM: send_proxy: fix initialisation of send_proxy_ofs
- MEDIUM: memory: add the ability to poison memory at run time
- BUG/MEDIUM: log: ensure that unique_id is properly initialized
- MINOR: cfgparse: use a common errmsg pointer for all parsers
- MEDIUM: cfgparse: make backend_parse_balance() use memprintf to report errors
- MEDIUM: cfgparse: use the new error reporting framework for remaining cfg_keywords
- MINOR: http: replace http_message_realign() with buffer_slow_realign()
These methods have been superseded by src and dst which support
multiple families. There is no point keeping them since they appeared
in a development version anyway.
For configurations using "src6", please use "src" instead. For "dst6",
use "dst" instead.
Make use of the new IPv6 pattern type so that acl_match_ip() knows how to
compare pattern and sample.
IPv6 may be entered in their usual form, with or without a netmask appended.
Only bit counts are accepted for IPv6 netmasks. In order to avoid any risk of
trouble with randomly resolved IP addresses, host names are never allowed in
IPv6 patterns.
HAProxy is also able to match IPv4 addresses with IPv6 addresses in the
following situations :
- tested address is IPv4, pattern address is IPv4, the match applies
in IPv4 using the supplied mask if any.
- tested address is IPv6, pattern address is IPv6, the match applies
in IPv6 using the supplied mask if any.
- tested address is IPv6, pattern address is IPv4, the match applies in IPv4
using the pattern's mask if the IPv6 address matches with 2002:IPV4::,
::IPV4 or ::ffff:IPV4, otherwise it fails.
- tested address is IPv4, pattern address is IPv6, the IPv4 address is first
converted to IPv6 by prefixing ::ffff: in front of it, then the match is
applied in IPv6 using the supplied IPv6 mask.
It's easy to merge pattern and ACL fetches of cookies. It allows us
to remove two distinct fetch functions. The new function internally
uses an occurrence number to serve both purposes, but it didn't appear
worth exposing it outside so there is no keyword argument to set it.
However one of the benefits is that the "cookie" fetch for stick tables
now automatically adapts to requests and responses, so there is no more
need for set-cookie().
HTTP header fetch is now done using smp_fetch_hdr() for both ACLs and
patterns. This one also supports an occurrence number, making it possible
to specify explicit occurrences for ACLs and patterns.
These ones were easy to adapt to ACL usage and may really be useful,
so let's make them available right now. It's likely that some extension
such as regex, string-to-IP and raw IP matching will be implemented in
the near future.
src_port, dst_port and url_param have converged between ACLs and patterns.
This means that src_port is now available in patterns and that urlp_* has
been added to ACLs. Some code has moved to accommodate for static function
definitions, but there were little changes.
All ACLs which need to process HTTP contents first call this function which
performs all the preliminary tests and also triggers the request parsing if
needed. A macro was written to simplify the code.
As a side effect, it's not required anymore to check for the HTTP ACL before
checking for HTTP contents.
- Typo on "dispatch" keyword arguments.
- Reindent some blocks for better parsing by automated tools.
- "option mysql-check" and "option pgsql-check" arguments were not documented
as the others.
This patch is a group commit simplify the parsing of the documenation :
- remove remaining tabulations
- realign some lines
- break lines at 80 columns
- add missing chapters in the summary
- fix chapter numbering format
The Unique ID, is an ID generated with several informations. You can use
a log-format string to customize it, with the "unique-id-format" keyword,
and insert it in the request header, with the "unique-id-header" keyword.
%Fi: Frontend IP
%Fp: Frontend Port
%Si: Server IP
%Sp: Server Port
%Ts: Timestamp
%rt: HTTP request counter
%H: hostname
%pid: PID
+X: Hexadecimal represenation
The +X mode in logformat displays hexadecimal for the following flags
%Ci %Cp %Fi %Fp %Bi %Bp %Si %Sp %Ts %ct %pid
rename logformat_write_string() to lf_text()
Optimize size computation
The ACL matches rely on the extract_cookie_value() function as used for
for patterns. This permits ACLs to match cookie values based on the cookie
name instead of having to perform substring matching on the cookie header.
Sometimes it is desirable to forward a particular request to a specific
server without having to declare a dedicated backend for this server. This
can be achieved using the "use-server" rules. These rules are evaluated after
the "redirect" rules and before evaluating cookies, and they have precedence
on them. There may be as many "use-server" rules as desired. All of these
rules are evaluated in their declaration order, and the first one which
matches will assign the server.
Released version 1.5-dev8 with the following main changes :
- MINOR: patch for minor typo (ressources/resources)
- MEDIUM: http: add support for sending the server's name in the outgoing request
- DOC: mention that default checks are TCP connections
- BUG/MINOR: fix options forwardfor if-none when an alternative header name is specified
- CLEANUP: Make check_statuses, analyze_statuses and process_chk static
- CLEANUP: Fix HCHK spelling errors
- BUG/MINOR: fix typo in processing of http-send-name-header
- MEDIUM: log: Use linked lists for loggers
- BUILD: fix declaration inside a scope block
- REORG: log: split send_log function
- MINOR: config: Parse the string of the log-format config keyword
- MINOR: add ultoa, ulltoa, ltoa, lltoa implementations
- MINOR: Date and time fonctions that don't use snprintf
- MEDIUM: log: make http_sess_log use log_format
- DOC: log-format documentation
- MEDIUM: log: use log_format for mode tcplog
- MEDIUM: log-format: backend source address %Bi %Bp
- BUG/MINOR: log-format: fix %o flag
- BUG/MEDIUM: bad length in log_format and __send_log
- MINOR: logformat %st is signed
- BUILD/MINOR: fix the source URL in the spec file
- DOC: acl is http_first_req, not http_req_first
- BUG/MEDIUM: don't trim last spaces from headers consisting only of spaces
- MINOR: acl: add new matches for header/path/url length
- BUILD: halog: make halog build on solaris
- BUG/MINOR: don't use a wrong port when connecting to a server with mapped ports
- MINOR: remove the client/server side distinction in SI addresses
- MINOR: halog: add support for matching queued requests
- DOC: indicate that cookie "prefix" and "indirect" should not be mixed
- OPTIM/MINOR: move struct sockaddr_storage to the tail of structs
- OPTIM/MINOR: make it possible to change pipe size (tune.pipesize)
- BUILD/MINOR: silent a build warning in src/pipe.c (fcntl)
- OPTIM/MINOR: move the hdr_idx pools out of the proxy struct
- MEDIUM: tune.http.maxhdr makes it possible to configure the maximum number of HTTP headers
- BUG/MINOR: fix a segfault when parsing a config with undeclared peers
- CLEANUP: rename possibly confusing struct field "tracked"
- BUG/MEDIUM: checks: fix slowstart behaviour when server tracking is in use
- MINOR: config: tolerate server "cookie" setting in non-HTTP mode
- MEDIUM: buffers: add some new primitives and rework existing ones
- BUG: buffers: don't return a negative value on buffer_total_space_res()
- MINOR: buffers: make buffer_pointer() support negative pointers too
- CLEANUP: kill buffer_replace() and use an inline instead
- BUG: tcp: option nolinger does not work on backends
- CLEANUP: ebtree: remove a few annoying signedness warnings
- CLEANUP: ebtree: clarify licence and update to 6.0.6
- CLEANUP: ebtree: remove 4-year old harmless typo in duplicates insertion code
- CLEANUP: ebtree: remove another typo, a wrong initialization in insertion code
- BUG: ebtree: ebst_lookup() could return the wrong entry
- OPTIM: stream_sock: reduce the amount of in-flight spliced data
- OPTIM: stream_sock: save a failed recv syscall when splice returns EAGAIN
- MINOR: acl: add support for TLS server name matching using SNI
- BUG: http: re-enable TCP quick-ack upon incomplete HTTP requests
- BUG: proto_tcp: don't try to bind to a foreign address if sin_family is unknown
- MINOR: pattern: export the global temporary pattern
- CLEANUP: patterns: get rid of pattern_data_setstring()
- MEDIUM: acl: use temp_pattern to store fetched information in the "method" match
- MINOR: acl: include pattern.h to make pattern migration more transparent
- MEDIUM: pattern: change the pattern data integer from unsigned to signed
- MEDIUM: acl: use temp_pattern to store any integer-type information
- MEDIUM: acl: use temp_pattern to store any address-type information
- CLEANUP: acl: integer part of acl_test is not used anymore
- MEDIUM: acl: use temp_pattern to store any string-type information
- CLEANUP: acl: remove last data fields from the acl_test struct
- MEDIUM: http: replace get_ip_from_hdr2() with http_get_hdr()
- MEDIUM: patterns: the hdr() pattern is now of type string
- DOC: add minimal documentation on how ACLs work internally
- DOC: add a coding-style file
- OPTIM: halog: keep a fast path for the lines-count only
- CLEANUP: silence a warning when building on sparc
- BUG: http: tighten the list of allowed characters in a URI
- MEDIUM: http: block non-ASCII characters in URIs by default
- DOC: add some documentation from RFC3986 about URI format
- BUG/MINOR: cli: correctly remove the whole table on "clear table"
- BUG/MEDIUM: correctly disable servers tracking another disabled servers.
- BUG/MEDIUM: zero-weight servers must not dequeue requests from the backend
- MINOR: halog: add some help on the command line
- BUILD: fix build error on FreeBSD
- BUG: fix double free in peers config error path
- MEDIUM: improve config check return codes
- BUILD: make it possible to look for pcre in the default system paths
- MINOR: config: emit a warning when 'default_backend' masks servers
- MINOR: backend: rework the LC definition to support other connection-based algos
- MEDIUM: backend: add the 'first' balancing algorithm
- BUG: fix httplog trailing LF
- MEDIUM: increase chunk-size limit to 2GB-1
- BUG: queue: fix dequeueing sequence on HTTP keep-alive sessions
- BUG: http: disable TCP delayed ACKs when forwarding content-length data
- BUG: checks: fix server maintenance exit sequence
- BUG/MINOR: stream_sock: don't remove BF_EXPECT_MORE and BF_SEND_DONTWAIT on partial writes
- DOC: enumerate valid status codes for "observe layer7"
- MINOR: buffer: switch a number of buffer args to const
- CLEANUP: silence signedness warning in acl.c
- BUG: stream_sock: si->release was not called upon shutw()
- MINOR: log: use "%ts" to log term status only and "%tsc" to log with cookie
- BUG/CRITICAL: log: fix risk of crash in development snapshot
- BUG/MAJOR: possible crash when using capture headers on TCP frontends
- MINOR: config: disable header captures in TCP mode and complain
The difference could be seen when logging a request in HTTP mode with option
tcplog, as it would keep emitting 4 chars. Better use two distinct flags to
clear the confusion.
%Bi return the backend source IP
%Bp return the backend source port
Add a function pointer in logformat_type to do additional configuration
during the log-format variable parsing.
Merge http_sess_log() and tcp_sess_log() to sess_log() and move it to
log.c
A new field in logformat_type define if you can use a logformat
variable in TCP or HTTP mode.
doc: log-format in tcp mode
Note that due to the way log buffer allocation currently works, trying to
log an HTTP request without "option httplog" is still not possible. This
will change in the near future.
The principle behind this load balancing algorithm was first imagined
and modeled by Steen Larsen then iteratively refined through several
work sessions until it would totally address its original goal.
The purpose of this algorithm is to always use the smallest number of
servers so that extra servers can be powered off during non-intensive
hours. Additional tools may be used to do that work, possibly by
locally monitoring the servers' activity.
The first server with available connection slots receives the connection.
The servers are choosen from the lowest numeric identifier to the highest
(see server parameter "id"), which defaults to the server's position in
the farm. Once a server reaches its maxconn value, the next server is used.
It does not make sense to use this algorithm without setting maxconn. Note
that it can however make sense to use minconn so that servers are not used
at full load before starting new servers, and so that introduction of new
servers requires a progressively increasing load (the number of servers
would more or less follow the square root of the load until maxconn is
reached). This algorithm ignores the server weight, and is more beneficial
to long sessions such as RDP or IMAP than HTTP, though it can be useful
there too.
These ones are invalid and blocked unless "option accept-invalid-http-request"
is specified in the frontend. In any case, the faulty request is logged.
Note that some of the remaining invalid chars are still not checked against,
those are the invalid ones between 32 and 127 :
34 ('"'), 60 ('<'), 62 ('>'), 92 ('\'), 94 ('^'),
96 ('`'), 123 ('{'), 124 ('|'), 125 ('}')
Using a lookup table might be better at some point.
New option "http-send-name-header" specifies the name of a header which
will hold the server name in outgoing requests. This is the name of the
server the connection is really sent to, which means that upon redispatches,
the header's value is updated so that it always matches the server's name.
This pattern previously was limited to type IP. With the new header
extraction function, it becomes possible to extract strings, so that
the header can be returned as a string. This will not change anything
to existing configs, as string will automatically be converted to IP
when needed. However, new configs will be able to use IPv6 addresses
from headers in stick-tables, as well as stick on any non-IP header
(eg: host, user-agent, ...).
Server Name Indication (SNI) is a TLS extension which makes a client
present the name of the server it is connecting to in the client hello.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.
The new ACL "req_ssl_sni" matches the name extracted from the TLS
handshake against a list of names which may be loaded from a file if
needed.
This patch settles the 2 loggers limitation.
Loggers are now stored in linked lists.
Using "global log", the global loggers list content is added at the end
of the current proxy list. Each "log" entries are added at the end of
the proxy list.
"no log" flush a logger list.
For a long time, the max number of headers was taken as a part of the buffer
size. Since the header size can be configured at runtime, it does not make
much sense anymore.
Nothing was making it necessary to have a static value, so let's turn this into
a tunable with a default value of 101 which equals what was previously used.
By default, pipes are the default size for the system. But sometimes when
using TCP splicing, it can improve performance to increase pipe sizes,
especially if it is suspected that pipes are not filled and that many
calls to splice() are performed. This has an impact on the kernel's
memory footprint, so this must not be changed if impacts are not understood.
When prefix and indirect are used together, a client which connects to
a server with a cookie will never get any cookie update from this server,
which will be removed by the "indirect" option.
This patch introduces hdr_len, path_len and url_len for matching these
respective parts lengths against integers. This can be used to detect
abuse or empty headers.
