mirrors/haproxy
1
0
Fork 0
mirror of https://git.haproxy.org/git/haproxy.git/ synced 2025-08-18 21:21:22 +02:00
Code Issues Projects Releases Wiki Activity
18,441 Commits 16 Branches 394 Tags
Commit Graph

1764 Commits

Author SHA1 Message Date
Willy Tarreau
26f6ae12c0 MAJOR: config: disable support for nbproc and nbthread in parallel 
When 1.8 was released, we wanted to support both nbthread and nbproc to
observe how things would go. Since then it appeared obvious that the two
are never used together because of the pain to configure affinity in this
case, and instead of bringing benefits, it brings the limitations of both
models, and causes multiple threads to compete for the same CPU. In
addition, it costs a lot to support both in parallel, so let's get rid
of this once for all.
 2019-02-27 14:27:04 +01:00
Willy Tarreau
c299e1e027 DOC: fix alphabetic ordering for "tune.fail-alloc" setting 
Last time I verified, the "f" letter was not between the "l" and the
"m", but between the "e" and the "g", so let's move this entry to the
right place.
 2019-02-27 14:18:51 +01:00
Willy Tarreau
6c1b667e57 [RELEASE] Released version 2.0-dev1 
Released version 2.0-dev1 with the following main changes :
    - MINOR: mux-h2: only increase the connection window with the first update
    - REGTESTS: remove the expected window updates from H2 handshakes
    - BUG/MINOR: mux-h2: make empty HEADERS frame return a connection error
    - BUG/MEDIUM: mux-h2: mark that we have too many CS once we have more than the max
    - MEDIUM: mux-h2: remove padlen during headers phase
    - MINOR: h2: add a bit-based frame type representation
    - MINOR: mux-h2: remove useless check for empty frame length in h2s_decode_headers()
    - MEDIUM: mux-h2: decode HEADERS frames before allocating the stream
    - MINOR: mux-h2: make h2c_send_rst_stream() use the dummy stream's error code
    - MINOR: mux-h2: add a new dummy stream for the REFUSED_STREAM error code
    - MINOR: mux-h2: fail stream creation more cleanly using RST_STREAM
    - MINOR: buffers: add a new b_move() function
    - MINOR: mux-h2: make h2_peek_frame_hdr() support an offset
    - MEDIUM: mux-h2: handle decoding of CONTINUATION frames
    - CLEANUP: mux-h2: remove misleading comments about CONTINUATION
    - BUG/MEDIUM: servers: Don't try to reuse connection if we switched server.
    - BUG/MEDIUM: tasks: Decrement tasks_run_queue in tasklet_free().
    - BUG/MINOR: htx: send the proper authenticate header when using http-request auth
    - BUG/MEDIUM: mux_h2: Don't add to the idle list if we're full.
    - BUG/MEDIUM: servers: Fail if we fail to allocate a conn_stream.
    - BUG/MAJOR: servers: Use the list api correctly to avoid crashes.
    - BUG/MAJOR: servers: Correctly use LIST_ELEM().
    - BUG/MAJOR: sessions: Use an unlimited number of servers for the conn list.
    - BUG/MEDIUM: servers: Flag the stream_interface on handshake error.
    - MEDIUM: servers: Be smarter when switching connections.
    - MEDIUM: sessions: Keep track of which connections are idle.
    - MINOR: payload: add sample fetch for TLS ALPN
    - BUG/MEDIUM: log: don't mark log FDs as non-blocking on terminals
    - MINOR: channel: Add the function channel_add_input
    - MINOR: stats/htx: Call channel_add_input instead of updating channel state by hand
    - BUG/MEDIUM: cache: Be sure to end the forwarding when XFER length is unknown
    - BUG/MAJOR: htx: Return the good block address after a defrag
    - MINOR: lb: allow redispatch when using consistent hash
    - CLEANUP: mux-h2: fix end-of-stream flag name when processing headers
    - BUG/MEDIUM: mux-h2: always restart reading if data are available
    - BUG/MINOR: mux-h2: set the stream-full flag when leaving h2c_decode_headers()
    - BUG/MINOR: mux-h2: don't check the CS count in h2c_bck_handle_headers()
    - BUG/MINOR: mux-h2: mark end-of-stream after processing response HEADERS, not before
    - BUG/MINOR: mux-h2: only update rxbuf's length for H1 headers
    - BUG/MEDIUM: mux-h1: use per-direction flags to indicate transitions
    - BUG/MEDIUM: mux-h1: make HTX chunking consistent with H2
    - BUG/MAJOR: stream-int: Update the stream expiration date in stream_int_notify()
    - BUG/MEDIUM: proto-htx: Set SI_FL_NOHALF on server side when request is done
    - BUG/MEDIUM: mux-h1: Add a task to handle connection timeouts
    - MINOR: mux-h2: make h2c_decode_headers() return a status, not a count
    - MINOR: mux-h2: add a new dummy stream : h2_error_stream
    - MEDIUM: mux-h2: make h2c_decode_headers() support recoverable errors
    - BUG/MINOR: mux-h2: detect when the HTX EOM block cannot be added after headers
    - MINOR: mux-h2: remove a misleading and impossible test
    - CLEANUP: mux-h2: clean the stream error path on HEADERS frame processing
    - MINOR: mux-h2: check for too many streams only for idle streams
    - MINOR: mux-h2: set H2_SF_HEADERS_RCVD when a HEADERS frame was decoded
    - BUG/MEDIUM: mux-h2: decode trailers in HEADERS frames
    - MINOR: h2: add h2_make_h1_trailers to turn H2 headers to H1 trailers
    - MEDIUM: mux-h2: pass trailers to H1 (legacy mode)
    - MINOR: htx: add a new function to add a block without filling it
    - MINOR: h2: add h2_make_htx_trailers to turn H2 headers to HTX trailers
    - MEDIUM: mux-h2: pass trailers to HTX
    - MINOR: mux-h1: parse the content-length header on output and set H1_MF_CLEN
    - BUG/MEDIUM: mux-h1: don't enforce chunked encoding on requests
    - MINOR: mux-h2: make HTX_BLK_EOM processing idempotent
    - MINOR: h1: make the H1 headers block parser able to parse headers only
    - MEDIUM: mux-h2: emit HEADERS frames when facing HTX trailers blocks
    - MINOR: stream/htx: Add info about the HTX structs in "show sess all" command
    - MINOR: stream: Add the subscription events of SIs in "show sess all" command
    - MINOR: mux-h1: Add the subscription events in "show fd" command
    - BUG/MEDIUM: h1: Get the h1m state when restarting the headers parsing
    - BUG/MINOR: cache/htx: Be sure to count partial trailers
    - BUG/MEDIUM: h1: In h1_init(), wake the tasklet instead of calling h1_recv().
    - BUG/MEDIUM: server: Defer the mux init until after xprt has been initialized.
    - MINOR: connections: Remove a stall comment.
    - BUG/MEDIUM: cli: make "show sess" really thread-safe
    - BUILD: add a new file "version.c" to carry version updates
    - MINOR: stream/htx: add the HTX flags output in "show sess all"
    - MINOR: stream/cli: fix the location of the waiting flag in "show sess all"
    - MINOR: stream/cli: report more info about the HTTP messages on "show sess all"
    - BUG/MINOR: lua: bad args are returned for Lua actions
    - BUG/MEDIUM: lua: dead lock when Lua tasks are trigerred
    - MINOR: htx: Add an helper function to get the max space usable for a block
    - MINOR: channel/htx: Add HTX version for some helper functions
    - BUG/MEDIUM: cache/htx: Respect the reserve when cached objects are served
    - BUG/MINOR: stats/htx: Respect the reserve when the stats page is dumped
    - DOC: regtest: make it clearer what the purpose of the "broken" series is
    - REGTEST: mailers: add new test for 'mailers' section
    - REGTEST: Add a reg test for health-checks over SSL/TLS.
    - BUG/MINOR: mux-h1: Close connection on shutr only when shutw was really done
    - MEDIUM: mux-h1: Clarify how shutr/shutw are handled
    - BUG/MINOR: compression: Disable it if another one is already in progress
    - BUG/MINOR: filters: Detect cache+compression config on legacy HTTP streams
    - BUG/MINOR: cache: Disable the cache if any compression filter precedes it
    - REGTEST: Add some informatoin to test results.
    - MINOR: htx: Add a function to truncate all blocks after a specific offset
    - MINOR: channel/htx: Add the HTX version of channel_truncate/erase
    - BUG/MINOR: proto_htx: Use HTX versions to truncate or erase a buffer
    - BUG/CRITICAL: mux-h2: re-check the frame length when PRIORITY is used
    - DOC: Fix typo in req.ssl_alpn example (commit 4afdd138424ab...)
    - DOC: http-request cache-use / http-response cache-store expects cache name
    - REGTEST: "capture (request|response)" regtest.
    - BUG/MINOR: lua/htx: Respect the reserve when data are send from an HTX applet
    - REGTEST: filters: add compression test
    - BUG/MEDIUM: init: Initialize idle_orphan_conns for first server in server-template
    - BUG/MEDIUM: ssl: Disable anti-replay protection and set max data with 0RTT.
    - DOC: Be a bit more explicit about allow-0rtt security implications.
    - MINOR: mux-h1: make the mux_h1_ops struct static
    - BUILD: makefile: add an EXTRA_OBJS variable to help build optional code
    - BUG/MEDIUM: connection: properly unregister the mux on failed initialization
    - BUG/MAJOR: cache: fix confusion between zero and uninitialized cache key
    - REGTESTS: test case for map_regm commit 271022150d
    - REGTESTS: Basic tests for concat,strcmp,word,field,ipmask converters
    - REGTESTS: Basic tests for using maps to redirect requests / select backend
    - DOC: REGTESTS README varnishtest -Dno-htx= define.
    - MINOR: spoe: Make the SPOE filter compatible with HTX proxies
    - MINOR: checks: Store the proxy in checks.
    - BUG/MEDIUM: checks: Avoid having an associated server for email checks.
    - REGTEST: Switch to vtest.
    - REGTEST: Adapt reg test doc files to vtest.
    - BUG/MEDIUM: h1: Make sure we destroy an inactive connectin that did shutw.
    - BUG/MINOR: base64: dec func ignores padding for output size checking
    - BUG/MEDIUM: ssl: missing allocation failure checks loading tls key file
    - MINOR: ssl: add support of aes256 bits ticket keys on file and cli.
    - BUG/MINOR: backend: don't use url_param_name as a hint for BE_LB_ALGO_PH
    - BUG/MINOR: backend: balance uri specific options were lost across defaults
    - BUG/MINOR: backend: BE_LB_LKUP_CHTREE is a value, not a bit
    - MINOR: backend: move url_param_name/len to lbprm.arg_str/len
    - MINOR: backend: make headers and RDP cookie also use arg_str/len
    - MINOR: backend: add new fields in lbprm to store more LB options
    - MINOR: backend: make the header hash use arg_opt1 for use_domain_only
    - MINOR: backend: remap the balance uri settings to lbprm.arg_opt{1,2,3}
    - MINOR: backend: move hash_balance_factor out of chash
    - MEDIUM: backend: move all LB algo parameters into an union
    - MINOR: backend: make the random algorithm support a number of draws
    - BUILD/MEDIUM: da: Necessary code changes for new buffer API.
    - BUG/MINOR: stick_table: Prevent conn_cur from underflowing
    - BUG: 51d: Changes to the buffer API in 1.9 were not applied to the 51Degrees code.
    - BUG/MEDIUM: stats: Get the right scope pointer depending on HTX is used or not
    - DOC: add a missing space in the documentation for bc_http_major
    - REGTEST: checks basic stats webpage functionality
    - BUG/MEDIUM: servers: Make assign_tproxy_address work when ALPN is set.
    - BUG/MEDIUM: connections: Add the CO_FL_CONNECTED flag if a send succeeded.
    - DOC: add github issue templates
    - MINOR: cfgparse: Extract some code to be re-used.
    - CLEANUP: cfgparse: Return asap from cfg_parse_peers().
    - CLEANUP: cfgparse: Code reindentation.
    - MINOR: cfgparse: Useless frontend initialization in "peers" sections.
    - MINOR: cfgparse: Rework peers frontend init.
    - MINOR: cfgparse: Simplication.
    - MINOR: cfgparse: Make "peer" lines be parsed as "server" lines.
    - MINOR: peers: Make outgoing connection to SSL/TLS peers work.
    - MINOR: cfgparse: SSL/TLS binding in "peers" sections.
    - DOC: peers: SSL/TLS documentation for "peers"
    - BUG/MINOR: startup: certain goto paths in init_pollers fail to free
    - BUG/MEDIUM: checks: fix recent regression on agent-check making it crash
    - BUG/MINOR: server: don't always trust srv_check_health when loading a server state
    - BUG/MINOR: check: Wake the check task if the check is finished in wake_srv_chk()
    - BUG/MEDIUM: ssl: Fix handling of TLS 1.3 KeyUpdate messages
    - DOC: mention the effect of nf_conntrack_tcp_loose on src/dst
    - BUG/MINOR: proto-htx: Return an error if all headers cannot be received at once
    - BUG/MEDIUM: mux-h2/htx: Respect the channel's reserve
    - BUG/MINOR: mux-h1: Apply the reserve on the channel's buffer only
    - BUG/MINOR: mux-h1: avoid copying output over itself in zero-copy
    - BUG/MAJOR: mux-h2: don't destroy the stream on failed allocation in h2_snd_buf()
    - BUG/MEDIUM: backend: also remove from idle list muxes that have no more room
    - BUG/MEDIUM: mux-h2: properly abort on trailers decoding errors
    - MINOR: h2: declare new sets of frame types
    - BUG/MINOR: mux-h2: CONTINUATION in closed state must always return GOAWAY
    - BUG/MINOR: mux-h2: headers-type frames in HREM are always a connection error
    - BUG/MINOR: mux-h2: make it possible to set the error code on an already closed stream
    - BUG/MINOR: hpack: return a compression error on invalid table size updates
    - MINOR: server: make sure pool-max-conn is >= -1
    - BUG/MINOR: stream: take care of synchronous errors when trying to send
    - CLEANUP: server: fix indentation mess on idle connections
    - BUG/MINOR: mux-h2: always check the stream ID limit in h2_avail_streams()
    - BUG/MINOR: mux-h2: refuse to allocate a stream with too high an ID
    - BUG/MEDIUM: backend: never try to attach to a mux having no more stream available
    - MINOR: server: add a max-reuse parameter
    - MINOR: mux-h2: always consider a server's max-reuse parameter
    - MEDIUM: stream-int: always mark pending outgoing SI_ST_CON
    - MINOR: stream: don't wait before retrying after a failed connection reuse
    - MEDIUM: h2: always parse and deduplicate the content-length header
    - BUG/MINOR: mux-h2: always compare content-length to the sum of DATA frames
    - CLEANUP: h2: Remove debug printf in mux_h2.c
    - MINOR: cfgparse: make the process/thread parser support a maximum value
    - MINOR: threads: make MAX_THREADS configurable at build time
    - DOC: nbthread is no longer experimental.
    - BUG/MINOR: listener: always fill the source address for accepted socketpairs
    - BUG/MINOR: mux-h2: do not report available outgoing streams after GOAWAY
    - BUG/MINOR: spoe: corrected fragmentation string size
    - BUG/MINOR: task: fix possibly missed event in inter-thread wakeups
    - BUG/MEDIUM: servers: Attempt to reuse an unfinished connection on retry.
    - BUG/MEDIUM: backend: always call si_detach_endpoint() on async connection failure
    - SCRIPTS: add the issue tracker URL to the announce script
    - MINOR: peers: Extract some code to be reused.
    - CLEANUP: peers: Indentation fixes.
    - MINOR: peers: send code factorization.
    - MINOR: peers: Add new functions to send code and reduce the I/O handler.
    - MEDIUM: peers: synchronizaiton code factorization to reduce the size of the I/O handler.
    - MINOR: peers: Move update receive code to reduce the size of the I/O handler.
    - MINOR: peers: Move ack, switch and definition receive code to reduce the size of the I/O handler.
    - MINOR: peers: Move high level receive code to reduce the size of I/O handler.
    - CLEANUP: peers: Be more generic.
    - MINOR: peers: move error handling to reduce the size of the I/O handler.
    - MINOR: peers: move messages treatment code to reduce the size of the I/O handler.
    - MINOR: peers: move send code to reduce the size of the I/O handler.
    - CLEANUP: peers: Remove useless statements.
    - MINOR: peers: move "hello" message treatment code to reduce the size of the I/O handler.
    - MINOR: peers: move peer initializations code to reduce the size of the I/O handler.
    - CLEANUP: peers: factor the error handling code in peer_treet_updatemsg()
    - CLEANUP: peers: factor error handling in peer_treat_definedmsg()
    - BUILD/MINOR: peers: shut up a build warning introduced during last cleanup
    - BUG/MEDIUM: mux-h2: only close connection on request frames on closed streams
    - CLEANUP: mux-h2: remove two useless but misleading assignments
    - BUG/MEDIUM: checks: Check that conn_install_mux succeeded.
    - BUG/MEDIUM: servers: Only destroy a conn_stream we just allocated.
    - BUG/MEDIUM: servers: Don't add an incomplete conn to the server idle list.
    - BUG/MEDIUM: checks: Don't try to set ALPN if connection failed.
    - BUG/MEDIUM: h2: In h2_send(), stop the loop if we failed to alloc a buf.
    - BUG/MEDIUM: peers: Handle mux creation failure.
    - BUG/MEDIUM: servers: Close the connection if we failed to install the mux.
    - BUG/MEDIUM: compression: Rewrite strong ETags
    - BUG/MINOR: deinit: tcp_rep.inspect_rules not deinit, add to deinit
    - CLEANUP: mux-h2: remove misleading leftover test on h2s' nullity
    - BUG/MEDIUM: mux-h2: wake up flow-controlled streams on initial window update
    - BUG/MEDIUM: mux-h2: fix two half-closed to closed transitions
    - BUG/MEDIUM: mux-h2: make sure never to send GOAWAY on too old streams
    - BUG/MEDIUM: mux-h2: do not abort HEADERS frame before decoding them
    - BUG/MINOR: mux-h2: make sure response HEADERS are not received in other states than OPEN and HLOC
    - MINOR: h2: add a generic frame checker
    - MEDIUM: mux-h2: check the frame validity before considering the stream state
    - CLEANUP: mux-h2: remove stream ID and frame length checks from the frame parsers
    - BUG/MINOR: mux-h2: make sure request trailers on aborted streams don't break the connection
    - DOC: compression: Update the reasons for disabled compression
    - BUG/MEDIUM: buffer: Make sure b_is_null handles buffers waiting for allocation.
    - DOC: htx: make it clear that htxbuf() and htx_from_buf() always return valid pointers
    - MINOR: htx: never check for null htx pointer in htx_is_{,not_}empty()
    - MINOR: mux-h2: consistently rely on the htx variable to detect the mode
    - BUG/MEDIUM: peers: Peer addresses parsing broken.
    - BUG/MEDIUM: mux-h1: Don't add "transfer-encoding" if message-body is forbidden
    - BUG/MEDIUM: connections: Don't forget to remove CO_FL_SESS_IDLE.
    - BUG/MINOR: stream: don't close the front connection when facing a backend error
    - BUG/MEDIUM: mux-h2: wait for the mux buffer to be empty before closing the connection
    - MINOR: stream-int: add a new flag to mention that we want the connection to be killed
    - MINOR: connstream: have a new flag CS_FL_KILL_CONN to kill a connection
    - BUG/MEDIUM: mux-h2: do not close the connection on aborted streams
    - BUG/MINOR: server: fix logic flaw in idle connection list management
    - MINOR: mux-h2: max-concurrent-streams should be unsigned
    - MINOR: mux-h2: make sure to only check concurrency limit on the frontend
    - MINOR: mux-h2: learn and store the peer's advertised MAX_CONCURRENT_STREAMS setting
    - BUG/MEDIUM: mux-h2: properly consider the peer's advertised max-concurrent-streams
    - MINOR: xref: Add missing barriers.
    - MINOR: muxes: Don't bother to LIST_DEL(&conn->list) before calling conn_free().
    - MINOR: debug: Add an option that causes random allocation failures.
    - BUG/MEDIUM: backend: always release the previous connection into its own target srv_list
    - BUG/MEDIUM: htx: check the HTX compatibility in dynamic use-backend rules
    - BUG/MINOR: tune.fail-alloc: Don't forget to initialize ret.
    - BUG/MINOR: backend: check srv_conn before dereferencing it
    - BUG/MEDIUM: mux-h2: always omit :scheme and :path for the CONNECT method
    - BUG/MEDIUM: mux-h2: always set :authority on request output
    - BUG/MEDIUM: stream: Don't forget to free s->unique_id in stream_free().
    - BUG/MINOR: threads: fix the process range of thread masks
    - BUG/MINOR: config: fix bind line thread mask validation
    - CLEANUP: threads: fix misleading comment about all_threads_mask
    - CLEANUP: threads: use nbits to calculate the thread mask
    - OPTIM: listener: optimize cache-line packing for struct listener
    - MINOR: tools: improve the popcount() operation
    - MINOR: config: keep an all_proc_mask like we have all_threads_mask
    - MINOR: global: add proc_mask() and thread_mask()
    - MINOR: config: simplify bind_proc processing using proc_mask()
    - MINOR: threads: make use of thread_mask() to simplify some thread calculations
    - BUG/MINOR: compression: properly report compression stats in HTX mode
    - BUG/MINOR: task: close a tiny race in the inter-thread wakeup
    - BUG/MAJOR: config: verify that targets of track-sc and stick rules are present
    - BUG/MAJOR: spoe: verify that backends used by SPOE cover all their callers' processes
    - BUG/MAJOR: htx/backend: Make all tests on HTTP messages compatible with HTX
    - BUG/MINOR: config: make sure to count the error on incorrect track-sc/stick rules
    - DOC: ssl: Clarify when pre TLSv1.3 cipher can be used
    - DOC: ssl: Stop documenting ciphers example to use
    - BUG/MINOR: spoe: do not assume agent->rt is valid on exit
    - BUG/MINOR: lua: initialize the correct idle conn lists for the SSL sockets
    - BUG/MEDIUM: spoe: initialization depending on nbthread must be done last
    - BUG/MEDIUM: server: initialize the idle conns list after parsing the config
    - BUG/MEDIUM: server: initialize the orphaned conns lists and tasks at the end
    - MINOR: config: make MAX_PROCS configurable at build time
    - BUG/MAJOR: spoe: Don't try to get agent config during SPOP healthcheck
    - BUG/MINOR: config: Reinforce validity check when a process number is parsed
    - BUG/MEDIUM: peers: check that p->srv actually exists before using p->srv->use_ssl
    - CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy
    - BUG/MINOR: mux-h1: verify the request's version before dropping connection: keep-alive
    - BUG: 51d: In Hash Trie, multi header matching was affected by the header names stored globaly.
    - MEDIUM: 51d: Enabled multi threaded operation in the 51Degrees module.
    - BUG/MAJOR: stream: avoid double free on unique_id
    - BUILD/MINOR: stream: avoid a build warning with threads disabled
    - BUILD/MINOR: tools: fix build warning in the date conversion functions
    - BUILD/MINOR: peers: remove an impossible null test in intencode()
    - BUILD/MINOR: htx: fix some potential null-deref warnings with http_find_stline
    - BUG/MEDIUM: peers: Missing peer initializations.
    - BUG/MEDIUM: http_fetch: fix the "base" and "base32" fetch methods in HTX mode
    - BUG/MEDIUM: proto_htx: Fix data size update if end of the cookie is removed
    - BUG/MEDIUM: http_fetch: fix "req.body_len" and "req.body_size" fetch methods in HTX mode
    - BUILD/MEDIUM: initcall: Fix build on MacOS.
    - BUG/MEDIUM: mux-h2/htx: Always set CS flags before exiting h2_rcv_buf()
    - MINOR: h2/htx: Set the flag HTX_SL_F_BODYLESS for messages without body
    - BUG/MINOR: mux-h1: Add "transfer-encoding" header on outgoing requests if needed
    - BUG/MINOR: mux-h2: Don't add ":status" pseudo-header on trailers
    - BUG/MINOR: proto-htx: Consider a XFER_LEN message as chunked by default
    - BUG/MEDIUM: h2/htx: Correctly handle interim responses when HTX is enabled
    - MINOR: mux-h2: Set HTX extra value when possible
    - BUG/MEDIUM: htx: count the amount of copied data towards the final count
    - MINOR: mux-h2: make the H2 MAX_FRAME_SIZE setting configurable
    - BUG/MEDIUM: mux-h2/htx: send an empty DATA frame on empty HTX trailers
    - BUG/MEDIUM: servers: Use atomic operations when handling curr_idle_conns.
    - BUG/MEDIUM: servers: Add a per-thread counter of idle connections.
    - MINOR: fd: add a new my_closefrom() function to close all FDs
    - MINOR: checks: use my_closefrom() to close all FDs
    - MINOR: fd: implement an optimised my_closefrom() function
    - BUG/MINOR: fd: make sure my_closefrom() doesn't miss some FDs
    - BUG/MAJOR: fd/threads, task/threads: ensure all spin locks are unlocked
    - BUG/MAJOR: listener: Make sure the listener exist before using it.
    - MINOR: fd: Use closefrom() as my_closefrom() if supported.
    - BUG/MEDIUM: mux-h1: Report the right amount of data xferred in h1_rcv_buf()
    - BUG/MINOR: channel: Set CF_WROTE_DATA when outgoing data are skipped
    - MINOR: htx: Add function to drain data from an HTX message
    - MINOR: channel/htx: Add function to skips output bytes from an HTX channel
    - BUG/MAJOR: cache/htx: Set the start-line offset when a cached object is served
    - BUG/MEDIUM: cache: Get objects from the cache only for GET and HEAD requests
    - BUG/MINOR: cache/htx: Return only the headers of cached objects to HEAD requests
    - BUG/MINOR: mux-h1: Always initilize h1m variable in h1_process_input()
    - BUG/MEDIUM: proto_htx: Fix functions applying regex filters on HTX messages
    - BUG/MEDIUM: h2: advertise to servers that we don't support push
    - MINOR: standard: Add a function to parse uints (dotted notation).
    - MINOR: arg: Add support for ARGT_PBUF_FNUM arg type.
    - MINOR: http_fetch: add "req.ungrpc" sample fetch for gRPC.
    - MINOR: sample: Add two sample converters for protocol buffers.
    - DOC: sample: Add gRPC related documentation.
 2019-02-26 16:43:49 +01:00
Frédéric Lécaille
ec891195d9 DOC: sample: Add gRPC related documentation. 
Add documentation for "req.ungrpc" sample fetch method and "varint"
and "svarint" two new protocol buffers specific converters.
 2019-02-26 16:27:05 +01:00
Willy Tarreau
a24b35ca18 MINOR: mux-h2: make the H2 MAX_FRAME_SIZE setting configurable 
This creates a new tunable "tune.h2.max-frame-size" to adjust the
advertised max frame size. When not set it still defaults to the buffer
size. It is convenient to advertise sizes lower than the buffer size,
for example when using very large buffers.
 2019-02-21 17:30:59 +01:00
Bertrand Jacquin
4f03ab06a9 DOC: ssl: Stop documenting ciphers example to use 
Since TLS ciphers are not well understand, it is very common pratice to
copy and paste parameters from documentation and use them as-is. Since RC4
should not be used anymore, it is wiser to link users to up to date
documnetation from Mozilla to avoid unsafe configuration in the wild.

Clarify the location of man pages for OpenSSL when missing.
 2019-02-06 23:03:40 +01:00
Bertrand Jacquin
8cf7c1eb61 DOC: ssl: Clarify when pre TLSv1.3 cipher can be used 
This is mainly driven by the fact TLSv1.3 will have a successor at some
point.
 2019-02-06 23:03:38 +01:00
Olivier Houchard
dc21ff778b MINOR: debug: Add an option that causes random allocation failures. 
When compiling with DEBUG_FAIL_ALLOC, add a new option, tune.fail-alloc,
that gives the percentage of chances an allocation fails.
This is useful to check that allocation failures are always handled
gracefully.
 2019-01-31 19:38:25 +01:00
Tim Duesterhus
bb48c9a0ee DOC: compression: Update the reasons for disabled compression 
- Update the list of status codes to include 201 - 203.
- Remove the fact about the temporary workaround for chunked responses
  (this is verified using reg-test compression/h00000.vtc).
- Add malformed ETags

see b229f018eedef4d18571ce6da23d8e153249a836

This commit should be backported together with b229f018eedef4d18571ce6da23d8e153249a836
the changes should be correct until 1.7 at the very least, possibly older.
 2019-01-31 04:33:07 +01:00
Tim Duesterhus
b229f018ee BUG/MEDIUM: compression: Rewrite strong ETags 
RFC 7232 section 2.3.3 states:

> Note: Content codings are a property of the representation data,
> so a strong entity-tag for a content-encoded representation has to
> be distinct from the entity tag of an unencoded representation to
> prevent potential conflicts during cache updates and range
> requests.  In contrast, transfer codings (Section 4 of [RFC7230])
> apply only during message transfer and do not result in distinct
> entity-tags.

Thus a strong ETag must be changed when compressing. Usually this is done
by converting it into a weak ETag, which represents a semantically, but not
byte-by-byte identical response. A conversion to a weak ETag still allows
If-None-Match to work.

This should be backported to 1.9 and might be backported to every supported
branch with compression.
 2019-01-29 20:26:06 +01:00
Willy Tarreau
1f672a8162 DOC: nbthread is no longer experimental. 
It was mentioned when releasing 1.8 but early bugs have long been
addressed and this comment discourages some users from using threads.

This should be backported to 1.9 and 1.8 now.
 2019-01-26 14:22:17 +01:00
Willy Tarreau
9c538e01c2 MINOR: server: add a max-reuse parameter 
Some servers may wish to limit the total number of requests they execute
over a connection because some of their components might leak resources.
In HTTP/1 it was easy, they just had to emit a "connection: close" header
field with the last response. In HTTP/2, it's less easy because the info
is not always shared with the component dealing with the H2 protocol and
it could be harder to advertise a GOAWAY with a stream limit.

This patch provides a solution to this by adding a new "max-reuse" parameter
to the server keyword. This parameter indicates how many times an idle
connection may be reused for new requests. The information is made available
and the underlying muxes will be able to use it at will.

This patch should be backported to 1.9.
 2019-01-24 19:06:43 +01:00
Willy Tarreau
64ded3db2c DOC: mention the effect of nf_conntrack_tcp_loose on src/dst 
On rare occasions the logs may report inverted src/dst when using
conntrack with this sysctl. Add a mention for it in the doc. More
info here :

     https://www.spinics.net/lists/netdev/msg544878.html
 2019-01-23 10:02:15 +01:00
Frédéric Lécaille
2f167b3543 DOC: peers: SSL/TLS documentation for "peers" 2019-01-18 14:26:21 +01:00
Jérôme Magnin
35e53a6a08 DOC: add a missing space in the documentation for bc_http_major 
add a missing space between sample fetch name and colon, and make
haproxy-dconv happier.
 2019-01-17 10:28:58 +01:00
Willy Tarreau
21c741a665 MINOR: backend: make the random algorithm support a number of draws 
When an argument <draws> is present, it must be an integer value one
or greater, indicating the number of draws before selecting the least
loaded of these servers. It was indeed demonstrated that picking the
least loaded of two servers is enough to significantly improve the
fairness of the algorithm, by always avoiding to pick the most loaded
server within a farm and getting rid of any bias that could be induced
by the unfair distribution of the consistent list. Higher values N will
take away N-1 of the highest loaded servers at the expense of performance.
With very high values, the algorithm will converge towards the leastconn's
result but much slower. The default value is 2, which generally shows very
good distribution and performance. This algorithm is also known as the
Power of Two Random Choices and is described here :

http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf
 2019-01-14 19:33:17 +01:00
Emeric Brun
9e7547740c MINOR: ssl: add support of aes256 bits ticket keys on file and cli. 
Openssl switched from aes128 to aes256 since may 2016  to compute
tls ticket secrets used by default. But Haproxy still handled only
128 bits keys for both tls key file and CLI.

This patch permit the user to set aes256 keys throught CLI or
the key file (80 bytes encoded in base64) in the same way that
aes128 keys were handled (48 bytes encoded in base64):
- first 16 bytes for the key name
- next 16/32 bytes for aes 128/256 key bits key
- last 16/32 bytes for hmac 128/256 bits

Both sizes are now supported (but keys from same file must be
of the same size and can but updated via CLI only using a key of
the same size).

Note: This feature need the fix "dec func ignores padding for output
size checking."
 2019-01-14 19:32:58 +01:00
Olivier Houchard
6975296494 DOC: Be a bit more explicit about allow-0rtt security implications. 
Document a bit better than allow-0rtt can trivially be used for replay attacks,
and so should only be used when it's safe to replay a request.

This should probably be backported to 1.8 and 1.9.
 2019-01-09 16:26:33 +01:00
Jarno Huuskonen
251a6b72a8 DOC: http-request cache-use / http-response cache-store expects cache name 
Adds missing cache name option to http-request cache-use and
http-response cache-store documentation.

Also adds optional if/unless condition to
10.2.2. Proxy section: http-request cache-use / http-response cache-store
 2019-01-08 14:02:39 +01:00
Jarno Huuskonen
e504f819d5 DOC: Fix typo in req.ssl_alpn example (commit 4afdd138424ab...) 
Also link to ssl_fc_alpn.
 2019-01-08 14:02:26 +01:00
Willy Tarreau
59884a646c MINOR: lb: allow redispatch when using consistent hash 
Redispatch traditionally only worked for cookie based persistence.

Adding redispatch support for consistent hash based persistence - also
update docs.

Reported by Oskar Stenman on discourse:
https://discourse.haproxy.org/t/balance-uri-consistent-hashing-redispatch-3-not-redispatching/3344

Should be backported to 1.8.

Cc: Lukas Tribus <lukas@ltri.eu>
 2019-01-02 20:22:17 +01:00
Alex Zorin
4afdd13842 MINOR: payload: add sample fetch for TLS ALPN 
Application-Layer Protocol Negotiation (ALPN, RFC7301) is a TLS
extension which allows a client to present a preference for which
protocols it wishes to connect to, when a single port supports multiple
multiple application protocols.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.

The new fetch "req.ssl_alpn" extracts the ALPN protocol names that may
be present in the ClientHello message.
 2019-01-01 09:15:01 +01:00
Willy Tarreau
fba74ea7b0 [RELEASE] Released version 2.0-dev0 
Released version 2.0-dev0 with the following main changes :
    - BUG/MAJOR: connections: Close the connection before freeing it.
    - REGTEST: Require the option LUA to run lua tests
    - REGTEST: script: Process script arguments before everything else
    - REGTEST: script: Evaluate the varnishtest command to allow quoted parameters
    - REGTEST: script: Add the option --clean to remove previous log direcotries
    - REGTEST: script: Add the option --debug to show logs on standard ouput
    - REGTEST: script: Add the option --keep-logs to keep all log directories
    - REGTEST: script: Add the option --use-htx to enable the HTX in regtests
    - REGTEST: script: Print only errors in the results report
    - REGTEST: Add option to use HTX prefixed by the macro 'no-htx'
    - REGTEST: Make reg-tests target support argument.
    - REGTEST: Fix a typo about barrier type.
    - REGTEST: Be less Linux specific with a syslog regex.
    - REGTEST: Missing enclosing quotes for ${tmpdir} macro.
    - REGTEST: Exclude freebsd target for some reg tests.
    - BUG/MEDIUM: h2: Don't forget to quit the sending_list if SUB_CALL_UNSUBSCRIBE.
    - BUG/MEDIUM: mux-h2: Don't forget to quit the send list on error reports
    - BUG/MEDIUM: dns: Don't prevent reading the last byte of the payload in dns_validate_response()
    - BUG/MEDIUM: dns: overflowed dns name start position causing invalid dns error
    - BUG/MINOR: compression/htx: Don't compress responses with unknown body length
    - BUG/MINOR: compression/htx: Don't add the last block of data if it is empty
    - MEDIUM: mux_h1: Implement h1_show_fd.
    - REGTEST: script: Add support of alternatives in requited options list
    - REGTEST: Add a basic test for the compression
    - BUG/MEDIUM: mux-h2: don't needlessly wake up the demux on short frames
    - REGTEST: A basic test for "http-buffer-request"
    - BUG/MEDIUM: server: Also copy "check-sni" for server templates.
    - MINOR: ssl: Add ssl_sock_set_alpn().
    - MEDIUM: checks: Add check-alpn.
 2018-12-22 11:20:35 +01:00
Olivier Houchard
921501443b MEDIUM: checks: Add check-alpn. 
Add a way to configure the ALPN used by check, with a new "check-alpn"
keyword. By default, the checks will use the server ALPN, but it may not
be convenient, for instance because the server may use HTTP/2, while checks
are unable to do HTTP/2 yet.
 2018-12-21 19:54:16 +01:00
Willy Tarreau
822305067b [RELEASE] Released version 1.9.0 
Released version 1.9.0 with the following main changes :
    - BUG/MEDIUM: compression: Use the right buffer pointers to compress input data
    - BUG/MINOR: mux_pt: Set CS_FL_WANT_ROOM when count is zero in rcv_buf() callback
    - BUG/MEDIUM: connection: Add a new CS_FL_ERR_PENDING flag to conn_streams.
    - CONTRIB: debug: teach the "flags" utility about new conn_stream flags
    - BUG/MEDIUM: stream-int: always clear CS_FL_WANT_ROOM before receiving
    - BUG/MEDIUM: mux-h2: also restart demuxing when data are pending in demux
    - BUG/MEDIUM: mux-h2: restart demuxing as soon as demux data are available
    - BUG/MEDIUM: h2: fix aggregated cookie length computation in HTX mode
    - MINOR: mux-h2: report more h2c, last h2s and cs information on "show fd"
    - CONTRIB: debug: report stream-int's flag SI_FL_CLEAN_ABRT
    - MINOR: cli/stream: add the conn_stream in "show sess" output
    - BUG/MINOR: mux-h2: don't report a fantom h2s in "show fd"
    - BUG/MINOR: cli/fd: don't isolate the thread for each individual fd
    - MINOR: objtype: report a few missing types in names and base pointers
    - BUG/MEDIUM: mux-h2: make sure to report synchronous errors after EOS
    - BUG/MEDIUM: mux-h2: report asynchronous errors in h2_wake_some_streams()
    - BUG/MEDIUM: mux-h2: make sure the demux also wakes streams up on errors
    - BUG/MINOR: mux-h1: report the correct frontend in error captures
    - BUG/MEDIUM: stream-int: also wake the stream up on end of transfer
    - MEDIUM: h2: properly check and deduplicate the content-length header in HTX
    - BUG/MEDIUM: stream: Forward the right amount of data before infinite forwarding
    - BUG/MINOR: proto_htx: Call the HTX version of the function managing client cookies
    - BUG/MEDIUM: lua/htx: Handle EOM in receive/get_line calls in HTTP applets
    - BUG/MINOR: lua: Return an error if a legacy HTTP applet doesn't send anything
    - MINOR: compression: Remove the thread_local variable buf_output
    - CLEANUP: connection: rename subscription events values and event field
    - CLEANUP: connection: rename conn->mux_ctx to conn->ctx
    - MINOR: connection: remove an unwelcome dependency on struct stream
    - CLEANUP: stream-int: consistently call the si/stream_int functions
    - BUG/MEDIUM: h1: Don't shutw/shutr the connection if we have keepalive.
    - BUG/MEDIUM: H2: Make sure htx is set even on empty frames.
    - BUG/MEDIUM: mux-h2: pass CS_FL_ERR_PENDING to h2_wake_some_streams()
    - MEDIUM: stream-int: always consider all CS errors on the send side
    - BUG/MEDIUM: h2: Make sure we don't set CS_FL_ERROR if there's still data.
    - CLEANUP: mux-h2: implement h2s_notify_{send,recv} to report events to subscribers
    - MINOR: mux-h2: add a new function h2s_alert() to call the data layer
    - BUG/MEDIUM: mux-h2: make use of h2s_alert() to report aborts
    - MINOR: connection: add cs_set_error() to set the error bits
    - CLEANUP: mux-h2: make use of cs_set_error()
    - BUG/MINOR: mux-h2: make sure we check the conn_stream in early data
    - BUG/MEDIUM: h2: Don't wait for flow control if the connection had a shutr.
    - MINOR: cli/show_fd: report that a connection is back or not
    - SCRIPTS: add the slack channel URL to the announce script
    - CLEANUP: remove my name and address from the copyright banner
    - DOC: mention in the readme that 1.9 is a stable version now
 2018-12-19 19:13:17 +01:00
Willy Tarreau
2a7d6502bf [RELEASE] Released version 1.9-dev11 
Released version 1.9-dev11 with the following main changes :
    - BUG/MEDIUM: connection: Don't use the provided conn_stream if it was tried.
    - REGTEST/MINOR: remove double body specification for server txresp
    - BUG/MEDIUM: connections: Remove error flags when retrying.
    - REGTEST/MINOR: skip seamless-reload test with abns socket on freebsd
    - REGTEST/MINOR: remove health-check that can make the test fail
    - DOC: clarify that check-sni needs an argument.
    - DOC: refer to check-sni in the documentation of sni
    - BUG/MEDIUM: mux-h2: fix encoding of non-GET/POST methods
    - BUG/MINOR: mux-h1: Fix conn_mode processing for headerless outgoing messages
    - BUG/MEDIUM: mux-h1: Add a BUSY mode to not loop on pipelinned requests
    - BUG/MEDIUM: mux-h1: Don't loop on the headers parsing if the read0 was received
    - BUG/MEDIUM: htx: Always do a defrag if a block value is replace by a bigger one
    - BUG/MEDIUM: mux-h2: Don't forget to set the CS_FL_EOS flag with htx.
    - BUG/MINOR: hpack: fix off-by-one in header name encoding length calculation
    - CLEANUP: hpack: no need to include chunk.h, only include buf.h
    - MINOR: hpack: simplify the len to bytes conversion
    - MINOR: hpack: use ist2bin() to copy header names in hpack_encode_header()
    - MINOR: hpack: optimize header encoding for short names
    - CONTRIB: hpack: add a compressed stream generator for the encoder
    - MEDIUM: hpack: make it possible to encode any static header name
    - MINOR: hpack: move the length computation and encoding functions to .h
    - MINOR: hpack: provide a function to encode a short indexed header
    - MINOR: hpack: provide a function to encode a long indexed header
    - MINOR: hpack: provide new functions to encode the ":status" header
    - MEDIUM: mux-h2: make use of standard HPACK encoding functions for the status
    - MINOR: hpack: provide a function to encode an HTTP method
    - MEDIUM: mux-h2: make use of hpack_encode_method() to encode the method
    - MINOR: hpack: provide a function to encode an HTTP scheme
    - MEDIUM: mux-h2: make use of hpack_encode_scheme() to encode the scheme
    - MINOR: hpack: provide a function to encode an HTTP path
    - MEDIUM: mux-h2: make use of hpack_encode_path() to encode the path
    - REGTEST: add the HTTP rules test involving HTX processing
    - REORG: connection: centralize the conn_set_{tos,mark,quickack} functions
    - MEDIUM: cli: rework the CLI proxy parser
    - MINOR: cli: parse prompt command in the CLI proxy
    - MINOR: cli: implements 'quit' in the CLI proxy
    - BUG/MINOR: cli: wait for payload data even without prompt
    - MEDIUM: cli: handle payload in CLI proxy
    - MINOR: cli: use pcli_flags for prompt activation
    - MINOR: compression: Rename the function check_legacy_http_comp_flt()
    - MINOR: cache/htx: Don't use the same cache on HTX and legacy HTTP proxies
    - MINOR: cache: Register the cache as a data filter only if response is cacheable
    - MEDIUM: cache/htx: Add the HTX support into the cache
    - MINOR: cache: Improve and simplify the cache configuration check
    - MINOR: filters: Export the name of known filters
    - MEDIUM: cache/compression: Add a way to safely combined compression and cache
    - MEDIUM: cache: Require an explicit filter declaration if other filters are used
    - REORG: htx: merge types+proto into common/htx.h
    - REORG: http: create http_msg.c to place there some legacy HTTP parts
    - REORG: h1: move legacy http functions to http_msg.c
    - REORG: h1: move the h1_state definition to proto_http
    - CLEANUP: h1: remove some occurrences of unneeded h1.h inclusions
    - REORG: h1: merge types+proto into common/h1.h
    - CLEANUP: stream: remove SF_TUNNEL, SF_INITIALIZED, SF_CONN_TAR
    - MEDIUM: mux-h1: implement true zero-copy of DATA blocks
    - MINOR: config: round up global.tune.bufsize to the next multiple of 2 void*
    - BUG/MINOR: mux-h2: refrain from muxing during the preface
    - BUG/MINOR: mux-h2: advertise a larger connection window size
    - DOC: master CLI documentation in management.txt
    - MINOR: mux-h2: avoid copying large blocks into full buffers
    - MEDIUM: mux-h2: implement true zero-copy send of large HTX DATA blocks
    - MINOR: mux-h2: force reads to be HTX-aligned in HTX mode
    - MINOR: cli: change 'show proc' output of old processes
    - BUG/MEDIUM: mux-h1: Fix the zero-copy on output for chunked messages
    - BUG: dns: Prevent stack-exhaustion via recursion loop in dns_read_name
    - BUG: dns: Prevent out-of-bounds read in dns_read_name()
    - BUG: dns: Prevent out-of-bounds read in dns_validate_dns_response()
    - BUG: dns: Fix out-of-bounds read via signedness error in dns_validate_dns_response()
    - BUG: dns: Fix off-by-one write in dns_validate_dns_response()
    - REGTEST: the cache regtest requires haproxy 1.9
    - MEDIUM: cli: store CLI level in the appctx
    - MEDIUM: cli: show and change CLI permissions
    - CLEANUP: cli: use dedicated define instead of appctx ones
    - MEDIUM: cli: handle CLI level from the master CLI
    - BUG/MEDIUM: cli: handle correctly prefix and payload
    - BUILD: Makefile: Implements the help target
    - REGTESTS: adjust the http-rules regtest to support window updates
    - BUG/MEDIUM: connections: Remove CS_FL_EOS | CS_FL_REOS on retry.
    - BUG/MEDIUM: stream_interface: Don't report read0 if we were not connected.
    - BUG/MEDIUM: connection: Just make sure we closed the fd on connection failure.
    - MEDIUM: mux: Add an optional "reset" method.
    - BUG/MEDIUM: mux-h1: Fix loop if server closes its connection with unparsed data
    - MINOR: mux-h1: Add helper functions to wake a stream from recv or send
    - BUG/MEDIUM: mux-h1: Wake the stream for send once the connection is established
    - BUG/MEDIUM: connections: Don't attempt to reuse an unusable connection.
    - MEDIUM: htx: Try to take a connection over if it has no owner.
    - REGTEST: Reg testing improvements.
    - REGTEST: Add a first test for health-checks.
    - REGTEST: Reg test for "check" health-check option.
    - REGTEST: level 1 health-check test 2.
    - REGTEST: Add miscellaneous reg tests for health-checks.
    - REGTEST: add a few HTTP messaging tests
    - MINOR: lb: make the leastconn algorithm more accurate
    - REGTEST: fix missing space in checks/s00001
    - REGTEST: http-messaging: add "option http-buffer-request" for H2 tests
    - BUG/MEDIUM: cache: fix random crash on filter parser's error path
    - MINOR: connection: realign empty buffers in muxes, not transport layers
    - MINOR: mux_h1/h2: simplify the zero-copy Rx alignment
    - MINOR: backend: count the number of connect and reuse per server and per backend
    - BUG/MINOR: stats: fix inversion of failed header rewrites and other statuses
    - MINOR: tools: increase the number of ITOA strings to 16
    - MINOR: cache: report the number of cache lookups and cache hits
    - MEDIUM: tasks: check the global task mask instead of the thread number
    - MINOR: mworker: set all_threads_mask and pid_bit to 1
    - BUG/MINOR: proto_htx: Fix htx_res_set_status to also set the reason
    - BUG/MINOR: stats: Parse post data for HTX streams
    - MINOR: payload/htx: Adapt smp_fetch_len to be HTX aware
    - MINOR: http_fecth: Implement body_len and body_size sample fetches for the HTX
    - MAJOR: lua: Forbid calls to Channel functions for LUA scripts in HTTP proxies
    - MEDIUM: lua/htx: Adapt functions of the HTTP to be compatible with HTX
    - MINOR: lua/htx: Adapt the functions get_in_length and is_full to be HTX aware
    - MAJOR: lua/htx: Adapt HTTP applets to support HTX messages
    - MINOR: lua: Remove useless check on the messages state in HTTP functions
    - BUG/MEDIUM: htx: When performing zero-copy, start from the right offset.
    - BUG/MINOR: mworker: don't use unitialized mworker_proc struct
    - MINOR: mworker/cli: indicate in the master prompt when a reload failed
    - MINOR: cli: implements 'reload' on master CLI
    - BUG/MEDIUM: log: Don't call sample_fetch_as_type if we don't have a stream.
    - BUG/MEDIUM: mux-h1: make sure we always have at least one HTX block to send
    - BUG/MAJOR: backend: only update server's counters when the server exists
    - MINOR: tools: preset the port of fd-based "sockets" to zero
    - BUG/MINOR: log: fix logging to both FD and IP
    - REGTEST: Add a reg test for HTTP cookies.
    - BUILD: ssl: Fix compilation without deprecated OpenSSL 1.1 APIs
    - BUILD: thread: properly report multi-thread support
    - BUG/MINOR: logs: leave startup-logs global and not per-thread
    - BUG/MEDIUM: threads: don't close the thread waker pipe if not init
    - BUG/MAJOR: compression/cache: Make it really works with these both filters
    - BUG/MEDIUM: h2: Don't forget to destroy the h2s after deferred shut.
    - MEDIUM: proxy: Set http-reuse safe as default.
    - MEDIUM: servers: Add a command to limit the number of idling connections.
    - MEDIUM: servers: Replace idle-timeout with pool-purge-delay.
    - MEDIUM: mux: Destroy the stream before trying to add the conn to the idle list.
    - MEDIUM: mux: provide the session to the init() and attach() method.
    - MEDIUM: sessions: Don't keep an infinite number of idling connections.
    - MEDIUM: servers: Be more agressive when adding H2 connection to idle lists.
    - MEDIUM: mux_h2: Always set CS_FL_NOT_FIRST for new conn_streams.
    - BUG/MEDIUM: htx/cache: use the correct class of error codes on abort
    - BUG/MINOR: cache: also consider CF_SHUTR to abort delivery
    - MINOR: pools: Cast to volatile int * instead of int *.
    - MINOR: debug: make the ABORT_NOW macro use a volatile int
    - BUG/MEDIUM: h2: Don't destroy the h2s if it still has a cs attached.
    - BUG/MEDIUM: mux-h1: don't try to process an empty input buffer
    - DOC: clarify the agent-check status line syntax
    - BUG/MAJOR: hpack: fix length check for short names encoding
    - DOC: split the README into README + INSTALL
 2018-12-16 22:35:06 +01:00
Willy Tarreau
7a0139e2b6 DOC: clarify the agent-check status line syntax 
Nick Ramirez reported that the wording is confusing and lets one think
that the CR or LF are both optional, which is not the case (either is
optional). Let's reformulate this.
 2018-12-16 08:42:56 +01:00
Olivier Houchard
a4d4fdfaa3 MEDIUM: sessions: Don't keep an infinite number of idling connections. 
In session, don't keep an infinite number of connection that can idle.
Add a new frontend parameter, "max-session-srv-conns" to set a max number,
with a default value of 5.
 2018-12-15 23:50:10 +01:00
Olivier Houchard
b7b3faa79c MEDIUM: servers: Replace idle-timeout with pool-purge-delay. 
Instead of the old "idle-timeout" mechanism, add a new option,
"pool-purge-delay", that sets the delay before purging idle connections.
Each time the delay happens, we destroy half of the idle connections.
 2018-12-15 23:50:09 +01:00
Olivier Houchard
006e3101f9 MEDIUM: servers: Add a command to limit the number of idling connections. 
Add a new command, "pool-max-conn" that sets the maximum number of connections
waiting in the orphan idling connections list (as activated with idle-timeout).
Using "-1" means unlimited. Using pools is now dependant on this.
 2018-12-15 23:50:08 +01:00
Olivier Houchard
86006a561e MEDIUM: proxy: Set http-reuse safe as default. 
Change the default for http-reuse from "never" to "safe", as it has been
the recommended setting for a few versions now and backend H2 makes little
sense without it.

Some warnings were removed from the config parser since it can dynamically
be disabled depending on the server's configuration, so there's no need to
disable it on a whole backend just for one server.
 2018-12-15 23:50:08 +01:00
Christopher Faulet
27d93c3f94 BUG/MAJOR: compression/cache: Make it really works with these both filters 
Caching the response with the compression enabled was totally broken. To fix the
problem, the compression must be done after caching the response. Otherwise it
needs to change the cache to store compressed and uncompressed objects for the
same ressource. So, because it is not possible for now, it is forbidden to
declare the compression filter before the cache one. To ease the configuration,
both can be implicitly declared (without "filter" keyword). The compression will
automatically be inserted after the cache.

Then, to make it works this way, the compression filter has been slighly
modified. Now, the response headers are updated after http-response rules
evaluations, instead of before. So, if the response contains a "Content-length"
header, it will be kept with the response stored in the cache. So this cached
response will be able to be served to clients not supporting the compression at
all.
 2018-12-15 23:50:07 +01:00
Willy Tarreau
c77d364905 MINOR: config: round up global.tune.bufsize to the next multiple of 2 void* 
Since HTX casts the buffer to a struct and stores relative pointers at the
end, it is mandatory that its end is properly aligned. This patch enforces
a buffer size rounding up to the next multiple of two void*, thus 8 on
32-bit and 16 on 64-bit, to match what malloc() already does on the beginning
of the buffer. In pratice it will never be really noticeable since default
sizes already are such multiples.
 2018-12-12 06:19:42 +01:00
Christopher Faulet
99a17a2d91 MEDIUM: cache: Require an explicit filter declaration if other filters are used 
As for the compression filter, the cache filter must be explicitly declared
(using the filter keyword) if other filters than cache are used. It is mandatory
to explicitly define the filters order.

Documentation has been updated accordingly.
 2018-12-11 17:09:31 +01:00
Jérôme Magnin
b36a6d21db DOC: refer to check-sni in the documentation of sni 
Make it obvious in the description of the sni directive that it can
not be used for health checks, and refer to the appropriate directive.

This can be backported to 1.8 as check-sni appeared in 1.8.
 2018-12-10 05:21:02 +01:00
Jérôme Magnin
ae9bb76001 DOC: clarify that check-sni needs an argument. 
Make it more obvious that check-sni requires an argument, and that
it can only be a string. Also refer to sni for proxied traffic.

This can be backported to 1.8 as check-sni appeared in 1.8.
 2018-12-10 05:20:00 +01:00
Willy Tarreau
72e9227385 [RELEASE] Released version 1.9-dev10 
Released version 1.9-dev10 with the following main changes :
    - MINOR: htx: Rename functions htx_*_to_str() to be H1 specific
    - BUG/MINOR: htx: Force HTTP/1.1 on H1 formatting when version is 1.1 or above
    - BUG/MINOR: fix ssl_fc_alpn and actually add ssl_bc_alpn
    - BUG/MEDIUM: mworker: stop proxies which have no listener in the master
    - BUG/MEDIUM: h1: Destroy a connection after detach if it has no owner.
    - BUG/MEDIUM: h2: Don't forget to wake the tasklet after shutr/shutw.
    - BUG/MINOR: flt_trace/compression: Use the right flag to add the HTX support
    - BUG/MEDIUM: stream_interface: Make REALLY sure we read all the data.
    - MEDIUM: mux-h1: Revamp the way subscriptions are handled.
    - BUG/MEDIUM: mux-h1: Always set CS_FL_RCV_MORE when data are received in h1_recv()
    - MINOR: mux-h1: Make sure to return 1 in h1_recv() when needed
    - BUG/MEDIUM: mux-h1: Release the mux H1 in h1_process() if there is no h1s
    - BUG/MINOR: proto_htx: Truncate the request when an error is detected
    - BUG/MEDIUM: h2: When sending in HTX, make sure the caller knows we sent all.
    - BUG/MEDIUM: mux-h2: properly update the window size in HTX mode
    - BUG/MEDIUM: mux-h2: make sure to always report HTX EOM when consumed by headers
    - BUG/MEDIUM: mux-h2: stop sending HTX once the mux is blocked
    - BUG/MEDIUM: mux-h2: don't send more HTX data than requested
    - MINOR: mux-h2: stop on non-DATA and non-EOM HTX blocks
    - BUG/MEDIUM: h1: Correctly report used data with no len.
    - MEDIUM: h1: Realign the ibuf before calling rcv_buf if needed.
    - BUG/MEDIUM: mux_pt: Always set CS_FL_RCV_MORE.
    - MINOR: htx: make htx_from_buf() adjust the size only on new buffers
    - MINOR: htx: add buf_room_for_htx_data() to help optimize buffer transfers
    - MEDIUM: mux-h1: make use of buf_room_for_htx_data() instead of b_room()
    - MEDIUM: mux-h1: attempt to zero-copy Rx DATA transfers
    - MEDIUM: mux-h1: avoid a double copy on the Tx path whenever possible
    - BUG/MEDIUM: stream-int: don't mark as blocked an empty buffer on Rx
    - BUG/MINOR: mux-h1: Check h1m flags to set the server conn_mode on request path
    - MEDIUM: htx: Rework conversion from a buffer to an htx structure
    - MEDIUM: channel/htx: Add functions for forward HTX data
    - MINOR: mux-h1: Don't adjust anymore the amount of data sent in h1_snd_buf()
    - CLEANUP: htx: Fix indentation here and there in HTX files
    - MINOR: mux-h1: Allow partial data consumption during outgoing data processing
    - BUG/MEDIUM: mux-h2: use the correct offset for the HTX start line
    - BUG/MEDIUM: mux-h2: stop sending using HTX on errors
    - MINOR: mux-h1: Drain obuf if the output is closed after sending data
    - BUG/MEDIUM: mworker: stop every tasks in the master
    - BUG/MEDIUM: htx: Set the right start-line offset after a defrag
    - BUG/MEDIUM: stream: Don't dereference s->txn when it is not there yet.
    - BUG/MEDIUM: connections: Reuse an already attached conn_stream.
    - MINOR: stream-int: add a new blocking condition on the remote connection
    - BUG/MEDIUM: stream-int: don't attempt to receive if the connection is not established
    - BUG/MEDIUM: lua: block on remote connection establishment
    - BUG/MEDIUM: mworker: fix several typos in mworker_cleantasks()
    - SCRIPTS/REGTEST: merge grep+sed into sed in run-regtests
    - BUG/MEDIUM: connections: Split CS_FL_RCV_MORE into 2 flags.
    - BUG/MEDIUM: h1: Don't free the connection if it's an outgoing connection.
    - BUG/MEDIUM: h1: Set CS_FL_REOS if we had a read0.
    - BUG/MEDIUM: mux-h1: Be sure to have a conn_stream to set CS_FL_REOS in h1_recv
    - REGTEST: Move LUA reg test 4 to level 1.
    - MINOR: ist: add functions to copy/uppercase/lowercase into a buffer or string
    - MEDIUM: ist: always turn header names to lower case
    - MINOR: h2: don't turn HTX header names to lower case anymore
    - MEDIUM: ist: use local conversion arrays to case conversion
    - MINOR: htx: switch to case sensitive search of lower case header names
    - MINOR: mux-h1: Set CS_FL_EOS when read0 is detected and no data are pending
    - BUG/MINOR: stream-int: Process read0 even if no data was received in si_cs_recv
    - REGTEST: fix the Lua test file name in test lua/h00002 :-)
    - REGTEST: add a basic test for HTTP rules manipulating headers
    - BUG/MEDIUM: sample: Don't treat SMP_T_METH as SMP_T_STR.
    - MINOR: sample: add bc_http_major
    - BUG/MEDIUM: htx: fix typo in htx_replace_stline() making it fail all the time
    - REGTEST: make the HTTP rules test compatible with HTTP/2 as well
    - BUG/MEDIUM: h2: Don't try to chunk data when using HTX.
    - MINOR: compiler: add a new macro ALREADY_CHECKED()
    - BUILD: h2: mark the start line already checked to avoid warnings
    - BUG/MINOR: mux-h1: Remove the connection header when it is useless
 2018-12-08 16:20:55 +01:00
Jérôme Magnin
8657742092 MINOR: sample: add bc_http_major 
This adds the sample fetch bc_http_major. It returns the backend connection's HTTP
version encoding, which may be 1 for HTTP/0.9 to HTTP/1.1 or 2 for HTTP/2.0. It is
based on the on-wire encoding, and not the version present in the request header.
 2018-12-07 15:34:39 +01:00
Willy Tarreau
da7e3be36f [RELEASE] Released version 1.9-dev9 
Released version 1.9-dev9 with the following main changes :
    - BUILD/MINOR: ssl: fix build with non-alpn/non-npn libssl
    - BUG/MINOR: mworker: Do not attempt to close(2) fd -1
    - BUILD: compression: fix build error with DEFAULT_MAXZLIBMEM
    - MINOR: compression: always create the compression pool
    - BUG/MEDIUM: mworker: fix FD leak upon reload
    - BUILD: htx: fix fprintf format inconsistency on 32-bit platforms
    - BUILD: buffers: buf.h requires unistd to get ssize_t on libmusl
    - MINOR: initcall: introduce a way to register init functions to call at boot
    - MINOR: init: process all initcalls in order at boot time
    - MEDIUM: init: convert all trivial registration calls to initcalls
    - MINOR: thread: provide a set of lock initialisers
    - MINOR: threads: add new macros to declare self-initializing locks
    - MEDIUM: init: use self-initializing spinlocks and rwlocks
    - MINOR: initcall: apply initcall to all register_build_opts() calls
    - MINOR: initcall: use initcalls for most post_{check,deinit} and per_thread*
    - MINOR: initcall: use initcalls for section parsers
    - MINOR: memory: add a callback function to create a pool
    - MEDIUM: init: use initcall for all fixed size pool creations
    - MEDIUM: memory: use pool_destroy_all() to destroy all pools on deinit()
    - MEDIUM: initcall: use initcalls for a few initialization functions
    - MEDIUM: memory: make the pool cache an array and not a thread_local
    - MINOR: ssl: free ctx when libssl doesn't support NPN
    - BUG/MINOR: proto_htx: only mark connections private if NTLM is detected
    - MINOR: h2: make struct h2_ops static
    - BUG/MEDIUM: mworker: avoid leak of client socket
    - REORG: mworker: declare master variable in global.h
    - BUG/MEDIUM: listeners: CLOEXEC flag is not correctly set
    - CLEANUP: http: Fix typo in init_http's comment
    - BUILD: Makefile: Disable -Wcast-function-type if it exists.
    - BUG/MEDIUM: h2: Don't bogusly error if the previous stream was closed.
    - REGTEST/MINOR: script: add run-regtests.sh script
    - REGTEST: Add a basic test for the cache.
    - BUG/MEDIUM: mux_pt: Don't forget to unsubscribe() on attach.
    - BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores session id
    - BUG/MEDIUM: connections: Wake the stream once the mux is chosen.
    - BUG/MEDIUM: connections: Don't forget to detach the connection from the SI.
    - BUG/MEDIUM: stream_interface: Don't check if the handshake is done.
    - BUG/MEDIUM: stream_interface: Make sure we read all the data available.
    - BUG/MEDIUM: h2: Call h2_process() if there's an error on the connection.
    - REGTEST: Fix several issues.
    - REGTEST: lua: check socket functionality from a lua-task
    - BUG/MEDIUM: session: Remove the session from the session_list in session_free.
    - BUG/MEDIUM: streams: Don't assume we have a CS in sess_update_st_con_tcp.
    - BUG/MEDIUM: connections: Don't assume we have a mux in connect_server().
    - BUG/MEDIUM: connections: Remove the connection from the idle list before destroy.
    - BUG/MEDIUM: session: properly clean the outgoing connection before freeing.
    - BUG/MEDIUM: mux_pt: Don't try to send if handshake is not done.
    - MEDIUM: connections: Put H2 connections in the idle list if http-reuse always.
    - MEDIUM: h2: Destroy a connection with no stream if it has no owner.
    - MAJOR: sessions: Store multiple outgoing connections in the session.
    - MEDIUM: session: Steal owner-less connections on end of transaction.
    - MEDIUM: server: Be smarter about deciding to reuse the last server.
    - BUG/MEDIUM: Special-case http_proxy when dealing with outgoing connections.
    - BUG/MINOR: cfgparse: Fix transition between 2 sections with the same name
    - BUG/MINOR: http: Use out buffer instead of trash to display error snapshot
    - BUG/MINOR: htx: Fix block size calculation when a start-line is added/replaced
    - BUG/MINOR: mux-h1: Fix processing of "Connection: " header on outgoing messages
    - BUG/MEDIUM: mux-h1: Reset the H1 parser when an outgoing message is processed
    - BUG/MINOR: proto_htx: Send outgoing data to client to start response processing
    - BUG/MINOR: htx: Stop a header or a start line lookup on the first EOH or EOM
    - BUG/MINOR: connection: report mux modes when HTX is supported
    - MINOR: htx: add a function to cut the beginning of a DATA block
    - MEDIUM: conn_stream: Add a way to get mux's info on a CS from the upper layer
    - MINOR: mux-h1: Implement get_cs_info() callback
    - MINOR: stream: Rely on CS's info if it exists and fallback on session's ones
    - MINOR: proto_htx: Use conn_stream's info to set t_idle duration when possible
    - MINOR: mux-h1: Don't rely on the stream anymore in h1_set_srv_conn_mode()
    - MINOR: mux-h1: Write last chunk and trailers if not found in the HTX message
    - MINOR: mux-h1: Be prepare to fail when EOM is added during trailers parsing
    - MINOR: mux-h1: Subscribe to send in h1_snd_buf() when not all data have been sent
    - MINOR: mux-h1: Consume channel's data in a loop in h1_snd_buf()
    - MEDIUM: mux-h1: Add keep-alive outgoing connections in connections list
    - MINOR: htx: Add function to add an HTX block just before another one
    - MINOR: htx: Add function to iterate on an HTX message using HTX blocks
    - MINOR: htx: Add a function to find the HTX block corresponding to a data offset
    - MINOR: stats: Don't add end-of-data marker and trailers in the HTX response
    - MEDIUM: htx: Change htx_sl to be a struct instead of an union
    - MINOR: htx: Add the start-line offset for the HTX message in the HTX structure
    - MEDIUM: htx: Don't rely on h1_sl anymore except during H1 header parsing
    - MINOR: proto-htx: Use the start-line flags to set the HTTP messsage ones
    - MINOR: htx: Add BODYLESS flags on the HTX start-line and the HTTP message
    - MINOR: proto_htx: Use full HTX messages to send 100-Continue responses
    - MINOR: proto_htx: Use full HTX messages to send 103-Early-Hints responses
    - MINOR: proto_htx: Use full HTX messages to send 401 and 407 responses
    - MINOR: proto_htx: Send valid HTX message when redir mode is enabled on a server
    - MINOR: proto_htx: Send valid HTX message to send 30x responses
    - MEDIUM: proto_htx: Convert all HTTP error messages into HTX
    - MINOR: mux-h1: Process conn_mode on the EOH when no connection header is found
    - MINOR: mux-h1: Change client conn_mode on an explicit close for the response
    - MINOR: mux-h1: Capture bad H1 messages
    - MAJOR: filters: Adapt filters API to be compatible with the HTX represenation
    - MEDIUM: proto_htx/filters: Add data filtering during the forwarding
    - MINOR: flt_trace: Adapt to be compatible with the HTX representation
    - MEDIUM: compression: Adapt to be compatible with the HTX representation
    - MINOR: h2: implement H2->HTX request header frame transcoding
    - MEDIUM: mux-h2: register mux for both HTTP and HTX modes
    - MEDIUM: mux-h2: make h2_rcv_buf() support HTX transfers
    - MEDIUM: mux-h2: make h2_snd_buf() HTX-aware
    - MEDIUM: mux-h2: add basic H2->HTX transcoding support for headers
    - MEDIUM: mux-h2: implement emission of H2 headers frames from HTX blocks
    - MEDIUM: mux-h2: implement the emission of DATA frames from HTX DATA blocks
    - MEDIUM: mux-h2: support passing H2 DATA frames to HTX blocks
    - BUG/MINOR: cfgparse: Fix the call to post parser of the last sections parsed
    - BUG/MEDIUM: mux-h2: don't lose the first response header in HTX mode
    - BUG/MEDIUM: mux-h2: remove the HTX EOM block on H2 response headers
    - MINOR: listener: the mux_proto entry in the bind_conf is const
    - MINOR: connection: create conn_get_best_mux_entry()
    - MINOR: server: the mux_proto entry in the server is const
    - MINOR: config: make sure to associate the proper mux to bind and servers
    - MINOR: hpack: add ":path" to the list of common header fields
    - MINOR: h2: add new functions to produce an HTX message from an H2 response
    - MINOR: mux-h2: mention that the mux is compatible with both sides
    - MINOR: mux-h2: implement an outgoing stream allocator : h2c_bck_stream_new()
    - MEDIUM: mux-h2: start to create the outgoing mux
    - MEDIUM: mux-h2: implement encoding of H2 request on the backend side
    - MEDIUM: mux-h2: make h2_frt_decode_headers() direction-agnostic
    - MEDIUM: mux-h2: make h2_process_demux() capable of processing responses as well
    - MEDIUM: mux-h2: Implement h2_attach().
    - MEDIUM: mux-h2: Don't bother flagging outgoing connections as TOOMANY.
    - REGTEST: Fix LEVEL 4 script 0 of "connection" module.
    - MINOR: connection: Fix a comment.
    - MINOR: mux: add a "max_streams" method.
    - MEDIUM: servers: Add a way to keep idle connections alive.
    - CLEANUP: fix typos in the htx subsystem
    - CLEANUP: Fix typo in the chunk headers file
    - CLEANUP: Fix typos in the h1 subsystem
    - CLEANUP: Fix typos in the h2 subsystem
    - CLEANUP: Fix a typo in the mini-clist header
    - CLEANUP: Fix a typo in the proto_htx subsystem
    - CLEANUP: Fix typos in the proto_tcp subsystem
    - CLEANUP: Fix a typo in the signal subsystem
    - CLEANUP: Fix a typo in the session subsystem
    - CLEANUP: Fix a typo in the queue subsystem
    - CLEANUP: Fix typos in the shctx subsystem
    - CLEANUP: Fix typos in the socket pair protocol subsystem
    - CLEANUP: Fix typos in the map management functions
    - CLEANUP: Fix typo in the fwrr subsystem
    - CLEANUP: Fix typos in the cli subsystem
    - CLEANUP: Fix typo in the 51d subsystem
    - CLEANUP: Fix a typo in the base64 subsystem
    - CLEANUP: Fix a typo in the connection subsystem
    - CLEANUP: Fix a typo in the protocol header file
    - CLEANUP: Fix a typo in the checks header file
    - CLEANUP: Fix typos in the file descriptor subsystem
    - CLEANUP: Fix a typo in the listener subsystem
    - BUG/MINOR: lb-map: fix unprotected update to server's score
    - BUILD: threads: fix minor build warnings when threads are disabled
 2018-12-02 19:31:37 +01:00
Olivier Houchard
0c18a6fe34 MEDIUM: servers: Add a way to keep idle connections alive. 
Add a new keyword for servers, "idle-timeout". If set, unused connections are
kept alive until the timeout happens, and will be picked for reuse if no
other connection is available.
 2018-12-02 18:16:53 +01:00
Willy Tarreau
0b936ad946 [RELEASE] Released version 1.9-dev8 
Released version 1.9-dev8 with the following main changes :
    - REORG: config: extract the global section parser into cfgparse-global
    - REORG: config: extract the proxy parser into cfgparse-listen.c
    - BUILD: update the list of supported targets and compilers in makefile and readme
    - BUILD: reorder the objects in the makefile
    - BUILD: Makefile: make "V=1" show some of the commands that are executed
    - BUILD: Makefile: add the quiet mode to a few more targets
    - BUILD: Makefile: add "$(Q)" to clean, tags and cscope targets
    - BUILD: Makefile: switch to quiet mode by default for CC/LD/AR
    - MINOR: cli: format `show proc` to be more readable
    - MINOR: cli: displays uptime in `show proc`
    - MINOR: cli: show master information in 'show proc'
    - BUG/MEDIUM: hpack: fix encoding of "accept-ranges" field
    - MAJOR: mux-h1: Remove the rxbuf and decode HTTP messages in channel's buffer
    - BUG/MINOR: mux-h1: Enable keep-alive on server side
    - BUG/MEDIUM: mux-h1: Fix freeze when the kernel splicing is used
    - BUG/MEDIUM: mux-h1: Don't set the flag CS_FL_RCV_MORE when nothing was parsed
    - BUG/MINOR: stats/htx: Remove channel's output when the request is eaten
    - BUG/MINOR: proto_htx: Fix request/response synchronisation on error
    - MINOR: stream-int: Notify caller when an error is reported after a rcv_pipe()
    - MINOR: stream-int: Notify caller when an error is reported after a rcv_buf()
    - BUG/MINOR: stream-int: Don't call snd_buf() if there are still data in the pipe
    - MINOR: stream-int: remove useless checks on CS and conn flags in si_cs_send()
    - BUG/MINOR: config: Be aware of the HTX during the check of mux protocols
    - BUG/MINOR: mux-htx: Fix bad test on h1c flags in h1_recv_allowed()
    - MEDIUM: mworker: wait mode use standard init code path
    - MINOR: log: introduce ha_notice()
    - MINOR: mworker: use ha_notice to announce a new worker
    - BUG/MEDIUM: http_fetch: Make sure name is initialized before http_find_header.
    - MINOR: cli: add mworker_accept_wrapper to 'show fd'
    - MEDIUM: signal: signal_unregister() removes every handlers
    - BUG/MEDIUM: mworker: unregister the signals of main()
    - MINOR: cli: add a few missing includes in proto/cli.h
    - REORG: time/activity: move activity measurements to activity.{c,h}
    - MINOR: activity: report the average loop time in "show activity"
    - MINOR: activity: add configuration and CLI support for "profiling.tasks"
    - MEDIUM: tasks: collect per-task CPU time and latency
    - MINOR: sample: add cpu_calls, cpu_ns_avg, cpu_ns_tot, lat_ns_avg, lat_ns_tot
    - MINOR: cli/activity: rename the stolen CPU time fields to mention milliseconds
    - BUG/MINOR: cli: Fix memory leak
    - BUG/MINOR: mworker: fix FD leak and memory leak in error path
    - MINOR: poller: move the call of tv_update_date() back to the pollers
    - MINOR: polling: add an option to support busy polling
    - MINOR: server: Add "alpn" and "npn" keywords.
    - MEDIUM: connection: Don't bother reactivating polling after connection retry.
    - MAJOR: connections: Defer mux creation for outgoing connection if alpn is set.
    - MEDIUM: ssl: Add ssl_bc_alpn and ssl_bc_npn sample fetches.
    - MINOR: servers: Free [idle|safe|priv]_conns on exit.
    - REGTEST: add the option to test only a specific set of files
    - REGTEST: add a test for connections to a "dispatch" address
    - BUG/MEDIUM: connections: Don't reset the conn flags in *connect_server().
    - MINOR: server: Only defined conn_complete_server if USE_OPENSSL is set.
    - BUG/MEDIUM: servers: Don't check if we have a conn_stream too soon.
    - BUG/MEDIUM: sessions: Set sess->origin to NULL if the origin was destroyed.
    - MEDIUM: servers: Store the connection in the SI until we have a mux.
    - BUG/MEDIUM: h2: wake the processing task up after demuxing
    - BUG/MEDIUM: h2: restart demuxing after releasing buffer space
 2018-11-25 09:16:46 +01:00
Olivier Houchard
6b77f49e78 MEDIUM: ssl: Add ssl_bc_alpn and ssl_bc_npn sample fetches. 
Add 2 new sample fetches, ssl_bc_alpn and ssl_bc_npn, that provides the
ALPN and the NPN for an outgoing connection.
 2018-11-22 19:52:44 +01:00
Olivier Houchard
c756600103 MINOR: server: Add "alpn" and "npn" keywords. 
Add new keywords to "server" lines, alpn and npn.
If set, when connecting through SSL, those alpn/npn will be negociated
during the SSL handshake.
 2018-11-22 19:50:08 +01:00
Willy Tarreau
beb859abce MINOR: polling: add an option to support busy polling 
In some situations, especially when dealing with low latency on processors
supporting a variable frequency or when running inside virtual machines,
each time the process waits for an I/O using the poller, the processor
goes back to sleep or is offered to another VM for a long time, and it
causes excessively high latencies.

A solution to this provided by this patch is to enable busy polling using
a global option. When busy polling is enabled, the pollers never sleep and
loop over themselves waiting for an I/O event to happen or for a timeout
to occur. On multi-processor machines it can significantly overheat the
processor but it usually results in much lower latencies.

A typical test consisting in injecting traffic over a single connection at
a time over the loopback shows a bump from 4640 to 8540 connections per
second on forwarded connections, indicating a latency reduction of 98
microseconds for each connection, and a bump from 12500 to 21250 for
locally terminated connections (redirects), indicating a reduction of
33 microseconds.

It is only usable with epoll and kqueue because select() and poll()'s
API is not convenient for such usages, and the level of performance they
are used in doesn't benefit from this anyway.

The option, which obviously remains disabled by default, can be turned
on using "busy-polling" in the global section, and turned off later
using "no busy-polling". Its status is reported in "show info" to help
troubleshooting suspicious CPU spikes.
 2018-11-22 19:47:30 +01:00
Willy Tarreau
70fe94419c MINOR: sample: add cpu_calls, cpu_ns_avg, cpu_ns_tot, lat_ns_avg, lat_ns_tot 
These sample fetch keywords report performance metrics about the task calling
them. They are useful to report in logs which requests consume too much CPU
time and what negative performane impact it has on other requests. Typically
logging cpu_ns_avg and lat_ns_avg will show culprits and victims.
 2018-11-22 16:07:39 +01:00
Willy Tarreau
75c62c2793 MINOR: activity: add configuration and CLI support for "profiling.tasks" 
This is a new global setting which enables or disables CPU profiling
per task. For now it only sets/resets the variable based on the global
option "profiling.tasks" and supports showing it as well as setting it
from the CLI using "show profiling" and "set profiling". The option will
be used by a future commit. It was done in a way which should ease future
addition of profiling options.
 2018-11-22 11:48:51 +01:00
Willy Tarreau
5c0e41b7cb [RELEASE] Released version 1.9-dev7 
Released version 1.9-dev7 with the following main changes :
    - BUILD: cache: fix a build warning regarding too large an integer for the age
    - CLEANUP: fix typos in the comments of the Makefile
    - CLEANUP: fix a typo in a comment for the contrib/halog subsystem
    - CLEANUP: fix typos in comments for the contrib/modsecurity subsystem
    - CLEANUP: fix typos in comments for contrib/spoa_example
    - CLEANUP: fix typos in comments for contrib/wireshark-dissectors
    - DOC: Fix typos in README and CONTRIBUTING
    - MINOR: log: slightly improve error message syntax on log failure
    - DOC: logs: the format directive was missing from the second log part
    - MINOR: log: report the number of dropped logs in the stats
    - MEDIUM: log: add support for logging to existing file descriptors
    - MEDIUM: log: support a new "short" format
    - MEDIUM: log: add a new "raw" format
    - BUG/MEDIUM: stream-int: change the way buffer room is requested by a stream-int
    - BUG/MEDIUM: stream-int: convert some co_data() checks to channel_is_empty()
    - MINOR: namespaces: don't build namespace.c if disabled
    - BUILD/MEDIUM: threads/affinity: DragonFly build fix
    - MINOR: http: Add new "early-hint" http-request action.
    - MINOR: http: Make new "early-hint" http-request action really be parsed.
    - MINOR: http: Implement "early-hint" http request rules.
    - MINOR: doc: Add information about "early-hint" http-request action.
    - DOC: early-hints: fix truncated line.
    - MINOR: mworker: only close std{in,out,err} in daemon mode
    - BUG/MEDIUM: log: don't CLOEXEC the inherited FDs
    - BUG/MEDIUM: Make sure stksess is properly aligned.
    - BUG/MEDIUM: stream-int: make failed splice_in always subscribe to recv
    - BUG/MEDIUM: stream-int: clear CO_FL_WAIT_ROOM after splicing data in
    - BUG/MINOR: stream-int: make sure not to go through the rcv_buf path after splice()
    - CONTRIB: debug: fix build related to conn_stream flags change
    - REGTEST: fix scripts 1 and 3 to accept development version
    - BUG/MINOR: http_fetch: Remove the version part when capturing the request uri
    - MINOR: http: Regroup return statements of http_req_get_intercept_rule at the end
    - MINOR: http: Regroup return statements of http_res_get_intercept_rule at the end
    - BUG/MINOR: http: Be sure to sent fully formed HTTP 103 responses
    - MEDIUM: jobs: support unstoppable jobs for soft stop
    - MEDIUM: listeners: support unstoppable listener
    - MEDIUM: cli: worker socketpair is unstoppable
    - BUG/MINOR: stream-int: set SI_FL_WANT_PUT in sess_establish()
    - MINOR: stream: move the conn_stream specific calls to the stream-int
    - BUG/MINOR: config: Copy default error messages when parsing of a backend starts
    - CLEANUP: h2: minimum documentation for recent API changes
    - MINOR: mux: implement a get_first_cs() method
    - MINOR: stream-int: make conn_si_send_proxy() use cs_get_first()
    - MINOR: stream-int: relax the forwarding rules in stream_int_notify()
    - MINOR: stream-int: expand the flags to 32-bit
    - MINOR: stream-int: rename SI_FL_WAIT_ROOM to SI_FL_RXBLK_ROOM
    - MINOR: stream-int: introduce new SI_FL_RXBLK flags
    - MINOR: stream-int: add new functions si_{rx,tx}_{blocked,endp_ready}()
    - MINOR: stream-int: replace SI_FL_WANT_PUT with !SI_FL_RX_WAIT_EP
    - MINOR: stream-int: use si_rx_blocked()/si_tx_blocked() to check readiness
    - MEDIUM: stream-int: use si_rx_buff_{rdy,blk} to report buffer readiness
    - MINOR: stream-int: replace si_{want,stop}_put() with si_rx_endp_{more,done}()
    - MEDIUM: stream-int: update the endp polling status only at the end of si_cs_recv()
    - MINOR: stream-int: make si_sync_recv() simply check ENDP before si_cs_recv()
    - MINOR: stream-int: automatically mark applets as ready if they block on the channel
    - MEDIUM: stream-int: fix the si_cant_put() calls used for end point readiness
    - MEDIUM: stream-int: fix the si_cant_put() calls used for buffer readiness
    - MEDIUM: stream-int: use si_rx_shut_blk() to indicate the SI is closed
    - MEDIUM: stream-int: unconditionally call si_chk_rcv() in update and notify
    - MEDIUM: stream-int: make use of si_rx_chan_{rdy,blk} to control the stream-int from the channel
    - MINOR: stream-int: replace si_cant_put() with si_rx_room_{blk,rdy}()
    - MEDIUM: connections: Wait until the connection is established to try to recv.
    - MEDIUM: mux: Teach the mux_pt how to deal with idle connections.
    - MINOR: mux: Add a new "avail_streams" method.
    - MINOR: mux: Add a destroy() method.
    - MINOR: sessions: Start to store the outgoing connection in sessions.
    - MAJOR: connections: Detach connections from streams.
    - MINOR: conn_stream: Add a flag to notify the mux it should flush its buffers
    - MINOR: htx: Add proto_htx.c file
    - MINOR: conn_stream: Add a flag to notify the mux it must respect the reserve
    - MINOR: http: Add standalone functions to parse a start-line or a header
    - MINOR: http: Call http_send_name_header with the stream instead of the txn
    - MINOR: conn_stream: Add a flag to notify the SI some data were received
    - MINOR: http: Add macros to check if a stream uses the HTX representation
    - MEDIUM: proto_htx: Add HTX analyzers and use it when the mux H1 is used
    - MEDIUM: mux-h1: Add dummy mux to handle HTTP/1.1 connections
    - MEDIUM: mux-h1: Add parsing of incoming and ougoing HTTP messages
    - MAJOR: mux-h1/proto_htx: Handle keep-alive connections in the mux
    - MEDIUM: mux-h1: Add support of the kernel TCP splicing to forward data
    - MEDIUM: htx: Add API to deal with the internal representation of HTTP messages
    - MINOR: http_htx: Add functions to manipulate HTX messages in http_htx.c
    - MINOR: proto_htx: Add some functions to handle HTX messages
    - MAJOR: mux-h1/proto_htx: Switch mux-h1 and HTX analyzers on the HTX representation
    - MINOR: http_htx: Add functions to replace part of the start-line
    - MINOR: http_htx: Add functions to retrieve a specific occurrence of a header
    - MINOR: proto_htx: Rewrite htx_apply_redirect_rule to handle HTX messages
    - MINOR: proto_htx: Add the internal function htx_del_hdr_value
    - MINOR: proto_htx: Add the internal function htx_fmt_res_line
    - MINOR: proto_htx: Add functions htx_transform_header and htx_transform_header_str
    - MINOR: proto_htx: Add functions htx_req_replace_stline and htx_res_set_status
    - MINOR: proto_htx: Add function to build and send HTTP 103 responses
    - MINOR: proto_htx: Add functions htx_req_get_intercept_rule and htx_res_get_intercept_rule
    - MINOR: proto_htx: Add functions to apply req* and rsp* rules on HTX messages
    - MINOR: proto_htx: Add functions to manage cookies on HTX messages
    - MINOR: proto_htx: Add functions to check the cacheability of HTX messages
    - MINOR: proto_htx: Add functions htx_send_name_header
    - MINOR: proto_htx: Add functions htx_perform_server_redirect
    - MINOR: proto_htx: Add functions to handle the stats applet
    - MEDIUM: proto_htx: Adapt htx_process_req_common to handle HTX messages
    - MEDIUM: proto_htx: Adapt htx_process_request to handle HTX messages
    - MINOR: proto_htx: Adapt htx_process_tarpit to handle HTX messages
    - MEDIUM: proto_htx: Adapt htx_wait_for_request_body to handle HTX messages
    - MEDIUM: proto_htx: Adapt htx_process_res_common to handle HTX messages
    - MINOR: http_fetch: Add smp_prefetch_htx
    - MEDIUM: http_fetch: Adapt all fetches to handle HTX messages
    - MEDIUM: mux-h1: Wait for connection establishment before consuming channel's data
    - MINOR: stats/htx: Adapt the stats applet to handle HTX messages
    - MINOR: stream: Don't reset sov value with HTX messages
    - MEDIUM: mux-h1: Handle errors and timeouts in the stream
    - MINOR: filters/htx: Forbid filters when the HTX is enabled on a proxy
    - MINOR: lua/htx: Forbid lua usage when the HTX is enabled on a proxy
    - CLEANUP: Fix some typos in the haproxy subsystem
    - CLEANUP: Fix typos in the dns subsystem
    - CLEANUP: Fix typos in the pattern subsystem
    - CLEANUP: fix 2 typos in the xxhash subsystem
    - CLEANUP: fix a few typos in the comments of the server subsystem
    - CLEANUP: fix a misspell in tests/filltab25.c
    - CLEANUP: fix a typo found in the stream subsystem
    - CLEANUP: fix typos in comments in ebtree
    - CLEANUP: fix typos in reg-tests
    - CLEANUP: fix typos in the comments of the vars subsystem
    - CLEANUP: fix typos in the hlua_fcn subsystem
    - CLEANUP: fix typos in the proto_http subsystem
    - CLEANUP: fix typos in the proxy subsystem
    - CLEANUP: fix typos in the ssl_sock subsystem
    - DOC: Fix typos in different subsections of the documentation
    - DOC: fix a few typos in the documentation
    - MINOR: Fix an error message thrown when we run out of memory
    - MINOR: Fix typos in error messages in the proxy subsystem
    - MINOR: fix typos in the examples files
    - CLEANUP: Fix a typo in the stats subsystem
    - CLEANUP: Fix typos in the acl subsystem
    - CLEANUP: Fix typos in the cache subsystem
    - CLEANUP: Fix typos in the cfgparse subsystem
    - CLEANUP: Fix typos in the filters subsystem
    - CLEANUP: Fix typos in the http subsystem
    - CLEANUP: Fix typos in the log subsystem
    - CLEANUP: Fix typos in the peers subsystem
    - CLEANUP: Fix typos in the regex subsystem
    - CLEANUP: Fix typos in the sample subsystem
    - CLEANUP: Fix typos in the spoe subsystem
    - CLEANUP: Fix typos in the standard subsystem
    - CLEANUP: Fix typos in the stick_table subsystem
    - CLEANUP: Fix typos in the task subsystem
    - MINOR: Fix typo in error message in the standard subsystem
    - CLEANUP: fix typos in the comments of hlua
    - MINOR: Fix typo in the error 500 output of hlua
    - MINOR: Fix a typo in a warning message in the spoe subsystem
 2018-11-18 22:33:00 +01:00
Joseph Herlant
71b4b150f2 DOC: fix a few typos in the documentation 
This commit deals with a few misspells in the documentation.
 2018-11-18 22:23:15 +01:00
Frédéric Lécaille
3aac1068f7 DOC: early-hints: fix truncated line. 2018-11-13 10:14:02 +01:00
Frédéric Lécaille
06f5b6435b MINOR: doc: Add information about "early-hint" http-request action. 2018-11-12 21:08:55 +01:00