From 26b7f4e979561aa1f17d6719fc1dad91e1a5d1b4 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Wed, 27 Mar 2024 16:04:05 +0300 Subject: libcurl: update to 8.7.1 --- libs/libcurl/docs/CHANGES | 13880 +++++++++++++++++++++++--------------------- libs/libcurl/docs/THANKS | 57 +- 2 files changed, 7160 insertions(+), 6777 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 8e70215477..80d1cc84b5 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,10691 +6,11019 @@ Changelog -Version 8.6.0 (31 Jan 2024) +Version 8.7.1 (27 Mar 2024) -Daniel Stenberg (31 Jan 2024) +Daniel Stenberg (27 Mar 2024) -- RELEASE-NOTES: synced +- RELEASE-PROCEDURE: remove old release dates, add new pending ones - curl 8.6.0 +Version 8.7.0 (27 Mar 2024) -- THANKS: new contributors from 8.5.0 +Daniel Stenberg (27 Mar 2024) -Jay Satiro (31 Jan 2024) +- RELEASE-NOTES: synced -- cd2nroff: use perl 'strict' and 'warnings' + curl 8.7.0 release - - Use strict and warnings pragmas. +- THANKS: new contributors from the 8.7.0 release - - If open() fails then show the reason. +- CURLOPT_POSTFIELDS.md: used for MQTT as well - - Set STDIN io layer :crlf so that input is properly read on Windows. + Closes #13189 - - When STDIN is used as input, the filename $f is now set to "STDIN". +- http: remove stale comment about rewindbeforesend - Various error messages in single() use $f for the filename and this way - it is not undefined when STDIN. + ... because that struct field exists no more. - Closes https://github.com/curl/curl/pull/12819 + Follow-up to 14bcea074a782272. -Daniel Stenberg (30 Jan 2024) + Closes #13187 -- cd2nroff: fix duplicate output issue +- DISTROS: add document with distro pointers - Assisted-by: Jay Satiro - Fixes https://github.com/curl/curl-www/issues/321 - Closes #12818 + Lots of organizations distribute curl packages to end users. This is a + collection of pointers to where to learn more about curl on and with + each distro. -- lib: error out on multissl + http3 + Assisted-by: Alan Coopersmith + Assisted-by: Andrew Kaster + Assisted-by: Andy Fiddaman + Assisted-by: Arjan van de Ven + Assisted-by: Brian Clemens + Assisted-by: chrysos349 on github + Assisted-by: Dan Fandrich + Assisted-by: Dan McDonald + Assisted-by: Gaelan Steele + Assisted-by: graywolf on github + Assisted-by: Jan Macku + Assisted-by: John Marshall + Assisted-by: Jonathan Perkin + Assisted-by: Kevin Daudt + Assisted-by: Marcus Müller + Assisted-by: Michał Górny + Assisted-by: Outvi V + Assisted-by: Ross Burton + Assisted-by: Sean Molenaar + Assisted-by: Till Wegmüller + Assisted-by: Viktor Szakats + Assisted-by: Winni Neessen - Since the QUIC/h3 code has no knowledge or handling of multissl it might - bring unintended consequences if we allow it. + Closes #13178 - configure, cmake and curl_setup.h all now reject this combination. +Fabian Keil (25 Mar 2024) - Assisted-by: Viktor Szakats - Assisted-by: Gisle Vanem - Ref: #12806 - Closes #12807 +- wolfSSL: do not call the stub function wolfSSL_BIO_set_init() -Patrick Monnerat (29 Jan 2024) + Calling the function isn't necessary and causes the build + to fail when wolfSSL has been compiled with NO_WOLFSSL_STUB: -- OS400: sync ILE/RPG binding + Making all in opts + CCLD curl + ld: error: undefined symbol: wolfSSL_BIO_set_init + >>> referenced by wolfssl.c:235 (vtls/wolfssl.c:235) + >>> libcurl_la-wolfssl.o:(wolfssl_bio_cf_create) in archiv + e ../lib/.libs/libcurl.a + cc: error: linker command failed with exit code 1 (use -v to see invocat + ion) + *** Error code 1 - Also do not force git CRLF line endings on *.cmd files for OS400. + Closes #13164 - Closes #12815 +Daniel Stenberg (25 Mar 2024) -Viktor Szakats (28 Jan 2024) +- cmdline-opts: shorter help texts -- build: delete/replace 3 more clang warning pragmas + In an effort to increase the readability of the "--help all" output on + narrow (80 column) terminals. - - tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma. + Co-authored-by: Jay Satiro - - whitespace formatting in `mprintf.h`, lib518, lib537. + Closes #13169 - - lib518: fix wrong variable in `sizeof()`. +Matt Jolly (25 Mar 2024) - - lib518: bump variables to `rlim_t`. - Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469 +- curl-rustls.m4: add pkg-config support to rustls detection - - lib518: sync error message with lib537 - Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce + Based on the existing openssl pkg-config detection, this commit tries to + use pkg-config to find `rustls` then falls back to the current approach + if that fails. - - lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas - by reworking test code. + We use the following logic: - Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812 - Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 - Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + - if no path is provided, just use pkg-config, if it's not there we have + a problem! + - if a path is provided, try pkg-config + + if pkg-config fails, try and find rustls directly - Reviewed-by: Daniel Stenberg - Closes #12814 + Closes #13179 -Richard Levitte (27 Jan 2024) +Mohammadreza Hendiani (25 Mar 2024) -- cmake: freshen up docs/INSTALL.cmake +- TODO: update 13.11 with more information - - Turn docs/INSTALL.cmake into a proper markdown file, - docs/INSTALL-CMAKE.md - - Move things around to divide the description into configuration, - building and installing sections - - Mention the more modern cmake options to configure, build and install, - but also retain the older variants as fallbacks + Closes #13173 - Closes #12772 +Daniel Stenberg (23 Mar 2024) -Viktor Szakats (27 Jan 2024) +- docs/libcurl: generate PROTOCOLS from meta-data -- build: delete/replace clang warning pragmas + Remove the PROTOCOLS section from the source files completely and + instead generate them based on the header data in the curldown files. - - delete redundant warning suppressions for `-Wformat-nonliteral`. - This now relies on `CURL_PRINTF()` and it's theoratically possible - that this macro isn't active but the warning is. We're ignoring this - as a corner-case here. + It also generates TLS backend information for options marked for TLS as + protocol. - - replace two pragmas with code changes to avoid the warnings. + Closes #13175 - Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 - Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 +- CURLMOPT_MAX*: mention what happens if changed mid-transfer - Reviewed-by: Daniel Stenberg - Closes #12812 + For CURLMOPT_MAXCONNECTS and CURLMOPT_MAX_HOST_CONNECTIONS -Daniel Stenberg (27 Jan 2024) + Ref: #13158 + Closes #13176 -- RELEASE-NOTES: synced +- docs/libcurl: add TLS backend info for all TLS options -- http: only act on 101 responses when they are HTTP/1.1 + All man pages that are listed to be for TLS now must also specify + exactly what TLS backends the option works for, or use All if they all + work. - For 101 responses claiming to be any other protocol, bail out. This - would previously trigger an assert. + cd2nroff makes sure this is done and that the listed backends exist. - Add test 1704 to verify. + Closes #13168 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184 - Closes #12811 +- docs/libcurl: cleanups -Scarlett McAllister (27 Jan 2024) + - CURLINFO_TLS_SESSION.md: remove mention of NSS + - CURLINFO_TLS_SSL_PTR.md: remove NSS leftover + - CURLOPT_CAINFO.md: drop mention of backends not supporting this + - CURLOPT_CAPATH.md: wolfSSL also supports this -- _VARIABLES.md: add missing 'be' into the sentence + Closes #13166 - Closes #12809 +- docs: make each libcurl man specify protocol(s) -Stefan Eissing (27 Jan 2024) + The mandatory header now has a mandatory list of protocols for which the + manpage is relevant. -- mqtt, remove remaining use of data->state.buffer + Most man pages already has a "PROTOCOLS" section, but this introduces a + stricter way to specify the relevant protocols. - Closes #12799 + cd2nroff verifies that at least one protocol is mentioned (which can be + `*`). -Daniel Stenberg (27 Jan 2024) + This information is not used just yet, but A) the PROTOCOLS section can + now instead get generated and get a unified wording across all manpages + and B) this allows us to more reliably filter/search for protocol + specific manpages/options. -- x509asn1: switch from malloc to dynbuf + Closes #13166 - Closes #12808 +Stefan Eissing (21 Mar 2024) -- x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy +- http2, http3: only return CURLE_PARTIAL_FILE when bytes were received - Closes #12808 + - should resolve spurious pytest failures when stream were reset + right after response header were received -- x509asn1: reduce malloc in Curl_extract_certinfo + Clsoes #13151 - Using dynbuf +- http: separate response parsing from response action - Closes #12808 + - move code that triggers on end-of-response into separate function from + parsing + - simplify some headp/headerlen usage + - add `httpversion` to SingleRequest to indicate the version of the + current response -Jay Satiro (27 Jan 2024) + Closes #13134 -- THANKS: add Alexander Bartel and Brennan Kinney +Daniel Stenberg (21 Mar 2024) - They reported and investigated #10259 which was fixed by 7b2d98df. +- http2: remove the third (unused) argument from http2_data_done() - Ref: https://github.com/curl/curl/issues/10259 + Closes #13154 -Daniel Stenberg (26 Jan 2024) +- RELEASE-NOTES: synced -- krb5: add prototype to silence clang warnings on mvsnprintf() +Evgeny Grin (Karlson2k) (21 Mar 2024) - "error: format string is not a string literal" +- RELEASE-NOTES: corrected - Follow-up to 09230127589eccc7 which made the warning appear + Corrected link for item 118 - Assisted-by: Viktor Szakats - Closes #12803 + Closes #13157 -- x509asn1: remove code for WANT_VERIFYHOST +Daniel Stenberg (19 Mar 2024) - No code ever sets this anymore since we dropped gskit +- CURLOPT_INTERFACE.md: remove spurious amp, add see-also - Follow-up to 78d6232f1f326b9ab4d + Closes #13149 - Closes #12804 +Stefan Eissing (19 Mar 2024) -- socks: reduce the buffer size to 600 (from 8K) +- http: improve response header handling, save cpu cycles - This is malloc'ed memory and it does not more. Test 742 helps us verify - this. + Saving some cpu cycles in http response header processing: + - pass the length of the header line along + - use string constant sizeof() instead of strlen() + - check line length if prefix is possible + - switch on first header char to limit checks - Closes #12789 + Closes #13143 -Stefan Eissing (26 Jan 2024) +Daniel Stenberg (19 Mar 2024) -- file+ftp: use stack buffers instead of data->state.buffer +- tool_getparam: accept a blank -w "" - Closes #12789 + Added test 468 to verify. -- vtls: receive max buffer + Regression from 07bcae89d5d00 (shipped in 8.6.0) + Reported-by: Thomas Pyle + Fixes #13144 + Closes #13145 - - do not only receive one TLS record, but try to fill - the passed buffer - - consider <4K remaning space is "filled". +Evgeny Grin (Karlson2k) (18 Mar 2024) - Closes #12801 +- curl_sha512_256: work around a NetBSD bug -Daniel Stenberg (26 Jan 2024) + Based on Michael Kaufmann analysis and suggestion -- docs: do not start lines/sentences with So, But nor And + Closes #13133 - Closes #12802 +Stefan Eissing (18 Mar 2024) -- docs: remove spurious ampersands from markdown +- http: expect 100 rework - They were leftovers from the nroff conversion. + Move all handling of HTTP's `Expect: 100-continue` feature into a client + reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer + sending on general events like a timer. - Follow-up to eefcc1bda4bccd800f5a5 + HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect: + 100-continue`. That reader works as follows: - Closes #12800 + - on first invocation, records time, starts the `EXPIRE_100_TIMEOUT` + timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0, + eos=FALSE like a paused upload. -Patrick Monnerat (26 Jan 2024) + - on subsequent invocation it checks if the timer has expired. If so, it + enables `KEEP_SEND` and switches to passing through reads to the + underlying readers. -- sasl: make login option string override http auth + Transfer handling's `readwrite()` will be invoked when a timer expires + (like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing + `KEEP_SEND_TIMER`, it will try to upload more data, which triggers + reading from the client readers again. Which then may lead to a new + pausing or cause the upload to start. - - Use http authentication mechanisms as a default, not a preset. + Flags and timestamps connected to this have been moved from + `SingleRequest` into the reader's context. - Consider http authentication options which are mapped to SASL options as - a default (overriding the hardcoded default mask for the protocol) that - is ignored if a login option string is given. + Closes #13110 - Prior to this change, if some HTTP auth options were given, sasl mapped - http authentication options to sasl ones but merged them with the login - options. +- mbedtls: fix pytest for newer versions - That caused problems with the cli tool that sets the http login option - CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag - maps to more than one sasl mechanisms and the latter cannot be cleared - individually by the login options string. + Fix the expectations in pytest for newer versions of mbedtls - New test 992 checks this. + Closes #13132 - Fixes https://github.com/curl/curl/issues/10259 - Closes https://github.com/curl/curl/pull/12790 +Daniel Stenberg (15 Mar 2024) -Stefan Eissing (26 Jan 2024) +- ipv6.md: mention IPv4 mapped addresses -- socks: use own buffer instead of data->state.buffer + Reported-by: Josh Soref + Assisted-by: Jay Satiro + Fixes #13112 + Closes #13131 - Closes #12788 +Stefan Eissing (15 Mar 2024) -Daniel Stenberg (26 Jan 2024) +- http: revisit http_perhapsrewind() -- socks: fix generic output string to say SOCKS instead of SOCKS4 + - use facilities provided by client readers better + - work also for non-uploading requests like GET/HEAD + - update documentation - ... since it was also logged for SOCKS5. + Closes #13117 - Closes #12797 +- test 1541: verify getinfo values on first header callback -- test742: test SOCKS5 with max length user, password and hostname + Reported-by: chensong1211 on github + Ref: #13125 + Closes #13128 - Adjusted the socksd server accordingly to allow for configuring that - long user name and password. +- TLS: start shutdown only when peer did not already close - Closes #12797 + - When curl sees a TCP close from the peer, do not start a TLS shutdown. + TLS shutdown is a handshake and if the peer already closed the + connection, it is not interested in participating. -Stefan Eissing (25 Jan 2024) + Reported-by: dfdity on github + Assisted-by: Jiří Bok + Assisted-by: Pēteris Caune + Fixes #10290 + Closes #13087 -- ssh: use stack scratch buffer for seeks +Daniel Stenberg (14 Mar 2024) - - instead of data->state.buffer +- RELEASE-NOTES: synced - Closes #12794 +- curl: make --libcurl output better CURLOPT_*SSLVERSION -Daniel Stenberg (25 Jan 2024) + The option is really two enums ORed together, so it needs special + attention to make the code output nice. -- krb5: access the response buffer correctly + Added test 1481 to verify. Both the server and the proxy versions. - As the pingpong code no longer uses the download buffer. + Reported-by: Boris Verkhovskiy + Fixes #13127 + Closes #13129 - Folllow-up to c2d973627bab12ab - Pointed-out-by: Stefan Eissing - Closes #12796 +- GHA/linux: add sysctl trick to work-around GitHub runner issue -Stefan Eissing (25 Jan 2024) + The GitHub image runner update from 20240304.1.0 to 20240310.1 + introduces a problem for clang-14. The issue is caused by + incompatibility between llvm 14 provided in ubuntu-22.04 image and the + much newer kernel configured with high-entropy ASLR. -- mqtt: use stack scratch buffer for recv+publish + As a work-around, we issue a sysctl command to lower the entropy and get + clang-14 to work again. - - instead of data->state.buffer + URL: https://github.com/actions/runner-images/issues/9491 - Closes #12792 + Closes #13124 -- telnet, use stack scratch buffer for do +- SPONSORS: describe the basics - - instead of data->state.buffer + Closes #13119 - Closes #12793 +- GOVERNANCE: document the core team -- http, use stack scratch buffer + Closes #13118 - - instead of data->state.buffer +Jay Satiro (13 Mar 2024) - Closes #12791 +- vquic-tls: fix the error code returned for bad CA file -- ntlm_wb: do not use data->state.buf any longer + - Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem + reading the cert file or path. - Closes #12787 + This is a follow-up to the parent commit aedbbdf1. -- gitignore: the generated `libcurl-symbols.md` + Reported-by: Karthikdasari0423@users.noreply.github.com - Closes #12795 + Fixes https://github.com/curl/curl/issues/13115 -Daniel Stenberg (25 Jan 2024) +Daniel Stenberg (12 Mar 2024) -- tool: fix the listhelp generation command +- vquic-tls: return appropirate errors on wolfSSL errors - The previous command line to generate the tool_listhelp.c source file - broke with 2494b8dd5175cee7. + Reported-by: Dexter Gerig + Closes #13107 - Make 'make listhelp' invoked in src/ generate it. Also update the - comment in the file to mention the right procedure. +Viktor Szakats (12 Mar 2024) - Closes #12786 +- tidy-up: one comment and EOF newlines -- http: check for "Host:" case insensitively + Reviewed-by: Daniel Stenberg + Closes #13108 - When checking if the user wants to replace the header, the check should - be case insensitive. +Daniel Stenberg (12 Mar 2024) - Adding test 461 to verify +- cmdline-opts: language cleanups - Found-by: Dan Fandrich - Ref: #12782 - Closes #12784 + Use imperative mood consistently for the first sentence describing an + option. -Tatsuhiro Tsujikawa (25 Jan 2024) + "Set this" instead "tell curl to set" or "this sets..." -- configure: add libngtcp2_crypto_boringssl detection + Plus some extra cleanups and rephrasing. - If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested, - try to detect libngtcp2_crypto_boringssl. + Closes #13106 - Reported-by: ウさん - Fixes #12724 - Closes #12769 +- managen: remove space before protocols -Daniel Stenberg (25 Jan 2024) + For options that are listed for specific protocols, the protocols (shown + first within parentheses) are now output without the leading space in the + manpage output. -- http: remove comment reference to a removed solution + Closes #13105 - Follow-up to 58974d25d +Jay Satiro (12 Mar 2024) - Closes #12785 +- mbedtls: properly cleanup the thread-shared entropy -Stefan Eissing (25 Jan 2024) + - Store the state of the thread-shared entropy for global init/cleanup. -- pytest: Scorecard tracking CPU and RSS + - Use curl's thread support of mbedtls for all Windows builds instead of + just when the threaded resolver is used via USE_THREADS_WIN32. - Closes #12765 + Prior to this change on global cleanup curl builds that have curl thread + support for mbedtls freed the entropy (8b1d2298) but failed to mark that + it had been freed, which caused problems on subsequent init + transfer. -Graham Campbell (25 Jan 2024) + Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105 + Reported-by: awesomekosm@users.noreply.github.com -- GHA: bump ngtcp2, gnutls, mod_h2, quiche + Closes https://github.com/curl/curl/pull/13071 - - ngtcp2 to v1.2.0 - - gnutls to 3.8.3 - - mod_h2 to 2.0.26 - - quiche to 0.20.0 +Daniel Stenberg (12 Mar 2024) - Closes #12778 - Closes #12779 - Closes #12780 - Closes #12781 +- tool_getparam: handle non-existing (out of range) short-options -Daniel Stenberg (25 Jan 2024) + ... correctly, even when they follow an existing one without a space in + between. -- ftpserver.pl: send 213 SIZE response without spurious newline + Verify with test 467 -- pingpong: stop using the download buffer + Follow-up to 07dd60c05b + Reported-by: Geeknik Labs + Fixes #13101 + Closes #13102 - The pingpong logic now uses its own dynbuf for receiving command - response data. +Stefan Eissing (11 Mar 2024) - When the "final" response header for a commanad has been received, that - final line is left first in the recvbuf for the protocols to parse at - will. If there is additional data behind the final response line, the - 'overflow' counter is indicate how many bytes. +- lib: move 'done' parameter to SingleRequests - Closes #12757 + A transfer may do several `SingleRequest`s for its success. This happens + regularly for authentication, follows and retries on failed connections. + The "readwrite()" calls and functions connected to those carried a `bool + *done` parameter to indicate that the current `SingleRequest` is over. + This may happen before `upload_done` or `download_done` bits of + `SingleRequest` are set. -- gen.pl: remove bold from .IP used for ## + The problem with that is now `write_resp()` protocol handlers are + invoked in places where the `bool *done` cannot be passed up to the + caller. Instead of being a bool in the call chain, it needs to become a + member of `SingleRequest`, reflecting its state. - Reported-by: Viktor Szakats - Fixes #12776 - Closes #12777 + This removes the `bool *done` parameter and adds the `done` bit to + `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a + `SingleRequest` in a follow up, clearing `done` and other + flags/counters. -Viktor Szakats (24 Jan 2024) + Closes #13096 -- cmake: rework options to enable curl and libcurl docs +- request: clarify message when request has been sent off - Rework CMake options for building/using curl tool and libcurl manuals. + Change the "uploaded and fine" message for requests without a body - - rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning: - to build man page and built-in manual for curl tool. + Reported-by: Karthikdasari0423 on github + Fixes #13093 + Closes #13095 - - rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning: - to build man pages for libcurl. +Daniel Stenberg (11 Mar 2024) - - `BUILD_LIBCURL_DOCS` now works without having to enable - `ENABLE_CURL_MANUAL` too. +- RELEASE-NOTES: synced - - drop support for existing CMake-level `USE_MANUAL` option to avoid - confusion. (It used to work with the effect of current - `ENABLE_CURL_MANUAL`, but only by accident.) +Stefan Eissing (9 Mar 2024) - Assisted-by: Richard Levitte - Ref: #12771 - Closes #12773 +- lib: keep conn IP information together -Daniel Stenberg (24 Jan 2024) + new struct ip_quadruple for holding local/remote addr+port -- urlapi: remove assert + - used in data->info and conn and cf-socket.c + - copy back and forth complete struct + - add 'secondary' to conn + - use secondary in reporting success for ftp 2nd connection - This assert triggers wrongly when CURLU_GUESS_SCHEME and - CURLU_NO_AUTHORITY are both set and the URL is a single path. + Reported-by: DasKutti on github + Fixes #13084 + Closes #13090 - I think this assert has played out its role. It was introduced in a - rather big refactor. +Daniel Stenberg (8 Mar 2024) - Follow-up to 4cfa5bcc9a +- scripts/managen: the new name and home for the manpage generator - Reported-by: promptfuzz_ on hackerone - Closes #12775 + It was previously docs/cmdline-opts/gen.pl -Patrick Monnerat (24 Jan 2024) + Closes #13089 -- tests: avoid int/size_t conversion size/sign warnings +- VULN-DISCLOSURE-POLICY.md: update detail about CVE requests - Closes #12768 + curl is a CNA now -Daniel Stenberg (24 Jan 2024) + Closes #13088 -- GHA: add a job scanning for "bad words" in markdown +Stefan Eissing (8 Mar 2024) - This means words, phrases or things we have decided not to use - words that - are spelled right according to the dictionary but we want to avoid. In the - name of consistency and better documentation. +- lib: client reader polish - Closes #12764 + - seek_func/seek_client, use transfer values only + - remove copies held in `struct connectdata`, use only + ever `data->set.seek_func` + - resolves possible issues in multiuse connections + - new mime post reader eliminates need to ever overwriting this -Viktor Szakats (23 Jan 2024) + - websockets, remove empty Curl_ws_done() function -- cmake: speed up curldown processing, enable by default + Closes #13079 - - cmake: enable `BUILD_DOCS` by default (this controls converting and - installing `.3` files from `.md` sources) +Marcel Raad (8 Mar 2024) - - cmake: speed up generating `.3` files by using a single command per - directory, instead of a single command per file. This reduces external - commands by about a thousand. (There remains some CMake logic kicking - in resulting in 500 -one per file- external `-E touch_nocreate` calls.) +- lib1598: fix `CURLOPT_POSTFIELDSIZE` usage - - cd2nroff: add ability to process multiple input files. + It requires a `long` argument. - - cd2nroff: add `-k` option to use the source filename to form the - output filename. (instead of the default in-file `Title:` line.) + Closes https://github.com/curl/curl/pull/13085 - Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c - Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753 - Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 +Daniel Stenberg (8 Mar 2024) - Closes #12762 +- docs/cmdline-opts: drop the curl.1 from the dist tarball -Richard Levitte (23 Jan 2024) + Since it is no longer needed for building tool_hugehelp.c and all the + docs is available in readable markdown format in the tarball, the peeps + that don't want to build the manpage still do good. -- docs: install curl.1 with cmake as well + Removing it also fixes the complexity of out-of-tree builds when the + curl.1 exists in the source tree. - Closes #12759 +- test1140/1173: extend wildcards to find curl.1 -Daniel Stenberg (23 Jan 2024) + ... in its new build path. -- osslq: remove the TLS library from the version output + Also update the test scripts to be more precise in error messages to + help us understand CI errors better. - Since we only support using a single TLS library at any one time, we - know that the TLS library for QUIC is the same that is also shown for - regular TLS. + Follow-up to f03c85635f35269f1 + Ref: #13029 + Closes #13083 - Fixes #12763 - Reported-by: Viktor Szakats - Closes #12767 +- http2: minor tweaks to optimize two struct sizes -Stefan Eissing (23 Jan 2024) + - use BIT() instead of bool + - place the struct fields in (roughly) size order -- CI: remove unnecessary OpenSSL 3 option `enable-tls1_3` + Closes #13082 - .. and switch OpenSSL 3 libdir from lib64 to lib for consistency. +- buildconf.bat: remove outdated groff/nroff use - Closes https://github.com/curl/curl/pull/12758 + - don't try to generate the real hugehelp file, because it requires + curl.txt which needs a build + - don't attempt to do anything in a c-ares subdirectory -- GHA: bump nghttp2 version to v1.59.0 + Follow-up to f03c85635f35269 + Closes #13078 - - Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version. +- http2: memory errors in the push callbacks are fatal - Closes https://github.com/curl/curl/pull/12766 + Use the correct nghttp2 error code accordingly. -Daniel Stenberg (23 Jan 2024) + Closes #13081 -- RELEASE-NOTES: synced +Viktor Szakats (7 Mar 2024) -- docs/cmdline: change to .md for cmdline docs +- mkhelp: rename variable to fix compiler warnings - - switch all invidual files documenting command line options into .md, - as the documentation is now markdown-looking. + ``` + src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global de + claration [_bld\src\curl.vcxproj] + _bld\src\tool_hugehelp.c(8,27): + see declaration of 'm' + src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75c + d2n0wj6lc#L614 - - made the parser treat 4-space indents as quotes + Reviewed-by: Daniel Stenberg + Closes #13077 - - switch to building the curl.1 manpage using the "mainpage.idx" file, - which lists the files to include to generate it, instead of using the - previous page-footer/headers. Also, those files are now also .md - ones, using the same format. I gave them underscore prefixes to make - them sort separately: - _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md, - _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md, - _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md, - _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md +Daniel Stenberg (7 Mar 2024) - - updated test cases accordingly +- KNOWN_BUGS: POP3 issue when reading small chunks - Closes #12751 + Closes #12063 -dependabot[bot] (23 Jan 2024) +- RELEASE-NOTES: synced -- CI: bump actions/cache from 3 to 4 +Robert Moreton (7 Mar 2024) - Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. - - [Release notes](https://github.com/actions/cache/releases) - - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - - [Commits](https://github.com/actions/cache/compare/v3...v4) +- asyn-ares: fix data race warning - --- - updated-dependencies: - - dependency-name: actions/cache - dependency-type: direct:production - update-type: version-update:semver-major - ... + - Store the c-ares version during global init. - Signed-off-by: dependabot[bot] - Closes #12756 + Prior to this change several threads could write the same data to a + static int variable at the same time. Though in practice it's not a + problem ThreadSanitizer may warn. -Daniel Stenberg (23 Jan 2024) + Reported-by: Nikita Taranov + Assisted-by: Jay Satiro -- openssl: when verifystatus fails, remove session id from cache + Fixes #13065 + Closes #13000 - To prevent that it gets used in a subsequent transfer that skips the - verifystatus check since that check can't be done when the session id is - reused. +Stefan Eissing (7 Mar 2024) - Reported-by: Hiroki Kurosawa - Closes #12760 +- hyper: implement unpausing via client reader -Viktor Szakats (23 Jan 2024) + Just a tidy up to contain 'ifdef' pollution of common + code parts with implementation specifics. -- cmake: add option to disable building docs + - remove the ifdef hyper unpausing in easy.c + - add hyper client reader for CURL_CR_PROTOCOL phase + that implements the unpause method for calling + the hyper waker if it is set -Richard Levitte (23 Jan 2024) + Closes #13075 -- cmake: use curldown to build man pages +- ngtcp2: no recvbuf for stream - This throws away the previous HTML and PDF producers, to mimic what - Makefile.am does as faithfully as possible. + - write response data directly to the transfer via + `Curl_xfer_write_resp()` like we do in HTTP/2. - Closes #12753 + Closes #13073 -Daniel Stenberg (23 Jan 2024) +- docs/cmdline-opts/.gitignore: ignore curl.txt -- mksymbolsmanpage.pl: provide references to where the symbol is used + Closes #13076 -- docs: introduce "curldown" for libcurl man page format +Evgeny Grin (Karlson2k) (7 Mar 2024) - curldown is this new file format for libcurl man pages. It is markdown - inspired with differences: +- sha512_256: add support for GnuTLS and OpenSSL - - Each file has a set of leading headers with meta-data - - Supports a small subset of markdown - - Uses .md file extensions for editors/IDE/GitHub to treat them nicely - - Generates man pages very similar to the previous ones - - Generates man pages that still convert nicely to HTML on the website - - Detects and highlights mentions of curl symbols automatically (when - their man page section is specified) + This is a follow-up for PR #12897. - tools: + Add support for SHA-512/256 digest calculation by TLS backends. + Currently only OpenSSL and GnuTLS (actually, nettle) support + SHA-512/256. - - cd2nroff: converts from curldown to nroff man page - - nroff2cd: convert an (old) nroff man page to curldown - - cdall: convert many nroff pages to curldown versions - - cd2cd: verifies and updates a curldown to latest curldown + Closes #13070 - This setup generates .3 versions of all the curldown versions at build time. +- digest: add check for hashing error - CI: + Closes #13072 - Since the documentation is now technically markdown in the eyes of many - things, the CI runs many more tests and checks on this documentation, - including proselint, link checkers and tests that make sure we capitalize the - first letter after a period... +Viktor Szakats (7 Mar 2024) - Closes #12730 +- cmake: enable `ENABLE_CURL_MANUAL` by default -Viktor Szakats (22 Jan 2024) + Meaning `curl.1` and `src/tool_hugehelp.c` are built by default, + and `--manual` in curl tool is also enabled by default. -- libssh2: use `libssh2_session_callback_set2()` with v1.11.1 + This syncs behaviour with autotools. - To avoid a local hack to pass function pointers and to avoid - deprecation warnings when building with libssh2 v1.11.1 or newer: - ``` - lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is depreca - ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- - declarations] - lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is depreca - ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- - declarations] - ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/2072082 - 1100#step:3:4982 + For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set + to a consistent date, e.g. the timestamp of `CHANGES`. - Ref: https://github.com/libssh2/libssh2/pull/1285 - Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db - 3cf1d4b0b4 - Reviewed-by: Daniel Stenberg - Closes #12754 + A pre-built manual (e.g. the one distributed in the official source + tarball) will be ignored and rebuilt after this patch, unless + explicitly disabling this option. -Daniel Stenberg (22 Jan 2024) + Fixes #13028 + Closes #13069 -- transfer: make the select_bits_paused condition check both directions +Stefan Eissing (7 Mar 2024) - If there is activity in a direction that is not paused, return false. +- http2: push headers better cleanup - Reported-by: Sergey Bronnikov - Bug: https://curl.se/mail/lib-2024-01/0049.html - Closes #12740 + - provide common cleanup method for push headers -Stefan Eissing (22 Jan 2024) + Closes #13054 -- http3: initial support for OpenSSL 3.2 QUIC stack +Daniel Stenberg (7 Mar 2024) - - HTTP/3 for curl using OpenSSL's own QUIC stack together - with nghttp3 - - configure with `--with-openssl-quic` to enable curl to - build this. This requires the nghttp3 library - - implementation with the following restrictions: - * macOS has to use an unconnected UDP socket due to an - issue in OpenSSL's datagram implementation - See https://github.com/openssl/openssl/issues/23251 - This makes connections to non-reponsive servers hang. - * GET requests will send the indicator that they have - no body in a separate QUIC packet. This may result - in processing delays or Transfer-Encodings on proxied - requests - * uploads that encounter blocks will use 100% cpu as - detection of these flow control issue is not working - (we have not figured out to pry that from OpenSSL). +- GIT-INFO: convert to markdown - Closes #12734 + Closes #13074 -Viktor Szakats (22 Jan 2024) +Richard Levitte (7 Mar 2024) -- cmake: fix `ENABLE_MANUAL` option +- cmake: fix libcurl.pc and curl-config library specifications - Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`. + Letting CMake figure out where libraries are located gives you full + paths. When generating libcurl.pc and curl-config, getting libraries as + full paths is unusual when one expects to get a list of -l. - Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the - option designed to enable building and using the built-in curl manual. - (`USE_MANUAL=ON` option worked for this instead, by accident). + To meet expectations, an effort is made to convert the full paths into + -l, possibly with -L before it. - Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409 - Closes #12749 + Fixes #6169 + Fixes #12748 + Closes #12930 -Mohammadreza Hendiani (19 Jan 2024) +Daniel Stenberg (7 Mar 2024) -- TODO: update broken link to ratelimit-headers draft +- test463: HTTP with -d @file with file containing CR, LF and null byte - Closes #12741 +- paramhlp: fix CRLF-stripping files with "-d @file" -Daniel Stenberg (19 Jan 2024) + All CR and LF bytes should be stripped, as documented, and all other + bytes are inluded in the data. Starting now, it also excludes null bytes + as they would otherwise also cut the data short. -- cmake: when USE_MANUAL=YES, build the curl.1 man page + Reported-by: Simon K + Fixes #13063 + Closes #13064 - Fixes KNOWN_BUG 15.4 +Viktor Szakats (7 Mar 2024) - Closes #12742 +- cmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled -- cmdline-opts/write-out.d: remove spurious double quotes + Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF` + when building without the Schannel TLS backend. -Stefan Eissing (19 Jan 2024) + This in turn may have caused Kerberos, SPNEGO and SSPI features + disappearing even with `CURL_WINDOWS_SSPI=ON` set. -- rtsp: Convert assertion into debug log + This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a + default for `CURL_WINDOWS_SSPI`, but allowing a manual override. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 + Also update the option text to better tell its purpose. - - write excess bytes to the client where the standard excess bytes - checks will report any wrongness and fail the transfer + Thanks-to: Andreas Loew + Reviewed-by: Daniel Stenberg + Ref: #13056 + Closes #13061 - Fixes #12738 - Closes #12739 +Jay Satiro (6 Mar 2024) -Daniel Stenberg (19 Jan 2024) +- KNOWN_BUGS: FTPS server compatibility on Windows with Schannel -- headers: remove assert from Curl_headers_push + - Remove "2.12 FTPS with Schannel times out file list operation" - The fuzzer managed to reach the function without a terminating CR or LF - so let's handle it normally. While there, remove the goto. + - Remove "7.12 FTPS directory listing hangs on Windows with Schannel" - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839 + - Add "7.12 FTPS server compatibility on Windows with Schannel" - Closes #12721 + This change adds a more generic bug description that explains FTPS with + the latest curl and Schannel is not widely used and may have more bugs + than other TLS backends. -- curl_easy_getinfo.3: remove the wrong time value count + The two removed FTPS Schannel bugs can't be reproduced any longer and + were likely fixed by 24d6c288. - It said "six" time values but they are eight by now. Remove the mention - of the amount. + Ref: https://github.com/curl/curl/issues/5284 + Ref: https://github.com/curl/curl/issues/9161 + Ref: https://github.com/curl/curl/issues/12894 - Closes #12727 + Closes https://github.com/curl/curl/pull/13032 -Viktor Szakats (18 Jan 2024) +- trace-config.md: remove the mutexed options list -- mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls` + - Remove the rendered manpage message that says: + "[--trace-config] is mutually exclusive to --trace and -v, --verbose". - - Silence warning in mbedTLS v3.5.1 public headers: - ``` - ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: r - edundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls] - ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: pre - vious declaration of 'psa_set_key_domain_parameters' was here - ``` - Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae - 785a513e157c - Ref: https://github.com/libssh2/libssh2/pull/1226 + Actually it can be used with either of those options, which are mutually + exclusive to each other but not to --trace-config. - - Fix compiler warnings seen with gcc 9.2.0 + cmake unity: - ``` - ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read': - ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull - -dereference] - 189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &res - ult); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~ - ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write': - ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull - -dereference] - 168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, & - result); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~ - ``` + Ref: https://curl.se/docs/manpage.html#--trace-config - - delete stray `#else`. + Closes https://github.com/curl/curl/pull/13031 - Closes #12720 +Daniel Stenberg (6 Mar 2024) -Daniel Stenberg (17 Jan 2024) +- mkhelp: simplify the generated hugehelp program -- docs: cleanup nroff format use + Use a plain array and puts() every line, also allows us to provide the + strings without ending newlines. - - remove use of .BI for code snippet - - stop using .br, just do a blank line - - remove use of .PP - - remove use for .sp - - remove backslash in .IP - - use .IP instead of .TP + - merge blank lines into the next one as a prefixed newline. + - turn eight consecutive spaces into a tab (since they can only be on the + left side of text) + - the newly generated tool_hugehelp is 3K lines shorter and 50K smaller + - modifies the top logo layout a little by reducing the indent - Closes #12731 + Closes #13047 -Stefan Eissing (17 Jan 2024) +- docs: ascii version of manpage without nroff -- test2307: fix expected failure code after ws refactoring + Create ASCII version of manpage without nroff - Fixes #12722 - Closes #12728 + - build src/tool_hugegelp.c from the ascii manpage + - move the the manpage and the ascii version build to docs/cmdline-opts + - remove all use of nroff from the build process + - should make the build entirely reproducible (by avoiding nroff) -Jay Satiro (17 Jan 2024) + - partly reverts 2620aa9 to build libcurl option man pages one by one + in cmake because the appveyor builds got all crazy until I did -- cf-socket: show errno in tcpkeepalive error messages + The ASCII version of the manpage - - If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set - then show the errno in the verbose error messages. + - is built with gen.pl, just like the manpage is + - has a right-justified column making the appearance similar to the previous + version + - uses a 4-space indent per level (instead of the old version's 7) + - does not do hyphenation of words (which nroff does) - Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652 + History - Closes https://github.com/curl/curl/pull/12726 + We first made the curl build use nroff for building the hugehelp file in + December 1998, for curl 5.2. -- tool_getparam: stop supporting `@filename` style for --cookie + Closes #13047 - The `@filename` style was never documented for --cookie - but prior to this change curl would accept it anyway and always treat a - @ prefixed string as a filename. +Stefan Eissing (6 Mar 2024) - That's a problem if the string also contains a = sign because then it is - documented to be interpreted as a cookie string and not a filename. - - Example: +- lib: add `void *ctx` to reader/writer instances - `--cookie @foo=bar` + - `struct Curl_cwriter` and `struct Curl_creader` now carry a + `void *ctx` member that points to the instance as allocated. + - using `r->ctx` and `w->ctx` as pointer to the instance specific + struct that has been allocated - Before: Interpreted as load cookies from filename foo=bar. + Reported-by: Rudi Heitbaum + Fixes #13035 + Closes #13059 - After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`). +- http: fix dead code in setting post client reader - Other curl options with a data/filename option-value use the `@filename` - to distinguish filenames which is probably how this happened. The - --cookie option has never been documented that way. + - postsize was always 0, thus the check's else never happened + after the mime client reader was introduced - Ref: https://curl.se/docs/manpage.html#-b + Follow-up to 0ba47146f7ff3d + Closes #13060 - Closes https://github.com/curl/curl/pull/12645 +- http2: fix push discard -Stefan Eissing (16 Jan 2024) + - fix logic in discarding a failed pushed stream so that + stream context is properly cleaned up -- websockets: refactor decode chain + Closes #13055 - - use client writer stack for decoding frames - - move websocket protocol handler to ws.c +- transfer.c: break receive loop in speed limited transfers - Closes #12713 + - the change breaks looping in transfer.c receive for transfers that are + speed limited on having gotten *some* bytes. + - the overall speed limit timing is done in multi.c -- websockets: check for negative payload lengths + Reported-by: Dmitry Karpov + Bug: https://curl.se/mail/lib-2024-03/0001.html + Closes #13050 - - in en- and decoding, check the websocket frame payload lengths for - negative values (from curl_off_t) and error the operation in that case - - add test 2307 to verify +- mime: add client reader - Closes #12707 + Add `mime` client reader. Encapsulates reading from mime parts, getting + their length, rewinding and unpausing. -Daniel Stenberg (16 Jan 2024) + - remove special mime handling from sendf.c and easy.c + - add general "unpause" method to client readers + - use new reader in http/imap/smtp + - make some mime functions static that are now only used internally -- docs: mention env vars not used by schannel + In addition: + - remove flag 'forbidchunk' as no longer needed - Ref: #12704 + Closes #13039 - Co-authored-by: Jay Satiro +Daniel Stenberg (5 Mar 2024) - Closes #12711 +- RELEASE-NOTES: synced -- tool_operate: make --remove-on-error only remove "real" files +- TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake" - Reported-by: Harry Sintonen - Assisted-by: Dan Fandrich + Follow-up to 8e741644a229c37 - Closes #12710 +Tal Regev (5 Mar 2024) -Jay Wu (16 Jan 2024) +- cmake: add USE_OPENSSL_QUIC support -- url: don't set default CA paths for Secure Transport backend + Closes #13034 - As the default for this backend is the native CA store. +Stefan Eissing (5 Mar 2024) - Closes #12704 +- TIMER_STARTTRANSFER: set the same for everyone -Lin Sun (16 Jan 2024) + - set TIMER_STARTTRANSFER on seeing the first response bytes + in the download client writer, not coming from a CONNECT + - initialized the timer the same way for all protocols + - remove explicit setting of TIMER_STARTTRANSFER in file.c + and c-hyper.c -- asyn-ares: with modern c-ares, use its default timeout + Closes #13052 - Closes #12703 +Michael Kaufmann (5 Mar 2024) -Daniel Stenberg (15 Jan 2024) +- http: better error message for HTTP/1.x response without status line -- tool_operate: stop setting the file comment on Amiga + If a response without a status line is received, and the connection is + known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status + line" instead of "Received HTTP/0.9 when not allowed". - - the URL is capped at 80 cols, which ruins it if longer - - it does not strip off URL credentials - - it is done unconditonally, not on --xattr - - we don't have Amiga in the CI which makes fixing it blindly fragile + Closes #13045 - Someone who builds and tests on Amiga can add it back correctly in a - future if there is a desire. +Viktor Szakats (5 Mar 2024) - Reported-by: Harry Sintonen - Closes #12709 +- KNOWN_BUGS: fix typo -Stefan Eissing (15 Jan 2024) + Reviewed-by: Daniel Stenberg + Closes #13051 + +Sebastian Neubauer (5 Mar 2024) + +- smpt: fix starttls + + In cases where the connection was fast, curl sometimes failed to open a + connection. This fixes a regression of c2d973627bab12abc5486a3f3. + + The regression triggered in these steps: + + 1. Create an smtp connection + 2. Use STARTTLS + 3. Receive the response + 4. We are inside the loop in `smtp_statemachine`, calling + `smtp_state_starttls_resp` + 5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and + run `smtp_perform_upgrade_tls` at the start of the function. + + In the bad flow, we stay in the while loop, calling + `Curl_pp_readresp`, which reads part of the TLS handshake and things + go wrong. + + The reason is that `Curl_pp_moredata` changed behavior and always + returns `true`, so we stay in the loop in `smtp_statemachine`. With a + slow connection `Curl_pp_readresp` cannot read new data and returns + `CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`. + + With a fast connection, `Curl_pp_readresp` reads new data from the tcp + connection, which is part of the TLS handshake. + + The fix is in `Curl_pp_moredata`, which needs to take the final line + into account and return `false` if only the final line is stored. + + Closes #13048 + +Stefan Eissing (5 Mar 2024) + +- lib: enhance client reader resume + rewind + + - update client reader documentation + - client reader, add rewind capabilities + - tell creader to rewind on next start + - Curl_client_reset() will keep reader for future rewind if requested + - add Curl_client_cleanup() for freeing all resources independent of + rewinds + - add Curl_client_start() to trigger rewinds + - move rewind code from multi.c to sendf.c and make part of + "cr-in"'s implementation + - http, move the "resume_from" handling into the client readers + - the setup of a HTTP request is reshuffled to follow: + * determine method, target, auth negotiation + * install the client reader(s) for the request, including crlf + conversions and "chunked" encoding + * apply ranges to client reader + * concat request headers, upgrades, cookies, etc. + * complete request by determining Content-Length of installed + readers in combination with method + * send + - add methods for client readers to + * return the overall length they will generate (or -1 when unknown) + * return the amount of data on the CLIENT level, so that + expect-100 can decide if it want to apply itself + * set a "resume_from" offset or fail if unsupported + - struct HTTP has become largely empty now + - rename `Client_reader_*` to `Curl_creader_*` + + Closes #13026 + +Viktor Szakats (5 Mar 2024) + +- openssl-quic: fix BIO leak and Windows warning + + Caused by an accidentally duplicated line in + d6825df334def106f735ce7e0c1a2ea87bddffb0. -- rtsp: deal with borked server responses + ``` + .../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses intege + r precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-6 + 4-to-32] + 1095 | bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE); + | ~~~~~~~~~~~~~ ~~~~~~~^~~~~~ + 1 warning and 2 errors generated. + ``` - - enforce a response body length of 0, if the - response has no Content-lenght. This is according - to the RTSP spec. - - excess bytes in a response body are forwarded to - the client writers which will report and fail the - transfer + Reviewed-by: Stefan Eissing + Closes #13043 - Follow-up to d7b6ce6 - Fixes #12701 - Closes #12706 +- openssl-quic: fix unity build, casing, indentation -Daniel Stenberg (14 Jan 2024) + - rename static functions to avoid duplicate symbols in unity mode. + - windows -> Windows/window in error message and comment. + - fix indentation. -- version: show only the libpsl version, not its dependencies + Reviewed-by: Stefan Eissing + Closes #13044 - The libpsl version output otherwise also includes version number for its - dependencies, like IDN lib, but since libcurl does not use libpsl's IDN - functionality those components are not important. +Daniel Stenberg (5 Mar 2024) - Ref: https://github.com/curl/curl-for-win/issues/63 - Closes #12700 +- gen.pl: make the "manpageification" faster -Brad Harder (14 Jan 2024) + The function that replaces occurances of "--longoption" with "-Z, + --longoption" etc with the proper highlight applied, no longer loops + over the options. -- curl.h: CURLOPT_DNS_SERVERS is only available with c-ares + Closes #13041 - Closes #12695 +- CONTRIBUTE: update the section on documentation format -Daniel Stenberg (14 Jan 2024) + ... since most of it is markdown now. -- cmdline-opts/gen.pl: error on initital blank line + Closes #13046 - After the "---" separator, there should be no blank line and this script - now errors out if one is detected. +- smtp: free a temp resource - Ref: #12696 - Closes #12698 + The returned address needs to be freed. -- cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper + Follow-up to e3905de8196d67b89df1602feb84c1f993211b20 + Spotted by Coverity - Follow-up to 693cd1679361828a which was incomplete + Closes #13038 - Ref #12680 - Closes #12697 +- _VARIABLES.md: improve the description -- curl_multi_fdset.3: remove mention of null pointer support + Closes #13040 - ... since this funtion has not supported null pointer fd_set arguments since - at least 2006. (That's when I stopped my git blame journey) +dependabot[bot] (4 Mar 2024) - Fixes #12691 - Reported-by: sfan5 on github - Closes #12692 +- build(deps): bump fsfe/reuse-action from 2 to 3 -Mark Huang (14 Jan 2024) + Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3. + - [Release notes](https://github.com/fsfe/reuse-action/releases) + - [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3) -- docs/cmdline: remove unnecessary line breaks + --- + updated-dependencies: + - dependency-name: fsfe/reuse-action + dependency-type: direct:production + update-type: version-update:semver-major + ... - Closes #12696 + Signed-off-by: dependabot[bot] -Daniel Stenberg (14 Jan 2024) +Stefan Eissing (4 Mar 2024) -- transfer: remove warning: Value stored to 'blen' is never read +- pytest: adapt to API change - Detected by scan-build + - pytest has changed the signature of the hook pytest_report_header() + for some obscure reason and that change landed in our CI now - Follow-up from 1cd2f0072f + - remove the changed param that we never used anyway - Closes #12693 + Closes #13037 -Stefan Eissing (13 Jan 2024) +Daniel Stenberg (4 Mar 2024) -- lib: replace readwrite with write_resp +- cookie: if psl fails, reject the cookie - This clarifies the handling of server responses by folding the code for - the complicated protocols into their protocol handlers. This concerns - mainly HTTP and its bastard sibling RTSP. + A libpsl install without data and no built-in database is now considered + bad enough to reject all cookies since they cannot be checked. It is + somewhat of a user error, but still. - The terms "read" and "write" are often used without clear context if - they refer to the connect or the client/application side of a - transfer. This PR uses "read/write" for operations on the client side - and "send/receive" for the connection, e.g. server side. If this is - considered useful, we can revisit renaming of further methods in another - PR. + Reported-by: Dan Fandrich + Closes #13033 - Curl's protocol handler `readwrite()` method been changed: +Stefan Eissing (4 Mar 2024) - ```diff - - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, - - const char *buf, size_t blen, - - size_t *pconsumed, bool *readmore); - + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t ble - n, - + bool is_eos, bool *done); - ``` +- lib: further send/upload handling polish - The name was changed to clarify that this writes reponse data to the - client side. The parameter changes are: + - Move all the "upload_done" handling to request.c - * `conn` removed as it always operates on `data->conn` - * `pconsumed` removed as the method needs to handle all data on success - * `readmore` removed as no longer necessary - * `is_eos` as indicator that this is the last call for the transfer - response (end-of-stream). - * `done` TRUE on return iff the transfer response is to be treated as - finished + - add possibility to abort sending of a request + - add `Curl_req_done_sending()` for checks + - transfer.c: readwrite_upload() now clean - This change affects many files only because of updated comments in - handlers that provide no implementation. The real change is that the - HTTP protocol handlers now provide an implementation. + - removing data->state.ulbuf and data->req.upload_fromhere - The HTTP protocol handlers `write_resp()` implementation will get passed - **all** raw data of a server response for the transfer. The HTTP/1.x - formatted status and headers, as well as the undecoded response - body. `Curl_http_write_resp_hds()` is used internally to parse the - response headers and pass them on. This method is public as the RTSP - protocol handler also uses it. + - as well as data->req.upload_present + - set data->req.upload_done on having read all from + the client and completely flushed the send buffer - HTTP/1.1 "chunked" transport encoding is now part of the general - *content encoding* writer stack, just like other encodings. A new flag - `CLIENTWRITE_EOS` was added for the last client write. This allows - writers to verify that they are in a valid end state. The chunked - decoder will check if it indeed has seen the last chunk. + - tftp, remove setting of data->req.upload_fromhere - The general response handling in `transfer.c:466` happens in function - `readwrite_data()`. This mainly operates now like: + - serves no purpose as `upload_present` is not set + and the data itself is directly `sendto()` anyway - ``` - static CURLcode readwrite_data(data, ...) - { - do { - Curl_xfer_recv_resp(data, buf) - ... - Curl_xfer_write_resp(data, buf) - ... - } while(interested); - ... - } - ``` + - smtp, make upload EOB conversion a client reader + - xfer_ulbuf addition - All the response data handling is implemented in - `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` - implementation if available, or does the default behaviour. + - add xfer_ulbuf for borrowing, similar to xfer_buf + - use in file upload + - use in c-hyper body sending - All raw response data needs to pass through this function. Which also - means that anyone in possession of such data may call - `Curl_xfer_write_resp()`. + - h1-proxy, remove init of data->state.uilbuf that is never used + - smb, add own send_buf instead of using data->state.ulbuf - Closes #12480 + Closes #13010 -Daniel Stenberg (13 Jan 2024) +Daniel Stenberg (4 Mar 2024) - RELEASE-NOTES: synced -- TODO: TFTP doesn't convert LF to CRLF for mode=netascii +kpcyrd (3 Mar 2024) - Closes #12655 - Closes #12690 +- rustls: fix two warnings related to number types -- gen: do italics/bold for a range of letters, not just single word + Reported-by: Gisle Vanem + Follow-up to #12989 + Closes #13017 - Previously it would match only on a sequence of non-space, which made it - miss to highlight for example "public suffix list". +Stefan Eissing (3 Mar 2024) - Updated the recent cookie.d edit from 5da57193b732 to use bold instead - of italics. +- bufq: writing into a softlimit queue cannot be partial - Closes #12689 + - when unable to obtain a new chunk on a softlimit bufq, + this is an allocation error and needs to be reported as + such. + - writes into a soflimit bufq never must be partial success -- docs: describe and highlight super cookies + Reported-by: Dan Fandrich + Fixes #13020 + Closes #13023 - Reported-by: Yadhu Krishna M +Dan Fandrich (2 Mar 2024) - Closes #12687 +- configure: Don't build shell completions when disabled -- configure: when enabling QUIC, check that TLS supports QUIC + With the recent changes to completion file building, the files were + built always and only installation was selectively disabled. Now, when + they are disabled they aren't even built, avoiding a build-time error in + environments where it's not possible to run the curl binary that was + just created (e.g. if library paths were not set up correctly). - Most importantly perhaps is when using OpenSSL that the used - build/flavor has the QUIC API: the vanilla OpenSSL does not, only - BoringSSL, libressl, AWS-LC and quictls do. + Follow-up to 0f7aba83c - Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7f - e57e#r136780413 + Reported-by: av223119 on github + Fixes #13027 + Closes #13030 - Closes #12683 +Jay Satiro (2 Mar 2024) -Stefan Eissing (11 Jan 2024) +- cmdline-opts/_EXITCODES: sync with libcurl-errors -- vquic: extract TLS setup into own source + - Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that + can be returned by the curl tool. - - separate ngtcp2 specific parts out - - provide callback during init to allow ngtcp2 to apply its defaults + Closes https://github.com/curl/curl/pull/13015 - Closes #12678 +Stefan Eissing (1 Mar 2024) -Sergey Markelov (11 Jan 2024) +- hyper: disable test1598 due to lack of trailer support -- multi: remove total timer reset in file_do() while fetching file:// + Follow-up to 50838095 - The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts - with resetting the timer that is a start point for further multi states. - If file://, MSTATE_DO calls file_do() that should not reset the total - timer. Otherwise, the total time is always less than the pre-transfer - and the start transfer times. + Closes #13016 - Closes #12682 +Dan Fandrich (1 Mar 2024) -Daniel Stenberg (11 Jan 2024) +- ftp: Mark a const buffer as const -- http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT +- appveyor: Properly skip if only CircleCI is changed - Extended test 80 to verify this. +- docs: Update minimal binary size in INSTALL.md - Reported-by: Stefan Eissing - Fixes #12680 - Closes #12681 + Include more options to reduce binary size. -- sectransp: do verify_cert without memdup for blobs +- configure: Don't make shell completions without perl - Since the information is then already stored in memory, this can avoid - an extra set of malloc + free calls. + The code that attempted to skip building the shell completions didn't + work properly and tried to build them even if perl wasn't available. + This step, as well as the install step, is now properly skipped without + perl. - Closes #12679 + Follow-up to 89733e2dd -- hsts: remove assert for zero length domain + Closes #13022 - A zero length domain can happen if the HSTS parser is given invalid - input data which is not unheard of and is done by the fuzzer. +RainRat (1 Mar 2024) - Follow-up from cfe7902111ae547873 +- misc: Fix typos in docs and lib - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661 + This fixes miscellaneous typos and duplicated words in the docs, lib + and test comments and a few user facing errorstrings. - Closes #12676 + Author: RainRat on Github + Reviewed-by: Daniel Gustafsson + Reviewed-by: Dan Fandrich + Closes: #13019 -- headers: make sure the trailing newline is not stored +Dan Fandrich (29 Feb 2024) - extended test1940 to verify blank header fields too +- configure: build & install shell completions when enabled - Bug: https://curl.se/mail/lib-2024-01/0019.html - Reported-by: Dmitry Karpov - Closes #12675 + The --with-fish-functions-dir and --with-zsh-functions-dir options + currently have no effect on a normal build because the scripts/ directory + where they're used is not built. Add scripts/ to a normal build and + change the completion options to default to off to preserve the existing + behaviour. -- curl_easy_header.3: tiny language fix + Closes: #12906 - Closes #12672 +- github/labeler: improve the match patterns -- examples/range.c: add +Stefan Eissing (28 Feb 2024) - Closes #12671 +- tests: add test1598 for POST with trailers -- examples/netrc.c: add + - test POST fields with trailers and chunked encoding - Closes #12671 + Ref: #12938 + Closes #13009 -- examples/ipv6.c: new example showing IPv6-only internet transfer +Daniel Stenberg (28 Feb 2024) - Closes #12671 +- cmdline-opts/_VERSION: provide %VERSION correctly -- examples/address-scope.c: renamed from ipv6.c + ... so that it does not get included verbatim in the output. Fixes a + regression shipped in 8.6.0. - It shows address scope use really + Also fix a format mistake in form.md - Closes #12671 + Closes #13008 -Stefan Eissing (9 Jan 2024) +Stefan Eissing (28 Feb 2024) -- multi: pollset adjust, init with FIRSTSOCKET during connect +- lib: Curl_read/Curl_write clarifications - - `conn->sockfd` is set by `Curl_setup_transfer()`, but that - is called *after* the connection has been established - - use `conn->sock[FIRSTSOCKET]` instead + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to + clarify when and at what level they operate + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer + setup of `conn->sockfd` and `conn->writesockfd` on which + connection filter chain to operate. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index + as parameter. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for + naming consistency + - clarify that the special CURLE_AGAIN hangling to return + `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` + and CURLE_AGAIN is returned by all other send() variants. + - fix a bug in websocket `curl_ws_recv()` that mixed up data + when it arrived in more than a single chunk (to be made + into a sperate PR, also) - Follow-up to a0f94800d507de - Closes #12664 + Added as documented [in + CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68 + aa06dc738cb75d4/docs/CLIENT-READERS.md). -Daniel Stenberg (9 Jan 2024) + - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` + - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` + - HTTP chunked uploads are now formatted in a client reader added when + needed. + - FTP line-end conversions are done in a client reader added when + needed. + - when sending requests headers, remaining buffer space is filled with + body data for sending in "one go". This is independent of the request + body size. Resolves #12938 as now small and large requests have the + same code path. -- WEBSOCKET.md: remove dead link + Changes done to test cases: -- CI: spellcheck/appveyor: invoke configure --without-libpsl + - test513: now fails before sending request headers as this initial + "client read" triggers the setup fault. Behaves now the same as in + hyper build + - test547, test555, test1620: fix the length check in the lib code to + only fail for reads *smaller* than expected. This was a bug in the + test code that never triggered in the old implementation. - Follow-up to 2998874bb61ac6 + Closes #12969 -- cmdline/docs/*.d: switch to using ## instead of .IP +Daniel Gustafsson (28 Feb 2024) - To make the editing easier. To write and to read. +- curldown: Fix email address in Copyright - Closes #12667 + The curldown conversion accidentally replaced daniel@haxx.se with + just daniel.se. This reverts back to the proper email address in + the curldown docs as well as in a few other stray places where it + was incorrect (while unrelated to curldown). -- gen.pl: support ## for doing .IP in table-like lists + Reviewed-by: Daniel Stenberg + Closes: #12997 - Warn on use of .RS/.IP/.RE +Daniel Stenberg (28 Feb 2024) - Closes #12667 +- getparam: make --ftp-ssl work again -Jay Satiro (9 Jan 2024) + Follow-up to 9e4e527 which accidentally broke it -- cookie.d: Document use of empty string to enable cookie engine + Reported-by: Jordan Brown + Fixes #13006 + Closes #13007 - - Explain that --cookie "" can be used to enable the cookie engine - without reading any initial cookies. +- KNOWN_BUGS: IMAPS connection fails with rustls error - As is documented in CURLOPT_COOKIEFILE. + Closes #10457 - Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html +- KNOWN_BUGS: FTPS upload, FileZilla, GnuTLS and close_notify - Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420 - Reported-by: janko-js@users.noreply.github.com + Closes #11383 - Closes https://github.com/curl/curl/pull/12646 +- KNOWN_BUGS: Implicit FTPS upload timeout -Daniel Stenberg (9 Jan 2024) + Closes #11720 -- setopt: use memdup0 when cloning COPYPOSTFIELDS +- KNOWN_BUGS: HTTP/2 prior knowledge over proxy - Closes #12651 + Closes #12641 -- telnet: use dynbuf instad of malloc for escape buffer +- TODO: build HTTP/3 with OpenSSL and nghttp3 using cmake - Previously, send_telnet_data() would malloc + free a buffer every time - for escaping IAC codes. Now, it reuses a dynbuf for this purpose. + Closes #12988 - Closes #12652 +- TODO: Select signature algorithms -- CI: install libpsl or configure --without-libpsl in builds + Closes #12982 - As a follow-up to the stricted libpsl check in configure +- examples: use present tense in comments -- configure: make libpsl detection failure cause error + remove "will" and some other word fixes - To force users to explictily disable it if they really don't want it - used and make it harder to accidentally miss it. + Closes #13003 - --without-libpsl is the option to use if PSL is not wanted. +- docs: more language cleanups - Closes #12661 + - present tense + - avoid bad words -- RELEASE-NOTES: synced + Closes #13003 -- pop3: replace calloc + memcpy with memdup0 +Daniel Gustafsson (27 Feb 2024) - ... and make sure to return error on out of memory. +- setopt: Fix disabling all protocols - Closes #12650 + When disabling all protocols without enabling any, the resulting + set of allowed protocols remained the default set. Clearing the + allowed set before inspecting the passed value from --proto make + the set empty even in the errorpath of no protocols enabled. -- lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT + Co-authored-by: Dan Fandrich + Reported-by: Dan Fandrich + Reviewed-by: Daniel Stenberg + Closes: #13004 - Closes #12658 +Andreas Kiefer (27 Feb 2024) -- mime: use memdup0 instead of malloc + memcpy +- fopen: fix narrowing conversion warning on 32-bit Android - Closes #12649 + This was fixed in commit 06dc599405f, but came back in commit + 03cb1ff4d62. -- tool_getparam: move the --rate logic into set_rate() + When building for 32-bit ARM or x86 Android, `st_mode` is defined as + `unsigned int` instead of `mode_t`, resulting in a + `-Wimplicit-int-conversion` clang warning because `mode_t` is + `unsigned short`. Add a cast to silence the warning, but only for + 32-bit Android builds, because other architectures and platforms are + not affected. -- tool_getparam: switch to an enum for every option + Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li + bc/include/sys/stat.h#86 + Closes https://github.com/curl/curl/pull/12998 - To make the big switch much easier to read/understand and to make it - easier to add new options. +Stefan Eissing (27 Feb 2024) -- tool_getparam: build post data using dynbuf (more) +- lib: Curl_read/Curl_write clarifications -- tool_getparam: replace malloc + copy by dynbuf for --data + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to + clarify when and at what level they operate + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer + setup of `conn->sockfd` and `conn->writesockfd` on which + connection filter chain to operate. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index + as parameter. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for + naming consistency + - clarify that the special CURLE_AGAIN hangling to return + `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` + and CURLE_AGAIN is returned by all other send() variants. + - fix a bug in websocket `curl_ws_recv()` that mixed up data + when it arrived in more than a single chunk -- tool_getparam: make data_urlencode avoid direct malloc + The method for sending not just raw bytes, but bytes that are either + "headers" or "body". The send abstraction stack, to to bottom, now is: - use aprintf() instead + * `Curl_req_send()`: has parameter to indicate amount of header bytes, + buffers all data. + * `Curl_xfer_send()`: knows on which socket index to send, returns + amount of bytes sent. + * `Curl_conn_send()`: called with socket index, returns amount of bytes + sent. -- tool_getparam: move the --url-query logic into url_query() + In addition there is `Curl_req_flush()` for writing out all buffered + bytes. - This function is not doing post at all so it was always weirdly placed. + `Curl_req_send()` is active for requests without body, + `Curl_buffer_send()` still being used for others. This is because the + special quirks need to be addressed in future parts: -- tool_getparam: move the --data logic into set_data() + * `expect-100` handling + * `Curl_fillreadbuffer()` needs to add directly to the new + `data->req.sendbuf` + * special body handlings, like `chunked` encodings and line end + conversions will be moved into something like a Client Reader. -- tool_getparam: unify the cmdline switch() into a single one + In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, + replace the `ssize_t` with a `size_t`. It makes no sense to allow for negativ + e + values as the returned `CURLcode` already specifies error conditions. This + allows easier handling of lengths without casting. - - easier to follow, easier to modify, easier to extend, possibly slightly - faster + Closes #12964 - - each case now has the long option as a comment +Daniel Stenberg (27 Feb 2024) -- tool_getparam: bsearch cmdline options +- multi: make add_handle free any multi_easy - - the option names are now alpha sorted and lookup is a lot faster + If the easy handle that is being added to a multi handle has previously + been used for curl_easy_perform(), there is a private multi handle here + that we can kill off. While it flushes some caches etc for the easy + handle would it be used for an easy interface transfer again after being + used in the multi stack, this cleanup simplifies behavior and uses less + memory. - - use case sensitive matching. It was previously case insensitive, but that - was not documented nor tested. + Closes #12992 - - remove "partial match" feature. It was not documented, not tested and - was always fragile as existing use could break when we add a new - option +- docs: use present tense - - lookup short options via a table + avoid "will", detect "will" as a bad word in the CI - Closes #12631 + Also line wrapped a bunch of paragraphs -Gabe (8 Jan 2024) + Closes #13001 -- COPYING: update copyright year +- CURLOPT_SSL_CTX_FUNCTION.md: no promises of lifetime after return - Closes #12654 + ... and cleanup other language. -Stefan Eissing (8 Jan 2024) + Closes #12999 -- url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD +Stefan Eissing (27 Feb 2024) - Also add more tracing to test 19 +- lib: send rework - Follow-up to a0f9480 + Curl_read/Curl_write clarifications - Fixes #12657 - Closes #12659 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify + when and at what level they operate -Daniel Stenberg (8 Jan 2024) + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer setup of + `conn->sockfd` and `conn->writesockfd` on which connection filter + chain to operate. -- connect: remove margin from eyeballer alloc + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index as + parameter. - Presumably leftovers from debugging + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming + consistency - Closes #12647 + - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` + with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is + returned by all other send() variants. -- ftp: only consider entry path if it has a length + SingleRequest reshuffling - Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb + - move functions into request.[ch] + - differentiate between reset and free + - add Curl_req_done() to perform last actions + - add a send `bufq` to SingleRequest for future use in keeping upload data - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631 + Closes #12963 - Avoids a NULL pointer deref. +Daniel Stenberg (26 Feb 2024) - Closes #12648 +- RELEASE-NOTES: synced -Stefan Eissing (7 Jan 2024) +- http_chunks: remove unused 'endptr' variable -- transfer: adjust_pollset improvements + Closes #12996 - - let `multi_getsock()` initialize the pollset in what the - transfer state requires in regards to SEND/RECV - - change connection filters `adjust_pollset()` implementation - to react on the presence of POLLIN/-OUT in the pollset and - no longer check CURL_WANT_SEND/CURL_WANT_RECV - - cf-socket will no longer add POLLIN on its own - - http2 and http/3 filters will only do adjustments if the - passed pollset wants to POLLIN/OUT for the transfer on - the socket. This is similar to the HTTP/2 proxy filter - and works in stacked filters. +Louis Solofrizzo (26 Feb 2024) - Closes #12640 +- lib: initialize output pointers to NULL before calling strto[ff,l,ul] -Daniel Stenberg (6 Jan 2024) + In order to make MSAN happy: -- ftp: use memdup0 to store the OS from a SYST 215 response + ==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x596f3b3ed246 in curlx_strtoofft [...]/libcurl/src/lib/strtoofft.c:23 + 9:11 + #1 0x596f3b402156 in Curl_httpchunk_read [...]/libcurl/src/lib/http_chunk + s.c:149:12 + #2 0x596f3b348550 in readwrite_data [...]/libcurl/src/lib/transfer.c:607: + 11 + [...] - avoid malloc + direct buffer fiddle + ==2202041==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x5a3fab66a72a in Curl_parse_port [...]/libcurl/src/lib/urlapi.c:547:8 + #1 0x5a3fab650645 in parse_authority [...]/libcurl/src/lib/urlapi.c:796:1 + 2 + #2 0x5a3fab6740f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 + #3 0x5a3fab664fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: + 1342:12 + [...] - Closes #12639 + ==2202320==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x569076a0d6b0 in ipv4_normalize [...]/libcurl/src/lib/urlapi.c:683:12 + #1 0x5690769f2820 in parse_authority [...]/libcurl/src/lib/urlapi.c:803:1 + 0 + #2 0x569076a160f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 + #3 0x569076a06fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: + 1342:12 + [...] -- ftp: use dynbuf to store entrypath + Signed-off-by: Louis Solofrizzo + Closes #12995 - avoid direct malloc +Stefan Eissing (26 Feb 2024) - Closes #12638 +- lib: move client writer into own source -Lealem Amedie (6 Jan 2024) + Refactoring of the client writer that passes the data to the + client/application's callback functions. -- wolfssl: load certificate *chain* for PEM client certs + - split out into own source cw-out.[ch] from sendf.c - Closes #12634 + - move tempwrite and tempcount from data->state into the context of the + client writer -Stefan Eissing (4 Jan 2024) + - redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On + paused transfers, this allows to "record" interleaved HEADER/BODY + chunks to be "played back" in the same order on unpausing. -- http: adjust_pollset fix + - keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. + On exceeding that, return CURLE_TOO_LARGE instead of + CURLE_OUT_OF_MEMORY as before. - do not add a socket for POLLIN when the transfer does not want to send - (for example is paused). + - add method to be called when a transfer is DONE to allow writing of + any data still buffered - Follow-up to 47f5b1a + - when paused, record HEADER writes exactly as they come for later + playback. HEADERs are documented to be written one-by-one. - Reported-by: bubbleguuum on github - Fixes #12632 - Closes #12633 + Closes #12898 -Daniel Stenberg (3 Jan 2024) +- urldata: move authneg bit from conn to Curl_easy -- tool: make parser reject blank arguments if not supported + - from `conn->bits.authneg` to `data->req.authneg` + - this is a property of the request about to be made + and not a property of the connection + - in multiuse connections, transfer could step on each others + toes here potentially. - Already in the getstr() function that clones the input argument. + Closes #12949 - Closes #12620 +- c-hyper: add header collection writer in hyper builds -dependabot[bot] (3 Jan 2024) + Closes #12880 -- build(deps): bump github/codeql-action from 2 to 3 +- http: move headers collecting to writer - Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 - to 3. - - [Release notes](https://github.com/github/codeql-action/releases) - - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - - [Commits](https://github.com/github/codeql-action/compare/v2...v3) + - add a client writer that does "push" response + headers written to the client if the headers api + is enabled + - remove special handling in sendf.c + - needs to be installed very early on connection + setup to catch CONNECT response headers - --- - updated-dependencies: - - dependency-name: github/codeql-action - dependency-type: direct:production - update-type: version-update:semver-major - ... + Closes #12880 - Signed-off-by: dependabot[bot] +- sendf: Curl_client_write(), make passed in buf const - Closes #12625 +Michał Antoniak (26 Feb 2024) -- build(deps): bump actions/checkout from 3 to 4 +- lib: remove curl_mimepart object when CURL_DISABLE_MIME - Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - - [Release notes](https://github.com/actions/checkout/releases) - - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - - [Commits](https://github.com/actions/checkout/compare/v3...v4) + Remove curl_mimepart object from UserDefined structure when + CURL_DISABLE_MIME flag is active. Reduce size of UserDefined structure. - --- - updated-dependencies: - - dependency-name: actions/checkout - dependency-type: direct:production - update-type: version-update:semver-major - ... + Also remove unreachable code: when CURL_DISABLE_MIME is set, httpreq can + never have HTTPREQ_POST_MIME value and the same goes for the + CURL_DISABLE_FORM_API flag and the HTTPREQ_POST_FORM value - Signed-off-by: dependabot[bot] + Closes #12948 - Closes #12624 +kpcyrd (26 Feb 2024) -- build(deps): bump actions/upload-artifact from 3 to 4 +- rustls: make curl compile with 0.12.0 - Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) f - rom 3 to 4. - - [Release notes](https://github.com/actions/upload-artifact/releases) - - [Commits](https://github.com/actions/upload-artifact/compare/v3...v4) + Closes #12989 - --- - updated-dependencies: - - dependency-name: actions/upload-artifact - dependency-type: direct:production - update-type: version-update:semver-major - ... +Daniel Stenberg (26 Feb 2024) - Signed-off-by: dependabot[bot] +- strtoofft: fix the overflow check - Closes #12627 + ... to not rely on wrapping, since it is an undefined behavior that is + not what always might happen. This is in our private strtoff() parser + function, used only on platforms without a native version. -- build(deps): bump actions/download-artifact from 3 to 4 + Reported-by: vulnerabilityspotter on hackerone + Closes #12990 - Bumps [actions/download-artifact](https://github.com/actions/download-artifac - t) from 3 to 4. - - [Release notes](https://github.com/actions/download-artifact/releases) - - [Commits](https://github.com/actions/download-artifact/compare/v3...v4) +- libssh/libssh2: return error on too big range - --- - updated-dependencies: - - dependency-name: actions/download-artifact - dependency-type: direct:production - update-type: version-update:semver-major - ... + If trying to get the range 0 - 2^63 and the remote file is 2^63 bytes or + larger. - Signed-off-by: dependabot[bot] + Fixes #12983 + Closes #12984 - Closes #12626 +Scott Talbert (24 Feb 2024) -Stefan Eissing (3 Jan 2024) +- setopt: fix check for CURLOPT_PROXY_TLSAUTH_TYPE value -- http3/quiche: fix result code on a stream reset + Prior to this change CURLOPT_PROXY_TLSAUTH_TYPE would return + CURLE_BAD_FUNCTION_ARGUMENT on any type other than NULL. Since there is + only one type of TLS auth and it is also the default (SRP) the TLS auth + would work anyway. - - fixes pytest failures in test 07_22 - - aligns CURLcode values on stream reset with ngtcp2 + Closes https://github.com/curl/curl/pull/12981 - Closes #12629 +Jay Satiro (24 Feb 2024) -Daniel Stenberg (2 Jan 2024) +- mprintf: fix format prefix I32/I64 for windows compilers -- setopt: clear mimepost when formp is freed + - Support I32 & I64 (eg: %I64d) for all Win32 builds. - A precaution to avoid a possibly dangling pointer left behind. + Prior to this change mprintf support for the I format prefix, which is a + Microsoft extension, was dependent on the compiler used. - Reported-by: Thomas Ferguson - Fixes #12608 - Closes #12621 + When Borland compiler support was removed in fd7ef00f the prefix was + then no longer supported for that compiler; however since it's still + possible to build with Borland I'm restoring support for the prefix in + this way. -Andy Alt (2 Jan 2024) + Reported-by: Paweł Witas -- CI: Add dependabot.yml + Fixes https://github.com/curl/curl/issues/12944 + Closes https://github.com/curl/curl/pull/12950 - This will cause dependabot to open a PR when various actions are - updated, provided that the action maintainer has issued a release. +Daniel Stenberg (23 Feb 2024) - Closes #12623 +- cd2nroff: gen: make `\>` in input to render as plain '>' in output -Gisle Vanem (2 Jan 2024) + The same (copy and pasted) fix/mistake as in gen.pl -- content_encoding: change return code to typedef'ed enum +- gen: make `\>` in input to render as plain '>' in output - ... to work around a clang ubsan warning. + Reported-by: Gisle Vanem + Fixes #12977 + Closes #12978 - Fixes #12618 - Closes #12622 +Fabrice Fontaine (23 Feb 2024) -Daniel Stenberg (2 Jan 2024) +- configure.ac: find libpsl with pkg-config -- tool: prepend output_dir in header callback + Find libpsl with pkg-config to avoid static build failures. - When Content-Disposition parsing is used and an output dir is prepended, - make sure to store that new file name correctly so that it can be used - for setting the file timestamp when --remote-time is used. + Ref: http://autobuild.buildroot.org/results/1fb15e1a99472c403d0d3b1a688902f32 + e78d002 - Extended test 3012 to verify. + Signed-off-by: Fabrice Fontaine + Closes #12947 - Co-Authored-by: Jay Satiro - Reported-by: hgdagon on github - Fixes #12614 - Closes #12617 +Daniel Stenberg (23 Feb 2024) -- test1254: fix typo in name plus shorten it +- BUG-BOUNTY.md: clarify that the curl security team decides -- RELEASE-NOTES: synced + Closes #12975 -Viktor Szakats (2 Jan 2024) +- THANKS: add bug reporter from #740 -- schannel: fix `-Warith-conversion` gcc 13 warning + Ref: https://github.com/curl/curl/issues/740 - ``` - lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' - may change the sign of the result [-Warith-conversion] - 1201 | *extension_len = *list_len + - | ^ - ``` +Stefan Eissing (22 Feb 2024) - Closes #12616 +- multi: fix multi_sock handling of select_bits -- asyn-thread: silence `-Wcast-align` warning for Windows + - OR the event bitmask to data->state.select_bits instead of overwriting + them. They are cleared again on use. - Seen with llvm/clang 17: - ``` - lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct - thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align] - 310 | CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlap - ped); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~ - .../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded fro - m macro 'CONTAINING_RECORD' - 717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(addre - ss) - (ULONG_PTR)(&((type *)0)->field))) - | ^~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ``` + Reported-by: 5533asdg on github + Fixes #12971 + Closes #12972 - Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482 +Daniel Stenberg (22 Feb 2024) - Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261 - Closes #12615 +- curlver: bump to 8.7.0 for next release -Daniel Stenberg (2 Jan 2024) +- RELEASE-NOTES: synced -- tool_listhelp: regenerate after recent .d updates +- write-out: add '%{proxy_used}' - Makes it survive test 1478 + Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to + for example determine if a `NOPROXY` pattern matched the hostname or + not. - Closes #12612 + Extended test 970 and 972 -- test1478: verify src/tool_listhelp.c +- CURLINFO_USED_PROXY: return bool whether the proxy was used - Verify that the source file on disk is identical to the output of gen.pl - listhelp, as otherwise they are out of sync and need attention. + Adds test536 to verify - Closes #12612 + Closes #12719 -- testutil: make runtests support %include +- sha512_256: remove the cast macro, minor language/format edits - Using this instruction, a test case can include the contents of a file - into the test during the preprocessing. + Follow-up to cbe41d151d6a100c - Closes #12612 + Closes #12966 -- runtests: for mode="text" on , fix newlines on both parts +Stefan Eissing (20 Feb 2024) - Closes #12612 +- DoH: add trace configuration -Jay Satiro (2 Jan 2024) + - refs #12397 where it is dicussed how to en-/disable verbose output + of DoH operations + - introducing `struct curl_trc_feat` to track a curl feature for + tracing + - adding `data->state.feat` optionally pointing to the feature a + transfer belongs to + - adding trace functions and verbosity checks on features + - using trace feature in DoH code + - documenting `doh` as feature for `--trace-config` -- quiche: return CURLE_HTTP3 on send to invalid stream + Closes #12411 - Prior to this change if a send failed on a stream in an invalid state - (according to quiche) and not marked as closed (according to libcurl) - then the send function would return CURLE_SEND_ERROR. +- websocket: fix curl_ws_recv() - We already have similar code for ngtcp2 to return CURLE_HTTP3 in this - case. + - when data arrived in several chunks, the collection into + the passed buffer always started at offset 0, overwriting + the data already there. - Caught by test test_07_upload.py: test_07_22_upload_parallel_fail. + adding test_20_07 to verify fix - Fixes https://github.com/curl/curl/issues/12590 - Closes https://github.com/curl/curl/pull/12597 + - debug environment var CURL_WS_CHUNK_SIZE can be used to + influence the buffer chunk size used for en-/decoding. -Daniel Stenberg (1 Jan 2024) + Closes #12945 -- cmdline-opts: update availability for the *-ca-native options +Evgeny Grin (Karlson2k) (20 Feb 2024) - Closes #12613 +- digest: support SHA-512/256 -Patrick Monnerat (31 Dec 2023) + Also fix the tests. New implementation tested with GNU libmicrohttpd. + The new numbers in tests are real SHA-512/256 numbers (not just some + random ;) numbers ). -- openldap: fix STARTTLS +- tests: add SHA-512/256 unit test - It was not working anymore since introduction of connection filters. +- SHA-512/256: implement hash algorithm - Also do not attempt to recover from a failing TLS negotiation with - CURLUSESSL_TRY. + Closes #12897 - Closes #12610 +- curl_setup.h: add curl_uint64_t internal type -Daniel Stenberg (31 Dec 2023) + The unsigned version of curl_off_t basically -- haproxy-clientip.d: document the arg +Daniel Stenberg (20 Feb 2024) - The arg keyword was missing and therefore not present in the man page. +- docs: dist curl*.1 and install without perl - Closes #12611 + Drop docs/mk-ca-bundle.1 from the tarball. It can be generated at will. -annalee (29 Dec 2023) + Closes #12959 + Fixes #12921 + Reported-by: Michael Forney -- configure: fix no default int compile error in ipv6 detection +Stefan Eissing (20 Feb 2024) - Closes #12607 +- OpenSSL QUIC: adapt to v3.3.x -Dan Fandrich (28 Dec 2023) + - set our idle timeout as transport parameter + - query negotiated idle timeout for connection alive checks + - query number of available bidi streams on a connection + - use write_ex2 with SSL_WRITE_FLAG_CONCLUDE to signal + EOF on last chunk write, so stream close does not + require an additional QUIC packet -- CI: Fix use of any-glob-to-all-files in the labeler + Closes #12933 - Despite its name, this atom acts like one-glob-to-all-files and a - different syntax with braces must be used to get - any-glob-to-all-files semantics. Unfortunately, this makes the file - completely unreadable. +Ramiro Garcia (19 Feb 2024) - Ref: https://github.com/actions/labeler/issues/731 +- MANUAL.md: fix typo -Daniel Stenberg (29 Dec 2023) + Closes #12965 -- CURLOPT_AUTOREFERER.3: mention CURLINFO_REFERER +Daniel Stenberg (19 Feb 2024) -- CURLINFO_REFERER.3: clarify that it is the *request* header +- BINDINGS: add mcurl, the python binding - That libcurl itself sent in the most recent request + Ref: #12956 + Closes #12962 - Closes #12605 +- mk-ca-bundle.md: cleanups and polish -Jay Satiro (28 Dec 2023) + Closes #12958 -- system_win32: fix a function pointer assignment warning +- spellcheck.yml: remove .1/.3 handling, clean all man page .md files - - Use CURLX_FUNCTION_CAST to suppress a function pointer assignment - warning. + Since we generate all .1 and .3 files from markdown now, we can limit + the spellcheck to the markdown versions only. - a6bbc87f added lookups of some Windows API functions and then cast them - like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn - about that as breaking strict-aliasing rules so this PR changes those - assignments to use CURLX_FUNCTION_CAST. + Closes #12960 - Bug: https://github.com/curl/curl/pull/12581#issuecomment-1869804317 - Reported-by: Marcel Raad +- libcurl-docs: cleanups - Closes https://github.com/curl/curl/pull/12602 + CURLMOPT_SOCKETDATA.md: fix typo + CURLMOPT_TIMERDATA.md: fix typo + CURLOPT_COOKIELIST.m: quote strings + CURLOPT_PREREQFUNCTION.md: quote variable names + CURLOPT_TCP_NODELAY.md: rephrased to please spell checker + CURLOPT_WILDCARDMATCH.md: rephrased + libcurl-tutorial.md: use correct option name + curl_global_init_mem.md: quote headers + curl_easy_getinfo.md: use correct symbol names in headers + curl_global_trace.md: quote some headers + curl_ws_meta.md: quote struct field names + libcurl-env.md: quote headers -- verify-examples.pl: fail verification on unescaped backslash +- cd2nroff: remove backticks from titles - - Check that all backslashes in EXAMPLE are properly escaped. +- RELEASE-NOTES: synced - eg manpage must always use `\\n` never `\n`. +Stefan Eissing (18 Feb 2024) - This is because the manpage requires we always double blackslash to show - a single backslash. Prior to this change an erroneous single backslash - would pass through and compile even though it would not show correctly - in the manpage. +- http_chunks: fix the accounting of consumed bytes - Co-authored-by: Daniel Stenberg + Prior to this change chunks were handled correctly although in verbose + mode libcurl could incorrectly warn of "Leftovers after chunking" even + if there were none. - Ref: https://github.com/curl/curl/pull/12588 + Reported-by: Michael Kaufmann - Closes https://github.com/curl/curl/pull/12589 + Fixes https://github.com/curl/curl/issues/12937 + Closes https://github.com/curl/curl/pull/12939 -- vtls: fix missing multissl version info +- file: use xfer buf for file:// transfers - - Fix erroneous buffer copy logic from ff74cef5. + - For file:// transfers use the multi handle's transfer buffer for + up- and downloads. - Prior to this change the MultiSSL version info returned to the user - was empty. + Prior to this change a6c9a33 (precedes 8.6.0) changed the file:// + transfers to use a smaller stack based buffer, and that caused a + significant performance decrease in Windows. - Closes https://github.com/curl/curl/pull/12599 + Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086 + Reported-by: edmcln@users.noreply.github.com -Daniel Stenberg (27 Dec 2023) + Closes https://github.com/curl/curl/pull/12932 -- KNOWN_BUGS: [RTSP] Some methods do not support response bodies +Karthikdasari0423 (18 Feb 2024) - Closes #12414 +- HTTP3.md: always run nghttp3 submodule init -Patrick Monnerat (27 Dec 2023) + - For consistency change all 'build nghttp3' commands to run submodule + init after cloning, even if the branch does not have submodules. -- openldap: fix an LDAP crash + Follow-up to 5a4b2f93 and 4f794558. - Reported-by: Ozan Cansel - Fixes #12593 - Closes #12600 + Closes https://github.com/curl/curl/pull/12928 -Daniel Stenberg (27 Dec 2023) +LeeRiva (18 Feb 2024) -- getinfo: CURLINFO_QUEUE_TIME_T +- CURLOPT_POSTQUOTE.md: fix typo - Returns the time, in microseconds, during which this transfer was held - in a waiting queue before it started "for real". A transfer might be put - in a queue if after getting started, it cannot create a new connection - etc due to set conditions and limits imposed by the application. + Closes https://github.com/curl/curl/pull/12926 - Ref: #12293 - Closes #12368 +Evgeny Grin (Karlson2k) (18 Feb 2024) -- RELEASE-NOTES: synced +- checksrc.pl: fix handling .checksrc with CRLF -Jay Satiro (26 Dec 2023) + - When parsing .checksrc chomp the (CR)LF line ending. -- examples/sendrecv: fix comment line length + Prior to this change on Windows checksrc.pl would not process the + symbols in .checksrc properly, since many git repos in Windows use auto + crlf to check out files with CRLF line endings. - Caught by checksrc. + Closes https://github.com/curl/curl/pull/12924 -Haydar Alaidrus (23 Dec 2023) +Richard Levitte (18 Feb 2024) -- CURLOPT_POSTFIELDS.3: fix incorrect C string escape in example +- cmake: fix install for older CMake versions - - Escape inner quotes with two backslashes. + - Generate the docs install list by using a foreach loop instead of + LIST:TRANSFORM since older CMake can't handle the latter. - Two backslashes escapes the backslash for the man page and will show as - a single backslash. + Reported-by: Dan Fandrich - eg: "{\\"name\\": \\"daniel\\"}" shows as "{\"name\": \"daniel\"}". + Fixes https://github.com/curl/curl/issues/12920 + Closes https://github.com/curl/curl/pull/12922 - Closes https://github.com/curl/curl/pull/12588 +Stefan Eissing (16 Feb 2024) -Viktor Szakats (23 Dec 2023) +- vtls: fix tls proxy peer verification -- appveyor: tidy-ups + - When verifying a proxy certificate for an ip address, use the correct + ip family. - - replace two remaining backslashes with forward slashes. - - tidy up the way we form and pass `TFLAGS`. + Prior to this change the "connection" ip family was used, which was not + necessarily the same. - Follow-up to 2d4d0c1fd32f5cc3f946c407c8eccd5477b287df #12572 + Reported-by: HsiehYuho@users.noreply.github.com - Closes #12582 + Fixes https://github.com/curl/curl/issues/12831 + Closes https://github.com/curl/curl/pull/12931 -Stefan Eissing (22 Dec 2023) +Dan Fandrich (15 Feb 2024) -- transfer: fix upload rate limiting, add test cases +- CI: Bump the Circle CI base Ubuntu image to the latest 20.04 - - add test cases for rate limiting uploads for all - http versions - - fix transfer loop handling of limits. Signal a re-receive - attempt only on exhausting maxloops without an EAGAIN - - fix `data->state.selectbits` forcing re-receive to also - set re-sending when transfer is doing this. + The previous ones are going to be removed soon, plus the new ones + include all the fixes since then. - Reported-by: Karthikdasari0423 on github - Fixes #12559 - Closes #12586 +Jay Satiro (13 Feb 2024) -Daniel Stenberg (22 Dec 2023) +- transfer: improve Windows SO_SNDBUF update limit -- mbedtls: free the entropy when threaded + - Change the 1 second SO_SNDBUF update limit from per transfer to per + connection. - The entropy_free was never done for threaded builds, causing a small - (fixed) memory leak. + Prior to this change many transfers over the same connection could cause + many SO_SNDBUF updates made to that connection per second, which was + unnecessary. - Reported-by: RevaliQaQ on github - Fixes #12584 - Closes #12585 + Closes https://github.com/curl/curl/pull/12911 -Stefan Eissing (22 Dec 2023) +- schannel: fix hang on unexpected server close -- http2: improved on_stream_close/data_done handling + - Treat TLS connection close (either due to a close_notify from the + server or just closed due to receiving 0) as pending data. - - there seems to be a code path that cleans up easy handles without - triggering DONE or DETACH events to the connection filters. This - would explain wh nghttp2 still holds stream user data - - add GOOD check to easy handle used in on_close_callback to - prevent crashes, ASSERTs in debug builds. - - NULL the stream user data early before submitting RST - - add checks in on_stream_close() to identify UNGOOD easy handles + This is because in some cases schannel_recv knows the connection is + closed but has to return actual pending data so it can't return 0 or an + error to indicate no more data. In this case schannel_recv must be + called again, which only happens if readwrite_data sees that there is + still pending data. - Reported-by: Hans-Christian Egtvedt - Fixes #10936 - Closes #12562 + Prior to this change if the total size of the body that libcurl expected + to receive from the server was unknown then it was possible under some + network conditions that libcurl would hang waiting to receive more data, + when in fact a close_notify alert indicating no more data would be sent + was already processed. -Daniel Stenberg (22 Dec 2023) + Fixes https://github.com/curl/curl/issues/12894 + Closes https://github.com/curl/curl/pull/12910 -- mprintf: overhaul and bugfixes +Daniel Stenberg (10 Feb 2024) - In a test case using lots of snprintf() calls using many commonly used - %-codes per call, this version is around 30% faster than previous - version. +- KNOWN_BUGS: FTP upload fails if remebered dir is deleted - It also fixes the #12561 bug which made it not behave correctly when - given unknown %-sequences. Fixing that flaw required a different take on - the problem, which resulted in the new two-arrays model. + Closes #12181 + Closes #12923 - lib557: extended - Verify the #12561 fix and test more printf features +Michał Antoniak (10 Feb 2024) - unit1398: fix test: It used a $ only for one argument, which is not - supported. +- mbedtls: use mbedtls_ssl_conf_{min|max}_tls_version - Fixes #12561 - Closes #12563 + ... instead of the deprecated mbedtls_ssl_conf_{min|max}_version -Viktor Szakats (21 Dec 2023) + Closes #12905 -- appveyor: replace PowerShell with bash + parallel autotools +Dan Fandrich (9 Feb 2024) - PowerShell works (after a steep development curve), but one property of - it stuck and kept causing unresolvable usability issues: With - `$ErrorActionPreference=Stop`, it does abort on failures, but shows only - the first line of the error message. In `Continue` mode, it shows the - full error message, but doesn't stop on all errors. Another issue is - PowerShell considering any stderr output as if the command failed (this - has been improved in 7.2 (2021-Nov), but fixed versions aren't running - in CI and will not be for a long time in all test images.) +- CI: bump to actions/cache@v4 to avoid warning - Thus, we're going with bash. +Evgeny Grin (Karlson2k) (9 Feb 2024) - Also: - - use `-j2` with autotools tests, making them finish 5-15 minutes per - job faster. - - omit `POSIX_PATH_PREFIX`. - - use `WINDIR`. - - prefer forward slashes. +- test1165: improve pattern matching - Follow-up to: 75078a415d9c769419aed4153d3d525a8eba95af #11999 - Ref: #12444 + * Fix excluded digits at the end of the symbols ('CURL_DISABLE_POP3' + was checked as 'CURL_DISABLE_POP') - Fixes #12560 - Closes #12572 + Closes #12903 -Pavel Pavlov (21 Dec 2023) +Dan Fandrich (9 Feb 2024) -- asyn-thread: use GetAddrInfoExW on >= Windows 8 +- scripts: Fix cijobs.pl for Azure and GHA - For doing async DNS resolution instead of starting a thread for each - request. + The spacing in the yaml files changed. - Fixes #12481 - Closes #12482 +Daniel Stenberg (9 Feb 2024) -Daniel Stenberg (21 Dec 2023) +- RELEASE-NOTES: synced -- strerror: repair get_winsock_error() +- TODO: use pkg-config to find libpsl - It would try to read longer than the provided string and crash. + Closes #12919 - Follow-up to ff74cef5d4a0cf60106517a1c7384 - Reported-by: calvin2021y on github - Fixes #12578 - Closes #12579 +- TODO: avoid nroff -- CURLOPT_SSH_*_KEYFILE: clarify + Instead of adjusting roffit, skip the nroff step. - Closes #12554 + Closes #12919 -ivanfywang (21 Dec 2023) +Dan Fandrich (9 Feb 2024) -- ngtcp2: put h3 at the front of alpn +- Revert "CI: run Circle macOS builds on x86 for now" - Closes #12576 + This reverts commit 2683de3078eadc86d9b182e7417f4ee75a247e2c. + ARM resources are now available in Circle CI, so run these builds on ARM + again. This platform needs explicit paths set to libpsl and its + dependency icu4c. -Daniel Stenberg (21 Dec 2023) + Follow-up to 2683de30 -- test460: verify a command line using --expand with no argument + Closes #12635 - This verifies the fix for #12565 +Viktor Szakats (9 Feb 2024) -- tool_getparam: do not try to expand without an argument +- cmake: add warning for using TLS libraries without 1.3 support - This would lead to a segfault. + Closes #12900 - Fixes #12565 - Reported-by: Geeknik Labs - Closes #12575 +Daniel Stenberg (9 Feb 2024) -- RELEASE-NOTES: synced +- configure: add warning for using TLS libraries without 1.3 support - Bumped version to 8.6.0 because of changes + Closes #12900 -- Makefile.am: fix the MSVC project generation +Michał Antoniak (9 Feb 2024) - It made the vcxproj files not get included in dist tarballs. +- mbedtls: fix building when MBEDTLS_X509_REMOVE_INFO flag is defined - Regression since 74423b5df4c8117891eb89 (8.5.0) + Closes #12904 - Reported-by: iAroc on github - Fixes #12564 - Closes #12567 +Stefan Eissing (9 Feb 2024) -zengwei2000 (21 Dec 2023) +- ftp: fix socket wait activity in ftp_domore_getsock -- altsvc: free 'as' when returning error + - when waiting on the data connection, always add the control socket to + the pollset on state STOP or let the pingpong add the socket according + to its needs. - Closes #12570 + Reported-by: Fabian Vogt + Fixes #12901 + Closes #12913 - Signed-off-by: zengwei +Daniel Stenberg (9 Feb 2024) -Viktor Szakats (20 Dec 2023) +- dist: make sure the http tests are in the tarball -- build: fix `-Wconversion`/`-Wsign-conversion` warnings + Fixes #12914 + Reported-by: Fabian Vogt + Closes #12917 - Fix remaining warnings in examples and tests which are not suppressed - by the pragma in `lib/curl_setup.h`. +Stefan Eissing (9 Feb 2024) - Silence a toolchain issue causing warnings in `FD_SET()` calls with - older Cygwin/MSYS2 builds. Likely fixed on 2020-08-03 by: - https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=5717262b8ecfed0f7f - ab63e2c09c78991e36f9dd +- multi: add xfer_buf to multi handle - Follow-up to 2dbe75bd7f3c36837aa06fd87a442bdf3fb7faef #12492 + - can be borrowed by transfer during recv-write operation + - needs to be released before borrowing again + - adjustis size to `data->set.buffer_size` + - used in transfer.c readwrite_data() - Closes #12557 + Closes #12805 -- build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings +Daniel Stenberg (9 Feb 2024) - - enable `-Wsign-conversion` warnings, but also setting them to not - raise errors. - - fix `-Warith-conversion` warnings seen in CI. - These are triggered by `-Wsign-converion` and causing errors unless - explicitly silenced. It makes more sense to fix them, there just a few - of them. - - fix some `-Wsign-conversion` warnings. - - hide `-Wsign-conversion` warnings with a `#pragma`. - - add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build - basis. - - update a CI job to unhide them with the above macro: - https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3 +- write-out.md: clarify error handling details - Closes #12492 + - it gets used even if the transfer fails -- cmake: tidy-up `OtherTests.cmake` + - it does not cause error to be returned even if it fails - - make more obvious which detection uses which prep steps. - - merge and streamline conditions. - - these should not alter detection results. + Closes #12909 - Also align log output messages from - `Macros.cmake` / `curl_internal_test` with rest of the build. +Stefan Eissing (8 Feb 2024) - Closes #12551 +- ftp: do lineend conversions in client writer -- appveyor: switch to out-of-tree builds + - remove the ftp special handling from sendf.c + - let ftp_do() add a client writer that does + the linened conversions + - change the lineend conversion to no longer + modify the passed buffer, but write smaller + chunks to the next cwriter instead. The + inefficiency of this will be mitigated once + we add output buffering for all client writes. - With cmake and autotools. + Closes #12878 - Closes #12550 +- ftp: tracing improvements -Daniel Stenberg (19 Dec 2023) + - trace socketindex for connection filters when not the first + - trace socket fd in tcp + - trace pollset adjusts in vtls -- DEPRECATE.md: mention that NTLM_WB no longer works + Closes #12902 - Ref: #12479 - Closes #12553 +Karthikdasari0423 (8 Feb 2024) -- CURLOPT_SERVER_RESPONSE_TIMEOUT_MS: add +- HTTP3.md: adjust the OpenSSL QUIC install instructions - Proposed-by: Yifei Kong - Ref: https://curl.se/mail/lib-2023-11/0023.html - Closes #12369 + tried installing with old steps but failed + tried with newly added setps and able to build + ``` + root@ubuntu:~/curl# ./src/curl -V + /root/curl/src/.libs/curl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSS + L_3.2.0' not found (required by /root/curl/lib/.libs/libcurl.so.4) + root@ubuntu:~/curl# + ``` + ``` + root@ubuntu:~/curl# ./src/curl -V + curl 8.6.1-DEV (x86_64-pc-linux-gnu) libcurl/8.6.1-DEV OpenSSL/3.2.0 zlib/1.2 + .11 brotli/1.0.9 libpsl/0.21.0 nghttp3/1.1.0 OpenLDAP/2.5.16 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp + Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz + NTLM PSL SSL threadsafe TLS-SRP UnixSockets + root@ubuntu:~/curl# + ``` -Viktor Szakats (18 Dec 2023) + Closes #12896 -- build: more `-Wformat` fixes +Daniel Stenberg (8 Feb 2024) - - memdebug: update to not trigger `-Wformat-nonliteral` warnings. - - imap: mark `imap_sendf()` with `CURL_PRINTF()`. - - tool_msgs: mark static function with `CURL_PRINTF()`. +- TODO: align the TOC with the header - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 +- docs: make sure curl.1 is included in dist tarballs - Closes #12540 + Ref: https://github.com/curl/curl/issues/12832#issuecomment-1933271873 -- windows: delete redundant headers + Closes #12892 - `winsock2.h` pulls in `windows.h`. `ws2tcpip.h` pulls in `winsock2.h`. - `winsock2.h` and `ws2tcpip.h` are also pulled by `curl/curl.h`. +Karthikdasari0423 (8 Feb 2024) - Keep only those headers that are not already included, or the code under - it uses something from that specific header. +- HTTP3.md: remove quiche word in Openssl 3.2 - Closes #12539 + Closes #12893 -- cmake: prefill/cache `HAVE_STRUCT_SOCKADDR_STORAGE` +Daniel Stenberg (7 Feb 2024) - Also add missing include to `OtherTests.cmake`. It didn't cause an issue - because the parent already included this earlier by chance. +- curl: when allocating variables, add the name into the struct - Closes #12537 + This saves the name from being an extra separate allocation. -Daniel Stenberg (18 Dec 2023) + Closes #12891 -- runner.pm: fix perl warning when running tests +- lib582: remove code causing warning that is never run - Use of uninitialized value $runner::gdbthis in numeric eq (==) at runner. - pm + The previous realloc code in this code could trigger a compiler warning, + but since that code path cannot happen in normal circumstances it now + instead exits with an error message there. - Follow-up from 3dcf301752a09d9 + Ref: #12887 + Closes #12890 - Closes #12549 +Stefan Eissing (7 Feb 2024) -- runtests: support -gl. Like -g but for lldb. +- vtls: revert "receive max buffer" + add test case - Follow-up to 63b5748 + - add test_05_04 for requests using http/1.0, http/1.1 and h2 against an + Apache resource that does an unclean TLS shutdown. + - revert special workarund in openssl.c for suppressing shutdown errors + on multiplexed connections + - vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53 - Invokes the test case via lldb instead of gdb. Since using gdb is such a - pain on mac, using lldb is sometimes less quirky. + Fixes #12885 + Fixes #12844 - Closes #12547 + Closes #12848 -- curl.h: add CURLE_TOO_LARGE +Daniel Stenberg (7 Feb 2024) - A new error code to be used when an internal field grows too large, like - when a dynbuf reaches its maximum. Previously it would return - CURLE_OUT_OF_MEMORY for this, which is highly misleading. +- tests: support setting/using blank content env variables - Ref: #12268 - Closes #12269 + - test450: remove --config from the keywords + - test2080: change return code + - test428: add --config as a keyword + - test428: disable on Windows due to CI problems -- CI/circleci: disable MQTT in the HTTP-only build +- curl: exit on config file parser errors - And remove the use of configure options that don't actually exist + Like when trying to import an environment variable that does not exist. - Closes #12546 + Also fix a bug for reading env variables when there is a default value + set. -Yedaya Katsman (18 Dec 2023) + Bug: https://curl.se/mail/archive-2024-02/0008.html + Reported-by: Brett Buddin -- tests: respect $TMPDIR when creating unix domain sockets + Add test 462 to verify. - When running on termux, where $TMPDIR isn't /tmp, running the tests - failed, since the server config tried creating sockets in /tmp, without - checking the temp dir config. Use the TMPDIR variable that makes it find - the correct directory everywhere [0] + Closes #12862 - [0] https://perldoc.perl.org/File::Temp#tempfile +Daniel Szmulewicz (7 Feb 2024) - Closes #12545 +- CURLOPT_WRITEFUNCTION.md: typo fix -Viktor Szakats (17 Dec 2023) + The maximum amount of body data that is passed to the write + callback is defined in the curl.h header file -- ssh: fix namespace of two local macros + Closes #12889 - Avoid using the libssh and libssh2 macro namespaces by prefixing - these local macro names with `CURL_`. +Daniel Stenberg (7 Feb 2024) - Follow-up to 413a0fedd02c8c6df1d294534b8c6e306fcca7a2 #12346 +- lib: convert Curl_get_line to use dynbuf - Reviewed-by: Daniel Stenberg - Closes #12544 + Create the line in a dynbuf. Aborts the reading of the file on + errors. Avoids having to always allocate maximum amount from the + start. Avoids direct malloc. -- cmake: whitespace tidy-up in `OtherTests.cmake` + Closes #12846 - Closes #12538 +- KNOWN_BUGS: unicode on Windows -Mark Sinkovics (16 Dec 2023) + Closes #11461 + Closes #12231 + Closes #12883 -- cmake: fix generation for system name iOS +- tool_operate: change precedence of server Retry-After time - This PR fixes a problem that happens during CMake configuration when - the `CMAKE_SYSTEM_NAME` set to `iOS` and not `Darwin`. This value is - available (as far as I remember) version 3.14. The final solution - (thanks to @vszakats) is to use `APPLE` which contains all the Apple - platforms https://cmake.org/cmake/help/latest/variable/APPLE.html. + - When calculating the retry time, no longer allow a server's requested + Retry-After time to take precedence over a longer retry time (either + default algorithmic or user-specified). - This issue was found when during vcpkg installation. Running command - `vcpkg install curl:arm64-ios` and `vcpkg install curl:x64-ios` failed - with message: - ``` - CMake Error: try_run() invoked in cross-compiling mode, please set the follow - ing cache variables appropriately: - HAVE_H_ERRNO_ASSIGNABLE_EXITCODE (advanced) - ``` - After this fix, I was able to compile the compile the binary without - any issue. + Prior to this change the server's Retry-After time took precedence over + curl's retry time in all cases, but that's not always practical for + short Retry-After times depending on how busy the server is. - In addition to that fix, this PR also contains an simplification to - check if the platform is not APPLE. + Bug: https://curl.se/mail/archive-2024-01/0022.html + Reported-by: Dirk Hünniger - Co-authored-by: Viktor Szakats - Closes #12515 + Closes https://github.com/curl/curl/pull/12871 -Daniel Stenberg (16 Dec 2023) +- cmdline-docs: quote and angle bracket cleanup -- RELEASE-NOTES: synced + - make sure angle brackets are escaped + - remove a lot of superfluous double quotes + - replace several double quotes with backticks -Baruch Siach (16 Dec 2023) + To make nicer-looking markdown. -- gnutls: fix build with --disable-verbose + Closes #12884 - infof() parameters must be defined event with --disable-verbose since - commit dac293cfb702 ("lib: apache style infof and trace - macros/functions"). +- badwords: use hostname, not host name - Move also 'ptr' definition under !CURL_DISABLE_VERBOSE_STRINGS. + and username, filename - consistently. Fixed the patterns in + badwords.txt to catch these. - Fixes the following build failure: + Closes #12888 - In file included from ../lib/sendf.h:29, - from vtls/gtls.c:44: - vtls/gtls.c: In function 'Curl_gtls_verifyserver': - vtls/gtls.c:841:34: error: 'version' undeclared (first use in this function); - did you mean 'session'? - 841 | gnutls_protocol_get_name(version), ptr); - | ^~~~~~~ +Viktor Szakats (6 Feb 2024) - Closes #12505 +- cmake: fix function description in comment [ci skip] -Viktor Szakats (16 Dec 2023) + Closes #12879 -- build: delete unused `HAVE_{GSSHEIMDAL,GSSMIT,HEIMDAL}` +Daniel Stenberg (6 Feb 2024) - Stop setting `HAVE_GSSHEIMDAL`, `HAVE_GSSMIT` and `HAVE_HEIMDAL`. - There was no place in the build system or source code that used them. +- header.md: remove backslash, make nicer markdown - Reviewed-by: Daniel Stenberg - Closes #12506 + - remove a leftover backslash before a dash + - use backticks for "code" strings -- build: remove redundant `CURL_PULL_*` settings + Closes #12877 - These macros were not propagated to the source code from CMake. +- docs: add mk-ca-bundle.1 to dist - autotools set only one of them (`CURL_PULL_SYS_POLL_H`), initially to - address an AIX issue [1]. This later broke when introducing `system.h` - [2] without the logic it enabled. A subsequent fix [3] re-added the - logic, and also enabled it for AIX before its use, directly in - `system.h`. + ... which also makes it get built. But don't build this or curl-config.1 + if build docs is disabled. - [1] 2012-11-23: 665adcd4b7bcdb7deb638cdc499fbe71f8d777f2 - [2] 2017-03-29: 9506d01ee50d5908138ebad0fd9fbd39b66bd64d #1373 - [3] 2017-08-25: 8a84fcc4b59e8b78d2acc6febf44a43d6bc81b59 #1828 #1833 + Closes #12875 - Reviewed-by: Daniel Stenberg - Closes #12502 +Stefan Eissing (6 Feb 2024) -- system.h: sync mingw `CURL_TYPEOF_CURL_SOCKLEN_T` with other compilers +- https-proxy: use IP address and cert with ip in alt names - Align mingw with the other Windows compilers and use the `int` type for - `CURL_TYPEOF_CURL_SOCKLEN_T` (and thus for `curl_socklent_t`). This - makes it unnecessary to make a mingw-specific trick and pull all Windows - headers early just for this type definition. This type is specific to - Windows, not to the compiler. mingw-w64's Windows header maps it to - `int` too. + - improve info logging when peer verification fails to indicate + if DNS name or ip address has been tried to match + - add test case for contacting https proxy with ip address + - add pytest env check on loaded credentials and re-issue + when they are no longer valid + - disable proxy ip address test for bearssl, since not supported there - With this we also delete all remaining uses of `CURL_PULL_WS2TCPIP_H`. + Ref: #12831 + Closes #12838 - [ The official solution is to use `socklen_t` for all Windows compilers. - In this case we may want to update `curl/curl.h` to pull in Windows - headers before `system.h`. ] +Jiawen Geng (6 Feb 2024) - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12501 +- docs: add necessary setup for nghttp3 -- windows: simplify detecting and using system headers + Now nghttp3 has submodules + https://github.com/ngtcp2/nghttp3/blob/main/.gitmodules - - autotools, cmake: assume that if we detect Windows, `windows.h`, - `winsock2.h` and `ws2tcpip.h` do exist. - - lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of - looking for `winsock2.h`. - - autotools: merge 3 Windows check methods into one. - - move Watt-32 and lwIP socket support to `setup-win32.h` from - `config-win32.h`. It opens up using these with all build tools. Also - merge logic with Windows Sockets. - - fix to assume Windows sockets with the mingw32ce toolchain. - Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4 - - cmake: delete unused variable `signature_call_conv` since - eb33ccd5332435fa50f1758e5debb869c6942b7f. - - autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection. - - examples/externalsocket: fix header order. - - cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue` - that wasn't used anymore. - - cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test - `SIZEOF_STRUCT_SOCKADDR_STORAGE`. + Closes #12859 - After this patch curl universally uses `_WIN32` to guard - Windows-specific logic. It guards Windows Sockets-specific logic with - `USE_WINSOCK` (this might need further work). +Peter Krefting (6 Feb 2024) - Reviewed-by: Jay Satiro - Closes #12495 +- version: allow building with ancient libpsl -- build: enable missing OpenSSF-recommended warnings, with fixes + The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7 + ships with version 0.7.0 which lacks this API. Revert to using the older + versioning API if we detect an old libpsl version. - https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening - -Guide-for-C-and-C++.html - as of 2023-11-29 [1]. + Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4 + Bug: https://curl.se/mail/archive-2024-02/0004.html + Reported-by: Scott Mutter + Closes #12872 - Enable new recommended warnings (except `-Wsign-conversion`): +Daniel Stenberg (6 Feb 2024) - - enable `-Wformat=2` for clang (in both cmake and autotools). - - add `CURL_PRINTF()` internal attribute and mark functions accepting - printf arguments with it. This is a copy of existing - `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible - with redefinting the `printf` symbol: - https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 - - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for - mingw-w64 and enable it on this platform. - - enable `-Wimplicit-fallthrough`. - - enable `-Wtrampolines`. - - add `-Wsign-conversion` commented with a FIXME. - - cmake: enable `-pedantic-errors` the way we do it with autotools. - Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 - - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format - checks. Previously it was always disabled due to the internal `printf` - macro. +- TODO: Support latest rustls - Fix them: + Closes #12737 + Closes #12874 - - fix bug where an `set_ipv6_v6only()` call was missed in builds with - `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. - - add internal `FALLTHROUGH()` macro. - - replace obsolete fall-through comments with `FALLTHROUGH()`. - - fix fallthrough markups: Delete redundant ones (showing up as - warnings in most cases). Add missing ones. Fix indentation. - - silence `-Wformat-nonliteral` warnings with llvm/clang. - - fix one `-Wformat-nonliteral` warning. - - fix new `-Wformat` and `-Wformat-security` warnings. - - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its - definition to `lib/curl_setup.h` allowing use in `tests/server`. - - lib: fix two wrongly passed string arguments in log outputs. - Co-authored-by: Jay Satiro - - fix new `-Wformat` warnings on mingw-w64. +- docs: make curldown do angle brackets like markdown - [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895b - fc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options- - Hardening-Guide-for-C-and-C%2B%2B.md + Make sure we use \< and \> in markdown all over so that it renders + correctly, on GitHub and elsewhere. cd2nroff now outputs a warning if it + finds an unescaled angle bracket. - Closes #12489 + Ref: #12854 + Closes #12869 -- Makefile.mk: drop Windows support +- docs: fix the --disable-docs for autotools - And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga. + Follow-up to 541321507e386 - We recommend CMake instead. With unity mode it's much faster, and about - the same without. + Closes #12870 - Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 - Reviewed-by: Daniel Stenberg - Closes #12224 +- RELEASE-NOTES: synced -Daniel Stenberg (16 Dec 2023) +- libcurl-security.md: Active FTP passes on the local IP address -- cmdline-docs: use .IP consistently + Reported-by: Harry Sintonen + Closes #12867 - Remove use of .TP and some .B. The idea is to reduce nroff syntax as - much as possible and to use it consistently. Ultimately, we should be - able to introduce our own easier-to-use-and-read syntax/formatting and - convert on generation time. +Stefan Eissing (5 Feb 2024) - Closes #12535 +- configure: do not link with nghttp3 unless necessary -Tatsuhiko Miyagawa (16 Dec 2023) + Fixes #12833 + Closes #12864 + Reported-by: Ryan Carsten Schmidt -- http: fix off-by-one error in request method length check +Daniel Stenberg (5 Feb 2024) - It should allow one more byte. +- THANKS: add Dmitry Tretyakov - Closes #12534 + ... since I missed to give credit to the report in the fix of #12861 -Daniel Stenberg (15 Dec 2023) +Stefan Eissing (5 Feb 2024) -- curl: show ipfs and ipns as supported "protocols" +- openssl-quic: check on Windows that socket conv to int is possible - They are accepted schemes in URLs passed to curl (the tool, not the - library). + Fixes #12861 + Closes #12865 - Also makes curl-config show the same list. +Daniel Stenberg (5 Feb 2024) - Co-Authored-by: Jay Satiro - Reported-by: Chara White - Bug: https://curl.se/mail/archive-2023-12/0026.html - Closes #12508 +- tool_cb_hdr: only parse etag + content-disposition for 2xx -- Revert "urldata: move async resolver state from easy handle to connectdata" + ... and ignore them for other response codes. - This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198) + Reported-by: Harry Sintonen + Closes #12866 - We want the c-ares channel to be held in the easy handle, not per - connection - for performance. +- md4: include strdup.h for the memdup proto - Closes #12524 + Reported-by: Erik Schnetter + Fixes #12849 + Closes #12863 -Viktor Szakats (15 Dec 2023) +Joel Depooter (5 Feb 2024) -- openssl: re-match LibreSSL deinit with init +- docs: add missing slashes to SChannel client certificate documentation - Earlier we switched to use modern initialization with LibreSSL v2.7.0 - and up, but did not touch deinitialization [1]. Fix it in this patch. + When setting the CURLOPT_SSLCERT option to a certificate thumprint, it + is required to have a backslash between the "store location", "store + name" and "thumbprint" tokens. These slashes were present in the + previous documentation, but were missed in the transition to markdown + documentation. - Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611 + Closes #12854 - [1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014 +Stefan Eissing (5 Feb 2024) - Reported-by: Mike Hommey - Reviewed-by: Daniel Stenberg - Fixes #12525 - Closes #12526 +- HTTP/2: write response directly -Daniel Stenberg (14 Dec 2023) + - use the new `Curl_xfer_write_resp()` to write incoming responses + directly to the client + - eliminates `stream->recvbuf` + - memory consumption on parallel transfers minimized -- libssh: supress warnings without version check + Closes #12828 - Define unconditionally. +Daniel Stenberg (5 Feb 2024) - Follow-up from d21bd2190c46ad7fa +- cookie.md: provide an example sending a fixed cookie - Closes #12523 + Closes #12868 -- hostip: return error immediately when Curl_ip2addr() fails +Lars Kellogg-Stedman (5 Feb 2024) - Closes #12522 +- ALTSVC.md: correct a typo -Theo (14 Dec 2023) + The ALPN documentation erroneously referred to a "host number" instead + of a "port number". -- libssh: improve the deprecation warning dismissal + Closes #12852 - Previous code was compiler dependant, and dismissed all deprecation warnings - indiscriminately. +Boris Verkhovskiy (5 Feb 2024) - libssh provides a way to disable the deprecation warnings for libssh only, an - d - naturally this is the preferred way. +- proxy1.0.md: fix example - This commit uses that, to prevent the erroneous hiding of potential, unrelate - d - deprecation warnings. + Closes #12856 - Fixes #12519 - Closes #12520 +Chris Webb (5 Feb 2024) -Daniel Stenberg (14 Dec 2023) +- configure: add --disable-docs flag -- test1474: removed + Building man pages from curldown sources now requires perl. Add a + --disable-docs flag to configure to enable building and installing + without documentation where perl is not available or man pages are not + required. This is selected automatically (with a warning) when perl is + not found by configure. - The test was already somewhat flaky and disabled on several platforms, - and after 1da640abb688 even more unstable. + Fixes #12832 + Closes #12857 -- readwrite_data: loop less +Faraz Fallahi (5 Feb 2024) - This function is made to loop in order to drain incoming data - faster. Completely removing the loop has a measerably negative impact on - transfer speeds. +- connect.c: fix typo - Downsides with the looping include + Closes #12858 - - it might call the progress callback much more seldom. Especially if - the write callback is slow. +Daniel Stenberg (1 Feb 2024) - - rate limiting becomes less exact +- sendf: ignore response body to HEAD - - a single transfer might "starve out" other parallel transfers + and mark the stream for close, but return OK since the response this far + was ok - if headers were received. Partly because this is what curl has + done traditionally. - - QUIC timers for other connections can't be maintained correctly + Test 499 verifies. Updates test 689. - The long term fix should be to remove the loop and optimize coming back - to avoid the transfer speed penalty. + Reported-by: Sergey Bronnikov + Bug: https://curl.se/mail/lib-2024-02/0000.html + Closes #12842 - This fix lower the max loop count to reduce the starvation problem, and - avoids the loop completely for when rate-limiting is in progress. +- ftp: treat a 226 arriving before data as a signal to read data - Ref: #12488 - Ref: https://curl.se/mail/lib-2023-12/0012.html - Closes #12504 + For active mode transfers. -Stefan Eissing (14 Dec 2023) + Due to some interesting timing, curl can sometimes get the 226 (transfer + complete) over the control channel first, before the data connection + signals readability. If this happens, use that as a signal to check the + data connection. -- lib: eliminate `conn->cselect_bits` + Additionally, set the socket filter in listen mode *before* the + PORT/EPRT command is issued, to reduce the risk that the little time gap + could interfere. - - use `data->state.dselect_bits` everywhere instead - - remove `bool *comeback` parameter as non-zero - `data->state.dselect_bits` will indicate that IO is - incomplete. + This issue never reproduced for me on Debian and takes several hundred + rounds for me to trigger on my mac. - Closes #12512 + Reported-by: Stefan Eissing + Fixes #12823 + Closes #12841 -- connect: refactor `Curl_timeleft()` +Patrick Monnerat (1 Feb 2024) - - less local vars, "better" readability - - added documentation +- OS400: avoid using awk in the build scripts - Closes #12518 + Awk is a PASE program and its use may cause a failure depending on the + CCSID of the calling script (IBM bug?). -Dmitry Karpov (14 Dec 2023) + For this reason, revert to an sed-only solution to extract the exported + symbols from the header files. -- cookie: avoid fopen with empty file name + Closes #12826 - Closes #12514 +Jan Macku (1 Feb 2024) -Viktor Szakats (13 Dec 2023) +- docs: remove `mk-ca-bundle.1` from `man_MANS` -- tests/server: delete workaround for old-mingw + It was accidentally added in https://github.com/curl/curl/pull/12730 - mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this. + Co-authored-by: Lukáš Zaoral + Signed-off-by: Jan Macku - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 + Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b + Closes #12843 - Reviewed-by: Jay Satiro - Closes #12510 +Daniel Stenberg (1 Feb 2024) -- cmake: delete obsolete TODOs more [ci skip] +- RELEASE-NOTES: synced - - manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288 - - soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023 - - bunch of others that are completed - - `NTLM_WB_ENABLED` is implemented in a basic form, and now also - scheduled for removal, so a TODO at this point isn't useful. + and bump to 8.6.1 for now - And this 'to-check' item: +- cmdline-docs/Makefile: avoid using a fixed temp file name - Q: "The cmake build selected to run gcc with -fPIC on my box while the - plain configure script did not." + By appending the pid number two different runs at the same time will not + trample over the same file. - A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 - and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly - enable PIC for libcurl shared lib. Or when building libcurl for - shared and static lib in a single pass. We do this by default for - Windows or when enabled by the user via `SHARE_LIB_OBJECT`. - Otherwise we don't touch this setting. Meaning the default set by - CMake (if any) or the toolchain is used. On Debian Bookworm, this - means that PIC is disabled for static libs by default. Some platforms - (like macOS), has PIC enabled by default. - autotools supports the double-pass mode only, and in that case - CMake seems to match PIC behaviour now (as tested on Linux with gcc.) + Reported-by: Jon Rumsey + Fixes #12829 + Closes #12839 - Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500 +- asyn-thread: use wakeup_close to close the read descriptor - Reviewed-by: Jay Satiro - Closes #12509 + Reported-by: Dan Fandrich + Ref: #12834 + Closes #12836 -Stefan Eissing (12 Dec 2023) +Stefan Eissing (1 Feb 2024) -- CLIENT-WRITERS: design and use documentation +- ntml_wb: fix buffer type typo - Closes #12507 + Closes #12825 -Viktor Szakats (12 Dec 2023) +Daniel Stenberg (1 Feb 2024) -- cmake: delete obsolete TODO items [ci skip] +- tool_operate: do not set CURLOPT_QUICK_EXIT in debug builds - There is always room for improvement, but CMake is up to par now with - autotools, so there is no longer a good reason to keep around these - inline TODO items. + Since it allows (small) memory leaks that interfere with torture tests + and regular memory-leak checks. - Answering one of questions: + Reported-by: Dan Fandrich + Fixes #12834 + Closes #12835 - Q: "The gcc command line use neither -g nor any -O options. As a - developer, I also treasure our configure scripts's --enable-debug - option that sets a long range of "picky" compiler options." +Boris Verkhovskiy (31 Jan 2024) - A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info - and optimization level. E.g.: - - `Release` = `-O3` + no debug info - - `MinSizeRel` = `-Os` + no debug info - - `Debug` = `-O0` + debug info +- form-string.md: correct the example - https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-deb - ug-release-relwithdebinfo-and-minsizerel/59314670#59314670 - https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#defaul - t-and-custom-configurations + Closes #12822 - For picky warnings we have the `PICKY_COMPILER` options, enabled by - default. +Version 8.6.0 (31 Jan 2024) - Closes #12500 +Daniel Stenberg (31 Jan 2024) -Stefan Eissing (11 Dec 2023) +- RELEASE-NOTES: synced -- CONNECTION-FILTERS: update documentation + curl 8.6.0 - Closes #12497 +- THANKS: new contributors from 8.5.0 -Daniel Stenberg (11 Dec 2023) +Jay Satiro (31 Jan 2024) -- lib: reduce use of strncpy +- cd2nroff: use perl 'strict' and 'warnings' - - bearssl: select cipher without buffer copies - - http_aws_sigv4: avoid strncpy, require exact timestamp length - - http_aws_sigv4: use memcpy isntead of strncpy - - openssl: avoid strncpy calls - - schannel: check for 1.3 algos without buffer copies - - strerror: avoid strncpy calls - - telnet: avoid strncpy, return error on too long inputs - - vtls: avoid strncpy in multissl_version() + - Use strict and warnings pragmas. - Closes #12499 + - If open() fails then show the reason. -- CI/distcheck: run full tests + - Set STDIN io layer :crlf so that input is properly read on Windows. - To be able to detect missing files better, this now runs the full CI - test suite. If done before, it would have detected #12462 before - release. + - When STDIN is used as input, the filename $f is now set to "STDIN". - Closes #12503 + Various error messages in single() use $f for the filename and this way + it is not undefined when STDIN. -- docs: clean up Protocols: for cmdline options + Closes https://github.com/curl/curl/pull/12819 - ... and some other minor polish. +Daniel Stenberg (30 Jan 2024) - Closes #12496 +- cd2nroff: fix duplicate output issue -- cmdline/gen: fix the sorting of the man page options + Assisted-by: Jay Satiro + Fixes https://github.com/curl/curl-www/issues/321 + Closes #12818 - They were previously sorted based on the file names, which use a .d - extension, making "data" get placed after "data-binary" etc. Making the - sort ignore the extention fixes the ordering. +- lib: error out on multissl + http3 - Reported-by: Boris Verkhovskiy - Bug: https://curl.se/mail/archive-2023-12/0014.html - Closes #12494 + Since the QUIC/h3 code has no knowledge or handling of multissl it might + bring unintended consequences if we allow it. -Daniel Gustafsson (9 Dec 2023) + configure, cmake and curl_setup.h all now reject this combination. -- doh: remove unused local variable + Assisted-by: Viktor Szakats + Assisted-by: Gisle Vanem + Ref: #12806 + Closes #12807 - The nurl variable is no longer used during probing following - a refactoring, so remove. +Patrick Monnerat (29 Jan 2024) - Closes #12491 +- OS400: sync ILE/RPG binding -Jay Satiro (8 Dec 2023) + Also do not force git CRLF line endings on *.cmd files for OS400. -- build: fix Windows ADDRESS_FAMILY detection + Closes #12815 - - Include winsock2.h for Windows ADDRESS_FAMILY detection. +Viktor Szakats (28 Jan 2024) - Prior to this change cmake detection didn't work because it included - ws2def.h by itself, which is missing needed types from winsock2.h. +- build: delete/replace 3 more clang warning pragmas - Prior to this change autotools detection didn't work because it did not - include any Windows header. + - tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma. - In both cases libcurl would fall back on unsigned short as the address - family type, which is the same as ADDRESS_FAMILY. + - whitespace formatting in `mprintf.h`, lib518, lib537. - Co-authored-by: Viktor Szakats + - lib518: fix wrong variable in `sizeof()`. - Closes https://github.com/curl/curl/pull/12441 + - lib518: bump variables to `rlim_t`. + Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469 -Daniel Stenberg (8 Dec 2023) + - lib518: sync error message with lib537 + Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce -- lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding + - lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas + by reworking test code. - Since the copy does not stop at a null byte, let's not call it anything - that makes you think it works like the common strndup() function. + Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812 + Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 + Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat + Reviewed-by: Daniel Stenberg + Closes #12814 - Closes #12490 +Richard Levitte (27 Jan 2024) -- convsrctest.pl: removed: not used, not shipped in tarballs +- cmake: freshen up docs/INSTALL.cmake -- tests: rename tests scripts to the test number + - Turn docs/INSTALL.cmake into a proper markdown file, + docs/INSTALL-CMAKE.md + - Move things around to divide the description into configuration, + building and installing sections + - Mention the more modern cmake options to configure, build and install, + but also retain the older variants as fallbacks - It is hard to name the scripts sensibly. Lots of them are similarly - named and the name did not tell which test that used them. + Closes #12772 - The new approach is rather to name them based on the test number that - runs them. Also helps us see which scripts are for individual tests - rather than for general test infra. +Viktor Szakats (27 Jan 2024) - - badsymbols.pl -> test1167.pl - - check-deprecated.pl -> test1222.pl - - check-translatable-options.pl -> test1544.pl - - disable-scan.pl -> test1165.pl - - error-codes.pl -> test1175.pl - - errorcodes.pl -> test1477.pl - - extern-scan.pl -> test1135.pl - - manpage-scan.pl -> test1139.pl - - manpage-syntax.pl -> test1173.pl - - markdown-uppercase.pl -> test1275.pl - - mem-include-scan.pl -> test1132.pl - - nroff-scan.pl -> test1140.pl - - option-check.pl -> test1276.pl - - options-scan.pl -> test971.pl - - symbol-scan.pl -> test1119.pl - - version-scan.pl -> test1177.pl +- build: delete/replace clang warning pragmas - Closes #12487 + - delete redundant warning suppressions for `-Wformat-nonliteral`. + This now relies on `CURL_PRINTF()` and it's theoratically possible + that this macro isn't active but the warning is. We're ignoring this + as a corner-case here. -Michał Antoniak (8 Dec 2023) + - replace two pragmas with code changes to avoid the warnings. -- sendf: fix compiler warning with CURL_DISABLE_HEADERS_API + Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 + Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - fix MSVC warning C4189: 'htype': local variable is initialized but not - referenced - when CURL_DISABLE_HEADERS_API is defined. + Reviewed-by: Daniel Stenberg + Closes #12812 - Closes #12485 +Daniel Stenberg (27 Jan 2024) -Viktor Szakats (8 Dec 2023) +- RELEASE-NOTES: synced -- tidy-up: whitespace +- http: only act on 101 responses when they are HTTP/1.1 - Closes #12484 + For 101 responses claiming to be any other protocol, bail out. This + would previously trigger an assert. -Stefan Eissing (7 Dec 2023) + Add test 1704 to verify. -- test_02_download: fix paramters to test_02_27 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184 + Closes #12811 - - it is a special client that only ever uses http/2 +Scarlett McAllister (27 Jan 2024) - Closes #12467 +- _VARIABLES.md: add missing 'be' into the sentence -Michał Antoniak (7 Dec 2023) + Closes #12809 -- vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY +Stefan Eissing (27 Jan 2024) - Closes #12459 +- mqtt, remove remaining use of data->state.buffer -Daniel Stenberg (7 Dec 2023) + Closes #12799 -- lib: strndup/memdup instead of malloc, memcpy and null-terminate +Daniel Stenberg (27 Jan 2024) - - bufref: use strndup - - cookie: use strndup - - formdata: use strndup - - ftp: use strndup - - gtls: use aprintf instead of malloc + strcpy * 2 - - http: use strndup - - mbedtls: use strndup - - md4: use memdup - - ntlm: use memdup - - ntlm_sspi: use strndup - - pingpong: use memdup - - rtsp: use strndup instead of malloc, memcpy and null-terminate - - sectransp: use strndup - - socks_gssapi.c: use memdup - - vtls: use dynbuf instead of malloc, snprintf and memcpy - - vtls: use strdup instead of malloc + memcpy - - wolfssh: use strndup - - Closes #12453 +- x509asn1: switch from malloc to dynbuf -- strdup: remove the memchr check from Curl_strndup + Closes #12808 - It makes it possible to clone a binary chunk of data. +- x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy - Closes #12453 + Closes #12808 -- ftp: handle the PORT parsing without allocation +- x509asn1: reduce malloc in Curl_extract_certinfo - Also reduces amount of *cpy() calls. + Using dynbuf - Closes #12456 + Closes #12808 -- RELEASE-NOTES: synced +Jay Satiro (27 Jan 2024) - Bumped to 8.5.1 +- THANKS: add Alexander Bartel and Brennan Kinney -- url: for disabled protocols, mention if found in redirect + They reported and investigated #10259 which was fixed by 7b2d98df. - To help users better understand where the URL (and denied scheme) comes - from. Also removed "in libcurl" from the message, since the disabling - can be done by the application. + Ref: https://github.com/curl/curl/issues/10259 - The error message now says "not supported" or "disabled" depending on - why it was denied: +Daniel Stenberg (26 Jan 2024) - Protocol "hej" not supported - Protocol "http" disabled +- krb5: add prototype to silence clang warnings on mvsnprintf() - And in redirects: + "error: format string is not a string literal" - Protocol "hej" not supported (in redirect) - Protocol "http" disabled (in redirect) + Follow-up to 09230127589eccc7 which made the warning appear - Reported-by: Mauricio Scheffer - Fixes #12465 - Closes #12469 + Assisted-by: Viktor Szakats + Closes #12803 -Stefan Eissing (6 Dec 2023) +- x509asn1: remove code for WANT_VERIFYHOST -- sectransp_ make TLSCipherNameForNumber() available in non-verbose config + No code ever sets this anymore since we dropped gskit - Reported-by: Cajus Pollmeier - Closes #12476 - Fixes #12474 + Follow-up to 78d6232f1f326b9ab4d -YX Hao (6 Dec 2023) + Closes #12804 -- lib: fix variable undeclared error caused by `infof` changes +- socks: reduce the buffer size to 600 (from 8K) - `--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined. - `infof` isn't `Curl_nop_stmt` anymore: dac293c. + This is malloc'ed memory and it does not more. Test 742 helps us verify + this. - Follow-up to dac293c + Closes #12789 - Closes #12470 +Stefan Eissing (26 Jan 2024) -Viktor Szakats (6 Dec 2023) +- file+ftp: use stack buffers instead of data->state.buffer -- tidy-up: fix yamllint whitespace issues in labeler.yml + Closes #12789 - Follow-up to bda212911457c6fadfbba50be61afc4ca513fa56 #12466 +- vtls: receive max buffer - Reviewed-by: Dan Fandrich - Closes #12475 + - do not only receive one TLS record, but try to fill + the passed buffer + - consider <4K remaning space is "filled". -- tidy-up: fix yamllint whitespace issues + Closes #12801 - Closes #12466 +Daniel Stenberg (26 Jan 2024) -Chris Sauer (6 Dec 2023) +- docs: do not start lines/sentences with So, But nor And -- cmake: fix typo + Closes #12802 - Follow-up to aace27b - Closes #12464 +- docs: remove spurious ampersands from markdown -Daniel Stenberg (6 Dec 2023) + They were leftovers from the nroff conversion. -- dist: add tests/errorcodes.pl to the tarball + Follow-up to eefcc1bda4bccd800f5a5 - Used by test 1477 + Closes #12800 - Reported-by: Xi Ruoyao - Follow-up to 0ca3a4ec9a7 - Fixes #12462 - Closes #12463 +Patrick Monnerat (26 Jan 2024) -Dan Fandrich (6 Dec 2023) +- sasl: make login option string override http auth -- github/labeler: update a missed key in the v5 upgrade + - Use http authentication mechanisms as a default, not a preset. - Follow-up to ce03fe3ba + Consider http authentication options which are mapped to SASL options as + a default (overriding the hardcoded default mask for the protocol) that + is ignored if a login option string is given. -Version 8.5.0 (6 Dec 2023) + Prior to this change, if some HTTP auth options were given, sasl mapped + http authentication options to sasl ones but merged them with the login + options. -Daniel Stenberg (6 Dec 2023) + That caused problems with the cli tool that sets the http login option + CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag + maps to more than one sasl mechanisms and the latter cannot be cleared + individually by the login options string. -- RELEASE-NOTES: synced + New test 992 checks this. - The curl 8.5.0 release. + Fixes https://github.com/curl/curl/issues/10259 + Closes https://github.com/curl/curl/pull/12790 -Dan Fandrich (5 Dec 2023) +Stefan Eissing (26 Jan 2024) -- github/labeler: switch from the beta to labeler v5 +- socks: use own buffer instead of data->state.buffer - Some keys were renamed and the dot option was made default. + Closes #12788 - Closes #12458 +Daniel Stenberg (26 Jan 2024) -Daniel Stenberg (5 Dec 2023) +- socks: fix generic output string to say SOCKS instead of SOCKS4 -- DEPRECATE: remove NTLM_WB in June 2024 + ... since it was also logged for SOCKS5. - Ref: https://curl.se/mail/lib-2023-12/0010.html + Closes #12797 - Closes #12451 +- test742: test SOCKS5 with max length user, password and hostname -Jacob Hoffman-Andrews (4 Dec 2023) + Adjusted the socksd server accordingly to allow for configuring that + long user name and password. -- rustls: implement connect_blocking + Closes #12797 - Closes #11647 +Stefan Eissing (25 Jan 2024) -Daniel Stenberg (4 Dec 2023) +- ssh: use stack scratch buffer for seeks -- examples/rtsp-options.c: add + - instead of data->state.buffer - Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and - CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. + Closes #12794 - Closes #12452 +Daniel Stenberg (25 Jan 2024) -Stefan Eissing (4 Dec 2023) +- krb5: access the response buffer correctly -- ngtcp2: ignore errors on unknown streams + As the pingpong code no longer uses the download buffer. - - expecially in is_alive checks on connections, we might - see incoming packets on streams already forgotten and closed, - leading to errors reported by nghttp3. Ignore those. + Folllow-up to c2d973627bab12ab + Pointed-out-by: Stefan Eissing + Closes #12796 - Closes #12449 +Stefan Eissing (25 Jan 2024) -Daniel Stenberg (4 Dec 2023) +- mqtt: use stack scratch buffer for recv+publish -- docs: make all examples in all libcurl man pages compile + - instead of data->state.buffer - Closes #12448 + Closes #12792 -- checksrc.pl: support #line instructions +- telnet, use stack scratch buffer for do - makes it identify the correct source file and line + - instead of data->state.buffer -- GHA/man-examples: verify libcurl man page examples + Closes #12793 -- verify-examples.pl: verify that all man page examples compile clean +- http, use stack scratch buffer -- RELEASE-NOTES: synced + - instead of data->state.buffer -Graham Campbell (2 Dec 2023) + Closes #12791 -- http3: bump ngtcp2 and nghttp3 versions +- ntlm_wb: do not use data->state.buf any longer - nghttp3 v1.1.0 - ngtcp2 v1.1.0 + Closes #12787 - In docs and CI +- gitignore: the generated `libcurl-symbols.md` - Closes #12446 + Closes #12795 -- CI/quiche: use `3.1.4+quic` consistently in CI workflows +Daniel Stenberg (25 Jan 2024) - Closes #12447 +- tool: fix the listhelp generation command -Viktor Szakats (2 Dec 2023) + The previous command line to generate the tool_listhelp.c source file + broke with 2494b8dd5175cee7. -- test1545: disable deprecation warnings + Make 'make listhelp' invoked in src/ generate it. Also update the + comment in the file to mention the right procedure. - Fixes: - https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr - p6pk#L1205 + Closes #12786 - Same with details: - https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil - b6wt#L1263 - ``` - tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 - .0. Use curl_mime_init() [-Werror=deprecated-declarations] - 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", - | ^~~~~~~~~~~~ - [...] - ``` +- http: check for "Host:" case insensitively - Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 + When checking if the user wants to replace the header, the check should + be case insensitive. - Fixes #12445 - Closes #12444 + Adding test 461 to verify -Daniel Stenberg (2 Dec 2023) + Found-by: Dan Fandrich + Ref: #12782 + Closes #12784 -- INSTALL: update list of ports and CPU archs +Tatsuhiro Tsujikawa (25 Jan 2024) -- symbols-in-versions: the CLOSEPOLICY options are deprecated +- configure: add libngtcp2_crypto_boringssl detection - The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. + If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested, + try to detect libngtcp2_crypto_boringssl. -z2_ (1 Dec 2023) + Reported-by: ウさん + Fixes #12724 + Closes #12769 -- build: fix builds that disable protocols but not digest auth +Daniel Stenberg (25 Jan 2024) - - Build base64 functions if digest auth is not disabled. +- http: remove comment reference to a removed solution - Prior to this change if some protocols were disabled but not digest auth - then a build error would occur due to missing base64 functions. + Follow-up to 58974d25d - Fixes https://github.com/curl/curl/issues/12440 - Closes https://github.com/curl/curl/pull/12442 + Closes #12785 -Michał Antoniak (1 Dec 2023) +Stefan Eissing (25 Jan 2024) -- connect: reduce number of transportation providers +- pytest: Scorecard tracking CPU and RSS - Use only the ones necessary - the ones that are built-in. Saves a few - bytes in the resulting code. + Closes #12765 - Closes #12438 +Graham Campbell (25 Jan 2024) -David Benjamin (1 Dec 2023) +- GHA: bump ngtcp2, gnutls, mod_h2, quiche -- vtls: consistently use typedef names for OpenSSL structs + - ngtcp2 to v1.2.0 + - gnutls to 3.8.3 + - mod_h2 to 2.0.26 + - quiche to 0.20.0 - The foo_st names don't appear in OpenSSL public API documentation. The - FOO typedefs are more common. This header was already referencing - SSL_CTX via . There is a comment about avoiding - , but OpenSSL actually declares all the typedefs in - , which is already included by (and - every other OpenSSL header), so just use that. Though I've included it - just to be explicit. + Closes #12778 + Closes #12779 + Closes #12780 + Closes #12781 - (I'm also fairly sure including already triggers the - Schannel conflicts anyway. The comment was probably just out of date.) +Daniel Stenberg (25 Jan 2024) - Closes #12439 +- ftpserver.pl: send 213 SIZE response without spurious newline -Lau (1 Dec 2023) +- pingpong: stop using the download buffer -- libcurl-security.3: fix typo + The pingpong logic now uses its own dynbuf for receiving command + response data. - Fixed minimal typo. + When the "final" response header for a commanad has been received, that + final line is left first in the recvbuf for the protocols to parse at + will. If there is additional data behind the final response line, the + 'overflow' counter is indicate how many bytes. - Closes #12437 + Closes #12757 -Stefan Eissing (1 Dec 2023) +- gen.pl: remove bold from .IP used for ## -- ngtcp2: fix races in stream handling + Reported-by: Viktor Szakats + Fixes #12776 + Closes #12777 - - fix cases where ngtcp2 invokes callbacks on streams that - nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND - in these cases as it is normal behaviour. +Viktor Szakats (24 Jan 2024) - Closes #12435 +- cmake: rework options to enable curl and libcurl docs -Emanuele Torre (1 Dec 2023) + Rework CMake options for building/using curl tool and libcurl manuals. -- tool_writeout_json: fix JSON encoding of non-ascii bytes + - rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning: + to build man page and built-in manual for curl tool. - char variables if unspecified can be either signed or unsigned depending - on the platform according to the C standard; in most platforms, they are - signed. + - rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning: + to build man pages for libcurl. - This meant that the *i<32 waas always true for bytes with the top bit - set. So they were always getting encoded as \uXXXX, and then since they - were also signed negative, they were getting extended with 1s causing - '\xe2' to be expanded to \uffffffe2, for example: + - `BUILD_LIBCURL_DOCS` now works without having to enable + `ENABLE_CURL_MANUAL` too. - $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul - l - \uffffffe2\uffffff80\uffffff9c + - drop support for existing CMake-level `USE_MANUAL` option to avoid + confusion. (It used to work with the effect of current + `ENABLE_CURL_MANUAL`, but only by accident.) - I fixed this bug by making the code use explicitly unsigned char* - variables instead of char* variables. + Assisted-by: Richard Levitte + Ref: #12771 + Closes #12773 - Test 268 verifies +Daniel Stenberg (24 Jan 2024) - Reported-by: iconoclasthero - Closes #12434 +- urlapi: remove assert -Stefan Eissing (1 Dec 2023) + This assert triggers wrongly when CURLU_GUESS_SCHEME and + CURLU_NO_AUTHORITY are both set and the URL is a single path. -- cf-socket: TCP trace output local address used in connect + I think this assert has played out its role. It was introduced in a + rather big refactor. - Closes #12427 + Follow-up to 4cfa5bcc9a -Jay Satiro (1 Dec 2023) + Reported-by: promptfuzz_ on hackerone + Closes #12775 -- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation +Patrick Monnerat (24 Jan 2024) - - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it - includes protocol-specific instructions that trigger a transfer. +- tests: avoid int/size_t conversion size/sign warnings - Prior to this change it explicitly said that it did not include those - instructions in the time, but that is incorrect. + Closes #12768 - The change is a copy of the fixed explanation already in - CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. +Daniel Stenberg (24 Jan 2024) - Reported-by: eeverettrbx@users.noreply.github.com +- GHA: add a job scanning for "bad words" in markdown - Fixes https://github.com/curl/curl/issues/12431 - Closes https://github.com/curl/curl/pull/12432 + This means words, phrases or things we have decided not to use - words that + are spelled right according to the dictionary but we want to avoid. In the + name of consistency and better documentation. -Daniel Stenberg (30 Nov 2023) + Closes #12764 -- multi: during ratelimit multi_getsock should return no sockets +Viktor Szakats (23 Jan 2024) - ... as there is nothing to wait for then, it just waits. Otherwise, this - causes much more CPU work and updates than necessary during ratelimit - periods. +- cmake: speed up curldown processing, enable by default - Ref: https://curl.se/mail/lib-2023-11/0056.html - Closes #12430 + - cmake: enable `BUILD_DOCS` by default (this controls converting and + installing `.3` files from `.md` sources) -Dmitry Karpov (30 Nov 2023) + - cmake: speed up generating `.3` files by using a single command per + directory, instead of a single command per file. This reduces external + commands by about a thousand. (There remains some CMake logic kicking + in resulting in 500 -one per file- external `-E touch_nocreate` calls.) -- transfer: abort pause send when connection is marked for closing + - cd2nroff: add ability to process multiple input files. - This handles cases of some bi-directional "upgrade" scenarios - (i.e. WebSockets) where sending is paused until some "upgrade" handshake - is completed, but server rejects the handshake and closes the - connection. + - cd2nroff: add `-k` option to use the source filename to form the + output filename. (instead of the default in-file `Title:` line.) - Closes #12428 + Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c + Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753 + Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 -Daniel Stenberg (28 Nov 2023) + Closes #12762 -- RELEASE-NOTES: synced +Richard Levitte (23 Jan 2024) -- openssl: when a session-ID is reused, skip OCSP stapling +- docs: install curl.1 with cmake as well - Fixes #12399 - Reported-by: Alexey Larikov - Closes #12418 + Closes #12759 -- test1545: test doing curl_formadd twice with missing file +Daniel Stenberg (23 Jan 2024) - Reproduces #12410 - Verifies the fix - Closes #12421 +- osslq: remove the TLS library from the version output -- Curl_http_body: cleanup properly when Curl_getformdata errors + Since we only support using a single TLS library at any one time, we + know that the TLS library for QUIC is the same that is also shown for + regular TLS. - Reported-by: yushicheng7788 on github - Based-on-work-by: yushicheng7788 on github - Fixes #12410 - Closes #12421 + Fixes #12763 + Reported-by: Viktor Szakats + Closes #12767 -- test1477: verify that libcurl-errors.3 and public headers are synced +Stefan Eissing (23 Jan 2024) - The script errorcodes.pl extracts all error codes from all headers and - checks that they are all documented, then checks that all documented - error codes are also specified in a header file. +- CI: remove unnecessary OpenSSL 3 option `enable-tls1_3` - Closes #12424 + .. and switch OpenSSL 3 libdir from lib64 to lib for consistency. -- libcurl-errors.3: sync with current public headers + Closes https://github.com/curl/curl/pull/12758 - Closes #12424 +- GHA: bump nghttp2 version to v1.59.0 -Stefan Eissing (28 Nov 2023) + - Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version. -- test459: fix for parallel runs + Closes https://github.com/curl/curl/pull/12766 - - change warniing message to work better with varying filename - length. - - adapt test output check to new formatting +Daniel Stenberg (23 Jan 2024) - Follow-up to 97ccc4479f77ba3191c6 - Closes #12423 +- RELEASE-NOTES: synced -Daniel Stenberg (27 Nov 2023) +- docs/cmdline: change to .md for cmdline docs -- tool_cb_prg: make the carriage return fit for wide progress bars + - switch all invidual files documenting command line options into .md, + as the documentation is now markdown-looking. - When the progress bar was made max width (256 columns), the fly() - function attempted to generate its output buffer too long so that the - trailing carriage return would not fit and then the output would show - wrongly. The fly function is called when the expected total transfer is - unknown, which could be one or more progress calls before the actual - progress meter get shown when the expected transfer size is provided. + - made the parser treat 4-space indents as quotes - This new take also replaces the msnprintf() call with a much simpler - memset() for speed. + - switch to building the curl.1 manpage using the "mainpage.idx" file, + which lists the files to include to generate it, instead of using the + previous page-footer/headers. Also, those files are now also .md + ones, using the same format. I gave them underscore prefixes to make + them sort separately: + _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md, + _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md, + _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md, + _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md - Reported-by: Tim Hill - Fixes #12407 - Closes #12415 + - updated test cases accordingly -- tool_parsecfg: make warning output propose double-quoting + Closes #12751 - When the config file parser detects a word that *probably* should be - quoted, mention double-quotes as a possible remedy. +dependabot[bot] (23 Jan 2024) - Test 459 verifies. +- CI: bump actions/cache from 3 to 4 - Proposed-by: Jiehong on github - Fixes #12409 - Closes #12412 + Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. + - [Release notes](https://github.com/actions/cache/releases) + - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) + - [Commits](https://github.com/actions/cache/compare/v3...v4) -Jay Satiro (26 Nov 2023) + --- + updated-dependencies: + - dependency-name: actions/cache + dependency-type: direct:production + update-type: version-update:semver-major + ... -- curl.rc: switch out the copyright symbol for plain ASCII + Signed-off-by: dependabot[bot] + Closes #12756 - .. like we already do for libcurl.rc. +Daniel Stenberg (23 Jan 2024) - libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" - warning so it was switched to ascii. +- openssl: when verifystatus fails, remove session id from cache - Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 + To prevent that it gets used in a subsequent transfer that skips the + verifystatus check since that check can't be done when the session id is + reused. - Suggested-by: Robert Southee + Reported-by: Hiroki Kurosawa + Closes #12760 - Closes https://github.com/curl/curl/pull/12403 +Viktor Szakats (23 Jan 2024) -Daniel Stenberg (26 Nov 2023) +- cmake: add option to disable building docs -- conncache: use the closure handle when disconnecting surplus connections +Richard Levitte (23 Jan 2024) - Use the closure handle for disconnecting connection cache entries so - that anything that happens during the disconnect is not stored and - associated with the 'data' handle which already just finished a transfer - and it is important that details from the unrelated disconnect does not - taint meta-data in the data handle. +- cmake: use curldown to build man pages - Like storing the response code. + This throws away the previous HTML and PDF producers, to mimic what + Makefile.am does as faithfully as possible. - This also adjust test 1506. Unfortunately it also removes a key part of - the test that verifies that a connection is closed since when this - output vanishes (because the closure handle is used), we don't know - exactly that the connection actually gets closed in this test... + Closes #12753 - Reported-by: ohyeaah on github - Fixes #12367 - Closes #12405 +Daniel Stenberg (23 Jan 2024) -- RELEASE-NOTES: synced +- mksymbolsmanpage.pl: provide references to where the symbol is used -Stefan Eissing (24 Nov 2023) +- docs: introduce "curldown" for libcurl man page format -- quic: make eyeballers connect retries stop at weird replies + curldown is this new file format for libcurl man pages. It is markdown + inspired with differences: - - when a connect immediately goes into DRAINING state, do - not attempt retries in the QUIC connection filter. Instead, - return CURLE_WEIRD_SERVER_REPLY - - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an - inconclusive answer. When all addresses have been attempted, - rewind the address list once on an inconclusive answer. - - refs #11832 where connects were retried indefinitely until - the overall timeout fired + - Each file has a set of leading headers with meta-data + - Supports a small subset of markdown + - Uses .md file extensions for editors/IDE/GitHub to treat them nicely + - Generates man pages very similar to the previous ones + - Generates man pages that still convert nicely to HTML on the website + - Detects and highlights mentions of curl symbols automatically (when + their man page section is specified) - Closes #12400 + tools: -Daniel Stenberg (24 Nov 2023) + - cd2nroff: converts from curldown to nroff man page + - nroff2cd: convert an (old) nroff man page to curldown + - cdall: convert many nroff pages to curldown versions + - cd2cd: verifies and updates a curldown to latest curldown -- CI: verify libcurl function SYNPOSIS sections + This setup generates .3 versions of all the curldown versions at build time. - With the .github/scripits/verify-synopsis.pl script + CI: - Closes #12402 + Since the documentation is now technically markdown in the eyes of many + things, the CI runs many more tests and checks on this documentation, + including proselint, link checkers and tests that make sure we capitalize the + first letter after a period... -- docs/libcurl: SYNSOPSIS cleanup + Closes #12730 - - use the correct include file - - make sure they are declared as in the header file - - fix minor nroff syntax mistakes (missing .fi) +Viktor Szakats (22 Jan 2024) - These are verified by verify-synopsis.pl, which extracts the SYNPOSIS - code and runs it through gcc. +- libssh2: use `libssh2_session_callback_set2()` with v1.11.1 - Closes #12402 + To avoid a local hack to pass function pointers and to avoid + deprecation warnings when building with libssh2 v1.11.1 or newer: + ``` + lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is depreca + ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- + declarations] + lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is depreca + ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- + declarations] + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/2072082 + 1100#step:3:4982 -- sendf: fix comment typo + Ref: https://github.com/libssh2/libssh2/pull/1285 + Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db + 3cf1d4b0b4 + Reviewed-by: Daniel Stenberg + Closes #12754 -- fopen: allocate the dir after fopen +Daniel Stenberg (22 Jan 2024) - Move the allocation of the directory name down to after the fopen() call - to allow that shortcut code path to avoid a superfluous malloc+free - cycle. +- transfer: make the select_bits_paused condition check both directions - Follow-up to 73b65e94f35311 + If there is activity in a direction that is not paused, return false. - Closes #12398 + Reported-by: Sergey Bronnikov + Bug: https://curl.se/mail/lib-2024-01/0049.html + Closes #12740 -Stefan Eissing (24 Nov 2023) +Stefan Eissing (22 Jan 2024) -- transfer: cleanup done+excess handling +- http3: initial support for OpenSSL 3.2 QUIC stack - - add `SingleRequest->download_done` as indicator that - all download bytes have been received - - remove `stop_reading` bool from readwrite functions - - move excess body handling into client download writer + - HTTP/3 for curl using OpenSSL's own QUIC stack together + with nghttp3 + - configure with `--with-openssl-quic` to enable curl to + build this. This requires the nghttp3 library + - implementation with the following restrictions: + * macOS has to use an unconnected UDP socket due to an + issue in OpenSSL's datagram implementation + See https://github.com/openssl/openssl/issues/23251 + This makes connections to non-reponsive servers hang. + * GET requests will send the indicator that they have + no body in a separate QUIC packet. This may result + in processing delays or Transfer-Encodings on proxied + requests + * uploads that encounter blocks will use 100% cpu as + detection of these flow control issue is not working + (we have not figured out to pry that from OpenSSL). - Closes #12371 + Closes #12734 -Daniel Stenberg (23 Nov 2023) +Viktor Szakats (22 Jan 2024) -- fopen: create new file using old file's mode +- cmake: fix `ENABLE_MANUAL` option - Because the function renames the temp file to the target name as a last - step, if the file was previously owned by a different user, not ORing - the old mode could otherwise end up creating a file that was no longer - readable by the original owner after save. + Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`. - Reported-by: Loïc Yhuel - Fixes #12299 - Closes #12395 + Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the + option designed to enable building and using the built-in curl manual. + (`USE_MANUAL=ON` option worked for this instead, by accident). -- test1476: require proxy + Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409 + Closes #12749 - Follow-up from 323df4261c3542 +Mohammadreza Hendiani (19 Jan 2024) - Closes #12394 +- TODO: update broken link to ratelimit-headers draft -- fopen: create short(er) temporary file name + Closes #12741 - Only using random letters in the name plus a ".tmp" extension. Not by - appending characters to the final file name. +Daniel Stenberg (19 Jan 2024) - Reported-by: Maksymilian Arciemowicz +- cmake: when USE_MANUAL=YES, build the curl.1 man page - Closes #12388 + Fixes KNOWN_BUG 15.4 -Stefan Eissing (23 Nov 2023) + Closes #12742 -- tests: git ignore generated second-hsts.txt file +- cmdline-opts/write-out.d: remove spurious double quotes - File is generated in test lib1900 +Stefan Eissing (19 Jan 2024) - Follow-up to 7cb03229d9e9c5 +- rtsp: Convert assertion into debug log - Closes #12393 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 -Viktor Szakats (23 Nov 2023) + - write excess bytes to the client where the standard excess bytes + checks will report any wrongness and fail the transfer -- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 + Fixes #12738 + Closes #12739 - Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to - OpenSSL 1.1.x, and LibreSSL 3.6 or upper. +Daniel Stenberg (19 Jan 2024) - With the caveat, that "group name" and "type name" are missing from - the log output with these TLS backends. +- headers: remove assert from Curl_headers_push - Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + The fuzzer managed to reach the function without a terminating CR or LF + so let's handle it normally. While there, remove the goto. - Reviewed-by: Daniel Stenberg - Closes #12385 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839 -Daniel Stenberg (23 Nov 2023) + Closes #12721 -- urldata: fix typo in comment +- curl_easy_getinfo.3: remove the wrong time value count -- CI: codespell + It said "six" time values but they are eight by now. Remove the mention + of the amount. - The list of words to ignore is in the file - .github/scripts/codespell-ignore.txt + Closes #12727 - Closes #12390 +Viktor Szakats (18 Jan 2024) -- lib: fix comment typos +- mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls` - Five separate ones, found by codespell + - Silence warning in mbedTLS v3.5.1 public headers: + ``` + ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: r + edundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls] + ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: pre + vious declaration of 'psa_set_key_domain_parameters' was here + ``` + Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae + 785a513e157c + Ref: https://github.com/libssh2/libssh2/pull/1226 - Closes #12390 + - Fix compiler warnings seen with gcc 9.2.0 + cmake unity: + ``` + ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read': + ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull + -dereference] + 189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &res + ult); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~ + ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write': + ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull + -dereference] + 168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, & + result); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~ + ``` -- test1476: verify cookie PSL mixed case + - delete stray `#else`. -- cookie: lowercase the domain names before PSL checks + Closes #12720 - Reported-by: Harry Sintonen +Daniel Stenberg (17 Jan 2024) - Closes #12387 +- docs: cleanup nroff format use -Viktor Szakats (23 Nov 2023) + - remove use of .BI for code snippet + - stop using .br, just do a blank line + - remove use of .PP + - remove use for .sp + - remove backslash in .IP + - use .IP instead of .TP -- openssl: fix building with v3 `no-deprecated` + add CI test + Closes #12731 - - build quictls with `no-deprecated` in CI to have test coverage for - this OpenSSL 3 configuration. +Stefan Eissing (17 Jan 2024) - - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. - The caller code is meant for OpenSSL 3, while these two functions were - only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 - if built with option `no-deprecated`, causing build errors: - ``` - vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ - algorithms'; ISO C99 and later do not support implicit function declaration - s [-Wimplicit-function-declaration] - vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ - digests'; ISO C99 and later do not support implicit function declarations [ - -Wimplicit-function-declaration] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f - ullLog=true#L7667 +- test2307: fix expected failure code after ws refactoring - Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 - Reviewed-by: Alex Bozarth + Fixes #12722 + Closes #12728 - - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with - `no-deprecated` quictls 3 builds. - Do it by moving an existing solution for this from `vtls/openssl.c` - to `vtls/openssl.h` and adjusting caller code. - ``` - vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g - et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit - -function-declaration] - ``` - Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s - tep:24:1178 +Jay Satiro (17 Jan 2024) - - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and - `-Wunused-function` when trying to build curl with NTLM enabled but - without the necessary TLS backend (with DES) support. +- cf-socket: show errno in tcpkeepalive error messages - Closes #12384 + - If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set + then show the errno in the verbose error messages. -- curl.h: delete Symbian OS references + Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652 - curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 - via #5989. Delete references to it from public headers, because there - is no fresh release to use those headers with. + Closes https://github.com/curl/curl/pull/12726 - Reviewed-by: Dan Fandrich - Reviewed-by: Jay Satiro - Closes #12378 +- tool_getparam: stop supporting `@filename` style for --cookie -- windows: use built-in `_WIN32` macro to detect Windows + The `@filename` style was never documented for --cookie + but prior to this change curl would accept it anyway and always treat a + @ prefixed string as a filename. - Windows compilers define `_WIN32` automatically. Windows SDK headers - or build env defines `WIN32`, or we have to take care of it. The - agreement seems to be that `_WIN32` is the preferred practice here. - Make the source code rely on that to detect we're building for Windows. + That's a problem if the string also contains a = sign because then it is + documented to be interpreted as a cookie string and not a filename. - Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for - Windows detection, next to the official `_WIN32`. After this patch it - only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. + Example: - There is a slight chance these break compatibility with Windows - compilers that fail to define `_WIN32`. I'm not aware of any obsolete - or modern compiler affected, but in case there is one, one possible - solution is to define this macro manually. + `--cookie @foo=bar` - grepping for `WIN32` remains useful to discover Windows-specific code. + Before: Interpreted as load cookies from filename foo=bar. - Also: + After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`). - - extend `checksrc` to ensure we're not using `WIN32` anymore. + Other curl options with a data/filename option-value use the `@filename` + to distinguish filenames which is probably how this happened. The + --cookie option has never been documented that way. - - apply minor formatting here and there. + Ref: https://curl.se/docs/manpage.html#-b - - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. + Closes https://github.com/curl/curl/pull/12645 - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg +Stefan Eissing (16 Jan 2024) - Closes #12376 +- websockets: refactor decode chain -Stefan Eissing (22 Nov 2023) + - use client writer stack for decoding frames + - move websocket protocol handler to ws.c -- url: ConnectionExists revisited + Closes #12713 - - have common pattern of `if not match, continue` - - revert pages long if()s to return early - - move dead connection check to later since it may - be relatively expensive - - check multiuse also when NOT building with NGHTTP2 - - for MULTIUSE bundles, verify that the inspected - connection indeed supports multiplexing when in use - (bundles may contain a mix of connection, afaict) +- websockets: check for negative payload lengths - Closes #12373 + - in en- and decoding, check the websocket frame payload lengths for + negative values (from curl_off_t) and error the operation in that case + - add test 2307 to verify -Daniel Stenberg (22 Nov 2023) + Closes #12707 -- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range +Daniel Stenberg (16 Jan 2024) - ... or use the default value. +- docs: mention env vars not used by schannel - Also clarify the documentation language somewhat. + Ref: #12704 - Closes #12382 + Co-authored-by: Jay Satiro -- urldata: make maxconnects a 32 bit value + Closes #12711 - "2^32 idle connections ought to be enough for anybody" +- tool_operate: make --remove-on-error only remove "real" files - Closes #12375 + Reported-by: Harry Sintonen + Assisted-by: Dan Fandrich -- FEATURES: update the URL phrasing + Closes #12710 - The URL is length limited since a while back so "no limit" simply is not - true anymore. Mention the URL RFC standard used instead. +Jay Wu (16 Jan 2024) - Closes #12383 +- url: don't set default CA paths for Secure Transport backend -- wolfssh: remove redundant static prototypes + As the default for this backend is the native CA store. - vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We - rror=redundant-decls] + Closes #12704 - Closes #12381 +Lin Sun (16 Jan 2024) -- setopt: remove superfluous use of ternary expressions +- asyn-ares: with modern c-ares, use its default timeout - Closes #12374 + Closes #12703 -- mime: store "form escape" as a single bit +Daniel Stenberg (15 Jan 2024) - Closes #12374 +- tool_operate: stop setting the file comment on Amiga -- setopt: check CURLOPT_TFTP_BLKSIZE range on set + - the URL is capped at 80 cols, which ruins it if longer + - it does not strip off URL credentials + - it is done unconditonally, not on --xattr + - we don't have Amiga in the CI which makes fixing it blindly fragile - ... instead of later when the transfer is about to happen. + Someone who builds and tests on Amiga can add it back correctly in a + future if there is a desire. - Closes #12374 + Reported-by: Harry Sintonen + Closes #12709 -Viktor Szakats (21 Nov 2023) +Stefan Eissing (15 Jan 2024) -- build: add more picky warnings and fix them +- rtsp: deal with borked server responses - Enable more picky compiler warnings. I've found these options in the - nghttp3 project when implementing the CMake quick picky warning - functionality for it [1]. + - enforce a response body length of 0, if the + response has no Content-lenght. This is according + to the RTSP spec. + - excess bytes in a response body are forwarded to + the client writers which will report and fail the + transfer - `-Wunused-macros` was too noisy to keep around, but fixed a few issues - it revealed while testing. + Follow-up to d7b6ce6 + Fixes #12701 + Closes #12706 - - autotools: reflect the more precisely-versioned clang warnings. - Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 - - autotools: sync between clang and gcc the way we set `no-multichar`. - - autotools: avoid setting `-Wstrict-aliasing=3` twice. - - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. - It triggers in libtool-generated stub code. +Daniel Stenberg (14 Jan 2024) - - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. +- version: show only the libpsl version, not its dependencies - - lib/curl_setup.h: delete duplicate declaration for `fileno`. - Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d - (1999-12-29). This suggests this may not be needed anymore, but if - it does, we may restore this for those specific (non-Windows) systems. - - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since - c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. - - lib: delete unused macro `isxdigit_ascii` since - f65f750742068f579f4ee6d8539ed9d5f0afcb85. - - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. - - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. - - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` - macro. - - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. - - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. - - lib/setopt: fix `-Wunreachable-code-break`. - - lib/system_win32 and lib/timeval: fix double declarations for - `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. - - lib/warnless: fix double declarations in CMake UNITY mode [5]. - This was due to force-disabling the header guard of `warnless.h` to - to reapply it to source code coming after `warnless.c` in UNITY - builds. This reapplied declarations too, causing the warnings. - Solved by adding a header guard for the lines that actually need - to be reapplied. - - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. - - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant - block. - - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. - - lib/vtls/sectransp: suppress `-Wunreachable-code`. - Detected in `else` branches of dynamic feature checks, with results - known at compile-time, e.g. - ```c - if(SecCertificateCopySubjectSummary) /* -> true */ - ``` - Likely fixable as a separate micro-project, but given SecureTransport - is deprecated anyway, let's just silence these locally. - - src/tool_help: delete duplicate declaration for `helptext`. - - src/tool_xattr: fix `-Wunreachable-code`. - - tests: delete duplicate declaration for `unitfail` [8]. - - tests: delete duplicate declaration for `strncasecompare`. - - tests/libtest: delete duplicate declaration for `gethostname`. - Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 - (2010-08-02). - Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc - If there are still systems around with warnings, we may restore the - prototype, but limited for those systems. - - tests/lib2305: delete duplicate declaration for - `libtest_debug_config`. - - tests/h2-download: fix `-Wunreachable-code-break`. + The libpsl version output otherwise also includes version number for its + dependencies, like IDN lib, but since libcurl does not use libpsl's IDN + functionality those components are not important. - [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 - f8bf9f/cmake/PickyWarningsC.cmake - [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui - qla5fj45?fullLog=true#L1675 - [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 - 2331#step:7:72 - https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 - 2331#step:7:109 - [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L204 - [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L218 - [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 - 2331#step:7:290 - [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 - 2331#step:9:1193 - [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 - 2331#step:33:1870 + Ref: https://github.com/curl/curl-for-win/issues/63 + Closes #12700 - Closes #12331 +Brad Harder (14 Jan 2024) -Daniel Stenberg (21 Nov 2023) +- curl.h: CURLOPT_DNS_SERVERS is only available with c-ares -- transfer: avoid unreachable expression + Closes #12695 - If curl_off_t and size_t have the same size (which is common on modern - 64 bit systems), a condition cannot occur which Coverity pointed - out. Avoid the warning by having the code conditionally only used if - curl_off_t actually is larger. +Daniel Stenberg (14 Jan 2024) - Follow-up to 1cd2f0072fa482e25baa2 +- cmdline-opts/gen.pl: error on initital blank line - Closes #12370 + After the "---" separator, there should be no blank line and this script + now errors out if one is detected. -Stefan Eissing (21 Nov 2023) + Ref: #12696 + Closes #12698 -- transfer: readwrite improvements +- cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper - - changed header/chunk/handler->readwrite prototypes to accept `buf`, - `blen` and a `pconsumed` pointer. They now get the buffer to work on - and report back how many bytes they consumed - - eliminated `k->str` in SingleRequest - - improved excess data handling to properly calculate with any body data - left in the headerb buffer - - eliminated `k->badheader` enum to only be a bool + Follow-up to 693cd1679361828a which was incomplete - Closes #12283 + Ref #12680 + Closes #12697 -Daniel Stenberg (21 Nov 2023) +- curl_multi_fdset.3: remove mention of null pointer support -- RELEASE-NOTES: synced + ... since this funtion has not supported null pointer fd_set arguments since + at least 2006. (That's when I stopped my git blame journey) -Jiří Hruška (21 Nov 2023) + Fixes #12691 + Reported-by: sfan5 on github + Closes #12692 -- transfer: avoid calling the read callback again after EOF +Mark Huang (14 Jan 2024) - Regression since 7f43f3dc5994d01b12 (7.84.0) +- docs/cmdline: remove unnecessary line breaks - Bug: https://curl.se/mail/lib-2023-11/0017.html + Closes #12696 - Closes #12363 +Daniel Stenberg (14 Jan 2024) -Daniel Stenberg (21 Nov 2023) +- transfer: remove warning: Value stored to 'blen' is never read -- doh: provide better return code for responses w/o addresses + Detected by scan-build - Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the - response did not contain any addresses. Now it more accurately returns - CURLE_COULDNT_RESOLVE_HOST. + Follow-up from 1cd2f0072f - Reported-by: lRoccoon on github + Closes #12693 - Fixes #12365 - Closes #12366 +Stefan Eissing (13 Jan 2024) -Stefan Eissing (21 Nov 2023) +- lib: replace readwrite with write_resp -- HTTP/2, HTTP/3: handle detach of onoing transfers + This clarifies the handling of server responses by folding the code for + the complicated protocols into their protocol handlers. This concerns + mainly HTTP and its bastard sibling RTSP. - - refs #12356 where a UAF is reported when closing a connection - with a stream whose easy handle was cleaned up already - - handle DETACH events same as DONE events in h2/h3 filters + The terms "read" and "write" are often used without clear context if + they refer to the connect or the client/application side of a + transfer. This PR uses "read/write" for operations on the client side + and "send/receive" for the connection, e.g. server side. If this is + considered useful, we can revisit renaming of further methods in another + PR. - Fixes #12356 - Reported-by: Paweł Wegner - Closes #12364 + Curl's protocol handler `readwrite()` method been changed: -Viktor Szakats (20 Nov 2023) + ```diff + - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, + - const char *buf, size_t blen, + - size_t *pconsumed, bool *readmore); + + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t ble + n, + + bool is_eos, bool *done); + ``` -- autotools: stop setting `-std=gnu89` with `--enable-warnings` + The name was changed to clarify that this writes reponse data to the + client side. The parameter changes are: - Do not alter the C standard when building with `--enable-warnings` when - building with gcc. + * `conn` removed as it always operates on `data->conn` + * `pconsumed` removed as the method needs to handle all data on success + * `readmore` removed as no longer necessary + * `is_eos` as indicator that this is the last call for the transfer + response (end-of-stream). + * `done` TRUE on return iff the transfer response is to be treated as + finished - On one hand this alters warning results compared to a default build. - On the other, it may produce different binaries, which is unexpected. + This change affects many files only because of updated comments in + handlers that provide no implementation. The real change is that the + HTTP protocol handlers now provide an implementation. - Also fix new warnings that appeared after removing `-std=gnu89`: + The HTTP protocol handlers `write_resp()` implementation will get passed + **all** raw data of a server response for the transfer. The HTTP/1.x + formatted status and headers, as well as the undecoded response + body. `Curl_http_write_resp_hds()` is used internally to parse the + response headers and pass them on. This method is public as the RTSP + protocol handler also uses it. - - include: fix public curl headers to use the correct printf mask for - `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 - and Visual Studio 2013 and newer. This fixes the printf mask warnings - in examples and tests. E.g. [1] + HTTP/1.1 "chunked" transport encoding is now part of the general + *content encoding* writer stack, just like other encodings. A new flag + `CLIENTWRITE_EOS` was added for the last client write. This allows + writers to verify that they are in a valid end state. The chunked + decoder will check if it indeed has seen the last chunk. - - conncache: fix printf format string [2]. + The general response handling in `transfer.c:466` happens in function + `readwrite_data()`. This mainly operates now like: - - http2: fix potential null pointer dereference [3]. - (seen on Slackware with gcc 11.) + ``` + static CURLcode readwrite_data(data, ...) + { + do { + Curl_xfer_recv_resp(data, buf) + ... + Curl_xfer_write_resp(data, buf) + ... + } while(interested); + ... + } + ``` - - libssh: fix printf format string in SFTP code [4]. - Also make MSVC builds compatible with old CRT versions. + All the response data handling is implemented in + `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` + implementation if available, or does the default behaviour. - - libssh2: fix printf format string in SFTP code for MSVC. - Applying the same fix as for libssh above. + All raw response data needs to pass through this function. Which also + means that anyone in possession of such data may call + `Curl_xfer_write_resp()`. - - unit1395: fix `argument is null` and related issues [5]: - - stop calling `strcmp()` with NULL to avoid undefined behaviour. - - fix checking results if some of them were NULL. - - do not pass NULL to printf `%s`. + Closes #12480 - - ci: keep a build job with `-std=gnu89` to continue testing for - C89-compliance. We can apply this to other gcc jobs as needed. - Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 +Daniel Stenberg (13 Jan 2024) - [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l - ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b - [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 - 2346#step:6:67 - [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 - 2346#step:30:214 - [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 - 2346#step:29:895 - [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 - 2346#step:33:1689 +- RELEASE-NOTES: synced - Closes #12346 +- TODO: TFTP doesn't convert LF to CRLF for mode=netascii -- autotools: fix/improve gcc and Apple clang version detection + Closes #12655 + Closes #12690 - - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu - may return `n-win32` (also with `-dumpfullversion`). Causing these - errors and failing to enable picky warnings: - ``` - ../configure: line 23845: test: : integer expression expected - ``` - Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 - 3718#step:5:143 +- gen: do italics/bold for a range of letters, not just single word - Fix that by stripping any dash-suffix and handling a dotless (major-only) - version number by assuming `.0` in that case. + Previously it would match only on a sequence of non-space, which made it + miss to highlight for example "public suffix list". - `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` - Ref: https://github.com/mamedev/mame/pull/9767 + Updated the recent cookie.d edit from 5da57193b732 to use bold instead + of italics. - - fix Apple clang version detection for releases between - 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the - version was under-detected as 3.7 llvm/clang equivalent. + Closes #12689 - - fix Apple clang version detection for 'Apple clang version 11.0.0' - and newer where the Apple clang version was detected, instead of its - llvm/clang equivalent. +- docs: describe and highlight super cookies - - display detected clang/gcc/icc compiler version. + Reported-by: Yadhu Krishna M - Via libssh2: - - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d - 89875ad - https://github.com/libssh2/libssh2/pull/1187 - - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 - 42b592e - https://github.com/libssh2/libssh2/pull/1232 + Closes #12687 - Closes #12362 +- configure: when enabling QUIC, check that TLS supports QUIC -- autotools: delete LCC compiler support bits + Most importantly perhaps is when using OpenSSL that the used + build/flavor has the QUIC API: the vanilla OpenSSL does not, only + BoringSSL, libressl, AWS-LC and quictls do. - Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 + Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7f + e57e#r136780413 - Closes #12357 + Closes #12683 -- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` +Stefan Eissing (11 Jan 2024) - - tests: verify CMake `DISABLE` options. +- vquic: extract TLS setup into own source - Make an exception for 2 CMake-only ones, and one more that's - using a different naming scheme, also in autotools and source. + - separate ngtcp2 specific parts out + - provide callback during init to allow ngtcp2 to apply its defaults - - cmake: add support for `CURL_DISABLE_HEADERS_API`. + Closes #12678 - Suggested-by: Daniel Stenberg - Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 +Sergey Markelov (11 Jan 2024) - Closes #12353 +- multi: remove total timer reset in file_do() while fetching file:// -Jacob Hoffman-Andrews (20 Nov 2023) + The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts + with resetting the timer that is a start point for further multi states. + If file://, MSTATE_DO calls file_do() that should not reset the total + timer. Otherwise, the total time is always less than the pre-transfer + and the start transfer times. -- hyper: temporarily remove HTTP/2 support + Closes #12682 - The current design of the Hyper integration requires rebuilding the - Hyper clientconn for each request. However, building the clientconn - requires resending the HTTP/2 connection preface, which is incorrect - from a protocol perspective. That in turn causes servers to send GOAWAY - frames, effectively degrading performance to "no connection reuse" in - the best case. It may also be triggering some bugs where requests get - dropped entirely and reconnects take too long. +Daniel Stenberg (11 Jan 2024) - This doesn't rule out HTTP/2 support with Hyper, but it may take a - redesign of the Hyper integration in order to make things work. +- http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT - Closes #12191 + Extended test 80 to verify this. -Jay Satiro (20 Nov 2023) + Reported-by: Stefan Eissing + Fixes #12680 + Closes #12681 -- schannel: fix unused variable warning +- sectransp: do verify_cert without memdup for blobs - Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 - Reported-by: Viktor Szakats + Since the information is then already stored in memory, this can avoid + an extra set of malloc + free calls. - Closes https://github.com/curl/curl/pull/12361 + Closes #12679 -Daniel Stenberg (19 Nov 2023) +- hsts: remove assert for zero length domain -- url: find scheme with a "perfect hash" + A zero length domain can happen if the HSTS parser is given invalid + input data which is not unheard of and is done by the fuzzer. - Instead of a loop to scan over the potentially 30+ scheme names, this - uses a "perfect hash" table. This works fine because the set of schemes - is known and cannot change in a build. The hash algorithm and table size - is made to only make a single scheme index per table entry. + Follow-up from cfe7902111ae547873 - The perfect hash is generated by a separate tool (scripts/schemetable.c) + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661 - Closes #12347 + Closes #12676 -- scripts: add schemetable.c +- headers: make sure the trailing newline is not stored - This tool generates a scheme-matching table. + extended test1940 to verify blank header fields too - It iterates over a number of different initial and shift values in order - to find the hash algorithm that needs the smallest possible table. + Bug: https://curl.se/mail/lib-2024-01/0019.html + Reported-by: Dmitry Karpov + Closes #12675 - The generated hash function, table and table size then needs to be used - by the url.c:Curl_getn_scheme_handler() function. +- curl_easy_header.3: tiny language fix -Stefan Eissing (19 Nov 2023) + Closes #12672 -- vtls/vquic, keep peer name information together +- examples/range.c: add - - add `struct ssl_peer` to keep hostname, dispname and sni - for a filter - - allocate `sni` for use in VTLS backend - - eliminate `Curl_ssl_snihost()` and its use of the download buffer - - use ssl_peer in SSL and QUIC filters + Closes #12671 - Closes #12349 +- examples/netrc.c: add -Viktor Szakats (18 Nov 2023) + Closes #12671 -- build: always revert `#pragma GCC diagnostic` after use +- examples/ipv6.c: new example showing IPv6-only internet transfer - Before this patch some source files were overriding gcc warning options, - but without restoring them at the end of the file. In CMake UNITY builds - these options spilled over to the remainder of the source code, - effecitvely disabling them for a larger portion of the codebase than - intended. + Closes #12671 - `#pragma clang diagnostic` didn't have such issue in the codebase. +- examples/address-scope.c: renamed from ipv6.c - Reviewed-by: Marcel Raad - Closes #12352 + It shows address scope use really -- tidy-up: casing typos, delete unused Windows version aliases + Closes #12671 - - cmake: fix casing of `UnixSockets` to match the rest of the codebase. +Stefan Eissing (9 Jan 2024) - - curl-compilers.m4: fix casing in a comment. +- multi: pollset adjust, init with FIRSTSOCKET during connect - - setup-win32: delete unused Windows version constant aliases. + - `conn->sockfd` is set by `Curl_setup_transfer()`, but that + is called *after* the connection has been established + - use `conn->sock[FIRSTSOCKET]` instead - Reviewed-by: Marcel Raad - Closes #12351 + Follow-up to a0f94800d507de + Closes #12664 -- keylog: disable if unused +Daniel Stenberg (9 Jan 2024) - Fully disable keylog code if there is no TLS or QUIC subsystem using it. +- WEBSOCKET.md: remove dead link - Closes #12350 +- CI: spellcheck/appveyor: invoke configure --without-libpsl -- cmake: add `CURL_DISABLE_BINDLOCAL` option + Follow-up to 2998874bb61ac6 - To match similar autotools option. +- cmdline/docs/*.d: switch to using ## instead of .IP - Default is `ON`. + To make the editing easier. To write and to read. - Reviewed-by: Daniel Stenberg - Closes #12345 + Closes #12667 -- url: fix `-Wzero-length-array` with no protocols +- gen.pl: support ## for doing .IP in table-like lists - Fixes: - ``` - ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ - -Wc2x-extensions] - 178 | static const struct Curl_handler * const protocols[] = { - | ^ - ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length - -array] - ``` + Warn on use of .RS/.IP/.RE - Closes #12344 + Closes #12667 -- url: fix builds with `CURL_DISABLE_HTTP` +Jay Satiro (9 Jan 2024) - Fixes: - ``` - ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' - 456 | Curl_mime_cleanpart(data->state.formp); - | ~~~~~~~~~~~ ^ - ``` +- cookie.d: Document use of empty string to enable cookie engine - Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 + - Explain that --cookie "" can be used to enable the cookie engine + without reading any initial cookies. - Closes #12343 + As is documented in CURLOPT_COOKIEFILE. -- http: fix `-Wunused-parameter` with no auth and no proxy + Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html - ``` - lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] - bool proxy) - ^ - ``` + Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420 + Reported-by: janko-js@users.noreply.github.com - Reviewed-by: Marcel Raad - Closes #12338 + Closes https://github.com/curl/curl/pull/12646 -Daniel Stenberg (16 Nov 2023) +Daniel Stenberg (9 Jan 2024) -- TODO: Some TLS options are not offered for HTTPS proxies +- setopt: use memdup0 when cloning COPYPOSTFIELDS - Closes #12286 - Closes #12342 + Closes #12651 -- RELEASE-NOTES: synced +- telnet: use dynbuf instad of malloc for escape buffer -- duphandle: make dupset() not return with pointers to old alloced data + Previously, send_telnet_data() would malloc + free a buffer every time + for escaping IAC codes. Now, it reuses a dynbuf for this purpose. - As the blob pointers are to be duplicated, the function must not return - mid-function with lingering pointers to the old handle's allocated data, - as that would lead to double-free in OOM situations. + Closes #12652 - Make sure to clear all destination pointers first to avoid this risk. +- CI: install libpsl or configure --without-libpsl in builds - Closes #12337 + As a follow-up to the stricted libpsl check in configure -Viktor Szakats (16 Nov 2023) +- configure: make libpsl detection failure cause error -- http: fix `-Wunused-variable` compiler warning + To force users to explictily disable it if they really don't want it + used and make it harder to accidentally miss it. - Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. + --without-libpsl is the option to use if PSL is not wanted. - E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + - `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + - `CURL_DISABLE_NTLM` on non-Windows. + Closes #12661 - ``` - ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl - e] - CURLcode result = CURLE_OK; - ^ - ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus - ed-but-set-variable] - unsigned long *availp; - ^ - ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse - d-but-set-variable] - struct auth *authp; - ^ - ``` +- RELEASE-NOTES: synced - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 +- pop3: replace calloc + memcpy with memdup0 - Fixes #12228 - Closes #12335 + ... and make sure to return error on out of memory. -Jay Satiro (16 Nov 2023) + Closes #12650 -- tool: support bold headers in Windows +- lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT - - If virtual terminal processing is enabled in Windows then use ANSI - escape codes Esc[1m and Esc[22m to turn bold on and off. + Closes #12658 - Suggested-by: Gisle Vanem +- mime: use memdup0 instead of malloc + memcpy - Ref: https://github.com/curl/curl/discussions/11770 + Closes #12649 - Closes https://github.com/curl/curl/pull/12321 +- tool_getparam: move the --rate logic into set_rate() -Viktor Szakats (15 Nov 2023) +- tool_getparam: switch to an enum for every option -- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` + To make the big switch much easier to read/understand and to make it + easier to add new options. - Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + - `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, - or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while - compiling due to a missing HMAC declaration. +- tool_getparam: build post data using dynbuf (more) - The reason is that HMAC is required by `lib/sha256.c` which publishes - `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when - building for libssh2 v1.8.2 (2019-05-25) or older. +- tool_getparam: replace malloc + copy by dynbuf for --data - Make sure to compile the HMAC bits for a successful build. +- tool_getparam: make data_urlencode avoid direct malloc - Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting - them into separate sources isn't practical. + use aprintf() instead - Fixes: - ``` - [...] - In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc - url_object.dir/Unity/unity_0_c.c:310: - ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s - truct HMAC_params' - 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { - | ^ - ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par - ams' - [...] - ``` +- tool_getparam: move the --url-query logic into url_query() - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 + This function is not doing post at all so it was always weirdly placed. - Fixes #12273 - Closes #12332 +- tool_getparam: move the --data logic into set_data() -Daniel Stenberg (15 Nov 2023) +- tool_getparam: unify the cmdline switch() into a single one -- duphandle: also free 'outcurl->cookies' in error path + - easier to follow, easier to modify, easier to extend, possibly slightly + faster - Fixes memory-leak when OOM mid-function + - each case now has the long option as a comment - Use plain free instead of safefree, since the entire struct is - freed below. +- tool_getparam: bsearch cmdline options - Remove some free calls that is already freed in Curl_freeset() + - the option names are now alpha sorted and lookup is a lot faster - Closes #12329 + - use case sensitive matching. It was previously case insensitive, but that + was not documented nor tested. -Viktor Szakats (15 Nov 2023) + - remove "partial match" feature. It was not documented, not tested and + was always fragile as existing use could break when we add a new + option -- config-win32: set `HAVE_SNPRINTF` for mingw-w64 + - lookup short options via a table - It's available in all mingw-w64 releases. We already pre-fill this - detection in CMake. + Closes #12631 - Closes #12325 +Gabe (8 Jan 2024) -- sasl: fix `-Wunused-function` compiler warning +- COPYING: update copyright year - In builds with disabled auths. + Closes #12654 - ``` - lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus - ed-function] - static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, - ^ - 1 warning generated. - ``` - Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st - ep:3:3822 +Stefan Eissing (8 Jan 2024) - Reviewed-by: Daniel Stenberg - Closes #12326 +- url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD -- build: picky warning updates + Also add more tracing to test 19 - - cmake: sync some picky gcc warnings with autotools. - - cmake, autotools: add `-Wold-style-definition` for clang too. - - cmake: more precise version info for old clang options. - - cmake: use `IN LISTS` syntax in `foreach()`. + Follow-up to a0f9480 - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #12324 + Fixes #12657 + Closes #12659 -Daniel Stenberg (15 Nov 2023) +Daniel Stenberg (8 Jan 2024) -- urldata: move cookielist from UserDefined to UrlState +- connect: remove margin from eyeballer alloc - 1. Because the value is not strictly set with a setopt option. + Presumably leftovers from debugging - 2. Because otherwise when duping a handle when all the set.* fields are - first copied and an error happens (think out of memory mid-function), - the function would easily free the list *before* it was deep-copied, - which could lead to a double-free. + Closes #12647 - Closes #12323 +- ftp: only consider entry path if it has a length -Viktor Szakats (14 Nov 2023) + Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb -- autotools: avoid passing `LDFLAGS` twice to libcurl + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631 - autotools passes `LDFLAGS` automatically linker commands. curl's - `lib/Makefile.am` customizes libcurl linker flags. In that - customization, it added `LDFLAGS` to the custom flags. This resulted in - passing `LDFLAGS` _twice_ to the `libtool` command. + Avoids a NULL pointer deref. - Most of the time this is benign, but some `LDFLAGS` options can break - the build when passed twice. One such example is passing `.o` files, - e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for - MUSL builds. + Closes #12648 - Passing them twice resulted in duplicate symbol errors: - ``` - libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a - arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] - ld.lld-15: error: duplicate symbol: _start - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - [...] - clang: error: linker command failed with exit code 1 (use -v to see invocatio - n) - ``` +Stefan Eissing (7 Jan 2024) - This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. - The patch was a works-for-me hack that ended up merged in curl: - https://sourceforge.net/p/curl/bugs/1217/#06ef - With the root cause remaining unclear. +- transfer: adjust_pollset improvements - Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring - `-L` first? This would be unusual and suggests a bug in either the - linker or in `libtool`. + - let `multi_getsock()` initialize the pollset in what the + transfer state requires in regards to SEND/RECV + - change connection filters `adjust_pollset()` implementation + to react on the presence of POLLIN/-OUT in the pollset and + no longer check CURL_WANT_SEND/CURL_WANT_RECV + - cf-socket will no longer add POLLIN on its own + - http2 and http/3 filters will only do adjustments if the + passed pollset wants to POLLIN/OUT for the transfer on + the socket. This is similar to the HTTP/2 proxy filter + and works in stacked filters. - The curl build does pass the list of detected libs via its own - `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` - command line, but it's the job of `libtool` to ensure that even - a peculiar linker gets the options in the expected order. Also because - autotools passes `LDFLAGS` last, making it hardly possible to pass - anything after it. + Closes #12640 - Perhaps in the 10 years since this issue, this already got a fix - upstream. +Daniel Stenberg (6 Jan 2024) - This patch deletes `LDFLAGS` from our customized libcurl options, - leaving a single copy of them as passed by autotools automatically. +- ftp: use memdup0 to store the OS from a SYST 215 response - Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - Closes #12310 + avoid malloc + direct buffer fiddle -- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` + Closes #12639 - To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and - curl tool (`CURL_LDFLAGS_BIN`). +- ftp: use dynbuf to store entrypath - This makes it possible to build libcurl and curl with a single - invocation with lib- and tool-specific custom linker flags. + avoid direct malloc - Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, - controlling static/shared, incl. requesting a static curl tool (with - `-static-libtool-libs`) while building both shared and static libcurl. + Closes #12638 - curl-for-win uses the above and some more. +Lealem Amedie (6 Jan 2024) - These options are already supported in `Makefile.mk`. CMake has built-in - variables for this. +- wolfssl: load certificate *chain* for PEM client certs - Closes #12312 + Closes #12634 -Jay Satiro (14 Nov 2023) +Stefan Eissing (4 Jan 2024) -- tool_cb_hdr: add an additional parsing check +- http: adjust_pollset fix - - Don't dereference the past-the-end element when parsing the server's - Content-disposition header. + do not add a socket for POLLIN when the transfer does not want to send + (for example is paused). - As 'p' is advanced it can point to the past-the-end element and prior - to this change 'p' could be dereferenced in that case. + Follow-up to 47f5b1a - Technically the past-the-end element is not out of bounds because dynbuf - (which manages the header line) automatically adds a null terminator to - every buffer and that is not included in the buffer length passed to - the header callback. + Reported-by: bubbleguuum on github + Fixes #12632 + Closes #12633 - Closes https://github.com/curl/curl/pull/12320 +Daniel Stenberg (3 Jan 2024) -Philip Heiduck (14 Nov 2023) +- tool: make parser reject blank arguments if not supported -- .cirrus.yml: freebsd 14 + Already in the getstr() function that clones the input argument. - ensure curl works on latest freebsd version + Closes #12620 - Closes #12053 +dependabot[bot] (3 Jan 2024) -Daniel Stenberg (13 Nov 2023) +- build(deps): bump github/codeql-action from 2 to 3 -- easy: in duphandle, init the cookies for the new handle + Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 + to 3. + - [Release notes](https://github.com/github/codeql-action/releases) + - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) + - [Commits](https://github.com/github/codeql-action/compare/v2...v3) - ... not the source handle. + --- + updated-dependencies: + - dependency-name: github/codeql-action + dependency-type: direct:production + update-type: version-update:semver-major + ... - Closes #12318 + Signed-off-by: dependabot[bot] -- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set + Closes #12625 - Previously it would unconditionally use the size, which is set to -1 - when strlen is requested. +- build(deps): bump actions/checkout from 3 to 4 - Updated test 544 to verify. + Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. + - [Release notes](https://github.com/actions/checkout/releases) + - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) + - [Commits](https://github.com/actions/checkout/compare/v3...v4) - Closes #12317 + --- + updated-dependencies: + - dependency-name: actions/checkout + dependency-type: direct:production + update-type: version-update:semver-major + ... -- RELEASE-NOTES: synced + Signed-off-by: dependabot[bot] -- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped + Closes #12624 - Closes #12315 +- build(deps): bump actions/upload-artifact from 3 to 4 -- urldata: move hstslist from 'set' to 'state' + Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) f + rom 3 to 4. + - [Release notes](https://github.com/actions/upload-artifact/releases) + - [Commits](https://github.com/actions/upload-artifact/compare/v3...v4) - To make it work properly with curl_easy_duphandle(). This, because - duphandle duplicates the entire 'UserDefined' struct by plain copy while - 'hstslist' is a linked curl_list of file names. This would lead to a - double-free when the second of the two involved easy handles were - closed. + --- + updated-dependencies: + - dependency-name: actions/upload-artifact + dependency-type: direct:production + update-type: version-update:semver-major + ... - Closes #12315 + Signed-off-by: dependabot[bot] -- test1900: verify duphandle with HSTS using multiple files + Closes #12627 - Closes #12315 +- build(deps): bump actions/download-artifact from 3 to 4 -Goro FUJI (13 Nov 2023) + Bumps [actions/download-artifact](https://github.com/actions/download-artifac + t) from 3 to 4. + - [Release notes](https://github.com/actions/download-artifact/releases) + - [Commits](https://github.com/actions/download-artifact/compare/v3...v4) -- http: allow longer HTTP/2 request method names + --- + updated-dependencies: + - dependency-name: actions/download-artifact + dependency-type: direct:production + update-type: version-update:semver-major + ... - - Increase the maximum request method name length from 11 to 23. + Signed-off-by: dependabot[bot] - For HTTP/1.1 and earlier there's not a specific limit in libcurl for - method length except that it is limited by the initial HTTP request - limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same - and there was no specific limit. + Closes #12626 - According to Internet Assigned Numbers Authority (IANA) the longest - registered method is UPDATEREDIRECTREF which is 17 characters. +Stefan Eissing (3 Jan 2024) - Also there are unregistered methods used by some companies that are - longer than 11 characters. +- http3/quiche: fix result code on a stream reset - The limit was originally added by 61f52a97 but not used until fc2f1e54. + - fixes pytest failures in test 07_22 + - aligns CURLcode values on stream reset with ngtcp2 - Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml + Closes #12629 - Closes https://github.com/curl/curl/pull/12311 +Daniel Stenberg (2 Jan 2024) -Jay Satiro (12 Nov 2023) +- setopt: clear mimepost when formp is freed -- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does + A precaution to avoid a possibly dangling pointer left behind. - - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB - and CURLOPT_PROXY_CAINFO_BLOB docs. + Reported-by: Thomas Ferguson + Fixes #12608 + Closes #12621 - All the other _BLOB option docs already have the same explanation. +Andy Alt (2 Jan 2024) - Closes https://github.com/curl/curl/pull/12277 +- CI: Add dependabot.yml -Viktor Szakats (11 Nov 2023) + This will cause dependabot to open a PR when various actions are + updated, provided that the action maintainer has issued a release. -- tidy-up: dedupe Windows system libs in cmake + Closes #12623 - Reviewed-by: Daniel Stenberg - Closes #12307 +Gisle Vanem (2 Jan 2024) -Junho Choi (11 Nov 2023) - -- ci: test with latest quiche release (0.19.0) +- content_encoding: change return code to typedef'ed enum - Closes #12180 + ... to work around a clang ubsan warning. -- quiche: use quiche_conn_peer_transport_params() + Fixes #12618 + Closes #12622 - In recent quiche, transport parameter API is separated - with quiche_conn_peer_transport_params(). - (https://github.com/cloudflare/quiche/pull/1575) - It breaks with bulding with latest(post 0.18.0) quiche. +Daniel Stenberg (2 Jan 2024) - Closes #12180 +- tool: prepend output_dir in header callback -Daniel Stenberg (11 Nov 2023) + When Content-Disposition parsing is used and an output dir is prepended, + make sure to store that new file name correctly so that it can be used + for setting the file timestamp when --remote-time is used. -- Makefile: generate the VC 14.20 project files at dist-time + Extended test 3012 to verify. - Follow-up to 28287092cc5a6d6ef8 (#12282) + Co-Authored-by: Jay Satiro + Reported-by: hgdagon on github + Fixes #12614 + Closes #12617 - Closes #12290 +- test1254: fix typo in name plus shorten it -Sam James (11 Nov 2023) +- RELEASE-NOTES: synced -- misc: fix -Walloc-size warnings +Viktor Szakats (2 Jan 2024) - GCC 14 introduces a new -Walloc-size included in -Wextra which gives: +- schannel: fix `-Warith-conversion` gcc 13 warning ``` - src/tool_operate.c: In function ‘add_per_transfer’: - src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo - r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] - 213 | p = calloc(sizeof(struct per_transfer), 1); - | ^ - src/var.c: In function ‘addvariable’: - src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type - struct var’ with size ‘32’ [-Walloc-size] - 361 | p = calloc(sizeof(struct var), 1); - | ^ + lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' + may change the sign of the result [-Warith-conversion] + 1201 | *extension_len = *list_len + + | ^ ``` - The calloc prototype is: + Closes #12616 + +- asyn-thread: silence `-Wcast-align` warning for Windows + + Seen with llvm/clang 17: + ``` + lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct + thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align] + 310 | CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlap + ped); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~ + .../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded fro + m macro 'CONTAINING_RECORD' + 717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(addre + ss) - (ULONG_PTR)(&((type *)0)->field))) + | ^~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``` - void *calloc(size_t nmemb, size_t size); - ``` - So, just swap the number of members and size arguments to match the - prototype, as we're initialising 1 struct of size `sizeof(struct - ...)`. GCC then sees we're not doing anything wrong. + Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482 - Closes #12292 + Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261 + Closes #12615 -Mark Gaiser (11 Nov 2023) +Daniel Stenberg (2 Jan 2024) -- IPFS: bugfixes +- tool_listhelp: regenerate after recent .d updates - - Fixed endianness bug in gateway file parsing - - Use IPFS_PATH in tests where IPFS_DATA was used - - Fixed typos from traling -> trailing - - Fixed broken link in IPFS.md + Makes it survive test 1478 - Follow-up to 859e88f6533f9e + Closes #12612 - Reported-by: Michael Kaufmann - Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 - Closes #12305 +- test1478: verify src/tool_listhelp.c -Daniel Stenberg (11 Nov 2023) + Verify that the source file on disk is identical to the output of gen.pl + listhelp, as otherwise they are out of sync and need attention. -- VULN-DISCLOSURE-POLIC: remove broken link to hackerone + Closes #12612 - It should ideally soon not be done from hackerone anyway +- testutil: make runtests support %include - Closes #12308 + Using this instruction, a test case can include the contents of a file + into the test during the preprocessing. -Andrew Kurushin (11 Nov 2023) + Closes #12612 -- schannel: add CA cache support for files and memory blobs +- runtests: for mode="text" on , fix newlines on both parts - - Support CA bundle and blob caching. + Closes #12612 - Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. +Jay Satiro (2 Jan 2024) - Closes https://github.com/curl/curl/pull/12261 +- quiche: return CURLE_HTTP3 on send to invalid stream -Daniel Stenberg (10 Nov 2023) + Prior to this change if a send failed on a stream in an invalid state + (according to quiche) and not marked as closed (according to libcurl) + then the send function would return CURLE_SEND_ERROR. -- RELEASE-NOTES: synced + We already have similar code for ngtcp2 to return CURLE_HTTP3 in this + case. -Charlie C (10 Nov 2023) + Caught by test test_07_upload.py: test_07_22_upload_parallel_fail. -- cmake: option to disable install & drop `curlu` target when unused + Fixes https://github.com/curl/curl/issues/12590 + Closes https://github.com/curl/curl/pull/12597 - This patch makes the following changes: - - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. - - Removes the target `curlu` when the option `BUILD_TESTING` is set to - `OFF` - to prevent it from being loaded in Visual Studio. +Daniel Stenberg (1 Jan 2024) - Closes #12287 +- cmdline-opts: update availability for the *-ca-native options -Kai Pastor (10 Nov 2023) + Closes #12613 -- cmake: fix multiple include of CURL package +Patrick Monnerat (31 Dec 2023) - Fixes errors on second `find_package(CURL)`. This is a frequent case - with transitive dependencies: - ``` - CMake Error at ...: - add_library cannot create ALIAS target "CURL::libcurl" because another - target with the same name already exists. - ``` +- openldap: fix STARTTLS - Test to reproduce: - ```cmake - cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher + It was not working anymore since introduction of connection filters. - project(curl) + Also do not attempt to recover from a failing TLS negotiation with + CURLUSESSL_TRY. - set(CURL_DIR "example/lib/cmake/CURL/") - find_package(CURL CONFIG REQUIRED) - find_package(CURL CONFIG REQUIRED) # fails + Closes #12610 - add_executable(main main.c) - target_link_libraries(main CURL::libcurl) - ``` +Daniel Stenberg (31 Dec 2023) - Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes - Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html - Ref: #12300 - Assisted-by: Harry Mallon - Closes #11913 +- haproxy-clientip.d: document the arg -Viktor Szakats (8 Nov 2023) + The arg keyword was missing and therefore not present in the man page. -- tidy-up: use `OPENSSL_VERSION_NUMBER` + Closes #12611 - Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. - Before this patch some places used `OPENSSL_VERSION_MAJOR`. +annalee (29 Dec 2023) - Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't - define any version number in these implementations: BoringSSL, AWS-LC, - LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to - `opensslv.h`. This wasn't causing a deeper problem because the code is - looking for v3, which is only provided by OpenSSL/quictls as of now. +- configure: fix no default int compile error in ipv6 detection - According to https://github.com/openssl/openssl/issues/17517, the macro - `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. + Closes #12607 - Reviewed-by: Marcel Raad - Closes #12298 +Dan Fandrich (28 Dec 2023) -Daniel Stenberg (8 Nov 2023) +- CI: Fix use of any-glob-to-all-files in the labeler -- resolve.d: drop a multi use-sentence + Despite its name, this atom acts like one-glob-to-all-files and a + different syntax with braces must be used to get + any-glob-to-all-files semantics. Unfortunately, this makes the file + completely unreadable. - Since the `multi:` keyword adds that message. + Ref: https://github.com/actions/labeler/issues/731 - Reported-by: 積丹尼 Dan Jacobson - Fixes https://github.com/curl/curl/discussions/12294 - Closes #12295 +Daniel Stenberg (29 Dec 2023) -- content_encoding: make Curl_all_content_encodings allocless +- CURLOPT_AUTOREFERER.3: mention CURLINFO_REFERER - - Fixes a memory leak pointed out by Coverity - - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? - id=63947 - - Avoids unncessary allocations +- CURLINFO_REFERER.3: clarify that it is the *request* header - Follow-up ad051e1cbec68b2456a22661b + That libcurl itself sent in the most recent request - Closes #12289 + Closes #12605 -Michael Kaufmann (7 Nov 2023) +Jay Satiro (28 Dec 2023) -- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 +- system_win32: fix a function pointer assignment warning - Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), - avoid it and use "http/1.1" instead. + - Use CURLX_FUNCTION_CAST to suppress a function pointer assignment + warning. - This reverts commit df856cb5c9 (#10183). + a6bbc87f added lookups of some Windows API functions and then cast them + like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn + about that as breaking strict-aliasing rules so this PR changes those + assignments to use CURLX_FUNCTION_CAST. - Fixes #12259 - Closes #12285 + Bug: https://github.com/curl/curl/pull/12581#issuecomment-1869804317 + Reported-by: Marcel Raad -Daniel Stenberg (7 Nov 2023) + Closes https://github.com/curl/curl/pull/12602 -- Makefile.am: drop vc10, vc11 and vc12 projects from dist +- verify-examples.pl: fail verification on unescaped backslash - They are end of life products. Support for generating them remain in the - repo for a while but this change drops them from distribution. + - Check that all backslashes in EXAMPLE are properly escaped. - Closes #12288 + eg manpage must always use `\\n` never `\n`. -David Suter (7 Nov 2023) + This is because the manpage requires we always double blackslash to show + a single backslash. Prior to this change an erroneous single backslash + would pass through and compile even though it would not show correctly + in the manpage. -- projects: add VC14.20 project files + Co-authored-by: Daniel Stenberg - Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. - OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why - this is missing. Updated the templates to produce a VC14.20 project. - Project opens in Visual Studio 2019 as expected. + Ref: https://github.com/curl/curl/pull/12588 - Closes #12282 + Closes https://github.com/curl/curl/pull/12589 -Daniel Stenberg (7 Nov 2023) +- vtls: fix missing multissl version info -- curl: move IPFS code into src/tool_ipfs.[ch] + - Fix erroneous buffer copy logic from ff74cef5. - - convert ensure_trailing into ensure_trailing_slash - - strdup the URL string to own it proper - - use shorter variable names - - combine some expressions - - simplify error handling in ipfs_gateway() - - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached - - ipfs-gateway.d polish and simplification - - shorten ipfs error message + make them "synthetic" + Prior to this change the MultiSSL version info returned to the user + was empty. - Closes #12281 + Closes https://github.com/curl/curl/pull/12599 -Viktor Szakats (6 Nov 2023) +Daniel Stenberg (27 Dec 2023) -- build: delete support bits for obsolete Windows compilers +- KNOWN_BUGS: [RTSP] Some methods do not support response bodies - - Pelles C: Unclear status, failed to obtain a fresh copy a few months - ago. Possible website is HTTP-only. ~10 years ago I left this compiler - dealing with crashes and other issues with no response on the forum - for years. It has seen some activity in curl back in 2021. - - LCC: Last stable release in September 2002. - - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned - in 2006. - - Borland C++: We dropped Borland C++ support in 2018. - - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 - (or possibly 2008) as a minimum. + Closes #12414 - Closes #12222 +Patrick Monnerat (27 Dec 2023) -- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` +- openldap: fix an LDAP crash - We use `stdint.h` unconditionally in all places except one. These uses - are imposed by external dependencies / features. nghttp2, quic, wolfSSL - and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that - any of these features make curl require a C99 compiler. (In case of - MSVC, this means Visual Studio 2010 or newer.) + Reported-by: Ozan Cansel + Fixes #12593 + Closes #12600 - This patch changes the single use of `stdint.h` guarded by - `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using - `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used - anywhere else, allowing to delete this feature check as well. +Daniel Stenberg (27 Dec 2023) - Closes #12275 +- getinfo: CURLINFO_QUEUE_TIME_T -Daniel Stenberg (6 Nov 2023) + Returns the time, in microseconds, during which this transfer was held + in a waiting queue before it started "for real". A transfer might be put + in a queue if after getting started, it cannot create a new connection + etc due to set conditions and limits imposed by the application. -- tool_operate: do not mix memory models + Ref: #12293 + Closes #12368 - Make sure 'inputpath' only points to memory allocated by libcurl so that - curl_free works correctly. +- RELEASE-NOTES: synced - Pointed out by Coverity +Jay Satiro (26 Dec 2023) - Follow-up to 859e88f6533f9e1f890 +- examples/sendrecv: fix comment line length - Closes #12280 + Caught by checksrc. -Stefan Eissing (6 Nov 2023) +Haydar Alaidrus (23 Dec 2023) -- lib: client writer, part 2, accounting + logging +- CURLOPT_POSTFIELDS.3: fix incorrect C string escape in example - This PR has these changes: + - Escape inner quotes with two backslashes. - Renaming of unencode_* to cwriter, e.g. client writers - - documentation of sendf.h functions - - move max decode stack checks back to content_encoding.c - - define writer phase which was used as order before - - introduce phases for monitoring inbetween decode phases - - offering default implementations for init/write/close + Two backslashes escapes the backslash for the man page and will show as + a single backslash. - Add type paramter to client writer's do_write() - - always pass all writes through the writer stack - - writers who only care about BODY data will pass other writes unchanged + eg: "{\\"name\\": \\"daniel\\"}" shows as "{\"name\": \"daniel\"}". - add RAW and PROTOCOL client writers - - RAW used for Curl_debug() logging of CURLINFO_DATA_IN - - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and - Curl_pgrsSetDownloadCounter() - - remove all updates of data->req.bytecount and calls to - Curl_pgrsSetDownloadCounter() and Curl_debug() from other code - - adjust test457 expected output to no longer see the excess write + Closes https://github.com/curl/curl/pull/12588 - Closes #12184 +Viktor Szakats (23 Dec 2023) -Daniel Stenberg (6 Nov 2023) +- appveyor: tidy-ups -- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw + - replace two remaining backslashes with forward slashes. + - tidy up the way we form and pass `TFLAGS`. - Closes #12278 + Follow-up to 2d4d0c1fd32f5cc3f946c407c8eccd5477b287df #12572 -Viktor Szakats (6 Nov 2023) + Closes #12582 -- rand: fix build error with autotools + LibreSSL +Stefan Eissing (22 Dec 2023) - autotools unexpectedly detects `arc4random` because it is also looking - into dependency libs. One dependency, LibreSSL, happens to publish an - `arc4random` function (via its shared lib before v3.7, also via static - lib as of v3.8.2). When trying to use this function in `lib/rand.c`, - its protoype is missing. To fix that, curl included a prototype, but - that used a C99 type without including `stdint.h`, causing: +- transfer: fix upload rate limiting, add test cases - ``` - ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' - 37 | uint32_t arc4random(void); - | ^ - 1 error generated. - ``` + - add test cases for rate limiting uploads for all + http versions + - fix transfer loop handling of limits. Signal a re-receive + attempt only on exhausting maxloops without an EAGAIN + - fix `data->state.selectbits` forcing re-receive to also + set re-sending when transfer is doing this. - This patch improves this by dropping the local prototype and instead - limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide - their own random source anyway. + Reported-by: Karthikdasari0423 on github + Fixes #12559 + Closes #12586 - The better fix would be to teach autotools to not link dependency libs - while detecting `arc4random`. +Daniel Stenberg (22 Dec 2023) - LibreSSL publishing a non-namespaced `arc4random` tracked here: - https://github.com/libressl/portable/issues/928 +- mbedtls: free the entropy when threaded - Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 + The entropy_free was never done for threaded builds, causing a small + (fixed) memory leak. - Reviewed-by: Daniel Stenberg - Fixes #12257 - Closes #12274 + Reported-by: RevaliQaQ on github + Fixes #12584 + Closes #12585 -Daniel Stenberg (5 Nov 2023) +Stefan Eissing (22 Dec 2023) -- RELEASE-NOTES: synced +- http2: improved on_stream_close/data_done handling -- strdup: do Curl_strndup without strncpy + - there seems to be a code path that cleans up easy handles without + triggering DONE or DETACH events to the connection filters. This + would explain wh nghttp2 still holds stream user data + - add GOOD check to easy handle used in on_close_callback to + prevent crashes, ASSERTs in debug builds. + - NULL the stream user data early before submitting RST + - add checks in on_stream_close() to identify UNGOOD easy handles - To avoid (false positive) gcc-13 compiler warnings. + Reported-by: Hans-Christian Egtvedt + Fixes #10936 + Closes #12562 - Follow-up to 4855debd8a2c1cb +Daniel Stenberg (22 Dec 2023) - Assisted-by: Jay Satiro - Reported-by: Viktor Szakats - Fixes #12258 +- mprintf: overhaul and bugfixes -Enno Boland (5 Nov 2023) + In a test case using lots of snprintf() calls using many commonly used + %-codes per call, this version is around 30% faster than previous + version. -- HTTP: fix empty-body warning + It also fixes the #12561 bug which made it not behave correctly when + given unknown %-sequences. Fixing that flaw required a different take on + the problem, which resulted in the new two-arrays model. - This change fixes a compiler warning with gcc-12.2.0 when - `-DCURL_DISABLE_BEARER_AUTH=ON` is used. + lib557: extended - Verify the #12561 fix and test more printf features - /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': - /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp - ty body in an 'else' statement [-Wempty-body] - 1147 | ; - | ^ + unit1398: fix test: It used a $ only for one argument, which is not + supported. - Closes #12262 + Fixes #12561 + Closes #12563 -Daniel Stenberg (5 Nov 2023) +Viktor Szakats (21 Dec 2023) -- openssl: identify the "quictls" backend correctly +- appveyor: replace PowerShell with bash + parallel autotools - Since vanilla OpenSSL does not support the QUIC API I think it helps - users to identify the correct OpenSSL fork in version output. The best - (crude) way to do that right now seems to be to check if ngtcp2 support - is enabled. + PowerShell works (after a steep development curve), but one property of + it stuck and kept causing unresolvable usability issues: With + `$ErrorActionPreference=Stop`, it does abort on failures, but shows only + the first line of the error message. In `Continue` mode, it shows the + full error message, but doesn't stop on all errors. Another issue is + PowerShell considering any stderr output as if the command failed (this + has been improved in 7.2 (2021-Nov), but fixed versions aren't running + in CI and will not be for a long time in all test images.) - Closes #12270 + Thus, we're going with bash. -Mark Gaiser (5 Nov 2023) + Also: + - use `-j2` with autotools tests, making them finish 5-15 minutes per + job faster. + - omit `POSIX_PATH_PREFIX`. + - use `WINDIR`. + - prefer forward slashes. -- curl: improved IPFS and IPNS URL support + Follow-up to: 75078a415d9c769419aed4153d3d525a8eba95af #11999 + Ref: #12444 - Previously just ipfs:// and ipns:// was supported, which is - too strict for some usecases. + Fixes #12560 + Closes #12572 - This patch allows paths and query arguments to be used too. - Making this work according to normal http semantics: +Pavel Pavlov (21 Dec 2023) - ipfs:///foo/bar?key=val - ipns:///foo/bar?key=val +- asyn-thread: use GetAddrInfoExW on >= Windows 8 - The gateway url support is changed. - It now only supports gateways in the form of: + For doing async DNS resolution instead of starting a thread for each + request. - http:///foo/bar - http:// + Fixes #12481 + Closes #12482 - Query arguments here are explicitly not allowed and trigger an intended - malformed url error. +Daniel Stenberg (21 Dec 2023) - There also was a crash when IPFS_PATH was set with a non trailing - forward slash. This has been fixed. +- strerror: repair get_winsock_error() - Lastly, a load of test cases have been added to verify the above. + It would try to read longer than the provided string and crash. - Reported-by: Steven Allen - Fixes #12148 - Closes #12152 + Follow-up to ff74cef5d4a0cf60106517a1c7384 + Reported-by: calvin2021y on github + Fixes #12578 + Closes #12579 -Harry Mallon (5 Nov 2023) +- CURLOPT_SSH_*_KEYFILE: clarify -- docs: KNOWN_BUGS cleanup + Closes #12554 - * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. - Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 +ivanfywang (21 Dec 2023) - * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. - Fixed in #11806 +- ngtcp2: put h3 at the front of alpn - * Remove mention of aws-sigv4 query empty value problems + Closes #12576 - * Remove mention of aws-sigv4 missing amz-content-sha256 - Fixed in #9995 +Daniel Stenberg (21 Dec 2023) -- http_aws_sigv4: canonicalise valueless query params +- test460: verify a command line using --expand with no argument - Fixes #8107 - Closes #12244 + This verifies the fix for #12565 -Michael Kaufmann (4 Nov 2023) +- tool_getparam: do not try to expand without an argument -- docs: preserve the modification date when copying the prebuilt man page + This would lead to a segfault. - The previously built man page "curl.1" must be copied with the original - modification date, otherwise the man page is never updated. + Fixes #12565 + Reported-by: Geeknik Labs + Closes #12575 - This fixes a bug that has been introduced with commit 2568441cab. +- RELEASE-NOTES: synced - Reviewed-by: Dan Fandrich - Reviewed-by: Daniel Stenberg + Bumped version to 8.6.0 because of changes - Closes #12199 +- Makefile.am: fix the MSVC project generation -Daniel Stenberg (4 Nov 2023) + It made the vcxproj files not get included in dist tarballs. -- docs: remove bold from some man page SYNOPSIS sections + Regression since 74423b5df4c8117891eb89 (8.5.0) - In the name of consistency + Reported-by: iAroc on github + Fixes #12564 + Closes #12567 - Closes #12267 +zengwei2000 (21 Dec 2023) -- openssl: two multi pointer checks should probably rather be asserts +- altsvc: free 'as' when returning error - ... so add the asserts now and consider removing the dynamic checks in a - future. + Closes #12570 - Ref: #12261 - Closes #12264 + Signed-off-by: zengwei -boilingoden (4 Nov 2023) +Viktor Szakats (20 Dec 2023) -- docs: add supported version for the json write-out +- build: fix `-Wconversion`/`-Wsign-conversion` warnings - xref: https://curl.se/changes.html#7_70_0 + Fix remaining warnings in examples and tests which are not suppressed + by the pragma in `lib/curl_setup.h`. - Closes #12266 + Silence a toolchain issue causing warnings in `FD_SET()` calls with + older Cygwin/MSYS2 builds. Likely fixed on 2020-08-03 by: + https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=5717262b8ecfed0f7f + ab63e2c09c78991e36f9dd -Viktor Szakats (3 Nov 2023) + Follow-up to 2dbe75bd7f3c36837aa06fd87a442bdf3fb7faef #12492 -- appveyor: make VS2008-built curl tool runnable + Closes #12557 - By linking the CRT statically. This avoids the error about missing - runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`. +- build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings - Closes #12263 + - enable `-Wsign-conversion` warnings, but also setting them to not + raise errors. + - fix `-Warith-conversion` warnings seen in CI. + These are triggered by `-Wsign-converion` and causing errors unless + explicitly silenced. It makes more sense to fix them, there just a few + of them. + - fix some `-Wsign-conversion` warnings. + - hide `-Wsign-conversion` warnings with a `#pragma`. + - add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build + basis. + - update a CI job to unhide them with the above macro: + https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3 -Stefan Eissing (3 Nov 2023) + Closes #12492 -- url: proxy ssl connection reuse fix +- cmake: tidy-up `OtherTests.cmake` - - tunnel https proxy used for http: transfers does - no check if proxy-ssl configuration matches - - test cases added, test_10_12 fails on 8.4.0 + - make more obvious which detection uses which prep steps. + - merge and streamline conditions. + - these should not alter detection results. - Closes #12255 + Also align log output messages from + `Macros.cmake` / `curl_internal_test` with rest of the build. -Jay Satiro (3 Nov 2023) + Closes #12551 -- curl_sspi: support more revocation error names in error messages +- appveyor: switch to out-of-tree builds - - Add these revocation errors to sspi error list: - CRYPT_E_NO_REVOCATION_DLL, CRYPT_E_NO_REVOCATION_CHECK, - CRYPT_E_REVOCATION_OFFLINE and CRYPT_E_NOT_IN_REVOCATION_DATABASE. + With cmake and autotools. - Prior to this change those error codes were not matched to their macro - name and instead shown as "unknown error". + Closes #12550 - Before: +Daniel Stenberg (19 Dec 2023) - schannel: next InitializeSecurityContext failed: - Unknown error (0x80092013) - The revocation function was - unable to check revocation because the revocation server was offline. +- DEPRECATE.md: mention that NTLM_WB no longer works - After: + Ref: #12479 + Closes #12553 - schannel: next InitializeSecurityContext failed: - CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was - unable to check revocation because the revocation server was offline. +- CURLOPT_SERVER_RESPONSE_TIMEOUT_MS: add - Bug: https://github.com/curl/curl/issues/12239 - Reported-by: Niracler Li + Proposed-by: Yifei Kong + Ref: https://curl.se/mail/lib-2023-11/0023.html + Closes #12369 - Closes https://github.com/curl/curl/pull/12241 +Viktor Szakats (18 Dec 2023) -- strdup: don't allow Curl_strndup to read past a null terminator +- build: more `-Wformat` fixes - - Use malloc + strncpy instead of Curl_memdup to dupe the string before - null terminating it. + - memdebug: update to not trigger `-Wformat-nonliteral` warnings. + - imap: mark `imap_sendf()` with `CURL_PRINTF()`. + - tool_msgs: mark static function with `CURL_PRINTF()`. - Prior to this change if Curl_strndup was passed a length longer than - the allocated string then it could copy out of bounds. + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - This change is for posterity. Curl_strndup was added in the parent - commit and currently none of the calls to it pass a length that would - cause it to read past the allocated length of the input. + Closes #12540 - Follow-up to d3b3ba35. +- windows: delete redundant headers - Closes https://github.com/curl/curl/pull/12254 + `winsock2.h` pulls in `windows.h`. `ws2tcpip.h` pulls in `winsock2.h`. + `winsock2.h` and `ws2tcpip.h` are also pulled by `curl/curl.h`. -Daniel Stenberg (2 Nov 2023) + Keep only those headers that are not already included, or the code under + it uses something from that specific header. -- lib: add and use Curl_strndup() + Closes #12539 - The Curl_strndup() function is similar to memdup(), but copies 'n' bytes - then adds a terminating null byte ('\0'). +- cmake: prefill/cache `HAVE_STRUCT_SOCKADDR_STORAGE` - Closes #12251 + Also add missing include to `OtherTests.cmake`. It didn't cause an issue + because the parent already included this earlier by chance. -- CURPOST_POSTFIELDS.3: add CURLOPT_COPYPOSTFIELDS in SEE ALSO + Closes #12537 -Stefan Eissing (2 Nov 2023) +Daniel Stenberg (18 Dec 2023) -- pytest: use lower count in repeat tests +- runner.pm: fix perl warning when running tests - - lower large iteration counts in some tests somewhat for - the same coverage with less duration + Use of uninitialized value $runner::gdbthis in numeric eq (==) at runner. + pm - Closes #12248 + Follow-up from 3dcf301752a09d9 -Daniel Stenberg (2 Nov 2023) + Closes #12549 -- RELEASE-NOTES: synced +- runtests: support -gl. Like -g but for lldb. -- docs: clarify that curl passes on input unfiltered + Follow-up to 63b5748 - ... for several options. + Invokes the test case via lldb instead of gdb. Since using gdb is such a + pain on mac, using lldb is sometimes less quirky. - Reported-by: Ophir Lojkine + Closes #12547 - Closes #12249 +- curl.h: add CURLE_TOO_LARGE -- urlapi: when URL encoding the fragment, pass in the right length + A new error code to be used when an internal field grows too large, like + when a dynbuf reaches its maximum. Previously it would return + CURLE_OUT_OF_MEMORY for this, which is highly misleading. - A benign bug because it would only add an extra null terminator. + Ref: #12268 + Closes #12269 - Made lib1560 get a test that runs this code. +- CI/circleci: disable MQTT in the HTTP-only build - Closes #12250 + And remove the use of configure options that don't actually exist -Stefan Eissing (2 Nov 2023) + Closes #12546 -- vtls: late clone of connection ssl config +Yedaya Katsman (18 Dec 2023) - - perform connection cache matching against `data->set.ssl.primary` - and proxy counterpart - - fully clone connection ssl config only when connection is used +- tests: respect $TMPDIR when creating unix domain sockets - Closes #12237 + When running on termux, where $TMPDIR isn't /tmp, running the tests + failed, since the server config tried creating sockets in /tmp, without + checking the temp dir config. Use the TMPDIR variable that makes it find + the correct directory everywhere [0] -- msh3: error when built with CURL_DISABLE_SOCKETPAIR set + [0] https://perldoc.perl.org/File::Temp#tempfile - Reported-by: Gisle Vanem - Closes #12252 - Fixes #12213 + Closes #12545 -Daniel Stenberg (2 Nov 2023) +Viktor Szakats (17 Dec 2023) -- hsts: skip single-dot hostname +- ssh: fix namespace of two local macros - Reported-by: Maksymilian Arciemowicz + Avoid using the libssh and libssh2 macro namespaces by prefixing + these local macro names with `CURL_`. - Closes #12247 + Follow-up to 413a0fedd02c8c6df1d294534b8c6e306fcca7a2 #12346 -- vtls: fix build without proxy + Reviewed-by: Daniel Stenberg + Closes #12544 - Follow-up to bf0e278a3c54bc7fee7360da17c +- cmake: whitespace tidy-up in `OtherTests.cmake` - closes #12243 + Closes #12538 -- docs/example/keepalive.c: show TCP keep-alive options +Mark Sinkovics (16 Dec 2023) - Closes #12242 +- cmake: fix generation for system name iOS -- lib1560: verify appending blank URL encoded query string + This PR fixes a problem that happens during CMake configuration when + the `CMAKE_SYSTEM_NAME` set to `iOS` and not `Darwin`. This value is + available (as far as I remember) version 3.14. The final solution + (thanks to @vszakats) is to use `APPLE` which contains all the Apple + platforms https://cmake.org/cmake/help/latest/variable/APPLE.html. -- urlapi: skip appending NULL pointer query + This issue was found when during vcpkg installation. Running command + `vcpkg install curl:arm64-ios` and `vcpkg install curl:x64-ios` failed + with message: + ``` + CMake Error: try_run() invoked in cross-compiling mode, please set the follow + ing cache variables appropriately: + HAVE_H_ERRNO_ASSIGNABLE_EXITCODE (advanced) + ``` + After this fix, I was able to compile the compile the binary without + any issue. - Reported-by: kirbyn17 on hackerone + In addition to that fix, this PR also contains an simplification to + check if the platform is not APPLE. - Closes #12240 + Co-authored-by: Viktor Szakats + Closes #12515 -- lib1560: verify setting host to "" with and without URL encode +Daniel Stenberg (16 Dec 2023) -- urlapi: avoid null deref if setting blank host to url encode +- RELEASE-NOTES: synced - Reported-by: kirbyn17 on hackerone +Baruch Siach (16 Dec 2023) - Closes #12240 +- gnutls: fix build with --disable-verbose -- dynbuf: assert for NULL pointer inputs + infof() parameters must be defined event with --disable-verbose since + commit dac293cfb702 ("lib: apache style infof and trace + macros/functions"). - Help us catch more mistakes. + Move also 'ptr' definition under !CURL_DISABLE_VERBOSE_STRINGS. - Closes #12238 - -- HTTP3: ngtcp2 builds are no longer experimental + Fixes the following build failure: - The other HTTP/3 backends are still experimental. + In file included from ../lib/sendf.h:29, + from vtls/gtls.c:44: + vtls/gtls.c: In function 'Curl_gtls_verifyserver': + vtls/gtls.c:841:34: error: 'version' undeclared (first use in this function); + did you mean 'session'? + 841 | gnutls_protocol_get_name(version), ptr); + | ^~~~~~~ - Closes #12235 + Closes #12505 -Stefan Eissing (31 Oct 2023) +Viktor Szakats (16 Dec 2023) -- vtls: cleanup SSL config management +- build: delete unused `HAVE_{GSSHEIMDAL,GSSMIT,HEIMDAL}` - - remove `Curl_ssl_get_config()`, no longer needed + Stop setting `HAVE_GSSHEIMDAL`, `HAVE_GSSMIT` and `HAVE_HEIMDAL`. + There was no place in the build system or source code that used them. - Closes #12204 + Reviewed-by: Daniel Stenberg + Closes #12506 -Daniel Stenberg (31 Oct 2023) +- build: remove redundant `CURL_PULL_*` settings -- libcurl-thread.3: simplify the TLS section + These macros were not propagated to the source code from CMake. - All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August - 2016. + autotools set only one of them (`CURL_PULL_SYS_POLL_H`), initially to + address an AIX issue [1]. This later broke when introducing `system.h` + [2] without the logic it enabled. A subsequent fix [3] re-added the + logic, and also enabled it for AIX before its use, directly in + `system.h`. - Closes #12233 + [1] 2012-11-23: 665adcd4b7bcdb7deb638cdc499fbe71f8d777f2 + [2] 2017-03-29: 9506d01ee50d5908138ebad0fd9fbd39b66bd64d #1373 + [3] 2017-08-25: 8a84fcc4b59e8b78d2acc6febf44a43d6bc81b59 #1828 #1833 -- configure: better --disable-http + Reviewed-by: Daniel Stenberg + Closes #12502 - - disable HTTPS-proxy as well, since it can't work without HTTP +- system.h: sync mingw `CURL_TYPEOF_CURL_SOCKLEN_T` with other compilers - - curl_setup: when HTTP is disabled, also disable all features that are - HTTP-only + Align mingw with the other Windows compilers and use the `int` type for + `CURL_TYPEOF_CURL_SOCKLEN_T` (and thus for `curl_socklent_t`). This + makes it unnecessary to make a mingw-specific trick and pull all Windows + headers early just for this type definition. This type is specific to + Windows, not to the compiler. mingw-w64's Windows header maps it to + `int` too. - - version: HTTPS-proxy only exists if HTTP support exists + With this we also delete all remaining uses of `CURL_PULL_WS2TCPIP_H`. - Closes #12223 + [ The official solution is to use `socklen_t` for all Windows compilers. + In this case we may want to update `curl/curl.h` to pull in Windows + headers before `system.h`. ] -- http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12501 - Finding a 'Content-Range:' in the response changed the handling. +- windows: simplify detecting and using system headers - Add test case 1475 to verify -C - with 416 and Content-Range: header, - which is almost exactly like test 194 which instead uses a fixed -C - offset. Adjusted test 194 to also be considered fine. + - autotools, cmake: assume that if we detect Windows, `windows.h`, + `winsock2.h` and `ws2tcpip.h` do exist. + - lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of + looking for `winsock2.h`. + - autotools: merge 3 Windows check methods into one. + - move Watt-32 and lwIP socket support to `setup-win32.h` from + `config-win32.h`. It opens up using these with all build tools. Also + merge logic with Windows Sockets. + - fix to assume Windows sockets with the mingw32ce toolchain. + Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4 + - cmake: delete unused variable `signature_call_conv` since + eb33ccd5332435fa50f1758e5debb869c6942b7f. + - autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection. + - examples/externalsocket: fix header order. + - cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue` + that wasn't used anymore. + - cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test + `SIZEOF_STRUCT_SOCKADDR_STORAGE`. - Fixes #10521 - Reported-by: Smackd0wn - Fixes #12174 - Reported-by: Anubhav Rai - Closes #12176 + After this patch curl universally uses `_WIN32` to guard + Windows-specific logic. It guards Windows Sockets-specific logic with + `USE_WINSOCK` (this might need further work). -Stefan Eissing (30 Oct 2023) + Reviewed-by: Jay Satiro + Closes #12495 -- GHA: fix checkout of quictls repository to use correct branch name +- build: enable missing OpenSSF-recommended warnings, with fixes - Follow-up to c868b0e30f10cd0ac7 + https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening + -Guide-for-C-and-C++.html + as of 2023-11-29 [1]. - Closes #12232 + Enable new recommended warnings (except `-Wsign-conversion`): -Daniel Stenberg (30 Oct 2023) + - enable `-Wformat=2` for clang (in both cmake and autotools). + - add `CURL_PRINTF()` internal attribute and mark functions accepting + printf arguments with it. This is a copy of existing + `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible + with redefinting the `printf` symbol: + https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 + - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for + mingw-w64 and enable it on this platform. + - enable `-Wimplicit-fallthrough`. + - enable `-Wtrampolines`. + - add `-Wsign-conversion` commented with a FIXME. + - cmake: enable `-pedantic-errors` the way we do it with autotools. + Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 + - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format + checks. Previously it was always disabled due to the internal `printf` + macro. -- docs/example/localport.c: show off CURLOPT_LOCALPORT + Fix them: - Closes #12230 + - fix bug where an `set_ipv6_v6only()` call was missed in builds with + `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. + - add internal `FALLTHROUGH()` macro. + - replace obsolete fall-through comments with `FALLTHROUGH()`. + - fix fallthrough markups: Delete redundant ones (showing up as + warnings in most cases). Add missing ones. Fix indentation. + - silence `-Wformat-nonliteral` warnings with llvm/clang. + - fix one `-Wformat-nonliteral` warning. + - fix new `-Wformat` and `-Wformat-security` warnings. + - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its + definition to `lib/curl_setup.h` allowing use in `tests/server`. + - lib: fix two wrongly passed string arguments in log outputs. + Co-authored-by: Jay Satiro + - fix new `-Wformat` warnings on mingw-w64. -- docs/examples/interface.c: show CURLOPT_INTERFACE use + [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895b + fc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options- + Hardening-Guide-for-C-and-C%2B%2B.md - Although super simple. + Closes #12489 - Closes #12229 +- Makefile.mk: drop Windows support -Viktor Szakats (30 Oct 2023) + And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga. -- build: fix compiler warning with auths disabled + We recommend CMake instead. With unity mode it's much faster, and about + the same without. - ``` - ./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' - [-Wunused-function] - static int is_valid_auth_separator(char ch) - ^ - 5 warnings generated. - ``` + Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 + Reviewed-by: Daniel Stenberg + Closes #12224 - Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 +Daniel Stenberg (16 Dec 2023) - Closes #12227 +- cmdline-docs: use .IP consistently -- build: require Windows XP or newer + Remove use of .TP and some .B. The idea is to reduce nroff syntax as + much as possible and to use it consistently. Ultimately, we should be + able to introduce our own easier-to-use-and-read syntax/formatting and + convert on generation time. - After this patch we assume availability of `getaddrinfo` and - `freeaddrinfo`, first introduced in Windows XP. Meaning curl - now requires building for Windows XP as a minimum. + Closes #12535 - TODO: assume these also in autotools. +Tatsuhiko Miyagawa (16 Dec 2023) - Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 - Closes #12225 +- http: fix off-by-one error in request method length check -- appveyor: bump one job to OpenSSL 3.1 (was 1.1.1) + It should allow one more byte. - Use 3.1 with the modern runner image. + Closes #12534 - We still use 1.1.1 in 8 jobs. +Daniel Stenberg (15 Dec 2023) - 1.1.1 is EOL since 2023-09-11: - https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/ +- curl: show ipfs and ipns as supported "protocols" - Also: - - add missing SSL-backend to job descriptions. - - tidy up CPU in job descriptions. + They are accepted schemes in URLs passed to curl (the tool, not the + library). - Closes #12226 + Also makes curl-config show the same list. -Daniel Stenberg (30 Oct 2023) + Co-Authored-by: Jay Satiro + Reported-by: Chara White + Bug: https://curl.se/mail/archive-2023-12/0026.html + Closes #12508 -- RELEASE-NOTES: synced +- Revert "urldata: move async resolver state from easy handle to connectdata" -- GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions + This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198) - ngtcp2 1.0.1 - nghttp3 1.0.0 - nghttp2 1.58.0 - quictls 3.1.4+quic + We want the c-ares channel to be held in the easy handle, not per + connection - for performance. - also sync HTTP3.md with these changes + Closes #12524 - Closes #12132 +Viktor Szakats (15 Dec 2023) -Kareem (29 Oct 2023) +- openssl: re-match LibreSSL deinit with init -- wolfssl: add default case for wolfssl_connect_step1 switch + Earlier we switched to use modern initialization with LibreSSL v2.7.0 + and up, but did not touch deinitialization [1]. Fix it in this patch. - Closes #12218 + Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611 -Jay Satiro (29 Oct 2023) + [1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014 -- curl_setup: disallow Windows IPv6 builds missing getaddrinfo + Reported-by: Mike Hommey + Reviewed-by: Daniel Stenberg + Fixes #12525 + Closes #12526 - - On Windows if IPv6 is enabled but getaddrinfo is missing then #error - the build. +Daniel Stenberg (14 Dec 2023) - curl can be built with IPv6 support (ENABLE_IPV6) but without the - ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On - Windows this is highly unlikely and should be considered a bad build - configuration. +- libssh: supress warnings without version check - Such a bad configuration has already given us a bug that was hard to - diagnose. See #12134 and #12136 for discussion. + Define unconditionally. - Ref: https://github.com/curl/curl/issues/12134 - Ref: https://github.com/curl/curl/pull/12136 + Follow-up from d21bd2190c46ad7fa - Closes https://github.com/curl/curl/pull/12221 + Closes #12523 -Nico Rieck (29 Oct 2023) +- hostip: return error immediately when Curl_ip2addr() fails -- openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs + Closes #12522 - - If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA - "CA" store after importing from root CA "ROOT" store. +Theo (14 Dec 2023) - This change allows curl to work in situations where a server does not - send all intermediate certs and they are present in the "CA" store (the - store with intermediate CAs). This is already allowed by the Schannel - backend. +- libssh: improve the deprecation warning dismissal - Also this change makes partial chain verification possible for those - certs since we allow partial chain verification by default for OpenSSL - (unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel - backend. + Previous code was compiler dependant, and dismissed all deprecation warnings + indiscriminately. - Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs. + libssh provides a way to disable the deprecation warnings for libssh only, an + d + naturally this is the preferred way. - Fixes https://github.com/curl/curl/issues/12155 - Closes https://github.com/curl/curl/pull/12185 + This commit uses that, to prevent the erroneous hiding of potential, unrelate + d + deprecation warnings. -Viktor Szakats (28 Oct 2023) + Fixes #12519 + Closes #12520 -- Makefile.mk: fix `-rtmp` option for non-Windows [ci skip] +Daniel Stenberg (14 Dec 2023) -Daniel Stenberg (28 Oct 2023) +- test1474: removed -- asyn-ares: handle no connection in the addrinfo callback + The test was already somewhat flaky and disabled on several platforms, + and after 1da640abb688 even more unstable. - To avoid crashing. +- readwrite_data: loop less - Follow-up from 56a4db2 - Closes #12219 + This function is made to loop in order to drain incoming data + faster. Completely removing the loop has a measerably negative impact on + transfer speeds. -Jay Satiro (28 Oct 2023) + Downsides with the looping include -- hostip6: fix DEBUG_ADDRINFO builds + - it might call the progress callback much more seldom. Especially if + the write callback is slow. - - Removed unused and incorrect parameter from dump_addrinfo(). + - rate limiting becomes less exact - Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442 - Reported-by: Gisle Vanem + - a single transfer might "starve out" other parallel transfers - Closes https://github.com/curl/curl/pull/12212 + - QUIC timers for other connections can't be maintained correctly -Viktor Szakats (28 Oct 2023) + The long term fix should be to remove the loop and optimize coming back + to avoid the transfer speed penalty. -- Makefile.mk: restore `_mingw.h` for default `_WIN32_WINNT` + This fix lower the max loop count to reduce the starvation problem, and + avoids the loop completely for when rate-limiting is in progress. - In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support. - Turns out `_mingw.h` had the side-effect of setting a default - `_WIN32_WINNT` value expected by `lib/config-win32.h` to enable - `getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused - disabling support for this unless specifying the value manually. + Ref: #12488 + Ref: https://curl.se/mail/lib-2023-12/0012.html + Closes #12504 - Restore this header and update its comment to tell why we continue - to need it. +Stefan Eissing (14 Dec 2023) - This triggered a regression in official Windows curl builds starting - with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.) +- lib: eliminate `conn->cselect_bits` - Regression from 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 + - use `data->state.dselect_bits` everywhere instead + - remove `bool *comeback` parameter as non-zero + `data->state.dselect_bits` will indicate that IO is + incomplete. - Reported-by: zhengqwe on github - Helped-by: Nico Rieck - Fixes #12134 - Fixes #12136 - Closes #12217 + Closes #12512 -- hostip: silence compiler warning `-Wparentheses-equality` +- connect: refactor `Curl_timeleft()` - Seen with LLVM 17. + - less local vars, "better" readability + - added documentation - ``` - hostip.c:1336:22: warning: equality comparison with extraneous parentheses [- - Wparentheses-equality] - 1336 | (a->ai_family == PF_INET)) { - | ~~~~~~~~~~~~~^~~~~~~~~~ - hostip.c:1336:22: note: remove extraneous parentheses around the comparison t - o silence this warning - 1336 | (a->ai_family == PF_INET)) { - | ~ ^ ~ - hostip.c:1336:22: note: use '=' to turn this equality comparison into an assi - gnment - 1336 | (a->ai_family == PF_INET)) { - | ^~ - | = - 1 warning generated. - ``` + Closes #12518 - Follow-up to b651aba0962bb31353f55de4dc35f745952a1b10 #12145 +Dmitry Karpov (14 Dec 2023) - Reviewed-by: Daniel Stenberg - Closes #12215 +- cookie: avoid fopen with empty file name -Stefan Eissing (27 Oct 2023) + Closes #12514 -- doh: use PIPEWAIT when HTTP/2 is attempted +Viktor Szakats (13 Dec 2023) - Closes #12214 +- tests/server: delete workaround for old-mingw -Daniel Stenberg (27 Oct 2023) + mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this. -- setopt: remove outdated cookie comment + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 - Closes #12206 + Reviewed-by: Jay Satiro + Closes #12510 -Stefan Eissing (27 Oct 2023) +- cmake: delete obsolete TODOs more [ci skip] -- cfilter: provide call to tell connection to forget a socket + - manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288 + - soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023 + - bunch of others that are completed + - `NTLM_WB_ENABLED` is implemented in a basic form, and now also + scheduled for removal, so a TODO at this point isn't useful. - - fixed libssh.c workaround for a socket being closed by - the library - - eliminate the terrible hack in cf-socket.c to guess when - this happened and try not closing the socket again. - - fixes race in eyeballing when socket could have failed to - be closed for a discarded connect attempt + And this 'to-check' item: - Closes #12207 + Q: "The cmake build selected to run gcc with -fPIC on my box while the + plain configure script did not." -- url: protocol handler lookup tidy-up + A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 + and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly + enable PIC for libcurl shared lib. Or when building libcurl for + shared and static lib in a single pass. We do this by default for + Windows or when enabled by the user via `SHARE_LIB_OBJECT`. + Otherwise we don't touch this setting. Meaning the default set by + CMake (if any) or the toolchain is used. On Debian Bookworm, this + means that PIC is disabled for static libs by default. Some platforms + (like macOS), has PIC enabled by default. + autotools supports the double-pass mode only, and in that case + CMake seems to match PIC behaviour now (as tested on Linux with gcc.) - - rename lookup to what it does - - use ARRAYSIZE instead of NULL check for end - - offer alternate lookup for 0-terminated strings + Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500 - Closes #12216 + Reviewed-by: Jay Satiro + Closes #12509 -Viktor Szakats (27 Oct 2023) +Stefan Eissing (12 Dec 2023) -- build: variadic macro tidy-ups +- CLIENT-WRITERS: design and use documentation - - delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks. - (both autotools and CMake.) - - delete duplicate `NULL` check in `Curl_trc_cf_infof()`. - - fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds. - ``` - ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parame - ter] - static void nosigpipe(struct Curl_easy *data, - ^ - ``` - - fix `#ifdef` comments in `lib/curl_trc.{c,h}`. - - fix indentation in some `infof()` calls. + Closes #12507 - Follow-up to dac293cfb7026b1ca4175d88b80f1432d3d3c684 #12167 +Viktor Szakats (12 Dec 2023) - Cherry-picked from #12105 - Closes #12210 +- cmake: delete obsolete TODO items [ci skip] -- cmake: speed up threads setup for Windows + There is always room for improvement, but CMake is up to par now with + autotools, so there is no longer a good reason to keep around these + inline TODO items. - Win32 threads are always available. We enabled them unconditionally - (with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection - logic has this condition hard-coded for Windows as well (since at least - 2007). + Answering one of questions: - Instead of doing all the work of detecting pthread combinations on - Windows, then discarding those results, skip these efforts and assume - built-in thread support when building for Windows. + Q: "The gcc command line use neither -g nor any -O options. As a + developer, I also treasure our configure scripts's --enable-debug + option that sets a long range of "picky" compiler options." - This saves 1-3 slow CMake configuration steps. + A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info + and optimization level. E.g.: + - `Release` = `-O3` + no debug info + - `MinSizeRel` = `-Os` + no debug info + - `Debug` = `-O0` + debug info - Reviewed-by: Daniel Stenberg - Closes #12202 + https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-deb + ug-release-relwithdebinfo-and-minsizerel/59314670#59314670 + https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#defaul + t-and-custom-configurations -- cmake: speed up zstd detection + For picky warnings we have the `PICKY_COMPILER` options, enabled by + default. - Before this patch we detected the presence of a specific zstd API to - see if we can use the library. zstd published that API in its first - stable release: v1.0.0 (2016-08-31). + Closes #12500 - Replace that method by detecting the zstd library version instead and - accepting if it's v1.0.0 or newer. Also display this detected version - and display a warning if the zstd found is unfit for curl. +Stefan Eissing (11 Dec 2023) - We use the same version detection method as zstd itself, via its public - C header. +- CONNECTION-FILTERS: update documentation - This deviates from autotools which keeps using the slow method of - looking for the API by building a test program. The outcome is the same - as long as zstd keeps offering this API. + Closes #12497 - Ref: https://github.com/facebook/zstd/commit/5a0c8e24395079f8e8cdc90aa1659cd5 - ab1b7427 (2016-08-12, committed) - Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first - released) - Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0 +Daniel Stenberg (11 Dec 2023) - Reviewed-by: Daniel Stenberg - Closes #12200 +- lib: reduce use of strncpy -Daniel Stenberg (26 Oct 2023) + - bearssl: select cipher without buffer copies + - http_aws_sigv4: avoid strncpy, require exact timestamp length + - http_aws_sigv4: use memcpy isntead of strncpy + - openssl: avoid strncpy calls + - schannel: check for 1.3 algos without buffer copies + - strerror: avoid strncpy calls + - telnet: avoid strncpy, return error on too long inputs + - vtls: avoid strncpy in multissl_version() -- openssl: fix infof() to avoid compiler warning for %s with null + Closes #12499 - vtls/openssl.c: In function ‘ossl_connect_step2’: - ../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror - =format-overflow=] - 120 | Curl_infof(data, __VA_ARGS__); } while(0) - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’ - 4008 | infof(data, "SSL connection using %s / %s / %s / %s", - | ^~~~~ - vtls/openssl.c:4008:49: note: format string is defined here - 4008 | infof(data, "SSL connection using %s / %s / %s / %s", - | ^~ +- CI/distcheck: run full tests - Follow-up to b6e6d4ff8f253c8b8055bab - Closes #12196 + To be able to detect missing files better, this now runs the full CI + test suite. If done before, it would have detected #12462 before + release. -Stefan Eissing (26 Oct 2023) + Closes #12503 -- lib: apache style infof and trace macros/functions +- docs: clean up Protocols: for cmdline options - - test for a simplified C99 variadic check - - args to infof() in --disable-verbose are no longer disregarded but - must compile. + ... and some other minor polish. - Closes #12167 - Fixes #12083 - Fixes #11880 - Fixes #11891 + Closes #12496 -Daniel Stenberg (26 Oct 2023) +- cmdline/gen: fix the sorting of the man page options -- RELEASE-NOTES: synced + They were previously sorted based on the file names, which use a .d + extension, making "data" get placed after "data-binary" etc. Making the + sort ignore the extention fixes the ordering. -Stefan Eissing (26 Oct 2023) + Reported-by: Boris Verkhovskiy + Bug: https://curl.se/mail/archive-2023-12/0014.html + Closes #12494 -- urldata: move async resolver state from easy handle to connectdata +Daniel Gustafsson (9 Dec 2023) - - resolving is done for a connection, not for every transfer - - save create/dup/free of a cares channel for each transfer - - check values of setopt calls against a local channel if no - connection has been attached yet, when needed. +- doh: remove unused local variable - Closes #12198 + The nurl variable is no longer used during probing following + a refactoring, so remove. -Daniel Stenberg (26 Oct 2023) + Closes #12491 -- CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERRO - R +Jay Satiro (8 Dec 2023) - It returns CURLE_WRITE_ERROR. It was not previously stated clearly. +- build: fix Windows ADDRESS_FAMILY detection - Reported-by: enWILLYado on github - Fixes #12201 - Closes #12203 + - Include winsock2.h for Windows ADDRESS_FAMILY detection. -Viktor Szakats (25 Oct 2023) + Prior to this change cmake detection didn't work because it included + ws2def.h by itself, which is missing needed types from winsock2.h. -- autotools: update references to deleted `crypt-auth` option + Prior to this change autotools detection didn't work because it did not + include any Windows header. - Delete leftovers of the `crypt-auth` `./configure` option and - add the new ones that replaced them. + In both cases libcurl would fall back on unsigned short as the address + family type, which is the same as ADDRESS_FAMILY. - Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 + Co-authored-by: Viktor Szakats - Reviewed-by: Daniel Stenberg - Closes #12194 + Closes https://github.com/curl/curl/pull/12441 -Stefan Eissing (25 Oct 2023) +Daniel Stenberg (8 Dec 2023) -- lib: introduce struct easy_poll_set for poll information +- lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding - Connection filter had a `get_select_socks()` method, inspired by the - various `getsocks` functions involved during the lifetime of a - transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), - return sockets to monitor and flag if this shall be done for POLLIN - and/or POLLOUT. + Since the copy does not stop at a null byte, let's not call it anything + that makes you think it works like the common strndup() function. - Due to this design, sockets and flags could only be added, not - removed. This led to problems in filters like HTTP/2 where flow control - prohibits the sending of data until the peer increases the flow - window. The general transfer loop wants to write, adds POLLOUT, the - socket is writeable but no data can be written. + Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat - This leads to cpu busy loops. To prevent that, HTTP/2 did set the - `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes - further attempts. This works if only one such filter is involved. If a - HTTP/2 transfer goes through a HTTP/2 proxy, two filters are - setting/clearing this flag and may step on each other's toes. + Closes #12490 - Connection filters `get_select_socks()` is replaced by - `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps - up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` - flags. This struct is initialized in `multi_getsock()` by calling the - various `getsocks()` implementations based on transfer state, as before. +- convsrctest.pl: removed: not used, not shipped in tarballs - After protocol handlers/transfer loop have set the sockets and flags - they want, the `easy_pollset` is *always* passed to the filters. Filters - "higher" in the chain are called first, starting at the first - not-yet-connection one. Each filter may add sockets and/or change - flags. When all flags are removed, the socket itself is removed from the - pollset. +- tests: rename tests scripts to the test number - Example: + It is hard to name the scripts sensibly. Lots of them are similarly + named and the name did not tell which test that used them. - * transfer wants to send, adds POLLOUT - * http/2 filter has a flow control block, removes POLLOUT and adds - POLLIN (it is waiting on a WINDOW_UPDATE from the server) - * TLS filter is connected and changes nothing - * h2-proxy filter also has a flow control block on its tunnel stream, - removes POLLOUT and adds POLLIN also. - * socket filter is connected and changes nothing - * The resulting pollset is then mixed together with all other transfers - and their pollsets, just as before. + The new approach is rather to name them based on the test number that + runs them. Also helps us see which scripts are for individual tests + rather than for general test infra. - Use of `SEND_HOLD` is no longer necessary in the filters. + - badsymbols.pl -> test1167.pl + - check-deprecated.pl -> test1222.pl + - check-translatable-options.pl -> test1544.pl + - disable-scan.pl -> test1165.pl + - error-codes.pl -> test1175.pl + - errorcodes.pl -> test1477.pl + - extern-scan.pl -> test1135.pl + - manpage-scan.pl -> test1139.pl + - manpage-syntax.pl -> test1173.pl + - markdown-uppercase.pl -> test1275.pl + - mem-include-scan.pl -> test1132.pl + - nroff-scan.pl -> test1140.pl + - option-check.pl -> test1276.pl + - options-scan.pl -> test971.pl + - symbol-scan.pl -> test1119.pl + - version-scan.pl -> test1177.pl - All filters are adapted for the changed method. The handling in - `multi.c` has been adjusted, but its state handling the the protocol - handlers' `getsocks` method are untouched. + Closes #12487 - The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS - filters needed to be adjusted for the connecting handshake read/write - handling. +Michał Antoniak (8 Dec 2023) - No noticeable difference in performance was detected in local scorecard - runs. +- sendf: fix compiler warning with CURL_DISABLE_HEADERS_API - Closes #11833 + fix MSVC warning C4189: 'htype': local variable is initialized but not + referenced - when CURL_DISABLE_HEADERS_API is defined. -Daniel Stenberg (25 Oct 2023) + Closes #12485 -- tests/README: SOCKS tests are not using OpenSSH, it has its own server +Viktor Szakats (8 Dec 2023) - Follow-up to 04fd67555cc +- tidy-up: whitespace - Closes #12195 + Closes #12484 -Jacob Hoffman-Andrews (25 Oct 2023) +Stefan Eissing (7 Dec 2023) -- tets: make test documentation more user-friendly +- test_02_download: fix paramters to test_02_27 - Put the instructions to run tests right at the top of tests/README.md. + - it is a special client that only ever uses http/2 - Give instructions to read the runtests.1 man page for information - about flags. Delete redundant copy of the flags documentation in the - README. + Closes #12467 - Add a mention in README.md of the important parallelism flag, to make - test runs go much faster. +Michał Antoniak (7 Dec 2023) - Move documentation of output line format into the runtests.1 man page, - and update it with missing flags. +- vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY - Fix the order of two flags in the man page. + Closes #12459 - Closes #12193 +Daniel Stenberg (7 Dec 2023) -Viktor Szakats (24 Oct 2023) +- lib: strndup/memdup instead of malloc, memcpy and null-terminate -- cmake: pre-fill rest of detection values for Windows + - bufref: use strndup + - cookie: use strndup + - formdata: use strndup + - ftp: use strndup + - gtls: use aprintf instead of malloc + strcpy * 2 + - http: use strndup + - mbedtls: use strndup + - md4: use memdup + - ntlm: use memdup + - ntlm_sspi: use strndup + - pingpong: use memdup + - rtsp: use strndup instead of malloc, memcpy and null-terminate + - sectransp: use strndup + - socks_gssapi.c: use memdup + - vtls: use dynbuf instead of malloc, snprintf and memcpy + - vtls: use strdup instead of malloc + memcpy + - wolfssh: use strndup - The goal of this patch is to avoid unnecessary feature detection work - when doing Windows builds with CMake. Do this by pre-filling well-known - detection results for Windows and specifically for mingw-w64 and MSVC - compilers. Also limit feature checks to platforms where the results are - actually used. Drop a few redundant ones. And some tidying up. + Closes #12453 - - pre-fill remaining detection values in Windows CMake builds. +- strdup: remove the memchr check from Curl_strndup - Based on actual detection results observed in CI runs, preceding - similar work over libssh2 and matching up values with - `lib/config-win32.h`. + It makes it possible to clone a binary chunk of data. - This brings down CMake configuration time from 58 to 14 seconds on the - same local machine. + Closes #12453 - On AppVeyor CI this translates to: - - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job): - https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecr - jpy7necb#L296 - https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2 - fe249uo8#L186 - - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job): - https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5iv - lcs7ub29?fullLog=true#L290 - https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjs - yc9kl13a?fullLog=true#L194 +- ftp: handle the PORT parsing without allocation - The formula is about 1-3 seconds delay for each detection. Almost all - of these trigger a full compile-link cycle behind the scenes, slow - even today, both cross and native, mingw-w64 and apparently MSVC too. - Enabling .map files or other custom build features slows it down - further. (Similar is expected for autotools configure.) + Also reduces amount of *cpy() calls. - - stop detecting `idn2.h` if idn2 was deselected. - autotools does this. + Closes #12456 - - stop detecting `idn2.h` if idn2 was not found. - This deviates from autotools. Source code requires both header and - lib, so this is still correct, but faster. +- RELEASE-NOTES: synced - - limit `ADDRESS_FAMILY` detection to Windows. + Bumped to 8.5.1 - - normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format. +- url: for disabled protocols, mention if found in redirect - - pre-fill `HAVE_WIN32_WINNT`-dependent detection results. - Saving 4 (slow) feature-detections in most builds: `getaddrinfo`, - `freeaddrinfo`, `inet_ntop`, `inet_pton` + To help users better understand where the URL (and denied scheme) comes + from. Also removed "in libcurl" from the message, since the disabling + can be done by the application. - - fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`, - `HAVE_GETTIMEOFDAY` for mingw-w64. - Luckily this do not change build results, as `WIN32` took - priority over `HAVE_GETTIMEOFDAY` with the current source - code. + The error message now says "not supported" or "disabled" depending on + why it was denied: - - limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and - `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows. - We're not using these in the source code for Windows. + Protocol "hej" not supported + Protocol "http" disabled - - reduce compiler warning noise in CMake internal logs: - - fix to include `winsock2.h` before `windows.h`. - Apply it to autotools test snippets too. - - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above. - - cleanup `CMake/CurlTests.c` to emit less warnings. + And in redirects: - - delete redundant `HAVE_MACRO_SIGSETJMP` feature check. - It was the same check as `HAVE_SIGSETJMP`. + Protocol "hej" not supported (in redirect) + Protocol "http" disabled (in redirect) - - delete 'experimental' marking from `CURL_USE_OPENSSL`. + Reported-by: Mauricio Scheffer + Fixes #12465 + Closes #12469 - - show CMake version via `CMakeLists.txt`. - Credit to the `zlib-ng` project for the idea: - https://github.com/zlib-ng/zlib-ng/blob/61e181c8ae93dbf56040336179c9954078b - d1399/CMakeLists.txt#L7 +Stefan Eissing (6 Dec 2023) - - make `CMake/CurlTests.c` pass `checksrc`. +- sectransp_ make TLSCipherNameForNumber() available in non-verbose config - - `CMake/WindowsCache.cmake` tidy-ups. + Reported-by: Cajus Pollmeier + Closes #12476 + Fixes #12474 - - replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`. +YX Hao (6 Dec 2023) - Closes #12044 +- lib: fix variable undeclared error caused by `infof` changes -Jay Satiro (24 Oct 2023) + `--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined. + `infof` isn't `Curl_nop_stmt` anymore: dac293c. -- page-footer: clarify exit code 25 + Follow-up to dac293c - - Clarify that curl tool exit code 25 means an upload failed to start. + Closes #12470 - Exit code 25 is equivalent to CURLE_UPLOAD_FAILED (25). Prior to this - change the documentation only mentioned the case of FTP STOR failing. +Viktor Szakats (6 Dec 2023) - Reported-by: Emanuele Torre +- tidy-up: fix yamllint whitespace issues in labeler.yml - Ref: https://github.com/curl/curl/blob/curl-8_4_0/docs/libcurl/libcurl-errors - .3#L113-L115 + Follow-up to bda212911457c6fadfbba50be61afc4ca513fa56 #12466 - Fixes https://github.com/curl/curl/issues/12189 - Closes https://github.com/curl/curl/pull/12190 + Reviewed-by: Dan Fandrich + Closes #12475 -Daniel Stenberg (24 Oct 2023) +- tidy-up: fix yamllint whitespace issues -- scripts/cijobs.pl: adjust for appveyor + Closes #12466 - Follow-up to a1d73a6bb +Chris Sauer (6 Dec 2023) -Alex Bozarth (24 Oct 2023) +- cmake: fix typo -- OpenSSL: Include SIG and KEM algorithms in verbose + Follow-up to aace27b + Closes #12464 - Currently the verbose output does not include which algorithms are used - for the signature and key exchange when using OpenSSL. Including the - algorithms used will enable better debugging when working on using new - algorithm implementations. Know what algorithms are used has become more - important with the fast growing research into new quantum-safe - algorithms. +Daniel Stenberg (6 Dec 2023) - This implementation includes a build time check for the OpenSSL version - to use a new function that will be included in OpenSSL 3.2 that was - introduced in openssl/openssl@6866824 +- dist: add tests/errorcodes.pl to the tarball - Based-on-patch-by: Martin Schmatz - Closes #12030 + Used by test 1477 -Daniel Stenberg (23 Oct 2023) + Reported-by: Xi Ruoyao + Follow-up to 0ca3a4ec9a7 + Fixes #12462 + Closes #12463 -- http2: provide an error callback and failf the message +Dan Fandrich (6 Dec 2023) - Getting nghttp2's error message helps users understand what's going - on. For example when the connection is brought down due a forbidden - header is used - as that header is then not displayed by curl itself. +- github/labeler: update a missed key in the v5 upgrade - Example: + Follow-up to ce03fe3ba - curl: (92) Invalid HTTP header field was received: frame type: 1, - stream: 1, name: [upgrade], value: [h2,h2c] +Version 8.5.0 (6 Dec 2023) - Ref: #12172 - Closes #12179 +Daniel Stenberg (6 Dec 2023) -Turiiya (23 Oct 2023) +- RELEASE-NOTES: synced -- BINDINGS: add V binding + The curl 8.5.0 release. - Closes #12182 +Dan Fandrich (5 Dec 2023) -Daniel Stenberg (22 Oct 2023) +- github/labeler: switch from the beta to labeler v5 -- configure: check for the fseeko declaration too + Some keys were renamed and the dot option was made default. - ... and make the code require both symbol and declaration. + Closes #12458 - This is because for Android, the symbol is always present in the lib at - build-time even when not actually available in run-time. +Daniel Stenberg (5 Dec 2023) - Assisted-by: Viktor Szakats - Reported-by: 12932 on github - Fixes #12086 - Closes #12158 +- DEPRECATE: remove NTLM_WB in June 2024 -Viktor Szakats (22 Oct 2023) + Ref: https://curl.se/mail/lib-2023-12/0010.html -- cmake: fix OpenSSL quic detection in quiche builds + Closes #12451 - An orphan call to `CheckQuicSupportInOpenSSL()` remained after a recent - update when checking QUIC for quiche. Move back QUIC detection to - a function and fixup callers to use that. Also make sure that quiche - gets QUIC from BoringSSL, because it doesn't support other forks at this - time. +Jacob Hoffman-Andrews (4 Dec 2023) - Regression from dee310d54261f9a8416e87d50bccfe2cbe404949 #11555 +- rustls: implement connect_blocking - Reported-by: Casey Bodley - Fixes #12160 - Closes #12162 + Closes #11647 -Daniel Stenberg (22 Oct 2023) +Daniel Stenberg (4 Dec 2023) -- RELEASE-NOTES: synced +- examples/rtsp-options.c: add - bump to 8.5.0 for pending release + Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and + CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. -Dan Fandrich (21 Oct 2023) + Closes #12452 -- test3103: add missing quotes around a test tag attribute +Stefan Eissing (4 Dec 2023) -Loïc Yhuel (21 Oct 2023) +- ngtcp2: ignore errors on unknown streams -- tool: fix --capath when proxy support is disabled + - expecially in is_alive checks on connections, we might + see incoming packets on streams already forgotten and closed, + leading to errors reported by nghttp3. Ignore those. - After 95e8515ca0, --capath always sets CURLOPT_PROXY_CAPATH, which fails - with CURLE_UNKNOWN_OPTION when proxy support is disabled. + Closes #12449 - Closes #12089 +Daniel Stenberg (4 Dec 2023) -Daniel Stenberg (21 Oct 2023) +- docs: make all examples in all libcurl man pages compile -- openldap: move the alloc of ldapconninfo to *connect() + Closes #12448 - Fixes a minor memory leak on LDAP connection reuse. +- checksrc.pl: support #line instructions - Doing the allocation already in *setup_connection() is wrong since that - connect struct might get discarded early when an existing connection is - reused instead. + makes it identify the correct source file and line - Closes #12166 +- GHA/man-examples: verify libcurl man page examples -- openldap: set the callback argument in oldap_do +- verify-examples.pl: verify that all man page examples compile clean - ... to make sure it has the current 'data' pointer and not a stale old - one. +- RELEASE-NOTES: synced - Reported-by: Dan Fandrich - Closes #12166 +Graham Campbell (2 Dec 2023) -- gnutls: support CURLSSLOPT_NATIVE_CA +- http3: bump ngtcp2 and nghttp3 versions - Remove the CURL_CA_FALLBACK logic. That build option was added to allow - primarily OpenSSL to use the default paths for loading the CA certs. For - GnuTLS it was instead made to load the "system certs", which is - different and not desirable. + nghttp3 v1.1.0 + ngtcp2 v1.1.0 - The native CA store loading is now asked for with this option. + In docs and CI - Follow-up to 7b55279d1d856 + Closes #12446 - Co-authored-by: Jay Satiro +- CI/quiche: use `3.1.4+quic` consistently in CI workflows - Closes #12137 + Closes #12447 -Stefan Eissing (21 Oct 2023) +Viktor Szakats (2 Dec 2023) -- RTSP: improved RTP parser +- test1545: disable deprecation warnings - - fix HTTP header parsing to report incomplete - lines it buffers as consumed! - - re-implement the RTP parser for interleave RTP - messages for robustness. It is now keeping its - state at the connection - - RTSP protocol handler "readwrite" implementation - now tracks if the response is before/in/after - header parsing or "in" a bod by calling - "Curl_http_readwrite_headers()" itself. This - allows it to know when non-RTP bytes are "junk" - or HEADER or BODY. - - tested with #12035 and various small receive - sizes where current master fails + Fixes: + https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr + p6pk#L1205 - Closes #12052 + Same with details: + https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil + b6wt#L1263 + ``` + tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 + .0. Use curl_mime_init() [-Werror=deprecated-declarations] + 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", + | ^~~~~~~~~~~~ + [...] + ``` -- http2: header conversion tightening + Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 - - fold the code to convert dynhds to the nghttp2 structs - into a dynhds internal method - - saves code duplication - - pacifies compiler analyzers + Fixes #12445 + Closes #12444 - Closes #12097 +Daniel Stenberg (2 Dec 2023) -Daniel Stenberg (21 Oct 2023) +- INSTALL: update list of ports and CPU archs -- curl_ntlm_wb: fix elif typo +- symbols-in-versions: the CLOSEPOLICY options are deprecated - Reported-by: Manfred Schwarb - Follow-up to d4314cdf65ae - Bug: https://github.com/curl/curl/commit/d4314cdf65aee295db627016934bd9eb621a - b077#r130551295 + The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. -Dan Fandrich (20 Oct 2023) +z2_ (1 Dec 2023) -- test1683: remove commented-out check alternatives +- build: fix builds that disable protocols but not digest auth - Python precheck/postcheck alternatives were included but commented out. - Since these are not used and perl is guaranteed to be available to run - the perl versions anyway, the Python ones are removed. + - Build base64 functions if digest auth is not disabled. -Daniel Stenberg (20 Oct 2023) + Prior to this change if some protocols were disabled but not digest auth + then a build error would occur due to missing base64 functions. -- hostip: show the list of IPs when resolving is done + Fixes https://github.com/curl/curl/issues/12440 + Closes https://github.com/curl/curl/pull/12442 - Getting 'curl.se' today then gets this verbose output which might help - debugging connectivity related matters. +Michał Antoniak (1 Dec 2023) - * Host curl.se:80 was resolved. - * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, - 2a04:4e42:600::347, 2a04:4e42:800::347, 2a04:4e42:a00::347, - 2a04:4e42:c00::347, 2a04:4e42:e00::347 - * IPv4: 151.101.193.91, 151.101.1.91, 151.101.65.91, 151.101.129.91 +- connect: reduce number of transportation providers - Co-authored-by: Jay Satiro - Closes #12145 + Use only the ones necessary - the ones that are built-in. Saves a few + bytes in the resulting code. -rilysh (20 Oct 2023) + Closes #12438 -- docs: fix function typo in curl_easy_option_next.3 +David Benjamin (1 Dec 2023) - Closes #12170 +- vtls: consistently use typedef names for OpenSSL structs -Daniel Stenberg (20 Oct 2023) + The foo_st names don't appear in OpenSSL public API documentation. The + FOO typedefs are more common. This header was already referencing + SSL_CTX via . There is a comment about avoiding + , but OpenSSL actually declares all the typedefs in + , which is already included by (and + every other OpenSSL header), so just use that. Though I've included it + just to be explicit. -- vssh: remove the #ifdef for Curl_ssh_init, use empty macro + (I'm also fairly sure including already triggers the + Schannel conflicts anyway. The comment was probably just out of date.) - In the same style as other init calls + Closes #12439 -- easy: remove duplicate wolfSSH init call +Lau (1 Dec 2023) - It is already done in Curl_ssh_init() where it belongs. +- libcurl-security.3: fix typo - Closes #12168 + Fixed minimal typo. -- socks: make SOCKS5 use the CURLOPT_IPRESOLVE choice + Closes #12437 - Fixes #11949 - Reported-by: Ammar Faizi - Closes #12163 +Stefan Eissing (1 Dec 2023) -- urldata: move the 'internal' boolean to the state struct +- ngtcp2: fix races in stream handling - ... where all the other state bits for the easy handles live. + - fix cases where ngtcp2 invokes callbacks on streams that + nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND + in these cases as it is normal behaviour. - Closes #12165 + Closes #12435 -- url: don't touch the multi handle when closing internal handles +Emanuele Torre (1 Dec 2023) - Reported-by: Maksymilian Arciemowicz - Closes #12165 +- tool_writeout_json: fix JSON encoding of non-ascii bytes -Faraz Fallahi (19 Oct 2023) + char variables if unspecified can be either signed or unsigned depending + on the platform according to the C standard; in most platforms, they are + signed. -- getenv: PlayStation doesn't have getenv() + This meant that the *i<32 waas always true for bytes with the top bit + set. So they were always getting encoded as \uXXXX, and then since they + were also signed negative, they were getting extended with 1s causing + '\xe2' to be expanded to \uffffffe2, for example: - Closes #12140 + $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul + l + \uffffffe2\uffffff80\uffffff9c -Daniel Stenberg (19 Oct 2023) + I fixed this bug by making the code use explicitly unsigned char* + variables instead of char* variables. -- transfer: only reset the FTP wildcard engine in CLEAR state + Test 268 verifies - To avoid the state machine to start over and redownload all the files - *again*. + Reported-by: iconoclasthero + Closes #12434 - Reported-by: lkordos on github - Regression from 843b3baa3e3cb228 (shipped in 8.1.0) - Bisect-by: Dan Fandrich - Fixes #11775 - Closes #12156 +Stefan Eissing (1 Dec 2023) -Stefan Eissing (19 Oct 2023) +- cf-socket: TCP trace output local address used in connect -- GHA: move mod_h2 version in CI to v2.0.25 + Closes #12427 - Closes #12157 +Jay Satiro (1 Dec 2023) -Daniel Stenberg (19 Oct 2023) +- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation -- ntlm_wb: use pipe instead of socketpair when possible + - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it + includes protocol-specific instructions that trigger a transfer. - Closes #12149 + Prior to this change it explicitly said that it did not include those + instructions in the time, but that is incorrect. -- RELEASE-NOTES: synced + The change is a copy of the fixed explanation already in + CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. -- asyn-thread: use pipe instead of socketpair for IPC when available + Reported-by: eeverettrbx@users.noreply.github.com - If pipe() is present. Less overhead. + Fixes https://github.com/curl/curl/issues/12431 + Closes https://github.com/curl/curl/pull/12432 - Helped-by: Viktor Szakats - Closes #12146 +Daniel Stenberg (30 Nov 2023) -Dan Fandrich (17 Oct 2023) +- multi: during ratelimit multi_getsock should return no sockets -- tests: Fix Windows test helper tool search & use it for handle64 + ... as there is nothing to wait for then, it just waits. Otherwise, this + causes much more CPU work and updates than necessary during ratelimit + periods. - The checkcmd() and checktestcmd() functions would not have worked on - Windows due to hard-coding the UNIX PATH separator character and not - adding .exe file extension. This meant that tools like stunnel, valgrind - and nghttpx would not have been found and used on Windows, and - inspection of previous test runs show none of those being found in pure - Windows CI builds. + Ref: https://curl.se/mail/lib-2023-11/0056.html + Closes #12430 - With this fixed, they can be used to detect the handle64.exe program - before attempting to use it. When handle64.exe was called - unconditionally without it existing, it caused perl to abort the test - run with the error +Dmitry Karpov (30 Nov 2023) - The running command stopped because the preference variable - "ErrorActionPreference" or common parameter is set to Stop: - sh: handle64.exe: command not found +- transfer: abort pause send when connection is marked for closing - Closes #12115 + This handles cases of some bi-directional "upgrade" scenarios + (i.e. WebSockets) where sending is paused until some "upgrade" handshake + is completed, but server rejects the handshake and closes the + connection. -Daniel Stenberg (17 Oct 2023) + Closes #12428 -- multi: use pipe instead of socketpair to *wakeup() +Daniel Stenberg (28 Nov 2023) - If pipe() is present. Less overhead. +- RELEASE-NOTES: synced - Closes #12142 +- openssl: when a session-ID is reused, skip OCSP stapling -Jay Satiro (17 Oct 2023) + Fixes #12399 + Reported-by: Alexey Larikov + Closes #12418 -- build: fix 'threadsafe' feature detection for older gcc +- test1545: test doing curl_formadd twice with missing file - - Add 'threadsafe' to the feature list shown during build if POSIX - threads are being used. + Reproduces #12410 + Verifies the fix + Closes #12421 - This is a follow-up to 5adb6000 which added support for building a - thread-safe libcurl with older versions of gcc where atomic is not - available but pthread is. +- Curl_http_body: cleanup properly when Curl_getformdata errors - Reported-by: Dan Fandrich - Co-authored-by: Dan Fandrich + Reported-by: yushicheng7788 on github + Based-on-work-by: yushicheng7788 on github + Fixes #12410 + Closes #12421 - Fixes https://github.com/curl/curl/issues/12125 - Closes https://github.com/curl/curl/pull/12127 +- test1477: verify that libcurl-errors.3 and public headers are synced -Daniel Stenberg (16 Oct 2023) + The script errorcodes.pl extracts all error codes from all headers and + checks that they are all documented, then checks that all documented + error codes are also specified in a header file. -- test729: verify socks4a with excessive proxy user name length + Closes #12424 -- socks: better buffer size checks for socks4a user and hostname +- libcurl-errors.3: sync with current public headers - Also limit the proxy user name to 255 bytes, which is the same limit as - in SOCKS5. + Closes #12424 - Reported-by: sd0 on hackerone - Closes #12139 +Stefan Eissing (28 Nov 2023) -- curl.h: on FreeBSD include sys/param.h instead of osreldate.h +- test459: fix for parallel runs - Should things build on Playstation as well + - change warniing message to work better with varying filename + length. + - adapt test output check to new formatting - Fixes #12107 - Reported-by: Faraz Fallahi - Closes #12123 + Follow-up to 97ccc4479f77ba3191c6 + Closes #12423 -Marcin Rataj (16 Oct 2023) +Daniel Stenberg (27 Nov 2023) -- tool_operate: fix links in ipfs errors +- tool_cb_prg: make the carriage return fit for wide progress bars - URL fragment links generated from headers in - https://curl.se/docs/ipfs.html are lowercase. + When the progress bar was made max width (256 columns), the fly() + function attempted to generate its output buffer too long so that the + trailing carriage return would not fit and then the output would show + wrongly. The fly function is called when the expected total transfer is + unknown, which could be one or more progress calls before the actual + progress meter get shown when the expected transfer size is provided. - Closes #12133 + This new take also replaces the msnprintf() call with a much simpler + memset() for speed. -Viktor Szakats (15 Oct 2023) + Reported-by: Tim Hill + Fixes #12407 + Closes #12415 -- cmake: replace `check_library_exists_concat()` +- tool_parsecfg: make warning output propose double-quoting - The idea of `check_library_exists_concat()` is that it detects an - optional component and adds it to the list of libs that we also use in - subsequent component checks. This caused problems when detecting - components with unnecessary dependencies that were not yet built. + When the config file parser detects a word that *probably* should be + quoted, mention double-quotes as a possible remedy. - CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used - for component checks, which we already use in most cases. That left 4 - uses of `check_library_exists_concat()`. Only one of these actually - needed the 'concat' feature (ldap/lber). + Test 459 verifies. - Delete this function and replace it with standard - `check_library_exists()` and manual management of our `CURL_LIBS` - list we use when linking build targets. And special logic to handle the - ldap/lber case. + Proposed-by: Jiehong on github + Fixes #12409 + Closes #12412 - (We have a similar function for headers: `check_include_file_concat()`. - It works, but problematic for performance reasons and because it hides - the actual headers required in `check_symbol_exists()` calls.) +Jay Satiro (26 Nov 2023) - Ref: #11537 #11558 - Fixes #11285 - Fixes #11648 - Closes #12070 +- curl.rc: switch out the copyright symbol for plain ASCII -LoRd_MuldeR (15 Oct 2023) + .. like we already do for libcurl.rc. -- tool_cb_wrt: fix write output for very old Windows versions + libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" + warning so it was switched to ascii. - - Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW() - function. + Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 - Apparently this parameter was *not* optional on older Windows versions. + Suggested-by: Robert Southee - Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1. - So at some point between those two Microsoft changed the behavior. + Closes https://github.com/curl/curl/pull/12403 - Prior to this change, on those versions if parameter is NULL then the - function call fails with error ERROR_INVALID_ACCESS. +Daniel Stenberg (26 Nov 2023) - Regression since af3f4e41. +- conncache: use the closure handle when disconnecting surplus connections - Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299 + Use the closure handle for disconnecting connection cache entries so + that anything that happens during the disconnect is not stored and + associated with the 'data' handle which already just finished a transfer + and it is important that details from the unrelated disconnect does not + taint meta-data in the data handle. - Fixes https://github.com/curl/curl/issues/12131 - Closes https://github.com/curl/curl/pull/12130 - -Jay Satiro (15 Oct 2023) + Like storing the response code. -- tool_urlglob: fix build for old gcc versions + This also adjust test 1506. Unfortunately it also removes a key part of + the test that verifies that a connection is closed since when this + output vanishes (because the closure handle is used), we don't know + exactly that the connection actually gets closed in this test... - - Don't use __builtin_mul_overflow for GCC 4 and earlier. + Reported-by: ohyeaah on github + Fixes #12367 + Closes #12405 - The function was added in GCC 5. +- RELEASE-NOTES: synced - Ref: https://gcc.gnu.org/gcc-5/changes.html +Stefan Eissing (24 Nov 2023) - Reported-by: Dan Fandrich +- quic: make eyeballers connect retries stop at weird replies - Fixes https://github.com/curl/curl/issues/12124 - Closes https://github.com/curl/curl/pull/12128 + - when a connect immediately goes into DRAINING state, do + not attempt retries in the QUIC connection filter. Instead, + return CURLE_WEIRD_SERVER_REPLY + - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an + inconclusive answer. When all addresses have been attempted, + rewind the address list once on an inconclusive answer. + - refs #11832 where connects were retried indefinitely until + the overall timeout fired -Carlos Henrique Lima Melara (14 Oct 2023) + Closes #12400 -- docs/libcurl: fix three minor man page format mistakes +Daniel Stenberg (24 Nov 2023) - Reported-by: Samuel Henrique +- CI: verify libcurl function SYNPOSIS sections - Closes https://github.com/curl/curl/pull/12126 + With the .github/scripits/verify-synopsis.pl script -Jay Satiro (14 Oct 2023) + Closes #12402 -- tests/server: add more SOCKS5 handshake error checking +- docs/libcurl: SYNSOPSIS cleanup - - Add additional checking for missing and too-short SOCKS5 handshake - messages. + - use the correct include file + - make sure they are declared as in the header file + - fix minor nroff syntax mistakes (missing .fi) - Prior to this change the SOCKS5 test server did not check that all parts - of the handshake were received successfully. If those parts were missing - or too short then the server would access uninitialized memory. + These are verified by verify-synopsis.pl, which extracts the SYNPOSIS + code and runs it through gcc. - This issue was discovered in CI job 'memory-sanitizer' test results. - Test 2055 was failing due to the SOCKS5 test server not running. It was - not running because either it crashed or memory sanitizer aborted it - during Test 728. Test 728 connects to the SOCKS5 test server on a - redirect but does not send any data on purpose. The test server was not - prepared for that. + Closes #12402 - Reported-by: Dan Fandrich +- sendf: fix comment typo - Fixes https://github.com/curl/curl/issues/12117 - Closes https://github.com/curl/curl/pull/12118 +- fopen: allocate the dir after fopen -Daniel Stenberg (14 Oct 2023) + Move the allocation of the directory name down to after the fopen() call + to allow that shortcut code path to avoid a superfluous malloc+free + cycle. -- RELEASE-NOTES: synced + Follow-up to 73b65e94f35311 -Sohom Datta (14 Oct 2023) + Closes #12398 -- tool_getparam: limit --rate to be smaller than number of ms +Stefan Eissing (24 Nov 2023) - Currently, curl allows users to specify absurd request rates that might - be higher than the number of milliseconds in the unit (ex: curl --rate - 3600050/h http://localhost:8080 does not error out despite there being - only 3600000ms in a hour). +- transfer: cleanup done+excess handling - This change adds a conditional check before the millisecond calculation - making sure that the number is not higher than the numerator (the unit) - If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE + - add `SingleRequest->download_done` as indicator that + all download bytes have been received + - remove `stop_reading` bool from readwrite functions + - move excess body handling into client download writer - Closes #12116 + Closes #12371 -Daniel Stenberg (14 Oct 2023) +Daniel Stenberg (23 Nov 2023) -- opts: fix two minor man page format mistakes +- fopen: create new file using old file's mode -Jay Satiro (14 Oct 2023) + Because the function renames the temp file to the target name as a last + step, if the file was previously owned by a different user, not ORing + the old mode could otherwise end up creating a file that was no longer + readable by the original owner after save. -- curl_trc: remove a bad assertion + Reported-by: Loïc Yhuel + Fixes #12299 + Closes #12395 - - Remove DEBUGASSERT that an internal handle must not have user - private_data set before calling the user's debug callback. +- test1476: require proxy - This is a follow-up to 0dc40b2a. The user can distinguish their easy - handle from an internal easy handle by setting CURLOPT_PRIVATE on their - easy handle. I had wrongly assumed that meant the user couldn't then - set CURLOPT_PRIVATE on an internal handle as well. + Follow-up from 323df4261c3542 - Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 - Reported-by: Daniel Stenberg + Closes #12394 - Closes https://github.com/curl/curl/pull/12104 +- fopen: create short(er) temporary file name -Dan Fandrich (13 Oct 2023) + Only using random letters in the name plus a ".tmp" extension. Not by + appending characters to the final file name. -- test613: stop showing an error on missing output file + Reported-by: Maksymilian Arciemowicz - This test would show an error message if the output was missing during - the log post-processing step, but the message was not captured by the - test harness and wasn't useful since the normal golden log file - comparison would the problem more clearly. + Closes #12388 -Stefan Eissing (13 Oct 2023) +Stefan Eissing (23 Nov 2023) -- quic: manage connection idle timeouts +- tests: git ignore generated second-hsts.txt file - - configure a 120s idle timeout on our side of the connection - - track the timestamp when actual socket IO happens - - check IO timestamp to our *and* the peer's idle timeouts - in "is this connection alive" checks + File is generated in test lib1900 - Reported-by: calvin2021y on github - Fixes #12064 - Closes #12077 + Follow-up to 7cb03229d9e9c5 -Dan Fandrich (13 Oct 2023) + Closes #12393 -- CI: ignore test 286 on Appveyor gcc 9 build +Viktor Szakats (23 Nov 2023) - This test fails sometimes with a super fast retry loop due to what may - just be a compiler bug. The test results are ignored on the one CI job - where it occurs because there seems to be nothing we can do to fix it. +- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 - Fixes #12040 - Closes #12106 + Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to + OpenSSL 1.1.x, and LibreSSL 3.6 or upper. -Viktor Szakats (13 Oct 2023) + With the caveat, that "group name" and "type name" are missing from + the log output with these TLS backends. -- lib: fix gcc warning in printf call + Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Do not pass NULL to printf %s. + Reviewed-by: Daniel Stenberg + Closes #12385 - Seen with gcc 13.2.0 on Debian: - ``` - .../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wfo - rmat-overflow=] - ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/1758442 - 6483#step:3:11104 +Daniel Stenberg (23 Nov 2023) - Ref: #10284 - Co-authored-by: Jay Satiro - Closes #12082 +- urldata: fix typo in comment -Alex Klyubin (13 Oct 2023) +- CI: codespell -- http2: safer invocation of populate_binsettings + The list of words to ignore is in the file + .github/scripts/codespell-ignore.txt - populate_binsettings now returns a negative value on error, instead of a - huge positive value. Both places which call this function have been - updated to handle this change in its contract. + Closes #12390 - The way populate_binsettings had been used prior to this change the huge - positive values -- due to signed->unsigned conversion of the potentially - negative result of nghttp2_pack_settings_payload which returns negative - values on error -- are not possible. But only because http2.c currently - always provides a large enough output buffer and provides H2 SETTINGS - IVs which pass the verification logic inside nghttp2. If the - verification logic were to change or if http2.c started passing in more - IVs without increasing the output buffer size, the overflow could become - reachable, and libcurl/curl might start leaking memory contents to - servers/proxies... +- lib: fix comment typos - Closes #12101 + Five separate ones, found by codespell -Daniel Stenberg (13 Oct 2023) + Closes #12390 -- openssl: avoid BN_num_bits() NULL pointer derefs +- test1476: verify cookie PSL mixed case - Reported-by: icy17 on github - Fixes #12099 - Closes #12100 +- cookie: lowercase the domain names before PSL checks -- wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA + Reported-by: Harry Sintonen - This define is set in wolfssl's options.h file when this function and - feature is present. Handles both builds with the feature explicitly - disabled and wolfSSL versions before 5.5.2 - which introduced this API - call. + Closes #12387 - Closes #12108 +Viktor Szakats (23 Nov 2023) -- tool_urlglob: make multiply() bail out on negative values +- openssl: fix building with v3 `no-deprecated` + add CI test - - Does not work correctly with negative values - - use __builtin_mul_overflow() on gcc + - build quictls with `no-deprecated` in CI to have test coverage for + this OpenSSL 3 configuration. - Reported-by: Torben Dury - Closes #12102 + - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. + The caller code is meant for OpenSSL 3, while these two functions were + only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 + if built with option `no-deprecated`, causing build errors: + ``` + vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ + algorithms'; ISO C99 and later do not support implicit function declaration + s [-Wimplicit-function-declaration] + vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ + digests'; ISO C99 and later do not support implicit function declarations [ + -Wimplicit-function-declaration] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f + ullLog=true#L7667 -Loïc Yhuel (13 Oct 2023) + Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 + Reviewed-by: Alex Bozarth -- cmake: fix CURL_DISABLE_GETOPTIONS + - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with + `no-deprecated` quictls 3 builds. + Do it by moving an existing solution for this from `vtls/openssl.c` + to `vtls/openssl.h` and adjusting caller code. + ``` + vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g + et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit + -function-declaration] + ``` + Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s + tep:24:1178 - - Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake. + - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and + `-Wunused-function` when trying to build curl with NTLM enabled but + without the necessary TLS backend (with DES) support. - Prior to this change the option had no effect because it was missing - from that file. + Closes #12384 - Closes https://github.com/curl/curl/pull/12091 +- curl.h: delete Symbian OS references -- easy_lock: add a pthread_mutex_t fallback + curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 + via #5989. Delete references to it from public headers, because there + is no fresh release to use those headers with. - This allows to keep the init threadsafe with gcc < 4.9.0 (no C11 - atomics). + Reviewed-by: Dan Fandrich + Reviewed-by: Jay Satiro + Closes #12378 - Closes https://github.com/curl/curl/pull/12090 +- windows: use built-in `_WIN32` macro to detect Windows -Viktor Szakats (12 Oct 2023) + Windows compilers define `_WIN32` automatically. Windows SDK headers + or build env defines `WIN32`, or we have to take care of it. The + agreement seems to be that `_WIN32` is the preferred practice here. + Make the source code rely on that to detect we're building for Windows. -- CI: add autotools, out-of-tree, debug build to distro check job + Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for + Windows detection, next to the official `_WIN32`. After this patch it + only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. - Add a job that builds curl from a generated source tarball sample, with - autotools, out-of-tree, in debug mode. + There is a slight chance these break compatibility with Windows + compilers that fail to define `_WIN32`. I'm not aware of any obsolete + or modern compiler affected, but in case there is one, one possible + solution is to define this macro manually. - Ref: #12085 - Closes #12088 + grepping for `WIN32` remains useful to discover Windows-specific code. -Daniel Stenberg (12 Oct 2023) + Also: -- http: avoid Expect: 100-continue if Upgrade: is used + - extend `checksrc` to ensure we're not using `WIN32` anymore. - Reported-by: Daniel Jelinski - Fixes #12022 - Closes #12062 + - apply minor formatting here and there. -Jan Alexander Steffens (heftig) (12 Oct 2023) + - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. -- docs: use SOURCE_DATE_EPOCH for generated manpages + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg - This should make builds from Git reproducible. + Closes #12376 - Closes #12092 +Stefan Eissing (22 Nov 2023) -Daniel Stenberg (12 Oct 2023) +- url: ConnectionExists revisited -- RELEASE-NOTES: synced + - have common pattern of `if not match, continue` + - revert pages long if()s to return early + - move dead connection check to later since it may + be relatively expensive + - check multiuse also when NOT building with NGHTTP2 + - for MULTIUSE bundles, verify that the inspected + connection indeed supports multiplexing when in use + (bundles may contain a mix of connection, afaict) - Bumped to 8.4.1 + Closes #12373 -Viktor Szakats (12 Oct 2023) +Daniel Stenberg (22 Nov 2023) -- cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection +- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range - Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet, - aligning this with autotools. This fixes an error when doing - cross-builds and also actually detects this feature. It affected systems - not allowlisted into this, e.g. SerenityOS. + ... or use the default value. - We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`. + Also clarify the documentation language somewhat. - Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979 - Ref: #12095 (closed in favour of this patch) - Ref: #11964 (effort to sync cmake detections with autotools) + Closes #12382 - Reported-by: Kartatz on Github - Assisted-by: Kartatz on Github - Fixes #12093 - Closes #12094 +- urldata: make maxconnects a 32 bit value -- build: add `src/.checksrc` to source tarball + "2^32 idle connections ought to be enough for anybody" - Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958 + Closes #12375 - Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071 - Reported-by: Romain Geissler - Fixes #12084 - Closes #12085 +- FEATURES: update the URL phrasing -Version 8.4.0 (11 Oct 2023) + The URL is length limited since a while back so "no limit" simply is not + true anymore. Mention the URL RFC standard used instead. -Daniel Stenberg (11 Oct 2023) + Closes #12383 -- RELEASE-NOTES: synced +- wolfssh: remove redundant static prototypes -- THANKS: add contributors from 8.4.0 + vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We + rror=redundant-decls] -Jay Satiro (11 Oct 2023) + Closes #12381 -- socks: return error if hostname too long for remote resolve +- setopt: remove superfluous use of ternary expressions - Prior to this change the state machine attempted to change the remote - resolve to a local resolve if the hostname was longer than 255 - characters. Unfortunately that did not work as intended and caused a - security issue. + Closes #12374 - Bug: https://curl.se/docs/CVE-2023-38545.html +- mime: store "form escape" as a single bit -Stefan Eissing (10 Oct 2023) + Closes #12374 -- CI: remove slowed-network tests +- setopt: check CURLOPT_TFTP_BLKSIZE range on set - - remove these tests as they are currently not reliable in our CI - setups. + ... instead of later when the transfer is about to happen. - curl handles the test cases, but CI sometimes fails on these due to - additional conditions. Rather than mix them in, an additional CI job - will be added in the future that is specific to them. + Closes #12374 - Closes https://github.com/curl/curl/pull/12075 +Viktor Szakats (21 Nov 2023) -Jay Satiro (10 Oct 2023) +- build: add more picky warnings and fix them -- libcurl-env-dbg.3: move debug variables from libcurl-env.3 + Enable more picky compiler warnings. I've found these options in the + nghttp3 project when implementing the CMake quick picky warning + functionality for it [1]. - - Move documentation of libcurl environment variables used only in debug - builds from libcurl-env into a separate document libcurl-env-dbg. + `-Wunused-macros` was too noisy to keep around, but fixed a few issues + it revealed while testing. - - Document more debug environment variables. + - autotools: reflect the more precisely-versioned clang warnings. + Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 + - autotools: sync between clang and gcc the way we set `no-multichar`. + - autotools: avoid setting `-Wstrict-aliasing=3` twice. + - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. + It triggers in libtool-generated stub code. - Previously undocumented or missing a description: + - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. - CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, - CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, - CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, - CURL_TIME. + - lib/curl_setup.h: delete duplicate declaration for `fileno`. + Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d + (1999-12-29). This suggests this may not be needed anymore, but if + it does, we may restore this for those specific (non-Windows) systems. + - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since + c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. + - lib: delete unused macro `isxdigit_ascii` since + f65f750742068f579f4ee6d8539ed9d5f0afcb85. + - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. + - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. + - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` + macro. + - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. + - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. + - lib/setopt: fix `-Wunreachable-code-break`. + - lib/system_win32 and lib/timeval: fix double declarations for + `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. + - lib/warnless: fix double declarations in CMake UNITY mode [5]. + This was due to force-disabling the header guard of `warnless.h` to + to reapply it to source code coming after `warnless.c` in UNITY + builds. This reapplied declarations too, causing the warnings. + Solved by adding a header guard for the lines that actually need + to be reapplied. + - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. + - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant + block. + - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. + - lib/vtls/sectransp: suppress `-Wunreachable-code`. + Detected in `else` branches of dynamic feature checks, with results + known at compile-time, e.g. + ```c + if(SecCertificateCopySubjectSummary) /* -> true */ + ``` + Likely fixable as a separate micro-project, but given SecureTransport + is deprecated anyway, let's just silence these locally. + - src/tool_help: delete duplicate declaration for `helptext`. + - src/tool_xattr: fix `-Wunreachable-code`. + - tests: delete duplicate declaration for `unitfail` [8]. + - tests: delete duplicate declaration for `strncasecompare`. + - tests/libtest: delete duplicate declaration for `gethostname`. + Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 + (2010-08-02). + Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc + If there are still systems around with warnings, we may restore the + prototype, but limited for those systems. + - tests/lib2305: delete duplicate declaration for + `libtest_debug_config`. + - tests/h2-download: fix `-Wunreachable-code-break`. - Closes https://github.com/curl/curl/pull/11811 + [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 + f8bf9f/cmake/PickyWarningsC.cmake + [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui + qla5fj45?fullLog=true#L1675 + [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 + 2331#step:7:72 + https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 + 2331#step:7:109 + [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L204 + [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L218 + [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 + 2331#step:7:290 + [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 + 2331#step:9:1193 + [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 + 2331#step:33:1870 -Dan Fandrich (9 Oct 2023) + Closes #12331 -- test670: increase the test timeout +Daniel Stenberg (21 Nov 2023) - This should make it more immune to loaded servers. +- transfer: avoid unreachable expression - Ref: #11328 + If curl_off_t and size_t have the same size (which is common on modern + 64 bit systems), a condition cannot occur which Coverity pointed + out. Avoid the warning by having the code conditionally only used if + curl_off_t actually is larger. -Stefan Eissing (9 Oct 2023) + Follow-up to 1cd2f0072fa482e25baa2 -- MQTT: improve receive of ACKs + Closes #12370 - - add `mq->recvbuf` to provide buffering of incomplete - ACK responses - - continue ACK reading until sufficient bytes available - - fixes test failures on low network receives +Stefan Eissing (21 Nov 2023) - Closes #12071 +- transfer: readwrite improvements -Viktor Szakats (9 Oct 2023) + - changed header/chunk/handler->readwrite prototypes to accept `buf`, + `blen` and a `pconsumed` pointer. They now get the buffer to work on + and report back how many bytes they consumed + - eliminated `k->str` in SingleRequest + - improved excess data handling to properly calculate with any body data + left in the headerb buffer + - eliminated `k->badheader` enum to only be a bool -- quic: fix BoringSSL build + Closes #12283 - Add guard around `SSL_CTX_set_ciphersuites()` use. +Daniel Stenberg (21 Nov 2023) - Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 +- RELEASE-NOTES: synced - Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 +Jiří Hruška (21 Nov 2023) - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes #12067 +- transfer: avoid calling the read callback again after EOF -Stefan Eissing (9 Oct 2023) + Regression since 7f43f3dc5994d01b12 (7.84.0) -- test1540: improve reliability - - - print that bytes have been received on pausing, but not how many + Bug: https://curl.se/mail/lib-2023-11/0017.html - Closes #12069 + Closes #12363 -- test2302: improve reliability +Daniel Stenberg (21 Nov 2023) - - make result print collected write data, unless - change in meta flags is detected - - will show same result even when data arrives via - several writecb invocations +- doh: provide better return code for responses w/o addresses - Closes #12068 + Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the + response did not contain any addresses. Now it more accurately returns + CURLE_COULDNT_RESOLVE_HOST. -Daniel Stenberg (9 Oct 2023) + Reported-by: lRoccoon on github -- curl_easy_pause: set "in callback" true on exit if true + Fixes #12365 + Closes #12366 - Because it might have called another callback in the mean time that then - set the bit FALSE on exit. +Stefan Eissing (21 Nov 2023) - Reported-by: Jay Satiro - Fixes #12059 - Closes #12061 +- HTTP/2, HTTP/3: handle detach of onoing transfers -Viktor Szakats (8 Oct 2023) + - refs #12356 where a UAF is reported when closing a connection + with a stream whose easy handle was cleaned up already + - handle DETACH events same as DONE events in h2/h3 filters -- h3: add support for ngtcp2 with AWS-LC builds + Fixes #12356 + Reported-by: Paweł Wegner + Closes #12364 - ``` - curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- - LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 - Release-Date: [unreleased] - Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps - mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss - Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi - SSL NTLM SSL threadsafe UnixSockets - ``` +Viktor Szakats (20 Nov 2023) - Also delete an obsolete GnuTLS TODO and update the header comment in - `FindNGTCP2.cmake`. +- autotools: stop setting `-std=gnu89` with `--enable-warnings` - Reviewed-by: Daniel Stenberg - Closes #12066 + Do not alter the C standard when building with `--enable-warnings` when + building with gcc. -- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros + On one hand this alters warning results compared to a default build. + On the other, it may produce different binaries, which is unexpected. - Syncing this up with CMake. + Also fix new warnings that appeared after removing `-std=gnu89`: - Source code uses the built-in `OPENSSL_IS_AWSLC` and - `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is - necessary from the build tools. + - include: fix public curl headers to use the correct printf mask for + `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 + and Visual Studio 2013 and newer. This fixes the printf mask warnings + in examples and tests. E.g. [1] - The one use of `HAVE_BORINGSSL` in the source turned out to be no longer - necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 + - conncache: fix printf format string [2]. - autotools detects this anyway for display purposes. - CMake detects this to decide whether to use the BoringSSL-specific - crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection - result just yet (planned in #12066). + - http2: fix potential null pointer dereference [3]. + (seen on Slackware with gcc 11.) - Ref: #11964 + - libssh: fix printf format string in SFTP code [4]. + Also make MSVC builds compatible with old CRT versions. - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12065 + - libssh2: fix printf format string in SFTP code for MSVC. + Applying the same fix as for libssh above. -Marc Hoersken (8 Oct 2023) + - unit1395: fix `argument is null` and related issues [5]: + - stop calling `strcmp()` with NULL to avoid undefined behaviour. + - fix checking results if some of them were NULL. + - do not pass NULL to printf `%s`. -- CI: move distcheck job from Azure Pipelines to GitHub Actions + - ci: keep a build job with `-std=gnu89` to continue testing for + C89-compliance. We can apply this to other gcc jobs as needed. + Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 - This will allow for more trigger excludes within Azure Pipelines. + [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l + ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b + [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 + 2346#step:6:67 + [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 + 2346#step:30:214 + [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 + 2346#step:29:895 + [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 + 2346#step:33:1689 - Also fixes seemingly broken check with scripts/installcheck.sh. - Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 + Closes #12346 - Assisted-by: Philip Heiduck - Closes #9532 +- autotools: fix/improve gcc and Apple clang version detection -Daniel Stenberg (8 Oct 2023) + - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu + may return `n-win32` (also with `-dumpfullversion`). Causing these + errors and failing to enable picky warnings: + ``` + ../configure: line 23845: test: : integer expression expected + ``` + Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 + 3718#step:5:143 -- url: fall back to http/https proxy env-variable if ws/wss not set + Fix that by stripping any dash-suffix and handling a dotless (major-only) + version number by assuming `.0` in that case. - Reported-by: Craig Andrews - Fixes #12031 - Closes #12058 + `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` + Ref: https://github.com/mamedev/mame/pull/9767 -Stefan Eissing (8 Oct 2023) + - fix Apple clang version detection for releases between + 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the + version was under-detected as 3.7 llvm/clang equivalent. -- cf-socket: simulate slow/blocked receives in debug + - fix Apple clang version detection for 'Apple clang version 11.0.0' + and newer where the Apple clang version was detected, instead of its + llvm/clang equivalent. - add 2 env variables for non-UDP sockets: - 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly - should return EAGAIN - 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket + - display detected clang/gcc/icc compiler version. - Closes #12035 + Via libssh2: + - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d + 89875ad + https://github.com/libssh2/libssh2/pull/1187 + - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 + 42b592e + https://github.com/libssh2/libssh2/pull/1232 -- http2: refused stream handling for retry + Closes #12362 - - answer HTTP/2 streams refused via a GOAWAY from the server to - respond with CURLE_RECV_ERROR in order to trigger a retry - on another connection +- autotools: delete LCC compiler support bits - Reported-by: black-desk on github - Ref #11859 - Closes #12054 + Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 -Jay Satiro (8 Oct 2023) + Closes #12357 -- CURLOPT_DEBUGFUNCTION.3: warn about internal handles +- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` - - Warn that the user's debug callback may be called with the handle - parameter set to an internal handle. + - tests: verify CMake `DISABLE` options. - Without this warning the user may assume that the only handles their - debug callback receives are the easy handles on which they set - CURLOPT_DEBUGFUNCTION. + Make an exception for 2 CMake-only ones, and one more that's + using a different naming scheme, also in autotools and source. - This is a follow-up to f8cee8cc which changed DoH handles to inherit - the debug callback function set in the user's easy handle. As a result - those handles are now passed to the user's debug callback function. + - cmake: add support for `CURL_DISABLE_HEADERS_API`. - Closes https://github.com/curl/curl/pull/12034 + Suggested-by: Daniel Stenberg + Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 -- url: fix typo + Closes #12353 -Daniel Stenberg (8 Oct 2023) +Jacob Hoffman-Andrews (20 Nov 2023) -- test458: verify --expand-output, expanding a file name accepting option +- hyper: temporarily remove HTTP/2 support - Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) + The current design of the Hyper integration requires rebuilding the + Hyper clientconn for each request. However, building the clientconn + requires resending the HTTP/2 connection preface, which is incorrect + from a protocol perspective. That in turn causes servers to send GOAWAY + frames, effectively degrading performance to "no connection reuse" in + the best case. It may also be triggering some bugs where requests get + dropped entirely and reconnects take too long. -- tool_getparam: accept variable expansion on file names too + This doesn't rule out HTTP/2 support with Hyper, but it may take a + redesign of the Hyper integration in order to make things work. - Reported-by: PBudmark on github - Fixes #12048 - Closes #12055 + Closes #12191 -- RELEASE-NOTES: synced +Jay Satiro (20 Nov 2023) -- multi: do CURLM_CALL_MULTI_PERFORM at two more places +- schannel: fix unused variable warning - ... when it does a state transition but there is no particular socket or - timer activity. This was made apparent when commit b5bb84c removed a - superfluous timer expiry. + Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 + Reported-by: Viktor Szakats - Reported-by: Dan Fandrich. - Fixes #12033 - Closes #12056 + Closes https://github.com/curl/curl/pull/12361 -Viktor Szakats (7 Oct 2023) +Daniel Stenberg (19 Nov 2023) -- GHA/linux: mbedtls 3.5.0 + minor dep bumps +- url: find scheme with a "perfect hash" - Closes #12057 + Instead of a loop to scan over the potentially 30+ scheme names, this + uses a "perfect hash" table. This works fine because the set of schemes + is known and cannot change in a build. The hash algorithm and table size + is made to only make a single scheme index per table entry. -Dan Fandrich (7 Oct 2023) + The perfect hash is generated by a separate tool (scripts/schemetable.c) -- CI: bump OpenLDAP package version on FreeBSD + Closes #12347 - The old one is no longer available. +- scripts: add schemetable.c -Marc Hoersken (7 Oct 2023) + This tool generates a scheme-matching table. -- docs/libcurl/opts/Makefile.inc: add missing manpage files + It iterates over a number of different initial and shift values in order + to find the hash algorithm that needs the smallest possible table. - Detected with #9532 + The generated hash function, table and table size then needs to be used + by the url.c:Curl_getn_scheme_handler() function. -Dan Fandrich (7 Oct 2023) +Stefan Eissing (19 Nov 2023) -- tests: fix a race condition in ftp server disconnect +- vtls/vquic, keep peer name information together - If a client disconnected and reconnected quickly, before the ftp server - had a chance to respond, the protocol message/ack (ping/pong) sequence - got out of sync, causing messages sent to the old client to be delivered - to the new. A disconnect must now be acknowledged and intermediate - requests thrown out until it is, which ensures that such synchronization - problems can't occur. This problem could affect ftp, pop3, imap and smtp - tests. + - add `struct ssl_peer` to keep hostname, dispname and sni + for a filter + - allocate `sni` for use in VTLS backend + - eliminate `Curl_ssl_snihost()` and its use of the download buffer + - use ssl_peer in SSL and QUIC filters - Fixes #12002 - Closes #12049 + Closes #12349 -Viktor Szakats (7 Oct 2023) +Viktor Szakats (18 Nov 2023) -- appveyor: bump mingw-w64 job to gcc 13 (was: 8) +- build: always revert `#pragma GCC diagnostic` after use - This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). - Adding a modern gcc version to the tests. + Before this patch some source files were overriding gcc warning options, + but without restoring them at the end of the file. In CMake UNITY builds + these options spilled over to the remainder of the source code, + effecitvely disabling them for a larger portion of the codebase than + intended. - (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 - finished in 32, 35, 34 minutes in the 3 test runs so far.) + `#pragma clang diagnostic` didn't have such issue in the codebase. - It also adds a modern CMake version and OS env to our mingw-w64 builds. + Reviewed-by: Marcel Raad + Closes #12352 - Closes #12051 +- tidy-up: casing typos, delete unused Windows version aliases -David Benjamin (6 Oct 2023) + - cmake: fix casing of `UnixSockets` to match the rest of the codebase. -- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR + - curl-compilers.m4: fix casing in a comment. - While the struct is still public in OpenSSL, there is a (somewhat - inconvenient) accessor. Use it to remain compatible if it becomes opaque - in the future. + - setup-win32: delete unused Windows version constant aliases. - Closes #12038 + Reviewed-by: Marcel Raad + Closes #12351 -Daniel Stenberg (6 Oct 2023) +- keylog: disable if unused -- curl_easy_pause.3: mention it works within callbacks + Fully disable keylog code if there is no TLS or QUIC subsystem using it. - Reported-by: Maxim Dzhura - Bug: https://curl.se/mail/lib-2023-10/0010.html - Closes #12046 + Closes #12350 -- curl_easy_pause.3: mention h2/h3 buffering +- cmake: add `CURL_DISABLE_BINDLOCAL` option - Asked-by: Maxim Dzhura - Ref: https://curl.se/mail/lib-2023-10/0011.html + To match similar autotools option. - Closes #12045 + Default is `ON`. -Viktor Szakats (6 Oct 2023) + Reviewed-by: Daniel Stenberg + Closes #12345 -- cmake: re-add missed C89 headers for specific detections +- url: fix `-Wzero-length-array` with no protocols - We removed C89 `setjmp.h` and `signal.h` detections and excluded them - from the global header list we use when detecting functions [1]. Then - missed to re-add these headers to the specific functions which need - them to be detected [2]. Fix this omission in this patch. + Fixes: + ``` + ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ + -Wc2x-extensions] + 178 | static const struct Curl_handler * const protocols[] = { + | ^ + ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length + -array] + ``` - [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 - [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + Closes #12344 - Closes #12043 +- url: fix builds with `CURL_DISABLE_HTTP` -Daniel Stenberg (6 Oct 2023) + Fixes: + ``` + ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' + 456 | Curl_mime_cleanpart(data->state.formp); + | ~~~~~~~~~~~ ^ + ``` -- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE + Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 - Since there is nothing to wait for there. Avoids the test 1233 hang - reported in #12033. + Closes #12343 - Reported-by: Dan Fandrich - Closes #12042 +- http: fix `-Wunused-parameter` with no auth and no proxy -Dan Fandrich (5 Oct 2023) + ``` + lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] + bool proxy) + ^ + ``` -- test1903: actually verify the cookies after the test + Reviewed-by: Marcel Raad + Closes #12338 - The test otherwise could do just about anything (except leak memory in - debug mode) and its bad behaviour wouldn't be detected. Now, check the - resulting cookie file to ensure the cookies are still there. +Daniel Stenberg (16 Nov 2023) - Closes #12041 +- TODO: Some TLS options are not offered for HTTPS proxies -- test: add missing s + Closes #12286 + Closes #12342 - The tests will otherwise fail if curl has them disabled. +- RELEASE-NOTES: synced -- test1906: set a lower timeout since it's hit on Windows +- duphandle: make dupset() not return with pointers to old alloced data - msys2 builds actually hit the connect timeout in normal operation, so - lower the timeout from 5 minutes to 5 seconds to reduce test time. + As the blob pointers are to be duplicated, the function must not return + mid-function with lingering pointers to the old handle's allocated data, + as that would lead to double-free in OOM situations. - Ref: #11328 - Closes #12036 + Make sure to clear all destination pointers first to avoid this risk. -Daniel Stenberg (5 Oct 2023) + Closes #12337 -- RELEASE-NOTES: synced +Viktor Szakats (16 Nov 2023) -Jay Satiro (5 Oct 2023) +- http: fix `-Wunused-variable` compiler warning -- idn: fix WinIDN null ptr deref on bad host + Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. - - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from - UTF-8 to UTF-16. + E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + + `CURL_DISABLE_NTLM` on non-Windows. - Prior to this change a failed conversion erroneously returned CURLE_OK - which meant 'decoded' pointer (what would normally point to the - punycode) would not be written to, remain NULL and be dereferenced - causing an access violation. + ``` + ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl + e] + CURLcode result = CURLE_OK; + ^ + ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus + ed-but-set-variable] + unsigned long *availp; + ^ + ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse + d-but-set-variable] + struct auth *authp; + ^ + ``` - Closes https://github.com/curl/curl/pull/11983 + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 -Dan Fandrich (4 Oct 2023) + Fixes #12228 + Closes #12335 -- tests: close the shell used to start sshd +Jay Satiro (16 Nov 2023) - This shell isn't needed once sshd starts, so use "exec" so it doesn't - stick around. +- tool: support bold headers in Windows - Closes #12032 + - If virtual terminal processing is enabled in Windows then use ANSI + escape codes Esc[1m and Esc[22m to turn bold on and off. -Daniel Stenberg (4 Oct 2023) + Suggested-by: Gisle Vanem -- base64: also build for curl + Ref: https://github.com/curl/curl/discussions/11770 - Since the tool itself now uses the base64 code using the curlx way, it - needs to build also when the tool needs it. Starting now, the tool build - defines BULDING_CURL to allow lib-side code to use it. + Closes https://github.com/curl/curl/pull/12321 - Follow-up to 2e160c9c6525 +Viktor Szakats (15 Nov 2023) - Closes #12010 +- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` -Eduard Strehlau (4 Oct 2023) + Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + + `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, + or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while + compiling due to a missing HMAC declaration. -- tests: Fix zombie processes left behind by FTP tests. + The reason is that HMAC is required by `lib/sha256.c` which publishes + `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when + building for libssh2 v1.8.2 (2019-05-25) or older. - ftpserver.pl correctly cleans up spawned server processes, - but forgets to wait for the shell used to spawn them. - This is barely noticeable during a normal testrun, - but causes process exhaustion and test failure - during a complete torture run of the FTP tests. + Make sure to compile the HMAC bits for a successful build. - Fixes #12018 - Closes #12020 + Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting + them into separate sources isn't practical. -Dan Fandrich (4 Oct 2023) + Fixes: + ``` + [...] + In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc + url_object.dir/Unity/unity_0_c.c:310: + ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s + truct HMAC_params' + 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { + | ^ + ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par + ams' + [...] + ``` -- github/labeler: improve labeler matches + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 -- test574: add a timeout to the test + Fixes #12273 + Closes #12332 - This one hangs occasionally, so this will speed up a test run and allow - logs to be seen when it does. +Daniel Stenberg (15 Nov 2023) - Closes #12025 +- duphandle: also free 'outcurl->cookies' in error path -- tests: propagate errors in libtests + Fixes memory-leak when OOM mid-function - Use the test macros to automatically propagate some errors, and check - and log others while running the tests. This can help in debugging - exactly why a test has failed. + Use plain free instead of safefree, since the entire struct is + freed below. -- tests: set --expect100-timeout to improve test reliability + Remove some free calls that is already freed in Curl_freeset() - On an overloaded server, the default 1 second timeout can go by without - the test server having a chance to respond with the expected headers, - causing tests to fail. Increase the 1 second timeout to 99 seconds so - this failure mode is no longer a problem on test 1129. Some other tests - already set a high value, but make them consistently 99 seconds so if - something goes wrong the test is stalled for less time. + Closes #12329 - Ref: #11328 +Viktor Szakats (15 Nov 2023) -- CI: ignore the "flaky" and "timing-dependent" test results in CMake +- config-win32: set `HAVE_SNPRINTF` for mingw-w64 - This was already done for automake builds but CMake builds were missed. - Test 1086 actually causes the test harness to crash with: + It's available in all mingw-w64 releases. We already pre-fill this + detection in CMake. - Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec - ts/curl/tests/ftpserver.pl line 527 + Closes #12325 - Rather than fix it now, this change leaves test 1086 entirely skipped on - those builds that show this problem. +- sasl: fix `-Wunused-function` compiler warning - Follow-up to 589dca761 + In builds with disabled auths. - Ref: #11865 + ``` + lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus + ed-function] + static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, + ^ + 1 warning generated. + ``` + Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st + ep:3:3822 -Viktor Szakats (4 Oct 2023) + Reviewed-by: Daniel Stenberg + Closes #12326 -- cmake: improve OpenLDAP builds +- build: picky warning updates - - cmake: detect OpenLDAP based on function `ldap_init_fd`. - autotools does this. autotools also publishes this detection result - in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source - doesn't use this value. (it might need to be remove-listed in - `scripts/cmp-config.pl` for future OpenLDAP test builds.) - This also deletes existing self-declaration method via the - CMake-specific `CURL_USE_OPENLDAP` configuration. + - cmake: sync some picky gcc warnings with autotools. + - cmake, autotools: add `-Wold-style-definition` for clang too. + - cmake: more precise version info for old clang options. + - cmake: use `IN LISTS` syntax in `foreach()`. - - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. - Like autotools does. This fixes a long list of these warnings: - ``` - /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i - s not defined, evaluates to 0 [-Wundef] - ``` + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Closes #12324 - - cmake: delete LDAP TODO comment no longer relevant. +Daniel Stenberg (15 Nov 2023) - Also: +- urldata: move cookielist from UserDefined to UrlState - - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature - detection functions. + 1. Because the value is not strictly set with a setopt option. - Ref: #11964 (effort to sync cmake detections with autotools) + 2. Because otherwise when duping a handle when all the set.* fields are + first copied and an error happens (think out of memory mid-function), + the function would easily free the list *before* it was deep-copied, + which could lead to a double-free. - Closes #12024 + Closes #12323 -- cmake: fix unity builds for more build combinations +Viktor Szakats (14 Nov 2023) - By using unique static function/variable names in source files - implementing these interfaces. +- autotools: avoid passing `LDFLAGS` twice to libcurl - - OpenLDAP combined with any SSH backend. + autotools passes `LDFLAGS` automatically linker commands. curl's + `lib/Makefile.am` customizes libcurl linker flags. In that + customization, it added `LDFLAGS` to the custom flags. This resulted in + passing `LDFLAGS` _twice_ to the `libtool` command. - - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. + Most of the time this is benign, but some `LDFLAGS` options can break + the build when passed twice. One such example is passing `.o` files, + e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for + MUSL builds. - Closes #12027 + Passing them twice resulted in duplicate symbol errors: + ``` + libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a + arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] + ld.lld-15: error: duplicate symbol: _start + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + [...] + clang: error: linker command failed with exit code 1 (use -v to see invocatio + n) + ``` -Daniel Stenberg (4 Oct 2023) + This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. + The patch was a works-for-me hack that ended up merged in curl: + https://sourceforge.net/p/curl/bugs/1217/#06ef + With the root cause remaining unclear. -- tests: remove leading spaces from some tags + Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring + `-L` first? This would be unusual and suggests a bug in either the + linker or in `libtool`. - The threee tags ``, `` and `` were frequently used - with a leading space that this removes. The reason this habbit is so - widespread in testcases is probably that they have been copy and pasted. + The curl build does pass the list of detected libs via its own + `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` + command line, but it's the job of `libtool` to ensure that even + a peculiar linker gets the options in the expected order. Also because + autotools passes `LDFLAGS` last, making it hardly possible to pass + anything after it. - Hence, fixing them all now might curb this practice from now on. + Perhaps in the 10 years since this issue, this already got a fix + upstream. - Closes #12028 + This patch deletes `LDFLAGS` from our customized libcurl options, + leaving a single copy of them as passed by autotools automatically. -Viktor Szakats (4 Oct 2023) + Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + Closes #12310 -- GHA: bump actions/checkout +- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` - Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 - Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 + To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and + curl tool (`CURL_LDFLAGS_BIN`). - Closes #12023 + This makes it possible to build libcurl and curl with a single + invocation with lib- and tool-specific custom linker flags. -- spelling: fix codespell 2.2.6 typos + Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, + controlling static/shared, incl. requesting a static curl tool (with + `-static-libtool-libs`) while building both shared and static libcurl. - Closes #12019 + curl-for-win uses the above and some more. -Daniel Stenberg (3 Oct 2023) + These options are already supported in `Makefile.mk`. CMake has built-in + variables for this. -- GHA: add workflow to compare configure vs cmake outputs + Closes #12312 - Uses scripts/cmp-config.pl two compare two curl_config.h files, - presumbly generated with configure and cmake. It displays the - differences and filters out a lot of known lines we ignore. +Jay Satiro (14 Nov 2023) - The script also shows the matches that were *not* used. Possibly - subjects for removal. +- tool_cb_hdr: add an additional parsing check - Closes #11964 + - Don't dereference the past-the-end element when parsing the server's + Content-disposition header. -- appveyor: enable test 571 + As 'p' is advanced it can point to the past-the-end element and prior + to this change 'p' could be dereferenced in that case. - Follow-up from 8a940fd55c175f7 / #12013 + Technically the past-the-end element is not out of bounds because dynbuf + (which manages the header line) automatically adds a null terminator to + every buffer and that is not included in the buffer length passed to + the header callback. - Closes #12017 + Closes https://github.com/curl/curl/pull/12320 -Viktor Szakats (3 Oct 2023) +Philip Heiduck (14 Nov 2023) -- build: alpha-sort source files for lib and src +- .cirrus.yml: freebsd 14 - Closes #12014 + ensure curl works on latest freebsd version -- cmake: delete old `HAVE_LDAP_URL_PARSE` logic + Closes #12053 - Left there by accident after adding proper detection for this. +Daniel Stenberg (13 Nov 2023) - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 +- easy: in duphandle, init the cookies for the new handle - Ref: #11964 (effort to sync cmake detections with autotools) + ... not the source handle. - Closes #12015 + Closes #12318 -Stefan Eissing (3 Oct 2023) +- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set -- tests: increase lib571 timeout from 3s to 30s + Previously it would unconditionally use the size, which is set to -1 + when strlen is requested. - - 3s is too short for our CI, making this test fail occasionally - - test usually experiences no delay run locally, so 30s wont hurt + Updated test 544 to verify. - Closes #12013 + Closes #12317 -Viktor Szakats (3 Oct 2023) +- RELEASE-NOTES: synced -- cmake: fix unity with Windows Unicode + TrackMemory +- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped - Found the root cause of the startup crash in unity builds with Unicode - and TrackMemory enabled at the same time. + Closes #12315 - We must make sure that the `memdebug.h` header doesn't apply to - `lib/curl_multibyte.c` (as even noted in a comment there.) In unity - builds all headers apply to all sources, including `curl_multibyte.c`. - This probably resulted in an infinite loop on startup. +- urldata: move hstslist from 'set' to 'state' - Exclude this source from unity compilation with TrackMemory enabled, - in both libcurl and curl tool. Enable unity mode for a debug Unicode - CI job to keep it tested. Also delete the earlier workaround that - fully disabled unity for affected builds. + To make it work properly with curl_easy_duphandle(). This, because + duphandle duplicates the entire 'UserDefined' struct by plain copy while + 'hstslist' is a linked curl_list of file names. This would lead to a + double-free when the second of the two involved easy handles were + closed. - Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 + Closes #12315 - Closes #11928 +- test1900: verify duphandle with HSTS using multiple files -- cmake: disable unity mode with Windows Unicode + TrackMemory + Closes #12315 - "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, - aka `-DCURLDEBUG`). +Goro FUJI (13 Nov 2023) - There is an issue with memory tracking and Unicode when built in "unity" - mode, which results in the curl tool crashing right on startup, even - without any command-line option. Interestingly this doesn't happen under - WINE (at least on the system I tested this on), but consistenly happens - on real Windows machines. Crash is 0xC0000374 heap corruption. Both - shared and static curl executables are affected. +- http: allow longer HTTP/2 request method names - This limitation probably won't hit too many people, but it remains - a TODO to find and fix the root cause and drop this workaround. + - Increase the maximum request method name length from 11 to 23. - Example builds and runs: - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd - 7iwj#L313 (static) - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby - qu9c#L317 (shared) + For HTTP/1.1 and earlier there's not a specific limit in libcurl for + method length except that it is limited by the initial HTTP request + limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same + and there was no specific limit. - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 + According to Internet Assigned Numbers Authority (IANA) the longest + registered method is UPDATEREDIRECTREF which is 17 characters. - Ref: #11928 - Closes #12005 + Also there are unregistered methods used by some companies that are + longer than 11 characters. -- cmake: tidy-up `NOT_NEED_LBER_H` detection + The limit was originally added by 61f52a97 but not used until fc2f1e54. - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 + Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml -- appveyor: rewrite batch in PowerShell + CI improvements + Closes https://github.com/curl/curl/pull/12311 - 1. Rewrite in PowerShell: +Jay Satiro (12 Nov 2023) - - rewrite MS-DOS batch build script in PowerShell. - - move some bash operations into native PowerShell. - - fixups for PowerShell insisting on failure when a command outputs - something to stderr. - - fix to actually run `curl -V` after every build. - (and exclude ARM64 builds.) - - also say why we skipped `curl -V` if we had to skip. - - fix CMake warnings about unused configuration variables, by adapting - these dynamically for build cases. - - dedupe OpenSSL path into a variable. - - disable `test1451` failing with a warning anyway due to missing python - impacket. (after trying and failing to install impacket) - PowerShell promotes these warnings to errors by PowerShell. We can also - suppress they wholesale if they start causing issues in the future, - like we already to with `autoreconf` and `./configure`. +- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does - PowerShell is better than MS-DOS batches, so the hope is this makes it - easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't - supported inline by AppVeyor on Windows build machines, but we are okay - to keep it in an external script, so it's also an option. + - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB + and CURLOPT_PROXY_CAINFO_BLOB docs. - 2. CI improvements: + All the other _BLOB option docs already have the same explanation. - - enable tests for a "unity" build job. - - speed-up CI initialization by using shallow clones of the curl repo. - - speed-up CMake MSVC jobs with `TrackFileAccess=false`. - - enable parallelism in `VisualStudioSolution` builds. - - display CMake version before builds. - - always show the CPU in job names. - - tell which jobs are build-only in job names. - - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. - - add `config.log` (autotools) to dumped logs (need to enable manually). + Closes https://github.com/curl/curl/pull/12277 - 3. Style: +Viktor Szakats (11 Nov 2023) - - use single-quotes in YAML like we do in other CI YAML files. - It also allows to drop quoting characters and lighter to write/read. - (keep double quotes for PowerShell strings needing expansion.) +- tidy-up: dedupe Windows system libs in cmake - Closes #11999 + Reviewed-by: Daniel Stenberg + Closes #12307 -- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows +Junho Choi (11 Nov 2023) - - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. - Before this patch we set it based it on the presence of `stricmp`, - which correctly enabled it on e.g. Windows, but was inaccurate for - other platforms. +- ci: test with latest quiche release (0.19.0) - - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and - LDAPS is not explicitly disabled. This mimics autotools behaviour. - Previously we set it only for Windows LDAP. After this fix, LDAPS is - correctly enabled in default macOS builds. + Closes #12180 - - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) - to avoid deprecation warnings for LDAP API. +- quiche: use quiche_conn_peer_transport_params() - - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. - This doesn't make much sense, but let's do it to sync behaviour with - autotools. + In recent quiche, transport parameter API is separated + with quiche_conn_peer_transport_params(). + (https://github.com/cloudflare/quiche/pull/1575) + It breaks with bulding with latest(post 0.18.0) quiche. - - fix benign typo in variable name. + Closes #12180 - Ref: #11964 (effort to sync cmake detections with autotools) +Daniel Stenberg (11 Nov 2023) - Closes #12006 +- Makefile: generate the VC 14.20 project files at dist-time -- autotools: restore `HAVE_IOCTL_*` detections + Follow-up to 28287092cc5a6d6ef8 (#12282) - This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in - 4d73854462f30948acab12984b611e9e33ee41e6 and - c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the - `HAVE_IOCTL` result it generated was unused in the source. But, - I did miss the fact that this had two dependent checks: - `CURL_CHECK_FUNC_IOCTL_FIONBIO`, - `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: - `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. + Closes #12290 - Regression from 4d73854462f30948acab12984b611e9e33ee41e6 +Sam James (11 Nov 2023) - Ref: #11964 (effort to sync cmake detections with autotools) +- misc: fix -Walloc-size warnings - Closes #12008 + GCC 14 introduces a new -Walloc-size included in -Wextra which gives: -Daniel Stenberg (2 Oct 2023) + ``` + src/tool_operate.c: In function ‘add_per_transfer’: + src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo + r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] + 213 | p = calloc(sizeof(struct per_transfer), 1); + | ^ + src/var.c: In function ‘addvariable’: + src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type + struct var’ with size ‘32’ [-Walloc-size] + 361 | p = calloc(sizeof(struct var), 1); + | ^ + ``` -- RELEASE-PROCEDURE.md: updated coming release dates + The calloc prototype is: + ``` + void *calloc(size_t nmemb, size_t size); + ``` -- RELEASE-NOTES: synced + So, just swap the number of members and size arguments to match the + prototype, as we're initialising 1 struct of size `sizeof(struct + ...)`. GCC then sees we're not doing anything wrong. -Viktor Szakats (1 Oct 2023) + Closes #12292 -- cmake: pre-cache `HAVE_POLL_FINE` on Windows +Mark Gaiser (11 Nov 2023) - Windows doesn't support `poll()`, so we can safely skip checking for - fine poll. +- IPFS: bugfixes - Closes #12003 + - Fixed endianness bug in gateway file parsing + - Use IPFS_PATH in tests where IPFS_DATA was used + - Fixed typos from traling -> trailing + - Fixed broken link in IPFS.md -- gha: bump actions to latest versions + Follow-up to 859e88f6533f9e - - actions@checkout@v4 (from v3 and v2) + Reported-by: Michael Kaufmann + Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 + Closes #12305 - - fsfe/reuse-action@v2 (from v1) +Daniel Stenberg (11 Nov 2023) - Closes #12000 +- VULN-DISCLOSURE-POLIC: remove broken link to hackerone -Stefan Eissing (30 Sep 2023) + It should ideally soon not be done from hackerone anyway -- h2: testcase and fix for pausing h2 streams + Closes #12308 - - refs #11982 where it was noted that paused transfers may - close successfully without delivering the complete data - - made sample poc into tests/http/client/h2-pausing.c and - added test_02_27 to reproduce +Andrew Kurushin (11 Nov 2023) - Closes #11989 - Fixes #11982 - Reported-by: Harry Sintonen +- schannel: add CA cache support for files and memory blobs -Viktor Szakats (30 Sep 2023) + - Support CA bundle and blob caching. -- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value + Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. - Before this patch CMake builds accepted any value and it was used at - runtime as-is. This patch make sure that the selected default backend - is also enabled in the build. It also enforces a full lowercase value. + Closes https://github.com/curl/curl/pull/12261 - This improves reproducibility and brings CMake in sync with autotools - which already worked like described above. +Daniel Stenberg (10 Nov 2023) - Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 +- RELEASE-NOTES: synced - Closes #11998 +Charlie C (10 Nov 2023) -- autotools: adjust `CURL_CA_PATH` value to CMake +- cmake: option to disable install & drop `curlu` target when unused - autotools was using the same value as CMake, but with an ending - slash. Delete the ending slash to match configurations. + This patch makes the following changes: + - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. + - Removes the target `curlu` when the option `BUILD_TESTING` is set to + `OFF` - to prevent it from being loaded in Visual Studio. - Ref: #11964 (effort to sync cmake detections with autotools) + Closes #12287 - Closes #11997 +Kai Pastor (10 Nov 2023) -- cmake: detect `sys/wait.h` and `netinet/udp.h` +- cmake: fix multiple include of CURL package - Ref: #11964 (effort to sync cmake detections with autotools) + Fixes errors on second `find_package(CURL)`. This is a frequent case + with transitive dependencies: + ``` + CMake Error at ...: + add_library cannot create ALIAS target "CURL::libcurl" because another + target with the same name already exists. + ``` - Closes #11996 + Test to reproduce: + ```cmake + cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher -Daniel Stenberg (30 Sep 2023) + project(curl) -- lib: provide and use Curl_hexencode + set(CURL_DIR "example/lib/cmake/CURL/") + find_package(CURL CONFIG REQUIRED) + find_package(CURL CONFIG REQUIRED) # fails - Generates a lower case ASCII hex output from a binary input. + add_executable(main main.c) + target_link_libraries(main CURL::libcurl) + ``` - Closes #11990 + Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes + Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html + Ref: #12300 + Assisted-by: Harry Mallon + Closes #11913 -- configure: check for the capath by default +Viktor Szakats (8 Nov 2023) - ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS - L +- tidy-up: use `OPENSSL_VERSION_NUMBER` - cmake: synced + Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. + Before this patch some places used `OPENSSL_VERSION_MAJOR`. - Assisted-by: Viktor Szakats - Closes #11987 + Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't + define any version number in these implementations: BoringSSL, AWS-LC, + LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to + `opensslv.h`. This wasn't causing a deeper problem because the code is + looking for v3, which is only provided by OpenSSL/quictls as of now. -- wolfssl: ignore errors in CA path + According to https://github.com/openssl/openssl/issues/17517, the macro + `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. - The default wolfSSL_CTX_load_verify_locations() function is quite picky - with the certificates it loads and will for example return error if just - one of the certs has expired. + Reviewed-by: Marcel Raad + Closes #12298 - With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it - behaves more similar to what OpenSSL does by default. +Daniel Stenberg (8 Nov 2023) - Even the set of default certs on my Debian unstable has several expired - ones. +- resolve.d: drop a multi use-sentence - Assisted-by: Juliusz Sosinowicz - Assisted-by: Michael Osipov + Since the `multi:` keyword adds that message. - Closes #11987 + Reported-by: 積丹尼 Dan Jacobson + Fixes https://github.com/curl/curl/discussions/12294 + Closes #12295 -- create-dirs.d: clarify it also uses --output-dirs +- content_encoding: make Curl_all_content_encodings allocless - Reported-by: Robert Simpson - Fixes #11991 - Closes #11995 + - Fixes a memory leak pointed out by Coverity + - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? + id=63947 + - Avoids unncessary allocations -Viktor Szakats (30 Sep 2023) + Follow-up ad051e1cbec68b2456a22661b -- appveyor: fix yamlint issues, indent + Closes #12289 - Also: - - use double quotes in all batch if statements. +Michael Kaufmann (7 Nov 2023) - Closes #11994 +- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 -- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` + Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), + avoid it and use "http/1.1" instead. - Based on existing autotools logic. + This reverts commit df856cb5c9 (#10183). - Ref: #11964 (effort to sync cmake detections with autotools) + Fixes #12259 + Closes #12285 - Closes #11981 +Daniel Stenberg (7 Nov 2023) -- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` +- Makefile.am: drop vc10, vc11 and vc12 projects from dist - Based on existing autotools logic. + They are end of life products. Support for generating them remain in the + repo for a while but this change drops them from distribution. - autotools checks for old versions of the allowlisted target OSes and - disables this feature when seeing them. In CMake we assume we're running - on newer systems and enable regardless of OS version. + Closes #12288 - autotools always runs all 3 probes for non-fast-tracked systems and - enables this feature if any one of them was successful. To save - configuration time, CMake stops at the first successful check. +David Suter (7 Nov 2023) - OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD - system. I haven't double-checked if this is correct, but looks odd. +- projects: add VC14.20 project files - Ref: #11964 (effort to sync cmake detections with autotools) + Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. + OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why + this is missing. Updated the templates to produce a VC14.20 project. + Project opens in Visual Studio 2019 as expected. - Closes #11979 + Closes #12282 -- cmake: fix `HAVE_WRITABLE_ARGV` detection +Daniel Stenberg (7 Nov 2023) - Move detection before the creation of detection results in - `curl_config.h`. +- curl: move IPFS code into src/tool_ipfs.[ch] - Ref: #11964 (effort to sync cmake detections with autotools) + - convert ensure_trailing into ensure_trailing_slash + - strdup the URL string to own it proper + - use shorter variable names + - combine some expressions + - simplify error handling in ipfs_gateway() + - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached + - ipfs-gateway.d polish and simplification + - shorten ipfs error message + make them "synthetic" - Closes #11978 + Closes #12281 -- appveyor: minor improvements +Viktor Szakats (6 Nov 2023) - - run `curl -V` after builds to see if they run and with what features. - Except for one job where a CRT DLL is missing. And ARM64 which should - fail, but is silently not launched instead. +- build: delete support bits for obsolete Windows compilers - - copy libcurl DLL next to curl tool and tests binaries in shared mode. - This makes it possible to run the tests. (We don't run tests after - these builds yet.) + - Pelles C: Unclear status, failed to obtain a fresh copy a few months + ago. Possible website is HTTP-only. ~10 years ago I left this compiler + dealing with crashes and other issues with no response on the forum + for years. It has seen some activity in curl back in 2021. + - LCC: Last stable release in September 2002. + - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned + in 2006. + - Borland C++: We dropped Borland C++ support in 2018. + - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 + (or possibly 2008) as a minimum. - - list the DLLs and EXEs present after the builds. + Closes #12222 - - add `DEBUG` variable for CMake builds to allow disabling it, for - testing non-debug builds. (currently enabled for all) +- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` - - add commented lines that dump CMake configuration logs for debugging - build/auto-detection issues. + We use `stdint.h` unconditionally in all places except one. These uses + are imposed by external dependencies / features. nghttp2, quic, wolfSSL + and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that + any of these features make curl require a C99 compiler. (In case of + MSVC, this means Visual Studio 2010 or newer.) - - add gcc version to jobs where missing. + This patch changes the single use of `stdint.h` guarded by + `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using + `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used + anywhere else, allowing to delete this feature check as well. - - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 - to the build mix. + Closes #12275 - - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. +Daniel Stenberg (6 Nov 2023) - - delete a duplicate backslash. +- tool_operate: do not mix memory models - Closes #11976 + Make sure 'inputpath' only points to memory allocated by libcurl so that + curl_free works correctly. -- configure: replace adhoc domain with `localhost` in tests + Pointed out by Coverity - Reviewed-by: Daniel Stenberg - Closes #11988 + Follow-up to 859e88f6533f9e1f890 -- tidy-up: use more example domains + Closes #12280 - Also make use of the example TLD: - https://en.wikipedia.org/wiki/.example +Stefan Eissing (6 Nov 2023) - Reviewed-by: Daniel Stenberg - Closes #11992 +- lib: client writer, part 2, accounting + logging -Dan Fandrich (29 Sep 2023) + This PR has these changes: -- runtests: display the test status if tests appear hung + Renaming of unencode_* to cwriter, e.g. client writers + - documentation of sendf.h functions + - move max decode stack checks back to content_encoding.c + - define writer phase which was used as order before + - introduce phases for monitoring inbetween decode phases + - offering default implementations for init/write/close - It sometimes happens that a test hangs during a test run and never - returns. The test harness will wait indefinitely for the results and on - CI servers the CI job will eventually be killed after an hour or two. - At the end of a test run, if results haven't come in within a couple of - minutes, display the status of all test runners and what tests they're - running to help in debugging the problem. + Add type paramter to client writer's do_write() + - always pass all writes through the writer stack + - writers who only care about BODY data will pass other writes unchanged - This feature is really only kick in with parallel testing enabled, which - is fine because without parallel testing it's usually easy to tell what - test has hung. + add RAW and PROTOCOL client writers + - RAW used for Curl_debug() logging of CURLINFO_DATA_IN + - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and + Curl_pgrsSetDownloadCounter() + - remove all updates of data->req.bytecount and calls to + Curl_pgrsSetDownloadCounter() and Curl_debug() from other code + - adjust test457 expected output to no longer see the excess write - Closes #11980 + Closes #12184 -- github/labeler: remove workaround for labeler +Daniel Stenberg (6 Nov 2023) - This was added due to what seemed to be a bug regarding the sync-labels: - config option, but it looks like it wasn't necessary. +- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw - Follow-up to b2b0534e7 + Closes #12278 -Viktor Szakats (29 Sep 2023) +Viktor Szakats (6 Nov 2023) -- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] +- rand: fix build error with autotools + LibreSSL -Daniel Stenberg (29 Sep 2023) + autotools unexpectedly detects `arc4random` because it is also looking + into dependency libs. One dependency, LibreSSL, happens to publish an + `arc4random` function (via its shared lib before v3.7, also via static + lib as of v3.8.2). When trying to use this function in `lib/rand.c`, + its protoype is missing. To fix that, curl included a prototype, but + that used a C99 type without including `stdint.h`, causing: -- docs: replace made up domains with example.com + ``` + ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' + 37 | uint32_t arc4random(void); + | ^ + 1 error generated. + ``` - in FAQ and MANUAL.md + This patch improves this by dropping the local prototype and instead + limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide + their own random source anyway. - - example.com was made for this purpose. + The better fix would be to teach autotools to not link dependency libs + while detecting `arc4random`. - - reduces the risk that one of those domains suddenly start hosting - something nasty and we provide links to them + LibreSSL publishing a non-namespaced `arc4random` tracked here: + https://github.com/libressl/portable/issues/928 - Closes #11986 + Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 -Michael Osipov (29 Sep 2023) + Reviewed-by: Daniel Stenberg + Fixes #12257 + Closes #12274 -- acinclude.m4: Document proper system truststore on FreeBSD +Daniel Stenberg (5 Nov 2023) - The default system truststore on FreeBSD has been /etc/ssl/certs for many - years now. It is managed canonically through certctl(8) and contains hashed - symlinks for OpenSSL and other TLS providers. - The previous ones require security/ca_root_nss which might not be installed o - r - will not contain any custom CA certificates. +- RELEASE-NOTES: synced - Closes #11985 +- strdup: do Curl_strndup without strncpy -Daniel Stenberg (29 Sep 2023) + To avoid (false positive) gcc-13 compiler warnings. -- FAQ: How do I upgrade curl.exe in Windows? + Follow-up to 4855debd8a2c1cb - This is a growing question, better answer it here to get somewhere to - point users to. + Assisted-by: Jay Satiro + Reported-by: Viktor Szakats + Fixes #12258 - Closes #11984 +Enno Boland (5 Nov 2023) -Viktor Szakats (28 Sep 2023) +- HTTP: fix empty-body warning -- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC + This change fixes a compiler warning with gcc-12.2.0 when + `-DCURL_DISABLE_BEARER_AUTH=ON` is used. - `basename` is present in mingw-w64, missing from MSVC. Pre-cache - accordingly to make configure faster. + /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': + /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp + ty body in an 'else' statement [-Wempty-body] + 1147 | ; + | ^ - Notice that `basename` has a bug so we later disable it even with - mingw-w64: - https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li - b/curl_setup.h#L820-L825 + Closes #12262 - Closes #11974 +Daniel Stenberg (5 Nov 2023) -Daniel Stenberg (28 Sep 2023) +- openssl: identify the "quictls" backend correctly -- cmake: add missing checks + Since vanilla OpenSSL does not support the QUIC API I think it helps + users to identify the correct OpenSSL fork in version output. The best + (crude) way to do that right now seems to be to check if ngtcp2 support + is enabled. - - check for arc4random. To make rand.c use it accordingly. - - check for fcntl - - fix fseek detection - - add SIZEOF_CURL_SOCKET_T - - fix USE_UNIX_SOCKETS - - define HAVE_SNPRINTF to 1 - - check for fnmatch - - check for sched_yield - - remove HAVE_GETPPID duplicate from curl_config.h - - add HAVE_SENDMSG + Closes #12270 - Ref: #11964 +Mark Gaiser (5 Nov 2023) - Co-authored-by: Viktor Szakats - Closes #11973 +- curl: improved IPFS and IPNS URL support -- configure: remove unused checks + Previously just ipfs:// and ipns:// was supported, which is + too strict for some usecases. - - for sys/uio.h - - for fork - - for connect + This patch allows paths and query arguments to be used too. + Making this work according to normal http semantics: - Ref: #11964 + ipfs:///foo/bar?key=val + ipns:///foo/bar?key=val - Closes #11973 + The gateway url support is changed. + It now only supports gateways in the form of: -- lib: remove TIME_WITH_SYS_TIME + http:///foo/bar + http:// - It is not used in any code anywhere. + Query arguments here are explicitly not allowed and trigger an intended + malformed url error. - Ref: #11964 - Closes #11975 + There also was a crash when IPFS_PATH was set with a non trailing + forward slash. This has been fixed. -- docs: update curl man page references + Lastly, a load of test cases have been added to verify the above. - Detected by the manpage-syntax update + Reported-by: Steven Allen + Fixes #12148 + Closes #12152 - Closes #11963 +Harry Mallon (5 Nov 2023) -- manpage-syntax: verify curl man page references +- docs: KNOWN_BUGS cleanup - 1. References to curl symbols are now checked that they indeed exist as - man pages. This for \f references as well as the names referenced in the - SEE ALSO section. + * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. + Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 - Allowlist curl.1 since it is not always built in builds + * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. + Fixed in #11806 - 2. References to curl symbols that lack section now causes warning, since tha - t - will prevent them from getting linked properly + * Remove mention of aws-sigv4 query empty value problems - 3. Check for "bare" references to curl functions and warn, they should be - references + * Remove mention of aws-sigv4 missing amz-content-sha256 + Fixed in #9995 - Closes #11963 +- http_aws_sigv4: canonicalise valueless query params -- cmake: add check for suseconds_t + Fixes #8107 + Closes #12244 - And fix the HAVE_LONGLONG define +Michael Kaufmann (4 Nov 2023) - Ref: #11964 - Closes #11977 +- docs: preserve the modification date when copying the prebuilt man page -Viktor Szakats (28 Sep 2023) + The previously built man page "curl.1" must be copied with the original + modification date, otherwise the man page is never updated. -- tidy-up: whitespace fixes + This fixes a bug that has been introduced with commit 2568441cab. - Closes #11972 + Reviewed-by: Dan Fandrich + Reviewed-by: Daniel Stenberg -- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS + Closes #12199 - With new option `CURL_DISABLE_SRP=ON` to force-disable it. - To match existing option and detection logic in autotools. +Daniel Stenberg (4 Nov 2023) - Also: - - fix detecting GnuTLS. - We assume `nettle` as a GnuTLS dependency. - - add CMake GnuTLS CI job. - - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) - TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. - - fix compiler warning when building with GnuTLS and disabled TLS-SRP. - - fix comment typos, whitespace. +- docs: remove bold from some man page SYNOPSIS sections - Ref: #11964 + In the name of consistency - Closes #11967 + Closes #12267 -- tool: use our own stderr variable +- openssl: two multi pointer checks should probably rather be asserts - Earlier this year we changed our own stderr variable to use the standard - name `stderr` (to avoid bugs where someone is using `stderr` instead of - the curl-tool specific variable). This solution needed to override the - standard `stderr` symbol via the preprocessor. This in turn didn't play - well with unity builds and caused curl tool to crash or stay silent due - to an uninitialized stderr. This was a hard to find issue, fixed by - manually breaking out one file from the unity sources. + ... so add the asserts now and consider removing the dynamic checks in a + future. - To avoid two these two tricks, this patch implements a different - solution: Restore using our own local variable for our stderr output and - leave `stderr` as-is. To avoid using `stderr` by mistake, add a - `checksrc` rule (based on logic we already used in lib for `strerror`) - that detects any `stderr` use in `src` and points to using our own - variable instead: `tool_stderr`. + Ref: #12261 + Closes #12264 - Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e - Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db +boilingoden (4 Nov 2023) - Closes #11958 +- docs: add supported version for the json write-out -Loïc Yhuel (28 Sep 2023) + xref: https://curl.se/changes.html#7_70_0 -- connect: only start the happy eyeballs timer when needed + Closes #12266 - The timeout is only used when there is a second address family, for the - delayed eyeballer. +Viktor Szakats (3 Nov 2023) - Closes #11939 +- appveyor: make VS2008-built curl tool runnable -Daniel Stenberg (28 Sep 2023) + By linking the CRT statically. This avoids the error about missing + runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`. -- tool_operate: free 'gateway' correctly + Closes #12263 - Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. +Stefan Eissing (3 Nov 2023) - Also removed repeated wording in IPFS related error messages. +- url: proxy ssl connection reuse fix - Closes #11969 + - tunnel https proxy used for http: transfers does + no check if proxy-ssl configuration matches + - test cases added, test_10_12 fails on 8.4.0 -Stefan Eissing (28 Sep 2023) + Closes #12255 -- lib: move handling of `data->req.writer_stack` into Curl_client_write() +Jay Satiro (3 Nov 2023) - - move definitions from content_encoding.h to sendf.h - - move create/cleanup/add code into sendf.c - - installed content_encoding writers will always be called - on Curl_client_write(CLIENTWRITE_BODY) - - Curl_client_cleanup() frees writers and tempbuffers from - paused transfers, irregardless of protocol +- curl_sspi: support more revocation error names in error messages - Closes #11908 + - Add these revocation errors to sspi error list: + CRYPT_E_NO_REVOCATION_DLL, CRYPT_E_NO_REVOCATION_CHECK, + CRYPT_E_REVOCATION_OFFLINE and CRYPT_E_NOT_IN_REVOCATION_DATABASE. -Loïc Yhuel (28 Sep 2023) + Prior to this change those error codes were not matched to their macro + name and instead shown as "unknown error". -- multi: round the timeout up to prevent early wakeups + Before: - Curl_timediff rounds down to the millisecond, so curl_multi_perform can - be called too early, then we get a timeout of 0 and call it again. + schannel: next InitializeSecurityContext failed: + Unknown error (0x80092013) - The revocation function was + unable to check revocation because the revocation server was offline. - The code already handled the case of timeouts which expired less than - 1ms in the future. By rounding up, we make sure we will never ask the - platform to wake up too early. + After: - Closes #11938 + schannel: next InitializeSecurityContext failed: + CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was + unable to check revocation because the revocation server was offline. -Daniel Stenberg (28 Sep 2023) + Bug: https://github.com/curl/curl/issues/12239 + Reported-by: Niracler Li -- RELEASE-NOTES: spell out that IPFS is via gateway + Closes https://github.com/curl/curl/pull/12241 -- RELEASE-NOTES: synced +- strdup: don't allow Curl_strndup to read past a null terminator -- tool_operate: avoid strlen() -1 on zero length content from file + - Use malloc + strncpy instead of Curl_memdup to dupe the string before + null terminating it. - Follow-up to 65b563a96a226649ba12cb1e + Prior to this change if Curl_strndup was passed a length longer than + the allocated string then it could copy out of bounds. - Closes #11959 + This change is for posterity. Curl_strndup was added in the parent + commit and currently none of the calls to it pass a length that would + cause it to read past the allocated length of the input. -- tool_operate: fix memory mixups + Follow-up to d3b3ba35. - Switch to plain getenv() from curl_getenv() to avoid the allocation and - having to keep track of which free() or curl_free() that need to be - used. + Closes https://github.com/curl/curl/pull/12254 - Coverity found issues and a memory leak. +Daniel Stenberg (2 Nov 2023) - Follow-up to 65b563a96a226649ba12cb1e +- lib: add and use Curl_strndup() - Closes #11959 + The Curl_strndup() function is similar to memdup(), but copies 'n' bytes + then adds a terminating null byte ('\0'). -Viktor Szakats (27 Sep 2023) + Closes #12251 -- curl-functions.m4: fixup recent bad edits +- CURPOST_POSTFIELDS.3: add CURLOPT_COPYPOSTFIELDS in SEE ALSO - Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 +Stefan Eissing (2 Nov 2023) - Closes #11966 +- pytest: use lower count in repeat tests -Daniel Stenberg (27 Sep 2023) + - lower large iteration counts in some tests somewhat for + the same coverage with less duration -- curl-functions.m4: fix include line + Closes #12248 - This made the getaddrinfo detection fail, but we did not spot it in the - CI because it graciously falled back to using legacy functions instead! +Daniel Stenberg (2 Nov 2023) - Follow-up to 96c29900bcec (#11940) +- RELEASE-NOTES: synced - Closes #11965 +- docs: clarify that curl passes on input unfiltered -- inet_ntop: add typecast to silence Coverity + ... for several options. - CID 1024653: Integer handling issues (SIGN_EXTENSION) + Reported-by: Ophir Lojkine - Suspicious implicit sign extension: "src[i]" with type "unsigned char - const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to - type "int" (32 bits, signed), then sign-extended to type "unsigned long" - (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than - 0x7FFFFFFF, the upper bits of the result will all be 1. + Closes #12249 - 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); +- urlapi: when URL encoding the fragment, pass in the right length - The value will not be greater than 0x7FFFFFFF so this still cannot - happen. + A benign bug because it would only add an extra null terminator. - Also, switch to ints here instead of longs. The values stored are 16 bit - so at least no need to use 64 bit variables. Also, longs are 32 bit on - some platforms so this logic still needs to work with 32 bits. + Made lib1560 get a test that runs this code. - Closes #11960 + Closes #12250 -- docs: adapt SEE ALSO sections to new requirements +Stefan Eissing (2 Nov 2023) - To please manpage-syntax.pl used by test 1173 +- vtls: late clone of connection ssl config - Closes #11957 + - perform connection cache matching against `data->set.ssl.primary` + and proxy counterpart + - fully clone connection ssl config only when connection is used -- manpage-syntax.pl: verify SEE ALSO syntax + Closes #12237 - - Enforce a single reference per .BR line - - Skip the quotes around the section number for example (3) - - Insist on trailing commas on all lines except the last - - Error on comma on the last SEE ALSO entry +- msh3: error when built with CURL_DISABLE_SOCKETPAIR set - - List the entries alpha-sorted, not enforced just recommended + Reported-by: Gisle Vanem + Closes #12252 + Fixes #12213 - Closes #11957 +Daniel Stenberg (2 Nov 2023) -- connect: expire the timeout when trying next +- hsts: skip single-dot hostname - ... so that it gets called again immediately and can continue trying - addresses to connect to. Otherwise it might unnecessarily wait for a - while there. + Reported-by: Maksymilian Arciemowicz - Fixes #11920 - Reported-by: Loïc Yhuel - Closes #11935 + Closes #12247 -- http: remove wrong comment for http_should_fail +- vtls: fix build without proxy - Reported-by: Christian Schmitz - Ref: #11936 - Closes #11941 + Follow-up to bf0e278a3c54bc7fee7360da17c -Dan Fandrich (26 Sep 2023) + closes #12243 -- tool_setopt: remove unused function tool_setopt_flags +- docs/example/keepalive.c: show TCP keep-alive options - This function is identical to tool_setopt_bitmask except that it treats - the argument as unsigned. + Closes #12242 - Closes #11943 +- lib1560: verify appending blank URL encoded query string -Viktor Szakats (26 Sep 2023) +- urlapi: skip appending NULL pointer query -- cmake: add feature checks for `memrchr` and `getifaddrs` + Reported-by: kirbyn17 on hackerone - - `HAVE_MEMRCHR` for `memrchr`. - - `HAVE_GETIFADDRS` for `getifaddrs`. - This was present in `lib/curl_config.h.cmake` but missed the detection - logic. + Closes #12240 - To match existing autotools feature checks. +- lib1560: verify setting host to "" with and without URL encode - Closes #11954 +- urlapi: avoid null deref if setting blank host to url encode -- cmake: move global headers to specific checks + Reported-by: kirbyn17 on hackerone - Before this patch we added standard headers unconditionally to the - global list of headers used for feature checks. This is unnecessary - and also doesn't help CMake 'Generate' performance. This patch moves - these headers to each feature check where they are actually needed. - Stop using `stddef.h`, as it seems unnecessary. + Closes #12240 - I've used autotools' `m4/curl-functions.m4` to figure out these - dependencies. +- dynbuf: assert for NULL pointer inputs - Also delete checking for the C89 standard header `time.h`, that I - missed in the earlier commit. + Help us catch more mistakes. - Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + Closes #12238 - Closes #11951 +- HTTP3: ngtcp2 builds are no longer experimental -- src/mkhelp: make generated code pass `checksrc` + The other HTTP/3 backends are still experimental. - Closes #11955 + Closes #12235 -- tests: show which curl tool `runtests.pl` is using +Stefan Eissing (31 Oct 2023) - To help debugging when there is issue finding or running it. +- vtls: cleanup SSL config management - Closes #11953 + - remove `Curl_ssl_get_config()`, no longer needed -- CI/azure: make `MAKEFLAGS` global to parallelize all jobs + Closes #12204 - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi - th -j3) +Daniel Stenberg (31 Oct 2023) - Closes #11952 +- libcurl-thread.3: simplify the TLS section -- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 + All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August + 2016. - Also delete an accidental variable reference. + Closes #12233 - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 +- configure: better --disable-http - Closes #11945 + - disable HTTPS-proxy as well, since it can't work without HTTP -Daniel Stenberg (26 Sep 2023) + - curl_setup: when HTTP is disabled, also disable all features that are + HTTP-only -- docs: add see also curl_multi_get_handles to some man pages + - version: HTTPS-proxy only exists if HTTP support exists - Assisted-by: Jay Satiro + Closes #12223 - Closes #11942 +- http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine -Viktor Szakats (26 Sep 2023) + Finding a 'Content-Range:' in the response changed the handling. -- cmake: assume `_fseeki64` and no `fseeko` on Windows + Add test case 1475 to verify -C - with 416 and Content-Range: header, + which is almost exactly like test 194 which instead uses a fixed -C + offset. Adjusted test 194 to also be considered fine. - `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and - at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. + Fixes #10521 + Reported-by: Smackd0wn + Fixes #12174 + Reported-by: Anubhav Rai + Closes #12176 - `fseeko` is not present in any of these. +Stefan Eissing (30 Oct 2023) - (mingw-w64 1.0 also offers `fseeko64`.) +- GHA: fix checkout of quictls repository to use correct branch name - [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 + Follow-up to c868b0e30f10cd0ac7 - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 + Closes #12232 - Closes #11950 +Daniel Stenberg (30 Oct 2023) -- build: delete checks for C89 standard headers +- docs/example/localport.c: show off CURLOPT_LOCALPORT - Delete checks and guards for standard C89 headers and assume these are - available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, - `stddef.h`, `signal.h`. + Closes #12230 - Some of these we already used unconditionally, some others we only used - for feature checks. +- docs/examples/interface.c: show CURLOPT_INTERFACE use - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i - n CMake) + Although super simple. - Closes #11940 + Closes #12229 -Stefan Eissing (26 Sep 2023) +Viktor Szakats (30 Oct 2023) -- multiif.h: remove Curl_multi_dump declaration +- build: fix compiler warning with auths disabled - Follow-up to d850eea2 which removed the Curl_multi_dump definition. + ``` + ./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' + [-Wunused-function] + static int is_valid_auth_separator(char ch) + ^ + 5 warnings generated. + ``` - Closes https://github.com/curl/curl/pull/11946 + Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 -Jay Satiro (26 Sep 2023) + Closes #12227 -- config-win32: define HAVE__FSEEKI64 +- build: require Windows XP or newer - Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that - calls _fseeki64 if it is available. + After this patch we assume availability of `getaddrinfo` and + `freeaddrinfo`, first introduced in Windows XP. Meaning curl + now requires building for Windows XP as a minimum. - Closes https://github.com/curl/curl/pull/11944 + TODO: assume these also in autotools. -- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER + Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 + Closes #12225 - - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place - even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. +- appveyor: bump one job to OpenSSL 3.1 (was 1.1.1) - The behavior is verified by test2048. + Use 3.1 with the modern runner image. - Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 - Reported-by: claudiusaiz@users.noreply.github.com + We still use 1.1.1 in 8 jobs. - Bug: https://github.com/curl/curl/discussions/11910 - Reported-by: Hakan Sunay Halil + 1.1.1 is EOL since 2023-09-11: + https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/ - Closes https://github.com/curl/curl/pull/11930 + Also: + - add missing SSL-backend to job descriptions. + - tidy up CPU in job descriptions. -Stefan Eissing (26 Sep 2023) + Closes #12226 -- openssl: improve ssl shutdown handling +Daniel Stenberg (30 Oct 2023) - - If SSL shutdown is not finished then make an additional call to - SSL_read to gather additional tracing. +- RELEASE-NOTES: synced - - Fix http2 and h2-proxy filters to forward do_close() calls to the next - filter. +- GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions - For example h2 and SSL shutdown before and after this change: + ngtcp2 1.0.1 + nghttp3 1.0.0 + nghttp2 1.58.0 + quictls 3.1.4+quic - Before: + also sync HTTP3.md with these changes - Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> - ssl_cf_destroy + Closes #12132 - After: +Kareem (29 Oct 2023) - Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> - ssl_cf_close +- wolfssl: add default case for wolfssl_connect_step1 switch - Note that currently the tracing does not show output on the connection - closure handle. Refer to discussion in #11878. + Closes #12218 - Ref: https://github.com/curl/curl/discussions/11878 +Jay Satiro (29 Oct 2023) - Closes https://github.com/curl/curl/pull/11858 +- curl_setup: disallow Windows IPv6 builds missing getaddrinfo -Loïc Yhuel (26 Sep 2023) + - On Windows if IPv6 is enabled but getaddrinfo is missing then #error + the build. -- multi: fix small timeouts + curl can be built with IPv6 support (ENABLE_IPV6) but without the + ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On + Windows this is highly unlikely and should be considered a bad build + configuration. - Since Curl_timediff rounds down to the millisecond, timeouts which - expire in less than 1ms are considered as outdated and removed from the - list. We can use Curl_timediff_us instead, big timeouts could saturate - but this is not an issue. + Such a bad configuration has already given us a bug that was hard to + diagnose. See #12134 and #12136 for discussion. - Closes #11937 + Ref: https://github.com/curl/curl/issues/12134 + Ref: https://github.com/curl/curl/pull/12136 -Viktor Szakats (25 Sep 2023) + Closes https://github.com/curl/curl/pull/12221 -- cmake: fix stderr initialization in unity builds +Nico Rieck (29 Oct 2023) - Before this patch, in certain build configurations the curl tool may - not have displayed anything (debug, macOS), or crashed at startup - (debug, Windows). +- openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 - Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db + - If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA + "CA" store after importing from root CA "ROOT" store. - Closes #11929 + This change allows curl to work in situations where a server does not + send all intermediate certs and they are present in the "CA" store (the + store with intermediate CAs). This is already allowed by the Schannel + backend. -- cmake: fix missing `zlib.h` when compiling `libcurltool` + Also this change makes partial chain verification possible for those + certs since we allow partial chain verification by default for OpenSSL + (unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel + backend. - Came up while testing debug/testing build for Windows. I'm not sure why - it didn't come up in earlier tests with similar config. - `tool_hugehelp.c` might indeed require `zlib.h` and without linking - `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected - dependency headers to the compiler command. + Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs. - ``` - [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang - --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 - 6_64 - -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO - DE - -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib - -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li - ne-argument - -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static - -libgcc - -lucrt [...] -O3 -DNDEBUG -municode -MD - -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d - -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x - 64/src/tool_hugehelp.c - .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f - ile not found - 6 | #include - | ^~~~~~~~ - ``` + Fixes https://github.com/curl/curl/issues/12155 + Closes https://github.com/curl/curl/pull/12185 - Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 +Viktor Szakats (28 Oct 2023) - Closes #11927 +- Makefile.mk: fix `-rtmp` option for non-Windows [ci skip] -- cmake: fix duplicate symbols when linking tests +Daniel Stenberg (28 Oct 2023) - The linker resolves this automatically in non-unity builds. In unity - builds the linker cannot drop a single object with the duplicates, - resulting in these errors. The root issue is that we started including - certain objects both via both libcurlu and libcurltool libs. +- asyn-ares: handle no connection in the addrinfo callback - Regression from 39e7c22bb459c2e818f079984989a26a09741860 + To avoid crashing. - Windows errors: - ``` - [ 3%] Linking C executable unit1303.exe - [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul - tibyte.c.obj - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch - ar': - C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve - rt_UTF8_to_wchar' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:44: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT - F8': - C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve - rt_wchar_to_UTF8' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:66: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': - C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 - _open' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:92: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': - C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 - 2_fopen' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:120: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': - [...] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 - aa4ehny5q#L247 + Follow-up from 56a4db2 + Closes #12219 + +Jay Satiro (28 Oct 2023) + +- hostip6: fix DEBUG_ADDRINFO builds + + - Removed unused and incorrect parameter from dump_addrinfo(). + + Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442 + Reported-by: Gisle Vanem + + Closes https://github.com/curl/curl/pull/12212 + +Viktor Szakats (28 Oct 2023) + +- Makefile.mk: restore `_mingw.h` for default `_WIN32_WINNT` + + In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support. + Turns out `_mingw.h` had the side-effect of setting a default + `_WIN32_WINNT` value expected by `lib/config-win32.h` to enable + `getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused + disabling support for this unless specifying the value manually. + + Restore this header and update its comment to tell why we continue + to need it. + + This triggered a regression in official Windows curl builds starting + with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.) + + Regression from 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 + + Reported-by: zhengqwe on github + Helped-by: Nico Rieck + Fixes #12134 + Fixes #12136 + Closes #12217 + +- hostip: silence compiler warning `-Wparentheses-equality` + + Seen with LLVM 17. - macOS errors: - ``` - [ 56%] Linking C executable unit1302 - duplicate symbol '_curlx_sotouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_sitouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_uztosz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - [...] - ``` - with config: ``` - -DCMAKE_UNITY_BUILD=ON \ - -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ - -DBUILD_SHARED_LIBS=ON \ - -DBUILD_STATIC_LIBS=OFF + hostip.c:1336:22: warning: equality comparison with extraneous parentheses [- + Wparentheses-equality] + 1336 | (a->ai_family == PF_INET)) { + | ~~~~~~~~~~~~~^~~~~~~~~~ + hostip.c:1336:22: note: remove extraneous parentheses around the comparison t + o silence this warning + 1336 | (a->ai_family == PF_INET)) { + | ~ ^ ~ + hostip.c:1336:22: note: use '=' to turn this equality comparison into an assi + gnment + 1336 | (a->ai_family == PF_INET)) { + | ^~ + | = + 1 warning generated. ``` - Closes #11926 + Follow-up to b651aba0962bb31353f55de4dc35f745952a1b10 #12145 -- cmake: lib `CURL_STATICLIB` fixes (Windows) + Reviewed-by: Daniel Stenberg + Closes #12215 - - always define `CURL_STATICLIB` when building libcurl for Windows. +Stefan Eissing (27 Oct 2023) - This disables `__declspec(dllexport)` for exported libcurl symbols. - In normal mode (hide symbols) these exported symbols are specified - via `libcurl.def`. When not hiding symbols, all symbols are exported - by default. +- doh: use PIPEWAIT when HTTP/2 is attempted - Regression from 1199308dbc902c52be67fc805c72dd2582520d30 + Closes #12214 - Fixes #11844 +Daniel Stenberg (27 Oct 2023) - - fix to omit `libcurl.def` when not hiding private symbols. +- setopt: remove outdated cookie comment - Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 + Closes #12206 - - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also - omitting `libcurl.def` in this case, and exporting all symbols - instead. This ensures that a shared curl tool can access all debug - functions which are not normally exported from libcurl DLL. +Stefan Eissing (27 Oct 2023) - - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" - target. +- cfilter: provide call to tell connection to forget a socket - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 + - fixed libssh.c workaround for a socket being closed by + the library + - eliminate the terrible hack in cf-socket.c to guess when + this happened and try not closing the socket again. + - fixes race in eyeballing when socket could have failed to + be closed for a discarded connect attempt - - delete duplicate `BUILDING_LIBCURL` definitions. + Closes #12207 - - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. +- url: protocol handler lookup tidy-up - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 + - rename lookup to what it does + - use ARRAYSIZE instead of NULL check for end + - offer alternate lookup for 0-terminated strings - Closes #11914 + Closes #12216 -Daniel Stenberg (25 Sep 2023) +Viktor Szakats (27 Oct 2023) + +- build: variadic macro tidy-ups + + - delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks. + (both autotools and CMake.) + - delete duplicate `NULL` check in `Curl_trc_cf_infof()`. + - fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds. + ``` + ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parame + ter] + static void nosigpipe(struct Curl_easy *data, + ^ + ``` + - fix `#ifdef` comments in `lib/curl_trc.{c,h}`. + - fix indentation in some `infof()` calls. + + Follow-up to dac293cfb7026b1ca4175d88b80f1432d3d3c684 #12167 + + Cherry-picked from #12105 + Closes #12210 + +- cmake: speed up threads setup for Windows + + Win32 threads are always available. We enabled them unconditionally + (with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection + logic has this condition hard-coded for Windows as well (since at least + 2007). + + Instead of doing all the work of detecting pthread combinations on + Windows, then discarding those results, skip these efforts and assume + built-in thread support when building for Windows. + + This saves 1-3 slow CMake configuration steps. + + Reviewed-by: Daniel Stenberg + Closes #12202 + +- cmake: speed up zstd detection + + Before this patch we detected the presence of a specific zstd API to + see if we can use the library. zstd published that API in its first + stable release: v1.0.0 (2016-08-31). + + Replace that method by detecting the zstd library version instead and + accepting if it's v1.0.0 or newer. Also display this detected version + and display a warning if the zstd found is unfit for curl. + + We use the same version detection method as zstd itself, via its public + C header. + + This deviates from autotools which keeps using the slow method of + looking for the API by building a test program. The outcome is the same + as long as zstd keeps offering this API. + + Ref: https://github.com/facebook/zstd/commit/5a0c8e24395079f8e8cdc90aa1659cd5 + ab1b7427 (2016-08-12, committed) + Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first + released) + Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0 + + Reviewed-by: Daniel Stenberg + Closes #12200 + +Daniel Stenberg (26 Oct 2023) + +- openssl: fix infof() to avoid compiler warning for %s with null + + vtls/openssl.c: In function ‘ossl_connect_step2’: + ../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror + =format-overflow=] + 120 | Curl_infof(data, __VA_ARGS__); } while(0) + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’ + 4008 | infof(data, "SSL connection using %s / %s / %s / %s", + | ^~~~~ + vtls/openssl.c:4008:49: note: format string is defined here + 4008 | infof(data, "SSL connection using %s / %s / %s / %s", + | ^~ + + Follow-up to b6e6d4ff8f253c8b8055bab + Closes #12196 + +Stefan Eissing (26 Oct 2023) + +- lib: apache style infof and trace macros/functions + + - test for a simplified C99 variadic check + - args to infof() in --disable-verbose are no longer disregarded but + must compile. + + Closes #12167 + Fixes #12083 + Fixes #11880 + Fixes #11891 + +Daniel Stenberg (26 Oct 2023) - RELEASE-NOTES: synced -Dan Fandrich (25 Sep 2023) +Stefan Eissing (26 Oct 2023) -- tests: fix log directory path in IPFS tests +- urldata: move async resolver state from easy handle to connectdata - Hard-coding the log directory name fails with parallel tests. + - resolving is done for a connection, not for every transfer + - save create/dup/free of a cares channel for each transfer + - check values of setopt calls against a local channel if no + connection has been attached yet, when needed. - Follow-up to 65b563a96 + Closes #12198 - Ref: #8805 +Daniel Stenberg (26 Oct 2023) -Daniel Stenberg (25 Sep 2023) +- CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERRO + R -- curl_multi_get_handles: get easy handles from a multi handle + It returns CURLE_WRITE_ERROR. It was not previously stated clearly. - Closes #11750 + Reported-by: enWILLYado on github + Fixes #12201 + Closes #12203 -Stefan Eissing (25 Sep 2023) +Viktor Szakats (25 Oct 2023) -- http: h1/h2 proxy unification +- autotools: update references to deleted `crypt-auth` option - - use shared code for setting up the CONNECT request - when tunneling, used in HTTP/1.x and HTTP/2 proxying - - eliminate use of Curl_buffer_send() and other manipulations - of `data->req` or `data->state.ulbuf` + Delete leftovers of the `crypt-auth` `./configure` option and + add the new ones that replaced them. - Closes #11808 + Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 -Natanael Copa (25 Sep 2023) + Reviewed-by: Daniel Stenberg + Closes #12194 -- lib: use wrapper for curl_mime_data fseek callback +Stefan Eissing (25 Oct 2023) - fseek uses long offset which does not match with curl_off_t. This leads - to undefined behavior when calling the callback and caused failure on - arm 32 bit. +- lib: introduce struct easy_poll_set for poll information - Use a wrapper to solve this and use fseeko which uses off_t instead of - long. + Connection filter had a `get_select_socks()` method, inspired by the + various `getsocks` functions involved during the lifetime of a + transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), + return sockets to monitor and flag if this shall be done for POLLIN + and/or POLLOUT. - Thanks to the nice people at Libera IRC #musl for helping finding this - out. + Due to this design, sockets and flags could only be added, not + removed. This led to problems in filters like HTTP/2 where flow control + prohibits the sending of data until the peer increases the flow + window. The general transfer loop wants to write, adds POLLOUT, the + socket is writeable but no data can be written. - Fixes #11882 - Fixes #11900 - Closes #11918 + This leads to cpu busy loops. To prevent that, HTTP/2 did set the + `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes + further attempts. This works if only one such filter is involved. If a + HTTP/2 transfer goes through a HTTP/2 proxy, two filters are + setting/clearing this flag and may step on each other's toes. -- configure: sort AC_CHECK_FUNCS + Connection filters `get_select_socks()` is replaced by + `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps + up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` + flags. This struct is initialized in `multi_getsock()` by calling the + various `getsocks()` implementations based on transfer state, as before. - No functional changes. + After protocol handlers/transfer loop have set the sockets and flags + they want, the `easy_pollset` is *always* passed to the filters. Filters + "higher" in the chain are called first, starting at the first + not-yet-connection one. Each filter may add sockets and/or change + flags. When all flags are removed, the socket itself is removed from the + pollset. -Daniel Stenberg (25 Sep 2023) + Example: -- warnless: remove unused functions + * transfer wants to send, adds POLLOUT + * http/2 filter has a flow control block, removes POLLOUT and adds + POLLIN (it is waiting on a WINDOW_UPDATE from the server) + * TLS filter is connected and changes nothing + * h2-proxy filter also has a flow control block on its tunnel stream, + removes POLLOUT and adds POLLIN also. + * socket filter is connected and changes nothing + * The resulting pollset is then mixed together with all other transfers + and their pollsets, just as before. - Previously put there for use with the intel compiler + Use of `SEND_HOLD` is no longer necessary in the filters. - Closes #11932 + All filters are adapted for the changed method. The handling in + `multi.c` has been adjusted, but its state handling the the protocol + handlers' `getsocks` method are untouched. -- GHA/linux: run singleuse to detect single-use global functions + The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS + filters needed to be adjusted for the connecting handshake read/write + handling. - Use --unit for configure --enable-debug builds + No noticeable difference in performance was detected in local scorecard + runs. - Closes #11932 + Closes #11833 -- singleuse: add scan for use in other source codes +Daniel Stenberg (25 Oct 2023) - This should reduce false-positive to almost zero. Checks for presence in - unit tests if --unit is specified, which is intended for debug builds - where unit testing is enabled. +- tests/README: SOCKS tests are not using OpenSSH, it has its own server - Closes #11932 + Follow-up to 04fd67555cc -- multi: remove Curl_multi_dump + Closes #12195 - A debug-only function that is basically never used. Removed to ease the - use of the singleuse script to detect non-static functions not used - outside the file where it is defined. +Jacob Hoffman-Andrews (25 Oct 2023) - Closes #11931 +- tets: make test documentation more user-friendly -Viktor Szakats (24 Sep 2023) + Put the instructions to run tests right at the top of tests/README.md. -- tests: fix compiler warnings + Give instructions to read the runtests.1 man page for information + about flags. Delete redundant copy of the flags documentation in the + README. - Seen with llvm 17 on Windows x64. + Add a mention in README.md of the important parallelism flag, to make + test runs go much faster. - ``` - .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration - for non-static variable 'logdir' [-Wmissing-variable-declarations] - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration - for non-static variable 'loglockfile' [-Wmissing-variable-declarations] - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat - ion for non-static variable 'logdir' [-Wmissing-variable-declarations] - 43 | const char *logdir = "log"; - | ^ - .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl - e is not intended to be used outside of this translation unit - 43 | const char *logdir = "log"; - | ^ - .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator - here [-Wcomma] - 350 | ++d, ++s; - | ^ - .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa - rning - 350 | ++d, ++s; - | ^~~ - | (void)( ) - ``` + Move documentation of output line format into the runtests.1 man page, + and update it with missing flags. - ``` - .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; - | ~ ^ ~~~~~~~~~~~~~ - 1 warning generated. + Fix the order of two flags in the man page. - .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison - 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] - 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. + Closes #12193 - .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > - 2147483647 is always false [-Wtautological-type-limit-compare] - 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - ``` +Viktor Szakats (24 Oct 2023) - Seen on macOS Intel: - ``` - .../curl/tests/server/sws.c:440:64: warning: field precision should have type - 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] - msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" - , - ~~^~ - 1 warning generated. - ``` +- cmake: pre-fill rest of detection values for Windows - Closes #11925 + The goal of this patch is to avoid unnecessary feature detection work + when doing Windows builds with CMake. Do this by pre-filling well-known + detection results for Windows and specifically for mingw-w64 and MSVC + compilers. Also limit feature checks to platforms where the results are + actually used. Drop a few redundant ones. And some tidying up. -Jay Satiro (24 Sep 2023) + - pre-fill remaining detection values in Windows CMake builds. -- url: fix netrc info message + Based on actual detection results observed in CI runs, preceding + similar work over libssh2 and matching up values with + `lib/config-win32.h`. - - Fix netrc info message to use the generic ".netrc" filename if the - user did not specify a netrc location. + This brings down CMake configuration time from 58 to 14 seconds on the + same local machine. - - Update --netrc doc to add that recent versions of curl on Windows - prefer .netrc over _netrc. + On AppVeyor CI this translates to: + - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job): + https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecr + jpy7necb#L296 + https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2 + fe249uo8#L186 + - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job): + https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5iv + lcs7ub29?fullLog=true#L290 + https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjs + yc9kl13a?fullLog=true#L194 - Before: - * Couldn't find host google.com in the (nil) file; using defaults + The formula is about 1-3 seconds delay for each detection. Almost all + of these trigger a full compile-link cycle behind the scenes, slow + even today, both cross and native, mingw-w64 and apparently MSVC too. + Enabling .map files or other custom build features slows it down + further. (Similar is expected for autotools configure.) + + - stop detecting `idn2.h` if idn2 was deselected. + autotools does this. + + - stop detecting `idn2.h` if idn2 was not found. + This deviates from autotools. Source code requires both header and + lib, so this is still correct, but faster. + + - limit `ADDRESS_FAMILY` detection to Windows. + + - normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format. + + - pre-fill `HAVE_WIN32_WINNT`-dependent detection results. + Saving 4 (slow) feature-detections in most builds: `getaddrinfo`, + `freeaddrinfo`, `inet_ntop`, `inet_pton` + + - fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`, + `HAVE_GETTIMEOFDAY` for mingw-w64. + Luckily this do not change build results, as `WIN32` took + priority over `HAVE_GETTIMEOFDAY` with the current source + code. + + - limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and + `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows. + We're not using these in the source code for Windows. + + - reduce compiler warning noise in CMake internal logs: + - fix to include `winsock2.h` before `windows.h`. + Apply it to autotools test snippets too. + - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above. + - cleanup `CMake/CurlTests.c` to emit less warnings. - After: - * Couldn't find host google.com in the .netrc file; using defaults + - delete redundant `HAVE_MACRO_SIGSETJMP` feature check. + It was the same check as `HAVE_SIGSETJMP`. - Closes https://github.com/curl/curl/pull/11904 + - delete 'experimental' marking from `CURL_USE_OPENSSL`. -Dan Fandrich (23 Sep 2023) + - show CMake version via `CMakeLists.txt`. + Credit to the `zlib-ng` project for the idea: + https://github.com/zlib-ng/zlib-ng/blob/61e181c8ae93dbf56040336179c9954078b + d1399/CMakeLists.txt#L7 -- wolfssh: do cleanup in Curl_ssh_cleanup + - make `CMake/CurlTests.c` pass `checksrc`. - Closes: #11921 + - `CMake/WindowsCache.cmake` tidy-ups. -Daniel Stenberg (24 Sep 2023) + - replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`. -- tool_listhelp: regenerated + Closes #12044 - Polished the --ipfs-gateway description +Jay Satiro (24 Oct 2023) - Fixed the --trace-config description +- page-footer: clarify exit code 25 - The script also fixed some other small mistakes + - Clarify that curl tool exit code 25 means an upload failed to start. - Closes #11923 + Exit code 25 is equivalent to CURLE_UPLOAD_FAILED (25). Prior to this + change the documentation only mentioned the case of FTP STOR failing. -Viktor Szakats (23 Sep 2023) + Reported-by: Emanuele Torre -- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) + Ref: https://github.com/curl/curl/blob/curl-8_4_0/docs/libcurl/libcurl-errors + .3#L113-L115 - Also fix to export all symbols in Windows debug builds, making - `-debug-dyn` builds work with `-DCURL_STATICLIB` set. + Fixes https://github.com/curl/curl/issues/12189 + Closes https://github.com/curl/curl/pull/12190 - Ref: https://github.com/curl/curl/pull/11914 (same for CMake) +Daniel Stenberg (24 Oct 2023) - Closes #11924 +- scripts/cijobs.pl: adjust for appveyor -Daniel Stenberg (23 Sep 2023) + Follow-up to a1d73a6bb -- quic: set ciphers/curves the same way regular TLS does +Alex Bozarth (24 Oct 2023) - for OpenSSL/BoringSSL +- OpenSSL: Include SIG and KEM algorithms in verbose - Fixes #11796 - Reported-by: Karthikdasari0423 on github - Assisted-by: Jay Satiro - Closes #11836 + Currently the verbose output does not include which algorithms are used + for the signature and key exchange when using OpenSSL. Including the + algorithms used will enable better debugging when working on using new + algorithm implementations. Know what algorithms are used has become more + important with the fast growing research into new quantum-safe + algorithms. -- test457: verify --max-filesize with chunked encoding + This implementation includes a build time check for the OpenSSL version + to use a new function that will be included in OpenSSL 3.2 that was + introduced in openssl/openssl@6866824 -- lib: let the max filesize option stop too big transfers too + Based-on-patch-by: Martin Schmatz + Closes #12030 - Previously it would only stop them from getting started if the size is - known to be too big then. +Daniel Stenberg (23 Oct 2023) - Update the libcurl and curl docs accordingly. +- http2: provide an error callback and failf the message - Fixes #11810 - Reported-by: Elliot Killick - Assisted-by: Jay Satiro - Closes #11820 + Getting nghttp2's error message helps users understand what's going + on. For example when the connection is brought down due a forbidden + header is used - as that header is then not displayed by curl itself. -Viktor Szakats (23 Sep 2023) + Example: -- mingw: delete support for legacy mingw.org toolchain + curl: (92) Invalid HTTP header field was received: frame type: 1, + stream: 1, name: [upgrade], value: [h2,h2c] - Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: - https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ - Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. - It supported the x86 CPU only and used a old Windows API header and - implib set, often causing issues. It also misses most modern Windows - features, offering old versions of both binutils and gcc (no llvm/clang - support). It was last updated 2 years ago. + Ref: #12172 + Closes #12179 - curl now relies on toolchains based on the mingw-w64 project: - https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ - https://www.msys2.org/ https://github.com/msys2/msys2 - https://github.com/mstorsjo/llvm-mingw - (Also available via Linux and macOS package managers.) +Turiiya (23 Oct 2023) - Closes #11625 +- BINDINGS: add V binding -Mark Gaiser (23 Sep 2023) + Closes #12182 -- curl: add support for the IPFS protocols: +Daniel Stenberg (22 Oct 2023) - - ipfs:// - - ipns:// +- configure: check for the fseeko declaration too - This allows you tu use ipfs in curl like: - curl ipfs:// - and - curl ipns:// + ... and make the code require both symbol and declaration. - For more information consult the readme at: - https://curl.se/docs/ipfs.html + This is because for Android, the symbol is always present in the lib at + build-time even when not actually available in run-time. - Closes #8805 + Assisted-by: Viktor Szakats + Reported-by: 12932 on github + Fixes #12086 + Closes #12158 -Daniel Stenberg (23 Sep 2023) +Viktor Szakats (22 Oct 2023) -- bufq: remove Curl_bufq_skip_and_shift (unused) +- cmake: fix OpenSSL quic detection in quiche builds - Closes #11915 + An orphan call to `CheckQuicSupportInOpenSSL()` remained after a recent + update when checking QUIC for quiche. Move back QUIC detection to + a function and fixup callers to use that. Also make sure that quiche + gets QUIC from BoringSSL, because it doesn't support other forks at this + time. -- scripts/singleuse.pl: add curl_global_trace + Regression from dee310d54261f9a8416e87d50bccfe2cbe404949 #11555 -Viktor Szakats (22 Sep 2023) + Reported-by: Casey Bodley + Fixes #12160 + Closes #12162 -- cmake: fix unity symbol collisions in h2 builds +Daniel Stenberg (22 Oct 2023) - Regression from 331b89a319d0067fa1e6441719307cfef9c7960f +- RELEASE-NOTES: synced - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #11912 + bump to 8.5.0 for pending release -Daniel Stenberg (22 Sep 2023) +Dan Fandrich (21 Oct 2023) -- RELEASE-NOTES: synced +- test3103: add missing quotes around a test tag attribute -Dan Fandrich (21 Sep 2023) +Loïc Yhuel (21 Oct 2023) -- github/labeler: improve the match patterns +- tool: fix --capath when proxy support is disabled - This includes new rules for setting the appleOS and logging labels and - matches on some example files. Also, enable dot mode for wildcard - matches in the .github directory. + After 95e8515ca0, --capath always sets CURLOPT_PROXY_CAPATH, which fails + with CURLE_UNKNOWN_OPTION when proxy support is disabled. -Daniel Stenberg (21 Sep 2023) + Closes #12089 -- upload-file.d: describe the file name slash/backslash handling +Daniel Stenberg (21 Oct 2023) - Closes #11911 +- openldap: move the alloc of ldapconninfo to *connect() -Jakub Jelen (21 Sep 2023) + Fixes a minor memory leak on LDAP connection reuse. -- libssh: cap SFTP packet size sent + Doing the allocation already in *setup_connection() is wrong since that + connect struct might get discarded early when an existing connection is + reused instead. - Due to libssh limitations + Closes #12166 - Signed-off-by: Jakub Jelen +- openldap: set the callback argument in oldap_do - Closes #11804 + ... to make sure it has the current 'data' pointer and not a stale old + one. -Daniel Stenberg (21 Sep 2023) + Reported-by: Dan Fandrich + Closes #12166 -- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 +- gnutls: support CURLSSLOPT_NATIVE_CA - Closes #11905 + Remove the CURL_CA_FALLBACK logic. That build option was added to allow + primarily OpenSSL to use the default paths for loading the CA certs. For + GnuTLS it was instead made to load the "system certs", which is + different and not desirable. -- mailmap: unify Michael Osipov under a single email + The native CA store loading is now asked for with this option. -Ted Lyngmo (21 Sep 2023) + Follow-up to 7b55279d1d856 -- docs: use CURLSSLBACKEND_NONE + Co-authored-by: Jay Satiro - [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in - documentation and examples. + Closes #12137 - Signed-off-by: Ted Lyngmo +Stefan Eissing (21 Oct 2023) - Closes #11909 +- RTSP: improved RTP parser -Dan Fandrich (21 Sep 2023) + - fix HTTP header parsing to report incomplete + lines it buffers as consumed! + - re-implement the RTP parser for interleave RTP + messages for robustness. It is now keeping its + state at the connection + - RTSP protocol handler "readwrite" implementation + now tracks if the response is before/in/after + header parsing or "in" a bod by calling + "Curl_http_readwrite_headers()" itself. This + allows it to know when non-RTP bytes are "junk" + or HEADER or BODY. + - tested with #12035 and various small receive + sizes where current master fails -- github/labeler: give the sync-labels config item a default value + Closes #12052 - This shouldn't be necessary and is likely a bug with this beta version - of the labeller. +- http2: header conversion tightening - Also, fix the negative matches for the documentation label. + - fold the code to convert dynhds to the nghttp2 structs + into a dynhds internal method + - saves code duplication + - pacifies compiler analyzers - Follow-up to dd12b452a - Closes #11907 + Closes #12097 -- github/labeler: fix up more the labeler config format +Daniel Stenberg (21 Oct 2023) - The new version didn't like the workaround we had for a bug in the - previous labeler version, and it should no longer be needed. +- curl_ntlm_wb: fix elif typo - Follow-up to dd12b452a - Closes #11906 + Reported-by: Manfred Schwarb + Follow-up to d4314cdf65ae + Bug: https://github.com/curl/curl/commit/d4314cdf65aee295db627016934bd9eb621a + b077#r130551295 -- github/labeler: fix indenting to try to appease labeller +Dan Fandrich (20 Oct 2023) - Follow-up to dd12b452a +- test1683: remove commented-out check alternatives -Jay Satiro (21 Sep 2023) + Python precheck/postcheck alternatives were included but commented out. + Since these are not used and perl is guaranteed to be available to run + the perl versions anyway, the Python ones are removed. -- libssh2: fix error message on failed pubkey-from-file +Daniel Stenberg (20 Oct 2023) - - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error - message "SSH public key authentication failed: Reason unknown (-1)". +- hostip: show the list of IPs when resolving is done - When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a - generic error and therefore doesn't set an error message. AFAICT that is - not documented behavior. + Getting 'curl.se' today then gets this verbose output which might help + debugging connectivity related matters. + + * Host curl.se:80 was resolved. + * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, + 2a04:4e42:600::347, 2a04:4e42:800::347, 2a04:4e42:a00::347, + 2a04:4e42:c00::347, 2a04:4e42:e00::347 + * IPv4: 151.101.193.91, 151.101.1.91, 151.101.65.91, 151.101.129.91 - Prior to this change libcurl retrieved the last set error message which - would be from a previous function failing. That resulted in misleading - auth failed error messages in verbose mode. + Co-authored-by: Jay Satiro + Closes #12145 - Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 - Reported-by: consulion@users.noreply.github.com +rilysh (20 Oct 2023) - Closes https://github.com/curl/curl/pull/11881 +- docs: fix function typo in curl_easy_option_next.3 -Stefan Eissing (21 Sep 2023) + Closes #12170 -- pytest: exclude test_03_goaway in CI runs due to timing dependency +Daniel Stenberg (20 Oct 2023) - Closes #11860 +- vssh: remove the #ifdef for Curl_ssh_init, use empty macro -- lib: disambiguate Curl_client_write flag semantics + In the same style as other init calls - - use CLIENTWRITE_BODY *only* when data is actually body data - - add CLIENTWRITE_INFO for meta data that is *not* a HEADER - - debug assertions that BODY/INFO/HEADER is not used mixed - - move `data->set.include_header` check into Curl_client_write - so protocol handlers no longer have to care - - add special in FTP for `data->set.include_header` for historic, - backward compatible reasons - - move unpausing of client writes from easy.c to sendf.c, so that - code is in one place and can forward flags correctly +- easy: remove duplicate wolfSSH init call - Closes #11885 + It is already done in Curl_ssh_init() where it belongs. -Patrick Monnerat (21 Sep 2023) + Closes #12168 -- tftpd: always use curl's own tftp.h +- socks: make SOCKS5 use the CURLOPT_IPRESOLVE choice - Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects - and reports a stringop-overread warning: + Fixes #11949 + Reported-by: Ammar Faizi + Closes #12163 - tftpd.c: In function ‘write_behind.isra’: - tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f - rom a region of size 0 [-Wstringop-overread] - 485 | return write(test->ofile, writebuf, count); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - In file included from tftpd.c:71: - /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 - 58 | char tu_data[0]; /* data or error stri - ng */ - | ^~~~~~~ +- urldata: move the 'internal' boolean to the state struct - This occurs because writebuf points to this field and the latter - cannot be considered as being of dynamic length because it is not - the last field in the structure. Thus it is bound to its declared - size. + ... where all the other state bits for the easy handles live. - This commit always uses curl's own version of tftp.h where the - target field is last in its structure, effectively avoiding the - warning. + Closes #12165 - As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for - arpa/tftp.h are removed. +- url: don't touch the multi handle when closing internal handles - Closes #11897 + Reported-by: Maksymilian Arciemowicz + Closes #12165 -Dan Fandrich (20 Sep 2023) +Faraz Fallahi (19 Oct 2023) -- test1474: make precheck more robust on non-Solaris systems +- getenv: PlayStation doesn't have getenv() - If uname -r returns something odd, perl could return an error code and - the test would be erroneously skipped. The qx// syntax avoid this. + Closes #12140 - Followup to 08f9b2148 +Daniel Stenberg (19 Oct 2023) -- github/labeler: switch to the 5 beta version +- transfer: only reset the FTP wildcard engine in CLEAR state - This version adds an important feature that will allow more PRs to be - labelled. Rather than being limited to labeling PRs with files that - match a single glob, it can now label them if multiple changed files - match any one of a number of globs. + To avoid the state machine to start over and redownload all the files + *again*. -Daniel Stenberg (20 Sep 2023) + Reported-by: lkordos on github + Regression from 843b3baa3e3cb228 (shipped in 8.1.0) + Bisect-by: Dan Fandrich + Fixes #11775 + Closes #12156 -- lib: enable hmac for digest as well +Stefan Eissing (19 Oct 2023) - Previously a build that disabled NTLM and aws-sigv4 would fail to build - since the hmac was disabled, but it is also needed for digest auth. +- GHA: move mod_h2 version in CI to v2.0.25 - Follow-up to e92edfbef64448ef + Closes #12157 - Fixes #11890 - Reported-by: Aleksander Mazur - Closes #11896 +Daniel Stenberg (19 Oct 2023) -- idn: if idn2_check_version returns NULL, return error +- ntlm_wb: use pipe instead of socketpair when possible - ... this avoids a NULL dereference for this unusual case. + Closes #12149 - Reported-by: s0urc3_ on hackerone - Closes #11898 +- RELEASE-NOTES: synced -- http: fix CURL_DISABLE_BEARER_AUTH breakage +- asyn-thread: use pipe instead of socketpair for IPC when available - When bearer auth was disabled, the if/else logic got wrong and caused - problems. + If pipe() is present. Less overhead. - Follow-up to e92edfbef64448ef461 - Fixes #11892 - Reported-by: Aleksander Mazur - Closes #11895 + Helped-by: Viktor Szakats + Closes #12146 -Michael Osipov (20 Sep 2023) +Dan Fandrich (17 Oct 2023) -- wolfssl: allow capath with CURLOPT_CAINFO_BLOB +- tests: Fix Windows test helper tool search & use it for handle64 - Remain consistent with OpenSSL. While CAfile is nulled as documented - with CURLOPT_CAINFO_BLOB, CApath remains intact. + The checkcmd() and checktestcmd() functions would not have worked on + Windows due to hard-coding the UNIX PATH separator character and not + adding .exe file extension. This meant that tools like stunnel, valgrind + and nghttpx would not have been found and used on Windows, and + inspection of previous test runs show none of those being found in pure + Windows CI builds. - Closes #11886 + With this fixed, they can be used to detect the handle64.exe program + before attempting to use it. When handle64.exe was called + unconditionally without it existing, it caused perl to abort the test + run with the error -- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c + The running command stopped because the preference variable + "ErrorActionPreference" or common parameter is set to Stop: + sh: handle64.exe: command not found - Closes #11886 + Closes #12115 -Dan Fandrich (19 Sep 2023) +Daniel Stenberg (17 Oct 2023) -- test1474: disable test on NetBSD, OpenBSD and Solaris 10 +- multi: use pipe instead of socketpair to *wakeup() - These kernels only send a fraction of the requested amount of the first - large block, invalidating the assumptions of the test and causing it to - fail. + If pipe() is present. Less overhead. - Assisted-by: Christian Weisgerber - Ref: https://curl.se/mail/lib-2023-09/0021.html - Closes #11888 + Closes #12142 -Ryan Schmidt (20 Sep 2023) +Jay Satiro (17 Oct 2023) -- cmake, configure: also link with CoreServices +- build: fix 'threadsafe' feature detection for older gcc - When linking with CoreFoundation, also link with CoreServices which is - apparently required to avoid an NSInvalidArgumentException in software - linking with libcurl on macOS Sonoma 14 and later. + - Add 'threadsafe' to the feature list shown during build if POSIX + threads are being used. - Fixes #11893 - Closes #11894 + This is a follow-up to 5adb6000 which added support for building a + thread-safe libcurl with older versions of gcc where atomic is not + available but pthread is. -Marc Hoersken (19 Sep 2023) + Reported-by: Dan Fandrich + Co-authored-by: Dan Fandrich -- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket + Fixes https://github.com/curl/curl/issues/12125 + Closes https://github.com/curl/curl/pull/12127 - These dependencies are now already included in the Docker image. +Daniel Stenberg (16 Oct 2023) - Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 - b41d15606e97f38cf312c1ce56 +- test729: verify socks4a with excessive proxy user name length - Closes #11889 +- socks: better buffer size checks for socks4a user and hostname -Daniel Stenberg (19 Sep 2023) + Also limit the proxy user name to 255 bytes, which is the same limit as + in SOCKS5. -- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files + Reported-by: sd0 on hackerone + Closes #12139 - Ref: #11883 - Reported-by: Michael Osipov - Closes #11884 +- curl.h: on FreeBSD include sys/param.h instead of osreldate.h -- RELEASE-NOTES: synced + Should things build on Playstation as well -- test3103: CURLOPT_COOKIELIST test + Fixes #12107 + Reported-by: Faraz Fallahi + Closes #12123 -- cookie: set ->running in cookie_init even if data is NULL +Marcin Rataj (16 Oct 2023) - This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) +- tool_operate: fix links in ipfs errors - Test 3103 verifies. + URL fragment links generated from headers in + https://curl.se/docs/ipfs.html are lowercase. - Fixes #11875 - Reported-by: wangp on github - Closes #11876 + Closes #12133 -- test498: total header size for all redirects is larger than accepted +Viktor Szakats (15 Oct 2023) -- http: use per-request counter to check too large headers +- cmake: replace `check_library_exists_concat()` - Not the counter that accumulates all headers over all redirects. + The idea of `check_library_exists_concat()` is that it detects an + optional component and adds it to the list of libs that we also use in + subsequent component checks. This caused problems when detecting + components with unnecessary dependencies that were not yet built. - Follow-up to 3ee79c1674fd6 + CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used + for component checks, which we already use in most cases. That left 4 + uses of `check_library_exists_concat()`. Only one of these actually + needed the 'concat' feature (ldap/lber). - Do a second check for 20 times the limit for the accumulated size for - all headers. + Delete this function and replace it with standard + `check_library_exists()` and manual management of our `CURL_LIBS` + list we use when linking build targets. And special logic to handle the + ldap/lber case. - Fixes #11871 - Reported-by: Joshix-1 on github - Closes #11872 + (We have a similar function for headers: `check_include_file_concat()`. + It works, but problematic for performance reasons and because it hides + the actual headers required in `check_symbol_exists()` calls.) -Jay Satiro (18 Sep 2023) + Ref: #11537 #11558 + Fixes #11285 + Fixes #11648 + Closes #12070 -- THANKS: add Eric Murphy +LoRd_MuldeR (15 Oct 2023) - He reported #11850 (quiche build error) but I forgot to add a - 'reported-by' entry in the fix 267e14f1. +- tool_cb_wrt: fix write output for very old Windows versions -Daniel Stenberg (18 Sep 2023) + - Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW() + function. -- h2-proxy: remove left-over mistake in drain_tunnel() + Apparently this parameter was *not* optional on older Windows versions. - Left-over from 331b89a319 + Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1. + So at some point between those two Microsoft changed the behavior. - Reported-by: 南宫雪珊 + Prior to this change, on those versions if parameter is NULL then the + function call fails with error ERROR_INVALID_ACCESS. - Closes https://github.com/curl/curl/pull/11877 + Regression since af3f4e41. -vvb2060 (18 Sep 2023) + Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299 -- lib: failf/infof compiler warnings + Fixes https://github.com/curl/curl/issues/12131 + Closes https://github.com/curl/curl/pull/12130 - Closes #11874 +Jay Satiro (15 Oct 2023) -Daniel Stenberg (17 Sep 2023) +- tool_urlglob: fix build for old gcc versions -- rand: fix 'alnum': array is too small to include a terminating null character + - Don't use __builtin_mul_overflow for GCC 4 and earlier. - It was that small on purpose, but this change now adds the null byte to - avoid the error. + The function was added in GCC 5. - Follow-up to 3aa3cc9b052353b1 + Ref: https://gcc.gnu.org/gcc-5/changes.html Reported-by: Dan Fandrich - Ref: #11838 - Closes #11870 - -Mathias Fuchs (16 Sep 2023) -- cmake: fix the help text to the static build option in CMakeLists.txt + Fixes https://github.com/curl/curl/issues/12124 + Closes https://github.com/curl/curl/pull/12128 - Closes #11843 +Carlos Henrique Lima Melara (14 Oct 2023) -John Haugabook (16 Sep 2023) +- docs/libcurl: fix three minor man page format mistakes -- MANUAL.md: change domain to example.com + Reported-by: Samuel Henrique - Closes #11866 + Closes https://github.com/curl/curl/pull/12126 -Daniel Stenberg (16 Sep 2023) +Jay Satiro (14 Oct 2023) -- doh: inherit DEBUGFUNCTION/DATA +- tests/server: add more SOCKS5 handshake error checking - When creating new transfers for doing DoH, they now inherit the debug - settings from the initiating transfer, so that the application can - redirect and handle the verbose output correctly even for the DoH - transfers. + - Add additional checking for missing and too-short SOCKS5 handshake + messages. - Reported-by: calvin2021y on github - Fixes #11864 - Closes #11869 + Prior to this change the SOCKS5 test server did not check that all parts + of the handshake were received successfully. If those parts were missing + or too short then the server would access uninitialized memory. -Dan Fandrich (16 Sep 2023) + This issue was discovered in CI job 'memory-sanitizer' test results. + Test 2055 was failing due to the SOCKS5 test server not running. It was + not running because either it crashed or memory sanitizer aborted it + during Test 728. Test 728 connects to the SOCKS5 test server on a + redirect but does not send any data on purpose. The test server was not + prepared for that. -- http_aws_sigv4: fix sorting with empty parts + Reported-by: Dan Fandrich - When comparing with an empty part, the non-empty one is always - considered greater-than. Previously, the two would be considered equal - which would randomly place empty parts amongst non-empty ones. This - showed as a test 439 failure on Solaris as it uses a different - implementation of qsort() that compares parts differently. + Fixes https://github.com/curl/curl/issues/12117 + Closes https://github.com/curl/curl/pull/12118 - Fixes #11855 - Closes #11868 +Daniel Stenberg (14 Oct 2023) -- CI: ignore the "flaky" and "timing-dependent" test results +- RELEASE-NOTES: synced - CI builds will now run these tests, but will ignore the results if they - fail. The relevant tests are ones that are sensitive to timing or - have edge conditions that make them more likely to fail on CI servers, - which are often heavily overloaded and slow. +Sohom Datta (14 Oct 2023) - This change only adds two additional tests to be ignored, since the - others already had the flaky keyword. +- tool_getparam: limit --rate to be smaller than number of ms - Closes #11865 + Currently, curl allows users to specify absurd request rates that might + be higher than the number of milliseconds in the unit (ex: curl --rate + 3600050/h http://localhost:8080 does not error out despite there being + only 3600000ms in a hour). -- runtests: eliminate a warning on old perl versions + This change adds a conditional check before the millisecond calculation + making sure that the number is not higher than the numerator (the unit) + If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE - The warning "Use of implicit split to @_ is deprecated" showed between - perl versions about 5.8 through 5.11. + Closes #12116 -- tests: log the test result code after each libtest +Daniel Stenberg (14 Oct 2023) - This makes it easier to determine the test status. Also, capitalize - FAILURE and ABORT messages in log lines to make them easier to spot. +- opts: fix two minor man page format mistakes -Harry Sintonen (16 Sep 2023) +Jay Satiro (14 Oct 2023) -- misc: better random strings +- curl_trc: remove a bad assertion - Generate alphanumerical random strings. + - Remove DEBUGASSERT that an internal handle must not have user + private_data set before calling the user's debug callback. - Prior this change curl used to create random hex strings. This was - mostly okay, but having alphanumerical random strings is better: The - strings have more entropy in the same space. + This is a follow-up to 0dc40b2a. The user can distinguish their easy + handle from an internal easy handle by setting CURLOPT_PRIVATE on their + easy handle. I had wrongly assumed that meant the user couldn't then + set CURLOPT_PRIVATE on an internal handle as well. - The MIME multipart boundary used to be mere 64-bits of randomness due - to being 16 hex chars. With these changes the boundary is 22 - alphanumerical chars, or little over 130 bits of randomness. + Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 + Reported-by: Daniel Stenberg - Closes #11838 + Closes https://github.com/curl/curl/pull/12104 -Daniel Stenberg (15 Sep 2023) +Dan Fandrich (13 Oct 2023) -- cookie: reduce variable scope, add const +- test613: stop showing an error on missing output file -- cookie: do not store the expire or max-age strings + This test would show an error message if the output was missing during + the log post-processing step, but the message was not captured by the + test harness and wasn't useful since the normal golden log file + comparison would the problem more clearly. - Convert it to an expire time at once and save memory. +Stefan Eissing (13 Oct 2023) - Closes #11862 +- quic: manage connection idle timeouts -- cookie: remove unnecessary struct fields + - configure a 120s idle timeout on our side of the connection + - track the timestamp when actual socket IO happens + - check IO timestamp to our *and* the peer's idle timeouts + in "is this connection alive" checks - Plus: reduce the hash table size from 256 to 63. It seems unlikely to - make much of a speed difference for most use cases but saves 1.5KB of - data per instance. + Reported-by: calvin2021y on github + Fixes #12064 + Closes #12077 - Closes #11862 +Dan Fandrich (13 Oct 2023) -- RELEASE-NOTES: synced +- CI: ignore test 286 on Appveyor gcc 9 build - Bumped to 8.4.0, the next presumed version + This test fails sometimes with a super fast retry loop due to what may + just be a compiler bug. The test results are ignored on the one CI job + where it occurs because there seems to be nothing we can do to fix it. -Dan Fandrich (14 Sep 2023) + Fixes #12040 + Closes #12106 -- test2600: remove special case handling for USE_ALARM_TIMEOUT +Viktor Szakats (13 Oct 2023) - This was originally added to handle platforms that supported only 1 - second granularity in connect timeouts, but after some recent changes - the test currently permafails on several Windows platforms. +- lib: fix gcc warning in printf call - The need for this special-case was removed in commit 8627416, which - increased the connect timeout in all cases to well above 1 second. + Do not pass NULL to printf %s. - Fixes #11767 - Closes #11849 + Seen with gcc 13.2.0 on Debian: + ``` + .../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wfo + rmat-overflow=] + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/1758442 + 6483#step:3:11104 -Daniel Stenberg (14 Sep 2023) + Ref: #10284 + Co-authored-by: Jay Satiro + Closes #12082 -- SECURITY-PROCESS.md. call it vulnerability disclosure policy +Alex Klyubin (13 Oct 2023) - SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md +- http2: safer invocation of populate_binsettings - This a name commonly used for a document like this. This name helps - users find it. + populate_binsettings now returns a negative value on error, instead of a + huge positive value. Both places which call this function have been + updated to handle this change in its contract. - Closes #11852 + The way populate_binsettings had been used prior to this change the huge + positive values -- due to signed->unsigned conversion of the potentially + negative result of nghttp2_pack_settings_payload which returns negative + values on error -- are not possible. But only because http2.c currently + always provides a large enough output buffer and provides H2 SETTINGS + IVs which pass the verification logic inside nghttp2. If the + verification logic were to change or if http2.c started passing in more + IVs without increasing the output buffer size, the overflow could become + reachable, and libcurl/curl might start leaking memory contents to + servers/proxies... -Junho Choi (14 Sep 2023) + Closes #12101 -- quiche: fix build error with --with-ca-fallback +Daniel Stenberg (13 Oct 2023) - - Fix build error when curl is built with --with-quiche - and --with-ca-fallback. +- openssl: avoid BN_num_bits() NULL pointer derefs - - Add --with-ca-fallback to the quiche CI job. + Reported-by: icy17 on github + Fixes #12099 + Closes #12100 - Fixes https://github.com/curl/curl/issues/11850 - Closes https://github.com/curl/curl/pull/11847 +- wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA -Jay Satiro (14 Sep 2023) + This define is set in wolfssl's options.h file when this function and + feature is present. Handles both builds with the feature explicitly + disabled and wolfSSL versions before 5.5.2 - which introduced this API + call. -- escape: replace Curl_isunreserved with ISUNRESERVED + Closes #12108 - - Use the ALLCAPS version of the macro so that it is clear a macro is - being called that evaluates the variable multiple times. +- tool_urlglob: make multiply() bail out on negative values - - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates - a variable multiple times. + - Does not work correctly with negative values + - use __builtin_mul_overflow() on gcc - This is a follow-up to 291d225a which changed Curl_isunreserved into an - alias macro for ISUNRESERVED. The problem is the former is not easily - identified as a macro by the caller, which could lead to a bug. + Reported-by: Torben Dury + Closes #12102 - For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but - Curl_isunreserved(*foo++) is not even though they both are the same. +Loïc Yhuel (13 Oct 2023) - Closes https://github.com/curl/curl/pull/11846 +- cmake: fix CURL_DISABLE_GETOPTIONS -Dan Fandrich (13 Sep 2023) + - Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake. -- tests: increase the default server logs lock timeout + Prior to this change the option had no effect because it was missing + from that file. - This timeout is used to wait for the server to finish writing its logs - before checking them against the expected values. An overloaded machine - could take more than the two seconds previously allocated, so increase - the timeout to 5 seconds. + Closes https://github.com/curl/curl/pull/12091 - Ref: #11328 - Closes #11834 +- easy_lock: add a pthread_mutex_t fallback -- tests: increase TEST_HANG_TIMEOUT in two tests + This allows to keep the init threadsafe with gcc < 4.9.0 (no C11 + atomics). - These tests had a 5 second timeout compared to 60 seconds for all other - tests. Make these consistent with the others for more reliability on - heavily-loaded machines. + Closes https://github.com/curl/curl/pull/12090 - Ref: #11328 +Viktor Szakats (12 Oct 2023) -- test1056: disable on Windows +- CI: add autotools, out-of-tree, debug build to distro check job - This test relies on the IPv6 scope field being ignored when connecting to - ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit - dodgy, but it works on all our test platforms except Windows. This - test was disabled manually on all Windows CI builds already, so instead - add an incompatible feature and precheck so it's skipped on Windows - everywhere automatically. + Add a job that builds curl from a generated source tarball sample, with + autotools, out-of-tree, in debug mode. -- test587: add a slight delay after test + Ref: #12085 + Closes #12088 - This test is designed to connect to the server, then immediately send a - few bytes and disconnect. In some situations, such as on a loaded - server, this doesn't give the server enough time to write its lock file - before its existence is checked. The test harness then fails to find the - server's input log file (because it hasn't been written yet) and fails - the test. By adding a short delay after the test, the HTTP server has - enough time to write its lock file which gives itself more time to write - its remaining files. +Daniel Stenberg (12 Oct 2023) - Ref: #11328 +- http: avoid Expect: 100-continue if Upgrade: is used -- tests: stop overriding the lock timeout + Reported-by: Daniel Jelinski + Fixes #12022 + Closes #12062 - These tests reduce the server lock wait timeout which can increase - flakiness on loaded machines. Since this is merely an optimization, - eliminate them in favour of reliability. +Jan Alexander Steffens (heftig) (12 Oct 2023) - Ref: #11328 +- docs: use SOURCE_DATE_EPOCH for generated manpages -- tests: add some --expect100-timeout to reduce timing dependencies + This should make builds from Git reproducible. - These tests can fail when the test machine is so slow that the test HTTP - server didn't get a chance to complete before the client's one second - 100-continue timeout triggered. Increase that 1 second to 999 seconds so - this situation doesn't happen. + Closes #12092 - Ref: #11328 +Daniel Stenberg (12 Oct 2023) -- test661: return from test early in case of curl error +- RELEASE-NOTES: synced -- tests: add the timing-dependent keyword on several tests + Bumped to 8.4.1 - These are ones likely to fail on heavily-loaded machines that alter the - normal test timing. Most of these tests already had the flaky keyword - since this condition makes them more likely to fail on CI. +Viktor Szakats (12 Oct 2023) -- test1592: greatly increase the maximum test timeout +- cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection - It was too short to be reliable on heavily loaded CI machines, and - as a fail-safe only, it didn't need to be short. + Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet, + aligning this with autotools. This fixes an error when doing + cross-builds and also actually detects this feature. It affected systems + not allowlisted into this, e.g. SerenityOS. - Ref: #11328 + We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`. -- test: minor test cleanups + Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979 + Ref: #12095 (closed in favour of this patch) + Ref: #11964 (effort to sync cmake detections with autotools) - Remove an obsolete block of code in tests 2032 & 576. - Add a comment in test 1474. + Reported-by: Kartatz on Github + Assisted-by: Kartatz on Github + Fixes #12093 + Closes #12094 -- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts +- build: add `src/.checksrc` to source tarball - This gives more of a margin for error when running on overloaded CI - servers. + Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958 - Ref: #11328 + Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071 + Reported-by: Romain Geissler + Fixes #12084 + Closes #12085 -- tests: improve SLOWDOWN test reliability by reducing sent data +Version 8.4.0 (11 Oct 2023) - These tests are run in SLOWDOWN mode which adds a 10 msec delay after - each character output, which means it takes at least 1.6 seconds (and - 320 kernel calls) just to get through the long welcome banner. On an - overloaded system, this can end up taking much more than 1.6 seconds, - and even more than the 7 or 16 second curl timeout that the tests rely - on, causing them to fail. Reducing the size of the welcome banner drops - the total number of characters sent before the transfer starts by more - than half, which reduces the opportunity for test-breaking slowdowns by - the same amount. +Daniel Stenberg (11 Oct 2023) - Ref: #11328 +- RELEASE-NOTES: synced -- test650: fix an end tag typo +- THANKS: add contributors from 8.4.0 -Jay Satiro (13 Sep 2023) +Jay Satiro (11 Oct 2023) -- tool_cb_wrt: fix debug assertion +- socks: return error if hostname too long for remote resolve - - Fix off-by-one out-of-bounds array index in Windows debug assertion. + Prior to this change the state machine attempted to change the remote + resolve to a local resolve if the hostname was longer than 255 + characters. Unfortunately that did not work as intended and caused a + security issue. - Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 - Reported-by: Gisle Vanem + Bug: https://curl.se/docs/CVE-2023-38545.html -Daniel Stenberg (13 Sep 2023) +Stefan Eissing (10 Oct 2023) -- ctype: add ISUNRESERVED() +- CI: remove slowed-network tests - ... and make Curl_isunreserved() use that macro instead of providing a - separate funtion for the purpose. + - remove these tests as they are currently not reliable in our CI + setups. - Closes #11840 + curl handles the test cases, but CI sometimes fails on these due to + additional conditions. Rather than mix them in, an additional CI job + will be added in the future that is specific to them. -Version 8.3.0 (13 Sep 2023) + Closes https://github.com/curl/curl/pull/12075 -Daniel Stenberg (13 Sep 2023) +Jay Satiro (10 Oct 2023) -- RELEASE-NOTES: syn ced +- libcurl-env-dbg.3: move debug variables from libcurl-env.3 - curl 8.3.0 release + - Move documentation of libcurl environment variables used only in debug + builds from libcurl-env into a separate document libcurl-env-dbg. -- THANKS: contributors from 8.3.0 + - Document more debug environment variables. -Thorsten Klein (12 Sep 2023) + Previously undocumented or missing a description: -- cmake: set SIZEOF_LONG_LONG in curl_config.h + CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, + CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, + CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, + CURL_TIME. - in order to support 32bit builds regarding wolfssl CTC_SETTINGS + Closes https://github.com/curl/curl/pull/11811 - Closes #11839 +Dan Fandrich (9 Oct 2023) -Jay Satiro (12 Sep 2023) +- test670: increase the test timeout -- curl_ngtcp2: fix error message + This should make it more immune to loaded servers. -- http_aws_sigv4: handle no-value user header entries + Ref: #11328 - - Handle user headers in format 'name:' and 'name;' with no value. +Stefan Eissing (9 Oct 2023) - The former is used when the user wants to remove an internal libcurl - header and the latter is used when the user actually wants to send a - no-value header in the format 'name:' (note the semi-colon is converted - by libcurl to a colon). +- MQTT: improve receive of ACKs - Prior to this change the AWS header import code did not special case - either of those and the generated AWS SignedHeaders would be incorrect. + - add `mq->recvbuf` to provide buffering of incomplete + ACK responses + - continue ACK reading until sufficient bytes available + - fixes test failures on low network receives - Reported-by: apparentorder@users.noreply.github.com + Closes #12071 - Ref: https://curl.se/docs/manpage.html#-H +Viktor Szakats (9 Oct 2023) - Fixes https://github.com/curl/curl/issues/11664 - Closes https://github.com/curl/curl/pull/11668 +- quic: fix BoringSSL build -Dan Fandrich (11 Sep 2023) + Add guard around `SSL_CTX_set_ciphersuites()` use. -- CI: run pytest with the -v option + Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 - This lists of the test cases being run so it can be tracked over time. + Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 - Closes #11824 + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes #12067 -Daniel Stenberg (11 Sep 2023) +Stefan Eissing (9 Oct 2023) -- HTTP3: the msquic backend is not functional +- test1540: improve reliability - I ask that we do not submit bugs for this backend just yet as we know it - does not fully work. + - print that bytes have been received on pausing, but not how many - Closes #11831 - Closes #11819 + Closes #12069 -- aws_sigv4: the query canon code miscounted URL encoded input +- test2302: improve reliability - Added some extra ampersands to test 439 to verify "blank" query parts + - make result print collected write data, unless + change in meta flags is detected + - will show same result even when data arrives via + several writecb invocations - Follow-up to fc76a24c53b08cdf + Closes #12068 - Closes #11829 +Daniel Stenberg (9 Oct 2023) -vvb2060 (11 Sep 2023) +- curl_easy_pause: set "in callback" true on exit if true -- quic: don't set SNI if hostname is an IP address + Because it might have called another callback in the mean time that then + set the bit FALSE on exit. - We already do this for TLS connections. + Reported-by: Jay Satiro + Fixes #12059 + Closes #12061 - RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in - "HostName". +Viktor Szakats (8 Oct 2023) - Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 +- h3: add support for ngtcp2 with AWS-LC builds - Fixes https://github.com/curl/curl/issues/11827 - Closes https://github.com/curl/curl/pull/11828 + ``` + curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- + LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps + mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss + Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi + SSL NTLM SSL threadsafe UnixSockets + ``` -Daniel Stenberg (10 Sep 2023) + Also delete an obsolete GnuTLS TODO and update the header comment in + `FindNGTCP2.cmake`. -- RELEASE-NOTES: synced + Reviewed-by: Daniel Stenberg + Closes #12066 -Benoit Pierre (10 Sep 2023) +- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros -- configure: fix `HAVE_TIME_T_UNSIGNED` check + Syncing this up with CMake. - The syntax was incorrect (need a proper main body), and the test - condition was wrong (resulting in a signed `time_t` detected as - unsigned). + Source code uses the built-in `OPENSSL_IS_AWSLC` and + `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is + necessary from the build tools. - Closes #11825 + The one use of `HAVE_BORINGSSL` in the source turned out to be no longer + necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 -Daniel Stenberg (9 Sep 2023) + autotools detects this anyway for display purposes. + CMake detects this to decide whether to use the BoringSSL-specific + crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection + result just yet (planned in #12066). -- THANKS-filter: pszlazak on github + Ref: #11964 -pszlazak (9 Sep 2023) + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12065 -- include.d: explain headers not printed with --fail before 7.75.0 +Marc Hoersken (8 Oct 2023) - Prior to 7.75.0 response headers were not printed if -f/--fail was used - and an error was reported by server. This was fixed in ab525c0 - (precedes 7.75.0). +- CI: move distcheck job from Azure Pipelines to GitHub Actions - Closes #11822 + This will allow for more trigger excludes within Azure Pipelines. -Daniel Stenberg (8 Sep 2023) + Also fixes seemingly broken check with scripts/installcheck.sh. + Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 -- http_aws_sigv4: skip the op if the query pair is zero bytes + Assisted-by: Philip Heiduck + Closes #9532 - Follow-up to fc76a24c53b08cdf +Daniel Stenberg (8 Oct 2023) - Spotted by OSS-Fuzz +- url: fall back to http/https proxy env-variable if ws/wss not set - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 - Closes #11823 + Reported-by: Craig Andrews + Fixes #12031 + Closes #12058 -- cmdline-docs: use present tense, not future +Stefan Eissing (8 Oct 2023) - + some smaller cleanups +- cf-socket: simulate slow/blocked receives in debug - Closes #11821 + add 2 env variables for non-UDP sockets: + 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly + should return EAGAIN + 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket -- cmdline-docs: make sure to phrase it as "added in ...." + Closes #12035 - References to things that were added or changed in a specific version - should be specified as "(added in [version]) for two reasons: +- http2: refused stream handling for retry - 1 - consistency + - answer HTTP/2 streams refused via a GOAWAY from the server to + respond with CURLE_RECV_ERROR in order to trigger a retry + on another connection - 2 - to allow gen.pl to strip them out if deemed referring to too old - versions + Reported-by: black-desk on github + Ref #11859 + Closes #12054 - Closes #11821 +Jay Satiro (8 Oct 2023) -Jay Satiro (8 Sep 2023) +- CURLOPT_DEBUGFUNCTION.3: warn about internal handles -- docs: mark --ssl-revoke-best-effort as Schannel specific + - Warn that the user's debug callback may be called with the handle + parameter set to an internal handle. - Closes https://github.com/curl/curl/pull/11760 + Without this warning the user may assume that the only handles their + debug callback receives are the easy handles on which they set + CURLOPT_DEBUGFUNCTION. -Nathan Moinvaziri (8 Sep 2023) + This is a follow-up to f8cee8cc which changed DoH handles to inherit + the debug callback function set in the user's easy handle. As a result + those handles are now passed to the user's debug callback function. -- schannel: fix ordering of cert chain info + Closes https://github.com/curl/curl/pull/12034 - - Use CERT_CONTEXT's pbCertEncoded to determine chain order. +- url: fix typo - CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains - end-entity/server certificate in pbCertEncoded. We can use this pointer - to determine the order of certificates when enumerating hCertStore using - CertEnumCertificatesInStore. +Daniel Stenberg (8 Oct 2023) - This change is to help ensure that the ordering of the certificate chain - requested by the user via CURLINFO_CERTINFO has the same ordering on all - versions of Windows. +- test458: verify --expand-output, expanding a file name accepting option - Prior to this change Schannel certificate order was reversed in 8986df80 - but that was later reverted in f540a39b when it was discovered that - Windows 11 22H2 does the reversal on its own. + Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) - Ref: https://github.com/curl/curl/issues/9706 +- tool_getparam: accept variable expansion on file names too - Closes https://github.com/curl/curl/pull/11632 + Reported-by: PBudmark on github + Fixes #12048 + Closes #12055 -Chris Talbot (8 Sep 2023) +- RELEASE-NOTES: synced -- digest: Use hostname to generate spn instead of realm +- multi: do CURLM_CALL_MULTI_PERFORM at two more places - In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 + ... when it does a state transition but there is no particular socket or + timer activity. This was made apparent when commit b5bb84c removed a + superfluous timer expiry. - digest-uri-value should be serv-type "/" host , where host is: + Reported-by: Dan Fandrich. + Fixes #12033 + Closes #12056 - The DNS host name or IP address for the service requested. The - DNS host name must be the fully-qualified canonical name of the - host. The DNS host name is the preferred form; see notes on server - processing of the digest-uri. +Viktor Szakats (7 Oct 2023) - Realm may not be the host, so we must specify the host explicitly. +- GHA/linux: mbedtls 3.5.0 + minor dep bumps - Note this change only affects the non-SSPI digest code. The digest code - used by SSPI builds already uses the hostname to generate the spn. + Closes #12057 - Ref: https://github.com/curl/curl/issues/11369 +Dan Fandrich (7 Oct 2023) - Closes https://github.com/curl/curl/pull/11395 +- CI: bump OpenLDAP package version on FreeBSD -Daniel Stenberg (7 Sep 2023) + The old one is no longer available. -- docs: remove use of the word 'very' +Marc Hoersken (7 Oct 2023) - It is mostly superfluous. proselint would complain. +- docs/libcurl/opts/Makefile.inc: add missing manpage files - Closes #11818 + Detected with #9532 -- curl_multi_remove_handle.3: clarify what happens with connection +Dan Fandrich (7 Oct 2023) - Closes #11817 +- tests: fix a race condition in ftp server disconnect -- RELEASE-NOTES: synced + If a client disconnected and reconnected quickly, before the ftp server + had a chance to respond, the protocol message/ack (ping/pong) sequence + got out of sync, causing messages sent to the old client to be delivered + to the new. A disconnect must now be acknowledged and intermediate + requests thrown out until it is, which ensures that such synchronization + problems can't occur. This problem could affect ftp, pop3, imap and smtp + tests. -- test439: verify query canonization for aws-sigv4 + Fixes #12002 + Closes #12049 -- tool_operate: make aws-sigv4 not require TLS to be used +Viktor Szakats (7 Oct 2023) - Maybe not used too often, but we want it for testing and it should work. +- appveyor: bump mingw-w64 job to gcc 13 (was: 8) -- http_aws_sigv4: canonicalize the query + This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). + Adding a modern gcc version to the tests. - Percent encoding needs to be done using uppercase, and most - non-alphanumerical must be percent-encoded. + (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 + finished in 32, 35, 34 minutes in the 3 test runs so far.) - Fixes #11794 - Reported-by: John Walker - Closes #11806 + It also adds a modern CMake version and OS env to our mingw-w64 builds. -Wyatt O'Day (7 Sep 2023) + Closes #12051 -- lib: add ability to disable auths individually +David Benjamin (6 Oct 2023) - Both with configure and cmake +- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR - Closes #11490 + While the struct is still public in OpenSSL, there is a (somewhat + inconvenient) accessor. Use it to remain compatible if it becomes opaque + in the future. -Stefan Eissing (7 Sep 2023) + Closes #12038 -- ngtcp2: fix handling of large requests +Daniel Stenberg (6 Oct 2023) - - requests >64K are send in parts to the filter - - fix parsing of the request to assemble it correctly - from several sends - - open a QUIC stream only when the complete request has - been collected +- curl_easy_pause.3: mention it works within callbacks - Closes #11815 + Reported-by: Maxim Dzhura + Bug: https://curl.se/mail/lib-2023-10/0010.html + Closes #12046 -- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before +- curl_easy_pause.3: mention h2/h3 buffering - - we delay loading the x509 store to shorten the handshake time. - However an application callback installed via CURLOPT_SSL_CTX_FUNCTION - may need to have the store loaded and try to manipulate it. - - load the x509 store before invoking the app callback + Asked-by: Maxim Dzhura + Ref: https://curl.se/mail/lib-2023-10/0011.html - Fixes #11800 - Reported-by: guoxinvmware on github - Cloes #11805 + Closes #12045 -Daniel Stenberg (7 Sep 2023) +Viktor Szakats (6 Oct 2023) -- krb5: fix "implicit conversion loses integer precision" warnings +- cmake: re-add missed C89 headers for specific detections - conversions to/from enum and unsigned chars + We removed C89 `setjmp.h` and `signal.h` detections and excluded them + from the global header list we use when detecting functions [1]. Then + missed to re-add these headers to the specific functions which need + them to be detected [2]. Fix this omission in this patch. - Closes #11814 + [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 + [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 -Stefan Eissing (7 Sep 2023) + Closes #12043 -- pytest: improvements +Daniel Stenberg (6 Oct 2023) - - set CURL_CI for pytest runs in CI environments - - exclude timing sensitive tests from CI runs - - for failed results, list only the log and stat of - the failed transfer +- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE - - fix type in http.c comment + Since there is nothing to wait for there. Avoids the test 1233 hang + reported in #12033. - Closes #11812 + Reported-by: Dan Fandrich + Closes #12042 -- CI: move on to ngtcp2 v0.19.1 +Dan Fandrich (5 Oct 2023) - Closes #11809 +- test1903: actually verify the cookies after the test -Dan Fandrich (5 Sep 2023) + The test otherwise could do just about anything (except leak memory in + debug mode) and its bad behaviour wouldn't be detected. Now, check the + resulting cookie file to ensure the cookies are still there. -- CI: run Circle macOS builds on x86 for now + Closes #12041 - The ARM machines aren't ready for us and requesting them now causes - warnings e-mails to be sent to some PR pushers. +- test: add missing s - Ref: #11771 + The tests will otherwise fail if curl has them disabled. -Viktor Szakats (5 Sep 2023) +- test1906: set a lower timeout since it's hit on Windows -- http3: adjust cast for ngtcp2 v0.19.0 + msys2 builds actually hit the connect timeout in normal operation, so + lower the timeout from 5 minutes to 5 seconds to reduce test time. - ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` - an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. + Ref: #11328 + Closes #12036 - Fixes: - ``` - ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in - teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned - char') [-Wimplicit-int-conversion] - pi.ecn = (uint32_t)ecn; - ~ ^~~~~~~~~~~~~ - ``` +Daniel Stenberg (5 Oct 2023) - Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our - docs and CI. +- RELEASE-NOTES: synced - Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 - 782894a3 - Ref: https://github.com/ngtcp2/ngtcp2/pull/877 - Closes #11798 +Jay Satiro (5 Oct 2023) -Stefan Eissing (5 Sep 2023) +- idn: fix WinIDN null ptr deref on bad host -- http: fix sending of large requests + - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from + UTF-8 to UTF-16. - - refs #11342 where errors with git https interactions - were observed - - problem was caused by 1st sends of size larger than 64KB - which resulted in later retries of 64KB only - - limit sending of 1st block to 64KB - - adjust h2/h3 filters to cope with parsing the HTTP/1.1 - formatted request in chunks + Prior to this change a failed conversion erroneously returned CURLE_OK + which meant 'decoded' pointer (what would normally point to the + punycode) would not be written to, remain NULL and be dereferenced + causing an access violation. - - introducing Curl_nwrite() as companion to Curl_write() - for the many cases where the sockindex is already known + Closes https://github.com/curl/curl/pull/11983 - Fixes #11342 (again) - Closes #11803 +Dan Fandrich (4 Oct 2023) -- pytest: fix check for slow_network skips to only apply when intended +- tests: close the shell used to start sshd - Closes #11801 + This shell isn't needed once sshd starts, so use "exec" so it doesn't + stick around. -Daniel Stenberg (5 Sep 2023) + Closes #12032 -- curl_url_get/set.3: add missing semicolon in SYNOPSIS +Daniel Stenberg (4 Oct 2023) -- CURLOPT_URL.3: explain curl_url_set() uses the same parser +- base64: also build for curl -- CURLOPT_URL.3: add two URL API calls in the see-also section + Since the tool itself now uses the base64 code using the curlx way, it + needs to build also when the tool needs it. Starting now, the tool build + defines BULDING_CURL to allow lib-side code to use it. -Dan Fandrich (4 Sep 2023) + Follow-up to 2e160c9c6525 -- CI: add a 32-bit i686 Linux build + Closes #12010 - This is done by cross-compiling under regular x86_64 Linux. Since the - kernel offers backwards compatibility, the binaries can be tested as - normal. +Eduard Strehlau (4 Oct 2023) - Closes #11799 +- tests: Fix zombie processes left behind by FTP tests. -- tests: fix a type warning on 32-bit x86 + ftpserver.pl correctly cleans up spawned server processes, + but forgets to wait for the shell used to spawn them. + This is barely noticeable during a normal testrun, + but causes process exhaustion and test failure + during a complete torture run of the FTP tests. -Viktor Szakats (4 Sep 2023) + Fixes #12018 + Closes #12020 -- tests: delete stray `.orig` file +Dan Fandrich (4 Oct 2023) - Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f - Closes #11797 +- github/labeler: improve labeler matches -Daniel Stenberg (4 Sep 2023) +- test574: add a timeout to the test -- RELEASE-NOTES: synced + This one hangs occasionally, so this will speed up a test run and allow + logs to be seen when it does. -Viktor Szakats (4 Sep 2023) + Closes #12025 -- lib: silence compiler warning in inet_ntop6 +- tests: propagate errors in libtests - ``` - ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her - e [-Wcomma] - cur.base = i, cur.len = 1; - ^ - ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin - g - cur.base = i, cur.len = 1; - ^~~~~~~~~~~~ - (void)( ) - ``` + Use the test macros to automatically propagate some errors, and check + and log others while running the tests. This can help in debugging + exactly why a test has failed. - Closes #11790 +- tests: set --expect100-timeout to improve test reliability -Daniel Stenberg (4 Sep 2023) + On an overloaded server, the default 1 second timeout can go by without + the test server having a chance to respond with the expected headers, + causing tests to fail. Increase the 1 second timeout to 99 seconds so + this failure mode is no longer a problem on test 1129. Some other tests + already set a high value, but make them consistently 99 seconds so if + something goes wrong the test is stalled for less time. -- transfer: also stop the sending on closed connection + Ref: #11328 - Previously this cleared the receiving bit only but in some cases it is - also still sending (like a request-body) when disconnected and neither - direction can continue then. +- CI: ignore the "flaky" and "timing-dependent" test results in CMake - Fixes #11769 - Reported-by: Oleg Jukovec - Closes #11795 + This was already done for automake builds but CMake builds were missed. + Test 1086 actually causes the test harness to crash with: -John Bampton (4 Sep 2023) + Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec + ts/curl/tests/ftpserver.pl line 527 -- docs: change `sub-domain` to `subdomain` + Rather than fix it now, this change leaves test 1086 entirely skipped on + those builds that show this problem. - https://en.wikipedia.org/wiki/Subdomain + Follow-up to 589dca761 - Closes #11793 + Ref: #11865 -Stefan Eissing (4 Sep 2023) +Viktor Szakats (4 Oct 2023) -- multi: more efficient pollfd count for poll +- cmake: improve OpenLDAP builds - - do not use separate pollfds for sockets that have POLLIN+POLLOUT + - cmake: detect OpenLDAP based on function `ldap_init_fd`. + autotools does this. autotools also publishes this detection result + in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source + doesn't use this value. (it might need to be remove-listed in + `scripts/cmp-config.pl` for future OpenLDAP test builds.) + This also deletes existing self-declaration method via the + CMake-specific `CURL_USE_OPENLDAP` configuration. - Closes #11792 + - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. + Like autotools does. This fixes a long list of these warnings: + ``` + /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i + s not defined, evaluates to 0 [-Wundef] + ``` -- http2: polish things around POST + - cmake: delete LDAP TODO comment no longer relevant. - - added test cases for various code paths - - fixed handling of blocked write when stream had - been closed inbetween attempts - - re-enabled DEBUGASSERT on send with smaller data size + Also: - - in debug builds, environment variables can be set to simulate a slow - network when sending data. cf-socket.c and vquic.c support - * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be - answered with a EAGAIN. TCP/UNIX sockets. - This is chosen randomly. - * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written - to the network. TCP/UNIX sockets. - Example: 80 means a send with 1000 bytes would only send 800 - This is applied to every send. - * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be - answered with EAGAIN. QUIC only. - This is chosen randomly. + - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature + detection functions. - Closes #11756 + Ref: #11964 (effort to sync cmake detections with autotools) -Daniel Stenberg (4 Sep 2023) + Closes #12024 -- docs: add curl_global_trace to some SEE ALSO sections +- cmake: fix unity builds for more build combinations - Closes #11791 + By using unique static function/variable names in source files + implementing these interfaces. -- os400: fix checksrc nits + - OpenLDAP combined with any SSH backend. - Closes #11789 + - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. -Nicholas Nethercote (3 Sep 2023) + Closes #12027 -- hyper: remove `hyptransfer->endtask` +Daniel Stenberg (4 Oct 2023) - `Curl_hyper_stream` needs to distinguish between two kinds of - `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and - (b) background tasks that hyper produces. It does this by recording the - address of any `foreach` task in `hyptransfer->endtask` before pushing - it into the executor, and then comparing that against the address of - tasks later polled out of the executor. +- tests: remove leading spaces from some tags - This works right now, but there is no guarantee from hyper that the - addresses are stable. `hyper_executor_push` says "The executor takes - ownership of the task, which should not be accessed again unless - returned back to the user with `hyper_executor_poll`". That wording is a - bit ambiguous but with my Rust programmer's hat on I read it as meaning - the task returned with `hyper_executor_poll` may be conceptually the - same as a task that was pushed, but that there are no other guarantees - and comparing addresses is a bad idea. + The threee tags ``, `` and `` were frequently used + with a leading space that this removes. The reason this habbit is so + widespread in testcases is probably that they have been copy and pasted. - This commit instead uses `hyper_task_set_userdata` to mark the `foreach` - task with a `USERDATA_RESP_BODY` value which can then be checked for, - removing the need for `hyptransfer->endtask`. This makes the code look - more like that hyper C API examples, which use userdata for every task - and never look at task addresses. + Hence, fixing them all now might curb this practice from now on. - Closes #11779 + Closes #12028 -Dave Cottlehuber (3 Sep 2023) +Viktor Szakats (4 Oct 2023) -- ws: fix spelling mistakes in examples and tests +- GHA: bump actions/checkout - Closes #11784 + Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 + Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 -Daniel Stenberg (3 Sep 2023) + Closes #12023 -- tool_filetime: make -z work with file dates before 1970 +- spelling: fix codespell 2.2.6 typos - Fixes #11785 - Reported-by: Harry Sintonen - Closes #11786 + Closes #12019 -Dan Fandrich (1 Sep 2023) +Daniel Stenberg (3 Oct 2023) -- build: fix portability of mancheck and checksrc targets +- GHA: add workflow to compare configure vs cmake outputs - At least FreeBSD preserves cwd across makefile lines, so rules - consisting of more than one "cd X; do_something" must be explicitly run - in a subshell to avoid this. This problem caused the Cirrus FreeBSD - build to fail when parallel make jobs were enabled. + Uses scripts/cmp-config.pl two compare two curl_config.h files, + presumbly generated with configure and cmake. It displays the + differences and filters out a lot of known lines we ignore. -- CI: adjust labeler match patterns for new & obsolete files + The script also shows the matches that were *not* used. Possibly + subjects for removal. -- configure: trust pkg-config when it's used for zlib + Closes #11964 - The library flags retrieved from pkg-config were later thrown out and - harded-coded, which negates the whole reason to use pkg-config. - Also, previously, the assumption was made that --libs-only-l and - --libs-only-L are the full decomposition of --libs, which is untrue and - would not allow linking against a static zlib. The new approach is - better in that it uses --libs, although only if --libs-only-l returns - nothing. +- appveyor: enable test 571 - Bug: https://curl.se/mail/lib-2023-08/0081.html - Reported-by: Randall - Closes #11778 + Follow-up from 8a940fd55c175f7 / #12013 -Stefan Eissing (1 Sep 2023) + Closes #12017 -- CI/ngtcp2: clear wolfssl for when cache is ignored +Viktor Szakats (3 Oct 2023) - Closes #11783 +- build: alpha-sort source files for lib and src -Daniel Stenberg (1 Sep 2023) + Closes #12014 -- RELEASE-NOTES: synced +- cmake: delete old `HAVE_LDAP_URL_PARSE` logic -Nicholas Nethercote (1 Sep 2023) + Left there by accident after adding proper detection for this. -- hyper: fix a progress upload counter bug + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - `Curl_pgrsSetUploadCounter` should be a passed a total count, not an - increment. + Ref: #11964 (effort to sync cmake detections with autotools) - This changes the failing diff for test 579 with hyper from this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 29 out of 0[LF] - ``` - to this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 40 out of 0[LF] - ``` - Presumably a step in the right direction. + Closes #12015 - Closes #11780 +Stefan Eissing (3 Oct 2023) -Daniel Stenberg (1 Sep 2023) +- tests: increase lib571 timeout from 3s to 30s -- awssiv4: avoid freeing the date pointer on error + - 3s is too short for our CI, making this test fail occasionally + - test usually experiences no delay run locally, so 30s wont hurt - Since it was not allocated, don't free it even if it was wrong syntax + Closes #12013 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 +Viktor Szakats (3 Oct 2023) - Follow-up to b137634ba3adb +- cmake: fix unity with Windows Unicode + TrackMemory - Closes #11782 + Found the root cause of the startup crash in unity builds with Unicode + and TrackMemory enabled at the same time. -Stefan Eissing (1 Sep 2023) + We must make sure that the `memdebug.h` header doesn't apply to + `lib/curl_multibyte.c` (as even noted in a comment there.) In unity + builds all headers apply to all sources, including `curl_multibyte.c`. + This probably resulted in an infinite loop on startup. -- CI: ngtcp2-linux: use separate caches for tls libraries + Exclude this source from unity compilation with TrackMemory enabled, + in both libcurl and curl tool. Enable unity mode for a debug Unicode + CI job to keep it tested. Also delete the earlier workaround that + fully disabled unity for affected builds. - allow ever changing master for wolfssl + Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Closes #11766 + Closes #11928 -- replace `master` as wolfssl-version with recent commit +- cmake: disable unity mode with Windows Unicode + TrackMemory -- wolfssl, use master again in CI + "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, + aka `-DCURLDEBUG`). - - with the shared session update fix landed in master, it - is time to use that in our CI again + There is an issue with memory tracking and Unicode when built in "unity" + mode, which results in the curl tool crashing right on startup, even + without any command-line option. Interestingly this doesn't happen under + WINE (at least on the system I tested this on), but consistenly happens + on real Windows machines. Crash is 0xC0000374 heap corruption. Both + shared and static curl executables are affected. -Nicholas Nethercote (31 Aug 2023) + This limitation probably won't hit too many people, but it remains + a TODO to find and fix the root cause and drop this workaround. -- tests: fix formatting errors in `FILEFORMAT.md`. + Example builds and runs: + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd + 7iwj#L313 (static) + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby + qu9c#L317 (shared) - Without the surrounding backticks, these tags get swallowed when the - markdown is rendered. + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Closes #11777 + Ref: #11928 + Closes #12005 -Viktor Szakats (31 Aug 2023) +- cmake: tidy-up `NOT_NEED_LBER_H` detection -- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - Allow overriding the default TLS backend via a CMake setting. +- appveyor: rewrite batch in PowerShell + CI improvements - E.g.: - `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` + 1. Rewrite in PowerShell: - Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, - schannel, secure-transport, wolfssl + - rewrite MS-DOS batch build script in PowerShell. + - move some bash operations into native PowerShell. + - fixups for PowerShell insisting on failure when a command outputs + something to stderr. + - fix to actually run `curl -V` after every build. + (and exclude ARM64 builds.) + - also say why we skipped `curl -V` if we had to skip. + - fix CMake warnings about unused configuration variables, by adapting + these dynamically for build cases. + - dedupe OpenSSL path into a variable. + - disable `test1451` failing with a warning anyway due to missing python + impacket. (after trying and failing to install impacket) + PowerShell promotes these warnings to errors by PowerShell. We can also + suppress they wholesale if they start causing issues in the future, + like we already to with `autoreconf` and `./configure`. - The passed string is baked into the curl/libcurl binaries. - The value is case-insensitive. + PowerShell is better than MS-DOS batches, so the hope is this makes it + easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't + supported inline by AppVeyor on Windows build machines, but we are okay + to keep it in an external script, so it's also an option. - We added a similar option to autotools in 2017 via - c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. + 2. CI improvements: - TODO: Convert to lowercase to improve reproducibility. + - enable tests for a "unity" build job. + - speed-up CI initialization by using shallow clones of the curl repo. + - speed-up CMake MSVC jobs with `TrackFileAccess=false`. + - enable parallelism in `VisualStudioSolution` builds. + - display CMake version before builds. + - always show the CPU in job names. + - tell which jobs are build-only in job names. + - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. + - add `config.log` (autotools) to dumped logs (need to enable manually). - Closes #11774 + 3. Style: -- sectransp: fix compiler warnings + - use single-quotes in YAML like we do in other CI YAML files. + It also allows to drop quoting characters and lighter to write/read. + (keep double quotes for PowerShell strings needing expansion.) - https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# - step:3:11046 - ``` - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 - 4: warning: unused variable 'success' [-Wunused-variable] - OSStatus success; - ^ - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 - 4: warning: unused parameter 'sha256len' [-Wunused-parameter] - size_t sha256len) - ^ - ``` + Closes #11999 - Closes #11773 +- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows -- tidy-up: mostly whitespace nits + - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. + Before this patch we set it based it on the presence of `stricmp`, + which correctly enabled it on e.g. Windows, but was inaccurate for + other platforms. - - delete completed TODO from `./CMakeLists.txt`. - - convert a C++ comment to C89 in `./CMake/CurlTests.c`. - - delete duplicate EOLs from EOF. - - add missing EOL at EOF. - - delete whitespace at EOL (except from expected test results). - - convert tabs to spaces. - - convert CRLF EOLs to LF in GHA yaml. - - text casing fixes in `./CMakeLists.txt`. - - fix a codespell typo in `packages/OS400/initscript.sh`. + - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and + LDAPS is not explicitly disabled. This mimics autotools behaviour. + Previously we set it only for Windows LDAP. After this fix, LDAPS is + correctly enabled in default macOS builds. - Closes #11772 + - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) + to avoid deprecation warnings for LDAP API. -Dan Fandrich (31 Aug 2023) + - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. + This doesn't make much sense, but let's do it to sync behaviour with + autotools. -- CI: remove Windows builds from Cirrus, without replacement + - fix benign typo in variable name. - If we don't do this, all coverage on Cirrus will cease in a few days. By - removing the Windows builds, the FreeBSD one should still continue - as before. The Windows builds will need be moved to another service to - maintain test coverage. + Ref: #11964 (effort to sync cmake detections with autotools) - Closes #11771 + Closes #12006 -- CI: switch macOS ARM build from Cirrus to Circle CI +- autotools: restore `HAVE_IOCTL_*` detections - Cirrus is drastically reducing their free tier on Sept. 1, so they will - no longer perform all these builds for us. All but one build has been - moved, with the LibreSSL one being dropped because of linking problems - on Circle. + This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in + 4d73854462f30948acab12984b611e9e33ee41e6 and + c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the + `HAVE_IOCTL` result it generated was unused in the source. But, + I did miss the fact that this had two dependent checks: + `CURL_CHECK_FUNC_IOCTL_FIONBIO`, + `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: + `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. - One important note about this change is that Circle CI is currently - directing all these builds to x86_64 hardware, despite them requesting - ARM. This is because ARM nodes are scheduled to be available on the - free tier only in December. This reduces our architectural diversity - until then but it should automatically come back once those machines are - enabled. + Regression from 4d73854462f30948acab12984b611e9e33ee41e6 -- CI: use the right variable for BSD make + Ref: #11964 (effort to sync cmake detections with autotools) - BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel - builds before. + Closes #12008 -- CI: drop the FreeBSD 12.X build +Daniel Stenberg (2 Oct 2023) - Cirrus' new free tier won't let us have many builds, so drop the - nonessential ones. The FreeBSD 13.X build will still give us the most - relevant FreeBSD coverage. +- RELEASE-PROCEDURE.md: updated coming release dates -- CI: move the Alpine build from Cirrus to GHA +- RELEASE-NOTES: synced - Cirrus is reducing their free tier to next to nothing, so we must move - builds elsewhere. +Viktor Szakats (1 Oct 2023) -Stefan Eissing (30 Aug 2023) +- cmake: pre-cache `HAVE_POLL_FINE` on Windows -- test_07_upload.py: fix test_07_34 curl args + Windows doesn't support `poll()`, so we can safely skip checking for + fine poll. - - Pass correct filename to --data-binary. + Closes #12003 - Prior to this change --data-binary was passed an incorrect filename due - to a missing separator in the arguments list. Since aacbeae7 curl will - error on incorrect filenames for POST. +- gha: bump actions to latest versions - Fixes https://github.com/curl/curl/issues/11761 - Closes https://github.com/curl/curl/pull/11763 + - actions@checkout@v4 (from v3 and v2) -Nicholas Nethercote (30 Aug 2023) + - fsfe/reuse-action@v2 (from v1) -- tests: document which tests fail due to hyper's lack of trailer support. + Closes #12000 - Closes #11762 +Stefan Eissing (30 Sep 2023) -- docs: removing "pausing transfers" from HYPER.md. +- h2: testcase and fix for pausing h2 streams - It's a reference to #8600, which was fixed by #9070. + - refs #11982 where it was noted that paused transfers may + close successfully without delivering the complete data + - made sample poc into tests/http/client/h2-pausing.c and + added test_02_27 to reproduce - Closes #11764 + Closes #11989 + Fixes #11982 + Reported-by: Harry Sintonen -Patrick Monnerat (30 Aug 2023) +Viktor Szakats (30 Sep 2023) -- os400: handle CURL_TEMP_PRINTF() while building bind source +- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value - Closes #11547 + Before this patch CMake builds accepted any value and it was used at + runtime as-is. This patch make sure that the selected default backend + is also enabled in the build. It also enforces a full lowercase value. -- os400: build test servers + This improves reproducibility and brings CMake in sync with autotools + which already worked like described above. - Also fix a non-compliant main prototype in disabled.c. + Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 - Closes #11547 + Closes #11998 -- tests: fix compilation error for os400 +- autotools: adjust `CURL_CA_PATH` value to CMake - OS400 uses BSD 4.3 setsockopt() prototype by default: this does not - define parameter as const, resulting in an error if actual parameter is - const. Remove the const keyword from the actual parameter cast: this - works in all conditions, even if the formal parameter uses it. + autotools was using the same value as CMake, but with an ending + slash. Delete the ending slash to match configurations. - Closes #11547 + Ref: #11964 (effort to sync cmake detections with autotools) -- os400: make programs and command name configurable + Closes #11997 - Closes #11547 +- cmake: detect `sys/wait.h` and `netinet/udp.h` -- os400: move build configuration parameters to a separate script + Ref: #11964 (effort to sync cmake detections with autotools) - They can then easily be overriden in a script named "config400.override" - that is not part of the distribution. + Closes #11996 - Closes #11547 +Daniel Stenberg (30 Sep 2023) -- os400: implement CLI tool +- lib: provide and use Curl_hexencode - This is provided as a QADRT (ascii) program, a link to it in the IFS and - a minimal CL command. + Generates a lower case ASCII hex output from a binary input. - Closes #11547 + Closes #11990 -Matthias Gatto (30 Aug 2023) +- configure: check for the capath by default -- lib: fix aws-sigv4 having date header twice in some cases + ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS + L - When the user was providing the header X-XXX-Date, the header was - re-added during signature computation, and we had it twice in the - request. + cmake: synced - Reported-by: apparentorder@users.noreply.github.com + Assisted-by: Viktor Szakats + Closes #11987 - Signed-off-by: Matthias Gatto +- wolfssl: ignore errors in CA path - Fixes: https://github.com/curl/curl/issues/11738 - Closes: https://github.com/curl/curl/pull/11754 + The default wolfSSL_CTX_load_verify_locations() function is quite picky + with the certificates it loads and will for example return error if just + one of the certs has expired. -Jay Satiro (30 Aug 2023) + With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it + behaves more similar to what OpenSSL does by default. -- multi: remove 'processing: ' debug message + Even the set of default certs on my Debian unstable has several expired + ones. - - Remove debug message added by e024d566. + Assisted-by: Juliusz Sosinowicz + Assisted-by: Michael Osipov - Closes https://github.com/curl/curl/pull/11759 + Closes #11987 -- ftp: fix temp write of ipv6 address +- create-dirs.d: clarify it also uses --output-dirs - - During the check to differentiate between a port and IPv6 address - without brackets, write the binary IPv6 address to an in6_addr. + Reported-by: Robert Simpson + Fixes #11991 + Closes #11995 - Prior to this change the binary IPv6 address was erroneously written to - a sockaddr_in6 'sa6' when it should have been written to its in6_addr - member 'sin6_addr'. There's no fallout because no members of 'sa6' are - accessed before it is later overwritten. +Viktor Szakats (30 Sep 2023) - Closes https://github.com/curl/curl/pull/11747 +- appveyor: fix yamlint issues, indent -- tool: change some fopen failures from warnings to errors + Also: + - use double quotes in all batch if statements. - - Error on missing input file for --data, --data-binary, - --data-urlencode, --header, --variable, --write-out. + Closes #11994 - Prior to this change if a user of the curl tool specified an input file - for one of the above options and that file could not be opened then it - would be treated as zero length data instead of an error. For example, a - POST using `--data @filenametypo` would cause a zero length POST which - is probably not what the user intended. +- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` - Closes https://github.com/curl/curl/pull/11677 + Based on existing autotools logic. -- hostip: fix typo + Ref: #11964 (effort to sync cmake detections with autotools) -Davide Masserut (29 Aug 2023) + Closes #11981 -- tool: avoid including leading spaces in the Location hyperlink +- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` - Co-authored-by: Dan Fandrich + Based on existing autotools logic. - Closes #11735 + autotools checks for old versions of the allowlisted target OSes and + disables this feature when seeing them. In CMake we assume we're running + on newer systems and enable regardless of OS version. -Daniel Stenberg (29 Aug 2023) + autotools always runs all 3 probes for non-fast-tracked systems and + enables this feature if any one of them was successful. To save + configuration time, CMake stops at the first successful check. -- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline + OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD + system. I haven't double-checked if this is correct, but looks odd. - Closes #11757 + Ref: #11964 (effort to sync cmake detections with autotools) -- connect: stop halving the remaining timeout when less than 600 ms left + Closes #11979 - When curl wants to connect to a host, it always has a TIMEOUT. The - maximum time it is allowed to spend until a connect is confirmed. +- cmake: fix `HAVE_WRITABLE_ARGV` detection - curl will try to connect to each of the IP adresses returned for the - host. Two loops, one for each IP family. + Move detection before the creation of detection results in + `curl_config.h`. - During the connect loop, while curl has more than one IP address left to - try within a single address family, curl has traditionally allowed (time - left/2) for *this* connect attempt. This, to not get stuck on the - initial addresses in case the timeout but still allow later addresses to - get attempted. + Ref: #11964 (effort to sync cmake detections with autotools) - This has the downside that when users set a very short timeout and the - host has a large number of IP addresses, the effective result might be - that every attempt gets a little too short time. + Closes #11978 - This change stop doing the divided-by-two if the total time left is - below a threshold. This threshold is 600 milliseconds. +- appveyor: minor improvements - Closes #11693 + - run `curl -V` after builds to see if they run and with what features. + Except for one job where a CRT DLL is missing. And ARM64 which should + fail, but is silently not launched instead. -- asyn-ares: reduce timeout to 2000ms + - copy libcurl DLL next to curl tool and tests binaries in shared mode. + This makes it possible to run the tests. (We don't run tests after + these builds yet.) - When UDP packets get lost this makes for slightly faster retries. This - lower timeout is used by @c-ares itself by default starting next - release. + - list the DLLs and EXEs present after the builds. - Closes #11753 + - add `DEBUG` variable for CMake builds to allow disabling it, for + testing non-debug builds. (currently enabled for all) -John Bampton (29 Aug 2023) + - add commented lines that dump CMake configuration logs for debugging + build/auto-detection issues. -- misc: remove duplicate words + - add gcc version to jobs where missing. - Closes #11740 + - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 + to the build mix. -Daniel Stenberg (29 Aug 2023) + - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. -- RELEASE-NOTES: synced + - delete a duplicate backslash. -- wolfSSL: avoid the OpenSSL compat API when not needed + Closes #11976 - ... and instead call wolfSSL functions directly. +- configure: replace adhoc domain with `localhost` in tests - Closes #11752 + Reviewed-by: Daniel Stenberg + Closes #11988 -Viktor Szakats (28 Aug 2023) +- tidy-up: use more example domains -- lib: fix null ptr derefs and uninitialized vars (h2/h3) + Also make use of the example TLD: + https://en.wikipedia.org/wiki/.example - Fixing compiler warnings with gcc 13.2.0 in unity builds. + Reviewed-by: Daniel Stenberg + Closes #11992 - Assisted-by: Jay Satiro - Assisted-by: Stefan Eissing - Closes #11739 +Dan Fandrich (29 Sep 2023) -Jay Satiro (28 Aug 2023) +- runtests: display the test status if tests appear hung -- secureserver.pl: fix stunnel version parsing + It sometimes happens that a test hangs during a test run and never + returns. The test harness will wait indefinitely for the results and on + CI servers the CI job will eventually be killed after an hour or two. + At the end of a test run, if results haven't come in within a couple of + minutes, display the status of all test runners and what tests they're + running to help in debugging the problem. - - Allow the stunnel minor-version version part to be zero. + This feature is really only kick in with parallel testing enabled, which + is fine because without parallel testing it's usually easy to tell what + test has hung. - Prior to this change with the stunnel version scheme of . - if either part was 0 then version parsing would fail, causing - secureserver.pl to fail with error "No stunnel", causing tests that use - the SSL protocol to be skipped. As a practical matter this bug can only - be caused by a minor-version part of 0, since the major-version part is - always greater than 0. + Closes #11980 - Closes https://github.com/curl/curl/pull/11722 +- github/labeler: remove workaround for labeler -- secureserver.pl: fix stunnel path quoting + This was added due to what seemed to be a bug regarding the sync-labels: + config option, but it looks like it wasn't necessary. - - Store the stunnel path in the private variable $stunnel unquoted and - instead quote it in the command strings. + Follow-up to b2b0534e7 - Prior to this change the quoted stunnel path was passed to perl's file - operators which cannot handle quoted paths. For example: +Viktor Szakats (29 Sep 2023) - $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; - if(-x $stunnel or -x "$stunnel") - # false even if path exists and is executable +- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] - Our other test scripts written in perl, unlike this one, use servers.pm - which has a global $stunnel variable with the path stored unquoted and - therefore those scripts don't have this problem. +Daniel Stenberg (29 Sep 2023) - Closes https://github.com/curl/curl/pull/11721 +- docs: replace made up domains with example.com -Daniel Stenberg (28 Aug 2023) + in FAQ and MANUAL.md -- altsvc: accept and parse IPv6 addresses in response headers + - example.com was made for this purpose. - Store numerical IPv6 addresses in the alt-svc file with the brackets - present. + - reduces the risk that one of those domains suddenly start hosting + something nasty and we provide links to them - Verify with test 437 and 438 + Closes #11986 - Fixes #11737 - Reported-by: oliverpool on github - Closes #11743 +Michael Osipov (29 Sep 2023) -- libtest: use curl_free() to free libcurl allocated data +- acinclude.m4: Document proper system truststore on FreeBSD - In several test programs. These mistakes are not detected or a problem - as long as memdebug.h is included, as that provides the debug wrappers - for all memory functions in the same style libcurl internals do it, - which makes curl_free and free effectively the same call. + The default system truststore on FreeBSD has been /etc/ssl/certs for many + years now. It is managed canonically through certctl(8) and contains hashed + symlinks for OpenSSL and other TLS providers. + The previous ones require security/ca_root_nss which might not be installed o + r + will not contain any custom CA certificates. - Reported-by: Nicholas Nethercote - Closes #11746 + Closes #11985 -Jay Satiro (28 Aug 2023) +Daniel Stenberg (29 Sep 2023) -- disable.d: explain --disable not implemented prior to 7.50.0 +- FAQ: How do I upgrade curl.exe in Windows? - Option -q/--disable was added in 5.0 but only -q was actually - implemented. Later --disable was implemented in e200034 (precedes - 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). + This is a growing question, better answer it here to get somewhere to + point users to. - Reported-by: pszlazak@users.noreply.github.com + Closes #11984 - Fixes https://github.com/curl/curl/issues/11710 - Closes #11712 +Viktor Szakats (28 Sep 2023) -Nicholas Nethercote (28 Aug 2023) +- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC -- hyper: fix ownership problems + `basename` is present in mingw-w64, missing from MSVC. Pre-cache + accordingly to make configure faster. - Some of these changes come from comparing `Curl_http` and - `start_CONNECT`, which are similar, and adding things to them that are - present in one and missing in another. + Notice that `basename` has a bug so we later disable it even with + mingw-w64: + https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li + b/curl_setup.h#L820-L825 - The most important changes: - - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the - happy path. - - In `start_CONNECT`, add a missing `hyper_request_free` on the error - path. - - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. - - In `bodysend`, remove an unnecessary `hyper_body_free` on a different - error path that would cause a double-free. - https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html - says of `hyper_request_set_body`: "This takes ownership of the - hyper_body *, you must not use it or free it after setting it on the - request." This is true even if `hyper_request_set_body` returns an - error; I confirmed this by looking at the hyper source code. + Closes #11974 - Other changes are minor but make things slightly nicer. +Daniel Stenberg (28 Sep 2023) - Closes #11745 +- cmake: add missing checks -Daniel Stenberg (28 Aug 2023) + - check for arc4random. To make rand.c use it accordingly. + - check for fcntl + - fix fseek detection + - add SIZEOF_CURL_SOCKET_T + - fix USE_UNIX_SOCKETS + - define HAVE_SNPRINTF to 1 + - check for fnmatch + - check for sched_yield + - remove HAVE_GETPPID duplicate from curl_config.h + - add HAVE_SENDMSG -- multi.h: the 'revents' field of curl_waitfd is supported + Ref: #11964 - Since 6d30f8ebed34e7276 + Co-authored-by: Viktor Szakats + Closes #11973 - Reported-by: Nicolás Ojeda Bär - Ref: #11748 - Closes #11749 +- configure: remove unused checks -Gerome Fournier (27 Aug 2023) + - for sys/uio.h + - for fork + - for connect -- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() + Ref: #11964 - Closes #11742 + Closes #11973 -Daniel Stenberg (27 Aug 2023) +- lib: remove TIME_WITH_SYS_TIME -- docs: mention critical files in same directories as curl saves + It is not used in any code anywhere. - ... cannot be fully protected. Don't do it. + Ref: #11964 + Closes #11975 - Co-authored-by: Jay Satiro - Reported-by: Harry Sintonen - Fixes #11530 - Closes #11701 +- docs: update curl man page references -John Hawthorn (26 Aug 2023) + Detected by the manpage-syntax update -- OpenSSL: clear error queue after SSL_shutdown + Closes #11963 - We've seen errors left in the OpenSSL error queue (specifically, - "shutdown while in init") by adding some logging it revealed that the - source was this file. +- manpage-syntax: verify curl man page references - Since we call SSL_read and SSL_shutdown here, but don't check the return - code for an error, we should clear the OpenSSL error queue in case one - was raised. + 1. References to curl symbols are now checked that they indeed exist as + man pages. This for \f references as well as the names referenced in the + SEE ALSO section. - This didn't affect curl because we call ERR_clear_error before every - write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when - libcurl is used in a process with other OpenSSL users, they may detect - an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their - own. + Allowlist curl.1 since it is not always built in builds - Co-authored-by: Satana de Sant'Ana + 2. References to curl symbols that lack section now causes warning, since tha + t + will prevent them from getting linked properly - Closes #11736 + 3. Check for "bare" references to curl functions and warn, they should be + references -Alexander Kanavin (25 Aug 2023) + Closes #11963 -- tests: update cookie expiry dates to far in the future +- cmake: add check for suseconds_t - This allows testing Y2038 with system time set to after that, so that - actual Y2038 issues can be exposed, and not masked by expiry errors. + And fix the HAVE_LONGLONG define - Fixes #11576 - Closes #11610 + Ref: #11964 + Closes #11977 -John Bampton (25 Aug 2023) +Viktor Szakats (28 Sep 2023) -- misc: fix spelling +- tidy-up: whitespace fixes - Closes #11733 + Closes #11972 -Daniel Stenberg (25 Aug 2023) +- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS -- cmdline-opts/page-header: clarify stronger that !opt == URL + With new option `CURL_DISABLE_SRP=ON` to force-disable it. + To match existing option and detection logic in autotools. - Everything provided on the command line that is not an option (or an - argument to an option) is treated as a URL. + Also: + - fix detecting GnuTLS. + We assume `nettle` as a GnuTLS dependency. + - add CMake GnuTLS CI job. + - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) + TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. + - fix compiler warning when building with GnuTLS and disabled TLS-SRP. + - fix comment typos, whitespace. - Closes #11734 + Ref: #11964 -- tests/runner: fix %else handling + Closes #11967 - Getting the show state proper for %else and %endif did not properly work - in nested cases. +- tool: use our own stderr variable - Follow-up to 3d089c41ea9 + Earlier this year we changed our own stderr variable to use the standard + name `stderr` (to avoid bugs where someone is using `stderr` instead of + the curl-tool specific variable). This solution needed to override the + standard `stderr` symbol via the preprocessor. This in turn didn't play + well with unity builds and caused curl tool to crash or stay silent due + to an uninitialized stderr. This was a hard to find issue, fixed by + manually breaking out one file from the unity sources. - Closes #11731 + To avoid two these two tricks, this patch implements a different + solution: Restore using our own local variable for our stderr output and + leave `stderr` as-is. To avoid using `stderr` by mistake, add a + `checksrc` rule (based on logic we already used in lib for `strerror`) + that detects any `stderr` use in `src` and points to using our own + variable instead: `tool_stderr`. -Nicholas Nethercote (25 Aug 2023) + Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e + Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db -- docs: Remove mention of #10803 from `KNOWN_BUGS`. + Closes #11958 - Because the leaks have been fixed. +Loïc Yhuel (28 Sep 2023) -- c-hyper: fix another memory leak in `Curl_http`. +- connect: only start the happy eyeballs timer when needed - There is a `hyper_clientconn_free` call on the happy path, but not one - on the error path. This commit adds one. + The timeout is only used when there is a second address family, for the + delayed eyeballer. - Fixes the second memory leak reported by Valgrind in #10803. + Closes #11939 - Fixes #10803 - Closes #11729 +Daniel Stenberg (28 Sep 2023) -- c-hyper: fix a memory leak in `Curl_http`. +- tool_operate: free 'gateway' correctly - A request created with `hyper_request_new` must be consumed by either - `hyper_clientconn_send` or `hyper_request_free`. + Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. - This is not terrifically clear from the hyper docs -- - `hyper_request_free` is documented only with "Free an HTTP request if - not going to send it on a client" -- but a perusal of the hyper code - confirms it. + Also removed repeated wording in IPFS related error messages. - This commit adds a `hyper_request_free` to the `error:` path in - `Curl_http` so that the request is consumed when an error occurs after - the request is created but before it is sent. + Closes #11969 - Fixes the first memory leak reported by Valgrind in #10803. +Stefan Eissing (28 Sep 2023) - Closes #11729 +- lib: move handling of `data->req.writer_stack` into Curl_client_write() -Daniel Stenberg (25 Aug 2023) + - move definitions from content_encoding.h to sendf.h + - move create/cleanup/add code into sendf.c + - installed content_encoding writers will always be called + on Curl_client_write(CLIENTWRITE_BODY) + - Curl_client_cleanup() frees writers and tempbuffers from + paused transfers, irregardless of protocol -- RELEASE-NOTES: synced + Closes #11908 -John Bampton (25 Aug 2023) +Loïc Yhuel (28 Sep 2023) -- misc: spellfixes +- multi: round the timeout up to prevent early wakeups - Closes #11730 + Curl_timediff rounds down to the millisecond, so curl_multi_perform can + be called too early, then we get a timeout of 0 and call it again. -Daniel Stenberg (25 Aug 2023) + The code already handled the case of timeouts which expired less than + 1ms in the future. By rounding up, we make sure we will never ask the + platform to wake up too early. -- tests: add support for nested %if conditions + Closes #11938 - Provides more flexiblity to test cases. +Daniel Stenberg (28 Sep 2023) - Also warn and bail out if there is an '%else' or %endif' without a - preceeding '%if'. +- RELEASE-NOTES: spell out that IPFS is via gateway - Ref: #11610 - Closes #11728 +- RELEASE-NOTES: synced -- time-cond.d: mention what happens on a missing file +- tool_operate: avoid strlen() -1 on zero length content from file - Closes #11727 + Follow-up to 65b563a96a226649ba12cb1e -Christian Hesse (24 Aug 2023) + Closes #11959 -- docs/cmdline-opts: match the current output +- tool_operate: fix memory mixups - The release date has been added in output, reflect that in documentation. + Switch to plain getenv() from curl_getenv() to avoid the allocation and + having to keep track of which free() or curl_free() that need to be + used. - Closes #11723 + Coverity found issues and a memory leak. -Daniel Stenberg (24 Aug 2023) + Follow-up to 65b563a96a226649ba12cb1e -- lib: minor comment corrections + Closes #11959 -- docs: rewrite to present tense +Viktor Szakats (27 Sep 2023) - ... instead of using future tense. +- curl-functions.m4: fixup recent bad edits - + numerous cleanups and improvements - + stick to "reuse" not "re-use" - + fewer contractions + Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Closes #11713 + Closes #11966 -- urlapi: setting a blank URL ("") is not an ok URL +Daniel Stenberg (27 Sep 2023) - Test it in 1560 - Fixes #11714 - Reported-by: ad0p on github - Closes #11715 +- curl-functions.m4: fix include line -- spelling: use 'reuse' not 're-use' in code and elsewhere + This made the getaddrinfo detection fail, but we did not spot it in the + CI because it graciously falled back to using legacy functions instead! - Unify the spelling as both versions were previously used intermittently + Follow-up to 96c29900bcec (#11940) - Closes #11717 + Closes #11965 -Michael Osipov (23 Aug 2023) +- inet_ntop: add typecast to silence Coverity -- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC + CID 1024653: Integer handling issues (SIGN_EXTENSION) - HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the - default one. Use "long long" in 32 bit mode and just "long" in 64 bit - mode. + Suspicious implicit sign extension: "src[i]" with type "unsigned char + const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to + type "int" (32 bits, signed), then sign-extended to type "unsigned long" + (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than + 0x7FFFFFFF, the upper bits of the result will all be 1. - Closes #11718 + 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); -Dan Fandrich (22 Aug 2023) + The value will not be greater than 0x7FFFFFFF so this still cannot + happen. -- tests: don't call HTTP errors OK in test cases + Also, switch to ints here instead of longs. The values stored are 16 bit + so at least no need to use 64 bit variables. Also, longs are 32 bit on + some platforms so this logic still needs to work with 32 bits. - Some HTTP errors codes were accompanied by the text OK, which causes - some cognitive dissonance when reading them. + Closes #11960 -- http: close the connection after a late 417 is received +- docs: adapt SEE ALSO sections to new requirements - In this situation, only part of the data has been sent before aborting - so the connection is no longer usable. + To please manpage-syntax.pl used by test 1173 - Assisted-by: Jay Satiro - Fixes #11678 - Closes #11679 + Closes #11957 -- runtests: slightly increase the longest log file displayed +- manpage-syntax.pl: verify SEE ALSO syntax - The new limit provides enough space for a 64 KiB data block to be logged - in a trace file, plus a few lines at the start and end for context. This - happens to be the amount of data sent at a time in a PUT request. + - Enforce a single reference per .BR line + - Skip the quotes around the section number for example (3) + - Insist on trailing commas on all lines except the last + - Error on comma on the last SEE ALSO entry -- tests: add delay command to the HTTP server + - List the entries alpha-sorted, not enforced just recommended - This adds a delay after client connect. + Closes #11957 -Daniel Stenberg (22 Aug 2023) +- connect: expire the timeout when trying next -- cirrus: install everthing with pkg, avoid pip + ... so that it gets called again immediately and can continue trying + addresses to connect to. Otherwise it might unnecessarily wait for a + while there. - Assisted-by: Sevan Janiyan + Fixes #11920 + Reported-by: Loïc Yhuel + Closes #11935 - Closes #11711 +- http: remove wrong comment for http_should_fail -- curl_url*.3: update function descriptions + Reported-by: Christian Schmitz + Ref: #11936 + Closes #11941 - - expand and clarify several descriptions - - avoid using future tense all over +Dan Fandrich (26 Sep 2023) - Closes #11708 +- tool_setopt: remove unused function tool_setopt_flags -- RELEASE-NOTES: synced + This function is identical to tool_setopt_bitmask except that it treats + the argument as unsigned. -Stefan Eissing (21 Aug 2023) + Closes #11943 -- CI/cirrus: disable python install on FreeBSD +Viktor Szakats (26 Sep 2023) - - python cryptography package does not build build FreeBSD - - install just mentions "error" - - this gets the build and the main test suite going again +- cmake: add feature checks for `memrchr` and `getifaddrs` - Closes #11705 + - `HAVE_MEMRCHR` for `memrchr`. + - `HAVE_GETIFADDRS` for `getifaddrs`. + This was present in `lib/curl_config.h.cmake` but missed the detection + logic. -- test2600: fix flakiness on low cpu + To match existing autotools feature checks. - - refs #11355 where failures to to low cpu resources in CI - are reported - - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations - to test cases - - trigger Curl_expire() in test filter to allow re-checks before - the usual 1second interval + Closes #11954 - Closes #11690 +- cmake: move global headers to specific checks -Maksim Sciepanienka (20 Aug 2023) + Before this patch we added standard headers unconditionally to the + global list of headers used for feature checks. This is unnecessary + and also doesn't help CMake 'Generate' performance. This patch moves + these headers to each feature check where they are actually needed. + Stop using `stddef.h`, as it seems unnecessary. -- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf + I've used autotools' `m4/curl-functions.m4` to figure out these + dependencies. - Closes #11698 + Also delete checking for the C89 standard header `time.h`, that I + missed in the earlier commit. -Daniel Stenberg (20 Aug 2023) + Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 -- test687/688: two more basic --xattr tests + Closes #11951 - Closes #11697 +- src/mkhelp: make generated code pass `checksrc` -- cmdline-opts/docs: mentioned the negative option part + Closes #11955 - ... for --no-alpn and --no-buffer in the same style done for other --no- - options: +- tests: show which curl tool `runtests.pl` is using - "Note that this is the negated option name documented." + To help debugging when there is issue finding or running it. - Closes #11695 + Closes #11953 -Emanuele Torre (19 Aug 2023) +- CI/azure: make `MAKEFLAGS` global to parallelize all jobs -- tool/var: also error when expansion result starts with NUL + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi + th -j3) - Expansions whose output starts with NUL were being expanded to the empty - string, and not being recognised as values that contain a NUL byte, and - should error. + Closes #11952 - Closes #11694 +- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 -Daniel Stenberg (19 Aug 2023) + Also delete an accidental variable reference. -- tests: add 'large-time' as a testable feature + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 - This allows test cases to require this feature to run and to be used in - %if conditions. + Closes #11945 - Large here means larger than 32 bits. Ie does not suffer from y2038. +Daniel Stenberg (26 Sep 2023) - Closes #11696 +- docs: add see also curl_multi_get_handles to some man pages -- tests/Makefile: add check-translatable-options.pl to tarball + Assisted-by: Jay Satiro - Used in test 1544 + Closes #11942 - Follow-up to ae806395abc8c +Viktor Szakats (26 Sep 2023) -- gen.pl: fix a long version generation mistake +- cmake: assume `_fseeki64` and no `fseeko` on Windows - Too excessive escaping made the parsing not find the correct long names - later and instead add "wrong" links. + `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and + at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. - Follow-up to 439ff2052e219 + `fseeko` is not present in any of these. - Reported-by: Lukas Tribus - Fixes #11688 - Closes #11689 + (mingw-w64 1.0 also offers `fseeko64`.) -- lib: move mimepost data from ->req.p.http to ->state + [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 - When the legacy CURLOPT_HTTPPOST option is used, it gets converted into - the modem mimpost struct at first use. This data is (now) kept for the - entire transfer and not only per single HTTP request. This re-enables - rewind in the beginning of the second request instead of in end of the - first, as brought by 1b39731. + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 - The request struct is per-request data only. + Closes #11950 - Extend test 650 to verify. +- build: delete checks for C89 standard headers - Fixes #11680 - Reported-by: yushicheng7788 on github - Closes #11682 + Delete checks and guards for standard C89 headers and assume these are + available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, + `stddef.h`, `signal.h`. -Patrick Monnerat (17 Aug 2023) + Some of these we already used unconditionally, some others we only used + for feature checks. -- os400: do not check translatable options at build time + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i + n CMake) - Now that there is a test for this, the build time check is not needed - anymore. + Closes #11940 - Closes #11650 +Stefan Eissing (26 Sep 2023) -- test1554: check translatable string options in OS400 wrapper +- multiif.h: remove Curl_multi_dump declaration - This test runs a perl script that checks all string options are properly - translated by the OS400 character code conversion wrapper. It also - verifies these options are listed in alphanumeric order in the wrapper - switch statement. + Follow-up to d850eea2 which removed the Curl_multi_dump definition. - Closes #11650 + Closes https://github.com/curl/curl/pull/11946 -Daniel Stenberg (17 Aug 2023) +Jay Satiro (26 Sep 2023) -- unit3200: skip testing if function is not present +- config-win32: define HAVE__FSEEKI64 - Fake a successful run since we have no easy mechanism to skip this test - for this advanced condition. + Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that + calls _fseeki64 if it is available. -- unit2600: fix build warning if built without verbose messages + Closes https://github.com/curl/curl/pull/11944 -- test1608: make it build and get skipped without shuffle DNS support +- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER -- lib: --disable-bindlocal builds curl without local binding support + - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place + even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. -- test1304: build and skip without netrc support + The behavior is verified by test2048. -- lib: build fixups when built with most things disabled + Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 + Reported-by: claudiusaiz@users.noreply.github.com - Closes #11687 + Bug: https://github.com/curl/curl/discussions/11910 + Reported-by: Hakan Sunay Halil -- workflows/macos.yml: disable zstd and alt-svc in the http-only build + Closes https://github.com/curl/curl/pull/11930 - Closes #11683 +Stefan Eissing (26 Sep 2023) -Stefan Eissing (17 Aug 2023) +- openssl: improve ssl shutdown handling -- bearssl: handshake fix, provide proper get_select_socks() implementation + - If SSL shutdown is not finished then make an additional call to + SSL_read to gather additional tracing. - - bring bearssl handshake times down from +200ms down to other TLS backends - - vtls: improve generic get_select_socks() implementation - - tests: provide Apache with a suitable ssl session cache + - Fix http2 and h2-proxy filters to forward do_close() calls to the next + filter. - Closes #11675 + For example h2 and SSL shutdown before and after this change: -- tests: TLS session sharing test + Before: - - test TLS session sharing with special test client - - expect failure with wolfSSL - - disable flaky wolfSSL test_02_07b + Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> + ssl_cf_destroy - Closes #11675 + After: -Daniel Stenberg (17 Aug 2023) + Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> + ssl_cf_close -- CURLOPT_*TIMEOUT*: extend and clarify + Note that currently the tracing does not show output on the connection + closure handle. Refer to discussion in #11878. - Closes #11686 + Ref: https://github.com/curl/curl/discussions/11878 -- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails + Closes https://github.com/curl/curl/pull/11858 - And document it. Only return out of memory when it actually is a memory - problem. +Loïc Yhuel (26 Sep 2023) - Pointed-out-by: Jacob Mealey - Closes #11674 +- multi: fix small timeouts -Mathew Benson (17 Aug 2023) + Since Curl_timediff rounds down to the millisecond, timeouts which + expire in less than 1ms are considered as outdated and removed from the + list. We can use Curl_timediff_us instead, big timeouts could saturate + but this is not an issue. -- cmake: add GnuTLS option + Closes #11937 - - Option to use GNUTLS was missing. Hence was not able to use GNUTLS - with ngtcp2 for http3. +Viktor Szakats (25 Sep 2023) - Closes #11685 +- cmake: fix stderr initialization in unity builds -Daniel Stenberg (16 Aug 2023) + Before this patch, in certain build configurations the curl tool may + not have displayed anything (debug, macOS), or crashed at startup + (debug, Windows). -- RELEASE-NOTES: synced + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 + Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db -- http: remove the p_pragma struct field + Closes #11929 - unused since 40e8b4e52 (2008) +- cmake: fix missing `zlib.h` when compiling `libcurltool` - Closes #11681 + Came up while testing debug/testing build for Windows. I'm not sure why + it didn't come up in earlier tests with similar config. + `tool_hugehelp.c` might indeed require `zlib.h` and without linking + `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected + dependency headers to the compiler command. -Jay Satiro (16 Aug 2023) + ``` + [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang + --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 + 6_64 + -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO + DE + -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib + -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li + ne-argument + -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static + -libgcc + -lucrt [...] -O3 -DNDEBUG -municode -MD + -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d + -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x + 64/src/tool_hugehelp.c + .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f + ile not found + 6 | #include + | ^~~~~~~~ + ``` -- CURLINFO_CERTINFO.3: better explain curl_certinfo struct + Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 - Closes https://github.com/curl/curl/pull/11666 + Closes #11927 -- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation +- cmake: fix duplicate symbols when linking tests - - Remove the out-of-date SSL backend list supported by - CURLOPT_SSL_CTX_FUNCTION. + The linker resolves this automatically in non-unity builds. In unity + builds the linker cannot drop a single object with the duplicates, + resulting in these errors. The root issue is that we started including + certain objects both via both libcurlu and libcurltool libs. - It makes more sense to just refer to that document instead of having - a separate list that has to be kept in sync. + Regression from 39e7c22bb459c2e818f079984989a26a09741860 - Closes https://github.com/curl/curl/pull/11665 + Windows errors: + ``` + [ 3%] Linking C executable unit1303.exe + [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul + tibyte.c.obj + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch + ar': + C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve + rt_UTF8_to_wchar' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:44: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT + F8': + C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve + rt_wchar_to_UTF8' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:66: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': + C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 + _open' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:92: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': + C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 + 2_fopen' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:120: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': + [...] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 + aa4ehny5q#L247 -- write-out.d: clarify %{time_starttransfer} + macOS errors: + ``` + [ 56%] Linking C executable unit1302 + duplicate symbol '_curlx_sotouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_sitouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_uztosz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + [...] + ``` + with config: + ``` + -DCMAKE_UNITY_BUILD=ON \ + -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ + -DBUILD_SHARED_LIBS=ON \ + -DBUILD_STATIC_LIBS=OFF + ``` - sync it up with CURLINFO_STARTTRANSFER_TIME_T + Closes #11926 -Daniel Stenberg (15 Aug 2023) +- cmake: lib `CURL_STATICLIB` fixes (Windows) -- transfer: don't set TIMER_STARTTRANSFER on first send + - always define `CURL_STATICLIB` when building libcurl for Windows. - The time stamp is for measuring the first *received* byte + This disables `__declspec(dllexport)` for exported libcurl symbols. + In normal mode (hide symbols) these exported symbols are specified + via `libcurl.def`. When not hiding symbols, all symbols are exported + by default. - Fixes #11669 - Reported-by: JazJas on github - Closes #11670 + Regression from 1199308dbc902c52be67fc805c72dd2582520d30 -trrui-huawei (15 Aug 2023) + Fixes #11844 -- quiche: enable quiche to handle timeout events + - fix to omit `libcurl.def` when not hiding private symbols. - In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` - interface for the application to invoke upon timer - expiration. Therefore, invoking the `on_timeout` function of the - Connection is crucial to ensure seamless functionality of quiche with - timeout events. + Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 - Closes #11654 + - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also + omitting `libcurl.def` in this case, and exporting all symbols + instead. This ensures that a shared curl tool can access all debug + functions which are not normally exported from libcurl DLL. -- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s + - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" + target. - Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 -Daniel Stenberg (15 Aug 2023) + - delete duplicate `BUILDING_LIBCURL` definitions. -- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang + - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. - Closes #9580 + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 -- imap: add a check for failing strdup() + Closes #11914 -- imap: remove the only sscanf() call in the IMAP code +Daniel Stenberg (25 Sep 2023) - Avoids the use of a stack buffer. +- RELEASE-NOTES: synced - Closes #11673 +Dan Fandrich (25 Sep 2023) -- imap: use a dynbuf in imap_atom +- tests: fix log directory path in IPFS tests - Avoid a calculation + malloc. Build the output in a dynbuf. + Hard-coding the log directory name fails with parallel tests. - Closes #11672 + Follow-up to 65b563a96 -Marin Hannache (14 Aug 2023) + Ref: #8805 -- http: do not require a user name when using CURLAUTH_NEGOTIATE +Daniel Stenberg (25 Sep 2023) - In order to get Negotiate (SPNEGO) authentication to work in HTTP you - used to be required to provide a (fake) user name (this concerned both - curl and the lib) because the code wrongly only considered - authentication if there was a user name provided, as in: +- curl_multi_get_handles: get easy handles from a multi handle - curl -u : --negotiate https://example.com/ + Closes #11750 - This commit leverages the `struct auth` want member to figure out if the - user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of - setting a user name both in curl and the lib. +Stefan Eissing (25 Sep 2023) - Signed-off-by: Marin Hannache - Reported-by: Enrico Scholz - Fixes https://sourceforge.net/p/curl/bugs/440/ - Fixes #1161 - Closes #9047 +- http: h1/h2 proxy unification -Viktor Szakats (13 Aug 2023) + - use shared code for setting up the CONNECT request + when tunneling, used in HTTP/1.x and HTTP/2 proxying + - eliminate use of Curl_buffer_send() and other manipulations + of `data->req` or `data->state.ulbuf` -- build: streamline non-UWP wincrypt detections + Closes #11808 - - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build - and disable our non-UWP-compatible use the Windows crypto API. This - allows to drop two dynamic feature checks. +Natanael Copa (25 Sep 2023) - `WINDOWS_STORE` is true when invoking CMake with - `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. +- lib: use wrapper for curl_mime_data fseek callback - Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html + fseek uses long offset which does not match with curl_off_t. This leads + to undefined behavior when calling the callback and caused failure on + arm 32 bit. - - with autotools, drop the separate feature check for `wincrypt.h`. On - one hand this header has been present for long (even Borland C 5.5 had - it from year 2000), on the other we used the check result solely to - enable another check for certain crypto functions. This fails anyway - with the header not present. We save one dynamic feature check at the - configure stage. + Use a wrapper to solve this and use fseeko which uses off_t instead of + long. - Reviewed-by: Marcel Raad - Closes #11657 + Thanks to the nice people at Libera IRC #musl for helping finding this + out. -Nicholas Nethercote (13 Aug 2023) + Fixes #11882 + Fixes #11900 + Closes #11918 -- docs/HYPER.md: update hyper build instructions +- configure: sort AC_CHECK_FUNCS - Nightly Rust and `-Z unstable-options` are not needed. + No functional changes. - The instructions here now match the hyper docs exactly: - https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 - 6f88 +Daniel Stenberg (25 Sep 2023) - Closes #11662 +- warnless: remove unused functions -Daniel Stenberg (13 Aug 2023) + Previously put there for use with the intel compiler -- RELEASE-NOTES: synced + Closes #11932 -- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name +- GHA/linux: run singleuse to detect single-use global functions - Asssisted-by: Jay Satiro - Closes #11655 + Use --unit for configure --enable-debug builds -- spellcheck: adapt to backslashed minuses + Closes #11932 - As the curl.1 has more backslashed minus, the cleanup sed lines xneed to - adapt. +- singleuse: add scan for use in other source codes - Adjusted some docs slighly. + This should reduce false-positive to almost zero. Checks for presence in + unit tests if --unit is specified, which is intended for debug builds + where unit testing is enabled. - Follow-up to 439ff2052e + Closes #11932 - Closes #11663 +- multi: remove Curl_multi_dump -- gen: escape more minus + A debug-only function that is basically never used. Removed to ease the + use of the singleuse script to detect non-static functions not used + outside the file where it is defined. - Detected since it was still hard to search for option names using dashes - in the middle in the man page. + Closes #11931 - Closes #11660 +Viktor Szakats (24 Sep 2023) -- cookie-jar.d: enphasize that this option is ONLY writing cookies +- tests: fix compiler warnings - Reported-by: Dan Jacobson - Tweaked-by: Jay Satiro - Ref: #11642 - Closes #11661 + Seen with llvm 17 on Windows x64. -Nicholas Nethercote (11 Aug 2023) + ``` + .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration + for non-static variable 'logdir' [-Wmissing-variable-declarations] + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration + for non-static variable 'loglockfile' [-Wmissing-variable-declarations] + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat + ion for non-static variable 'logdir' [-Wmissing-variable-declarations] + 43 | const char *logdir = "log"; + | ^ + .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl + e is not intended to be used outside of this translation unit + 43 | const char *logdir = "log"; + | ^ + .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator + here [-Wcomma] + 350 | ++d, ++s; + | ^ + .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa + rning + 350 | ++d, ++s; + | ^~~ + | (void)( ) + ``` -- docs/HYPER.md: document a workaround for a link error + ``` + .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; + | ~ ^ ~~~~~~~~~~~~~ + 1 warning generated. - Closes #11653 + .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison + 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] + 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. -Jay Satiro (11 Aug 2023) + .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > + 2147483647 is always false [-Wtautological-type-limit-compare] + 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + ``` -- schannel: verify hostname independent of verify cert + Seen on macOS Intel: + ``` + .../curl/tests/server/sws.c:440:64: warning: field precision should have type + 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] + msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" + , + ~~^~ + 1 warning generated. + ``` - Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off - and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the - hostname in schannel code. + Closes #11925 - This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and - verify hostname". We discussed a fix several years ago in #3285 but it - went stale. +Jay Satiro (24 Sep 2023) - Assisted-by: Daniel Stenberg +- url: fix netrc info message - Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html - Reported-by: Martin Galvan + - Fix netrc info message to use the generic ".netrc" filename if the + user did not specify a netrc location. - Ref: https://github.com/curl/curl/pull/3285 + - Update --netrc doc to add that recent versions of curl on Windows + prefer .netrc over _netrc. - Fixes https://github.com/curl/curl/issues/3284 - Closes https://github.com/curl/curl/pull/10056 + Before: + * Couldn't find host google.com in the (nil) file; using defaults -Daniel Stenberg (11 Aug 2023) + After: + * Couldn't find host google.com in the .netrc file; using defaults -- curl_quiche: remove superfluous NULL check + Closes https://github.com/curl/curl/pull/11904 - 'stream' is always non-NULL at this point +Dan Fandrich (23 Sep 2023) - Pointed out by Coverity +- wolfssh: do cleanup in Curl_ssh_cleanup - Closes #11656 + Closes: #11921 -- curl/urlapi.h: tiny typo +Daniel Stenberg (24 Sep 2023) -- github/labeler: make HYPER.md set Hyper and not TLS +- tool_listhelp: regenerated -- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 + Polished the --ipfs-gateway description - 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need - to specify version changes for earlier releases in the generated output. + Fixed the --trace-config description - This ups the limit from the previous 7.30.0 (Apr 12 2013) + The script also fixed some other small mistakes - This hides roughly 35 "added in" mentions. + Closes #11923 - Closes #11651 +Viktor Szakats (23 Sep 2023) -Jay Satiro (10 Aug 2023) +- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) -- bug_report: require reporters to specify curl and os versions + Also fix to export all symbols in Windows debug builds, making + `-debug-dyn` builds work with `-DCURL_STATICLIB` set. - - Change curl version and os sections from single-line input to - multi-line textarea. + Ref: https://github.com/curl/curl/pull/11914 (same for CMake) - - Require curl version and os sections to be filled out before report - can be submitted. + Closes #11924 - Closes https://github.com/curl/curl/pull/11636 +Daniel Stenberg (23 Sep 2023) -Daniel Stenberg (9 Aug 2023) +- quic: set ciphers/curves the same way regular TLS does -- gen.pl: replace all single quotes with aq + for OpenSSL/BoringSSL - - this prevents man from using a unicode sequence for them - - which then allows search to work properly + Fixes #11796 + Reported-by: Karthikdasari0423 on github + Assisted-by: Jay Satiro + Closes #11836 - Closes #11645 +- test457: verify --max-filesize with chunked encoding -Viktor Szakats (9 Aug 2023) +- lib: let the max filesize option stop too big transfers too -- cmake: fix to use variable for the curl namespace + Previously it would only stop them from getting started if the size is + known to be too big then. - Replace (wrong) literal with a variable to specify the curl - namespace. + Update the libcurl and curl docs accordingly. - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 + Fixes #11810 + Reported-by: Elliot Killick + Assisted-by: Jay Satiro + Closes #11820 - Reported-by: balikalina on Github - Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 - 20d30#r123923098 - Closes #11629 +Viktor Szakats (23 Sep 2023) -- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms +- mingw: delete support for legacy mingw.org toolchain - 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing - libcurl objects for shared and static targets. + Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: + https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ + Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. + It supported the x86 CPU only and used a old Windows API header and + implib set, often causing issues. It also misses most modern Windows + features, offering old versions of both binutils and gcc (no llvm/clang + support). It was last updated 2 years ago. - The above automatically enabled for Windows builds, with an option to - disable with `SHARE_LIB_OBJECT=OFF`. + curl now relies on toolchains based on the mingw-w64 project: + https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ + https://www.msys2.org/ https://github.com/msys2/msys2 + https://github.com/mstorsjo/llvm-mingw + (Also available via Linux and macOS package managers.) - This patch extend this feature to all platforms as a manual option. - You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects - are built in PIC mode, meaning the static lib will also have PIC code. + Closes #11625 - [EXPERIMENTAL] +Mark Gaiser (23 Sep 2023) - Closes #11627 +- curl: add support for the IPFS protocols: -- cmake: assume `wldap32` availability on Windows + - ipfs:// + - ipns:// - This system library first shipped with Windows ME, available as an extra - install for some older releases (according to [1]). The import library - was present already in old MinGW 3.4.2 (year 2007). + This allows you tu use ipfs in curl like: + curl ipfs:// + and + curl ipns:// - Drop the feature check and its associated `HAVE_WLDAP32` variable. + For more information consult the readme at: + https://curl.se/docs/ipfs.html - To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` - CMake option, like before. + Closes #8805 - [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html +Daniel Stenberg (23 Sep 2023) - Reviewed-by: Jay Satiro - Closes #11624 +- bufq: remove Curl_bufq_skip_and_shift (unused) -Daniel Stenberg (9 Aug 2023) + Closes #11915 -- page-header: move up a URL paragraph from GLOBBING to URL +- scripts/singleuse.pl: add curl_global_trace -- variable.d: output the function names table style +Viktor Szakats (22 Sep 2023) - Also correct the url function name in the header +- cmake: fix unity symbol collisions in h2 builds - Closes #11641 + Regression from 331b89a319d0067fa1e6441719307cfef9c7960f -- haproxy-clientip.d: remove backticks + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #11912 - This is not markdown +Daniel Stenberg (22 Sep 2023) - Follow-up to 0a75964d0d94a4 +- RELEASE-NOTES: synced - Closes #11639 +Dan Fandrich (21 Sep 2023) -- RELEASE-NOTES: synced +- github/labeler: improve the match patterns -- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens + This includes new rules for setting the appleOS and logging labels and + matches on some example files. Also, enable dot mode for wildcard + matches in the .github directory. - Reported-by: FC Stegerman - Fixes #11635 - Closes #11637 +Daniel Stenberg (21 Sep 2023) -- cmdline-opts/page-header: reorder, clean up +- upload-file.d: describe the file name slash/backslash handling - - removed some unnecessary blurb to focus - - moved up the more important URL details - - put "globbing" into its own subtitle and moved down a little - - mention the online man page in the version section + Closes #11911 - Closes #11638 +Jakub Jelen (21 Sep 2023) -- c-hyper: adjust the hyper to curlcode conversion +- libssh: cap SFTP packet size sent - Closes #11621 + Due to libssh limitations -- test2306: make it use a persistent connection + Signed-off-by: Jakub Jelen - + enable verbose already from the start + Closes #11804 - Closes #11621 +Daniel Stenberg (21 Sep 2023) -eppesuig (8 Aug 2023) +- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 -- list-only.d: mention SFTP as supported protocol + Closes #11905 - Closes #11628 +- mailmap: unify Michael Osipov under a single email -Daniel Stenberg (8 Aug 2023) +Ted Lyngmo (21 Sep 2023) -- request.d: use .TP for protocol "labels" +- docs: use CURLSSLBACKEND_NONE - To render the section nicer in man page. + [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in + documentation and examples. - Closes #11630 + Signed-off-by: Ted Lyngmo -- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP + Closes #11909 - ... as documented. +Dan Fandrich (21 Sep 2023) - Update test 3201 and 3202 accordingly. +- github/labeler: give the sync-labels config item a default value - Reported-by: Markus Sommer - Fixes #11619 - Closes #11626 + This shouldn't be necessary and is likely a bug with this beta version + of the labeller. -- page-footer: QLOGDIR works with ngtcp2 and quiche + Also, fix the negative matches for the documentation label. - It previously said "both" backends which is confusing as we currently - have three... + Follow-up to dd12b452a + Closes #11907 - Closes #11631 +- github/labeler: fix up more the labeler config format -Stefan Eissing (8 Aug 2023) + The new version didn't like the workaround we had for a bug in the + previous labeler version, and it should no longer be needed. -- http3: quiche, handshake optimization, trace cleanup + Follow-up to dd12b452a + Closes #11906 - - load x509 store after clienthello - - cleanup of tracing +- github/labeler: fix indenting to try to appease labeller - Closes #11618 + Follow-up to dd12b452a -Daniel Stenberg (8 Aug 2023) +Jay Satiro (21 Sep 2023) -- ngtcp2: remove dead code +- libssh2: fix error message on failed pubkey-from-file - 'result' is always zero (CURLE_OK) at this point + - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error + message "SSH public key authentication failed: Reason unknown (-1)". - Detected by Coverity + When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a + generic error and therefore doesn't set an error message. AFAICT that is + not documented behavior. - Closes #11622 + Prior to this change libcurl retrieved the last set error message which + would be from a previous function failing. That resulted in misleading + auth failed error messages in verbose mode. -Viktor Szakats (8 Aug 2023) + Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 + Reported-by: consulion@users.noreply.github.com -- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` + Closes https://github.com/curl/curl/pull/11881 - OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the - popular forks (yet). Use the macro itself to detect its presence, - replacing the hard-wired fork-specific conditions. +Stefan Eissing (21 Sep 2023) - This way the feature will enable automatically when forks implement it, - while also shorter and possibly requiring less future maintenance. +- pytest: exclude test_03_goaway in CI runs due to timing dependency - Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 + Closes #11860 - Reviewed-by: Jay Satiro - Closes #11617 +- lib: disambiguate Curl_client_write flag semantics -- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 + - use CLIENTWRITE_BODY *only* when data is actually body data + - add CLIENTWRITE_INFO for meta data that is *not* a HEADER + - debug assertions that BODY/INFO/HEADER is not used mixed + - move `data->set.include_header` check into Curl_client_write + so protocol handlers no longer have to care + - add special in FTP for `data->set.include_header` for historic, + backward compatible reasons + - move unpausing of client writes from easy.c to sendf.c, so that + code is in one place and can forward flags correctly - LibreSSL 3.4.1 (2021-10-14) added support for - `SSL_CTX_set_ciphersuites`. + Closes #11885 - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt +Patrick Monnerat (21 Sep 2023) - Reviewed-by: Jay Satiro - Closes #11616 +- tftpd: always use curl's own tftp.h -- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 + Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects + and reports a stringop-overread warning: - LibreSSL 3.5.0 (2022-02-24) added support for - `SSL_CTX_set_keylog_callback`. + tftpd.c: In function ‘write_behind.isra’: + tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f + rom a region of size 0 [-Wstringop-overread] + 485 | return write(test->ofile, writebuf, count); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + In file included from tftpd.c:71: + /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 + 58 | char tu_data[0]; /* data or error stri + ng */ + | ^~~~~~~ - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt + This occurs because writebuf points to this field and the latter + cannot be considered as being of dynamic length because it is not + the last field in the structure. Thus it is bound to its declared + size. - Reviewed-by: Jay Satiro - Closes #11615 + This commit always uses curl's own version of tftp.h where the + target field is last in its structure, effectively avoiding the + warning. -- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks + As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for + arpa/tftp.h are removed. - - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is - also not used by curl, but it is by its optional dependency `librtmp`. - Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This - library has been available since the early days of Windows. + Closes #11897 - - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present - since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and - other existing logic already assumed this lib being present, so delete - the check and replace the detection variable with `WIN32` and always - add `ws2_32` on Windows. +Dan Fandrich (20 Sep 2023) - Closes #11612 +- test1474: make precheck more robust on non-Solaris systems -Daniel Gustafsson (8 Aug 2023) + If uname -r returns something odd, perl could return an error code and + the test would be erroneously skipped. The qx// syntax avoid this. -- crypto: ensure crypto initialization works + Followup to 08f9b2148 - Make sure that context initialization during hash setup works to avoid - going forward with the risk of a null pointer dereference. +- github/labeler: switch to the 5 beta version - Reported-by: Philippe Antoine on HackerOne - Assisted-by: Jay Satiro - Assisted-by: Daniel Stenberg + This version adds an important feature that will allow more PRs to be + labelled. Rather than being limited to labeling PRs with files that + match a single glob, it can now label them if multiple changed files + match any one of a number of globs. - Closes #11614 +Daniel Stenberg (20 Sep 2023) -Viktor Szakats (7 Aug 2023) +- lib: enable hmac for digest as well -- openssl: switch to modern init for LibreSSL 2.7.0+ + Previously a build that disabled NTLM and aws-sigv4 would fail to build + since the hmac was disabled, but it is also needed for digest auth. - LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, - `OPENSSL_init_ssl()` function and deprecated the old, manual init - method, as seen in OpenSSL 1.1.0. Switch to the modern method when - available. + Follow-up to e92edfbef64448ef - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt + Fixes #11890 + Reported-by: Aleksander Mazur + Closes #11896 - Reviewed-by: Daniel Stenberg - Closes #11611 +- idn: if idn2_check_version returns NULL, return error -Daniel Stenberg (7 Aug 2023) + ... this avoids a NULL dereference for this unusual case. -- gskit: remove + Reported-by: s0urc3_ on hackerone + Closes #11898 - We remove support for building curl with gskit. +- http: fix CURL_DISABLE_BEARER_AUTH breakage - - This is a niche TLS library, only running on some IBM systems - - no regular curl contributors use this backend - - no CI builds use or verify this backend - - gskit, or the curl adaption for it, lacks many modern TLS features - making it an inferior solution - - build breakages in this code take weeks or more to get detected - - fixing gskit code is mostly done "flying blind" + When bearer auth was disabled, the if/else logic got wrong and caused + problems. - This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has - been mentioned on the curl-library mailing list. + Follow-up to e92edfbef64448ef461 + Fixes #11892 + Reported-by: Aleksander Mazur + Closes #11895 - It could be brought back, this is not a ban. Given proper effort and - will, gskit support is welcome back into the curl TLS backend family. +Michael Osipov (20 Sep 2023) - Closes #11460 +- wolfssl: allow capath with CURLOPT_CAINFO_BLOB -- RELEASE-NOTES: synced + Remain consistent with OpenSSL. While CAfile is nulled as documented + with CURLOPT_CAINFO_BLOB, CApath remains intact. -Dan Fandrich (7 Aug 2023) + Closes #11886 -- THANKS-filter: add a name typo +- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c -Stefan Eissing (7 Aug 2023) + Closes #11886 -- http3/ngtcp2: shorten handshake, trace cleanup +Dan Fandrich (19 Sep 2023) - - shorten handshake timing by delayed x509 store load (OpenSSL) - as we do for HTTP/2 - - cleanup of trace output, align with HTTP/2 output +- test1474: disable test on NetBSD, OpenBSD and Solaris 10 - Closes #11609 + These kernels only send a fraction of the requested amount of the first + large block, invalidating the assumptions of the test and causing it to + fail. -Daniel Stenberg (7 Aug 2023) + Assisted-by: Christian Weisgerber + Ref: https://curl.se/mail/lib-2023-09/0021.html + Closes #11888 -- headers: accept leading whitespaces on first response header +Ryan Schmidt (20 Sep 2023) - This is a bad header fold but since the popular browsers accept this - violation, so does curl now. Unless built with hyper. +- cmake, configure: also link with CoreServices - Add test 1473 to verify and adjust test 2306. + When linking with CoreFoundation, also link with CoreServices which is + apparently required to avoid an NSInvalidArgumentException in software + linking with libcurl on macOS Sonoma 14 and later. - Reported-by: junsik on github - Fixes #11605 - Closes #11607 + Fixes #11893 + Closes #11894 -- include/curl/mprintf.h: add __attribute__ for the prototypes +Marc Hoersken (19 Sep 2023) - - if gcc or clang is used - - if __STDC_VERSION__ >= 199901L, which means greater than C90 - - if not using mingw - - if CURL_NO_FMT_CHECKS is not defined +- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket - Closes #11589 + These dependencies are now already included in the Docker image. -- tests: fix bad printf format flags in test code + Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 + b41d15606e97f38cf312c1ce56 -- tests: fix header scan tools for attribute edits in mprintf.h + Closes #11889 -- cf-socket: log successful interface bind +Daniel Stenberg (19 Sep 2023) - When the setsockopt SO_BINDTODEVICE operation succeeds, output that in - the verbose output. +- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files - Ref: #11599 - Closes #11608 + Ref: #11883 + Reported-by: Michael Osipov + Closes #11884 -- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled +- RELEASE-NOTES: synced - Ref: #11457 - Closes #11606 +- test3103: CURLOPT_COOKIELIST test -- CURLOPT_SSL_VERIFYPEER.3: add two more see also options +- cookie: set ->running in cookie_init even if data is NULL - CURLINFO_CAINFO and CURLINFO_CAPATH + This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) - Closes #11603 + Test 3103 verifies. -- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice + Fixes #11875 + Reported-by: wangp on github + Closes #11876 - Closes #11007 +- test498: total header size for all redirects is larger than accepted -Graham Campbell (6 Aug 2023) +- http: use per-request counter to check too large headers -- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 + Not the counter that accumulates all headers over all redirects. - Closes #11585 + Follow-up to 3ee79c1674fd6 -Daniel Stenberg (6 Aug 2023) + Do a second check for 20 times the limit for the accumulated size for + all headers. -- TODO: add *5* entries for aws-sigv4 + Fixes #11871 + Reported-by: Joshix-1 on github + Closes #11872 - Closes #7559 - Closes #8107 - Closes #8810 - Closes #9717 - Closes #10129 +Jay Satiro (18 Sep 2023) -- TODO: LDAP Certificate-Based Authentication +- THANKS: add Eric Murphy - Closes #9641 + He reported #11850 (quiche build error) but I forgot to add a + 'reported-by' entry in the fix 267e14f1. -Stefan Eissing (6 Aug 2023) +Daniel Stenberg (18 Sep 2023) -- http2: cleanup trace messages +- h2-proxy: remove left-over mistake in drain_tunnel() - - more compact format with bracketed stream id - - all frames traced in and out + Left-over from 331b89a319 - Closes #11592 + Reported-by: 南宫雪珊 -Daniel Stenberg (6 Aug 2023) + Closes https://github.com/curl/curl/pull/11877 -- tests/tftpd+mqttd: make variables static to silence picky warnings +vvb2060 (18 Sep 2023) - Closes #11594 +- lib: failf/infof compiler warnings -- docs/cmdline: remove repeated working for negotiate + ntlm + Closes #11874 - The extra wording is added automatically by the gen.pl tool +Daniel Stenberg (17 Sep 2023) - Closes #11597 +- rand: fix 'alnum': array is too small to include a terminating null character -- docs/cmdline: add small "warning" to verbose options + It was that small on purpose, but this change now adds the null byte to + avoid the error. - "Note that verbose output of curl activities and network traffic might - contain sensitive data, including user names, credentials or secret data - content. Be aware and be careful when sharing trace logs with others." + Follow-up to 3aa3cc9b052353b1 - Closes #11596 + Reported-by: Dan Fandrich + Ref: #11838 + Closes #11870 -- RELEASE-NOTES: synced +Mathias Fuchs (16 Sep 2023) -- pingpong: don't use *bump_headersize +- cmake: fix the help text to the static build option in CMakeLists.txt - We use that for HTTP(S) only. + Closes #11843 - Follow-up to 3ee79c1674fd6 +John Haugabook (16 Sep 2023) - Closes #11590 +- MANUAL.md: change domain to example.com -- urldata: remove spurious parenthesis to unbreak no-proxy build + Closes #11866 - Follow-up to e12b39e13382 +Daniel Stenberg (16 Sep 2023) - Closes #11591 +- doh: inherit DEBUGFUNCTION/DATA -- easy: don't call Curl_trc_opt() in disabled-verbose builds + When creating new transfers for doing DoH, they now inherit the debug + settings from the initiating transfer, so that the application can + redirect and handle the verbose output correctly even for the DoH + transfers. - Follow-up to e12b39e133822c6a0 + Reported-by: calvin2021y on github + Fixes #11864 + Closes #11869 - Closes #11588 +Dan Fandrich (16 Sep 2023) -- http: use %u for printfing int +- http_aws_sigv4: fix sorting with empty parts - Follow-up to 3ee79c1674fd6f99e8efca5 + When comparing with an empty part, the non-empty one is always + considered greater-than. Previously, the two would be considered equal + which would randomly place empty parts amongst non-empty ones. This + showed as a test 439 failure on Solaris as it uses a different + implementation of qsort() that compares parts differently. - Closes #11587 + Fixes #11855 + Closes #11868 -Goro FUJI (3 Aug 2023) +- CI: ignore the "flaky" and "timing-dependent" test results -- vquic: show stringified messages for errno + CI builds will now run these tests, but will ignore the results if they + fail. The relevant tests are ones that are sensitive to timing or + have edge conditions that make them more likely to fail on CI servers, + which are often heavily overloaded and slow. - Closes #11584 + This change only adds two additional tests to be ignored, since the + others already had the flaky keyword. -Stefan Eissing (3 Aug 2023) + Closes #11865 -- trace: make tracing available in non-debug builds +- runtests: eliminate a warning on old perl versions - Add --trace-config to curl + The warning "Use of implicit split to @_ is deprecated" showed between + perl versions about 5.8 through 5.11. - Add curl_global_trace() to libcurl +- tests: log the test result code after each libtest - Closes #11421 + This makes it easier to determine the test status. Also, capitalize + FAILURE and ABORT messages in log lines to make them easier to spot. -Daniel Stenberg (3 Aug 2023) +Harry Sintonen (16 Sep 2023) -- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" +- misc: better random strings - See also https://github.com/curl/curl/pull/7507 + Generate alphanumerical random strings. -- TODO: add "WebSocket read callback" + Prior this change curl used to create random hex strings. This was + mostly okay, but having alphanumerical random strings is better: The + strings have more entropy in the same space. - remove "Upgrade to websockets" as we already have this + The MIME multipart boundary used to be mere 64-bits of randomness due + to being 16 hex chars. With these changes the boundary is 22 + alphanumerical chars, or little over 130 bits of randomness. - Closes #11402 + Closes #11838 -- test497: verify rejecting too large incoming headers +Daniel Stenberg (15 Sep 2023) -- http: return error when receiving too large header set +- cookie: reduce variable scope, add const - To avoid abuse. The limit is set to 300 KB for the accumulated size of - all received HTTP headers for a single response. Incomplete research - suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to - 1MB. +- cookie: do not store the expire or max-age strings - Closes #11582 + Convert it to an expire time at once and save memory. -Stefan Eissing (3 Aug 2023) + Closes #11862 -- http2: upgrade tests and add fix for non-existing stream +- cookie: remove unnecessary struct fields - - check in h2 filter recv that stream actually exists - and return error if not - - add test for parallel, extreme h2 upgrades that fail if - connections get reused before fully switched - - add h2 upgrade upload test just for completeness + Plus: reduce the hash table size from 256 to 63. It seems unlikely to + make much of a speed difference for most use cases but saves 1.5KB of + data per instance. - Closes #11563 + Closes #11862 -Viktor Szakats (3 Aug 2023) +- RELEASE-NOTES: synced -- tests: ensure `libcurl.def` contains all exports + Bumped to 8.4.0, the next presumed version - Add `test1279` to verify that `libcurl.def` lists all exported API - functions found in libcurl headers. +Dan Fandrich (14 Sep 2023) - Also: +- test2600: remove special case handling for USE_ALARM_TIMEOUT - - extend test suite XML `stdout` tag with the `loadfile` attribute. + This was originally added to handle platforms that supported only 1 + second granularity in connect timeouts, but after some recent changes + the test currently permafails on several Windows platforms. - - fix `tests/extern-scan.pl` and `test1135` to include websocket API. + The need for this special-case was removed in commit 8627416, which + increased the connect timeout in all cases to well above 1 second. - - use all headers (sorted) in `test1135` instead of a manual list. + Fixes #11767 + Closes #11849 - - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. +Daniel Stenberg (14 Sep 2023) - - add `libcurl.def` to the auto-labeler GHA task. +- SECURITY-PROCESS.md. call it vulnerability disclosure policy - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 + SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md - Closes #11570 + This a name commonly used for a document like this. This name helps + users find it. -Daniel Stenberg (2 Aug 2023) + Closes #11852 -- url: change default value for CURLOPT_MAXREDIRS to 30 +Junho Choi (14 Sep 2023) - It was previously unlimited by default, but that's not a sensible - default. While changing this has a remote risk of breaking an existing - use case, I figure it is more likely to actually save users from loops. +- quiche: fix build error with --with-ca-fallback - Closes #11581 + - Fix build error when curl is built with --with-quiche + and --with-ca-fallback. -- lib: fix a few *printf() flag mistakes + - Add --with-ca-fallback to the quiche CI job. - Reported-by: Gisle Vanem - Ref: #11574 - Closes #11579 + Fixes https://github.com/curl/curl/issues/11850 + Closes https://github.com/curl/curl/pull/11847 -Samuel Chiang (2 Aug 2023) +Jay Satiro (14 Sep 2023) -- openssl: make aws-lc version support OCSP +- escape: replace Curl_isunreserved with ISUNRESERVED - And bump version in CI + - Use the ALLCAPS version of the macro so that it is clear a macro is + being called that evaluates the variable multiple times. - Closes #11568 + - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates + a variable multiple times. -Daniel Stenberg (2 Aug 2023) + This is a follow-up to 291d225a which changed Curl_isunreserved into an + alias macro for ISUNRESERVED. The problem is the former is not easily + identified as a macro by the caller, which could lead to a bug. -- tool: make the length argument an int for printf()-.* flags + For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but + Curl_isunreserved(*foo++) is not even though they both are the same. - Closes #11578 + Closes https://github.com/curl/curl/pull/11846 -- tool_operate: fix memory leak when SSL_CERT_DIR is used +Dan Fandrich (13 Sep 2023) - Detected by Coverity +- tests: increase the default server logs lock timeout - Follow-up to 29bce9857a12b6cfa726a5 + This timeout is used to wait for the server to finish writing its logs + before checking them against the expected values. An overloaded machine + could take more than the two seconds previously allocated, so increase + the timeout to 5 seconds. - Closes #11577 + Ref: #11328 + Closes #11834 -- tool/var: free memory on OOM +- tests: increase TEST_HANG_TIMEOUT in two tests - Coverity detected this memory leak in OOM situation + These tests had a 5 second timeout compared to 60 seconds for all other + tests. Make these consistent with the others for more reliability on + heavily-loaded machines. - Follow-up to 2e160c9c652504e + Ref: #11328 - Closes #11575 +- test1056: disable on Windows -Viktor Szakats (2 Aug 2023) + This test relies on the IPv6 scope field being ignored when connecting to + ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit + dodgy, but it works on all our test platforms except Windows. This + test was disabled manually on all Windows CI builds already, so instead + add an incompatible feature and precheck so it's skipped on Windows + everywhere automatically. -- gha: bump libressl and mbedtls versions +- test587: add a slight delay after test - Closes #11573 + This test is designed to connect to the server, then immediately send a + few bytes and disconnect. In some situations, such as on a loaded + server, this doesn't give the server enough time to write its lock file + before its existence is checked. The test harness then fails to find the + server's input log file (because it hasn't been written yet) and fails + the test. By adding a short delay after the test, the HTTP server has + enough time to write its lock file which gives itself more time to write + its remaining files. -Jay Satiro (2 Aug 2023) + Ref: #11328 -- schannel: fix user-set legacy algorithms in Windows 10 & 11 +- tests: stop overriding the lock timeout - - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then - use the SCHANNEL_CRED legacy structure to pass the list to Schannel. + These tests reduce the server lock wait timeout which can increase + flakiness on loaded machines. Since this is merely an optimization, + eliminate them in favour of reliability. - - If the user set both a legacy algorithm list and a TLS 1.3 cipher list - then abort. + Ref: #11328 - Although MS doesn't document it, Schannel will not negotiate TLS 1.3 - when SCHANNEL_CRED is used. That means setting a legacy algorithm list - limits the user to earlier versions of TLS. +- tests: add some --expect100-timeout to reduce timing dependencies - Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would - ignore legacy algorithms in Windows 10 1809 and later. + These tests can fail when the test machine is so slow that the test HTTP + server didn't get a chance to complete before the client's one second + 100-continue timeout triggered. Increase that 1 second to 999 seconds so + this situation doesn't happen. - Reported-by: zhihaoy@users.noreply.github.com + Ref: #11328 - Fixes https://github.com/curl/curl/pull/10741 - Closes https://github.com/curl/curl/pull/10746 +- test661: return from test early in case of curl error -Daniel Stenberg (2 Aug 2023) +- tests: add the timing-dependent keyword on several tests -- variable.d: setting a variable again overwrites it + These are ones likely to fail on heavily-loaded machines that alter the + normal test timing. Most of these tests already had the flaky keyword + since this condition makes them more likely to fail on CI. - Reported-by: Niall McGee - Bug: https://twitter.com/niallmcgee/status/1686523075423322113 - Closes #11571 +- test1592: greatly increase the maximum test timeout -Jay Satiro (2 Aug 2023) + It was too short to be reliable on heavily loaded CI machines, and + as a fail-safe only, it didn't need to be short. -- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting + Ref: #11328 - - Re-wrap CURLSSLOPT_ALLOW_BEAST description. +- test: minor test cleanups -Daniel Stenberg (2 Aug 2023) + Remove an obsolete block of code in tests 2032 & 576. + Add a comment in test 1474. -- RELEASE-NOTES: synced +- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts -- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set + This gives more of a margin for error when running on overloaded CI + servers. - Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not - used, thus unnecessarily asking for addresses that will not be used. + Ref: #11328 - Reported-by: Joseph Tharayil - Fixes #11564 - Closes #11565 +- tests: improve SLOWDOWN test reliability by reducing sent data -- docs: link to the website versions instead of markdowns + These tests are run in SLOWDOWN mode which adds a 10 msec delay after + each character output, which means it takes at least 1.6 seconds (and + 320 kernel calls) just to get through the long welcome banner. On an + overloaded system, this can end up taking much more than 1.6 seconds, + and even more than the 7 or 16 second curl timeout that the tests rely + on, causing them to fail. Reducing the size of the welcome banner drops + the total number of characters sent before the transfer starts by more + than half, which reduces the opportunity for test-breaking slowdowns by + the same amount. - ... to make the links work when the markdown is converted to webpages on - https://curl.se + Ref: #11328 - Reported-by: Maurício Meneghini Fauth - Fixes https://github.com/curl/curl-www/issues/272 - Closes #11569 +- test650: fix an end tag typo -Viktor Szakats (1 Aug 2023) +Jay Satiro (13 Sep 2023) -- cmake: cache more config and delete unused ones +- tool_cb_wrt: fix debug assertion - - cache more Windows config results for faster initialization. + - Fix off-by-one out-of-bounds array index in Windows debug assertion. - - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. + Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 + Reported-by: Gisle Vanem - - delete dead references to `sys/utsname.h`. +Daniel Stenberg (13 Sep 2023) - Closes #11551 +- ctype: add ISUNRESERVED() -- egd: delete feature detection and related source code + ... and make Curl_isunreserved() use that macro instead of providing a + separate funtion for the purpose. - EGD is Entropy Gathering Daemon, a socket-based entropy source supported - by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it - a while ago. + Closes #11840 - Its detection in CMake was broken all along because OpenSSL libs were - not linked at the point of feature check. +Version 8.3.0 (13 Sep 2023) - Delete detection from both cmake and autotools, along with the related - source snippet, and the `--with-egd-socket=` `./configure` option. +Daniel Stenberg (13 Sep 2023) - Closes #11556 +- RELEASE-NOTES: syn ced -Stefan Eissing (1 Aug 2023) + curl 8.3.0 release -- tests: fix h3 server check and parallel instances +- THANKS: contributors from 8.3.0 - - fix check for availability of nghttpx server - - add `tcp` frontend config for same port as quic, as - without this, port 3000 is bound which clashes for parallel - testing +Thorsten Klein (12 Sep 2023) - Closes #11553 +- cmake: set SIZEOF_LONG_LONG in curl_config.h -Daniel Stenberg (1 Aug 2023) + in order to support 32bit builds regarding wolfssl CTC_SETTINGS -- docs/cmdline-opts: spellfixes, typos and polish + Closes #11839 - To make them accepted by the spell checker +Jay Satiro (12 Sep 2023) - Closes #11562 +- curl_ngtcp2: fix error message -- CI/spellcheck: build curl.1 and spellcheck it +- http_aws_sigv4: handle no-value user header entries - Added acceptable words + - Handle user headers in format 'name:' and 'name;' with no value. - Closes #11562 + The former is used when the user wants to remove an internal libcurl + header and the latter is used when the user actually wants to send a + no-value header in the format 'name:' (note the semi-colon is converted + by libcurl to a colon). -Alexander Jaeger (1 Aug 2023) + Prior to this change the AWS header import code did not special case + either of those and the generated AWS SignedHeaders would be incorrect. -- misc: fix various typos + Reported-by: apparentorder@users.noreply.github.com - Closes #11561 + Ref: https://curl.se/docs/manpage.html#-H -Daniel Stenberg (1 Aug 2023) + Fixes https://github.com/curl/curl/issues/11664 + Closes https://github.com/curl/curl/pull/11668 -- http2: avoid too early connection re-use/multiplexing +Dan Fandrich (11 Sep 2023) - HTTP/1 connections that are upgraded to HTTP/2 should not be picked up - for reuse and multiplexing by other handles until the 101 switching - process is completed. +- CI: run pytest with the -v option - Lots-of-debgging-by: Stefan Eissing - Reported-by: Richard W.M. Jones - Bug: https://curl.se/mail/lib-2023-07/0045.html - Closes #11557 + This lists of the test cases being run so it can be tracked over time. -- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" + Closes #11824 - This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. +Daniel Stenberg (11 Sep 2023) - It's a user error for supplying incomplete information to the build system. +- HTTP3: the msquic backend is not functional - Reported-by: Ryan Schmidt - Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 + I ask that we do not submit bugs for this backend just yet as we know it + does not fully work. -Viktor Szakats (1 Aug 2023) + Closes #11831 + Closes #11819 -- cmake: add support for single libcurl compilation pass +- aws_sigv4: the query canon code miscounted URL encoded input - Before this patch CMake builds used two separate compilation passes to - build the shared and static libcurl respectively. This patch allows to - reduce that to a single pass if the target platform and build settings - allow it. + Added some extra ampersands to test 439 to verify "blank" query parts - This reduces CMake build times when building both static and shared - libcurl at the same time, making these dual builds an almost zero-cost - option. + Follow-up to fc76a24c53b08cdf - Enable this feature for Windows builds, where the difference between the - two passes was the use of `__declspec(dllexport)` attribute for exported - API functions for the shared builds. This patch replaces this method - with the use of `libcurl.def` at DLL link time. + Closes #11829 - Also update `Makefile.mk` to use `libcurl.def` to export libcurl API - symbols on Windows. This simplifies (or fixes) this build method (e.g. - in curl-for-win, which generated a `libcurl.def` from `.h` files using - an elaborate set of transformations). +vvb2060 (11 Sep 2023) - `libcurl.def` has the maintenance cost of keeping the list of public - libcurl API symbols up-to-date. This list seldom changes, so the cost - is low. +- quic: don't set SNI if hostname is an IP address - Closes #11546 + We already do this for TLS connections. -- cmake: detect `SSL_set0_wbio` in OpenSSL + RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in + "HostName". - Present in OpenSSL 1.1.0 and BoringSSL. - Missing from LibreSSL 3.8.0. + Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 - Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 + Fixes https://github.com/curl/curl/issues/11827 + Closes https://github.com/curl/curl/pull/11828 - While here, also fix `RAND_egd()` detection which was broken, likely all - along. This feature is probably broken with CMake builds and also - requires a sufficiently obsolete OpenSSL version, so this part of the - update was not tested. +Daniel Stenberg (10 Sep 2023) - Closes #11555 +- RELEASE-NOTES: synced diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index 0c1d0c7589..ecd6e9eb9a 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -11,6 +11,7 @@ 1ocalhost on github 3dyd on github 3eka on github +5533asdg on github 8U61ife on github a1346054 on github Aaro Koskinen @@ -47,11 +48,13 @@ ahodesuka on github ajak in #curl Ajit Dhumale Akhil Kedia +Akhilesh Nema Aki Koskinen Akos Pasztory Akshay Vernekar Alain Danteny Alain Miniussi +Alan Coopersmith Alan Jenkins Alan Pinstein Albert Chin-A-Young @@ -155,6 +158,7 @@ Andreas Falkenhahn Andreas Farber Andreas Fischer Andreas Huebner +Andreas Kiefer Andreas Kostyrka Andreas Malzahn Andreas Ntaflos @@ -186,6 +190,7 @@ Andrew de los Reyes Andrew Francis Andrew Fuller Andrew Ishchuk +Andrew Kaster Andrew Krieger Andrew Kurushin Andrew Lambert @@ -238,6 +243,7 @@ apparentorder on github April King arainchik on github Archangel_SDY on github +Arjan van de Ven Arkadiusz Miskiewicz Armel Asselin Arnaud Compan @@ -262,7 +268,9 @@ Ates Goral atjg on github Augustus Saunders Austin Green +av223119 on github Avery Fay +awesomekosm on github awesomenode on github Axel Chong Axel Morawietz @@ -378,12 +386,14 @@ BratSinot on github Brendan Jurd Brennan Kinney Brent Beardsley +Brett Buddin Brian Akins Brian Bergeron Brian Carpenter Brian Chaplin Brian Childs Brian Chrisman +Brian Clemens Brian Dessent Brian E. Gallew Brian Green @@ -453,6 +463,7 @@ Charles Kerr Charles Romestant Charlie C Chen Prog +chensong1211 on github Cherish98 on github Chester Liu Chih-Chung Chang @@ -475,6 +486,7 @@ Chris Roberts Chris Sauer Chris Smowton Chris Talbot +Chris Webb Chris Young Christian Fillion Christian Grothoff @@ -502,6 +514,7 @@ Christopher R. Palmer Christopher Reid Christopher Sauer Christopher Stone +chrysos349 on github Chungtsun Li Ciprian Badescu civodul on github @@ -575,6 +588,7 @@ Dan Fandrich Dan Johnson Dan Kenigsberg Dan Locks +Dan McDonald Dan McNulty Dan Nelson Dan Petitt @@ -608,6 +622,7 @@ Daniel Shahaf Daniel Silverstone Daniel Steinberg Daniel Stenberg +Daniel Szmulewicz Daniel Theron Daniel Valenzuela Daniel Woelfel @@ -618,6 +633,7 @@ Darryl House Darshan Mody Darío Hereñú dasimx on github +DasKutti on github Dave Cottlehuber Dave Dribin Dave Halbakken @@ -689,12 +705,13 @@ Denis Laxalde Denis Ollier Dennis Clarke Dennis Felsing -dependabot[bot] Derek Higgins Derzsi Dániel Desmond O. Chang destman on github Detlef Schmier +Dexter Gerig +dfdity on github Dheeraj Sangamkar Didier Brisebourg Diego Bes @@ -714,6 +731,7 @@ Diogo Teles Sant'Anna Dion Williams Dirk Eddelbuettel Dirk Feytons +Dirk Hünniger Dirk Manske Dirk Rosenkranz Dirk Wetter @@ -738,6 +756,7 @@ Dmitry Mikhirev Dmitry Popov Dmitry Rechkin Dmitry S. Baikov +Dmitry Tretyakov Dmitry Wagin dnivras on github Dolbneff A.V @@ -788,6 +807,7 @@ ed0d2b2ce19451f2 Eddie Lumpkin Edgaras Janušauskas Edin Kadribasic +edmcln on github Edmond Yu Edoardo Lolletti Eduard Bloch @@ -860,6 +880,7 @@ Erik Janssen Erik Johansson Erik Minekus Erik Olsson +Erik Schnetter Erik Stenlund Ernest Beinrohr Ernst Sjöstrand @@ -884,6 +905,7 @@ Fabian Frank Fabian Hiernaux Fabian Keil Fabian Ruff +Fabian Vogt Fabian Yamaguchi Fabrice Fontaine Fabrizio Ammollo @@ -957,6 +979,7 @@ Gabriel Corona Gabriel Kuri Gabriel Simmer Gabriel Sjoberg +Gaelan Steele Gambit Communications Ganesh Kamath gaoxingwang on github @@ -1018,6 +1041,7 @@ Gou Lingfeng Graham Campbell Grant Erickson Grant Pannell +graywolf on github Greg Hewgill Greg Morse Greg Onufer @@ -1111,6 +1135,7 @@ Hongyi Zhao Howard Blaise Howard Chu hsiao yi +HsiehYuho on github htasta on github Hubert Kario Hugh Macdonald @@ -1312,6 +1337,7 @@ Jesse Noller Jesse Tan jethrogb on github jhoyla on github +Jiawen Geng Jie He Jiehong on github Jilayne Lovejoy @@ -1327,6 +1353,7 @@ Jiri Dvorak Jiri Hruska Jiri Jaburek Jishan Shaikh +Jiří Bok Jiří Malák jmdavitt on github jnbr on github @@ -1416,6 +1443,7 @@ Jonathan Cardoso Machado Jonathan Hseu Jonathan Moerman Jonathan Nieder +Jonathan Perkin Jonathan Watt Jonathan Wernberg Jongki Suwandi @@ -1536,6 +1564,7 @@ Kev Jackson Kevin Adler Kevin Baughman Kevin Burke +Kevin Daudt Kevin Fisk Kevin Ji Kevin Lussier @@ -1563,10 +1592,12 @@ Koichi Shiraishi kokke on github Konstantin Isakov Konstantin Kushnir +Konstantin Vlasov KotlinIsland on github kotoriのねこ kouzhudong on github Kovalkov Dmitrii +kpcyrd kreshano on github Kris Kennaway Krishnendu Majumdar @@ -1606,6 +1637,7 @@ Lars Francke Lars Gustafsson Lars J. Aas Lars Johannesen +Lars Kellogg-Stedman Lars Nilsson Lars Torben Wilson Lau @@ -1620,6 +1652,7 @@ Lawrence Wagerfield Leah Neukirchen Lealem Amedie Leandro Coutinho +LeeRiva Legoff Vincent Lehel Bernadt Leif W @@ -1666,6 +1699,7 @@ Loic Dachary LoRd_MuldeR Loren Kirkby Lorenzo Miniero +Louis Solofrizzo Loïc Yhuel lRoccoon on github Luan Cestari @@ -1690,6 +1724,7 @@ Luke Amery Luke Call Luke Dashjr Luke Granger-Brown +Lukáš Zaoral luminixinc on github Luo Jinghua Luong Dinh Dung @@ -1746,6 +1781,7 @@ Marco Maggi Marcos Diazr Marcus Hoffmann Marcus Klein +Marcus Müller Marcus Sundberg Marcus T Marcus Webster @@ -2060,6 +2096,7 @@ Nicolás Ojeda Bär Niels Martignène Niels van Tongeren Nikita Schmidt +Nikita Taranov Nikitinskit Dmitriy Niklas Angebrand Niklas Hambüchen @@ -2132,6 +2169,7 @@ Oscar Norlander Oskar Liljeblad Oskar Sigvardsson Oumph on github +Outvi V ovidiu-benea on github Ozan Cansel P R Schaffner @@ -2198,6 +2236,7 @@ Pawel A. Gajda Pawel Kierski Paweł Kowalski Paweł Wegner +Paweł Witas PBudmark on github Pedro Henrique Pedro Larroy @@ -2219,6 +2258,7 @@ Peter Goodman Peter Heuchert Peter Hjalmarsson Peter Korsgaard +Peter Krefting Peter Körner Peter Lamare Peter Lamberg @@ -2293,6 +2333,7 @@ pszemus on github pszlazak on github puckipedia on github Puneet Pawaia +Pēteris Caune qiandu2006 on github Quagmire Quanah Gibson-Mount @@ -2315,6 +2356,7 @@ Rainer Canavan Rainer Jung Rainer Koenig Rainer Müller +RainRat Raito Bezarius Rajesh Naganathan Rajkumar Mandal @@ -2323,6 +2365,7 @@ Ralph Beckmann Ralph Langendam Ralph Mitchell Ram Krushna Mishra +Ramiro Garcia ramsay-jones on github Ran Mozes RanBarLavie on github @@ -2428,6 +2471,7 @@ Robert Foreman Robert Iakobashvili Robert Kolcun Robert Linden +Robert Moreton Robert Olson Robert Prag Robert Ronto @@ -2476,6 +2520,7 @@ roughtex on github Roy Bellingan Roy Li Roy Shan +Rudi Heitbaum Rui LIU Rui Pinheiro Rune Kleveland @@ -2488,6 +2533,7 @@ RuurdBeerstra on github rwmjones on github Ryan Beck-Buysse Ryan Braud +Ryan Carsten Schmidt Ryan Chan Ryan Mast Ryan Nelson @@ -2545,6 +2591,8 @@ Scott Barrett Scott Cantor Scott Davis Scott McCreary +Scott Mutter +Scott Talbert sd0 on hackerone Sean Boudreau Sean Burford @@ -2555,6 +2603,7 @@ Sean Molenaar Sebastiaan van Erk Sebastian Haglund Sebastian Mundry +Sebastian Neubauer Sebastian Pohlschmidt Sebastian Rasmussen Sebastian Sterk @@ -2618,6 +2667,7 @@ Simon Chalifoux Simon Dick Simon H. Simon Josefsson +Simon K Simon Legner Simon Liu Simon Warta @@ -2731,6 +2781,7 @@ T200proX7 on github Tadej Vengust Tae Hyoung Ahn Taiyu Len +Tal Regev Taneli Vähäkangas Tanguy Fautre Taras Kushnir @@ -2766,6 +2817,7 @@ Thomas L. Shinnick Thomas Lopatic Thomas M. DuBuisson Thomas Petazzoni +Thomas Pyle Thomas Ruecker Thomas Schwinge Thomas Taylor @@ -2778,6 +2830,7 @@ Thorsten Klein Thorsten Schöning Tiit Pikma Till Maas +Till Wegmüller Tim Ansell Tim Baker Tim Bartley @@ -2948,6 +3001,7 @@ Vojtěch Král Volker Schmid Vsevolod Novikov vshmuk on hackerone +vulnerabilityspotter on hackerone vvb2060 vvb2060 on github Vyron Tsingaras @@ -2981,6 +3035,7 @@ William A. Rowe Jr William Ahern William Desportes William Tang +Winni Neessen wmsch on github wncboy on github Wojciech Zwiefka -- cgit v1.2.3