From ff76fe6c8f1e3b34c5571437612a038077f29860 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Fri, 24 May 2024 23:43:03 +0300 Subject: libcurl: update to 8.8.0 --- libs/libcurl/docs/CHANGES | 13963 ++++++++++++++++++++++---------------------- libs/libcurl/docs/THANKS | 41 +- 2 files changed, 6864 insertions(+), 7140 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 80d1cc84b5..5553b0a845 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,11019 +6,10704 @@ Changelog -Version 8.7.1 (27 Mar 2024) +Version 8.8.0 (22 May 2024) -Daniel Stenberg (27 Mar 2024) +Daniel Stenberg (22 May 2024) -- RELEASE-PROCEDURE: remove old release dates, add new pending ones +- RELEASE-NOTES: synced -Version 8.7.0 (27 Mar 2024) +- THANKS: add contributors from 8.8.0 -Daniel Stenberg (27 Mar 2024) +Nathan Moinvaziri (21 May 2024) -- RELEASE-NOTES: synced +- url: remove duplicate call to Curl_conncache_remove_conn when pruning - curl 8.7.0 release + - remove unnecessary prunedead struct from prune_dead_connections + - rename extract_if_dead to prune_if_dead for clarity -- THANKS: new contributors from the 8.7.0 release + Closes #13710 -- CURLOPT_POSTFIELDS.md: used for MQTT as well +Joseph Chen (21 May 2024) - Closes #13189 +- curl_setup.h: add support for IAR compiler -- http: remove stale comment about rewindbeforesend + Closes #13728 - ... because that struct field exists no more. +Stephen Farrell (21 May 2024) - Follow-up to 14bcea074a782272. +- docs/ECH: typo/clarification - Closes #13187 + Closes #13727 -- DISTROS: add document with distro pointers +Viktor Szakats (21 May 2024) - Lots of organizations distribute curl packages to end users. This is a - collection of pointers to where to learn more about curl on and with - each distro. +- hash: delete unused debug function - Assisted-by: Alan Coopersmith - Assisted-by: Andrew Kaster - Assisted-by: Andy Fiddaman - Assisted-by: Arjan van de Ven - Assisted-by: Brian Clemens - Assisted-by: chrysos349 on github - Assisted-by: Dan Fandrich - Assisted-by: Dan McDonald - Assisted-by: Gaelan Steele - Assisted-by: graywolf on github - Assisted-by: Jan Macku - Assisted-by: John Marshall - Assisted-by: Jonathan Perkin - Assisted-by: Kevin Daudt - Assisted-by: Marcus Müller - Assisted-by: Michał Górny - Assisted-by: Outvi V - Assisted-by: Ross Burton - Assisted-by: Sean Molenaar - Assisted-by: Till Wegmüller - Assisted-by: Viktor Szakats - Assisted-by: Winni Neessen + It had no use in the curl codebase and was also protected by the macro + `AGGRESSIVE_TEST` (renamed in 2020), also with no local reference. - Closes #13178 + Added in ca6e77083768858aa34207f8c5dce38b3c05336d (2002-11-11) -Fabian Keil (25 Mar 2024) + Closes #13729 -- wolfSSL: do not call the stub function wolfSSL_BIO_set_init() +Stefan Eissing (21 May 2024) - Calling the function isn't necessary and causes the build - to fail when wolfSSL has been compiled with NO_WOLFSSL_STUB: +- content_encoding: reject transfer-encoding after chunked - Making all in opts - CCLD curl - ld: error: undefined symbol: wolfSSL_BIO_set_init - >>> referenced by wolfssl.c:235 (vtls/wolfssl.c:235) - >>> libcurl_la-wolfssl.o:(wolfssl_bio_cf_create) in archiv - e ../lib/.libs/libcurl.a - cc: error: linker command failed with exit code 1 (use -v to see invocat - ion) - *** Error code 1 + reject a response that applies a transfer-encoding after a 'chunked' + encoding. RFC 9112 ch. 6.1 required chunked to be the final encoding. - Closes #13164 + Closes #13733 -Daniel Stenberg (25 Mar 2024) +- http: HEAD response body tolerance -- cmdline-opts: shorter help texts + - as reported in #13725, some servers wrongly send body bytes in + responses to a HEAD request. This used to be tolerated in curl + 8.4 and before and leads to failed transfers in newer versions. + - restore previous behaviour for HTTP/1.1 and HTTP/2: + * 1.1: do not add 'Transfer-Encoding' writers from HEAD + responses. RFC 9112 says they do not apply. + * 2: when the transfer expects 'no_body', to not report stream + resets as error when all response headers have been received. - In an effort to increase the readability of the "--help all" output on - narrow (80 column) terminals. + Reported-by: Jeroen Ooms + Fixes #13725 + Closes #13732 - Co-authored-by: Jay Satiro +Viktor Szakats (20 May 2024) - Closes #13169 +- tests: fix TFTP test 2305 on Windows -Matt Jolly (25 Mar 2024) + Ref: #13692 + Closes #13724 -- curl-rustls.m4: add pkg-config support to rustls detection +Jay Satiro (20 May 2024) - Based on the existing openssl pkg-config detection, this commit tries to - use pkg-config to find `rustls` then falls back to the current approach - if that fails. +- openssl: revert keylog_callback support for LibreSSL - We use the following logic: + - Revert to the legacy TLS 1.2 key logging code for LibreSSL. - - if no path is provided, just use pkg-config, if it's not there we have - a problem! - - if a path is provided, try pkg-config - + if pkg-config fails, try and find rustls directly + - Document SSLKEYLOGFILE for LibreSSL is TLS 1.2 max. - Closes #13179 + Prior to this change if the user specified a filename in the + SSLKEYLOGFILE environment variable and was using LibreSSL 3.5.0+ then + an empty file would be created and no keys would be logged. -Mohammadreza Hendiani (25 Mar 2024) + This is effectively a revert of e43474b4 which changed openssl.c to use + SSL_CTX_set_keylog_callback for LibreSSL 3.5.0+. Unfortunately LibreSSL + added that function only as a stub that doesn't actually do anything. -- TODO: update 13.11 with more information + Reported-by: Gonçalo Carvalho - Closes #13173 + Fixes https://github.com/curl/curl/issues/13672 + Closes https://github.com/curl/curl/pull/13682 -Daniel Stenberg (23 Mar 2024) +renovate[bot] (19 May 2024) -- docs/libcurl: generate PROTOCOLS from meta-data +- GHA: pin dependencies - Remove the PROTOCOLS section from the source files completely and - instead generate them based on the header data in the curldown files. + Closes #13712 - It also generates TLS backend information for options marked for TLS as - protocol. +Viktor Szakats (19 May 2024) - Closes #13175 +- appveyor: drop unnecessary `--clean-first` cmake option -- CURLMOPT_MAX*: mention what happens if changed mid-transfer + In CI all machines are fresh on startup, making the `clean` operation + unnecessary. This can save some time/energy for each job run. - For CURLMOPT_MAXCONNECTS and CURLMOPT_MAX_HOST_CONNECTIONS + Closes #13707 - Ref: #13158 - Closes #13176 +- cmake: merge two `if(BUILD_TESTING)` branches -- docs/libcurl: add TLS backend info for all TLS options + Closes #13708 - All man pages that are listed to be for TLS now must also specify - exactly what TLS backends the option works for, or use All if they all - work. +Tatsuhiro Tsujikawa (19 May 2024) - cd2nroff makes sure this is done and that the listed backends exist. +- GHA: bump nghttp2 to v1.62.1 - Closes #13168 + Use gcc-12 explicitly to compile C++20 source files. -- docs/libcurl: cleanups + Closes #13702 - - CURLINFO_TLS_SESSION.md: remove mention of NSS - - CURLINFO_TLS_SSL_PTR.md: remove NSS leftover - - CURLOPT_CAINFO.md: drop mention of backends not supporting this - - CURLOPT_CAPATH.md: wolfSSL also supports this +Viktor Szakats (19 May 2024) - Closes #13166 +- GHA: add NetBSD, OpenBSD, FreeBSD/arm64 and OmniOS jobs -- docs: make each libcurl man specify protocol(s) + Add these jobs to GHA: + - NetBSD, cmake-unity, clang, OpenSSL, x86_64, with tests, w/o python, + no parallelism (was flaky sometimes) + - OpenBSD, cmake-unity, clang, LibreSSL, x86_64, with tests, + with python, -j8, TFTP results ignored due to #13623. + - FreeBSD, cmake-unity and autotools, clang, OpenSSL, arm64 + (Tests disabled for arm64, because they are slow. It's available for + x86_64 with python, -j12.) + Configuration matches our existing Cirrus CI one. + - OmniOS, autotools, gcc, OpenSSL, x86_64, with tests, -j12. - The mandatory header now has a mandatory list of protocols for which the - manpage is relevant. + All build with websockets and examples. - Most man pages already has a "PROTOCOLS" section, but this introduces a - stricter way to specify the relevant protocols. + Closes #13583 - cd2nroff verifies that at least one protocol is mentioned (which can be - `*`). +- GHA: disable TFTP test on native Windows - This information is not used just yet, but A) the PROTOCOLS section can - now instead get generated and get a unified wording across all manpages - and B) this allows us to more reliably filter/search for protocol - specific manpages/options. + Some TFTP tests seem to enter into a loop and maybe hang? - Closes #13166 + E.g. 1007, 1009, 1238 -Stefan Eissing (21 Mar 2024) + Try fixing it by skipping all TFTP tests. -- http2, http3: only return CURLE_PARTIAL_FILE when bytes were received + Ref: https://github.com/curl/curl/actions/runs/9141987545/job/25137038249?pr= + 13698 - - should resolve spurious pytest failures when stream were reset - right after response header were received + Also drop mingw-w64 test exclusions copy-pasted from MSYS jobs. - Clsoes #13151 + Possibly related: cffbcc3110c1eda2e333f9cfe2e269154618793a #5364 -- http: separate response parsing from response action + Close #13699 - - move code that triggers on end-of-response into separate function from - parsing - - simplify some headp/headerlen usage - - add `httpversion` to SingleRequest to indicate the version of the - current response +renovate[bot] (18 May 2024) - Closes #13134 +- GHA: pin dependencies -Daniel Stenberg (21 Mar 2024) + Closes #13691 -- http2: remove the third (unused) argument from http2_data_done() +Viktor Szakats (18 May 2024) - Closes #13154 +- cmake: do not pass linker flags to the static library tool -- RELEASE-NOTES: synced + Do not add linker flags to the global CMake static library tool (aka + "static linker") (e.g. `ar`) flags list. They don't mix well. This was + only done after successfully detecting GSSAPI. -Evgeny Grin (Karlson2k) (21 Mar 2024) + Linker flags seen on Old Linux CI: + ``` + -- |GSS_LINKER_FLAGS|-Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib/x86_64-li + nux-gnu/heimdal| + -- |CMAKE_STATIC_LINKER_FLAGS| -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib + /x86_64-linux-gnu/heimdal| + ``` + Ref: https://github.com/curl/curl/actions/runs/9138988036/job/25130791712#ste + p:6:85 -- RELEASE-NOTES: corrected + Causing: + ``` + /usr/bin/ar qc libcurltool.a -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib/ + x86_64-linux-gnu/heimdal + CMakeFiles/curltool.dir/slist_wc.c.o CMakeFiles/curltool.dir/tool_binmode.c + .o CMakeFiles/curltool.dir/tool_bname.c.o + [...] + CMakeFiles/curltool.dir/tool_writeout_json.c.o CMakeFiles/curltool.dir/tool + _xattr.c.o CMakeFiles/curltool.dir/var.c.o + CMakeFiles/curltool.dir/__/lib/base64.c.o CMakeFiles/curltool.dir/__/lib/dy + nbuf.c.o + /usr/bin/ar: invalid option -- 'W' + Usage: /usr/bin/ar [emulation options] [-]{dmpqrstx}[abcDfilMNoPsSTuvV] [--pl + ugin ] [member-name] [count] archive-file file... + /usr/bin/ar -M [now - c->timestamp; + | ^~~ + curl/lib/hostip.c: In function 'Curl_hostcache_prune': + curl/lib/hostip.c:241:10: note: 'now' was declared here + 241 | time_t now; + | ^~~ + In function 'hostcache_timestamp_remove', + inlined from 'fetch_addr' at curl/lib/hostip.c:310:8: + curl/lib/hostip.c:205:23: error: 'user.now' may be used uninitialized [-Werro + r=maybe-uninitialized] + 205 | time_t age = prune->now - c->timestamp; + | ~~~~~^~~~~ + curl/lib/hostip.c: In function 'fetch_addr': + curl/lib/hostip.c:304:33: note: 'user' declared here + 304 | struct hostcache_prune_data user; + | ^~~~ + In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity + _0_c.c:40: + curl/lib/cf-socket.c: In function 'cf_socket_send': + curl/lib/cf-socket.c:1294:10: error: 'c' may be used uninitialized [-Werror=m + aybe-uninitialized] + 1294 | if(c >= ((100-ctx->wblock_percent)*256/100)) { + | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + curl/lib/cf-socket.c:1292:19: note: 'c' was declared here + 1292 | unsigned char c; + | ^ + In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity + _0_c.c:364: + In function 'tftp_state_timeout', + inlined from 'tftp_multi_statemach' at curl/lib/tftp.c:1230:27: + curl/lib/tftp.c:1208:5: error: 'current' may be used uninitialized [-Werror=m + aybe-uninitialized] + 1208 | if(current > state->rx_time + state->retry_time) { + | ^ + curl/lib/tftp.c: In function 'tftp_multi_statemach': + curl/lib/tftp.c:1192:10: note: 'current' was declared here + 1192 | time_t current; + | ^~~~~~~ + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49792835/job/91c8dj5 + qb36spfe0#L112 + Ref: https://github.com/curl/curl/actions/runs/9082968838/job/24960616145#ste + p:12:62 -Stefan Eissing (15 Mar 2024) + Ref: #13592 + Closes #13643 -- http: revisit http_perhapsrewind() +Andrew (16 May 2024) - - use facilities provided by client readers better - - work also for non-uploading requests like GET/HEAD - - update documentation +- wakeup_create: use FD_CLOEXEC/SOCK_CLOEXEC - Closes #13117 + for `pipe()`/`socketpair()` -- test 1541: verify getinfo values on first header callback + Fixes #13618 + Closes #13625 - Reported-by: chensong1211 on github - Ref: #13125 - Closes #13128 +Stefan Eissing (16 May 2024) -- TLS: start shutdown only when peer did not already close +- rustls: fix partial send handling - - When curl sees a TCP close from the peer, do not start a TLS shutdown. - TLS shutdown is a handshake and if the peer already closed the - connection, it is not interested in participating. + When TLS bytes could not completely sent off, the amount of plain bytes + already added to rustls were forgotten. This lead to send those byte + duplicate, corrupting the request send to the server. - Reported-by: dfdity on github - Assisted-by: Jiří Bok - Assisted-by: Pēteris Caune - Fixes #10290 - Closes #13087 + Closes #13676 -Daniel Stenberg (14 Mar 2024) +- pytest: add DELETE tests, check server version -- RELEASE-NOTES: synced + - add tests for DELETE working + - check apache version in keepalive test + - fix some comments -- curl: make --libcurl output better CURLOPT_*SSLVERSION + Closes #13679 - The option is really two enums ORed together, so it needs special - attention to make the code output nice. +Juliusz Sosinowicz (16 May 2024) - Added test 1481 to verify. Both the server and the proxy versions. +- vquic-tls: use correct cert name check API for wolfSSL - Reported-by: Boris Verkhovskiy - Fixes #13127 - Closes #13129 + wolfSSL_X509_check_host checks the peer name against the alt names and + the common name. -- GHA/linux: add sysctl trick to work-around GitHub runner issue + Fixes #13487 + Closes #13680 - The GitHub image runner update from 20240304.1.0 to 20240310.1 - introduces a problem for clang-14. The issue is caused by - incompatibility between llvm 14 provided in ubuntu-22.04 image and the - much newer kernel configured with high-entropy ASLR. +Viktor Szakats (16 May 2024) - As a work-around, we issue a sysctl command to lower the entropy and get - clang-14 to work again. +- cmake: initialize `BUILD_TESTING` before first use - URL: https://github.com/actions/runner-images/issues/9491 + Before this patch `BUILD_TESTING` was used once, then initialized, then + used again. This caused the `curlu` library not being built when relying + on an implicit `BUILD_TESTING=ON` setting, and ending up with a link + error when building the `testdeps` target. - Closes #13124 + It did not cause issues when `BUILD_TESTING` was explicitly set. -- SPONSORS: describe the basics + Move the initialization before the first use to fix it. - Closes #13119 + Regression from aace27b0965c10394544d1dacc9c2cb2fe0de3d3 #12287 + Closes #13668 -- GOVERNANCE: document the core team +Daniel Stenberg (16 May 2024) - Closes #13118 +- libtest: 2308 verifies CURLE_WRITE_ERROR after write callback error -Jay Satiro (13 Mar 2024) + Verifies that the issue in #13669 actually is fixed. This return code is + what the CURLOPT_WRITEFUNCTION manpage documents should be returned. -- vquic-tls: fix the error code returned for bad CA file + This code is mostly from the + Source-written-by: Trumeet on github + Closes #13671 - - Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem - reading the cert file or path. +Antoine Bollengier (16 May 2024) - This is a follow-up to the parent commit aedbbdf1. +- socketpair: fix compilation when USE_UNIX_SOCKETS is not defined - Reported-by: Karthikdasari0423@users.noreply.github.com + Closes #13666 - Fixes https://github.com/curl/curl/issues/13115 +Stefan Eissing (16 May 2024) -Daniel Stenberg (12 Mar 2024) +- rustsls: fix error code on receive -- vquic-tls: return appropirate errors on wolfSSL errors + - use CURLE_RECV_ERROR instead of CURLE_READ_ERROR when receiving + data fails. - Reported-by: Dexter Gerig - Closes #13107 + Closes #13670 -Viktor Szakats (12 Mar 2024) +Max Dymond (16 May 2024) -- tidy-up: one comment and EOF newlines +- ci: disable Renovate dashboard - Reviewed-by: Daniel Stenberg - Closes #13108 + The Renovate dashboard insists on an open issue, + which is a problem. Disable the dashboard. Status + can still be seen at https://developer.mend.io/github/curl/curl. -Daniel Stenberg (12 Mar 2024) + Fixes #13630 + Closes #13673 -- cmdline-opts: language cleanups +Daniel Stenberg (16 May 2024) - Use imperative mood consistently for the first sentence describing an - option. +- RELEASE-NOTES: synced - "Set this" instead "tell curl to set" or "this sets..." +renovate[bot] (16 May 2024) - Plus some extra cleanups and rephrasing. +- GHA: update awslabs/aws-lc to v1.27.0 - Closes #13106 + Closes #13667 -- managen: remove space before protocols +Daniel Stenberg (15 May 2024) - For options that are listed for specific protocols, the protocols (shown - first within parentheses) are now output without the leading space in the - manpage output. +- curl_easy_pause.md: use correct defines in example - Closes #13105 + Spotted-by: Harry Sintonen + Closes #13664 -Jay Satiro (12 Mar 2024) +Viktor Szakats (15 May 2024) -- mbedtls: properly cleanup the thread-shared entropy +- appveyor: more tidy-ups - - Store the state of the thread-shared entropy for global init/cleanup. + - use `--disable` when calling `curl --version`. Just in case. - - Use curl's thread support of mbedtls for all Windows builds instead of - just when the threaded resolver is used via USE_THREADS_WIN32. + - use single-quotes for a constant. - Prior to this change on global cleanup curl builds that have curl thread - support for mbedtls freed the entropy (8b1d2298) but failed to mark that - it had been freed, which caused problems on subsequent init + transfer. + Closes #13662 - Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105 - Reported-by: awesomekosm@users.noreply.github.com +- reuse: migrate standalone license file to dep5 - Closes https://github.com/curl/curl/pull/13071 + Follow-up to 73a36021207284ad2b4340ffde34a51b0ba4d47a + Closes #13660 -Daniel Stenberg (12 Mar 2024) +- appveyor: guard against crash-build with VS2008 -- tool_getparam: handle non-existing (out of range) short-options + The combination of `-DDEBUGBUILD`, a shared `curl.exe`, and the VS2008 + compiler creates a `curl.exe` segfaulting on startup: - ... correctly, even when they follow an existing one without a space in - between. + ``` + + _bld/src/curl.exe --version + ./appveyor.sh: line 122: 793 Segmentation fault "${curl}" --version + Command exited with code 139 + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49817266/job/651iy6q + n1e238pqj#L191 - Verify with test 467 + Add job that triggers the issue and add the necessary logic to skip + running the affected `curl.exe`. - Follow-up to 07dd60c05b - Reported-by: Geeknik Labs - Fixes #13101 - Closes #13102 + Ref: #13592 + Closes #13654 -Stefan Eissing (11 Mar 2024) +renovate[bot] (15 May 2024) -- lib: move 'done' parameter to SingleRequests +- GHA: pin dependencies - A transfer may do several `SingleRequest`s for its success. This happens - regularly for authentication, follows and retries on failed connections. - The "readwrite()" calls and functions connected to those carried a `bool - *done` parameter to indicate that the current `SingleRequest` is over. - This may happen before `upload_done` or `download_done` bits of - `SingleRequest` are set. + Closes #13628 - The problem with that is now `write_resp()` protocol handlers are - invoked in places where the `bool *done` cannot be passed up to the - caller. Instead of being a bool in the call chain, it needs to become a - member of `SingleRequest`, reflecting its state. +Orgad Shaneh (15 May 2024) - This removes the `bool *done` parameter and adds the `done` bit to - `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a - `SingleRequest` in a follow up, clearing `done` and other - flags/counters. +- socket: remove redundant call to getsockname - Closes #13096 + The result "add" is unused. -- request: clarify message when request has been sent off + Closes #13655 - Change the "uploaded and fine" message for requests without a body +renovate[bot] (15 May 2024) - Reported-by: Karthikdasari0423 on github - Fixes #13093 - Closes #13095 +- CI: renovate updates -Daniel Stenberg (11 Mar 2024) + - GHA: update actions/checkout action to v4 + - GHA: update wolfSSL/wolfssh to v1.4.17 + - GHA: update wolfSSL/wolfssl to v5.7.0 + - Update the regex config in renovate.json -- RELEASE-NOTES: synced + Closes #13632 + Closes #13641 + Closes #13658 + Closes #13659 -Stefan Eissing (9 Mar 2024) +Max Dymond (15 May 2024) -- lib: keep conn IP information together +- ci: fix renovate config for WolfSSL/WolfSSH tagging scheme - new struct ip_quadruple for holding local/remote addr+port + WolfSSL/WolfSSH use a different versioning scheme; + stable builds end with `-stable`. Renovate requires + some extra configuration to extract the version + from these types of tags. - - used in data->info and conn and cf-socket.c - - copy back and forth complete struct - - add 'secondary' to conn - - use secondary in reporting success for ftp 2nd connection + Closes #13644 - Reported-by: DasKutti on github - Fixes #13084 - Closes #13090 +- ci: set semantic type as CI and include digests as CI operations -Daniel Stenberg (8 Mar 2024) + Replace "chore" with "ci" for renovate's semantic + type, and include digests with "pin" and + "pinDigest" as ci operations. -- scripts/managen: the new name and home for the manpage generator + Closes #13644 - It was previously docs/cmdline-opts/gen.pl +Daniel Stenberg (15 May 2024) - Closes #13089 +- DEPRECATE.md: TLS libraries without 1.3 support -- VULN-DISCLOSURE-POLICY.md: update detail about CVE requests + curl drops support for TLS libraries without TLS 1.3 capability after + May 2025. - curl is a CNA now + It requires that a curl build using the library should be able to + negotiate and use TLS 1.3, or else it is not good enough. We support a + vast amount of other TLS libraries that are likely to satisfy users + better. - Closes #13088 + Closes #13544 -Stefan Eissing (8 Mar 2024) +- Revert "ci: update nghttp2/nghttp2 to v1.62.0" -- lib: client reader polish + This reverts commit 14f2c767555b7598d7783ccd9093670b84d28488. - - seek_func/seek_client, use transfer values only - - remove copies held in `struct connectdata`, use only - ever `data->set.seek_func` - - resolves possible issues in multiuse connections - - new mime post reader eliminates need to ever overwriting this + We need to also upgrade the C++ compiler for that bump to work. - - websockets, remove empty Curl_ws_done() function + Closes #13656 - Closes #13079 +renovate[bot] (15 May 2024) -Marcel Raad (8 Mar 2024) +- Dockerfile: update debian digest to 911821c -- lib1598: fix `CURLOPT_POSTFIELDSIZE` usage + Closes #13629 - It requires a `long` argument. +- ci: update gnutls/gnutls to v3.8.5 - Closes https://github.com/curl/curl/pull/13085 + Closes #13640 -Daniel Stenberg (8 Mar 2024) +- ci: update awslabs/aws-lc to v1.26.0 -- docs/cmdline-opts: drop the curl.1 from the dist tarball + Closes #13647 - Since it is no longer needed for building tool_hugehelp.c and all the - docs is available in readable markdown format in the tarball, the peeps - that don't want to build the manpage still do good. +- ci: update cloudflare/quiche to v0.21.0 - Removing it also fixes the complexity of out-of-tree builds when the - curl.1 exists in the source tree. + Closes #13648 -- test1140/1173: extend wildcards to find curl.1 +- ci: update libressl-portable/portable to v3.9.2 - ... in its new build path. + Closes #13649 - Also update the test scripts to be more precise in error messages to - help us understand CI errors better. +- ci: update nghttp2/nghttp2 to v1.62.0 - Follow-up to f03c85635f35269f1 - Ref: #13029 - Closes #13083 + Closes #13650 -- http2: minor tweaks to optimize two struct sizes +- ci: update ngtcp2/nghttp3 to v1.3.0 - - use BIT() instead of bool - - place the struct fields in (roughly) size order + Closes #13651 - Closes #13082 +- ci: update ngtcp2/ngtcp2 to v1.5.0 -- buildconf.bat: remove outdated groff/nroff use + Closes #13652 - - don't try to generate the real hugehelp file, because it requires - curl.txt which needs a build - - don't attempt to do anything in a c-ares subdirectory +Max Dymond (14 May 2024) - Follow-up to f03c85635f35269 - Closes #13078 +- ci: handle git submodules for mbedTLS -- http2: memory errors in the push callbacks are fatal +- ci: reconfigure renovate - Use the correct nghttp2 error code accordingly. + - set prefix for github actions updates to be gha: + - set prefix for other renovate actions to be ci: + - disable debian updates in linux-old.yml - Closes #13081 +Viktor Szakats (14 May 2024) -Viktor Szakats (7 Mar 2024) +- tidy-up: whitespace [ci skip] -- mkhelp: rename variable to fix compiler warnings +- warnless: delete orphan declarations - ``` - src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global de - claration [_bld\src\curl.vcxproj] - _bld\src\tool_hugehelp.c(8,27): - see declaration of 'm' - src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global d - eclaration [_bld\src\curl.vcxproj] - src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global d - eclaration [_bld\src\curl.vcxproj] - src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global d - eclaration [_bld\src\curl.vcxproj] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75c - d2n0wj6lc#L614 + Follow-up to 358f7e757781857c4b498a68634726609fa3884a #11932 + Closes #13639 - Reviewed-by: Daniel Stenberg - Closes #13077 +Daniel Stenberg (14 May 2024) -Daniel Stenberg (7 Mar 2024) +- BUG-BOUNTY.md: clarify the third party situation -- KNOWN_BUGS: POP3 issue when reading small chunks + We do not pay bounties for problems in other libraries. - Closes #12063 + Closes #13560 -- RELEASE-NOTES: synced +Stefan Eissing (14 May 2024) -Robert Moreton (7 Mar 2024) +- http tests: in CI skip test_02_23* for quiche -- asyn-ares: fix data race warning + For unknown reasons, these tests fail in CI often, but run fine locally. + Skip them in CI to avoid unrelated PRs to have failures. - - Store the c-ares version during global init. + Closes #13638 - Prior to this change several threads could write the same data to a - static int variable at the same time. Though in practice it's not a - problem ThreadSanitizer may warn. +Daniel Gustafsson (14 May 2024) - Reported-by: Nikita Taranov - Assisted-by: Jay Satiro +- hsts: explicitly skip blank lines - Fixes #13065 - Closes #13000 + Keep blank lines or lines containing only whitespace to make it all + the way to the more expensive sscanf call in hsts_add. -Stefan Eissing (7 Mar 2024) + Closes: #13603 + Reviewed-by: Daniel Stenberg -- hyper: implement unpausing via client reader +- autotools: Only probe for SGI MIPS compilers on IRIX - Just a tidy up to contain 'ifdef' pollution of common - code parts with implementation specifics. + MIPSPro and the predecessor compiler which was part of the IDO (IRIS + Development Option) were only ever shipped on the SGI IRIX operating + system (with MIPSPro on 6.0+ which was released in 1994). Limit the + autoconf check to IRIX when probing for these compilers to save some + cycles on other platforms. - - remove the ifdef hyper unpausing in easy.c - - add hyper client reader for CURL_CR_PROTOCOL phase - that implements the unpause method for calling - the hyper waker if it is set + Closes: #13611 + Reviewed-by: Daniel Stenberg - Closes #13075 +Viktor Szakats (14 May 2024) -- ngtcp2: no recvbuf for stream +- tests: fix test 1167 to skip digit-only symbols - - write response data directly to the transfer via - `Curl_xfer_write_resp()` like we do in HTTP/2. + This avoids mistaking symbols with their numeric value when using + certain C preprocessors which output these numeric values at the + beginning of the line as part of an expression. - Closes #13073 + Seen on OpenBSD 7.5 + clang. -- docs/cmdline-opts/.gitignore: ignore curl.txt + Example `test1167.pl -v` output, before this patch: + ``` + Source: cpp /home/runner/work/curl/curl/tests/../include/curl/curl.h + Symbol: 20000 + Line #3835: 20000 + 142, + [...] + Bad symbols in public header files: + 20000 + [...] + ``` + Ref: https://github.com/curl/curl/actions/runs/9069136530/job/24918015357#ste + p:3:7513 - Closes #13076 + Ref: #13583 + Closes #13634 -Evgeny Grin (Karlson2k) (7 Mar 2024) +Daniel Stenberg (14 May 2024) -- sha512_256: add support for GnuTLS and OpenSSL +- lib: call Curl_strntolower instead of doing crafted loops - This is a follow-up for PR #12897. + Closes #13627 - Add support for SHA-512/256 digest calculation by TLS backends. - Currently only OpenSSL and GnuTLS (actually, nettle) support - SHA-512/256. +- setopt: acknowledge errors proper for CURLOPT_COOKIEJAR - Closes #13070 + Error out on error, do not continue. -- digest: add check for hashing error + Closes #13624 - Closes #13072 +- vtls: remove duplicate assign -Viktor Szakats (7 Mar 2024) + Curl_ssl_peer_cleanup() already clears the ->sni field, no point in + assigning it again. -- cmake: enable `ENABLE_CURL_MANUAL` by default + Spotted by CodeSonar - Meaning `curl.1` and `src/tool_hugehelp.c` are built by default, - and `--manual` in curl tool is also enabled by default. + Closes #13626 - This syncs behaviour with autotools. +Max Dymond (13 May 2024) - For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set - to a consistent date, e.g. the timestamp of `CHANGES`. +- Group all non-major updates together to reduce PR spam - A pre-built manual (e.g. the one distributed in the official source - tarball) will be ignored and rebuilt after this patch, unless - explicitly disabling this option. +- Add the remainder of the workflows - Fixes #13028 - Closes #13069 +- Add some basic versioning for some workflows to check whether this is detecte + d properly -Stefan Eissing (7 Mar 2024) +renovate[bot] (13 May 2024) -- http2: push headers better cleanup +- Add renovate.json - - provide common cleanup method for push headers +Daniel Stenberg (13 May 2024) - Closes #13054 +- vauth: make two functions void that always just returned OK -Daniel Stenberg (7 Mar 2024) + Removes the need to check return values when they can never fail. -- GIT-INFO: convert to markdown + Pointed out by CodeSonar - Closes #13074 + Closes #13621 -Richard Levitte (7 Mar 2024) +- setopt: remove check for 'option' that is always true -- cmake: fix libcurl.pc and curl-config library specifications + - make sure that passing in option set to NULL clears the fields + correctly - Letting CMake figure out where libraries are located gives you full - paths. When generating libcurl.pc and curl-config, getting libraries as - full paths is unusual when one expects to get a list of -l. + - remove the weird second take if Curl_parse_login_details() returns + error - To meet expectations, an effort is made to convert the full paths into - -l, possibly with -L before it. + Follow-up to 7333faf00bf25db7cd1e0012d6b140 - Fixes #6169 - Fixes #12748 - Closes #12930 + Spotted by CodeSonar -Daniel Stenberg (7 Mar 2024) + Closes #13619 -- test463: HTTP with -d @file with file containing CR, LF and null byte +Viktor Szakats (13 May 2024) -- paramhlp: fix CRLF-stripping files with "-d @file" +- tests: tidy up types in server code - All CR and LF bytes should be stripped, as documented, and all other - bytes are inluded in the data. Starting now, it also excludes null bytes - as they would otherwise also cut the data short. + Cherry-picked from #13489 + Closes #13610 - Reported-by: Simon K - Fixes #13063 - Closes #13064 +Daniel Stenberg (13 May 2024) -Viktor Szakats (7 Mar 2024) +- setopt: make the setstropt_userpwd args compulsory -- cmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled + They were always used so no point in allowing them to be optional. - Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF` - when building without the Schannel TLS backend. + follow-up to 0e37b42dc956bd8a - This in turn may have caused Kerberos, SPNEGO and SSPI features - disappearing even with `CURL_WINDOWS_SSPI=ON` set. + Closes #13608 + Reviewed-by: Daniel Gustafsson - This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a - default for `CURL_WINDOWS_SSPI`, but allowing a manual override. +- RELEASE-NOTES: synced - Also update the option text to better tell its purpose. +Daniel Gustafsson (13 May 2024) - Thanks-to: Andreas Loew - Reviewed-by: Daniel Stenberg - Ref: #13056 - Closes #13061 +- websocket: Avoid memory leak in error path -Jay Satiro (6 Mar 2024) + In the errorpath for randstr being too long to copy into the buffer + we leak the randstr when returning CURLE_FAILED_INIT. Fix by using + an explicit free on randstr in the errorpath. -- KNOWN_BUGS: FTPS server compatibility on Windows with Schannel + Closes: #13602 + Reviewed-by: Daniel Stenberg - - Remove "2.12 FTPS with Schannel times out file list operation" +- hsts: Remove single-use single-line function - - Remove "7.12 FTPS directory listing hangs on Windows with Schannel" + The hsts_entry() function contains of a single line and is only + used in a single place in the code, so move the allocation into + hsts_create instead to improve code readability. C code usually + don't use the factory abstraction for object creation, and this + small example wasn't following our usual code style. - - Add "7.12 FTPS server compatibility on Windows with Schannel" + Closes: #13604 + Reviewed-by: Daniel Stenberg - This change adds a more generic bug description that explains FTPS with - the latest curl and Schannel is not widely used and may have more bugs - than other TLS backends. +Viktor Szakats (12 May 2024) - The two removed FTPS Schannel bugs can't be reproduced any longer and - were likely fixed by 24d6c288. +- lib: bump hash sizes to `size_t` - Ref: https://github.com/curl/curl/issues/5284 - Ref: https://github.com/curl/curl/issues/9161 - Ref: https://github.com/curl/curl/issues/12894 + Follow-up to cc907e80a2498c0599253271a6f657f614b52a4e #13502 + Cherry-picked from #13489 + Closes #13601 - Closes https://github.com/curl/curl/pull/13032 +- tests: make the unit test result type `CURLcode` -- trace-config.md: remove the mutexed options list + Before this patch, the result code was a mixture of `int` and + `CURLcode`. - - Remove the rendered manpage message that says: - "[--trace-config] is mutually exclusive to --trace and -v, --verbose". + Also adjust casts and fix a couple of minor issues found along the way. - Actually it can be used with either of those options, which are mutually - exclusive to each other but not to --trace-config. + Cherry-picked from #13489 + Closes #13600 - Ref: https://curl.se/docs/manpage.html#--trace-config +- appveyor: tidy-ups - Closes https://github.com/curl/curl/pull/13031 + - delete a duplicate line. + - simplify a `make` call. + - merge two `if` branches. + - reorder autotools options for clarity. + - add `--enable-warnings` where missing (it's also the default.) + - add empty lines to YAML for readability. + - use lowercase install prefix/directory. -Daniel Stenberg (6 Mar 2024) + Closes #13598 -- mkhelp: simplify the generated hugehelp program +Daniel Stenberg (12 May 2024) - Use a plain array and puts() every line, also allows us to provide the - strings without ending newlines. +- docs/cmdline-opts: mention STARTTLS for --ssl and --ssl-reqd - - merge blank lines into the next one as a prefixed newline. - - turn eight consecutive spaces into a tab (since they can only be on the - left side of text) - - the newly generated tool_hugehelp is 3K lines shorter and 50K smaller - - modifies the top logo layout a little by reducing the indent + ... since users might look for those terms in the manpage. - Closes #13047 + Closes #13590 -- docs: ascii version of manpage without nroff +- setopt: warn on Curl_set*opt() uses not using the return value - Create ASCII version of manpage without nroff + And switch the invokes that would "set" NULL to instead just plainly + free the pointer, as those were otherwise the invokes that would ignore + the return code. And possibly confuse static code analyzers. - - build src/tool_hugegelp.c from the ascii manpage - - move the the manpage and the ascii version build to docs/cmdline-opts - - remove all use of nroff from the build process - - should make the build entirely reproducible (by avoiding nroff) + Closes #13591 - - partly reverts 2620aa9 to build libcurl option man pages one by one - in cmake because the appveyor builds got all crazy until I did +Orgad Shaneh (12 May 2024) - The ASCII version of the manpage +- autotools: delete unused functions - - is built with gen.pl, just like the manpage is - - has a right-justified column making the appearance similar to the previous - version - - uses a 4-space indent per level (instead of the old version's 7) - - does not do hyphenation of words (which nroff does) + Closes #13605 - History +Viktor Szakats (11 May 2024) - We first made the curl build use nroff for building the hugehelp file in - December 1998, for curl 5.2. +- examples: fix/silence `-Wsign-conversion` - Closes #13047 + - extend `FD_SET()` hack to all platforms (was only Cygwin). + Warnings may also happen in other envs, e.g. OmniOS. + Ref: https://github.com/libssh2/libssh2/actions/runs/8854199687/job/2431676 + 2831#step:3:2021 -Stefan Eissing (6 Mar 2024) + - tidy-up `CURLcode` vs `int` use. -- lib: add `void *ctx` to reader/writer instances + - cast an unsigned to `long` before passing to `curl_easy_setopt()`. - - `struct Curl_cwriter` and `struct Curl_creader` now carry a - `void *ctx` member that points to the instance as allocated. - - using `r->ctx` and `w->ctx` as pointer to the instance specific - struct that has been allocated + Cherry-picked from #13489 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + Closes #13501 - Reported-by: Rudi Heitbaum - Fixes #13035 - Closes #13059 +Orgad Shaneh (11 May 2024) -- http: fix dead code in setting post client reader +- cmake: fix `HAVE_IOCTLSOCKET_FIONBIO` test with gcc 14 - - postsize was always 0, thus the check's else never happened - after the mime client reader was introduced + The function signature has had u_long flags since ever. This is how it + is defined in the documentation, and implemented in MinGW. - Follow-up to 0ba47146f7ff3d - Closes #13060 + The code that uses ioctlsocket in nonblock.c also has unsigned long. -- http2: fix push discard + Error: + CurlTests.c:275:41: error: passing argument 3 of 'ioctlsocket' from incompati + ble pointer type [-Wincompatible-pointer-types] + 275 | if(0 != ioctlsocket(0, FIONBIO, &flags)) + | ^~~~~~ + | | + | int * + In file included from CurlTests.c:266: + /opt/mxe/usr/i686-w64-mingw32.static/include/winsock2.h:1007:76: note: expect + ed 'u_long *' {aka 'long unsigned int *'} but argument is of type 'int *' + 1007 | WINSOCK_API_LINKAGE int WSAAPI ioctlsocket(SOCKET s,__LONG32 cmd,u_ + long *argp); + | ~~ + ~~~~~~^~~~ - - fix logic in discarding a failed pushed stream so that - stream context is properly cleaned up + Closes #13578 - Closes #13055 +Jay Satiro (10 May 2024) -- transfer.c: break receive loop in speed limited transfers +- ftp: fix build for CURL_DISABLE_VERBOSE_STRINGS - - the change breaks looping in transfer.c receive for transfers that are - speed limited on having gotten *some* bytes. - - the overall speed limit timing is done in multi.c + This is a follow-up to b7c7dffe which changed the FTP state change + verbose debug text (aka infof) to tracing debug text (aka trc). - Reported-by: Dmitry Karpov - Bug: https://curl.se/mail/lib-2024-03/0001.html - Closes #13050 + Prior to this change if libcurl was without DEBUGBUILD and built with + CURL_DISABLE_VERBOSE_STRINGS (ie --disable-verbose) the build would + error. -- mime: add client reader + Caught by Circle CI job openssl-no-verbose. - Add `mime` client reader. Encapsulates reading from mime parts, getting - their length, rewinding and unpausing. +- lib: clear the easy handle's saved errno before transfer - - remove special mime handling from sendf.c and easy.c - - add general "unpause" method to client readers - - use new reader in http/imap/smtp - - make some mime functions static that are now only used internally + - Clear data->state.os_errno before transfer. - In addition: - - remove flag 'forbidchunk' as no longer needed + - Explain the change in behavior in the CURLINFO_OS_ERRNO doc. - Closes #13039 + - Add to the CURLINFO_OS_ERRNO doc the list of libcurl network-related + errors that may cause the errno to be saved. -Daniel Stenberg (5 Mar 2024) + data->state.os_errno is saved before libcurl returns a network-related + failure such as connection failure. It is accessible to the user via + CURLINFO_OS_ERRNO so they can get more information about the failure. -- RELEASE-NOTES: synced + Prior to this change it wasn't cleared before transfer, so if a user + retrieved the saved errno it could be from a previous transfer. That is + because an errno is not always saved for network-related errors. -- TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake" + Closes https://github.com/curl/curl/pull/13574 - Follow-up to 8e741644a229c37 +Stefan Eissing (10 May 2024) -Tal Regev (5 Mar 2024) +- ftp: add tracing support -- cmake: add USE_OPENSSL_QUIC support + - add `Curl_trc_feat_ftp` for tracing via trace config + - add macro CURL_TRC_FTP(data, fmt, ...) + - replace DEBUGF(infof()) statements in ftp.c by CURL_TRC_FTP() + - always trace FTP connection state - Closes #13034 + Closes #13580 -Stefan Eissing (5 Mar 2024) +Daniel Stenberg (10 May 2024) -- TIMER_STARTTRANSFER: set the same for everyone +- http: remove redundant check - - set TIMER_STARTTRANSFER on seeing the first response bytes - in the download client writer, not coming from a CONNECT - - initialized the timer the same way for all protocols - - remove explicit setting of TIMER_STARTTRANSFER in file.c - and c-hyper.c + Spotted by CodeSonar - Closes #13052 + Closes #13582 -Michael Kaufmann (5 Mar 2024) +Viktor Szakats (10 May 2024) -- http: better error message for HTTP/1.x response without status line +- ldap: fix unused variables (seen on OmniOS) - If a response without a status line is received, and the connection is - known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status - line" instead of "Received HTTP/0.9 when not allowed". + ``` + ../../lib/ldap.c: In function 'ldap_do': + ../../lib/ldap.c:380:11: error: unused variable 'ldap_ca' [-Werror=unused-v + ariable] + 380 | char *ldap_ca = conn->ssl_config.CAfile; + | ^~~~~~~ + ../../lib/ldap.c:379:9: error: unused variable 'ldap_option' [-Werror=unuse + d-variable] + 379 | int ldap_option; + | ^~~~~~~~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9033564377/job/24824192730#ste + p:3:6059 - Closes #13045 + Ref: #13583 + Closes #13588 -Viktor Szakats (5 Mar 2024) +Daniel Stenberg (10 May 2024) -- KNOWN_BUGS: fix typo +- url: make parse_login_details use memdup0 - Reviewed-by: Daniel Stenberg - Closes #13051 + Also make the user and password arguments mandatory, since all code + paths in libcurl used them anyway. -Sebastian Neubauer (5 Mar 2024) + Adapted unit test case 1620 to the new rules. -- smpt: fix starttls + Closes #13584 - In cases where the connection was fast, curl sometimes failed to open a - connection. This fixes a regression of c2d973627bab12abc5486a3f3. +Orgad Shaneh (10 May 2024) - The regression triggered in these steps: +- digest: replace strcpy for empty string with simple assignment - 1. Create an smtp connection - 2. Use STARTTLS - 3. Receive the response - 4. We are inside the loop in `smtp_statemachine`, calling - `smtp_state_starttls_resp` - 5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and - run `smtp_perform_upgrade_tls` at the start of the function. + Closes #13586 - In the bad flow, we stay in the while loop, calling - `Curl_pp_readresp`, which reads part of the TLS handshake and things - go wrong. +Viktor Szakats (10 May 2024) - The reason is that `Curl_pp_moredata` changed behavior and always - returns `true`, so we stay in the loop in `smtp_statemachine`. With a - slow connection `Curl_pp_readresp` cannot read new data and returns - `CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`. +- autotools: fix `HAVE_IOCTLSOCKET_FIONBIO` test for gcc 14 - With a fast connection, `Curl_pp_readresp` reads new data from the tcp - connection, which is part of the TLS handshake. + ``` + conftest.c:152:41: error: passing argument 3 of 'ioctlsocket' from incompatib + le pointer type [-Wincompatible-pointer-types] + 152 | if(0 != ioctlsocket(0, FIONBIO, &flags)) + | ^~~~~~ + | | + | int * + ``` - The fix is in `Curl_pp_moredata`, which needs to take the final line - into account and return `false` if only the final line is stored. + Reported-by: LigH + Fixes #13579 + Closes #13587 - Closes #13048 +- CI: ignore test 286 on Appveyor gcc 7 build -Stefan Eissing (5 Mar 2024) + Disabled earlier for gcc 9 builds. gcc 7 uses the same runner and + prone to similar intermittent failures. -- lib: enhance client reader resume + rewind + Follow-up to f1e05a6e6e7225fa09952abb2c935ae1abe44f45 #12106 #12040 + Closes #13575 - - update client reader documentation - - client reader, add rewind capabilities - - tell creader to rewind on next start - - Curl_client_reset() will keep reader for future rewind if requested - - add Curl_client_cleanup() for freeing all resources independent of - rewinds - - add Curl_client_start() to trigger rewinds - - move rewind code from multi.c to sendf.c and make part of - "cr-in"'s implementation - - http, move the "resume_from" handling into the client readers - - the setup of a HTTP request is reshuffled to follow: - * determine method, target, auth negotiation - * install the client reader(s) for the request, including crlf - conversions and "chunked" encoding - * apply ranges to client reader - * concat request headers, upgrades, cookies, etc. - * complete request by determining Content-Length of installed - readers in combination with method - * send - - add methods for client readers to - * return the overall length they will generate (or -1 when unknown) - * return the amount of data on the CLIENT level, so that - expect-100 can decide if it want to apply itself - * set a "resume_from" offset or fail if unsupported - - struct HTTP has become largely empty now - - rename `Client_reader_*` to `Curl_creader_*` +Daniel Stenberg (10 May 2024) - Closes #13026 +- cf-socket: don't try getting local IP without socket -Viktor Szakats (5 Mar 2024) + In cf_tcp_connect(), it might fail and not get a socket assigned to + ctx->sock but set_local_ip() is still called which would make + getsockname() get invoked with a negative file desriptor and fail. -- openssl-quic: fix BIO leak and Windows warning + By adding this check, set_local_ip() will now instead blank out the + fields correctly. - Caused by an accidentally duplicated line in - d6825df334def106f735ce7e0c1a2ea87bddffb0. + Spotted by CodeSonar - ``` - .../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses intege - r precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-6 - 4-to-32] - 1095 | bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE); - | ~~~~~~~~~~~~~ ~~~~~~~^~~~~~ - 1 warning and 2 errors generated. - ``` + Closes #13577 - Reviewed-by: Stefan Eissing - Closes #13043 +- tool_getparam: remove two redundant conditions -- openssl-quic: fix unity build, casing, indentation + When getstr() does not return error, it returns a valid pointer. - - rename static functions to avoid duplicate symbols in unity mode. - - windows -> Windows/window in error message and comment. - - fix indentation. + Spotted by CodeSonar - Reviewed-by: Stefan Eissing - Closes #13044 + Closes #13576 -Daniel Stenberg (5 Mar 2024) +Stefan Eissing (10 May 2024) -- gen.pl: make the "manpageification" faster +- quiche: trust its timeout handling - The function that replaces occurances of "--longoption" with "-Z, - --longoption" etc with the proper highlight applied, no longer loops - over the options. + - set the idle timeout transport parameter + in milliseconds as documented by quiche + - do not calculate the idle timeout, rely on + quiche handling it - Closes #13041 + Closes #13581 -- CONTRIBUTE: update the section on documentation format +Daniel Stenberg (10 May 2024) - ... since most of it is markdown now. +- dmaketgz: accept a SOURCE_DATE_EPOCH as an second argument - Closes #13046 + to make it easier to reproduce a tarball -- smtp: free a temp resource + Closes #13573 - The returned address needs to be freed. +- RELEASE-NOTES: synced - Follow-up to e3905de8196d67b89df1602feb84c1f993211b20 - Spotted by Coverity +Stefan Eissing (10 May 2024) - Closes #13038 +- h3/ngtcp2: improve error handling -- _VARIABLES.md: improve the description + - identify ngtcp2 and nghttp3 error codes that are fatal + - close quic connection on fatal errors + - refuse further filter operations once connection is closed + - confusion about the nghttp3 API. We should close the QUIC stream on + cancel and not use the nghttp3 calls intended to be invoked when the + QUIC stream was closed by the peer. - Closes #13040 + Closes #13562 -dependabot[bot] (4 Mar 2024) +Jay Satiro (10 May 2024) -- build(deps): bump fsfe/reuse-action from 2 to 3 +- docs: fix some CURLINFO examples - Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3. - - [Release notes](https://github.com/fsfe/reuse-action/releases) - - [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3) + - improve getinfo result check for example sections: + CURLINFO_ACTIVESOCKET, CURLINFO_LASTSOCKET, CURLINFO_SSL_VERIFYRESULT, + CURLINFO_PROXY_SSL_VERIFYRESULT - --- - updated-dependencies: - - dependency-name: fsfe/reuse-action - dependency-type: direct:production - update-type: version-update:semver-major - ... + - fix getinfo result check for example sections: + CURLINFO_NUM_CONNECTS, CURLINFO_OS_ERRNO - Signed-off-by: dependabot[bot] + - fix verify result check for example sections: + CURLINFO_PROXY_SSL_VERIFYRESULT -Stefan Eissing (4 Mar 2024) + Bug: https://github.com/curl/curl/discussions/13557#discussion-6625507 + Reported-by: farazrbx@users.noreply.github.com -- pytest: adapt to API change + Closes https://github.com/curl/curl/pull/13559 - - pytest has changed the signature of the hook pytest_report_header() - for some obscure reason and that change landed in our CI now +Daniel Stenberg (9 May 2024) - - remove the changed param that we never used anyway +- KNOWN_BUGS: gssapi library name + version is missing in curl_version_info() - Closes #13037 + Closes #13492 + Closes #13570 -Daniel Stenberg (4 Mar 2024) +- krb5: use dynbuf -- cookie: if psl fails, reject the cookie + Closes #13568 - A libpsl install without data and no built-in database is now considered - bad enough to reject all cookies since they cannot be checked. It is - somewhat of a user error, but still. +- managen: fix the option sort order - Reported-by: Dan Fandrich - Closes #13033 + ... it used to strip off the .d file extension to sort correctly but + ever since the extension changed to .md the operation failed and the + sort got wrong. -Stefan Eissing (4 Mar 2024) + Follow-up to 2494b8dd5175cee7f2e -- lib: further send/upload handling polish + Closes #13567 - - Move all the "upload_done" handling to request.c +Stefan Eissing (8 May 2024) - - add possibility to abort sending of a request - - add `Curl_req_done_sending()` for checks - - transfer.c: readwrite_upload() now clean +- GHA: repair the linux-old job - - removing data->state.ulbuf and data->req.upload_fromhere + package libc6_2.28-10+deb10u2_amd64.deb changed to + libc6_2.28-10+deb10u3_amd64.deb - - as well as data->req.upload_present - - set data->req.upload_done on having read all from - the client and completely flushed the send buffer + Closes #13564 - - tftp, remove setting of data->req.upload_fromhere +Viktor Szakats (8 May 2024) - - serves no purpose as `upload_present` is not set - and the data itself is directly `sendto()` anyway +- appveyor: make gcc 6 mingw64 job build-only - - smtp, make upload EOB conversion a client reader - - xfer_ulbuf addition + This job has proven to be the flakiest of all, and it's also the oldest + Windows runner we had tests running on: 'Visual Studio 2015', that is + running on Windows Server 2012 R2: + https://www.appveyor.com/docs/windows-images-software/ - - add xfer_ulbuf for borrowing, similar to xfer_buf - - use in file upload - - use in c-hyper body sending + Turn off tests on this job to help stabilizing CI runs. - - h1-proxy, remove init of data->state.uilbuf that is never used - - smb, add own send_buf instead of using data->state.ulbuf + This was also one of the slowest running job amongst the AppVeyor CI ones. - Closes #13010 + Flakiness data: + https://testclutch.curl.se/static/reports/summary.html + Entries: + Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity + (curl) [current] + Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static (curl) [fo + rmer] -Daniel Stenberg (4 Mar 2024) + Closes #13566 -- RELEASE-NOTES: synced +Stefan Eissing (8 May 2024) -kpcyrd (3 Mar 2024) +- unit2604: use alloc instead of overlong string const -- rustls: fix two warnings related to number types + Closes #13563 - Reported-by: Gisle Vanem - Follow-up to #12989 - Closes #13017 +Daniel Gustafsson (8 May 2024) -Stefan Eissing (3 Mar 2024) +- bufq: remove duplicate word in comment -- bufq: writing into a softlimit queue cannot be partial + Inspired by 13552. - - when unable to obtain a new chunk on a softlimit bufq, - this is an allocation error and needs to be reported as - such. - - writes into a soflimit bufq never must be partial success + Closes: #13554 + Reviewed-by: Daniel Stenberg - Reported-by: Dan Fandrich - Fixes #13020 - Closes #13023 +Viktor Szakats (8 May 2024) -Dan Fandrich (2 Mar 2024) +- lib/cf-h1-proxy: silence compiler warnings (gcc 14) -- configure: Don't build shell completions when disabled + They came up ealier with gcc 12 (Windows), but apparently gcc 14 is + still reporting them, also under Linux. - With the recent changes to completion file building, the files were - built always and only installation was selectively disabled. Now, when - they are disabled they aren't even built, avoiding a build-time error in - environments where it's not possible to run the curl binary that was - just created (e.g. if library paths were not set up correctly). + ``` + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c: In functi + on 'cf_h1_proxy_close': + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1060:17: w + arning: null pointer dereference [-Wnull-dereference] + 1060 | cf->connected = FALSE; + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1061:8: wa + rning: null pointer dereference [-Wnull-dereference] + 1061 | if(cf->ctx) { + | ~~^~~~~ + In function 'tunnel_free', + inlined from 'cf_h1_proxy_destroy' at /home/runner/work/curl-for-win/curl + -for-win/curl/lib/cf-h1-proxy.c:1053:3: + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:198:27: wa + rning: null pointer dereference [-Wnull-dereference] + 198 | struct h1_tunnel_state *ts = cf->ctx; + | ^~ + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/8985369476/job/2467921 + 9528#step:3:6320 - Follow-up to 0f7aba83c + Fixes #13237 + Closes #13555 - Reported-by: av223119 on github - Fixes #13027 - Closes #13030 +Michał Antoniak (8 May 2024) -Jay Satiro (2 Mar 2024) +- mbedtls: support TLS 1.3 -- cmdline-opts/_EXITCODES: sync with libcurl-errors + Closes #13539 - - Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that - can be returned by the curl tool. +Daniel Stenberg (8 May 2024) - Closes https://github.com/curl/curl/pull/13015 +- version: use msnprintf instead of strncpy -Stefan Eissing (1 Mar 2024) + - to ensure a terminating null byte + - to avoid zero-padding the target -- hyper: disable test1598 due to lack of trailer support + debug code only - Follow-up to 50838095 + Closes #13549 - Closes #13016 +- curl_path: make Curl_get_pathname use dynbuf -Dan Fandrich (1 Mar 2024) + ... instead of malloc and memcpy -- ftp: Mark a const buffer as const + - unit test 2604 verifies Curl_get_pathname() -- appveyor: Properly skip if only CircleCI is changed + Closes #13550 -- docs: Update minimal binary size in INSTALL.md +- lib: make protocol handlers store scheme name lowercase - Include more options to reduce binary size. + - saves a lowercase operation when the "[scheme]_proxy" name is + generated + - appears less "shouting" + - update test 970, 972, 1438 and 1536 -- configure: Don't make shell completions without perl + Closes #13553 - The code that attempted to skip building the shell completions didn't - work properly and tried to build them even if perl wasn't available. - This step, as well as the install step, is now properly skipped without - perl. +- lib: remove two instances of "only only" messages - Follow-up to 89733e2dd + Fixes #13551 + Reported-by: Lucas Nussbaum + Closes #13552 - Closes #13022 +Pavel Pavlov (7 May 2024) -RainRat (1 Mar 2024) +- asyn-thread: fix curl_global_cleanup crash in Windows -- misc: Fix typos in docs and lib + - Make sure that asynchronous resolves handled by Winsock are stopped + before WSACleanup is called. - This fixes miscellaneous typos and duplicated words in the docs, lib - and test comments and a few user facing errorstrings. + This is implemented by ensuring that when Curl_resolver_kill is called + (eg via multi_done) it will cancel the Winsock asynchronous resolve and + wait for the cancellation to complete. Winsock runs the asynchronous + completion routine immediately when a resolve is canceled. - Author: RainRat on Github - Reviewed-by: Daniel Gustafsson - Reviewed-by: Dan Fandrich - Closes: #13019 + Prior to this change it was possible that during curl_global_cleanup + "a DNS resolver thread created by GetAddrInfoExW did not terminate yet, + however curl is already shutting down, deinitializing Winsock with + WSACleanup() leading to an access violation." -Dan Fandrich (29 Feb 2024) + Background: -- configure: build & install shell completions when enabled + If libcurl is built with the asynchronous threaded resolver option for + Windows then it resolves in one of two ways. For Windows 8.1 and later, + libcurl resolves by using the Winsock asynchronous resolver which does + its own thread management. For older versions of Windows, libcurl + resolves by creating a separate thread that calls getaddrinfo. This + change only affects the former and it's already handled for the latter. - The --with-fish-functions-dir and --with-zsh-functions-dir options - currently have no effect on a normal build because the scripts/ directory - where they're used is not built. Add scripts/ to a normal build and - change the completion options to default to off to preserve the existing - behaviour. + Reported-by: Ch40zz@users.noreply.github.com - Closes: #12906 + Fixes https://github.com/curl/curl/issues/13509 + Closes https://github.com/curl/curl/pull/13518 -- github/labeler: improve the match patterns +Jay Satiro (7 May 2024) -Stefan Eissing (28 Feb 2024) +- asyn-thread: fix Curl_thread_create result check -- tests: add test1598 for POST with trailers + - Compare to curl_thread_t_null instead of 0 for error. - - test POST fields with trailers and chunked encoding + Currently for both supported thread libraries (pthreads and Windows) + curl_thread_t_null is defined as 0. However, the pattern throughout the + code is to check against curl_thread_t_null and not 0 since for + posterity some thread library may not use 0 for error. - Ref: #12938 - Closes #13009 + Closes https://github.com/curl/curl/pull/13542 -Daniel Stenberg (28 Feb 2024) +- curl_multibyte: remove access() function wrapper for Windows -- cmdline-opts/_VERSION: provide %VERSION correctly + - Remove curlx_win32_access() which was a wrapper to use access() in + Windows. - ... so that it does not get included verbatim in the output. Fixes a - regression shipped in 8.6.0. + This is a follow-up to 602fc213, one of two commits which removed + access() calls from the codebase and banned use of the function. - Also fix a format mistake in form.md + Closes https://github.com/curl/curl/pull/13529 - Closes #13008 +Daniel Gustafsson (6 May 2024) -Stefan Eissing (28 Feb 2024) +- tls: Remove EXAMPLEs from deprecated options -- lib: Curl_read/Curl_write clarifications + CURLOPT_EGDSOCKET and CURLOPT_RANDOM_FILE are both completely dead + so remove their example sections since the code there is useless. + There is still a way to inject a random file for OpenSSL older than + 1.1.0 but it's not what the example showed (and it's not even done + with this option) so we refrain from documenting it here. - - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to - clarify when and at what level they operate - - send/recv of transfer related data is now done via - `Curl_xfer_send()/Curl_xfer_recv()` which no longer has - socket/socketindex as parameter. It decides on the transfer - setup of `conn->sockfd` and `conn->writesockfd` on which - connection filter chain to operate. - - send/recv on a specific connection filter chain is done via - `Curl_conn_send()/Curl_conn_recv()` which get the socket index - as parameter. - - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for - naming consistency - - clarify that the special CURLE_AGAIN hangling to return - `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` - and CURLE_AGAIN is returned by all other send() variants. - - fix a bug in websocket `curl_ws_recv()` that mixed up data - when it arrived in more than a single chunk (to be made - into a sperate PR, also) + Closes: #13540 + Reviewed-by: Daniel Stenberg - Added as documented [in - CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68 - aa06dc738cb75d4/docs/CLIENT-READERS.md). +- tests: Only require EXAMPLE for non-deprecated options - - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` - - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` - - HTTP chunked uploads are now formatted in a client reader added when - needed. - - FTP line-end conversions are done in a client reader added when - needed. - - when sending requests headers, remaining buffer space is filled with - body data for sending in "one go". This is independent of the request - body size. Resolves #12938 as now small and large requests have the - same code path. + Manpages which document deprecated CURLOPT_ or CURLINFO_ are not + required to have an EXAMPLE section since they might effectively + be dead no-ops which we don't want to trick users into believing + they can use by copying example code. - Changes done to test cases: + Closes: #13540 + Reviewed-by: Daniel Stenberg - - test513: now fails before sending request headers as this initial - "client read" triggers the setup fault. Behaves now the same as in - hyper build - - test547, test555, test1620: fix the length check in the lib code to - only fail for reads *smaller* than expected. This was a bug in the - test code that never triggered in the old implementation. +Daniel Stenberg (6 May 2024) - Closes #12969 +- EXPERIMENTAL: add graduation requirements for each feature -Daniel Gustafsson (28 Feb 2024) + Starting now, experimental features should have a set of documentated + requirements of what is needed for the feature to graduate. -- curldown: Fix email address in Copyright + This adds requirements to all existing experiments. - The curldown conversion accidentally replaced daniel@haxx.se with - just daniel.se. This reverts back to the proper email address in - the curldown docs as well as in a few other stray places where it - was incorrect (while unrelated to curldown). + Closes #13541 - Reviewed-by: Daniel Stenberg - Closes: #12997 +Ivan (6 May 2024) -Daniel Stenberg (28 Feb 2024) +- misc: fix typos, quoting and spelling -- getparam: make --ftp-ssl work again + Fix wording of comments, and misquotings where `' is markdown parsed + where it shouldn't be, and remove a misspelled preprocessor comment + which really isn't needed (and removing it makes it match surrounding + code better). - Follow-up to 9e4e527 which accidentally broke it + Closes: #13538 + Reviewed-by: Daniel Gustafsson - Reported-by: Jordan Brown - Fixes #13006 - Closes #13007 +Daniel Gustafsson (6 May 2024) -- KNOWN_BUGS: IMAPS connection fails with rustls error +- tests: Mark tftpd timer function as noreturn - Closes #10457 + This avoids the below compiler warning: -- KNOWN_BUGS: FTPS upload, FileZilla, GnuTLS and close_notify + tftpd.c:280:1: warning: function 'timer' could be declared with + attribute 'noreturn' [-Wmissing-noreturn] - Closes #11383 + Closes: #13534 + Reviewed-by: Daniel Stenberg -- KNOWN_BUGS: Implicit FTPS upload timeout +- doh: Remove unused function prototype - Closes #11720 + Closes: #13536 + Reviewed-by: Daniel Stenberg -- KNOWN_BUGS: HTTP/2 prior knowledge over proxy +Daniel Stenberg (6 May 2024) - Closes #12641 +- doh: cleanups in ECH related functions -- TODO: build HTTP/3 with OpenSSL and nghttp3 using cmake + - make local_decode_rdata_name use dynbuf instead of calloc + memcpy + - avoid extra memdup in local_decode_rdata_alpn + - no need to if() before free() + - use memdup instead of calloc + memcpy in Curl_doh_decode_httpsrr - Closes #12988 + Reviewed-by: Stephen Farrell + Closes #13526 -- TODO: Select signature algorithms +Viktor Szakats (5 May 2024) - Closes #12982 +- libssh2: delete redundant feature guard -- examples: use present tense in comments + Delete `HAVE_LIBSSH2_VERSION` (equivalent to + `LIBSSH2_VERSION_NUM` > 0x010100) guard surrounding + a `LIBSSH2_VERSION_NUM` > 0x010B00 one. - remove "will" and some other word fixes + Reviewed-by: Daniel Gustafsson + Closes #13537 - Closes #13003 +Jan Venekamp (5 May 2024) -- docs: more language cleanups +- tool_cfgable: free {proxy_}cipher13_list on exit - - present tense - - avoid bad words + Author: Jan Venekamp + Reviewed-by: Daniel Gustafsson + Closes: #13531 - Closes #13003 +RainRat (4 May 2024) -Daniel Gustafsson (27 Feb 2024) +- doh: Fix typo in comment -- setopt: Fix disabling all protocols + Closes: #13504 + Author: RainRat on Github + Reviewed-by: Daniel Stenberg + Reviewed-by: Daniel Gustafsson - When disabling all protocols without enabling any, the resulting - set of allowed protocols remained the default set. Clearing the - allowed set before inspecting the passed value from --proto make - the set empty even in the errorpath of no protocols enabled. +Christian Schmitz (4 May 2024) - Co-authored-by: Dan Fandrich - Reported-by: Dan Fandrich - Reviewed-by: Daniel Stenberg - Closes: #13004 +- dynbuf: Fix returncode on memory error -Andreas Kiefer (27 Feb 2024) + Curl_dyn_vaddf should return a proper error code in case allocating + memory failed. -- fopen: fix narrowing conversion warning on 32-bit Android + Closes: #13533 + Author: Christian Schmitz + Reviewed-by: Daniel Gustafsson - This was fixed in commit 06dc599405f, but came back in commit - 03cb1ff4d62. +Daniel Stenberg (3 May 2024) - When building for 32-bit ARM or x86 Android, `st_mode` is defined as - `unsigned int` instead of `mode_t`, resulting in a - `-Wimplicit-int-conversion` clang warning because `mode_t` is - `unsigned short`. Add a cast to silence the warning, but only for - 32-bit Android builds, because other architectures and platforms are - not affected. +- RELEASE-NOTES: synced - Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li - bc/include/sys/stat.h#86 - Closes https://github.com/curl/curl/pull/12998 +Jan Venekamp (2 May 2024) -Stefan Eissing (27 Feb 2024) +- bearssl: use common code for cipher suite lookup -- lib: Curl_read/Curl_write clarifications + Take advantage of the Curl_cipher_suite_walk_str() and + Curl_cipher_suite_get_str() functions introduced in commit fba9afeb. - - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to - clarify when and at what level they operate - - send/recv of transfer related data is now done via - `Curl_xfer_send()/Curl_xfer_recv()` which no longer has - socket/socketindex as parameter. It decides on the transfer - setup of `conn->sockfd` and `conn->writesockfd` on which - connection filter chain to operate. - - send/recv on a specific connection filter chain is done via - `Curl_conn_send()/Curl_conn_recv()` which get the socket index - as parameter. - - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for - naming consistency - - clarify that the special CURLE_AGAIN hangling to return - `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` - and CURLE_AGAIN is returned by all other send() variants. - - fix a bug in websocket `curl_ws_recv()` that mixed up data - when it arrived in more than a single chunk + This also fixes CURLOPT_SSL_CIPHER_LIST not working at all for bearssl + due to commit ff74cef5. - The method for sending not just raw bytes, but bytes that are either - "headers" or "body". The send abstraction stack, to to bottom, now is: + Closes #13464 - * `Curl_req_send()`: has parameter to indicate amount of header bytes, - buffers all data. - * `Curl_xfer_send()`: knows on which socket index to send, returns - amount of bytes sent. - * `Curl_conn_send()`: called with socket index, returns amount of bytes - sent. +Daniel Stenberg (2 May 2024) - In addition there is `Curl_req_flush()` for writing out all buffered - bytes. +- curl.h: change CURL_SSLVERSION_* from enum to defines - `Curl_req_send()` is active for requests without body, - `Curl_buffer_send()` still being used for others. This is because the - special quirks need to be addressed in future parts: + C++20 and later compilers emit a deprecation warning if values from two + different enums are combined with a bitwise operation the way the + CURL_SSLVERSION_* values were previously created. - * `expect-100` handling - * `Curl_fillreadbuffer()` needs to add directly to the new - `data->req.sendbuf` - * special body handlings, like `chunked` encodings and line end - conversions will be moved into something like a Client Reader. + Reported-by: Michael Kaufmann + Fixes #13510 + Closes #13511 - In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, - replace the `ssize_t` with a `size_t`. It makes no sense to allow for negativ - e - values as the returned `CURLcode` already specifies error conditions. This - allows easier handling of lengths without casting. +- configure: error on missing perl if docs or manual is enabled - Closes #12964 + Fixes #13508 + Reported-by: Harmen Stoppels + Closes #13514 -Daniel Stenberg (27 Feb 2024) +- tool_cb_rea: limit rate unpause for -T . uploads -- multi: make add_handle free any multi_easy + To avoid getting stuck in a busy-loop when nothing is read from stdin, + this function now checks the call rate and might enforce a short sleep + when called repeatedly without uploading anything. It is a crude + work-around to avoid a 100% busy CPU. - If the easy handle that is being added to a multi handle has previously - been used for curl_easy_perform(), there is a private multi handle here - that we can kill off. While it flushes some caches etc for the easy - handle would it be used for an easy interface transfer again after being - used in the multi stack, this cleanup simplifies behavior and uses less - memory. + Reported-by: magisterquis on hackerone + Fixes #13174 + Closes #13506 - Closes #12992 +Viktor Szakats (1 May 2024) -- docs: use present tense +- appveyor: enable websockets for VS2017 jobs - avoid "will", detect "will" as a bad word in the CI + Follow-up to eb4fe6c6340c3d5b0c347c6e30be004d4f9117d7 #13232 + Closes #13513 - Also line wrapped a bunch of paragraphs +Daniel Stenberg (30 Apr 2024) - Closes #13001 +- if2ip: make the buf_size arg a size_t -- CURLOPT_SSL_CTX_FUNCTION.md: no promises of lifetime after return + sizes should be size_t - ... and cleanup other language. + Ref: #13489 + Closes #13505 - Closes #12999 +- cf-https-connect: use timeouts as unsigned ints -Stefan Eissing (27 Feb 2024) + To match the type used in 'set.happy_eyeballs_timeout'. -- lib: send rework + Ref: #13489 + Closes #13503 - Curl_read/Curl_write clarifications +- hash: change 'slots' to size_t from int - - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify - when and at what level they operate + - an unsigned type makes more sense + - size_t seems suitable + - on 64 bit args, the struct alignment makes the new Curl_hash remain + the same size - - send/recv of transfer related data is now done via - `Curl_xfer_send()/Curl_xfer_recv()` which no longer has - socket/socketindex as parameter. It decides on the transfer setup of - `conn->sockfd` and `conn->writesockfd` on which connection filter - chain to operate. + Closes #13502 - - send/recv on a specific connection filter chain is done via - `Curl_conn_send()/Curl_conn_recv()` which get the socket index as - parameter. +Viktor Szakats (30 Apr 2024) - - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming - consistency +- libssh2: replace `access()` with `stat()` - - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` - with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is - returned by all other send() variants. + Prefer `stat()` to verify the presence of key files. - SingleRequest reshuffling + This drops the last uses of `access()` in the codebase, which was + reported to cause issues in some cases. - - move functions into request.[ch] - - differentiate between reset and free - - add Curl_req_done() to perform last actions - - add a send `bufq` to SingleRequest for future use in keeping upload data + Also add `access()` to the list of banned functions in checksrc. - Closes #12963 + Ref: https://github.com/curl/curl/pull/13412#issuecomment-2065505415 + Ref: https://github.com/curl/curl/pull/13482#issuecomment-2078980522 + Ref: #13497 + Co-authored-by: Jay Satiro + Closes #13498 -Daniel Stenberg (26 Feb 2024) +Daniel Stenberg (30 Apr 2024) + +- multi: remove useless assignment + + Spotted by CodeSonar + + Closes #13500 - RELEASE-NOTES: synced -- http_chunks: remove unused 'endptr' variable +fuzzard (29 Apr 2024) - Closes #12996 +- cmake: FindNGHTTP2 add static lib name to find_library call -Louis Solofrizzo (26 Feb 2024) + Add the static library name, nghttp2_static as a name to search. -- lib: initialize output pointers to NULL before calling strto[ff,l,ul] + This provides cmake parity with the winbuild Makefile.vc allowing + the cmake build to find and allow the link to static nghttp2 library. - In order to make MSAN happy: +Viktor Szakats (29 Apr 2024) - ==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x596f3b3ed246 in curlx_strtoofft [...]/libcurl/src/lib/strtoofft.c:23 - 9:11 - #1 0x596f3b402156 in Curl_httpchunk_read [...]/libcurl/src/lib/http_chunk - s.c:149:12 - #2 0x596f3b348550 in readwrite_data [...]/libcurl/src/lib/transfer.c:607: - 11 - [...] +- DISTROS: add patch and issues link for curl-for-win - ==2202041==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x5a3fab66a72a in Curl_parse_port [...]/libcurl/src/lib/urlapi.c:547:8 - #1 0x5a3fab650645 in parse_authority [...]/libcurl/src/lib/urlapi.c:796:1 - 2 - #2 0x5a3fab6740f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 - #3 0x5a3fab664fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: - 1342:12 - [...] + curl-for-win sometimes includes curl patches that were already merged in + master, but not yet part of a stable release. - ==2202320==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x569076a0d6b0 in ipv4_normalize [...]/libcurl/src/lib/urlapi.c:683:12 - #1 0x5690769f2820 in parse_authority [...]/libcurl/src/lib/urlapi.c:803:1 - 0 - #2 0x569076a160f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 - #3 0x569076a06fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: - 1342:12 - [...] + Also include the Issues link. Build-specific issues are handled there. - Signed-off-by: Louis Solofrizzo - Closes #12995 + Ref: #13493 + Closes #13499 -Stefan Eissing (26 Feb 2024) +Daniel Stenberg (29 Apr 2024) -- lib: move client writer into own source +- mime: avoid using access() - Refactoring of the client writer that passes the data to the - client/application's callback functions. + If stat() fails, there is no point in calling access() - - split out into own source cw-out.[ch] from sendf.c + Also: return error immediately if the stat() fails. - - move tempwrite and tempcount from data->state into the context of the - client writer + Ref: #13482 + Closes #13497 - - redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On - paused transfers, this allows to "record" interleaved HEADER/BODY - chunks to be "played back" in the same order on unpausing. +Stefan Eissing (29 Apr 2024) - - keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. - On exceeding that, return CURLE_TOO_LARGE instead of - CURLE_OUT_OF_MEMORY as before. +- tests: add SNI and peer name checks - - add method to be called when a transfer is DONE to allow writing of - any data still buffered + - connect to DNS names with trailing dot + - connect to DNS names with double trailing dot + - rustls, always give `peer->hostname` and let it + figure out SNI itself + - add SNI tests for ip address and localhost + - document in code and TODO that QUIC with ngtcp2+wolfssl + does not do proper peer verification of the certificate + - mbedtls, skip tests with ip address verification as not + supported by the library - - when paused, record HEADER writes exactly as they come for later - playback. HEADERs are documented to be written one-by-one. + Closes #13486 - Closes #12898 +Daniel Stenberg (29 Apr 2024) -- urldata: move authneg bit from conn to Curl_easy +- curl_getdate.md: document two-digit year handling - - from `conn->bits.authneg` to `data->req.authneg` - - this is a property of the request about to be made - and not a property of the connection - - in multiuse connections, transfer could step on each others - toes here potentially. + Mentioned-by: Paul Gilmartin + Ref: https://curl.se/mail/archive-2024-04/0014.html + Closes #13494 - Closes #12949 +Viktor Szakats (29 Apr 2024) -- c-hyper: add header collection writer in hyper builds +- cmake: add `BUILD_EXAMPLES` option to build examples - Closes #12880 + You can enable it with `-DBUILD_EXAMPLES=ON`. -- http: move headers collecting to writer + To match autotools' `make examples` feature. + Windows (static) builds not tested. - - add a client writer that does "push" response - headers written to the client if the headers api - is enabled - - remove special handling in sendf.c - - needs to be installed very early on connection - setup to catch CONNECT response headers + Also enable examples in a pair of CI jobs. - Closes #12880 + Apply related updates to the macOS CI workflow: + - drop unused `CXX` envs. + - drop no longer needed `-Wno-error=undef -Wno-error=conversion` flags. + - pass `-Wno-deprecated-declarations` to GCC too (for `BUILD_EXAMPLES`). + - document why `-Wno-deprecated-declarations` is necessary. -- sendf: Curl_client_write(), make passed in buf const + Closes #13491 -Michał Antoniak (26 Feb 2024) +Stefan Eissing (26 Apr 2024) -- lib: remove curl_mimepart object when CURL_DISABLE_MIME +- http3: quiche+ngtcp2 improvements - Remove curl_mimepart object from UserDefined structure when - CURL_DISABLE_MIME flag is active. Reduce size of UserDefined structure. + - quiche: error transfers that try to receive on a closed + or draining connection + - ngtcp2: use callback for extending max bidi streams. This + allows more precise calculation of MAX_CONCURRENT as we + only can start a new stream when the server acknowledges + the close - not when we locally have closed it. + - remove a fprintf() from h2-download client to avoid excess + log files on tests timing out. - Also remove unreachable code: when CURL_DISABLE_MIME is set, httpreq can - never have HTTPREQ_POST_MIME value and the same goes for the - CURL_DISABLE_FORM_API flag and the HTTPREQ_POST_FORM value + Closes #13475 - Closes #12948 +- vtls: TLS session storage overhaul -kpcyrd (26 Feb 2024) + - add session with destructor callback + - remove vtls `session_free` method + - let `Curl_ssl_addsessionid()` take ownership + of session object, freeing it also on failures + - change tls backend use + - test_17, add tests for SSL session resumption -- rustls: make curl compile with 0.12.0 + Closes #13386 - Closes #12989 +- multi: multi_wait improvements -Daniel Stenberg (26 Feb 2024) + - only call `multi_getsock()` once for all transfers + - realloc pollset array on demand + - fold repeated sockets -- strtoofft: fix the overflow check + Closes #13150 - ... to not rely on wrapping, since it is an undefined behavior that is - not what always might happen. This is in our private strtoff() parser - function, used only on platforms without a native version. +Philip Heiduck (25 Apr 2024) - Reported-by: vulnerabilityspotter on hackerone - Closes #12990 +- ci: remove microsoft-prod.list -- libssh/libssh2: return error on too big range + This is added by default, and it is often broken, but we don't need + anything from it. - If trying to get the range 0 - 2^63 and the remote file is 2^63 bytes or - larger. + Closes #13473 - Fixes #12983 - Closes #12984 +Evgeny Grin (Karlson2k) (25 Apr 2024) -Scott Talbert (24 Feb 2024) +- curl_setup.h: detect 'inline' support -- setopt: fix check for CURLOPT_PROXY_TLSAUTH_TYPE value + Closes #13355 - Prior to this change CURLOPT_PROXY_TLSAUTH_TYPE would return - CURLE_BAD_FUNCTION_ARGUMENT on any type other than NULL. Since there is - only one type of TLS auth and it is also the default (SRP) the TLS auth - would work anyway. +Daniel Stenberg (25 Apr 2024) - Closes https://github.com/curl/curl/pull/12981 +- multi: avoid memory-leak risk -Jay Satiro (24 Feb 2024) + 'newurl' is allocated in some conditions and used in a few scenarios, + but there were theoretical combinations in which it would not get freed. + Move the free to happen unconditionally. Never triggered by tests, but + spotted by Coverity. -- mprintf: fix format prefix I32/I64 for windows compilers + Closes #13471 - - Support I32 & I64 (eg: %I64d) for all Win32 builds. +Johann Sebastian Schicho (25 Apr 2024) - Prior to this change mprintf support for the I format prefix, which is a - Microsoft extension, was dependent on the compiler used. +- sendf: Curl_cwriter_write: remove comment disallowing zero length writes - When Borland compiler support was removed in fd7ef00f the prefix was - then no longer supported for that compiler; however since it's still - possible to build with Borland I'm restoring support for the prefix in - this way. + They are needed to pass CLIENTWRITE_EOS. - Reported-by: Paweł Witas + Closes #13477 - Fixes https://github.com/curl/curl/issues/12944 - Closes https://github.com/curl/curl/pull/12950 +Stefan Eissing (25 Apr 2024) -Daniel Stenberg (23 Feb 2024) +- CI: macos fixes for new ARM GHA images -- cd2nroff: gen: make `\>` in input to render as plain '>' in output + - based on #13478 with additions from #13476 + - make homebrew install path flexible + - fix OpenSSL pkgconfig files libdir + - add path to --with-libssh2 target + - disable gcc securetransport due to linker + errors (missing symbols), probably because + the os version is no longer low enough - The same (copy and pasted) fix/mistake as in gen.pl + Assisted-by: Viktor Szakats -- gen: make `\>` in input to render as plain '>' in output + Closes #13479 - Reported-by: Gisle Vanem - Fixes #12977 - Closes #12978 +- content_encoding: ignore duplicate chunked encoding -Fabrice Fontaine (23 Feb 2024) + - ignore duplicate "chunked" transfer-encodings from + a server to accomodate for broken implementations + - add test1482 and test1483 -- configure.ac: find libpsl with pkg-config + Reported-by: Mel Zuser + Fixes #13451 + Closes #13461 - Find libpsl with pkg-config to avoid static build failures. +Daniel Stenberg (25 Apr 2024) - Ref: http://autobuild.buildroot.org/results/1fb15e1a99472c403d0d3b1a688902f32 - e78d002 +- tool: move tool_ftruncate64 to tool_util.c - Signed-off-by: Fabrice Fontaine - Closes #12947 + ... and the prototype to tool_setup.h, to make them both available more + widely and accurately. -Daniel Stenberg (23 Feb 2024) + Follow-up to 00bef95946d3511 -- BUG-BOUNTY.md: clarify that the curl security team decides + Fixes #13458 + Closes #13459 - Closes #12975 +Viktor Szakats (24 Apr 2024) -- THANKS: add bug reporter from #740 +- lib: silence `-Wsign-conversion` in base64, strcase, mprintf - Ref: https://github.com/curl/curl/issues/740 + Closes #13467 -Stefan Eissing (22 Feb 2024) +- CI: retain failure code after `./configure` with Circle CI -- multi: fix multi_sock handling of select_bits + Suggested-by: Dan Fandrich + Follow-up to 43299e93c06b96fea8a8dc9b1c2e49c82bc21801 #13462 + Follow-up to d7332e3e46c3ef401b34e6a1a129eb4dd846c452 #12635 + Closes #13468 - - OR the event bitmask to data->state.select_bits instead of overwriting - them. They are cleared again on use. +Daniel Stenberg (24 Apr 2024) - Reported-by: 5533asdg on github - Fixes #12971 - Closes #12972 +- RELEASE-NOTES: synced -Daniel Stenberg (22 Feb 2024) +Jan Venekamp (24 Apr 2024) -- curlver: bump to 8.7.0 for next release +- mbedTLS: implement CURLOPT_SSL_CIPHER_LIST option -- RELEASE-NOTES: synced + Use a lookup list to set the cipher suites, allowing the + ciphers to be set by either openssl or IANA names. -- write-out: add '%{proxy_used}' + To keep the binary size of the lookup list down we compress + each entry in the cipher list down to 2 + 6 bytes using the + C preprocessor. - Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to - for example determine if a `NOPROXY` pattern matched the hostname or - not. + Closes #13442 - Extended test 970 and 972 +Viktor Szakats (24 Apr 2024) -- CURLINFO_USED_PROXY: return bool whether the proxy was used +- CI: show more failed `config.log` on Circle CI - Adds test536 to verify + Show last 1000 lines of `config.log` if `./configure` fails. This was + already done for one job, this patch extends it to all. - Closes #12719 + Ref: #13438 + Closes #13462 -- sha512_256: remove the cast macro, minor language/format edits +Daniel Stenberg (24 Apr 2024) - Follow-up to cbe41d151d6a100c +- telnet: check return code from fileno() - Closes #12966 + and return error if necessary -Stefan Eissing (20 Feb 2024) + Spotted by CodeSonar -- DoH: add trace configuration + Closes #13457 - - refs #12397 where it is dicussed how to en-/disable verbose output - of DoH operations - - introducing `struct curl_trc_feat` to track a curl feature for - tracing - - adding `data->state.feat` optionally pointing to the feature a - transfer belongs to - - adding trace functions and verbosity checks on features - - using trace feature in DoH code - - documenting `doh` as feature for `--trace-config` +Viktor Szakats (24 Apr 2024) - Closes #12411 +- tls: fix SecureTransport + BearSSL cmake unity builds -- websocket: fix curl_ws_recv() + Avoid clashing static function names by namespacing them. - - when data arrived in several chunks, the collection into - the passed buffer always started at offset 0, overwriting - the data already there. + Pointed-out-by: Jan Venekamp + Ref: https://github.com/curl/curl/pull/13442#discussion_r1576350700 + Closes #13450 - adding test_20_07 to verify fix +Jay Satiro (24 Apr 2024) - - debug environment var CURL_WS_CHUNK_SIZE can be used to - influence the buffer chunk size used for en-/decoding. +- dllmain: Call OpenSSL thread cleanup for Windows and Cygwin - Closes #12945 + - Call OPENSSL_thread_stop on thread termination (DLL_THREAD_DETACH) + to prevent a memory leak in case OpenSSL is linked statically. -Evgeny Grin (Karlson2k) (20 Feb 2024) + - Warn in libcurl-thread.3 that if OpenSSL is linked statically then it + may require thread cleanup. -- digest: support SHA-512/256 + OpenSSL may need per-thread cleanup to stop a memory leak. For Windows + and Cygwin if libcurl was built as a DLL then we can do that for the + user by calling OPENSSL_thread_stop on thread termination. However, if + libcurl was built statically then we do not have notification of thread + termination and cannot do that for the user. - Also fix the tests. New implementation tested with GNU libmicrohttpd. - The new numbers in tests are real SHA-512/256 numbers (not just some - random ;) numbers ). + Also, there are several other unusual cases where it may be necessary + for the user to call OPENSSL_thread_stop, so in the libcurl-thread + warning I added a link to the OpenSSL documentation. -- tests: add SHA-512/256 unit test + Co-authored-by: Viktor Szakats -- SHA-512/256: implement hash algorithm + Reported-by: southernedge@users.noreply.github.com + Reported-by: zmcx16@users.noreply.github.com - Closes #12897 + Ref: https://www.openssl.org/docs/man3.0/man3/OPENSSL_thread_stop.html#NOTES -- curl_setup.h: add curl_uint64_t internal type + Fixes https://github.com/curl/curl/issues/12327 + Closes https://github.com/curl/curl/pull/12408 - The unsigned version of curl_off_t basically +Jan Venekamp (24 Apr 2024) -Daniel Stenberg (20 Feb 2024) +- rustls: remove incorrect SSLSUPP_TLS13_CIPHERSUITES flag -- docs: dist curl*.1 and install without perl + The rustls backend advertises SSLSUPP_TLS13_CIPHERSUITES, but + the code does not actually seem to support it (yet?). Removed + the flag and corrected documentation. - Drop docs/mk-ca-bundle.1 from the tarball. It can be generated at will. + Closes #13452 - Closes #12959 - Fixes #12921 - Reported-by: Michael Forney +Stefan Eissing (24 Apr 2024) -Stefan Eissing (20 Feb 2024) +- quiche: expire all active transfers on connection close -- OpenSSL QUIC: adapt to v3.3.x + - when a connection close is detected, all ongoing transfers + need to expire bc no more POLL events are likely to happen + for them. - - set our idle timeout as transport parameter - - query negotiated idle timeout for connection alive checks - - query number of available bidi streams on a connection - - use write_ex2 with SSL_WRITE_FLAG_CONCLUDE to signal - EOF on last chunk write, so stream close does not - require an additional QUIC packet + Fixes #13439 + Reported-by: Jay Satiro + Closes #13447 - Closes #12933 +Dan Fandrich (23 Apr 2024) -Ramiro Garcia (19 Feb 2024) +- tests: fix feature case in test1481 -- MANUAL.md: fix typo + This test was being skipped everywhere because the feature never + matched. - Closes #12965 + Closes #13445 -Daniel Stenberg (19 Feb 2024) +Gusted (23 Apr 2024) -- BINDINGS: add mcurl, the python binding +- tool_operate: don't truncate the etag save file by default - Ref: #12956 - Closes #12962 + This fixes a regression of 75d79a4486b279100209ddf8c7fdb12955fb66e9. The + code in tool-operate truncated the etag save file, under the assumption + that the file would be written with a new etag value. However since + 75d79a4486b279100209ddf8c7fdb12955fb66e9 that might not be the case + anymore and could result in the file being truncated when --etag-compare + and --etag-save was used and that the etag value matched with what the + server responded. Instead the truncation should not be done when a new + etag value should be written. -- mk-ca-bundle.md: cleanups and polish + Test 3204 was added to verify that the file with the etag value doesn't + change the contents when used by --etag-compare and --etage-save and + that value matches with what the server returns on a non 2xx response. - Closes #12958 + Closes #13432 -- spellcheck.yml: remove .1/.3 handling, clean all man page .md files +Abdullah Alyan (22 Apr 2024) - Since we generate all .1 and .3 files from markdown now, we can limit - the spellcheck to the markdown versions only. +- tests: enable test 1117 for hyper - Closes #12960 + Closes #13436 -- libcurl-docs: cleanups +Daniel Stenberg (22 Apr 2024) - CURLMOPT_SOCKETDATA.md: fix typo - CURLMOPT_TIMERDATA.md: fix typo - CURLOPT_COOKIELIST.m: quote strings - CURLOPT_PREREQFUNCTION.md: quote variable names - CURLOPT_TCP_NODELAY.md: rephrased to please spell checker - CURLOPT_WILDCARDMATCH.md: rephrased - libcurl-tutorial.md: use correct option name - curl_global_init_mem.md: quote headers - curl_easy_getinfo.md: use correct symbol names in headers - curl_global_trace.md: quote some headers - curl_ws_meta.md: quote struct field names - libcurl-env.md: quote headers +- sendf: useless assignment in cr_lc_read() -- cd2nroff: remove backticks from titles + Spotted by CodeSonar -- RELEASE-NOTES: synced + Closes #13437 -Stefan Eissing (18 Feb 2024) +- tool_paramhlp: remove duplicate assign -- http_chunks: fix the accounting of consumed bytes + Spotted by CodeSonar - Prior to this change chunks were handled correctly although in verbose - mode libcurl could incorrectly warn of "Leftovers after chunking" even - if there were none. + Closes #13433 - Reported-by: Michael Kaufmann +- transfer: remove useless assignment - Fixes https://github.com/curl/curl/issues/12937 - Closes https://github.com/curl/curl/pull/12939 + in Curl_xfer_recv_resp -- file: use xfer buf for file:// transfers + Spotted by CodeSonar - - For file:// transfers use the multi handle's transfer buffer for - up- and downloads. + Closes #13435 - Prior to this change a6c9a33 (precedes 8.6.0) changed the file:// - transfers to use a smaller stack based buffer, and that caused a - significant performance decrease in Windows. +- http: acknowledge a returned error code - Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086 - Reported-by: edmcln@users.noreply.github.com + ... and do not overwrite it with a new value that could then hide the + problem. - Closes https://github.com/curl/curl/pull/12932 + Spotted by CodeSonar -Karthikdasari0423 (18 Feb 2024) + Closes #13434 -- HTTP3.md: always run nghttp3 submodule init +- tool_operate: init vars unconditionally in post_per_transfer - - For consistency change all 'build nghttp3' commands to run submodule - init after cloning, even if the branch does not have submodules. + In case of (the unlikely) early return, they could otherwise remain + uninitialized - Follow-up to 5a4b2f93 and 4f794558. + Spotted by CodeSonar - Closes https://github.com/curl/curl/pull/12928 + Closes #13430 -LeeRiva (18 Feb 2024) +- RELEASE-NOTES: synced -- CURLOPT_POSTQUOTE.md: fix typo +- urlapi: allow setting port number zero - Closes https://github.com/curl/curl/pull/12926 + Also set and check errno when strtoul() parsing numbers for better error + checking. -Evgeny Grin (Karlson2k) (18 Feb 2024) + Updated test 1560 -- checksrc.pl: fix handling .checksrc with CRLF + Closes #13427 - - When parsing .checksrc chomp the (CR)LF line ending. +- http_aws_sigv4: remove useless assignment - Prior to this change on Windows checksrc.pl would not process the - symbols in .checksrc properly, since many git repos in Windows use auto - crlf to check out files with CRLF line endings. + This code assigned the variable the same value it already had - Closes https://github.com/curl/curl/pull/12924 + Spotted by CodeSonar -Richard Levitte (18 Feb 2024) + Closes #13426 -- cmake: fix install for older CMake versions +- file: remove useless assignment - - Generate the docs install list by using a foreach loop instead of - LIST:TRANSFORM since older CMake can't handle the latter. + This code assigned the variable the same value it already had. - Reported-by: Dan Fandrich + Spotted by CodeSonar - Fixes https://github.com/curl/curl/issues/12920 - Closes https://github.com/curl/curl/pull/12922 + Closes #13425 -Stefan Eissing (16 Feb 2024) +- test2406: verify -f with HTTP/2 -- vtls: fix tls proxy peer verification +Stefan Eissing (19 Apr 2024) - - When verifying a proxy certificate for an ip address, use the correct - ip family. +- http2 + ngtcp2: pass CURLcode errors from callbacks - Prior to this change the "connection" ip family was used, which was not - necessarily the same. + - errors returned by Curl_xfer_write_resp() and the header variant are + not errors in the protocol. The result needs to be returned on the + next recv() from the protocol filter. - Reported-by: HsiehYuho@users.noreply.github.com + - make xfer write errors for response data cause the stream to be + cancelled - Fixes https://github.com/curl/curl/issues/12831 - Closes https://github.com/curl/curl/pull/12931 + - added pytest test_02_14 and test_02_15 to verify that also for + parallel processing -Dan Fandrich (15 Feb 2024) + Reported-by: Laramie Leavitt + Fixes #13411 + Closes #13424 -- CI: Bump the Circle CI base Ubuntu image to the latest 20.04 +Daniel Stenberg (19 Apr 2024) - The previous ones are going to be removed soon, plus the new ones - include all the fixes since then. +- request: make Curl_req_init return void -Jay Satiro (13 Feb 2024) + Since it could not return error and therefore this change removes dead + code for the caller. -- transfer: improve Windows SO_SNDBUF update limit + Spotted by CodeSonar. - - Change the 1 second SO_SNDBUF update limit from per transfer to per - connection. + Closes #13423 - Prior to this change many transfers over the same connection could cause - many SO_SNDBUF updates made to that connection per second, which was - unnecessary. +- multi: remove the unused Curl_preconnect function - Closes https://github.com/curl/curl/pull/12911 + The implementation has been removed, no point in keeping it around. -- schannel: fix hang on unexpected server close + Follow-up to 476adfeac019ed - - Treat TLS connection close (either due to a close_notify from the - server or just closed due to receiving 0) as pending data. + Closes #13422 - This is because in some cases schannel_recv knows the connection is - closed but has to return actual pending data so it can't return 0 or an - error to indicate no more data. In this case schannel_recv must be - called again, which only happens if readwrite_data sees that there is - still pending data. +- Curl_creader_read: init two variables to avoid using them uninited - Prior to this change if the total size of the body that libcurl expected - to receive from the server was unknown then it was possible under some - network conditions that libcurl would hang waiting to receive more data, - when in fact a close_notify alert indicating no more data would be sent - was already processed. + Spotted by CodeSonar - Fixes https://github.com/curl/curl/issues/12894 - Closes https://github.com/curl/curl/pull/12910 + Closes #13419 -Daniel Stenberg (10 Feb 2024) +- http: reject HTTP major version switch mid connection -- KNOWN_BUGS: FTP upload fails if remebered dir is deleted + A connection that has seen an HTTP major version now refuses any other + major HTTP version in future responses. Previously, a HTTP/1.x + connection would just silently accept HTTP/2 or HTTP/3 in the status + lines as long as it had support for those built-in. It would then just + lead to confusion and badness. - Closes #12181 - Closes #12923 + Indirectly Spotted by CodeSonar which identified a duplicate assignment + in this function. -Michał Antoniak (10 Feb 2024) + Add test 471 to verify -- mbedtls: use mbedtls_ssl_conf_{min|max}_tls_version + Closes #13421 - ... instead of the deprecated mbedtls_ssl_conf_{min|max}_version +- mqtt: when Curl_xfer_recv returns error, don't use nread - Closes #12905 + A returned error code makes other return value unreliable, and in this + case potentially uninitialized. On error, do not read other return + values like the nread counter. -Dan Fandrich (9 Feb 2024) + Spotted by CodeSonar -- CI: bump to actions/cache@v4 to avoid warning + Closes #13418 -Evgeny Grin (Karlson2k) (9 Feb 2024) +- ftp: fix socket leak on rare error -- test1165: improve pattern matching + In the function AcceptServerConnect() the newly created socket would + leak if Curl_conn_tcp_accepted_set() returns error. Which basically + should never happen. - * Fix excluded digits at the end of the symbols ('CURL_DISABLE_POP3' - was checked as 'CURL_DISABLE_POP') + Spotted by CodeSonar. - Closes #12903 + Closes #13417 -Dan Fandrich (9 Feb 2024) +- urlapi: remove unused flags argument from Curl_url_set_authority -- scripts: Fix cijobs.pl for Azure and GHA + The function is only called from a single place (for HTTP/2 server push) + so might as well just assume this fixed option every time. - The spacing in the yaml files changed. + Closes #13409 -Daniel Stenberg (9 Feb 2024) +- github/ISSUE_TEMPLATE: tweak the commericual support text -- RELEASE-NOTES: synced +- github/ISSUE_TEMPLATE: link the GitHub discussions too -- TODO: use pkg-config to find libpsl + ... and move the feature request line to the bottom. - Closes #12919 +- curl_url_get.md: clarify queries and fragments and CURLU_GET_EMPTY -- TODO: avoid nroff + Follow-up to 3eac21d86bc5 - Instead of adjusting roffit, skip the nroff step. + Closes #13407 - Closes #12919 +Stefan Eissing (18 Apr 2024) -Dan Fandrich (9 Feb 2024) +- tests: check caddy server version to match test expectations -- Revert "CI: run Circle macOS builds on x86 for now" + - new caddy servers no longer return 200 on POSTs, but 405 + as they should - This reverts commit 2683de3078eadc86d9b182e7417f4ee75a247e2c. - ARM resources are now available in Circle CI, so run these builds on ARM - again. This platform needs explicit paths set to libpsl and its - dependency icu4c. + Closes #13405 - Follow-up to 2683de30 +Daniel Stenberg (18 Apr 2024) - Closes #12635 +- curl_url_set.md: extended -Viktor Szakats (9 Feb 2024) + Closes #13404 -- cmake: add warning for using TLS libraries without 1.3 support +- urlapi: add CURLU_GET_EMPTY for empty queries and fragments - Closes #12900 + By default the API inhibits empty queries and fragments extracted. + Unless this new flag is set. -Daniel Stenberg (9 Feb 2024) + This also makes the behavior more consistent: without it set, zero + length queries and fragments are considered not present in the URL. With + the flag set, they are returned as a zero length strings if they were in + fact present in the URL. -- configure: add warning for using TLS libraries without 1.3 support + This applies when extracting the individual query and fragment + components and for the full URL. - Closes #12900 + Closes #13396 -Michał Antoniak (9 Feb 2024) +- RELEASE-NOTES: synced -- mbedtls: fix building when MBEDTLS_X509_REMOVE_INFO flag is defined +- lib1560: test with leading zeroes and more IPv4 versions - Closes #12904 + Inspired by WHATWG URL Spec test inputs -Stefan Eissing (9 Feb 2024) + Closes #13400 -- ftp: fix socket wait activity in ftp_domore_getsock +Christian Schmitz (17 Apr 2024) - - when waiting on the data connection, always add the control socket to - the pollset on state STOP or let the pingpong add the socket according - to its needs. +- smtp: result of Curl_bufq_cread was not used - Reported-by: Fabian Vogt - Fixes #12901 - Closes #12913 + return the result back to the caller. -Daniel Stenberg (9 Feb 2024) + Closes #13398 -- dist: make sure the http tests are in the tarball +Daniel Stenberg (17 Apr 2024) - Fixes #12914 - Reported-by: Fabian Vogt - Closes #12917 +- urlapi: fix relative redirects to fragment-only -Stefan Eissing (9 Feb 2024) + Using the URL API for a redirect URL when the redirected-to string + starts with a hash, ie is only a fragment, the API would produce the + wrong final URL. -- multi: add xfer_buf to multi handle + Adjusted test 1560 to test for several new redirect cases. - - can be borrowed by transfer during recv-write operation - - needs to be released before borrowing again - - adjustis size to `data->set.buffer_size` - - used in transfer.c readwrite_data() + Closes #13394 - Closes #12805 +Jiwoo Park (17 Apr 2024) -Daniel Stenberg (9 Feb 2024) +- url: fix use of an uninitialized variable -- write-out.md: clarify error handling details + Closes #13399 - - it gets used even if the transfer fails +Patrick Monnerat (17 Apr 2024) - - it does not cause error to be returned even if it fails +- os400: sync with latest changes - Closes #12909 + - Conversion support for new version info character field rtmp_version. + - New ILE/RPG declarations. -Stefan Eissing (8 Feb 2024) + Closes #13402 -- ftp: do lineend conversions in client writer +Daniel Stenberg (17 Apr 2024) - - remove the ftp special handling from sendf.c - - let ftp_do() add a client writer that does - the linened conversions - - change the lineend conversion to no longer - modify the passed buffer, but write smaller - chunks to the next cwriter instead. The - inefficiency of this will be mitigated once - we add output buffering for all client writes. +- ngtcp2: fix macro use - Closes #12878 + macro "H3_STREAM_CTX" requires 2 arguments, but only 1 given -- ftp: tracing improvements + Follow-up to c6655f7029ec5c128561e3ecf1f93db3ed0432a4 - - trace socketindex for connection filters when not the first - - trace socket fd in tcp - - trace pollset adjusts in vtls + Closes #13401 - Closes #12902 +Christian Schmitz (17 Apr 2024) -Karthikdasari0423 (8 Feb 2024) +- sendf: fix two typos in comments -- HTTP3.md: adjust the OpenSSL QUIC install instructions + The parameters are named data, not date. - tried installing with old steps but failed - tried with newly added setps and able to build - ``` - root@ubuntu:~/curl# ./src/curl -V - /root/curl/src/.libs/curl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSS - L_3.2.0' not found (required by /root/curl/lib/.libs/libcurl.so.4) - root@ubuntu:~/curl# - ``` - ``` - root@ubuntu:~/curl# ./src/curl -V - curl 8.6.1-DEV (x86_64-pc-linux-gnu) libcurl/8.6.1-DEV OpenSSL/3.2.0 zlib/1.2 - .11 brotli/1.0.9 libpsl/0.21.0 nghttp3/1.1.0 OpenLDAP/2.5.16 - Release-Date: [unreleased] - Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns - ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp - Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz - NTLM PSL SSL threadsafe TLS-SRP UnixSockets - root@ubuntu:~/curl# - ``` + Closes #13393 - Closes #12896 +- lib: silence warnings on comma misuse -Daniel Stenberg (8 Feb 2024) + Building curl with -Wcomma, I see warnings about "possible misuse of + comma operator here" and moving fields assignment out of the for() fixes + it. -- TODO: align the TOC with the header + Closes #13392 -- docs: make sure curl.1 is included in dist tarballs +Stefan Eissing (17 Apr 2024) - Ref: https://github.com/curl/curl/issues/12832#issuecomment-1933271873 +- http/2, http/3: decouple stream state from easy handle - Closes #12892 + - add `Curl_hash_offt` as hashmap between a `curl_off_t` and + an object. Use this in h2+h3 connection filters to associate + `data->id` with the internal stream state. + - changed implementations of all affected connection filters + - removed `h2_ctx*` and `h3_ctx*` from `struct HTTP` and thus + the easy handle + - solves the problem of attaching "foreign protocol" easy handles + during connection shutdown -Karthikdasari0423 (8 Feb 2024) + Test 1616 verifies the new hash functions. -- HTTP3.md: remove quiche word in Openssl 3.2 + Closes #13204 - Closes #12893 +Daniel Stenberg (17 Apr 2024) -Daniel Stenberg (7 Feb 2024) +- ROADMAP: remove completed entries, mention websocket -- curl: when allocating variables, add the name into the struct +- THANKS-filter: name fixes - This saves the name from being an extra separate allocation. +Christian Schmitz (17 Apr 2024) - Closes #12891 +- winbuild: add ENABLE_WEBSOCKETS option -- lib582: remove code causing warning that is never run + Closes #13232 - The previous realloc code in this code could trigger a compiler warning, - but since that code path cannot happen in normal circumstances it now - instead exits with an error message there. +Daniel Stenberg (17 Apr 2024) - Ref: #12887 - Closes #12890 +- dmaketgz: compacter -Stefan Eissing (7 Feb 2024) + Removes the need for disabling shellcheck warnings. -- vtls: revert "receive max buffer" + add test case + Follow-up to d28f74913c2 + Proposed-by: Viktor Szakats + Closes #13391 - - add test_05_04 for requests using http/1.0, http/1.1 and h2 against an - Apache resource that does an unclean TLS shutdown. - - revert special workarund in openssl.c for suppressing shutdown errors - on multiplexed connections - - vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53 +Dan Fandrich (16 Apr 2024) - Fixes #12885 - Fixes #12844 +- tests: Fix uninitialized value warning - Closes #12848 + The check for an option must be predicated on options existing at all. -Daniel Stenberg (7 Feb 2024) + Follow-up to f7cc9e91 -- tests: support setting/using blank content env variables +Christian Schmitz (17 Apr 2024) - - test450: remove --config from the keywords - - test2080: change return code - - test428: add --config as a keyword - - test428: disable on Windows due to CI problems +- idn: add native AppleIDN (icucore) support for macOS/iOS -- curl: exit on config file parser errors + I implemented the IDN functions for macOS and iOS using Unicode + libraries coming with macOS and iOS. - Like when trying to import an environment variable that does not exist. + Builds and runs here on macOS 14.2.1. Also verified to load and + run on older macOS version 10.13. - Also fix a bug for reading env variables when there is a default value - set. + Build requires macOS SDK 13 or equivalent. - Bug: https://curl.se/mail/archive-2024-02/0008.html - Reported-by: Brett Buddin + Set `-DUSE_APPLE_IDN=ON` CMake option to enable it. + With autotools and other build tools, set these manual options: + ``` + CPPFLAGS=-DUSE_APPLE_IDN + LIBS=-licucore + ``` - Add test 462 to verify. + Completes TODO 1.6. - Closes #12862 + TODO: add autotools option and feature-detection. -Daniel Szmulewicz (7 Feb 2024) + Refs: #5330 #5371 + Co-authored-by: Viktor Szakats + Closes #13246 -- CURLOPT_WRITEFUNCTION.md: typo fix +Stefan Eissing (16 Apr 2024) - The maximum amount of body data that is passed to the write - callback is defined in the curl.h header file +- http3: extend download abort tests, fixes in ngtcp2 - Closes #12889 + - fix flow handling in ngtcp2 to ACK data on streams + we abort ourself. + - extend test_02_23* cases to also run for h3 + - skip test_02_23* for OpenSSL QUIC as it gets stalled + on progressing the connection -Daniel Stenberg (7 Feb 2024) + Closes #13374 -- lib: convert Curl_get_line to use dynbuf +Daniel Stenberg (16 Apr 2024) - Create the line in a dynbuf. Aborts the reading of the file on - errors. Avoids having to always allocate maximum amount from the - start. Avoids direct malloc. +- tests: add -q as first option when invoking curl for tests - Closes #12846 + To reduce the risk that the user running the tests has a .curlrc present + that messes things up. -- KNOWN_BUGS: unicode on Windows + Support 'option="no-q"' for the tag to switch it off on demand. + Use this new feature in test 433 and 436. - Closes #11461 - Closes #12231 - Closes #12883 + Ref: #13284 + Closes #13387 -- tool_operate: change precedence of server Retry-After time +- dmaketgz: release tarball generation using docker - - When calculating the retry time, no longer allow a server's requested - Retry-After time to take precedence over a longer retry time (either - default algorithmic or user-specified). + For easier reproducibility. - Prior to this change the server's Retry-After time took precedence over - curl's retry time in all cases, but that's not always practical for - short Retry-After times depending on how busy the server is. + Mention using this script in RELEASE-PROCEDURE - Bug: https://curl.se/mail/archive-2024-01/0022.html - Reported-by: Dirk Hünniger + Closes #13388 - Closes https://github.com/curl/curl/pull/12871 +Viktor Szakats (16 Apr 2024) -- cmdline-docs: quote and angle bracket cleanup +- cmake: update ECH code and minor fixups - - make sure angle brackets are escaped - - remove a lot of superfluous double quotes - - replace several double quotes with backticks + - `openssl_check_symbol_exists()` expects a 4th argument now. + Follow-up to edc2702a1fe3a4a5386ffd9aa4f240f0c0197fa2 #13373 - To make nicer-looking markdown. + - minor comment/script touch-ups. + Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 - Closes #12884 + - fix indentation. -- badwords: use hostname, not host name + Closes #13383 - and username, filename - consistently. Fixed the patterns in - badwords.txt to catch these. +- tests: fix shellcheck issues in `ech_tests.sh` - Closes #12888 + Add double-quotes where missing. -Viktor Szakats (6 Feb 2024) + Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 + Closes #13382 -- cmake: fix function description in comment [ci skip] +- dist: add ECH files to tarball - Closes #12879 + Also sort `EXTRA_DIST` list in `tests/Makefile.am` and make it diffable. -Daniel Stenberg (6 Feb 2024) + Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 + Closes #13381 -- header.md: remove backslash, make nicer markdown +- openvms: look for `USE_IPV6` in `config.h` (was: `ENABLE_IPV6`) - - remove a leftover backslash before a dash - - use backticks for "code" strings + The OpenVMS script `config_h.com` is parsing the config header + generated by autotools. Let's make it look for the macro name we now + use universally across the codebase. - Closes #12877 + Follow-up to e411c98f702f0fb38dceec95e7507ef15a00d12c #13349 + Closes #13360 -- docs: add mk-ca-bundle.1 to dist +daniel-j-h (16 Apr 2024) - ... which also makes it get built. But don't build this or curl-config.1 - if build docs is disabled. +- Dockerfile: for release automation and reproducibility - Closes #12875 + Closes #13250 -Stefan Eissing (6 Feb 2024) +Stefan Eissing (16 Apr 2024) -- https-proxy: use IP address and cert with ip in alt names +- cw-out: improved error handling - - improve info logging when peer verification fails to indicate - if DNS name or ip address has been tried to match - - add test case for contacting https proxy with ip address - - add pytest env check on loaded credentials and re-issue - when they are no longer valid - - disable proxy ip address test for bearssl, since not supported there + - remember error encountered in invoking write callback and always fail + afterwards without further invokes - Ref: #12831 - Closes #12838 + - check behaviour in test_02_17 with h2-pausing client -Jiawen Geng (6 Feb 2024) + Reported-by: Pavel Kropachev + Fixes #13337 + Closes #13340 -- docs: add necessary setup for nghttp3 +Daniel Stenberg (16 Apr 2024) - Now nghttp3 has submodules - https://github.com/ngtcp2/nghttp3/blob/main/.gitmodules +- version: add "ECH" as a feature - Closes #12859 + If available -Peter Krefting (6 Feb 2024) + Follow-up to a362962b7 + Closes #13378 -- version: allow building with ancient libpsl +- CURLOPT_ECH: polish - The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7 - ships with version 0.7.0 which lacks this API. Revert to using the older - versioning API if we detect an old libpsl version. + - remove the pointer to build instructions, it won't work in manpages + - add see-also + - minor white space edits - Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4 - Bug: https://curl.se/mail/archive-2024-02/0004.html - Reported-by: Scott Mutter - Closes #12872 + Closes #13379 -Daniel Stenberg (6 Feb 2024) +Viktor Szakats (16 Apr 2024) -- TODO: Support latest rustls +- tidy-up: whitespace [ci skip] - Closes #12737 - Closes #12874 +- mbedtls: fix building with v3 in CMake Unity mode -- docs: make curldown do angle brackets like markdown + Before this patch the internal feature detection macro + `HAS_MBEDTLS_RESULT_CODE_BASED_FUNCTIONS` was defined in three files, + with an incomplete logic in one of them. In Unity mode that spilled + into another source file and broke the build. - Make sure we use \< and \> in markdown all over so that it renders - correctly, on GitHub and elsewhere. cd2nroff now outputs a warning if it - finds an unescaled angle bracket. + Closes #13377 - Ref: #12854 - Closes #12869 +- cmake: add librtmp/rtmpdump option and detection -- docs: fix the --disable-docs for autotools + Add CMake option `USE_LIBRTMP`. Disabled by default. - Follow-up to 541321507e386 + This library requires OpenSSL TLS-backend when linked statically. - Closes #12870 + Follow-up to 6eb9e65781fa1fd8a0bcfe0715187a3a35f09ae4 #13364 + Closes #13373 -- RELEASE-NOTES: synced +Stephen Farrell (16 Apr 2024) -- libcurl-security.md: Active FTP passes on the local IP address +- TLS: add support for ECH (Encrypted Client Hello) - Reported-by: Harry Sintonen - Closes #12867 + An EXPERIMENTAL feature used with CURLOPT_ECH and --ech. -Stefan Eissing (5 Feb 2024) + Closes #11922 -- configure: do not link with nghttp3 unless necessary +Daniel Stenberg (15 Apr 2024) - Fixes #12833 - Closes #12864 - Reported-by: Ryan Carsten Schmidt +- RELEASE-NOTES: synced -Daniel Stenberg (5 Feb 2024) +- multi: introduce SETUP state for better timeouts -- THANKS: add Dmitry Tretyakov + Since we can go to the CONNECT state from PENDING, potentially multiple + times for a single transfer, this change introdues a SETUP state that + happens before CONNECT when doing a new transfer. - ... since I missed to give credit to the report in the fix of #12861 + Now, doing a redirect on a handle goes back to SETUP (not CONNECT like + before) and we initilize the connect timeout etc in SETUP. Previously, + we would do it in CONNECT but that would make it unreliable in cases + where a transfer goes in and out between CONNECT and PENDING multiple + times. -Stefan Eissing (5 Feb 2024) + SETUP is transient, so the handle never actually stays in that state. -- openssl-quic: check on Windows that socket conv to int is possible + Additionally: take care of timeouts of PENDING transfers in + curl_multi_perform() - Fixes #12861 - Closes #12865 + Ref: #13227 + Closes #13371 -Daniel Stenberg (5 Feb 2024) +Tal Regev (15 Apr 2024) -- tool_cb_hdr: only parse etag + content-disposition for 2xx +- cmake: forward `USE_LIBRTMP` option to C - ... and ignore them for other response codes. + Define in C `USE_LIBRTMP` if user requested it from cmake. - Reported-by: Harry Sintonen - Closes #12866 + Closes #13364 -- md4: include strdup.h for the memdup proto +Daniel Stenberg (15 Apr 2024) - Reported-by: Erik Schnetter - Fixes #12849 - Closes #12863 +- curl_version_info: provide librtmp version -Joel Depooter (5 Feb 2024) + Ref: https://github.com/curl/curl/pull/13364#issuecomment-2054151942 + Reported-by: talregev on github + Closes #13368 -- docs: add missing slashes to SChannel client certificate documentation +blankie (15 Apr 2024) - When setting the CURLOPT_SSLCERT option to a certificate thumprint, it - is required to have a backslash between the "store location", "store - name" and "thumbprint" tokens. These slashes were present in the - previous documentation, but were missed in the transition to markdown - documentation. +- docs: clarify CURLOPT_MAXFILESIZE and CURLOPT_MAXFILESIZE_LARGE - Closes #12854 + The bounds of the size parameter were not specified, and nor was it + specified how to disable the maximum file size check. -Stefan Eissing (5 Feb 2024) + The documentation also incorrectly stated that CURLOPT_MAXFILESIZE + always returns CURLE_OK and that CURLOPT_MAXFILESIZE_LARGE only returns + CURLE_OK or CURLE_UNKNOWN_OPTION. -- HTTP/2: write response directly + It also did not mention what the default value is, which is zero. This + commit updates the documentation to make note of all these things. - - use the new `Curl_xfer_write_resp()` to write incoming responses - directly to the client - - eliminates `stream->recvbuf` - - memory consumption on parallel transfers minimized + Closes #13372 - Closes #12828 +Patrick Monnerat (15 Apr 2024) -Daniel Stenberg (5 Feb 2024) +- OS400: post-shellcheck changes adjustments -- cookie.md: provide an example sending a fixed cookie + Build scripts must be executed by the os/400 shell (sh), not bash which + is a PASE program. - Closes #12868 + Shell function get_make_vars() escaping reworked to match $() subcommand + construct. -Lars Kellogg-Stedman (5 Feb 2024) + Follow-up to 8a622baf9e9233241bbe93d6599c99cb46478614 + Closes #13366 -- ALTSVC.md: correct a typo +Viktor Szakats (15 Apr 2024) - The ALPN documentation erroneously referred to a "host number" instead - of a "port number". +- OS400: tidy-up - Closes #12852 + Drop/fixup mods trying to make some syntax highlighters happier. -Boris Verkhovskiy (5 Feb 2024) + Follow-up to 8a622baf9e9233241bbe93d6599c99cb46478614 #13309 + Closes #13362 -- proxy1.0.md: fix example +Daniel Stenberg (15 Apr 2024) - Closes #12856 +- multi: timeout handles even without connection -Chris Webb (5 Feb 2024) + When there is a "change" in a multi handle and pending handles are moved + back to the main list to be retested if they can proceed further (for + example a previous transfer completed or a connection has a confirmed + multiplexed state), the timeout check in multi_runsingle() would not + trigger because it required an established connection. -- configure: add --disable-docs flag + This could make a pending tranfer go back to pending state even though + it had been "in progress" for a longer time than permitted. By removing + the requirement for an associated connection, the timeout check will be + done proper even for transfers that has not yet been assigned one. - Building man pages from curldown sources now requires perl. Add a - --disable-docs flag to configure to enable building and installing - without documentation where perl is not available or man pages are not - required. This is selected automatically (with a warning) when perl is - not found by configure. + Ref #13227 + Reported-by: Rahul Krishna M + Closes #13276 - Fixes #12832 - Closes #12857 +Patrick Monnerat (15 Apr 2024) -Faraz Fallahi (5 Feb 2024) +- mprintf: check fputc error rather than matching returned character -- connect.c: fix typo + OS/400 ascii fputc wrapper deviates from the posix standard by the + fact that it returns the ebcdic encoding of the original ascii + character. Testing for a matching value for success will then always + fail. - Closes #12858 + This commit replaces the chariacter comparison by an explicit error + return check. -Daniel Stenberg (1 Feb 2024) + Follow-up to ef2cf58 + Closes #13367 -- sendf: ignore response body to HEAD +Viktor Szakats (14 Apr 2024) - and mark the stream for close, but return OK since the response this far - was ok - if headers were received. Partly because this is what curl has - done traditionally. +- ci: add CMake build variation, fixup libssh detection in `linux-old` - Test 499 verifies. Updates test 689. + To test without c-ares and hit `easy_lock.h` on an old system. Use this + new build step to introduce small variations, and also test libssh2. - Reported-by: Sergey Bronnikov - Bug: https://curl.se/mail/lib-2024-02/0000.html - Closes #12842 + Also add workaround to existing job to enable libssh. (CMake's generic + auto-detection doesn't seem to work here.): + ``` + CMake Warning at CMakeLists.txt:908 (find_package): + Could not find a package configuration file provided by "libssh" with any + of the following names: -- ftp: treat a 226 arriving before data as a signal to read data + libsshConfig.cmake + libssh-config.cmake + ``` + Ref: https://github.com/curl/curl/actions/runs/8661316091/job/23750974358#ste + p:5:69 - For active mode transfers. + Closes #13361 - Due to some interesting timing, curl can sometimes get the 226 (transfer - complete) over the control channel first, before the data connection - signals readability. If this happens, use that as a signal to check the - data connection. +- lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3` - Additionally, set the socket filter in listen mode *before* the - PORT/EPRT command is issued, to reduce the risk that the little time gap - could interfere. + Before this patch `lib/curl_setup.h` defined these two macros right + next to each other, then the source code used them interchangeably. - This issue never reproduced for me on Debian and takes several hundred - rounds for me to trigger on my mac. + After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features. + (Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used. - Reported-by: Stefan Eissing - Fixes #12823 - Closes #12841 + This patch doesn't change the way HTTP/3 is enabled via autotools + or CMake. Builders who enabled HTTP/3 manually by defining both of + these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`. -Patrick Monnerat (1 Feb 2024) + Closes #13352 -- OS400: avoid using awk in the build scripts +- build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`) - Awk is a PASE program and its use may cause a failure depending on the - CCSID of the calling script (IBM bug?). + Before this patch, two macros were used to guard IPv6 features in curl + sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use + the latter for consistency with other similar switches. - For this reason, revert to an sed-only solution to extract the exported - symbols from the header files. + `-DENABLE_IPV6` remains accepted for compatibility as a synonym for + `-DUSE_IPV6`, when passed to the compiler. - Closes #12826 + `ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc` + options to control this feature. -Jan Macku (1 Feb 2024) + Closes #13349 -- docs: remove `mk-ca-bundle.1` from `man_MANS` +Dan Fandrich (12 Apr 2024) - It was accidentally added in https://github.com/curl/curl/pull/12730 +- DISTROS: mark rolling release distros - Co-authored-by: Lukáš Zaoral - Signed-off-by: Jan Macku + These are ones that are unlikely to have back-ported curl patches. - Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b - Closes #12843 + Closes #13353 -Daniel Stenberg (1 Feb 2024) +Daniel Stenberg (12 Apr 2024) -- RELEASE-NOTES: synced +- mbedtls: cut off trailing newlines from debug logs - and bump to 8.6.1 for now + To avoid double newlines in the output. -- cmdline-docs/Makefile: avoid using a fixed temp file name + Reported-by: Gisle Vanem + Fixes #13321 + Closes #13356 - By appending the pid number two different runs at the same time will not - trample over the same file. +- RELEASE-NOTES: synced - Reported-by: Jon Rumsey - Fixes #12829 - Closes #12839 +Stefan Eissing (12 Apr 2024) -- asyn-thread: use wakeup_close to close the read descriptor +- CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported - Reported-by: Dan Fandrich - Ref: #12834 - Closes #12836 + - tests for 'size_request' and other stats reported, for + presence and consistency -Stefan Eissing (1 Feb 2024) + Reported-by: Jonatan Vela + Fixes #13269 + Closes #13275 -- ntml_wb: fix buffer type typo +Viktor Szakats (11 Apr 2024) - Closes #12825 +- dist: add files missing from release tarball -Daniel Stenberg (1 Feb 2024) + Closes #13346 -- tool_operate: do not set CURLOPT_QUICK_EXIT in debug builds +- ci: parallelize more, tidy up cmake commands (distcheck, macos) - Since it allows (small) memory leaks that interfere with torture tests - and regular memory-leak checks. + Also enable `-DCURL_WERROR=ON` in the Linux cmake build test. - Reported-by: Dan Fandrich - Fixes #12834 - Closes #12835 + Closes #13343 -Boris Verkhovskiy (31 Jan 2024) +Toon Claes (11 Apr 2024) -- form-string.md: correct the example +- docs: add CURLOPT_NOPROGRESS to CURLOPT_XFERINFOFUNCTION example - Closes #12822 + It's important to set `CURLOPT_NOPROGRESS` to `0` if you want your + transfer callback function, set by `CURLOPT_XFERINFOFUNCTION`, getting + called. To emphasize this to the users, add this to the code example. -Version 8.6.0 (31 Jan 2024) + Closes #13348 -Daniel Stenberg (31 Jan 2024) +RainRat (11 Apr 2024) -- RELEASE-NOTES: synced +- misc: fix typos - curl 8.6.0 + Closes #13344 -- THANKS: new contributors from 8.5.0 +Colin Leroy-Mira (11 Apr 2024) -Jay Satiro (31 Jan 2024) +- file: add support for getting basic directory listings -- cd2nroff: use perl 'strict' and 'warnings' + Not supported on Windows (yet) - - Use strict and warnings pragmas. + Closes #13137 - - If open() fails then show the reason. +Viktor Szakats (11 Apr 2024) - - Set STDIN io layer :crlf so that input is properly read on Windows. +- ci: add curl-for-win builds: Linux MUSL, macOS, Windows - - When STDIN is used as input, the filename $f is now set to "STDIN". + Linux MUSL (llvm/clang), macOS Apple clang, Windows (llvm/clang). - Various error messages in single() use $f for the filename and this way - it is not undefined when STDIN. + Configured with HTTP/2 and HTTP/3 and other dependencies (the default + curl-for-win) for a comprehensive build test. - Closes https://github.com/curl/curl/pull/12819 + ``` + curl 8.8.0-DEV (x86_64-unknown-linux-musl) libcurl/8.8.0-DEV LibreSSL/3.9.1 z + lib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 + ngtcp2/1.4.0 nghttp3/1.2.0 + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss + Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefil + e libz NTLM PSL SSL threadsafe UnixSockets zstd -Daniel Stenberg (30 Jan 2024) + curl 8.8.0-DEV (x86_64-apple-darwin) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1. + 3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp + 2/1.4.0 nghttp3/1.2.0 + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws w + ss + Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefil + e libz NTLM PSL SSL threadsafe UnixSockets zstd + + curl 8.8.0-DEV (x86_64-w64-mingw32) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3 + .1 brotli/1.1.0 zstd/1.5.6 WinIDN libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 + ngtcp2/1.4.0 nghttp3/1.2.0 + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws w + ss + Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerb + eros Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd + ``` -- cd2nroff: fix duplicate output issue + Limited to x64, because for build testing the additional CPUs don't add + much value compared to the extra build time. They can be enabled easily + if deemed useful. - Assisted-by: Jay Satiro - Fixes https://github.com/curl/curl-www/issues/321 - Closes #12818 + To the extent of curl-for-win configuration options, it's trivial to add + further build combinations. -- lib: error out on multissl + http3 + Closes #13335 - Since the QUIC/h3 code has no knowledge or handling of multissl it might - bring unintended consequences if we allow it. +- OS400: fix shellcheck warnings in scripts - configure, cmake and curl_setup.h all now reject this combination. + - use `$()` instead of backticks, and re-arrange double-quotes inside. + - add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu` + .) + - add `-n` to a few `if`s. + - shorten redirections by using `{} >` (as shellcheck recommended). + - silence warnings where variables were detected as unused (SC2034). + - a couple misc updates to silence warnings. + - switch to bash shebang for `-ot` feature. + - split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, ` + $(dirname \`) - Assisted-by: Viktor Szakats - Assisted-by: Gisle Vanem - Ref: #12806 - Closes #12807 + Also enable CI checks for OS/400 shell scripts. -Patrick Monnerat (29 Jan 2024) + Ref: #13307 + Closes #13309 -- OS400: sync ILE/RPG binding +Stefan Eissing (11 Apr 2024) - Also do not force git CRLF line endings on *.cmd files for OS400. +- lib: add Curl_xfer_write_resp_hd - Closes #12815 + Add method in protocol handlers to allow writing of a single, + 0-terminated header line. Avoids parsing and copying these lines. -Viktor Szakats (28 Jan 2024) + Closes #13165 -- build: delete/replace 3 more clang warning pragmas +- llist: add Curl_llist_append() - - tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma. + - use for better readability in all places where the "insert_next" + actually performs an append to the list + - add some tests in unit1300 - - whitespace formatting in `mprintf.h`, lib518, lib537. + Closes #13336 - - lib518: fix wrong variable in `sizeof()`. +- gnutls: lazy init the trust settings - - lib518: bump variables to `rlim_t`. - Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469 + - delay loading of trust anchors and CRLs after the ClientHello + has been sent off + - add tracing to IO operations + - on IO errors, return the CURLcode of the underlying filter - - lib518: sync error message with lib537 - Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce + Closes #13339 - - lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas - by reworking test code. +Marcel Raad (10 Apr 2024) - Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812 - Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 - Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 +- http_negotiate: fix `CURL_DISABLE_PROXY` build - Reviewed-by: Daniel Stenberg - Closes #12814 + `proxyuserpwd` was removed from `dynamically_allocated_data` in commit + f46385d36df. -Richard Levitte (27 Jan 2024) + Closes https://github.com/curl/curl/pull/13334 -- cmake: freshen up docs/INSTALL.cmake +Viktor Szakats (10 Apr 2024) - - Turn docs/INSTALL.cmake into a proper markdown file, - docs/INSTALL-CMAKE.md - - Move things around to divide the description into configuration, - building and installing sections - - Mention the more modern cmake options to configure, build and install, - but also retain the older variants as fallbacks +- quic: fixup duplicate static function name (for cmake unity) - Closes #12772 + Visible in daily curl-for-win builds: + https://github.com/curl/curl-for-win/actions/runs/8621925870 -Viktor Szakats (27 Jan 2024) + ``` + lib/vquic/curl_ngtcp2.c:1916:12: error: redefinition of 'ossl_new_session_cb' + static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid) + ^ + lib/vtls/openssl.c:2978:12: note: previous definition is here + static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid) + ^ + ``` + https://github.com/curl/curl-for-win/actions/runs/8621925870/job/23631885439# + step:3:6965 -- build: delete/replace clang warning pragmas + Follow-up to 3210101088dfa3d6a125d213226b092f2f866722 #13172 + Closes #13332 - - delete redundant warning suppressions for `-Wformat-nonliteral`. - This now relies on `CURL_PRINTF()` and it's theoratically possible - that this macro isn't active but the warning is. We're ignoring this - as a corner-case here. +- appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings - - replace two pragmas with code changes to avoid the warnings. + Tests were consistently flaky for a while. - Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 - Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers: + ``` + C:/projects/curl/lib/vtls/schannel.c(688): + warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater s + ize + C:/projects/curl/lib/vtls/schannel_verify.c(642): + warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater s + ize + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44 + kymgc0nif#L106 - Reviewed-by: Daniel Stenberg - Closes #12812 + Closes #13330 -Daniel Stenberg (27 Jan 2024) +Daniel Stenberg (10 Apr 2024) -- RELEASE-NOTES: synced +- projects: drop MSVC project files for recent versions -- http: only act on 101 responses when they are HTTP/1.1 + We encourage users to generate visual studio project files using CMake. - For 101 responses claiming to be any other protocol, bail out. This - would previously trigger an assert. + We keep project files in git for ancient visual studio versions that + cmake cannot generate files for, but we no longer ship the project files + in the tarballs. - Add test 1704 to verify. + appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013) - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184 - Closes #12811 + Co-Authored-by: Viktor Szakats + Co-Authored-by: Jay Satiro -Scarlett McAllister (27 Jan 2024) + Closes #13311 -- _VARIABLES.md: add missing 'be' into the sentence +Viktor Szakats (9 Apr 2024) - Closes #12809 +- cmake: use namespaced custom target names -Stefan Eissing (27 Jan 2024) + Rename custom target to namespaced (unique) names to avoid colliding + with 3rd-party projects (e.g. libzip) built together with curl. -- mqtt, remove remaining use of data->state.buffer + Reported-by: hammlee96 on github + Fixes #13324 + Closes #13326 - Closes #12799 +- appveyor: re-enable OpenSSL 3, bump to 3.2.1 -Daniel Stenberg (27 Jan 2024) + Ref: b62454a875d70f93ab5347c050903596feb45a23 #13266 + Closes #13329 -- x509asn1: switch from malloc to dynbuf +Stefan Eissing (9 Apr 2024) - Closes #12808 +- CI: upgrade openssl version to 3.3.0 for openssl-quic -- x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy + Closes #13328 - Closes #12808 +Daniel Stenberg (9 Apr 2024) -- x509asn1: reduce malloc in Curl_extract_certinfo +- RELEASE-NOTES: synced - Using dynbuf + Bump to 8.8.0-DEV - Closes #12808 +- curl_multi_waitfds.md: add protocol mention -Jay Satiro (27 Jan 2024) + Follow-up to 02beac6bb6b -- THANKS: add Alexander Bartel and Brennan Kinney +Dmitry Karpov (9 Apr 2024) - They reported and investigated #10259 which was fixed by 7b2d98df. +- lib: add curl_multi_waitfds - Ref: https://github.com/curl/curl/issues/10259 + New function call, similar to curl_multi_fdset() -Daniel Stenberg (26 Jan 2024) + Closes #13135 -- krb5: add prototype to silence clang warnings on mvsnprintf() +Viktor Szakats (9 Apr 2024) - "error: format string is not a string literal" +- dist: verify tarball reproducibility in CI - Follow-up to 09230127589eccc7 which made the warning appear + Closes #13327 - Assisted-by: Viktor Szakats - Closes #12803 +Stefan Eissing (9 Apr 2024) -- x509asn1: remove code for WANT_VERIFYHOST +- tests: stabilitze test_02_23* - No code ever sets this anymore since we dropped gskit + - h2-download now always opens the output file on first write callback + invocation, if it will pause the transfer or not. + - Checks on output files then does not depend on the amount of data curl + has collected for the first write. - Follow-up to 78d6232f1f326b9ab4d + Closes #13323 - Closes #12804 +- tls: fix compile issues on old-linux CI -- socks: reduce the buffer size to 600 (from 8K) + Follow-up to 3210101088dfa + Closes #13325 - This is malloc'ed memory and it does not more. Test 742 helps us verify - this. +Viktor Szakats (9 Apr 2024) - Closes #12789 +- dist: add reproducible dir entries to tarballs -Stefan Eissing (26 Jan 2024) + In the initial implementation of reproducible tarballs, they were + missing directory entries, while .zip archives had them. It meant + that on extracting the tarball, on-disk directory entries got the + current timestamp. -- file+ftp: use stack buffers instead of data->state.buffer + This patch fixes this by including directory entries in the tarball, + with reproducible timestamps. It also moves sorting inside tar, + to ensure reproducible directory entry timestamps on extract + (without the need of `--delay-directory-restore` option, when + extracting with GNU tar. BSD tar got that right by default.) - Closes #12789 + GNU tar 1.28 (2014-07-28) introduced `--sort=`. -- vtls: receive max buffer + Ref: https://github.com/curl/curl/pull/13299#discussion_r1555957350 + Follow-up to 860cd5fc2dc8e165fadd2c19a9b7c73b3ae5069d #13299 + Closes #13322 - - do not only receive one TLS record, but try to fill - the passed buffer - - consider <4K remaning space is "filled". +Stefan Eissing (9 Apr 2024) - Closes #12801 +- tls: use shared init code for TCP+QUIC -Daniel Stenberg (26 Jan 2024) + Closes #13172 -- docs: do not start lines/sentences with So, But nor And +Daniel Stenberg (9 Apr 2024) - Closes #12802 +- .mailmap: update Gisle's preferred email -- docs: remove spurious ampersands from markdown +Jan Macku (9 Apr 2024) - They were leftovers from the nroff conversion. +- doc: pytest `--repeat` -> `--count` - Follow-up to eefcc1bda4bccd800f5a5 + Pytest doesn't have a `--repeat` option, but it does have a `--count` + option. - Closes #12800 + ``` + --count=COUNT Number of times to repeat each test + ``` -Patrick Monnerat (26 Jan 2024) + Closes #13218 -- sasl: make login option string override http auth +Daniel Stenberg (9 Apr 2024) - - Use http authentication mechanisms as a default, not a preset. +- src/Makefile.am: access curl.txt using a relative path, not abs - Consider http authentication options which are mapped to SASL options as - a default (overriding the hardcoded default mask for the protocol) that - is ignored if a login option string is given. + ... to make it work when mounted using different mount points. Like when + generated/used inside and outside of a docker image. - Prior to this change, if some HTTP auth options were given, sasl mapped - http authentication options to sasl ones but merged them with the login - options. + Closes #13320 - That caused problems with the cli tool that sets the http login option - CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag - maps to more than one sasl mechanisms and the latter cannot be cleared - individually by the login options string. +- build: remove MacOSX-Framework script - New test 992 checks this. + I don't think this is much used these days. - Fixes https://github.com/curl/curl/issues/10259 - Closes https://github.com/curl/curl/pull/12790 + Also remove the libcurl.plist file used (only) by this script -Stefan Eissing (26 Jan 2024) + Closes #13313 -- socks: use own buffer instead of data->state.buffer +- release-tools.sh: store the timestamp and release tag too - Closes #12788 + When maketgz invokes this script to generate the docs/RELEASE-TOOLS.md + file that gets bundled in the release, it now also passes on the exact + timestamp and version number so that those details also get mentioned in + the document. They will help users reproduce an identical tarball. -Daniel Stenberg (26 Jan 2024) + Closes #13319 -- socks: fix generic output string to say SOCKS instead of SOCKS4 +Viktor Szakats (8 Apr 2024) - ... since it was also logged for SOCKS5. +- GHA: disable permissions where missing - Closes #12797 + Reviewed-by: Daniel Stenberg + Closes #13306 -- test742: test SOCKS5 with max length user, password and hostname +Stefan Eissing (8 Apr 2024) - Adjusted the socksd server accordingly to allow for configuring that - long user name and password. +- CI: update component versions - Closes #12797 + - ngtcp2: v1.4.0 + - nghttp3: v1.2.0 + - nghttp2: v1.61.0 + - mod_h2: v2.0.27 -Stefan Eissing (25 Jan 2024) + Closes #13316 -- ssh: use stack scratch buffer for seeks +Jérôme Leclercq (8 Apr 2024) - - instead of data->state.buffer +- CMake: check fseeko after detecting HAVE_FILE_OFFSET_BITS - Closes #12794 + Closes #13264 -Daniel Stenberg (25 Jan 2024) +Stefan Eissing (8 Apr 2024) -- krb5: access the response buffer correctly +- http2: emit RST when client write fails - As the pingpong code no longer uses the download buffer. + - When the writing of response data fails, reset the stream + and do not return a callback error to nghttp2. That would + be a fatal error for the connection and harm other requests. + - add test cases for various abort scenarios - Folllow-up to c2d973627bab12ab - Pointed-out-by: Stefan Eissing - Closes #12796 + Reported-by: Konstantin Kuzov + Fixes #13292 + Closes #13298 -Stefan Eissing (25 Jan 2024) +Kailun Qin (8 Apr 2024) -- mqtt: use stack scratch buffer for recv+publish +- mbedtls: call mbedtls_ssl_setup() after RNG callback is set - - instead of data->state.buffer + Since mbedTLS v3.6.0, the RNG check added in ssl_conf_check() will fail + if no RNG is provided when calling mbedtls_ssl_setup(). - Closes #12792 + Therefore, mbedtls_ssl_conf_rng() needs to be called before the SSL + context is passed to mbedtls_ssl_setup(). -- telnet, use stack scratch buffer for do + Ref: https://github.com/Mbed-TLS/mbedtls/commit/b422cab052b51ec84758638d6783d + 6ba4fc60613 - - instead of data->state.buffer + Signed-off-by: Kailun Qin + Closes #13314 - Closes #12793 +Daniel Stenberg (8 Apr 2024) -- http, use stack scratch buffer +- NTLM_WB: drop support - - instead of data->state.buffer + The feature has not worked for months and has been marked as DEPRECATED + for six+ months. - Closes #12791 + Closes #13249 -- ntlm_wb: do not use data->state.buf any longer +- curl_trc: fix build error when lacking verbose messages - Closes #12787 + Follow-up from 0b28ece657b2273 + Closes #13312 -- gitignore: the generated `libcurl-symbols.md` +Viktor Szakats (8 Apr 2024) - Closes #12795 +- contrithanks: honor `CURLWWW` variable -Daniel Stenberg (25 Jan 2024) + Reviewed-by: Daniel Stenberg + Closes #13315 -- tool: fix the listhelp generation command +- GHA: add shellcheck job and fix warnings, shell tidy-ups - The previous command line to generate the tool_listhelp.c source file - broke with 2494b8dd5175cee7. + Reviewed-by: Daniel Stenberg + Closes #13307 - Make 'make listhelp' invoked in src/ generate it. Also update the - comment in the file to mention the right procedure. +- dist: do not require Perl in `maketgz` - Closes #12786 + Perl remains required for the tarball build process. -- http: check for "Host:" case insensitively + Follow-up to 860cd5fc2dc8e165fadd2c19a9b7c73b3ae5069d #13299 - When checking if the user wants to replace the header, the check should - be case insensitive. + Reviewed-by: Daniel Stenberg + Closes #13310 - Adding test 461 to verify +Daniel Stenberg (8 Apr 2024) - Found-by: Dan Fandrich - Ref: #12782 - Closes #12784 +- RELEASE-NOTES: synced -Tatsuhiro Tsujikawa (25 Jan 2024) +- docs/cmdline-opts: invoke managen using a relative path -- configure: add libngtcp2_crypto_boringssl detection + ... no need to use an absolute path, that makes the build unncessarily + fail if invoked using a different mount point. managen now takes options + to find the input files. - If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested, - try to detect libngtcp2_crypto_boringssl. + Update test1478 to provide the dir arguments to managen - Reported-by: ウさん - Fixes #12724 - Closes #12769 + Closes #13281 -Daniel Stenberg (25 Jan 2024) +- GHA: add valgrind to a wolfSSL build -- http: remove comment reference to a removed solution + Closes #13274 - Follow-up to 58974d25d +Viktor Szakats (7 Apr 2024) - Closes #12785 +- dist: `set -eu`, fix shellcheck, make reproducible and smaller tarballs -Stefan Eissing (25 Jan 2024) + - set bash `-eu` and fix fallouts. + - fix shellcheck warnings. + - set and use `SOURCE_DATE_EPOCH` for reproducibility. + Authored-by: Daniel J. H. + Ref: #13280 + - set `TZ=UTC` and `LC_ALL=C` for reproducibility. + - make file timestamps in tarball/zip reproducible. + - make directory timestamps in zip reproducible. + - make timestamps of tarballs/zip reproducible. + - make file order in tarball/zip reproducible. + - omit extra file metadata from zip for reproducibility. + - use maximum zip compression. + - use POSIX `ustar` tarball format to avoid supply chain vulnerability: + https://seclists.org/oss-sec/2021/q4/0 + - make uid/gid in tarball reproducible. + - omit owner user/group names from tarball for reproducibility and privacy. + - omit current timestamp from .gz header for reproducibility. + - display SHA-256 hashes of produced tarballs/zip. + - fix whitespace. -- pytest: Scorecard tracking CPU and RSS + `.tar.gz` also became smaller in the process: 4,462,311 -> 4,148,249 bytes (8 + .7.1) - Closes #12765 + Requires GNU tar, GNU date, `sha256sum`. -Graham Campbell (25 Jan 2024) + Reviewed-by: Daniel Stenberg + Ref: #13250 + Closes #13299 -- GHA: bump ngtcp2, gnutls, mod_h2, quiche +Gisle Vanem (7 Apr 2024) - - ngtcp2 to v1.2.0 - - gnutls to 3.8.3 - - mod_h2 to 2.0.26 - - quiche to 0.20.0 +- tests/http: fix compiler warning - Closes #12778 - Closes #12779 - Closes #12780 - Closes #12781 + - Init result code variable to fix clang warning that it may be used + uninitialized. -Daniel Stenberg (25 Jan 2024) + Fixes https://github.com/curl/curl/issues/13301 + Closes https://github.com/curl/curl/pull/13304 -- ftpserver.pl: send 213 SIZE response without spurious newline +Stefan Eissing (6 Apr 2024) -- pingpong: stop using the download buffer +- vquic: use new curl_int64_t type - The pingpong logic now uses its own dynbuf for receiving command - response data. + - add curl_int64_t signed 64-bit type for lib use - When the "final" response header for a commanad has been received, that - final line is left first in the recvbuf for the protocols to parse at - will. If there is additional data behind the final response line, the - 'overflow' counter is indicate how many bytes. + - define CURL_PRId64, CURL_PRIu64 format ids - Closes #12757 + - use curl_int64_t in vquic -- gen.pl: remove bold from .IP used for ## + curl_int64_t signed complements the existing curl_uint64_t unsigned. - Reported-by: Viktor Szakats - Fixes #12776 - Closes #12777 + Note that `curl_int64_t` and `int64_t` are assignable from each other + but not identical. Some platforms with 64 long type defint int64_t as + "long long" (staring at macOS) which messes up things like pointers and + format identifiers. -Viktor Szakats (24 Jan 2024) + Closes https://github.com/curl/curl/pull/13293 -- cmake: rework options to enable curl and libcurl docs +Jay Satiro (5 Apr 2024) - Rework CMake options for building/using curl tool and libcurl manuals. +- lib: use multi instead of multi_easy for the active multi - - rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning: - to build man page and built-in manual for curl tool. + - Use data->multi and not data->multi_easy to refer to the active multi. - - rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning: - to build man pages for libcurl. + The easy handle's active multi is always data->multi. - - `BUILD_LIBCURL_DOCS` now works without having to enable - `ENABLE_CURL_MANUAL` too. + This is a follow up to 757dfdf which changed curl so that an easy handle + used with the easy interface and then multi interface cannot have two + different multi handles associated with it at the same time + (data->multi_easy from the easy interface and data->multi from the multi + interface). - - drop support for existing CMake-level `USE_MANUAL` option to avoid - confusion. (It used to work with the effect of current - `ENABLE_CURL_MANUAL`, but only by accident.) + Closes https://github.com/curl/curl/pull/12665 - Assisted-by: Richard Levitte - Ref: #12771 - Closes #12773 +Viktor Szakats (5 Apr 2024) -Daniel Stenberg (24 Jan 2024) +- tidy-up: whitespace [ci skip] -- urlapi: remove assert +Daniel Stenberg (5 Apr 2024) - This assert triggers wrongly when CURLU_GUESS_SCHEME and - CURLU_NO_AUTHORITY are both set and the URL is a single path. +- makefile: remove the sorting from the vc-ide action - I think this assert has played out its role. It was introduced in a - rather big refactor. + This target generates the MSVC project files. This change removes the + extra sorting and instead makes the script use the order of the files as + listed in the variables - which are mostly sorted anyway. - Follow-up to 4cfa5bcc9a + This is an attempt to make the project file generation more easily + reproducible. - Reported-by: promptfuzz_ on hackerone - Closes #12775 + Ref: #13250 + Closes #13294 -Patrick Monnerat (24 Jan 2024) +Gisle Vanem (5 Apr 2024) -- tests: avoid int/size_t conversion size/sign warnings +- bearssl: fix compiler warnings - Closes #12768 + "variables may be uninitialized when used" -Daniel Stenberg (24 Jan 2024) + Fixes #13290 + Closes #13297 -- GHA: add a job scanning for "bad words" in markdown +Daniel Stenberg (5 Apr 2024) - This means words, phrases or things we have decided not to use - words that - are spelled right according to the dictionary but we want to avoid. In the - name of consistency and better documentation. +- DISTROS: Cygwin updates - Closes #12764 + Brought-by: Brian Inglis + Fixes #13258 + Co-authored-by: Viktor Szakats + Closes #13279 -Viktor Szakats (23 Jan 2024) +Stefan Eissing (5 Apr 2024) -- cmake: speed up curldown processing, enable by default +- lib: add trace support for client reads and writes - - cmake: enable `BUILD_DOCS` by default (this controls converting and - installing `.3` files from `.md` sources) + - add `CURL_TRC_READ()` and `CURL_TRC_WRITE()` + - use in generic client writers and readers, as well + as http headers, chunking and websockets - - cmake: speed up generating `.3` files by using a single command per - directory, instead of a single command per file. This reduces external - commands by about a thousand. (There remains some CMake logic kicking - in resulting in 500 -one per file- external `-E touch_nocreate` calls.) + Closes #13223 - - cd2nroff: add ability to process multiple input files. +Michał Antoniak (5 Apr 2024) - - cd2nroff: add `-k` option to use the source filename to form the - output filename. (instead of the default in-file `Title:` line.) +- urldata: remove fields not used depending on used features - Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c - Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753 - Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 + Reduced size of dynamically_allocated_data structure. - Closes #12762 + Reduced number of stored values in enum dupstring and enum dupblob. This + affects the reduced array placed in the UserDefined structure. -Richard Levitte (23 Jan 2024) + Closes #13188 -- docs: install curl.1 with cmake as well +Viktor Szakats (5 Apr 2024) - Closes #12759 +- cmake: enable `-pedantic-errors` for clang when `CURL_WERROR=ON` -Daniel Stenberg (23 Jan 2024) + clang doesn't have the issues of GCC and old CMake versions. -- osslq: remove the TLS library from the version output + Note: This introduces asymmetry with autotools, which only enables + this for GCC. - Since we only support using a single TLS library at any one time, we - know that the TLS library for QUIC is the same that is also shown for - regular TLS. + Reviewed-by: Daniel Stenberg + Closes #13286 - Fixes #12763 - Reported-by: Viktor Szakats - Closes #12767 +- cmake: fix `CURL_WERROR=ON` for old CMake and use it in GHA/linux-old -Stefan Eissing (23 Jan 2024) + - cmake: fix `-pedantic-errors` for old CMake with `CURL_WERROR=ON` set. -- CI: remove unnecessary OpenSSL 3 option `enable-tls1_3` + `-pedantic-errors` option throws a warning with GCC (all versions) and + makes `check_symbol_exists()` fail in CMake versions older than + v3.23.0 (2022-03-29), when CMake introduced a workaround: - .. and switch OpenSSL 3 libdir from lib64 to lib for consistency. + https://gitlab.kitware.com/cmake/cmake/-/issues/13208 + https://gitlab.kitware.com/cmake/cmake/-/commit/eeb45401163d831b8c841ef6eba + 81466b4067b68 + https://gitlab.kitware.com/cmake/cmake/-/commit/1ab7c3cd28b27ca162c4559e102 + 6e5cad1898ade - Closes https://github.com/curl/curl/pull/12758 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 -- GHA: bump nghttp2 version to v1.59.0 + - set `CURL_WERROR=ON` for the `linux-old` job in CI. - - Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version. + Closes #13282 - Closes https://github.com/curl/curl/pull/12766 +- lib: use `#error` instead of invalid syntax in `curl_setup_once.h` -Daniel Stenberg (23 Jan 2024) + Reviewed-by: Daniel Stenberg + Closes #13287 -- RELEASE-NOTES: synced +Daniel Stenberg (5 Apr 2024) -- docs/cmdline: change to .md for cmdline docs +- GHA: on macOS remove $HOME/.curlrc - - switch all invidual files documenting command line options into .md, - as the documentation is now markdown-looking. + A recent image upgrade added a $HOME/.curlrc by default using --ipv4. - - made the parser treat 4-space indents as quotes + Ref: https://github.com/actions/runner-images/pull/9586 + Fixes #13284 + Closes #13285 - - switch to building the curl.1 manpage using the "mainpage.idx" file, - which lists the files to include to generate it, instead of using the - previous page-footer/headers. Also, those files are now also .md - ones, using the same format. I gave them underscore prefixes to make - them sort separately: - _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md, - _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md, - _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md, - _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md +Viktor Szakats (4 Apr 2024) - - updated test cases accordingly +- cmake: fixup `DEPENDS` filename - Closes #12751 + Fixing: + ``` + make[2]: Circular docs/curl-config.1 <- docs/curl-config.1 dependency dropped + . + make[2]: Circular docs/mk-ca-bundle.1 <- docs/mk-ca-bundle.1 dependency dropp + ed. + ``` + Ref: https://github.com/curl/curl/actions/runs/8559617487/job/23456740844?pr= + 13282#step:6:18 -dependabot[bot] (23 Jan 2024) + Follow-up to 5023ffad2c27d4b916ddb91800f99ecc5d3aad07 #13197 + Closes #13283 -- CI: bump actions/cache from 3 to 4 +- GHA: enable unity mode for cmake jobs + tidy-ups - Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. - - [Release notes](https://github.com/actions/cache/releases) - - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - - [Commits](https://github.com/actions/cache/compare/v3...v4) + Unity mode is not supported by CMake v3.7.2 used in linux-old, but + enable it anyway for consistency and to kick in automatically once + migrating to a newer old Linux in the future. - --- - updated-dependencies: - - dependency-name: actions/cache - dependency-type: direct:production - update-type: version-update:semver-major - ... + Also: + - replace `CMAKE_COMPILE_WARNING_AS_ERROR` with `CURL_WERROR`. + - delete default build option `PICKY_COMPILER=ON`. - Signed-off-by: dependabot[bot] - Closes #12756 + Closes #13277 -Daniel Stenberg (23 Jan 2024) +Dan Fandrich (4 Apr 2024) -- openssl: when verifystatus fails, remove session id from cache +- CI: Add CI build on Debian stretch to test old support - To prevent that it gets used in a subsequent transfer that skips the - verifystatus check since that check can't be done when the session id is - reused. + This version still has ELTS support and contains some old versions of + key components like cmake to help prevent us from breaking that support. - Reported-by: Hiroki Kurosawa - Closes #12760 + Closes #13029 -Viktor Szakats (23 Jan 2024) +Stefan Eissing (4 Apr 2024) -- cmake: add option to disable building docs +- request: paused upload on completed download, assess connection -Richard Levitte (23 Jan 2024) + A transfer with a completed download that is still uploading needs to + check the connection state when it is PAUSEd, since connection + close/errors would otherwise go unnoticed. -- cmake: use curldown to build man pages + Reported-by: Sergey Bronnikov + Fixes #13260 + Closes #13271 - This throws away the previous HTML and PDF producers, to mimic what - Makefile.am does as faithfully as possible. +Daniel Stenberg (4 Apr 2024) - Closes #12753 +- url: do not URL decode proxy crendentials -Daniel Stenberg (23 Jan 2024) + The two options CURLOPT_PROXYUSERNAME and CURLOPT_PROXYPASSWORD set the + actual names as-is, not URL encoded. -- mksymbolsmanpage.pl: provide references to where the symbol is used + Modified test 503 to use percent-encoded strings in the credential + strings that should be passed on as-is. -- docs: introduce "curldown" for libcurl man page format + Reported-by: Sergey Ogryzkov + Fixes #13265 + Closes #13270 - curldown is this new file format for libcurl man pages. It is markdown - inspired with differences: +Viktor Szakats (4 Apr 2024) - - Each file has a set of leading headers with meta-data - - Supports a small subset of markdown - - Uses .md file extensions for editors/IDE/GitHub to treat them nicely - - Generates man pages very similar to the previous ones - - Generates man pages that still convert nicely to HTML on the website - - Detects and highlights mentions of curl symbols automatically (when - their man page section is specified) +- appveyor: enable cmake unity mode by default - tools: + Leave one non-unity cmake job. This makes the jobs finish slightly + quicker, while giving more coverage for unity issues. - - cd2nroff: converts from curldown to nroff man page - - nroff2cd: convert an (old) nroff man page to curldown - - cdall: convert many nroff pages to curldown versions - - cd2cd: verifies and updates a curldown to latest curldown + Before: + https://ci.appveyor.com/project/curlorg/curl/builds/49496977 + https://ci.appveyor.com/project/curlorg/curl/builds/49500372 + After: + https://ci.appveyor.com/project/curlorg/curl/builds/49500338 - This setup generates .3 versions of all the curldown versions at build time. + Also fixup unrelated whitespace. - CI: + Reviewed-by: Daniel Stenberg + Closes #13217 - Since the documentation is now technically markdown in the eyes of many - things, the CI runs many more tests and checks on this documentation, - including proselint, link checkers and tests that make sure we capitalize the - first letter after a period... +Daniel Stenberg (4 Apr 2024) - Closes #12730 +- RELEASE-NOTES: synced -Viktor Szakats (22 Jan 2024) +Viktor Szakats (4 Apr 2024) -- libssh2: use `libssh2_session_callback_set2()` with v1.11.1 +- cmake: speed up libcurl doc building again - To avoid a local hack to pass function pointers and to avoid - deprecation warnings when building with libssh2 v1.11.1 or newer: - ``` - lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is depreca - ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- - declarations] - lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is depreca - ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- - declarations] - ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/2072082 - 1100#step:3:4982 + This time limit the number of files per command to avoid exceeding + limitations of certain OS/shell envs. - Ref: https://github.com/libssh2/libssh2/pull/1285 - Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db - 3cf1d4b0b4 - Reviewed-by: Daniel Stenberg - Closes #12754 + Such known env is Windows with the `cmd.exe` shell, which features an + 8K command-line length limit to this day. -Daniel Stenberg (22 Jan 2024) + Allowlisting `UNIX` to have no limit and using a limit of 200 for other + envs to be safe. If there is a way to detect `cmd.exe` and/or we know + which precise envs are sensitive to this, we can tweak these conditions + further. -- transfer: make the select_bits_paused condition check both directions + Even with the low limit, this patch reduces external commands by 200x, + making builds much faster. - If there is activity in a direction that is not paused, return false. + Ref: #12762 2620aa930bc73af1e4c70b10e3125b957b96ecfb (initial) + Ref: #13047 f03c85635f35269f1f45b983bf216624f541760a (revert) - Reported-by: Sergey Bronnikov - Bug: https://curl.se/mail/lib-2024-01/0049.html - Closes #12740 + Reviewed-by: Daniel Stenberg + Closes #13207 -Stefan Eissing (22 Jan 2024) +- cmake: tidy-up to use `WORKING_DIRECTORY` -- http3: initial support for OpenSSL 3.2 QUIC stack + Reviewed-by: Daniel Stenberg + Closes #13206 - - HTTP/3 for curl using OpenSSL's own QUIC stack together - with nghttp3 - - configure with `--with-openssl-quic` to enable curl to - build this. This requires the nghttp3 library - - implementation with the following restrictions: - * macOS has to use an unconnected UDP socket due to an - issue in OpenSSL's datagram implementation - See https://github.com/openssl/openssl/issues/23251 - This makes connections to non-reponsive servers hang. - * GET requests will send the indicator that they have - no body in a separate QUIC packet. This may result - in processing delays or Transfer-Encodings on proxied - requests - * uploads that encounter blocks will use 100% cpu as - detection of these flow control issue is not working - (we have not figured out to pry that from OpenSSL). +- cmake: generate misc manpages and install `mk-ca-bundle.pl` - Closes #12734 + - install `mk-ca-bundle.pl` like autotools does. -Viktor Szakats (22 Jan 2024) + - generate and install `mk-ca-bundle.1` and `curl-config.1` like + autotools. This fixes tests 1140 and 1173. -- cmake: fix `ENABLE_MANUAL` option + Reported-by: Dan Fandrich + Fixes #13194 - Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`. + - add option `BUILD_MISC_DOCS` to control building the above two + manpages. Enabled by default. - Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the - option designed to enable building and using the built-in curl manual. - (`USE_MANUAL=ON` option worked for this instead, by accident). + - appveyor: stop disabling tests 1140 and 1173. - Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409 - Closes #12749 + Reviewed-by: Daniel Stenberg + Closes #13197 -Mohammadreza Hendiani (19 Jan 2024) +Fabian Keil (4 Apr 2024) -- TODO: update broken link to ratelimit-headers draft +- wolfssl: plug memory leak in wolfssl_connect_step2() - Closes #12741 + Fixes: -Daniel Stenberg (19 Jan 2024) + test 2034...[simple HTTPS GET with DER public key pinning] + ==61829== 22,610 (3,744 direct, 18,866 indirect) bytes in 1 blocks are d + efinitely lost in loss record 51 of 54 + ==61829== at 0x484BB74: malloc (vg_replace_malloc.c:446) + ==61829== by 0x4B53A80: wolfSSL_Malloc (memory.c:344) + ==61829== by 0x4C1C8E1: wolfSSL_X509_new (x509.c:5326) + ==61829== by 0x4C3977D: d2i_X509orX509REQ (x509.c:3628) + ==61829== by 0x4C1D1F4: wolfSSL_X509_d2i (x509.c:3664) + ==61829== by 0x4C1C37B: wolfSSL_X509_dup (x509.c:13425) + ==61829== by 0x4C197DB: wolfSSL_get_peer_certificate (ssl.c:18765) + ==61829== by 0x33297C: wolfssl_connect_step2 (wolfssl.c:875) + ==61829== by 0x331669: wolfssl_connect_common (wolfssl.c:1287) + ==61829== by 0x3303E9: wolfssl_connect_nonblocking (wolfssl.c:1319) + ==61829== by 0x32FE89: ssl_connect_nonblocking (vtls.c:510) + ==61829== by 0x32DBE5: ssl_cf_connect (vtls.c:1679) + ==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307) + ==61829== by 0x27D9CF: cf_setup_connect (connect.c:1199) + ==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307) + ==61829== by 0x283CEA: cf_hc_baller_connect (cf-https-connect.c:135) -- cmake: when USE_MANUAL=YES, build the curl.1 man page + Closes #13272 - Fixes KNOWN_BUG 15.4 +Viktor Szakats (3 Apr 2024) - Closes #12742 +- appveyor: OpenSSL 3 no longer found by CMake, revert to 1.1.1 -- cmdline-opts/write-out.d: remove spurious double quotes + OpenSSL moved directories, and bumped versions in AppVeyor CI. -Stefan Eissing (19 Jan 2024) + Downgrading is not an ideal solution, but however trivial the solution + may be, I failed to come with anything that made CMake recognize either + OpenSSL 3.1 or 3.2. -- rtsp: Convert assertion into debug log + Possibly caused by: + https://github.com/appveyor/build-images/commit/702e8cdca01f28f6a40687783f493 + c786cebbe2c + https://github.com/appveyor/build-images/pull/149 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 + Closes #13266 - - write excess bytes to the client where the standard excess bytes - checks will report any wrongness and fail the transfer +hongfei.li (3 Apr 2024) - Fixes #12738 - Closes #12739 +- winbuild: use $(RC) correctly -Daniel Stenberg (19 Jan 2024) + Cloes #13267 -- headers: remove assert from Curl_headers_push +Daniel Stenberg (3 Apr 2024) - The fuzzer managed to reach the function without a terminating CR or LF - so let's handle it normally. While there, remove the goto. +- dist: remove the curl-config.1 from the tarball - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839 + The markdown file is already there and the .1 file gets generated in the + build. - Closes #12721 + Ref: #13250 + Closes #13268 -- curl_easy_getinfo.3: remove the wrong time value count +- curl_global_trace.md: shorten the description - It said "six" time values but they are eight by now. Remove the mention - of the amount. + Closes #13263 - Closes #12727 +- test1901: verify chunked POST from callback with CURLOPT_POSTFIELDSIZE set -Viktor Szakats (18 Jan 2024) + Follow-up to 721941aadf4ad -- mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls` + Ref: #13257 + Closes #13262 - - Silence warning in mbedTLS v3.5.1 public headers: - ``` - ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: r - edundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls] - ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: pre - vious declaration of 'psa_set_key_domain_parameters' was here - ``` - Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae - 785a513e157c - Ref: https://github.com/libssh2/libssh2/pull/1226 +Stefan Eissing (2 Apr 2024) - - Fix compiler warnings seen with gcc 9.2.0 + cmake unity: - ``` - ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read': - ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull - -dereference] - 189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &res - ult); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~ - ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write': - ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull - -dereference] - 168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, & - result); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~ - ``` +- http: with chunked POST forced, disable length check on read callback - - delete stray `#else`. + - when an application forces HTTP/1.1 chunked transfer encoding + by setting the corresponding header and instructs curl to use + the CURLOPT_READFUNCTION, disregard any POST length information. + - this establishes backward compatibility with previous curl versions - Closes #12720 + Applications are encouraged to not force "chunked", but rather + set length information for a POST. By setting -1, curl will + auto-select chunked on HTTP/1.1 and work properly on other HTTP + versions. -Daniel Stenberg (17 Jan 2024) + Reported-by: Jeff King + Fixes #13229 + Closes #13257 -- docs: cleanup nroff format use +Jay Satiro (1 Apr 2024) - - remove use of .BI for code snippet - - stop using .br, just do a blank line - - remove use of .PP - - remove use for .sp - - remove backslash in .IP - - use .IP instead of .TP +- INSTALL-CMAKE.md: explain `cmake -G ` - Closes #12731 + - Explain that CMake's -G option can be used to specify which build + system to generate files for. -Stefan Eissing (17 Jan 2024) + Example: cmake ../curl -G "MinGW Makefiles" -- test2307: fix expected failure code after ws refactoring + Ref: https://github.com/curl/curl/pull/12224#issuecomment-2026813645 - Fixes #12722 - Closes #12728 + Closes https://github.com/curl/curl/pull/13244 -Jay Satiro (17 Jan 2024) +Daniel Stenberg (1 Apr 2024) -- cf-socket: show errno in tcpkeepalive error messages +- libcurl-opts: mention pipelining less - - If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set - then show the errno in the verbose error messages. + libcurl has not supported HTTP pipelining since many years. Remove a few + (more) mentions of the feature. - Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652 + Closes #13254 - Closes https://github.com/curl/curl/pull/12726 +Daniel McCarney (31 Mar 2024) -- tool_getparam: stop supporting `@filename` style for --cookie +- m4: reposition USE_RUSTLS="yes" for pkg-config - The `@filename` style was never documented for --cookie - but prior to this change curl would accept it anyway and always treat a - @ prefixed string as a filename. + It's necessary to set this var to "yes" _after_ AC_DEFINE and AC_SUBST + in order for a later `test` to pass so that `check_for_ca_bundle=1` ends + up being set. This is in turn required for the default CA certificate + bundle to be set when building w/ rustls & pkg-config. - That's a problem if the string also contains a = sign because then it is - documented to be interpreted as a cookie string and not a filename. + Reported-by: Matt Jolly + Fixes #13248 + Closes #13251 - Example: +Daniel Stenberg (31 Mar 2024) - `--cookie @foo=bar` +- maketgz: put docs/RELEASE-TOOL.md into the tarball - Before: Interpreted as load cookies from filename foo=bar. + Generated with scripts/release-tools.sh - After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`). + The script lists the exact Debian package names and version numbers for + the tools that are used to generate the tarball. - Other curl options with a data/filename option-value use the `@filename` - to distinguish filenames which is probably how this happened. The - --cookie option has never been documented that way. + Closes #13239 - Ref: https://curl.se/docs/manpage.html#-b +- cd2nroff/manage: use UTC when SOURCE_DATE_EPOCH is set - Closes https://github.com/curl/curl/pull/12645 + Make them independent of the TZ setting. Also set a date string like + YYYY-MM-DD to avoid a local month name in the date. -Stefan Eissing (16 Jan 2024) + Reported-by: Carlos Henrique Lima Melara + Fixes #13242 + Closes #13243 -- websockets: refactor decode chain +- RELEASE-NOTES: synced - - use client writer stack for decoding frames - - move websocket protocol handler to ws.c +- docs/MAIL-ETIQUETTE: convert to markdown - Closes #12713 + To render nicer. To get spellchecked. -- websockets: check for negative payload lengths + Closes #13247 - - in en- and decoding, check the websocket frame payload lengths for - negative values (from curl_off_t) and error the operation in that case - - add test 2307 to verify +- reuse: add copyright + license info to individual docs/*.md files - Closes #12707 + Instead of use 'docs/*.md' in dep5. For clarity and avoiding a wide- + matching wildcard. -Daniel Stenberg (16 Jan 2024) + + Remove mention of old files from .reuse/dep5 + + add info to .github/dependabot.yml + + make scripts/copyright.pl warn on non-matching patterns -- docs: mention env vars not used by schannel + Closes #13245 - Ref: #12704 +- test470: warn about unicode quote character read from config file - Co-authored-by: Jay Satiro + Idea-by: Emanuele Torre - Closes #12711 +- test469: verify warning when argument has unicode quote -- tool_operate: make --remove-on-error only remove "real" files +- tool_getparam: output warning for leading unicode quote character - Reported-by: Harry Sintonen - Assisted-by: Dan Fandrich + ... in the option argument. - Closes #12710 + Typically this is a mistake done when copying example command lines from + online documentation using the wrong quote character. -Jay Wu (16 Jan 2024) + Presumably there are also other potential quote characters that might be + used, and this check is done without even knowing that unicode is used! -- url: don't set default CA paths for Secure Transport backend + Reported-by: Sanjay Pujare + Fixes #13214 + Closes #13215 - As the default for this backend is the native CA store. +- tool: follow-up getenv fix - Closes #12704 + Remove a double free. Change the IPFS env use to a plain getenv() simply + because coverity gets confused. -Lin Sun (16 Jan 2024) + Follow-up to 9126b141c9398fe + Closes #13241 -- asyn-ares: with modern c-ares, use its default timeout +- idn: make Curl_idnconvert_hostname() use Curl_idn_decode() - Closes #12703 + In the name of less code duplication -Daniel Stenberg (15 Jan 2024) + Closes #13236 -- tool_operate: stop setting the file comment on Amiga +- curl-confopts.m4: define CARES_NO_DEPRECATED when c-ares is used - - the URL is capped at 80 cols, which ruins it if longer - - it does not strip off URL credentials - - it is done unconditonally, not on --xattr - - we don't have Amiga in the CI which makes fixing it blindly fragile + Starting in 1.28.0 c-ares added deprecation warnings for some API calls + libcurl uses. - Someone who builds and tests on Amiga can add it back correctly in a - future if there is a desire. + Closes #13240 - Reported-by: Harry Sintonen - Closes #12709 +- vquic: use CURL_FORMAT_CURL_OFF_T for 64 bit printf output -Stefan Eissing (15 Jan 2024) + Reported-by: Keitagit-kun on github + Fixes #13224 + Closes #13231 -- rtsp: deal with borked server responses +- openldap: create ldap URLs correctly for IPv6 addresses - - enforce a response body length of 0, if the - response has no Content-lenght. This is according - to the RTSP spec. - - excess bytes in a response body are forwarded to - the client writers which will report and fail the - transfer + Reported-by: Sergio Durigan Junior + Fixes #13228 + Closes #13235 - Follow-up to d7b6ce6 - Fixes #12701 - Closes #12706 +- curl: use curl_getenv instead of the curlx_ version -Daniel Stenberg (14 Jan 2024) + The curlx one was once introduced when we still considered dropping the + libcurl function at some point. To reduce confusion and to make it + easier to understand when curl_free() should be used, use the actual + libcurl function call directly instead. -- version: show only the libpsl version, not its dependencies + Closes #13230 - The libpsl version output otherwise also includes version number for its - dependencies, like IDN lib, but since libcurl does not use libpsl's IDN - functionality those components are not important. +Evgeny Grin (Karlson2k) (30 Mar 2024) - Ref: https://github.com/curl/curl-for-win/issues/63 - Closes #12700 +- curl_sha512_256: do not use workaround for NetBSD when not needed -Brad Harder (14 Jan 2024) + Assisted-by: riastradh on github + Assisted-by: Michael Kaufmann + Closes #13225 -- curl.h: CURLOPT_DNS_SERVERS is only available with c-ares +Matt Jolly (30 Mar 2024) - Closes #12695 +- m4: fix rustls pkg-config codepath -Daniel Stenberg (14 Jan 2024) + The previous pkg-config code would successfully detect rustls but did + not set all appropriate variables and call the right macros to properly + configure cURL. -- cmdline-opts/gen.pl: error on initital blank line + Reported-by: kpcyrd on github + Fixes #13200 + Closes #13202 - After the "---" separator, there should be no blank line and this script - now errors out if one is detected. +Daniel McCarney (30 Mar 2024) - Ref: #12696 - Closes #12698 +- deps: update librustls 0.12.0 -> 0.13.0 -- cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper + This commit updates the optional rustls-ffi librustls dependency from + 0.12.0 to 0.13.0. This version is based on the latest available rustls + release (0.23.4). - Follow-up to 693cd1679361828a which was incomplete + The breaking API changes from 0.12.0 to 0.13.0 are in API surface unused + by curl, so this is an in-place update without any code changes. - Ref #12680 - Closes #12697 + The `RUSTLS.md` documentation is updated to reflect the new version in + use, and to clarify that `cbindgen` isn't required to build `librustls` + - it's only used by developers to update the vendored `rustls.h` header + file maintained upstream. -- curl_multi_fdset.3: remove mention of null pointer support + Closes #13238 - ... since this funtion has not supported null pointer fd_set arguments since - at least 2006. (That's when I stopped my git blame journey) +Daniel Stenberg (28 Mar 2024) - Fixes #12691 - Reported-by: sfan5 on github - Closes #12692 +- RELEASE-NOTES: synced -Mark Huang (14 Jan 2024) +- tool_xattr: "guess" URL scheme if none is provided -- docs/cmdline: remove unnecessary line breaks + ... when figuring out the source URL to store. - Closes #12696 + Reported-by: Dagfinn Ilmari Mannsåker + Fixes #13205 + Closes #13221 -Daniel Stenberg (14 Jan 2024) +- tool_xattr: in debug builds, act normally if CURL_FAKE_XATTR is not set -- transfer: remove warning: Value stored to 'blen' is never read + Closes #13220 - Detected by scan-build +Stefan Eissing (28 Mar 2024) - Follow-up from 1cd2f0072f +- content_encoding: brotli and others, pass through 0-length writes - Closes #12693 + - curl's transfer handling may write 0-length chunks at the end of the + download with an EOS flag. (HTTP/2 does this commonly) -Stefan Eissing (13 Jan 2024) + - content encoders need to pass-through such a write and not count this + as error in case they are finished decoding -- lib: replace readwrite with write_resp + Fixes #13209 + Fixes #13212 + Closes #13219 - This clarifies the handling of server responses by folding the code for - the complicated protocols into their protocol handlers. This concerns - mainly HTTP and its bastard sibling RTSP. +Tobias Stoeckmann (28 Mar 2024) - The terms "read" and "write" are often used without clear context if - they refer to the connect or the client/application side of a - transfer. This PR uses "read/write" for operations on the client side - and "send/receive" for the connection, e.g. server side. If this is - considered useful, we can revisit renaming of further methods in another - PR. +- libssh2: set length to 0 if strdup failed - Curl's protocol handler `readwrite()` method been changed: + Internally, libssh2 dereferences the NULL pointer if length is non-zero. + The callback function cannot return the error condition, so at least + prevent subsequent crash. - ```diff - - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, - - const char *buf, size_t blen, - - size_t *pconsumed, bool *readmore); - + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t ble - n, - + bool is_eos, bool *done); - ``` + Closes #13213 - The name was changed to clarify that this writes reponse data to the - client side. The parameter changes are: +Daniel Stenberg (28 Mar 2024) - * `conn` removed as it always operates on `data->conn` - * `pconsumed` removed as the method needs to handle all data on success - * `readmore` removed as no longer necessary - * `is_eos` as indicator that this is the last call for the transfer - response (end-of-stream). - * `done` TRUE on return iff the transfer response is to be treated as - finished +- RELEASE-PROCEDURE: mention an initial working build - This change affects many files only because of updated comments in - handlers that provide no implementation. The real change is that the - HTTP protocol handlers now provide an implementation. + This is the step that was not done and caused the 8.7.0 mishap (it + lacked the correctly generated hugehelp file). - The HTTP protocol handlers `write_resp()` implementation will get passed - **all** raw data of a server response for the transfer. The HTTP/1.x - formatted status and headers, as well as the undecoded response - body. `Curl_http_write_resp_hds()` is used internally to parse the - response headers and pass them on. This method is public as the RTSP - protocol handler also uses it. + Remove the mention of the copyright script as this is verified by a CI + job these days: the REUSE one. - HTTP/1.1 "chunked" transport encoding is now part of the general - *content encoding* writer stack, just like other encodings. A new flag - `CLIENTWRITE_EOS` was added for the last client write. This allows - writers to verify that they are in a valid end state. The chunked - decoder will check if it indeed has seen the last chunk. + Closes #13216 - The general response handling in `transfer.c:466` happens in function - `readwrite_data()`. This mainly operates now like: +Paul Howarth (28 Mar 2024) - ``` - static CURLcode readwrite_data(data, ...) - { - do { - Curl_xfer_recv_resp(data, buf) - ... - Curl_xfer_write_resp(data, buf) - ... - } while(interested); - ... - } - ``` +- curl_sha512_255: fix detection of OpenSSL 1.1.1 or later - All the response data handling is implemented in - `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` - implementation if available, or does the default behaviour. + Use the same OPENSSL_VERSION_NUMBER comparison as in lib/vtls/openssl.c. - All raw response data needs to pass through this function. Which also - means that anyone in possession of such data may call - `Curl_xfer_write_resp()`. + Closes #13208 - Closes #12480 +Robert Moreton (28 Mar 2024) -Daniel Stenberg (13 Jan 2024) +- cf-socket: remove references to l_ip, l_port -- RELEASE-NOTES: synced + Fixes #13210 + Closes #13211 -- TODO: TFTP doesn't convert LF to CRLF for mode=netascii +Daniel Stenberg (28 Mar 2024) - Closes #12655 - Closes #12690 +- openssl: do not set SSL_MODE_RELEASE_BUFFERS -- gen: do italics/bold for a range of letters, not just single word + While it might save some memory, it causes OpenSSL to instead do a huge + amount of allocations. - Previously it would match only on a sequence of non-space, which made it - miss to highlight for example "public suffix list". + Ref: #13136 + Closes #13203 - Updated the recent cookie.d edit from 5da57193b732 to use bold instead - of italics. +- curl: make --help adapt to the terminal width - Closes #12689 + Instead of assuming and working with 80 colums, try figuring out what + width is actually used. -- docs: describe and highlight super cookies + Ref: #13141 - Reported-by: Yadhu Krishna M + Closes #13171 - Closes #12687 +- RELEASE-NOTES: synced -- configure: when enabling QUIC, check that TLS supports QUIC + and bump to 8.7.2 for now - Most importantly perhaps is when using OpenSSL that the used - build/flavor has the QUIC API: the vanilla OpenSSL does not, only - BoringSSL, libressl, AWS-LC and quictls do. +- configure: make --disable-docs imply --disable-manual - Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7f - e57e#r136780413 + Because when the docs is not built, the necesary curl.txt file is not + present so then the manual cannot get built. - Closes #12683 + Reported-by: Harry Sintonen + Closes #13191 -Stefan Eissing (11 Jan 2024) +Chris Webb (27 Mar 2024) -- vquic: extract TLS setup into own source +- cmdline-docs: fix make install with configure --disable-docs - - separate ngtcp2 specific parts out - - provide callback during init to allow ngtcp2 to apply its defaults + make -C docs/cmdline-opts install depends on all-am, which in turn + depends on $(MANS), unconditionally defined to be $(man_MANS). - Closes #12678 + As with CLEANFILES, only add curl.1 to man_MANS when BUILD_DOCS is true + so we don't try to build curl.1 unnecessarily. -Sergey Markelov (11 Jan 2024) + Closes #13198 -- multi: remove total timer reset in file_do() while fetching file:// +Version 8.7.1 (27 Mar 2024) - The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts - with resetting the timer that is a start point for further multi states. - If file://, MSTATE_DO calls file_do() that should not reset the total - timer. Otherwise, the total time is always less than the pre-transfer - and the start transfer times. +Daniel Stenberg (27 Mar 2024) - Closes #12682 +- RELEASE-PROCEDURE: remove old release dates, add new pending ones -Daniel Stenberg (11 Jan 2024) +Version 8.7.0 (27 Mar 2024) -- http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT +Daniel Stenberg (27 Mar 2024) - Extended test 80 to verify this. +- RELEASE-NOTES: synced - Reported-by: Stefan Eissing - Fixes #12680 - Closes #12681 + curl 8.7.0 release -- sectransp: do verify_cert without memdup for blobs +- THANKS: new contributors from the 8.7.0 release - Since the information is then already stored in memory, this can avoid - an extra set of malloc + free calls. +- CURLOPT_POSTFIELDS.md: used for MQTT as well - Closes #12679 + Closes #13189 -- hsts: remove assert for zero length domain +- http: remove stale comment about rewindbeforesend - A zero length domain can happen if the HSTS parser is given invalid - input data which is not unheard of and is done by the fuzzer. + ... because that struct field exists no more. - Follow-up from cfe7902111ae547873 + Follow-up to 14bcea074a782272. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661 + Closes #13187 - Closes #12676 +- DISTROS: add document with distro pointers -- headers: make sure the trailing newline is not stored + Lots of organizations distribute curl packages to end users. This is a + collection of pointers to where to learn more about curl on and with + each distro. - extended test1940 to verify blank header fields too + Assisted-by: Alan Coopersmith + Assisted-by: Andrew Kaster + Assisted-by: Andy Fiddaman + Assisted-by: Arjan van de Ven + Assisted-by: Brian Clemens + Assisted-by: chrysos349 on github + Assisted-by: Dan Fandrich + Assisted-by: Dan McDonald + Assisted-by: Gaelan Steele + Assisted-by: graywolf on github + Assisted-by: Jan Macku + Assisted-by: John Marshall + Assisted-by: Jonathan Perkin + Assisted-by: Kevin Daudt + Assisted-by: Marcus Müller + Assisted-by: Michał Górny + Assisted-by: Outvi V + Assisted-by: Ross Burton + Assisted-by: Sean Molenaar + Assisted-by: Till Wegmüller + Assisted-by: Viktor Szakats + Assisted-by: Winni Neessen - Bug: https://curl.se/mail/lib-2024-01/0019.html - Reported-by: Dmitry Karpov - Closes #12675 + Closes #13178 -- curl_easy_header.3: tiny language fix +Fabian Keil (25 Mar 2024) - Closes #12672 +- wolfSSL: do not call the stub function wolfSSL_BIO_set_init() -- examples/range.c: add + Calling the function isn't necessary and causes the build + to fail when wolfSSL has been compiled with NO_WOLFSSL_STUB: - Closes #12671 + Making all in opts + CCLD curl + ld: error: undefined symbol: wolfSSL_BIO_set_init + >>> referenced by wolfssl.c:235 (vtls/wolfssl.c:235) + >>> libcurl_la-wolfssl.o:(wolfssl_bio_cf_create) in archiv + e ../lib/.libs/libcurl.a + cc: error: linker command failed with exit code 1 (use -v to see invocat + ion) + *** Error code 1 -- examples/netrc.c: add + Closes #13164 - Closes #12671 +Daniel Stenberg (25 Mar 2024) -- examples/ipv6.c: new example showing IPv6-only internet transfer +- cmdline-opts: shorter help texts - Closes #12671 + In an effort to increase the readability of the "--help all" output on + narrow (80 column) terminals. -- examples/address-scope.c: renamed from ipv6.c + Co-authored-by: Jay Satiro - It shows address scope use really + Closes #13169 - Closes #12671 +Matt Jolly (25 Mar 2024) -Stefan Eissing (9 Jan 2024) +- curl-rustls.m4: add pkg-config support to rustls detection -- multi: pollset adjust, init with FIRSTSOCKET during connect + Based on the existing openssl pkg-config detection, this commit tries to + use pkg-config to find `rustls` then falls back to the current approach + if that fails. - - `conn->sockfd` is set by `Curl_setup_transfer()`, but that - is called *after* the connection has been established - - use `conn->sock[FIRSTSOCKET]` instead + We use the following logic: - Follow-up to a0f94800d507de - Closes #12664 + - if no path is provided, just use pkg-config, if it's not there we have + a problem! + - if a path is provided, try pkg-config + + if pkg-config fails, try and find rustls directly -Daniel Stenberg (9 Jan 2024) + Closes #13179 -- WEBSOCKET.md: remove dead link +Mohammadreza Hendiani (25 Mar 2024) -- CI: spellcheck/appveyor: invoke configure --without-libpsl +- TODO: update 13.11 with more information - Follow-up to 2998874bb61ac6 + Closes #13173 -- cmdline/docs/*.d: switch to using ## instead of .IP +Daniel Stenberg (23 Mar 2024) - To make the editing easier. To write and to read. +- docs/libcurl: generate PROTOCOLS from meta-data - Closes #12667 + Remove the PROTOCOLS section from the source files completely and + instead generate them based on the header data in the curldown files. -- gen.pl: support ## for doing .IP in table-like lists + It also generates TLS backend information for options marked for TLS as + protocol. - Warn on use of .RS/.IP/.RE + Closes #13175 - Closes #12667 +- CURLMOPT_MAX*: mention what happens if changed mid-transfer -Jay Satiro (9 Jan 2024) + For CURLMOPT_MAXCONNECTS and CURLMOPT_MAX_HOST_CONNECTIONS -- cookie.d: Document use of empty string to enable cookie engine + Ref: #13158 + Closes #13176 - - Explain that --cookie "" can be used to enable the cookie engine - without reading any initial cookies. +- docs/libcurl: add TLS backend info for all TLS options - As is documented in CURLOPT_COOKIEFILE. + All man pages that are listed to be for TLS now must also specify + exactly what TLS backends the option works for, or use All if they all + work. - Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html + cd2nroff makes sure this is done and that the listed backends exist. - Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420 - Reported-by: janko-js@users.noreply.github.com + Closes #13168 - Closes https://github.com/curl/curl/pull/12646 +- docs/libcurl: cleanups -Daniel Stenberg (9 Jan 2024) + - CURLINFO_TLS_SESSION.md: remove mention of NSS + - CURLINFO_TLS_SSL_PTR.md: remove NSS leftover + - CURLOPT_CAINFO.md: drop mention of backends not supporting this + - CURLOPT_CAPATH.md: wolfSSL also supports this -- setopt: use memdup0 when cloning COPYPOSTFIELDS + Closes #13166 - Closes #12651 +- docs: make each libcurl man specify protocol(s) -- telnet: use dynbuf instad of malloc for escape buffer + The mandatory header now has a mandatory list of protocols for which the + manpage is relevant. - Previously, send_telnet_data() would malloc + free a buffer every time - for escaping IAC codes. Now, it reuses a dynbuf for this purpose. + Most man pages already has a "PROTOCOLS" section, but this introduces a + stricter way to specify the relevant protocols. - Closes #12652 + cd2nroff verifies that at least one protocol is mentioned (which can be + `*`). -- CI: install libpsl or configure --without-libpsl in builds + This information is not used just yet, but A) the PROTOCOLS section can + now instead get generated and get a unified wording across all manpages + and B) this allows us to more reliably filter/search for protocol + specific manpages/options. - As a follow-up to the stricted libpsl check in configure + Closes #13166 -- configure: make libpsl detection failure cause error +Stefan Eissing (21 Mar 2024) - To force users to explictily disable it if they really don't want it - used and make it harder to accidentally miss it. +- http2, http3: only return CURLE_PARTIAL_FILE when bytes were received - --without-libpsl is the option to use if PSL is not wanted. + - should resolve spurious pytest failures when stream were reset + right after response header were received - Closes #12661 + Clsoes #13151 -- RELEASE-NOTES: synced +- http: separate response parsing from response action -- pop3: replace calloc + memcpy with memdup0 + - move code that triggers on end-of-response into separate function from + parsing + - simplify some headp/headerlen usage + - add `httpversion` to SingleRequest to indicate the version of the + current response - ... and make sure to return error on out of memory. + Closes #13134 - Closes #12650 +Daniel Stenberg (21 Mar 2024) -- lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT +- http2: remove the third (unused) argument from http2_data_done() - Closes #12658 + Closes #13154 -- mime: use memdup0 instead of malloc + memcpy +- RELEASE-NOTES: synced - Closes #12649 +Evgeny Grin (Karlson2k) (21 Mar 2024) -- tool_getparam: move the --rate logic into set_rate() +- RELEASE-NOTES: corrected -- tool_getparam: switch to an enum for every option + Corrected link for item 118 - To make the big switch much easier to read/understand and to make it - easier to add new options. + Closes #13157 -- tool_getparam: build post data using dynbuf (more) +Daniel Stenberg (19 Mar 2024) -- tool_getparam: replace malloc + copy by dynbuf for --data +- CURLOPT_INTERFACE.md: remove spurious amp, add see-also -- tool_getparam: make data_urlencode avoid direct malloc + Closes #13149 - use aprintf() instead +Stefan Eissing (19 Mar 2024) -- tool_getparam: move the --url-query logic into url_query() +- http: improve response header handling, save cpu cycles - This function is not doing post at all so it was always weirdly placed. + Saving some cpu cycles in http response header processing: + - pass the length of the header line along + - use string constant sizeof() instead of strlen() + - check line length if prefix is possible + - switch on first header char to limit checks -- tool_getparam: move the --data logic into set_data() + Closes #13143 -- tool_getparam: unify the cmdline switch() into a single one +Daniel Stenberg (19 Mar 2024) - - easier to follow, easier to modify, easier to extend, possibly slightly - faster +- tool_getparam: accept a blank -w "" - - each case now has the long option as a comment + Added test 468 to verify. -- tool_getparam: bsearch cmdline options + Regression from 07bcae89d5d00 (shipped in 8.6.0) + Reported-by: Thomas Pyle + Fixes #13144 + Closes #13145 - - the option names are now alpha sorted and lookup is a lot faster +Evgeny Grin (Karlson2k) (18 Mar 2024) - - use case sensitive matching. It was previously case insensitive, but that - was not documented nor tested. +- curl_sha512_256: work around a NetBSD bug - - remove "partial match" feature. It was not documented, not tested and - was always fragile as existing use could break when we add a new - option + Based on Michael Kaufmann analysis and suggestion - - lookup short options via a table + Closes #13133 - Closes #12631 +Stefan Eissing (18 Mar 2024) -Gabe (8 Jan 2024) +- http: expect 100 rework -- COPYING: update copyright year + Move all handling of HTTP's `Expect: 100-continue` feature into a client + reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer + sending on general events like a timer. - Closes #12654 + HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect: + 100-continue`. That reader works as follows: -Stefan Eissing (8 Jan 2024) + - on first invocation, records time, starts the `EXPIRE_100_TIMEOUT` + timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0, + eos=FALSE like a paused upload. -- url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD + - on subsequent invocation it checks if the timer has expired. If so, it + enables `KEEP_SEND` and switches to passing through reads to the + underlying readers. - Also add more tracing to test 19 + Transfer handling's `readwrite()` will be invoked when a timer expires + (like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing + `KEEP_SEND_TIMER`, it will try to upload more data, which triggers + reading from the client readers again. Which then may lead to a new + pausing or cause the upload to start. - Follow-up to a0f9480 + Flags and timestamps connected to this have been moved from + `SingleRequest` into the reader's context. - Fixes #12657 - Closes #12659 + Closes #13110 -Daniel Stenberg (8 Jan 2024) +- mbedtls: fix pytest for newer versions -- connect: remove margin from eyeballer alloc + Fix the expectations in pytest for newer versions of mbedtls - Presumably leftovers from debugging + Closes #13132 - Closes #12647 +Daniel Stenberg (15 Mar 2024) -- ftp: only consider entry path if it has a length +- ipv6.md: mention IPv4 mapped addresses - Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb + Reported-by: Josh Soref + Assisted-by: Jay Satiro + Fixes #13112 + Closes #13131 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631 +Stefan Eissing (15 Mar 2024) - Avoids a NULL pointer deref. +- http: revisit http_perhapsrewind() - Closes #12648 + - use facilities provided by client readers better + - work also for non-uploading requests like GET/HEAD + - update documentation -Stefan Eissing (7 Jan 2024) + Closes #13117 -- transfer: adjust_pollset improvements +- test 1541: verify getinfo values on first header callback - - let `multi_getsock()` initialize the pollset in what the - transfer state requires in regards to SEND/RECV - - change connection filters `adjust_pollset()` implementation - to react on the presence of POLLIN/-OUT in the pollset and - no longer check CURL_WANT_SEND/CURL_WANT_RECV - - cf-socket will no longer add POLLIN on its own - - http2 and http/3 filters will only do adjustments if the - passed pollset wants to POLLIN/OUT for the transfer on - the socket. This is similar to the HTTP/2 proxy filter - and works in stacked filters. + Reported-by: chensong1211 on github + Ref: #13125 + Closes #13128 - Closes #12640 +- TLS: start shutdown only when peer did not already close -Daniel Stenberg (6 Jan 2024) + - When curl sees a TCP close from the peer, do not start a TLS shutdown. + TLS shutdown is a handshake and if the peer already closed the + connection, it is not interested in participating. -- ftp: use memdup0 to store the OS from a SYST 215 response + Reported-by: dfdity on github + Assisted-by: Jiří Bok + Assisted-by: Pēteris Caune + Fixes #10290 + Closes #13087 - avoid malloc + direct buffer fiddle +Daniel Stenberg (14 Mar 2024) - Closes #12639 +- RELEASE-NOTES: synced -- ftp: use dynbuf to store entrypath +- curl: make --libcurl output better CURLOPT_*SSLVERSION - avoid direct malloc + The option is really two enums ORed together, so it needs special + attention to make the code output nice. - Closes #12638 + Added test 1481 to verify. Both the server and the proxy versions. -Lealem Amedie (6 Jan 2024) + Reported-by: Boris Verkhovskiy + Fixes #13127 + Closes #13129 -- wolfssl: load certificate *chain* for PEM client certs +- GHA/linux: add sysctl trick to work-around GitHub runner issue - Closes #12634 + The GitHub image runner update from 20240304.1.0 to 20240310.1 + introduces a problem for clang-14. The issue is caused by + incompatibility between llvm 14 provided in ubuntu-22.04 image and the + much newer kernel configured with high-entropy ASLR. -Stefan Eissing (4 Jan 2024) + As a work-around, we issue a sysctl command to lower the entropy and get + clang-14 to work again. -- http: adjust_pollset fix + URL: https://github.com/actions/runner-images/issues/9491 - do not add a socket for POLLIN when the transfer does not want to send - (for example is paused). + Closes #13124 - Follow-up to 47f5b1a +- SPONSORS: describe the basics - Reported-by: bubbleguuum on github - Fixes #12632 - Closes #12633 + Closes #13119 -Daniel Stenberg (3 Jan 2024) +- GOVERNANCE: document the core team -- tool: make parser reject blank arguments if not supported + Closes #13118 - Already in the getstr() function that clones the input argument. +Jay Satiro (13 Mar 2024) - Closes #12620 +- vquic-tls: fix the error code returned for bad CA file -dependabot[bot] (3 Jan 2024) + - Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem + reading the cert file or path. -- build(deps): bump github/codeql-action from 2 to 3 + This is a follow-up to the parent commit aedbbdf1. - Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 - to 3. - - [Release notes](https://github.com/github/codeql-action/releases) - - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - - [Commits](https://github.com/github/codeql-action/compare/v2...v3) + Reported-by: Karthikdasari0423@users.noreply.github.com - --- - updated-dependencies: - - dependency-name: github/codeql-action - dependency-type: direct:production - update-type: version-update:semver-major - ... + Fixes https://github.com/curl/curl/issues/13115 - Signed-off-by: dependabot[bot] +Daniel Stenberg (12 Mar 2024) - Closes #12625 +- vquic-tls: return appropirate errors on wolfSSL errors -- build(deps): bump actions/checkout from 3 to 4 + Reported-by: Dexter Gerig + Closes #13107 - Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - - [Release notes](https://github.com/actions/checkout/releases) - - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - - [Commits](https://github.com/actions/checkout/compare/v3...v4) +Viktor Szakats (12 Mar 2024) - --- - updated-dependencies: - - dependency-name: actions/checkout - dependency-type: direct:production - update-type: version-update:semver-major - ... - - Signed-off-by: dependabot[bot] - - Closes #12624 +- tidy-up: one comment and EOF newlines -- build(deps): bump actions/upload-artifact from 3 to 4 + Reviewed-by: Daniel Stenberg + Closes #13108 - Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) f - rom 3 to 4. - - [Release notes](https://github.com/actions/upload-artifact/releases) - - [Commits](https://github.com/actions/upload-artifact/compare/v3...v4) +Daniel Stenberg (12 Mar 2024) - --- - updated-dependencies: - - dependency-name: actions/upload-artifact - dependency-type: direct:production - update-type: version-update:semver-major - ... +- cmdline-opts: language cleanups - Signed-off-by: dependabot[bot] + Use imperative mood consistently for the first sentence describing an + option. - Closes #12627 + "Set this" instead "tell curl to set" or "this sets..." -- build(deps): bump actions/download-artifact from 3 to 4 + Plus some extra cleanups and rephrasing. - Bumps [actions/download-artifact](https://github.com/actions/download-artifac - t) from 3 to 4. - - [Release notes](https://github.com/actions/download-artifact/releases) - - [Commits](https://github.com/actions/download-artifact/compare/v3...v4) + Closes #13106 - --- - updated-dependencies: - - dependency-name: actions/download-artifact - dependency-type: direct:production - update-type: version-update:semver-major - ... +- managen: remove space before protocols - Signed-off-by: dependabot[bot] + For options that are listed for specific protocols, the protocols (shown + first within parentheses) are now output without the leading space in the + manpage output. - Closes #12626 + Closes #13105 -Stefan Eissing (3 Jan 2024) +Jay Satiro (12 Mar 2024) -- http3/quiche: fix result code on a stream reset +- mbedtls: properly cleanup the thread-shared entropy - - fixes pytest failures in test 07_22 - - aligns CURLcode values on stream reset with ngtcp2 + - Store the state of the thread-shared entropy for global init/cleanup. - Closes #12629 + - Use curl's thread support of mbedtls for all Windows builds instead of + just when the threaded resolver is used via USE_THREADS_WIN32. -Daniel Stenberg (2 Jan 2024) + Prior to this change on global cleanup curl builds that have curl thread + support for mbedtls freed the entropy (8b1d2298) but failed to mark that + it had been freed, which caused problems on subsequent init + transfer. -- setopt: clear mimepost when formp is freed + Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105 + Reported-by: awesomekosm@users.noreply.github.com - A precaution to avoid a possibly dangling pointer left behind. + Closes https://github.com/curl/curl/pull/13071 - Reported-by: Thomas Ferguson - Fixes #12608 - Closes #12621 +Daniel Stenberg (12 Mar 2024) -Andy Alt (2 Jan 2024) +- tool_getparam: handle non-existing (out of range) short-options -- CI: Add dependabot.yml + ... correctly, even when they follow an existing one without a space in + between. - This will cause dependabot to open a PR when various actions are - updated, provided that the action maintainer has issued a release. + Verify with test 467 - Closes #12623 + Follow-up to 07dd60c05b + Reported-by: Geeknik Labs + Fixes #13101 + Closes #13102 -Gisle Vanem (2 Jan 2024) +Stefan Eissing (11 Mar 2024) -- content_encoding: change return code to typedef'ed enum +- lib: move 'done' parameter to SingleRequests - ... to work around a clang ubsan warning. + A transfer may do several `SingleRequest`s for its success. This happens + regularly for authentication, follows and retries on failed connections. + The "readwrite()" calls and functions connected to those carried a `bool + *done` parameter to indicate that the current `SingleRequest` is over. + This may happen before `upload_done` or `download_done` bits of + `SingleRequest` are set. - Fixes #12618 - Closes #12622 + The problem with that is now `write_resp()` protocol handlers are + invoked in places where the `bool *done` cannot be passed up to the + caller. Instead of being a bool in the call chain, it needs to become a + member of `SingleRequest`, reflecting its state. -Daniel Stenberg (2 Jan 2024) + This removes the `bool *done` parameter and adds the `done` bit to + `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a + `SingleRequest` in a follow up, clearing `done` and other + flags/counters. -- tool: prepend output_dir in header callback + Closes #13096 - When Content-Disposition parsing is used and an output dir is prepended, - make sure to store that new file name correctly so that it can be used - for setting the file timestamp when --remote-time is used. +- request: clarify message when request has been sent off - Extended test 3012 to verify. + Change the "uploaded and fine" message for requests without a body - Co-Authored-by: Jay Satiro - Reported-by: hgdagon on github - Fixes #12614 - Closes #12617 + Reported-by: Karthikdasari0423 on github + Fixes #13093 + Closes #13095 -- test1254: fix typo in name plus shorten it +Daniel Stenberg (11 Mar 2024) - RELEASE-NOTES: synced -Viktor Szakats (2 Jan 2024) - -- schannel: fix `-Warith-conversion` gcc 13 warning - - ``` - lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' - may change the sign of the result [-Warith-conversion] - 1201 | *extension_len = *list_len + - | ^ - ``` +Stefan Eissing (9 Mar 2024) - Closes #12616 +- lib: keep conn IP information together -- asyn-thread: silence `-Wcast-align` warning for Windows + new struct ip_quadruple for holding local/remote addr+port - Seen with llvm/clang 17: - ``` - lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct - thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align] - 310 | CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlap - ped); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~ - .../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded fro - m macro 'CONTAINING_RECORD' - 717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(addre - ss) - (ULONG_PTR)(&((type *)0)->field))) - | ^~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ``` + - used in data->info and conn and cf-socket.c + - copy back and forth complete struct + - add 'secondary' to conn + - use secondary in reporting success for ftp 2nd connection - Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482 + Reported-by: DasKutti on github + Fixes #13084 + Closes #13090 - Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261 - Closes #12615 +Daniel Stenberg (8 Mar 2024) -Daniel Stenberg (2 Jan 2024) +- scripts/managen: the new name and home for the manpage generator -- tool_listhelp: regenerate after recent .d updates + It was previously docs/cmdline-opts/gen.pl - Makes it survive test 1478 + Closes #13089 - Closes #12612 +- VULN-DISCLOSURE-POLICY.md: update detail about CVE requests -- test1478: verify src/tool_listhelp.c + curl is a CNA now - Verify that the source file on disk is identical to the output of gen.pl - listhelp, as otherwise they are out of sync and need attention. + Closes #13088 - Closes #12612 +Stefan Eissing (8 Mar 2024) -- testutil: make runtests support %include +- lib: client reader polish - Using this instruction, a test case can include the contents of a file - into the test during the preprocessing. + - seek_func/seek_client, use transfer values only + - remove copies held in `struct connectdata`, use only + ever `data->set.seek_func` + - resolves possible issues in multiuse connections + - new mime post reader eliminates need to ever overwriting this - Closes #12612 + - websockets, remove empty Curl_ws_done() function -- runtests: for mode="text" on , fix newlines on both parts + Closes #13079 - Closes #12612 +Marcel Raad (8 Mar 2024) -Jay Satiro (2 Jan 2024) +- lib1598: fix `CURLOPT_POSTFIELDSIZE` usage -- quiche: return CURLE_HTTP3 on send to invalid stream + It requires a `long` argument. - Prior to this change if a send failed on a stream in an invalid state - (according to quiche) and not marked as closed (according to libcurl) - then the send function would return CURLE_SEND_ERROR. + Closes https://github.com/curl/curl/pull/13085 - We already have similar code for ngtcp2 to return CURLE_HTTP3 in this - case. +Daniel Stenberg (8 Mar 2024) - Caught by test test_07_upload.py: test_07_22_upload_parallel_fail. +- docs/cmdline-opts: drop the curl.1 from the dist tarball - Fixes https://github.com/curl/curl/issues/12590 - Closes https://github.com/curl/curl/pull/12597 + Since it is no longer needed for building tool_hugehelp.c and all the + docs is available in readable markdown format in the tarball, the peeps + that don't want to build the manpage still do good. -Daniel Stenberg (1 Jan 2024) + Removing it also fixes the complexity of out-of-tree builds when the + curl.1 exists in the source tree. -- cmdline-opts: update availability for the *-ca-native options +- test1140/1173: extend wildcards to find curl.1 - Closes #12613 + ... in its new build path. -Patrick Monnerat (31 Dec 2023) + Also update the test scripts to be more precise in error messages to + help us understand CI errors better. -- openldap: fix STARTTLS + Follow-up to f03c85635f35269f1 + Ref: #13029 + Closes #13083 - It was not working anymore since introduction of connection filters. +- http2: minor tweaks to optimize two struct sizes - Also do not attempt to recover from a failing TLS negotiation with - CURLUSESSL_TRY. + - use BIT() instead of bool + - place the struct fields in (roughly) size order - Closes #12610 + Closes #13082 -Daniel Stenberg (31 Dec 2023) +- buildconf.bat: remove outdated groff/nroff use -- haproxy-clientip.d: document the arg + - don't try to generate the real hugehelp file, because it requires + curl.txt which needs a build + - don't attempt to do anything in a c-ares subdirectory - The arg keyword was missing and therefore not present in the man page. + Follow-up to f03c85635f35269 + Closes #13078 - Closes #12611 +- http2: memory errors in the push callbacks are fatal -annalee (29 Dec 2023) + Use the correct nghttp2 error code accordingly. -- configure: fix no default int compile error in ipv6 detection + Closes #13081 - Closes #12607 +Viktor Szakats (7 Mar 2024) -Dan Fandrich (28 Dec 2023) +- mkhelp: rename variable to fix compiler warnings -- CI: Fix use of any-glob-to-all-files in the labeler + ``` + src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global de + claration [_bld\src\curl.vcxproj] + _bld\src\tool_hugehelp.c(8,27): + see declaration of 'm' + src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75c + d2n0wj6lc#L614 - Despite its name, this atom acts like one-glob-to-all-files and a - different syntax with braces must be used to get - any-glob-to-all-files semantics. Unfortunately, this makes the file - completely unreadable. + Reviewed-by: Daniel Stenberg + Closes #13077 - Ref: https://github.com/actions/labeler/issues/731 +Daniel Stenberg (7 Mar 2024) -Daniel Stenberg (29 Dec 2023) +- KNOWN_BUGS: POP3 issue when reading small chunks -- CURLOPT_AUTOREFERER.3: mention CURLINFO_REFERER + Closes #12063 -- CURLINFO_REFERER.3: clarify that it is the *request* header +- RELEASE-NOTES: synced - That libcurl itself sent in the most recent request +Robert Moreton (7 Mar 2024) - Closes #12605 +- asyn-ares: fix data race warning -Jay Satiro (28 Dec 2023) + - Store the c-ares version during global init. -- system_win32: fix a function pointer assignment warning + Prior to this change several threads could write the same data to a + static int variable at the same time. Though in practice it's not a + problem ThreadSanitizer may warn. - - Use CURLX_FUNCTION_CAST to suppress a function pointer assignment - warning. + Reported-by: Nikita Taranov + Assisted-by: Jay Satiro - a6bbc87f added lookups of some Windows API functions and then cast them - like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn - about that as breaking strict-aliasing rules so this PR changes those - assignments to use CURLX_FUNCTION_CAST. + Fixes #13065 + Closes #13000 - Bug: https://github.com/curl/curl/pull/12581#issuecomment-1869804317 - Reported-by: Marcel Raad +Stefan Eissing (7 Mar 2024) - Closes https://github.com/curl/curl/pull/12602 +- hyper: implement unpausing via client reader -- verify-examples.pl: fail verification on unescaped backslash + Just a tidy up to contain 'ifdef' pollution of common + code parts with implementation specifics. - - Check that all backslashes in EXAMPLE are properly escaped. + - remove the ifdef hyper unpausing in easy.c + - add hyper client reader for CURL_CR_PROTOCOL phase + that implements the unpause method for calling + the hyper waker if it is set - eg manpage must always use `\\n` never `\n`. + Closes #13075 - This is because the manpage requires we always double blackslash to show - a single backslash. Prior to this change an erroneous single backslash - would pass through and compile even though it would not show correctly - in the manpage. +- ngtcp2: no recvbuf for stream - Co-authored-by: Daniel Stenberg + - write response data directly to the transfer via + `Curl_xfer_write_resp()` like we do in HTTP/2. - Ref: https://github.com/curl/curl/pull/12588 + Closes #13073 - Closes https://github.com/curl/curl/pull/12589 +- docs/cmdline-opts/.gitignore: ignore curl.txt -- vtls: fix missing multissl version info + Closes #13076 - - Fix erroneous buffer copy logic from ff74cef5. +Evgeny Grin (Karlson2k) (7 Mar 2024) - Prior to this change the MultiSSL version info returned to the user - was empty. +- sha512_256: add support for GnuTLS and OpenSSL - Closes https://github.com/curl/curl/pull/12599 + This is a follow-up for PR #12897. -Daniel Stenberg (27 Dec 2023) + Add support for SHA-512/256 digest calculation by TLS backends. + Currently only OpenSSL and GnuTLS (actually, nettle) support + SHA-512/256. -- KNOWN_BUGS: [RTSP] Some methods do not support response bodies + Closes #13070 - Closes #12414 +- digest: add check for hashing error -Patrick Monnerat (27 Dec 2023) + Closes #13072 -- openldap: fix an LDAP crash +Viktor Szakats (7 Mar 2024) - Reported-by: Ozan Cansel - Fixes #12593 - Closes #12600 +- cmake: enable `ENABLE_CURL_MANUAL` by default -Daniel Stenberg (27 Dec 2023) + Meaning `curl.1` and `src/tool_hugehelp.c` are built by default, + and `--manual` in curl tool is also enabled by default. -- getinfo: CURLINFO_QUEUE_TIME_T + This syncs behaviour with autotools. - Returns the time, in microseconds, during which this transfer was held - in a waiting queue before it started "for real". A transfer might be put - in a queue if after getting started, it cannot create a new connection - etc due to set conditions and limits imposed by the application. + For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set + to a consistent date, e.g. the timestamp of `CHANGES`. - Ref: #12293 - Closes #12368 + A pre-built manual (e.g. the one distributed in the official source + tarball) will be ignored and rebuilt after this patch, unless + explicitly disabling this option. -- RELEASE-NOTES: synced + Fixes #13028 + Closes #13069 -Jay Satiro (26 Dec 2023) +Stefan Eissing (7 Mar 2024) -- examples/sendrecv: fix comment line length +- http2: push headers better cleanup - Caught by checksrc. + - provide common cleanup method for push headers -Haydar Alaidrus (23 Dec 2023) + Closes #13054 -- CURLOPT_POSTFIELDS.3: fix incorrect C string escape in example +Daniel Stenberg (7 Mar 2024) - - Escape inner quotes with two backslashes. +- GIT-INFO: convert to markdown - Two backslashes escapes the backslash for the man page and will show as - a single backslash. + Closes #13074 - eg: "{\\"name\\": \\"daniel\\"}" shows as "{\"name\": \"daniel\"}". +Richard Levitte (7 Mar 2024) - Closes https://github.com/curl/curl/pull/12588 +- cmake: fix libcurl.pc and curl-config library specifications -Viktor Szakats (23 Dec 2023) + Letting CMake figure out where libraries are located gives you full + paths. When generating libcurl.pc and curl-config, getting libraries as + full paths is unusual when one expects to get a list of -l. -- appveyor: tidy-ups + To meet expectations, an effort is made to convert the full paths into + -l, possibly with -L before it. - - replace two remaining backslashes with forward slashes. - - tidy up the way we form and pass `TFLAGS`. + Fixes #6169 + Fixes #12748 + Closes #12930 - Follow-up to 2d4d0c1fd32f5cc3f946c407c8eccd5477b287df #12572 +Daniel Stenberg (7 Mar 2024) - Closes #12582 +- test463: HTTP with -d @file with file containing CR, LF and null byte -Stefan Eissing (22 Dec 2023) +- paramhlp: fix CRLF-stripping files with "-d @file" -- transfer: fix upload rate limiting, add test cases + All CR and LF bytes should be stripped, as documented, and all other + bytes are inluded in the data. Starting now, it also excludes null bytes + as they would otherwise also cut the data short. - - add test cases for rate limiting uploads for all - http versions - - fix transfer loop handling of limits. Signal a re-receive - attempt only on exhausting maxloops without an EAGAIN - - fix `data->state.selectbits` forcing re-receive to also - set re-sending when transfer is doing this. + Reported-by: Simon K + Fixes #13063 + Closes #13064 - Reported-by: Karthikdasari0423 on github - Fixes #12559 - Closes #12586 +Viktor Szakats (7 Mar 2024) -Daniel Stenberg (22 Dec 2023) +- cmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled -- mbedtls: free the entropy when threaded + Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF` + when building without the Schannel TLS backend. - The entropy_free was never done for threaded builds, causing a small - (fixed) memory leak. + This in turn may have caused Kerberos, SPNEGO and SSPI features + disappearing even with `CURL_WINDOWS_SSPI=ON` set. - Reported-by: RevaliQaQ on github - Fixes #12584 - Closes #12585 + This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a + default for `CURL_WINDOWS_SSPI`, but allowing a manual override. -Stefan Eissing (22 Dec 2023) + Also update the option text to better tell its purpose. -- http2: improved on_stream_close/data_done handling + Thanks-to: Andreas Loew + Reviewed-by: Daniel Stenberg + Ref: #13056 + Closes #13061 - - there seems to be a code path that cleans up easy handles without - triggering DONE or DETACH events to the connection filters. This - would explain wh nghttp2 still holds stream user data - - add GOOD check to easy handle used in on_close_callback to - prevent crashes, ASSERTs in debug builds. - - NULL the stream user data early before submitting RST - - add checks in on_stream_close() to identify UNGOOD easy handles +Jay Satiro (6 Mar 2024) - Reported-by: Hans-Christian Egtvedt - Fixes #10936 - Closes #12562 +- KNOWN_BUGS: FTPS server compatibility on Windows with Schannel -Daniel Stenberg (22 Dec 2023) + - Remove "2.12 FTPS with Schannel times out file list operation" -- mprintf: overhaul and bugfixes + - Remove "7.12 FTPS directory listing hangs on Windows with Schannel" - In a test case using lots of snprintf() calls using many commonly used - %-codes per call, this version is around 30% faster than previous - version. + - Add "7.12 FTPS server compatibility on Windows with Schannel" - It also fixes the #12561 bug which made it not behave correctly when - given unknown %-sequences. Fixing that flaw required a different take on - the problem, which resulted in the new two-arrays model. + This change adds a more generic bug description that explains FTPS with + the latest curl and Schannel is not widely used and may have more bugs + than other TLS backends. - lib557: extended - Verify the #12561 fix and test more printf features + The two removed FTPS Schannel bugs can't be reproduced any longer and + were likely fixed by 24d6c288. - unit1398: fix test: It used a $ only for one argument, which is not - supported. + Ref: https://github.com/curl/curl/issues/5284 + Ref: https://github.com/curl/curl/issues/9161 + Ref: https://github.com/curl/curl/issues/12894 - Fixes #12561 - Closes #12563 + Closes https://github.com/curl/curl/pull/13032 -Viktor Szakats (21 Dec 2023) +- trace-config.md: remove the mutexed options list -- appveyor: replace PowerShell with bash + parallel autotools + - Remove the rendered manpage message that says: + "[--trace-config] is mutually exclusive to --trace and -v, --verbose". - PowerShell works (after a steep development curve), but one property of - it stuck and kept causing unresolvable usability issues: With - `$ErrorActionPreference=Stop`, it does abort on failures, but shows only - the first line of the error message. In `Continue` mode, it shows the - full error message, but doesn't stop on all errors. Another issue is - PowerShell considering any stderr output as if the command failed (this - has been improved in 7.2 (2021-Nov), but fixed versions aren't running - in CI and will not be for a long time in all test images.) + Actually it can be used with either of those options, which are mutually + exclusive to each other but not to --trace-config. - Thus, we're going with bash. + Ref: https://curl.se/docs/manpage.html#--trace-config - Also: - - use `-j2` with autotools tests, making them finish 5-15 minutes per - job faster. - - omit `POSIX_PATH_PREFIX`. - - use `WINDIR`. - - prefer forward slashes. + Closes https://github.com/curl/curl/pull/13031 - Follow-up to: 75078a415d9c769419aed4153d3d525a8eba95af #11999 - Ref: #12444 - - Fixes #12560 - Closes #12572 - -Pavel Pavlov (21 Dec 2023) - -- asyn-thread: use GetAddrInfoExW on >= Windows 8 +Daniel Stenberg (6 Mar 2024) - For doing async DNS resolution instead of starting a thread for each - request. +- mkhelp: simplify the generated hugehelp program - Fixes #12481 - Closes #12482 + Use a plain array and puts() every line, also allows us to provide the + strings without ending newlines. -Daniel Stenberg (21 Dec 2023) + - merge blank lines into the next one as a prefixed newline. + - turn eight consecutive spaces into a tab (since they can only be on the + left side of text) + - the newly generated tool_hugehelp is 3K lines shorter and 50K smaller + - modifies the top logo layout a little by reducing the indent -- strerror: repair get_winsock_error() + Closes #13047 - It would try to read longer than the provided string and crash. +- docs: ascii version of manpage without nroff - Follow-up to ff74cef5d4a0cf60106517a1c7384 - Reported-by: calvin2021y on github - Fixes #12578 - Closes #12579 + Create ASCII version of manpage without nroff -- CURLOPT_SSH_*_KEYFILE: clarify + - build src/tool_hugegelp.c from the ascii manpage + - move the the manpage and the ascii version build to docs/cmdline-opts + - remove all use of nroff from the build process + - should make the build entirely reproducible (by avoiding nroff) - Closes #12554 + - partly reverts 2620aa9 to build libcurl option man pages one by one + in cmake because the appveyor builds got all crazy until I did -ivanfywang (21 Dec 2023) + The ASCII version of the manpage -- ngtcp2: put h3 at the front of alpn + - is built with gen.pl, just like the manpage is + - has a right-justified column making the appearance similar to the previous + version + - uses a 4-space indent per level (instead of the old version's 7) + - does not do hyphenation of words (which nroff does) - Closes #12576 + History -Daniel Stenberg (21 Dec 2023) + We first made the curl build use nroff for building the hugehelp file in + December 1998, for curl 5.2. -- test460: verify a command line using --expand with no argument + Closes #13047 - This verifies the fix for #12565 +Stefan Eissing (6 Mar 2024) -- tool_getparam: do not try to expand without an argument +- lib: add `void *ctx` to reader/writer instances - This would lead to a segfault. + - `struct Curl_cwriter` and `struct Curl_creader` now carry a + `void *ctx` member that points to the instance as allocated. + - using `r->ctx` and `w->ctx` as pointer to the instance specific + struct that has been allocated - Fixes #12565 - Reported-by: Geeknik Labs - Closes #12575 + Reported-by: Rudi Heitbaum + Fixes #13035 + Closes #13059 -- RELEASE-NOTES: synced +- http: fix dead code in setting post client reader - Bumped version to 8.6.0 because of changes + - postsize was always 0, thus the check's else never happened + after the mime client reader was introduced -- Makefile.am: fix the MSVC project generation + Follow-up to 0ba47146f7ff3d + Closes #13060 - It made the vcxproj files not get included in dist tarballs. +- http2: fix push discard - Regression since 74423b5df4c8117891eb89 (8.5.0) + - fix logic in discarding a failed pushed stream so that + stream context is properly cleaned up - Reported-by: iAroc on github - Fixes #12564 - Closes #12567 + Closes #13055 -zengwei2000 (21 Dec 2023) +- transfer.c: break receive loop in speed limited transfers -- altsvc: free 'as' when returning error + - the change breaks looping in transfer.c receive for transfers that are + speed limited on having gotten *some* bytes. + - the overall speed limit timing is done in multi.c - Closes #12570 + Reported-by: Dmitry Karpov + Bug: https://curl.se/mail/lib-2024-03/0001.html + Closes #13050 - Signed-off-by: zengwei +- mime: add client reader -Viktor Szakats (20 Dec 2023) + Add `mime` client reader. Encapsulates reading from mime parts, getting + their length, rewinding and unpausing. -- build: fix `-Wconversion`/`-Wsign-conversion` warnings + - remove special mime handling from sendf.c and easy.c + - add general "unpause" method to client readers + - use new reader in http/imap/smtp + - make some mime functions static that are now only used internally - Fix remaining warnings in examples and tests which are not suppressed - by the pragma in `lib/curl_setup.h`. + In addition: + - remove flag 'forbidchunk' as no longer needed - Silence a toolchain issue causing warnings in `FD_SET()` calls with - older Cygwin/MSYS2 builds. Likely fixed on 2020-08-03 by: - https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=5717262b8ecfed0f7f - ab63e2c09c78991e36f9dd + Closes #13039 - Follow-up to 2dbe75bd7f3c36837aa06fd87a442bdf3fb7faef #12492 +Daniel Stenberg (5 Mar 2024) - Closes #12557 +- RELEASE-NOTES: synced -- build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings +- TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake" - - enable `-Wsign-conversion` warnings, but also setting them to not - raise errors. - - fix `-Warith-conversion` warnings seen in CI. - These are triggered by `-Wsign-converion` and causing errors unless - explicitly silenced. It makes more sense to fix them, there just a few - of them. - - fix some `-Wsign-conversion` warnings. - - hide `-Wsign-conversion` warnings with a `#pragma`. - - add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build - basis. - - update a CI job to unhide them with the above macro: - https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3 + Follow-up to 8e741644a229c37 - Closes #12492 +Tal Regev (5 Mar 2024) -- cmake: tidy-up `OtherTests.cmake` +- cmake: add USE_OPENSSL_QUIC support - - make more obvious which detection uses which prep steps. - - merge and streamline conditions. - - these should not alter detection results. + Closes #13034 - Also align log output messages from - `Macros.cmake` / `curl_internal_test` with rest of the build. +Stefan Eissing (5 Mar 2024) - Closes #12551 +- TIMER_STARTTRANSFER: set the same for everyone -- appveyor: switch to out-of-tree builds + - set TIMER_STARTTRANSFER on seeing the first response bytes + in the download client writer, not coming from a CONNECT + - initialized the timer the same way for all protocols + - remove explicit setting of TIMER_STARTTRANSFER in file.c + and c-hyper.c - With cmake and autotools. + Closes #13052 - Closes #12550 +Michael Kaufmann (5 Mar 2024) -Daniel Stenberg (19 Dec 2023) +- http: better error message for HTTP/1.x response without status line -- DEPRECATE.md: mention that NTLM_WB no longer works + If a response without a status line is received, and the connection is + known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status + line" instead of "Received HTTP/0.9 when not allowed". - Ref: #12479 - Closes #12553 + Closes #13045 -- CURLOPT_SERVER_RESPONSE_TIMEOUT_MS: add +Viktor Szakats (5 Mar 2024) - Proposed-by: Yifei Kong - Ref: https://curl.se/mail/lib-2023-11/0023.html - Closes #12369 +- KNOWN_BUGS: fix typo -Viktor Szakats (18 Dec 2023) + Reviewed-by: Daniel Stenberg + Closes #13051 -- build: more `-Wformat` fixes +Sebastian Neubauer (5 Mar 2024) - - memdebug: update to not trigger `-Wformat-nonliteral` warnings. - - imap: mark `imap_sendf()` with `CURL_PRINTF()`. - - tool_msgs: mark static function with `CURL_PRINTF()`. +- smpt: fix starttls - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + In cases where the connection was fast, curl sometimes failed to open a + connection. This fixes a regression of c2d973627bab12abc5486a3f3. - Closes #12540 + The regression triggered in these steps: -- windows: delete redundant headers + 1. Create an smtp connection + 2. Use STARTTLS + 3. Receive the response + 4. We are inside the loop in `smtp_statemachine`, calling + `smtp_state_starttls_resp` + 5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and + run `smtp_perform_upgrade_tls` at the start of the function. - `winsock2.h` pulls in `windows.h`. `ws2tcpip.h` pulls in `winsock2.h`. - `winsock2.h` and `ws2tcpip.h` are also pulled by `curl/curl.h`. + In the bad flow, we stay in the while loop, calling + `Curl_pp_readresp`, which reads part of the TLS handshake and things + go wrong. - Keep only those headers that are not already included, or the code under - it uses something from that specific header. + The reason is that `Curl_pp_moredata` changed behavior and always + returns `true`, so we stay in the loop in `smtp_statemachine`. With a + slow connection `Curl_pp_readresp` cannot read new data and returns + `CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`. - Closes #12539 + With a fast connection, `Curl_pp_readresp` reads new data from the tcp + connection, which is part of the TLS handshake. -- cmake: prefill/cache `HAVE_STRUCT_SOCKADDR_STORAGE` + The fix is in `Curl_pp_moredata`, which needs to take the final line + into account and return `false` if only the final line is stored. - Also add missing include to `OtherTests.cmake`. It didn't cause an issue - because the parent already included this earlier by chance. + Closes #13048 - Closes #12537 +Stefan Eissing (5 Mar 2024) -Daniel Stenberg (18 Dec 2023) +- lib: enhance client reader resume + rewind -- runner.pm: fix perl warning when running tests + - update client reader documentation + - client reader, add rewind capabilities + - tell creader to rewind on next start + - Curl_client_reset() will keep reader for future rewind if requested + - add Curl_client_cleanup() for freeing all resources independent of + rewinds + - add Curl_client_start() to trigger rewinds + - move rewind code from multi.c to sendf.c and make part of + "cr-in"'s implementation + - http, move the "resume_from" handling into the client readers + - the setup of a HTTP request is reshuffled to follow: + * determine method, target, auth negotiation + * install the client reader(s) for the request, including crlf + conversions and "chunked" encoding + * apply ranges to client reader + * concat request headers, upgrades, cookies, etc. + * complete request by determining Content-Length of installed + readers in combination with method + * send + - add methods for client readers to + * return the overall length they will generate (or -1 when unknown) + * return the amount of data on the CLIENT level, so that + expect-100 can decide if it want to apply itself + * set a "resume_from" offset or fail if unsupported + - struct HTTP has become largely empty now + - rename `Client_reader_*` to `Curl_creader_*` - Use of uninitialized value $runner::gdbthis in numeric eq (==) at runner. - pm + Closes #13026 - Follow-up from 3dcf301752a09d9 +Viktor Szakats (5 Mar 2024) - Closes #12549 +- openssl-quic: fix BIO leak and Windows warning -- runtests: support -gl. Like -g but for lldb. + Caused by an accidentally duplicated line in + d6825df334def106f735ce7e0c1a2ea87bddffb0. - Follow-up to 63b5748 + ``` + .../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses intege + r precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-6 + 4-to-32] + 1095 | bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE); + | ~~~~~~~~~~~~~ ~~~~~~~^~~~~~ + 1 warning and 2 errors generated. + ``` - Invokes the test case via lldb instead of gdb. Since using gdb is such a - pain on mac, using lldb is sometimes less quirky. + Reviewed-by: Stefan Eissing + Closes #13043 - Closes #12547 +- openssl-quic: fix unity build, casing, indentation -- curl.h: add CURLE_TOO_LARGE + - rename static functions to avoid duplicate symbols in unity mode. + - windows -> Windows/window in error message and comment. + - fix indentation. - A new error code to be used when an internal field grows too large, like - when a dynbuf reaches its maximum. Previously it would return - CURLE_OUT_OF_MEMORY for this, which is highly misleading. + Reviewed-by: Stefan Eissing + Closes #13044 - Ref: #12268 - Closes #12269 +Daniel Stenberg (5 Mar 2024) -- CI/circleci: disable MQTT in the HTTP-only build +- gen.pl: make the "manpageification" faster - And remove the use of configure options that don't actually exist + The function that replaces occurances of "--longoption" with "-Z, + --longoption" etc with the proper highlight applied, no longer loops + over the options. - Closes #12546 + Closes #13041 -Yedaya Katsman (18 Dec 2023) +- CONTRIBUTE: update the section on documentation format -- tests: respect $TMPDIR when creating unix domain sockets + ... since most of it is markdown now. - When running on termux, where $TMPDIR isn't /tmp, running the tests - failed, since the server config tried creating sockets in /tmp, without - checking the temp dir config. Use the TMPDIR variable that makes it find - the correct directory everywhere [0] + Closes #13046 - [0] https://perldoc.perl.org/File::Temp#tempfile +- smtp: free a temp resource - Closes #12545 + The returned address needs to be freed. -Viktor Szakats (17 Dec 2023) + Follow-up to e3905de8196d67b89df1602feb84c1f993211b20 + Spotted by Coverity -- ssh: fix namespace of two local macros + Closes #13038 - Avoid using the libssh and libssh2 macro namespaces by prefixing - these local macro names with `CURL_`. +- _VARIABLES.md: improve the description - Follow-up to 413a0fedd02c8c6df1d294534b8c6e306fcca7a2 #12346 + Closes #13040 - Reviewed-by: Daniel Stenberg - Closes #12544 +dependabot[bot] (4 Mar 2024) -- cmake: whitespace tidy-up in `OtherTests.cmake` +- build(deps): bump fsfe/reuse-action from 2 to 3 - Closes #12538 + Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3. + - [Release notes](https://github.com/fsfe/reuse-action/releases) + - [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3) -Mark Sinkovics (16 Dec 2023) + --- + updated-dependencies: + - dependency-name: fsfe/reuse-action + dependency-type: direct:production + update-type: version-update:semver-major + ... -- cmake: fix generation for system name iOS + Signed-off-by: dependabot[bot] - This PR fixes a problem that happens during CMake configuration when - the `CMAKE_SYSTEM_NAME` set to `iOS` and not `Darwin`. This value is - available (as far as I remember) version 3.14. The final solution - (thanks to @vszakats) is to use `APPLE` which contains all the Apple - platforms https://cmake.org/cmake/help/latest/variable/APPLE.html. +Stefan Eissing (4 Mar 2024) - This issue was found when during vcpkg installation. Running command - `vcpkg install curl:arm64-ios` and `vcpkg install curl:x64-ios` failed - with message: - ``` - CMake Error: try_run() invoked in cross-compiling mode, please set the follow - ing cache variables appropriately: - HAVE_H_ERRNO_ASSIGNABLE_EXITCODE (advanced) - ``` - After this fix, I was able to compile the compile the binary without - any issue. +- pytest: adapt to API change - In addition to that fix, this PR also contains an simplification to - check if the platform is not APPLE. + - pytest has changed the signature of the hook pytest_report_header() + for some obscure reason and that change landed in our CI now - Co-authored-by: Viktor Szakats - Closes #12515 + - remove the changed param that we never used anyway -Daniel Stenberg (16 Dec 2023) + Closes #13037 -- RELEASE-NOTES: synced +Daniel Stenberg (4 Mar 2024) -Baruch Siach (16 Dec 2023) +- cookie: if psl fails, reject the cookie -- gnutls: fix build with --disable-verbose + A libpsl install without data and no built-in database is now considered + bad enough to reject all cookies since they cannot be checked. It is + somewhat of a user error, but still. - infof() parameters must be defined event with --disable-verbose since - commit dac293cfb702 ("lib: apache style infof and trace - macros/functions"). + Reported-by: Dan Fandrich + Closes #13033 - Move also 'ptr' definition under !CURL_DISABLE_VERBOSE_STRINGS. +Stefan Eissing (4 Mar 2024) - Fixes the following build failure: +- lib: further send/upload handling polish - In file included from ../lib/sendf.h:29, - from vtls/gtls.c:44: - vtls/gtls.c: In function 'Curl_gtls_verifyserver': - vtls/gtls.c:841:34: error: 'version' undeclared (first use in this function); - did you mean 'session'? - 841 | gnutls_protocol_get_name(version), ptr); - | ^~~~~~~ + - Move all the "upload_done" handling to request.c - Closes #12505 + - add possibility to abort sending of a request + - add `Curl_req_done_sending()` for checks + - transfer.c: readwrite_upload() now clean -Viktor Szakats (16 Dec 2023) + - removing data->state.ulbuf and data->req.upload_fromhere -- build: delete unused `HAVE_{GSSHEIMDAL,GSSMIT,HEIMDAL}` + - as well as data->req.upload_present + - set data->req.upload_done on having read all from + the client and completely flushed the send buffer - Stop setting `HAVE_GSSHEIMDAL`, `HAVE_GSSMIT` and `HAVE_HEIMDAL`. - There was no place in the build system or source code that used them. + - tftp, remove setting of data->req.upload_fromhere - Reviewed-by: Daniel Stenberg - Closes #12506 + - serves no purpose as `upload_present` is not set + and the data itself is directly `sendto()` anyway -- build: remove redundant `CURL_PULL_*` settings + - smtp, make upload EOB conversion a client reader + - xfer_ulbuf addition - These macros were not propagated to the source code from CMake. + - add xfer_ulbuf for borrowing, similar to xfer_buf + - use in file upload + - use in c-hyper body sending - autotools set only one of them (`CURL_PULL_SYS_POLL_H`), initially to - address an AIX issue [1]. This later broke when introducing `system.h` - [2] without the logic it enabled. A subsequent fix [3] re-added the - logic, and also enabled it for AIX before its use, directly in - `system.h`. + - h1-proxy, remove init of data->state.uilbuf that is never used + - smb, add own send_buf instead of using data->state.ulbuf - [1] 2012-11-23: 665adcd4b7bcdb7deb638cdc499fbe71f8d777f2 - [2] 2017-03-29: 9506d01ee50d5908138ebad0fd9fbd39b66bd64d #1373 - [3] 2017-08-25: 8a84fcc4b59e8b78d2acc6febf44a43d6bc81b59 #1828 #1833 + Closes #13010 - Reviewed-by: Daniel Stenberg - Closes #12502 +Daniel Stenberg (4 Mar 2024) -- system.h: sync mingw `CURL_TYPEOF_CURL_SOCKLEN_T` with other compilers +- RELEASE-NOTES: synced - Align mingw with the other Windows compilers and use the `int` type for - `CURL_TYPEOF_CURL_SOCKLEN_T` (and thus for `curl_socklent_t`). This - makes it unnecessary to make a mingw-specific trick and pull all Windows - headers early just for this type definition. This type is specific to - Windows, not to the compiler. mingw-w64's Windows header maps it to - `int` too. +kpcyrd (3 Mar 2024) - With this we also delete all remaining uses of `CURL_PULL_WS2TCPIP_H`. +- rustls: fix two warnings related to number types - [ The official solution is to use `socklen_t` for all Windows compilers. - In this case we may want to update `curl/curl.h` to pull in Windows - headers before `system.h`. ] + Reported-by: Gisle Vanem + Follow-up to #12989 + Closes #13017 - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12501 +Stefan Eissing (3 Mar 2024) -- windows: simplify detecting and using system headers +- bufq: writing into a softlimit queue cannot be partial - - autotools, cmake: assume that if we detect Windows, `windows.h`, - `winsock2.h` and `ws2tcpip.h` do exist. - - lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of - looking for `winsock2.h`. - - autotools: merge 3 Windows check methods into one. - - move Watt-32 and lwIP socket support to `setup-win32.h` from - `config-win32.h`. It opens up using these with all build tools. Also - merge logic with Windows Sockets. - - fix to assume Windows sockets with the mingw32ce toolchain. - Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4 - - cmake: delete unused variable `signature_call_conv` since - eb33ccd5332435fa50f1758e5debb869c6942b7f. - - autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection. - - examples/externalsocket: fix header order. - - cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue` - that wasn't used anymore. - - cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test - `SIZEOF_STRUCT_SOCKADDR_STORAGE`. + - when unable to obtain a new chunk on a softlimit bufq, + this is an allocation error and needs to be reported as + such. + - writes into a soflimit bufq never must be partial success - After this patch curl universally uses `_WIN32` to guard - Windows-specific logic. It guards Windows Sockets-specific logic with - `USE_WINSOCK` (this might need further work). + Reported-by: Dan Fandrich + Fixes #13020 + Closes #13023 - Reviewed-by: Jay Satiro - Closes #12495 +Dan Fandrich (2 Mar 2024) -- build: enable missing OpenSSF-recommended warnings, with fixes +- configure: Don't build shell completions when disabled - https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening - -Guide-for-C-and-C++.html - as of 2023-11-29 [1]. + With the recent changes to completion file building, the files were + built always and only installation was selectively disabled. Now, when + they are disabled they aren't even built, avoiding a build-time error in + environments where it's not possible to run the curl binary that was + just created (e.g. if library paths were not set up correctly). - Enable new recommended warnings (except `-Wsign-conversion`): + Follow-up to 0f7aba83c - - enable `-Wformat=2` for clang (in both cmake and autotools). - - add `CURL_PRINTF()` internal attribute and mark functions accepting - printf arguments with it. This is a copy of existing - `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible - with redefinting the `printf` symbol: - https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 - - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for - mingw-w64 and enable it on this platform. - - enable `-Wimplicit-fallthrough`. - - enable `-Wtrampolines`. - - add `-Wsign-conversion` commented with a FIXME. - - cmake: enable `-pedantic-errors` the way we do it with autotools. - Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 - - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format - checks. Previously it was always disabled due to the internal `printf` - macro. + Reported-by: av223119 on github + Fixes #13027 + Closes #13030 - Fix them: +Jay Satiro (2 Mar 2024) - - fix bug where an `set_ipv6_v6only()` call was missed in builds with - `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. - - add internal `FALLTHROUGH()` macro. - - replace obsolete fall-through comments with `FALLTHROUGH()`. - - fix fallthrough markups: Delete redundant ones (showing up as - warnings in most cases). Add missing ones. Fix indentation. - - silence `-Wformat-nonliteral` warnings with llvm/clang. - - fix one `-Wformat-nonliteral` warning. - - fix new `-Wformat` and `-Wformat-security` warnings. - - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its - definition to `lib/curl_setup.h` allowing use in `tests/server`. - - lib: fix two wrongly passed string arguments in log outputs. - Co-authored-by: Jay Satiro - - fix new `-Wformat` warnings on mingw-w64. +- cmdline-opts/_EXITCODES: sync with libcurl-errors - [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895b - fc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options- - Hardening-Guide-for-C-and-C%2B%2B.md + - Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that + can be returned by the curl tool. - Closes #12489 + Closes https://github.com/curl/curl/pull/13015 -- Makefile.mk: drop Windows support +Stefan Eissing (1 Mar 2024) - And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga. +- hyper: disable test1598 due to lack of trailer support - We recommend CMake instead. With unity mode it's much faster, and about - the same without. + Follow-up to 50838095 - Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 - Reviewed-by: Daniel Stenberg - Closes #12224 + Closes #13016 -Daniel Stenberg (16 Dec 2023) +Dan Fandrich (1 Mar 2024) -- cmdline-docs: use .IP consistently +- ftp: Mark a const buffer as const - Remove use of .TP and some .B. The idea is to reduce nroff syntax as - much as possible and to use it consistently. Ultimately, we should be - able to introduce our own easier-to-use-and-read syntax/formatting and - convert on generation time. +- appveyor: Properly skip if only CircleCI is changed - Closes #12535 +- docs: Update minimal binary size in INSTALL.md -Tatsuhiko Miyagawa (16 Dec 2023) + Include more options to reduce binary size. -- http: fix off-by-one error in request method length check +- configure: Don't make shell completions without perl - It should allow one more byte. + The code that attempted to skip building the shell completions didn't + work properly and tried to build them even if perl wasn't available. + This step, as well as the install step, is now properly skipped without + perl. - Closes #12534 + Follow-up to 89733e2dd -Daniel Stenberg (15 Dec 2023) + Closes #13022 -- curl: show ipfs and ipns as supported "protocols" +RainRat (1 Mar 2024) - They are accepted schemes in URLs passed to curl (the tool, not the - library). +- misc: Fix typos in docs and lib - Also makes curl-config show the same list. + This fixes miscellaneous typos and duplicated words in the docs, lib + and test comments and a few user facing errorstrings. - Co-Authored-by: Jay Satiro - Reported-by: Chara White - Bug: https://curl.se/mail/archive-2023-12/0026.html - Closes #12508 + Author: RainRat on Github + Reviewed-by: Daniel Gustafsson + Reviewed-by: Dan Fandrich + Closes: #13019 -- Revert "urldata: move async resolver state from easy handle to connectdata" +Dan Fandrich (29 Feb 2024) - This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198) +- configure: build & install shell completions when enabled - We want the c-ares channel to be held in the easy handle, not per - connection - for performance. + The --with-fish-functions-dir and --with-zsh-functions-dir options + currently have no effect on a normal build because the scripts/ directory + where they're used is not built. Add scripts/ to a normal build and + change the completion options to default to off to preserve the existing + behaviour. - Closes #12524 + Closes: #12906 -Viktor Szakats (15 Dec 2023) +- github/labeler: improve the match patterns -- openssl: re-match LibreSSL deinit with init +Stefan Eissing (28 Feb 2024) - Earlier we switched to use modern initialization with LibreSSL v2.7.0 - and up, but did not touch deinitialization [1]. Fix it in this patch. +- tests: add test1598 for POST with trailers - Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611 + - test POST fields with trailers and chunked encoding - [1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014 + Ref: #12938 + Closes #13009 - Reported-by: Mike Hommey - Reviewed-by: Daniel Stenberg - Fixes #12525 - Closes #12526 +Daniel Stenberg (28 Feb 2024) -Daniel Stenberg (14 Dec 2023) +- cmdline-opts/_VERSION: provide %VERSION correctly -- libssh: supress warnings without version check + ... so that it does not get included verbatim in the output. Fixes a + regression shipped in 8.6.0. - Define unconditionally. + Also fix a format mistake in form.md - Follow-up from d21bd2190c46ad7fa + Closes #13008 - Closes #12523 +Stefan Eissing (28 Feb 2024) -- hostip: return error immediately when Curl_ip2addr() fails +- lib: Curl_read/Curl_write clarifications - Closes #12522 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to + clarify when and at what level they operate + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer + setup of `conn->sockfd` and `conn->writesockfd` on which + connection filter chain to operate. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index + as parameter. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for + naming consistency + - clarify that the special CURLE_AGAIN hangling to return + `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` + and CURLE_AGAIN is returned by all other send() variants. + - fix a bug in websocket `curl_ws_recv()` that mixed up data + when it arrived in more than a single chunk (to be made + into a sperate PR, also) -Theo (14 Dec 2023) + Added as documented [in + CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68 + aa06dc738cb75d4/docs/CLIENT-READERS.md). -- libssh: improve the deprecation warning dismissal + - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` + - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` + - HTTP chunked uploads are now formatted in a client reader added when + needed. + - FTP line-end conversions are done in a client reader added when + needed. + - when sending requests headers, remaining buffer space is filled with + body data for sending in "one go". This is independent of the request + body size. Resolves #12938 as now small and large requests have the + same code path. - Previous code was compiler dependant, and dismissed all deprecation warnings - indiscriminately. + Changes done to test cases: - libssh provides a way to disable the deprecation warnings for libssh only, an - d - naturally this is the preferred way. + - test513: now fails before sending request headers as this initial + "client read" triggers the setup fault. Behaves now the same as in + hyper build + - test547, test555, test1620: fix the length check in the lib code to + only fail for reads *smaller* than expected. This was a bug in the + test code that never triggered in the old implementation. - This commit uses that, to prevent the erroneous hiding of potential, unrelate - d - deprecation warnings. + Closes #12969 - Fixes #12519 - Closes #12520 +Daniel Gustafsson (28 Feb 2024) -Daniel Stenberg (14 Dec 2023) +- curldown: Fix email address in Copyright -- test1474: removed + The curldown conversion accidentally replaced daniel@haxx.se with + just daniel.se. This reverts back to the proper email address in + the curldown docs as well as in a few other stray places where it + was incorrect (while unrelated to curldown). - The test was already somewhat flaky and disabled on several platforms, - and after 1da640abb688 even more unstable. + Reviewed-by: Daniel Stenberg + Closes: #12997 -- readwrite_data: loop less +Daniel Stenberg (28 Feb 2024) - This function is made to loop in order to drain incoming data - faster. Completely removing the loop has a measerably negative impact on - transfer speeds. +- getparam: make --ftp-ssl work again - Downsides with the looping include + Follow-up to 9e4e527 which accidentally broke it - - it might call the progress callback much more seldom. Especially if - the write callback is slow. + Reported-by: Jordan Brown + Fixes #13006 + Closes #13007 - - rate limiting becomes less exact +- KNOWN_BUGS: IMAPS connection fails with rustls error - - a single transfer might "starve out" other parallel transfers + Closes #10457 - - QUIC timers for other connections can't be maintained correctly +- KNOWN_BUGS: FTPS upload, FileZilla, GnuTLS and close_notify - The long term fix should be to remove the loop and optimize coming back - to avoid the transfer speed penalty. + Closes #11383 - This fix lower the max loop count to reduce the starvation problem, and - avoids the loop completely for when rate-limiting is in progress. +- KNOWN_BUGS: Implicit FTPS upload timeout - Ref: #12488 - Ref: https://curl.se/mail/lib-2023-12/0012.html - Closes #12504 + Closes #11720 -Stefan Eissing (14 Dec 2023) +- KNOWN_BUGS: HTTP/2 prior knowledge over proxy -- lib: eliminate `conn->cselect_bits` + Closes #12641 - - use `data->state.dselect_bits` everywhere instead - - remove `bool *comeback` parameter as non-zero - `data->state.dselect_bits` will indicate that IO is - incomplete. +- TODO: build HTTP/3 with OpenSSL and nghttp3 using cmake - Closes #12512 + Closes #12988 -- connect: refactor `Curl_timeleft()` +- TODO: Select signature algorithms - - less local vars, "better" readability - - added documentation + Closes #12982 - Closes #12518 +- examples: use present tense in comments -Dmitry Karpov (14 Dec 2023) + remove "will" and some other word fixes -- cookie: avoid fopen with empty file name + Closes #13003 - Closes #12514 +- docs: more language cleanups -Viktor Szakats (13 Dec 2023) + - present tense + - avoid bad words -- tests/server: delete workaround for old-mingw + Closes #13003 - mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this. +Daniel Gustafsson (27 Feb 2024) - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 +- setopt: Fix disabling all protocols - Reviewed-by: Jay Satiro - Closes #12510 + When disabling all protocols without enabling any, the resulting + set of allowed protocols remained the default set. Clearing the + allowed set before inspecting the passed value from --proto make + the set empty even in the errorpath of no protocols enabled. -- cmake: delete obsolete TODOs more [ci skip] + Co-authored-by: Dan Fandrich + Reported-by: Dan Fandrich + Reviewed-by: Daniel Stenberg + Closes: #13004 - - manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288 - - soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023 - - bunch of others that are completed - - `NTLM_WB_ENABLED` is implemented in a basic form, and now also - scheduled for removal, so a TODO at this point isn't useful. +Andreas Kiefer (27 Feb 2024) - And this 'to-check' item: - - Q: "The cmake build selected to run gcc with -fPIC on my box while the - plain configure script did not." +- fopen: fix narrowing conversion warning on 32-bit Android - A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 - and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly - enable PIC for libcurl shared lib. Or when building libcurl for - shared and static lib in a single pass. We do this by default for - Windows or when enabled by the user via `SHARE_LIB_OBJECT`. - Otherwise we don't touch this setting. Meaning the default set by - CMake (if any) or the toolchain is used. On Debian Bookworm, this - means that PIC is disabled for static libs by default. Some platforms - (like macOS), has PIC enabled by default. - autotools supports the double-pass mode only, and in that case - CMake seems to match PIC behaviour now (as tested on Linux with gcc.) + This was fixed in commit 06dc599405f, but came back in commit + 03cb1ff4d62. - Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500 + When building for 32-bit ARM or x86 Android, `st_mode` is defined as + `unsigned int` instead of `mode_t`, resulting in a + `-Wimplicit-int-conversion` clang warning because `mode_t` is + `unsigned short`. Add a cast to silence the warning, but only for + 32-bit Android builds, because other architectures and platforms are + not affected. - Reviewed-by: Jay Satiro - Closes #12509 + Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li + bc/include/sys/stat.h#86 + Closes https://github.com/curl/curl/pull/12998 -Stefan Eissing (12 Dec 2023) +Stefan Eissing (27 Feb 2024) -- CLIENT-WRITERS: design and use documentation +- lib: Curl_read/Curl_write clarifications - Closes #12507 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to + clarify when and at what level they operate + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer + setup of `conn->sockfd` and `conn->writesockfd` on which + connection filter chain to operate. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index + as parameter. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for + naming consistency + - clarify that the special CURLE_AGAIN hangling to return + `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` + and CURLE_AGAIN is returned by all other send() variants. + - fix a bug in websocket `curl_ws_recv()` that mixed up data + when it arrived in more than a single chunk -Viktor Szakats (12 Dec 2023) + The method for sending not just raw bytes, but bytes that are either + "headers" or "body". The send abstraction stack, to to bottom, now is: -- cmake: delete obsolete TODO items [ci skip] + * `Curl_req_send()`: has parameter to indicate amount of header bytes, + buffers all data. + * `Curl_xfer_send()`: knows on which socket index to send, returns + amount of bytes sent. + * `Curl_conn_send()`: called with socket index, returns amount of bytes + sent. - There is always room for improvement, but CMake is up to par now with - autotools, so there is no longer a good reason to keep around these - inline TODO items. + In addition there is `Curl_req_flush()` for writing out all buffered + bytes. - Answering one of questions: + `Curl_req_send()` is active for requests without body, + `Curl_buffer_send()` still being used for others. This is because the + special quirks need to be addressed in future parts: - Q: "The gcc command line use neither -g nor any -O options. As a - developer, I also treasure our configure scripts's --enable-debug - option that sets a long range of "picky" compiler options." + * `expect-100` handling + * `Curl_fillreadbuffer()` needs to add directly to the new + `data->req.sendbuf` + * special body handlings, like `chunked` encodings and line end + conversions will be moved into something like a Client Reader. - A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info - and optimization level. E.g.: - - `Release` = `-O3` + no debug info - - `MinSizeRel` = `-Os` + no debug info - - `Debug` = `-O0` + debug info + In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, + replace the `ssize_t` with a `size_t`. It makes no sense to allow for negativ + e + values as the returned `CURLcode` already specifies error conditions. This + allows easier handling of lengths without casting. - https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-deb - ug-release-relwithdebinfo-and-minsizerel/59314670#59314670 - https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#defaul - t-and-custom-configurations + Closes #12964 - For picky warnings we have the `PICKY_COMPILER` options, enabled by - default. +Daniel Stenberg (27 Feb 2024) - Closes #12500 +- multi: make add_handle free any multi_easy -Stefan Eissing (11 Dec 2023) + If the easy handle that is being added to a multi handle has previously + been used for curl_easy_perform(), there is a private multi handle here + that we can kill off. While it flushes some caches etc for the easy + handle would it be used for an easy interface transfer again after being + used in the multi stack, this cleanup simplifies behavior and uses less + memory. -- CONNECTION-FILTERS: update documentation + Closes #12992 - Closes #12497 +- docs: use present tense -Daniel Stenberg (11 Dec 2023) + avoid "will", detect "will" as a bad word in the CI -- lib: reduce use of strncpy + Also line wrapped a bunch of paragraphs - - bearssl: select cipher without buffer copies - - http_aws_sigv4: avoid strncpy, require exact timestamp length - - http_aws_sigv4: use memcpy isntead of strncpy - - openssl: avoid strncpy calls - - schannel: check for 1.3 algos without buffer copies - - strerror: avoid strncpy calls - - telnet: avoid strncpy, return error on too long inputs - - vtls: avoid strncpy in multissl_version() + Closes #13001 - Closes #12499 +- CURLOPT_SSL_CTX_FUNCTION.md: no promises of lifetime after return -- CI/distcheck: run full tests + ... and cleanup other language. - To be able to detect missing files better, this now runs the full CI - test suite. If done before, it would have detected #12462 before - release. + Closes #12999 - Closes #12503 +Stefan Eissing (27 Feb 2024) -- docs: clean up Protocols: for cmdline options +- lib: send rework - ... and some other minor polish. + Curl_read/Curl_write clarifications - Closes #12496 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify + when and at what level they operate -- cmdline/gen: fix the sorting of the man page options + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer setup of + `conn->sockfd` and `conn->writesockfd` on which connection filter + chain to operate. - They were previously sorted based on the file names, which use a .d - extension, making "data" get placed after "data-binary" etc. Making the - sort ignore the extention fixes the ordering. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index as + parameter. - Reported-by: Boris Verkhovskiy - Bug: https://curl.se/mail/archive-2023-12/0014.html - Closes #12494 + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming + consistency -Daniel Gustafsson (9 Dec 2023) + - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` + with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is + returned by all other send() variants. -- doh: remove unused local variable + SingleRequest reshuffling - The nurl variable is no longer used during probing following - a refactoring, so remove. + - move functions into request.[ch] + - differentiate between reset and free + - add Curl_req_done() to perform last actions + - add a send `bufq` to SingleRequest for future use in keeping upload data - Closes #12491 + Closes #12963 -Jay Satiro (8 Dec 2023) +Daniel Stenberg (26 Feb 2024) -- build: fix Windows ADDRESS_FAMILY detection +- RELEASE-NOTES: synced - - Include winsock2.h for Windows ADDRESS_FAMILY detection. +- http_chunks: remove unused 'endptr' variable - Prior to this change cmake detection didn't work because it included - ws2def.h by itself, which is missing needed types from winsock2.h. + Closes #12996 - Prior to this change autotools detection didn't work because it did not - include any Windows header. +Louis Solofrizzo (26 Feb 2024) - In both cases libcurl would fall back on unsigned short as the address - family type, which is the same as ADDRESS_FAMILY. +- lib: initialize output pointers to NULL before calling strto[ff,l,ul] - Co-authored-by: Viktor Szakats + In order to make MSAN happy: - Closes https://github.com/curl/curl/pull/12441 + ==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x596f3b3ed246 in curlx_strtoofft [...]/libcurl/src/lib/strtoofft.c:23 + 9:11 + #1 0x596f3b402156 in Curl_httpchunk_read [...]/libcurl/src/lib/http_chunk + s.c:149:12 + #2 0x596f3b348550 in readwrite_data [...]/libcurl/src/lib/transfer.c:607: + 11 + [...] -Daniel Stenberg (8 Dec 2023) + ==2202041==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x5a3fab66a72a in Curl_parse_port [...]/libcurl/src/lib/urlapi.c:547:8 + #1 0x5a3fab650645 in parse_authority [...]/libcurl/src/lib/urlapi.c:796:1 + 2 + #2 0x5a3fab6740f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 + #3 0x5a3fab664fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: + 1342:12 + [...] -- lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding + ==2202320==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x569076a0d6b0 in ipv4_normalize [...]/libcurl/src/lib/urlapi.c:683:12 + #1 0x5690769f2820 in parse_authority [...]/libcurl/src/lib/urlapi.c:803:1 + 0 + #2 0x569076a160f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 + #3 0x569076a06fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: + 1342:12 + [...] - Since the copy does not stop at a null byte, let's not call it anything - that makes you think it works like the common strndup() function. + Signed-off-by: Louis Solofrizzo + Closes #12995 - Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat +Stefan Eissing (26 Feb 2024) - Closes #12490 +- lib: move client writer into own source -- convsrctest.pl: removed: not used, not shipped in tarballs + Refactoring of the client writer that passes the data to the + client/application's callback functions. -- tests: rename tests scripts to the test number + - split out into own source cw-out.[ch] from sendf.c - It is hard to name the scripts sensibly. Lots of them are similarly - named and the name did not tell which test that used them. + - move tempwrite and tempcount from data->state into the context of the + client writer - The new approach is rather to name them based on the test number that - runs them. Also helps us see which scripts are for individual tests - rather than for general test infra. + - redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On + paused transfers, this allows to "record" interleaved HEADER/BODY + chunks to be "played back" in the same order on unpausing. - - badsymbols.pl -> test1167.pl - - check-deprecated.pl -> test1222.pl - - check-translatable-options.pl -> test1544.pl - - disable-scan.pl -> test1165.pl - - error-codes.pl -> test1175.pl - - errorcodes.pl -> test1477.pl - - extern-scan.pl -> test1135.pl - - manpage-scan.pl -> test1139.pl - - manpage-syntax.pl -> test1173.pl - - markdown-uppercase.pl -> test1275.pl - - mem-include-scan.pl -> test1132.pl - - nroff-scan.pl -> test1140.pl - - option-check.pl -> test1276.pl - - options-scan.pl -> test971.pl - - symbol-scan.pl -> test1119.pl - - version-scan.pl -> test1177.pl + - keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. + On exceeding that, return CURLE_TOO_LARGE instead of + CURLE_OUT_OF_MEMORY as before. - Closes #12487 - -Michał Antoniak (8 Dec 2023) + - add method to be called when a transfer is DONE to allow writing of + any data still buffered -- sendf: fix compiler warning with CURL_DISABLE_HEADERS_API + - when paused, record HEADER writes exactly as they come for later + playback. HEADERs are documented to be written one-by-one. - fix MSVC warning C4189: 'htype': local variable is initialized but not - referenced - when CURL_DISABLE_HEADERS_API is defined. + Closes #12898 - Closes #12485 +- urldata: move authneg bit from conn to Curl_easy -Viktor Szakats (8 Dec 2023) + - from `conn->bits.authneg` to `data->req.authneg` + - this is a property of the request about to be made + and not a property of the connection + - in multiuse connections, transfer could step on each others + toes here potentially. -- tidy-up: whitespace + Closes #12949 - Closes #12484 +- c-hyper: add header collection writer in hyper builds -Stefan Eissing (7 Dec 2023) + Closes #12880 -- test_02_download: fix paramters to test_02_27 +- http: move headers collecting to writer - - it is a special client that only ever uses http/2 + - add a client writer that does "push" response + headers written to the client if the headers api + is enabled + - remove special handling in sendf.c + - needs to be installed very early on connection + setup to catch CONNECT response headers - Closes #12467 + Closes #12880 -Michał Antoniak (7 Dec 2023) +- sendf: Curl_client_write(), make passed in buf const -- vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY +Michał Antoniak (26 Feb 2024) - Closes #12459 +- lib: remove curl_mimepart object when CURL_DISABLE_MIME -Daniel Stenberg (7 Dec 2023) + Remove curl_mimepart object from UserDefined structure when + CURL_DISABLE_MIME flag is active. Reduce size of UserDefined structure. -- lib: strndup/memdup instead of malloc, memcpy and null-terminate + Also remove unreachable code: when CURL_DISABLE_MIME is set, httpreq can + never have HTTPREQ_POST_MIME value and the same goes for the + CURL_DISABLE_FORM_API flag and the HTTPREQ_POST_FORM value - - bufref: use strndup - - cookie: use strndup - - formdata: use strndup - - ftp: use strndup - - gtls: use aprintf instead of malloc + strcpy * 2 - - http: use strndup - - mbedtls: use strndup - - md4: use memdup - - ntlm: use memdup - - ntlm_sspi: use strndup - - pingpong: use memdup - - rtsp: use strndup instead of malloc, memcpy and null-terminate - - sectransp: use strndup - - socks_gssapi.c: use memdup - - vtls: use dynbuf instead of malloc, snprintf and memcpy - - vtls: use strdup instead of malloc + memcpy - - wolfssh: use strndup + Closes #12948 - Closes #12453 +kpcyrd (26 Feb 2024) -- strdup: remove the memchr check from Curl_strndup +- rustls: make curl compile with 0.12.0 - It makes it possible to clone a binary chunk of data. + Closes #12989 - Closes #12453 +Daniel Stenberg (26 Feb 2024) -- ftp: handle the PORT parsing without allocation +- strtoofft: fix the overflow check - Also reduces amount of *cpy() calls. + ... to not rely on wrapping, since it is an undefined behavior that is + not what always might happen. This is in our private strtoff() parser + function, used only on platforms without a native version. - Closes #12456 + Reported-by: vulnerabilityspotter on hackerone + Closes #12990 -- RELEASE-NOTES: synced +- libssh/libssh2: return error on too big range - Bumped to 8.5.1 + If trying to get the range 0 - 2^63 and the remote file is 2^63 bytes or + larger. -- url: for disabled protocols, mention if found in redirect + Fixes #12983 + Closes #12984 - To help users better understand where the URL (and denied scheme) comes - from. Also removed "in libcurl" from the message, since the disabling - can be done by the application. +Scott Talbert (24 Feb 2024) - The error message now says "not supported" or "disabled" depending on - why it was denied: +- setopt: fix check for CURLOPT_PROXY_TLSAUTH_TYPE value - Protocol "hej" not supported - Protocol "http" disabled + Prior to this change CURLOPT_PROXY_TLSAUTH_TYPE would return + CURLE_BAD_FUNCTION_ARGUMENT on any type other than NULL. Since there is + only one type of TLS auth and it is also the default (SRP) the TLS auth + would work anyway. - And in redirects: + Closes https://github.com/curl/curl/pull/12981 - Protocol "hej" not supported (in redirect) - Protocol "http" disabled (in redirect) +Jay Satiro (24 Feb 2024) - Reported-by: Mauricio Scheffer - Fixes #12465 - Closes #12469 +- mprintf: fix format prefix I32/I64 for windows compilers -Stefan Eissing (6 Dec 2023) + - Support I32 & I64 (eg: %I64d) for all Win32 builds. -- sectransp_ make TLSCipherNameForNumber() available in non-verbose config + Prior to this change mprintf support for the I format prefix, which is a + Microsoft extension, was dependent on the compiler used. - Reported-by: Cajus Pollmeier - Closes #12476 - Fixes #12474 + When Borland compiler support was removed in fd7ef00f the prefix was + then no longer supported for that compiler; however since it's still + possible to build with Borland I'm restoring support for the prefix in + this way. -YX Hao (6 Dec 2023) + Reported-by: Paweł Witas -- lib: fix variable undeclared error caused by `infof` changes + Fixes https://github.com/curl/curl/issues/12944 + Closes https://github.com/curl/curl/pull/12950 - `--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined. - `infof` isn't `Curl_nop_stmt` anymore: dac293c. +Daniel Stenberg (23 Feb 2024) - Follow-up to dac293c +- cd2nroff: gen: make `\>` in input to render as plain '>' in output - Closes #12470 + The same (copy and pasted) fix/mistake as in gen.pl -Viktor Szakats (6 Dec 2023) +- gen: make `\>` in input to render as plain '>' in output -- tidy-up: fix yamllint whitespace issues in labeler.yml + Reported-by: Gisle Vanem + Fixes #12977 + Closes #12978 - Follow-up to bda212911457c6fadfbba50be61afc4ca513fa56 #12466 +Fabrice Fontaine (23 Feb 2024) - Reviewed-by: Dan Fandrich - Closes #12475 +- configure.ac: find libpsl with pkg-config -- tidy-up: fix yamllint whitespace issues + Find libpsl with pkg-config to avoid static build failures. - Closes #12466 + Ref: http://autobuild.buildroot.org/results/1fb15e1a99472c403d0d3b1a688902f32 + e78d002 -Chris Sauer (6 Dec 2023) + Signed-off-by: Fabrice Fontaine + Closes #12947 -- cmake: fix typo +Daniel Stenberg (23 Feb 2024) - Follow-up to aace27b - Closes #12464 +- BUG-BOUNTY.md: clarify that the curl security team decides -Daniel Stenberg (6 Dec 2023) + Closes #12975 -- dist: add tests/errorcodes.pl to the tarball +- THANKS: add bug reporter from #740 - Used by test 1477 + Ref: https://github.com/curl/curl/issues/740 - Reported-by: Xi Ruoyao - Follow-up to 0ca3a4ec9a7 - Fixes #12462 - Closes #12463 +Stefan Eissing (22 Feb 2024) -Dan Fandrich (6 Dec 2023) +- multi: fix multi_sock handling of select_bits -- github/labeler: update a missed key in the v5 upgrade + - OR the event bitmask to data->state.select_bits instead of overwriting + them. They are cleared again on use. - Follow-up to ce03fe3ba + Reported-by: 5533asdg on github + Fixes #12971 + Closes #12972 -Version 8.5.0 (6 Dec 2023) +Daniel Stenberg (22 Feb 2024) -Daniel Stenberg (6 Dec 2023) +- curlver: bump to 8.7.0 for next release - RELEASE-NOTES: synced - The curl 8.5.0 release. +- write-out: add '%{proxy_used}' -Dan Fandrich (5 Dec 2023) + Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to + for example determine if a `NOPROXY` pattern matched the hostname or + not. -- github/labeler: switch from the beta to labeler v5 + Extended test 970 and 972 - Some keys were renamed and the dot option was made default. +- CURLINFO_USED_PROXY: return bool whether the proxy was used - Closes #12458 + Adds test536 to verify -Daniel Stenberg (5 Dec 2023) + Closes #12719 -- DEPRECATE: remove NTLM_WB in June 2024 +- sha512_256: remove the cast macro, minor language/format edits - Ref: https://curl.se/mail/lib-2023-12/0010.html + Follow-up to cbe41d151d6a100c - Closes #12451 + Closes #12966 -Jacob Hoffman-Andrews (4 Dec 2023) +Stefan Eissing (20 Feb 2024) -- rustls: implement connect_blocking +- DoH: add trace configuration - Closes #11647 + - refs #12397 where it is dicussed how to en-/disable verbose output + of DoH operations + - introducing `struct curl_trc_feat` to track a curl feature for + tracing + - adding `data->state.feat` optionally pointing to the feature a + transfer belongs to + - adding trace functions and verbosity checks on features + - using trace feature in DoH code + - documenting `doh` as feature for `--trace-config` -Daniel Stenberg (4 Dec 2023) + Closes #12411 -- examples/rtsp-options.c: add +- websocket: fix curl_ws_recv() - Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and - CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. + - when data arrived in several chunks, the collection into + the passed buffer always started at offset 0, overwriting + the data already there. - Closes #12452 + adding test_20_07 to verify fix -Stefan Eissing (4 Dec 2023) + - debug environment var CURL_WS_CHUNK_SIZE can be used to + influence the buffer chunk size used for en-/decoding. -- ngtcp2: ignore errors on unknown streams + Closes #12945 - - expecially in is_alive checks on connections, we might - see incoming packets on streams already forgotten and closed, - leading to errors reported by nghttp3. Ignore those. +Evgeny Grin (Karlson2k) (20 Feb 2024) - Closes #12449 +- digest: support SHA-512/256 -Daniel Stenberg (4 Dec 2023) + Also fix the tests. New implementation tested with GNU libmicrohttpd. + The new numbers in tests are real SHA-512/256 numbers (not just some + random ;) numbers ). -- docs: make all examples in all libcurl man pages compile +- tests: add SHA-512/256 unit test - Closes #12448 +- SHA-512/256: implement hash algorithm -- checksrc.pl: support #line instructions + Closes #12897 - makes it identify the correct source file and line +- curl_setup.h: add curl_uint64_t internal type -- GHA/man-examples: verify libcurl man page examples + The unsigned version of curl_off_t basically -- verify-examples.pl: verify that all man page examples compile clean +Daniel Stenberg (20 Feb 2024) -- RELEASE-NOTES: synced +- docs: dist curl*.1 and install without perl -Graham Campbell (2 Dec 2023) + Drop docs/mk-ca-bundle.1 from the tarball. It can be generated at will. -- http3: bump ngtcp2 and nghttp3 versions + Closes #12959 + Fixes #12921 + Reported-by: Michael Forney - nghttp3 v1.1.0 - ngtcp2 v1.1.0 +Stefan Eissing (20 Feb 2024) - In docs and CI +- OpenSSL QUIC: adapt to v3.3.x - Closes #12446 + - set our idle timeout as transport parameter + - query negotiated idle timeout for connection alive checks + - query number of available bidi streams on a connection + - use write_ex2 with SSL_WRITE_FLAG_CONCLUDE to signal + EOF on last chunk write, so stream close does not + require an additional QUIC packet -- CI/quiche: use `3.1.4+quic` consistently in CI workflows + Closes #12933 - Closes #12447 +Ramiro Garcia (19 Feb 2024) -Viktor Szakats (2 Dec 2023) +- MANUAL.md: fix typo -- test1545: disable deprecation warnings + Closes #12965 - Fixes: - https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr - p6pk#L1205 +Daniel Stenberg (19 Feb 2024) - Same with details: - https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil - b6wt#L1263 - ``` - tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 - .0. Use curl_mime_init() [-Werror=deprecated-declarations] - 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", - | ^~~~~~~~~~~~ - [...] - ``` +- BINDINGS: add mcurl, the python binding - Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 + Ref: #12956 + Closes #12962 - Fixes #12445 - Closes #12444 +- mk-ca-bundle.md: cleanups and polish -Daniel Stenberg (2 Dec 2023) + Closes #12958 -- INSTALL: update list of ports and CPU archs +- spellcheck.yml: remove .1/.3 handling, clean all man page .md files -- symbols-in-versions: the CLOSEPOLICY options are deprecated + Since we generate all .1 and .3 files from markdown now, we can limit + the spellcheck to the markdown versions only. - The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. + Closes #12960 -z2_ (1 Dec 2023) +- libcurl-docs: cleanups -- build: fix builds that disable protocols but not digest auth + CURLMOPT_SOCKETDATA.md: fix typo + CURLMOPT_TIMERDATA.md: fix typo + CURLOPT_COOKIELIST.m: quote strings + CURLOPT_PREREQFUNCTION.md: quote variable names + CURLOPT_TCP_NODELAY.md: rephrased to please spell checker + CURLOPT_WILDCARDMATCH.md: rephrased + libcurl-tutorial.md: use correct option name + curl_global_init_mem.md: quote headers + curl_easy_getinfo.md: use correct symbol names in headers + curl_global_trace.md: quote some headers + curl_ws_meta.md: quote struct field names + libcurl-env.md: quote headers - - Build base64 functions if digest auth is not disabled. +- cd2nroff: remove backticks from titles - Prior to this change if some protocols were disabled but not digest auth - then a build error would occur due to missing base64 functions. +- RELEASE-NOTES: synced - Fixes https://github.com/curl/curl/issues/12440 - Closes https://github.com/curl/curl/pull/12442 +Stefan Eissing (18 Feb 2024) -Michał Antoniak (1 Dec 2023) +- http_chunks: fix the accounting of consumed bytes -- connect: reduce number of transportation providers + Prior to this change chunks were handled correctly although in verbose + mode libcurl could incorrectly warn of "Leftovers after chunking" even + if there were none. - Use only the ones necessary - the ones that are built-in. Saves a few - bytes in the resulting code. + Reported-by: Michael Kaufmann - Closes #12438 + Fixes https://github.com/curl/curl/issues/12937 + Closes https://github.com/curl/curl/pull/12939 -David Benjamin (1 Dec 2023) +- file: use xfer buf for file:// transfers -- vtls: consistently use typedef names for OpenSSL structs + - For file:// transfers use the multi handle's transfer buffer for + up- and downloads. - The foo_st names don't appear in OpenSSL public API documentation. The - FOO typedefs are more common. This header was already referencing - SSL_CTX via . There is a comment about avoiding - , but OpenSSL actually declares all the typedefs in - , which is already included by (and - every other OpenSSL header), so just use that. Though I've included it - just to be explicit. + Prior to this change a6c9a33 (precedes 8.6.0) changed the file:// + transfers to use a smaller stack based buffer, and that caused a + significant performance decrease in Windows. - (I'm also fairly sure including already triggers the - Schannel conflicts anyway. The comment was probably just out of date.) + Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086 + Reported-by: edmcln@users.noreply.github.com - Closes #12439 + Closes https://github.com/curl/curl/pull/12932 -Lau (1 Dec 2023) +Karthikdasari0423 (18 Feb 2024) -- libcurl-security.3: fix typo +- HTTP3.md: always run nghttp3 submodule init - Fixed minimal typo. + - For consistency change all 'build nghttp3' commands to run submodule + init after cloning, even if the branch does not have submodules. - Closes #12437 + Follow-up to 5a4b2f93 and 4f794558. -Stefan Eissing (1 Dec 2023) + Closes https://github.com/curl/curl/pull/12928 -- ngtcp2: fix races in stream handling +LeeRiva (18 Feb 2024) - - fix cases where ngtcp2 invokes callbacks on streams that - nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND - in these cases as it is normal behaviour. +- CURLOPT_POSTQUOTE.md: fix typo - Closes #12435 + Closes https://github.com/curl/curl/pull/12926 -Emanuele Torre (1 Dec 2023) +Evgeny Grin (Karlson2k) (18 Feb 2024) -- tool_writeout_json: fix JSON encoding of non-ascii bytes +- checksrc.pl: fix handling .checksrc with CRLF - char variables if unspecified can be either signed or unsigned depending - on the platform according to the C standard; in most platforms, they are - signed. + - When parsing .checksrc chomp the (CR)LF line ending. - This meant that the *i<32 waas always true for bytes with the top bit - set. So they were always getting encoded as \uXXXX, and then since they - were also signed negative, they were getting extended with 1s causing - '\xe2' to be expanded to \uffffffe2, for example: + Prior to this change on Windows checksrc.pl would not process the + symbols in .checksrc properly, since many git repos in Windows use auto + crlf to check out files with CRLF line endings. - $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul - l - \uffffffe2\uffffff80\uffffff9c + Closes https://github.com/curl/curl/pull/12924 - I fixed this bug by making the code use explicitly unsigned char* - variables instead of char* variables. +Richard Levitte (18 Feb 2024) - Test 268 verifies +- cmake: fix install for older CMake versions - Reported-by: iconoclasthero - Closes #12434 + - Generate the docs install list by using a foreach loop instead of + LIST:TRANSFORM since older CMake can't handle the latter. -Stefan Eissing (1 Dec 2023) + Reported-by: Dan Fandrich -- cf-socket: TCP trace output local address used in connect + Fixes https://github.com/curl/curl/issues/12920 + Closes https://github.com/curl/curl/pull/12922 - Closes #12427 +Stefan Eissing (16 Feb 2024) -Jay Satiro (1 Dec 2023) +- vtls: fix tls proxy peer verification -- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation + - When verifying a proxy certificate for an ip address, use the correct + ip family. - - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it - includes protocol-specific instructions that trigger a transfer. + Prior to this change the "connection" ip family was used, which was not + necessarily the same. - Prior to this change it explicitly said that it did not include those - instructions in the time, but that is incorrect. + Reported-by: HsiehYuho@users.noreply.github.com - The change is a copy of the fixed explanation already in - CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. + Fixes https://github.com/curl/curl/issues/12831 + Closes https://github.com/curl/curl/pull/12931 - Reported-by: eeverettrbx@users.noreply.github.com +Dan Fandrich (15 Feb 2024) - Fixes https://github.com/curl/curl/issues/12431 - Closes https://github.com/curl/curl/pull/12432 +- CI: Bump the Circle CI base Ubuntu image to the latest 20.04 -Daniel Stenberg (30 Nov 2023) + The previous ones are going to be removed soon, plus the new ones + include all the fixes since then. -- multi: during ratelimit multi_getsock should return no sockets +Jay Satiro (13 Feb 2024) - ... as there is nothing to wait for then, it just waits. Otherwise, this - causes much more CPU work and updates than necessary during ratelimit - periods. +- transfer: improve Windows SO_SNDBUF update limit - Ref: https://curl.se/mail/lib-2023-11/0056.html - Closes #12430 + - Change the 1 second SO_SNDBUF update limit from per transfer to per + connection. -Dmitry Karpov (30 Nov 2023) + Prior to this change many transfers over the same connection could cause + many SO_SNDBUF updates made to that connection per second, which was + unnecessary. -- transfer: abort pause send when connection is marked for closing + Closes https://github.com/curl/curl/pull/12911 - This handles cases of some bi-directional "upgrade" scenarios - (i.e. WebSockets) where sending is paused until some "upgrade" handshake - is completed, but server rejects the handshake and closes the - connection. +- schannel: fix hang on unexpected server close - Closes #12428 + - Treat TLS connection close (either due to a close_notify from the + server or just closed due to receiving 0) as pending data. -Daniel Stenberg (28 Nov 2023) + This is because in some cases schannel_recv knows the connection is + closed but has to return actual pending data so it can't return 0 or an + error to indicate no more data. In this case schannel_recv must be + called again, which only happens if readwrite_data sees that there is + still pending data. -- RELEASE-NOTES: synced + Prior to this change if the total size of the body that libcurl expected + to receive from the server was unknown then it was possible under some + network conditions that libcurl would hang waiting to receive more data, + when in fact a close_notify alert indicating no more data would be sent + was already processed. -- openssl: when a session-ID is reused, skip OCSP stapling + Fixes https://github.com/curl/curl/issues/12894 + Closes https://github.com/curl/curl/pull/12910 - Fixes #12399 - Reported-by: Alexey Larikov - Closes #12418 +Daniel Stenberg (10 Feb 2024) -- test1545: test doing curl_formadd twice with missing file +- KNOWN_BUGS: FTP upload fails if remebered dir is deleted - Reproduces #12410 - Verifies the fix - Closes #12421 + Closes #12181 + Closes #12923 -- Curl_http_body: cleanup properly when Curl_getformdata errors +Michał Antoniak (10 Feb 2024) - Reported-by: yushicheng7788 on github - Based-on-work-by: yushicheng7788 on github - Fixes #12410 - Closes #12421 +- mbedtls: use mbedtls_ssl_conf_{min|max}_tls_version -- test1477: verify that libcurl-errors.3 and public headers are synced + ... instead of the deprecated mbedtls_ssl_conf_{min|max}_version - The script errorcodes.pl extracts all error codes from all headers and - checks that they are all documented, then checks that all documented - error codes are also specified in a header file. + Closes #12905 - Closes #12424 +Dan Fandrich (9 Feb 2024) -- libcurl-errors.3: sync with current public headers +- CI: bump to actions/cache@v4 to avoid warning - Closes #12424 +Evgeny Grin (Karlson2k) (9 Feb 2024) -Stefan Eissing (28 Nov 2023) +- test1165: improve pattern matching -- test459: fix for parallel runs + * Fix excluded digits at the end of the symbols ('CURL_DISABLE_POP3' + was checked as 'CURL_DISABLE_POP') - - change warniing message to work better with varying filename - length. - - adapt test output check to new formatting + Closes #12903 - Follow-up to 97ccc4479f77ba3191c6 - Closes #12423 +Dan Fandrich (9 Feb 2024) -Daniel Stenberg (27 Nov 2023) +- scripts: Fix cijobs.pl for Azure and GHA -- tool_cb_prg: make the carriage return fit for wide progress bars + The spacing in the yaml files changed. - When the progress bar was made max width (256 columns), the fly() - function attempted to generate its output buffer too long so that the - trailing carriage return would not fit and then the output would show - wrongly. The fly function is called when the expected total transfer is - unknown, which could be one or more progress calls before the actual - progress meter get shown when the expected transfer size is provided. +Daniel Stenberg (9 Feb 2024) - This new take also replaces the msnprintf() call with a much simpler - memset() for speed. +- RELEASE-NOTES: synced - Reported-by: Tim Hill - Fixes #12407 - Closes #12415 +- TODO: use pkg-config to find libpsl -- tool_parsecfg: make warning output propose double-quoting + Closes #12919 - When the config file parser detects a word that *probably* should be - quoted, mention double-quotes as a possible remedy. +- TODO: avoid nroff - Test 459 verifies. + Instead of adjusting roffit, skip the nroff step. - Proposed-by: Jiehong on github - Fixes #12409 - Closes #12412 + Closes #12919 -Jay Satiro (26 Nov 2023) +Dan Fandrich (9 Feb 2024) -- curl.rc: switch out the copyright symbol for plain ASCII +- Revert "CI: run Circle macOS builds on x86 for now" - .. like we already do for libcurl.rc. + This reverts commit 2683de3078eadc86d9b182e7417f4ee75a247e2c. + ARM resources are now available in Circle CI, so run these builds on ARM + again. This platform needs explicit paths set to libpsl and its + dependency icu4c. - libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" - warning so it was switched to ascii. + Follow-up to 2683de30 - Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 + Closes #12635 - Suggested-by: Robert Southee +Viktor Szakats (9 Feb 2024) - Closes https://github.com/curl/curl/pull/12403 +- cmake: add warning for using TLS libraries without 1.3 support -Daniel Stenberg (26 Nov 2023) + Closes #12900 -- conncache: use the closure handle when disconnecting surplus connections +Daniel Stenberg (9 Feb 2024) - Use the closure handle for disconnecting connection cache entries so - that anything that happens during the disconnect is not stored and - associated with the 'data' handle which already just finished a transfer - and it is important that details from the unrelated disconnect does not - taint meta-data in the data handle. +- configure: add warning for using TLS libraries without 1.3 support - Like storing the response code. + Closes #12900 - This also adjust test 1506. Unfortunately it also removes a key part of - the test that verifies that a connection is closed since when this - output vanishes (because the closure handle is used), we don't know - exactly that the connection actually gets closed in this test... +Michał Antoniak (9 Feb 2024) - Reported-by: ohyeaah on github - Fixes #12367 - Closes #12405 +- mbedtls: fix building when MBEDTLS_X509_REMOVE_INFO flag is defined -- RELEASE-NOTES: synced + Closes #12904 -Stefan Eissing (24 Nov 2023) +Stefan Eissing (9 Feb 2024) -- quic: make eyeballers connect retries stop at weird replies +- ftp: fix socket wait activity in ftp_domore_getsock - - when a connect immediately goes into DRAINING state, do - not attempt retries in the QUIC connection filter. Instead, - return CURLE_WEIRD_SERVER_REPLY - - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an - inconclusive answer. When all addresses have been attempted, - rewind the address list once on an inconclusive answer. - - refs #11832 where connects were retried indefinitely until - the overall timeout fired + - when waiting on the data connection, always add the control socket to + the pollset on state STOP or let the pingpong add the socket according + to its needs. - Closes #12400 + Reported-by: Fabian Vogt + Fixes #12901 + Closes #12913 -Daniel Stenberg (24 Nov 2023) +Daniel Stenberg (9 Feb 2024) -- CI: verify libcurl function SYNPOSIS sections +- dist: make sure the http tests are in the tarball - With the .github/scripits/verify-synopsis.pl script + Fixes #12914 + Reported-by: Fabian Vogt + Closes #12917 - Closes #12402 +Stefan Eissing (9 Feb 2024) -- docs/libcurl: SYNSOPSIS cleanup +- multi: add xfer_buf to multi handle - - use the correct include file - - make sure they are declared as in the header file - - fix minor nroff syntax mistakes (missing .fi) + - can be borrowed by transfer during recv-write operation + - needs to be released before borrowing again + - adjustis size to `data->set.buffer_size` + - used in transfer.c readwrite_data() - These are verified by verify-synopsis.pl, which extracts the SYNPOSIS - code and runs it through gcc. + Closes #12805 - Closes #12402 +Daniel Stenberg (9 Feb 2024) -- sendf: fix comment typo +- write-out.md: clarify error handling details -- fopen: allocate the dir after fopen + - it gets used even if the transfer fails - Move the allocation of the directory name down to after the fopen() call - to allow that shortcut code path to avoid a superfluous malloc+free - cycle. + - it does not cause error to be returned even if it fails - Follow-up to 73b65e94f35311 + Closes #12909 - Closes #12398 +Stefan Eissing (8 Feb 2024) -Stefan Eissing (24 Nov 2023) +- ftp: do lineend conversions in client writer -- transfer: cleanup done+excess handling + - remove the ftp special handling from sendf.c + - let ftp_do() add a client writer that does + the linened conversions + - change the lineend conversion to no longer + modify the passed buffer, but write smaller + chunks to the next cwriter instead. The + inefficiency of this will be mitigated once + we add output buffering for all client writes. - - add `SingleRequest->download_done` as indicator that - all download bytes have been received - - remove `stop_reading` bool from readwrite functions - - move excess body handling into client download writer + Closes #12878 - Closes #12371 +- ftp: tracing improvements -Daniel Stenberg (23 Nov 2023) + - trace socketindex for connection filters when not the first + - trace socket fd in tcp + - trace pollset adjusts in vtls -- fopen: create new file using old file's mode + Closes #12902 - Because the function renames the temp file to the target name as a last - step, if the file was previously owned by a different user, not ORing - the old mode could otherwise end up creating a file that was no longer - readable by the original owner after save. +Karthikdasari0423 (8 Feb 2024) - Reported-by: Loïc Yhuel - Fixes #12299 - Closes #12395 +- HTTP3.md: adjust the OpenSSL QUIC install instructions -- test1476: require proxy + tried installing with old steps but failed + tried with newly added setps and able to build + ``` + root@ubuntu:~/curl# ./src/curl -V + /root/curl/src/.libs/curl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSS + L_3.2.0' not found (required by /root/curl/lib/.libs/libcurl.so.4) + root@ubuntu:~/curl# + ``` + ``` + root@ubuntu:~/curl# ./src/curl -V + curl 8.6.1-DEV (x86_64-pc-linux-gnu) libcurl/8.6.1-DEV OpenSSL/3.2.0 zlib/1.2 + .11 brotli/1.0.9 libpsl/0.21.0 nghttp3/1.1.0 OpenLDAP/2.5.16 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp + Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz + NTLM PSL SSL threadsafe TLS-SRP UnixSockets + root@ubuntu:~/curl# + ``` - Follow-up from 323df4261c3542 + Closes #12896 - Closes #12394 +Daniel Stenberg (8 Feb 2024) -- fopen: create short(er) temporary file name +- TODO: align the TOC with the header - Only using random letters in the name plus a ".tmp" extension. Not by - appending characters to the final file name. +- docs: make sure curl.1 is included in dist tarballs - Reported-by: Maksymilian Arciemowicz + Ref: https://github.com/curl/curl/issues/12832#issuecomment-1933271873 - Closes #12388 + Closes #12892 -Stefan Eissing (23 Nov 2023) +Karthikdasari0423 (8 Feb 2024) -- tests: git ignore generated second-hsts.txt file +- HTTP3.md: remove quiche word in Openssl 3.2 - File is generated in test lib1900 + Closes #12893 - Follow-up to 7cb03229d9e9c5 +Daniel Stenberg (7 Feb 2024) - Closes #12393 +- curl: when allocating variables, add the name into the struct -Viktor Szakats (23 Nov 2023) + This saves the name from being an extra separate allocation. -- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 + Closes #12891 - Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to - OpenSSL 1.1.x, and LibreSSL 3.6 or upper. +- lib582: remove code causing warning that is never run - With the caveat, that "group name" and "type name" are missing from - the log output with these TLS backends. + The previous realloc code in this code could trigger a compiler warning, + but since that code path cannot happen in normal circumstances it now + instead exits with an error message there. - Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + Ref: #12887 + Closes #12890 - Reviewed-by: Daniel Stenberg - Closes #12385 +Stefan Eissing (7 Feb 2024) -Daniel Stenberg (23 Nov 2023) +- vtls: revert "receive max buffer" + add test case -- urldata: fix typo in comment + - add test_05_04 for requests using http/1.0, http/1.1 and h2 against an + Apache resource that does an unclean TLS shutdown. + - revert special workarund in openssl.c for suppressing shutdown errors + on multiplexed connections + - vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53 -- CI: codespell + Fixes #12885 + Fixes #12844 - The list of words to ignore is in the file - .github/scripts/codespell-ignore.txt + Closes #12848 - Closes #12390 +Daniel Stenberg (7 Feb 2024) -- lib: fix comment typos +- tests: support setting/using blank content env variables - Five separate ones, found by codespell - - Closes #12390 - -- test1476: verify cookie PSL mixed case - -- cookie: lowercase the domain names before PSL checks - - Reported-by: Harry Sintonen - - Closes #12387 - -Viktor Szakats (23 Nov 2023) - -- openssl: fix building with v3 `no-deprecated` + add CI test - - - build quictls with `no-deprecated` in CI to have test coverage for - this OpenSSL 3 configuration. - - - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. - The caller code is meant for OpenSSL 3, while these two functions were - only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 - if built with option `no-deprecated`, causing build errors: - ``` - vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ - algorithms'; ISO C99 and later do not support implicit function declaration - s [-Wimplicit-function-declaration] - vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ - digests'; ISO C99 and later do not support implicit function declarations [ - -Wimplicit-function-declaration] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f - ullLog=true#L7667 - - Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 - Reviewed-by: Alex Bozarth - - - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with - `no-deprecated` quictls 3 builds. - Do it by moving an existing solution for this from `vtls/openssl.c` - to `vtls/openssl.h` and adjusting caller code. - ``` - vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g - et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit - -function-declaration] - ``` - Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s - tep:24:1178 - - - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and - `-Wunused-function` when trying to build curl with NTLM enabled but - without the necessary TLS backend (with DES) support. - - Closes #12384 - -- curl.h: delete Symbian OS references - - curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 - via #5989. Delete references to it from public headers, because there - is no fresh release to use those headers with. - - Reviewed-by: Dan Fandrich - Reviewed-by: Jay Satiro - Closes #12378 - -- windows: use built-in `_WIN32` macro to detect Windows - - Windows compilers define `_WIN32` automatically. Windows SDK headers - or build env defines `WIN32`, or we have to take care of it. The - agreement seems to be that `_WIN32` is the preferred practice here. - Make the source code rely on that to detect we're building for Windows. - - Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for - Windows detection, next to the official `_WIN32`. After this patch it - only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. - - There is a slight chance these break compatibility with Windows - compilers that fail to define `_WIN32`. I'm not aware of any obsolete - or modern compiler affected, but in case there is one, one possible - solution is to define this macro manually. - - grepping for `WIN32` remains useful to discover Windows-specific code. - - Also: - - - extend `checksrc` to ensure we're not using `WIN32` anymore. - - - apply minor formatting here and there. - - - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. - - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg - - Closes #12376 - -Stefan Eissing (22 Nov 2023) - -- url: ConnectionExists revisited - - - have common pattern of `if not match, continue` - - revert pages long if()s to return early - - move dead connection check to later since it may - be relatively expensive - - check multiuse also when NOT building with NGHTTP2 - - for MULTIUSE bundles, verify that the inspected - connection indeed supports multiplexing when in use - (bundles may contain a mix of connection, afaict) - - Closes #12373 - -Daniel Stenberg (22 Nov 2023) - -- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range - - ... or use the default value. - - Also clarify the documentation language somewhat. - - Closes #12382 - -- urldata: make maxconnects a 32 bit value - - "2^32 idle connections ought to be enough for anybody" - - Closes #12375 - -- FEATURES: update the URL phrasing - - The URL is length limited since a while back so "no limit" simply is not - true anymore. Mention the URL RFC standard used instead. - - Closes #12383 - -- wolfssh: remove redundant static prototypes - - vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We - rror=redundant-decls] - - Closes #12381 - -- setopt: remove superfluous use of ternary expressions - - Closes #12374 - -- mime: store "form escape" as a single bit - - Closes #12374 - -- setopt: check CURLOPT_TFTP_BLKSIZE range on set - - ... instead of later when the transfer is about to happen. - - Closes #12374 - -Viktor Szakats (21 Nov 2023) - -- build: add more picky warnings and fix them - - Enable more picky compiler warnings. I've found these options in the - nghttp3 project when implementing the CMake quick picky warning - functionality for it [1]. - - `-Wunused-macros` was too noisy to keep around, but fixed a few issues - it revealed while testing. - - - autotools: reflect the more precisely-versioned clang warnings. - Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 - - autotools: sync between clang and gcc the way we set `no-multichar`. - - autotools: avoid setting `-Wstrict-aliasing=3` twice. - - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. - It triggers in libtool-generated stub code. - - - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. - - - lib/curl_setup.h: delete duplicate declaration for `fileno`. - Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d - (1999-12-29). This suggests this may not be needed anymore, but if - it does, we may restore this for those specific (non-Windows) systems. - - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since - c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. - - lib: delete unused macro `isxdigit_ascii` since - f65f750742068f579f4ee6d8539ed9d5f0afcb85. - - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. - - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. - - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` - macro. - - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. - - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. - - lib/setopt: fix `-Wunreachable-code-break`. - - lib/system_win32 and lib/timeval: fix double declarations for - `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. - - lib/warnless: fix double declarations in CMake UNITY mode [5]. - This was due to force-disabling the header guard of `warnless.h` to - to reapply it to source code coming after `warnless.c` in UNITY - builds. This reapplied declarations too, causing the warnings. - Solved by adding a header guard for the lines that actually need - to be reapplied. - - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. - - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant - block. - - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. - - lib/vtls/sectransp: suppress `-Wunreachable-code`. - Detected in `else` branches of dynamic feature checks, with results - known at compile-time, e.g. - ```c - if(SecCertificateCopySubjectSummary) /* -> true */ - ``` - Likely fixable as a separate micro-project, but given SecureTransport - is deprecated anyway, let's just silence these locally. - - src/tool_help: delete duplicate declaration for `helptext`. - - src/tool_xattr: fix `-Wunreachable-code`. - - tests: delete duplicate declaration for `unitfail` [8]. - - tests: delete duplicate declaration for `strncasecompare`. - - tests/libtest: delete duplicate declaration for `gethostname`. - Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 - (2010-08-02). - Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc - If there are still systems around with warnings, we may restore the - prototype, but limited for those systems. - - tests/lib2305: delete duplicate declaration for - `libtest_debug_config`. - - tests/h2-download: fix `-Wunreachable-code-break`. - - [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 - f8bf9f/cmake/PickyWarningsC.cmake - [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui - qla5fj45?fullLog=true#L1675 - [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 - 2331#step:7:72 - https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 - 2331#step:7:109 - [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L204 - [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L218 - [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 - 2331#step:7:290 - [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 - 2331#step:9:1193 - [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 - 2331#step:33:1870 - - Closes #12331 - -Daniel Stenberg (21 Nov 2023) - -- transfer: avoid unreachable expression - - If curl_off_t and size_t have the same size (which is common on modern - 64 bit systems), a condition cannot occur which Coverity pointed - out. Avoid the warning by having the code conditionally only used if - curl_off_t actually is larger. - - Follow-up to 1cd2f0072fa482e25baa2 - - Closes #12370 - -Stefan Eissing (21 Nov 2023) - -- transfer: readwrite improvements - - - changed header/chunk/handler->readwrite prototypes to accept `buf`, - `blen` and a `pconsumed` pointer. They now get the buffer to work on - and report back how many bytes they consumed - - eliminated `k->str` in SingleRequest - - improved excess data handling to properly calculate with any body data - left in the headerb buffer - - eliminated `k->badheader` enum to only be a bool - - Closes #12283 - -Daniel Stenberg (21 Nov 2023) - -- RELEASE-NOTES: synced - -Jiří Hruška (21 Nov 2023) - -- transfer: avoid calling the read callback again after EOF - - Regression since 7f43f3dc5994d01b12 (7.84.0) - - Bug: https://curl.se/mail/lib-2023-11/0017.html - - Closes #12363 - -Daniel Stenberg (21 Nov 2023) + - test450: remove --config from the keywords + - test2080: change return code + - test428: add --config as a keyword + - test428: disable on Windows due to CI problems -- doh: provide better return code for responses w/o addresses +- curl: exit on config file parser errors - Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the - response did not contain any addresses. Now it more accurately returns - CURLE_COULDNT_RESOLVE_HOST. + Like when trying to import an environment variable that does not exist. - Reported-by: lRoccoon on github + Also fix a bug for reading env variables when there is a default value + set. - Fixes #12365 - Closes #12366 + Bug: https://curl.se/mail/archive-2024-02/0008.html + Reported-by: Brett Buddin -Stefan Eissing (21 Nov 2023) + Add test 462 to verify. -- HTTP/2, HTTP/3: handle detach of onoing transfers + Closes #12862 - - refs #12356 where a UAF is reported when closing a connection - with a stream whose easy handle was cleaned up already - - handle DETACH events same as DONE events in h2/h3 filters +Daniel Szmulewicz (7 Feb 2024) - Fixes #12356 - Reported-by: Paweł Wegner - Closes #12364 +- CURLOPT_WRITEFUNCTION.md: typo fix -Viktor Szakats (20 Nov 2023) + The maximum amount of body data that is passed to the write + callback is defined in the curl.h header file -- autotools: stop setting `-std=gnu89` with `--enable-warnings` + Closes #12889 - Do not alter the C standard when building with `--enable-warnings` when - building with gcc. +Daniel Stenberg (7 Feb 2024) - On one hand this alters warning results compared to a default build. - On the other, it may produce different binaries, which is unexpected. +- lib: convert Curl_get_line to use dynbuf - Also fix new warnings that appeared after removing `-std=gnu89`: + Create the line in a dynbuf. Aborts the reading of the file on + errors. Avoids having to always allocate maximum amount from the + start. Avoids direct malloc. - - include: fix public curl headers to use the correct printf mask for - `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 - and Visual Studio 2013 and newer. This fixes the printf mask warnings - in examples and tests. E.g. [1] + Closes #12846 - - conncache: fix printf format string [2]. +- KNOWN_BUGS: unicode on Windows - - http2: fix potential null pointer dereference [3]. - (seen on Slackware with gcc 11.) + Closes #11461 + Closes #12231 + Closes #12883 - - libssh: fix printf format string in SFTP code [4]. - Also make MSVC builds compatible with old CRT versions. +- tool_operate: change precedence of server Retry-After time - - libssh2: fix printf format string in SFTP code for MSVC. - Applying the same fix as for libssh above. + - When calculating the retry time, no longer allow a server's requested + Retry-After time to take precedence over a longer retry time (either + default algorithmic or user-specified). - - unit1395: fix `argument is null` and related issues [5]: - - stop calling `strcmp()` with NULL to avoid undefined behaviour. - - fix checking results if some of them were NULL. - - do not pass NULL to printf `%s`. + Prior to this change the server's Retry-After time took precedence over + curl's retry time in all cases, but that's not always practical for + short Retry-After times depending on how busy the server is. - - ci: keep a build job with `-std=gnu89` to continue testing for - C89-compliance. We can apply this to other gcc jobs as needed. - Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 + Bug: https://curl.se/mail/archive-2024-01/0022.html + Reported-by: Dirk Hünniger - [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l - ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b - [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 - 2346#step:6:67 - [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 - 2346#step:30:214 - [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 - 2346#step:29:895 - [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 - 2346#step:33:1689 + Closes https://github.com/curl/curl/pull/12871 - Closes #12346 +- cmdline-docs: quote and angle bracket cleanup -- autotools: fix/improve gcc and Apple clang version detection + - make sure angle brackets are escaped + - remove a lot of superfluous double quotes + - replace several double quotes with backticks - - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu - may return `n-win32` (also with `-dumpfullversion`). Causing these - errors and failing to enable picky warnings: - ``` - ../configure: line 23845: test: : integer expression expected - ``` - Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 - 3718#step:5:143 + To make nicer-looking markdown. - Fix that by stripping any dash-suffix and handling a dotless (major-only) - version number by assuming `.0` in that case. + Closes #12884 - `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` - Ref: https://github.com/mamedev/mame/pull/9767 +- badwords: use hostname, not host name - - fix Apple clang version detection for releases between - 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the - version was under-detected as 3.7 llvm/clang equivalent. + and username, filename - consistently. Fixed the patterns in + badwords.txt to catch these. - - fix Apple clang version detection for 'Apple clang version 11.0.0' - and newer where the Apple clang version was detected, instead of its - llvm/clang equivalent. + Closes #12888 - - display detected clang/gcc/icc compiler version. +Viktor Szakats (6 Feb 2024) - Via libssh2: - - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d - 89875ad - https://github.com/libssh2/libssh2/pull/1187 - - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 - 42b592e - https://github.com/libssh2/libssh2/pull/1232 +- cmake: fix function description in comment [ci skip] - Closes #12362 + Closes #12879 -- autotools: delete LCC compiler support bits +Daniel Stenberg (6 Feb 2024) - Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 +- header.md: remove backslash, make nicer markdown - Closes #12357 + - remove a leftover backslash before a dash + - use backticks for "code" strings -- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` + Closes #12877 - - tests: verify CMake `DISABLE` options. +- docs: add mk-ca-bundle.1 to dist - Make an exception for 2 CMake-only ones, and one more that's - using a different naming scheme, also in autotools and source. + ... which also makes it get built. But don't build this or curl-config.1 + if build docs is disabled. - - cmake: add support for `CURL_DISABLE_HEADERS_API`. + Closes #12875 - Suggested-by: Daniel Stenberg - Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 +Stefan Eissing (6 Feb 2024) - Closes #12353 +- https-proxy: use IP address and cert with ip in alt names -Jacob Hoffman-Andrews (20 Nov 2023) + - improve info logging when peer verification fails to indicate + if DNS name or ip address has been tried to match + - add test case for contacting https proxy with ip address + - add pytest env check on loaded credentials and re-issue + when they are no longer valid + - disable proxy ip address test for bearssl, since not supported there -- hyper: temporarily remove HTTP/2 support + Ref: #12831 + Closes #12838 - The current design of the Hyper integration requires rebuilding the - Hyper clientconn for each request. However, building the clientconn - requires resending the HTTP/2 connection preface, which is incorrect - from a protocol perspective. That in turn causes servers to send GOAWAY - frames, effectively degrading performance to "no connection reuse" in - the best case. It may also be triggering some bugs where requests get - dropped entirely and reconnects take too long. +Jiawen Geng (6 Feb 2024) - This doesn't rule out HTTP/2 support with Hyper, but it may take a - redesign of the Hyper integration in order to make things work. +- docs: add necessary setup for nghttp3 - Closes #12191 + Now nghttp3 has submodules + https://github.com/ngtcp2/nghttp3/blob/main/.gitmodules -Jay Satiro (20 Nov 2023) + Closes #12859 -- schannel: fix unused variable warning +Peter Krefting (6 Feb 2024) - Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 - Reported-by: Viktor Szakats +- version: allow building with ancient libpsl - Closes https://github.com/curl/curl/pull/12361 + The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7 + ships with version 0.7.0 which lacks this API. Revert to using the older + versioning API if we detect an old libpsl version. -Daniel Stenberg (19 Nov 2023) + Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4 + Bug: https://curl.se/mail/archive-2024-02/0004.html + Reported-by: Scott Mutter + Closes #12872 -- url: find scheme with a "perfect hash" +Daniel Stenberg (6 Feb 2024) - Instead of a loop to scan over the potentially 30+ scheme names, this - uses a "perfect hash" table. This works fine because the set of schemes - is known and cannot change in a build. The hash algorithm and table size - is made to only make a single scheme index per table entry. +- TODO: Support latest rustls - The perfect hash is generated by a separate tool (scripts/schemetable.c) + Closes #12737 + Closes #12874 - Closes #12347 +- docs: make curldown do angle brackets like markdown -- scripts: add schemetable.c + Make sure we use \< and \> in markdown all over so that it renders + correctly, on GitHub and elsewhere. cd2nroff now outputs a warning if it + finds an unescaled angle bracket. - This tool generates a scheme-matching table. + Ref: #12854 + Closes #12869 - It iterates over a number of different initial and shift values in order - to find the hash algorithm that needs the smallest possible table. +- docs: fix the --disable-docs for autotools - The generated hash function, table and table size then needs to be used - by the url.c:Curl_getn_scheme_handler() function. + Follow-up to 541321507e386 -Stefan Eissing (19 Nov 2023) + Closes #12870 -- vtls/vquic, keep peer name information together +- RELEASE-NOTES: synced - - add `struct ssl_peer` to keep hostname, dispname and sni - for a filter - - allocate `sni` for use in VTLS backend - - eliminate `Curl_ssl_snihost()` and its use of the download buffer - - use ssl_peer in SSL and QUIC filters +- libcurl-security.md: Active FTP passes on the local IP address - Closes #12349 + Reported-by: Harry Sintonen + Closes #12867 -Viktor Szakats (18 Nov 2023) +Stefan Eissing (5 Feb 2024) -- build: always revert `#pragma GCC diagnostic` after use +- configure: do not link with nghttp3 unless necessary - Before this patch some source files were overriding gcc warning options, - but without restoring them at the end of the file. In CMake UNITY builds - these options spilled over to the remainder of the source code, - effecitvely disabling them for a larger portion of the codebase than - intended. + Fixes #12833 + Closes #12864 + Reported-by: Ryan Carsten Schmidt - `#pragma clang diagnostic` didn't have such issue in the codebase. +Daniel Stenberg (5 Feb 2024) - Reviewed-by: Marcel Raad - Closes #12352 +- THANKS: add Dmitry Tretyakov -- tidy-up: casing typos, delete unused Windows version aliases + ... since I missed to give credit to the report in the fix of #12861 - - cmake: fix casing of `UnixSockets` to match the rest of the codebase. +Stefan Eissing (5 Feb 2024) - - curl-compilers.m4: fix casing in a comment. +- openssl-quic: check on Windows that socket conv to int is possible - - setup-win32: delete unused Windows version constant aliases. + Fixes #12861 + Closes #12865 - Reviewed-by: Marcel Raad - Closes #12351 +Daniel Stenberg (5 Feb 2024) -- keylog: disable if unused +- tool_cb_hdr: only parse etag + content-disposition for 2xx - Fully disable keylog code if there is no TLS or QUIC subsystem using it. + ... and ignore them for other response codes. - Closes #12350 + Reported-by: Harry Sintonen + Closes #12866 -- cmake: add `CURL_DISABLE_BINDLOCAL` option +- md4: include strdup.h for the memdup proto - To match similar autotools option. + Reported-by: Erik Schnetter + Fixes #12849 + Closes #12863 - Default is `ON`. +Joel Depooter (5 Feb 2024) - Reviewed-by: Daniel Stenberg - Closes #12345 +- docs: add missing slashes to SChannel client certificate documentation -- url: fix `-Wzero-length-array` with no protocols + When setting the CURLOPT_SSLCERT option to a certificate thumprint, it + is required to have a backslash between the "store location", "store + name" and "thumbprint" tokens. These slashes were present in the + previous documentation, but were missed in the transition to markdown + documentation. - Fixes: - ``` - ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ - -Wc2x-extensions] - 178 | static const struct Curl_handler * const protocols[] = { - | ^ - ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length - -array] - ``` + Closes #12854 - Closes #12344 +Stefan Eissing (5 Feb 2024) -- url: fix builds with `CURL_DISABLE_HTTP` +- HTTP/2: write response directly - Fixes: - ``` - ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' - 456 | Curl_mime_cleanpart(data->state.formp); - | ~~~~~~~~~~~ ^ - ``` + - use the new `Curl_xfer_write_resp()` to write incoming responses + directly to the client + - eliminates `stream->recvbuf` + - memory consumption on parallel transfers minimized - Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 + Closes #12828 - Closes #12343 +Daniel Stenberg (5 Feb 2024) -- http: fix `-Wunused-parameter` with no auth and no proxy +- cookie.md: provide an example sending a fixed cookie - ``` - lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] - bool proxy) - ^ - ``` + Closes #12868 - Reviewed-by: Marcel Raad - Closes #12338 +Lars Kellogg-Stedman (5 Feb 2024) -Daniel Stenberg (16 Nov 2023) +- ALTSVC.md: correct a typo -- TODO: Some TLS options are not offered for HTTPS proxies + The ALPN documentation erroneously referred to a "host number" instead + of a "port number". - Closes #12286 - Closes #12342 + Closes #12852 -- RELEASE-NOTES: synced +Boris Verkhovskiy (5 Feb 2024) -- duphandle: make dupset() not return with pointers to old alloced data +- proxy1.0.md: fix example - As the blob pointers are to be duplicated, the function must not return - mid-function with lingering pointers to the old handle's allocated data, - as that would lead to double-free in OOM situations. + Closes #12856 - Make sure to clear all destination pointers first to avoid this risk. +Chris Webb (5 Feb 2024) - Closes #12337 +- configure: add --disable-docs flag -Viktor Szakats (16 Nov 2023) + Building man pages from curldown sources now requires perl. Add a + --disable-docs flag to configure to enable building and installing + without documentation where perl is not available or man pages are not + required. This is selected automatically (with a warning) when perl is + not found by configure. -- http: fix `-Wunused-variable` compiler warning + Fixes #12832 + Closes #12857 - Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. +Faraz Fallahi (5 Feb 2024) - E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + - `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + - `CURL_DISABLE_NTLM` on non-Windows. +- connect.c: fix typo - ``` - ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl - e] - CURLcode result = CURLE_OK; - ^ - ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus - ed-but-set-variable] - unsigned long *availp; - ^ - ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse - d-but-set-variable] - struct auth *authp; - ^ - ``` + Closes #12858 + +Daniel Stenberg (1 Feb 2024) - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 +- sendf: ignore response body to HEAD - Fixes #12228 - Closes #12335 + and mark the stream for close, but return OK since the response this far + was ok - if headers were received. Partly because this is what curl has + done traditionally. -Jay Satiro (16 Nov 2023) + Test 499 verifies. Updates test 689. -- tool: support bold headers in Windows + Reported-by: Sergey Bronnikov + Bug: https://curl.se/mail/lib-2024-02/0000.html + Closes #12842 - - If virtual terminal processing is enabled in Windows then use ANSI - escape codes Esc[1m and Esc[22m to turn bold on and off. +- ftp: treat a 226 arriving before data as a signal to read data - Suggested-by: Gisle Vanem + For active mode transfers. - Ref: https://github.com/curl/curl/discussions/11770 + Due to some interesting timing, curl can sometimes get the 226 (transfer + complete) over the control channel first, before the data connection + signals readability. If this happens, use that as a signal to check the + data connection. - Closes https://github.com/curl/curl/pull/12321 + Additionally, set the socket filter in listen mode *before* the + PORT/EPRT command is issued, to reduce the risk that the little time gap + could interfere. -Viktor Szakats (15 Nov 2023) + This issue never reproduced for me on Debian and takes several hundred + rounds for me to trigger on my mac. -- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` + Reported-by: Stefan Eissing + Fixes #12823 + Closes #12841 - Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + - `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, - or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while - compiling due to a missing HMAC declaration. +Patrick Monnerat (1 Feb 2024) - The reason is that HMAC is required by `lib/sha256.c` which publishes - `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when - building for libssh2 v1.8.2 (2019-05-25) or older. +- OS400: avoid using awk in the build scripts - Make sure to compile the HMAC bits for a successful build. + Awk is a PASE program and its use may cause a failure depending on the + CCSID of the calling script (IBM bug?). - Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting - them into separate sources isn't practical. + For this reason, revert to an sed-only solution to extract the exported + symbols from the header files. - Fixes: - ``` - [...] - In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc - url_object.dir/Unity/unity_0_c.c:310: - ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s - truct HMAC_params' - 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { - | ^ - ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par - ams' - [...] - ``` + Closes #12826 - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 +Jan Macku (1 Feb 2024) - Fixes #12273 - Closes #12332 +- docs: remove `mk-ca-bundle.1` from `man_MANS` -Daniel Stenberg (15 Nov 2023) + It was accidentally added in https://github.com/curl/curl/pull/12730 -- duphandle: also free 'outcurl->cookies' in error path + Co-authored-by: Lukáš Zaoral + Signed-off-by: Jan Macku - Fixes memory-leak when OOM mid-function + Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b + Closes #12843 - Use plain free instead of safefree, since the entire struct is - freed below. +Daniel Stenberg (1 Feb 2024) - Remove some free calls that is already freed in Curl_freeset() +- RELEASE-NOTES: synced - Closes #12329 + and bump to 8.6.1 for now -Viktor Szakats (15 Nov 2023) +- cmdline-docs/Makefile: avoid using a fixed temp file name -- config-win32: set `HAVE_SNPRINTF` for mingw-w64 + By appending the pid number two different runs at the same time will not + trample over the same file. - It's available in all mingw-w64 releases. We already pre-fill this - detection in CMake. + Reported-by: Jon Rumsey + Fixes #12829 + Closes #12839 - Closes #12325 +- asyn-thread: use wakeup_close to close the read descriptor -- sasl: fix `-Wunused-function` compiler warning + Reported-by: Dan Fandrich + Ref: #12834 + Closes #12836 - In builds with disabled auths. +Stefan Eissing (1 Feb 2024) - ``` - lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus - ed-function] - static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, - ^ - 1 warning generated. - ``` - Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st - ep:3:3822 +- ntml_wb: fix buffer type typo - Reviewed-by: Daniel Stenberg - Closes #12326 + Closes #12825 -- build: picky warning updates +Daniel Stenberg (1 Feb 2024) - - cmake: sync some picky gcc warnings with autotools. - - cmake, autotools: add `-Wold-style-definition` for clang too. - - cmake: more precise version info for old clang options. - - cmake: use `IN LISTS` syntax in `foreach()`. +- tool_operate: do not set CURLOPT_QUICK_EXIT in debug builds - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #12324 + Since it allows (small) memory leaks that interfere with torture tests + and regular memory-leak checks. -Daniel Stenberg (15 Nov 2023) + Reported-by: Dan Fandrich + Fixes #12834 + Closes #12835 -- urldata: move cookielist from UserDefined to UrlState +Boris Verkhovskiy (31 Jan 2024) - 1. Because the value is not strictly set with a setopt option. +- form-string.md: correct the example - 2. Because otherwise when duping a handle when all the set.* fields are - first copied and an error happens (think out of memory mid-function), - the function would easily free the list *before* it was deep-copied, - which could lead to a double-free. + Closes #12822 - Closes #12323 +Version 8.6.0 (31 Jan 2024) -Viktor Szakats (14 Nov 2023) +Daniel Stenberg (31 Jan 2024) -- autotools: avoid passing `LDFLAGS` twice to libcurl +- RELEASE-NOTES: synced - autotools passes `LDFLAGS` automatically linker commands. curl's - `lib/Makefile.am` customizes libcurl linker flags. In that - customization, it added `LDFLAGS` to the custom flags. This resulted in - passing `LDFLAGS` _twice_ to the `libtool` command. + curl 8.6.0 - Most of the time this is benign, but some `LDFLAGS` options can break - the build when passed twice. One such example is passing `.o` files, - e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for - MUSL builds. +- THANKS: new contributors from 8.5.0 - Passing them twice resulted in duplicate symbol errors: - ``` - libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a - arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] - ld.lld-15: error: duplicate symbol: _start - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - [...] - clang: error: linker command failed with exit code 1 (use -v to see invocatio - n) - ``` +Jay Satiro (31 Jan 2024) - This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. - The patch was a works-for-me hack that ended up merged in curl: - https://sourceforge.net/p/curl/bugs/1217/#06ef - With the root cause remaining unclear. +- cd2nroff: use perl 'strict' and 'warnings' - Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring - `-L` first? This would be unusual and suggests a bug in either the - linker or in `libtool`. + - Use strict and warnings pragmas. - The curl build does pass the list of detected libs via its own - `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` - command line, but it's the job of `libtool` to ensure that even - a peculiar linker gets the options in the expected order. Also because - autotools passes `LDFLAGS` last, making it hardly possible to pass - anything after it. + - If open() fails then show the reason. - Perhaps in the 10 years since this issue, this already got a fix - upstream. + - Set STDIN io layer :crlf so that input is properly read on Windows. - This patch deletes `LDFLAGS` from our customized libcurl options, - leaving a single copy of them as passed by autotools automatically. + - When STDIN is used as input, the filename $f is now set to "STDIN". - Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - Closes #12310 + Various error messages in single() use $f for the filename and this way + it is not undefined when STDIN. -- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` + Closes https://github.com/curl/curl/pull/12819 - To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and - curl tool (`CURL_LDFLAGS_BIN`). +Daniel Stenberg (30 Jan 2024) - This makes it possible to build libcurl and curl with a single - invocation with lib- and tool-specific custom linker flags. +- cd2nroff: fix duplicate output issue - Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, - controlling static/shared, incl. requesting a static curl tool (with - `-static-libtool-libs`) while building both shared and static libcurl. + Assisted-by: Jay Satiro + Fixes https://github.com/curl/curl-www/issues/321 + Closes #12818 - curl-for-win uses the above and some more. +- lib: error out on multissl + http3 - These options are already supported in `Makefile.mk`. CMake has built-in - variables for this. + Since the QUIC/h3 code has no knowledge or handling of multissl it might + bring unintended consequences if we allow it. - Closes #12312 + configure, cmake and curl_setup.h all now reject this combination. -Jay Satiro (14 Nov 2023) + Assisted-by: Viktor Szakats + Assisted-by: Gisle Vanem + Ref: #12806 + Closes #12807 -- tool_cb_hdr: add an additional parsing check +Patrick Monnerat (29 Jan 2024) - - Don't dereference the past-the-end element when parsing the server's - Content-disposition header. +- OS400: sync ILE/RPG binding - As 'p' is advanced it can point to the past-the-end element and prior - to this change 'p' could be dereferenced in that case. + Also do not force git CRLF line endings on *.cmd files for OS400. - Technically the past-the-end element is not out of bounds because dynbuf - (which manages the header line) automatically adds a null terminator to - every buffer and that is not included in the buffer length passed to - the header callback. + Closes #12815 - Closes https://github.com/curl/curl/pull/12320 +Viktor Szakats (28 Jan 2024) -Philip Heiduck (14 Nov 2023) +- build: delete/replace 3 more clang warning pragmas -- .cirrus.yml: freebsd 14 + - tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma. - ensure curl works on latest freebsd version + - whitespace formatting in `mprintf.h`, lib518, lib537. - Closes #12053 + - lib518: fix wrong variable in `sizeof()`. -Daniel Stenberg (13 Nov 2023) + - lib518: bump variables to `rlim_t`. + Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469 -- easy: in duphandle, init the cookies for the new handle + - lib518: sync error message with lib537 + Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce + + - lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas + by reworking test code. - ... not the source handle. + Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812 + Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 + Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - Closes #12318 + Reviewed-by: Daniel Stenberg + Closes #12814 -- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set +Richard Levitte (27 Jan 2024) - Previously it would unconditionally use the size, which is set to -1 - when strlen is requested. +- cmake: freshen up docs/INSTALL.cmake - Updated test 544 to verify. + - Turn docs/INSTALL.cmake into a proper markdown file, + docs/INSTALL-CMAKE.md + - Move things around to divide the description into configuration, + building and installing sections + - Mention the more modern cmake options to configure, build and install, + but also retain the older variants as fallbacks - Closes #12317 + Closes #12772 -- RELEASE-NOTES: synced +Viktor Szakats (27 Jan 2024) -- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped +- build: delete/replace clang warning pragmas - Closes #12315 + - delete redundant warning suppressions for `-Wformat-nonliteral`. + This now relies on `CURL_PRINTF()` and it's theoratically possible + that this macro isn't active but the warning is. We're ignoring this + as a corner-case here. -- urldata: move hstslist from 'set' to 'state' + - replace two pragmas with code changes to avoid the warnings. - To make it work properly with curl_easy_duphandle(). This, because - duphandle duplicates the entire 'UserDefined' struct by plain copy while - 'hstslist' is a linked curl_list of file names. This would lead to a - double-free when the second of the two involved easy handles were - closed. + Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 + Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - Closes #12315 + Reviewed-by: Daniel Stenberg + Closes #12812 -- test1900: verify duphandle with HSTS using multiple files +Daniel Stenberg (27 Jan 2024) - Closes #12315 +- RELEASE-NOTES: synced -Goro FUJI (13 Nov 2023) +- http: only act on 101 responses when they are HTTP/1.1 -- http: allow longer HTTP/2 request method names + For 101 responses claiming to be any other protocol, bail out. This + would previously trigger an assert. - - Increase the maximum request method name length from 11 to 23. + Add test 1704 to verify. - For HTTP/1.1 and earlier there's not a specific limit in libcurl for - method length except that it is limited by the initial HTTP request - limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same - and there was no specific limit. + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184 + Closes #12811 - According to Internet Assigned Numbers Authority (IANA) the longest - registered method is UPDATEREDIRECTREF which is 17 characters. +Scarlett McAllister (27 Jan 2024) - Also there are unregistered methods used by some companies that are - longer than 11 characters. +- _VARIABLES.md: add missing 'be' into the sentence - The limit was originally added by 61f52a97 but not used until fc2f1e54. + Closes #12809 - Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml +Stefan Eissing (27 Jan 2024) - Closes https://github.com/curl/curl/pull/12311 +- mqtt, remove remaining use of data->state.buffer -Jay Satiro (12 Nov 2023) + Closes #12799 -- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does +Daniel Stenberg (27 Jan 2024) - - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB - and CURLOPT_PROXY_CAINFO_BLOB docs. +- x509asn1: switch from malloc to dynbuf - All the other _BLOB option docs already have the same explanation. + Closes #12808 - Closes https://github.com/curl/curl/pull/12277 +- x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy -Viktor Szakats (11 Nov 2023) + Closes #12808 -- tidy-up: dedupe Windows system libs in cmake +- x509asn1: reduce malloc in Curl_extract_certinfo - Reviewed-by: Daniel Stenberg - Closes #12307 + Using dynbuf -Junho Choi (11 Nov 2023) + Closes #12808 -- ci: test with latest quiche release (0.19.0) +Jay Satiro (27 Jan 2024) - Closes #12180 +- THANKS: add Alexander Bartel and Brennan Kinney -- quiche: use quiche_conn_peer_transport_params() + They reported and investigated #10259 which was fixed by 7b2d98df. - In recent quiche, transport parameter API is separated - with quiche_conn_peer_transport_params(). - (https://github.com/cloudflare/quiche/pull/1575) - It breaks with bulding with latest(post 0.18.0) quiche. + Ref: https://github.com/curl/curl/issues/10259 - Closes #12180 +Daniel Stenberg (26 Jan 2024) -Daniel Stenberg (11 Nov 2023) +- krb5: add prototype to silence clang warnings on mvsnprintf() -- Makefile: generate the VC 14.20 project files at dist-time + "error: format string is not a string literal" - Follow-up to 28287092cc5a6d6ef8 (#12282) + Follow-up to 09230127589eccc7 which made the warning appear - Closes #12290 + Assisted-by: Viktor Szakats + Closes #12803 -Sam James (11 Nov 2023) +- x509asn1: remove code for WANT_VERIFYHOST -- misc: fix -Walloc-size warnings + No code ever sets this anymore since we dropped gskit - GCC 14 introduces a new -Walloc-size included in -Wextra which gives: + Follow-up to 78d6232f1f326b9ab4d - ``` - src/tool_operate.c: In function ‘add_per_transfer’: - src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo - r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] - 213 | p = calloc(sizeof(struct per_transfer), 1); - | ^ - src/var.c: In function ‘addvariable’: - src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type - struct var’ with size ‘32’ [-Walloc-size] - 361 | p = calloc(sizeof(struct var), 1); - | ^ - ``` + Closes #12804 - The calloc prototype is: - ``` - void *calloc(size_t nmemb, size_t size); - ``` +- socks: reduce the buffer size to 600 (from 8K) - So, just swap the number of members and size arguments to match the - prototype, as we're initialising 1 struct of size `sizeof(struct - ...)`. GCC then sees we're not doing anything wrong. + This is malloc'ed memory and it does not more. Test 742 helps us verify + this. - Closes #12292 + Closes #12789 -Mark Gaiser (11 Nov 2023) +Stefan Eissing (26 Jan 2024) -- IPFS: bugfixes +- file+ftp: use stack buffers instead of data->state.buffer - - Fixed endianness bug in gateway file parsing - - Use IPFS_PATH in tests where IPFS_DATA was used - - Fixed typos from traling -> trailing - - Fixed broken link in IPFS.md + Closes #12789 - Follow-up to 859e88f6533f9e +- vtls: receive max buffer - Reported-by: Michael Kaufmann - Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 - Closes #12305 + - do not only receive one TLS record, but try to fill + the passed buffer + - consider <4K remaning space is "filled". -Daniel Stenberg (11 Nov 2023) + Closes #12801 -- VULN-DISCLOSURE-POLIC: remove broken link to hackerone +Daniel Stenberg (26 Jan 2024) - It should ideally soon not be done from hackerone anyway +- docs: do not start lines/sentences with So, But nor And - Closes #12308 + Closes #12802 -Andrew Kurushin (11 Nov 2023) +- docs: remove spurious ampersands from markdown -- schannel: add CA cache support for files and memory blobs + They were leftovers from the nroff conversion. - - Support CA bundle and blob caching. + Follow-up to eefcc1bda4bccd800f5a5 - Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. + Closes #12800 - Closes https://github.com/curl/curl/pull/12261 +Patrick Monnerat (26 Jan 2024) -Daniel Stenberg (10 Nov 2023) +- sasl: make login option string override http auth -- RELEASE-NOTES: synced + - Use http authentication mechanisms as a default, not a preset. -Charlie C (10 Nov 2023) + Consider http authentication options which are mapped to SASL options as + a default (overriding the hardcoded default mask for the protocol) that + is ignored if a login option string is given. -- cmake: option to disable install & drop `curlu` target when unused + Prior to this change, if some HTTP auth options were given, sasl mapped + http authentication options to sasl ones but merged them with the login + options. - This patch makes the following changes: - - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. - - Removes the target `curlu` when the option `BUILD_TESTING` is set to - `OFF` - to prevent it from being loaded in Visual Studio. + That caused problems with the cli tool that sets the http login option + CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag + maps to more than one sasl mechanisms and the latter cannot be cleared + individually by the login options string. - Closes #12287 + New test 992 checks this. -Kai Pastor (10 Nov 2023) + Fixes https://github.com/curl/curl/issues/10259 + Closes https://github.com/curl/curl/pull/12790 -- cmake: fix multiple include of CURL package +Stefan Eissing (26 Jan 2024) - Fixes errors on second `find_package(CURL)`. This is a frequent case - with transitive dependencies: - ``` - CMake Error at ...: - add_library cannot create ALIAS target "CURL::libcurl" because another - target with the same name already exists. - ``` +- socks: use own buffer instead of data->state.buffer - Test to reproduce: - ```cmake - cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher + Closes #12788 - project(curl) +Daniel Stenberg (26 Jan 2024) - set(CURL_DIR "example/lib/cmake/CURL/") - find_package(CURL CONFIG REQUIRED) - find_package(CURL CONFIG REQUIRED) # fails +- socks: fix generic output string to say SOCKS instead of SOCKS4 - add_executable(main main.c) - target_link_libraries(main CURL::libcurl) - ``` + ... since it was also logged for SOCKS5. - Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes - Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html - Ref: #12300 - Assisted-by: Harry Mallon - Closes #11913 + Closes #12797 -Viktor Szakats (8 Nov 2023) +- test742: test SOCKS5 with max length user, password and hostname -- tidy-up: use `OPENSSL_VERSION_NUMBER` + Adjusted the socksd server accordingly to allow for configuring that + long user name and password. - Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. - Before this patch some places used `OPENSSL_VERSION_MAJOR`. + Closes #12797 - Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't - define any version number in these implementations: BoringSSL, AWS-LC, - LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to - `opensslv.h`. This wasn't causing a deeper problem because the code is - looking for v3, which is only provided by OpenSSL/quictls as of now. +Stefan Eissing (25 Jan 2024) - According to https://github.com/openssl/openssl/issues/17517, the macro - `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. +- ssh: use stack scratch buffer for seeks - Reviewed-by: Marcel Raad - Closes #12298 + - instead of data->state.buffer -Daniel Stenberg (8 Nov 2023) + Closes #12794 -- resolve.d: drop a multi use-sentence +Daniel Stenberg (25 Jan 2024) - Since the `multi:` keyword adds that message. +- krb5: access the response buffer correctly - Reported-by: 積丹尼 Dan Jacobson - Fixes https://github.com/curl/curl/discussions/12294 - Closes #12295 + As the pingpong code no longer uses the download buffer. -- content_encoding: make Curl_all_content_encodings allocless + Folllow-up to c2d973627bab12ab + Pointed-out-by: Stefan Eissing + Closes #12796 - - Fixes a memory leak pointed out by Coverity - - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? - id=63947 - - Avoids unncessary allocations +Stefan Eissing (25 Jan 2024) - Follow-up ad051e1cbec68b2456a22661b +- mqtt: use stack scratch buffer for recv+publish - Closes #12289 + - instead of data->state.buffer -Michael Kaufmann (7 Nov 2023) + Closes #12792 -- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 +- telnet, use stack scratch buffer for do - Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), - avoid it and use "http/1.1" instead. + - instead of data->state.buffer - This reverts commit df856cb5c9 (#10183). + Closes #12793 - Fixes #12259 - Closes #12285 +- http, use stack scratch buffer -Daniel Stenberg (7 Nov 2023) + - instead of data->state.buffer -- Makefile.am: drop vc10, vc11 and vc12 projects from dist + Closes #12791 - They are end of life products. Support for generating them remain in the - repo for a while but this change drops them from distribution. +- ntlm_wb: do not use data->state.buf any longer - Closes #12288 + Closes #12787 -David Suter (7 Nov 2023) +- gitignore: the generated `libcurl-symbols.md` -- projects: add VC14.20 project files + Closes #12795 - Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. - OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why - this is missing. Updated the templates to produce a VC14.20 project. - Project opens in Visual Studio 2019 as expected. +Daniel Stenberg (25 Jan 2024) - Closes #12282 +- tool: fix the listhelp generation command -Daniel Stenberg (7 Nov 2023) + The previous command line to generate the tool_listhelp.c source file + broke with 2494b8dd5175cee7. -- curl: move IPFS code into src/tool_ipfs.[ch] + Make 'make listhelp' invoked in src/ generate it. Also update the + comment in the file to mention the right procedure. - - convert ensure_trailing into ensure_trailing_slash - - strdup the URL string to own it proper - - use shorter variable names - - combine some expressions - - simplify error handling in ipfs_gateway() - - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached - - ipfs-gateway.d polish and simplification - - shorten ipfs error message + make them "synthetic" + Closes #12786 - Closes #12281 +- http: check for "Host:" case insensitively -Viktor Szakats (6 Nov 2023) + When checking if the user wants to replace the header, the check should + be case insensitive. -- build: delete support bits for obsolete Windows compilers + Adding test 461 to verify - - Pelles C: Unclear status, failed to obtain a fresh copy a few months - ago. Possible website is HTTP-only. ~10 years ago I left this compiler - dealing with crashes and other issues with no response on the forum - for years. It has seen some activity in curl back in 2021. - - LCC: Last stable release in September 2002. - - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned - in 2006. - - Borland C++: We dropped Borland C++ support in 2018. - - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 - (or possibly 2008) as a minimum. + Found-by: Dan Fandrich + Ref: #12782 + Closes #12784 - Closes #12222 +Tatsuhiro Tsujikawa (25 Jan 2024) -- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` +- configure: add libngtcp2_crypto_boringssl detection - We use `stdint.h` unconditionally in all places except one. These uses - are imposed by external dependencies / features. nghttp2, quic, wolfSSL - and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that - any of these features make curl require a C99 compiler. (In case of - MSVC, this means Visual Studio 2010 or newer.) + If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested, + try to detect libngtcp2_crypto_boringssl. - This patch changes the single use of `stdint.h` guarded by - `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using - `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used - anywhere else, allowing to delete this feature check as well. + Reported-by: ウさん + Fixes #12724 + Closes #12769 - Closes #12275 +Daniel Stenberg (25 Jan 2024) -Daniel Stenberg (6 Nov 2023) +- http: remove comment reference to a removed solution -- tool_operate: do not mix memory models + Follow-up to 58974d25d - Make sure 'inputpath' only points to memory allocated by libcurl so that - curl_free works correctly. + Closes #12785 - Pointed out by Coverity +Stefan Eissing (25 Jan 2024) - Follow-up to 859e88f6533f9e1f890 +- pytest: Scorecard tracking CPU and RSS - Closes #12280 + Closes #12765 -Stefan Eissing (6 Nov 2023) +Graham Campbell (25 Jan 2024) -- lib: client writer, part 2, accounting + logging +- GHA: bump ngtcp2, gnutls, mod_h2, quiche - This PR has these changes: + - ngtcp2 to v1.2.0 + - gnutls to 3.8.3 + - mod_h2 to 2.0.26 + - quiche to 0.20.0 - Renaming of unencode_* to cwriter, e.g. client writers - - documentation of sendf.h functions - - move max decode stack checks back to content_encoding.c - - define writer phase which was used as order before - - introduce phases for monitoring inbetween decode phases - - offering default implementations for init/write/close + Closes #12778 + Closes #12779 + Closes #12780 + Closes #12781 - Add type paramter to client writer's do_write() - - always pass all writes through the writer stack - - writers who only care about BODY data will pass other writes unchanged +Daniel Stenberg (25 Jan 2024) - add RAW and PROTOCOL client writers - - RAW used for Curl_debug() logging of CURLINFO_DATA_IN - - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and - Curl_pgrsSetDownloadCounter() - - remove all updates of data->req.bytecount and calls to - Curl_pgrsSetDownloadCounter() and Curl_debug() from other code - - adjust test457 expected output to no longer see the excess write +- ftpserver.pl: send 213 SIZE response without spurious newline - Closes #12184 +- pingpong: stop using the download buffer -Daniel Stenberg (6 Nov 2023) + The pingpong logic now uses its own dynbuf for receiving command + response data. -- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw + When the "final" response header for a commanad has been received, that + final line is left first in the recvbuf for the protocols to parse at + will. If there is additional data behind the final response line, the + 'overflow' counter is indicate how many bytes. - Closes #12278 + Closes #12757 -Viktor Szakats (6 Nov 2023) +- gen.pl: remove bold from .IP used for ## -- rand: fix build error with autotools + LibreSSL + Reported-by: Viktor Szakats + Fixes #12776 + Closes #12777 - autotools unexpectedly detects `arc4random` because it is also looking - into dependency libs. One dependency, LibreSSL, happens to publish an - `arc4random` function (via its shared lib before v3.7, also via static - lib as of v3.8.2). When trying to use this function in `lib/rand.c`, - its protoype is missing. To fix that, curl included a prototype, but - that used a C99 type without including `stdint.h`, causing: +Viktor Szakats (24 Jan 2024) - ``` - ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' - 37 | uint32_t arc4random(void); - | ^ - 1 error generated. - ``` +- cmake: rework options to enable curl and libcurl docs - This patch improves this by dropping the local prototype and instead - limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide - their own random source anyway. + Rework CMake options for building/using curl tool and libcurl manuals. - The better fix would be to teach autotools to not link dependency libs - while detecting `arc4random`. + - rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning: + to build man page and built-in manual for curl tool. - LibreSSL publishing a non-namespaced `arc4random` tracked here: - https://github.com/libressl/portable/issues/928 + - rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning: + to build man pages for libcurl. - Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 + - `BUILD_LIBCURL_DOCS` now works without having to enable + `ENABLE_CURL_MANUAL` too. - Reviewed-by: Daniel Stenberg - Fixes #12257 - Closes #12274 + - drop support for existing CMake-level `USE_MANUAL` option to avoid + confusion. (It used to work with the effect of current + `ENABLE_CURL_MANUAL`, but only by accident.) -Daniel Stenberg (5 Nov 2023) + Assisted-by: Richard Levitte + Ref: #12771 + Closes #12773 -- RELEASE-NOTES: synced +Daniel Stenberg (24 Jan 2024) -- strdup: do Curl_strndup without strncpy +- urlapi: remove assert - To avoid (false positive) gcc-13 compiler warnings. + This assert triggers wrongly when CURLU_GUESS_SCHEME and + CURLU_NO_AUTHORITY are both set and the URL is a single path. - Follow-up to 4855debd8a2c1cb + I think this assert has played out its role. It was introduced in a + rather big refactor. - Assisted-by: Jay Satiro - Reported-by: Viktor Szakats - Fixes #12258 + Follow-up to 4cfa5bcc9a -Enno Boland (5 Nov 2023) + Reported-by: promptfuzz_ on hackerone + Closes #12775 -- HTTP: fix empty-body warning +Patrick Monnerat (24 Jan 2024) - This change fixes a compiler warning with gcc-12.2.0 when - `-DCURL_DISABLE_BEARER_AUTH=ON` is used. +- tests: avoid int/size_t conversion size/sign warnings - /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': - /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp - ty body in an 'else' statement [-Wempty-body] - 1147 | ; - | ^ + Closes #12768 - Closes #12262 +Daniel Stenberg (24 Jan 2024) -Daniel Stenberg (5 Nov 2023) +- GHA: add a job scanning for "bad words" in markdown -- openssl: identify the "quictls" backend correctly + This means words, phrases or things we have decided not to use - words that + are spelled right according to the dictionary but we want to avoid. In the + name of consistency and better documentation. - Since vanilla OpenSSL does not support the QUIC API I think it helps - users to identify the correct OpenSSL fork in version output. The best - (crude) way to do that right now seems to be to check if ngtcp2 support - is enabled. + Closes #12764 - Closes #12270 +Viktor Szakats (23 Jan 2024) + +- cmake: speed up curldown processing, enable by default -Mark Gaiser (5 Nov 2023) + - cmake: enable `BUILD_DOCS` by default (this controls converting and + installing `.3` files from `.md` sources) -- curl: improved IPFS and IPNS URL support + - cmake: speed up generating `.3` files by using a single command per + directory, instead of a single command per file. This reduces external + commands by about a thousand. (There remains some CMake logic kicking + in resulting in 500 -one per file- external `-E touch_nocreate` calls.) - Previously just ipfs:// and ipns:// was supported, which is - too strict for some usecases. + - cd2nroff: add ability to process multiple input files. - This patch allows paths and query arguments to be used too. - Making this work according to normal http semantics: + - cd2nroff: add `-k` option to use the source filename to form the + output filename. (instead of the default in-file `Title:` line.) - ipfs:///foo/bar?key=val - ipns:///foo/bar?key=val + Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c + Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753 + Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 - The gateway url support is changed. - It now only supports gateways in the form of: + Closes #12762 - http:///foo/bar - http:// +Richard Levitte (23 Jan 2024) - Query arguments here are explicitly not allowed and trigger an intended - malformed url error. +- docs: install curl.1 with cmake as well - There also was a crash when IPFS_PATH was set with a non trailing - forward slash. This has been fixed. + Closes #12759 - Lastly, a load of test cases have been added to verify the above. +Daniel Stenberg (23 Jan 2024) - Reported-by: Steven Allen - Fixes #12148 - Closes #12152 +- osslq: remove the TLS library from the version output -Harry Mallon (5 Nov 2023) + Since we only support using a single TLS library at any one time, we + know that the TLS library for QUIC is the same that is also shown for + regular TLS. -- docs: KNOWN_BUGS cleanup + Fixes #12763 + Reported-by: Viktor Szakats + Closes #12767 - * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. - Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 +Stefan Eissing (23 Jan 2024) - * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. - Fixed in #11806 +- CI: remove unnecessary OpenSSL 3 option `enable-tls1_3` - * Remove mention of aws-sigv4 query empty value problems + .. and switch OpenSSL 3 libdir from lib64 to lib for consistency. - * Remove mention of aws-sigv4 missing amz-content-sha256 - Fixed in #9995 + Closes https://github.com/curl/curl/pull/12758 -- http_aws_sigv4: canonicalise valueless query params +- GHA: bump nghttp2 version to v1.59.0 - Fixes #8107 - Closes #12244 + - Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version. -Michael Kaufmann (4 Nov 2023) + Closes https://github.com/curl/curl/pull/12766 -- docs: preserve the modification date when copying the prebuilt man page +Daniel Stenberg (23 Jan 2024) - The previously built man page "curl.1" must be copied with the original - modification date, otherwise the man page is never updated. +- RELEASE-NOTES: synced - This fixes a bug that has been introduced with commit 2568441cab. +- docs/cmdline: change to .md for cmdline docs - Reviewed-by: Dan Fandrich - Reviewed-by: Daniel Stenberg + - switch all invidual files documenting command line options into .md, + as the documentation is now markdown-looking. - Closes #12199 + - made the parser treat 4-space indents as quotes -Daniel Stenberg (4 Nov 2023) + - switch to building the curl.1 manpage using the "mainpage.idx" file, + which lists the files to include to generate it, instead of using the + previous page-footer/headers. Also, those files are now also .md + ones, using the same format. I gave them underscore prefixes to make + them sort separately: + _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md, + _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md, + _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md, + _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md -- docs: remove bold from some man page SYNOPSIS sections + - updated test cases accordingly - In the name of consistency + Closes #12751 - Closes #12267 +dependabot[bot] (23 Jan 2024) -- openssl: two multi pointer checks should probably rather be asserts +- CI: bump actions/cache from 3 to 4 - ... so add the asserts now and consider removing the dynamic checks in a - future. + Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. + - [Release notes](https://github.com/actions/cache/releases) + - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) + - [Commits](https://github.com/actions/cache/compare/v3...v4) - Ref: #12261 - Closes #12264 + --- + updated-dependencies: + - dependency-name: actions/cache + dependency-type: direct:production + update-type: version-update:semver-major + ... -boilingoden (4 Nov 2023) + Signed-off-by: dependabot[bot] + Closes #12756 -- docs: add supported version for the json write-out +Daniel Stenberg (23 Jan 2024) - xref: https://curl.se/changes.html#7_70_0 +- openssl: when verifystatus fails, remove session id from cache - Closes #12266 + To prevent that it gets used in a subsequent transfer that skips the + verifystatus check since that check can't be done when the session id is + reused. -Viktor Szakats (3 Nov 2023) + Reported-by: Hiroki Kurosawa + Closes #12760 -- appveyor: make VS2008-built curl tool runnable +Viktor Szakats (23 Jan 2024) - By linking the CRT statically. This avoids the error about missing - runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`. +- cmake: add option to disable building docs - Closes #12263 +Richard Levitte (23 Jan 2024) -Stefan Eissing (3 Nov 2023) +- cmake: use curldown to build man pages -- url: proxy ssl connection reuse fix + This throws away the previous HTML and PDF producers, to mimic what + Makefile.am does as faithfully as possible. - - tunnel https proxy used for http: transfers does - no check if proxy-ssl configuration matches - - test cases added, test_10_12 fails on 8.4.0 + Closes #12753 - Closes #12255 +Daniel Stenberg (23 Jan 2024) -Jay Satiro (3 Nov 2023) +- mksymbolsmanpage.pl: provide references to where the symbol is used -- curl_sspi: support more revocation error names in error messages +- docs: introduce "curldown" for libcurl man page format - - Add these revocation errors to sspi error list: - CRYPT_E_NO_REVOCATION_DLL, CRYPT_E_NO_REVOCATION_CHECK, - CRYPT_E_REVOCATION_OFFLINE and CRYPT_E_NOT_IN_REVOCATION_DATABASE. + curldown is this new file format for libcurl man pages. It is markdown + inspired with differences: - Prior to this change those error codes were not matched to their macro - name and instead shown as "unknown error". + - Each file has a set of leading headers with meta-data + - Supports a small subset of markdown + - Uses .md file extensions for editors/IDE/GitHub to treat them nicely + - Generates man pages very similar to the previous ones + - Generates man pages that still convert nicely to HTML on the website + - Detects and highlights mentions of curl symbols automatically (when + their man page section is specified) - Before: + tools: - schannel: next InitializeSecurityContext failed: - Unknown error (0x80092013) - The revocation function was - unable to check revocation because the revocation server was offline. + - cd2nroff: converts from curldown to nroff man page + - nroff2cd: convert an (old) nroff man page to curldown + - cdall: convert many nroff pages to curldown versions + - cd2cd: verifies and updates a curldown to latest curldown - After: + This setup generates .3 versions of all the curldown versions at build time. - schannel: next InitializeSecurityContext failed: - CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was - unable to check revocation because the revocation server was offline. + CI: - Bug: https://github.com/curl/curl/issues/12239 - Reported-by: Niracler Li + Since the documentation is now technically markdown in the eyes of many + things, the CI runs many more tests and checks on this documentation, + including proselint, link checkers and tests that make sure we capitalize the + first letter after a period... - Closes https://github.com/curl/curl/pull/12241 + Closes #12730 -- strdup: don't allow Curl_strndup to read past a null terminator +Viktor Szakats (22 Jan 2024) - - Use malloc + strncpy instead of Curl_memdup to dupe the string before - null terminating it. +- libssh2: use `libssh2_session_callback_set2()` with v1.11.1 - Prior to this change if Curl_strndup was passed a length longer than - the allocated string then it could copy out of bounds. + To avoid a local hack to pass function pointers and to avoid + deprecation warnings when building with libssh2 v1.11.1 or newer: + ``` + lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is depreca + ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- + declarations] + lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is depreca + ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- + declarations] + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/2072082 + 1100#step:3:4982 - This change is for posterity. Curl_strndup was added in the parent - commit and currently none of the calls to it pass a length that would - cause it to read past the allocated length of the input. + Ref: https://github.com/libssh2/libssh2/pull/1285 + Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db + 3cf1d4b0b4 + Reviewed-by: Daniel Stenberg + Closes #12754 - Follow-up to d3b3ba35. +Daniel Stenberg (22 Jan 2024) - Closes https://github.com/curl/curl/pull/12254 +- transfer: make the select_bits_paused condition check both directions -Daniel Stenberg (2 Nov 2023) + If there is activity in a direction that is not paused, return false. -- lib: add and use Curl_strndup() + Reported-by: Sergey Bronnikov + Bug: https://curl.se/mail/lib-2024-01/0049.html + Closes #12740 - The Curl_strndup() function is similar to memdup(), but copies 'n' bytes - then adds a terminating null byte ('\0'). +Stefan Eissing (22 Jan 2024) - Closes #12251 +- http3: initial support for OpenSSL 3.2 QUIC stack -- CURPOST_POSTFIELDS.3: add CURLOPT_COPYPOSTFIELDS in SEE ALSO + - HTTP/3 for curl using OpenSSL's own QUIC stack together + with nghttp3 + - configure with `--with-openssl-quic` to enable curl to + build this. This requires the nghttp3 library + - implementation with the following restrictions: + * macOS has to use an unconnected UDP socket due to an + issue in OpenSSL's datagram implementation + See https://github.com/openssl/openssl/issues/23251 + This makes connections to non-reponsive servers hang. + * GET requests will send the indicator that they have + no body in a separate QUIC packet. This may result + in processing delays or Transfer-Encodings on proxied + requests + * uploads that encounter blocks will use 100% cpu as + detection of these flow control issue is not working + (we have not figured out to pry that from OpenSSL). -Stefan Eissing (2 Nov 2023) + Closes #12734 -- pytest: use lower count in repeat tests +Viktor Szakats (22 Jan 2024) - - lower large iteration counts in some tests somewhat for - the same coverage with less duration +- cmake: fix `ENABLE_MANUAL` option - Closes #12248 + Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`. -Daniel Stenberg (2 Nov 2023) + Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the + option designed to enable building and using the built-in curl manual. + (`USE_MANUAL=ON` option worked for this instead, by accident). -- RELEASE-NOTES: synced + Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409 + Closes #12749 -- docs: clarify that curl passes on input unfiltered +Mohammadreza Hendiani (19 Jan 2024) - ... for several options. +- TODO: update broken link to ratelimit-headers draft - Reported-by: Ophir Lojkine + Closes #12741 - Closes #12249 +Daniel Stenberg (19 Jan 2024) -- urlapi: when URL encoding the fragment, pass in the right length +- cmake: when USE_MANUAL=YES, build the curl.1 man page - A benign bug because it would only add an extra null terminator. + Fixes KNOWN_BUG 15.4 - Made lib1560 get a test that runs this code. + Closes #12742 - Closes #12250 +- cmdline-opts/write-out.d: remove spurious double quotes -Stefan Eissing (2 Nov 2023) +Stefan Eissing (19 Jan 2024) -- vtls: late clone of connection ssl config +- rtsp: Convert assertion into debug log - - perform connection cache matching against `data->set.ssl.primary` - and proxy counterpart - - fully clone connection ssl config only when connection is used + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 - Closes #12237 + - write excess bytes to the client where the standard excess bytes + checks will report any wrongness and fail the transfer -- msh3: error when built with CURL_DISABLE_SOCKETPAIR set + Fixes #12738 + Closes #12739 - Reported-by: Gisle Vanem - Closes #12252 - Fixes #12213 +Daniel Stenberg (19 Jan 2024) -Daniel Stenberg (2 Nov 2023) +- headers: remove assert from Curl_headers_push -- hsts: skip single-dot hostname + The fuzzer managed to reach the function without a terminating CR or LF + so let's handle it normally. While there, remove the goto. - Reported-by: Maksymilian Arciemowicz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839 - Closes #12247 + Closes #12721 -- vtls: fix build without proxy +- curl_easy_getinfo.3: remove the wrong time value count - Follow-up to bf0e278a3c54bc7fee7360da17c + It said "six" time values but they are eight by now. Remove the mention + of the amount. - closes #12243 + Closes #12727 -- docs/example/keepalive.c: show TCP keep-alive options +Viktor Szakats (18 Jan 2024) - Closes #12242 +- mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls` -- lib1560: verify appending blank URL encoded query string + - Silence warning in mbedTLS v3.5.1 public headers: + ``` + ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: r + edundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls] + ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: pre + vious declaration of 'psa_set_key_domain_parameters' was here + ``` + Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae + 785a513e157c + Ref: https://github.com/libssh2/libssh2/pull/1226 -- urlapi: skip appending NULL pointer query + - Fix compiler warnings seen with gcc 9.2.0 + cmake unity: + ``` + ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read': + ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull + -dereference] + 189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &res + ult); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~ + ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write': + ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull + -dereference] + 168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, & + result); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~ + ``` - Reported-by: kirbyn17 on hackerone + - delete stray `#else`. - Closes #12240 + Closes #12720 -- lib1560: verify setting host to "" with and without URL encode +Daniel Stenberg (17 Jan 2024) -- urlapi: avoid null deref if setting blank host to url encode +- docs: cleanup nroff format use - Reported-by: kirbyn17 on hackerone + - remove use of .BI for code snippet + - stop using .br, just do a blank line + - remove use of .PP + - remove use for .sp + - remove backslash in .IP + - use .IP instead of .TP - Closes #12240 + Closes #12731 -- dynbuf: assert for NULL pointer inputs +Stefan Eissing (17 Jan 2024) - Help us catch more mistakes. +- test2307: fix expected failure code after ws refactoring - Closes #12238 + Fixes #12722 + Closes #12728 -- HTTP3: ngtcp2 builds are no longer experimental +Jay Satiro (17 Jan 2024) - The other HTTP/3 backends are still experimental. +- cf-socket: show errno in tcpkeepalive error messages - Closes #12235 + - If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set + then show the errno in the verbose error messages. -Stefan Eissing (31 Oct 2023) + Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652 -- vtls: cleanup SSL config management + Closes https://github.com/curl/curl/pull/12726 - - remove `Curl_ssl_get_config()`, no longer needed +- tool_getparam: stop supporting `@filename` style for --cookie - Closes #12204 + The `@filename` style was never documented for --cookie + but prior to this change curl would accept it anyway and always treat a + @ prefixed string as a filename. -Daniel Stenberg (31 Oct 2023) + That's a problem if the string also contains a = sign because then it is + documented to be interpreted as a cookie string and not a filename. -- libcurl-thread.3: simplify the TLS section + Example: - All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August - 2016. + `--cookie @foo=bar` - Closes #12233 + Before: Interpreted as load cookies from filename foo=bar. -- configure: better --disable-http + After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`). - - disable HTTPS-proxy as well, since it can't work without HTTP + Other curl options with a data/filename option-value use the `@filename` + to distinguish filenames which is probably how this happened. The + --cookie option has never been documented that way. - - curl_setup: when HTTP is disabled, also disable all features that are - HTTP-only + Ref: https://curl.se/docs/manpage.html#-b - - version: HTTPS-proxy only exists if HTTP support exists + Closes https://github.com/curl/curl/pull/12645 - Closes #12223 +Stefan Eissing (16 Jan 2024) -- http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine +- websockets: refactor decode chain - Finding a 'Content-Range:' in the response changed the handling. + - use client writer stack for decoding frames + - move websocket protocol handler to ws.c - Add test case 1475 to verify -C - with 416 and Content-Range: header, - which is almost exactly like test 194 which instead uses a fixed -C - offset. Adjusted test 194 to also be considered fine. + Closes #12713 - Fixes #10521 - Reported-by: Smackd0wn - Fixes #12174 - Reported-by: Anubhav Rai - Closes #12176 +- websockets: check for negative payload lengths -Stefan Eissing (30 Oct 2023) + - in en- and decoding, check the websocket frame payload lengths for + negative values (from curl_off_t) and error the operation in that case + - add test 2307 to verify -- GHA: fix checkout of quictls repository to use correct branch name + Closes #12707 - Follow-up to c868b0e30f10cd0ac7 +Daniel Stenberg (16 Jan 2024) - Closes #12232 +- docs: mention env vars not used by schannel -Daniel Stenberg (30 Oct 2023) + Ref: #12704 -- docs/example/localport.c: show off CURLOPT_LOCALPORT + Co-authored-by: Jay Satiro - Closes #12230 + Closes #12711 -- docs/examples/interface.c: show CURLOPT_INTERFACE use +- tool_operate: make --remove-on-error only remove "real" files - Although super simple. + Reported-by: Harry Sintonen + Assisted-by: Dan Fandrich - Closes #12229 + Closes #12710 -Viktor Szakats (30 Oct 2023) +Jay Wu (16 Jan 2024) -- build: fix compiler warning with auths disabled +- url: don't set default CA paths for Secure Transport backend - ``` - ./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' - [-Wunused-function] - static int is_valid_auth_separator(char ch) - ^ - 5 warnings generated. - ``` + As the default for this backend is the native CA store. - Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 + Closes #12704 - Closes #12227 +Lin Sun (16 Jan 2024) -- build: require Windows XP or newer +- asyn-ares: with modern c-ares, use its default timeout - After this patch we assume availability of `getaddrinfo` and - `freeaddrinfo`, first introduced in Windows XP. Meaning curl - now requires building for Windows XP as a minimum. + Closes #12703 - TODO: assume these also in autotools. +Daniel Stenberg (15 Jan 2024) - Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 - Closes #12225 +- tool_operate: stop setting the file comment on Amiga -- appveyor: bump one job to OpenSSL 3.1 (was 1.1.1) + - the URL is capped at 80 cols, which ruins it if longer + - it does not strip off URL credentials + - it is done unconditonally, not on --xattr + - we don't have Amiga in the CI which makes fixing it blindly fragile - Use 3.1 with the modern runner image. + Someone who builds and tests on Amiga can add it back correctly in a + future if there is a desire. - We still use 1.1.1 in 8 jobs. + Reported-by: Harry Sintonen + Closes #12709 - 1.1.1 is EOL since 2023-09-11: - https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/ +Stefan Eissing (15 Jan 2024) - Also: - - add missing SSL-backend to job descriptions. - - tidy up CPU in job descriptions. +- rtsp: deal with borked server responses - Closes #12226 + - enforce a response body length of 0, if the + response has no Content-lenght. This is according + to the RTSP spec. + - excess bytes in a response body are forwarded to + the client writers which will report and fail the + transfer -Daniel Stenberg (30 Oct 2023) + Follow-up to d7b6ce6 + Fixes #12701 + Closes #12706 -- RELEASE-NOTES: synced +Daniel Stenberg (14 Jan 2024) -- GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions +- version: show only the libpsl version, not its dependencies - ngtcp2 1.0.1 - nghttp3 1.0.0 - nghttp2 1.58.0 - quictls 3.1.4+quic + The libpsl version output otherwise also includes version number for its + dependencies, like IDN lib, but since libcurl does not use libpsl's IDN + functionality those components are not important. - also sync HTTP3.md with these changes + Ref: https://github.com/curl/curl-for-win/issues/63 + Closes #12700 - Closes #12132 +Brad Harder (14 Jan 2024) -Kareem (29 Oct 2023) +- curl.h: CURLOPT_DNS_SERVERS is only available with c-ares -- wolfssl: add default case for wolfssl_connect_step1 switch + Closes #12695 - Closes #12218 +Daniel Stenberg (14 Jan 2024) -Jay Satiro (29 Oct 2023) +- cmdline-opts/gen.pl: error on initital blank line -- curl_setup: disallow Windows IPv6 builds missing getaddrinfo + After the "---" separator, there should be no blank line and this script + now errors out if one is detected. - - On Windows if IPv6 is enabled but getaddrinfo is missing then #error - the build. + Ref: #12696 + Closes #12698 - curl can be built with IPv6 support (ENABLE_IPV6) but without the - ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On - Windows this is highly unlikely and should be considered a bad build - configuration. +- cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper - Such a bad configuration has already given us a bug that was hard to - diagnose. See #12134 and #12136 for discussion. + Follow-up to 693cd1679361828a which was incomplete - Ref: https://github.com/curl/curl/issues/12134 - Ref: https://github.com/curl/curl/pull/12136 + Ref #12680 + Closes #12697 - Closes https://github.com/curl/curl/pull/12221 +- curl_multi_fdset.3: remove mention of null pointer support -Nico Rieck (29 Oct 2023) + ... since this funtion has not supported null pointer fd_set arguments since + at least 2006. (That's when I stopped my git blame journey) -- openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs + Fixes #12691 + Reported-by: sfan5 on github + Closes #12692 - - If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA - "CA" store after importing from root CA "ROOT" store. +Mark Huang (14 Jan 2024) - This change allows curl to work in situations where a server does not - send all intermediate certs and they are present in the "CA" store (the - store with intermediate CAs). This is already allowed by the Schannel - backend. +- docs/cmdline: remove unnecessary line breaks - Also this change makes partial chain verification possible for those - certs since we allow partial chain verification by default for OpenSSL - (unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel - backend. + Closes #12696 - Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs. +Daniel Stenberg (14 Jan 2024) - Fixes https://github.com/curl/curl/issues/12155 - Closes https://github.com/curl/curl/pull/12185 +- transfer: remove warning: Value stored to 'blen' is never read -Viktor Szakats (28 Oct 2023) + Detected by scan-build -- Makefile.mk: fix `-rtmp` option for non-Windows [ci skip] + Follow-up from 1cd2f0072f -Daniel Stenberg (28 Oct 2023) + Closes #12693 -- asyn-ares: handle no connection in the addrinfo callback +Stefan Eissing (13 Jan 2024) - To avoid crashing. +- lib: replace readwrite with write_resp - Follow-up from 56a4db2 - Closes #12219 + This clarifies the handling of server responses by folding the code for + the complicated protocols into their protocol handlers. This concerns + mainly HTTP and its bastard sibling RTSP. -Jay Satiro (28 Oct 2023) + The terms "read" and "write" are often used without clear context if + they refer to the connect or the client/application side of a + transfer. This PR uses "read/write" for operations on the client side + and "send/receive" for the connection, e.g. server side. If this is + considered useful, we can revisit renaming of further methods in another + PR. -- hostip6: fix DEBUG_ADDRINFO builds + Curl's protocol handler `readwrite()` method been changed: - - Removed unused and incorrect parameter from dump_addrinfo(). + ```diff + - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, + - const char *buf, size_t blen, + - size_t *pconsumed, bool *readmore); + + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t ble + n, + + bool is_eos, bool *done); + ``` - Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442 - Reported-by: Gisle Vanem + The name was changed to clarify that this writes reponse data to the + client side. The parameter changes are: - Closes https://github.com/curl/curl/pull/12212 + * `conn` removed as it always operates on `data->conn` + * `pconsumed` removed as the method needs to handle all data on success + * `readmore` removed as no longer necessary + * `is_eos` as indicator that this is the last call for the transfer + response (end-of-stream). + * `done` TRUE on return iff the transfer response is to be treated as + finished -Viktor Szakats (28 Oct 2023) + This change affects many files only because of updated comments in + handlers that provide no implementation. The real change is that the + HTTP protocol handlers now provide an implementation. -- Makefile.mk: restore `_mingw.h` for default `_WIN32_WINNT` + The HTTP protocol handlers `write_resp()` implementation will get passed + **all** raw data of a server response for the transfer. The HTTP/1.x + formatted status and headers, as well as the undecoded response + body. `Curl_http_write_resp_hds()` is used internally to parse the + response headers and pass them on. This method is public as the RTSP + protocol handler also uses it. - In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support. - Turns out `_mingw.h` had the side-effect of setting a default - `_WIN32_WINNT` value expected by `lib/config-win32.h` to enable - `getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused - disabling support for this unless specifying the value manually. + HTTP/1.1 "chunked" transport encoding is now part of the general + *content encoding* writer stack, just like other encodings. A new flag + `CLIENTWRITE_EOS` was added for the last client write. This allows + writers to verify that they are in a valid end state. The chunked + decoder will check if it indeed has seen the last chunk. - Restore this header and update its comment to tell why we continue - to need it. + The general response handling in `transfer.c:466` happens in function + `readwrite_data()`. This mainly operates now like: - This triggered a regression in official Windows curl builds starting - with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.) + ``` + static CURLcode readwrite_data(data, ...) + { + do { + Curl_xfer_recv_resp(data, buf) + ... + Curl_xfer_write_resp(data, buf) + ... + } while(interested); + ... + } + ``` - Regression from 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 + All the response data handling is implemented in + `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` + implementation if available, or does the default behaviour. - Reported-by: zhengqwe on github - Helped-by: Nico Rieck - Fixes #12134 - Fixes #12136 - Closes #12217 + All raw response data needs to pass through this function. Which also + means that anyone in possession of such data may call + `Curl_xfer_write_resp()`. -- hostip: silence compiler warning `-Wparentheses-equality` + Closes #12480 - Seen with LLVM 17. +Daniel Stenberg (13 Jan 2024) - ``` - hostip.c:1336:22: warning: equality comparison with extraneous parentheses [- - Wparentheses-equality] - 1336 | (a->ai_family == PF_INET)) { - | ~~~~~~~~~~~~~^~~~~~~~~~ - hostip.c:1336:22: note: remove extraneous parentheses around the comparison t - o silence this warning - 1336 | (a->ai_family == PF_INET)) { - | ~ ^ ~ - hostip.c:1336:22: note: use '=' to turn this equality comparison into an assi - gnment - 1336 | (a->ai_family == PF_INET)) { - | ^~ - | = - 1 warning generated. - ``` +- RELEASE-NOTES: synced - Follow-up to b651aba0962bb31353f55de4dc35f745952a1b10 #12145 +- TODO: TFTP doesn't convert LF to CRLF for mode=netascii - Reviewed-by: Daniel Stenberg - Closes #12215 + Closes #12655 + Closes #12690 -Stefan Eissing (27 Oct 2023) +- gen: do italics/bold for a range of letters, not just single word -- doh: use PIPEWAIT when HTTP/2 is attempted + Previously it would match only on a sequence of non-space, which made it + miss to highlight for example "public suffix list". - Closes #12214 + Updated the recent cookie.d edit from 5da57193b732 to use bold instead + of italics. -Daniel Stenberg (27 Oct 2023) + Closes #12689 -- setopt: remove outdated cookie comment +- docs: describe and highlight super cookies - Closes #12206 + Reported-by: Yadhu Krishna M -Stefan Eissing (27 Oct 2023) + Closes #12687 -- cfilter: provide call to tell connection to forget a socket +- configure: when enabling QUIC, check that TLS supports QUIC - - fixed libssh.c workaround for a socket being closed by - the library - - eliminate the terrible hack in cf-socket.c to guess when - this happened and try not closing the socket again. - - fixes race in eyeballing when socket could have failed to - be closed for a discarded connect attempt + Most importantly perhaps is when using OpenSSL that the used + build/flavor has the QUIC API: the vanilla OpenSSL does not, only + BoringSSL, libressl, AWS-LC and quictls do. - Closes #12207 + Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7f + e57e#r136780413 -- url: protocol handler lookup tidy-up + Closes #12683 - - rename lookup to what it does - - use ARRAYSIZE instead of NULL check for end - - offer alternate lookup for 0-terminated strings +Stefan Eissing (11 Jan 2024) - Closes #12216 +- vquic: extract TLS setup into own source -Viktor Szakats (27 Oct 2023) + - separate ngtcp2 specific parts out + - provide callback during init to allow ngtcp2 to apply its defaults -- build: variadic macro tidy-ups + Closes #12678 - - delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks. - (both autotools and CMake.) - - delete duplicate `NULL` check in `Curl_trc_cf_infof()`. - - fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds. - ``` - ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parame - ter] - static void nosigpipe(struct Curl_easy *data, - ^ - ``` - - fix `#ifdef` comments in `lib/curl_trc.{c,h}`. - - fix indentation in some `infof()` calls. +Sergey Markelov (11 Jan 2024) - Follow-up to dac293cfb7026b1ca4175d88b80f1432d3d3c684 #12167 +- multi: remove total timer reset in file_do() while fetching file:// - Cherry-picked from #12105 - Closes #12210 + The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts + with resetting the timer that is a start point for further multi states. + If file://, MSTATE_DO calls file_do() that should not reset the total + timer. Otherwise, the total time is always less than the pre-transfer + and the start transfer times. -- cmake: speed up threads setup for Windows + Closes #12682 - Win32 threads are always available. We enabled them unconditionally - (with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection - logic has this condition hard-coded for Windows as well (since at least - 2007). +Daniel Stenberg (11 Jan 2024) - Instead of doing all the work of detecting pthread combinations on - Windows, then discarding those results, skip these efforts and assume - built-in thread support when building for Windows. +- http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT - This saves 1-3 slow CMake configuration steps. + Extended test 80 to verify this. - Reviewed-by: Daniel Stenberg - Closes #12202 + Reported-by: Stefan Eissing + Fixes #12680 + Closes #12681 -- cmake: speed up zstd detection +- sectransp: do verify_cert without memdup for blobs - Before this patch we detected the presence of a specific zstd API to - see if we can use the library. zstd published that API in its first - stable release: v1.0.0 (2016-08-31). + Since the information is then already stored in memory, this can avoid + an extra set of malloc + free calls. - Replace that method by detecting the zstd library version instead and - accepting if it's v1.0.0 or newer. Also display this detected version - and display a warning if the zstd found is unfit for curl. + Closes #12679 - We use the same version detection method as zstd itself, via its public - C header. +- hsts: remove assert for zero length domain - This deviates from autotools which keeps using the slow method of - looking for the API by building a test program. The outcome is the same - as long as zstd keeps offering this API. + A zero length domain can happen if the HSTS parser is given invalid + input data which is not unheard of and is done by the fuzzer. - Ref: https://github.com/facebook/zstd/commit/5a0c8e24395079f8e8cdc90aa1659cd5 - ab1b7427 (2016-08-12, committed) - Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first - released) - Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0 + Follow-up from cfe7902111ae547873 - Reviewed-by: Daniel Stenberg - Closes #12200 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661 -Daniel Stenberg (26 Oct 2023) + Closes #12676 -- openssl: fix infof() to avoid compiler warning for %s with null +- headers: make sure the trailing newline is not stored - vtls/openssl.c: In function ‘ossl_connect_step2’: - ../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror - =format-overflow=] - 120 | Curl_infof(data, __VA_ARGS__); } while(0) - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’ - 4008 | infof(data, "SSL connection using %s / %s / %s / %s", - | ^~~~~ - vtls/openssl.c:4008:49: note: format string is defined here - 4008 | infof(data, "SSL connection using %s / %s / %s / %s", - | ^~ + extended test1940 to verify blank header fields too - Follow-up to b6e6d4ff8f253c8b8055bab - Closes #12196 + Bug: https://curl.se/mail/lib-2024-01/0019.html + Reported-by: Dmitry Karpov + Closes #12675 -Stefan Eissing (26 Oct 2023) +- curl_easy_header.3: tiny language fix -- lib: apache style infof and trace macros/functions + Closes #12672 - - test for a simplified C99 variadic check - - args to infof() in --disable-verbose are no longer disregarded but - must compile. +- examples/range.c: add - Closes #12167 - Fixes #12083 - Fixes #11880 - Fixes #11891 + Closes #12671 -Daniel Stenberg (26 Oct 2023) +- examples/netrc.c: add -- RELEASE-NOTES: synced + Closes #12671 -Stefan Eissing (26 Oct 2023) +- examples/ipv6.c: new example showing IPv6-only internet transfer -- urldata: move async resolver state from easy handle to connectdata + Closes #12671 - - resolving is done for a connection, not for every transfer - - save create/dup/free of a cares channel for each transfer - - check values of setopt calls against a local channel if no - connection has been attached yet, when needed. +- examples/address-scope.c: renamed from ipv6.c - Closes #12198 + It shows address scope use really -Daniel Stenberg (26 Oct 2023) + Closes #12671 -- CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERRO - R +Stefan Eissing (9 Jan 2024) - It returns CURLE_WRITE_ERROR. It was not previously stated clearly. +- multi: pollset adjust, init with FIRSTSOCKET during connect - Reported-by: enWILLYado on github - Fixes #12201 - Closes #12203 + - `conn->sockfd` is set by `Curl_setup_transfer()`, but that + is called *after* the connection has been established + - use `conn->sock[FIRSTSOCKET]` instead -Viktor Szakats (25 Oct 2023) + Follow-up to a0f94800d507de + Closes #12664 -- autotools: update references to deleted `crypt-auth` option +Daniel Stenberg (9 Jan 2024) - Delete leftovers of the `crypt-auth` `./configure` option and - add the new ones that replaced them. +- WEBSOCKET.md: remove dead link - Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 +- CI: spellcheck/appveyor: invoke configure --without-libpsl - Reviewed-by: Daniel Stenberg - Closes #12194 - -Stefan Eissing (25 Oct 2023) - -- lib: introduce struct easy_poll_set for poll information - - Connection filter had a `get_select_socks()` method, inspired by the - various `getsocks` functions involved during the lifetime of a - transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), - return sockets to monitor and flag if this shall be done for POLLIN - and/or POLLOUT. - - Due to this design, sockets and flags could only be added, not - removed. This led to problems in filters like HTTP/2 where flow control - prohibits the sending of data until the peer increases the flow - window. The general transfer loop wants to write, adds POLLOUT, the - socket is writeable but no data can be written. - - This leads to cpu busy loops. To prevent that, HTTP/2 did set the - `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes - further attempts. This works if only one such filter is involved. If a - HTTP/2 transfer goes through a HTTP/2 proxy, two filters are - setting/clearing this flag and may step on each other's toes. - - Connection filters `get_select_socks()` is replaced by - `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps - up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` - flags. This struct is initialized in `multi_getsock()` by calling the - various `getsocks()` implementations based on transfer state, as before. - - After protocol handlers/transfer loop have set the sockets and flags - they want, the `easy_pollset` is *always* passed to the filters. Filters - "higher" in the chain are called first, starting at the first - not-yet-connection one. Each filter may add sockets and/or change - flags. When all flags are removed, the socket itself is removed from the - pollset. + Follow-up to 2998874bb61ac6 - Example: +- cmdline/docs/*.d: switch to using ## instead of .IP - * transfer wants to send, adds POLLOUT - * http/2 filter has a flow control block, removes POLLOUT and adds - POLLIN (it is waiting on a WINDOW_UPDATE from the server) - * TLS filter is connected and changes nothing - * h2-proxy filter also has a flow control block on its tunnel stream, - removes POLLOUT and adds POLLIN also. - * socket filter is connected and changes nothing - * The resulting pollset is then mixed together with all other transfers - and their pollsets, just as before. + To make the editing easier. To write and to read. - Use of `SEND_HOLD` is no longer necessary in the filters. + Closes #12667 - All filters are adapted for the changed method. The handling in - `multi.c` has been adjusted, but its state handling the the protocol - handlers' `getsocks` method are untouched. +- gen.pl: support ## for doing .IP in table-like lists - The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS - filters needed to be adjusted for the connecting handshake read/write - handling. + Warn on use of .RS/.IP/.RE - No noticeable difference in performance was detected in local scorecard - runs. + Closes #12667 - Closes #11833 +Jay Satiro (9 Jan 2024) -Daniel Stenberg (25 Oct 2023) +- cookie.d: Document use of empty string to enable cookie engine -- tests/README: SOCKS tests are not using OpenSSH, it has its own server + - Explain that --cookie "" can be used to enable the cookie engine + without reading any initial cookies. - Follow-up to 04fd67555cc + As is documented in CURLOPT_COOKIEFILE. - Closes #12195 + Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html -Jacob Hoffman-Andrews (25 Oct 2023) + Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420 + Reported-by: janko-js@users.noreply.github.com -- tets: make test documentation more user-friendly + Closes https://github.com/curl/curl/pull/12646 - Put the instructions to run tests right at the top of tests/README.md. +Daniel Stenberg (9 Jan 2024) - Give instructions to read the runtests.1 man page for information - about flags. Delete redundant copy of the flags documentation in the - README. +- setopt: use memdup0 when cloning COPYPOSTFIELDS - Add a mention in README.md of the important parallelism flag, to make - test runs go much faster. + Closes #12651 - Move documentation of output line format into the runtests.1 man page, - and update it with missing flags. +- telnet: use dynbuf instad of malloc for escape buffer - Fix the order of two flags in the man page. + Previously, send_telnet_data() would malloc + free a buffer every time + for escaping IAC codes. Now, it reuses a dynbuf for this purpose. - Closes #12193 + Closes #12652 -Viktor Szakats (24 Oct 2023) +- CI: install libpsl or configure --without-libpsl in builds -- cmake: pre-fill rest of detection values for Windows + As a follow-up to the stricted libpsl check in configure - The goal of this patch is to avoid unnecessary feature detection work - when doing Windows builds with CMake. Do this by pre-filling well-known - detection results for Windows and specifically for mingw-w64 and MSVC - compilers. Also limit feature checks to platforms where the results are - actually used. Drop a few redundant ones. And some tidying up. +- configure: make libpsl detection failure cause error - - pre-fill remaining detection values in Windows CMake builds. + To force users to explictily disable it if they really don't want it + used and make it harder to accidentally miss it. - Based on actual detection results observed in CI runs, preceding - similar work over libssh2 and matching up values with - `lib/config-win32.h`. + --without-libpsl is the option to use if PSL is not wanted. - This brings down CMake configuration time from 58 to 14 seconds on the - same local machine. + Closes #12661 - On AppVeyor CI this translates to: - - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job): - https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecr - jpy7necb#L296 - https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2 - fe249uo8#L186 - - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job): - https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5iv - lcs7ub29?fullLog=true#L290 - https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjs - yc9kl13a?fullLog=true#L194 +- RELEASE-NOTES: synced - The formula is about 1-3 seconds delay for each detection. Almost all - of these trigger a full compile-link cycle behind the scenes, slow - even today, both cross and native, mingw-w64 and apparently MSVC too. - Enabling .map files or other custom build features slows it down - further. (Similar is expected for autotools configure.) +- pop3: replace calloc + memcpy with memdup0 - - stop detecting `idn2.h` if idn2 was deselected. - autotools does this. + ... and make sure to return error on out of memory. - - stop detecting `idn2.h` if idn2 was not found. - This deviates from autotools. Source code requires both header and - lib, so this is still correct, but faster. + Closes #12650 - - limit `ADDRESS_FAMILY` detection to Windows. +- lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT - - normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format. + Closes #12658 - - pre-fill `HAVE_WIN32_WINNT`-dependent detection results. - Saving 4 (slow) feature-detections in most builds: `getaddrinfo`, - `freeaddrinfo`, `inet_ntop`, `inet_pton` +- mime: use memdup0 instead of malloc + memcpy - - fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`, - `HAVE_GETTIMEOFDAY` for mingw-w64. - Luckily this do not change build results, as `WIN32` took - priority over `HAVE_GETTIMEOFDAY` with the current source - code. + Closes #12649 - - limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and - `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows. - We're not using these in the source code for Windows. +- tool_getparam: move the --rate logic into set_rate() - - reduce compiler warning noise in CMake internal logs: - - fix to include `winsock2.h` before `windows.h`. - Apply it to autotools test snippets too. - - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above. - - cleanup `CMake/CurlTests.c` to emit less warnings. +- tool_getparam: switch to an enum for every option - - delete redundant `HAVE_MACRO_SIGSETJMP` feature check. - It was the same check as `HAVE_SIGSETJMP`. + To make the big switch much easier to read/understand and to make it + easier to add new options. - - delete 'experimental' marking from `CURL_USE_OPENSSL`. +- tool_getparam: build post data using dynbuf (more) - - show CMake version via `CMakeLists.txt`. - Credit to the `zlib-ng` project for the idea: - https://github.com/zlib-ng/zlib-ng/blob/61e181c8ae93dbf56040336179c9954078b - d1399/CMakeLists.txt#L7 +- tool_getparam: replace malloc + copy by dynbuf for --data - - make `CMake/CurlTests.c` pass `checksrc`. +- tool_getparam: make data_urlencode avoid direct malloc - - `CMake/WindowsCache.cmake` tidy-ups. + use aprintf() instead - - replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`. +- tool_getparam: move the --url-query logic into url_query() - Closes #12044 + This function is not doing post at all so it was always weirdly placed. -Jay Satiro (24 Oct 2023) +- tool_getparam: move the --data logic into set_data() -- page-footer: clarify exit code 25 +- tool_getparam: unify the cmdline switch() into a single one - - Clarify that curl tool exit code 25 means an upload failed to start. + - easier to follow, easier to modify, easier to extend, possibly slightly + faster - Exit code 25 is equivalent to CURLE_UPLOAD_FAILED (25). Prior to this - change the documentation only mentioned the case of FTP STOR failing. + - each case now has the long option as a comment - Reported-by: Emanuele Torre +- tool_getparam: bsearch cmdline options - Ref: https://github.com/curl/curl/blob/curl-8_4_0/docs/libcurl/libcurl-errors - .3#L113-L115 + - the option names are now alpha sorted and lookup is a lot faster - Fixes https://github.com/curl/curl/issues/12189 - Closes https://github.com/curl/curl/pull/12190 + - use case sensitive matching. It was previously case insensitive, but that + was not documented nor tested. -Daniel Stenberg (24 Oct 2023) + - remove "partial match" feature. It was not documented, not tested and + was always fragile as existing use could break when we add a new + option -- scripts/cijobs.pl: adjust for appveyor + - lookup short options via a table - Follow-up to a1d73a6bb + Closes #12631 -Alex Bozarth (24 Oct 2023) +Gabe (8 Jan 2024) -- OpenSSL: Include SIG and KEM algorithms in verbose +- COPYING: update copyright year - Currently the verbose output does not include which algorithms are used - for the signature and key exchange when using OpenSSL. Including the - algorithms used will enable better debugging when working on using new - algorithm implementations. Know what algorithms are used has become more - important with the fast growing research into new quantum-safe - algorithms. + Closes #12654 - This implementation includes a build time check for the OpenSSL version - to use a new function that will be included in OpenSSL 3.2 that was - introduced in openssl/openssl@6866824 +Stefan Eissing (8 Jan 2024) - Based-on-patch-by: Martin Schmatz - Closes #12030 +- url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD -Daniel Stenberg (23 Oct 2023) + Also add more tracing to test 19 -- http2: provide an error callback and failf the message + Follow-up to a0f9480 - Getting nghttp2's error message helps users understand what's going - on. For example when the connection is brought down due a forbidden - header is used - as that header is then not displayed by curl itself. + Fixes #12657 + Closes #12659 - Example: +Daniel Stenberg (8 Jan 2024) - curl: (92) Invalid HTTP header field was received: frame type: 1, - stream: 1, name: [upgrade], value: [h2,h2c] +- connect: remove margin from eyeballer alloc - Ref: #12172 - Closes #12179 + Presumably leftovers from debugging -Turiiya (23 Oct 2023) + Closes #12647 -- BINDINGS: add V binding +- ftp: only consider entry path if it has a length - Closes #12182 + Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb -Daniel Stenberg (22 Oct 2023) + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631 -- configure: check for the fseeko declaration too + Avoids a NULL pointer deref. - ... and make the code require both symbol and declaration. + Closes #12648 - This is because for Android, the symbol is always present in the lib at - build-time even when not actually available in run-time. +Stefan Eissing (7 Jan 2024) - Assisted-by: Viktor Szakats - Reported-by: 12932 on github - Fixes #12086 - Closes #12158 +- transfer: adjust_pollset improvements -Viktor Szakats (22 Oct 2023) + - let `multi_getsock()` initialize the pollset in what the + transfer state requires in regards to SEND/RECV + - change connection filters `adjust_pollset()` implementation + to react on the presence of POLLIN/-OUT in the pollset and + no longer check CURL_WANT_SEND/CURL_WANT_RECV + - cf-socket will no longer add POLLIN on its own + - http2 and http/3 filters will only do adjustments if the + passed pollset wants to POLLIN/OUT for the transfer on + the socket. This is similar to the HTTP/2 proxy filter + and works in stacked filters. -- cmake: fix OpenSSL quic detection in quiche builds + Closes #12640 - An orphan call to `CheckQuicSupportInOpenSSL()` remained after a recent - update when checking QUIC for quiche. Move back QUIC detection to - a function and fixup callers to use that. Also make sure that quiche - gets QUIC from BoringSSL, because it doesn't support other forks at this - time. +Daniel Stenberg (6 Jan 2024) - Regression from dee310d54261f9a8416e87d50bccfe2cbe404949 #11555 +- ftp: use memdup0 to store the OS from a SYST 215 response - Reported-by: Casey Bodley - Fixes #12160 - Closes #12162 + avoid malloc + direct buffer fiddle -Daniel Stenberg (22 Oct 2023) + Closes #12639 -- RELEASE-NOTES: synced +- ftp: use dynbuf to store entrypath - bump to 8.5.0 for pending release + avoid direct malloc -Dan Fandrich (21 Oct 2023) + Closes #12638 -- test3103: add missing quotes around a test tag attribute +Lealem Amedie (6 Jan 2024) -Loïc Yhuel (21 Oct 2023) +- wolfssl: load certificate *chain* for PEM client certs -- tool: fix --capath when proxy support is disabled + Closes #12634 - After 95e8515ca0, --capath always sets CURLOPT_PROXY_CAPATH, which fails - with CURLE_UNKNOWN_OPTION when proxy support is disabled. +Stefan Eissing (4 Jan 2024) - Closes #12089 +- http: adjust_pollset fix -Daniel Stenberg (21 Oct 2023) + do not add a socket for POLLIN when the transfer does not want to send + (for example is paused). -- openldap: move the alloc of ldapconninfo to *connect() + Follow-up to 47f5b1a - Fixes a minor memory leak on LDAP connection reuse. + Reported-by: bubbleguuum on github + Fixes #12632 + Closes #12633 - Doing the allocation already in *setup_connection() is wrong since that - connect struct might get discarded early when an existing connection is - reused instead. +Daniel Stenberg (3 Jan 2024) - Closes #12166 +- tool: make parser reject blank arguments if not supported -- openldap: set the callback argument in oldap_do + Already in the getstr() function that clones the input argument. - ... to make sure it has the current 'data' pointer and not a stale old - one. + Closes #12620 - Reported-by: Dan Fandrich - Closes #12166 +dependabot[bot] (3 Jan 2024) -- gnutls: support CURLSSLOPT_NATIVE_CA +- build(deps): bump github/codeql-action from 2 to 3 - Remove the CURL_CA_FALLBACK logic. That build option was added to allow - primarily OpenSSL to use the default paths for loading the CA certs. For - GnuTLS it was instead made to load the "system certs", which is - different and not desirable. + Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 + to 3. + - [Release notes](https://github.com/github/codeql-action/releases) + - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) + - [Commits](https://github.com/github/codeql-action/compare/v2...v3) - The native CA store loading is now asked for with this option. + --- + updated-dependencies: + - dependency-name: github/codeql-action + dependency-type: direct:production + update-type: version-update:semver-major + ... - Follow-up to 7b55279d1d856 + Signed-off-by: dependabot[bot] - Co-authored-by: Jay Satiro + Closes #12625 - Closes #12137 +- build(deps): bump actions/checkout from 3 to 4 -Stefan Eissing (21 Oct 2023) + Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. + - [Release notes](https://github.com/actions/checkout/releases) + - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) + - [Commits](https://github.com/actions/checkout/compare/v3...v4) -- RTSP: improved RTP parser + --- + updated-dependencies: + - dependency-name: actions/checkout + dependency-type: direct:production + update-type: version-update:semver-major + ... - - fix HTTP header parsing to report incomplete - lines it buffers as consumed! - - re-implement the RTP parser for interleave RTP - messages for robustness. It is now keeping its - state at the connection - - RTSP protocol handler "readwrite" implementation - now tracks if the response is before/in/after - header parsing or "in" a bod by calling - "Curl_http_readwrite_headers()" itself. This - allows it to know when non-RTP bytes are "junk" - or HEADER or BODY. - - tested with #12035 and various small receive - sizes where current master fails + Signed-off-by: dependabot[bot] - Closes #12052 + Closes #12624 -- http2: header conversion tightening +- build(deps): bump actions/upload-artifact from 3 to 4 - - fold the code to convert dynhds to the nghttp2 structs - into a dynhds internal method - - saves code duplication - - pacifies compiler analyzers + Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) f + rom 3 to 4. + - [Release notes](https://github.com/actions/upload-artifact/releases) + - [Commits](https://github.com/actions/upload-artifact/compare/v3...v4) - Closes #12097 + --- + updated-dependencies: + - dependency-name: actions/upload-artifact + dependency-type: direct:production + update-type: version-update:semver-major + ... -Daniel Stenberg (21 Oct 2023) + Signed-off-by: dependabot[bot] -- curl_ntlm_wb: fix elif typo + Closes #12627 - Reported-by: Manfred Schwarb - Follow-up to d4314cdf65ae - Bug: https://github.com/curl/curl/commit/d4314cdf65aee295db627016934bd9eb621a - b077#r130551295 +- build(deps): bump actions/download-artifact from 3 to 4 -Dan Fandrich (20 Oct 2023) + Bumps [actions/download-artifact](https://github.com/actions/download-artifac + t) from 3 to 4. + - [Release notes](https://github.com/actions/download-artifact/releases) + - [Commits](https://github.com/actions/download-artifact/compare/v3...v4) -- test1683: remove commented-out check alternatives + --- + updated-dependencies: + - dependency-name: actions/download-artifact + dependency-type: direct:production + update-type: version-update:semver-major + ... - Python precheck/postcheck alternatives were included but commented out. - Since these are not used and perl is guaranteed to be available to run - the perl versions anyway, the Python ones are removed. + Signed-off-by: dependabot[bot] -Daniel Stenberg (20 Oct 2023) + Closes #12626 -- hostip: show the list of IPs when resolving is done +Stefan Eissing (3 Jan 2024) - Getting 'curl.se' today then gets this verbose output which might help - debugging connectivity related matters. +- http3/quiche: fix result code on a stream reset - * Host curl.se:80 was resolved. - * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, - 2a04:4e42:600::347, 2a04:4e42:800::347, 2a04:4e42:a00::347, - 2a04:4e42:c00::347, 2a04:4e42:e00::347 - * IPv4: 151.101.193.91, 151.101.1.91, 151.101.65.91, 151.101.129.91 + - fixes pytest failures in test 07_22 + - aligns CURLcode values on stream reset with ngtcp2 - Co-authored-by: Jay Satiro - Closes #12145 + Closes #12629 -rilysh (20 Oct 2023) +Daniel Stenberg (2 Jan 2024) -- docs: fix function typo in curl_easy_option_next.3 +- setopt: clear mimepost when formp is freed - Closes #12170 + A precaution to avoid a possibly dangling pointer left behind. -Daniel Stenberg (20 Oct 2023) + Reported-by: Thomas Ferguson + Fixes #12608 + Closes #12621 -- vssh: remove the #ifdef for Curl_ssh_init, use empty macro +Andy Alt (2 Jan 2024) - In the same style as other init calls +- CI: Add dependabot.yml -- easy: remove duplicate wolfSSH init call + This will cause dependabot to open a PR when various actions are + updated, provided that the action maintainer has issued a release. - It is already done in Curl_ssh_init() where it belongs. + Closes #12623 - Closes #12168 +Gisle Vanem (2 Jan 2024) -- socks: make SOCKS5 use the CURLOPT_IPRESOLVE choice +- content_encoding: change return code to typedef'ed enum - Fixes #11949 - Reported-by: Ammar Faizi - Closes #12163 + ... to work around a clang ubsan warning. -- urldata: move the 'internal' boolean to the state struct + Fixes #12618 + Closes #12622 - ... where all the other state bits for the easy handles live. +Daniel Stenberg (2 Jan 2024) - Closes #12165 +- tool: prepend output_dir in header callback -- url: don't touch the multi handle when closing internal handles + When Content-Disposition parsing is used and an output dir is prepended, + make sure to store that new file name correctly so that it can be used + for setting the file timestamp when --remote-time is used. - Reported-by: Maksymilian Arciemowicz - Closes #12165 + Extended test 3012 to verify. -Faraz Fallahi (19 Oct 2023) + Co-Authored-by: Jay Satiro + Reported-by: hgdagon on github + Fixes #12614 + Closes #12617 -- getenv: PlayStation doesn't have getenv() +- test1254: fix typo in name plus shorten it - Closes #12140 +- RELEASE-NOTES: synced -Daniel Stenberg (19 Oct 2023) +Viktor Szakats (2 Jan 2024) -- transfer: only reset the FTP wildcard engine in CLEAR state +- schannel: fix `-Warith-conversion` gcc 13 warning - To avoid the state machine to start over and redownload all the files - *again*. + ``` + lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' + may change the sign of the result [-Warith-conversion] + 1201 | *extension_len = *list_len + + | ^ + ``` - Reported-by: lkordos on github - Regression from 843b3baa3e3cb228 (shipped in 8.1.0) - Bisect-by: Dan Fandrich - Fixes #11775 - Closes #12156 + Closes #12616 -Stefan Eissing (19 Oct 2023) +- asyn-thread: silence `-Wcast-align` warning for Windows -- GHA: move mod_h2 version in CI to v2.0.25 + Seen with llvm/clang 17: + ``` + lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct + thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align] + 310 | CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlap + ped); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~ + .../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded fro + m macro 'CONTAINING_RECORD' + 717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(addre + ss) - (ULONG_PTR)(&((type *)0)->field))) + | ^~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ``` - Closes #12157 + Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482 -Daniel Stenberg (19 Oct 2023) + Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261 + Closes #12615 -- ntlm_wb: use pipe instead of socketpair when possible +Daniel Stenberg (2 Jan 2024) - Closes #12149 +- tool_listhelp: regenerate after recent .d updates -- RELEASE-NOTES: synced + Makes it survive test 1478 -- asyn-thread: use pipe instead of socketpair for IPC when available + Closes #12612 - If pipe() is present. Less overhead. +- test1478: verify src/tool_listhelp.c - Helped-by: Viktor Szakats - Closes #12146 + Verify that the source file on disk is identical to the output of gen.pl + listhelp, as otherwise they are out of sync and need attention. -Dan Fandrich (17 Oct 2023) + Closes #12612 -- tests: Fix Windows test helper tool search & use it for handle64 +- testutil: make runtests support %include - The checkcmd() and checktestcmd() functions would not have worked on - Windows due to hard-coding the UNIX PATH separator character and not - adding .exe file extension. This meant that tools like stunnel, valgrind - and nghttpx would not have been found and used on Windows, and - inspection of previous test runs show none of those being found in pure - Windows CI builds. + Using this instruction, a test case can include the contents of a file + into the test during the preprocessing. - With this fixed, they can be used to detect the handle64.exe program - before attempting to use it. When handle64.exe was called - unconditionally without it existing, it caused perl to abort the test - run with the error + Closes #12612 - The running command stopped because the preference variable - "ErrorActionPreference" or common parameter is set to Stop: - sh: handle64.exe: command not found +- runtests: for mode="text" on , fix newlines on both parts - Closes #12115 + Closes #12612 -Daniel Stenberg (17 Oct 2023) +Jay Satiro (2 Jan 2024) -- multi: use pipe instead of socketpair to *wakeup() +- quiche: return CURLE_HTTP3 on send to invalid stream - If pipe() is present. Less overhead. + Prior to this change if a send failed on a stream in an invalid state + (according to quiche) and not marked as closed (according to libcurl) + then the send function would return CURLE_SEND_ERROR. - Closes #12142 + We already have similar code for ngtcp2 to return CURLE_HTTP3 in this + case. -Jay Satiro (17 Oct 2023) + Caught by test test_07_upload.py: test_07_22_upload_parallel_fail. -- build: fix 'threadsafe' feature detection for older gcc + Fixes https://github.com/curl/curl/issues/12590 + Closes https://github.com/curl/curl/pull/12597 - - Add 'threadsafe' to the feature list shown during build if POSIX - threads are being used. +Daniel Stenberg (1 Jan 2024) - This is a follow-up to 5adb6000 which added support for building a - thread-safe libcurl with older versions of gcc where atomic is not - available but pthread is. +- cmdline-opts: update availability for the *-ca-native options - Reported-by: Dan Fandrich - Co-authored-by: Dan Fandrich + Closes #12613 - Fixes https://github.com/curl/curl/issues/12125 - Closes https://github.com/curl/curl/pull/12127 +Patrick Monnerat (31 Dec 2023) -Daniel Stenberg (16 Oct 2023) +- openldap: fix STARTTLS -- test729: verify socks4a with excessive proxy user name length + It was not working anymore since introduction of connection filters. -- socks: better buffer size checks for socks4a user and hostname + Also do not attempt to recover from a failing TLS negotiation with + CURLUSESSL_TRY. - Also limit the proxy user name to 255 bytes, which is the same limit as - in SOCKS5. + Closes #12610 - Reported-by: sd0 on hackerone - Closes #12139 +Daniel Stenberg (31 Dec 2023) -- curl.h: on FreeBSD include sys/param.h instead of osreldate.h +- haproxy-clientip.d: document the arg - Should things build on Playstation as well + The arg keyword was missing and therefore not present in the man page. - Fixes #12107 - Reported-by: Faraz Fallahi - Closes #12123 + Closes #12611 -Marcin Rataj (16 Oct 2023) +annalee (29 Dec 2023) -- tool_operate: fix links in ipfs errors +- configure: fix no default int compile error in ipv6 detection - URL fragment links generated from headers in - https://curl.se/docs/ipfs.html are lowercase. + Closes #12607 - Closes #12133 +Dan Fandrich (28 Dec 2023) -Viktor Szakats (15 Oct 2023) +- CI: Fix use of any-glob-to-all-files in the labeler -- cmake: replace `check_library_exists_concat()` + Despite its name, this atom acts like one-glob-to-all-files and a + different syntax with braces must be used to get + any-glob-to-all-files semantics. Unfortunately, this makes the file + completely unreadable. - The idea of `check_library_exists_concat()` is that it detects an - optional component and adds it to the list of libs that we also use in - subsequent component checks. This caused problems when detecting - components with unnecessary dependencies that were not yet built. + Ref: https://github.com/actions/labeler/issues/731 - CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used - for component checks, which we already use in most cases. That left 4 - uses of `check_library_exists_concat()`. Only one of these actually - needed the 'concat' feature (ldap/lber). +Daniel Stenberg (29 Dec 2023) - Delete this function and replace it with standard - `check_library_exists()` and manual management of our `CURL_LIBS` - list we use when linking build targets. And special logic to handle the - ldap/lber case. +- CURLOPT_AUTOREFERER.3: mention CURLINFO_REFERER - (We have a similar function for headers: `check_include_file_concat()`. - It works, but problematic for performance reasons and because it hides - the actual headers required in `check_symbol_exists()` calls.) +- CURLINFO_REFERER.3: clarify that it is the *request* header - Ref: #11537 #11558 - Fixes #11285 - Fixes #11648 - Closes #12070 + That libcurl itself sent in the most recent request -LoRd_MuldeR (15 Oct 2023) + Closes #12605 -- tool_cb_wrt: fix write output for very old Windows versions +Jay Satiro (28 Dec 2023) - - Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW() - function. +- system_win32: fix a function pointer assignment warning - Apparently this parameter was *not* optional on older Windows versions. + - Use CURLX_FUNCTION_CAST to suppress a function pointer assignment + warning. - Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1. - So at some point between those two Microsoft changed the behavior. + a6bbc87f added lookups of some Windows API functions and then cast them + like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn + about that as breaking strict-aliasing rules so this PR changes those + assignments to use CURLX_FUNCTION_CAST. - Prior to this change, on those versions if parameter is NULL then the - function call fails with error ERROR_INVALID_ACCESS. + Bug: https://github.com/curl/curl/pull/12581#issuecomment-1869804317 + Reported-by: Marcel Raad - Regression since af3f4e41. + Closes https://github.com/curl/curl/pull/12602 - Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299 +- verify-examples.pl: fail verification on unescaped backslash - Fixes https://github.com/curl/curl/issues/12131 - Closes https://github.com/curl/curl/pull/12130 + - Check that all backslashes in EXAMPLE are properly escaped. -Jay Satiro (15 Oct 2023) + eg manpage must always use `\\n` never `\n`. -- tool_urlglob: fix build for old gcc versions + This is because the manpage requires we always double blackslash to show + a single backslash. Prior to this change an erroneous single backslash + would pass through and compile even though it would not show correctly + in the manpage. - - Don't use __builtin_mul_overflow for GCC 4 and earlier. + Co-authored-by: Daniel Stenberg - The function was added in GCC 5. + Ref: https://github.com/curl/curl/pull/12588 - Ref: https://gcc.gnu.org/gcc-5/changes.html + Closes https://github.com/curl/curl/pull/12589 - Reported-by: Dan Fandrich +- vtls: fix missing multissl version info - Fixes https://github.com/curl/curl/issues/12124 - Closes https://github.com/curl/curl/pull/12128 + - Fix erroneous buffer copy logic from ff74cef5. -Carlos Henrique Lima Melara (14 Oct 2023) + Prior to this change the MultiSSL version info returned to the user + was empty. -- docs/libcurl: fix three minor man page format mistakes + Closes https://github.com/curl/curl/pull/12599 - Reported-by: Samuel Henrique +Daniel Stenberg (27 Dec 2023) - Closes https://github.com/curl/curl/pull/12126 +- KNOWN_BUGS: [RTSP] Some methods do not support response bodies -Jay Satiro (14 Oct 2023) + Closes #12414 -- tests/server: add more SOCKS5 handshake error checking +Patrick Monnerat (27 Dec 2023) - - Add additional checking for missing and too-short SOCKS5 handshake - messages. +- openldap: fix an LDAP crash - Prior to this change the SOCKS5 test server did not check that all parts - of the handshake were received successfully. If those parts were missing - or too short then the server would access uninitialized memory. + Reported-by: Ozan Cansel + Fixes #12593 + Closes #12600 - This issue was discovered in CI job 'memory-sanitizer' test results. - Test 2055 was failing due to the SOCKS5 test server not running. It was - not running because either it crashed or memory sanitizer aborted it - during Test 728. Test 728 connects to the SOCKS5 test server on a - redirect but does not send any data on purpose. The test server was not - prepared for that. +Daniel Stenberg (27 Dec 2023) - Reported-by: Dan Fandrich +- getinfo: CURLINFO_QUEUE_TIME_T - Fixes https://github.com/curl/curl/issues/12117 - Closes https://github.com/curl/curl/pull/12118 + Returns the time, in microseconds, during which this transfer was held + in a waiting queue before it started "for real". A transfer might be put + in a queue if after getting started, it cannot create a new connection + etc due to set conditions and limits imposed by the application. -Daniel Stenberg (14 Oct 2023) + Ref: #12293 + Closes #12368 - RELEASE-NOTES: synced -Sohom Datta (14 Oct 2023) - -- tool_getparam: limit --rate to be smaller than number of ms - - Currently, curl allows users to specify absurd request rates that might - be higher than the number of milliseconds in the unit (ex: curl --rate - 3600050/h http://localhost:8080 does not error out despite there being - only 3600000ms in a hour). - - This change adds a conditional check before the millisecond calculation - making sure that the number is not higher than the numerator (the unit) - If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE - - Closes #12116 +Jay Satiro (26 Dec 2023) -Daniel Stenberg (14 Oct 2023) +- examples/sendrecv: fix comment line length -- opts: fix two minor man page format mistakes + Caught by checksrc. -Jay Satiro (14 Oct 2023) +Haydar Alaidrus (23 Dec 2023) -- curl_trc: remove a bad assertion +- CURLOPT_POSTFIELDS.3: fix incorrect C string escape in example - - Remove DEBUGASSERT that an internal handle must not have user - private_data set before calling the user's debug callback. + - Escape inner quotes with two backslashes. - This is a follow-up to 0dc40b2a. The user can distinguish their easy - handle from an internal easy handle by setting CURLOPT_PRIVATE on their - easy handle. I had wrongly assumed that meant the user couldn't then - set CURLOPT_PRIVATE on an internal handle as well. + Two backslashes escapes the backslash for the man page and will show as + a single backslash. - Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 - Reported-by: Daniel Stenberg + eg: "{\\"name\\": \\"daniel\\"}" shows as "{\"name\": \"daniel\"}". - Closes https://github.com/curl/curl/pull/12104 + Closes https://github.com/curl/curl/pull/12588 -Dan Fandrich (13 Oct 2023) +Viktor Szakats (23 Dec 2023) -- test613: stop showing an error on missing output file +- appveyor: tidy-ups - This test would show an error message if the output was missing during - the log post-processing step, but the message was not captured by the - test harness and wasn't useful since the normal golden log file - comparison would the problem more clearly. + - replace two remaining backslashes with forward slashes. + - tidy up the way we form and pass `TFLAGS`. -Stefan Eissing (13 Oct 2023) + Follow-up to 2d4d0c1fd32f5cc3f946c407c8eccd5477b287df #12572 -- quic: manage connection idle timeouts + Closes #12582 - - configure a 120s idle timeout on our side of the connection - - track the timestamp when actual socket IO happens - - check IO timestamp to our *and* the peer's idle timeouts - in "is this connection alive" checks +Stefan Eissing (22 Dec 2023) - Reported-by: calvin2021y on github - Fixes #12064 - Closes #12077 +- transfer: fix upload rate limiting, add test cases -Dan Fandrich (13 Oct 2023) + - add test cases for rate limiting uploads for all + http versions + - fix transfer loop handling of limits. Signal a re-receive + attempt only on exhausting maxloops without an EAGAIN + - fix `data->state.selectbits` forcing re-receive to also + set re-sending when transfer is doing this. -- CI: ignore test 286 on Appveyor gcc 9 build + Reported-by: Karthikdasari0423 on github + Fixes #12559 + Closes #12586 - This test fails sometimes with a super fast retry loop due to what may - just be a compiler bug. The test results are ignored on the one CI job - where it occurs because there seems to be nothing we can do to fix it. +Daniel Stenberg (22 Dec 2023) - Fixes #12040 - Closes #12106 +- mbedtls: free the entropy when threaded -Viktor Szakats (13 Oct 2023) + The entropy_free was never done for threaded builds, causing a small + (fixed) memory leak. -- lib: fix gcc warning in printf call + Reported-by: RevaliQaQ on github + Fixes #12584 + Closes #12585 - Do not pass NULL to printf %s. +Stefan Eissing (22 Dec 2023) - Seen with gcc 13.2.0 on Debian: - ``` - .../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wfo - rmat-overflow=] - ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/1758442 - 6483#step:3:11104 +- http2: improved on_stream_close/data_done handling - Ref: #10284 - Co-authored-by: Jay Satiro - Closes #12082 + - there seems to be a code path that cleans up easy handles without + triggering DONE or DETACH events to the connection filters. This + would explain wh nghttp2 still holds stream user data + - add GOOD check to easy handle used in on_close_callback to + prevent crashes, ASSERTs in debug builds. + - NULL the stream user data early before submitting RST + - add checks in on_stream_close() to identify UNGOOD easy handles -Alex Klyubin (13 Oct 2023) + Reported-by: Hans-Christian Egtvedt + Fixes #10936 + Closes #12562 -- http2: safer invocation of populate_binsettings +Daniel Stenberg (22 Dec 2023) - populate_binsettings now returns a negative value on error, instead of a - huge positive value. Both places which call this function have been - updated to handle this change in its contract. +- mprintf: overhaul and bugfixes - The way populate_binsettings had been used prior to this change the huge - positive values -- due to signed->unsigned conversion of the potentially - negative result of nghttp2_pack_settings_payload which returns negative - values on error -- are not possible. But only because http2.c currently - always provides a large enough output buffer and provides H2 SETTINGS - IVs which pass the verification logic inside nghttp2. If the - verification logic were to change or if http2.c started passing in more - IVs without increasing the output buffer size, the overflow could become - reachable, and libcurl/curl might start leaking memory contents to - servers/proxies... + In a test case using lots of snprintf() calls using many commonly used + %-codes per call, this version is around 30% faster than previous + version. - Closes #12101 + It also fixes the #12561 bug which made it not behave correctly when + given unknown %-sequences. Fixing that flaw required a different take on + the problem, which resulted in the new two-arrays model. -Daniel Stenberg (13 Oct 2023) + lib557: extended - Verify the #12561 fix and test more printf features -- openssl: avoid BN_num_bits() NULL pointer derefs + unit1398: fix test: It used a $ only for one argument, which is not + supported. - Reported-by: icy17 on github - Fixes #12099 - Closes #12100 + Fixes #12561 + Closes #12563 -- wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA +Viktor Szakats (21 Dec 2023) - This define is set in wolfssl's options.h file when this function and - feature is present. Handles both builds with the feature explicitly - disabled and wolfSSL versions before 5.5.2 - which introduced this API - call. +- appveyor: replace PowerShell with bash + parallel autotools - Closes #12108 + PowerShell works (after a steep development curve), but one property of + it stuck and kept causing unresolvable usability issues: With + `$ErrorActionPreference=Stop`, it does abort on failures, but shows only + the first line of the error message. In `Continue` mode, it shows the + full error message, but doesn't stop on all errors. Another issue is + PowerShell considering any stderr output as if the command failed (this + has been improved in 7.2 (2021-Nov), but fixed versions aren't running + in CI and will not be for a long time in all test images.) -- tool_urlglob: make multiply() bail out on negative values + Thus, we're going with bash. - - Does not work correctly with negative values - - use __builtin_mul_overflow() on gcc + Also: + - use `-j2` with autotools tests, making them finish 5-15 minutes per + job faster. + - omit `POSIX_PATH_PREFIX`. + - use `WINDIR`. + - prefer forward slashes. - Reported-by: Torben Dury - Closes #12102 + Follow-up to: 75078a415d9c769419aed4153d3d525a8eba95af #11999 + Ref: #12444 -Loïc Yhuel (13 Oct 2023) + Fixes #12560 + Closes #12572 -- cmake: fix CURL_DISABLE_GETOPTIONS +Pavel Pavlov (21 Dec 2023) - - Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake. +- asyn-thread: use GetAddrInfoExW on >= Windows 8 - Prior to this change the option had no effect because it was missing - from that file. + For doing async DNS resolution instead of starting a thread for each + request. - Closes https://github.com/curl/curl/pull/12091 + Fixes #12481 + Closes #12482 -- easy_lock: add a pthread_mutex_t fallback +Daniel Stenberg (21 Dec 2023) - This allows to keep the init threadsafe with gcc < 4.9.0 (no C11 - atomics). +- strerror: repair get_winsock_error() - Closes https://github.com/curl/curl/pull/12090 + It would try to read longer than the provided string and crash. -Viktor Szakats (12 Oct 2023) + Follow-up to ff74cef5d4a0cf60106517a1c7384 + Reported-by: calvin2021y on github + Fixes #12578 + Closes #12579 -- CI: add autotools, out-of-tree, debug build to distro check job +- CURLOPT_SSH_*_KEYFILE: clarify - Add a job that builds curl from a generated source tarball sample, with - autotools, out-of-tree, in debug mode. + Closes #12554 - Ref: #12085 - Closes #12088 +ivanfywang (21 Dec 2023) -Daniel Stenberg (12 Oct 2023) +- ngtcp2: put h3 at the front of alpn -- http: avoid Expect: 100-continue if Upgrade: is used + Closes #12576 - Reported-by: Daniel Jelinski - Fixes #12022 - Closes #12062 +Daniel Stenberg (21 Dec 2023) -Jan Alexander Steffens (heftig) (12 Oct 2023) +- test460: verify a command line using --expand with no argument -- docs: use SOURCE_DATE_EPOCH for generated manpages + This verifies the fix for #12565 - This should make builds from Git reproducible. +- tool_getparam: do not try to expand without an argument - Closes #12092 + This would lead to a segfault. -Daniel Stenberg (12 Oct 2023) + Fixes #12565 + Reported-by: Geeknik Labs + Closes #12575 - RELEASE-NOTES: synced - Bumped to 8.4.1 + Bumped version to 8.6.0 because of changes -Viktor Szakats (12 Oct 2023) +- Makefile.am: fix the MSVC project generation -- cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection + It made the vcxproj files not get included in dist tarballs. - Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet, - aligning this with autotools. This fixes an error when doing - cross-builds and also actually detects this feature. It affected systems - not allowlisted into this, e.g. SerenityOS. + Regression since 74423b5df4c8117891eb89 (8.5.0) - We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`. + Reported-by: iAroc on github + Fixes #12564 + Closes #12567 - Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979 - Ref: #12095 (closed in favour of this patch) - Ref: #11964 (effort to sync cmake detections with autotools) +zengwei2000 (21 Dec 2023) - Reported-by: Kartatz on Github - Assisted-by: Kartatz on Github - Fixes #12093 - Closes #12094 +- altsvc: free 'as' when returning error -- build: add `src/.checksrc` to source tarball + Closes #12570 - Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958 + Signed-off-by: zengwei - Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071 - Reported-by: Romain Geissler - Fixes #12084 - Closes #12085 +Viktor Szakats (20 Dec 2023) -Version 8.4.0 (11 Oct 2023) +- build: fix `-Wconversion`/`-Wsign-conversion` warnings -Daniel Stenberg (11 Oct 2023) + Fix remaining warnings in examples and tests which are not suppressed + by the pragma in `lib/curl_setup.h`. -- RELEASE-NOTES: synced + Silence a toolchain issue causing warnings in `FD_SET()` calls with + older Cygwin/MSYS2 builds. Likely fixed on 2020-08-03 by: + https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=5717262b8ecfed0f7f + ab63e2c09c78991e36f9dd -- THANKS: add contributors from 8.4.0 + Follow-up to 2dbe75bd7f3c36837aa06fd87a442bdf3fb7faef #12492 -Jay Satiro (11 Oct 2023) + Closes #12557 -- socks: return error if hostname too long for remote resolve +- build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings - Prior to this change the state machine attempted to change the remote - resolve to a local resolve if the hostname was longer than 255 - characters. Unfortunately that did not work as intended and caused a - security issue. + - enable `-Wsign-conversion` warnings, but also setting them to not + raise errors. + - fix `-Warith-conversion` warnings seen in CI. + These are triggered by `-Wsign-converion` and causing errors unless + explicitly silenced. It makes more sense to fix them, there just a few + of them. + - fix some `-Wsign-conversion` warnings. + - hide `-Wsign-conversion` warnings with a `#pragma`. + - add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build + basis. + - update a CI job to unhide them with the above macro: + https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3 - Bug: https://curl.se/docs/CVE-2023-38545.html + Closes #12492 -Stefan Eissing (10 Oct 2023) +- cmake: tidy-up `OtherTests.cmake` -- CI: remove slowed-network tests + - make more obvious which detection uses which prep steps. + - merge and streamline conditions. + - these should not alter detection results. - - remove these tests as they are currently not reliable in our CI - setups. + Also align log output messages from + `Macros.cmake` / `curl_internal_test` with rest of the build. - curl handles the test cases, but CI sometimes fails on these due to - additional conditions. Rather than mix them in, an additional CI job - will be added in the future that is specific to them. + Closes #12551 - Closes https://github.com/curl/curl/pull/12075 +- appveyor: switch to out-of-tree builds -Jay Satiro (10 Oct 2023) + With cmake and autotools. -- libcurl-env-dbg.3: move debug variables from libcurl-env.3 + Closes #12550 - - Move documentation of libcurl environment variables used only in debug - builds from libcurl-env into a separate document libcurl-env-dbg. +Daniel Stenberg (19 Dec 2023) - - Document more debug environment variables. +- DEPRECATE.md: mention that NTLM_WB no longer works - Previously undocumented or missing a description: + Ref: #12479 + Closes #12553 - CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, - CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, - CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, - CURL_TIME. +- CURLOPT_SERVER_RESPONSE_TIMEOUT_MS: add - Closes https://github.com/curl/curl/pull/11811 + Proposed-by: Yifei Kong + Ref: https://curl.se/mail/lib-2023-11/0023.html + Closes #12369 -Dan Fandrich (9 Oct 2023) +Viktor Szakats (18 Dec 2023) -- test670: increase the test timeout +- build: more `-Wformat` fixes - This should make it more immune to loaded servers. + - memdebug: update to not trigger `-Wformat-nonliteral` warnings. + - imap: mark `imap_sendf()` with `CURL_PRINTF()`. + - tool_msgs: mark static function with `CURL_PRINTF()`. - Ref: #11328 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 -Stefan Eissing (9 Oct 2023) + Closes #12540 -- MQTT: improve receive of ACKs +- windows: delete redundant headers - - add `mq->recvbuf` to provide buffering of incomplete - ACK responses - - continue ACK reading until sufficient bytes available - - fixes test failures on low network receives + `winsock2.h` pulls in `windows.h`. `ws2tcpip.h` pulls in `winsock2.h`. + `winsock2.h` and `ws2tcpip.h` are also pulled by `curl/curl.h`. - Closes #12071 + Keep only those headers that are not already included, or the code under + it uses something from that specific header. -Viktor Szakats (9 Oct 2023) + Closes #12539 -- quic: fix BoringSSL build +- cmake: prefill/cache `HAVE_STRUCT_SOCKADDR_STORAGE` - Add guard around `SSL_CTX_set_ciphersuites()` use. + Also add missing include to `OtherTests.cmake`. It didn't cause an issue + because the parent already included this earlier by chance. - Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 + Closes #12537 - Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 +Daniel Stenberg (18 Dec 2023) - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes #12067 +- runner.pm: fix perl warning when running tests -Stefan Eissing (9 Oct 2023) + Use of uninitialized value $runner::gdbthis in numeric eq (==) at runner. + pm -- test1540: improve reliability + Follow-up from 3dcf301752a09d9 - - print that bytes have been received on pausing, but not how many + Closes #12549 - Closes #12069 +- runtests: support -gl. Like -g but for lldb. -- test2302: improve reliability + Follow-up to 63b5748 - - make result print collected write data, unless - change in meta flags is detected - - will show same result even when data arrives via - several writecb invocations + Invokes the test case via lldb instead of gdb. Since using gdb is such a + pain on mac, using lldb is sometimes less quirky. - Closes #12068 + Closes #12547 -Daniel Stenberg (9 Oct 2023) +- curl.h: add CURLE_TOO_LARGE -- curl_easy_pause: set "in callback" true on exit if true + A new error code to be used when an internal field grows too large, like + when a dynbuf reaches its maximum. Previously it would return + CURLE_OUT_OF_MEMORY for this, which is highly misleading. - Because it might have called another callback in the mean time that then - set the bit FALSE on exit. + Ref: #12268 + Closes #12269 - Reported-by: Jay Satiro - Fixes #12059 - Closes #12061 +- CI/circleci: disable MQTT in the HTTP-only build -Viktor Szakats (8 Oct 2023) + And remove the use of configure options that don't actually exist -- h3: add support for ngtcp2 with AWS-LC builds + Closes #12546 - ``` - curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- - LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 - Release-Date: [unreleased] - Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps - mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss - Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi - SSL NTLM SSL threadsafe UnixSockets - ``` +Yedaya Katsman (18 Dec 2023) - Also delete an obsolete GnuTLS TODO and update the header comment in - `FindNGTCP2.cmake`. +- tests: respect $TMPDIR when creating unix domain sockets - Reviewed-by: Daniel Stenberg - Closes #12066 + When running on termux, where $TMPDIR isn't /tmp, running the tests + failed, since the server config tried creating sockets in /tmp, without + checking the temp dir config. Use the TMPDIR variable that makes it find + the correct directory everywhere [0] -- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros + [0] https://perldoc.perl.org/File::Temp#tempfile - Syncing this up with CMake. + Closes #12545 - Source code uses the built-in `OPENSSL_IS_AWSLC` and - `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is - necessary from the build tools. +Viktor Szakats (17 Dec 2023) - The one use of `HAVE_BORINGSSL` in the source turned out to be no longer - necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 +- ssh: fix namespace of two local macros - autotools detects this anyway for display purposes. - CMake detects this to decide whether to use the BoringSSL-specific - crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection - result just yet (planned in #12066). + Avoid using the libssh and libssh2 macro namespaces by prefixing + these local macro names with `CURL_`. - Ref: #11964 + Follow-up to 413a0fedd02c8c6df1d294534b8c6e306fcca7a2 #12346 Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12065 - -Marc Hoersken (8 Oct 2023) + Closes #12544 -- CI: move distcheck job from Azure Pipelines to GitHub Actions +- cmake: whitespace tidy-up in `OtherTests.cmake` - This will allow for more trigger excludes within Azure Pipelines. + Closes #12538 - Also fixes seemingly broken check with scripts/installcheck.sh. - Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 +Mark Sinkovics (16 Dec 2023) - Assisted-by: Philip Heiduck - Closes #9532 +- cmake: fix generation for system name iOS -Daniel Stenberg (8 Oct 2023) + This PR fixes a problem that happens during CMake configuration when + the `CMAKE_SYSTEM_NAME` set to `iOS` and not `Darwin`. This value is + available (as far as I remember) version 3.14. The final solution + (thanks to @vszakats) is to use `APPLE` which contains all the Apple + platforms https://cmake.org/cmake/help/latest/variable/APPLE.html. -- url: fall back to http/https proxy env-variable if ws/wss not set + This issue was found when during vcpkg installation. Running command + `vcpkg install curl:arm64-ios` and `vcpkg install curl:x64-ios` failed + with message: + ``` + CMake Error: try_run() invoked in cross-compiling mode, please set the follow + ing cache variables appropriately: + HAVE_H_ERRNO_ASSIGNABLE_EXITCODE (advanced) + ``` + After this fix, I was able to compile the compile the binary without + any issue. - Reported-by: Craig Andrews - Fixes #12031 - Closes #12058 + In addition to that fix, this PR also contains an simplification to + check if the platform is not APPLE. -Stefan Eissing (8 Oct 2023) + Co-authored-by: Viktor Szakats + Closes #12515 -- cf-socket: simulate slow/blocked receives in debug +Daniel Stenberg (16 Dec 2023) - add 2 env variables for non-UDP sockets: - 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly - should return EAGAIN - 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket +- RELEASE-NOTES: synced - Closes #12035 +Baruch Siach (16 Dec 2023) -- http2: refused stream handling for retry +- gnutls: fix build with --disable-verbose - - answer HTTP/2 streams refused via a GOAWAY from the server to - respond with CURLE_RECV_ERROR in order to trigger a retry - on another connection + infof() parameters must be defined event with --disable-verbose since + commit dac293cfb702 ("lib: apache style infof and trace + macros/functions"). - Reported-by: black-desk on github - Ref #11859 - Closes #12054 + Move also 'ptr' definition under !CURL_DISABLE_VERBOSE_STRINGS. -Jay Satiro (8 Oct 2023) + Fixes the following build failure: -- CURLOPT_DEBUGFUNCTION.3: warn about internal handles + In file included from ../lib/sendf.h:29, + from vtls/gtls.c:44: + vtls/gtls.c: In function 'Curl_gtls_verifyserver': + vtls/gtls.c:841:34: error: 'version' undeclared (first use in this function); + did you mean 'session'? + 841 | gnutls_protocol_get_name(version), ptr); + | ^~~~~~~ - - Warn that the user's debug callback may be called with the handle - parameter set to an internal handle. + Closes #12505 - Without this warning the user may assume that the only handles their - debug callback receives are the easy handles on which they set - CURLOPT_DEBUGFUNCTION. +Viktor Szakats (16 Dec 2023) - This is a follow-up to f8cee8cc which changed DoH handles to inherit - the debug callback function set in the user's easy handle. As a result - those handles are now passed to the user's debug callback function. +- build: delete unused `HAVE_{GSSHEIMDAL,GSSMIT,HEIMDAL}` - Closes https://github.com/curl/curl/pull/12034 + Stop setting `HAVE_GSSHEIMDAL`, `HAVE_GSSMIT` and `HAVE_HEIMDAL`. + There was no place in the build system or source code that used them. -- url: fix typo + Reviewed-by: Daniel Stenberg + Closes #12506 -Daniel Stenberg (8 Oct 2023) +- build: remove redundant `CURL_PULL_*` settings -- test458: verify --expand-output, expanding a file name accepting option + These macros were not propagated to the source code from CMake. - Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) + autotools set only one of them (`CURL_PULL_SYS_POLL_H`), initially to + address an AIX issue [1]. This later broke when introducing `system.h` + [2] without the logic it enabled. A subsequent fix [3] re-added the + logic, and also enabled it for AIX before its use, directly in + `system.h`. -- tool_getparam: accept variable expansion on file names too + [1] 2012-11-23: 665adcd4b7bcdb7deb638cdc499fbe71f8d777f2 + [2] 2017-03-29: 9506d01ee50d5908138ebad0fd9fbd39b66bd64d #1373 + [3] 2017-08-25: 8a84fcc4b59e8b78d2acc6febf44a43d6bc81b59 #1828 #1833 - Reported-by: PBudmark on github - Fixes #12048 - Closes #12055 + Reviewed-by: Daniel Stenberg + Closes #12502 -- RELEASE-NOTES: synced +- system.h: sync mingw `CURL_TYPEOF_CURL_SOCKLEN_T` with other compilers -- multi: do CURLM_CALL_MULTI_PERFORM at two more places + Align mingw with the other Windows compilers and use the `int` type for + `CURL_TYPEOF_CURL_SOCKLEN_T` (and thus for `curl_socklent_t`). This + makes it unnecessary to make a mingw-specific trick and pull all Windows + headers early just for this type definition. This type is specific to + Windows, not to the compiler. mingw-w64's Windows header maps it to + `int` too. - ... when it does a state transition but there is no particular socket or - timer activity. This was made apparent when commit b5bb84c removed a - superfluous timer expiry. + With this we also delete all remaining uses of `CURL_PULL_WS2TCPIP_H`. - Reported-by: Dan Fandrich. - Fixes #12033 - Closes #12056 + [ The official solution is to use `socklen_t` for all Windows compilers. + In this case we may want to update `curl/curl.h` to pull in Windows + headers before `system.h`. ] -Viktor Szakats (7 Oct 2023) + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12501 -- GHA/linux: mbedtls 3.5.0 + minor dep bumps +- windows: simplify detecting and using system headers - Closes #12057 + - autotools, cmake: assume that if we detect Windows, `windows.h`, + `winsock2.h` and `ws2tcpip.h` do exist. + - lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of + looking for `winsock2.h`. + - autotools: merge 3 Windows check methods into one. + - move Watt-32 and lwIP socket support to `setup-win32.h` from + `config-win32.h`. It opens up using these with all build tools. Also + merge logic with Windows Sockets. + - fix to assume Windows sockets with the mingw32ce toolchain. + Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4 + - cmake: delete unused variable `signature_call_conv` since + eb33ccd5332435fa50f1758e5debb869c6942b7f. + - autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection. + - examples/externalsocket: fix header order. + - cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue` + that wasn't used anymore. + - cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test + `SIZEOF_STRUCT_SOCKADDR_STORAGE`. -Dan Fandrich (7 Oct 2023) + After this patch curl universally uses `_WIN32` to guard + Windows-specific logic. It guards Windows Sockets-specific logic with + `USE_WINSOCK` (this might need further work). -- CI: bump OpenLDAP package version on FreeBSD + Reviewed-by: Jay Satiro + Closes #12495 - The old one is no longer available. +- build: enable missing OpenSSF-recommended warnings, with fixes -Marc Hoersken (7 Oct 2023) + https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening + -Guide-for-C-and-C++.html + as of 2023-11-29 [1]. -- docs/libcurl/opts/Makefile.inc: add missing manpage files + Enable new recommended warnings (except `-Wsign-conversion`): - Detected with #9532 + - enable `-Wformat=2` for clang (in both cmake and autotools). + - add `CURL_PRINTF()` internal attribute and mark functions accepting + printf arguments with it. This is a copy of existing + `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible + with redefinting the `printf` symbol: + https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 + - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for + mingw-w64 and enable it on this platform. + - enable `-Wimplicit-fallthrough`. + - enable `-Wtrampolines`. + - add `-Wsign-conversion` commented with a FIXME. + - cmake: enable `-pedantic-errors` the way we do it with autotools. + Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 + - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format + checks. Previously it was always disabled due to the internal `printf` + macro. -Dan Fandrich (7 Oct 2023) + Fix them: -- tests: fix a race condition in ftp server disconnect + - fix bug where an `set_ipv6_v6only()` call was missed in builds with + `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. + - add internal `FALLTHROUGH()` macro. + - replace obsolete fall-through comments with `FALLTHROUGH()`. + - fix fallthrough markups: Delete redundant ones (showing up as + warnings in most cases). Add missing ones. Fix indentation. + - silence `-Wformat-nonliteral` warnings with llvm/clang. + - fix one `-Wformat-nonliteral` warning. + - fix new `-Wformat` and `-Wformat-security` warnings. + - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its + definition to `lib/curl_setup.h` allowing use in `tests/server`. + - lib: fix two wrongly passed string arguments in log outputs. + Co-authored-by: Jay Satiro + - fix new `-Wformat` warnings on mingw-w64. - If a client disconnected and reconnected quickly, before the ftp server - had a chance to respond, the protocol message/ack (ping/pong) sequence - got out of sync, causing messages sent to the old client to be delivered - to the new. A disconnect must now be acknowledged and intermediate - requests thrown out until it is, which ensures that such synchronization - problems can't occur. This problem could affect ftp, pop3, imap and smtp - tests. + [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895b + fc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options- + Hardening-Guide-for-C-and-C%2B%2B.md - Fixes #12002 - Closes #12049 + Closes #12489 -Viktor Szakats (7 Oct 2023) +- Makefile.mk: drop Windows support -- appveyor: bump mingw-w64 job to gcc 13 (was: 8) + And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga. - This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). - Adding a modern gcc version to the tests. + We recommend CMake instead. With unity mode it's much faster, and about + the same without. - (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 - finished in 32, 35, 34 minutes in the 3 test runs so far.) + Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 + Reviewed-by: Daniel Stenberg + Closes #12224 - It also adds a modern CMake version and OS env to our mingw-w64 builds. +Daniel Stenberg (16 Dec 2023) - Closes #12051 +- cmdline-docs: use .IP consistently -David Benjamin (6 Oct 2023) + Remove use of .TP and some .B. The idea is to reduce nroff syntax as + much as possible and to use it consistently. Ultimately, we should be + able to introduce our own easier-to-use-and-read syntax/formatting and + convert on generation time. -- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR + Closes #12535 - While the struct is still public in OpenSSL, there is a (somewhat - inconvenient) accessor. Use it to remain compatible if it becomes opaque - in the future. +Tatsuhiko Miyagawa (16 Dec 2023) - Closes #12038 +- http: fix off-by-one error in request method length check -Daniel Stenberg (6 Oct 2023) + It should allow one more byte. -- curl_easy_pause.3: mention it works within callbacks + Closes #12534 - Reported-by: Maxim Dzhura - Bug: https://curl.se/mail/lib-2023-10/0010.html - Closes #12046 +Daniel Stenberg (15 Dec 2023) -- curl_easy_pause.3: mention h2/h3 buffering +- curl: show ipfs and ipns as supported "protocols" - Asked-by: Maxim Dzhura - Ref: https://curl.se/mail/lib-2023-10/0011.html + They are accepted schemes in URLs passed to curl (the tool, not the + library). - Closes #12045 + Also makes curl-config show the same list. -Viktor Szakats (6 Oct 2023) + Co-Authored-by: Jay Satiro + Reported-by: Chara White + Bug: https://curl.se/mail/archive-2023-12/0026.html + Closes #12508 -- cmake: re-add missed C89 headers for specific detections +- Revert "urldata: move async resolver state from easy handle to connectdata" - We removed C89 `setjmp.h` and `signal.h` detections and excluded them - from the global header list we use when detecting functions [1]. Then - missed to re-add these headers to the specific functions which need - them to be detected [2]. Fix this omission in this patch. + This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198) - [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 - [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + We want the c-ares channel to be held in the easy handle, not per + connection - for performance. - Closes #12043 + Closes #12524 -Daniel Stenberg (6 Oct 2023) +Viktor Szakats (15 Dec 2023) -- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE +- openssl: re-match LibreSSL deinit with init - Since there is nothing to wait for there. Avoids the test 1233 hang - reported in #12033. + Earlier we switched to use modern initialization with LibreSSL v2.7.0 + and up, but did not touch deinitialization [1]. Fix it in this patch. - Reported-by: Dan Fandrich - Closes #12042 + Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611 -Dan Fandrich (5 Oct 2023) + [1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014 -- test1903: actually verify the cookies after the test + Reported-by: Mike Hommey + Reviewed-by: Daniel Stenberg + Fixes #12525 + Closes #12526 - The test otherwise could do just about anything (except leak memory in - debug mode) and its bad behaviour wouldn't be detected. Now, check the - resulting cookie file to ensure the cookies are still there. +Daniel Stenberg (14 Dec 2023) - Closes #12041 +- libssh: supress warnings without version check -- test: add missing s + Define unconditionally. - The tests will otherwise fail if curl has them disabled. + Follow-up from d21bd2190c46ad7fa -- test1906: set a lower timeout since it's hit on Windows + Closes #12523 - msys2 builds actually hit the connect timeout in normal operation, so - lower the timeout from 5 minutes to 5 seconds to reduce test time. +- hostip: return error immediately when Curl_ip2addr() fails - Ref: #11328 - Closes #12036 + Closes #12522 -Daniel Stenberg (5 Oct 2023) +Theo (14 Dec 2023) -- RELEASE-NOTES: synced +- libssh: improve the deprecation warning dismissal -Jay Satiro (5 Oct 2023) + Previous code was compiler dependant, and dismissed all deprecation warnings + indiscriminately. -- idn: fix WinIDN null ptr deref on bad host + libssh provides a way to disable the deprecation warnings for libssh only, an + d + naturally this is the preferred way. - - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from - UTF-8 to UTF-16. + This commit uses that, to prevent the erroneous hiding of potential, unrelate + d + deprecation warnings. - Prior to this change a failed conversion erroneously returned CURLE_OK - which meant 'decoded' pointer (what would normally point to the - punycode) would not be written to, remain NULL and be dereferenced - causing an access violation. + Fixes #12519 + Closes #12520 - Closes https://github.com/curl/curl/pull/11983 +Daniel Stenberg (14 Dec 2023) -Dan Fandrich (4 Oct 2023) +- test1474: removed -- tests: close the shell used to start sshd + The test was already somewhat flaky and disabled on several platforms, + and after 1da640abb688 even more unstable. - This shell isn't needed once sshd starts, so use "exec" so it doesn't - stick around. +- readwrite_data: loop less - Closes #12032 + This function is made to loop in order to drain incoming data + faster. Completely removing the loop has a measerably negative impact on + transfer speeds. -Daniel Stenberg (4 Oct 2023) + Downsides with the looping include -- base64: also build for curl + - it might call the progress callback much more seldom. Especially if + the write callback is slow. - Since the tool itself now uses the base64 code using the curlx way, it - needs to build also when the tool needs it. Starting now, the tool build - defines BULDING_CURL to allow lib-side code to use it. + - rate limiting becomes less exact - Follow-up to 2e160c9c6525 + - a single transfer might "starve out" other parallel transfers - Closes #12010 + - QUIC timers for other connections can't be maintained correctly -Eduard Strehlau (4 Oct 2023) + The long term fix should be to remove the loop and optimize coming back + to avoid the transfer speed penalty. -- tests: Fix zombie processes left behind by FTP tests. + This fix lower the max loop count to reduce the starvation problem, and + avoids the loop completely for when rate-limiting is in progress. - ftpserver.pl correctly cleans up spawned server processes, - but forgets to wait for the shell used to spawn them. - This is barely noticeable during a normal testrun, - but causes process exhaustion and test failure - during a complete torture run of the FTP tests. + Ref: #12488 + Ref: https://curl.se/mail/lib-2023-12/0012.html + Closes #12504 - Fixes #12018 - Closes #12020 +Stefan Eissing (14 Dec 2023) -Dan Fandrich (4 Oct 2023) +- lib: eliminate `conn->cselect_bits` -- github/labeler: improve labeler matches + - use `data->state.dselect_bits` everywhere instead + - remove `bool *comeback` parameter as non-zero + `data->state.dselect_bits` will indicate that IO is + incomplete. -- test574: add a timeout to the test + Closes #12512 - This one hangs occasionally, so this will speed up a test run and allow - logs to be seen when it does. +- connect: refactor `Curl_timeleft()` - Closes #12025 + - less local vars, "better" readability + - added documentation -- tests: propagate errors in libtests + Closes #12518 - Use the test macros to automatically propagate some errors, and check - and log others while running the tests. This can help in debugging - exactly why a test has failed. +Dmitry Karpov (14 Dec 2023) -- tests: set --expect100-timeout to improve test reliability +- cookie: avoid fopen with empty file name - On an overloaded server, the default 1 second timeout can go by without - the test server having a chance to respond with the expected headers, - causing tests to fail. Increase the 1 second timeout to 99 seconds so - this failure mode is no longer a problem on test 1129. Some other tests - already set a high value, but make them consistently 99 seconds so if - something goes wrong the test is stalled for less time. + Closes #12514 - Ref: #11328 +Viktor Szakats (13 Dec 2023) -- CI: ignore the "flaky" and "timing-dependent" test results in CMake +- tests/server: delete workaround for old-mingw - This was already done for automake builds but CMake builds were missed. - Test 1086 actually causes the test harness to crash with: + mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this. - Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec - ts/curl/tests/ftpserver.pl line 527 + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 - Rather than fix it now, this change leaves test 1086 entirely skipped on - those builds that show this problem. + Reviewed-by: Jay Satiro + Closes #12510 - Follow-up to 589dca761 +- cmake: delete obsolete TODOs more [ci skip] - Ref: #11865 + - manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288 + - soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023 + - bunch of others that are completed + - `NTLM_WB_ENABLED` is implemented in a basic form, and now also + scheduled for removal, so a TODO at this point isn't useful. -Viktor Szakats (4 Oct 2023) + And this 'to-check' item: -- cmake: improve OpenLDAP builds + Q: "The cmake build selected to run gcc with -fPIC on my box while the + plain configure script did not." - - cmake: detect OpenLDAP based on function `ldap_init_fd`. - autotools does this. autotools also publishes this detection result - in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source - doesn't use this value. (it might need to be remove-listed in - `scripts/cmp-config.pl` for future OpenLDAP test builds.) - This also deletes existing self-declaration method via the - CMake-specific `CURL_USE_OPENLDAP` configuration. + A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 + and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly + enable PIC for libcurl shared lib. Or when building libcurl for + shared and static lib in a single pass. We do this by default for + Windows or when enabled by the user via `SHARE_LIB_OBJECT`. + Otherwise we don't touch this setting. Meaning the default set by + CMake (if any) or the toolchain is used. On Debian Bookworm, this + means that PIC is disabled for static libs by default. Some platforms + (like macOS), has PIC enabled by default. + autotools supports the double-pass mode only, and in that case + CMake seems to match PIC behaviour now (as tested on Linux with gcc.) - - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. - Like autotools does. This fixes a long list of these warnings: - ``` - /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i - s not defined, evaluates to 0 [-Wundef] - ``` + Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500 - - cmake: delete LDAP TODO comment no longer relevant. + Reviewed-by: Jay Satiro + Closes #12509 - Also: +Stefan Eissing (12 Dec 2023) - - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature - detection functions. +- CLIENT-WRITERS: design and use documentation - Ref: #11964 (effort to sync cmake detections with autotools) + Closes #12507 - Closes #12024 +Viktor Szakats (12 Dec 2023) -- cmake: fix unity builds for more build combinations +- cmake: delete obsolete TODO items [ci skip] - By using unique static function/variable names in source files - implementing these interfaces. + There is always room for improvement, but CMake is up to par now with + autotools, so there is no longer a good reason to keep around these + inline TODO items. - - OpenLDAP combined with any SSH backend. + Answering one of questions: - - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. + Q: "The gcc command line use neither -g nor any -O options. As a + developer, I also treasure our configure scripts's --enable-debug + option that sets a long range of "picky" compiler options." - Closes #12027 + A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info + and optimization level. E.g.: + - `Release` = `-O3` + no debug info + - `MinSizeRel` = `-Os` + no debug info + - `Debug` = `-O0` + debug info -Daniel Stenberg (4 Oct 2023) + https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-deb + ug-release-relwithdebinfo-and-minsizerel/59314670#59314670 + https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#defaul + t-and-custom-configurations -- tests: remove leading spaces from some tags + For picky warnings we have the `PICKY_COMPILER` options, enabled by + default. - The threee tags ``, `` and `` were frequently used - with a leading space that this removes. The reason this habbit is so - widespread in testcases is probably that they have been copy and pasted. + Closes #12500 - Hence, fixing them all now might curb this practice from now on. +Stefan Eissing (11 Dec 2023) - Closes #12028 +- CONNECTION-FILTERS: update documentation -Viktor Szakats (4 Oct 2023) + Closes #12497 -- GHA: bump actions/checkout +Daniel Stenberg (11 Dec 2023) - Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 - Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 +- lib: reduce use of strncpy - Closes #12023 + - bearssl: select cipher without buffer copies + - http_aws_sigv4: avoid strncpy, require exact timestamp length + - http_aws_sigv4: use memcpy isntead of strncpy + - openssl: avoid strncpy calls + - schannel: check for 1.3 algos without buffer copies + - strerror: avoid strncpy calls + - telnet: avoid strncpy, return error on too long inputs + - vtls: avoid strncpy in multissl_version() -- spelling: fix codespell 2.2.6 typos + Closes #12499 - Closes #12019 +- CI/distcheck: run full tests -Daniel Stenberg (3 Oct 2023) + To be able to detect missing files better, this now runs the full CI + test suite. If done before, it would have detected #12462 before + release. -- GHA: add workflow to compare configure vs cmake outputs + Closes #12503 - Uses scripts/cmp-config.pl two compare two curl_config.h files, - presumbly generated with configure and cmake. It displays the - differences and filters out a lot of known lines we ignore. +- docs: clean up Protocols: for cmdline options - The script also shows the matches that were *not* used. Possibly - subjects for removal. + ... and some other minor polish. - Closes #11964 + Closes #12496 -- appveyor: enable test 571 +- cmdline/gen: fix the sorting of the man page options - Follow-up from 8a940fd55c175f7 / #12013 + They were previously sorted based on the file names, which use a .d + extension, making "data" get placed after "data-binary" etc. Making the + sort ignore the extention fixes the ordering. - Closes #12017 + Reported-by: Boris Verkhovskiy + Bug: https://curl.se/mail/archive-2023-12/0014.html + Closes #12494 -Viktor Szakats (3 Oct 2023) +Daniel Gustafsson (9 Dec 2023) -- build: alpha-sort source files for lib and src +- doh: remove unused local variable - Closes #12014 + The nurl variable is no longer used during probing following + a refactoring, so remove. -- cmake: delete old `HAVE_LDAP_URL_PARSE` logic + Closes #12491 - Left there by accident after adding proper detection for this. +Jay Satiro (8 Dec 2023) - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 +- build: fix Windows ADDRESS_FAMILY detection - Ref: #11964 (effort to sync cmake detections with autotools) + - Include winsock2.h for Windows ADDRESS_FAMILY detection. - Closes #12015 + Prior to this change cmake detection didn't work because it included + ws2def.h by itself, which is missing needed types from winsock2.h. -Stefan Eissing (3 Oct 2023) + Prior to this change autotools detection didn't work because it did not + include any Windows header. -- tests: increase lib571 timeout from 3s to 30s + In both cases libcurl would fall back on unsigned short as the address + family type, which is the same as ADDRESS_FAMILY. - - 3s is too short for our CI, making this test fail occasionally - - test usually experiences no delay run locally, so 30s wont hurt + Co-authored-by: Viktor Szakats - Closes #12013 + Closes https://github.com/curl/curl/pull/12441 -Viktor Szakats (3 Oct 2023) +Daniel Stenberg (8 Dec 2023) -- cmake: fix unity with Windows Unicode + TrackMemory +- lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding - Found the root cause of the startup crash in unity builds with Unicode - and TrackMemory enabled at the same time. + Since the copy does not stop at a null byte, let's not call it anything + that makes you think it works like the common strndup() function. - We must make sure that the `memdebug.h` header doesn't apply to - `lib/curl_multibyte.c` (as even noted in a comment there.) In unity - builds all headers apply to all sources, including `curl_multibyte.c`. - This probably resulted in an infinite loop on startup. + Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat - Exclude this source from unity compilation with TrackMemory enabled, - in both libcurl and curl tool. Enable unity mode for a debug Unicode - CI job to keep it tested. Also delete the earlier workaround that - fully disabled unity for affected builds. + Closes #12490 - Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 +- convsrctest.pl: removed: not used, not shipped in tarballs - Closes #11928 +- tests: rename tests scripts to the test number -- cmake: disable unity mode with Windows Unicode + TrackMemory + It is hard to name the scripts sensibly. Lots of them are similarly + named and the name did not tell which test that used them. - "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, - aka `-DCURLDEBUG`). + The new approach is rather to name them based on the test number that + runs them. Also helps us see which scripts are for individual tests + rather than for general test infra. - There is an issue with memory tracking and Unicode when built in "unity" - mode, which results in the curl tool crashing right on startup, even - without any command-line option. Interestingly this doesn't happen under - WINE (at least on the system I tested this on), but consistenly happens - on real Windows machines. Crash is 0xC0000374 heap corruption. Both - shared and static curl executables are affected. + - badsymbols.pl -> test1167.pl + - check-deprecated.pl -> test1222.pl + - check-translatable-options.pl -> test1544.pl + - disable-scan.pl -> test1165.pl + - error-codes.pl -> test1175.pl + - errorcodes.pl -> test1477.pl + - extern-scan.pl -> test1135.pl + - manpage-scan.pl -> test1139.pl + - manpage-syntax.pl -> test1173.pl + - markdown-uppercase.pl -> test1275.pl + - mem-include-scan.pl -> test1132.pl + - nroff-scan.pl -> test1140.pl + - option-check.pl -> test1276.pl + - options-scan.pl -> test971.pl + - symbol-scan.pl -> test1119.pl + - version-scan.pl -> test1177.pl - This limitation probably won't hit too many people, but it remains - a TODO to find and fix the root cause and drop this workaround. + Closes #12487 - Example builds and runs: - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd - 7iwj#L313 (static) - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby - qu9c#L317 (shared) +Michał Antoniak (8 Dec 2023) - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 +- sendf: fix compiler warning with CURL_DISABLE_HEADERS_API - Ref: #11928 - Closes #12005 + fix MSVC warning C4189: 'htype': local variable is initialized but not + referenced - when CURL_DISABLE_HEADERS_API is defined. -- cmake: tidy-up `NOT_NEED_LBER_H` detection + Closes #12485 - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 +Viktor Szakats (8 Dec 2023) -- appveyor: rewrite batch in PowerShell + CI improvements +- tidy-up: whitespace - 1. Rewrite in PowerShell: + Closes #12484 - - rewrite MS-DOS batch build script in PowerShell. - - move some bash operations into native PowerShell. - - fixups for PowerShell insisting on failure when a command outputs - something to stderr. - - fix to actually run `curl -V` after every build. - (and exclude ARM64 builds.) - - also say why we skipped `curl -V` if we had to skip. - - fix CMake warnings about unused configuration variables, by adapting - these dynamically for build cases. - - dedupe OpenSSL path into a variable. - - disable `test1451` failing with a warning anyway due to missing python - impacket. (after trying and failing to install impacket) - PowerShell promotes these warnings to errors by PowerShell. We can also - suppress they wholesale if they start causing issues in the future, - like we already to with `autoreconf` and `./configure`. +Stefan Eissing (7 Dec 2023) - PowerShell is better than MS-DOS batches, so the hope is this makes it - easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't - supported inline by AppVeyor on Windows build machines, but we are okay - to keep it in an external script, so it's also an option. +- test_02_download: fix paramters to test_02_27 - 2. CI improvements: + - it is a special client that only ever uses http/2 - - enable tests for a "unity" build job. - - speed-up CI initialization by using shallow clones of the curl repo. - - speed-up CMake MSVC jobs with `TrackFileAccess=false`. - - enable parallelism in `VisualStudioSolution` builds. - - display CMake version before builds. - - always show the CPU in job names. - - tell which jobs are build-only in job names. - - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. - - add `config.log` (autotools) to dumped logs (need to enable manually). + Closes #12467 - 3. Style: +Michał Antoniak (7 Dec 2023) - - use single-quotes in YAML like we do in other CI YAML files. - It also allows to drop quoting characters and lighter to write/read. - (keep double quotes for PowerShell strings needing expansion.) +- vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY - Closes #11999 + Closes #12459 -- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows +Daniel Stenberg (7 Dec 2023) - - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. - Before this patch we set it based it on the presence of `stricmp`, - which correctly enabled it on e.g. Windows, but was inaccurate for - other platforms. +- lib: strndup/memdup instead of malloc, memcpy and null-terminate - - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and - LDAPS is not explicitly disabled. This mimics autotools behaviour. - Previously we set it only for Windows LDAP. After this fix, LDAPS is - correctly enabled in default macOS builds. + - bufref: use strndup + - cookie: use strndup + - formdata: use strndup + - ftp: use strndup + - gtls: use aprintf instead of malloc + strcpy * 2 + - http: use strndup + - mbedtls: use strndup + - md4: use memdup + - ntlm: use memdup + - ntlm_sspi: use strndup + - pingpong: use memdup + - rtsp: use strndup instead of malloc, memcpy and null-terminate + - sectransp: use strndup + - socks_gssapi.c: use memdup + - vtls: use dynbuf instead of malloc, snprintf and memcpy + - vtls: use strdup instead of malloc + memcpy + - wolfssh: use strndup - - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) - to avoid deprecation warnings for LDAP API. + Closes #12453 - - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. - This doesn't make much sense, but let's do it to sync behaviour with - autotools. +- strdup: remove the memchr check from Curl_strndup - - fix benign typo in variable name. + It makes it possible to clone a binary chunk of data. - Ref: #11964 (effort to sync cmake detections with autotools) + Closes #12453 - Closes #12006 +- ftp: handle the PORT parsing without allocation -- autotools: restore `HAVE_IOCTL_*` detections + Also reduces amount of *cpy() calls. - This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in - 4d73854462f30948acab12984b611e9e33ee41e6 and - c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the - `HAVE_IOCTL` result it generated was unused in the source. But, - I did miss the fact that this had two dependent checks: - `CURL_CHECK_FUNC_IOCTL_FIONBIO`, - `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: - `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. + Closes #12456 - Regression from 4d73854462f30948acab12984b611e9e33ee41e6 +- RELEASE-NOTES: synced - Ref: #11964 (effort to sync cmake detections with autotools) + Bumped to 8.5.1 - Closes #12008 +- url: for disabled protocols, mention if found in redirect -Daniel Stenberg (2 Oct 2023) + To help users better understand where the URL (and denied scheme) comes + from. Also removed "in libcurl" from the message, since the disabling + can be done by the application. -- RELEASE-PROCEDURE.md: updated coming release dates + The error message now says "not supported" or "disabled" depending on + why it was denied: -- RELEASE-NOTES: synced + Protocol "hej" not supported + Protocol "http" disabled -Viktor Szakats (1 Oct 2023) + And in redirects: -- cmake: pre-cache `HAVE_POLL_FINE` on Windows + Protocol "hej" not supported (in redirect) + Protocol "http" disabled (in redirect) - Windows doesn't support `poll()`, so we can safely skip checking for - fine poll. + Reported-by: Mauricio Scheffer + Fixes #12465 + Closes #12469 - Closes #12003 +Stefan Eissing (6 Dec 2023) -- gha: bump actions to latest versions +- sectransp_ make TLSCipherNameForNumber() available in non-verbose config - - actions@checkout@v4 (from v3 and v2) + Reported-by: Cajus Pollmeier + Closes #12476 + Fixes #12474 - - fsfe/reuse-action@v2 (from v1) +YX Hao (6 Dec 2023) - Closes #12000 +- lib: fix variable undeclared error caused by `infof` changes -Stefan Eissing (30 Sep 2023) + `--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined. + `infof` isn't `Curl_nop_stmt` anymore: dac293c. -- h2: testcase and fix for pausing h2 streams + Follow-up to dac293c - - refs #11982 where it was noted that paused transfers may - close successfully without delivering the complete data - - made sample poc into tests/http/client/h2-pausing.c and - added test_02_27 to reproduce + Closes #12470 - Closes #11989 - Fixes #11982 - Reported-by: Harry Sintonen +Viktor Szakats (6 Dec 2023) -Viktor Szakats (30 Sep 2023) +- tidy-up: fix yamllint whitespace issues in labeler.yml -- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value + Follow-up to bda212911457c6fadfbba50be61afc4ca513fa56 #12466 - Before this patch CMake builds accepted any value and it was used at - runtime as-is. This patch make sure that the selected default backend - is also enabled in the build. It also enforces a full lowercase value. + Reviewed-by: Dan Fandrich + Closes #12475 - This improves reproducibility and brings CMake in sync with autotools - which already worked like described above. +- tidy-up: fix yamllint whitespace issues - Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 + Closes #12466 - Closes #11998 +Chris Sauer (6 Dec 2023) -- autotools: adjust `CURL_CA_PATH` value to CMake +- cmake: fix typo - autotools was using the same value as CMake, but with an ending - slash. Delete the ending slash to match configurations. + Follow-up to aace27b + Closes #12464 - Ref: #11964 (effort to sync cmake detections with autotools) +Daniel Stenberg (6 Dec 2023) - Closes #11997 +- dist: add tests/errorcodes.pl to the tarball -- cmake: detect `sys/wait.h` and `netinet/udp.h` + Used by test 1477 - Ref: #11964 (effort to sync cmake detections with autotools) + Reported-by: Xi Ruoyao + Follow-up to 0ca3a4ec9a7 + Fixes #12462 + Closes #12463 - Closes #11996 +Dan Fandrich (6 Dec 2023) -Daniel Stenberg (30 Sep 2023) +- github/labeler: update a missed key in the v5 upgrade -- lib: provide and use Curl_hexencode + Follow-up to ce03fe3ba - Generates a lower case ASCII hex output from a binary input. +Version 8.5.0 (6 Dec 2023) - Closes #11990 +Daniel Stenberg (6 Dec 2023) -- configure: check for the capath by default +- RELEASE-NOTES: synced - ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS - L + The curl 8.5.0 release. - cmake: synced +Dan Fandrich (5 Dec 2023) - Assisted-by: Viktor Szakats - Closes #11987 +- github/labeler: switch from the beta to labeler v5 -- wolfssl: ignore errors in CA path + Some keys were renamed and the dot option was made default. - The default wolfSSL_CTX_load_verify_locations() function is quite picky - with the certificates it loads and will for example return error if just - one of the certs has expired. + Closes #12458 - With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it - behaves more similar to what OpenSSL does by default. +Daniel Stenberg (5 Dec 2023) - Even the set of default certs on my Debian unstable has several expired - ones. +- DEPRECATE: remove NTLM_WB in June 2024 - Assisted-by: Juliusz Sosinowicz - Assisted-by: Michael Osipov + Ref: https://curl.se/mail/lib-2023-12/0010.html - Closes #11987 + Closes #12451 -- create-dirs.d: clarify it also uses --output-dirs +Jacob Hoffman-Andrews (4 Dec 2023) - Reported-by: Robert Simpson - Fixes #11991 - Closes #11995 +- rustls: implement connect_blocking -Viktor Szakats (30 Sep 2023) + Closes #11647 -- appveyor: fix yamlint issues, indent +Daniel Stenberg (4 Dec 2023) - Also: - - use double quotes in all batch if statements. +- examples/rtsp-options.c: add - Closes #11994 + Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and + CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. -- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` + Closes #12452 - Based on existing autotools logic. +Stefan Eissing (4 Dec 2023) - Ref: #11964 (effort to sync cmake detections with autotools) +- ngtcp2: ignore errors on unknown streams - Closes #11981 + - expecially in is_alive checks on connections, we might + see incoming packets on streams already forgotten and closed, + leading to errors reported by nghttp3. Ignore those. -- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` + Closes #12449 - Based on existing autotools logic. +Daniel Stenberg (4 Dec 2023) - autotools checks for old versions of the allowlisted target OSes and - disables this feature when seeing them. In CMake we assume we're running - on newer systems and enable regardless of OS version. +- docs: make all examples in all libcurl man pages compile - autotools always runs all 3 probes for non-fast-tracked systems and - enables this feature if any one of them was successful. To save - configuration time, CMake stops at the first successful check. + Closes #12448 - OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD - system. I haven't double-checked if this is correct, but looks odd. +- checksrc.pl: support #line instructions - Ref: #11964 (effort to sync cmake detections with autotools) + makes it identify the correct source file and line - Closes #11979 +- GHA/man-examples: verify libcurl man page examples -- cmake: fix `HAVE_WRITABLE_ARGV` detection +- verify-examples.pl: verify that all man page examples compile clean - Move detection before the creation of detection results in - `curl_config.h`. +- RELEASE-NOTES: synced - Ref: #11964 (effort to sync cmake detections with autotools) +Graham Campbell (2 Dec 2023) - Closes #11978 +- http3: bump ngtcp2 and nghttp3 versions -- appveyor: minor improvements + nghttp3 v1.1.0 + ngtcp2 v1.1.0 - - run `curl -V` after builds to see if they run and with what features. - Except for one job where a CRT DLL is missing. And ARM64 which should - fail, but is silently not launched instead. + In docs and CI - - copy libcurl DLL next to curl tool and tests binaries in shared mode. - This makes it possible to run the tests. (We don't run tests after - these builds yet.) + Closes #12446 - - list the DLLs and EXEs present after the builds. +- CI/quiche: use `3.1.4+quic` consistently in CI workflows - - add `DEBUG` variable for CMake builds to allow disabling it, for - testing non-debug builds. (currently enabled for all) + Closes #12447 - - add commented lines that dump CMake configuration logs for debugging - build/auto-detection issues. +Viktor Szakats (2 Dec 2023) - - add gcc version to jobs where missing. +- test1545: disable deprecation warnings - - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 - to the build mix. + Fixes: + https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr + p6pk#L1205 - - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. + Same with details: + https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil + b6wt#L1263 + ``` + tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 + .0. Use curl_mime_init() [-Werror=deprecated-declarations] + 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", + | ^~~~~~~~~~~~ + [...] + ``` - - delete a duplicate backslash. + Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 - Closes #11976 + Fixes #12445 + Closes #12444 -- configure: replace adhoc domain with `localhost` in tests +Daniel Stenberg (2 Dec 2023) - Reviewed-by: Daniel Stenberg - Closes #11988 +- INSTALL: update list of ports and CPU archs -- tidy-up: use more example domains +- symbols-in-versions: the CLOSEPOLICY options are deprecated - Also make use of the example TLD: - https://en.wikipedia.org/wiki/.example + The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. - Reviewed-by: Daniel Stenberg - Closes #11992 +z2_ (1 Dec 2023) -Dan Fandrich (29 Sep 2023) +- build: fix builds that disable protocols but not digest auth -- runtests: display the test status if tests appear hung + - Build base64 functions if digest auth is not disabled. - It sometimes happens that a test hangs during a test run and never - returns. The test harness will wait indefinitely for the results and on - CI servers the CI job will eventually be killed after an hour or two. - At the end of a test run, if results haven't come in within a couple of - minutes, display the status of all test runners and what tests they're - running to help in debugging the problem. + Prior to this change if some protocols were disabled but not digest auth + then a build error would occur due to missing base64 functions. - This feature is really only kick in with parallel testing enabled, which - is fine because without parallel testing it's usually easy to tell what - test has hung. + Fixes https://github.com/curl/curl/issues/12440 + Closes https://github.com/curl/curl/pull/12442 - Closes #11980 +Michał Antoniak (1 Dec 2023) -- github/labeler: remove workaround for labeler +- connect: reduce number of transportation providers - This was added due to what seemed to be a bug regarding the sync-labels: - config option, but it looks like it wasn't necessary. + Use only the ones necessary - the ones that are built-in. Saves a few + bytes in the resulting code. - Follow-up to b2b0534e7 + Closes #12438 -Viktor Szakats (29 Sep 2023) +David Benjamin (1 Dec 2023) -- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] +- vtls: consistently use typedef names for OpenSSL structs -Daniel Stenberg (29 Sep 2023) + The foo_st names don't appear in OpenSSL public API documentation. The + FOO typedefs are more common. This header was already referencing + SSL_CTX via . There is a comment about avoiding + , but OpenSSL actually declares all the typedefs in + , which is already included by (and + every other OpenSSL header), so just use that. Though I've included it + just to be explicit. -- docs: replace made up domains with example.com + (I'm also fairly sure including already triggers the + Schannel conflicts anyway. The comment was probably just out of date.) - in FAQ and MANUAL.md + Closes #12439 - - example.com was made for this purpose. +Lau (1 Dec 2023) - - reduces the risk that one of those domains suddenly start hosting - something nasty and we provide links to them +- libcurl-security.3: fix typo - Closes #11986 + Fixed minimal typo. -Michael Osipov (29 Sep 2023) + Closes #12437 -- acinclude.m4: Document proper system truststore on FreeBSD +Stefan Eissing (1 Dec 2023) - The default system truststore on FreeBSD has been /etc/ssl/certs for many - years now. It is managed canonically through certctl(8) and contains hashed - symlinks for OpenSSL and other TLS providers. - The previous ones require security/ca_root_nss which might not be installed o - r - will not contain any custom CA certificates. +- ngtcp2: fix races in stream handling - Closes #11985 + - fix cases where ngtcp2 invokes callbacks on streams that + nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND + in these cases as it is normal behaviour. -Daniel Stenberg (29 Sep 2023) + Closes #12435 -- FAQ: How do I upgrade curl.exe in Windows? +Emanuele Torre (1 Dec 2023) - This is a growing question, better answer it here to get somewhere to - point users to. +- tool_writeout_json: fix JSON encoding of non-ascii bytes - Closes #11984 + char variables if unspecified can be either signed or unsigned depending + on the platform according to the C standard; in most platforms, they are + signed. -Viktor Szakats (28 Sep 2023) + This meant that the *i<32 waas always true for bytes with the top bit + set. So they were always getting encoded as \uXXXX, and then since they + were also signed negative, they were getting extended with 1s causing + '\xe2' to be expanded to \uffffffe2, for example: -- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC + $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul + l + \uffffffe2\uffffff80\uffffff9c - `basename` is present in mingw-w64, missing from MSVC. Pre-cache - accordingly to make configure faster. + I fixed this bug by making the code use explicitly unsigned char* + variables instead of char* variables. - Notice that `basename` has a bug so we later disable it even with - mingw-w64: - https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li - b/curl_setup.h#L820-L825 + Test 268 verifies - Closes #11974 + Reported-by: iconoclasthero + Closes #12434 -Daniel Stenberg (28 Sep 2023) +Stefan Eissing (1 Dec 2023) -- cmake: add missing checks +- cf-socket: TCP trace output local address used in connect - - check for arc4random. To make rand.c use it accordingly. - - check for fcntl - - fix fseek detection - - add SIZEOF_CURL_SOCKET_T - - fix USE_UNIX_SOCKETS - - define HAVE_SNPRINTF to 1 - - check for fnmatch - - check for sched_yield - - remove HAVE_GETPPID duplicate from curl_config.h - - add HAVE_SENDMSG + Closes #12427 - Ref: #11964 +Jay Satiro (1 Dec 2023) - Co-authored-by: Viktor Szakats - Closes #11973 +- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation -- configure: remove unused checks + - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it + includes protocol-specific instructions that trigger a transfer. - - for sys/uio.h - - for fork - - for connect + Prior to this change it explicitly said that it did not include those + instructions in the time, but that is incorrect. - Ref: #11964 + The change is a copy of the fixed explanation already in + CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. - Closes #11973 + Reported-by: eeverettrbx@users.noreply.github.com -- lib: remove TIME_WITH_SYS_TIME + Fixes https://github.com/curl/curl/issues/12431 + Closes https://github.com/curl/curl/pull/12432 - It is not used in any code anywhere. +Daniel Stenberg (30 Nov 2023) - Ref: #11964 - Closes #11975 +- multi: during ratelimit multi_getsock should return no sockets -- docs: update curl man page references + ... as there is nothing to wait for then, it just waits. Otherwise, this + causes much more CPU work and updates than necessary during ratelimit + periods. - Detected by the manpage-syntax update + Ref: https://curl.se/mail/lib-2023-11/0056.html + Closes #12430 - Closes #11963 +Dmitry Karpov (30 Nov 2023) -- manpage-syntax: verify curl man page references +- transfer: abort pause send when connection is marked for closing - 1. References to curl symbols are now checked that they indeed exist as - man pages. This for \f references as well as the names referenced in the - SEE ALSO section. + This handles cases of some bi-directional "upgrade" scenarios + (i.e. WebSockets) where sending is paused until some "upgrade" handshake + is completed, but server rejects the handshake and closes the + connection. - Allowlist curl.1 since it is not always built in builds + Closes #12428 - 2. References to curl symbols that lack section now causes warning, since tha - t - will prevent them from getting linked properly +Daniel Stenberg (28 Nov 2023) - 3. Check for "bare" references to curl functions and warn, they should be - references +- RELEASE-NOTES: synced - Closes #11963 +- openssl: when a session-ID is reused, skip OCSP stapling -- cmake: add check for suseconds_t + Fixes #12399 + Reported-by: Alexey Larikov + Closes #12418 - And fix the HAVE_LONGLONG define +- test1545: test doing curl_formadd twice with missing file - Ref: #11964 - Closes #11977 + Reproduces #12410 + Verifies the fix + Closes #12421 -Viktor Szakats (28 Sep 2023) +- Curl_http_body: cleanup properly when Curl_getformdata errors -- tidy-up: whitespace fixes + Reported-by: yushicheng7788 on github + Based-on-work-by: yushicheng7788 on github + Fixes #12410 + Closes #12421 - Closes #11972 +- test1477: verify that libcurl-errors.3 and public headers are synced -- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS + The script errorcodes.pl extracts all error codes from all headers and + checks that they are all documented, then checks that all documented + error codes are also specified in a header file. - With new option `CURL_DISABLE_SRP=ON` to force-disable it. - To match existing option and detection logic in autotools. + Closes #12424 - Also: - - fix detecting GnuTLS. - We assume `nettle` as a GnuTLS dependency. - - add CMake GnuTLS CI job. - - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) - TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. - - fix compiler warning when building with GnuTLS and disabled TLS-SRP. - - fix comment typos, whitespace. +- libcurl-errors.3: sync with current public headers - Ref: #11964 + Closes #12424 - Closes #11967 +Stefan Eissing (28 Nov 2023) -- tool: use our own stderr variable +- test459: fix for parallel runs - Earlier this year we changed our own stderr variable to use the standard - name `stderr` (to avoid bugs where someone is using `stderr` instead of - the curl-tool specific variable). This solution needed to override the - standard `stderr` symbol via the preprocessor. This in turn didn't play - well with unity builds and caused curl tool to crash or stay silent due - to an uninitialized stderr. This was a hard to find issue, fixed by - manually breaking out one file from the unity sources. + - change warniing message to work better with varying filename + length. + - adapt test output check to new formatting - To avoid two these two tricks, this patch implements a different - solution: Restore using our own local variable for our stderr output and - leave `stderr` as-is. To avoid using `stderr` by mistake, add a - `checksrc` rule (based on logic we already used in lib for `strerror`) - that detects any `stderr` use in `src` and points to using our own - variable instead: `tool_stderr`. + Follow-up to 97ccc4479f77ba3191c6 + Closes #12423 - Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e - Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db +Daniel Stenberg (27 Nov 2023) - Closes #11958 +- tool_cb_prg: make the carriage return fit for wide progress bars -Loïc Yhuel (28 Sep 2023) + When the progress bar was made max width (256 columns), the fly() + function attempted to generate its output buffer too long so that the + trailing carriage return would not fit and then the output would show + wrongly. The fly function is called when the expected total transfer is + unknown, which could be one or more progress calls before the actual + progress meter get shown when the expected transfer size is provided. -- connect: only start the happy eyeballs timer when needed + This new take also replaces the msnprintf() call with a much simpler + memset() for speed. - The timeout is only used when there is a second address family, for the - delayed eyeballer. + Reported-by: Tim Hill + Fixes #12407 + Closes #12415 - Closes #11939 +- tool_parsecfg: make warning output propose double-quoting -Daniel Stenberg (28 Sep 2023) + When the config file parser detects a word that *probably* should be + quoted, mention double-quotes as a possible remedy. -- tool_operate: free 'gateway' correctly + Test 459 verifies. - Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. + Proposed-by: Jiehong on github + Fixes #12409 + Closes #12412 - Also removed repeated wording in IPFS related error messages. +Jay Satiro (26 Nov 2023) - Closes #11969 +- curl.rc: switch out the copyright symbol for plain ASCII -Stefan Eissing (28 Sep 2023) + .. like we already do for libcurl.rc. -- lib: move handling of `data->req.writer_stack` into Curl_client_write() + libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" + warning so it was switched to ascii. - - move definitions from content_encoding.h to sendf.h - - move create/cleanup/add code into sendf.c - - installed content_encoding writers will always be called - on Curl_client_write(CLIENTWRITE_BODY) - - Curl_client_cleanup() frees writers and tempbuffers from - paused transfers, irregardless of protocol + Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 - Closes #11908 + Suggested-by: Robert Southee -Loïc Yhuel (28 Sep 2023) + Closes https://github.com/curl/curl/pull/12403 -- multi: round the timeout up to prevent early wakeups +Daniel Stenberg (26 Nov 2023) - Curl_timediff rounds down to the millisecond, so curl_multi_perform can - be called too early, then we get a timeout of 0 and call it again. +- conncache: use the closure handle when disconnecting surplus connections - The code already handled the case of timeouts which expired less than - 1ms in the future. By rounding up, we make sure we will never ask the - platform to wake up too early. + Use the closure handle for disconnecting connection cache entries so + that anything that happens during the disconnect is not stored and + associated with the 'data' handle which already just finished a transfer + and it is important that details from the unrelated disconnect does not + taint meta-data in the data handle. - Closes #11938 + Like storing the response code. -Daniel Stenberg (28 Sep 2023) + This also adjust test 1506. Unfortunately it also removes a key part of + the test that verifies that a connection is closed since when this + output vanishes (because the closure handle is used), we don't know + exactly that the connection actually gets closed in this test... -- RELEASE-NOTES: spell out that IPFS is via gateway + Reported-by: ohyeaah on github + Fixes #12367 + Closes #12405 - RELEASE-NOTES: synced -- tool_operate: avoid strlen() -1 on zero length content from file - - Follow-up to 65b563a96a226649ba12cb1e - - Closes #11959 +Stefan Eissing (24 Nov 2023) -- tool_operate: fix memory mixups +- quic: make eyeballers connect retries stop at weird replies - Switch to plain getenv() from curl_getenv() to avoid the allocation and - having to keep track of which free() or curl_free() that need to be - used. + - when a connect immediately goes into DRAINING state, do + not attempt retries in the QUIC connection filter. Instead, + return CURLE_WEIRD_SERVER_REPLY + - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an + inconclusive answer. When all addresses have been attempted, + rewind the address list once on an inconclusive answer. + - refs #11832 where connects were retried indefinitely until + the overall timeout fired - Coverity found issues and a memory leak. + Closes #12400 - Follow-up to 65b563a96a226649ba12cb1e +Daniel Stenberg (24 Nov 2023) - Closes #11959 +- CI: verify libcurl function SYNPOSIS sections -Viktor Szakats (27 Sep 2023) + With the .github/scripits/verify-synopsis.pl script -- curl-functions.m4: fixup recent bad edits + Closes #12402 - Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 +- docs/libcurl: SYNSOPSIS cleanup - Closes #11966 + - use the correct include file + - make sure they are declared as in the header file + - fix minor nroff syntax mistakes (missing .fi) -Daniel Stenberg (27 Sep 2023) + These are verified by verify-synopsis.pl, which extracts the SYNPOSIS + code and runs it through gcc. -- curl-functions.m4: fix include line + Closes #12402 - This made the getaddrinfo detection fail, but we did not spot it in the - CI because it graciously falled back to using legacy functions instead! +- sendf: fix comment typo - Follow-up to 96c29900bcec (#11940) +- fopen: allocate the dir after fopen - Closes #11965 + Move the allocation of the directory name down to after the fopen() call + to allow that shortcut code path to avoid a superfluous malloc+free + cycle. -- inet_ntop: add typecast to silence Coverity + Follow-up to 73b65e94f35311 - CID 1024653: Integer handling issues (SIGN_EXTENSION) + Closes #12398 - Suspicious implicit sign extension: "src[i]" with type "unsigned char - const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to - type "int" (32 bits, signed), then sign-extended to type "unsigned long" - (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than - 0x7FFFFFFF, the upper bits of the result will all be 1. +Stefan Eissing (24 Nov 2023) - 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); +- transfer: cleanup done+excess handling - The value will not be greater than 0x7FFFFFFF so this still cannot - happen. + - add `SingleRequest->download_done` as indicator that + all download bytes have been received + - remove `stop_reading` bool from readwrite functions + - move excess body handling into client download writer - Also, switch to ints here instead of longs. The values stored are 16 bit - so at least no need to use 64 bit variables. Also, longs are 32 bit on - some platforms so this logic still needs to work with 32 bits. + Closes #12371 - Closes #11960 +Daniel Stenberg (23 Nov 2023) -- docs: adapt SEE ALSO sections to new requirements +- fopen: create new file using old file's mode - To please manpage-syntax.pl used by test 1173 + Because the function renames the temp file to the target name as a last + step, if the file was previously owned by a different user, not ORing + the old mode could otherwise end up creating a file that was no longer + readable by the original owner after save. - Closes #11957 + Reported-by: Loïc Yhuel + Fixes #12299 + Closes #12395 -- manpage-syntax.pl: verify SEE ALSO syntax +- test1476: require proxy - - Enforce a single reference per .BR line - - Skip the quotes around the section number for example (3) - - Insist on trailing commas on all lines except the last - - Error on comma on the last SEE ALSO entry + Follow-up from 323df4261c3542 - - List the entries alpha-sorted, not enforced just recommended + Closes #12394 - Closes #11957 +- fopen: create short(er) temporary file name -- connect: expire the timeout when trying next + Only using random letters in the name plus a ".tmp" extension. Not by + appending characters to the final file name. - ... so that it gets called again immediately and can continue trying - addresses to connect to. Otherwise it might unnecessarily wait for a - while there. + Reported-by: Maksymilian Arciemowicz - Fixes #11920 - Reported-by: Loïc Yhuel - Closes #11935 + Closes #12388 -- http: remove wrong comment for http_should_fail +Stefan Eissing (23 Nov 2023) - Reported-by: Christian Schmitz - Ref: #11936 - Closes #11941 +- tests: git ignore generated second-hsts.txt file -Dan Fandrich (26 Sep 2023) + File is generated in test lib1900 -- tool_setopt: remove unused function tool_setopt_flags + Follow-up to 7cb03229d9e9c5 - This function is identical to tool_setopt_bitmask except that it treats - the argument as unsigned. + Closes #12393 - Closes #11943 +Viktor Szakats (23 Nov 2023) -Viktor Szakats (26 Sep 2023) +- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 -- cmake: add feature checks for `memrchr` and `getifaddrs` + Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to + OpenSSL 1.1.x, and LibreSSL 3.6 or upper. - - `HAVE_MEMRCHR` for `memrchr`. - - `HAVE_GETIFADDRS` for `getifaddrs`. - This was present in `lib/curl_config.h.cmake` but missed the detection - logic. + With the caveat, that "group name" and "type name" are missing from + the log output with these TLS backends. - To match existing autotools feature checks. + Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Closes #11954 + Reviewed-by: Daniel Stenberg + Closes #12385 -- cmake: move global headers to specific checks +Daniel Stenberg (23 Nov 2023) - Before this patch we added standard headers unconditionally to the - global list of headers used for feature checks. This is unnecessary - and also doesn't help CMake 'Generate' performance. This patch moves - these headers to each feature check where they are actually needed. - Stop using `stddef.h`, as it seems unnecessary. +- urldata: fix typo in comment - I've used autotools' `m4/curl-functions.m4` to figure out these - dependencies. +- CI: codespell - Also delete checking for the C89 standard header `time.h`, that I - missed in the earlier commit. + The list of words to ignore is in the file + .github/scripts/codespell-ignore.txt - Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + Closes #12390 - Closes #11951 +- lib: fix comment typos -- src/mkhelp: make generated code pass `checksrc` + Five separate ones, found by codespell - Closes #11955 + Closes #12390 -- tests: show which curl tool `runtests.pl` is using +- test1476: verify cookie PSL mixed case - To help debugging when there is issue finding or running it. +- cookie: lowercase the domain names before PSL checks - Closes #11953 + Reported-by: Harry Sintonen -- CI/azure: make `MAKEFLAGS` global to parallelize all jobs + Closes #12387 - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi - th -j3) +Viktor Szakats (23 Nov 2023) - Closes #11952 +- openssl: fix building with v3 `no-deprecated` + add CI test -- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 + - build quictls with `no-deprecated` in CI to have test coverage for + this OpenSSL 3 configuration. - Also delete an accidental variable reference. + - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. + The caller code is meant for OpenSSL 3, while these two functions were + only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 + if built with option `no-deprecated`, causing build errors: + ``` + vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ + algorithms'; ISO C99 and later do not support implicit function declaration + s [-Wimplicit-function-declaration] + vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ + digests'; ISO C99 and later do not support implicit function declarations [ + -Wimplicit-function-declaration] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f + ullLog=true#L7667 - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 + Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 + Reviewed-by: Alex Bozarth - Closes #11945 + - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with + `no-deprecated` quictls 3 builds. + Do it by moving an existing solution for this from `vtls/openssl.c` + to `vtls/openssl.h` and adjusting caller code. + ``` + vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g + et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit + -function-declaration] + ``` + Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s + tep:24:1178 -Daniel Stenberg (26 Sep 2023) + - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and + `-Wunused-function` when trying to build curl with NTLM enabled but + without the necessary TLS backend (with DES) support. -- docs: add see also curl_multi_get_handles to some man pages + Closes #12384 - Assisted-by: Jay Satiro +- curl.h: delete Symbian OS references - Closes #11942 + curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 + via #5989. Delete references to it from public headers, because there + is no fresh release to use those headers with. -Viktor Szakats (26 Sep 2023) + Reviewed-by: Dan Fandrich + Reviewed-by: Jay Satiro + Closes #12378 -- cmake: assume `_fseeki64` and no `fseeko` on Windows +- windows: use built-in `_WIN32` macro to detect Windows - `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and - at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. + Windows compilers define `_WIN32` automatically. Windows SDK headers + or build env defines `WIN32`, or we have to take care of it. The + agreement seems to be that `_WIN32` is the preferred practice here. + Make the source code rely on that to detect we're building for Windows. - `fseeko` is not present in any of these. + Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for + Windows detection, next to the official `_WIN32`. After this patch it + only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. - (mingw-w64 1.0 also offers `fseeko64`.) + There is a slight chance these break compatibility with Windows + compilers that fail to define `_WIN32`. I'm not aware of any obsolete + or modern compiler affected, but in case there is one, one possible + solution is to define this macro manually. - [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 + grepping for `WIN32` remains useful to discover Windows-specific code. - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 + Also: - Closes #11950 + - extend `checksrc` to ensure we're not using `WIN32` anymore. -- build: delete checks for C89 standard headers + - apply minor formatting here and there. - Delete checks and guards for standard C89 headers and assume these are - available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, - `stddef.h`, `signal.h`. + - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. - Some of these we already used unconditionally, some others we only used - for feature checks. + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i - n CMake) + Closes #12376 - Closes #11940 +Stefan Eissing (22 Nov 2023) -Stefan Eissing (26 Sep 2023) +- url: ConnectionExists revisited -- multiif.h: remove Curl_multi_dump declaration + - have common pattern of `if not match, continue` + - revert pages long if()s to return early + - move dead connection check to later since it may + be relatively expensive + - check multiuse also when NOT building with NGHTTP2 + - for MULTIUSE bundles, verify that the inspected + connection indeed supports multiplexing when in use + (bundles may contain a mix of connection, afaict) - Follow-up to d850eea2 which removed the Curl_multi_dump definition. + Closes #12373 - Closes https://github.com/curl/curl/pull/11946 +Daniel Stenberg (22 Nov 2023) -Jay Satiro (26 Sep 2023) +- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range -- config-win32: define HAVE__FSEEKI64 + ... or use the default value. - Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that - calls _fseeki64 if it is available. + Also clarify the documentation language somewhat. - Closes https://github.com/curl/curl/pull/11944 + Closes #12382 -- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER +- urldata: make maxconnects a 32 bit value - - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place - even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. + "2^32 idle connections ought to be enough for anybody" - The behavior is verified by test2048. + Closes #12375 - Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 - Reported-by: claudiusaiz@users.noreply.github.com +- FEATURES: update the URL phrasing - Bug: https://github.com/curl/curl/discussions/11910 - Reported-by: Hakan Sunay Halil + The URL is length limited since a while back so "no limit" simply is not + true anymore. Mention the URL RFC standard used instead. - Closes https://github.com/curl/curl/pull/11930 + Closes #12383 -Stefan Eissing (26 Sep 2023) +- wolfssh: remove redundant static prototypes -- openssl: improve ssl shutdown handling + vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We + rror=redundant-decls] - - If SSL shutdown is not finished then make an additional call to - SSL_read to gather additional tracing. + Closes #12381 - - Fix http2 and h2-proxy filters to forward do_close() calls to the next - filter. +- setopt: remove superfluous use of ternary expressions - For example h2 and SSL shutdown before and after this change: + Closes #12374 - Before: +- mime: store "form escape" as a single bit - Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> - ssl_cf_destroy + Closes #12374 - After: +- setopt: check CURLOPT_TFTP_BLKSIZE range on set - Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> - ssl_cf_close + ... instead of later when the transfer is about to happen. - Note that currently the tracing does not show output on the connection - closure handle. Refer to discussion in #11878. + Closes #12374 - Ref: https://github.com/curl/curl/discussions/11878 +Viktor Szakats (21 Nov 2023) - Closes https://github.com/curl/curl/pull/11858 +- build: add more picky warnings and fix them -Loïc Yhuel (26 Sep 2023) + Enable more picky compiler warnings. I've found these options in the + nghttp3 project when implementing the CMake quick picky warning + functionality for it [1]. -- multi: fix small timeouts + `-Wunused-macros` was too noisy to keep around, but fixed a few issues + it revealed while testing. - Since Curl_timediff rounds down to the millisecond, timeouts which - expire in less than 1ms are considered as outdated and removed from the - list. We can use Curl_timediff_us instead, big timeouts could saturate - but this is not an issue. + - autotools: reflect the more precisely-versioned clang warnings. + Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 + - autotools: sync between clang and gcc the way we set `no-multichar`. + - autotools: avoid setting `-Wstrict-aliasing=3` twice. + - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. + It triggers in libtool-generated stub code. - Closes #11937 + - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. -Viktor Szakats (25 Sep 2023) + - lib/curl_setup.h: delete duplicate declaration for `fileno`. + Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d + (1999-12-29). This suggests this may not be needed anymore, but if + it does, we may restore this for those specific (non-Windows) systems. + - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since + c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. + - lib: delete unused macro `isxdigit_ascii` since + f65f750742068f579f4ee6d8539ed9d5f0afcb85. + - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. + - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. + - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` + macro. + - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. + - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. + - lib/setopt: fix `-Wunreachable-code-break`. + - lib/system_win32 and lib/timeval: fix double declarations for + `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. + - lib/warnless: fix double declarations in CMake UNITY mode [5]. + This was due to force-disabling the header guard of `warnless.h` to + to reapply it to source code coming after `warnless.c` in UNITY + builds. This reapplied declarations too, causing the warnings. + Solved by adding a header guard for the lines that actually need + to be reapplied. + - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. + - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant + block. + - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. + - lib/vtls/sectransp: suppress `-Wunreachable-code`. + Detected in `else` branches of dynamic feature checks, with results + known at compile-time, e.g. + ```c + if(SecCertificateCopySubjectSummary) /* -> true */ + ``` + Likely fixable as a separate micro-project, but given SecureTransport + is deprecated anyway, let's just silence these locally. + - src/tool_help: delete duplicate declaration for `helptext`. + - src/tool_xattr: fix `-Wunreachable-code`. + - tests: delete duplicate declaration for `unitfail` [8]. + - tests: delete duplicate declaration for `strncasecompare`. + - tests/libtest: delete duplicate declaration for `gethostname`. + Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 + (2010-08-02). + Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc + If there are still systems around with warnings, we may restore the + prototype, but limited for those systems. + - tests/lib2305: delete duplicate declaration for + `libtest_debug_config`. + - tests/h2-download: fix `-Wunreachable-code-break`. -- cmake: fix stderr initialization in unity builds + [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 + f8bf9f/cmake/PickyWarningsC.cmake + [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui + qla5fj45?fullLog=true#L1675 + [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 + 2331#step:7:72 + https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 + 2331#step:7:109 + [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L204 + [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L218 + [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 + 2331#step:7:290 + [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 + 2331#step:9:1193 + [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 + 2331#step:33:1870 - Before this patch, in certain build configurations the curl tool may - not have displayed anything (debug, macOS), or crashed at startup - (debug, Windows). + Closes #12331 - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 - Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db +Daniel Stenberg (21 Nov 2023) - Closes #11929 +- transfer: avoid unreachable expression -- cmake: fix missing `zlib.h` when compiling `libcurltool` + If curl_off_t and size_t have the same size (which is common on modern + 64 bit systems), a condition cannot occur which Coverity pointed + out. Avoid the warning by having the code conditionally only used if + curl_off_t actually is larger. - Came up while testing debug/testing build for Windows. I'm not sure why - it didn't come up in earlier tests with similar config. - `tool_hugehelp.c` might indeed require `zlib.h` and without linking - `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected - dependency headers to the compiler command. + Follow-up to 1cd2f0072fa482e25baa2 - ``` - [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang - --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 - 6_64 - -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO - DE - -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib - -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li - ne-argument - -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static - -libgcc - -lucrt [...] -O3 -DNDEBUG -municode -MD - -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d - -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x - 64/src/tool_hugehelp.c - .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f - ile not found - 6 | #include - | ^~~~~~~~ - ``` + Closes #12370 - Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 +Stefan Eissing (21 Nov 2023) - Closes #11927 +- transfer: readwrite improvements -- cmake: fix duplicate symbols when linking tests + - changed header/chunk/handler->readwrite prototypes to accept `buf`, + `blen` and a `pconsumed` pointer. They now get the buffer to work on + and report back how many bytes they consumed + - eliminated `k->str` in SingleRequest + - improved excess data handling to properly calculate with any body data + left in the headerb buffer + - eliminated `k->badheader` enum to only be a bool - The linker resolves this automatically in non-unity builds. In unity - builds the linker cannot drop a single object with the duplicates, - resulting in these errors. The root issue is that we started including - certain objects both via both libcurlu and libcurltool libs. + Closes #12283 - Regression from 39e7c22bb459c2e818f079984989a26a09741860 +Daniel Stenberg (21 Nov 2023) - Windows errors: - ``` - [ 3%] Linking C executable unit1303.exe - [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul - tibyte.c.obj - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch - ar': - C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve - rt_UTF8_to_wchar' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:44: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT - F8': - C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve - rt_wchar_to_UTF8' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:66: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': - C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 - _open' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:92: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': - C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 - 2_fopen' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:120: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': - [...] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 - aa4ehny5q#L247 +- RELEASE-NOTES: synced - macOS errors: - ``` - [ 56%] Linking C executable unit1302 - duplicate symbol '_curlx_sotouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_sitouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_uztosz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - [...] - ``` - with config: - ``` - -DCMAKE_UNITY_BUILD=ON \ - -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ - -DBUILD_SHARED_LIBS=ON \ - -DBUILD_STATIC_LIBS=OFF - ``` +Jiří Hruška (21 Nov 2023) - Closes #11926 +- transfer: avoid calling the read callback again after EOF -- cmake: lib `CURL_STATICLIB` fixes (Windows) + Regression since 7f43f3dc5994d01b12 (7.84.0) - - always define `CURL_STATICLIB` when building libcurl for Windows. + Bug: https://curl.se/mail/lib-2023-11/0017.html - This disables `__declspec(dllexport)` for exported libcurl symbols. - In normal mode (hide symbols) these exported symbols are specified - via `libcurl.def`. When not hiding symbols, all symbols are exported - by default. + Closes #12363 - Regression from 1199308dbc902c52be67fc805c72dd2582520d30 +Daniel Stenberg (21 Nov 2023) - Fixes #11844 +- doh: provide better return code for responses w/o addresses - - fix to omit `libcurl.def` when not hiding private symbols. + Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the + response did not contain any addresses. Now it more accurately returns + CURLE_COULDNT_RESOLVE_HOST. - Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 + Reported-by: lRoccoon on github - - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also - omitting `libcurl.def` in this case, and exporting all symbols - instead. This ensures that a shared curl tool can access all debug - functions which are not normally exported from libcurl DLL. + Fixes #12365 + Closes #12366 - - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" - target. +Stefan Eissing (21 Nov 2023) - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 +- HTTP/2, HTTP/3: handle detach of onoing transfers - - delete duplicate `BUILDING_LIBCURL` definitions. + - refs #12356 where a UAF is reported when closing a connection + with a stream whose easy handle was cleaned up already + - handle DETACH events same as DONE events in h2/h3 filters - - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. + Fixes #12356 + Reported-by: Paweł Wegner + Closes #12364 - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 +Viktor Szakats (20 Nov 2023) - Closes #11914 +- autotools: stop setting `-std=gnu89` with `--enable-warnings` -Daniel Stenberg (25 Sep 2023) + Do not alter the C standard when building with `--enable-warnings` when + building with gcc. -- RELEASE-NOTES: synced + On one hand this alters warning results compared to a default build. + On the other, it may produce different binaries, which is unexpected. -Dan Fandrich (25 Sep 2023) + Also fix new warnings that appeared after removing `-std=gnu89`: -- tests: fix log directory path in IPFS tests + - include: fix public curl headers to use the correct printf mask for + `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 + and Visual Studio 2013 and newer. This fixes the printf mask warnings + in examples and tests. E.g. [1] - Hard-coding the log directory name fails with parallel tests. + - conncache: fix printf format string [2]. - Follow-up to 65b563a96 + - http2: fix potential null pointer dereference [3]. + (seen on Slackware with gcc 11.) - Ref: #8805 + - libssh: fix printf format string in SFTP code [4]. + Also make MSVC builds compatible with old CRT versions. -Daniel Stenberg (25 Sep 2023) + - libssh2: fix printf format string in SFTP code for MSVC. + Applying the same fix as for libssh above. -- curl_multi_get_handles: get easy handles from a multi handle + - unit1395: fix `argument is null` and related issues [5]: + - stop calling `strcmp()` with NULL to avoid undefined behaviour. + - fix checking results if some of them were NULL. + - do not pass NULL to printf `%s`. - Closes #11750 + - ci: keep a build job with `-std=gnu89` to continue testing for + C89-compliance. We can apply this to other gcc jobs as needed. + Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 -Stefan Eissing (25 Sep 2023) + [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l + ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b + [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 + 2346#step:6:67 + [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 + 2346#step:30:214 + [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 + 2346#step:29:895 + [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 + 2346#step:33:1689 -- http: h1/h2 proxy unification + Closes #12346 - - use shared code for setting up the CONNECT request - when tunneling, used in HTTP/1.x and HTTP/2 proxying - - eliminate use of Curl_buffer_send() and other manipulations - of `data->req` or `data->state.ulbuf` +- autotools: fix/improve gcc and Apple clang version detection - Closes #11808 + - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu + may return `n-win32` (also with `-dumpfullversion`). Causing these + errors and failing to enable picky warnings: + ``` + ../configure: line 23845: test: : integer expression expected + ``` + Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 + 3718#step:5:143 -Natanael Copa (25 Sep 2023) + Fix that by stripping any dash-suffix and handling a dotless (major-only) + version number by assuming `.0` in that case. -- lib: use wrapper for curl_mime_data fseek callback + `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` + Ref: https://github.com/mamedev/mame/pull/9767 - fseek uses long offset which does not match with curl_off_t. This leads - to undefined behavior when calling the callback and caused failure on - arm 32 bit. + - fix Apple clang version detection for releases between + 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the + version was under-detected as 3.7 llvm/clang equivalent. - Use a wrapper to solve this and use fseeko which uses off_t instead of - long. + - fix Apple clang version detection for 'Apple clang version 11.0.0' + and newer where the Apple clang version was detected, instead of its + llvm/clang equivalent. - Thanks to the nice people at Libera IRC #musl for helping finding this - out. + - display detected clang/gcc/icc compiler version. - Fixes #11882 - Fixes #11900 - Closes #11918 + Via libssh2: + - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d + 89875ad + https://github.com/libssh2/libssh2/pull/1187 + - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 + 42b592e + https://github.com/libssh2/libssh2/pull/1232 -- configure: sort AC_CHECK_FUNCS + Closes #12362 - No functional changes. +- autotools: delete LCC compiler support bits -Daniel Stenberg (25 Sep 2023) + Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 -- warnless: remove unused functions + Closes #12357 - Previously put there for use with the intel compiler +- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` - Closes #11932 + - tests: verify CMake `DISABLE` options. -- GHA/linux: run singleuse to detect single-use global functions + Make an exception for 2 CMake-only ones, and one more that's + using a different naming scheme, also in autotools and source. - Use --unit for configure --enable-debug builds + - cmake: add support for `CURL_DISABLE_HEADERS_API`. - Closes #11932 + Suggested-by: Daniel Stenberg + Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 -- singleuse: add scan for use in other source codes + Closes #12353 - This should reduce false-positive to almost zero. Checks for presence in - unit tests if --unit is specified, which is intended for debug builds - where unit testing is enabled. +Jacob Hoffman-Andrews (20 Nov 2023) - Closes #11932 +- hyper: temporarily remove HTTP/2 support -- multi: remove Curl_multi_dump + The current design of the Hyper integration requires rebuilding the + Hyper clientconn for each request. However, building the clientconn + requires resending the HTTP/2 connection preface, which is incorrect + from a protocol perspective. That in turn causes servers to send GOAWAY + frames, effectively degrading performance to "no connection reuse" in + the best case. It may also be triggering some bugs where requests get + dropped entirely and reconnects take too long. - A debug-only function that is basically never used. Removed to ease the - use of the singleuse script to detect non-static functions not used - outside the file where it is defined. + This doesn't rule out HTTP/2 support with Hyper, but it may take a + redesign of the Hyper integration in order to make things work. - Closes #11931 + Closes #12191 -Viktor Szakats (24 Sep 2023) +Jay Satiro (20 Nov 2023) -- tests: fix compiler warnings +- schannel: fix unused variable warning - Seen with llvm 17 on Windows x64. + Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 + Reported-by: Viktor Szakats - ``` - .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration - for non-static variable 'logdir' [-Wmissing-variable-declarations] - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration - for non-static variable 'loglockfile' [-Wmissing-variable-declarations] - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat - ion for non-static variable 'logdir' [-Wmissing-variable-declarations] - 43 | const char *logdir = "log"; - | ^ - .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl - e is not intended to be used outside of this translation unit - 43 | const char *logdir = "log"; - | ^ - .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator - here [-Wcomma] - 350 | ++d, ++s; - | ^ - .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa - rning - 350 | ++d, ++s; - | ^~~ - | (void)( ) - ``` + Closes https://github.com/curl/curl/pull/12361 - ``` - .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; - | ~ ^ ~~~~~~~~~~~~~ - 1 warning generated. +Daniel Stenberg (19 Nov 2023) - .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison - 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] - 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. +- url: find scheme with a "perfect hash" - .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > - 2147483647 is always false [-Wtautological-type-limit-compare] - 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - ``` + Instead of a loop to scan over the potentially 30+ scheme names, this + uses a "perfect hash" table. This works fine because the set of schemes + is known and cannot change in a build. The hash algorithm and table size + is made to only make a single scheme index per table entry. - Seen on macOS Intel: - ``` - .../curl/tests/server/sws.c:440:64: warning: field precision should have type - 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] - msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" - , - ~~^~ - 1 warning generated. - ``` + The perfect hash is generated by a separate tool (scripts/schemetable.c) - Closes #11925 + Closes #12347 -Jay Satiro (24 Sep 2023) +- scripts: add schemetable.c -- url: fix netrc info message + This tool generates a scheme-matching table. - - Fix netrc info message to use the generic ".netrc" filename if the - user did not specify a netrc location. + It iterates over a number of different initial and shift values in order + to find the hash algorithm that needs the smallest possible table. - - Update --netrc doc to add that recent versions of curl on Windows - prefer .netrc over _netrc. + The generated hash function, table and table size then needs to be used + by the url.c:Curl_getn_scheme_handler() function. - Before: - * Couldn't find host google.com in the (nil) file; using defaults +Stefan Eissing (19 Nov 2023) - After: - * Couldn't find host google.com in the .netrc file; using defaults +- vtls/vquic, keep peer name information together - Closes https://github.com/curl/curl/pull/11904 + - add `struct ssl_peer` to keep hostname, dispname and sni + for a filter + - allocate `sni` for use in VTLS backend + - eliminate `Curl_ssl_snihost()` and its use of the download buffer + - use ssl_peer in SSL and QUIC filters -Dan Fandrich (23 Sep 2023) + Closes #12349 -- wolfssh: do cleanup in Curl_ssh_cleanup +Viktor Szakats (18 Nov 2023) - Closes: #11921 +- build: always revert `#pragma GCC diagnostic` after use -Daniel Stenberg (24 Sep 2023) + Before this patch some source files were overriding gcc warning options, + but without restoring them at the end of the file. In CMake UNITY builds + these options spilled over to the remainder of the source code, + effecitvely disabling them for a larger portion of the codebase than + intended. -- tool_listhelp: regenerated + `#pragma clang diagnostic` didn't have such issue in the codebase. - Polished the --ipfs-gateway description + Reviewed-by: Marcel Raad + Closes #12352 - Fixed the --trace-config description +- tidy-up: casing typos, delete unused Windows version aliases - The script also fixed some other small mistakes + - cmake: fix casing of `UnixSockets` to match the rest of the codebase. - Closes #11923 + - curl-compilers.m4: fix casing in a comment. -Viktor Szakats (23 Sep 2023) + - setup-win32: delete unused Windows version constant aliases. -- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) + Reviewed-by: Marcel Raad + Closes #12351 - Also fix to export all symbols in Windows debug builds, making - `-debug-dyn` builds work with `-DCURL_STATICLIB` set. +- keylog: disable if unused - Ref: https://github.com/curl/curl/pull/11914 (same for CMake) + Fully disable keylog code if there is no TLS or QUIC subsystem using it. - Closes #11924 + Closes #12350 -Daniel Stenberg (23 Sep 2023) +- cmake: add `CURL_DISABLE_BINDLOCAL` option -- quic: set ciphers/curves the same way regular TLS does + To match similar autotools option. - for OpenSSL/BoringSSL + Default is `ON`. - Fixes #11796 - Reported-by: Karthikdasari0423 on github - Assisted-by: Jay Satiro - Closes #11836 + Reviewed-by: Daniel Stenberg + Closes #12345 -- test457: verify --max-filesize with chunked encoding +- url: fix `-Wzero-length-array` with no protocols -- lib: let the max filesize option stop too big transfers too + Fixes: + ``` + ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ + -Wc2x-extensions] + 178 | static const struct Curl_handler * const protocols[] = { + | ^ + ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length + -array] + ``` - Previously it would only stop them from getting started if the size is - known to be too big then. + Closes #12344 - Update the libcurl and curl docs accordingly. +- url: fix builds with `CURL_DISABLE_HTTP` - Fixes #11810 - Reported-by: Elliot Killick - Assisted-by: Jay Satiro - Closes #11820 + Fixes: + ``` + ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' + 456 | Curl_mime_cleanpart(data->state.formp); + | ~~~~~~~~~~~ ^ + ``` -Viktor Szakats (23 Sep 2023) + Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 -- mingw: delete support for legacy mingw.org toolchain + Closes #12343 - Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: - https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ - Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. - It supported the x86 CPU only and used a old Windows API header and - implib set, often causing issues. It also misses most modern Windows - features, offering old versions of both binutils and gcc (no llvm/clang - support). It was last updated 2 years ago. +- http: fix `-Wunused-parameter` with no auth and no proxy - curl now relies on toolchains based on the mingw-w64 project: - https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ - https://www.msys2.org/ https://github.com/msys2/msys2 - https://github.com/mstorsjo/llvm-mingw - (Also available via Linux and macOS package managers.) + ``` + lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] + bool proxy) + ^ + ``` - Closes #11625 + Reviewed-by: Marcel Raad + Closes #12338 -Mark Gaiser (23 Sep 2023) +Daniel Stenberg (16 Nov 2023) -- curl: add support for the IPFS protocols: +- TODO: Some TLS options are not offered for HTTPS proxies - - ipfs:// - - ipns:// + Closes #12286 + Closes #12342 - This allows you tu use ipfs in curl like: - curl ipfs:// - and - curl ipns:// +- RELEASE-NOTES: synced - For more information consult the readme at: - https://curl.se/docs/ipfs.html +- duphandle: make dupset() not return with pointers to old alloced data - Closes #8805 + As the blob pointers are to be duplicated, the function must not return + mid-function with lingering pointers to the old handle's allocated data, + as that would lead to double-free in OOM situations. -Daniel Stenberg (23 Sep 2023) + Make sure to clear all destination pointers first to avoid this risk. -- bufq: remove Curl_bufq_skip_and_shift (unused) + Closes #12337 - Closes #11915 +Viktor Szakats (16 Nov 2023) -- scripts/singleuse.pl: add curl_global_trace +- http: fix `-Wunused-variable` compiler warning -Viktor Szakats (22 Sep 2023) + Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. -- cmake: fix unity symbol collisions in h2 builds + E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + + `CURL_DISABLE_NTLM` on non-Windows. - Regression from 331b89a319d0067fa1e6441719307cfef9c7960f + ``` + ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl + e] + CURLcode result = CURLE_OK; + ^ + ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus + ed-but-set-variable] + unsigned long *availp; + ^ + ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse + d-but-set-variable] + struct auth *authp; + ^ + ``` - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #11912 + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 -Daniel Stenberg (22 Sep 2023) + Fixes #12228 + Closes #12335 -- RELEASE-NOTES: synced +Jay Satiro (16 Nov 2023) -Dan Fandrich (21 Sep 2023) +- tool: support bold headers in Windows -- github/labeler: improve the match patterns + - If virtual terminal processing is enabled in Windows then use ANSI + escape codes Esc[1m and Esc[22m to turn bold on and off. - This includes new rules for setting the appleOS and logging labels and - matches on some example files. Also, enable dot mode for wildcard - matches in the .github directory. + Suggested-by: Gisle Vanem -Daniel Stenberg (21 Sep 2023) + Ref: https://github.com/curl/curl/discussions/11770 -- upload-file.d: describe the file name slash/backslash handling + Closes https://github.com/curl/curl/pull/12321 - Closes #11911 +Viktor Szakats (15 Nov 2023) -Jakub Jelen (21 Sep 2023) +- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` -- libssh: cap SFTP packet size sent + Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + + `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, + or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while + compiling due to a missing HMAC declaration. - Due to libssh limitations + The reason is that HMAC is required by `lib/sha256.c` which publishes + `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when + building for libssh2 v1.8.2 (2019-05-25) or older. - Signed-off-by: Jakub Jelen + Make sure to compile the HMAC bits for a successful build. - Closes #11804 + Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting + them into separate sources isn't practical. -Daniel Stenberg (21 Sep 2023) + Fixes: + ``` + [...] + In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc + url_object.dir/Unity/unity_0_c.c:310: + ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s + truct HMAC_params' + 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { + | ^ + ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par + ams' + [...] + ``` -- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 - Closes #11905 + Fixes #12273 + Closes #12332 -- mailmap: unify Michael Osipov under a single email +Daniel Stenberg (15 Nov 2023) -Ted Lyngmo (21 Sep 2023) +- duphandle: also free 'outcurl->cookies' in error path -- docs: use CURLSSLBACKEND_NONE + Fixes memory-leak when OOM mid-function - [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in - documentation and examples. + Use plain free instead of safefree, since the entire struct is + freed below. - Signed-off-by: Ted Lyngmo + Remove some free calls that is already freed in Curl_freeset() - Closes #11909 + Closes #12329 -Dan Fandrich (21 Sep 2023) +Viktor Szakats (15 Nov 2023) -- github/labeler: give the sync-labels config item a default value +- config-win32: set `HAVE_SNPRINTF` for mingw-w64 - This shouldn't be necessary and is likely a bug with this beta version - of the labeller. + It's available in all mingw-w64 releases. We already pre-fill this + detection in CMake. - Also, fix the negative matches for the documentation label. + Closes #12325 - Follow-up to dd12b452a - Closes #11907 +- sasl: fix `-Wunused-function` compiler warning -- github/labeler: fix up more the labeler config format + In builds with disabled auths. - The new version didn't like the workaround we had for a bug in the - previous labeler version, and it should no longer be needed. + ``` + lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus + ed-function] + static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, + ^ + 1 warning generated. + ``` + Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st + ep:3:3822 - Follow-up to dd12b452a - Closes #11906 + Reviewed-by: Daniel Stenberg + Closes #12326 -- github/labeler: fix indenting to try to appease labeller +- build: picky warning updates - Follow-up to dd12b452a + - cmake: sync some picky gcc warnings with autotools. + - cmake, autotools: add `-Wold-style-definition` for clang too. + - cmake: more precise version info for old clang options. + - cmake: use `IN LISTS` syntax in `foreach()`. -Jay Satiro (21 Sep 2023) + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Closes #12324 -- libssh2: fix error message on failed pubkey-from-file +Daniel Stenberg (15 Nov 2023) - - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error - message "SSH public key authentication failed: Reason unknown (-1)". +- urldata: move cookielist from UserDefined to UrlState - When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a - generic error and therefore doesn't set an error message. AFAICT that is - not documented behavior. + 1. Because the value is not strictly set with a setopt option. - Prior to this change libcurl retrieved the last set error message which - would be from a previous function failing. That resulted in misleading - auth failed error messages in verbose mode. + 2. Because otherwise when duping a handle when all the set.* fields are + first copied and an error happens (think out of memory mid-function), + the function would easily free the list *before* it was deep-copied, + which could lead to a double-free. - Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 - Reported-by: consulion@users.noreply.github.com + Closes #12323 - Closes https://github.com/curl/curl/pull/11881 +Viktor Szakats (14 Nov 2023) -Stefan Eissing (21 Sep 2023) +- autotools: avoid passing `LDFLAGS` twice to libcurl -- pytest: exclude test_03_goaway in CI runs due to timing dependency + autotools passes `LDFLAGS` automatically linker commands. curl's + `lib/Makefile.am` customizes libcurl linker flags. In that + customization, it added `LDFLAGS` to the custom flags. This resulted in + passing `LDFLAGS` _twice_ to the `libtool` command. - Closes #11860 + Most of the time this is benign, but some `LDFLAGS` options can break + the build when passed twice. One such example is passing `.o` files, + e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for + MUSL builds. -- lib: disambiguate Curl_client_write flag semantics + Passing them twice resulted in duplicate symbol errors: + ``` + libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a + arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] + ld.lld-15: error: duplicate symbol: _start + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + [...] + clang: error: linker command failed with exit code 1 (use -v to see invocatio + n) + ``` - - use CLIENTWRITE_BODY *only* when data is actually body data - - add CLIENTWRITE_INFO for meta data that is *not* a HEADER - - debug assertions that BODY/INFO/HEADER is not used mixed - - move `data->set.include_header` check into Curl_client_write - so protocol handlers no longer have to care - - add special in FTP for `data->set.include_header` for historic, - backward compatible reasons - - move unpausing of client writes from easy.c to sendf.c, so that - code is in one place and can forward flags correctly + This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. + The patch was a works-for-me hack that ended up merged in curl: + https://sourceforge.net/p/curl/bugs/1217/#06ef + With the root cause remaining unclear. - Closes #11885 + Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring + `-L` first? This would be unusual and suggests a bug in either the + linker or in `libtool`. -Patrick Monnerat (21 Sep 2023) + The curl build does pass the list of detected libs via its own + `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` + command line, but it's the job of `libtool` to ensure that even + a peculiar linker gets the options in the expected order. Also because + autotools passes `LDFLAGS` last, making it hardly possible to pass + anything after it. -- tftpd: always use curl's own tftp.h + Perhaps in the 10 years since this issue, this already got a fix + upstream. - Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects - and reports a stringop-overread warning: + This patch deletes `LDFLAGS` from our customized libcurl options, + leaving a single copy of them as passed by autotools automatically. - tftpd.c: In function ‘write_behind.isra’: - tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f - rom a region of size 0 [-Wstringop-overread] - 485 | return write(test->ofile, writebuf, count); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - In file included from tftpd.c:71: - /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 - 58 | char tu_data[0]; /* data or error stri - ng */ - | ^~~~~~~ + Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + Closes #12310 - This occurs because writebuf points to this field and the latter - cannot be considered as being of dynamic length because it is not - the last field in the structure. Thus it is bound to its declared - size. +- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` - This commit always uses curl's own version of tftp.h where the - target field is last in its structure, effectively avoiding the - warning. + To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and + curl tool (`CURL_LDFLAGS_BIN`). - As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for - arpa/tftp.h are removed. + This makes it possible to build libcurl and curl with a single + invocation with lib- and tool-specific custom linker flags. - Closes #11897 + Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, + controlling static/shared, incl. requesting a static curl tool (with + `-static-libtool-libs`) while building both shared and static libcurl. -Dan Fandrich (20 Sep 2023) + curl-for-win uses the above and some more. -- test1474: make precheck more robust on non-Solaris systems + These options are already supported in `Makefile.mk`. CMake has built-in + variables for this. - If uname -r returns something odd, perl could return an error code and - the test would be erroneously skipped. The qx// syntax avoid this. + Closes #12312 - Followup to 08f9b2148 +Jay Satiro (14 Nov 2023) -- github/labeler: switch to the 5 beta version +- tool_cb_hdr: add an additional parsing check - This version adds an important feature that will allow more PRs to be - labelled. Rather than being limited to labeling PRs with files that - match a single glob, it can now label them if multiple changed files - match any one of a number of globs. + - Don't dereference the past-the-end element when parsing the server's + Content-disposition header. -Daniel Stenberg (20 Sep 2023) + As 'p' is advanced it can point to the past-the-end element and prior + to this change 'p' could be dereferenced in that case. -- lib: enable hmac for digest as well + Technically the past-the-end element is not out of bounds because dynbuf + (which manages the header line) automatically adds a null terminator to + every buffer and that is not included in the buffer length passed to + the header callback. - Previously a build that disabled NTLM and aws-sigv4 would fail to build - since the hmac was disabled, but it is also needed for digest auth. + Closes https://github.com/curl/curl/pull/12320 - Follow-up to e92edfbef64448ef +Philip Heiduck (14 Nov 2023) - Fixes #11890 - Reported-by: Aleksander Mazur - Closes #11896 +- .cirrus.yml: freebsd 14 -- idn: if idn2_check_version returns NULL, return error + ensure curl works on latest freebsd version - ... this avoids a NULL dereference for this unusual case. + Closes #12053 - Reported-by: s0urc3_ on hackerone - Closes #11898 +Daniel Stenberg (13 Nov 2023) -- http: fix CURL_DISABLE_BEARER_AUTH breakage +- easy: in duphandle, init the cookies for the new handle - When bearer auth was disabled, the if/else logic got wrong and caused - problems. + ... not the source handle. - Follow-up to e92edfbef64448ef461 - Fixes #11892 - Reported-by: Aleksander Mazur - Closes #11895 + Closes #12318 -Michael Osipov (20 Sep 2023) +- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set -- wolfssl: allow capath with CURLOPT_CAINFO_BLOB + Previously it would unconditionally use the size, which is set to -1 + when strlen is requested. - Remain consistent with OpenSSL. While CAfile is nulled as documented - with CURLOPT_CAINFO_BLOB, CApath remains intact. + Updated test 544 to verify. - Closes #11886 + Closes #12317 -- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c +- RELEASE-NOTES: synced - Closes #11886 +- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped -Dan Fandrich (19 Sep 2023) + Closes #12315 -- test1474: disable test on NetBSD, OpenBSD and Solaris 10 +- urldata: move hstslist from 'set' to 'state' - These kernels only send a fraction of the requested amount of the first - large block, invalidating the assumptions of the test and causing it to - fail. + To make it work properly with curl_easy_duphandle(). This, because + duphandle duplicates the entire 'UserDefined' struct by plain copy while + 'hstslist' is a linked curl_list of file names. This would lead to a + double-free when the second of the two involved easy handles were + closed. - Assisted-by: Christian Weisgerber - Ref: https://curl.se/mail/lib-2023-09/0021.html - Closes #11888 + Closes #12315 -Ryan Schmidt (20 Sep 2023) +- test1900: verify duphandle with HSTS using multiple files -- cmake, configure: also link with CoreServices + Closes #12315 - When linking with CoreFoundation, also link with CoreServices which is - apparently required to avoid an NSInvalidArgumentException in software - linking with libcurl on macOS Sonoma 14 and later. +Goro FUJI (13 Nov 2023) - Fixes #11893 - Closes #11894 +- http: allow longer HTTP/2 request method names -Marc Hoersken (19 Sep 2023) + - Increase the maximum request method name length from 11 to 23. -- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket + For HTTP/1.1 and earlier there's not a specific limit in libcurl for + method length except that it is limited by the initial HTTP request + limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same + and there was no specific limit. - These dependencies are now already included in the Docker image. + According to Internet Assigned Numbers Authority (IANA) the longest + registered method is UPDATEREDIRECTREF which is 17 characters. - Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 - b41d15606e97f38cf312c1ce56 + Also there are unregistered methods used by some companies that are + longer than 11 characters. - Closes #11889 + The limit was originally added by 61f52a97 but not used until fc2f1e54. -Daniel Stenberg (19 Sep 2023) + Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml -- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files + Closes https://github.com/curl/curl/pull/12311 - Ref: #11883 - Reported-by: Michael Osipov - Closes #11884 +Jay Satiro (12 Nov 2023) -- RELEASE-NOTES: synced +- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does -- test3103: CURLOPT_COOKIELIST test + - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB + and CURLOPT_PROXY_CAINFO_BLOB docs. -- cookie: set ->running in cookie_init even if data is NULL + All the other _BLOB option docs already have the same explanation. - This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) + Closes https://github.com/curl/curl/pull/12277 - Test 3103 verifies. +Viktor Szakats (11 Nov 2023) - Fixes #11875 - Reported-by: wangp on github - Closes #11876 +- tidy-up: dedupe Windows system libs in cmake -- test498: total header size for all redirects is larger than accepted + Reviewed-by: Daniel Stenberg + Closes #12307 -- http: use per-request counter to check too large headers +Junho Choi (11 Nov 2023) - Not the counter that accumulates all headers over all redirects. +- ci: test with latest quiche release (0.19.0) - Follow-up to 3ee79c1674fd6 + Closes #12180 - Do a second check for 20 times the limit for the accumulated size for - all headers. +- quiche: use quiche_conn_peer_transport_params() - Fixes #11871 - Reported-by: Joshix-1 on github - Closes #11872 + In recent quiche, transport parameter API is separated + with quiche_conn_peer_transport_params(). + (https://github.com/cloudflare/quiche/pull/1575) + It breaks with bulding with latest(post 0.18.0) quiche. -Jay Satiro (18 Sep 2023) + Closes #12180 -- THANKS: add Eric Murphy +Daniel Stenberg (11 Nov 2023) - He reported #11850 (quiche build error) but I forgot to add a - 'reported-by' entry in the fix 267e14f1. +- Makefile: generate the VC 14.20 project files at dist-time -Daniel Stenberg (18 Sep 2023) + Follow-up to 28287092cc5a6d6ef8 (#12282) -- h2-proxy: remove left-over mistake in drain_tunnel() + Closes #12290 - Left-over from 331b89a319 +Sam James (11 Nov 2023) - Reported-by: 南宫雪珊 +- misc: fix -Walloc-size warnings - Closes https://github.com/curl/curl/pull/11877 + GCC 14 introduces a new -Walloc-size included in -Wextra which gives: -vvb2060 (18 Sep 2023) + ``` + src/tool_operate.c: In function ‘add_per_transfer’: + src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo + r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] + 213 | p = calloc(sizeof(struct per_transfer), 1); + | ^ + src/var.c: In function ‘addvariable’: + src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type + struct var’ with size ‘32’ [-Walloc-size] + 361 | p = calloc(sizeof(struct var), 1); + | ^ + ``` -- lib: failf/infof compiler warnings + The calloc prototype is: + ``` + void *calloc(size_t nmemb, size_t size); + ``` - Closes #11874 + So, just swap the number of members and size arguments to match the + prototype, as we're initialising 1 struct of size `sizeof(struct + ...)`. GCC then sees we're not doing anything wrong. -Daniel Stenberg (17 Sep 2023) + Closes #12292 -- rand: fix 'alnum': array is too small to include a terminating null character +Mark Gaiser (11 Nov 2023) - It was that small on purpose, but this change now adds the null byte to - avoid the error. +- IPFS: bugfixes - Follow-up to 3aa3cc9b052353b1 + - Fixed endianness bug in gateway file parsing + - Use IPFS_PATH in tests where IPFS_DATA was used + - Fixed typos from traling -> trailing + - Fixed broken link in IPFS.md - Reported-by: Dan Fandrich - Ref: #11838 - Closes #11870 + Follow-up to 859e88f6533f9e -Mathias Fuchs (16 Sep 2023) + Reported-by: Michael Kaufmann + Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 + Closes #12305 -- cmake: fix the help text to the static build option in CMakeLists.txt +Daniel Stenberg (11 Nov 2023) - Closes #11843 +- VULN-DISCLOSURE-POLIC: remove broken link to hackerone -John Haugabook (16 Sep 2023) + It should ideally soon not be done from hackerone anyway -- MANUAL.md: change domain to example.com + Closes #12308 - Closes #11866 +Andrew Kurushin (11 Nov 2023) -Daniel Stenberg (16 Sep 2023) +- schannel: add CA cache support for files and memory blobs -- doh: inherit DEBUGFUNCTION/DATA + - Support CA bundle and blob caching. - When creating new transfers for doing DoH, they now inherit the debug - settings from the initiating transfer, so that the application can - redirect and handle the verbose output correctly even for the DoH - transfers. + Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. - Reported-by: calvin2021y on github - Fixes #11864 - Closes #11869 + Closes https://github.com/curl/curl/pull/12261 -Dan Fandrich (16 Sep 2023) +Daniel Stenberg (10 Nov 2023) -- http_aws_sigv4: fix sorting with empty parts +- RELEASE-NOTES: synced - When comparing with an empty part, the non-empty one is always - considered greater-than. Previously, the two would be considered equal - which would randomly place empty parts amongst non-empty ones. This - showed as a test 439 failure on Solaris as it uses a different - implementation of qsort() that compares parts differently. +Charlie C (10 Nov 2023) - Fixes #11855 - Closes #11868 +- cmake: option to disable install & drop `curlu` target when unused -- CI: ignore the "flaky" and "timing-dependent" test results + This patch makes the following changes: + - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. + - Removes the target `curlu` when the option `BUILD_TESTING` is set to + `OFF` - to prevent it from being loaded in Visual Studio. - CI builds will now run these tests, but will ignore the results if they - fail. The relevant tests are ones that are sensitive to timing or - have edge conditions that make them more likely to fail on CI servers, - which are often heavily overloaded and slow. + Closes #12287 - This change only adds two additional tests to be ignored, since the - others already had the flaky keyword. +Kai Pastor (10 Nov 2023) - Closes #11865 +- cmake: fix multiple include of CURL package -- runtests: eliminate a warning on old perl versions + Fixes errors on second `find_package(CURL)`. This is a frequent case + with transitive dependencies: + ``` + CMake Error at ...: + add_library cannot create ALIAS target "CURL::libcurl" because another + target with the same name already exists. + ``` - The warning "Use of implicit split to @_ is deprecated" showed between - perl versions about 5.8 through 5.11. + Test to reproduce: + ```cmake + cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher -- tests: log the test result code after each libtest + project(curl) - This makes it easier to determine the test status. Also, capitalize - FAILURE and ABORT messages in log lines to make them easier to spot. + set(CURL_DIR "example/lib/cmake/CURL/") + find_package(CURL CONFIG REQUIRED) + find_package(CURL CONFIG REQUIRED) # fails -Harry Sintonen (16 Sep 2023) + add_executable(main main.c) + target_link_libraries(main CURL::libcurl) + ``` -- misc: better random strings + Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes + Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html + Ref: #12300 + Assisted-by: Harry Mallon + Closes #11913 - Generate alphanumerical random strings. +Viktor Szakats (8 Nov 2023) - Prior this change curl used to create random hex strings. This was - mostly okay, but having alphanumerical random strings is better: The - strings have more entropy in the same space. +- tidy-up: use `OPENSSL_VERSION_NUMBER` - The MIME multipart boundary used to be mere 64-bits of randomness due - to being 16 hex chars. With these changes the boundary is 22 - alphanumerical chars, or little over 130 bits of randomness. + Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. + Before this patch some places used `OPENSSL_VERSION_MAJOR`. - Closes #11838 + Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't + define any version number in these implementations: BoringSSL, AWS-LC, + LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to + `opensslv.h`. This wasn't causing a deeper problem because the code is + looking for v3, which is only provided by OpenSSL/quictls as of now. -Daniel Stenberg (15 Sep 2023) + According to https://github.com/openssl/openssl/issues/17517, the macro + `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. -- cookie: reduce variable scope, add const + Reviewed-by: Marcel Raad + Closes #12298 -- cookie: do not store the expire or max-age strings +Daniel Stenberg (8 Nov 2023) - Convert it to an expire time at once and save memory. +- resolve.d: drop a multi use-sentence - Closes #11862 + Since the `multi:` keyword adds that message. -- cookie: remove unnecessary struct fields + Reported-by: 積丹尼 Dan Jacobson + Fixes https://github.com/curl/curl/discussions/12294 + Closes #12295 - Plus: reduce the hash table size from 256 to 63. It seems unlikely to - make much of a speed difference for most use cases but saves 1.5KB of - data per instance. +- content_encoding: make Curl_all_content_encodings allocless - Closes #11862 + - Fixes a memory leak pointed out by Coverity + - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? + id=63947 + - Avoids unncessary allocations -- RELEASE-NOTES: synced + Follow-up ad051e1cbec68b2456a22661b - Bumped to 8.4.0, the next presumed version + Closes #12289 -Dan Fandrich (14 Sep 2023) +Michael Kaufmann (7 Nov 2023) -- test2600: remove special case handling for USE_ALARM_TIMEOUT +- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 - This was originally added to handle platforms that supported only 1 - second granularity in connect timeouts, but after some recent changes - the test currently permafails on several Windows platforms. + Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), + avoid it and use "http/1.1" instead. - The need for this special-case was removed in commit 8627416, which - increased the connect timeout in all cases to well above 1 second. + This reverts commit df856cb5c9 (#10183). - Fixes #11767 - Closes #11849 + Fixes #12259 + Closes #12285 -Daniel Stenberg (14 Sep 2023) +Daniel Stenberg (7 Nov 2023) -- SECURITY-PROCESS.md. call it vulnerability disclosure policy +- Makefile.am: drop vc10, vc11 and vc12 projects from dist - SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md + They are end of life products. Support for generating them remain in the + repo for a while but this change drops them from distribution. - This a name commonly used for a document like this. This name helps - users find it. + Closes #12288 - Closes #11852 +David Suter (7 Nov 2023) -Junho Choi (14 Sep 2023) +- projects: add VC14.20 project files -- quiche: fix build error with --with-ca-fallback + Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. + OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why + this is missing. Updated the templates to produce a VC14.20 project. + Project opens in Visual Studio 2019 as expected. - - Fix build error when curl is built with --with-quiche - and --with-ca-fallback. + Closes #12282 - - Add --with-ca-fallback to the quiche CI job. +Daniel Stenberg (7 Nov 2023) - Fixes https://github.com/curl/curl/issues/11850 - Closes https://github.com/curl/curl/pull/11847 +- curl: move IPFS code into src/tool_ipfs.[ch] -Jay Satiro (14 Sep 2023) + - convert ensure_trailing into ensure_trailing_slash + - strdup the URL string to own it proper + - use shorter variable names + - combine some expressions + - simplify error handling in ipfs_gateway() + - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached + - ipfs-gateway.d polish and simplification + - shorten ipfs error message + make them "synthetic" -- escape: replace Curl_isunreserved with ISUNRESERVED + Closes #12281 - - Use the ALLCAPS version of the macro so that it is clear a macro is - being called that evaluates the variable multiple times. +Viktor Szakats (6 Nov 2023) - - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates - a variable multiple times. +- build: delete support bits for obsolete Windows compilers - This is a follow-up to 291d225a which changed Curl_isunreserved into an - alias macro for ISUNRESERVED. The problem is the former is not easily - identified as a macro by the caller, which could lead to a bug. + - Pelles C: Unclear status, failed to obtain a fresh copy a few months + ago. Possible website is HTTP-only. ~10 years ago I left this compiler + dealing with crashes and other issues with no response on the forum + for years. It has seen some activity in curl back in 2021. + - LCC: Last stable release in September 2002. + - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned + in 2006. + - Borland C++: We dropped Borland C++ support in 2018. + - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 + (or possibly 2008) as a minimum. - For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but - Curl_isunreserved(*foo++) is not even though they both are the same. + Closes #12222 - Closes https://github.com/curl/curl/pull/11846 +- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` -Dan Fandrich (13 Sep 2023) + We use `stdint.h` unconditionally in all places except one. These uses + are imposed by external dependencies / features. nghttp2, quic, wolfSSL + and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that + any of these features make curl require a C99 compiler. (In case of + MSVC, this means Visual Studio 2010 or newer.) -- tests: increase the default server logs lock timeout + This patch changes the single use of `stdint.h` guarded by + `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using + `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used + anywhere else, allowing to delete this feature check as well. - This timeout is used to wait for the server to finish writing its logs - before checking them against the expected values. An overloaded machine - could take more than the two seconds previously allocated, so increase - the timeout to 5 seconds. + Closes #12275 - Ref: #11328 - Closes #11834 +Daniel Stenberg (6 Nov 2023) -- tests: increase TEST_HANG_TIMEOUT in two tests +- tool_operate: do not mix memory models - These tests had a 5 second timeout compared to 60 seconds for all other - tests. Make these consistent with the others for more reliability on - heavily-loaded machines. + Make sure 'inputpath' only points to memory allocated by libcurl so that + curl_free works correctly. - Ref: #11328 + Pointed out by Coverity -- test1056: disable on Windows + Follow-up to 859e88f6533f9e1f890 - This test relies on the IPv6 scope field being ignored when connecting to - ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit - dodgy, but it works on all our test platforms except Windows. This - test was disabled manually on all Windows CI builds already, so instead - add an incompatible feature and precheck so it's skipped on Windows - everywhere automatically. + Closes #12280 -- test587: add a slight delay after test +Stefan Eissing (6 Nov 2023) - This test is designed to connect to the server, then immediately send a - few bytes and disconnect. In some situations, such as on a loaded - server, this doesn't give the server enough time to write its lock file - before its existence is checked. The test harness then fails to find the - server's input log file (because it hasn't been written yet) and fails - the test. By adding a short delay after the test, the HTTP server has - enough time to write its lock file which gives itself more time to write - its remaining files. +- lib: client writer, part 2, accounting + logging - Ref: #11328 + This PR has these changes: -- tests: stop overriding the lock timeout + Renaming of unencode_* to cwriter, e.g. client writers + - documentation of sendf.h functions + - move max decode stack checks back to content_encoding.c + - define writer phase which was used as order before + - introduce phases for monitoring inbetween decode phases + - offering default implementations for init/write/close - These tests reduce the server lock wait timeout which can increase - flakiness on loaded machines. Since this is merely an optimization, - eliminate them in favour of reliability. + Add type paramter to client writer's do_write() + - always pass all writes through the writer stack + - writers who only care about BODY data will pass other writes unchanged - Ref: #11328 + add RAW and PROTOCOL client writers + - RAW used for Curl_debug() logging of CURLINFO_DATA_IN + - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and + Curl_pgrsSetDownloadCounter() + - remove all updates of data->req.bytecount and calls to + Curl_pgrsSetDownloadCounter() and Curl_debug() from other code + - adjust test457 expected output to no longer see the excess write -- tests: add some --expect100-timeout to reduce timing dependencies + Closes #12184 - These tests can fail when the test machine is so slow that the test HTTP - server didn't get a chance to complete before the client's one second - 100-continue timeout triggered. Increase that 1 second to 999 seconds so - this situation doesn't happen. +Daniel Stenberg (6 Nov 2023) - Ref: #11328 +- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw -- test661: return from test early in case of curl error + Closes #12278 -- tests: add the timing-dependent keyword on several tests +Viktor Szakats (6 Nov 2023) - These are ones likely to fail on heavily-loaded machines that alter the - normal test timing. Most of these tests already had the flaky keyword - since this condition makes them more likely to fail on CI. +- rand: fix build error with autotools + LibreSSL -- test1592: greatly increase the maximum test timeout + autotools unexpectedly detects `arc4random` because it is also looking + into dependency libs. One dependency, LibreSSL, happens to publish an + `arc4random` function (via its shared lib before v3.7, also via static + lib as of v3.8.2). When trying to use this function in `lib/rand.c`, + its protoype is missing. To fix that, curl included a prototype, but + that used a C99 type without including `stdint.h`, causing: - It was too short to be reliable on heavily loaded CI machines, and - as a fail-safe only, it didn't need to be short. + ``` + ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' + 37 | uint32_t arc4random(void); + | ^ + 1 error generated. + ``` - Ref: #11328 + This patch improves this by dropping the local prototype and instead + limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide + their own random source anyway. -- test: minor test cleanups + The better fix would be to teach autotools to not link dependency libs + while detecting `arc4random`. - Remove an obsolete block of code in tests 2032 & 576. - Add a comment in test 1474. + LibreSSL publishing a non-namespaced `arc4random` tracked here: + https://github.com/libressl/portable/issues/928 -- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts + Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 - This gives more of a margin for error when running on overloaded CI - servers. + Reviewed-by: Daniel Stenberg + Fixes #12257 + Closes #12274 - Ref: #11328 +Daniel Stenberg (5 Nov 2023) -- tests: improve SLOWDOWN test reliability by reducing sent data +- RELEASE-NOTES: synced - These tests are run in SLOWDOWN mode which adds a 10 msec delay after - each character output, which means it takes at least 1.6 seconds (and - 320 kernel calls) just to get through the long welcome banner. On an - overloaded system, this can end up taking much more than 1.6 seconds, - and even more than the 7 or 16 second curl timeout that the tests rely - on, causing them to fail. Reducing the size of the welcome banner drops - the total number of characters sent before the transfer starts by more - than half, which reduces the opportunity for test-breaking slowdowns by - the same amount. +- strdup: do Curl_strndup without strncpy - Ref: #11328 + To avoid (false positive) gcc-13 compiler warnings. -- test650: fix an end tag typo + Follow-up to 4855debd8a2c1cb -Jay Satiro (13 Sep 2023) + Assisted-by: Jay Satiro + Reported-by: Viktor Szakats + Fixes #12258 -- tool_cb_wrt: fix debug assertion +Enno Boland (5 Nov 2023) - - Fix off-by-one out-of-bounds array index in Windows debug assertion. +- HTTP: fix empty-body warning - Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 - Reported-by: Gisle Vanem + This change fixes a compiler warning with gcc-12.2.0 when + `-DCURL_DISABLE_BEARER_AUTH=ON` is used. -Daniel Stenberg (13 Sep 2023) + /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': + /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp + ty body in an 'else' statement [-Wempty-body] + 1147 | ; + | ^ -- ctype: add ISUNRESERVED() + Closes #12262 - ... and make Curl_isunreserved() use that macro instead of providing a - separate funtion for the purpose. +Daniel Stenberg (5 Nov 2023) - Closes #11840 +- openssl: identify the "quictls" backend correctly -Version 8.3.0 (13 Sep 2023) + Since vanilla OpenSSL does not support the QUIC API I think it helps + users to identify the correct OpenSSL fork in version output. The best + (crude) way to do that right now seems to be to check if ngtcp2 support + is enabled. -Daniel Stenberg (13 Sep 2023) + Closes #12270 -- RELEASE-NOTES: syn ced +Mark Gaiser (5 Nov 2023) - curl 8.3.0 release +- curl: improved IPFS and IPNS URL support -- THANKS: contributors from 8.3.0 + Previously just ipfs:// and ipns:// was supported, which is + too strict for some usecases. -Thorsten Klein (12 Sep 2023) + This patch allows paths and query arguments to be used too. + Making this work according to normal http semantics: -- cmake: set SIZEOF_LONG_LONG in curl_config.h + ipfs:///foo/bar?key=val + ipns:///foo/bar?key=val - in order to support 32bit builds regarding wolfssl CTC_SETTINGS + The gateway url support is changed. + It now only supports gateways in the form of: - Closes #11839 + http:///foo/bar + http:// -Jay Satiro (12 Sep 2023) + Query arguments here are explicitly not allowed and trigger an intended + malformed url error. -- curl_ngtcp2: fix error message + There also was a crash when IPFS_PATH was set with a non trailing + forward slash. This has been fixed. -- http_aws_sigv4: handle no-value user header entries + Lastly, a load of test cases have been added to verify the above. - - Handle user headers in format 'name:' and 'name;' with no value. + Reported-by: Steven Allen + Fixes #12148 + Closes #12152 - The former is used when the user wants to remove an internal libcurl - header and the latter is used when the user actually wants to send a - no-value header in the format 'name:' (note the semi-colon is converted - by libcurl to a colon). +Harry Mallon (5 Nov 2023) - Prior to this change the AWS header import code did not special case - either of those and the generated AWS SignedHeaders would be incorrect. +- docs: KNOWN_BUGS cleanup - Reported-by: apparentorder@users.noreply.github.com + * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. + Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 - Ref: https://curl.se/docs/manpage.html#-H + * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. + Fixed in #11806 - Fixes https://github.com/curl/curl/issues/11664 - Closes https://github.com/curl/curl/pull/11668 + * Remove mention of aws-sigv4 query empty value problems -Dan Fandrich (11 Sep 2023) + * Remove mention of aws-sigv4 missing amz-content-sha256 + Fixed in #9995 -- CI: run pytest with the -v option +- http_aws_sigv4: canonicalise valueless query params - This lists of the test cases being run so it can be tracked over time. + Fixes #8107 + Closes #12244 - Closes #11824 +Michael Kaufmann (4 Nov 2023) -Daniel Stenberg (11 Sep 2023) +- docs: preserve the modification date when copying the prebuilt man page -- HTTP3: the msquic backend is not functional + The previously built man page "curl.1" must be copied with the original + modification date, otherwise the man page is never updated. - I ask that we do not submit bugs for this backend just yet as we know it - does not fully work. + This fixes a bug that has been introduced with commit 2568441cab. - Closes #11831 - Closes #11819 + Reviewed-by: Dan Fandrich + Reviewed-by: Daniel Stenberg -- aws_sigv4: the query canon code miscounted URL encoded input + Closes #12199 - Added some extra ampersands to test 439 to verify "blank" query parts +Daniel Stenberg (4 Nov 2023) - Follow-up to fc76a24c53b08cdf +- docs: remove bold from some man page SYNOPSIS sections - Closes #11829 + In the name of consistency -vvb2060 (11 Sep 2023) + Closes #12267 -- quic: don't set SNI if hostname is an IP address +- openssl: two multi pointer checks should probably rather be asserts - We already do this for TLS connections. + ... so add the asserts now and consider removing the dynamic checks in a + future. - RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in - "HostName". + Ref: #12261 + Closes #12264 - Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 +boilingoden (4 Nov 2023) - Fixes https://github.com/curl/curl/issues/11827 - Closes https://github.com/curl/curl/pull/11828 +- docs: add supported version for the json write-out -Daniel Stenberg (10 Sep 2023) + xref: https://curl.se/changes.html#7_70_0 -- RELEASE-NOTES: synced + Closes #12266 diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index ecd6e9eb9a..7910255eda 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -19,6 +19,7 @@ Aaron Oneal Aaron Orenstein Aaron Scarisbrick aasivov on github +Abdullah Alyan Abhinav Singh Abram Pousada accountantM on github @@ -181,6 +182,7 @@ Andrei Sedoi Andrei Valeriu BICA Andrei Virtosu Andrej E Baranov +Andrew Andrew Barnert Andrew Barnes Andrew Benham @@ -225,6 +227,7 @@ Anthony Hu Anthony Ramine Anthony Shaw Antoine Aubert +Antoine Bollengier Antoine Calando Antoine du Hamel Antoine Pietri @@ -357,6 +360,7 @@ Björn Stenberg black-desk on github Blaise Potard Blake Burkhart +blankie bnfp on github Bo Anderson Bob Relyea @@ -455,6 +459,7 @@ cbartl on github cclauss on github Cering on github Cesar Eduardo Barros +Ch40zz on github Chad Monroe Chandrakant Bagul Chara White @@ -540,6 +545,7 @@ Colin Blair Colin Cross Colin Hogben Colin Leroy +Colin Leroy-Mira Colin O'Dell Colin Watson Colm Buckley @@ -573,6 +579,7 @@ d912e3 on github Da-Yoon Chung daboul on github Dag Ekengren +Dagfinn Ilmari Mannsåker Dagobert Michelsen Daiki Ueno Dair Grant @@ -604,6 +611,7 @@ Daniel Faust Daniel Gustafsson Daniel Hallberg Daniel Hwang +Daniel J. H. Daniel Jeliński Daniel Johnson Daniel Kahn Gillmor @@ -613,6 +621,7 @@ Daniel Kurečka Daniel Lee Hwang Daniel Lublin Daniel Marjamäki +Daniel McCarney Daniel Melani Daniel Mentz Daniel Romero @@ -912,6 +921,7 @@ Fabrizio Ammollo Fahim Chandurwala Faizur Rahman Faraz Fallahi +farazrbx on github Farzin on github Fata Nugraha Fawad Mirza @@ -974,6 +984,7 @@ FuccDucc on github Fujii Hironori fullincome on github fundawang on github +fuzzard Gabe Gabriel Corona Gabriel Kuri @@ -1033,6 +1044,7 @@ Glenn Sheridan Glenn Strauss godmar on github Godwin Stewart +Gonçalo Carvalho Google Inc. Gordon Marler Gorilla Maguila @@ -1067,6 +1079,7 @@ Gunter Knauf guoxinvmware on github Gustaf Hui Gustavo Grieco +Gusted Guy Poizat GwanYeong Kim Gwen Shapira @@ -1079,6 +1092,7 @@ Hagai Auro Haibo Huang Hakan Sunay Halil Hamish Mackenzie +hammlee96 on github hamstergene on github Han Han Han Qiao @@ -1095,6 +1109,7 @@ Hans-Jurgen May Hao Wu Hardeep Singh Haris Okanovic +Harmen Stoppels Harold Stuart Harry Mallon Harry Sarson @@ -1130,6 +1145,7 @@ Hiroki Kurosawa Hirotaka Tagawa Ho-chi Chen Hoi-Ho Chan +Hongfei Li Hongli Lai Hongyi Zhao Howard Blaise @@ -1192,6 +1208,7 @@ Isaac Boukris Isaiah Norton Ishan SinghLevett Ithubg on github +Ivan Ivan Avdeev Ivan Tsybulin ivanfywang @@ -1353,6 +1370,7 @@ Jiri Dvorak Jiri Hruska Jiri Jaburek Jishan Shaikh +Jiwoo Park Jiří Bok Jiří Malák jmdavitt on github @@ -1372,6 +1390,7 @@ Johan Anderson Johan Lantz Johan Nilsson Johan van Selst +Johann Sebastian Schicho Johann150 on github Johannes Bauer Johannes Ernst @@ -1513,6 +1532,7 @@ jzinn on github János Fekete Jérémy Rabasco Jérémy Rocher +Jérôme Leclercq Jörg Mueller-Tolk Jörn Hartroth Jürgen Gmach @@ -1524,7 +1544,9 @@ Kai Noda Kai Pastor Kai Sommerfeld Kai-Uwe Rommel +Kailun Qin Kalle Vahlman +kalvdans on github Kamil Dudka Kane York Kang Lin @@ -1592,12 +1614,13 @@ Koichi Shiraishi kokke on github Konstantin Isakov Konstantin Kushnir +Konstantin Kuzov Konstantin Vlasov KotlinIsland on github kotoriのねこ kouzhudong on github Kovalkov Dmitrii -kpcyrd +kpcyrd on github kreshano on github Kris Kennaway Krishnendu Majumdar @@ -1709,6 +1732,7 @@ Luca Niccoli Lucas Adamski Lucas Clemente Vella Lucas Holt +Lucas Nussbaum Lucas Pardue Lucas Servén Marín Lucas Severo @@ -1740,6 +1764,7 @@ Maciej Puzio Maciej W. Rozycki madblobfish on github MaeIsBad on github +magisterquis on hackerone Mahmoud Samir Fayed Maks Naumov Maksim Arhipov @@ -1917,6 +1942,7 @@ Median Median Stride mehatzri on github Mehmet Bozkurt Mekonikum +Mel Zuser Melissa Mears Melroy van den Berg Mert Yazıcıoğlu @@ -1948,6 +1974,7 @@ Michael Kolechkin Michael Kujawa Michael König Michael Lee +Michael Litwak Michael Maltese Michael Mealling Michael Mueller @@ -2203,6 +2230,7 @@ Pau Garcia i Quiles Paul B. Omta Paul Donohue Paul Dreik +Paul Gilmartin Paul Groke Paul Harrington Paul Harris @@ -2224,6 +2252,7 @@ Paulo Roberto Tomasi Pavel Cenek Pavel Gushchin Pavel Kalyugin +Pavel Kropachev Pavel Löbl Pavel Mayorov Pavel Orehov @@ -2352,6 +2381,7 @@ Rafael Sagula Rafayel Mkrtchyan Rafaël Carré Rafał Mikrut +Rahul Krishna M Rainer Canavan Rainer Jung Rainer Koenig @@ -2398,10 +2428,12 @@ Renaud Guillard Renaud Lehoux Rene Bernhardt Rene Rebe +renovate[bot] Reuven Wachtfogel RevaliQaQ on github Reza Arbab Rianov Viacheslav +riastradh on github Ricardo Cadime Ricardo Gomes Ricardo M. Correia @@ -2572,6 +2604,7 @@ SandakovMM on github Sander Gates Sandor Feldi Sandro Jaeckel +Sanjay Pujare Santhana Todatry Santino Keupp Saqib Ali @@ -2683,6 +2716,7 @@ Sohom Datta Somnath Kundu Song Ma Sonia Subramanian +southernedge on github Spacen Jasset Spezifant on github Spiridonoff A.V @@ -2730,6 +2764,7 @@ Stephane Pellegrino Stephen Boost Stephen Brokenshire Stephen Collyer +Stephen Farrell Stephen Kick Stephen M. Coakley Stephen More @@ -2915,6 +2950,7 @@ Ton Voon Toni Moreno Tony Kelman tonystz on Github +Toon Claes Toon Verwaest Tor Arntsen Torben Dannhauer @@ -2931,6 +2967,8 @@ Trivikram Kamat Troels Walsted Hansen Troy Engel trrui-huawei +Trumeet on github +Trzik on github Tseng Jun Tuomas Siipola Tuomo Rinne @@ -3117,6 +3155,7 @@ zhihaoy on github Zhouyihai Ding ZimCodes on github zloi-user on github +zmcx16 on github Zmey Petroff Zvi Har'El zzq1015 on github -- cgit v1.2.3